-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
76 changed files
with
550 additions
and
11,686 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,25 +1,10 @@ | ||
# Honu | ||
|
||
[![Go Reference](https://pkg.go.dev/badge/github.com/rotationalio/honu.svg)](https://pkg.go.dev/github.com/rotationalio/honu) | ||
[![Go Report Card](https://goreportcard.com/badge/github.com/rotationalio/honu)](https://goreportcard.com/report/github.com/rotationalio/honu) | ||
![GitHub Actions CI](https://github.com/rotationalio/honu/actions/workflows/build.yaml/badge.svg?branch=main) | ||
![GitHub Actions CI](https://github.com/rotationalio/honu/actions/workflows/tests.yaml/badge.svg?branch=main) | ||
|
||
**HonuDB is a replicated key-value store intended for large systems that are distributed globally. Although HonuDB uses an eventually consistent gossip protocol for replication, it uses reinforcement learning with multi-armed bandits to optimize replication. Adaptive consistency reduces costs (ingress and egress data transfer) as well as improves consistency by lowering the likelihood of stale reads or forked writes.** | ||
The Honu Database is an eventually consistent replicated document database that intended for large systems that are distributed globally. Honu uses smart anti-entropy replication to quickly replicate collections across multiple nodes. | ||
|
||
## Benchmarks | ||
Smart anti-entropy uses reinforcement learning with multi-armed bandits to optimize replication. Adaptive consistency reduces costs (ingress and egress data transfer) as well as improves consistency by lowering the likelihood of stale reads or forked writes. | ||
|
||
Baseline comparison of honu database vs. pure leveldb performance to determine the overhead that honu adds: | ||
|
||
``` | ||
BenchmarkHonuGet-8 466665 2405 ns/op | ||
BenchmarkLevelDBGet-8 1243785 955.3 ns/op | ||
BenchmarkHonuPut-8 21448 55125 ns/op | ||
BenchmarkLevelDBPut-8 24276 52422 ns/op | ||
BenchmarkHonuDelete-8 42298 25136 ns/op | ||
BenchmarkLevelDBDelete-8 76857 16101 ns/op | ||
BenchmarkHonuIter-8 63772 18349 ns/op | ||
BenchmarkLevelDBIter-8 255319 4271 ns/op | ||
BenchmarkHonuObject-8 480555 2362 ns/op | ||
``` | ||
|
||
[![Baseline Benchmarks](docs/benchmark.png)](docs/benchmark.png) | ||
The goal of the database is to provide scalable data retrieval both in terms of number of nodes (e.g. scale to 100s of nodes) and amount of data (hundreds of terabytes). In addition to scale, this database provides data access controls, privacy and provenance, and other security related features. In short, HonuDB is a distributed data governance database for machine learning and artificial intelligence workloads. |
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.