From 4f14e369e68b1f03ebfc4dba7d6f368b1c46f4a5 Mon Sep 17 00:00:00 2001
From: Ryan Fox-Tyler <60440289+ryanfoxtyler@users.noreply.github.com>
Date: Tue, 4 Feb 2025 07:34:51 -0500
Subject: [PATCH 1/3] docs: migrate Badger docs
---
.trunk/trunk.yaml | 4 +-
badger/design.mdx | 62 ++
badger/overview.mdx | 19 +
badger/quickstart.mdx | 726 ++++++++++++++++++
badger/troubleshooting.mdx | 166 ++++
community-and-support.mdx | 1 +
mint.json | 12 +
styles/config/vocabularies/general/accept.txt | 1 +
8 files changed, 989 insertions(+), 2 deletions(-)
create mode 100644 badger/design.mdx
create mode 100644 badger/overview.mdx
create mode 100644 badger/quickstart.mdx
create mode 100644 badger/troubleshooting.mdx
diff --git a/.trunk/trunk.yaml b/.trunk/trunk.yaml
index a6973a49..71bd6af7 100644
--- a/.trunk/trunk.yaml
+++ b/.trunk/trunk.yaml
@@ -18,11 +18,11 @@ runtimes:
lint:
enabled:
- - renovate@39.149.0
+ - renovate@39.156.1
- golangci-lint@1.63.4
- vale@3.9.4
- actionlint@1.7.7
- - checkov@3.2.360
+ - checkov@3.2.361
- git-diff-check
- markdownlint@0.44.0
- oxipng@9.1.3
diff --git a/badger/design.mdx b/badger/design.mdx
new file mode 100644
index 00000000..4589f07d
--- /dev/null
+++ b/badger/design.mdx
@@ -0,0 +1,62 @@
+---
+title: Design
+description: "Architected for fast key-value storage in Go"
+"og:title": "Design - Badger"
+---
+
+We wrote Badger with these design goals in mind:
+
+- Write a key-value database in pure Go
+- Use latest research to build the fastest KV database for data sets spanning
+ terabytes
+- Optimize for modern storage devices
+
+Badger’s design is based on a paper titled
+[WiscKey: Separating Keys from Values in SSD-conscious Storage](https://www.usenix.org/system/files/conference/fast16/fast16-papers-lu.pdf).
+
+## References
+
+The following blog posts are a great starting point for learning more about
+Badger and the underlying design principles:
+
+- [Introducing Badger: A fast key-value store written natively in Go](https://dgraph.io/blog/post/badger/)
+- [Make Badger crash resilient with ALICE](https://dgraph.io/blog/post/alice/)
+- [Badger vs LMDB vs BoltDB: Benchmarking key-value databases in Go](https://dgraph.io/blog/post/badger-lmdb-boltdb/)
+- [Concurrent ACID Transactions in Badger](https://dgraph.io/blog/post/badger-txn/)
+
+## Comparisons
+
+| Feature | Badger | RocksDB | BoltDB |
+| ----------------------------- | -------------------------------------- | ---------------------------- | ------- |
+| Design | LSM tree with value log | LSM tree only | B+ tree |
+| High Read throughput | Yes | No | Yes |
+| High Write throughput | Yes | Yes | No |
+| Designed for SSDs | Yes (with latest research1) | Not specifically2 | No |
+| Embeddable | Yes | Yes | Yes |
+| Sorted KV access | Yes | Yes | Yes |
+| Pure Go (no Cgo) | Yes | No | Yes |
+| Transactions | Yes | Yes | Yes |
+| ACID-compliant | Yes, concurrent with SSI3 | No | Yes |
+| Snapshots | Yes | Yes | Yes |
+| TTL support | Yes | Yes | No |
+| 3D access (key-value-version) | Yes4 | No | No |
+
+1 The WiscKey paper (on which Badger is based) saw big wins with
+separating values from keys, significantly reducing the write amplification
+compared to a typical LSM tree.
+
+2 RocksDB is an SSD-optimized version of LevelDB, which was designed
+specifically for rotating disks. As such RocksDB's design isn't aimed at SSDs.
+
+3 SSI: Serializable Snapshot Isolation. For more details, see the
+blog post [Concurrent ACID Transactions in
+Badger](https://dgraph.io/blog/post/badger-txn/)
+
+4 Badger provides direct access to value versions via its Iterator
+API. Users can also specify how many versions to keep per key via Options.
+
+## Benchmarks
+
+We've run comprehensive benchmarks against RocksDB, BoltDB, and LMDB. The
+benchmarking code with detailed logs are in the
+[badger-bench](https://github.com/dgraph-io/badger-bench) repo.
diff --git a/badger/overview.mdx b/badger/overview.mdx
new file mode 100644
index 00000000..9ac81c14
--- /dev/null
+++ b/badger/overview.mdx
@@ -0,0 +1,19 @@
+---
+title: Overview
+description: "Welcome to the Badger docs!"
+mode: "wide"
+"og:title": "Overview - Badger"
+---
+
+## What is Badger? {/* */}
+
+BadgerDB is an embeddable, persistent, and fast key-value (KV) database written
+in pure Go. It's the underlying database for [Dgraph](https://dgraph.io), a
+fast, distributed graph database. It's meant to be an efficient alternative to
+non-Go-based key-value stores like RocksDB.
+
+## Changelog
+
+We keep the
+[repo Changelog](https://github.com/dgraph-io/badger/blob/main/CHANGELOG.md) up
+to date with each release.
diff --git a/badger/quickstart.mdx b/badger/quickstart.mdx
new file mode 100644
index 00000000..8f39dd89
--- /dev/null
+++ b/badger/quickstart.mdx
@@ -0,0 +1,726 @@
+---
+title: Quickstart
+description: "Everything you need to get started with Badger"
+"og:title": "Quickstart - Badger"
+---
+
+## Prerequisites
+
+- [Go](https://go.dev/doc/install) - v1.23 or higher
+- Text editor - we recommend [VS Code](https://code.visualstudio.com/)
+- Terminal - access Badger through a command-line interface (CLI)
+
+## Installing
+
+To start using Badger, run the following command to retrieve the library.
+
+```sh
+go get github.com/dgraph-io/badger/v4
+```
+
+Then, install the Badger command line utility into your $GOBIN path.
+
+```sh
+go install github.com/dgraph-io/badger/v4/badger@latest
+```
+
+## Opening a database
+
+The top-level object in Badger is a `DB`. It represents multiple files on disk
+in specific directories, which contain the data for a single database.
+
+To open your database, use the `badger.Open()` function, with the appropriate
+options. The `Dir` and `ValueDir` options are mandatory and you must specify
+them in your client. To simplify, you can set both options to the same value.
+
+
+ Badger obtains a lock on the directories. Multiple processes can't open the
+ same database at the same time.
+
+
+```go
+package main
+
+import (
+ "log"
+
+ badger "github.com/dgraph-io/badger/v4"
+)
+
+func main() {
+ // Open the Badger database located in the /tmp/badger directory.
+ // It will be created if it doesn't exist.
+ db, err := badger.Open(badger.DefaultOptions("/tmp/badger"))
+ if err != nil {
+ log.Fatal(err)
+ }
+ defer db.Close()
+ // Your code here…
+}
+```
+
+### In-memory/diskless mode
+
+By default, Badger ensures all data persists to disk. It also supports a pure
+in-memory mode. When Badger is running in this mode, all data remains in memory
+only. Reads and writes are much faster, but all data stored in Badger is lost in
+the case of a crash or close. To open badger in in-memory mode, set the
+`InMemory` option.
+
+```go
+opt := badger.DefaultOptions("").WithInMemory(true)
+```
+
+### Encryption mode
+
+If you enable encryption on Badger, you also need to set the index cache size.
+
+
+ Having a cache improves the performance. Otherwise, your reads would be very
+ slow while encryption is enabled.
+
+
+For example, to set a `100 Mb` cache:
+
+```go
+opts.IndexCache = 100 << 20 // 100 mb or some other size based on the amount of data
+```
+
+## Transactions
+
+### Read-only transactions
+
+To start a read-only transaction, you can use the `DB.View()` method:
+
+```go
+err := db.View(func(txn *badger.Txn) error {
+ // Your code here…
+ return nil
+})
+```
+
+You can't perform any writes or deletes within this transaction. Badger ensures
+that you get a consistent view of the database within this closure. Any writes
+that happen elsewhere after the transaction has started aren't seen by calls
+made within the closure.
+
+### Read-write transactions
+
+To start a read-write transaction, you can use the `DB.Update()` method:
+
+```go
+err := db.Update(func(txn *badger.Txn) error {
+ // Your code here…
+ return nil
+})
+```
+
+All database operations are allowed inside a read-write transaction.
+
+Always check the returned error value. If you return an error within your
+closure it will be passed through.
+
+An `ErrConflict` error will be reported in case of a conflict. Depending on the
+state of your app, you have the option to retry the operation if you receive
+this error.
+
+An `ErrTxnTooBig` will be reported in case the number of pending writes/deletes
+in the transaction exceeds a certain limit. In that case, it's best to commit
+the transaction and start a new transaction immediately. Here is an example (we
+aren't checking for errors in some places for simplicity):
+
+```go
+updates := make(map[string]string)
+txn := db.NewTransaction(true)
+for k,v := range updates {
+ if err := txn.Set([]byte(k),[]byte(v)); err == badger.ErrTxnTooBig {
+ _ = txn.Commit()
+ txn = db.NewTransaction(true)
+ _ = txn.Set([]byte(k),[]byte(v))
+ }
+}
+_ = txn.Commit()
+```
+
+### Managing transactions manually
+
+The `DB.View()` and `DB.Update()` methods are wrappers around the
+`DB.NewTransaction()` and `Txn.Commit()` methods (or `Txn.Discard()` in case of
+read-only transactions). These helper methods start the transaction, execute a
+function, and then safely discard your transaction if an error is returned. This
+is the recommended way to use Badger transactions.
+
+However, sometimes you may want to manually create and commit your transactions.
+You can use the `DB.NewTransaction()` function directly, which takes in a
+boolean argument to specify whether a read-write transaction is required. For
+read-write transactions, it's necessary to call `Txn.Commit()` to ensure the
+transaction is committed. For read-only transactions, calling `Txn.Discard()` is
+sufficient. `Txn.Commit()` also calls `Txn.Discard()` internally to cleanup the
+transaction, so just calling `Txn.Commit()` is sufficient for read-write
+transaction. However, if your code doesn’t call `Txn.Commit()` for some reason
+(for e.g it returns prematurely with an error), then please make sure you call
+`Txn.Discard()` in a `defer` block. Refer to the code below.
+
+```go
+// Start a writable transaction.
+txn := db.NewTransaction(true)
+defer txn.Discard()
+
+// Use the transaction...
+err := txn.Set([]byte("answer"), []byte("42"))
+if err != nil {
+ return err
+}
+
+// Commit the transaction and check for error.
+if err := txn.Commit(); err != nil {
+ return err
+}
+```
+
+The first argument to `DB.NewTransaction()` is a boolean stating if the
+transaction should be writable.
+
+Badger allows an optional callback to the `Txn.Commit()` method. Normally, the
+callback can be set to `nil`, and the method will return after all the writes
+have succeeded. However, if this callback is provided, the `Txn.Commit()` method
+returns as soon as it has checked for any conflicts. The actual writing to the
+disk happens asynchronously, and the callback is invoked once the writing has
+finished, or an error has occurred. This can improve the throughput of the app
+in some cases. But it also means that a transaction isn't durable until the
+callback has been invoked with a `nil` error value.
+
+## Using key/value pairs
+
+To save a key/value pair, use the `Txn.Set()` method:
+
+```go
+err := db.Update(func(txn *badger.Txn) error {
+ err := txn.Set([]byte("answer"), []byte("42"))
+ return err
+})
+```
+
+Key/Value pair can also be saved by first creating `Entry`, then setting this
+`Entry` using `Txn.SetEntry()`. `Entry` also exposes methods to set properties
+on it.
+
+```go
+err := db.Update(func(txn *badger.Txn) error {
+ e := badger.NewEntry([]byte("answer"), []byte("42"))
+ err := txn.SetEntry(e)
+ return err
+})
+```
+
+This sets the value of the `"answer"` key to `"42"`. To retrieve this value, we
+can use the `Txn.Get()` method:
+
+```go
+err := db.View(func(txn *badger.Txn) error {
+ item, err := txn.Get([]byte("answer"))
+ handle(err)
+
+ var valNot, valCopy []byte
+ err := item.Value(func(val []byte) error {
+ // This func with val would only be called if item.Value encounters no error.
+
+ // Accessing val here is valid.
+ fmt.Printf("The answer is: %s\n", val)
+
+ // Copying or parsing val is valid.
+ valCopy = append([]byte{}, val...)
+
+ // Assigning val slice to another variable is NOT OK.
+ valNot = val // Do not do this.
+ return nil
+ })
+ handle(err)
+
+ // DO NOT access val here. It is the most common cause of bugs.
+ fmt.Printf("NEVER do this. %s\n", valNot)
+
+ // You must copy it to use it outside item.Value(...).
+ fmt.Printf("The answer is: %s\n", valCopy)
+
+ // Alternatively, you could also use item.ValueCopy().
+ valCopy, err = item.ValueCopy(nil)
+ handle(err)
+ fmt.Printf("The answer is: %s\n", valCopy)
+
+ return nil
+})
+```
+
+`Txn.Get()` returns `ErrKeyNotFound` if the value isn't found.
+
+Please note that values returned from `Get()` are only valid while the
+transaction is open. If you need to use a value outside of the transaction then
+you must use `copy()` to copy it to another byte slice.
+
+Use the `Txn.Delete()` method to delete a key.
+
+## Monotonically increasing integers
+
+To get unique monotonically increasing integers with strong durability, you can
+use the `DB.GetSequence` method. This method returns a `Sequence` object, which
+is thread-safe and can be used concurrently via various goroutines.
+
+Badger would lease a range of integers to hand out from memory, with the
+bandwidth provided to `DB.GetSequence`. The frequency at which disk writes are
+done is determined by this lease bandwidth and the frequency of `Next`
+invocations. Setting a bandwidth too low would do more disk writes, setting it
+too high would result in wasted integers if Badger is closed or crashes. To
+avoid wasted integers, call `Release` before closing Badger.
+
+```go
+seq, err := db.GetSequence(key, 1000)
+defer seq.Release()
+for {
+ num, err := seq.Next()
+}
+```
+
+## Merge operations
+
+Badger provides support for ordered merge operations. You can define a func of
+type `MergeFunc` which takes in an existing value, and a value to be _merged_
+with it. It returns a new value which is the result of the _merge_ operation.
+All values are specified in byte arrays. For e.g., here is a merge function
+(`add`) which appends a `[]byte` value to an existing `[]byte` value.
+
+```go
+// Merge function to append one byte slice to another
+func add(originalValue, newValue []byte) []byte {
+ return append(originalValue, newValue...)
+}
+```
+
+This function can then be passed to the `DB.GetMergeOperator()` method, along
+with a key, and a duration value. The duration specifies how often the merge
+function is run on values that have been added using the `MergeOperator.Add()`
+method.
+
+`MergeOperator.Get()` method can be used to retrieve the cumulative value of the
+key associated with the merge operation.
+
+```go
+key := []byte("merge")
+
+m := db.GetMergeOperator(key, add, 200*time.Millisecond)
+defer m.Stop()
+
+m.Add([]byte("A"))
+m.Add([]byte("B"))
+m.Add([]byte("C"))
+
+res, _ := m.Get() // res should have value ABC encoded
+```
+
+Example: merge operator which increments a counter
+
+```go
+func uint64ToBytes(i uint64) []byte {
+ var buf [8]byte
+ binary.BigEndian.PutUint64(buf[:], i)
+ return buf[:]
+}
+
+func bytesToUint64(b []byte) uint64 {
+ return binary.BigEndian.Uint64(b)
+}
+
+// Merge function to add two uint64 numbers
+func add(existing, new []byte) []byte {
+ return uint64ToBytes(bytesToUint64(existing) + bytesToUint64(new))
+}
+```
+
+It can be used as
+
+```go
+key := []byte("merge")
+
+m := db.GetMergeOperator(key, add, 200*time.Millisecond)
+defer m.Stop()
+
+m.Add(uint64ToBytes(1))
+m.Add(uint64ToBytes(2))
+m.Add(uint64ToBytes(3))
+
+res, _ := m.Get() // res should have value 6 encoded
+```
+
+## Setting time to live (TTL) and user metadata on keys
+
+Badger allows setting an optional Time to Live (TTL) value on keys. Once the TTL
+has elapsed, the key is no longer retrievable and is eligible for garbage
+collection. A TTL can be set as a `time.Duration` value using the
+`Entry.WithTTL()` and `Txn.SetEntry()` API methods.
+
+```go
+err := db.Update(func(txn *badger.Txn) error {
+ e := badger.NewEntry([]byte("answer"), []byte("42")).WithTTL(time.Hour)
+ err := txn.SetEntry(e)
+ return err
+})
+```
+
+An optional user metadata value can be set on each key. A user metadata value is
+represented by a single byte. It can be used to set certain bits along with the
+key to aid in interpreting or decoding the key-value pair. User metadata can be
+set using `Entry.WithMeta()` and `Txn.SetEntry()` API methods.
+
+```go
+err := db.Update(func(txn *badger.Txn) error {
+ e := badger.NewEntry([]byte("answer"), []byte("42")).WithMeta(byte(1))
+ err := txn.SetEntry(e)
+ return err
+})
+```
+
+`Entry` APIs can be used to add the user metadata and TTL for same key. This
+`Entry` then can be set using `Txn.SetEntry()`.
+
+```go
+err := db.Update(func(txn *badger.Txn) error {
+ e := badger.NewEntry([]byte("answer"), []byte("42")).WithMeta(byte(1)).WithTTL(time.Hour)
+ err := txn.SetEntry(e)
+ return err
+})
+```
+
+## Iterating over keys
+
+To iterate over keys, we can use an `Iterator`, which can be obtained using the
+`Txn.NewIterator()` method. Iteration happens in byte-wise lexicographical
+sorting order.
+
+```go
+err := db.View(func(txn *badger.Txn) error {
+ opts := badger.DefaultIteratorOptions
+ opts.PrefetchSize = 10
+ it := txn.NewIterator(opts)
+ defer it.Close()
+ for it.Rewind(); it.Valid(); it.Next() {
+ item := it.Item()
+ k := item.Key()
+ err := item.Value(func(v []byte) error {
+ fmt.Printf("key=%s, value=%s\n", k, v)
+ return nil
+ })
+ if err != nil {
+ return err
+ }
+ }
+ return nil
+})
+```
+
+The iterator allows you to move to a specific point in the list of keys and move
+forward or backward through the keys one at a time.
+
+By default, Badger prefetches the values of the next 100 items. You can adjust
+that with the `IteratorOptions.PrefetchSize` field. However, setting it to a
+value higher than `GOMAXPROCS` (which we recommend to be 128 or higher)
+shouldn’t give any additional benefits. You can also turn off the fetching of
+values altogether. See section below on key-only iteration.
+
+### Prefix scans
+
+To iterate over a key prefix, you can combine `Seek()` and `ValidForPrefix()`:
+
+```go
+db.View(func(txn *badger.Txn) error {
+ it := txn.NewIterator(badger.DefaultIteratorOptions)
+ defer it.Close()
+ prefix := []byte("1234")
+ for it.Seek(prefix); it.ValidForPrefix(prefix); it.Next() {
+ item := it.Item()
+ k := item.Key()
+ err := item.Value(func(v []byte) error {
+ fmt.Printf("key=%s, value=%s\n", k, v)
+ return nil
+ })
+ if err != nil {
+ return err
+ }
+ }
+ return nil
+})
+```
+
+### Possible pagination implementation using Prefix scans
+
+Considering that iteration happens in **byte-wise lexicographical sorting**
+order, it's possible to create a sorting-sensitive key. For example, a simple
+blog post key might look like:`feed:userUuid:timestamp:postUuid`. Here, the
+`timestamp` part of the key is treated as an attribute, and items will be stored
+in the corresponding order:
+
+| Order ASC | Key |
+| :-------: | :------------------------------------------------------------ |
+| 1 | feed:tQpnEDVRoCxTFQDvyQEzdo:1733127889:tQpnEDVRoCxTFQDvyQEzdo |
+| 2 | feed:tQpnEDVRoCxTFQDvyQEzdo:1733127533:1Mryrou1xoekEaxzrFiHwL |
+| 3 | feed:tQpnEDVRoCxTFQDvyQEzdo:1733127486:pprRrNL2WP4yfVXsSNBSx6 |
+
+It's important to properly configure keys for lexicographical sorting to avoid
+incorrect ordering.
+
+A **prefix scan** through the preceding keys can be achieved using the prefix
+`feed:tQpnEDVRoCxTFQDvyQEzdo`. All matching keys are returned, sorted by
+`timestamp`.
+Sorting can be done in ascending or descending order based on `timestamp` or
+`reversed timestamp` as needed:
+
+```go
+reversedTimestamp := math.MaxInt64-time.Now().Unix()
+```
+
+This makes it possible to implement simple pagination by using a limit for the
+number of keys and a cursor (the last key from the previous iteration) to
+identify where to resume.
+
+```go
+// startCursor may look like 'feed:tQpnEDVRoCxTFQDvyQEzdo:1733127486'.
+// A prefix scan with this cursor will locate the specific key where
+// the previous iteration stopped.
+err = db.badger.View(func(txn *badger.Txn) error {
+ it := txn.NewIterator(opts)
+ defer it.Close()
+
+ // Prefix example 'feed:tQpnEDVRoCxTFQDvyQEzdo'
+ // if no cursor provided prefix scan starts from the beginning
+ p := prefix
+ if startCursor != nil {
+ p = startCursor
+ }
+ iterNum := 0 // Tracks the number of iterations to enforce the limit.
+ for it.Seek(p); it.ValidForPrefix(p); it.Next() {
+ // The method it.ValidForPrefix ensures that iteration continues
+ // as long as keys match the prefix.
+ // For example, if p = 'feed:tQpnEDVRoCxTFQDvyQEzdo:1733127486',
+ // it matches keys like
+ // 'feed:tQpnEDVRoCxTFQDvyQEzdo:1733127889:pprRrNL2WP4yfVXsSNBSx6'.
+
+ // Once the starting point for iteration is found, revert the prefix
+ // back to 'feed:tQpnEDVRoCxTFQDvyQEzdo' to continue iterating sequentially.
+ // Otherwise, iteration would stop after a single prefix-key match.
+ p = prefix
+
+ item := it.Item()
+ key := string(item.Key())
+
+ if iterNum > limit { // Limit reached.
+ nextCursor = key // Save the next cursor for future iterations.
+ return nil
+ }
+ iterNum++ // Increment iteration count.
+
+ err := item.Value(func(v []byte) error {
+ fmt.Printf("key=%s, value=%s\n", k, v)
+ return nil
+ })
+ if err != nil {
+ return err
+ }
+ }
+ // If the number of iterations is less than the limit,
+ // it means there are no more items for the prefix.
+ if iterNum < limit {
+ nextCursor = ""
+ }
+ return nil
+ })
+return nextCursor, err
+```
+
+### Key-only iteration
+
+Badger supports a unique mode of iteration called _key-only_ iteration. It's
+several order of magnitudes faster than regular iteration, because it involves
+access to the LSM-tree only, which is usually resident entirely in RAM. To
+enable key-only iteration, you need to set the `IteratorOptions.PrefetchValues`
+field to `false`. This can also be used to do sparse reads for selected keys
+during an iteration, by calling `item.Value()` only when required.
+
+```go
+err := db.View(func(txn *badger.Txn) error {
+ opts := badger.DefaultIteratorOptions
+ opts.PrefetchValues = false
+ it := txn.NewIterator(opts)
+ defer it.Close()
+ for it.Rewind(); it.Valid(); it.Next() {
+ item := it.Item()
+ k := item.Key()
+ fmt.Printf("key=%s\n", k)
+ }
+ return nil
+})
+```
+
+## Stream
+
+Badger provides a Stream framework, which concurrently iterates over all or a
+portion of the DB, converting data into custom key-values, and streams it out
+serially to be sent over network, written to disk, or even written back to
+Badger. This is a lot faster way to iterate over Badger than using a single
+Iterator. Stream supports Badger in both managed and normal mode.
+
+Stream uses the natural boundaries created by SSTables within the LSM tree, to
+quickly generate key ranges. Each goroutine then picks a range and runs an
+iterator to iterate over it. Each iterator iterates over all versions of values
+and is created from the same transaction, thus working over a snapshot of the
+DB. Every time a new key is encountered, it calls `ChooseKey(item)`, followed by
+`KeyToList(key, itr)`. This allows a user to select or reject that key, and if
+selected, convert the value versions into custom key-values. The goroutine
+batches up 4 MB worth of key-values, before sending it over to a channel.
+Another goroutine further batches up data from this channel using _smart
+batching_ algorithm and calls `Send` serially.
+
+This framework is designed for high throughput key-value iteration, spreading
+the work of iteration across many goroutines. `DB.Backup` uses this framework to
+provide full and incremental backups quickly. Dgraph is a heavy user of this
+framework. In fact, this framework was developed and used within Dgraph, before
+getting ported over to Badger.
+
+```go
+stream := db.NewStream()
+// db.NewStreamAt(readTs) for managed mode.
+
+// -- Optional settings
+stream.NumGo = 16 // Set number of goroutines to use for iteration.
+stream.Prefix = []byte("some-prefix") // Leave nil for iteration over the whole DB.
+stream.LogPrefix = "Badger.Streaming" // For identifying stream logs. Outputs to Logger.
+
+// ChooseKey is called concurrently for every key. If left nil, assumes true by default.
+stream.ChooseKey = func(item *badger.Item) bool {
+ return bytes.HasSuffix(item.Key(), []byte("er"))
+}
+
+// KeyToList is called concurrently for chosen keys. This can be used to convert
+// Badger data into custom key-values. If nil, uses stream.ToList, a default
+// implementation, which picks all valid key-values.
+stream.KeyToList = nil
+
+// -- End of optional settings.
+
+// Send is called serially, while Stream.Orchestrate is running.
+stream.Send = func(list *pb.KVList) error {
+ return proto.MarshalText(w, list) // Write to w.
+}
+
+// Run the stream
+if err := stream.Orchestrate(context.Background()); err != nil {
+ return err
+}
+// Done.
+```
+
+## Garbage collection
+
+Badger values need to be garbage collected, because of two reasons:
+
+- Badger keeps values separately from the LSM tree. This means that the
+ compaction operations that clean up the LSM tree do not touch the values at
+ all. Values need to be cleaned up separately.
+
+- Concurrent read/write transactions could leave behind multiple values for a
+ single key, because they're stored with different versions. These could
+ accumulate, and take up unneeded space beyond the time these older versions
+ are needed.
+
+Badger relies on the client to perform garbage collection at a time of their
+choosing. It provides the following method, which can be invoked at an
+appropriate time:
+
+- `DB.RunValueLogGC()`: This method is designed to do garbage collection while
+ Badger is online. Along with randomly picking a file, it uses statistics
+ generated by the LSM-tree compactions to pick files that are likely to lead to
+ maximum space reclamation. It's recommended to be called during periods of low
+ activity in your system, or periodically. One call would only result in
+ removal of at max one log file. As an optimization, you could also immediately
+ re-run it whenever it returns nil error (indicating a successful value log
+ GC), as shown below.
+
+ ```go
+ ticker := time.NewTicker(5 * time.Minute)
+ defer ticker.Stop()
+ for range ticker.C {
+ again:
+ err := db.RunValueLogGC(0.7)
+ if err == nil {
+ goto again
+ }
+ }
+ ```
+
+- `DB.PurgeOlderVersions()`: This method is **DEPRECATED** since v1.5.0. Now,
+ Badger's LSM tree automatically discards older/invalid versions of keys.
+
+
+ The `RunValueLogGC` method would not garbage collect the latest value log.
+
+
+## Database backup
+
+There are two public API methods `DB.Backup()` and `DB.Load()` which can be used
+to do online backups and restores. Badger v0.9 provides a CLI tool `badger`,
+which can do offline backup/restore. Make sure you have `$GOPATH/bin` in your
+PATH to use this tool.
+
+The command below creates a version-agnostic backup of the database, to a file
+`badger.bak` in the current working directory
+
+```sh
+badger backup --dir
+```
+
+To restore `badger.bak` in the current working directory to a new database:
+
+```sh
+badger restore --dir
+```
+
+See `badger --help` for more details.
+
+If you have a Badger database that was created using v0.8 (or below), you can
+use the `badger_backup` tool provided in v0.8.1, and then restore it using the
+preceding command to upgrade your database to work with the latest version.
+
+```sh
+badger_backup --dir --backup-file badger.bak
+```
+
+We recommend all users to use the `Backup` and `Restore` APIs and tools.
+However, Badger is also rsync-friendly because all files are immutable, barring
+the latest value log which is append-only. So, rsync can be used as rudimentary
+way to perform a backup. In the following script, we repeat rsync to ensure that
+the LSM tree remains consistent with the MANIFEST file while doing a full
+backup.
+
+```sh
+#!/bin/bash
+set -o history
+set -o histexpand
+# Makes a complete copy of a Badger database directory.
+# Repeat rsync if the MANIFEST and SSTables are updated.
+rsync -avz --delete db/ dst
+while !! | grep -q "(MANIFEST\|\.sst)$"; do :; done
+```
+
+## Memory usage
+
+Badger's memory usage can be managed by tweaking several options available in
+the `Options` struct that's passed in when opening the database using `DB.Open`.
+
+- Number of memtables (`Options.NumMemtables`)
+ - If you modify `Options.NumMemtables`, also adjust
+ `Options.NumLevelZeroTables` and `Options.NumLevelZeroTablesStall`
+ accordingly.
+- Number of concurrent compactions (`Options.NumCompactors`)
+- Size of table (`Options.BaseTableSize`)
+- Size of value log file (`Options.ValueLogFileSize`)
+
+If you want to decrease the memory usage of Badger instance, tweak these options
+(ideally one at a time) until you achieve the desired memory usage.
diff --git a/badger/troubleshooting.mdx b/badger/troubleshooting.mdx
new file mode 100644
index 00000000..612f0a1c
--- /dev/null
+++ b/badger/troubleshooting.mdx
@@ -0,0 +1,166 @@
+---
+title: Troubleshooting
+description: "Common issues and solutions with Badger"
+"og:title": "Troubleshooting - Badger"
+---
+
+## My writes are getting stuck
+
+**Update: with the new `Value(func(v []byte))` API, this deadlock can no longer
+happen.**
+
+The following is true for users on Badger v1.x.
+
+This can happen if a long running iteration with `Prefetch` is set to false, but
+a `Item::Value` call is made internally in the loop. That causes Badger to
+acquire read locks over the value log files to avoid value log GC removing the
+file from underneath. As a side effect, this also blocks a new value log GC file
+from being created, when the value log file boundary is hit.
+
+Please see GitHub issues [#293](https://github.com/dgraph-io/badger/issues/293)
+and [#315](https://github.com/dgraph-io/badger/issues/315).
+
+There are multiple workarounds during iteration:
+
+1. Use `Item::ValueCopy` instead of `Item::Value` when retrieving value.
+1. Set `Prefetch` to true. Badger would then copy over the value and release the
+ file lock immediately.
+1. When `Prefetch` is false, don't call `Item::Value` and do a pure key-only
+ iteration. This might be useful if you just want to delete a lot of keys.
+1. Do the writes in a separate transaction after the reads.
+
+## My writes are really slow
+
+Are you creating a new transaction for every single key update, and waiting for
+it to `Commit` fully before creating a new one? This leads to very low
+throughput.
+
+We've created `WriteBatch` API which provides a way to batch up many updates
+into a single transaction and `Commit` that transaction using callbacks to avoid
+blocking. This amortizes the cost of a transaction really well, and provides the
+most efficient way to do bulk writes.
+
+```go
+wb := db.NewWriteBatch()
+defer wb.Cancel()
+
+for i := 0; i < N; i++ {
+ err := wb.Set(key(i), value(i), 0) // Will create txns as needed.
+ handle(err)
+}
+handle(wb.Flush()) // Wait for all txns to finish.
+```
+
+Note that `WriteBatch` API doesn't allow any reads. For read-modify-write
+workloads, you should be using the `Transaction` API.
+
+## I don't see any disk writes
+
+If you're using Badger with `SyncWrites=false`, then your writes might not be
+written to value log and won't get synced to disk immediately. Writes to LSM
+tree are done inmemory first, before they get compacted to disk. The compaction
+would only happen once `BaseTableSize` has been reached. So, if you're doing a
+few writes and then checking, you might not see anything on disk. Once you
+`Close` the database, you'll see these writes on disk.
+
+## Reverse iteration doesn't give me the right results
+
+Just like forward iteration goes to the first key which is equal or greater than
+the SEEK key, reverse iteration goes to the first key which is equal or lesser
+than the SEEK key. Therefore, SEEK key would not be part of the results. You can
+typically add a `0xff` byte as a suffix to the SEEK key to include it in the
+results. See the following issues:
+[#436](https://github.com/dgraph-io/badger/issues/436) and
+[#347](https://github.com/dgraph-io/badger/issues/347).
+
+## Which instances should I use for Badger?
+
+We recommend using instances which provide local SSD storage, without any limit
+on the maximum IOPS. In AWS, these are storage optimized instances like i3. They
+provide local SSDs which clock 100K IOPS over 4KB blocks easily.
+
+## I'm getting a closed channel error
+
+```sh
+panic: close of closed channel
+panic: send on closed channel
+```
+
+If you're seeing panics like above, this would be because you're operating on a
+closed DB. This can happen, if you call `Close()` before sending a write, or
+multiple times. You should ensure that you only call `Close()` once, and all
+your read/write operations finish before closing.
+
+## Are there any Go specific settings that I should use?
+
+We _highly_ recommend setting a high number for `GOMAXPROCS`, which allows Go to
+observe the full IOPS throughput provided by modern SSDs. In Dgraph, we have set
+it to 128. For more details,
+[see this thread](https://groups.google.com/d/topic/golang-nuts/jPb_h3TvlKE/discussion).
+
+## Are there any Linux specific settings that I should use?
+
+We recommend setting `max file descriptors` to a high number depending upon the
+expected size of your data. On Linux and Mac, you can check the file descriptor
+limit with `ulimit -n -H` for the hard limit and `ulimit -n -S` for the soft
+limit. A soft limit of `65535` is a good lower bound. You can adjust the limit
+as needed.
+
+## I see "manifest has unsupported version: X (we support Y)" error
+
+This error means you have a badger directory which was created by an older
+version of badger and you're trying to open in a newer version of badger. The
+underlying data format can change across badger versions and users have to
+migrate their data directory. Badger data can be migrated from version X of
+badger to version Y of badger by following the steps listed below. Assume you
+were on badger v1.6.0 and you wish to migrate to v2.0.0 version.
+
+1. Install badger version v1.6.0
+
+ - `cd $GOPATH/src/github.com/dgraph-io/badger`
+ - `git checkout v1.6.0`
+ - `cd badger && go install`
+
+ This should install the old badger binary in your $GOBIN.
+
+2. Create Backup
+ - `badger backup --dir path/to/badger/directory -f badger.backup`
+3. Install badger version v2.0.0
+
+ - `cd $GOPATH/src/github.com/dgraph-io/badger`
+ - `git checkout v2.0.0`
+ - `cd badger && go install`
+
+ This should install new badger binary in your $GOBIN
+
+4. Restore data from backup
+
+ - `badger restore --dir path/to/new/badger/directory -f badger.backup`
+
+ This creates a new directory on `path/to/new/badger/directory` and add
+ badger data in newer format to it.
+
+NOTE - The preceding steps shouldn't cause any data loss but please ensure the
+new data is valid before deleting the old badger directory.
+
+## Why do I need gcc to build badger? Does badger need Cgo?
+
+Badger doesn't directly use Cgo but it relies on https://github.com/DataDog/zstd
+library for zstd compression and the library requires
+[`gcc/cgo`](https://pkg.go.dev/cmd/cgo). You can build Badger without Cgo by
+running `CGO_ENABLED=0 go build`. This builds Badger without the support for
+ZSTD compression algorithm.
+
+As of Badger versions
+[v2.2007.4](https://github.com/dgraph-io/badger/releases/tag/v2.2007.4) and
+[v3.2103.1](https://github.com/dgraph-io/badger/releases/tag/v3.2103.1) the
+DataDog ZSTD library was replaced by pure Golang version and Cgo is no longer
+required. The new library is
+[backwards compatible in nearly all cases](https://discuss.dgraph.io/t/use-pure-go-zstd-implementation/8670/10):
+
+
+ Yes they're compatible both ways. The only exception is 0 bytes of input which
+ gives 0 bytes output with the Go zstd. But you already have the
+ zstd.WithZeroFrames(true) which will wrap 0 bytes in a header so it can be fed
+ to DD zstd. This is only relevant when downgrading.
+
diff --git a/community-and-support.mdx b/community-and-support.mdx
index 32ded252..14e63358 100644
--- a/community-and-support.mdx
+++ b/community-and-support.mdx
@@ -19,6 +19,7 @@ relevant GitHub repository:
- [Modus](https://github.com/hypermodeinc/modus/issues)
- [Hyp CLI](https://github.com/hypermodeinc/hyp-cli/issues)
+- [Badger](https://github.com/dgraph-io/badger/issues)
All paid Hypermode packages include commercial support. Customers can reach out
via the Hypermode Console or through email at
diff --git a/mint.json b/mint.json
index a2af2870..febe6714 100644
--- a/mint.json
+++ b/mint.json
@@ -37,6 +37,10 @@
{
"name": "Modus",
"url": "modus"
+ },
+ {
+ "name": "Badger",
+ "url": "badger"
}
],
"redirects": [
@@ -177,6 +181,14 @@
{
"group": "Resources",
"pages": ["modus/changelog"]
+ },
+ {
+ "group": "Getting Started",
+ "pages": ["badger/overview", "badger/quickstart"]
+ },
+ {
+ "group": "Resources",
+ "pages": ["badger/troubleshooting", "badger/design"]
}
],
"feedback": {
diff --git a/styles/config/vocabularies/general/accept.txt b/styles/config/vocabularies/general/accept.txt
index fe45eb26..872936f5 100644
--- a/styles/config/vocabularies/general/accept.txt
+++ b/styles/config/vocabularies/general/accept.txt
@@ -49,6 +49,7 @@ CRUD
Debugf
Dgraph|dgraph
DQL
+embeddable
Errorf
GitHub|github
GraphiQL
From 7071d25d12b3c2d394d07219cfc7f0bdba497ae0 Mon Sep 17 00:00:00 2001
From: Ryan Fox-Tyler <60440289+ryanfoxtyler@users.noreply.github.com>
Date: Wed, 5 Feb 2025 09:42:18 -0500
Subject: [PATCH 2/3] repo links
---
README.md | 10 +++++-----
badger/overview.mdx | 4 ++--
badger/troubleshooting.mdx | 13 +++++++------
community-and-support.mdx | 2 +-
4 files changed, 15 insertions(+), 14 deletions(-)
diff --git a/README.md b/README.md
index f23310ef..03a35d8a 100644
--- a/README.md
+++ b/README.md
@@ -10,12 +10,12 @@ documentation for the open source projects that Hypermode leads:
- [Modus](https://github.com/hypermodeinc/modus) - serverless framework for
building functions and APIs, powered by WebAssembly
-- [Dgraph](https://github.com/dgraph-io/dgraph) - distributed, transactional
+- [Dgraph](https://github.com/hypermodeinc/dgraph) - distributed, transactional
graph database for real-time use cases
-- [Badger](https://github.com/dgraph-io/badger) - embeddable key-value store in
- Go
-- [Ristretto](https://github.com/dgraph-io/ristretto) - embeddable memory-bound
- cache in Go
+- [Badger](https://github.com/hypermodeinc/badger) - embeddable key-value store
+ in Go
+- [Ristretto](https://github.com/hypermodeinc/ristretto) - embeddable
+ memory-bound cache in Go
## Found an issue?
diff --git a/badger/overview.mdx b/badger/overview.mdx
index 9ac81c14..a01d8653 100644
--- a/badger/overview.mdx
+++ b/badger/overview.mdx
@@ -15,5 +15,5 @@ non-Go-based key-value stores like RocksDB.
## Changelog
We keep the
-[repo Changelog](https://github.com/dgraph-io/badger/blob/main/CHANGELOG.md) up
-to date with each release.
+[repo Changelog](https://github.com/hypermodeinc/badger/blob/main/CHANGELOG.md)
+up to date with each release.
diff --git a/badger/troubleshooting.mdx b/badger/troubleshooting.mdx
index 612f0a1c..7aa85abb 100644
--- a/badger/troubleshooting.mdx
+++ b/badger/troubleshooting.mdx
@@ -17,8 +17,9 @@ acquire read locks over the value log files to avoid value log GC removing the
file from underneath. As a side effect, this also blocks a new value log GC file
from being created, when the value log file boundary is hit.
-Please see GitHub issues [#293](https://github.com/dgraph-io/badger/issues/293)
-and [#315](https://github.com/dgraph-io/badger/issues/315).
+Please see GitHub issues
+[#293](https://github.com/hypermodeinc/badger/issues/293) and
+[#315](https://github.com/hypermodeinc/badger/issues/315).
There are multiple workarounds during iteration:
@@ -70,8 +71,8 @@ the SEEK key, reverse iteration goes to the first key which is equal or lesser
than the SEEK key. Therefore, SEEK key would not be part of the results. You can
typically add a `0xff` byte as a suffix to the SEEK key to include it in the
results. See the following issues:
-[#436](https://github.com/dgraph-io/badger/issues/436) and
-[#347](https://github.com/dgraph-io/badger/issues/347).
+[#436](https://github.com/hypermodeinc/badger/issues/436) and
+[#347](https://github.com/hypermodeinc/badger/issues/347).
## Which instances should I use for Badger?
@@ -152,8 +153,8 @@ running `CGO_ENABLED=0 go build`. This builds Badger without the support for
ZSTD compression algorithm.
As of Badger versions
-[v2.2007.4](https://github.com/dgraph-io/badger/releases/tag/v2.2007.4) and
-[v3.2103.1](https://github.com/dgraph-io/badger/releases/tag/v3.2103.1) the
+[v2.2007.4](https://github.com/hypermodeinc/badger/releases/tag/v2.2007.4) and
+[v3.2103.1](https://github.com/hypermodeinc/badger/releases/tag/v3.2103.1) the
DataDog ZSTD library was replaced by pure Golang version and Cgo is no longer
required. The new library is
[backwards compatible in nearly all cases](https://discuss.dgraph.io/t/use-pure-go-zstd-implementation/8670/10):
diff --git a/community-and-support.mdx b/community-and-support.mdx
index 14e63358..59d02231 100644
--- a/community-and-support.mdx
+++ b/community-and-support.mdx
@@ -19,7 +19,7 @@ relevant GitHub repository:
- [Modus](https://github.com/hypermodeinc/modus/issues)
- [Hyp CLI](https://github.com/hypermodeinc/hyp-cli/issues)
-- [Badger](https://github.com/dgraph-io/badger/issues)
+- [Badger](https://github.com/hypermodeinc/badger/issues)
All paid Hypermode packages include commercial support. Customers can reach out
via the Hypermode Console or through email at
From 4ea6b2f67ad34496c89c40943d305edd484effcb Mon Sep 17 00:00:00 2001
From: Ryan Fox-Tyler <60440289+ryanfoxtyler@users.noreply.github.com>
Date: Mon, 10 Feb 2025 19:11:45 -0800
Subject: [PATCH 3/3] .
---
.trunk/trunk.yaml | 14 +++++++-------
badger/design.mdx | 2 +-
badger/overview.mdx | 2 +-
badger/quickstart.mdx | 39 ++++++++++++++++++++------------------
badger/troubleshooting.mdx | 12 ++++++------
5 files changed, 36 insertions(+), 33 deletions(-)
diff --git a/.trunk/trunk.yaml b/.trunk/trunk.yaml
index 71bd6af7..db0d5e9b 100644
--- a/.trunk/trunk.yaml
+++ b/.trunk/trunk.yaml
@@ -2,7 +2,7 @@
version: 0.1
cli:
- version: 1.22.9
+ version: 1.22.10
plugins:
sources:
@@ -18,20 +18,20 @@ runtimes:
lint:
enabled:
- - renovate@39.156.1
+ - renovate@39.164.1
- golangci-lint@1.63.4
- - vale@3.9.4
+ - vale@3.9.5
- actionlint@1.7.7
- - checkov@3.2.361
+ - checkov@3.2.369
- git-diff-check
- markdownlint@0.44.0
- oxipng@9.1.3
- - prettier@3.4.2:
+ - prettier@3.5.0:
packages:
- "@mintlify/prettier-config@1.0.6"
- svgo@3.3.2
- - trivy@0.59.0
- - trufflehog@3.88.4
+ - trivy@0.59.1
+ - trufflehog@3.88.5
- yamllint@1.35.1
ignore:
- linters: [ALL]
diff --git a/badger/design.mdx b/badger/design.mdx
index 4589f07d..42264a2d 100644
--- a/badger/design.mdx
+++ b/badger/design.mdx
@@ -1,6 +1,6 @@
---
title: Design
-description: "Architected for fast key-value storage in Go"
+description: Architected for fast key-value storage in Go
"og:title": "Design - Badger"
---
diff --git a/badger/overview.mdx b/badger/overview.mdx
index a01d8653..c6343f92 100644
--- a/badger/overview.mdx
+++ b/badger/overview.mdx
@@ -1,6 +1,6 @@
---
title: Overview
-description: "Welcome to the Badger docs!"
+description: Welcome to the Badger docs!
mode: "wide"
"og:title": "Overview - Badger"
---
diff --git a/badger/quickstart.mdx b/badger/quickstart.mdx
index 8f39dd89..f2b7622d 100644
--- a/badger/quickstart.mdx
+++ b/badger/quickstart.mdx
@@ -1,6 +1,6 @@
---
title: Quickstart
-description: "Everything you need to get started with Badger"
+description: Everything you need to get started with Badger
"og:title": "Quickstart - Badger"
---
@@ -18,7 +18,7 @@ To start using Badger, run the following command to retrieve the library.
go get github.com/dgraph-io/badger/v4
```
-Then, install the Badger command line utility into your $GOBIN path.
+Then, install the Badger command line utility into your `$GOBIN` path.
```sh
go install github.com/dgraph-io/badger/v4/badger@latest
@@ -54,8 +54,10 @@ func main() {
if err != nil {
log.Fatal(err)
}
+
defer db.Close()
- // Your code here…
+
+ // your code here
}
```
@@ -63,9 +65,9 @@ func main() {
By default, Badger ensures all data persists to disk. It also supports a pure
in-memory mode. When Badger is running in this mode, all data remains in memory
-only. Reads and writes are much faster, but all data stored in Badger is lost in
-the case of a crash or close. To open badger in in-memory mode, set the
-`InMemory` option.
+only. Reads and writes are much faster, but Badger loses all stored data in the
+case of a crash or close. To open badger in in-memory mode, set the `InMemory`
+option.
```go
opt := badger.DefaultOptions("").WithInMemory(true)
@@ -73,11 +75,11 @@ opt := badger.DefaultOptions("").WithInMemory(true)
### Encryption mode
-If you enable encryption on Badger, you also need to set the index cache size.
+If you enable encryption in Badger, you also need to set the index cache size.
- Having a cache improves the performance. Otherwise, your reads would be very
- slow while encryption is enabled.
+ The cache improves the performance. Otherwise, reads can be very slow with
+ encryption enabled.
For example, to set a `100 Mb` cache:
@@ -94,7 +96,8 @@ To start a read-only transaction, you can use the `DB.View()` method:
```go
err := db.View(func(txn *badger.Txn) error {
- // Your code here…
+ // your code here
+
return nil
})
```
@@ -115,18 +118,18 @@ err := db.Update(func(txn *badger.Txn) error {
})
```
-All database operations are allowed inside a read-write transaction.
+Badger allows all database operations inside a read-write transaction.
Always check the returned error value. If you return an error within your
-closure it will be passed through.
+closure it's passed through.
-An `ErrConflict` error will be reported in case of a conflict. Depending on the
-state of your app, you have the option to retry the operation if you receive
-this error.
+An `ErrConflict` error is reported in case of a conflict. Depending on the state
+of your app, you have the option to retry the operation if you receive this
+error.
-An `ErrTxnTooBig` will be reported in case the number of pending writes/deletes
-in the transaction exceeds a certain limit. In that case, it's best to commit
-the transaction and start a new transaction immediately. Here is an example (we
+An `ErrTxnTooBig` is reported in case the number of pending writes/deletes in
+the transaction exceeds a certain limit. In that case, it's best to commit the
+transaction and start a new transaction immediately. Here is an example (we
aren't checking for errors in some places for simplicity):
```go
diff --git a/badger/troubleshooting.mdx b/badger/troubleshooting.mdx
index 7aa85abb..554e4ebb 100644
--- a/badger/troubleshooting.mdx
+++ b/badger/troubleshooting.mdx
@@ -4,7 +4,7 @@ description: "Common issues and solutions with Badger"
"og:title": "Troubleshooting - Badger"
---
-## My writes are getting stuck
+## Writes are getting stuck
**Update: with the new `Value(func(v []byte))` API, this deadlock can no longer
happen.**
@@ -12,7 +12,7 @@ happen.**
The following is true for users on Badger v1.x.
This can happen if a long running iteration with `Prefetch` is set to false, but
-a `Item::Value` call is made internally in the loop. That causes Badger to
+an `Item::Value` call is made internally in the loop. That causes Badger to
acquire read locks over the value log files to avoid value log GC removing the
file from underneath. As a side effect, this also blocks a new value log GC file
from being created, when the value log file boundary is hit.
@@ -30,7 +30,7 @@ There are multiple workarounds during iteration:
iteration. This might be useful if you just want to delete a lot of keys.
1. Do the writes in a separate transaction after the reads.
-## My writes are really slow
+## Writes are really slow
Are you creating a new transaction for every single key update, and waiting for
it to `Commit` fully before creating a new one? This leads to very low
@@ -64,7 +64,7 @@ would only happen once `BaseTableSize` has been reached. So, if you're doing a
few writes and then checking, you might not see anything on disk. Once you
`Close` the database, you'll see these writes on disk.
-## Reverse iteration doesn't give me the right results
+## Reverse iteration doesn't produce the right results
Just like forward iteration goes to the first key which is equal or greater than
the SEEK key, reverse iteration goes to the first key which is equal or lesser
@@ -122,7 +122,7 @@ were on badger v1.6.0 and you wish to migrate to v2.0.0 version.
- `git checkout v1.6.0`
- `cd badger && go install`
- This should install the old badger binary in your $GOBIN.
+ This should install the old badger binary in your `$GOBIN`.
2. Create Backup
- `badger backup --dir path/to/badger/directory -f badger.backup`
@@ -132,7 +132,7 @@ were on badger v1.6.0 and you wish to migrate to v2.0.0 version.
- `git checkout v2.0.0`
- `cd badger && go install`
- This should install new badger binary in your $GOBIN
+ This should install the new badger binary in your `$GOBIN`.
4. Restore data from backup