Repository navigation
csdb: LMDB block database for pruned storage - #26
Merged
Merged
Conversation
On mainnet the first pruning cycle of a node syncing from block 0 (blocks before 99'000 removed at the 200'000 checkpoint) terminated the node: compacting the 600 MB blockchain.db in one Db::compact call needed more of the environment's buffer cache than it holds, and BerkeleyDB threw (BDB3017 unable to allocate space from the buffer cache; terminate called after throwing an instance of 'DbException', what(): Db::compact: Cannot allocate memory). DatabaseBerkeleyDB::compact now works through each database 64 pages per call, resuming from the key the previous call stopped at, and turns any exception into an error result: compaction only returns space to the file system (BerkeleyDB reuses freed pages anyway), so pruneStep logs the failure and the node carries on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BerkeleyDB's default buffer cache (about 64 pages of 4 KB) is usually full of freshly written pages while a node syncs, and Db::compact then cannot allocate pages even in small steps (seen on mainnet at the 300'000 checkpoint). Nothing is wrong with the storage: compaction only returns space, and the pages freed by pruning are reused for new blocks (blockchain.db grew from 207 to 232 MB while 100'000 blocks were added and 100'000 removed). Log it as a warning instead of setting the storage error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On mainnet a pruned node syncing fast terminated in Db::del: BerkeleyDB's default buffer cache was exhausted (BDB3017) and the exception escaped DatabaseBerkeleyDB::remove. remove now turns any exception into an error result, and pruneStep keeps a block it could not remove and retries it on the next step instead of counting it as pruned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On mainnet a pruned node's blockchain.db kept growing even though the blocks it keeps stayed the same: BerkeleyDB's Recno keeps a placeholder for every removed record, so the pages of pruned blocks are never freed (db_stat on a copy: 104'123 records on 284'131 pages, 8% full, nothing on the free list), and compacting such a file exhausts the environment's locks and log region even with larger limits. DatabaseLmdb implements the same csdb::Database contract on LMDB, which the node already uses for its caches: blocks by 0-based sequence (MDB_INTEGERKEY), a hash -> sequence index and contract data, one transaction per operation, MDB_NOTLS (transactions are not tied to threads) and MDB_NOSYNC (as BerkeleyDB's DB_TXN_NOSYNC). LMDB frees the pages of removed records and reuses them, so the file stays the size of the blocks kept. [storage] block_db=lmdb selects it (default berkeleydb); the database lives in <db path>/lmdb, so an existing BerkeleyDB database is left alone. Nodes without the option are unchanged. Tests DatabaseLmdb.StoresFindsRemovesAndIterates and DatabaseLmdb.ReusesTheSpaceOfRemovedBlocks (ten windows of blocks written and pruned keep the file under twice one window); nodetests 136/136. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On mainnet, a pruned node's
blockchain.db(BerkeleyDB) kept growing while the number of blocks it keeps stayed the same. Between 1.5 and 3 million blocks it went from ~1.0 to ~2.0 GB.BerkeleyDB's Recno keeps a placeholder for every removed record, so the pages of pruned blocks are never freed.
db_staton a copy showed 104,123 records on 284,131 pages, 8% full, with nothing on the free list.Compaction does not fix it. On that file it exhausted the environment's locks and log region on the first step, even with 200k locks and a 4 MB log region. A run left alone for 70 minutes wrote ~4 GB of logs and the file grew.
Fix
DatabaseLmdb. It implements the samecsdb::Databasecontract on LMDB, which the node already uses for its caches. It stores:MDB_INTEGERKEY);It runs one transaction per operation, with
MDB_NOTLS(transactions are not tied to threads) andMDB_NOSYNC(as BerkeleyDB'sDB_TXN_NOSYNC). LMDB frees the pages of removed records and reuses them.Engine selection.
[storage] block_db=lmdbselects it; the default staysberkeleydb. The database lives in<db path>/lmdb, so an existing BerkeleyDB database is left alone. Nodes without the option are unchanged.Stacked on csdb: compaction of pruned storage can no longer terminate the node #25. This branch builds on csdb: compaction of pruned storage can no longer terminate the node #25: compaction and block removal can no longer terminate the node, and a block that could not be removed is retried. The diff therefore includes csdb: compaction of pruned storage can no longer terminate the node #25's commits.
Verification
DatabaseLmdb.StoresFindsRemovesAndIterates: the storage contract, reopening included.DatabaseLmdb.ReusesTheSpaceOfRemovedBlocks: ten windows of blocks written and pruned keep the file under twice one window.nodetests136/136.prune_keep_blocks=1000):The node's whole data (blocks, caches, checkpoints, logs) is 504 MB at block 2,979,150, with blocks before 2,799,000 pruned.
🤖 Generated with Claude Code