From 91764f19bfd12c0d8b2a8a8a943b1d9e6dbc4ddc Mon Sep 17 00:00:00 2001 From: EnzinoBB Date: Sat, 7 Nov 2026 09:04:40 +0100 Subject: [PATCH 1/2] packaging: rocksdb by default for full nodes; docs and set-up texts aligned - Docker entrypoint and guided set-up (setup.sh, setup.ps1): a new full node defaults to rocksdb, which starts from a snapshot; pruned stays on lmdb. Existing installs keep their backend: a database already in db/, or the db_backend of config.ini (a full [storage] without one keeps berkeleydb, the node's own default). The entrypoint also detects the backend of a database whose config.ini was removed. - Full node disk: about 250 to 300 GB with rocksdb (entrypoint log, .env, set-up questions 1 and 3b); backfill: about 2 to 3 days (more on a slow disk or few cores). - docker-compose.yml: cpus and mem_limit from CREDITS_CPUS (1.5) and CREDITS_MEMORY (4g) in .env; a full node sets 4 and 8g. - README: linux-x64 needs glibc 2.38, linux-x64-glibc2.31 covers older systems. CHANGES intro: ghcr.io/creditscommunity/credits-node. - make-package.sh: the config.pruned.ini comment describes the snapshot start ([snapshot] fetch=auto, the default) instead of a sync from block 0. - quick-start, storage.md, Docker README updated. - ci: setup-test checks the rocksdb default and that a full berkeleydb configuration or database is kept; the Docker entrypoint test checks the rocksdb default and an existing berkeleydb install. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/setup-test.yml | 41 +++++++++++++++++++----- README.md | 2 +- docs/CHANGES.md | 2 +- docs/quick-start.md | 13 +++++--- docs/storage.md | 6 ++-- packaging/docker/.env.example | 14 ++++++-- packaging/docker/README.md | 2 +- packaging/docker/docker-compose.yml | 7 ++-- packaging/docker/entrypoint.sh | 20 +++++++++--- packaging/docker/test/test-entrypoint.sh | 33 +++++++++++++++---- packaging/make-package.sh | 6 ++-- packaging/setup/setup.ps1 | 30 +++++++++++------ packaging/setup/setup.sh | 30 +++++++++++------ 13 files changed, 150 insertions(+), 56 deletions(-) diff --git a/.github/workflows/setup-test.yml b/.github/workflows/setup-test.yml index 534d380c3..19faa1557 100644 --- a/.github/workflows/setup-test.yml +++ b/.github/workflows/setup-test.yml @@ -2,8 +2,8 @@ name: Set-up script test # The guided set-up (packaging/setup) in the published packages of Linux, Windows and macOS, used as a # user would: answers typed on stdin, the snapshot downloaded, the node started up to the mainnet -# head; then a second run that must change nothing, and a full berkeleydb configuration with a wrong -# answer first. The wallet of packaging/wallet-tag is added to the package (downloaded from its public +# head; then a second run that must change nothing, a full berkeleydb configuration with a wrong +# answer first (kept by a later run on Enter), and a full node on rocksdb, the default for full. The wallet of packaging/wallet-tag is added to the package (downloaded from its public # release and checked, packaging/fetch-wallet.sh): the set-up writes its profile (wallet.toml) and # the token cache, the wallet reads the synced node through that profile (status, the network check, # the token list from the cache, the node's own key as the account "node"); on a terminal the set-up @@ -244,9 +244,11 @@ jobs: - name: Full berkeleydb set-up with a wrong answer first run: | - # storage (wrong, then full), database, share (no), executor, wallet, token cache, start, write - bash setup-run.sh csnode-clean 'maybe\nfull\n\nno\n\nno\n\nno\n\n' 2>&1 | tee setup3.txt | tail -20 + # storage (wrong, then full), database (berkeleydb: the full default is rocksdb), share (no), + # executor, wallet, token cache, start, write + bash setup-run.sh csnode-clean 'maybe\nfull\nberkeleydb\nno\n\nno\n\nno\n\n' 2>&1 | tee setup3.txt | tail -20 grep -a "Please answer one of" setup3.txt + grep -a "Database? (rocksdb/lmdb/berkeleydb) \[rocksdb\]" setup3.txt python check-config.py csnode-clean/config.ini snapshot "serve=off fetch=off" storage "db_backend=berkeleydb" # the rest of config.ini is the package's python - <<'EOF' @@ -264,6 +266,26 @@ jobs: print("the rest of config.ini is unchanged") EOF + - name: A full berkeleydb configuration is kept on Enter + run: | + # config.ini names berkeleydb (no database yet): storage (Enter: full), database (Enter: + # berkeleydb), share (Enter: no), executor, wallet, token cache, start, write + cp -a csnode-clean csnode-bdb && rm -f csnode-bdb/config.ini.bak-* + bash setup-run.sh csnode-bdb '\n\n\n\nno\n\nno\n\n' 2>&1 | tee setup3b.txt | tail -12 + grep -a "Pruned or full node? (pruned/full) \[full\]" setup3b.txt + grep -a "Database? (rocksdb/lmdb/berkeleydb) \[berkeleydb\]" setup3b.txt + python check-config.py csnode-bdb/config.ini snapshot "serve=off fetch=off" storage "db_backend=berkeleydb" + # a berkeleydb database in a folder whose config.ini names no backend: kept too + cp -a csnode-clean csnode-bdb2 && rm -f csnode-bdb2/config.ini.bak-* + cp "$(ls csnode-clean/config.ini.bak-* | head -1)" csnode-bdb2/config.ini + mkdir -p csnode-bdb2/db && touch csnode-bdb2/db/blockchain.db csnode-bdb2/db/sequence.db + # storage (Enter: full), database (Enter: berkeleydb), share, executor, wallet, token cache, + # start, write (no first-start question: the database exists) + bash setup-run.sh csnode-bdb2 '\n\nno\n\nno\n\nno\n\n' 2>&1 | tee setup3c.txt | tail -12 + grep -a "already holds a berkeleydb database" setup3c.txt + grep -a "Database? (rocksdb/lmdb/berkeleydb) \[berkeleydb\]" setup3c.txt + python check-config.py csnode-bdb2/config.ini storage "db_backend=berkeleydb" + - name: A stray "n" never makes the node's key run: | cp -a csnode-clean csnode-n && rm -f csnode-n/config.ini.bak-* csnode-n/wallet.toml csnode-n/Node*.txt @@ -296,10 +318,13 @@ jobs: - name: Full rocksdb set-up from a snapshot, with and without the backfill run: | - cp -a csnode-clean csnode-full && rm -f csnode-full/config.ini.bak-* - # storage, database, first start (network), backfill (Enter: yes), share (no), executor, wallet, - # token cache, start, write - bash setup-run.sh csnode-full 'full\nrocksdb\nnetwork\n\nno\n\nno\n\nno\n\n' 2>&1 | tee setup6.txt | tail -14 + # the package's config.ini, kept aside by the full berkeleydb set-up + cp -a csnode-clean csnode-full && cp "$(ls csnode-clean/config.ini.bak-* | head -1)" csnode-full/config.ini + rm -f csnode-full/config.ini.bak-* + # storage, database (Enter: rocksdb, the full default), first start (network), backfill + # (Enter: yes), share (no), executor, wallet, token cache, start, write + bash setup-run.sh csnode-full 'full\n\nnetwork\n\nno\n\nno\n\nno\n\n' 2>&1 | tee setup6.txt | tail -14 + grep -a "Database? (rocksdb/lmdb/berkeleydb) \[rocksdb\]" setup6.txt grep -a "fetches the older blocks from" setup6.txt grep -a "backfill: yes" setup6.txt python check-config.py csnode-full/config.ini snapshot "serve=off fetch=auto" storage "db_backend=rocksdb backfill=on" diff --git a/README.md b/README.md index 072a6ab8d..ceaf6c1f9 100644 --- a/README.md +++ b/README.md @@ -56,7 +56,7 @@ Indicative requirements: | | Pruned node | Full node | |---|---|---| -| Operating system | Linux x86-64 (glibc 2.31 or later: Ubuntu 20.04, Debian 11 and later), Windows 10 or later (x64), macOS on Apple Silicon | same | +| Operating system | Linux x86-64: the `linux-x64` package needs glibc 2.38 or later (Ubuntu 24.04, Debian 13 and later), the `linux-x64-glibc2.31` package runs on glibc 2.31 or later (Ubuntu 20.04, Debian 11 and later); Windows 10 or later (x64); macOS on Apple Silicon | same | | CPU | 2 cores | 4 cores or more (the backfill checks signatures on several threads) | | Memory | 4 GB (8 GB recommended) | 8 GB | | Disk | about 10 GB | about 250 to 300 GB with RocksDB, 400 GB or more with LMDB or BerkeleyDB; an SSD | diff --git a/docs/CHANGES.md b/docs/CHANGES.md index e911aeabf..f05389dbe 100644 --- a/docs/CHANGES.md +++ b/docs/CHANGES.md @@ -5,7 +5,7 @@ Every pull request merged into this fork since it branched from the first community release that contains it. For what these changes mean when running a node, see [differences.md](differences.md). Each release is published as `community-v4.2.532-community.N`, with full and pruned packages and the Docker image -`ghcr.io/enzinobb/credits-node`. +`ghcr.io/creditscommunity/credits-node`. Add a line here with every merged pull request. When a release is cut, move the "Not released yet" lines under the new release. diff --git a/docs/quick-start.md b/docs/quick-start.md index f7c68c0f9..22f5acbe5 100644 --- a/docs/quick-start.md +++ b/docs/quick-start.md @@ -76,13 +76,16 @@ The node listens on 9090 and 9088 on every interface: block them in the firewall CREDITS_SNAPSHOT=github CREDITS_SERVE_SNAPSHOT=on CREDITS_API=on + CREDITS_CPUS=4 + CREDITS_MEMORY=8g # CREDITS_BACKFILL=off # uncomment to keep only the snapshot's blocks ``` - Set `CREDITS_DB_BACKEND=rocksdb` for a full node: without it the full default is `berkeleydb`, - which cannot start from a snapshot and syncs from block 0. `CREDITS_API=on` is needed by the - wallet. For a full node, also raise or remove `cpus: 1.5` and `mem_limit: 4g` in - `docker-compose.yml`. + `rocksdb` is the default for a full node; the line only makes it explicit. (`berkeleydb`, the + official node's format, cannot start from a snapshot and syncs from block 0.) `CREDITS_API=on` + is needed by the wallet. `CREDITS_CPUS` and `CREDITS_MEMORY` set the container's limits in + `docker-compose.yml`: the defaults (1.5 cores, `4g`) fit a pruned node, a full node needs 4 + cores and 8 GB. `.env` is read on the **first start only**, to write `~/.credits/config.ini`. After that, edit `config.ini`. @@ -151,7 +154,7 @@ The set-up asks a few questions. Enter keeps the default shown in brackets. | # | Question | Pruned | Full | |---|---|---|---| | 1 | Pruned or full node? | `pruned` | `full` | -| 2 | Database? | `lmdb` (default) | `rocksdb` (the default is `berkeleydb`: change it) | +| 2 | Database? | `lmdb` (default) | `rocksdb` (default) | | 3 | How should the node get the chain? | `github` (default) | `github` (default) | | 3b | Fetch the older blocks? (full only) | not asked | `yes` (default); `no` keeps only the snapshot's blocks | | 4 | Share snapshots with other nodes? | `yes` (default) | `yes` (default) | diff --git a/docs/storage.md b/docs/storage.md index 1325ad741..fc72520e8 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -123,8 +123,8 @@ Without Docker, in the folder of the full package (`config.ini`): `BACKFILL: full node with blocks N..M: fetching 0..N-1 from peers in the background`. With an empty `db/` and `[snapshot] fetch=auto` (the default), the node also takes a snapshot from -its p2p neighbours by itself at its first start. In Docker: `CREDITS_STORAGE=full` with -`CREDITS_DB_BACKEND=lmdb` or `rocksdb` (packaging/docker/README.md). +its p2p neighbours by itself at its first start. In Docker: `CREDITS_STORAGE=full`, on rocksdb +(the default for full) or with `CREDITS_DB_BACKEND=lmdb` (packaging/docker/README.md). ## Start-up check @@ -247,7 +247,7 @@ db: stop it first` (exit 3). | `.env` | Meaning | |---|---| | `CREDITS_STORAGE` | `pruned` (default) or `full`. | -| `CREDITS_DB_BACKEND` | `lmdb` (default for pruned), `berkeleydb` (default for full) or `rocksdb`. | +| `CREDITS_DB_BACKEND` | `lmdb` (default for pruned), `rocksdb` (default for full) or `berkeleydb` (full only; it syncs from block 0). An existing `config.ini` keeps the backend it names. | | `CREDITS_CONVERT_DB=yes` | Pass `--convert-db` on every start, so that a change of `db_backend` converts the database (and an interrupted conversion resumes). Remove it afterwards. | | `CREDITS_RESET=1` | Move the data aside to `~/.credits/old-` and start over. | diff --git a/packaging/docker/.env.example b/packaging/docker/.env.example index d581063e0..3dc8798b3 100644 --- a/packaging/docker/.env.example +++ b/packaging/docker/.env.example @@ -6,11 +6,13 @@ CREDITS_VERSION=latest # host directory with the data, configuration and keys CREDITS_HOME=~/.credits -# pruned (recent blocks only, starts from a snapshot) or full (every block: on lmdb or rocksdb it +# pruned (recent blocks only, starts from a snapshot) or full (every block: on rocksdb or lmdb it # starts from a snapshot and fetches the older blocks in the background; on berkeleydb it syncs from # block 0) CREDITS_STORAGE=pruned -# block database: lmdb (default for pruned), berkeleydb (default for full), rocksdb +# block database: lmdb (default for pruned), rocksdb (default for full), berkeleydb (the official +# node's format; a full node only, syncs from block 0). Read on the first start only: an existing +# config.ini keeps its db_backend #CREDITS_DB_BACKEND= # first start with an empty data directory (pruned, or full on lmdb/rocksdb): github downloads the # latest published snapshot (then the neighbours, then block 0), p2p asks only the neighbours, off @@ -34,9 +36,15 @@ CREDITS_API=off CREDITS_EXECUTOR=on # full nodes, first start: off keeps only the snapshot's blocks instead of fetching the whole history -# from the neighbours in the background (about 400 GB with lmdb, 130 GB with rocksdb, days on mainnet) +# from the neighbours in the background (on mainnet about 2 to 3 days, more on a slow disk or few +# cores, and about 250 to 300 GB with rocksdb, 400 GB or more with lmdb) #CREDITS_BACKFILL=on +# container limits (docker-compose.yml). The defaults fit a pruned node on a shared server; a full +# node needs 4 cores and 8 GB: set CREDITS_CPUS=4 and CREDITS_MEMORY=8g +#CREDITS_CPUS=1.5 +#CREDITS_MEMORY=4g + # set to 1 once to move the current data aside (~/.credits/old-) and start over, # e.g. after changing the block database; then remove it #CREDITS_RESET=1 diff --git a/packaging/docker/README.md b/packaging/docker/README.md index e20e7d69b..2417799aa 100644 --- a/packaging/docker/README.md +++ b/packaging/docker/README.md @@ -9,7 +9,7 @@ docker compose up -d && docker compose logs -f A pruned node (the default) starts from a snapshot instead of syncing from block 0 for days. On its first start, with `CREDITS_SNAPSHOT=github` (the default), the container runs `node --download-snapshot`: it downloads the latest snapshot published on GitHub, checks every file against its signed manifest and restores it. When that fails, the node looks for a snapshot among its p2p neighbours (the community nodes that serve one) and, after `fetch_wait` (10 minutes) without one, syncs from block 0. `CREDITS_SNAPSHOT=p2p` skips the GitHub download, `off` syncs from block 0. The node checks a restored snapshot's state digest at its first start and stops if it differs. -A full node on lmdb or rocksdb (`CREDITS_STORAGE=full`, `CREDITS_DB_BACKEND=lmdb` or `rocksdb`) starts from the snapshot the same way. Then it fetches the older blocks from its neighbours in the background, down to block 0 (backward sync). It takes part in the network meanwhile, and the API serves each block and history as it arrives. On mainnet that is about 186 million blocks: about two to three days, and the disk of a full node, 400 GB or more with lmdb or about 250 to 300 GB with rocksdb. `CREDITS_BACKFILL=off` keeps only the snapshot's blocks. A full node on berkeleydb (the full default) syncs from block 0, as the official node does: a berkeleydb database cannot start above block 0. See `docs/storage.md`, "Backward sync". +A full node (`CREDITS_STORAGE=full`) on rocksdb, its default, or on lmdb (`CREDITS_DB_BACKEND=lmdb`) starts from the snapshot the same way. Then it fetches the older blocks from its neighbours in the background, down to block 0 (backward sync). It takes part in the network meanwhile, and the API serves each block and history as it arrives. On mainnet that is about 186 million blocks: about 2 to 3 days (more on a slow disk or few cores), and the disk of a full node, about 250 to 300 GB with rocksdb or 400 GB or more with lmdb. `CREDITS_BACKFILL=off` keeps only the snapshot's blocks. A full node on berkeleydb (`CREDITS_DB_BACKEND=berkeleydb`, the official node's format) syncs from block 0, as the official node does: a berkeleydb database cannot start above block 0. Until community.17 the full default was berkeleydb; an existing `config.ini` keeps the backend it names. The container's limits in `docker-compose.yml` (`CREDITS_CPUS`, default 1.5, and `CREDITS_MEMORY`, default `4g`, in `.env`) fit a pruned node: for a full node set `CREDITS_CPUS=4` and `CREDITS_MEMORY=8g`. See `docs/storage.md`, "Backward sync". The node also serves its own snapshots to its neighbours (`CREDITS_SERVE_SNAPSHOT=on`, the default; at most 2 MB/s, half while it is a confidant). See `docs/snapshots.md`. diff --git a/packaging/docker/docker-compose.yml b/packaging/docker/docker-compose.yml index 42e644dd5..cafc2d617 100644 --- a/packaging/docker/docker-compose.yml +++ b/packaging/docker/docker-compose.yml @@ -19,6 +19,7 @@ services: options: max-size: 50m max-file: "2" - # what a pruned node used while syncing on a shared server; remove or raise as needed - cpus: 1.5 - mem_limit: 4g + # limits from .env: the defaults fit a pruned node on a shared server; for a full node set + # CREDITS_CPUS=4 and CREDITS_MEMORY=8g (or more) in .env + cpus: ${CREDITS_CPUS:-1.5} + mem_limit: ${CREDITS_MEMORY:-4g} diff --git a/packaging/docker/entrypoint.sh b/packaging/docker/entrypoint.sh index d683cb4b2..1a6111d16 100755 --- a/packaging/docker/entrypoint.sh +++ b/packaging/docker/entrypoint.sh @@ -6,16 +6,16 @@ # values on the first start only; afterwards it is the operator's file and the source of truth. # A node with an empty data directory first downloads the latest published snapshot # (CREDITS_SNAPSHOT=github, the default: node --download-snapshot); when that fails, the node itself -# looks for a snapshot among its p2p neighbours, then syncs from block 0. A full node on lmdb or -# rocksdb does the same, then fetches the older blocks from its neighbours in the background -# (backward sync, CREDITS_BACKFILL); a full node on berkeleydb syncs from block 0. +# looks for a snapshot among its p2p neighbours, then syncs from block 0. A full node on rocksdb (the +# full default) or lmdb does the same, then fetches the older blocks from its neighbours in the +# background (backward sync, CREDITS_BACKFILL); a full node on berkeleydb syncs from block 0. # See packaging/docker/README.md for the variables. set -euo pipefail OPT=${CREDITS_OPT:-/opt/credits} DATA=${CREDITS_DATA:-/data} STORAGE=${CREDITS_STORAGE:-pruned} -case "$STORAGE" in pruned) default_backend=lmdb ;; full) default_backend=berkeleydb ;; +case "$STORAGE" in pruned) default_backend=lmdb ;; full) default_backend=rocksdb ;; *) echo "[entrypoint] CREDITS_STORAGE must be pruned or full, not '$STORAGE'"; exit 2 ;; esac BACKEND=${CREDITS_DB_BACKEND:-$default_backend} SNAPSHOT=${CREDITS_SNAPSHOT:-github} @@ -49,6 +49,16 @@ for v in "$API" "$EXECUTOR" "$SERVE" "$BACKFILL" "$TOKENS_CACHE"; do case "$v" i [[ "$P2P_PORT" =~ ^[0-9]+$ ]] || { log "CREDITS_P2P_PORT must be a number"; exit 2; } cd "$DATA" +# a data directory with a database but no config.ini (removed by hand): without CREDITS_DB_BACKEND, +# config.ini is written for the backend already there, not for the default +if [ -z "${CREDITS_DB_BACKEND:-}" ] && [ ! -f config.ini ] && [ -d db ] && [ "$RESET" != 1 ]; then + if [ -f .storage ]; then BACKEND=$(sed -nE 's/.*backend=([a-z]+).*/\1/p' .storage) + elif [ -f db/blockchain.db ] || [ -f db/sequence.db ]; then BACKEND=berkeleydb + elif [ -f db/lmdb/data.mdb ]; then BACKEND=lmdb + elif [ -f db/CURRENT ]; then BACKEND=rocksdb + fi + case "$BACKEND" in berkeleydb|lmdb|rocksdb) ;; *) BACKEND=$default_backend ;; esac +fi # left by the restore script of earlier releases; the node cleans up its own interrupted imports rm -rf .restore-tmp .digest-before if [ -e .restore-incomplete ]; then @@ -159,7 +169,7 @@ elif [ "$SNAPSHOT" = github ]; then else download=1 # below, once the keys exist [ "$cfg_mode" = pruned ] \ - || log "a full node starts from the snapshot, then fetches the older blocks from its neighbours in the background (about 400 GB with lmdb, 130 GB with rocksdb, days on mainnet; CREDITS_BACKFILL=off keeps the snapshot's blocks only)" + || log "a full node starts from the snapshot, then fetches the older blocks from its neighbours in the background (on mainnet about 2 to 3 days, and about 250 to 300 GB with rocksdb, 400 GB or more with lmdb; CREDITS_BACKFILL=off keeps the snapshot's blocks only)" fi fi echo "mode=$cfg_mode backend=$cfg_backend" > .storage diff --git a/packaging/docker/test/test-entrypoint.sh b/packaging/docker/test/test-entrypoint.sh index 511efc61b..ed6fe5ef4 100755 --- a/packaging/docker/test/test-entrypoint.sh +++ b/packaging/docker/test/test-entrypoint.sh @@ -128,19 +128,22 @@ run "$d" -e CREDITS_SNAPSHOT=auto -e CREDITS_SNAPSHOT_URL=https://example.org/re grep -q -- '--download-snapshot https://example.org/rel/snapshot.json' "$d/stub-download" || fail "old latest.json address" pass "p2p, off, serve and the old values" -# 6b. a full node on berkeleydb (the full default) does not download and does not fetch: berkeleydb +# 6b. a full node on berkeleydb (on request) does not download and does not fetch: berkeleydb # cannot hold a database that starts above block 0 d=$(newdata 6f) -run "$d" -e CREDITS_STORAGE=full >/dev/null +run "$d" -e CREDITS_STORAGE=full -e CREDITS_DB_BACKEND=berkeleydb >/dev/null [ -e "$d/stub-download" ] && fail "a full berkeleydb node downloads" grep -q '^fetch=off' "$d/config.ini" || fail "a full berkeleydb node fetches" +grep -q '^db_backend=berkeleydb' "$d/config.ini" || fail "a full berkeleydb node: db_backend" pass "a full berkeleydb node syncs from block 0" -# 6g. a full node on lmdb or rocksdb starts from the snapshot and backfills (backward sync); -# CREDITS_BACKFILL=off writes backfill=off -for b in lmdb rocksdb; do +# 6g. a full node on rocksdb (the full default, "default") or lmdb starts from the snapshot and +# backfills (backward sync); CREDITS_BACKFILL=off writes backfill=off +for b in default lmdb rocksdb; do d=$(newdata "6g-$b") - out=$(run "$d" -e CREDITS_STORAGE=full -e CREDITS_DB_BACKEND=$b) + if [ "$b" = default ]; then out=$(run "$d" -e CREDITS_STORAGE=full); b=rocksdb + else out=$(run "$d" -e CREDITS_STORAGE=full -e CREDITS_DB_BACKEND=$b); fi + grep -q "^db_backend=$b" "$d/config.ini" || fail "a full $b node: db_backend" grep -q -- '--download-snapshot' "$d/stub-download" 2>/dev/null || { echo "$out"; fail "a full $b node does not download"; } grep -q 'fetches the older blocks from its neighbours' <<<"$out" || { echo "$out"; fail "a full $b node: no backfill note"; } grep -q '^fetch=auto' "$d/config.ini" || fail "a full $b node does not fetch" @@ -153,6 +156,24 @@ run "$d" -e CREDITS_STORAGE=full -e CREDITS_DB_BACKEND=lmdb -e CREDITS_BACKFILL= grep -q '^backfill=off' "$d/config.ini" || fail "CREDITS_BACKFILL=off not written" pass "a full lmdb or rocksdb node starts from the snapshot and backfills" +# 6h. an existing full berkeleydb install keeps berkeleydb: a config.ini without db_backend (the +# node's own default) and, with config.ini removed, a berkeleydb database already in db/ +d=$(newdata 6h) +run "$d" -e CREDITS_STORAGE=full -e CREDITS_DB_BACKEND=berkeleydb >/dev/null +sed -i '/^db_backend=/d' "$d/config.ini" +mkdir -p "$d/db"; touch "$d/db/blockchain.db" "$d/db/sequence.db" +rm -f "$d/stub-download" +set +e; out=$(run "$d" -e CREDITS_STORAGE=full); rc=$?; set -e +[ "$rc" = 0 ] || { echo "$out"; fail "an old full berkeleydb install: exit $rc"; } +grep -q 'mode=full backend=berkeleydb' "$d/.storage" || fail "an old full berkeleydb install: record" +grep -q 'warning: CREDITS_DB_BACKEND' <<<"$out" && fail "an old full berkeleydb install: backend warning" +rm -f "$d/config.ini" "$d/.storage" +set +e; out=$(run "$d" -e CREDITS_STORAGE=full); rc=$?; set -e +[ "$rc" = 0 ] || { echo "$out"; fail "a berkeleydb database without config.ini: exit $rc"; } +grep -q '^db_backend=berkeleydb' "$d/config.ini" || fail "a berkeleydb database without config.ini: db_backend" +[ -e "$d/stub-download" ] && fail "an old full berkeleydb install downloads" +pass "an existing full berkeleydb install keeps berkeleydb" + # 6c. a snapshot mismatch reported by the node: the next start is refused d=$(newdata 6m) run "$d" -e CREDITS_SNAPSHOT=off >/dev/null diff --git a/packaging/make-package.sh b/packaging/make-package.sh index 249458f7b..46332f84b 100755 --- a/packaging/make-package.sh +++ b/packaging/make-package.sh @@ -87,8 +87,10 @@ cp "$dir/config.ini" "$dir/config.pruned.ini" cat >> "$dir/config.pruned.ini" <<'INI' [storage] -; pruned storage: the node syncs from block 0 by itself and keeps only recent blocks -; (from the last checkpoint but one), in an LMDB database that reuses freed space +; pruned storage: only recent blocks are kept (from the last checkpoint but one), in an LMDB +; database that reuses freed space. On an empty database the node starts from a snapshot: the +; published one with --download-snapshot, otherwise one from its neighbours ([snapshot] fetch=auto, +; the default); it syncs from block 0 only when no snapshot comes within fetch_wait (10 minutes) checkpoint_every=100000 checkpoint_keep=2 prune_keep_blocks=1000 diff --git a/packaging/setup/setup.ps1 b/packaging/setup/setup.ps1 index fc6e0e455..250913d08 100644 --- a/packaging/setup/setup.ps1 +++ b/packaging/setup/setup.ps1 @@ -181,7 +181,8 @@ if (($curBackend + $curPrune) -and (-not $curPrune -or $curPrune -eq '0')) { $st if ($have -eq 'berkeleydb') { $storageDefault = 'full' } Write-Host '1. Storage' Write-Host ' pruned: keeps only recent blocks (a few GB), starts in minutes from a snapshot. Enough to run a node.' -Write-Host ' full: keeps every block since the start of the chain (tens of GB), for history and explorers.' +Write-Host ' full: keeps every block since the start of the chain (about 250 to 300 GB with rocksdb),' +Write-Host ' for history and explorers.' $storage = Ask ' Pruned or full node? (pruned/full)' $storageDefault @('pruned', 'full') $prunedToFull = $false if ($storage -eq 'full' -and $have -and $curPrune -and $curPrune -ne '0') { @@ -199,13 +200,24 @@ if ($storage -eq 'pruned') { Write-Host ' rocksdb: compressed, smaller on disk, a little more CPU.' $backend = Ask ' Database? (lmdb/rocksdb)' $backendDefault @('lmdb', 'rocksdb') } else { - $backendDefault = 'berkeleydb' - if (($curBackend -eq 'lmdb' -or $curBackend -eq 'rocksdb') -and (-not $curPrune -or $curPrune -eq '0')) { $backendDefault = $curBackend } + # rocksdb for a new full node; a full config.ini keeps its backend, and an existing database + # keeps its own + $backendDefault = 'rocksdb' + if (-not $curPrune -or $curPrune -eq '0') { + switch ($curBackend) { + { $_ -in @('lmdb', 'rocksdb', 'berkeleydb') } { $backendDefault = $curBackend } + 'rocks' { $backendDefault = 'rocksdb' } + 'berkeley' { $backendDefault = 'berkeleydb' } + } + # a full [storage] without db_backend runs the node's own default, berkeleydb + if (-not $curBackend -and $curPrune -eq '0') { $backendDefault = 'berkeleydb' } + } if ($have) { $backendDefault = $have } - Write-Host " berkeleydb: the official node's format (the official node can open this folder too)." - Write-Host ' rocksdb: the smallest full node (about a third of berkeleydb).' - Write-Host ' lmdb: fast, larger on disk.' - $backend = Ask ' Database? (berkeleydb/rocksdb/lmdb)' $backendDefault @('berkeleydb', 'rocksdb', 'lmdb') + Write-Host ' rocksdb: the smallest full node (about a third of berkeleydb); starts from a snapshot.' + Write-Host ' lmdb: fast, larger on disk; starts from a snapshot.' + Write-Host " berkeleydb: the official node's format (the official node can open this folder too);" + Write-Host ' it cannot start from a snapshot and syncs from block 0.' + $backend = Ask ' Database? (rocksdb/lmdb/berkeleydb)' $backendDefault @('rocksdb', 'lmdb', 'berkeleydb') } Write-Host '' @@ -247,8 +259,8 @@ if ($storage -eq 'full' -and ($first -eq 'github' -or $first -eq 'network' -or $ Write-Host '3b. History (backfill)' Write-Host ' yes: fetch every older block down to block 0 in the background, each checked against the chain;' Write-Host ' the node runs and takes part in the network meanwhile. On mainnet about 186 million blocks:' - Write-Host ' one to five days, and the disk of a full node (about 130 GB or more with rocksdb, 400 GB' - Write-Host ' with lmdb or berkeleydb).' + Write-Host ' about 2 to 3 days (more on a slow disk or few cores), and the disk of a full node (about' + Write-Host ' 250 to 300 GB with rocksdb, 400 GB or more with lmdb or berkeleydb).' Write-Host ' no: keep only the blocks from the snapshot on (a few GB); the history before is not served.' $backfill = Ask ' Fetch the older blocks? (yes/no)' $backfillDefault @('yes', 'no') Write-Host '' diff --git a/packaging/setup/setup.sh b/packaging/setup/setup.sh index f66485620..1c2f24c47 100755 --- a/packaging/setup/setup.sh +++ b/packaging/setup/setup.sh @@ -212,7 +212,8 @@ if [ -n "$cur_backend$cur_prune" ] && { [ -z "$cur_prune" ] || [ "$cur_prune" = [ "$have" != berkeleydb ] || storage_default=full echo "1. Storage" echo " pruned: keeps only recent blocks (a few GB), starts in minutes from a snapshot. Enough to run a node." -echo " full: keeps every block since the start of the chain (tens of GB), for history and explorers." +echo " full: keeps every block since the start of the chain (about 250 to 300 GB with rocksdb)," +echo " for history and explorers." ask " Pruned or full node? (pruned/full)" "$storage_default" pruned full storage=$answer pruned_to_full=no @@ -232,14 +233,25 @@ if [ "$storage" = pruned ]; then echo " rocksdb: compressed, smaller on disk, a little more CPU." ask " Database? (lmdb/rocksdb)" "$backend_default" lmdb rocksdb else - backend_default=berkeleydb - case "$cur_backend" in lmdb|rocksdb) [ "$cur_prune" != "" ] && [ "$cur_prune" != 0 ] || backend_default=$cur_backend ;; esac + # rocksdb for a new full node; a full config.ini keeps its backend, and an existing database + # keeps its own + backend_default=rocksdb + if [ -z "$cur_prune" ] || [ "$cur_prune" = 0 ]; then + case "$cur_backend" in + lmdb|rocksdb|berkeleydb) backend_default=$cur_backend ;; + rocks) backend_default=rocksdb ;; + berkeley) backend_default=berkeleydb ;; + # a full [storage] without db_backend runs the node's own default, berkeleydb + "") [ "$cur_prune" != 0 ] || backend_default=berkeleydb ;; + esac + fi [ -z "$have" ] || backend_default=$have echo "2. Database" - echo " berkeleydb: the official node's format (the official node can open this folder too)." - echo " rocksdb: the smallest full node (about a third of berkeleydb)." - echo " lmdb: fast, larger on disk." - ask " Database? (berkeleydb/rocksdb/lmdb)" "$backend_default" berkeleydb rocksdb lmdb + echo " rocksdb: the smallest full node (about a third of berkeleydb); starts from a snapshot." + echo " lmdb: fast, larger on disk; starts from a snapshot." + echo " berkeleydb: the official node's format (the official node can open this folder too);" + echo " it cannot start from a snapshot and syncs from block 0." + ask " Database? (rocksdb/lmdb/berkeleydb)" "$backend_default" rocksdb lmdb berkeleydb fi backend=$answer echo @@ -292,8 +304,8 @@ if [ "$storage" = full ] && { [ "$first" = github ] || [ "$first" = network ] || echo "3b. History (backfill)" echo " yes: fetch every older block down to block 0 in the background, each checked against the chain;" echo " the node runs and takes part in the network meanwhile. On mainnet about 186 million blocks:" - echo " one to five days, and the disk of a full node (about 130 GB or more with rocksdb, 400 GB" - echo " with lmdb or berkeleydb)." + echo " about 2 to 3 days (more on a slow disk or few cores), and the disk of a full node (about" + echo " 250 to 300 GB with rocksdb, 400 GB or more with lmdb or berkeleydb)." echo " no: keep only the blocks from the snapshot on (a few GB); the history before is not served." ask " Fetch the older blocks? (yes/no)" "$backfill_default" yes no backfill=$answer From eaa6746509f92655ed5b1e9c10ec2779b835e6ec Mon Sep 17 00:00:00 2001 From: EnzinoBB Date: Sat, 7 Nov 2026 09:05:12 +0100 Subject: [PATCH 2/2] docs: CHANGES row for #173 Co-Authored-By: Claude Opus 5.5 --- docs/CHANGES.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/CHANGES.md b/docs/CHANGES.md index f05389dbe..9e146632c 100644 --- a/docs/CHANGES.md +++ b/docs/CHANGES.md @@ -45,6 +45,7 @@ lines under the new release. | PR | Change | |---|---| | #172 | docs: `docs/quick-start.md`, a short quick-start guide (Docker and native Linux, Windows and macOS; pruned on lmdb or full on rocksdb from the snapshot, snapshot serving on; requirements, ports, first checks, the wallet, upgrading with the database kept, common problems), linked from the top of the README | +| #173 | packaging: a new full node defaults to rocksdb (it starts from a snapshot) in Docker and in the guided set-up (setup.sh, setup.ps1); pruned stays on lmdb; existing installs keep their backend (the database in `db/`, the `db_backend` of `config.ini`, berkeleydb for a full `[storage]` without one; the entrypoint also detects the database of a data folder whose `config.ini` was removed); `docker-compose.yml` takes its limits from `CREDITS_CPUS` (1.5) and `CREDITS_MEMORY` (4g) in `.env` (a full node: 4 and 8g); texts aligned: a full node on rocksdb about 250 to 300 GB, backfill about 2 to 3 days, glibc 2.38 for `linux-x64` (2.31 for `-glibc2.31`) in the README, the image `ghcr.io/creditscommunity/credits-node`, the `config.pruned.ini` comment on the snapshot start; quick-start, storage.md and the Docker README updated; ci: setup-test and the Docker entrypoint test check the rocksdb default and that a full berkeleydb install is kept | ## community.17 — 2026-11-05