Skip to content

feat: bitkit staging without blocktank - #36

Merged
ovitrif merged 7 commits into
mainfrom
feat/bitkit-staging
Nov 9, 2026
Merged

ovitrif merged 7 commits into
mainfrom
feat/bitkit-staging

Conversation

@ovitrif

@ovitrif ovitrif commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Refs:

Description

  • Adds staging/, a deployable public regtest staging with no Blocktank part, so Bitkit's staging builds and E2E runs stop depending on *.stag0.blocktank.to. It goes with the Blocktank sunset, and its outage broke every new staging wallet and the E2E runs.
  • It runs VSS and LNURL-auth (the existing vss-server submodule and synonymdev/lnurl-auth-server), a regtest bitcoind, mempool (frontend, backend, its electrs fork) as explorer, Esplora REST and Electrum, the fx-rates relay and the RN backup server, behind one Caddy. Caddy also serves Electrum over TLS on 9999.
  • Keeps stag0's paths (/vss_rs_auth, /lnurl_auth/auth, /fx/rates/btc, /backups-ldk, /electrs, mempool's /api), so an app or test moves by changing the host.
  • Adds a small faucet and miner with Blocktank's /regtest/chain/deposit and /regtest/chain/mine contract (with or without the /blocktank/api/v2 prefix), so the E2E scripts only change BLOCKTANK_URL.
  • The faucet requires HTTP basic auth (FAUCET_AUTH, generated by init.sh, kept by callers as a CI secret) on deposit and mine, refuses to start without it, and rate-limits each client (FAUCET_RATE_PER_MIN) on top of per-call caps. stag0's open mine endpoint is what stalled its chain, Electrum and VSS.
  • init.sh generates the passwords and keys on the deploy host. The repository holds no credentials, and only Caddy publishes ports.
  • Adds docs/staging.md with what the five Bitkit repositories still call on stag0, which of those must outlive Blocktank, the settings to change, a migration checklist for a fresh build that reuses nothing from stag0 (new hostnames, credentials, chain and data; staging wallets start fresh) and the deploy and smoke steps.
  • Spark is not hosted here: it stays on Lightspark's hosted regtest, which runs its own chain. docs/staging.md notes that split as an open call.

Out of Scope

  • Spark: stays on Lightspark's hosted regtest; the local Spark stack is a separate profile.
  • Blocktank LSP endpoints (orders, info, push and wake, regtest/channel/pay), RGS and scorer: they go with Blocktank and LDK.
  • App, test and bitkit-core setting changes: listed in docs/staging.md, made in their own repositories once DevOps picks the hostnames.
  • The root docker-compose.yml and its services: unchanged.

Design

N/A — no UI changes.

Preview

N/A — no user-visible changes.

QA Notes

Journeys

N/A

Manual Tests

  • cd staging && ./init.sh, set the hostnames in .env, docker compose up -d --build → all services up, faucet logs faucet ready — deploy host with DNS and ports 80, 443, 9999
  • smoke checks in docs/staging.md against the deployed hosts → each answers — needs the public deployment

Automated Checks

  • ran the stack locally with Caddy's own CA (CADDY_GLOBAL_OPTIONS=local_certs):
    • bitkit-e2e-tests' fund-address.sh and mine-blocks.sh with BLOCKTANK_URL on the faucet, with and without the /blocktank/api/v2 prefix (plain-text txid, 2 blocks mined; bad address and over-cap counts give 400)
    • Esplora blocks/tip/height, address/…/utxo, tx/…/raw, tx/…/merkle-proof and block-height under /electrs and on the mempool host
    • mempool UI, /api/v1/fees/recommended and /api/blocks/tip/height
    • Electrum server.version and blockchain.headers.subscribe over TLS on 9999
    • /fx/rates/btc relay
    • /backups-ldk (same response as stag0 for an unsigned challenge)
    • LNURL-auth challenge → signed callback → JWT → VSS putObjects and getObject through /vss_rs_auth; a bad token gives 401
    • faucet auth and limits: no or wrong credentials give 401, right ones 200, past the per-minute limit 429 while another client still gets 200; the faucet refuses to start without FAUCET_AUTH; /health stays open
    • docker compose down and up: VSS data, chain and faucet wallet persist

Adds staging/, a deployable public regtest staging with VSS, LNURL-auth, bitcoind, mempool with its electrs, a faucet and miner at Blocktank's regtest paths, the fx-rates relay and the RN backup server behind one Caddy, keeping stag0's paths. docs/staging.md has the stag0 inventory, settings, migration checklist and deploy steps.
@ovitrif ovitrif self-assigned this Nov 9, 2026
@ovitrif
ovitrif requested review from a team, ben-kaufman and pwltr and removed request for a team November 9, 2026 14:03
@ovitrif

ovitrif commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator Author

Pushed b113492: the faucet now takes optional HTTP basic auth (FAUCET_AUTH, generated by init.sh) on deposit and mine, since an open mine endpoint is what flooded stag0's chain and DevOps is closing it there the same way. docs/staging.md adds the callers' secret to the settings and a fresh-chain option to the migration checklist.

@ovitrif

ovitrif commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator Author

Pushed e9a9b23:

  • The faucet now requires FAUCET_AUTH (no open mode) and rate-limits each client, on top of the per-call caps.
  • docs/staging.md now plans a fresh build that reuses nothing from stag0: new hostnames, credentials, chain and data, with staging wallets starting fresh. It replaces the earlier plan to copy stag0's data and keep its names.

@talosmachina talosmachina left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No findings. Adds staging/, a self-contained regtest staging (VSS, LNURL-auth, bitcoind, mempool and electrs, an authenticated faucet and miner at Blocktank's regtest paths, fx relay, RN backups) behind one Caddy, plus docs/staging.md. Reviewed e9a9b23, full tier, static.

What I checked, and 5 candidates I ruled out

Read in full: staging/compose.yml, staging/Caddyfile, staging/faucet/server.mjs, staging/init.sh, staging/.env.example, both Dockerfiles, docs/staging.md, the README and CHANGELOG hunks
Cross-checked: vss/Dockerfile and the root compose's VSS_JWT_PUBLIC_KEY_FILE usage; the rnBackupServerHost callers in bitkit-android and bitkit-ios for the /backups-ldk path
CI: none on this repository
Not run here: my sandboxed runner could not confirm the author's permission and refused to run the branch, so nothing was brought up; the review is from the code

Ruled out

  • Rate limit keyed on a spoofed X-Forwarded-For: Caddy only trusts an incoming X-Forwarded-For from configured trusted_proxies, none are set, so it overwrites the header with the peer address.
  • Unauthenticated body or limiter growth: the auth check runs before readJson and before limited, so an unauthenticated caller neither reaches the body parser nor fills the hits map.
  • bitcoind rpcallowip=0.0.0.0/0, whitelist=0.0.0.0/0: no port is published for it; only Caddy publishes 80, 443 and 9999.
  • Faucet amounts: Number.isInteger and the caps reject fractions and overflow, and bitcoind's invalid-address error (-5) maps to 400.
  • init.sh LDK key extraction: ran the private-key pipeline 1,500 times on OpenSSL 3.0, always 64 hex characters.

Merge confidence: 4/5, a deploy stack and a cross-repo contract with no CI, reviewed without bringing it up.

@ovitrif
ovitrif requested a review from mdozhdev November 9, 2026 14:36
Comment thread README.md Resolved
@ovitrif
ovitrif requested a review from ben-kaufman November 9, 2026 14:46

@pwltr pwltr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed commit 9dbf50f. The new staging/ stack replaces stag0 with a public regtest backend behind one Caddy: only Caddy publishes ports, images are pinned, every password and key is generated on the deploy host, and the faucet requires basic auth and a per-client rate limit. The README now covers the clone layout the build expects and a health check.

Low — init.sh can write an empty backup public key that a rerun never repairs (staging/init.sh:27).

The public key is written with echo "LDK_BACKUP_PUBLIC_KEY=$(… | xxd -p -c 33)" after the secret key is saved. A failing command inside that substitution does not stop the set -euo pipefail script, so on a host without xxd the line is written empty and the script still reports ready; a rerun skips the block because the secret key is set, and the listed prerequisites do not mention xxd. Checking for xxd up front, deriving the hex with od, or failing on an empty value would avoid it.

Review details

General review
Model: Claude Opus 5.5
Reasoning effort: high

@ovitrif

ovitrif commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator Author

Low — init.sh can write an empty backup public key that a rerun never repairs (staging/init.sh:27).

@pwltr Fixed in 57c11c0: init.sh now reads both keys from the DER output with od (no xxd), checks their lengths before writing either, and stops on a rerun when the public key is missing, saying how to regenerate the pair. Checked: five fresh runs give matching key pairs, a rerun passes, and an emptied public key stops the script.

@ovitrif
ovitrif requested a review from pwltr November 9, 2026 14:57
@ovitrif

ovitrif commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator Author

Merged main into this PR at 5dcc360 to resolve its one conflict: CHANGELOG.md, where #37 added its Spark line next to ours. Both lines stay; nothing under staging/ changed.

@ovitrif
ovitrif requested a review from ben-kaufman November 9, 2026 16:42

@pwltr pwltr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked commit 5dcc360, a merge of main. Its only conflict, in the changelog, keeps both entries, and this PR's own changes are unchanged since 57c11c0 and stay clear of the new Spark regtest profile. No issues.

Review details

General review
Model: Claude Opus 5.5
Reasoning effort: high

@ovitrif

ovitrif commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator Author

Merging with a merge commit: @pwltr approved the current head and no review threads are open.

@ovitrif
ovitrif merged commit 694c0e4 into main Nov 9, 2026
@ovitrif
ovitrif deleted the feat/bitkit-staging branch November 9, 2026 18:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants