Repository navigation
feat(backup): pull Jellyfin's state off smaug from a ZFS snapshot, encrypt it here and copy it to oracle (#484) - #554
Merged
Conversation
This was referenced Sep 19, 2026
…t, encrypted here and copied to oracle — the backup three files claimed and nothing performed (#484) Three files said erebor/apps was backed up and named backup-volumes.sh as the mechanism; that script had no sentinel for the volume, cannot see smaug's Docker, and the volume was on erebor/ix-apps anyway, so the dataset §4 called backed up held the compose file and nothing Jellyfin writes. ADR-0045 settles the mechanism and the three rejections, and this builds it. The pull runs on the monitoring host over the 99 → 40:22 pass ADR-0016 wrote for exactly this and left inert: scripts/backup-nas.sh logs in as one unprivileged user, tars Jellyfin's /config out of the newest periodic snapshot of erebor/apps, streams it into age here, verifies it against a sentinel read off a boot of the pinned image (./data/jellyfin.db appears about thirty seconds into a first start; the WAL beside it is where the writes are), and copies the set to oracle by the helpers #535 landed today. Jellyfin never stops: the snapshot is the quiesce, atomic across db, -wal and -shm, and a snapshot older than two days fails the run by name rather than producing a backup that stopped being current in silence. /config becomes a bind mount on erebor/apps so the dataset the table names is the one that holds the state; the runbook migrates what the named volume already holds. backup-volumes.sh gains a source guard just before its argument parsing, so the tables, verify(), the set helpers and the far-side helpers are one library for both scripts. Sets land in backups/nas/, apart from the volume sets, because the nightly verification checks a set against the current stack's volume list; a new `make verify-backups` walks both directories and the existing timer runs it. homelab-backup-nas.timer is Saturday 03:30, jitter and replay allowed since nothing is stopped. Benched end to end against a directory on oracle over real ssh, with the copy to a bench directory there: the run, --list, --verify-only --all, --copy-only, and the four failure modes (stale snapshot, wrong zone, unreachable host, unreadable path) each behave as the runbook says. Nothing here has touched smaug, whose SSH is still off: build-the-nas.md §6.2 is the deploy checklist — snapshot task, user frodo, ACL, SSH on, host key, the tar-to-/dev/null proof, first run, timers — and the issue closes on its Done block, not on this merge. Refs #484, refs #535, refs #413. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Gerrrt
force-pushed
the
gerrrt/erebor-apps-backup-193704
branch
from
September 20, 2026 00:28
112d078 to
ca494ed
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
erebor/appswas declared backed up in three files and backed up by nothing (#484). This builds the mechanism ADR-0045 decides:scripts/backup-nas.shon the monitoring host pulls Jellyfin's/configout of the newest periodic ZFS snapshot oferebor/appsover the inert10.0.99.20 → 10.0.40.30:22pass ADR-0016 wrote for exactly this, streams it intoagehere (plaintext only inside the ssh session), verifies it against a measured sentinel, and copies the set tooracleunderbackups/nasby the helpers make backup's volume sets stay on the host they protect: ADR-0015's copy to oracle is decided and unbuilt #535 landed today. Jellyfin never stops: the snapshot is the quiesce. A snapshot older than two days fails the run by name./configbecomes a bind mount at/mnt/erebor/apps/jellyfin/config, so the dataset §4 names is the one that holds the state;jellyfin-cachestays a named volume and isDISPOSABLEby name.STACK=media backup-volumes.sh --inventorynow reports nothing to archive, which is the true answer on this host.backup-volumes.shis sourceable: one guard before its argument parsing makes the tables,verify(), the set helpers and the far-side helpers a library for both scripts.jellyfin-config's sentinel is./data/jellyfin.db, read off a boot of the pinned image (appears ~30 s into a first start; the WAL beside it is where the writes are; a clean stop checkpoints it).homelab-backup-nas.timer(Saturdays 03:30, jitter and replay allowed since nothing is stopped), a JOBS row, andmake verify-backups, which the existing nightly unit now runs so one job proves both directories, here and onoracle.age, the sentinel check and CI; encrypting on the NAS, which ADR-0016 said and gets a third NOTE clause);build-the-nas.md§4.1 (the first recorded snapshot schedule, ADR-0040 d.4), §6 migration block, §6.2 deploy checklist, §6.3 restore; the media stack's compose, env and README say what is and is not backed up;security.md,network.md,roadmap.mdsay port 22 is inert until §6.2; the schedule and handover runbooks gain the job.Why
Two documents asserting a backup no script performs is worse than no backup — the next reader has no reason to look — and it is the shape #428 found in the sensitive tier, arriving a second time. Refs #484, refs #535, refs #413.
Blast radius
smaug, by handsecrets/*.sops.yaml—frodoonsmauggets the operator's public key;oraclereceives age ciphertext to the estate's two recipientsVerification
make validatepasses;shellcheckclean on both scripts;install-timers.sh --checkpasses the 2× rule forbackup-nasoracleover real ssh, with the copy to a bench directory there: run (23 entries, sentinel present, copied and hashed on the far side),--listboth sides,--verify-only --allboth sides,--copy-only,make verify-backups(both halves run even when the first fails). Negative tests: a stale snapshot, a wrong zone (name in the future), an unreachable host, an unreadable path — each dies naming its cause. Bench directories removed on both sides; no realbackups/naswas written onoracle.smaug's SSH is off and nothing on it is reachable from the monitoring host.build-the-nas.md§6.2 is the checklist — snapshot task (§4.1), userfrodo, ACL, SSH on, host key, the tar-to-/dev/nullproof, migration of the existing volume (§6), firstmake backup-nas, thenmake install-timers. Until the first run, the nightlyverify-backupswill report no complete sets in backups/nas; §6.2 orders the first run before the reinstall for that reason. The issue closes on §6.2's Done block.🤖 Generated with Claude Code