diff --git a/README.md b/README.md index 970114b2..039faf41 100644 --- a/README.md +++ b/README.md @@ -68,13 +68,19 @@ documents for different readers. `promtool`, `amtool`, `alloy fmt`, a real Loki boot to parse the LogQL rules, dashboard-JSON and datasource checks, every dashboard's PromQL parsed, plus `gitleaks` over the full history. -- **CI that validates the documentation too.** Six assertions cross-check - this prose against the configs it describes — rule and panel counts, the - SNMP inventory against `docs/network.md`, the host/stack and ports tables - against `compose.yaml`, and a ban on image versions in prose — Dependabot - edits only `compose.yaml`, so a version written anywhere else is stale - from the next bump. A document that disagrees with the repository fails - the build. +- **CI that validates the documentation too.** Ten assertions cross-check + this prose against the configs it describes — counted claims (rules, + dashboards, panels, Alloy agents, VLANs, ADRs, runbooks), the SNMP + inventory and the compute table against `docs/network.md`, the host/stack + and ports tables against `compose.yaml`, ADR numbering, firewall posture + against `docs/firewall-claims.yaml`, guest rows against each other, this + file's outstanding-purchase count against the roadmap's buy table, and a + ban on image versions in prose — Dependabot edits only `compose.yaml`, so + a version written anywhere else is stale from the next bump. A document + that disagrees with the repository fails the build. That opening count is + now one of the claims, read from the check registry rather than kept by + hand: it said six while ten ran, and the list beside it had been overtaken + by four — drift in the sentence advertising that drift gets caught. - **Supply chain pinned by digest.** Every image carries both a tag and a `sha256:` digest, so a moved tag cannot change what deploys. CI enforces it; `make pin-digests` re-resolves them from the registry. Every `docker run` in @@ -332,7 +338,9 @@ Open work is tracked in why it is in that order. The current top items: rehearse the firewall restore on the ProDesk bought on -2026-09-08, and then build the sensitive tier on that same box ([#404](https://github.com/Gerrrt/HomeLab/issues/404), [ADR-0034](adr/0034-run-the-sensitive-tier-on-the-prodesk-and-make-it-the-spare-hardware.md)). **Every purchase still outstanding is in one place**, the +2026-09-08 — on hand since 2026-09-15, so what the rehearsal waits on is the +I226 card and the installer stick and no longer the box — and then build the +sensitive tier on that same box ([#404](https://github.com/Gerrrt/HomeLab/issues/404), [ADR-0034](adr/0034-run-the-sensitive-tier-on-the-prodesk-and-make-it-the-spare-hardware.md)). **Every purchase still outstanding is in one place**, the roadmap's [*Everything still to buy*](docs/roadmap.md#everything-still-to-buy): two items now, one later, and a rule that nothing joins them without a decision. This sentence used to carry the list itself, name three purchases diff --git a/docs/hardware.md b/docs/hardware.md index 7f599e06..2b7d6a6b 100644 --- a/docs/hardware.md +++ b/docs/hardware.md @@ -102,8 +102,9 @@ revisions of this repository treated `shiva` as the hypervisor itself. 2026-09-08, carrying the unmanaged switch that feeds `prometheus` and `oracle` ([#110](https://github.com/Gerrrt/HomeLab/issues/110)) - HP ProDesk 600 G4 Micro — i5-8500T, 32 GB, 512 GB SSD, the same model as - `morpheus` — ordered 2026-09-08, in transit. This is `trinity`, and it is - **one box with two jobs, not two boxes**: the sensitive tier's host, and + `morpheus` — ordered 2026-09-08, **in hand since 2026-09-15** and not yet + opened. This is `trinity`, and it is **one box with two jobs, not two + boxes**: the sensitive tier's host, and the firewall's spare hardware in a disaster ([ADR-0034](adr/0034-run-the-sensitive-tier-on-the-prodesk-and-make-it-the-spare-hardware.md)). The documents call it "the tier's host" in one place and "the spare" in @@ -113,7 +114,13 @@ revisions of this repository treated `shiva` as the hypervisor itself. [#404](https://github.com/Gerrrt/HomeLab/issues/404) builds it, after the firewall restore has been rehearsed on it. It ships with the onboard NIC only; the I226 card the restore depends on was a separate purchase, made - 2026-09-11 and the entry below. + 2026-09-11 and the entry below — which has **not** landed. So what the + rehearsal waits on is no longer this box: it is that card and the installer + stick. **Checked on arrival, not assumed:** the i5-8500T, the 32 GB and the + 512 GB above are the listing's claims and not yet the machine's; the serial + and the product number off the case; and the question the entry below turns + on — whether that 512 GB SSD is an M.2 stick or a 2.5" drive in a carrier. + Those go here when it is opened. - Intel I226-V 2.5 GbE card on an M.2 B+M-key adapter — bought 2026-09-11, in transit. The second port on the ProDesk Micro above, matching the card in `morpheus` so a pfSense restore onto the spare brings the LAN up as @@ -125,8 +132,11 @@ revisions of this repository treated `shiva` as the hypervisor itself. adapter comes up as `ure0`, which is the one thing a restore must not be asked about. It goes in the G4 Micro's second M.2 slot — **and if that machine's 512 GB SSD turns out to be a 2.5" drive rather than M.2, the - drive carrier and this card contend for the same space.** Checked on - arrival, not assumed. Part number and the port's MAC go here when it lands. + drive carrier and this card contend for the same space.** That question + stopped waiting on this card when the G4 Micro landed on 2026-09-15: + opening the box answers it, and answering it before the card arrives is + the difference between a return and a part that fits nothing. Part number + and the port's MAC go here when it lands. - 2 TB USB portable hard drive — on hand, previously a games console's storage. Becomes the photo library's disk on `trinity` ([#404](https://github.com/Gerrrt/HomeLab/issues/404)): Immich's originals @@ -137,17 +147,23 @@ revisions of this repository treated `shiva` as the hypervisor itself. drive, which this replaces. - Lenovo ThinkServer TS150 — Xeon E3-1225 v6 (4 cores, 3.3 GHz, Intel HD P630 with Quick Sync), 8 GB ECC, four 3.5" bays, no drives, no OS — bought - 2026-09-09, in transit. The NAS `smaug` of + 2026-09-09, **in hand since 2026-09-15**. The NAS `smaug` of [ADR-0016](adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md), tracked under [#413](https://github.com/Gerrrt/HomeLab/issues/413). A tower, not a rack unit, and a 73 W part where the ADRs pictured an N100; it enters the Compute table when it is racked — or rather placed — addressed and in - `network.md`. The two drives for the mirror were bought 2026-09-11 and are - the entry below, as are the boot disk ADR-0016's Ubuntu Server install - wants and the bracket that carries it in the optical bay. Nothing for this - machine is outstanding on the roadmap's - [list](roadmap.md#everything-still-to-buy) any more; what is left is a - build. + `network.md`. The boot disk ADR-0016's Ubuntu Server install wants and the + bracket that carries it in the optical bay are the entries below and landed + with it; the two drives for the mirror, bought 2026-09-11, have not. Nothing + for this machine is outstanding on the roadmap's + [list](roadmap.md#everything-still-to-buy) any more; what is left is those + two drives landing, and a build. + **Checked on arrival, not assumed:** the Xeon E3-1225 v6 and the 8 GB of ECC + above came from a listing and not from the machine, so read them off POST or + the BIOS; the machine type-model and the serial off the chassis label; how + many of the four bays came with a caddy, because a bay without one holds no + drive; and that the 5.25" optical bay is empty, which the boot disk's bracket + assumes. Those go here when it is opened. - 2× Seagate Exos X20 18 TB (`ST18000NM003D`), 3.5" SATA — bought 2026-09-11, in transit. `smaug`'s ZFS mirror ([ADR-0016](adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md), @@ -158,18 +174,25 @@ revisions of this repository treated `shiva` as the hypervisor itself. before the mirror is built on them. Serials go here when they land. They enter the Compute table with the NAS, which is not built. - Intel DC S3520 240 GB, 2.5" SATA 6 Gb/s enterprise SSD with power-loss - protection — bought 2026-09-11, in transit. `smaug`'s boot disk, carrying - ADR-0016's Ubuntu Server and its one compose stack, deliberately not the - mirror. A data-centre part where the roadmap asked only for "a 240–256 GB - 2.5" SATA SSD": the endurance is beside the point for a boot disk, but + protection — bought 2026-09-11, **in hand since 2026-09-15**. `smaug`'s boot + disk, carrying ADR-0016's Ubuntu Server and its one compose stack, + deliberately not the mirror. A data-centre part where the roadmap asked only + for "a 240–256 GB 2.5" SATA SSD": the endurance is beside the point for a + boot disk, but the power-loss protection is the same property the SM863a pair was bought for, and a boot disk that survives a power cut is worth more here than one that is merely fast. The TS150's optical bay is 5.25" and this is a 2.5" drive, so it cannot be fitted bare: **a bracket for the bay and - double-sided tape to mount it were bought 2026-09-11**, alongside the drive. - Neither is named here yet, for the same reason the drive's serial is not — - nothing is recorded before it is in hand. Model, part number and serial go - here when it lands, with SMART read back before the install. + double-sided tape to mount it were bought 2026-09-11**, alongside the drive, + and both landed with it on 2026-09-15. Neither is named here yet — that was + deferred because nothing is recorded before it is in hand, and they now are, + so it is a naming this document owes rather than one it is waiting on. + **Checked on arrival, not assumed:** the drive's model, part number and + serial, and `smartctl -a` read back **before** the install rather than after + — the same caution the Exos pair below is bought under, and for the same + reason: a listing's hours are a claim. That the bracket is a 5.25"-to-2.5" + adapter and not the 3.5" version, which would fit neither the drive nor the + bay, goes here with them. - 2× Samsung SM863a 960 GB (`MZ-7KM960N`), 2.5" SATA 6 Gb/s enterprise SSDs with power-loss protection[^SM863a] — purchased 2026-09-09, in transit, for the ProLiant's SFF bays. Bought against the number every sizing diff --git a/docs/roadmap.md b/docs/roadmap.md index 591739df..c4afb9e0 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -89,22 +89,38 @@ capacity are still chosen at the till and recorded in [`hardware.md`](hardware.md) afterwards, per the ADR — nothing is named here before it is bought. -**Already paid for**, in transit or on hand: the ProDesk 600 G4 (the tier's -host and the firewall's spare hardware), and the I226 card on its M.2 adapter -that a restore onto it needs -([#404](https://github.com/Gerrrt/HomeLab/issues/404)); the CRS326 that -replaces `neo` ([#444](https://github.com/Gerrrt/HomeLab/issues/444)); the -TS150 NAS, the two -Exos X20 18 TB drives for its mirror, and its boot SSD with the bracket and -tape that mount it in the optical bay -([#413](https://github.com/Gerrrt/HomeLab/issues/413)); -two SM863a SSDs for -`Saruman` ([#418](https://github.com/Gerrrt/HomeLab/issues/418)); the pfSense -installer stick; the 2 TB USB drive that becomes the photo library's disk on -`trinity`; the removable medium the second age recipient lives on; and the A1437 cell +**Already paid for and on hand**, as of 2026-09-15: the ProDesk 600 G4 — +`trinity`, the tier's host and the firewall's spare hardware +([#404](https://github.com/Gerrrt/HomeLab/issues/404)); the TS150 NAS and its +boot SSD with the bracket and tape that mount it in the optical bay +([#413](https://github.com/Gerrrt/HomeLab/issues/413)); the 2 TB USB drive +that becomes the photo library's disk on `trinity`; and the removable medium +the second age recipient lives on. + +**Already paid for and still moving**: the I226 card on its M.2 adapter that a +restore onto `trinity` needs +([#404](https://github.com/Gerrrt/HomeLab/issues/404)) and the pfSense +installer stick — between them, the whole of what the rehearsal now waits on; +the two Exos X20 18 TB drives for `smaug`'s mirror +([#413](https://github.com/Gerrrt/HomeLab/issues/413)); the CRS326 that +replaces `neo` ([#444](https://github.com/Gerrrt/HomeLab/issues/444)); two +SM863a SSDs for `Saruman` +([#418](https://github.com/Gerrrt/HomeLab/issues/418)); and the A1437 cell that replaces `prometheus`'s thirteen-year-old one ([#454](https://github.com/Gerrrt/HomeLab/issues/454)). +Four things crossed from the second paragraph to the first on **2026-09-15**: +the TS150, the ProDesk, the boot SSD, and the bracket and tape that mount it. +**No row on the table above moves** — arriving is not buying, and this section +tracks money. Nothing is racked, fitted or built by it either. What it changes +is one sentence: the two builds are no longer waiting on a van for their +machines, only for the parts named above. And what those boxes actually are is +still unread — serials, the spec off POST rather than off a listing, SMART +before an install, and whether `trinity`'s 512 GB SSD is an M.2 stick or a +2.5" drive that would contend with the I226 card for the same slot. +[`hardware.md`](hardware.md) records each of those as owed rather than +assuming the listing was right. + **Only if a decision is taken, and none is pending** — these are not on the list, and each names what would put it there: @@ -687,8 +703,12 @@ what left this one unfireable for months. ECC memory under a ZFS mirror, four real bays, and a Kaby Lake iGPU whose Quick Sync is what #138's transcoding needs. The drives were bought on 2026-09-11 — two Seagate Exos X20 18 TB, chosen on cost per terabyte per - the ADR — and the boot disk with them, an Intel DC S3520 240 GB, - leaving only a bracket to carry it in the optical bay. The + the ADR — and the boot disk with them, an Intel DC S3520 240 GB, with the + bracket and the tape that carry it in the optical bay bought the same day. + This paragraph said a bracket was the only thing left to buy, which had not + been true since 2026-09-11. **The box landed 2026-09-15**, and the boot disk, + the bracket and the tape with it; the two Exos drives did not, so what the + build waits on is those and a bench, not money. The OS is decided: Ubuntu Server and one compose stack, not an appliance. Nothing is configured. Reading the enforced ruleset first changed two of the answers. **50→40 is not @@ -795,8 +815,9 @@ what left this one unfireable for months. did not exist, and PVE already does the snapshots the local-only answer needs — so PBS would add a service for a capability that exists. `smaug` is bought since 2026-09-09 — the TS150 of - [#413](https://github.com/Gerrrt/HomeLab/issues/413), in transit, its two - drives bought on 2026-09-11 — and not built, so the deferral's trigger has moved from + [#413](https://github.com/Gerrrt/HomeLab/issues/413), in hand since + 2026-09-15, its two drives bought on 2026-09-11 and still moving — and not + built, so the deferral's trigger has moved from a purchase to a build: PBS follows the NAS answering on `10.0.40.30`, not the box arriving. The lab has **revert and not backup** until then, and the ADR names what gets backed up when it arrives — which is also why ADR-0029 gives PBS no disk diff --git a/docs/runbooks/restore-the-firewall.md b/docs/runbooks/restore-the-firewall.md index d1aa48ae..f120b303 100644 --- a/docs/runbooks/restore-the-firewall.md +++ b/docs/runbooks/restore-the-firewall.md @@ -330,16 +330,25 @@ answer — which of the two ports a fresh install makes WAN and which LAN, whether the I226 card comes back as `igc0` on the spare, how long the whole thing takes — and to write the answers back into §3. -**Before the box arrives**, so the rehearsal does not stall on day one. All of -it was measured on 2026-09-09: - -- **The I226 card** — bought 2026-09-11, in transit - ([`hardware.md`](../hardware.md)). The spare is a - stock refurbished G4 with the onboard NIC only. If its 512 GB SSD turns out - to be a 2.5" drive, the drive carrier and a card in the second M.2 slot - contend for the same space; an M.2 SSD does not. -- **An installer for 2.9.0 or newer** — §3 step 1. The stick in transit is - whatever version it is; read it before the day. +**What the rehearsal is still missing.** This list was written on 2026-09-09 +to be lined up before the box arrived. The box arrived on 2026-09-15 and the +card and the installer stick did not, so it stops being a countdown and +becomes a list of what is blocking: + +- **The spare itself** — **here since 2026-09-15**, and unopened + ([`hardware.md`](../hardware.md)). It is a stock refurbished G4 that the + listing says has the onboard NIC only; nothing has been read off the machine + to confirm that. **Open it before the card arrives**, because of the bullet + below: if its 512 GB SSD is a 2.5" drive, the drive carrier and a card in + the second M.2 slot contend for the same space, and an M.2 SSD does not. + That was an unknown about a box in a van and is now an unanswered question + about a box in the room — answering it while the card can still be returned + costs nothing, and answering it on the day costs the rehearsal. +- **The I226 card** — bought 2026-09-11, **still in transit** + ([`hardware.md`](../hardware.md)). One of the two things the rehearsal now + waits on. +- **An installer for 2.9.0 or newer** — §3 step 1, and the other. The stick + in transit is whatever version it is; read it before the day. - **The newest export on `oracle`**, and the age key's offline copy. - **The numbers to check against**: `make backup-firewall ARGS=--verify-only` prints the schema and the rule count — `24.6` and 92 rules on 2026-09-09. diff --git a/scripts/check_docs.py b/scripts/check_docs.py index 3a027fe8..0c92d892 100755 --- a/scripts/check_docs.py +++ b/scripts/check_docs.py @@ -20,7 +20,9 @@ Ten assertions, each comparing prose against something machine-readable: 1. Counted claims rules, unit-test coverage, dashboards, panels, - Alloy agents, ADRs, runbooks + Alloy agents, ADRs, runbooks, and the size of this + list — README.md says how many assertions run, and + that number is read from the registry in main() 2. SNMP targets snmp.yaml <-> docs/network.md 3. Host and stack table docs/architecture.md <-> docs/network.md, stacks/ 4. Ports table docs/architecture.md <-> compose.yaml @@ -573,6 +575,25 @@ def check_counts(f: dict) -> list[str]: # a reference and not an inventory. (rf"{COUNT}" + WS + r"ADRs", {f["adrs"]}, "ADRs"), (rf"{COUNT}" + WS + r"runbooks", {f["runbooks"]}, "runbooks"), + # How many assertions this file runs — and the same shape as #72, #81 + # and #367: a counted claim that no pattern named, going stale where + # nothing could see it. README.md said "Six assertions" while main() + # registered ten, and the enumeration beside it had been overtaken by + # four checks — ADR numbering, firewall posture, guest claims and the + # buy list were all running and none was mentioned. Drift in the one + # file whose subject is catching drift, which makes it a false claim + # about the checking rather than a stale fact about the checks. + # + # The ADRs comment above settles the "just delete the number" argument + # and it applies here with more force: the sentence exists to say how + # thoroughly these documents are checked, so the number is the claim. + # + # Sourced from len(checks) in main() rather than a constant here, so + # an eleventh assertion fails this line on the way in instead of + # waiting for someone to reread the bullet. main() writes it into the + # facts dict after building the registry; facts() cannot compute it, + # because the registry closes over the dict facts() is still building. + (rf"{COUNT}" + WS + r"assertions", {f["assertions"]}, "assertions"), ) problems = [] for rel in PROSE: @@ -1260,6 +1281,13 @@ def main() -> int: check_buy_list), ) + # The registry is the source for README's "N assertions", so adding a + # check above updates the claim rather than invalidating it. Written here + # rather than in facts() because the tuple closes over `f`, so the count + # does not exist until after the dict does; check_counts reads it through + # the same closure, which runs below this line. + f["assertions"] = len(checks) + total = 0 for label, check in checks: problems = check()