diff --git a/README.md b/README.md index d42d1266..45e7ee57 100644 --- a/README.md +++ b/README.md @@ -334,7 +334,7 @@ why it is in that order. The current top items: rehearse the firewall restore on the ProDesk bought on 2026-09-08, and then build the sensitive tier on that same box ([#404](https://github.com/Gerrrt/HomeLab/issues/404), [ADR-0034](adr/0034-run-the-sensitive-tier-on-the-prodesk-and-make-it-the-spare-hardware.md)). **Every purchase still outstanding is in one place**, the roadmap's [*Everything still to buy*](docs/roadmap.md#everything-still-to-buy): -three items now, one later, and a rule that nothing joins them without a +six items now, one later, and a rule that nothing joins them without a decision. This sentence used to carry the list itself, name three purchases coupled to the UPS work and omit the tier's host entirely, which is how one ProDesk came to be bought for two jobs. diff --git a/docs/architecture.md b/docs/architecture.md index a3933d97..ee53fea4 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -189,7 +189,7 @@ hole from the monitoring VLAN into the monitored one. | `alexander` (10.0.30.40) | ๐ŸŸข 30 | [`stacks/lab`](../stacks/lab) | Prometheus, Loki, Grafana, Alloy โ€” the lab's own observability, which never remote-writes to VLAN 99 ([ADR-0007](adr/0007-defensive-estate-and-offensive-range.md), [ADR-0020](adr/0020-run-the-lab-stack-in-a-guest-with-its-own-prometheus.md)). A guest on `Saruman`, not the hypervisor; Alloy agent (Docker) | | `odin` (10.0.30.60) | ๐ŸŸข 30 | [`stacks/soc`](../stacks/soc) | **Not built yet** โ€” Wazuh (indexer, manager, dashboard), Velociraptor and Alloy: the security half of ADR-0007, placed by [ADR-0030](adr/0030-give-the-security-tooling-its-own-guest-and-its-own-stack.md) on a second guest on `Saruman` because `alexander`'s 8 GiB cannot hold both. The stack is authored and CI-validated ahead of the guest, the way `stacks/lab` was ahead of `alexander`; the build is [`build-the-soc-guest.md`](runbooks/build-the-soc-guest.md), behind [#414](https://github.com/Gerrrt/HomeLab/issues/414). The six domain machines report to it as agents; its Alloy pushes to `alexander`, never to VLAN 99, and the indexer's health is the one series that crosses into the lab's Prometheus. Alloy agent (Docker) | | `oracle` (10.0.99.30) | ๐Ÿ”ด 99 | *(none โ€” hand-run containers)* | The Lemmiwinks wiki and its Postgres, since 2025-11-12 ([ADR-0011](adr/0011-keep-the-wiki-internal.md)); Alloy agent (Docker, `scripts/deploy-agent.sh`); the off-host copy of the firewall export (`make backup-firewall`). The estate's host for small off-host jobs โ€” [ADR-0015](adr/0015-give-oracle-the-off-host-jobs.md) | -| `trinity` (10.0.99.40) | ๐Ÿ”ด 99 | [`stacks/sensitive`](../stacks/sensitive) | **Not built yet** โ€” ADR-0008's sensitive tier on the ProDesk 600 G4 of [ADR-0034](adr/0034-run-the-sensitive-tier-on-the-prodesk-and-make-it-the-spare-hardware.md), after the firewall restore is rehearsed on it ([#404](https://github.com/Gerrrt/HomeLab/issues/404)). The foundation is authored: Caddy as the published HTTPS port and step-ca as an intermediate beneath the lab CA ([#129](https://github.com/Gerrrt/HomeLab/issues/129), [#130](https://github.com/Gerrrt/HomeLab/issues/130)), with AdGuard Home behind Caddy and publishing 53 to the firewall's forwarder alone ([#135](https://github.com/Gerrrt/HomeLab/issues/135), [ADR-0010](adr/0010-keep-the-resolver-on-the-gateway.md)); Home Assistant ([#134](https://github.com/Gerrrt/HomeLab/issues/134)), Immich โ€” four containers behind Caddy with a memory limit on each ([#132](https://github.com/Gerrrt/HomeLab/issues/132)) โ€” Paperless-ngx with a Postgres and a Valkey of its own ([#133](https://github.com/Gerrrt/HomeLab/issues/133)) and Vaultwarden ([#131](https://github.com/Gerrrt/HomeLab/issues/131)) are authored as well; the rest follow. Alloy agent (Docker, `scripts/deploy-agent.sh`), pushing to `prometheus` like `oracle`'s | +| `trinity` (10.0.99.40) | ๐Ÿ”ด 99 | [`stacks/sensitive`](../stacks/sensitive) | **Not built yet** โ€” ADR-0008's sensitive tier on the ProDesk 600 G4 of [ADR-0034](adr/0034-run-the-sensitive-tier-on-the-prodesk-and-make-it-the-spare-hardware.md), after the firewall restore is rehearsed on it ([#404](https://github.com/Gerrrt/HomeLab/issues/404)). The foundation is authored: Caddy as the published HTTPS port and step-ca issuing beneath the tier's own root rather than the estate's, which is left untouched ([#129](https://github.com/Gerrrt/HomeLab/issues/129), [#130](https://github.com/Gerrrt/HomeLab/issues/130), [ADR-0037](adr/0037-give-the-sensitive-tier-its-own-root-and-issue-beneath-it-over-acme.md)), with AdGuard Home behind Caddy and publishing 53 to the firewall's forwarder alone ([#135](https://github.com/Gerrrt/HomeLab/issues/135), [ADR-0010](adr/0010-keep-the-resolver-on-the-gateway.md)); Home Assistant ([#134](https://github.com/Gerrrt/HomeLab/issues/134)), Immich โ€” four containers behind Caddy with a memory limit on each ([#132](https://github.com/Gerrrt/HomeLab/issues/132)) โ€” Paperless-ngx with a Postgres and a Valkey of its own ([#133](https://github.com/Gerrrt/HomeLab/issues/133)) and Vaultwarden ([#131](https://github.com/Gerrrt/HomeLab/issues/131)) are authored as well; the rest follow. Alloy agent (Docker, `scripts/deploy-agent.sh`), pushing to `prometheus` like `oracle`'s | | `bahamut` (10.0.30.50) | ๐ŸŸข 30 | *(none โ€” Windows)* | **Not built yet** โ€” Windows Server 2025 domain controller, PDC emulator and DNS for `ad.matrix.elysium` โ€” Tier 0. Static, because every member finds a DC through DNS and the DCs *are* the DNS. Scraped by `alexander` on `9182`; it pushes nothing, and runs no Alloy ([ADR-0029](adr/0029-size-the-lab-domain-and-separate-its-namespace-and-clock.md)) | | `leviathan` (10.0.30.51) | ๐ŸŸข 30 | *(none โ€” Windows)* | **Not built yet** โ€” Windows Server 2025 second domain controller and DNS โ€” Tier 0. Static, for the same reason. Scraped by `alexander` on `9182`; it pushes nothing, and runs no Alloy ([ADR-0029](adr/0029-size-the-lab-domain-and-separate-its-namespace-and-clock.md)) | | `titan` (10.0.30.52) | ๐ŸŸข 30 | *(none โ€” Windows)* | **Not built yet** โ€” Windows Server 2025 file and member server โ€” the shares, and the NTLM relay target that only exists because 2025 requires outbound SMB signing and not inbound โ€” Tier 1. Scraped by `alexander` on `9182`; it pushes nothing, and runs no Alloy ([ADR-0029](adr/0029-size-the-lab-domain-and-separate-its-namespace-and-clock.md)) | diff --git a/docs/hardware.md b/docs/hardware.md index 6a4c0973..2f2e47e1 100644 --- a/docs/hardware.md +++ b/docs/hardware.md @@ -119,7 +119,30 @@ revisions of this repository treated `shiva` as the hypervisor itself. tracked under [#413](https://github.com/Gerrrt/HomeLab/issues/413). A tower, not a rack unit, and a 73 W part where the ADRs pictured an N100; it enters the Compute table when it is racked โ€” or rather placed โ€” addressed and in - `network.md`. Two drives for the mirror are still to buy. + `network.md`. The two drives for the mirror were bought 2026-09-11 and are + the entry below, as is the boot disk ADR-0016's Ubuntu Server install + wants. Only a bracket to carry that disk in the optical bay is still to + buy, on the roadmap's [list](roadmap.md#everything-still-to-buy). +- 2ร— Seagate Exos X20 18 TB (`ST18000NM003D`), 3.5" SATA โ€” bought + 2026-09-11, in transit. `zion`'s ZFS mirror + ([ADR-0016](adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md), + [#413](https://github.com/Gerrrt/HomeLab/issues/413)). A mirror of two is + one drive's capacity, so this is 18 TB usable, not 36. The listing's + **zero power-on hours is a claim, not a fact** โ€” read it back with + `smartctl -a` on arrival and record what the drives actually report here, + before the mirror is built on them. Serials go here when they land. They + enter the Compute table with the NAS, which is not built. +- Intel DC S3520 240 GB, 2.5" SATA 6 Gb/s enterprise SSD with power-loss + protection โ€” bought 2026-09-11, in transit. `zion`'s boot disk, carrying + ADR-0016's Ubuntu Server and its one compose stack, deliberately not the + mirror. A data-centre part where the roadmap asked only for "a 240โ€“256 GB + 2.5" SATA SSD": the endurance is beside the point for a boot disk, but + the power-loss protection is the same property the SM863a pair was bought + for, and a boot disk that survives a power cut is worth more here than one + that is merely fast. **It has no bracket** โ€” the TS150's optical bay is + 5.25" and this is a 2.5" drive, so an adapter is outstanding and the drive + cannot be fitted without it. Model, part number and serial go here when it + lands, with SMART read back before the install. - 2ร— Samsung SM863a 960 GB (`MZ-7KM960N`), 2.5" SATA 6 Gb/s enterprise SSDs with power-loss protection[^SM863a] โ€” purchased 2026-09-09, in transit, for the ProLiant's SFF bays. Bought against the number every sizing diff --git a/docs/roadmap.md b/docs/roadmap.md index 564fa097..4019352f 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -34,8 +34,7 @@ battery it was never written about. | Item | For | Decided by | When it is needed | | --- | --- | --- | --- | -| Two 3.5" NAS drives, capacity chosen at the till | `zion`'s ZFS mirror | [ADR-0016](adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md), [#413](https://github.com/Gerrrt/HomeLab/issues/413) | Before the NAS is built | -| A 240โ€“256 GB 2.5" SATA SSD, and a bracket for the optical bay | `zion`'s boot disk. ADR-0016 chose Ubuntu Server and one compose stack rather than an appliance, so the OS wants a disk that is not the mirror โ€” and that never got written down | [ADR-0016](adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md), [#413](https://github.com/Gerrrt/HomeLab/issues/413) | With the drives | +| A bracket to carry a 2.5" SSD in the optical bay | Mounting `zion`'s boot disk. ADR-0016 chose Ubuntu Server and one compose stack rather than an appliance, so the OS wants a disk that is not the mirror โ€” and that never got written down. The SSD itself was bought 2026-09-11; this row is what is left of it | [ADR-0016](adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md), [#413](https://github.com/Gerrrt/HomeLab/issues/413) | With the drives | | One Intel I226 2.5 GbE card on an M.2 B+M-key adapter โ€” the part `morpheus` has, not a USB NIC | The ProDesk's second port, so a restore onto it comes up as `igc0` and asks nothing; for the rehearsal and any restore after it | [`restore-the-firewall.md`](runbooks/restore-the-firewall.md) ยง3, [#404](https://github.com/Gerrrt/HomeLab/issues/404) | Before the rehearsal | | Two Windows 11 Pro keys | The lab domain's two endpoints; the four servers are free evaluations | [ADR-0029](adr/0029-size-the-lab-domain-and-separate-its-namespace-and-clock.md), [#414](https://github.com/Gerrrt/HomeLab/issues/414) | When the domain build reaches the endpoints, not before | | A managed switch with a TLS management interface | Replacing `neo`, whose firmware will not persist a community deletion ([#84](https://github.com/Gerrrt/HomeLab/issues/84)) and which serves its admin UI over plain HTTP โ€” through the device the password protects, and that credential is read-write. **SNMPv3 is no longer the argument**: [ADR-0036](adr/0036-poll-the-ilo-and-the-ups-card-over-snmpv3-and-keep-the-firewall-on-bsnmpd.md) found the switch's agent answers v3 on the wire and was never what blocked #85, so TLS management is what a replacement is actually bought for | [ADR-0018](adr/0018-name-the-switch-and-leave-its-ui-on-plain-http.md), [#444](https://github.com/Gerrrt/HomeLab/issues/444) | Whenever a cabling window suits โ€” it is independent of everything else here | @@ -59,7 +58,10 @@ capacity are still chosen at the till and recorded in before it is bought. **Already paid for**, in transit or on hand: the ProDesk 600 G4 (the tier's -host and the firewall's spare hardware); the TS150 NAS; two SM863a SSDs for +host and the firewall's spare hardware); the TS150 NAS and the two Exos X20 +18 TB drives for its mirror, and its boot SSD +([#413](https://github.com/Gerrrt/HomeLab/issues/413)); +two SM863a SSDs for `Saruman` ([#418](https://github.com/Gerrrt/HomeLab/issues/418)); the pfSense installer stick; the 2 TB USB drive that becomes the photo library's disk on `trinity`; the removable medium the second age recipient lives on. @@ -632,8 +634,10 @@ what left this one unfireable for months. 73 W desktop Xeon in a tower against a 6 W part in a shoebox, so it draws and makes more, and it will not rack. What it has that an N100 box does not is ECC memory under a ZFS mirror, four real bays, and a Kaby Lake iGPU whose - Quick Sync is what #138's transcoding needs. The drives are the outstanding - purchase โ€” chosen at the till on cost per terabyte, per the ADR โ€” and the + Quick Sync is what #138's transcoding needs. The drives were bought on + 2026-09-11 โ€” two Seagate Exos X20 18 TB, chosen on cost per terabyte per + the ADR โ€” and the boot disk with them, an Intel DC S3520 240 GB, + leaving only a bracket to carry it in the optical bay. The OS is decided: Ubuntu Server and one compose stack, not an appliance. Nothing is configured. Reading the enforced ruleset first changed two of the answers. **50โ†’40 is not @@ -741,7 +745,7 @@ what left this one unfireable for months. so PBS would add a service for a capability that exists. `zion` is bought since 2026-09-09 โ€” the TS150 of [#413](https://github.com/Gerrrt/HomeLab/issues/413), in transit, its two - drives still to buy โ€” and not built, so the deferral's trigger has moved from + drives bought on 2026-09-11 โ€” and not built, so the deferral's trigger has moved from a purchase to a build: PBS follows the NAS answering on `10.0.40.30`, not the box arriving. The lab has **revert and not backup** until then, and the ADR names what gets backed up when it arrives โ€” which is also why ADR-0029 gives PBS no disk diff --git a/scripts/check_docs.py b/scripts/check_docs.py index 534e2ea7..3a027fe8 100755 --- a/scripts/check_docs.py +++ b/scripts/check_docs.py @@ -17,7 +17,7 @@ The device list must live in exactly one place. It is currently spread across five ... and --check asserts the other copies still agree. -Nine assertions, each comparing prose against something machine-readable: +Ten assertions, each comparing prose against something machine-readable: 1. Counted claims rules, unit-test coverage, dashboards, panels, Alloy agents, ADRs, runbooks @@ -37,6 +37,10 @@ 9. Guest claims a host that says it runs no guests, while another row says it is a guest on it. Checkable only against its siblings, which is why it is its own assertion. + 10. Outstanding buys README.md <-> the buy table in docs/roadmap.md. The + roadmap is a source here, not a target: that table + is the one place a purchase may enter or leave, so + it is what README's count answers to. Only present-tense documents are checked. `docs/roadmap.md` and `docs/adr/` record what was true when the work landed โ€” `roadmap.md` still says "(34 rules)" @@ -84,6 +88,7 @@ NETWORK_MD = REPO / "docs/network.md" ARCH_MD = REPO / "docs/architecture.md" HARDWARE_MD = REPO / "docs/hardware.md" +ROADMAP_MD = REPO / "docs/roadmap.md" # Present-tense documents. See the module docstring for why roadmap.md and # adr/ are deliberately absent โ€” they are records, not claims about now. @@ -1175,6 +1180,69 @@ def named(prefix: str) -> set[str] | None: return problems +def check_buy_list() -> list[str]: + """README's count of outstanding purchases against the roadmap's table. + + `roadmap.md` is a record and not checked as prose โ€” see the module + docstring. This assertion reads it as a *source* instead: the buy table is + the one place a purchase is allowed to enter or leave, by that section's + own rule, so it is the only thing README's count can honestly be checked + against. + + It exists because the count went stale inside a day. The reconciliation on + 2026-09-10 grew the table from three rows to seven and left README saying + "three items now", in the sentence that exists *because* a purchase had + been missed once already. Every other count in this repository is asserted + here; this one was prose on both ends, so nothing caught it. + + The "one later" half is checked too. `ifrit` is a paragraph rather than a + row, and a second such paragraph would make the word wrong in exactly the + same silent way. + """ + problems: list[str] = [] + roadmap = ROADMAP_MD.read_text() + + tables = tables_under(roadmap, re.compile(r"^## Everything still to buy\b")) + if not tables: + return [ + "docs/roadmap.md has no table under 'Everything still to buy' โ€” " + "README's count of outstanding purchases has nothing to check " + "against" + ] + + # The section's first table is the "Buy these" list; rows[0] is its header. + rows_now = len(tables[0]) - 1 + rows_later = len(re.findall(r"\*\*One more, later", roadmap)) + + num = "|".join(["[0-9]+", *NUMBER_WORDS]) + pattern = re.compile( + rf"({num}){WS}items?{WS}now,{WS}({num}){WS}later", re.IGNORECASE + ) + text = (REPO / "README.md").read_text() + match = pattern.search(text) + if not match: + return [ + "README.md no longer says 'N items now, M later' of the roadmap's " + "outstanding purchases โ€” the claim moved or was reworded, and this " + "assertion cannot follow it. Update the pattern or drop the check " + "deliberately" + ] + + line = text[: match.start()].count("\n") + 1 + said_now, said_later = number(match.group(1)), number(match.group(2)) + if said_now != rows_now: + problems.append( + f"README.md:{line} says {match.group(1)} outstanding purchase(s), " + f"but docs/roadmap.md's buy table has {rows_now} row(s)" + ) + if said_later != rows_later: + problems.append( + f"README.md:{line} says {match.group(2)} purchase(s) later, but " + f"docs/roadmap.md has {rows_later} 'One more, later' paragraph(s)" + ) + return problems + + def main() -> int: f = facts() checks = ( @@ -1188,6 +1256,8 @@ def main() -> int: ("firewall posture prose against docs/firewall-claims.yaml", check_firewall_posture), ("guest claims against each other", check_guest_claims), + ("README's outstanding-purchase count against the roadmap's table", + check_buy_list), ) total = 0 diff --git a/stacks/sensitive/Caddyfile b/stacks/sensitive/Caddyfile index 507a2413..29223be1 100644 --- a/stacks/sensitive/Caddyfile +++ b/stacks/sensitive/Caddyfile @@ -25,7 +25,7 @@ auto_https disable_redirects # Every certificate this file serves comes from step-ca over ACME, from - # the tier's own intermediate (ADR-0035, #130). `step-ca` resolves on the + # the tier's own intermediate (ADR-0037, #130). `step-ca` resolves on the # compose network; the directory URL's middle segment is the provisioner # name scripts/tier-ca.sh creates. trusted_roots is the TIER'S root โ€” the # file compose.yaml mounts from certificates/tier-ca.pem โ€” not the estate's