Sensitive tier: ntfy, to bring the notification path in-house #136

Sub-issue of #102.

What and why

ntfy Apache-2.0/GPL-2.0 Go/Docker/K8S — awesome-selfhosted README.md:355.

The alerting path already assumes ntfy. docs/roadmap.md records under Done that Alertmanager pointed at the ntfy.example.invalid placeholder for the entire life of the stack, so no alert had ever been delivered, and that it now points at a real receiver. #67 added a dead man's switch on top of it (PR #119), and #120 is waiting on the secrets to deploy that.

This issue is the last piece: run the receiver rather than depend on someone else's.

The one thing to think about first

#67 identified the constraint that matters here:

The awkward part is that the watcher has to live somewhere other than this host, or it fails at the same time as the thing it is watching.

Self-hosting ntfy pulls the notification path closer to the thing it reports on. Today prometheus (VLAN 99) alerts through a service that is not in the house; afterwards it would alert through a service on the same segment, on a box that is one of two shelf laptops behind a switch with no battery (#110).

That is not an argument against doing it — a mains cut already takes the alerting host down, so the marginal loss is small. But it means:

• The dead man's switch watcher must stay external, whatever else moves in-house. A self-hosted ntfy watched by a self-hosted checker is a closed loop that reports success right up until both are off.
• Or keep ntfy.sh as the delivery path for the watchdog specifically, and self-host everything else. Worth deciding explicitly rather than by omission.

What it needs

• Auth configured — an ntfy instance with open publish is an open relay for anything that can reach it.
• Topic names and any token in secrets/*.sops.yaml, replacing the values Fill in the three new notification secrets, then deploy #118 and #119 #120 is currently waiting on.
• Alertmanager's receiver URL repointed, and the alert path re-verified end to end afterwards, per docs/runbooks/verify-the-alert-path.md. The lesson from the placeholder is that a notification path is only believed after it has been observed working — a 200 is not evidence.
• Phones need the app pointed at the internal instance, which only works on-network given there is no external exposure.

Which VLAN this touches

🔴 99 — Winterfell (management)

Security review

• Does not weaken the default-deny posture between VLANs — publishers are on 99 and 50, both already permitted; no new rule
• Any new credential goes in secrets/*.sops.yaml — and note docs/security.md records a webhook URL previously found in a session transcript and rotated; treat topic URLs as credentials
• Any new device is added to docs/network.md with an OUI-truncated MAC — n/a

Depends on

#129 (Caddy) and the mini PC in #102. Interacts with #120 (secrets pending), #119/#67 (dead man's switch), #110 (the shelf switch has no battery).

Corrected 2026-09-20

Three premises above are spent. #120 closed 2026-09-04 (the receivers are filled in and alerts deliver); #110 closed 2026-09-08 (the shelf switch is racked in U4 on UPS power, so "a switch with no battery" is gone); and the ntfy.sh quota was found to be spent daily by the heartbeat (#407), which moved the dead man's switch to healthchecks.io on 2026-09-09 (#408). That last one answers the question this issue said to decide explicitly: the watcher is already external and stays external; what an in-house ntfy would replace is the delivery path for alerts, not the heartbeat. "Depends on #102" → #404. Not authored, and it is seq/7 in trinity on purpose — it has nothing to publish to until the tier serves.