Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .changeset/rewrite-agent-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
---
---
168 changes: 168 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
# nostream — working on `rewrite/main`

The Rust rewrite replaces the TypeScript relay **in place, crate by crate, on this
branch**. `main` receives it only at cutover. Until then `src/` (TypeScript) and
`crates/` (Rust) coexist in one tree and are separate pipelines.

Rules below are the ones an agent gets wrong. Generic Rust guidance is not repeated
here.

## Version control

- **Work on `rewrite/main` (or a branch off it), never `main`.**
- A PR into `rewrite/main` must carry **one empty changeset**
(`.changeset/<name>.md` with empty frontmatter) — `Changeset Required` runs on every
PR and a Rust-only change touches no published package. The changeset *content* is
changed only when the TypeScript package itself changes.
- Commits are commitlint-checked (`Lint commits`); conventional-commit form is required.
- **The husky `pre-commit` hook runs the TypeScript suite**
(`pnpm run lint`, `build:check`, `test:unit`) on every commit. A Rust-only change
cannot fail those, and CI runs the same three as `Lint code`, `Build check` and
`Unit Tests And Coverage` against the same change — so `--no-verify` is reasonable for
a Rust-only commit and never for one that touches `src/`, `test/` or a JS manifest.

## Validation — the exact commands

CI (`Rust Checks` in `.github/workflows/checks.yml`) is the authority. Run the same
five, in this order, and they must all pass before a change is claimed done:

```
cargo fmt --all --check
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo clippy --workspace --lib --bins --all-features -- \
-D warnings -D clippy::unwrap_used -D clippy::expect_used \
-D clippy::todo -D clippy::unimplemented -D clippy::panic
cargo test --workspace --all-features
cargo build --workspace --all-features --release
```

- **`--workspace`, never a single crate.** A crate-local `cargo test -p` proves nothing
about a workspace lint that another member trips.
- **`--all-features` is not optional.** `nostreamd`'s `otlp` feature gates three
dependencies; a change that only compiles without it is broken.
- **Two clippy passes, and both are required.** The lints `unwrap_used`, `expect_used`
and `panic` are declared in `[workspace.lints.clippy]` as well as passed on the strict
command line, so a local `cargo clippy` fails on the same code CI rejects. An
integration-test target that needs them carries
`#![allow(clippy::unwrap_used, clippy::expect_used)]` (plus `clippy::panic` where it
asserts by panicking) at the top of the file — naming the file that may unwrap, not
exempting test targets workspace-wide. A new test file therefore has to ask.
- The `#[allow(unsafe_code)]` carve-out is in `crates/nostream-storage/src/backend.rs`
around heed's `EnvOpenOptions::open`. Do not add a second one; do not widen
`[workspace.lints.rust] unsafe_code` from `deny` to `forbid` (it would reject that
audited call).

### The toolchain

`rust-toolchain.toml` pins it — channel `1.99.1`, `rustfmt` + `clippy`, `minimal` — and
every `cargo` invocation picks it up from the working directory. **Never pass a
version or `+stable` to a command**; edit the pin file if the toolchain must move, so
CI, contributors and build agents all move together.

## Crate layout

`crates/*`, one crate per module family, `edition = "2024"`, `rust-version = "1.99"`,
resolver 3:

| crate | owns |
|---|---|
| `nostream-core` | domain vocabulary: events, filters, kinds, wire messages, NIP type surfaces, `ShutdownState` |
| `nostream-config` | typed settings, loader, deep merge, validator, dot-path editor, redaction, guided catalogue |
| `nostream-storage` | event-store backends (LMDB via heed, Postgres), cache |
| `nostream-relay` | acceptor, connections, wire protocol, NIP-77 engine |
| `nostream-services` | NIP-66 monitors, Tor, schedulers, outbound work |
| `nostream-admin` | admin API and auth |
| `nostream-cli` | operator command line (`nostream-cli config validate`) |
| `nostreamd` | the daemon binary: runtime, telemetry, signals |

- Every crate manifest repeats `version.workspace = true`, `edition.workspace = true`,
`rust-version.workspace = true`, `license.workspace = true`,
`repository.workspace = true` and `[lints] workspace = true`. A new crate that omits
the last one silently opts out of every workspace lint.
- **Dependencies go in `[workspace.dependencies]` and are referenced as
`name.workspace = true`.** A version literal in a crate manifest is a second place to
bump; do not add one.
- Each crate's `lib.rs` doc comment names the TypeScript paths it owns at the capture
commit, and separately names what it deliberately does **not** own. Preserve both: a
reader must be able to tell an omission from a gap.

## Error model

- **One error enum per crate, named `<Crate>Error`** (`CoreError`, `ConfigError`,
`StorageError`, `DaemonError`, `CliError`), `thiserror::Error`, one variant per
failure class, and a message written for an operator. **Every library module returns its
own crate's type** — `nostream-config` returns `ConfigError`, and the CLI's library
modules (`crates/nostream-cli/src/`) return `CliError`. A library never returns another
crate's error type, so no crate has to depend on `nostream-cli` to name a failure.
- **`anyhow` appears only in `crates/nostreamd/src/main.rs` and
`crates/nostream-cli/src/main.rs`.** The binaries compose, log and exit; they are not
part of any library's contract. The check CI's reviewers use — it searches **every**
source file, not just the `lib.rs` roots, and skips comment lines that merely name the
crate:

```
grep -rn 'anyhow' crates --include=*.rs \
| grep -vE 'crates/nostreamd/src/main\.rs|crates/nostream-cli/src/main\.rs' \
| grep -vE '^[^:]+:[0-9]+:\s*(//|///|//!)'
```

→ no output. Re-proven 2026-10-18 by adding `fn probe() -> anyhow::Result<()>` to
`crates/nostream-config/src/load.rs`, which the check printed.
- **A subsystem that is not implemented returns
`Error::NotImplemented { module: "path::to::call" }` — never `todo!()`,
`unimplemented!()` or `panic!()`.** The daemon's `main` maps that variant to exit code
`1` with its message, and `DaemonError::InvalidSettings` to exit code `2`; keep those
two distinct.
- Validation findings are **not** errors: `nostream_config::Issue { path, message }`
carries a dot-path and the capture's exact message, and `Display` renders
`{path}: {message}`. A validator that returns `Err` on a bad setting instead of an
`Issue` changes the CLI's output contract.
- Settings validation messages are **byte-identical to the capture's**.
`crates/nostream-config/tests/ts_parity.rs` replays every case in
`tests/fixtures/ts-issues.json` (measured data, generated by
`tests/capture/gen-ts-issues.mjs` against the capture commit) and requires the same
issues, in the same order, with the same messages. **Exactly one path is exempt —
`eventStore.backend`, because the rewrite adds the LMDB backend — and the exemption is
recorded per case.** When a message or an exemption must change, change the fixture in
the same commit and say why; never loosen the comparison to make a case pass.

## Porting discipline (this branch, phase 1)

- **Stubs are labelled, not hidden.** Every unimplemented item carries
`/// Phase 1 stub — job 005; <what it will do>; see <TS path>` on the item, takes typed
arguments, and returns `NotImplemented`. `grep -rn 'Phase 1 stub' crates` must count
the stubs.
- **No silent `Ok(())`, no plausible fake data, no partial port.** A function that would
need real behaviour returns the error. A `Result`-returning stub whose body is
`Ok(Default::default())` is the failure mode this rule exists to prevent.
- The TypeScript tree is the specification. When Rust behaviour and a TS file disagree,
the TS file wins unless a decision record says otherwise; the ADRs live in
`docs/rewrite/decisions/` (`0001-runtime-and-telemetry.md` covers runtime, acceptor
shape and telemetry) — read the relevant one before changing what it decided.

## Tests

- Unit tests live in `#[cfg(test)]` modules; integration tests in `crates/<crate>/tests/`.
Prefer whole-behaviour tests at the public API: `crates/nostream-config/tests/load.rs`
writes real fixtures to a `tempfile::TempDir` and calls the public loader, rather than
asserting on internals.
- `unwrap`/`expect` are permitted in tests (the strict clippy pass excludes them) — use
`expect("why this cannot fail")` so a failure names the assumption.
- A parity test that reads `resources/default-settings.yaml` or a fixture must fail, not
skip, when the fixture is missing.

## Path filters — what triggers `Rust Checks`

`crates/**`, `Cargo.toml`, `Cargo.lock`, `rust-toolchain.toml`, `scripts/rust.sh`,
`resources/default-settings.yaml` and the workflow file itself. A change to any other
file does **not** run the Rust job, so a change that alters what `cargo` sees (a new
fixture directory, a `.cargo/config.toml`) needs the filter extended in the same PR.
`resources/default-settings.yaml` joins **both** the TS and Rust filters because both
pipelines read it.

## Branch protection

`rewrite/main` has its own ruleset with eight required contexts: `Rust Checks`,
`Lint commits`, `Lint code`, `Lint PR title`, `Changeset Required`, `Build check`,
`Unit Tests And Coverage`, `Integration Tests and Coverage`. Adding or renaming a CI job
means updating that ruleset in the same release, or the branch becomes unmergeable.
9 changes: 9 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,12 @@ unsafe_code = "deny"
todo = "deny"
unimplemented = "deny"
dbg_macro = "deny"
# Carried here as well as on the strict `--lib --bins` command line in CI. Without them
# a crate that inherits `[lints] workspace = true` passes a local `cargo clippy` and is
# then rejected by the gate, which is the one thing a lint configuration must never do.
# Tests still opt out per crate with `#![allow(clippy::unwrap_used, clippy::expect_used)]`
# in the test target, which is narrower than a command-line flag: it names the files that
# may unwrap instead of exempting every target in the workspace.
unwrap_used = "deny"
expect_used = "deny"
panic = "deny"
5 changes: 5 additions & 0 deletions crates/nostream-cli/tests/config_cli.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
//! End-to-end tests for the `config` subcommands.

// This integration-test target may unwrap: a failing assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used)]

use std::fs;
use std::path::PathBuf;

Expand Down
5 changes: 5 additions & 0 deletions crates/nostream-config/tests/backend_values.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
//! `eventStore.backend` accepts the rewrite's LMDB backend (spec D7) and still rejects
//! anything outside the two-member set, with the two-member message.

// This integration-test target may unwrap: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used)]

use std::fs;

use nostream_config::path::set_by_path;
Expand Down
5 changes: 5 additions & 0 deletions crates/nostream-config/tests/defaults_parity.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
//! Defaults parity with `test/unit/utils/settings.spec.ts:226-341` at the capture commit:
//! six defaults assertions and four override cases, value for value.

// This integration-test target may unwrap and panic: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]

use std::fs;

use nostream_config::load::merge;
Expand Down
5 changes: 5 additions & 0 deletions crates/nostream-config/tests/defaults_roundtrip.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@
//! untagged enum is right, and it is why `validate(&Settings)` may delegate to the document
//! validator.

// This integration-test target may unwrap: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used)]

use std::fs;

use nostream_config::settings::Settings;
Expand Down
5 changes: 5 additions & 0 deletions crates/nostream-config/tests/guided.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
//! The guided settings catalogue, ported from the `settings-guided-schema` block of
//! `test/unit/utils/settings-config.spec.ts`.

// This integration-test target may unwrap: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used)]

use nostream_config::guided::{
GuidedSettingFieldType, GuidedSettingValidator, get_guided_setting_category,
get_guided_setting_field, guided_setting_categories, require_non_empty_setting_value,
Expand Down
5 changes: 5 additions & 0 deletions crates/nostream-config/tests/load.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@
//! from `test/unit/utils/settings.spec.ts` and the precedence block of
//! `test/unit/utils/settings-config.spec.ts`.

// This integration-test target may unwrap: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used)]

use std::fs;
use std::path::{Path, PathBuf};

Expand Down
5 changes: 5 additions & 0 deletions crates/nostream-config/tests/notifications_merge.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
//! Admin-notification merge and patch semantics, ported from
//! `test/unit/utils/admin-notifications-settings.spec.ts`.

// This integration-test target may unwrap: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used)]

use nostream_config::notifications::{
AdminNotificationsPatch, AdminNotificationsSettings, get_merged_admin_notifications,
merge_admin_notifications_patch,
Expand Down
5 changes: 5 additions & 0 deletions crates/nostream-config/tests/path_editor.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
//! The dot-path editor and value parser, ported from `test/unit/utils/settings-config.spec.ts`.

// This integration-test target may unwrap and panic: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]

use std::fs;

use nostream_config::path::{
Expand Down
5 changes: 5 additions & 0 deletions crates/nostream-config/tests/redaction.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
//! Redaction, ported from `test/unit/utils/settings-redaction.spec.ts`.

// This integration-test target may unwrap: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used)]

use nostream_config::path::get_by_path;
use nostream_config::redaction::{
is_sensitive_settings_path, is_write_protected_settings_path, redact_settings_secrets,
Expand Down
5 changes: 5 additions & 0 deletions crates/nostream-config/tests/ts_parity.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@
//! `eventStore.backend`, because the rewrite adds the LMDB backend (spec D7) — and the
//! exemption is recorded per case and asserted here.

// This integration-test target may unwrap and panic: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]

use std::collections::BTreeSet;
use std::fs;

Expand Down
5 changes: 5 additions & 0 deletions crates/nostream-config/tests/validator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@
//! adaptive-pow matrix, the event-store cases, the shape rules and the admin-notification
//! rules. Each test names what `test/unit/utils/settings-config.spec.ts` asserted.

// This integration-test target may unwrap: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used)]

use std::fs;

use nostream_config::load::merge;
Expand Down
5 changes: 5 additions & 0 deletions crates/nostream-core/tests/lifecycle.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
//! The drain flag must survive a drain that starts before any subscriber exists.

// This integration-test target may unwrap: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used)]

use nostream_core::lifecycle::ShutdownState;

#[tokio::test]
Expand Down
5 changes: 5 additions & 0 deletions crates/nostream-core/tests/secret.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
//! `Secret` redacts through every formatting path that reaches a log line.

// This integration-test target may unwrap: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used)]

use nostream_core::Secret;

#[test]
Expand Down
5 changes: 5 additions & 0 deletions crates/nostream-storage/tests/storage.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
//! The LMDB environment slice and the backend stubs.

// This integration-test target may unwrap: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used)]

use std::path::Path;

use nostream_storage::{Backend, LmdbEnv, StorageError};
Expand Down
5 changes: 5 additions & 0 deletions crates/nostreamd/tests/acceptor.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
//! The acceptor's drain behaviour.

// This integration-test target may unwrap: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used)]

use std::time::Duration;

use nostream_core::lifecycle::ShutdownState;
Expand Down
5 changes: 5 additions & 0 deletions crates/nostreamd/tests/check_flag.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@
//! waits for a signal. Regression test for round 3's P0: the gate alone passed while `run()`
//! fell through into `serve`.

// This integration-test target may unwrap and panic: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]

use std::path::PathBuf;
use std::process::{Command, Stdio};
use std::time::{Duration, Instant};
Expand Down
5 changes: 5 additions & 0 deletions crates/nostreamd/tests/settings_gate.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
//! A malformed user settings file must serve on the defaults, not abort (R7's fallback rule).

// This integration-test target may unwrap: a failed assertion here is a failed test,
// which is the point. The allowance names this file rather than exempting test targets
// workspace-wide, so a new test file still has to ask.
#![allow(clippy::unwrap_used, clippy::expect_used)]

use std::fs;
use std::path::PathBuf;

Expand Down
Loading