From 7eda967c61b8306b83270f44a576d84df2d431fe Mon Sep 17 00:00:00 2001 From: David Abram Date: Tue, 10 Nov 2026 12:56:42 +0100 Subject: [PATCH 1/9] context: Add config schema series compatibility plan Define the implementation plan for accepting config schema declarations within a major.minor release series and repairing older canonical declarations during setup. The plan covers series-derived validation, typed classification, guarded schema-only upgrades, setup ordering and reporting, end-to-end checks, and documentation. Plan: config-schema-series-compatibility Tasks: T01-T08 Co-authored-by: SCE --- .../config-schema-series-compatibility.md | 366 ++++++++++++++++++ 1 file changed, 366 insertions(+) create mode 100644 context/plans/config-schema-series-compatibility.md diff --git a/context/plans/config-schema-series-compatibility.md b/context/plans/config-schema-series-compatibility.md new file mode 100644 index 000000000..62e5f372c --- /dev/null +++ b/context/plans/config-schema-series-compatibility.md @@ -0,0 +1,366 @@ +# Plan: config-schema-series-compatibility + +## Change summary + +Config files currently break on every release. This plan makes SCE config +`$schema` declarations compatible within a `major.minor` release series. It also +makes `sce setup` upgrade an older-series declaration in the canonical repository +and global config files. The upgrade replaces only `$schema`, validates the full +candidate before saving, and saves through the existing guarded atomic-write +path. Validation, `sce config validate`, `sce doctor`, and runtime config +loading stay read-only and keep validating every field against the embedded +schema. + +This plan extends existing behavior. It is unrelated to PR #351 +(`fix/sce-login-status`, open, all checks passing), which changes saved-login +status reporting in doctor, token storage, the `sce-info` plugin, and hook +assets. None of its 26 changed files overlap this plan's Rust config or setup +files. This branch (`fix/schema-versions`) currently points at PR #351's head +(`3e9e313f`), so the plan stacks on that work. Rebase onto `main` after #351 +merges. Do not fold this work into that PR. + +### Current behavior and causes of release-to-release incompatibility + +1. **Exact-version `const`.** `config/pkl/base/sce-config-schema.pkl` builds + `configSchemaUrl` from the full `.version` string + (`https://sce.crocoder.dev/v0.4.0-pre-alpha-v14/config.json`). It uses that + URL as both the schema `$id` and the `const` of the `$schema` property. Each + `nix run .#bump-version` changes `.version`, so the next CLI's embedded + schema rejects every declaration written by an earlier release, including + earlier releases in the same series. +2. **Setup never revisits an existing declaration.** + `bootstrap_repo_local_config` (`cli/src/services/setup/mod.rs`) returns + early when `.sce/config.json` exists. The access writer + (`cli/src/services/setup/access_preferences.rs`) only sets `$schema` when it + creates a missing repository file. Setup never writes the global file. A + stale declaration therefore remains stale after setup. +3. **A stale declaration invalidates the whole file.** `parse_file_config` + (`cli/src/services/config/schema.rs`) fails on the `/$schema` `const` + mismatch. In `resolve_runtime_config_with` + (`cli/src/services/config/resolver.rs`), a failing default-discovered file + is skipped and recorded in `validation_errors`. Every setting in that file + is ignored, including `auth.mode`, `telemetry.enabled`, + `agent_trace.repository_remote`, integrations, and custom policies. The + access policy degrades to `TelemetryDisabledReason::InvalidConfig` and + `background_cloud_allowed = false`. An explicit `--config` or + `SCE_CONFIG_FILE` target fails hard. +4. **Setup preflights read the file before an upgrade could run.** In + `SetupCommand::execute` (`cli/src/services/setup/command.rs`): + - `resolve_setup_repository` calls + `resolve_agent_trace_storage_runtime_config`. A stale local file is + skipped, so the named-remote preflight checks `origin` instead of the + configured `agent_trace.repository_remote`. It can fail or check the wrong + remote. + - `prepare_dispatch` then reads the following inputs, all through + `parse_file_config`: + - the access policy, which loses any global or repository restriction in + the stale file; + - `read_repository_access_preferences`, which returns + `AccessPreferences::Invalid` and skips access prompts; + - `persisted_integration_targets`, `persisted_optional_workflows`, and + `persisted_setup_confirmations`, which drop saved selections. + - `persist_repository_access_preferences` refuses an invalid file. + - `persist_integration_targets` skips persistence for an invalid local file. + - App startup (`cli/src/app.rs` → `app_support::invalid_discovered_config_guidance`) + logs the generic "Local `.sce` config is invalid" guidance. It does not say + that `sce setup` can repair the problem. + +### Target behavior for each declaration shape + +| Declaration in an existing canonical file | Validation (all read paths) | `sce setup` | +|---|---|---| +| `$schema` absent | Accepted; other fields validated as today | Unchanged; no declaration added; no report line | +| Canonical URL in the CLI series (for a `0.4` CLI: `0.4.0-pre-alpha-v1`, `-v14`, `0.4.0`, `0.4.7`, `0.4.7+build.5`) | Accepted | Unchanged, even when patch or prerelease differs; no report line | +| Canonical URL in an older series (`0.3.3`, `0.1.0`; with a `0.5` CLI, any `0.4.x`) | Rejected with an actionable message naming both series and `sce setup` | Candidate with only `$schema` set to the CLI's concrete URL is validated. A passing candidate is saved and reported as upgraded. A failing candidate leaves the file byte-identical and reports the validation errors with remediation | +| Canonical URL in a newer series (`0.5.0`, `1.4.0` for a `0.4` CLI) | Rejected with a message to upgrade the SCE CLI | Never downgraded; byte-identical; reported as left unchanged | +| Malformed or foreign value (`https://example.com/s.json`, unversioned `https://sce.crocoder.dev/config.json`, `…/v0.4/config.json`, `http://…`, non-string) | Rejected with a message naming the current canonical URL | Never rewritten; byte-identical; reported as unrecognized | +| Explicit `--config` / `SCE_CONFIG_FILE` file at a non-canonical path | Same validation rules | Never written | + +When both canonical files exist, setup evaluates and saves each one +independently. One invalid file does not block upgrading the other. A file +that is still invalid keeps the existing degraded-config behavior. An invalid +local file skips access prompts and target persistence. An invalid global file +is skipped by resolution. A missing global file is never created to upgrade a +schema. Repository bootstrap and the access writer create missing repository +files with the current URL as they do today. + +## Acceptance criteria + +How this plan is proven complete. Each criterion is observable and names the +check that proves it. `/validate` runs these checks; no task in the stack +performs final validation. + +- [ ] AC1: The embedded config schema accepts every canonical `https://sce.crocoder.dev/v..[-pre][+build]/config.json` declaration in the CLI's `major.minor` series. It rejects other series, malformed or foreign URLs, and non-string values. Every other field is still validated. + - Validate: `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema_declaration`, which runs the embedded-schema compatibility matrix and the classifier parity tests, plus the `$schema`-valid/other-field-invalid case. +- [ ] AC2: The accepted series comes from `.version` alone. Changing the release series changes the generated `$schema` rule and the Rust classifier without editing any other constant. + - Validate: `nix run .#pkl-check-generated`, which runs the Pkl series-pattern fixtures for `0.4.0-pre-alpha-v14`, `0.5.0`, and `1.4.2`. Also confirm that `nix shell nixpkgs#ripgrep -c rg -n '0\\.4' config/pkl/base/sce-config-schema*.pkl cli/src/services/config/schema_declaration.rs` finds no hard-coded series outside test fixtures. +- [ ] AC3: `sce config validate`, `sce doctor` (including `--fix`), and ordinary command startup never modify a config file with an older, newer, or foreign declaration. Each prints an actionable `$schema` message. + - Validate: the T07 end-to-end scenario's read-only phase compares file hashes before and after each command. +- [ ] AC4: `sce setup` upgrades an older-series declaration in the existing repository `.sce/config.json` and the existing global config. Only the `$schema` value changes, and every other byte is preserved, including auth policy, telemetry consent, integrations, and custom policies. A second run leaves both files byte-identical. + - Validate: `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema_upgrade` plus the T07 scenario's upgrade and idempotency phases. +- [ ] AC5: Setup never writes a newer-series, malformed, foreign, absent, or same-series declaration. It never creates a missing global config for a schema upgrade, and it never writes an explicit `--config`/`SCE_CONFIG_FILE` file at a non-canonical path. + - Validate: `schema_upgrade` unit matrix plus the `setup` command tests from T05. +- [ ] AC6: When an older-series candidate fails validation, the file stays byte-identical and setup reports the path, the validation errors, and the remediation. Any other canonical file is still upgraded. + - Validate: `schema_upgrade` candidate-invalid test plus the T07 mixed-file phase. +- [ ] AC7: Upgrade writes keep the existing protections. A symlinked file, whether its target is valid or dangling, is refused and left untouched. A file whose bytes change between planning and commit is refused. The write is atomic and keeps the file mode. + - Validate: `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema_upgrade` (symlink, dangling-symlink, concurrent-modification, and mode cases) and `… test … access_preferences`, which must stay unchanged after the T03 refactor. +- [ ] AC8: An older-series declaration does not change setup's preflights. The remote preflight checks the remote configured in a stale but otherwise valid local file. Access prompts, the access gate, and saved selections come from the upgraded files. A failed Git or remote preflight writes nothing. + - Validate: `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml setup` (T05 ordering tests). + +### Full validation + +Repository-wide checks `/validate` runs after the last task, regardless of +which criterion they map to. + +- `nix flake check` +- `nix run .#pkl-check-generated` +- `nix develop -c ./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` +- `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml` + +### Context sync + +- `context/sce/config-schema-publication.md`: series-scoped `$schema` rule; `$id` unchanged; snapshot policy. +- `context/cli/config-precedence-contract.md`: `$schema` compatibility rule, actionable messages, and the read-only guarantee. Also repair the drifted unversioned `https://sce.crocoder.dev/config.json` mention. +- `context/sce/setup-repo-local-config-bootstrap.md`: schema-upgrade ordering, the global `$schema`-only write, the `Config schema:` report section, and the replacement for the statement that the global config is only read. +- `context/glossary.md`: "config schema compatibility series". +- `context/decisions/`: a new ADR for setup writing only the global `$schema` value. It amends the read-only global-config statement in `2026-10-22-setup-writes-access-answers-to-repository-config.md` without changing the rule that access answers go only to the repository config. + +## Task context synchronization lifecycle + +Persist this field in every plan; this is durable plan state, not chat state: + +- **Task context synchronization:** every task carries `pending | synced | blocked`. + A completed task must be `synced` before another task can start or the plan can + finish. +- For `blocked`, record **Blocker**, **Required action**, and **Retry condition** + beside the status. Never infer `synced` from conversation history; write every + lifecycle transition to the plan file. + +## Constraints and non-goals + +- **In scope:** + - Pkl schema: `config/pkl/base/sce-config-schema.pkl`, a new `config/pkl/base/sce-config-schema-url.pkl` helper, and `config/pkl/renderers/generation-contract-check.pkl`. + - Config: `cli/src/services/config/schema.rs`, a new `cli/src/services/config/schema_declaration.rs`, and `cli/src/services/config/mod.rs`. In `cli/src/services/config/resolver.rs`, only the visibility of `resolve_agent_trace_storage_runtime_config_with` changes. + - Setup: `cli/src/services/setup/access_preferences.rs`, `cli/src/services/setup/atomic_write.rs`, a new `cli/src/services/setup/config_file_write.rs`, a new `cli/src/services/setup/schema_upgrade.rs`, `cli/src/services/setup/mod.rs`, and `cli/src/services/setup/command.rs`. + - App startup: the `cli/src/services/app_support.rs` guidance line. + - End-to-end: a new `scripts/test-setup-schema-upgrade.sh` and its `flake.nix` check. + - Context files listed under Context sync. +- **Out of scope:** + - Rewriting historical `schema/v*/config.json` snapshots, including the released `v0.4.0-pre-alpha-v14` snapshot. + - Changes to `nix/bump-version.sh`; it already regenerates the snapshot from `.version`. + - Field migrations. + - Network schema fetches. + - Upgrades from `sce doctor --fix` or `sce config`. + - The Agent Trace schema. + - PR #351's login-status work. +- **Constraints:** + - No new crate dependency. Version parsing is a small typed parser; `semver` is not added. + - Static dispatch and typed error enums, as `AGENTS.md` requires. + - Every candidate is validated through `parse_file_config` with the file's own `ConfigPathSource`, using the same validator as all other read paths. + - Writes go through the existing reread-compare, symlink-refusal, and `write_atomically` sequence. + - The Git-repo gate and named-remote preflight still run before every setup write. + - Startup, validate, and doctor resolution stay read-only. +- **Non-goal:** + - A general config migration framework, or accepting declarations from other series "leniently". + - Adding `$schema` to existing files that omit it. + - Normalizing formatting of an upgraded file. + +## Assumptions + +- The canonical declaration form is exactly `https://sce.crocoder.dev/v/config.json`. Every historical `schema/v*/config.json` snapshot (`v0.1.0` through `v0.4.0-pre-alpha-v14`) uses it, so it is the only form setup recognizes as an upgradable older declaration. +- The CLI series comes from `services::version::PACKAGE_VERSION`. It reads `SCE_VERSION` from `.version` and falls back to `CARGO_PKG_VERSION`, which `bump-version` and the version-parity check keep equal. The Pkl series comes from the same `.version`. +- The schema `$id` stays the CLI's concrete versioned URL. Only the `$schema` property changes from `const` to a series `pattern`. Published snapshots therefore keep a concrete identity while accepting same-series declarations. +- Schema-upgrade outcomes are reported but never fatal. This matches the existing degraded handling of invalid discovered config: setup continues and exits successfully, and a still-invalid file keeps today's degraded behavior. Upgrade write failures do not reuse the access writer's fatal `SCE-ERR-RUNTIME` path, because a global config managed through dotfile symlinks should not block repository setup. +- Upgrades run in every normal setup mode (interactive, non-interactive, target flags, and hooks-only) but not in context-only (`--bootstrap-context`). Context-only setup performs no config work. +- An upgraded file keeps its original bytes apart from the declaration string. When the old URL string occurs exactly once and swapping it yields a structurally identical document apart from `$schema`, only that literal is replaced. Otherwise the candidate uses the same `serde_json::to_string_pretty` plus trailing newline that the access writer already uses. + +## Task stack + +- [ ] T01: `Derive a series-scoped $schema rule from .version in the Pkl schema` (status:todo) + - Task ID: T01 + - Scope: + - In: + - Add `config/pkl/base/sce-config-schema-url.pkl`. From a version string, it produces the concrete URL and the series `pattern`, with `major.minor` escaped. The pattern accepts patch `0|[1-9][0-9]*`, optional `-` prerelease, and optional `+` build, anchored to `^https://sce\.crocoder\.dev/v…/config\.json$`. + - Switch `sce-config-schema.pkl` to use the helper. `$id` uses the concrete URL. The `$schema` property uses `type: string` plus the `pattern`, and a description that names the series. + - Add fixture assertions to `generation-contract-check.pkl` for `0.4.0-pre-alpha-v14`, `0.5.0`, and `1.4.2`. + - Replace `accepts_versioned_schema_declaration` in `schema.rs` with an embedded-schema matrix built from `PACKAGE_VERSION`'s series. The matrix accepts same-series stable, prerelease, and build declarations. It rejects other series, malformed values, and foreign values. It keeps the case where `$schema` is valid but another field is invalid. + - Out: the Rust classifier (T02) and historical snapshots. + - Dependencies: none + - Done when: + - The generated schema's `$schema` property is a series pattern derived from `.version`. + - The Pkl fixtures prove that a different `.version` series yields a different pattern. + - The embedded-schema matrix passes against the current series with no literal `0.4` outside fixtures. + - Verify: + - `nix run .#pkl-check-generated` + - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema` + - Context synchronization: pending + +- [ ] T02: `Add typed schema-declaration classification and actionable $schema errors` (status:todo) + - Task ID: T02 + - Scope: + - In: + - Add `cli/src/services/config/schema_declaration.rs` with these types: + - `SchemaVersion`: major, minor, and patch, plus a prerelease and build that are parsed but not compared. + - `SchemaSeries`. + - `current_schema_series()`, parsed from `PACKAGE_VERSION`. + - `classify_schema_declaration(value: Option<&Value>, current: SchemaSeries) -> SchemaDeclaration`, returning `Absent | CurrentSeries | OlderSeries(SchemaVersion) | NewerSeries(SchemaVersion) | Unrecognized`. + - Add a parity test. For the same URL matrix, embedded-schema acceptance of `{"$schema": url}` must equal `CurrentSeries`. Add classifier tests with an injected `0.5` series. They show that `0.4.0-pre-alpha-v1`, `0.4.7`, and `0.3.3` are `OlderSeries`, and that `1.4.0` is `NewerSeries`. + - In `validate_config_value_against_schema`, replace the generic `/$schema` pattern error with a message for each class: + - Older: "declares SCE config schema series X.Y; this CLI uses A.B; run 'sce setup' to upgrade the repository and global config, or set `$schema` to ". + - Newer: "… was written by a newer SCE release; upgrade the SCE CLI". + - Unrecognized: "… is not a canonical SCE schema URL; use ". + - Keep every other schema error and the sorted join. + - In `app_support::invalid_discovered_config_guidance`, mention `sce setup` when the only failure is an older-series declaration. + - Out: any write path, and setup wiring. + - Dependencies: T01 + - Done when: + - The classifier and embedded schema agree on the full matrix. + - `sce config validate` on a file declaring `0.3.3`, `0.5.0`, or a foreign URL reports the matching message, and the file is unchanged. + - Other field errors are still reported alongside the `$schema` message. + - Verify: + - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema_declaration` + - `… test … config` + - Context synchronization: pending + +- [ ] T03: `Extract the guarded config-file write shared by setup writers` (status:todo) + - Task ID: T03 + - Scope: + - In: + - Move the `ConfigLayer` enum and the read-original, reread-compare, symlink-refusal (`fs::symlink_metadata`), and `write_atomically` sequence into `cli/src/services/setup/config_file_write.rs`. Expose them through a typed `GuardedWriteError { ConcurrentModification, SymlinkManaged, Io(…) }` and a `write_guarded(path, layer, original: Option<&[u8]>, serialized: &[u8], reread)` function. + - Port `access_preferences.rs` to use the new module with byte-identical behavior and messages. + - Out: any new behavior. + - Dependencies: none + - Done when: + - `access_preferences` uses the shared helper. + - All existing access-preference tests pass unchanged, including the symlink, dangling-symlink, concurrent-modification, invalid-file, and mode-preservation tests. + - Verify: + - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml access_preferences` + - `… test … setup` + - Context synchronization: pending + +- [ ] T04: `Plan and commit a validated $schema-only upgrade for one config file` (status:todo) + - Task ID: T04 + - Scope: + - In: + - Add `cli/src/services/setup/schema_upgrade.rs`. It defines: + - `plan_schema_upgrade(path, layer) -> SchemaUpgradePlan`, with variants `NotPresent | NoDeclaration | Current | NewerSeries(version) | Unrecognized | Unreadable(reason) | Ready { original, candidate, from }` and `CandidateInvalid { from, reason }`. + - `commit_schema_upgrade(path, layer, plan) -> SchemaUpgradeOutcome`, with variants `Upgraded { from } | Unchanged | Skipped(SchemaUpgradeSkip)`. + - A typed `SchemaUpgradeSkip { NewerSeries, Unrecognized, CandidateInvalid, SymlinkManaged, ConcurrentModification, Io }`. + - Planning reads the file, classifies `$schema` with T02, and builds the candidate. The candidate uses the byte-preserving literal replacement, with a structural-equality guard and a pretty-serialization fallback. Planning validates the candidate with `parse_file_config` and the layer's `ConfigPathSource`. Planning never writes. + - Committing calls T03's `write_guarded` against the planned original bytes. + - Unit tests cover: + - absent file; + - absent `$schema`; + - same-series patch and prerelease values preserved; + - `0.3.3` upgraded with all other bytes preserved, using a fixture that contains `auth.mode`, `telemetry.enabled`, `integrations`, and custom `policies.bash`; + - a second plan returning `Current`; + - newer and foreign values untouched; + - an older file with an unknown key yielding `CandidateInvalid` with a byte-identical file; + - symlink and dangling-symlink refusal; + - concurrent modification between plan and commit; + - mode preservation; + - the global layer validated as `DefaultDiscoveredGlobal`. + - Out: setup ordering and reporting. + - Dependencies: T02, T03 + - Done when: + - Each declaration shape in the change summary table produces the specified plan and outcome. + - Only the `Ready` plan writes, and it writes only after successful validation. + - All protections hold. + - Verify: + - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema_upgrade` + - Context synchronization: pending + +- [ ] T05: `Run schema upgrades before setup's preflight readers` (status:todo) + - Task ID: T05 + - Scope: + - In: + - In `SetupCommand::execute`, plan upgrades after `ensure_git_repository`, unless the run is context-only. The local path comes from `RepoPaths::sce_config_file()`. The global path comes from `resolve_default_global_config_path()` and is planned only if the file exists; if the path does not resolve, setup skips the global upgrade. + - Make `resolve_agent_trace_storage_runtime_config_with` `pub(crate)`. Call it with a `read_file` overlay that serves `Ready` candidate bytes for exactly those two paths. The named-remote preflight then sees the validated candidate, and the resolver itself stays unchanged. + - After the remote preflight passes, commit the plans and keep the outcomes. + - Run `prepare_dispatch`, the access writer, lifecycle bootstrap, and target persistence after the commit, so they read the upgraded files. + - Split `resolve_setup_repository` as needed. Never pass an explicit config path to the planner. + - Add tests: + - A stale local file with `agent_trace.repository_remote: "upstream"` in a repo that has only `upstream` passes the preflight and is upgraded. + - The same repo with a failing remote preflight writes nothing. + - A stale global file with `auth.mode: "disabled"` locks the access gate after the upgrade. + - A stale local file's saved `auth.mode`, telemetry, targets, and optional workflows drive dispatch. + - A missing global file is not created. + - Context-only mode writes nothing. + - With an invalid stale local file and a valid stale global file, the global file is upgraded and the local file stays byte-identical, with prompts skipped as today. + - Out: report rendering (T06). + - Dependencies: T04 + - Done when: AC8 holds and the existing `setup` tests pass. + - Verify: + - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml setup` + - Context synchronization: pending + +- [ ] T06: `Report schema upgrade outcomes in setup output` (status:todo) + - Task ID: T06 + - Scope: + - In: + - Render a `Config schema:` section as the first setup section. It renders only when some file is `Upgraded` or `Skipped`, with one line for each file, using fixed wording: + - Upgraded: `upgraded '' from v to v`. + - Newer series: `left '' unchanged; it declares newer schema series X.Y; upgrade the SCE CLI`. + - Unrecognized: `left '' unchanged; '$schema' is not a canonical SCE schema URL; set it to `. + - Candidate invalid: `left '' unchanged; upgrading '$schema' to v would fail validation: . Fix the listed fields, then rerun 'sce setup'.` + - Symlink-managed or concurrent: the existing wording style, telling the user to update the source directly or rerun. + - Add renderer unit tests for each line. + - The exit code stays successful. + - Out: changes to other sections. + - Dependencies: T05 + - Done when: + - Every non-silent outcome renders its fixed line. + - Silent outcomes (absent file, no declaration, current series) render nothing. + - Verify: + - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml setup` + - Context synchronization: pending + +- [ ] T07: `Add an end-to-end setup schema-upgrade scenario check` (status:todo) + - Task ID: T07 + - Scope: + - In: + - Add `scripts/test-setup-schema-upgrade.sh`, following the isolation pattern in `scripts/stress-test-conversation-trace-firehose-mixed.sh`: a built `sce` binary, a temporary Git repo with a dummy `origin`, and isolated `HOME`, `XDG_CONFIG_HOME`, and `XDG_STATE_HOME`. Wire it into `flake.nix` as a `checks` entry. + - The script runs these phases: + 1. **Read-only.** With `0.3.3` declarations in both the global and local files, and in a newer and a foreign variant, `sce version`, `sce config validate`, `sce config show`, `sce doctor`, and `sce doctor --fix` leave file hashes unchanged and print the `$schema` messages. + 2. **Upgrade.** `sce setup --hooks --repo ` upgrades both files. The diff against the originals is exactly the `$schema` line, and `sce config validate` then passes. + 3. **Idempotency.** A second run leaves both files byte-identical and prints no `Config schema:` section. + 4. **Mixed.** With a stale invalid local file and a stale valid global file, the global file is upgraded and the local file is byte-identical, and the report names the local file's errors. + 5. **Newer.** A newer-series local file stays byte-identical. + 6. **Symlink.** A symlinked global config is reported and left untouched. + - Out: VM or npm scenarios. + - Dependencies: T06 + - Done when: the script passes locally and under `nix flake check`, covering AC3 through AC7 end to end. + - Verify: + - `nix develop -c ./scripts/test-setup-schema-upgrade.sh` + - `nix flake check` + - Context synchronization: pending + +- [ ] T08: `Document series compatibility and setup schema upgrades` (status:todo) + - Task ID: T08 + - Scope: + - In: + - Update the four context files listed under Context sync. This includes fixing the drifted unversioned schema URL line in `config-precedence-contract.md`, and replacing "The global config is only read" in `setup-repo-local-config-bootstrap.md` with the `$schema`-only global write. + - Add the glossary entry. + - Add an ADR in `context/decisions/` titled roughly `setup-upgrades-config-schema-declarations`. It records: + - acceptance by series; + - that setup is the only writer; + - the global write limited to `$schema`; + - no downgrade; + - no field migrations; + - non-fatal reporting; + - historical snapshots left unchanged, with the next `nix run .#bump-version` publishing the first series-pattern snapshot. + - Out: README and npm docs. Neither currently documents `$schema`. + - Dependencies: T07 + - Done when: + - Context describes the implemented behavior with no remaining claim of an exact-version `const`, a read-only global config, or an unversioned declaration. + - The ADR is linked from the setup bootstrap and schema publication files. + - Verify: + - `nix shell nixpkgs#ripgrep -c rg -n 'const.*config.json|global config is only read|sce.crocoder.dev/config.json' context --glob '!context/plans/**' --glob '!context/decisions/**'` returns no stale claims. + - Context synchronization: pending + +## Open questions + +None. The one judgment call is that schema-upgrade failures are reported but +not fatal. It is recorded under Assumptions with its rationale. It follows the +documented degraded-invalid-config setup behavior. Revise the plan if setup +should instead exit non-zero when a stale canonical file cannot be upgraded. From 1059b0c87e321f5b5bd2a7ded2dc29fe90280356 Mon Sep 17 00:00:00 2001 From: David Abram Date: Tue, 10 Nov 2026 13:03:37 +0100 Subject: [PATCH 2/9] config: Accept same-series $schema declarations Allow canonical config schema URLs from the CLI's major.minor release series instead of requiring an exact version, while retaining strict semver URL validation and concrete schema IDs. Add Pkl URL helpers and generated-schema contract checks, plus Rust acceptance coverage for valid, foreign, malformed, and non-string declarations. Document the compatibility decision and update the config contracts, glossary, context map, and completed plan evidence. Plan: `context/plans/config-schema-series-compatibility.md` (T01) Co-authored-by: SCE --- cli/src/services/config/schema.rs | 129 ++++++++++++++++-- config/pkl/base/sce-config-schema-url.pkl | 17 +++ config/pkl/base/sce-config-schema.pkl | 7 +- .../renderers/generation-contract-check.pkl | 30 ++++ context/cli/config-precedence-contract.md | 2 +- context/context-map.md | 2 +- ...ema-accepts-release-series-declarations.md | 57 ++++++++ context/glossary.md | 2 +- .../config-schema-series-compatibility.md | 23 +++- context/sce/config-schema-publication.md | 13 +- 10 files changed, 263 insertions(+), 19 deletions(-) create mode 100644 config/pkl/base/sce-config-schema-url.pkl create mode 100644 context/decisions/2026-11-10-config-schema-accepts-release-series-declarations.md diff --git a/cli/src/services/config/schema.rs b/cli/src/services/config/schema.rs index 3c8c3369f..2743b2532 100644 --- a/cli/src/services/config/schema.rs +++ b/cli/src/services/config/schema.rs @@ -970,15 +970,6 @@ mod agent_trace_config_tests { ) } - #[test] - fn accepts_versioned_schema_declaration() { - let schema_url = format!( - "https://sce.crocoder.dev/v{}/config.json", - env!("CARGO_PKG_VERSION") - ); - parse(&format!(r#"{{"$schema":"{schema_url}"}}"#)).unwrap(); - } - #[test] fn parses_agent_trace_repository_identity_keys() { let config = parse( @@ -1114,6 +1105,126 @@ mod agent_trace_config_tests { } } +#[cfg(test)] +mod schema_declaration_acceptance_tests { + use std::path::Path; + + use serde_json::{json, Value}; + + use super::{parse_file_config, ConfigPathSource}; + use crate::services::version::PACKAGE_VERSION; + + fn current_series() -> (u64, u64) { + let mut parts = PACKAGE_VERSION.split('.'); + let major = parts.next().unwrap().parse().unwrap(); + let minor = parts.next().unwrap().parse().unwrap(); + (major, minor) + } + + fn schema_url(version: &str) -> String { + format!("https://sce.crocoder.dev/v{version}/config.json") + } + + fn parse_value(value: &Value) -> anyhow::Result { + parse_file_config( + &value.to_string(), + Path::new("/tmp/sce-config.json"), + ConfigPathSource::Flag, + ) + } + + fn declaration_accepted(declaration: Value) -> bool { + parse_value(&json!({ "$schema": declaration })).is_ok() + } + + #[test] + fn schema_declaration_accepts_current_series_releases() { + let (major, minor) = current_series(); + let accepted = [ + PACKAGE_VERSION.to_string(), + format!("{major}.{minor}.0"), + format!("{major}.{minor}.7"), + format!("{major}.{minor}.0-pre-alpha-v1"), + format!("{major}.{minor}.0-pre-alpha-v14"), + format!("{major}.{minor}.7+build.5"), + format!("{major}.{minor}.12-rc.1+build.5"), + ]; + + for version in accepted { + let url = schema_url(&version); + assert!(declaration_accepted(json!(url)), "expected {url} accepted"); + } + } + + #[test] + fn schema_declaration_rejects_other_series() { + let (major, minor) = current_series(); + let mut rejected = vec![ + format!("{major}.{}.0", minor + 1), + format!("{}.{minor}.0", major + 1), + format!("{}.0.0", major + 1), + ]; + if minor > 0 { + rejected.push(format!("{major}.{}.3", minor - 1)); + } + if major > 0 { + rejected.push(format!("{}.{minor}.0", major - 1)); + } + + for version in rejected { + let url = schema_url(&version); + assert!(!declaration_accepted(json!(url)), "expected {url} rejected"); + } + } + + #[test] + fn schema_declaration_rejects_malformed_and_foreign_values() { + let (major, minor) = current_series(); + let rejected = [ + json!("https://example.com/s.json"), + json!("https://sce.crocoder.dev/config.json"), + json!(schema_url(&format!("{major}.{minor}"))), + json!(schema_url(&format!("{major}.{minor}.01"))), + json!(schema_url(&format!("{major}.{minor}.x"))), + json!(format!( + "http://sce.crocoder.dev/v{major}.{minor}.0/config.json" + )), + json!(format!( + "https://sce.crocoder.dev/v{major}.{minor}.0/config.json.bak" + )), + json!(format!( + "https://sce.crocoder.dev/v{major}x{minor}.0/config.json" + )), + json!(format!( + " https://sce.crocoder.dev/v{major}.{minor}.0/config.json" + )), + json!(1), + json!(null), + json!({ "url": schema_url(PACKAGE_VERSION) }), + ]; + + for value in rejected { + assert!( + !declaration_accepted(value.clone()), + "expected {value} rejected" + ); + } + } + + #[test] + fn schema_declaration_valid_still_validates_other_fields() { + let error = parse_value(&json!({ + "$schema": schema_url(PACKAGE_VERSION), + "log_level": "loud", + })) + .unwrap_err() + .to_string(); + + assert!(error.contains("log_level"), "unexpected error: {error}"); + assert!(!error.contains("$schema"), "unexpected error: {error}"); + } +} + #[cfg(test)] mod database_retry_config_tests { use std::path::Path; diff --git a/config/pkl/base/sce-config-schema-url.pkl b/config/pkl/base/sce-config-schema-url.pkl new file mode 100644 index 000000000..7a19ec2cf --- /dev/null +++ b/config/pkl/base/sce-config-schema-url.pkl @@ -0,0 +1,17 @@ +local semverPattern = #"(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?"# + +local function versionParts(version: String): List = + if (version.matches(Regex(semverPattern))) version.split(".") + else throw("SCE version '\(version)' is not a semantic version") + +function concreteUrl(version: String): String = + "https://sce.crocoder.dev/v\(version)/config.json" + +function series(version: String): String = + let (parts = versionParts(version)) "\(parts[0]).\(parts[1])" + +function seriesPattern(version: String): String = + let (parts = versionParts(version)) + #"^https://sce\.crocoder\.dev/v"# + + "\(parts[0])\\.\(parts[1])" + + #"\.(0|[1-9][0-9]*)(-[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?/config\.json$"# diff --git a/config/pkl/base/sce-config-schema.pkl b/config/pkl/base/sce-config-schema.pkl index 5af7ba8f1..7afcc71c1 100644 --- a/config/pkl/base/sce-config-schema.pkl +++ b/config/pkl/base/sce-config-schema.pkl @@ -1,9 +1,11 @@ import "bash-policy-presets.pkl" as bash_policy_presets +import "sce-config-schema-url.pkl" as schema_url import "workflow-catalog.pkl" as catalog import "../deps/org.json_schema/JsonSchema.pkl" local presetIds = bash_policy_presets.presets.toList().map((preset) -> preset.id).toListing() -local configSchemaUrl = "https://sce.crocoder.dev/v\(read("../../../.version").text.trim())/config.json" +local sceVersion = read("../../../.version").text.trim() +local configSchemaUrl = schema_url.concreteUrl(sceVersion) local optionalWorkflowIds = catalog.workflows.toMap() .filter((_, workflow) -> workflow.optional) @@ -98,7 +100,8 @@ local sceConfigSchema = new JsonSchema { properties { ["$schema"] = new JsonSchema { type = "string" - `const` = configSchemaUrl + description = "Canonical SCE config schema URL. Accepts any release in the \(schema_url.series(sceVersion)) series." + pattern = schema_url.seriesPattern(sceVersion) } ["log_level"] = new JsonSchema { type = "string" diff --git a/config/pkl/renderers/generation-contract-check.pkl b/config/pkl/renderers/generation-contract-check.pkl index 84ec26a8a..9dd0a991a 100644 --- a/config/pkl/renderers/generation-contract-check.pkl +++ b/config/pkl/renderers/generation-contract-check.pkl @@ -5,6 +5,7 @@ import "claude-content.pkl" as claude import "pi-content.pkl" as pi import "codex-content.pkl" as codex import "codex-metadata.pkl" as codexMetadata +import "../base/sce-config-schema-url.pkl" as schemaUrl hidden expectedArtifactPaths = new Mapping { for (_, document in opencode.agents) { @@ -1243,12 +1244,41 @@ hidden assertWorkflowLanguageConsistency = (artifacts: Mapping) -> ) "workflow language: shared vocabulary, phase naming, report naming, and document format are consistent" else throw("generated workflow language/format drifted from the shared T11 conventions") +hidden assertConfigSchemaSeriesPattern = (artifacts: Mapping) -> + let (generatedSchema = artifacts["config/schema/sce-config.schema.json"]) + let (pattern04 = schemaUrl.seriesPattern("0.4.0-pre-alpha-v14")) + let (pattern05 = schemaUrl.seriesPattern("0.5.0")) + let (pattern14 = schemaUrl.seriesPattern("1.4.2")) + if ( + pattern04 == #"^https://sce\.crocoder\.dev/v0\.4\.(0|[1-9][0-9]*)(-[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?/config\.json$"# + && pattern05.startsWith(#"^https://sce\.crocoder\.dev/v0\.5\."#) + && pattern14.startsWith(#"^https://sce\.crocoder\.dev/v1\.4\."#) + && pattern04 != pattern05 + && pattern05 != pattern14 + && schemaUrl.series("0.4.0-pre-alpha-v14") == "0.4" + && schemaUrl.series("0.5.0") == "0.5" + && schemaUrl.series("1.4.2") == "1.4" + && schemaUrl.concreteUrl("0.4.0-pre-alpha-v14") == "https://sce.crocoder.dev/v0.4.0-pre-alpha-v14/config.json" + && schemaUrl.concreteUrl("1.4.2") == "https://sce.crocoder.dev/v1.4.2/config.json" + && "https://sce.crocoder.dev/v0.4.0-pre-alpha-v1/config.json".matches(Regex(pattern04)) + && "https://sce.crocoder.dev/v0.4.7+build.5/config.json".matches(Regex(pattern04)) + && !"https://sce.crocoder.dev/v0.5.0/config.json".matches(Regex(pattern04)) + && !"https://sce.crocoder.dev/v0.4/config.json".matches(Regex(pattern04)) + && "https://sce.crocoder.dev/v0.5.0/config.json".matches(Regex(pattern05)) + && "https://sce.crocoder.dev/v1.4.0/config.json".matches(Regex(pattern14)) + && !"https://sce.crocoder.dev/v1.5.0/config.json".matches(Regex(pattern14)) + && generatedSchema.contains("\"pattern\"") + && !generatedSchema.contains("\"const\": \"https://sce.crocoder.dev/") + ) "config schema: $schema accepts a .version-derived major.minor series pattern" + else throw("generated config schema $schema rule must be the series pattern derived from .version, not an exact-version const") + contractChecks { ["workflow-language-consistency"] = assertWorkflowLanguageConsistency.apply(generatedArtifacts) ["workflow-vocabulary-decision-result"] = if (generatedArtifacts["config/.pi/skills/sce-decision/SKILL.md"].contains("internal result") && !generatedArtifacts["config/.pi/skills/sce-decision/SKILL.md"].contains("Return internal state") && !generatedArtifacts["config/.pi/skills/sce-decision/SKILL.md"].contains("internal handoff")) "sce-decision: uses internal-result vocabulary" else throw("sce-decision vocabulary drifted from workflow-vocabulary.md") ["compact-execution-contracts"] = assertCompactExecutionContracts.apply(workflowDocuments) ["compact-execution-contract-negative-fixtures"] = assertCompactExecutionContractFixtures ["artifact-paths"] = assertExactArtifactPaths.apply(generatedArtifacts) + ["config-schema-series-pattern"] = assertConfigSchemaSeriesPattern.apply(generatedArtifacts) ["sce-info-runtime"] = assertSceInfoRuntimeContract.apply(generatedArtifacts) ["codex-hook-invocation"] = assertCodexHookInvocationContract.apply(generatedArtifacts) ["codex-hook-helper-fail-closed"] = assertCodexHelperFailClosedMode.apply(generatedArtifacts) diff --git a/context/cli/config-precedence-contract.md b/context/cli/config-precedence-contract.md index 74dd0231b..dd5cc91e3 100644 --- a/context/cli/config-precedence-contract.md +++ b/context/cli/config-precedence-contract.md @@ -79,7 +79,7 @@ When a default-discovered global or repo-local config file exists but fails JSON - `sce config validate` and `sce doctor` both validate config-file structure against that shared generated schema before applying Rust-owned semantic checks such as duplicate custom `argv_prefix` detection and redundancy warnings. - Each reported schema-validation error is prefixed with the failing value's JSON-pointer location when it has one (for example `/integrations/optional_workflows/0: "nonesuch" is not one of "brownfield"`), so a rejected value names the key it came from; root-level errors keep their unprefixed text. Errors remain sorted and joined with ` | `. - After schema validation, `cli/src/services/config/schema.rs` deserializes top-level and nested config structure (`policies`, `policies.bash`, `policies.attribution_hooks`) into typed serde DTOs and applies focused Rust-owned mapping helpers for enum conversion and source attribution; policy-specific semantic checks are owned by `cli/src/services/config/policy.rs`. -- The canonical top-level schema declaration `"$schema": "https://sce.crocoder.dev/v/config.json"` (where `` is the CLI release version) is a supported config key for both explicit and discovered `sce/config.json` files, including command-startup paths like `sce version` and other config-loading commands that parse config before normal command dispatch. +- The canonical top-level schema declaration `"$schema": "https://sce.crocoder.dev/v/config.json"` is accepted for any `` in the CLI's `major.minor` release series ([series-acceptance decision](../decisions/2026-11-10-config-schema-accepts-release-series-declarations.md)). It is a supported config key for both explicit and discovered `sce/config.json` files, including command-startup paths like `sce version` and other config-loading commands that parse config before normal command dispatch. - Startup/runtime config resolution degrades gracefully for default-discovered files across ordinary startup, setup, and Agent Trace storage: invalid discovered files are skipped and reported via collected `validation_errors`, while explicit `--config` / `SCE_CONFIG_FILE` targets still fail immediately on the same parse or validation errors. - Config file content must be valid JSON with a top-level object. diff --git a/context/context-map.md b/context/context-map.md index c0ea8e312..d189c7c3c 100644 --- a/context/context-map.md +++ b/context/context-map.md @@ -116,7 +116,7 @@ Feature/domain context: - `context/sce/cli-npm-distribution-contract.md` (implemented `sce` npm launcher package, release-manifest/checksum-verified native binary install flow with manifest identity checks, bounded validating archive extraction, and atomic `runtime/sce` install, the supported darwin/arm64 plus linux x64+arm64 npm platform matrix, the executable launcher-mode requirement and offline packed-package smoke test (`npm/test/pack-smoke.sh` on the shared `npm/test/release-fixture.sh` prepare/serve harness, flake check `npm-pack-smoke`, and the `release-npm-package` post-pack gate, plus the opt-in x86_64-linux NixOS VM real-install test `.#npm-nixos-vm` with its rebuild-recovery, failed-reinstall, real `sce setup --bootstrap-context`, and minimal-runtime scenario subtests, and the statement that these hermetic fixtures use a throwaway key and repack, so they do not prove the exact release artifact or production key, plus the `npm/test/release-canary.sh` release-security canary that installs the unmodified tarball with the production key in local-directory, GitHub-release, and registry modes wired as the B1 `release-canary`, B2 `github-release-canary`, and post-publish `registry-canary` gates), the prerequisite that npm-consumed native archives are portable and free of forbidden `/nix/store/` runtime references before upload, and dedicated `.github/workflows/publish-npm.yml` downstream npm publish-stage contract (`workflow_call` from `release-sce.yml`, `NPM_TOKEN` scoped to the publish step, manual reruns, idempotent republish) including prerelease publication via npm `next` dist-tag instead of `latest`) - `context/sce/cli-cargo-distribution-contract.md` (implemented `sce` Cargo publication posture plus supported crates.io and wrapper-driven local-checkout install guidance, explicit unsupported `cargo install --git` boundary, dedicated crates.io publish workflow with semver prerelease guard, canonical generated-input declaration/producer ownership consumed by the Cargo wrapper, generated-output check, package-fallback preparation, and Nix pre-Cargo derivation, plus the deterministic combined checksummed fallback used for Pkl-free downstream crate builds) - `context/sce/flake-build-performance.md` (current native-vs-release package split, release-only commit embedding, shared-producer-backed pre-Cargo generated-input derivation consumed by native/release/test/Clippy but excluded from dependency/format derivations, slim default/database devShell contract, normal-vs-expensive CI tiers, durable before/after benchmark summary, validation evidence, and remaining cold-build trade-offs) -- `context/sce/config-schema-publication.md` (current embedded-only publication contract for the generated SCE config JSON Schema and the canonical SCE web application URL used in config declarations) +- `context/sce/config-schema-publication.md` (current embedded-only publication contract for the generated SCE config JSON Schema, the series-scoped `$schema` acceptance rule derived from `.version`, and the canonical SCE web application URL used in config declarations) - `context/sce/flatpak-distribution-patterns.md` (source-built Flatpak conventions, including Nix-side ephemeral fallback preparation for the Pkl-free build sandbox, manifest generation, local/release builds, release assets, and host-git bridge) - Setup behavior selection contract: [local bootstrap](sce/setup-repo-local-config-bootstrap.md) is canonical for the independent interactive confirmations, explicit nested config values, non-interactive existing-config safety, and unchanged runtime gates; see also [config precedence](cli/config-precedence-contract.md), [automatic sync](cli/agent-trace-auto-sync.md), [hook routing](sce/agent-trace-hooks-command-routing.md), and [commit attribution](sce/agent-trace-commit-msg-coauthor-policy.md). diff --git a/context/decisions/2026-11-10-config-schema-accepts-release-series-declarations.md b/context/decisions/2026-11-10-config-schema-accepts-release-series-declarations.md new file mode 100644 index 000000000..9cd17e90d --- /dev/null +++ b/context/decisions/2026-11-10-config-schema-accepts-release-series-declarations.md @@ -0,0 +1,57 @@ +# Decision: Config schema accepts `$schema` declarations by release series + +Date: 2026-11-10 +Status: Accepted +Plan: `context/plans/config-schema-series-compatibility.md` +Task: `T01` + +## Context + +The generated SCE config schema used the CLI's full release version, read from `.version`, as an exact `const` on the `$schema` property. Every `nix run .#bump-version` therefore made the next CLI reject every config file written by an earlier release, including earlier releases in the same `major.minor` series. In a default-discovered file, the rejection discarded every setting in that file. In an explicit `--config` / `SCE_CONFIG_FILE` file, the rejection was fatal. + +## Decision + +The SCE config schema accepts any canonical `$schema` declaration of the form `https://sce.crocoder.dev/v..[-prerelease][+build]/config.json` within the CLI's `major.minor` release series. + +- The series is derived solely from `.version`, through `config/pkl/base/sce-config-schema-url.pkl`. +- The `$schema` property is `type: string` with an anchored series `pattern`, replacing the exact-version `const`. +- The schema `$id` stays the CLI's concrete versioned URL. +- Declarations from other series, malformed or foreign URLs, and non-string values remain rejected. +- Every other config field is still validated against the embedded schema. + +## Rationale + +Patch and prerelease releases within one series do not change the config contract enough to invalidate existing files. Deriving the series from `.version` keeps a single version source, so a release bump moves the accepted series without editing constants. Keeping a concrete `$id` keeps each published snapshot uniquely identified. + +## Alternatives considered + +- **Keep the exact-version `const`** — Not selected: it breaks every existing config file on every release. +- **Accept any `sce.crocoder.dev` versioned URL** — Not selected: it would silently accept declarations from incompatible series. + +## Compatibility and risks + +- Files declaring an older or newer series are still rejected. The plan's later tasks add actionable messages and a setup-driven `$schema` upgrade for older-series files. +- Historical `schema/v*/config.json` snapshots keep their exact-version `const`. The next `nix run .#bump-version` publishes the first series-pattern snapshot. + +## Guardrails + +- The accepted series comes only from `.version`; no other hard-coded series exists outside test fixtures. +- Only the `major.minor` series is relaxed. The URL host, scheme, path shape, and semver form stay strict. + +## Consequences + +- A same-series release no longer invalidates existing config files. +- A series bump (`major.minor`) still requires users to update their declarations. + +## Follow-up + +- Remaining tasks T02–T08 of the plan: a typed classifier, actionable `$schema` errors, the setup `$schema` upgrade, end-to-end checks, and documentation. + +## References + +- Plan: [`config-schema-series-compatibility`](../plans/config-schema-series-compatibility.md) +- Task: `T01` +- Current-state context: [`config-schema-publication.md`](../sce/config-schema-publication.md) +- Current-state context: [`config-precedence-contract.md`](../cli/config-precedence-contract.md) +- Evidence: [`sce-config-schema-url.pkl`](../../config/pkl/base/sce-config-schema-url.pkl) +- Evidence: [`generation-contract-check.pkl`](../../config/pkl/renderers/generation-contract-check.pkl) diff --git a/context/glossary.md b/context/glossary.md index ce106b60e..9fc90d620 100644 --- a/context/glossary.md +++ b/context/glossary.md @@ -162,7 +162,7 @@ - `config policy semantic validation seam`: Canonical ownership in `cli/src/services/config/policy.rs` for the CLI's bash-policy and attribution-hooks semantic validation, merge helpers, and policy rendering: built-in/custom bash-policy catalog types and OnceLock (`BuiltinBashPolicyCatalog`, `BuiltinBashPolicyPreset`, `BuiltinBashPolicyMatcher`, `BuiltinBashPolicyRedundancyWarning`, `BUILTIN_BASH_POLICY_CATALOG`, `BASH_POLICY_PRESET_CATALOG_JSON`), policy config types (`BashPolicyConfig`, `CustomBashPolicyEntry`), catalog accessors (`builtin_bash_policy_catalog`, `builtin_bash_policy_preset_ids`, `is_builtin_bash_policy_preset_id`), policy parsing and validation (`parse_bash_policy_presets`, `parse_custom_bash_policies`, `parse_custom_bash_policy_entry`, `parse_custom_bash_policy_match`, `parse_custom_bash_policy_argv_prefix`), policy resolution (`resolve_bash_policy_config`, `build_validation_warnings`), and policy rendering (`format_bash_policies_text`, `format_bash_policies_json`); `mod.rs` imports `BashPolicyConfig`, `build_validation_warnings`, `format_bash_policies_json`, `format_bash_policies_text`, and `resolve_bash_policy_config` from `policy` for resolution and rendering consumers. - `config runtime resolver seam`: Canonical ownership in `cli/src/services/config/resolver.rs` for config-file discovery, file-layer merging, env/flag/default precedence resolution, shared auth-key resolution (`workos_client_id`, `control_plane_base_url`), observability runtime resolution, attribution-hooks gate resolution, and default-discovered invalid-file degradation; `control_plane_base_url` defaults to `https://sce.crocoderlab.dev` while the schema and SCE web URL remain `https://sce.crocoder.dev`; `cli/src/services/config/mod.rs` delegates `sce config show|validate` runtime resolution to this seam while facade re-exports preserve startup/auth/hooks callers through `services::config`. - `config render seam`: Canonical ownership in `cli/src/services/config/render.rs` for `sce config show` and `sce config validate` text/JSON output construction, including rendering-specific config-path formatting, resolved-value formatting, validation issue/warning rendering, and auth display-value redaction/abbreviation helpers; `cli/src/services/config/mod.rs` delegates rendering to this private submodule after resolver-owned runtime config resolution. -- `sce config schema artifact`: Canonical JSON Schema for global and repo-local `sce/config.json` files, authored in `config/pkl/base/sce-config-schema.pkl`, generated to payload-relative `config/schema/sce-config.schema.json` beneath Cargo `OUT_DIR` or packaging fallback roots, and embedded by `cli/src/services/config/schema.rs` for shared `sce config validate` and doctor config validation. The current schema accepts the canonical `$schema` declaration, flat logging keys (`log_level`, `log_format`, `log_dir`, positive-integer `log_file_retention_limit`), and existing auth/config keys. +- `sce config schema artifact`: Canonical JSON Schema for global and repo-local `sce/config.json` files, authored in `config/pkl/base/sce-config-schema.pkl`, generated to payload-relative `config/schema/sce-config.schema.json` beneath Cargo `OUT_DIR` or packaging fallback roots, and embedded by `cli/src/services/config/schema.rs` for shared `sce config validate` and doctor config validation. The current schema accepts any canonical `$schema` declaration in the CLI's `major.minor` config schema compatibility series (derived from `.version`), flat logging keys (`log_level`, `log_format`, `log_dir`, positive-integer `log_file_retention_limit`), and existing auth/config keys. - `log_file_retention_limit`: Flat top-level `sce/config.json` key validated as an integer with minimum `1`. It has config-file > default precedence only, defaults to `10`, merges global before local, appears in `sce config show` with provenance, and is carried by `ResolvedObservabilityRuntimeConfig`; the concrete logger stores the resolved value and uses it for primary and v2 creation-triggered cleanup. - `bash tool policy config surface`: Nested repo config namespace under `.sce/config.json` at `policies.bash`, currently supporting unique built-in `presets` plus repo-owned `custom` argv-prefix rules with deterministic validation, merged global/local resolution, and first-class `sce config show|validate` reporting. - `attribution hooks gate`: Enabled-by-default local hook runtime gate resolved through shared config precedence in `cli/src/services/config/mod.rs` (with parsing in `schema.rs`): opt-out env `SCE_ATTRIBUTION_HOOKS_DISABLED` overrides repo/global config key `policies.attribution_hooks.enabled` with inverted semantics, and the current enabled path activates commit-msg-only attribution gated by the staged-diff AI-overlap preflight. diff --git a/context/plans/config-schema-series-compatibility.md b/context/plans/config-schema-series-compatibility.md index 62e5f372c..b4b482959 100644 --- a/context/plans/config-schema-series-compatibility.md +++ b/context/plans/config-schema-series-compatibility.md @@ -176,7 +176,7 @@ Persist this field in every plan; this is durable plan state, not chat state: ## Task stack -- [ ] T01: `Derive a series-scoped $schema rule from .version in the Pkl schema` (status:todo) +- [x] T01: `Derive a series-scoped $schema rule from .version in the Pkl schema` (status:done) - Task ID: T01 - Scope: - In: @@ -193,7 +193,26 @@ Persist this field in every plan; this is durable plan state, not chat state: - Verify: - `nix run .#pkl-check-generated` - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema` - - Context synchronization: pending + - Completed: 2026-11-10 + - Files changed: + - `config/pkl/base/sce-config-schema-url.pkl` (new) + - `config/pkl/base/sce-config-schema.pkl` + - `config/pkl/renderers/generation-contract-check.pkl` + - `cli/src/services/config/schema.rs` + - Result: + - The new helper exposes `concreteUrl`, `series`, and `seriesPattern`, and throws on a non-semver version. + - The generated `$schema` property is now `type: string` with an anchored `^https://sce\.crocoder\.dev/v\.\.(0|[1-9][0-9]*)(-prerelease)?(+build)?/config\.json$` pattern. Its description names the series. `$id` is still the concrete URL. + - New contract check `config-schema-series-pattern` covers `0.4.0-pre-alpha-v14`, `0.5.0`, and `1.4.2`. It also asserts that the generated schema has no exact-version `const`. + - `accepts_versioned_schema_declaration` is replaced by module `schema_declaration_acceptance_tests`, with 4 tests built from the series in `PACKAGE_VERSION`: same-series acceptance; rejection of other series; rejection of malformed, foreign, and non-string values; and the case where `$schema` is valid but another field is invalid. + - Per a user instruction during the run, the new Pkl helper has no comments. + - Verify: + - `nix run .#pkl-check-generated`: passed (145 files; contract checks include `config-schema-series-pattern`). + - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema`: passed (29 passed, 0 failed). + - Supplementary checks: + - `cargo fmt --check`: passed. + - `rg -n '0\.4'` over the schema Pkl files and `schema.rs`: no matches. + - Context impact: localized. The generated config schema now accepts the whole series. `context/sce/config-schema-publication.md` still describes the exact-version `const` until T08 updates it. + - Context synchronization: synced - [ ] T02: `Add typed schema-declaration classification and actionable $schema errors` (status:todo) - Task ID: T02 diff --git a/context/sce/config-schema-publication.md b/context/sce/config-schema-publication.md index 8fcb36fc5..09d3a4c07 100644 --- a/context/sce/config-schema-publication.md +++ b/context/sce/config-schema-publication.md @@ -11,9 +11,16 @@ schema, not the separate Agent Trace schema. ## Current behavior -- The schema's `$id` and the config `$schema` property use - `https://sce.crocoder.dev/v/config.json`, where `` is the - CLI release version. +- The schema's `$id` is `https://sce.crocoder.dev/v/config.json`, + where `` is the CLI release version. +- The config `$schema` property accepts any canonical + `https://sce.crocoder.dev/v..[-pre][+build]/config.json` + declaration in the CLI's `major.minor` release series through an anchored + `pattern`. `config/pkl/base/sce-config-schema-url.pkl` derives the concrete + URL and the series pattern from `.version`. Other series, malformed or foreign + URLs, and non-string values are rejected. See + [the series-acceptance decision](../decisions/2026-11-10-config-schema-accepts-release-series-declarations.md). +- Snapshots published before this rule keep their exact-version `const`. - `sce setup` writes that same versioned declaration into a newly created repo-local `.sce/config.json`. - The generated working-tree schema remains ephemeral. Its version is read From 59181606461c3c8c12a7d58de8ca30ef0c058e9a Mon Sep 17 00:00:00 2001 From: David Abram Date: Tue, 10 Nov 2026 13:16:01 +0100 Subject: [PATCH 3/9] config: Add actionable schema-series validation guidance Classify `$schema` declarations by release series and replace generic schema errors with actionable guidance for older, newer, and unrecognized declarations. Improve startup diagnostics for older discovered-config series and cover the behavior with parity and guidance tests. Document the new config ownership, validation contract, and completed implementation task. Plan: config-schema-series-compatibility (T02) Co-authored-by: SCE --- cli/src/services/app_support.rs | 120 ++++- cli/src/services/config/mod.rs | 1 + cli/src/services/config/schema.rs | 54 ++- cli/src/services/config/schema_declaration.rs | 410 ++++++++++++++++++ context/architecture.md | 2 +- context/cli/config-precedence-contract.md | 5 + context/overview.md | 2 +- .../config-schema-series-compatibility.md | 33 +- 8 files changed, 589 insertions(+), 38 deletions(-) create mode 100644 cli/src/services/config/schema_declaration.rs diff --git a/cli/src/services/app_support.rs b/cli/src/services/app_support.rs index 364a36ab6..f59503374 100644 --- a/cli/src/services/app_support.rs +++ b/cli/src/services/app_support.rs @@ -83,26 +83,35 @@ pub(crate) fn classify_observability_configuration_error(error: &anyhow::Error) pub(crate) fn invalid_discovered_config_guidance( observability_config: &services::config::ResolvedObservabilityRuntimeConfig, ) -> Option { - if observability_config.validation_errors.is_empty() { + discovered_config_guidance( + &observability_config.loaded_config_paths, + &observability_config.validation_errors, + ) +} + +fn discovered_config_guidance( + loaded_config_paths: &[services::config::LoadedConfigPath], + validation_errors: &[String], +) -> Option { + if validation_errors.is_empty() { return None; } - let has_invalid_local_config = - observability_config - .loaded_config_paths - .iter() - .any(|loaded_path| { - loaded_path.source == services::config::ConfigPathSource::DefaultDiscoveredLocal - && observability_config - .validation_errors - .iter() - .any(|error| error.contains(loaded_path.path.to_string_lossy().as_ref())) - }); - - Some(if has_invalid_local_config { - "Local `.sce` config is invalid. Fix `.sce` and run `sce config validate`.".to_string() - } else { - "A discovered config file is invalid. Fix it and run `sce config validate`.".to_string() + let has_invalid_local_config = loaded_config_paths.iter().any(|loaded_path| { + loaded_path.source == services::config::ConfigPathSource::DefaultDiscoveredLocal + && validation_errors + .iter() + .any(|error| error.contains(loaded_path.path.to_string_lossy().as_ref())) + }); + let only_older_schema_series = validation_errors + .iter() + .all(|error| services::config::schema_declaration::is_older_series_only_failure(error)); + + Some(match (has_invalid_local_config, only_older_schema_series) { + (true, true) => "Local `.sce` config declares an older config schema series. Run `sce setup` to upgrade it.".to_string(), + (false, true) => "A discovered config file declares an older config schema series. Run `sce setup` to upgrade it.".to_string(), + (true, false) => "Local `.sce` config is invalid. Fix `.sce` and run `sce config validate`.".to_string(), + (false, false) => "A discovered config file is invalid. Fix it and run `sce config validate`.".to_string(), }) } @@ -226,3 +235,80 @@ fn write_error_diagnostic_with_color_policy( fn write_startup_diagnostic(writer: &mut W, diagnostic: &str) { let _ = writeln!(writer, "{}", services::style::error_code(diagnostic)); } + +#[cfg(test)] +mod discovered_config_guidance_tests { + use std::path::{Path, PathBuf}; + + use serde_json::json; + + use super::discovered_config_guidance; + use crate::services::config::schema::parse_file_config; + use crate::services::config::{ConfigPathSource, LoadedConfigPath}; + + const LOCAL_CONFIG_PATH: &str = "/repo/.sce/config.json"; + + fn local_loaded_path() -> LoadedConfigPath { + LoadedConfigPath { + path: PathBuf::from(LOCAL_CONFIG_PATH), + source: ConfigPathSource::DefaultDiscoveredLocal, + } + } + + fn validation_error(value: &serde_json::Value) -> String { + parse_file_config( + &value.to_string(), + Path::new(LOCAL_CONFIG_PATH), + ConfigPathSource::DefaultDiscoveredLocal, + ) + .unwrap_err() + .to_string() + } + + #[test] + fn older_schema_series_only_guidance_mentions_setup() { + let error = validation_error(&json!({ + "$schema": "https://sce.crocoder.dev/v0.0.1/config.json", + })); + + let guidance = discovered_config_guidance(&[local_loaded_path()], &[error]).unwrap(); + + assert!(guidance.contains("sce setup"), "{guidance}"); + assert!(guidance.starts_with("Local `.sce` config"), "{guidance}"); + } + + #[test] + fn older_schema_series_with_other_errors_keeps_generic_guidance() { + let error = validation_error(&json!({ + "$schema": "https://sce.crocoder.dev/v0.0.1/config.json", + "log_level": "loud", + })); + + let guidance = discovered_config_guidance(&[local_loaded_path()], &[error]).unwrap(); + + assert_eq!( + guidance, + "Local `.sce` config is invalid. Fix `.sce` and run `sce config validate`." + ); + } + + #[test] + fn foreign_schema_declaration_keeps_generic_guidance() { + let error = validation_error(&json!({ "$schema": "https://example.com/s.json" })); + + let guidance = discovered_config_guidance(&[], &[error]).unwrap(); + + assert_eq!( + guidance, + "A discovered config file is invalid. Fix it and run `sce config validate`." + ); + } + + #[test] + fn no_validation_errors_yields_no_guidance() { + assert_eq!( + discovered_config_guidance(&[local_loaded_path()], &[]), + None + ); + } +} diff --git a/cli/src/services/config/mod.rs b/cli/src/services/config/mod.rs index 53587a4f0..fc10da82b 100644 --- a/cli/src/services/config/mod.rs +++ b/cli/src/services/config/mod.rs @@ -3,6 +3,7 @@ pub mod lifecycle; pub mod policy; pub mod resolver; pub mod schema; +pub(crate) mod schema_declaration; pub mod types; mod render; diff --git a/cli/src/services/config/schema.rs b/cli/src/services/config/schema.rs index 2743b2532..1c4776a2e 100644 --- a/cli/src/services/config/schema.rs +++ b/cli/src/services/config/schema.rs @@ -7,6 +7,10 @@ use serde::Deserialize; use serde_json::Value; use super::policy::{parse_bash_policy_presets, parse_custom_bash_policies, CustomBashPolicyEntry}; +use super::schema_declaration::{ + classify_schema_declaration, current_schema_series, current_schema_version, + schema_declaration_issue, SCHEMA_DECLARATION_POINTER, +}; use super::types::{ parse_optional_workflow_id, AgentTraceDbRetryConfig, AuthMode, ConfigPathSource, DatabaseRetryConfig, IntegrationTargetId, IntegrationsConfig, LogFormat, LogLevel, @@ -234,17 +238,33 @@ pub(crate) type ParsedFilePolicies = ( ); pub(crate) fn validate_config_value_against_schema(value: &Value, path: &Path) -> Result<()> { - let mut errors = config_schema_validator() - .iter_errors(value) - .map(|error| { - let location = error.instance_path().to_string(); - if location.is_empty() { - error.to_string() - } else { - format!("{location}: {error}") + let current_version = current_schema_version(); + let declaration_issue = schema_declaration_issue( + &classify_schema_declaration( + value.get(CONFIG_SCHEMA_DECLARATION_KEY), + current_schema_series(), + ), + ¤t_version, + ); + let mut declaration_reported = false; + let mut errors = Vec::new(); + for error in config_schema_validator().iter_errors(value) { + let location = error.instance_path().to_string(); + if location.is_empty() { + errors.push(error.to_string()); + continue; + } + if location == SCHEMA_DECLARATION_POINTER { + if let Some(issue) = &declaration_issue { + if !declaration_reported { + errors.push(issue.clone()); + declaration_reported = true; + } + continue; } - }) - .collect::>(); + } + errors.push(format!("{location}: {error}")); + } if errors.is_empty() { return Ok(()); @@ -1133,7 +1153,7 @@ mod schema_declaration_acceptance_tests { ) } - fn declaration_accepted(declaration: Value) -> bool { + fn declaration_accepted(declaration: &Value) -> bool { parse_value(&json!({ "$schema": declaration })).is_ok() } @@ -1152,7 +1172,7 @@ mod schema_declaration_acceptance_tests { for version in accepted { let url = schema_url(&version); - assert!(declaration_accepted(json!(url)), "expected {url} accepted"); + assert!(declaration_accepted(&json!(url)), "expected {url} accepted"); } } @@ -1173,7 +1193,10 @@ mod schema_declaration_acceptance_tests { for version in rejected { let url = schema_url(&version); - assert!(!declaration_accepted(json!(url)), "expected {url} rejected"); + assert!( + !declaration_accepted(&json!(url)), + "expected {url} rejected" + ); } } @@ -1204,10 +1227,7 @@ mod schema_declaration_acceptance_tests { ]; for value in rejected { - assert!( - !declaration_accepted(value.clone()), - "expected {value} rejected" - ); + assert!(!declaration_accepted(&value), "expected {value} rejected"); } } diff --git a/cli/src/services/config/schema_declaration.rs b/cli/src/services/config/schema_declaration.rs new file mode 100644 index 000000000..8c083b371 --- /dev/null +++ b/cli/src/services/config/schema_declaration.rs @@ -0,0 +1,410 @@ +use std::fmt; + +use serde_json::Value; + +use crate::services::agent_trace::sce_config_schema_url; +use crate::services::version::PACKAGE_VERSION; + +const SCHEMA_URL_PREFIX: &str = "https://sce.crocoder.dev/v"; +const SCHEMA_URL_SUFFIX: &str = "/config.json"; + +pub(crate) const SCHEMA_DECLARATION_POINTER: &str = "/$schema"; + +const OLDER_SERIES_ISSUE_PREFIX: &str = "/$schema: declares SCE config schema version "; +const OLDER_SERIES_REMEDIATION: &str = + "run 'sce setup' to upgrade the repository and global config"; + +const ISSUE_SEPARATOR: &str = " | "; + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub(crate) struct SchemaSeries { + pub(crate) major: u64, + pub(crate) minor: u64, +} + +impl fmt::Display for SchemaSeries { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + write!(formatter, "{}.{}", self.major, self.minor) + } +} + +#[derive(Clone, Debug, Eq, PartialEq)] +pub(crate) struct SchemaVersion { + pub(crate) major: u64, + pub(crate) minor: u64, + pub(crate) patch: u64, + pub(crate) prerelease: Option, + pub(crate) build: Option, +} + +impl SchemaVersion { + /// Accepts exactly the semantic-version grammar used by the Pkl + /// `seriesPattern`, so classification stays in parity with the embedded schema. + pub(crate) fn parse(raw: &str) -> Option { + let (rest, build) = match raw.split_once('+') { + Some((rest, build)) => (rest, Some(parse_identifiers(build)?)), + None => (raw, None), + }; + let (core, prerelease) = match rest.split_once('-') { + Some((core, prerelease)) => (core, Some(parse_identifiers(prerelease)?)), + None => (rest, None), + }; + + let mut numbers = core.split('.'); + let major = parse_numeric(numbers.next()?)?; + let minor = parse_numeric(numbers.next()?)?; + let patch = parse_numeric(numbers.next()?)?; + if numbers.next().is_some() { + return None; + } + + Some(Self { + major, + minor, + patch, + prerelease, + build, + }) + } + + pub(crate) fn series(&self) -> SchemaSeries { + SchemaSeries { + major: self.major, + minor: self.minor, + } + } + + pub(crate) fn release(&self) -> String { + format!("{}.{}.{}", self.major, self.minor, self.patch) + } +} + +impl fmt::Display for SchemaVersion { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + write!(formatter, "{}.{}.{}", self.major, self.minor, self.patch)?; + if let Some(prerelease) = &self.prerelease { + write!(formatter, "-{prerelease}")?; + } + if let Some(build) = &self.build { + write!(formatter, "+{build}")?; + } + Ok(()) + } +} + +fn parse_numeric(raw: &str) -> Option { + let valid = raw == "0" + || (raw.starts_with(|first: char| ('1'..='9').contains(&first)) + && raw.bytes().all(|byte| byte.is_ascii_digit())); + if valid { + raw.parse().ok() + } else { + None + } +} + +fn parse_identifiers(raw: &str) -> Option { + let valid = raw.split('.').all(|identifier| { + !identifier.is_empty() + && identifier + .bytes() + .all(|byte| byte.is_ascii_alphanumeric() || byte == b'-') + }); + valid.then(|| raw.to_string()) +} + +#[derive(Clone, Debug, Eq, PartialEq)] +pub(crate) enum SchemaDeclaration { + Absent, + CurrentSeries, + OlderSeries(SchemaVersion), + NewerSeries(SchemaVersion), + Unrecognized, +} + +pub(crate) fn current_schema_version() -> SchemaVersion { + SchemaVersion::parse(PACKAGE_VERSION).expect("SCE package version should be a semantic version") +} + +pub(crate) fn current_schema_series() -> SchemaSeries { + current_schema_version().series() +} + +pub(crate) fn classify_schema_declaration( + value: Option<&Value>, + current: SchemaSeries, +) -> SchemaDeclaration { + let Some(value) = value else { + return SchemaDeclaration::Absent; + }; + let Some(version) = value + .as_str() + .and_then(|url| url.strip_prefix(SCHEMA_URL_PREFIX)) + .and_then(|rest| rest.strip_suffix(SCHEMA_URL_SUFFIX)) + .and_then(SchemaVersion::parse) + else { + return SchemaDeclaration::Unrecognized; + }; + + let declared = version.series(); + match (declared.major, declared.minor).cmp(&(current.major, current.minor)) { + std::cmp::Ordering::Equal => SchemaDeclaration::CurrentSeries, + std::cmp::Ordering::Less => SchemaDeclaration::OlderSeries(version), + std::cmp::Ordering::Greater => SchemaDeclaration::NewerSeries(version), + } +} + +/// Returns the single validation issue reported for a rejected `$schema` +/// declaration, or `None` when the declaration itself is acceptable. +pub(crate) fn schema_declaration_issue( + declaration: &SchemaDeclaration, + current: &SchemaVersion, +) -> Option { + let current_url = sce_config_schema_url(); + let current_release = current.release(); + let current_series = current.series(); + match declaration { + SchemaDeclaration::Absent | SchemaDeclaration::CurrentSeries => None, + SchemaDeclaration::OlderSeries(version) => Some(format!( + "{OLDER_SERIES_ISSUE_PREFIX}{}; this CLI uses {current_release} and accepts {current_series}.x; {OLDER_SERIES_REMEDIATION}, or set `$schema` to {current_url}", + version.release() + )), + SchemaDeclaration::NewerSeries(version) => Some(format!( + "{SCHEMA_DECLARATION_POINTER}: declares SCE config schema version {}, which was written by a newer SCE release; this CLI uses {current_release} and accepts {current_series}.x; upgrade the SCE CLI", + version.release() + )), + SchemaDeclaration::Unrecognized => Some(format!( + "{SCHEMA_DECLARATION_POINTER}: is not a canonical SCE schema URL; use {current_url}" + )), + } +} + +/// Recognizes a config validation error whose only issue is an older-series +/// `$schema` declaration. Issues are sorted and joined with `" | "`, so any +/// additional issue leaves a separator after the older-series issue. +pub(crate) fn is_older_series_only_failure(error: &str) -> bool { + let Some(start) = error.rfind(OLDER_SERIES_ISSUE_PREFIX) else { + return false; + }; + let issue = &error[start..]; + issue.contains(OLDER_SERIES_REMEDIATION) + && !issue.contains(ISSUE_SEPARATOR) + && !error[..start].contains(ISSUE_SEPARATOR) +} + +#[cfg(test)] +mod schema_declaration_tests { + use std::path::Path; + + use serde_json::{json, Value}; + + use super::{ + classify_schema_declaration, current_schema_series, current_schema_version, + is_older_series_only_failure, SchemaDeclaration, SchemaSeries, SchemaVersion, + }; + use crate::services::config::schema::parse_file_config; + use crate::services::config::ConfigPathSource; + use crate::services::version::PACKAGE_VERSION; + + const INJECTED_SERIES: SchemaSeries = SchemaSeries { major: 0, minor: 5 }; + + fn schema_url(version: &str) -> String { + format!("https://sce.crocoder.dev/v{version}/config.json") + } + + fn older_than_current() -> String { + let SchemaSeries { major, minor } = current_schema_series(); + if minor > 0 { + format!("{major}.{}.3", minor - 1) + } else { + format!("{}.9.0", major - 1) + } + } + + fn parse_error(value: &Value) -> Option { + parse_file_config( + &value.to_string(), + Path::new("/tmp/sce-config.json"), + ConfigPathSource::Flag, + ) + .err() + .map(|error| error.to_string()) + } + + fn declaration_matrix() -> Vec { + let SchemaSeries { major, minor } = current_schema_series(); + let mut versions = vec![ + PACKAGE_VERSION.to_string(), + format!("{major}.{minor}.0"), + format!("{major}.{minor}.7"), + format!("{major}.{minor}.0-pre-alpha-v1"), + format!("{major}.{minor}.0-pre-alpha-v14"), + format!("{major}.{minor}.7+build.5"), + format!("{major}.{minor}.12-rc.1+build.5"), + format!("{major}.{minor}.0-rc..1"), + format!("{major}.{minor}.0-"), + format!("{major}.{minor}.0+"), + format!("{major}.{minor}.0-rc_1"), + format!("{major}.{minor}"), + format!("{major}.{minor}.01"), + format!("{major}.{minor}.x"), + format!("{major}.{minor}.0.0"), + format!("{major}.{}.0", minor + 1), + format!("{}.{minor}.0", major + 1), + format!("{}.0.0", major + 1), + "0.3.3".to_string(), + "0.1.0".to_string(), + "1.4.0".to_string(), + format!("0{major}.{minor}.0"), + ]; + if minor > 0 { + versions.push(format!("{major}.{}.3", minor - 1)); + } + + let mut values: Vec = versions + .iter() + .map(|version| json!(schema_url(version))) + .collect(); + values.extend([ + json!("https://example.com/s.json"), + json!("https://sce.crocoder.dev/config.json"), + json!(format!( + "https://sce.crocoder.dev/v{major}.{minor}/config.json" + )), + json!(format!( + "http://sce.crocoder.dev/v{major}.{minor}.0/config.json" + )), + json!(format!( + "https://sce.crocoder.dev/v{major}.{minor}.0/config.json.bak" + )), + json!(format!( + " https://sce.crocoder.dev/v{major}.{minor}.0/config.json" + )), + json!(1), + json!(null), + json!({ "url": schema_url(PACKAGE_VERSION) }), + ]); + values + } + + #[test] + fn schema_declaration_classifier_matches_embedded_schema() { + let current = current_schema_series(); + for value in declaration_matrix() { + let accepted = parse_error(&json!({ "$schema": value.clone() })).is_none(); + let classified = classify_schema_declaration(Some(&value), current); + assert_eq!( + accepted, + classified == SchemaDeclaration::CurrentSeries, + "parity mismatch for {value}: classified {classified:?}" + ); + } + } + + #[test] + fn schema_declaration_absent_is_accepted() { + assert_eq!( + classify_schema_declaration(None, current_schema_series()), + SchemaDeclaration::Absent + ); + assert!(parse_error(&json!({})).is_none()); + } + + #[test] + fn schema_declaration_classifies_against_injected_series() { + let older = ["0.4.0-pre-alpha-v1", "0.4.7", "0.3.3"]; + for version in older { + let classified = + classify_schema_declaration(Some(&json!(schema_url(version))), INJECTED_SERIES); + assert_eq!( + classified, + SchemaDeclaration::OlderSeries(SchemaVersion::parse(version).unwrap()), + "{version}" + ); + } + + assert_eq!( + classify_schema_declaration(Some(&json!(schema_url("1.4.0"))), INJECTED_SERIES), + SchemaDeclaration::NewerSeries(SchemaVersion::parse("1.4.0").unwrap()) + ); + assert_eq!( + classify_schema_declaration(Some(&json!(schema_url("0.5.3+b.1"))), INJECTED_SERIES), + SchemaDeclaration::CurrentSeries + ); + assert_eq!( + classify_schema_declaration(Some(&json!(schema_url("0.5"))), INJECTED_SERIES), + SchemaDeclaration::Unrecognized + ); + } + + #[test] + fn schema_declaration_version_round_trips_prerelease_and_build() { + let version = SchemaVersion::parse("0.4.0-pre-alpha-v14+build.5").unwrap(); + assert_eq!(version.prerelease.as_deref(), Some("pre-alpha-v14")); + assert_eq!(version.build.as_deref(), Some("build.5")); + assert_eq!(version.to_string(), "0.4.0-pre-alpha-v14+build.5"); + } + + #[test] + fn schema_declaration_older_series_reports_setup_remediation() { + let current = current_schema_version(); + let older = older_than_current(); + let error = parse_error(&json!({ "$schema": schema_url(&older) })).unwrap(); + + let declared = SchemaVersion::parse(&older).unwrap().release(); + assert!( + error.contains(&format!( + "/$schema: declares SCE config schema version {declared}; this CLI uses {} and accepts {}.x; run 'sce setup'", + current.release(), + current.series() + )), + "{error}" + ); + assert!(error.contains(&schema_url(PACKAGE_VERSION)), "{error}"); + assert!(is_older_series_only_failure(&error), "{error}"); + } + + #[test] + fn schema_declaration_newer_series_reports_cli_upgrade() { + let current = current_schema_series(); + let newer = format!("{}.{}.0", current.major, current.minor + 1); + let error = parse_error(&json!({ "$schema": schema_url(&newer) })).unwrap(); + + assert!( + error.contains(&format!("declares SCE config schema version {newer}, which was written by a newer SCE release; this CLI uses")), + "{error}" + ); + assert!(error.contains("upgrade the SCE CLI"), "{error}"); + assert!(!is_older_series_only_failure(&error), "{error}"); + } + + #[test] + fn schema_declaration_unrecognized_reports_current_url() { + for value in [json!("https://example.com/s.json"), json!(1)] { + let error = parse_error(&json!({ "$schema": value })).unwrap(); + assert!( + error.contains(&format!( + "/$schema: is not a canonical SCE schema URL; use {}", + schema_url(PACKAGE_VERSION) + )), + "{error}" + ); + assert_eq!(error.matches("/$schema").count(), 1, "{error}"); + assert!(!is_older_series_only_failure(&error), "{error}"); + } + } + + #[test] + fn schema_declaration_issue_is_reported_alongside_other_field_errors() { + let error = parse_error(&json!({ + "$schema": schema_url(&older_than_current()), + "log_level": "loud", + })) + .unwrap(); + + assert!(error.contains("/$schema: "), "{error}"); + assert!(error.contains("/log_level: "), "{error}"); + assert!(error.contains(" | "), "{error}"); + assert!(!is_older_series_only_failure(&error), "{error}"); + } +} diff --git a/context/architecture.md b/context/architecture.md index 317c717ff..025cddc67 100644 --- a/context/architecture.md +++ b/context/architecture.md @@ -58,7 +58,7 @@ The repository includes a new placeholder Rust binary crate at `cli/`. - `cli/src/cli_schema.rs` is now the canonical owner for top-level command metadata for the real clap-backed command set (`auth`, `config`, `setup`, `doctor`, `hooks`, `policy`, `sync`, `version`, `completion`), including the slim top-level help purpose text and per-command visibility on `sce`, `sce help`, and `sce --help`; `cli/src/command_surface.rs` remains the custom top-level help renderer and known-command classifier, adding the synthetic `help` row plus the ASCII banner while consuming that shared metadata instead of maintaining a parallel command catalog. - `cli/src/services/default_paths.rs` is the canonical production path catalog for the CLI: it resolves config/state/cache roots with platform-aware XDG or `dirs` fallbacks through an internal `roots` seam, exposes named default paths for current persisted artifacts and database/log files (global config, auth tokens, auth DB, local DB, default observability log directory, and the sole Agent Trace DB path helper `agent_trace_db_path_for_repository` under `repos//agent-trace.db`; the former global-sentinel and per-checkout Agent Trace path helpers were removed by the `retire-legacy-agent-trace-db` plan), and owns canonical repo-relative, embedded-asset, install, hook, and context-path accessors so non-test production path definitions have one shared owner. Compile-time generated payload paths are owned by `build.rs` under `OUT_DIR`, not by the default-path catalog. Current production consumers such as config discovery, observability config resolution, doctor reporting, setup/install flows, database adapters, checkout identity, Agent Trace storage resolution, and local hook runtime path resolution consume this shared catalog rather than defining owned path literals in their own modules. - `cli/src/services/agent_trace.rs` is the Rust CLI owner for the SCE web base URL (`SCE_WEB_BASE_URL`) and exposes helpers for SCE-owned URL construction: Agent Trace conversation lookup URLs, persisted Agent Trace trace URLs, Agent Trace session URLs, and setup-created config schema URLs. Production Rust code should consume those helpers instead of repeating `sce.crocoder.dev` literals. The config resolver separately owns the `control_plane_base_url` runtime seam, whose baked `sce sync` default is `https://sce.crocoderlab.dev`; this control-plane host is not a web URL or schema owner. -- `cli/src/services/config/mod.rs` is the config service facade and `sce config` orchestration surface (`show`, `validate`, `--help`), with bare `sce config` routed by `cli/src/app.rs` to the same help payload as `sce config --help`. Focused submodules own the implementation slices: `types.rs` owns shared config/runtime primitives, `schema.rs` owns generated schema embedding plus typed file parsing, `policy.rs` owns bash-policy semantic validation plus policy-specific formatting and runtime preset-catalog access for the Rust evaluator, `resolver.rs` owns deterministic config-file discovery, file-layer merging, explicit value precedence (`flags > env > config file > defaults` where flag-backed), shared auth-key resolution, observability-runtime resolution, attribution-hooks runtime gate and `agent_trace.auto_sync` resolution, database-retry config resolution and `DATABASE_RETRY_CONFIG` `OnceLock` initialization, default-discovered invalid-file degradation, strict invalid-discovered-layer errors for Agent Trace storage, and explicit-path fatal errors for `--config` / `SCE_CONFIG_FILE`, and private `render.rs` owns `sce config show` / `sce config validate` text and JSON output construction plus rendering-specific display-value helpers. The facade preserves existing `services::config` imports for startup/auth/hooks callers while delegating command execution to resolution plus rendering submodules. +- `cli/src/services/config/mod.rs` is the config service facade and `sce config` orchestration surface (`show`, `validate`, `--help`), with bare `sce config` routed by `cli/src/app.rs` to the same help payload as `sce config --help`. Focused submodules own the implementation slices: `types.rs` owns shared config/runtime primitives, `schema.rs` owns generated schema embedding plus typed file parsing, `schema_declaration.rs` owns typed `$schema` release-series classification and the actionable `/$schema` validation messages, `policy.rs` owns bash-policy semantic validation plus policy-specific formatting and runtime preset-catalog access for the Rust evaluator, `resolver.rs` owns deterministic config-file discovery, file-layer merging, explicit value precedence (`flags > env > config file > defaults` where flag-backed), shared auth-key resolution, observability-runtime resolution, attribution-hooks runtime gate and `agent_trace.auto_sync` resolution, database-retry config resolution and `DATABASE_RETRY_CONFIG` `OnceLock` initialization, default-discovered invalid-file degradation, strict invalid-discovered-layer errors for Agent Trace storage, and explicit-path fatal errors for `--config` / `SCE_CONFIG_FILE`, and private `render.rs` owns `sce config show` / `sce config validate` text and JSON output construction plus rendering-specific display-value helpers. The facade preserves existing `services::config` imports for startup/auth/hooks callers while delegating command execution to resolution plus rendering submodules. - `cli/src/services/output_format.rs` defines the canonical shared CLI output-format contract (`OutputFormat`) for supporting commands, with deterministic `text|json` parsing and command-scoped actionable invalid-value guidance. - `cli/src/services/config/types.rs` is the canonical owner for the shared runtime/config primitive seam used by the CLI: `LogLevel`, `LogFormat`, `SCE_LOG_LEVEL`, `SCE_LOG_FORMAT`, `SCE_LOG_DIR`, `DEFAULT_LOG_FILE_RETENTION_LIMIT`, and the shared bool parsing helpers used by both config resolution and observability bootstrap; `cli/src/services/config/mod.rs` re-exports those primitives through the facade. - `cli/src/services/capabilities.rs` defines the current broad CLI capability traits consumed by the borrowed, compile-time-typed `AppContext`: `FsOps` with `StdFsOps` as a method-less filesystem capability marker and `GitOps` with `ProcessGitOps` for git command execution and availability checks. Existing service internals do not consume these traits directly yet; command execution uses narrow accessors and repo-root-scoped context derivation. diff --git a/context/cli/config-precedence-contract.md b/context/cli/config-precedence-contract.md index dd5cc91e3..05df9046e 100644 --- a/context/cli/config-precedence-contract.md +++ b/context/cli/config-precedence-contract.md @@ -80,6 +80,11 @@ When a default-discovered global or repo-local config file exists but fails JSON - Each reported schema-validation error is prefixed with the failing value's JSON-pointer location when it has one (for example `/integrations/optional_workflows/0: "nonesuch" is not one of "brownfield"`), so a rejected value names the key it came from; root-level errors keep their unprefixed text. Errors remain sorted and joined with ` | `. - After schema validation, `cli/src/services/config/schema.rs` deserializes top-level and nested config structure (`policies`, `policies.bash`, `policies.attribution_hooks`) into typed serde DTOs and applies focused Rust-owned mapping helpers for enum conversion and source attribution; policy-specific semantic checks are owned by `cli/src/services/config/policy.rs`. - The canonical top-level schema declaration `"$schema": "https://sce.crocoder.dev/v/config.json"` is accepted for any `` in the CLI's `major.minor` release series ([series-acceptance decision](../decisions/2026-11-10-config-schema-accepts-release-series-declarations.md)). It is a supported config key for both explicit and discovered `sce/config.json` files, including command-startup paths like `sce version` and other config-loading commands that parse config before normal command dispatch. +- A rejected `$schema` value is reported as one classified `/$schema` issue, sorted and joined with any other field errors. `cli/src/services/config/schema_declaration.rs` classifies the declaration against the CLI series as absent, current series, older series, newer series, or unrecognized, with the same semantic-version grammar as the Pkl series pattern; parity tests keep it aligned with the embedded schema. Versions in these messages are printed as `major.minor.patch`: + - Older series: `declares SCE config schema version 0.3.3; this CLI uses 0.4.0 and accepts 0.4.x; run 'sce setup' to upgrade the repository and global config, or set `$schema` to `. + - Newer series: `declares SCE config schema version , which was written by a newer SCE release; …; upgrade the SCE CLI`. + - Malformed, foreign, or non-string: `is not a canonical SCE schema URL; use `. +- When every collected discovered-config validation error is only an older-series `$schema` issue, the startup diagnostic says the local or discovered config "declares an older config schema series" and tells the user to run `sce setup`; any other failure keeps the generic "is invalid … run `sce config validate`" guidance. Validation itself never rewrites the file. - Startup/runtime config resolution degrades gracefully for default-discovered files across ordinary startup, setup, and Agent Trace storage: invalid discovered files are skipped and reported via collected `validation_errors`, while explicit `--config` / `SCE_CONFIG_FILE` targets still fail immediately on the same parse or validation errors. - Config file content must be valid JSON with a top-level object. diff --git a/context/overview.md b/context/overview.md index 494621b01..da4dde842 100644 --- a/context/overview.md +++ b/context/overview.md @@ -77,7 +77,7 @@ The setup service also provides repository-root install orchestration: it resolv The CLI now also applies baseline security hardening for reliability-driven automation: diagnostics/logging paths use deterministic secret redaction,`sce setup --hooks --repo ` canonicalizes and validates repository paths before execution, and setup write flows run explicit directory write-permission probes before staging/swap operations. The config service now provides deterministic runtime config resolution with explicit precedence (`flags > env > config file > defaults`), strict config-file validation (`$schema`, `log_level`, `log_format`, `log_to_file`, `log_dir`, `workos_client_id`, and nested `policies.bash`, `policies.attribution_hooks.enabled`, plus `policies.database_retry` with per-DB `connection_open`/`query` retry policy specs and Agent Trace-only `agent_trace_db.busy_timeout_ms`/`contention_deadline_ms`), deterministic default discovery/merge of global+local config files (`${config*root}/sce/config.json`then`.sce/config.json`with local override, where`config_root` comes from the shared default-path seam with XDG/`dirs::config_dir()` config-root resolution), defaults for the resolved observability value set (`log_level=error`, `log_format=text`, `log_dir=/sce/logs`), shared auth-key resolution with optional baked defaults starting at `workos_client_id`, first-class bash-policy preset/custom parsing with deterministic conflict and duplicate-prefix validation, custom-policy `satisfied_by`wrapper exemption (a policy does not fire when the matched command was unwrapped from a declared wrapper such as`nix shell nixpkgs#ripgrep`), and a canonical Pkl-authored `sce/config.json`JSON Schema generated beneath Cargo`OUT_DIR`and embedded by`cli/src/services/config/mod.rs`for both`sce config validate`and doctor-time config checks. Runtime startup config loading keeps parity with that schema by accepting its`$schema`declaration in repo-local and global config files, so startup commands such as`sce version`no longer fail before dispatch on that field; the canonical declaration is versioned as`"https://sce.crocoder.dev/v/config.json"` using the CLI release version; this schema URL is separate from the `https://sce.crocoderlab.dev` baked default used by `sce sync` for control-plane ingestion. App-runtime observability now consumes flat logging keys through the shared resolver, so env values still override config-file values while config files provide deterministic fallback for `log_dir`; positive-integer `log_file_retention_limit` uses config-file/default precedence, defaults to `10`, and controls creation-triggered cleanup for primary and v2 log files; `sce config show` reports resolved observability/auth/policy values with provenance, while `sce config validate` is now a trimmed validation surface that reports only pass/fail plus validation errors or warnings in text and JSON modes. The canonical preset catalog and matching contract live in `config/pkl/base/bash-policy-presets.pkl` and `context/sce/bash-tool-policy-enforcement-contract.md`. Invalid default-discovered config files now also degrade gracefully at startup: `sce` keeps running with degraded observability defaults, logs `sce.config.invalid_config` warnings, and reserves hard failures for explicit `--config` / `SCE_CONFIG_FILE` targets or other truly invalid runtime observability inputs. -`cli/src/services/config/mod.rs` is now a module facade that declares focused config submodules (`types`, `schema`, `policy`, `resolver`, private `render`, `command`, and `lifecycle`), re-exporting `pub use types::*`and`pub(crate) use schema::validate_config_file`. Shared config primitive ownership is delegated to `cli/src/services/config/types.rs`; schema loading and file parsing to `cli/src/services/config/schema.rs`; bash-policy semantic validation and policy-specific formatting to `cli/src/services/config/policy.rs`; runtime discovery/precedence to `cli/src/services/config/resolver.rs`; and `sce config show`/`sce config validate`text+JSON output construction to`cli/src/services/config/render.rs`. Downstream modules continue importing through `services::config`unchanged. The CLI now has a generic borrowed`AppContext`dependency view in`cli/src/app.rs`; `AppRuntime`owns concrete production logger/telemetry/fs/git dependencies, and command execution receives context views that borrow those dependencies plus an optional`repo_root: Option`. `AppContext::with_repo_root(...)`/`ContextWithRepoRoot`derives repo-root-scoped views while preserving the borrowed runtime dependencies, and command execution is generic over associated-type narrow accessor traits where practical. The broad capability seam lives in`cli/src/services/capabilities.rs`, where `FsOps`/`StdFsOps` form a method-less filesystem marker and `GitOps`/`ProcessGitOps` wrap git command execution and availability. +`cli/src/services/config/mod.rs` is now a module facade that declares focused config submodules (`types`, `schema`, `schema_declaration`, `policy`, `resolver`, private `render`, `command`, and `lifecycle`), re-exporting `pub use types::*`and`pub(crate) use schema::validate_config_file`. Shared config primitive ownership is delegated to `cli/src/services/config/types.rs`; schema loading and file parsing to `cli/src/services/config/schema.rs`; `$schema` declaration classification against the CLI release series to `cli/src/services/config/schema_declaration.rs`; bash-policy semantic validation and policy-specific formatting to `cli/src/services/config/policy.rs`; runtime discovery/precedence to `cli/src/services/config/resolver.rs`; and `sce config show`/`sce config validate`text+JSON output construction to`cli/src/services/config/render.rs`. Downstream modules continue importing through `services::config`unchanged. The CLI now has a generic borrowed`AppContext`dependency view in`cli/src/app.rs`; `AppRuntime`owns concrete production logger/telemetry/fs/git dependencies, and command execution receives context views that borrow those dependencies plus an optional`repo_root: Option`. `AppContext::with_repo_root(...)`/`ContextWithRepoRoot`derives repo-root-scoped views while preserving the borrowed runtime dependencies, and command execution is generic over associated-type narrow accessor traits where practical. The broad capability seam lives in`cli/src/services/capabilities.rs`, where `FsOps`/`StdFsOps` form a method-less filesystem marker and `GitOps`/`ProcessGitOps` wrap git command execution and availability. The shared default path service in`cli/src/services/default_paths.rs`is now the canonical owner for production CLI path definitions. It resolves per-user config/state/cache roots through a dedicated internal`roots`seam, exposes the current persisted-artifact inventory (global config and auth tokens), and also defines named DB paths (auth DB, local DB, Agent Trace DB) plus the repo-relative, install, hook, and context-path accessors consumed across current CLI production code. Non-test production modules should consume this shared catalog instead of hardcoding owned path literals. No default cache-backed persisted artifact currently exists, so cache-root resolution remains available without speculative cache-path features and no legacy default-path fallback is supported. The Rust CLI also centralizes SCE-owned web URI construction in`cli/src/services/agent_trace.rs`, with `SCE_WEB_BASE_URL`as the single Rust owner for`https://sce.crocoder.dev` and helpers consumed by Agent Trace conversation URLs, Agent Trace persisted trace URLs, Agent Trace session URLs, and setup-created repo-local config schema URLs. The config resolver separately owns `control_plane_base_url` and its `https://sce.crocoderlab.dev` baked sync default; the two URL owners must not be conflated. The current user-facing synchronization entrypoint is `sce sync`; references to the former nested spelling in historical records do not describe an available command. diff --git a/context/plans/config-schema-series-compatibility.md b/context/plans/config-schema-series-compatibility.md index b4b482959..d92a2eb97 100644 --- a/context/plans/config-schema-series-compatibility.md +++ b/context/plans/config-schema-series-compatibility.md @@ -214,7 +214,7 @@ Persist this field in every plan; this is durable plan state, not chat state: - Context impact: localized. The generated config schema now accepts the whole series. `context/sce/config-schema-publication.md` still describes the exact-version `const` until T08 updates it. - Context synchronization: synced -- [ ] T02: `Add typed schema-declaration classification and actionable $schema errors` (status:todo) +- [x] T02: `Add typed schema-declaration classification and actionable $schema errors` (status:done) - Task ID: T02 - Scope: - In: @@ -239,7 +239,36 @@ Persist this field in every plan; this is durable plan state, not chat state: - Verify: - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema_declaration` - `… test … config` - - Context synchronization: pending + - Completed: 2026-11-10 + - Files changed: + - `cli/src/services/config/schema_declaration.rs` (new) + - `cli/src/services/config/mod.rs` + - `cli/src/services/config/schema.rs` + - `cli/src/services/app_support.rs` + - Result: + - New `schema_declaration` module defines: + - `SchemaVersion`, with a typed parser that mirrors the Pkl `seriesPattern` grammar, plus `series()` and `release()`. + - `SchemaSeries`. + - `current_schema_version()` and `current_schema_series()`, both parsed from `PACKAGE_VERSION`. + - `classify_schema_declaration`, returning `Absent | CurrentSeries | OlderSeries | NewerSeries | Unrecognized`. + - `schema_declaration_issue`. + - `is_older_series_only_failure`. + - `validate_config_value_against_schema` collapses every `/$schema` schema error into one classified issue. Other errors and the sorted ` | ` join are unchanged. + - `invalid_discovered_config_guidance` delegates to `discovered_config_guidance`. When every validation error is only an older-series declaration, the guidance names `sce setup`, for both the local and the discovered-file wording. + - Deviation (user instruction during the run): the messages print full `major.minor.patch` versions rather than `X.Y` series. Prerelease and build are dropped. Final wording: + - Older: `/$schema: declares SCE config schema version 0.3.3; this CLI uses 0.4.0 and accepts 0.4.x; run 'sce setup' to upgrade the repository and global config, or set `$schema` to `. + - Newer: `… version 0.5.0, which was written by a newer SCE release; this CLI uses 0.4.0 and accepts 0.4.x; upgrade the SCE CLI`. + - Unrecognized: `/$schema: is not a canonical SCE schema URL; use `. + - Fixed a T01 test helper (`declaration_accepted` now takes `&Value`) to clear the clippy `needless_pass_by_value` failure under `--all-targets -D warnings`. + - Verify: + - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema_declaration`: passed (13 passed, 0 failed). This includes the classifier/embedded-schema parity matrix, the injected-`0.5` classifier tests, and the guidance tests. + - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml config`: passed (109 passed, 0 failed). + - Supplementary checks: + - `clippy --all-targets -- -D warnings`: passed. + - `cargo fmt --check`: passed. + - Behavioral check: `sce config validate` against a discovered `.sce/config.json` declaring `0.3.3`, `0.5.0`, or `https://example.com/s.json` printed the matching message, and a sha256 check confirmed each file was unchanged. `0.3.3` plus `log_level: "loud"` reported both issues. + - Context impact: localized. `$schema` validation messages and the startup guidance changed. `context/cli/config-precedence-contract.md` does not yet describe the actionable messages; T08 updates it. + - Context synchronization: synced - [ ] T03: `Extract the guarded config-file write shared by setup writers` (status:todo) - Task ID: T03 From e612db002bb9a7bbe4fc76ac44fee16a50c9b219 Mon Sep 17 00:00:00 2001 From: David Abram Date: Tue, 10 Nov 2026 13:21:04 +0100 Subject: [PATCH 4/9] setup: Extract guarded config-file writes Share setup config-layer metadata and guarded reread/symlink checks through a dedicated helper while retaining the existing atomic replacement path and error behavior. Add focused helper tests for missing files, concurrent creation, and dangling symlinks, and record completion of T03. Plan: config-schema-series-compatibility (T03) Co-authored-by: SCE --- cli/src/services/setup/access_preferences.rs | 89 +++-------- cli/src/services/setup/config_file_write.rs | 140 ++++++++++++++++++ cli/src/services/setup/mod.rs | 1 + context/architecture.md | 2 +- .../config-schema-series-compatibility.md | 27 +++- 5 files changed, 191 insertions(+), 68 deletions(-) create mode 100644 cli/src/services/setup/config_file_write.rs diff --git a/cli/src/services/setup/access_preferences.rs b/cli/src/services/setup/access_preferences.rs index 4f35b1a5d..2dc0e4c9a 100644 --- a/cli/src/services/setup/access_preferences.rs +++ b/cli/src/services/setup/access_preferences.rs @@ -1,16 +1,15 @@ use std::{ - fmt, fs, io, + fmt, path::{Path, PathBuf}, }; use anyhow::Context; use serde_json::{Map, Value}; -use crate::services::agent_trace::sce_config_schema_url; -use crate::services::config::schema::{parse_file_config, CONFIG_SCHEMA_DECLARATION_KEY}; -use crate::services::config::{AuthMode, ConfigPathSource}; +use crate::services::config::schema::parse_file_config; +use crate::services::config::AuthMode; -use super::atomic_write::write_atomically; +use super::config_file_write::{read_optional, write_guarded, ConfigLayer, GuardedWriteError}; const AUTH_KEY: &str = "auth"; const AUTH_MODE_KEY: &str = "mode"; @@ -66,39 +65,6 @@ impl fmt::Display for RepositoryAccessPreferencesError { impl std::error::Error for RepositoryAccessPreferencesError {} -#[derive(Clone, Copy, Debug, Eq, PartialEq)] -enum ConfigLayer { - Global, - Repository, -} - -impl ConfigLayer { - fn path_source(self) -> ConfigPathSource { - match self { - Self::Global => ConfigPathSource::DefaultDiscoveredGlobal, - Self::Repository => ConfigPathSource::DefaultDiscoveredLocal, - } - } - - fn label(self) -> &'static str { - match self { - Self::Global => "global SCE config", - Self::Repository => "repository SCE config", - } - } - - fn bootstrap_value(self) -> Value { - match self { - Self::Global => Value::Object(Map::from_iter([( - CONFIG_SCHEMA_DECLARATION_KEY.to_string(), - Value::String(sce_config_schema_url()), - )])), - Self::Repository => serde_json::from_str(&super::repo_local_config_bootstrap_payload()) - .expect("repo-local config bootstrap payload must be valid JSON"), - } - } -} - #[derive(Debug)] enum WriteFailure { Invalid(String), @@ -107,6 +73,16 @@ enum WriteFailure { Io(anyhow::Error), } +impl From for WriteFailure { + fn from(error: GuardedWriteError) -> Self { + match error { + GuardedWriteError::ConcurrentModification => Self::ConcurrentModification, + GuardedWriteError::SymlinkManaged => Self::SymlinkManaged, + GuardedWriteError::Io(error) => Self::Io(error), + } + } +} + pub(crate) fn read_global_access_preferences(path: &Path) -> anyhow::Result { read_access_preferences(path, ConfigLayer::Global) } @@ -229,15 +205,13 @@ where parse_file_config(&serialized, path, layer.path_source()) .map_err(|error| WriteFailure::Invalid(format!("{error:#}")))?; - let current = reread(path).map_err(WriteFailure::Io)?; - if current != original { - return Err(WriteFailure::ConcurrentModification); - } - if is_symlink(path, layer).map_err(WriteFailure::Io)? { - return Err(WriteFailure::SymlinkManaged); - } - - write_atomically(path, serialized.as_bytes()).map_err(WriteFailure::Io)?; + write_guarded( + path, + layer, + original.as_deref(), + serialized.as_bytes(), + reread, + )?; Ok(AccessWriteOutcome::Written) } @@ -261,28 +235,13 @@ fn set_nested_value(object: &mut Map, section: &str, key: &str, v } } -fn is_symlink(path: &Path, layer: ConfigLayer) -> anyhow::Result { - match fs::symlink_metadata(path) { - Ok(metadata) => Ok(metadata.file_type().is_symlink()), - Err(error) if error.kind() == io::ErrorKind::NotFound => Ok(false), - Err(error) => Err(error) - .with_context(|| format!("Failed to inspect {} '{}'", layer.label(), path.display())), - } -} - -fn read_optional(path: &Path, layer: ConfigLayer) -> anyhow::Result>> { - match fs::read(path) { - Ok(bytes) => Ok(Some(bytes)), - Err(error) if error.kind() == io::ErrorKind::NotFound => Ok(None), - Err(error) => Err(error) - .with_context(|| format!("Failed to read {} '{}'", layer.label(), path.display())), - } -} - #[cfg(test)] mod tests { use super::*; + use crate::services::agent_trace::sce_config_schema_url; + use crate::services::config::schema::CONFIG_SCHEMA_DECLARATION_KEY; use serde_json::json; + use std::fs; fn repository_config_path(dir: &Path) -> PathBuf { dir.join(".sce").join("config.json") diff --git a/cli/src/services/setup/config_file_write.rs b/cli/src/services/setup/config_file_write.rs new file mode 100644 index 000000000..b9baecf51 --- /dev/null +++ b/cli/src/services/setup/config_file_write.rs @@ -0,0 +1,140 @@ +use std::{fs, io, path::Path}; + +use anyhow::Context; +use serde_json::{Map, Value}; + +use crate::services::agent_trace::sce_config_schema_url; +use crate::services::config::schema::CONFIG_SCHEMA_DECLARATION_KEY; +use crate::services::config::ConfigPathSource; + +use super::atomic_write::write_atomically; + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub(super) enum ConfigLayer { + Global, + Repository, +} + +impl ConfigLayer { + pub(super) fn path_source(self) -> ConfigPathSource { + match self { + Self::Global => ConfigPathSource::DefaultDiscoveredGlobal, + Self::Repository => ConfigPathSource::DefaultDiscoveredLocal, + } + } + + pub(super) fn label(self) -> &'static str { + match self { + Self::Global => "global SCE config", + Self::Repository => "repository SCE config", + } + } + + pub(super) fn bootstrap_value(self) -> Value { + match self { + Self::Global => Value::Object(Map::from_iter([( + CONFIG_SCHEMA_DECLARATION_KEY.to_string(), + Value::String(sce_config_schema_url()), + )])), + Self::Repository => serde_json::from_str(&super::repo_local_config_bootstrap_payload()) + .expect("repo-local config bootstrap payload must be valid JSON"), + } + } +} + +#[derive(Debug)] +pub(super) enum GuardedWriteError { + ConcurrentModification, + SymlinkManaged, + Io(anyhow::Error), +} + +/// Writes `serialized` only when the file still holds exactly `original` and is +/// not a symlink, so setup never clobbers concurrent edits or symlink-managed +/// config sources. +pub(super) fn write_guarded( + path: &Path, + layer: ConfigLayer, + original: Option<&[u8]>, + serialized: &[u8], + reread: R, +) -> Result<(), GuardedWriteError> +where + R: FnOnce(&Path) -> anyhow::Result>>, +{ + let current = reread(path).map_err(GuardedWriteError::Io)?; + if current.as_deref() != original { + return Err(GuardedWriteError::ConcurrentModification); + } + if is_symlink(path, layer).map_err(GuardedWriteError::Io)? { + return Err(GuardedWriteError::SymlinkManaged); + } + + write_atomically(path, serialized).map_err(GuardedWriteError::Io) +} + +pub(super) fn read_optional(path: &Path, layer: ConfigLayer) -> anyhow::Result>> { + match fs::read(path) { + Ok(bytes) => Ok(Some(bytes)), + Err(error) if error.kind() == io::ErrorKind::NotFound => Ok(None), + Err(error) => Err(error) + .with_context(|| format!("Failed to read {} '{}'", layer.label(), path.display())), + } +} + +fn is_symlink(path: &Path, layer: ConfigLayer) -> anyhow::Result { + match fs::symlink_metadata(path) { + Ok(metadata) => Ok(metadata.file_type().is_symlink()), + Err(error) if error.kind() == io::ErrorKind::NotFound => Ok(false), + Err(error) => Err(error) + .with_context(|| format!("Failed to inspect {} '{}'", layer.label(), path.display())), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn reread(path: &Path) -> anyhow::Result>> { + read_optional(path, ConfigLayer::Global) + } + + #[test] + fn write_guarded_creates_missing_file_when_still_missing() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("config.json"); + + write_guarded(&path, ConfigLayer::Global, None, b"{}\n", reread).unwrap(); + + assert_eq!(fs::read(&path).unwrap(), b"{}\n"); + } + + #[test] + fn write_guarded_refuses_file_created_after_planning() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("config.json"); + fs::write(&path, b"{\"concurrent\": true}\n").unwrap(); + + let error = write_guarded(&path, ConfigLayer::Global, None, b"{}\n", reread).unwrap_err(); + + assert!(matches!(error, GuardedWriteError::ConcurrentModification)); + assert_eq!(fs::read(&path).unwrap(), b"{\"concurrent\": true}\n"); + } + + #[cfg(unix)] + #[test] + fn write_guarded_refuses_dangling_symlink_and_leaves_it_unchanged() { + use std::os::unix::fs::symlink; + + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("config.json"); + let target = dir.path().join("missing-target.json"); + symlink(&target, &path).unwrap(); + + let error = write_guarded(&path, ConfigLayer::Global, None, b"{}\n", reread).unwrap_err(); + + assert!(matches!(error, GuardedWriteError::SymlinkManaged)); + assert_eq!(fs::read_link(&path).unwrap(), target); + assert!(!target.exists()); + } +} diff --git a/cli/src/services/setup/mod.rs b/cli/src/services/setup/mod.rs index 29b0de097..1b36f37e0 100644 --- a/cli/src/services/setup/mod.rs +++ b/cli/src/services/setup/mod.rs @@ -14,6 +14,7 @@ pub(crate) mod access_preferences; mod atomic_write; pub(crate) mod claude_telemetry; pub mod command; +mod config_file_write; pub(crate) mod config_merge; pub(crate) mod hook_merge; diff --git a/context/architecture.md b/context/architecture.md index 025cddc67..a0c6c7807 100644 --- a/context/architecture.md +++ b/context/architecture.md @@ -71,7 +71,7 @@ The repository includes a new placeholder Rust binary crate at `cli/`. - `cli/src/services/agent_trace_db/mod.rs` owns the shared Agent Trace insert payloads, SQL constants, and typed row helpers (diff-trace/intersection/Agent Trace/message/part) plus `ensure_schema_ready_for_hooks()` consumed by the repository adapter. `cli/src/services/agent_trace_db/repository.rs` defines the sole `RepositoryAgentTraceDb` adapter over `TursoDb` with one fresh `agent-trace-repository/001_repository_schema.sql` baseline for `diff_traces` (including `payload_type`), `post_commit_patch_intersections`, `agent_traces`, `messages`, `parts`, indexes, and triggers, `repository_metadata` validation, no trace-table `checkout_id` columns, `agent_traces.agent_trace_id NOT NULL UNIQUE`, and `recent_diff_trace_patches(cutoff_time_ms, end_time_ms)` using the inclusive chronological parser without checkout filtering; structured-row reconstruction applies the persisted row `model_id` to every hunk and the persisted canonical `session_id` to every touched line before downstream combination and intersection. Active hook runtime, setup/lifecycle storage, and `sce sync` resolve through `agent_trace_storage` and use `RepositoryAgentTraceDb`. The checkout-scoped `AgentTraceDb`/`AgentTraceDbSpec` adapter, its `agent_trace_db_path()`/`agent_trace_db_path_for_checkout()` helpers, the 15-file `cli/migrations/agent-trace/` chain, and the former `sce trace --legacy` surface were removed by the `retire-legacy-agent-trace-db` plan. - `cli/src/services/setup/mod.rs` defines the setup command contract (`SetupMode`, `SetupTarget`, `SetupRequest`, CLI flag parser/validator), an `inquire`-backed interactive target prompter (`InquireSetupTargetPrompter`), setup dispatch outcomes (proceed/cancelled), additive durable-context bootstrap (`bootstrap_context_baseline` for standalone `--bootstrap-context` and every normal successful setup path), and compile-time embedded asset access (`EmbeddedAsset`, target-scoped iterators, required-hook asset iterators/lookups). It also owns the install-time optional-workflow seam: the `OptionalWorkflow` type plus the build-generated `OPTIONAL_WORKFLOWS` catalog, a per-target `WorkflowAssetLayout` built from the existing `default_paths` command/skill directory constants (`command`/`commands`/`prompts` plus `skills`), and `iter_embedded_assets_for_setup_target_with_selection(target, selection)`, which yields every embedded asset except the `{command_dir}/{command_slug}.md` file and `{skills_dir}/{skill_slug}/` subtree of each optional workflow the selection omits. Membership is derived from the catalog's slugs rather than an enumerated file list, so a new optional workflow needs no Rust change. This filtered iterator is the only way embedded assets are enumerated; setup installs through it and doctor inspects through it, so there is no unfiltered enumeration path that could reintroduce an unselected workflow. The non-interactive selection flows through the repeatable `--workflow ` flag into `SetupRequest.optional_workflows: Option>` (`None` meaning the flag was absent); `validate_optional_workflow_slugs` checks each slug against `OPTIONAL_WORKFLOWS` during request resolution, before any write, and `run_setup_for_mode` resolves `None` to the persisted `integrations.optional_workflows` (exported as `persisted_optional_workflows`) before installing through the filtered iterator and persisting the resolved selection. The interactive selection flows through the same seam: `SetupTargetPrompter` carries `prompt_targets(defaults)`, `prompt_optional_workflows(defaults)`, the two behavior confirmations `prompt_agent_trace_auto_sync(default)` / `prompt_attribution_hooks_enabled(default)` (dispatch passes `true`), and the access prompts `prompt_auth_mode(default)` (an `inquire::Select` with fixed Optional/Mandatory/Disabled rows), `prompt_telemetry_consent(default)`, and `prompt_login()` (Yes default), which dispatch does not call yet; every method returns `None` for a cancelled prompt and fails with the fixed TTY guidance on `NotTTY`, `SetupDispatch::Proceed { mode, optional_workflows }` carries a prompted selection alongside the resolved mode, and `resolve_setup_dispatch(mode, prompter, defaults)` runs the workflow prompt only after an interactive target prompt, mapping either cancellation to `SetupDispatch::Cancelled`. The prompt module builds its `inquire::MultiSelect` from `optional_workflow_prompt_inputs(catalog, defaults)`, which returns `None` for an empty catalog (skipping the prompt) and otherwise catalog-ordered rows plus the indices to pre-check, ignoring ids absent from the catalog. `setup/command.rs` therefore resolves the repository root before dispatch, mapping the setup domain's typed positive non-Git classification to `setup.not_git_repository` and other resolution failures to `general.unexpected_failure`, so a non-Git directory fails before any prompt without CLI-layer string matching. For repository builds, `cli/build.rs` validates the `SCE_CLI_GENERATED_INPUT_DIR` payload and canonical-input inventories, copies the payload into Cargo `OUT_DIR/pkl-generated`, stages `cli/assets/hooks/**` under `OUT_DIR/static`, requires the staged `config/optional-workflows.json`, and generates both the setup manifest and the optional-workflow catalog (`optional_workflows.rs`, rejecting a manifest whose `schemaVersion` is not 1 or whose entries lack a non-empty `id`/`title`/`description`/`commandSlug`/`skillSlug`) in `OUT_DIR`; focused internal seams separate install-flow from prompt-flow logic; `cli/src/services/setup/command.rs` owns the `SetupCommand` payload used by the static `RuntimeCommand` enum and executes against any context implementing repo-root scoping. Its install engine/orchestrator (`install_assets_for_concrete_target_with_rename`) installs each embedded asset individually: it stages the asset's content next to its final destination and swaps it into place by renaming the staging file directly over the destination — never unlinking the destination first, since `fs::rename` already replaces an existing file atomically — with deterministic recovery guidance naming the failing asset's path on swap failure (the pre-existing destination content, if any, is untouched) and no backup artifact creation; it never removes an integration target directory as a whole, so files a repository owns inside `.opencode`/`.claude`/`.pi` — including nested inside an SCE-owned subdirectory such as `skills/` or `commands/` — survive a setup run; Codex has no single target directory this way, since `CODEX_EMBEDDED_ASSETS` relative paths already carry their own `.agents/`/`.codex/` prefix and `InstallTargetPaths::codex_target_dir()` resolves to the repository root itself, but the same per-asset stage/atomic-swap and never-delete-what-it-did-not-author guarantees still apply file by file. For the three assets that are merge targets — the Claude target's `settings.json`, the OpenCode target's `opencode.json`, and Codex's user-owned `.codex/hooks.json` — the content staged is not always the embedded asset's bytes. Claude and OpenCode use `cli/src/services/setup/config_merge.rs`, while Codex uses the shared `cli/src/services/codex_hook_config.rs` service for structural validation and canonical-registration merging. For the Claude and OpenCode targets, the content staged is the result of `cli/src/services/setup/config_merge.rs::merge_or_create_claude_settings`/`merge_or_create_opencode_config(existing_bytes, generated_bytes, source_path)`: each returns the generated document verbatim when no file exists yet, otherwise parses the existing file as JSON (a parse failure is a hard error naming `source_path`, and nothing is written) and merges it with a pure per-shape function that copies `$schema` from the generated document and preserves every other key from the existing file untouched. `merge_claude_settings` replaces, per hook event key the generated document declares, only the entries whose command contains the ownership marker `run-sce-or-show-install-guidance.sh`, preserving every event key and non-SCE hook entry from the existing file. `merge_opencode_config` merges the `plugin` array as a set: existing entries whose path starts with the ownership marker `./plugins/sce-` are dropped structurally — so a stale plugin path an older or renamed catalog once installed is still recognized and pruned even after the current generated document stops declaring it — and the generated document's `plugin` entries are appended after the surviving entries. Codex's merge validates the document shape and recognizes ownership by the generated helper path plus one of two trailing command contracts — `sce hooks codex` or `sce hooks codex-mutation-scope` — attributing each handler to whichever it matches; it replaces stale or duplicate handlers with exactly one current handler per registration (the four `sce hooks codex` registrations plus six additive `sce hooks codex-mutation-scope` registrations, within that mutation-scope set, `PreToolUse` and `PostToolUse` use matcher `^(Bash|apply_patch)$`, while `Stop`, `Interrupt`, `SubagentStop`, and `SessionEnd` are appended in unmatched groups after the existing groups so an already-trusted handler keeps its `(event, matcher, group index, handler index)` identity and computed Codex trust key), touching only the matching command's handlers and preserving unrelated valid Codex fields, groups, and handlers. After that per-asset install loop, `prune_stale_assets_for_concrete_target` deletes every path the full embedded catalog for the concrete target claims but the resolved selection did not install (a deselected optional workflow, or an asset a newer catalog renamed or dropped), and `remove_empty_ancestor_directories` removes any parent directory left empty by that deletion, stopping at the target root or at a directory that still holds something such as a user file. It formats deterministic completion messaging; required-hook install orchestration (`install_required_git_hooks`, backed by the rename-injectable `install_required_git_hooks_with_rename`) is a third content-computation seam alongside the two JSON merge targets: `install_single_required_hook_with_rename` computes the bytes to stage with `cli/src/services/setup/hook_merge.rs::merge_or_create_hook(existing_bytes, canonical_bytes, hook_name)` rather than writing `hook_asset.bytes` verbatim — a foreign hook (no SCE managed block, no legacy guidance-URL marker) is kept as an exact byte prefix with the canonical block appended after it, an SCE-owned hook has only its block spliced in place or left unchanged, and a legacy pre-marker hook is replaced wholesale — then follows the same per-file stage/atomic-swap choreography as config-asset install: the staging file is renamed directly over the existing hook without unlinking it first, so a rename failure leaves the prior hook's bytes and executable bit intact, with deterministic recovery guidance on swap failure. `Installed`/`Updated`/`Skipped` are decided against the merged bytes plus the executable bit rather than the canonical asset's raw bytes, so an already-current foreign-plus-block hook reports `Skipped`; `RequiredHookInstallResult.unreachable_block_advisory` is set, and rendered as a named advisory line in setup's hook output, when an appended block follows a foreign hook's zero-indent `exec`/`exit` and so would never run. After the Git gate, setup always ensures the context baseline; context-only requests return there, while normal modes derive a repo-root-scoped context before aggregating static lifecycle provider `setup` dispatch across providers (config → local_db → auth_db → agent_trace_db → hooks when requested), so setup providers consume only repo-root access from the scoped context. - `cli/src/services/setup/mod.rs` keeps those responsibilities inside one file for now, but the current ownership split is explicit: the inline `install` module owns repository-path normalization, staging/swap install behavior, required-hook installation, and filesystem safety guards, while the inline `prompt` module owns interactive target selection and prompt styling. -- `cli/src/services/setup/access_preferences.rs` owns setup's access preferences (`auth.mode`, `telemetry.enabled`): the repository-layer reader and atomic writer for `.sce/config.json` (`read_repository_access_preferences` / `persist_repository_access_preferences`, a guarded write over the shared temp-file-plus-rename helper in `setup/atomic_write.rs`), plus a read-only global reader used for the Claude telemetry removal check. `setup/command.rs` calls the writer after dispatch and before the target install, and re-resolves the policy for the `SCE access` report and for the setup login step (`setup_login_action` / `run_setup_login_step`, which reuses `auth_command::run_login_in` rather than a separate login path); precedence stays in `config/resolver.rs` (see `context/cli/access-policy.md#setup`). +- `cli/src/services/setup/access_preferences.rs` owns setup's access preferences (`auth.mode`, `telemetry.enabled`): the repository-layer reader and atomic writer for `.sce/config.json` (`read_repository_access_preferences` / `persist_repository_access_preferences`, through `write_guarded` in `setup/config_file_write.rs`, which owns `ConfigLayer` and the setup config-file write guard: reread-compare, symlink refusal, then the shared temp-file-plus-rename helper in `setup/atomic_write.rs`), plus a read-only global reader used for the Claude telemetry removal check. `setup/command.rs` calls the writer after dispatch and before the target install, and re-resolves the policy for the `SCE access` report and for the setup login step (`setup_login_action` / `run_setup_login_step`, which reuses `auth_command::run_login_in` rather than a separate login path); precedence stays in `config/resolver.rs` (see `context/cli/access-policy.md#setup`). - `cli/src/services/security.rs` provides shared security utilities for deterministic secret redaction (`redact_sensitive_text`) and directory write-permission probes (`ensure_directory_is_writable`) used by app/setup/observability surfaces. - `cli/src/services/doctor/mod.rs` owns the current doctor request/report surface while focused submodules (`doctor/inspect.rs`, `doctor/render.rs`, `doctor/fixes.rs`, `doctor/types.rs`) split report fact collection, rendering, manual fix reporting, and doctor-owned domain types into smaller seams; `cli/src/services/doctor/command.rs` owns the `DoctorCommand` payload used by the static `RuntimeCommand` enum and executes against any context implementing repo-root scoping. Runtime doctor execution resolves a repository root, derives a scoped context, requests the shared static lifecycle provider catalog with hooks included for service-owned `diagnose` and `fix` behavior, adapts lifecycle-owned health/fix records into doctor-owned problem/fix records, and then renders stable text/JSON problem records with category/severity/fixability/remediation fields plus deterministic fix-result reporting in fix mode. Agent Trace database inspection is no longer a doctor-adjacent command surface; doctor owns repository-scoped DB health and checkout identity facts, while `sce sync` owns control-plane synchronization. Report fact collection preserves environment/repository/hook/integration display data and adds a non-launching `post_commit_auto_sync` fact based on canonical post-commit managed-block currency plus resolved `agent_trace.auto_sync` source/value; this fact does not launch `sce sync`, and existing hook problem/remediation/readiness semantics remain authoritative. Service-owned lifecycle providers own config validation, local DB and repository-scoped Agent Trace DB readiness/bootstrap, and hook rollout diagnosis/repair. Integration inspection in `doctor/inspect.rs` is scoped twice over: `resolve_doctor_integration_targets` picks which targets to inspect, and `persisted_optional_workflows` (reused from setup) resolves which optional workflows the repository selected, which the OpenCode/Claude/Pi/Codex child collectors apply through `iter_embedded_assets_for_setup_target_with_selection`. An unselected optional workflow therefore contributes no expected children at all, so no row and no missing/mismatch problem can be produced for it, while a selected one keeps the unchanged presence and content-hash checks. Codex's collector (`collect_codex_integration_groups`) is the one exception to the single-target-directory shape the other three share: since `CODEX_EMBEDDED_ASSETS` relative paths already carry their own `.agents/`/`.codex/` prefix, it resolves its integration root through `InstallTargetPaths::codex_target_dir()` (the repository root itself) rather than a `RepoPaths` per-target subdirectory, and splits assets into `Skills` (`.agents/skills/` prefix) and `Hooks` (`.codex/` prefix) groups instead of Pi's `Extensions`/`Prompts`/`Skills` split. Its `.codex/hooks.json` reporting is per-registration rather than one whole-file child: `codex_hook_config::diagnose_document` classifies each required registration — the four `sce hooks codex` registrations plus the six `sce hooks codex-mutation-scope` ones, the latter reported under distinct `#(mutation-scope)` rows — as `PresentAndCurrent`/`Missing`/`Stale` (or the whole document as `Malformed` when it cannot be structurally validated) without writing anything, so unrelated user handlers never create a false whole-document mismatch. For a structurally current registration, `codex_hook_trust` separately reads (never writes) Codex's own durable `$CODEX_HOME/config.toml` hook-trust state — reproducing upstream's `hook_hash`/`hook_key`/`hook_trust_status` exactly — and reports `Trusted`/`Untrusted`/`Modified`/`Disabled`/`Unknown`, and `codex_hook_policy` probes Codex's effective hook-discovery policy once per invocation; only a structurally current registration that is trusted *and* policy-allowed renders healthy. `sce doctor --fix` repairs a structurally unhealthy `.codex/hooks.json` through the existing merge-install path, but a registration that is current yet not-yet-trusted or policy-blocked is never "fixed", since SCE cannot grant Codex hook trust or change managed policy. - `cli/src/services/version/mod.rs` defines the version command parser/rendering contract (`parse_version_request`, `render_version`) with deterministic text output and stable JSON runtime-identification fields; `cli/src/services/version/command.rs` owns the `VersionCommand` payload used by the static `RuntimeCommand` enum. diff --git a/context/plans/config-schema-series-compatibility.md b/context/plans/config-schema-series-compatibility.md index d92a2eb97..efd504070 100644 --- a/context/plans/config-schema-series-compatibility.md +++ b/context/plans/config-schema-series-compatibility.md @@ -270,7 +270,7 @@ Persist this field in every plan; this is durable plan state, not chat state: - Context impact: localized. `$schema` validation messages and the startup guidance changed. `context/cli/config-precedence-contract.md` does not yet describe the actionable messages; T08 updates it. - Context synchronization: synced -- [ ] T03: `Extract the guarded config-file write shared by setup writers` (status:todo) +- [x] T03: `Extract the guarded config-file write shared by setup writers` (status:done) - Task ID: T03 - Scope: - In: @@ -284,7 +284,30 @@ Persist this field in every plan; this is durable plan state, not chat state: - Verify: - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml access_preferences` - `… test … setup` - - Context synchronization: pending + - Completed: 2026-11-10 + - Files changed: + - `cli/src/services/setup/config_file_write.rs` (new) + - `cli/src/services/setup/access_preferences.rs` + - `cli/src/services/setup/mod.rs` + - Result: + - New `config_file_write` module (`pub(super)`) owns: + - `ConfigLayer`, with `path_source`, `label`, and `bootstrap_value`; + - `read_optional`; + - the private `is_symlink`; + - `GuardedWriteError { ConcurrentModification, SymlinkManaged, Io(anyhow::Error) }`; + - `write_guarded(path, layer, original: Option<&[u8]>, serialized: &[u8], reread)`. It rereads and compares against `original`, refuses a symlink through `fs::symlink_metadata`, then calls `write_atomically`, in the same order as before. + - `access_preferences.rs` now uses the shared helper. Its private `WriteFailure` converts from `GuardedWriteError` through `From`. Every public type, error variant, and message is unchanged. The test bodies are unchanged; only the test module imports moved. + - `atomic_write.rs` is unchanged, and `claude_telemetry.rs` still uses it directly. + - Deviation: the done check names a dangling-symlink test, but `access_preferences` had none. Three helper tests were added in `config_file_write` instead: creating a missing file, refusing a file created after planning, and refusing a dangling symlink while leaving it intact. + - Verify: + - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml access_preferences`: passed (7 passed, 0 failed). This covers the symlink, concurrent-modification, invalid-file, and mode-preservation tests. + - `… test … setup`: passed (81 passed, 0 failed). + - Supplementary checks: + - `… test … config_file_write`: passed (3 passed, 0 failed). + - `clippy --all-targets -- -D warnings`: passed. + - `cargo fmt --check`: passed. + - Context impact: localized. This is an internal refactor of setup's write helpers with no behavior change. + - Context synchronization: synced - [ ] T04: `Plan and commit a validated $schema-only upgrade for one config file` (status:todo) - Task ID: T04 From 1071c675b42b355c50527a0895976d97dbe8b61b Mon Sep 17 00:00:00 2001 From: David Abram Date: Tue, 10 Nov 2026 13:28:02 +0100 Subject: [PATCH 5/9] setup: Add validated schema-series config upgrades Prepare a read-only planner and guarded commit path for upgrading older `$schema` series declarations in repository and global config files. Preserve the original bytes when possible by swapping only the schema literal, validate fallback serialization with the layer-specific parser, and skip newer, unrecognized, invalid, unreadable, symlinked, or concurrently changed files without rewriting them. Document the new module in architecture context and complete plan `config-schema-series-compatibility`, task T04, with its implementation and verification evidence. The setup flow remains unwired until T05 and T06. Co-authored-by: SCE --- cli/src/services/setup/mod.rs | 5 + cli/src/services/setup/schema_upgrade.rs | 576 ++++++++++++++++++ context/architecture.md | 1 + .../config-schema-series-compatibility.md | 40 +- 4 files changed, 620 insertions(+), 2 deletions(-) create mode 100644 cli/src/services/setup/schema_upgrade.rs diff --git a/cli/src/services/setup/mod.rs b/cli/src/services/setup/mod.rs index 1b36f37e0..bc80389c6 100644 --- a/cli/src/services/setup/mod.rs +++ b/cli/src/services/setup/mod.rs @@ -17,6 +17,11 @@ pub mod command; mod config_file_write; pub(crate) mod config_merge; pub(crate) mod hook_merge; +#[expect( + dead_code, + reason = "setup ordering and the setup report consume the planner and its skip details" +)] +mod schema_upgrade; #[derive(Debug)] struct MissingGitRemoteError { diff --git a/cli/src/services/setup/schema_upgrade.rs b/cli/src/services/setup/schema_upgrade.rs new file mode 100644 index 000000000..18be6fe5f --- /dev/null +++ b/cli/src/services/setup/schema_upgrade.rs @@ -0,0 +1,576 @@ +use std::path::Path; + +use anyhow::{anyhow, Context}; +use serde_json::Value; + +use crate::services::agent_trace::sce_config_schema_url; +use crate::services::config::schema::{parse_file_config, CONFIG_SCHEMA_DECLARATION_KEY}; +use crate::services::config::schema_declaration::{ + classify_schema_declaration, current_schema_series, SchemaDeclaration, SchemaVersion, +}; + +use super::config_file_write::{read_optional, write_guarded, ConfigLayer, GuardedWriteError}; + +#[derive(Debug)] +pub(crate) enum SchemaUpgradePlan { + NotPresent, + NoDeclaration, + Current, + NewerSeries(SchemaVersion), + Unrecognized, + Unreadable(anyhow::Error), + Ready { + original: Vec, + candidate: Vec, + from: SchemaVersion, + }, + CandidateInvalid { + from: SchemaVersion, + reason: String, + }, +} + +#[derive(Debug)] +pub(crate) enum SchemaUpgradeOutcome { + Upgraded { from: SchemaVersion }, + Unchanged, + Skipped(SchemaUpgradeSkip), +} + +#[derive(Debug)] +pub(crate) enum SchemaUpgradeSkip { + NewerSeries(SchemaVersion), + Unrecognized, + CandidateInvalid { from: SchemaVersion, reason: String }, + SymlinkManaged, + ConcurrentModification, + Io(anyhow::Error), +} + +impl From for SchemaUpgradeSkip { + fn from(error: GuardedWriteError) -> Self { + match error { + GuardedWriteError::ConcurrentModification => Self::ConcurrentModification, + GuardedWriteError::SymlinkManaged => Self::SymlinkManaged, + GuardedWriteError::Io(error) => Self::Io(error), + } + } +} + +/// Reads and classifies one canonical config file and, for an older-series +/// declaration, builds and validates the `$schema`-only candidate. Never writes. +pub(crate) fn plan_schema_upgrade(path: &Path, layer: ConfigLayer) -> SchemaUpgradePlan { + let original = match read_optional(path, layer) { + Ok(Some(bytes)) => bytes, + Ok(None) => return SchemaUpgradePlan::NotPresent, + Err(error) => return SchemaUpgradePlan::Unreadable(error), + }; + let document = match parse_document(path, layer, &original) { + Ok(document) => document, + Err(error) => return SchemaUpgradePlan::Unreadable(error), + }; + + let declaration = document + .value + .as_object() + .and_then(|object| object.get(CONFIG_SCHEMA_DECLARATION_KEY)); + let from = match classify_schema_declaration(declaration, current_schema_series()) { + SchemaDeclaration::Absent => return SchemaUpgradePlan::NoDeclaration, + SchemaDeclaration::CurrentSeries => return SchemaUpgradePlan::Current, + SchemaDeclaration::NewerSeries(version) => return SchemaUpgradePlan::NewerSeries(version), + SchemaDeclaration::Unrecognized => return SchemaUpgradePlan::Unrecognized, + SchemaDeclaration::OlderSeries(version) => version, + }; + + let candidate = match build_candidate(&document, &sce_config_schema_url()) { + Ok(candidate) => candidate, + Err(error) => return SchemaUpgradePlan::Unreadable(error), + }; + if let Err(error) = parse_file_config(&candidate, path, layer.path_source()) { + return SchemaUpgradePlan::CandidateInvalid { + from, + reason: format!("{error:#}"), + }; + } + + SchemaUpgradePlan::Ready { + original, + candidate: candidate.into_bytes(), + from, + } +} + +pub(crate) fn commit_schema_upgrade( + path: &Path, + layer: ConfigLayer, + plan: SchemaUpgradePlan, +) -> SchemaUpgradeOutcome { + commit_schema_upgrade_with(path, layer, plan, |path| read_optional(path, layer)) +} + +fn commit_schema_upgrade_with( + path: &Path, + layer: ConfigLayer, + plan: SchemaUpgradePlan, + reread: R, +) -> SchemaUpgradeOutcome +where + R: FnOnce(&Path) -> anyhow::Result>>, +{ + let skip = match plan { + SchemaUpgradePlan::NotPresent + | SchemaUpgradePlan::NoDeclaration + | SchemaUpgradePlan::Current => return SchemaUpgradeOutcome::Unchanged, + SchemaUpgradePlan::NewerSeries(version) => SchemaUpgradeSkip::NewerSeries(version), + SchemaUpgradePlan::Unrecognized => SchemaUpgradeSkip::Unrecognized, + SchemaUpgradePlan::Unreadable(error) => SchemaUpgradeSkip::Io(error), + SchemaUpgradePlan::CandidateInvalid { from, reason } => { + SchemaUpgradeSkip::CandidateInvalid { from, reason } + } + SchemaUpgradePlan::Ready { + original, + candidate, + from, + } => match write_guarded(path, layer, Some(&original), &candidate, reread) { + Ok(()) => return SchemaUpgradeOutcome::Upgraded { from }, + Err(error) => error.into(), + }, + }; + SchemaUpgradeOutcome::Skipped(skip) +} + +struct ConfigDocument<'a> { + raw: &'a str, + value: Value, +} + +fn parse_document<'a>( + path: &Path, + layer: ConfigLayer, + bytes: &'a [u8], +) -> anyhow::Result> { + let raw = std::str::from_utf8(bytes) + .with_context(|| format!("{} '{}' is not valid UTF-8", layer.label(), path.display()))?; + let value: Value = serde_json::from_str(raw) + .with_context(|| format!("{} '{}' is not valid JSON", layer.label(), path.display()))?; + if !value.is_object() { + return Err(anyhow!( + "{} '{}' must contain a top-level JSON object", + layer.label(), + path.display() + )); + } + Ok(ConfigDocument { raw, value }) +} + +/// Prefers swapping the single encoded `$schema` literal so every other byte is +/// preserved; the swap is kept only when the result is structurally identical +/// to the original apart from `$schema`. +fn build_candidate(document: &ConfigDocument<'_>, target_url: &str) -> anyhow::Result { + let mut expected = document.value.clone(); + let object = expected + .as_object_mut() + .context("config document must be a JSON object")?; + let previous = object.insert( + CONFIG_SCHEMA_DECLARATION_KEY.to_string(), + Value::String(target_url.to_string()), + ); + + if let Some(Value::String(previous_url)) = previous { + let previous_literal = serde_json::to_string(&previous_url)?; + if document.raw.matches(&previous_literal).count() == 1 { + let target_literal = serde_json::to_string(target_url)?; + let swapped = document.raw.replacen(&previous_literal, &target_literal, 1); + if serde_json::from_str::(&swapped).ok().as_ref() == Some(&expected) { + return Ok(swapped); + } + } + } + + let mut serialized = serde_json::to_string_pretty(&expected)?; + serialized.push('\n'); + Ok(serialized) +} + +#[cfg(test)] +mod tests { + use std::fs; + use std::path::PathBuf; + + use serde_json::json; + + use super::*; + use crate::services::config::schema_declaration::SchemaSeries; + use crate::services::config::ConfigPathSource; + + fn schema_url(version: &str) -> String { + format!("https://sce.crocoder.dev/v{version}/config.json") + } + + fn older_series_version() -> String { + let SchemaSeries { major, minor } = current_schema_series(); + if minor > 0 { + format!("{major}.{}.3", minor - 1) + } else { + format!("{}.9.3", major - 1) + } + } + + fn newer_series_version() -> String { + let SchemaSeries { major, minor } = current_schema_series(); + format!("{major}.{}.0", minor + 1) + } + + fn config_path(dir: &Path) -> PathBuf { + dir.join(".sce").join("config.json") + } + + fn write_config(path: &Path, contents: &str) { + fs::create_dir_all(path.parent().unwrap()).unwrap(); + fs::write(path, contents).unwrap(); + } + + fn stale_config(declaration: &str) -> String { + format!( + r#"{{ + "$schema": "{declaration}", + "auth": {{ "mode": "disabled" }}, + "telemetry": {{"enabled": false}}, + "integrations": {{ + "target": ["claude", "opencode"] + }}, + "policies": {{ + "bash": {{ + "custom": [ + {{ "id": "no-curl", "match": {{ "argv_prefix": ["curl"] }}, "message": "Use the fetch helper." }} + ] + }} + }} +}} +"# + ) + } + + fn plan_and_commit(path: &Path, layer: ConfigLayer) -> SchemaUpgradeOutcome { + commit_schema_upgrade(path, layer, plan_schema_upgrade(path, layer)) + } + + #[test] + fn schema_upgrade_ignores_missing_file() { + let dir = tempfile::tempdir().unwrap(); + let path = config_path(dir.path()); + + assert!(matches!( + plan_schema_upgrade(&path, ConfigLayer::Global), + SchemaUpgradePlan::NotPresent + )); + assert!(matches!( + plan_and_commit(&path, ConfigLayer::Global), + SchemaUpgradeOutcome::Unchanged + )); + assert!(!path.exists()); + } + + #[test] + fn schema_upgrade_leaves_file_without_declaration_unchanged() { + let dir = tempfile::tempdir().unwrap(); + let path = config_path(dir.path()); + let contents = "{ \"auth\": { \"mode\": \"optional\" } }\n"; + write_config(&path, contents); + + assert!(matches!( + plan_schema_upgrade(&path, ConfigLayer::Repository), + SchemaUpgradePlan::NoDeclaration + )); + assert!(matches!( + plan_and_commit(&path, ConfigLayer::Repository), + SchemaUpgradeOutcome::Unchanged + )); + assert_eq!(fs::read_to_string(&path).unwrap(), contents); + } + + #[test] + fn schema_upgrade_preserves_same_series_declarations() { + let SchemaSeries { major, minor } = current_schema_series(); + let dir = tempfile::tempdir().unwrap(); + let path = config_path(dir.path()); + for version in [ + format!("{major}.{minor}.0-pre-alpha-v1"), + format!("{major}.{minor}.7"), + format!("{major}.{minor}.7+build.5"), + ] { + let contents = stale_config(&schema_url(&version)); + write_config(&path, &contents); + + assert!(matches!( + plan_schema_upgrade(&path, ConfigLayer::Repository), + SchemaUpgradePlan::Current + )); + assert!(matches!( + plan_and_commit(&path, ConfigLayer::Repository), + SchemaUpgradeOutcome::Unchanged + )); + assert_eq!(fs::read_to_string(&path).unwrap(), contents); + } + } + + #[test] + fn schema_upgrade_replaces_only_older_series_declaration() { + let dir = tempfile::tempdir().unwrap(); + let path = config_path(dir.path()); + for older in [older_series_version(), "0.1.0".to_string()] { + let contents = stale_config(&schema_url(&older)); + write_config(&path, &contents); + + let outcome = plan_and_commit(&path, ConfigLayer::Repository); + + let SchemaUpgradeOutcome::Upgraded { from } = outcome else { + panic!("expected an upgrade, got {outcome:?}"); + }; + assert_eq!(from.to_string(), older); + assert_eq!( + fs::read_to_string(&path).unwrap(), + stale_config(&sce_config_schema_url()) + ); + assert!(matches!( + plan_schema_upgrade(&path, ConfigLayer::Repository), + SchemaUpgradePlan::Current + )); + let upgraded = fs::read(&path).unwrap(); + assert!(matches!( + plan_and_commit(&path, ConfigLayer::Repository), + SchemaUpgradeOutcome::Unchanged + )); + assert_eq!(fs::read(&path).unwrap(), upgraded); + } + } + + #[test] + fn schema_upgrade_falls_back_to_pretty_serialization_when_literal_is_ambiguous() { + let older_url = schema_url(&older_series_version()); + let raw = format!( + "{{\"$schema\":\"{older_url}\",\"policies\":{{\"bash\":{{\"custom\":[{{\"id\":\"x\",\"match\":{{\"argv_prefix\":[\"{older_url}\"]}},\"message\":\"m\"}}]}}}}}}" + ); + let document = ConfigDocument { + raw: &raw, + value: serde_json::from_str(&raw).unwrap(), + }; + + let candidate = build_candidate(&document, &sce_config_schema_url()).unwrap(); + + let candidate_value: Value = serde_json::from_str(&candidate).unwrap(); + assert_eq!(candidate_value["$schema"], json!(sce_config_schema_url())); + assert_eq!( + candidate_value["policies"]["bash"]["custom"][0]["match"]["argv_prefix"], + json!([older_url]) + ); + assert!(candidate.ends_with("}\n")); + } + + #[test] + fn schema_upgrade_never_rewrites_newer_or_unrecognized_declarations() { + let dir = tempfile::tempdir().unwrap(); + let path = config_path(dir.path()); + + let newer = stale_config(&schema_url(&newer_series_version())); + write_config(&path, &newer); + assert!(matches!( + plan_and_commit(&path, ConfigLayer::Repository), + SchemaUpgradeOutcome::Skipped(SchemaUpgradeSkip::NewerSeries(version)) + if version.to_string() == newer_series_version() + )); + assert_eq!(fs::read_to_string(&path).unwrap(), newer); + + for foreign in [ + "https://example.com/s.json".to_string(), + "https://sce.crocoder.dev/config.json".to_string(), + schema_url("0.4"), + format!( + "http://sce.crocoder.dev/v{}/config.json", + older_series_version() + ), + ] { + let contents = stale_config(&foreign); + write_config(&path, &contents); + assert!(matches!( + plan_and_commit(&path, ConfigLayer::Repository), + SchemaUpgradeOutcome::Skipped(SchemaUpgradeSkip::Unrecognized) + )); + assert_eq!(fs::read_to_string(&path).unwrap(), contents); + } + + let non_string = "{ \"$schema\": 4 }\n"; + write_config(&path, non_string); + assert!(matches!( + plan_and_commit(&path, ConfigLayer::Repository), + SchemaUpgradeOutcome::Skipped(SchemaUpgradeSkip::Unrecognized) + )); + assert_eq!(fs::read_to_string(&path).unwrap(), non_string); + } + + #[test] + fn schema_upgrade_reports_unreadable_files_without_writing() { + let dir = tempfile::tempdir().unwrap(); + let path = config_path(dir.path()); + for invalid in [b"{ not json".to_vec(), b"[]\n".to_vec(), vec![0xff, 0xfe]] { + write_config(&path, ""); + fs::write(&path, &invalid).unwrap(); + + assert!(matches!( + plan_schema_upgrade(&path, ConfigLayer::Repository), + SchemaUpgradePlan::Unreadable(_) + )); + assert!(matches!( + plan_and_commit(&path, ConfigLayer::Repository), + SchemaUpgradeOutcome::Skipped(SchemaUpgradeSkip::Io(_)) + )); + assert_eq!(fs::read(&path).unwrap(), invalid); + } + } + + #[test] + fn schema_upgrade_keeps_file_when_candidate_fails_validation() { + let dir = tempfile::tempdir().unwrap(); + let path = config_path(dir.path()); + let older = older_series_version(); + let contents = format!( + "{{\n \"$schema\": \"{}\",\n \"unknown_key\": true\n}}\n", + schema_url(&older) + ); + write_config(&path, &contents); + + let plan = plan_schema_upgrade(&path, ConfigLayer::Repository); + let SchemaUpgradePlan::CandidateInvalid { from, reason } = &plan else { + panic!("expected an invalid candidate, got {plan:?}"); + }; + assert_eq!(from.to_string(), older); + assert!(reason.contains("unknown_key"), "{reason}"); + assert!(!reason.contains("/$schema"), "{reason}"); + + assert!(matches!( + commit_schema_upgrade(&path, ConfigLayer::Repository, plan), + SchemaUpgradeOutcome::Skipped(SchemaUpgradeSkip::CandidateInvalid { .. }) + )); + assert_eq!(fs::read_to_string(&path).unwrap(), contents); + } + + #[test] + fn schema_upgrade_validates_global_candidate_as_discovered_global() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("sce").join("config.json"); + let contents = stale_config(&schema_url(&older_series_version())); + write_config(&path, &contents); + + let SchemaUpgradePlan::Ready { candidate, .. } = + plan_schema_upgrade(&path, ConfigLayer::Global) + else { + panic!("expected a ready global plan"); + }; + let parsed = parse_file_config( + std::str::from_utf8(&candidate).unwrap(), + &path, + ConfigLayer::Global.path_source(), + ) + .unwrap(); + assert_eq!( + parsed.auth_mode.unwrap().source, + ConfigPathSource::DefaultDiscoveredGlobal + ); + + assert!(matches!( + plan_and_commit(&path, ConfigLayer::Global), + SchemaUpgradeOutcome::Upgraded { .. } + )); + assert_eq!( + fs::read_to_string(&path).unwrap(), + stale_config(&sce_config_schema_url()) + ); + } + + #[test] + fn schema_upgrade_refuses_concurrent_modification_between_plan_and_commit() { + let dir = tempfile::tempdir().unwrap(); + let path = config_path(dir.path()); + write_config(&path, &stale_config(&schema_url(&older_series_version()))); + let plan = plan_schema_upgrade(&path, ConfigLayer::Repository); + assert!(matches!(plan, SchemaUpgradePlan::Ready { .. })); + let concurrent = stale_config(&schema_url("0.1.0")); + + let outcome = commit_schema_upgrade_with(&path, ConfigLayer::Repository, plan, |path| { + fs::write(path, &concurrent).unwrap(); + read_optional(path, ConfigLayer::Repository) + }); + + assert!(matches!( + outcome, + SchemaUpgradeOutcome::Skipped(SchemaUpgradeSkip::ConcurrentModification) + )); + assert_eq!(fs::read_to_string(&path).unwrap(), concurrent); + } + + #[cfg(unix)] + #[test] + fn schema_upgrade_refuses_symlinked_file_and_leaves_it_unchanged() { + use std::os::unix::fs::symlink; + + let dir = tempfile::tempdir().unwrap(); + let path = config_path(dir.path()); + let target = dir.path().join("dotfiles-config.json"); + let contents = stale_config(&schema_url(&older_series_version())); + fs::write(&target, &contents).unwrap(); + fs::create_dir_all(path.parent().unwrap()).unwrap(); + symlink(&target, &path).unwrap(); + + assert!(matches!( + plan_and_commit(&path, ConfigLayer::Global), + SchemaUpgradeOutcome::Skipped(SchemaUpgradeSkip::SymlinkManaged) + )); + assert_eq!(fs::read_link(&path).unwrap(), target); + assert_eq!(fs::read_to_string(&target).unwrap(), contents); + } + + #[cfg(unix)] + #[test] + fn schema_upgrade_leaves_dangling_symlink_untouched() { + use std::os::unix::fs::symlink; + + let dir = tempfile::tempdir().unwrap(); + let path = config_path(dir.path()); + let target = dir.path().join("missing-target.json"); + fs::create_dir_all(path.parent().unwrap()).unwrap(); + symlink(&target, &path).unwrap(); + + assert!(matches!( + plan_and_commit(&path, ConfigLayer::Global), + SchemaUpgradeOutcome::Unchanged + )); + assert_eq!(fs::read_link(&path).unwrap(), target); + assert!(!target.exists()); + } + + #[cfg(unix)] + #[test] + fn schema_upgrade_replaces_file_atomically_and_keeps_its_mode() { + use std::os::unix::fs::PermissionsExt; + + let dir = tempfile::tempdir().unwrap(); + let path = config_path(dir.path()); + write_config(&path, &stale_config(&schema_url(&older_series_version()))); + fs::set_permissions(&path, fs::Permissions::from_mode(0o640)).unwrap(); + + assert!(matches!( + plan_and_commit(&path, ConfigLayer::Repository), + SchemaUpgradeOutcome::Upgraded { .. } + )); + + assert_eq!( + fs::metadata(&path).unwrap().permissions().mode() & 0o7777, + 0o640 + ); + let entries: Vec<_> = fs::read_dir(path.parent().unwrap()) + .unwrap() + .map(|entry| entry.unwrap().file_name()) + .collect(); + assert_eq!(entries, vec![std::ffi::OsString::from("config.json")]); + } +} diff --git a/context/architecture.md b/context/architecture.md index a0c6c7807..7f20fe97c 100644 --- a/context/architecture.md +++ b/context/architecture.md @@ -72,6 +72,7 @@ The repository includes a new placeholder Rust binary crate at `cli/`. - `cli/src/services/setup/mod.rs` defines the setup command contract (`SetupMode`, `SetupTarget`, `SetupRequest`, CLI flag parser/validator), an `inquire`-backed interactive target prompter (`InquireSetupTargetPrompter`), setup dispatch outcomes (proceed/cancelled), additive durable-context bootstrap (`bootstrap_context_baseline` for standalone `--bootstrap-context` and every normal successful setup path), and compile-time embedded asset access (`EmbeddedAsset`, target-scoped iterators, required-hook asset iterators/lookups). It also owns the install-time optional-workflow seam: the `OptionalWorkflow` type plus the build-generated `OPTIONAL_WORKFLOWS` catalog, a per-target `WorkflowAssetLayout` built from the existing `default_paths` command/skill directory constants (`command`/`commands`/`prompts` plus `skills`), and `iter_embedded_assets_for_setup_target_with_selection(target, selection)`, which yields every embedded asset except the `{command_dir}/{command_slug}.md` file and `{skills_dir}/{skill_slug}/` subtree of each optional workflow the selection omits. Membership is derived from the catalog's slugs rather than an enumerated file list, so a new optional workflow needs no Rust change. This filtered iterator is the only way embedded assets are enumerated; setup installs through it and doctor inspects through it, so there is no unfiltered enumeration path that could reintroduce an unselected workflow. The non-interactive selection flows through the repeatable `--workflow ` flag into `SetupRequest.optional_workflows: Option>` (`None` meaning the flag was absent); `validate_optional_workflow_slugs` checks each slug against `OPTIONAL_WORKFLOWS` during request resolution, before any write, and `run_setup_for_mode` resolves `None` to the persisted `integrations.optional_workflows` (exported as `persisted_optional_workflows`) before installing through the filtered iterator and persisting the resolved selection. The interactive selection flows through the same seam: `SetupTargetPrompter` carries `prompt_targets(defaults)`, `prompt_optional_workflows(defaults)`, the two behavior confirmations `prompt_agent_trace_auto_sync(default)` / `prompt_attribution_hooks_enabled(default)` (dispatch passes `true`), and the access prompts `prompt_auth_mode(default)` (an `inquire::Select` with fixed Optional/Mandatory/Disabled rows), `prompt_telemetry_consent(default)`, and `prompt_login()` (Yes default), which dispatch does not call yet; every method returns `None` for a cancelled prompt and fails with the fixed TTY guidance on `NotTTY`, `SetupDispatch::Proceed { mode, optional_workflows }` carries a prompted selection alongside the resolved mode, and `resolve_setup_dispatch(mode, prompter, defaults)` runs the workflow prompt only after an interactive target prompt, mapping either cancellation to `SetupDispatch::Cancelled`. The prompt module builds its `inquire::MultiSelect` from `optional_workflow_prompt_inputs(catalog, defaults)`, which returns `None` for an empty catalog (skipping the prompt) and otherwise catalog-ordered rows plus the indices to pre-check, ignoring ids absent from the catalog. `setup/command.rs` therefore resolves the repository root before dispatch, mapping the setup domain's typed positive non-Git classification to `setup.not_git_repository` and other resolution failures to `general.unexpected_failure`, so a non-Git directory fails before any prompt without CLI-layer string matching. For repository builds, `cli/build.rs` validates the `SCE_CLI_GENERATED_INPUT_DIR` payload and canonical-input inventories, copies the payload into Cargo `OUT_DIR/pkl-generated`, stages `cli/assets/hooks/**` under `OUT_DIR/static`, requires the staged `config/optional-workflows.json`, and generates both the setup manifest and the optional-workflow catalog (`optional_workflows.rs`, rejecting a manifest whose `schemaVersion` is not 1 or whose entries lack a non-empty `id`/`title`/`description`/`commandSlug`/`skillSlug`) in `OUT_DIR`; focused internal seams separate install-flow from prompt-flow logic; `cli/src/services/setup/command.rs` owns the `SetupCommand` payload used by the static `RuntimeCommand` enum and executes against any context implementing repo-root scoping. Its install engine/orchestrator (`install_assets_for_concrete_target_with_rename`) installs each embedded asset individually: it stages the asset's content next to its final destination and swaps it into place by renaming the staging file directly over the destination — never unlinking the destination first, since `fs::rename` already replaces an existing file atomically — with deterministic recovery guidance naming the failing asset's path on swap failure (the pre-existing destination content, if any, is untouched) and no backup artifact creation; it never removes an integration target directory as a whole, so files a repository owns inside `.opencode`/`.claude`/`.pi` — including nested inside an SCE-owned subdirectory such as `skills/` or `commands/` — survive a setup run; Codex has no single target directory this way, since `CODEX_EMBEDDED_ASSETS` relative paths already carry their own `.agents/`/`.codex/` prefix and `InstallTargetPaths::codex_target_dir()` resolves to the repository root itself, but the same per-asset stage/atomic-swap and never-delete-what-it-did-not-author guarantees still apply file by file. For the three assets that are merge targets — the Claude target's `settings.json`, the OpenCode target's `opencode.json`, and Codex's user-owned `.codex/hooks.json` — the content staged is not always the embedded asset's bytes. Claude and OpenCode use `cli/src/services/setup/config_merge.rs`, while Codex uses the shared `cli/src/services/codex_hook_config.rs` service for structural validation and canonical-registration merging. For the Claude and OpenCode targets, the content staged is the result of `cli/src/services/setup/config_merge.rs::merge_or_create_claude_settings`/`merge_or_create_opencode_config(existing_bytes, generated_bytes, source_path)`: each returns the generated document verbatim when no file exists yet, otherwise parses the existing file as JSON (a parse failure is a hard error naming `source_path`, and nothing is written) and merges it with a pure per-shape function that copies `$schema` from the generated document and preserves every other key from the existing file untouched. `merge_claude_settings` replaces, per hook event key the generated document declares, only the entries whose command contains the ownership marker `run-sce-or-show-install-guidance.sh`, preserving every event key and non-SCE hook entry from the existing file. `merge_opencode_config` merges the `plugin` array as a set: existing entries whose path starts with the ownership marker `./plugins/sce-` are dropped structurally — so a stale plugin path an older or renamed catalog once installed is still recognized and pruned even after the current generated document stops declaring it — and the generated document's `plugin` entries are appended after the surviving entries. Codex's merge validates the document shape and recognizes ownership by the generated helper path plus one of two trailing command contracts — `sce hooks codex` or `sce hooks codex-mutation-scope` — attributing each handler to whichever it matches; it replaces stale or duplicate handlers with exactly one current handler per registration (the four `sce hooks codex` registrations plus six additive `sce hooks codex-mutation-scope` registrations, within that mutation-scope set, `PreToolUse` and `PostToolUse` use matcher `^(Bash|apply_patch)$`, while `Stop`, `Interrupt`, `SubagentStop`, and `SessionEnd` are appended in unmatched groups after the existing groups so an already-trusted handler keeps its `(event, matcher, group index, handler index)` identity and computed Codex trust key), touching only the matching command's handlers and preserving unrelated valid Codex fields, groups, and handlers. After that per-asset install loop, `prune_stale_assets_for_concrete_target` deletes every path the full embedded catalog for the concrete target claims but the resolved selection did not install (a deselected optional workflow, or an asset a newer catalog renamed or dropped), and `remove_empty_ancestor_directories` removes any parent directory left empty by that deletion, stopping at the target root or at a directory that still holds something such as a user file. It formats deterministic completion messaging; required-hook install orchestration (`install_required_git_hooks`, backed by the rename-injectable `install_required_git_hooks_with_rename`) is a third content-computation seam alongside the two JSON merge targets: `install_single_required_hook_with_rename` computes the bytes to stage with `cli/src/services/setup/hook_merge.rs::merge_or_create_hook(existing_bytes, canonical_bytes, hook_name)` rather than writing `hook_asset.bytes` verbatim — a foreign hook (no SCE managed block, no legacy guidance-URL marker) is kept as an exact byte prefix with the canonical block appended after it, an SCE-owned hook has only its block spliced in place or left unchanged, and a legacy pre-marker hook is replaced wholesale — then follows the same per-file stage/atomic-swap choreography as config-asset install: the staging file is renamed directly over the existing hook without unlinking it first, so a rename failure leaves the prior hook's bytes and executable bit intact, with deterministic recovery guidance on swap failure. `Installed`/`Updated`/`Skipped` are decided against the merged bytes plus the executable bit rather than the canonical asset's raw bytes, so an already-current foreign-plus-block hook reports `Skipped`; `RequiredHookInstallResult.unreachable_block_advisory` is set, and rendered as a named advisory line in setup's hook output, when an appended block follows a foreign hook's zero-indent `exec`/`exit` and so would never run. After the Git gate, setup always ensures the context baseline; context-only requests return there, while normal modes derive a repo-root-scoped context before aggregating static lifecycle provider `setup` dispatch across providers (config → local_db → auth_db → agent_trace_db → hooks when requested), so setup providers consume only repo-root access from the scoped context. - `cli/src/services/setup/mod.rs` keeps those responsibilities inside one file for now, but the current ownership split is explicit: the inline `install` module owns repository-path normalization, staging/swap install behavior, required-hook installation, and filesystem safety guards, while the inline `prompt` module owns interactive target selection and prompt styling. - `cli/src/services/setup/access_preferences.rs` owns setup's access preferences (`auth.mode`, `telemetry.enabled`): the repository-layer reader and atomic writer for `.sce/config.json` (`read_repository_access_preferences` / `persist_repository_access_preferences`, through `write_guarded` in `setup/config_file_write.rs`, which owns `ConfigLayer` and the setup config-file write guard: reread-compare, symlink refusal, then the shared temp-file-plus-rename helper in `setup/atomic_write.rs`), plus a read-only global reader used for the Claude telemetry removal check. `setup/command.rs` calls the writer after dispatch and before the target install, and re-resolves the policy for the `SCE access` report and for the setup login step (`setup_login_action` / `run_setup_login_step`, which reuses `auth_command::run_login_in` rather than a separate login path); precedence stays in `config/resolver.rs` (see `context/cli/access-policy.md#setup`). +- `cli/src/services/setup/schema_upgrade.rs` owns the `$schema`-only upgrade of one canonical config file (`ConfigLayer::Repository` or `ConfigLayer::Global`). `plan_schema_upgrade` is read-only. It classifies the declaration with `config/schema_declaration.rs`. For an older-series declaration, it builds a candidate that changes only `$schema`: it swaps the single URL literal when the result stays structurally identical, and otherwise falls back to pretty serialization. It then validates the candidate through `parse_file_config` with the layer's path source, yielding `Ready` or `CandidateInvalid`. `commit_schema_upgrade` writes only a `Ready` plan, through `write_guarded` against the planned original bytes. Every other plan becomes `Unchanged` or a typed `Skipped(SchemaUpgradeSkip)`. Newer-series, unrecognized, and invalid-candidate files are never rewritten, and a missing file is never created. `setup/command.rs` does not invoke it yet. - `cli/src/services/security.rs` provides shared security utilities for deterministic secret redaction (`redact_sensitive_text`) and directory write-permission probes (`ensure_directory_is_writable`) used by app/setup/observability surfaces. - `cli/src/services/doctor/mod.rs` owns the current doctor request/report surface while focused submodules (`doctor/inspect.rs`, `doctor/render.rs`, `doctor/fixes.rs`, `doctor/types.rs`) split report fact collection, rendering, manual fix reporting, and doctor-owned domain types into smaller seams; `cli/src/services/doctor/command.rs` owns the `DoctorCommand` payload used by the static `RuntimeCommand` enum and executes against any context implementing repo-root scoping. Runtime doctor execution resolves a repository root, derives a scoped context, requests the shared static lifecycle provider catalog with hooks included for service-owned `diagnose` and `fix` behavior, adapts lifecycle-owned health/fix records into doctor-owned problem/fix records, and then renders stable text/JSON problem records with category/severity/fixability/remediation fields plus deterministic fix-result reporting in fix mode. Agent Trace database inspection is no longer a doctor-adjacent command surface; doctor owns repository-scoped DB health and checkout identity facts, while `sce sync` owns control-plane synchronization. Report fact collection preserves environment/repository/hook/integration display data and adds a non-launching `post_commit_auto_sync` fact based on canonical post-commit managed-block currency plus resolved `agent_trace.auto_sync` source/value; this fact does not launch `sce sync`, and existing hook problem/remediation/readiness semantics remain authoritative. Service-owned lifecycle providers own config validation, local DB and repository-scoped Agent Trace DB readiness/bootstrap, and hook rollout diagnosis/repair. Integration inspection in `doctor/inspect.rs` is scoped twice over: `resolve_doctor_integration_targets` picks which targets to inspect, and `persisted_optional_workflows` (reused from setup) resolves which optional workflows the repository selected, which the OpenCode/Claude/Pi/Codex child collectors apply through `iter_embedded_assets_for_setup_target_with_selection`. An unselected optional workflow therefore contributes no expected children at all, so no row and no missing/mismatch problem can be produced for it, while a selected one keeps the unchanged presence and content-hash checks. Codex's collector (`collect_codex_integration_groups`) is the one exception to the single-target-directory shape the other three share: since `CODEX_EMBEDDED_ASSETS` relative paths already carry their own `.agents/`/`.codex/` prefix, it resolves its integration root through `InstallTargetPaths::codex_target_dir()` (the repository root itself) rather than a `RepoPaths` per-target subdirectory, and splits assets into `Skills` (`.agents/skills/` prefix) and `Hooks` (`.codex/` prefix) groups instead of Pi's `Extensions`/`Prompts`/`Skills` split. Its `.codex/hooks.json` reporting is per-registration rather than one whole-file child: `codex_hook_config::diagnose_document` classifies each required registration — the four `sce hooks codex` registrations plus the six `sce hooks codex-mutation-scope` ones, the latter reported under distinct `#(mutation-scope)` rows — as `PresentAndCurrent`/`Missing`/`Stale` (or the whole document as `Malformed` when it cannot be structurally validated) without writing anything, so unrelated user handlers never create a false whole-document mismatch. For a structurally current registration, `codex_hook_trust` separately reads (never writes) Codex's own durable `$CODEX_HOME/config.toml` hook-trust state — reproducing upstream's `hook_hash`/`hook_key`/`hook_trust_status` exactly — and reports `Trusted`/`Untrusted`/`Modified`/`Disabled`/`Unknown`, and `codex_hook_policy` probes Codex's effective hook-discovery policy once per invocation; only a structurally current registration that is trusted *and* policy-allowed renders healthy. `sce doctor --fix` repairs a structurally unhealthy `.codex/hooks.json` through the existing merge-install path, but a registration that is current yet not-yet-trusted or policy-blocked is never "fixed", since SCE cannot grant Codex hook trust or change managed policy. - `cli/src/services/version/mod.rs` defines the version command parser/rendering contract (`parse_version_request`, `render_version`) with deterministic text output and stable JSON runtime-identification fields; `cli/src/services/version/command.rs` owns the `VersionCommand` payload used by the static `RuntimeCommand` enum. diff --git a/context/plans/config-schema-series-compatibility.md b/context/plans/config-schema-series-compatibility.md index efd504070..676b58776 100644 --- a/context/plans/config-schema-series-compatibility.md +++ b/context/plans/config-schema-series-compatibility.md @@ -309,7 +309,7 @@ Persist this field in every plan; this is durable plan state, not chat state: - Context impact: localized. This is an internal refactor of setup's write helpers with no behavior change. - Context synchronization: synced -- [ ] T04: `Plan and commit a validated $schema-only upgrade for one config file` (status:todo) +- [x] T04: `Plan and commit a validated $schema-only upgrade for one config file` (status:done) - Task ID: T04 - Scope: - In: @@ -339,7 +339,43 @@ Persist this field in every plan; this is durable plan state, not chat state: - All protections hold. - Verify: - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema_upgrade` - - Context synchronization: pending + - Completed: 2026-11-10 + - Files changed: + - `cli/src/services/setup/schema_upgrade.rs` (new) + - `cli/src/services/setup/mod.rs` + - Result: + - New `schema_upgrade` module defines: + - `SchemaUpgradePlan { NotPresent, NoDeclaration, Current, NewerSeries(SchemaVersion), Unrecognized, Unreadable(anyhow::Error), Ready { original, candidate, from }, CandidateInvalid { from, reason } }`; + - `SchemaUpgradeOutcome { Upgraded { from }, Unchanged, Skipped(SchemaUpgradeSkip) }`; + - `SchemaUpgradeSkip { NewerSeries, Unrecognized, CandidateInvalid { from, reason }, SymlinkManaged, ConcurrentModification, Io }`, which converts from `GuardedWriteError`. + - `plan_schema_upgrade(path, layer)` reads the file with `read_optional`. It classifies `$schema` with `classify_schema_declaration` against `current_schema_series()`. For an older series, it builds the candidate and validates it with `parse_file_config` and `layer.path_source()`. It never writes. + - Building the candidate: the JSON-encoded old URL literal is swapped for the current URL only when it occurs exactly once and the reparsed result equals the original with only `$schema` replaced. Otherwise the candidate falls back to `to_string_pretty` plus a trailing newline. + - `commit_schema_upgrade(path, layer, plan)` writes only for `Ready`, through `write_guarded` against the planned original bytes. `NotPresent`, `NoDeclaration`, and `Current` map to `Unchanged`. The other plans map to `Skipped`. + - Approved assumptions: + - `Unreadable` (a read failure, non-UTF-8, non-JSON, or a non-object) maps to `Skipped(Io)` rather than adding a new skip variant. + - A dangling symlink reads as `NotPresent`. It yields `Unchanged`, and the link is left intact; there is nothing to upgrade. + - `mod schema_upgrade` carries `#[expect(dead_code, reason = …)]` until T05/T06 consume the planner and its skip details. Because it is an `expect`, the build fails once the module is fully used, which forces its removal. + - 13 unit tests cover: + - a missing file and no declaration; + - same-series prerelease, patch, and build values left unchanged; + - older-series `0.3.3`/`0.1.0` upgrades that change only the `$schema` bytes, using a fixture with `auth.mode`, `telemetry.enabled`, `integrations.target`, and custom `policies.bash` and irregular whitespace; + - a second plan returning `Current` and leaving the file byte-identical; + - the ambiguous-literal fallback; + - newer-series, foreign, unversioned, `v0.4`, `http://`, and non-string values left untouched; + - unreadable files; + - an unknown key yielding `CandidateInvalid` with the file byte-identical; + - the global layer validated as `DefaultDiscoveredGlobal`; + - concurrent modification between plan and commit; + - symlink and dangling-symlink handling; + - an atomic write that keeps the file mode. + - Verify: + - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema_upgrade`: passed (13 passed, 0 failed). + - Supplementary checks: + - `… test … setup`: passed (94 passed, 0 failed), including `access_preferences` and `config_file_write`. + - `clippy --all-targets -- -D warnings`: passed. + - `cargo fmt --check`: passed. + - Context impact: localized. This adds a new internal setup module that is not yet wired into the `sce setup` flow, so user-visible behavior is unchanged until T05 and T06. + - Context synchronization: synced - [ ] T05: `Run schema upgrades before setup's preflight readers` (status:todo) - Task ID: T05 From 8947c94a09feeec4699e193d6eae19a84a9c52c3 Mon Sep 17 00:00:00 2001 From: David Abram Date: Tue, 10 Nov 2026 13:36:59 +0100 Subject: [PATCH 6/9] setup: Upgrade canonical config schemas before setup preflight Plan read-only upgrades for the canonical repository and global config files after the Git gate, overlay validated candidates while resolving the named remote, and commit upgrades only after that preflight succeeds. Keep context-only and explicit-config behavior unchanged while exposing per-path upgrade outcomes for setup. Update the setup architecture and repository-config contract, and record completion of T05 in the config-schema-series-compatibility plan. Plan: config-schema-series-compatibility.md (T05) Co-authored-by: SCE --- cli/src/services/config/resolver.rs | 2 +- cli/src/services/setup/command.rs | 319 +++++++++++++++++- cli/src/services/setup/schema_upgrade.rs | 58 +++- context/architecture.md | 2 +- context/context-map.md | 2 +- .../config-schema-series-compatibility.md | 38 ++- .../sce/setup-repo-local-config-bootstrap.md | 7 +- 7 files changed, 413 insertions(+), 15 deletions(-) diff --git a/cli/src/services/config/resolver.rs b/cli/src/services/config/resolver.rs index 5564b4b8e..148ee893a 100644 --- a/cli/src/services/config/resolver.rs +++ b/cli/src/services/config/resolver.rs @@ -209,7 +209,7 @@ pub(crate) fn resolve_agent_trace_storage_runtime_config( ) } -fn resolve_agent_trace_storage_runtime_config_with( +pub(crate) fn resolve_agent_trace_storage_runtime_config_with( cwd: &Path, env_lookup: FEnv, read_file: FRead, diff --git a/cli/src/services/setup/command.rs b/cli/src/services/setup/command.rs index 1d4d3b456..2018def00 100644 --- a/cli/src/services/setup/command.rs +++ b/cli/src/services/setup/command.rs @@ -17,6 +17,7 @@ use crate::services::setup::access_preferences::{ RepositoryAccessPreferencesError, }; use crate::services::setup::claude_telemetry::CLAUDE_TELEMETRY_DISABLED_BY_CONFIG_LINE; +use crate::services::setup::schema_upgrade::{CanonicalSchemaUpgrades, SchemaUpgradeReport}; use crate::services::{config, managed_telemetry, setup}; const SCE_ACCESS_HEADING: &str = "SCE access:"; @@ -50,7 +51,8 @@ impl SetupCommand { .map_err(unexpected_failure)?, }; - let repository_root = resolve_setup_repository(&setup_start_path)?; + let repository = resolve_setup_repository(&setup_start_path, self.request.context_only)?; + let repository_root = repository.root; let (setup_dispatch, interactive_access) = match self.prepare_dispatch(&repository_root)? { PreparedSetup::Cancelled => return Ok(setup::setup_cancelled_text()), @@ -730,7 +732,40 @@ where format_claude_telemetry_report(&installed, authorization) } -fn resolve_setup_repository(start_path: &std::path::Path) -> Result { +struct SetupRepository { + root: PathBuf, + #[cfg_attr( + not(test), + expect( + dead_code, + reason = "the setup report renders the schema upgrade outcomes" + ) + )] + schema_upgrades: Vec, +} + +fn resolve_setup_repository( + start_path: &Path, + context_only: bool, +) -> Result { + resolve_setup_repository_with( + start_path, + context_only, + |key| std::env::var(key).ok(), + config::resolver::resolve_default_global_config_path, + ) +} + +fn resolve_setup_repository_with( + start_path: &Path, + context_only: bool, + env_lookup: FEnv, + resolve_global_config_path: FGlobalPath, +) -> Result +where + FEnv: Fn(&str) -> Option, + FGlobalPath: Fn() -> anyhow::Result, +{ let repository_root = setup::ensure_git_repository(start_path).map_err(|source| { let user_error = match source { setup::GitRepositoryResolutionError::NotGitRepository(_) => UserError::NotGitRepository, @@ -740,8 +775,27 @@ fn resolve_setup_repository(start_path: &std::path::Path) -> Result String::from_utf8(candidate.to_vec()).map_err(anyhow::Error::from), + None => std::fs::read_to_string(path) + .with_context(|| format!("Failed to read config file '{}'.", path.display())), + }, + Path::exists, + resolve_global_config_path, + ) + .map_err(unexpected_failure)?; setup::ensure_git_remote(&repository_root, &storage_config.repository_remote).map_err( |source| { @@ -753,7 +807,10 @@ fn resolve_setup_repository(start_path: &std::path::Path) -> Result) -> CliError { @@ -1682,6 +1739,258 @@ mod tests { ); } + fn git(repo: &Path, args: &[&str]) { + let output = std::process::Command::new("git") + .args(args) + .current_dir(repo) + .output() + .expect("git should run"); + assert!(output.status.success(), "git {args:?} failed"); + } + + fn schema_url(version: &str) -> String { + format!("https://sce.crocoder.dev/v{version}/config.json") + } + + fn older_series_url() -> String { + let config::schema_declaration::SchemaSeries { major, minor } = + config::schema_declaration::current_schema_series(); + if minor > 0 { + schema_url(&format!("{major}.{}.3", minor - 1)) + } else { + schema_url(&format!("{}.9.3", major - 1)) + } + } + + fn stale(body: &str) -> String { + format!( + "{{\n \"$schema\": \"{}\",\n{body}\n}}\n", + older_series_url() + ) + } + + fn upgraded(stale: &str) -> String { + stale.replace( + &older_series_url(), + &crate::services::agent_trace::sce_config_schema_url(), + ) + } + + impl AccessFixture { + fn git_repository(remotes: &[&str]) -> Self { + let fixture = Self::new(); + fs::remove_dir(fixture.repo.join(".git")).unwrap(); + git(&fixture.repo, &["init", "-q"]); + for remote in remotes { + git( + &fixture.repo, + &[ + "remote", + "add", + remote, + &format!("https://example.com/{remote}.git"), + ], + ); + } + fixture + } + + fn resolve_repository(&self, context_only: bool) -> Result { + resolve_setup_repository_with( + &self.repo, + context_only, + |_| None, + || Ok(self.global.clone()), + ) + } + + fn snapshot(&self) -> (Option>, Option>) { + ( + fs::read(self.repo_config_path()).ok(), + fs::read(&self.global).ok(), + ) + } + } + + fn upgraded_paths(repository: &SetupRepository) -> Vec { + repository + .schema_upgrades + .iter() + .filter(|report| { + matches!( + report.outcome, + setup::schema_upgrade::SchemaUpgradeOutcome::Upgraded { .. } + ) + }) + .map(|report| report.path.clone()) + .collect() + } + + #[test] + fn setup_preflight_checks_remote_from_stale_local_config_and_upgrades_it() { + let fixture = AccessFixture::git_repository(&["upstream"]); + let local = stale(r#" "agent_trace": { "repository_remote": "upstream" }"#); + fixture.write_repo_config(&local); + + let repository = fixture.resolve_repository(false).unwrap(); + + assert_eq!( + fs::read_to_string(fixture.repo_config_path()).unwrap(), + upgraded(&local) + ); + assert_eq!(upgraded_paths(&repository), [fixture.repo_config_path()]); + assert!(!fixture.global.exists()); + } + + #[test] + fn setup_failed_remote_preflight_writes_no_schema_upgrade() { + let fixture = AccessFixture::git_repository(&["upstream"]); + fixture.write_repo_config(&stale( + r#" "agent_trace": { "repository_remote": "missing" }"#, + )); + fs::write( + &fixture.global, + stale(r#" "auth": { "mode": "optional" }"#), + ) + .unwrap(); + let before = fixture.snapshot(); + + let Err(error) = fixture.resolve_repository(false) else { + panic!("preflight should fail for a missing remote"); + }; + + assert!( + matches!( + error, + CliError::User { + error: UserError::NotGitRemote, + .. + } + ), + "{error:?}" + ); + assert_eq!(fixture.snapshot(), before); + } + + #[test] + fn setup_failed_git_gate_writes_no_schema_upgrade() { + let fixture = AccessFixture::new(); + fs::remove_dir(fixture.repo.join(".git")).unwrap(); + fs::write( + &fixture.global, + stale(r#" "auth": { "mode": "optional" }"#), + ) + .unwrap(); + let before = fixture.snapshot(); + + assert!(fixture.resolve_repository(false).is_err()); + assert_eq!(fixture.snapshot(), before); + } + + #[test] + fn setup_access_gate_locks_on_stale_global_disabled_after_upgrade() { + let fixture = AccessFixture::git_repository(&["origin"]); + let global = stale(r#" "auth": { "mode": "disabled" }"#); + fs::write(&fixture.global, &global).unwrap(); + assert_eq!( + SetupAccessGate::from_policy(&fixture.policy(), false).auth_locked_by, + None + ); + + let repository = fixture.resolve_repository(false).unwrap(); + + assert_eq!( + upgraded_paths(&repository), + std::slice::from_ref(&fixture.global) + ); + assert_eq!( + fs::read_to_string(&fixture.global).unwrap(), + upgraded(&global) + ); + assert_eq!( + SetupAccessGate::from_policy(&fixture.policy(), false).auth_locked_by, + Some(AuthPolicyConfigFile::Global) + ); + let prompter = FakePrompter::answering(None, false); + fixture.prepare(&prompter, false).unwrap(); + assert!(prompter.asked().is_empty(), "{:?}", prompter.asked()); + } + + #[test] + fn setup_dispatch_uses_saved_answers_from_upgraded_local_config() { + let fixture = AccessFixture::git_repository(&["origin"]); + fixture.write_repo_config(&stale( + r#" "auth": { "mode": "optional" }, + "telemetry": { "enabled": false }, + "integrations": { "target": ["claude"], "optional_workflows": ["brownfield"] }"#, + )); + assert!(setup::persisted_integration_targets(&fixture.repo).is_empty()); + + fixture.resolve_repository(false).unwrap(); + + assert_eq!( + setup::persisted_integration_targets(&fixture.repo), + [ConcreteSetupTarget::Claude] + ); + assert_eq!( + setup::persisted_optional_workflows(&fixture.repo), + ["brownfield"] + ); + let prompter = FakePrompter::answering(None, false); + let access_save = ready_access_save(fixture.prepare(&prompter, false).unwrap()); + assert_eq!(prompter.asked(), ["login"]); + assert_eq!(access_save.outcome, AccessWriteOutcome::Unchanged); + } + + #[test] + fn setup_context_only_plans_no_schema_upgrade() { + let fixture = AccessFixture::git_repository(&["origin"]); + fixture.write_repo_config(&stale(r#" "auth": { "mode": "optional" }"#)); + fs::write( + &fixture.global, + stale(r#" "auth": { "mode": "optional" }"#), + ) + .unwrap(); + let before = fixture.snapshot(); + + let repository = fixture.resolve_repository(true).unwrap(); + + assert!(repository.schema_upgrades.is_empty()); + assert_eq!(fixture.snapshot(), before); + } + + #[test] + fn setup_upgrades_valid_global_when_stale_local_candidate_is_invalid() { + let fixture = AccessFixture::git_repository(&["origin"]); + let local = stale(r#" "unknown_key": true"#); + fixture.write_repo_config(&local); + let global = stale(r#" "auth": { "mode": "optional" }"#); + fs::write(&fixture.global, &global).unwrap(); + + let repository = fixture.resolve_repository(false).unwrap(); + + assert_eq!( + upgraded_paths(&repository), + std::slice::from_ref(&fixture.global) + ); + assert_eq!( + fs::read_to_string(&fixture.global).unwrap(), + upgraded(&global) + ); + assert_eq!( + fs::read_to_string(fixture.repo_config_path()).unwrap(), + local + ); + let prompter = FakePrompter::answering(None, false); + let access_save = ready_access_save(fixture.prepare(&prompter, false).unwrap()); + assert_eq!(prompter.asked(), ["login"]); + assert_eq!(access_save.outcome, AccessWriteOutcome::Unchanged); + assert_eq!( + fs::read_to_string(fixture.repo_config_path()).unwrap(), + local + ); + } + fn walk_files(root: &Path) -> Vec { let mut files = Vec::new(); for entry in fs::read_dir(root).unwrap() { diff --git a/cli/src/services/setup/schema_upgrade.rs b/cli/src/services/setup/schema_upgrade.rs index 18be6fe5f..12e9f4a6e 100644 --- a/cli/src/services/setup/schema_upgrade.rs +++ b/cli/src/services/setup/schema_upgrade.rs @@ -1,4 +1,4 @@ -use std::path::Path; +use std::path::{Path, PathBuf}; use anyhow::{anyhow, Context}; use serde_json::Value; @@ -139,6 +139,61 @@ where SchemaUpgradeOutcome::Skipped(skip) } +#[derive(Debug)] +pub(crate) struct SchemaUpgradeReport { + pub(crate) path: PathBuf, + pub(crate) outcome: SchemaUpgradeOutcome, +} + +#[derive(Debug)] +struct PlannedSchemaUpgrade { + path: PathBuf, + layer: ConfigLayer, + plan: SchemaUpgradePlan, +} + +#[derive(Debug, Default)] +pub(crate) struct CanonicalSchemaUpgrades { + files: Vec, +} + +impl CanonicalSchemaUpgrades { + pub(crate) fn plan(repository_config_path: &Path, global_config_path: Option<&Path>) -> Self { + let files = [ + Some((repository_config_path, ConfigLayer::Repository)), + global_config_path.map(|path| (path, ConfigLayer::Global)), + ] + .into_iter() + .flatten() + .map(|(path, layer)| PlannedSchemaUpgrade { + path: path.to_path_buf(), + layer, + plan: plan_schema_upgrade(path, layer), + }) + .collect(); + Self { files } + } + + pub(crate) fn candidate(&self, path: &Path) -> Option<&[u8]> { + self.files.iter().find_map(|file| match &file.plan { + SchemaUpgradePlan::Ready { candidate, .. } if file.path == path => { + Some(candidate.as_slice()) + } + _ => None, + }) + } + + pub(crate) fn commit(self) -> Vec { + self.files + .into_iter() + .map(|file| SchemaUpgradeReport { + outcome: commit_schema_upgrade(&file.path, file.layer, file.plan), + path: file.path, + }) + .collect() + } +} + struct ConfigDocument<'a> { raw: &'a str, value: Value, @@ -195,7 +250,6 @@ fn build_candidate(document: &ConfigDocument<'_>, target_url: &str) -> anyhow::R #[cfg(test)] mod tests { use std::fs; - use std::path::PathBuf; use serde_json::json; diff --git a/context/architecture.md b/context/architecture.md index 7f20fe97c..edf30ea47 100644 --- a/context/architecture.md +++ b/context/architecture.md @@ -72,7 +72,7 @@ The repository includes a new placeholder Rust binary crate at `cli/`. - `cli/src/services/setup/mod.rs` defines the setup command contract (`SetupMode`, `SetupTarget`, `SetupRequest`, CLI flag parser/validator), an `inquire`-backed interactive target prompter (`InquireSetupTargetPrompter`), setup dispatch outcomes (proceed/cancelled), additive durable-context bootstrap (`bootstrap_context_baseline` for standalone `--bootstrap-context` and every normal successful setup path), and compile-time embedded asset access (`EmbeddedAsset`, target-scoped iterators, required-hook asset iterators/lookups). It also owns the install-time optional-workflow seam: the `OptionalWorkflow` type plus the build-generated `OPTIONAL_WORKFLOWS` catalog, a per-target `WorkflowAssetLayout` built from the existing `default_paths` command/skill directory constants (`command`/`commands`/`prompts` plus `skills`), and `iter_embedded_assets_for_setup_target_with_selection(target, selection)`, which yields every embedded asset except the `{command_dir}/{command_slug}.md` file and `{skills_dir}/{skill_slug}/` subtree of each optional workflow the selection omits. Membership is derived from the catalog's slugs rather than an enumerated file list, so a new optional workflow needs no Rust change. This filtered iterator is the only way embedded assets are enumerated; setup installs through it and doctor inspects through it, so there is no unfiltered enumeration path that could reintroduce an unselected workflow. The non-interactive selection flows through the repeatable `--workflow ` flag into `SetupRequest.optional_workflows: Option>` (`None` meaning the flag was absent); `validate_optional_workflow_slugs` checks each slug against `OPTIONAL_WORKFLOWS` during request resolution, before any write, and `run_setup_for_mode` resolves `None` to the persisted `integrations.optional_workflows` (exported as `persisted_optional_workflows`) before installing through the filtered iterator and persisting the resolved selection. The interactive selection flows through the same seam: `SetupTargetPrompter` carries `prompt_targets(defaults)`, `prompt_optional_workflows(defaults)`, the two behavior confirmations `prompt_agent_trace_auto_sync(default)` / `prompt_attribution_hooks_enabled(default)` (dispatch passes `true`), and the access prompts `prompt_auth_mode(default)` (an `inquire::Select` with fixed Optional/Mandatory/Disabled rows), `prompt_telemetry_consent(default)`, and `prompt_login()` (Yes default), which dispatch does not call yet; every method returns `None` for a cancelled prompt and fails with the fixed TTY guidance on `NotTTY`, `SetupDispatch::Proceed { mode, optional_workflows }` carries a prompted selection alongside the resolved mode, and `resolve_setup_dispatch(mode, prompter, defaults)` runs the workflow prompt only after an interactive target prompt, mapping either cancellation to `SetupDispatch::Cancelled`. The prompt module builds its `inquire::MultiSelect` from `optional_workflow_prompt_inputs(catalog, defaults)`, which returns `None` for an empty catalog (skipping the prompt) and otherwise catalog-ordered rows plus the indices to pre-check, ignoring ids absent from the catalog. `setup/command.rs` therefore resolves the repository root before dispatch, mapping the setup domain's typed positive non-Git classification to `setup.not_git_repository` and other resolution failures to `general.unexpected_failure`, so a non-Git directory fails before any prompt without CLI-layer string matching. For repository builds, `cli/build.rs` validates the `SCE_CLI_GENERATED_INPUT_DIR` payload and canonical-input inventories, copies the payload into Cargo `OUT_DIR/pkl-generated`, stages `cli/assets/hooks/**` under `OUT_DIR/static`, requires the staged `config/optional-workflows.json`, and generates both the setup manifest and the optional-workflow catalog (`optional_workflows.rs`, rejecting a manifest whose `schemaVersion` is not 1 or whose entries lack a non-empty `id`/`title`/`description`/`commandSlug`/`skillSlug`) in `OUT_DIR`; focused internal seams separate install-flow from prompt-flow logic; `cli/src/services/setup/command.rs` owns the `SetupCommand` payload used by the static `RuntimeCommand` enum and executes against any context implementing repo-root scoping. Its install engine/orchestrator (`install_assets_for_concrete_target_with_rename`) installs each embedded asset individually: it stages the asset's content next to its final destination and swaps it into place by renaming the staging file directly over the destination — never unlinking the destination first, since `fs::rename` already replaces an existing file atomically — with deterministic recovery guidance naming the failing asset's path on swap failure (the pre-existing destination content, if any, is untouched) and no backup artifact creation; it never removes an integration target directory as a whole, so files a repository owns inside `.opencode`/`.claude`/`.pi` — including nested inside an SCE-owned subdirectory such as `skills/` or `commands/` — survive a setup run; Codex has no single target directory this way, since `CODEX_EMBEDDED_ASSETS` relative paths already carry their own `.agents/`/`.codex/` prefix and `InstallTargetPaths::codex_target_dir()` resolves to the repository root itself, but the same per-asset stage/atomic-swap and never-delete-what-it-did-not-author guarantees still apply file by file. For the three assets that are merge targets — the Claude target's `settings.json`, the OpenCode target's `opencode.json`, and Codex's user-owned `.codex/hooks.json` — the content staged is not always the embedded asset's bytes. Claude and OpenCode use `cli/src/services/setup/config_merge.rs`, while Codex uses the shared `cli/src/services/codex_hook_config.rs` service for structural validation and canonical-registration merging. For the Claude and OpenCode targets, the content staged is the result of `cli/src/services/setup/config_merge.rs::merge_or_create_claude_settings`/`merge_or_create_opencode_config(existing_bytes, generated_bytes, source_path)`: each returns the generated document verbatim when no file exists yet, otherwise parses the existing file as JSON (a parse failure is a hard error naming `source_path`, and nothing is written) and merges it with a pure per-shape function that copies `$schema` from the generated document and preserves every other key from the existing file untouched. `merge_claude_settings` replaces, per hook event key the generated document declares, only the entries whose command contains the ownership marker `run-sce-or-show-install-guidance.sh`, preserving every event key and non-SCE hook entry from the existing file. `merge_opencode_config` merges the `plugin` array as a set: existing entries whose path starts with the ownership marker `./plugins/sce-` are dropped structurally — so a stale plugin path an older or renamed catalog once installed is still recognized and pruned even after the current generated document stops declaring it — and the generated document's `plugin` entries are appended after the surviving entries. Codex's merge validates the document shape and recognizes ownership by the generated helper path plus one of two trailing command contracts — `sce hooks codex` or `sce hooks codex-mutation-scope` — attributing each handler to whichever it matches; it replaces stale or duplicate handlers with exactly one current handler per registration (the four `sce hooks codex` registrations plus six additive `sce hooks codex-mutation-scope` registrations, within that mutation-scope set, `PreToolUse` and `PostToolUse` use matcher `^(Bash|apply_patch)$`, while `Stop`, `Interrupt`, `SubagentStop`, and `SessionEnd` are appended in unmatched groups after the existing groups so an already-trusted handler keeps its `(event, matcher, group index, handler index)` identity and computed Codex trust key), touching only the matching command's handlers and preserving unrelated valid Codex fields, groups, and handlers. After that per-asset install loop, `prune_stale_assets_for_concrete_target` deletes every path the full embedded catalog for the concrete target claims but the resolved selection did not install (a deselected optional workflow, or an asset a newer catalog renamed or dropped), and `remove_empty_ancestor_directories` removes any parent directory left empty by that deletion, stopping at the target root or at a directory that still holds something such as a user file. It formats deterministic completion messaging; required-hook install orchestration (`install_required_git_hooks`, backed by the rename-injectable `install_required_git_hooks_with_rename`) is a third content-computation seam alongside the two JSON merge targets: `install_single_required_hook_with_rename` computes the bytes to stage with `cli/src/services/setup/hook_merge.rs::merge_or_create_hook(existing_bytes, canonical_bytes, hook_name)` rather than writing `hook_asset.bytes` verbatim — a foreign hook (no SCE managed block, no legacy guidance-URL marker) is kept as an exact byte prefix with the canonical block appended after it, an SCE-owned hook has only its block spliced in place or left unchanged, and a legacy pre-marker hook is replaced wholesale — then follows the same per-file stage/atomic-swap choreography as config-asset install: the staging file is renamed directly over the existing hook without unlinking it first, so a rename failure leaves the prior hook's bytes and executable bit intact, with deterministic recovery guidance on swap failure. `Installed`/`Updated`/`Skipped` are decided against the merged bytes plus the executable bit rather than the canonical asset's raw bytes, so an already-current foreign-plus-block hook reports `Skipped`; `RequiredHookInstallResult.unreachable_block_advisory` is set, and rendered as a named advisory line in setup's hook output, when an appended block follows a foreign hook's zero-indent `exec`/`exit` and so would never run. After the Git gate, setup always ensures the context baseline; context-only requests return there, while normal modes derive a repo-root-scoped context before aggregating static lifecycle provider `setup` dispatch across providers (config → local_db → auth_db → agent_trace_db → hooks when requested), so setup providers consume only repo-root access from the scoped context. - `cli/src/services/setup/mod.rs` keeps those responsibilities inside one file for now, but the current ownership split is explicit: the inline `install` module owns repository-path normalization, staging/swap install behavior, required-hook installation, and filesystem safety guards, while the inline `prompt` module owns interactive target selection and prompt styling. - `cli/src/services/setup/access_preferences.rs` owns setup's access preferences (`auth.mode`, `telemetry.enabled`): the repository-layer reader and atomic writer for `.sce/config.json` (`read_repository_access_preferences` / `persist_repository_access_preferences`, through `write_guarded` in `setup/config_file_write.rs`, which owns `ConfigLayer` and the setup config-file write guard: reread-compare, symlink refusal, then the shared temp-file-plus-rename helper in `setup/atomic_write.rs`), plus a read-only global reader used for the Claude telemetry removal check. `setup/command.rs` calls the writer after dispatch and before the target install, and re-resolves the policy for the `SCE access` report and for the setup login step (`setup_login_action` / `run_setup_login_step`, which reuses `auth_command::run_login_in` rather than a separate login path); precedence stays in `config/resolver.rs` (see `context/cli/access-policy.md#setup`). -- `cli/src/services/setup/schema_upgrade.rs` owns the `$schema`-only upgrade of one canonical config file (`ConfigLayer::Repository` or `ConfigLayer::Global`). `plan_schema_upgrade` is read-only. It classifies the declaration with `config/schema_declaration.rs`. For an older-series declaration, it builds a candidate that changes only `$schema`: it swaps the single URL literal when the result stays structurally identical, and otherwise falls back to pretty serialization. It then validates the candidate through `parse_file_config` with the layer's path source, yielding `Ready` or `CandidateInvalid`. `commit_schema_upgrade` writes only a `Ready` plan, through `write_guarded` against the planned original bytes. Every other plan becomes `Unchanged` or a typed `Skipped(SchemaUpgradeSkip)`. Newer-series, unrecognized, and invalid-candidate files are never rewritten, and a missing file is never created. `setup/command.rs` does not invoke it yet. +- `cli/src/services/setup/schema_upgrade.rs` owns the `$schema`-only upgrade of one canonical config file (`ConfigLayer::Repository` or `ConfigLayer::Global`). `plan_schema_upgrade` is read-only. It classifies the declaration with `config/schema_declaration.rs`. For an older-series declaration, it builds a candidate that changes only `$schema`: it swaps the single URL literal when the result stays structurally identical, and otherwise falls back to pretty serialization. It then validates the candidate through `parse_file_config` with the layer's path source, yielding `Ready` or `CandidateInvalid`. `commit_schema_upgrade` writes only a `Ready` plan, through `write_guarded` against the planned original bytes. Every other plan becomes `Unchanged` or a typed `Skipped(SchemaUpgradeSkip)`. Newer-series, unrecognized, and invalid-candidate files are never rewritten, and a missing file is never created. `CanonicalSchemaUpgrades` plans the canonical repository and global files, exposes `Ready` candidates as a read overlay for the setup remote preflight, and commits them into per-path `SchemaUpgradeReport`s; `setup/command.rs` calls it between the Git-repo gate and the named-remote check and commits after the check passes. - `cli/src/services/security.rs` provides shared security utilities for deterministic secret redaction (`redact_sensitive_text`) and directory write-permission probes (`ensure_directory_is_writable`) used by app/setup/observability surfaces. - `cli/src/services/doctor/mod.rs` owns the current doctor request/report surface while focused submodules (`doctor/inspect.rs`, `doctor/render.rs`, `doctor/fixes.rs`, `doctor/types.rs`) split report fact collection, rendering, manual fix reporting, and doctor-owned domain types into smaller seams; `cli/src/services/doctor/command.rs` owns the `DoctorCommand` payload used by the static `RuntimeCommand` enum and executes against any context implementing repo-root scoping. Runtime doctor execution resolves a repository root, derives a scoped context, requests the shared static lifecycle provider catalog with hooks included for service-owned `diagnose` and `fix` behavior, adapts lifecycle-owned health/fix records into doctor-owned problem/fix records, and then renders stable text/JSON problem records with category/severity/fixability/remediation fields plus deterministic fix-result reporting in fix mode. Agent Trace database inspection is no longer a doctor-adjacent command surface; doctor owns repository-scoped DB health and checkout identity facts, while `sce sync` owns control-plane synchronization. Report fact collection preserves environment/repository/hook/integration display data and adds a non-launching `post_commit_auto_sync` fact based on canonical post-commit managed-block currency plus resolved `agent_trace.auto_sync` source/value; this fact does not launch `sce sync`, and existing hook problem/remediation/readiness semantics remain authoritative. Service-owned lifecycle providers own config validation, local DB and repository-scoped Agent Trace DB readiness/bootstrap, and hook rollout diagnosis/repair. Integration inspection in `doctor/inspect.rs` is scoped twice over: `resolve_doctor_integration_targets` picks which targets to inspect, and `persisted_optional_workflows` (reused from setup) resolves which optional workflows the repository selected, which the OpenCode/Claude/Pi/Codex child collectors apply through `iter_embedded_assets_for_setup_target_with_selection`. An unselected optional workflow therefore contributes no expected children at all, so no row and no missing/mismatch problem can be produced for it, while a selected one keeps the unchanged presence and content-hash checks. Codex's collector (`collect_codex_integration_groups`) is the one exception to the single-target-directory shape the other three share: since `CODEX_EMBEDDED_ASSETS` relative paths already carry their own `.agents/`/`.codex/` prefix, it resolves its integration root through `InstallTargetPaths::codex_target_dir()` (the repository root itself) rather than a `RepoPaths` per-target subdirectory, and splits assets into `Skills` (`.agents/skills/` prefix) and `Hooks` (`.codex/` prefix) groups instead of Pi's `Extensions`/`Prompts`/`Skills` split. Its `.codex/hooks.json` reporting is per-registration rather than one whole-file child: `codex_hook_config::diagnose_document` classifies each required registration — the four `sce hooks codex` registrations plus the six `sce hooks codex-mutation-scope` ones, the latter reported under distinct `#(mutation-scope)` rows — as `PresentAndCurrent`/`Missing`/`Stale` (or the whole document as `Malformed` when it cannot be structurally validated) without writing anything, so unrelated user handlers never create a false whole-document mismatch. For a structurally current registration, `codex_hook_trust` separately reads (never writes) Codex's own durable `$CODEX_HOME/config.toml` hook-trust state — reproducing upstream's `hook_hash`/`hook_key`/`hook_trust_status` exactly — and reports `Trusted`/`Untrusted`/`Modified`/`Disabled`/`Unknown`, and `codex_hook_policy` probes Codex's effective hook-discovery policy once per invocation; only a structurally current registration that is trusted *and* policy-allowed renders healthy. `sce doctor --fix` repairs a structurally unhealthy `.codex/hooks.json` through the existing merge-install path, but a registration that is current yet not-yet-trusted or policy-blocked is never "fixed", since SCE cannot grant Codex hook trust or change managed policy. - `cli/src/services/version/mod.rs` defines the version command parser/rendering contract (`parse_version_request`, `render_version`) with deterministic text output and stable JSON runtime-identification fields; `cli/src/services/version/command.rs` owns the `VersionCommand` payload used by the static `RuntimeCommand` enum. diff --git a/context/context-map.md b/context/context-map.md index d189c7c3c..24f7e368c 100644 --- a/context/context-map.md +++ b/context/context-map.md @@ -87,7 +87,7 @@ Feature/domain context: - `context/sce/setup-githooks-hook-asset-packaging.md` (compile-time `sce setup --hooks` required-hook template packaging contract, including all-hook non-blocking missing-`sce` install guidance, available-CLI argument forwarding, post-commit-only origin remote lookup plus remote-URL forwarding/fallback behavior, setup-service accessor surface, and current validation posture) - `context/sce/setup-githooks-install-flow.md` (setup-service required-hook install orchestration with git-truth hooks-path resolution, managed-block merge content computation that preserves foreign hook content, per-hook installed/updated/skipped outcomes decided against the merged content, the unreachable-block advisory, and atomic-swap replacement with recovery guidance) - `context/sce/setup-githooks-cli-ux.md` (T04 composable `sce setup` target+`--hooks` / `--repo` command-surface contract, option compatibility validation, and deterministic setup/hook output semantics) -- `context/sce/setup-repo-local-config-bootstrap.md` (setup local bootstrap behavior: Git-root and named-remote preflights before prompts/context/lifecycle/hooks/assets, shared degraded handling for invalid default-discovered repo-local config with byte-preserving skip of integration persistence, explicit-config fatality, additive durable-context baseline via `sce setup --bootstrap-context` and every normal setup path, repo-local `.sce/config.json` create-if-missing via config lifecycle, additive `integrations.target` persistence after successful target installs, `integrations.optional_workflows` selection persistence with prompt-over-flag-over-persisted precedence, plus lifecycle-owned local DB initialization before hooks/config asset dispatch, and interactive global-only `auth.mode` / `telemetry.enabled` persistence before any repository write with the re-resolved `SCE access` report, the interactive setup login step decided by the effective auth mode with `auth.setup_login_required` on mandatory failure, and interactive-only `sce setup --reconfigure` re-asking every saved answer with the saved value as the default) +- `context/sce/setup-repo-local-config-bootstrap.md` (setup local bootstrap behavior: Git-root and named-remote preflights before prompts/context/lifecycle/hooks/assets, canonical `$schema` upgrades planned before the named-remote check and committed after it, shared degraded handling for invalid default-discovered repo-local config with byte-preserving skip of integration persistence, explicit-config fatality, additive durable-context baseline via `sce setup --bootstrap-context` and every normal setup path, repo-local `.sce/config.json` create-if-missing via config lifecycle, additive `integrations.target` persistence after successful target installs, `integrations.optional_workflows` selection persistence with prompt-over-flag-over-persisted precedence, plus lifecycle-owned local DB initialization before hooks/config asset dispatch, and interactive global-only `auth.mode` / `telemetry.enabled` persistence before any repository write with the re-resolved `SCE access` report, the interactive setup login step decided by the effective auth mode with `auth.setup_login_required` on mandatory failure, and interactive-only `sce setup --reconfigure` re-asking every saved answer with the saved value as the default) - `context/sce/cli-security-hardening-contract.md` (T06 CLI redaction contract, setup `--repo` canonicalization/validation, and setup write-permission probe behavior) - `context/sce/agent-trace-post-rewrite-local-remap-ingestion.md` (current post-rewrite no-op baseline plus historical remap-ingestion reference) - `context/sce/agent-trace-rewrite-trace-transformation.md` (current post-rewrite no-op baseline plus historical rewrite-transformation reference) diff --git a/context/plans/config-schema-series-compatibility.md b/context/plans/config-schema-series-compatibility.md index 676b58776..7c281c80f 100644 --- a/context/plans/config-schema-series-compatibility.md +++ b/context/plans/config-schema-series-compatibility.md @@ -377,7 +377,7 @@ Persist this field in every plan; this is durable plan state, not chat state: - Context impact: localized. This adds a new internal setup module that is not yet wired into the `sce setup` flow, so user-visible behavior is unchanged until T05 and T06. - Context synchronization: synced -- [ ] T05: `Run schema upgrades before setup's preflight readers` (status:todo) +- [x] T05: `Run schema upgrades before setup's preflight readers` (status:done) - Task ID: T05 - Scope: - In: @@ -399,7 +399,41 @@ Persist this field in every plan; this is durable plan state, not chat state: - Done when: AC8 holds and the existing `setup` tests pass. - Verify: - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml setup` - - Context synchronization: pending + - Completed: 2026-11-10 + - Files changed: + - `cli/src/services/setup/command.rs` + - `cli/src/services/setup/schema_upgrade.rs` + - `cli/src/services/config/resolver.rs` + - Result: + - `schema_upgrade.rs` adds: + - `CanonicalSchemaUpgrades`, which plans the repository file and, when its path resolves, the global file through `plan_schema_upgrade`. A missing file plans as `NotPresent` and is never created. + - `candidate(path)`, which returns the `Ready` candidate bytes for a planned path. + - `commit(self) -> Vec`. + - `resolve_setup_repository` now takes `context_only` and delegates to `resolve_setup_repository_with`, which takes injectable env lookup and global-path resolvers. It runs these steps in order: + 1. `ensure_git_repository`. + 2. Plan the upgrades, which is skipped in context-only mode. Only the canonical local and global paths are planned; an explicit config path never is. + 3. `resolve_agent_trace_storage_runtime_config_with`, now `pub(crate)`, with a `read_file` overlay that serves candidate bytes for the planned paths. + 4. `ensure_git_remote`. + 5. Commit the upgrades. + - It returns a `SetupRepository { root, schema_upgrades }`. `execute` then runs `prepare_dispatch` and every later reader or writer against the committed files. + - `SetupRepository.schema_upgrades` carries `#[cfg_attr(not(test), expect(dead_code, …))]` until T06 renders it. The module-level `expect(dead_code)` on `schema_upgrade` stays, because the skip details are still unread. + - 7 new tests in `setup::command::tests`: + - The remote preflight uses `upstream` from a stale local file in an `upstream`-only repo. The file is upgraded with only `$schema` changed, and the global file is not created. + - A failing remote preflight leaves both stale files byte-identical. + - A failing Git gate leaves the stale global file byte-identical. + - A stale global `auth.mode: disabled` does not lock the gate before the upgrade, locks it as `Global` after the upgrade, and suppresses the access prompts. + - A stale local file's targets, optional workflows, and saved access answers drive dispatch after the upgrade: only `login` is prompted, and the access write is `Unchanged`. + - Context-only mode plans and writes nothing. + - With an invalid stale local file and a valid stale global file, only the global file is upgraded, the local file stays byte-identical, and prompts are skipped as today. + - Approved assumption: the global file is passed to the planner whenever its path resolves. `NotPresent` covers "planned only if the file exists" without a separate existence check. + - Verify: + - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml setup`: passed (101 passed, 0 failed). + - Supplementary checks: + - `… test … schema_upgrade`: passed (16 passed, 0 failed). + - `clippy --all-targets -- -D warnings`: passed. + - `cargo fmt --check`: passed. + - Context impact: localized. The `sce setup` ordering now upgrades stale canonical `$schema` declarations before the remote preflight and before dispatch, which changes setup behavior for stale files. Setup output is unchanged until T06, and durable documentation is scheduled for T08. + - Context synchronization: synced - [ ] T06: `Report schema upgrade outcomes in setup output` (status:todo) - Task ID: T06 diff --git a/context/sce/setup-repo-local-config-bootstrap.md b/context/sce/setup-repo-local-config-bootstrap.md index ab725439f..7b575197e 100644 --- a/context/sce/setup-repo-local-config-bootstrap.md +++ b/context/sce/setup-repo-local-config-bootstrap.md @@ -41,14 +41,15 @@ Task `setup-repo-gate-and-local-config-bootstrap` T02, `turso-local-db-sync` T04 - The parent `.sce/` directory is created via `fs::create_dir_all` if missing. - The setup flow also bootstraps the canonical local DB through `LocalDbLifecycle::setup` and the Agent Trace DB through `AgentTraceDbLifecycle::setup`; both use the shared `TursoDb` adapter. - After both repository preflights (`ensure_git_repository` and the effective named-remote URL check), setup consumes the shared resolver's degraded result for an invalid default-discovered repo-local `.sce/config.json`; the file remains untouched and the run continues through prompts, context baseline bootstrap, lifecycle providers, hooks, and target assets. An absent config continues through the normal bootstrap path, while explicit config selections remain fatal. +- Between the Git-repo gate and the named-remote check, normal setup modes (not context-only) plan a `$schema`-only upgrade of the canonical repository `.sce/config.json` and, when its path resolves, the default global config (`cli/src/services/setup/schema_upgrade.rs`). An explicit `--config` / `SCE_CONFIG_FILE` file is never planned, and a missing file is never created. The named-remote check resolves `agent_trace.repository_remote` through a read overlay that serves each validated older-series candidate, so a stale but otherwise valid local file selects its configured remote. The planned upgrades are committed only after that check passes, so a failed Git or remote preflight writes nothing. Dispatch, access prompts and the access gate, the access writer, lifecycle bootstrap, and target persistence all read the committed files. Each file is upgraded or skipped independently: an invalid candidate leaves its file byte-identical and degraded as before, without blocking the other file. - Config/DB bootstrap runs after those preflights and after context baseline bootstrap, before config/hooks dispatch, so it applies to all normal setup modes: config-only, hooks-only, combined, and interactive. If the repo-local config is invalid, target and optional-workflow persistence is skipped rather than rewriting it. ## Repository access preferences Setup saves `auth.mode` and `telemetry.enabled` only to the repository's committed `/.sce/config.json`, so each answer applies only to that repository ([ADR](../decisions/2026-10-22-setup-writes-access-answers-to-repository-config.md)). It never writes them to the global user config or an explicit config file, and it does not migrate access values that earlier setup runs left in the global config; those stay in effect. This applies to interactive runs only; non-interactive, hooks-only, and context-only runs never write access keys and print no `SCE access` section. -- **Order:** the write runs right after dispatch and before `bootstrap_context_baseline`, so it is setup's first write. Cancelling a prompt still writes nothing. -- **Writer:** `persist_repository_access_preferences` (`cli/src/services/setup/access_preferences.rs`). It writes explicit answers only, preserves unrelated keys, validates the result with `parse_file_config`, and skips the write when the values are already current. A missing file starts from the repo-local bootstrap payload. The write is a sibling temp file plus a rename, keeping the existing mode. It is refused when the file's bytes changed after the first read. When a write is needed and the path is a symlink (valid or dangling, detected with `fs::symlink_metadata`), the writer refuses with `SymlinkManaged` and leaves the link and its target unchanged, because a rename would replace the link. A symlinked file whose values are already current is still a no-op. The global config is only read, for the [Claude telemetry removal](#setup-login) check. +- **Order:** the write runs right after dispatch and before `bootstrap_context_baseline`, so it is setup's first write after any `$schema` upgrade. Cancelling a prompt still writes nothing. +- **Writer:** `persist_repository_access_preferences` (`cli/src/services/setup/access_preferences.rs`). It writes explicit answers only, preserves unrelated keys, validates the result with `parse_file_config`, and skips the write when the values are already current. A missing file starts from the repo-local bootstrap payload. The write is a sibling temp file plus a rename, keeping the existing mode. It is refused when the file's bytes changed after the first read. When a write is needed and the path is a symlink (valid or dangling, detected with `fs::symlink_metadata`), the writer refuses with `SymlinkManaged` and leaves the link and its target unchanged, because a rename would replace the link. A symlinked file whose values are already current is still a no-op. The access writer only reads the global config, for the [Claude telemetry removal](#setup-login) check; setup's only global write is the `$schema` upgrade in [Behavior](#behavior). - **Invalid repository config:** when the file is already invalid at read time, the access prompts are skipped and nothing is written; the file stays byte-identical. - **Write failure** (unwritable, became invalid, or concurrently modified): setup exits with a runtime error (`SCE-ERR-RUNTIME`) before the target install, with no other repository file changed. The error is three fixed lines: `Failed to save SCE access preferences to the repository SCE config ''.`, then the writer's `Repository SCE config '' ...` error, then `No repository files were changed. Run 'sce config validate' to check the file, then rerun 'sce setup'.` For a symlink-managed file, the writer's error says SCE cannot modify it and to update its source directly, and the last line is `No repository files were changed. Rerun 'sce setup' after updating the config source.` - **Report:** after the target install, setup re-resolves the policy for the repository root and prints an `SCE access:` section between the install section and the Claude telemetry stage: @@ -137,7 +138,7 @@ The same write also records the run's resolved optional-workflow selection under - `cli/src/services/agent_trace_db/lifecycle.rs` implements `AgentTraceDbLifecycle::setup()` for Agent Trace DB initialization. - Repo-local config bootstrap uses `RepoPaths::sce_config_file()` and `RepoPaths::sce_dir()`; context baseline bootstrap uses the shared context accessors including `RepoPaths::context_tmp_gitignore_file()`. - The canonical payload constant is `REPO_LOCAL_CONFIG_BOOTSTRAP_PAYLOAD`. -- `cli/src/services/setup/command.rs` resolves the effective `agent_trace.repository_remote` and runs both repository preflights before `bootstrap_context_baseline`; default-discovered invalid config is handled by the shared resolver and is not rewritten. Context-only requests return after the baseline. Normal modes then derive a repo-root-scoped `AppContext` and aggregate lifecycle providers in config → local_db → auth_db → agent_trace_db → hooks order; `ConfigLifecycle::setup()` calls `bootstrap_repo_local_config(...)`, `LocalDbLifecycle::setup()` initializes the local DB, `AuthDbLifecycle::setup()` initializes the auth DB, and `AgentTraceDbLifecycle::setup()` initializes the Agent Trace DB. +- `cli/src/services/setup/command.rs` resolves the effective `agent_trace.repository_remote` and runs both repository preflights before `bootstrap_context_baseline`; `resolve_setup_repository_with` plans canonical `$schema` upgrades after the Git-repo gate, serves their candidates to the named-remote resolution, and commits them after the remote check; default-discovered invalid config is handled by the shared resolver and is not rewritten. Context-only requests return after the baseline. Normal modes then derive a repo-root-scoped `AppContext` and aggregate lifecycle providers in config → local_db → auth_db → agent_trace_db → hooks order; `ConfigLifecycle::setup()` calls `bootstrap_repo_local_config(...)`, `LocalDbLifecycle::setup()` initializes the local DB, `AuthDbLifecycle::setup()` initializes the auth DB, and `AgentTraceDbLifecycle::setup()` initializes the Agent Trace DB. ## Relationship to other setup contracts From 884147e99dc479cdc3b6dbbae25f029edbcfeae0 Mon Sep 17 00:00:00 2001 From: David Abram Date: Tue, 10 Nov 2026 13:47:59 +0100 Subject: [PATCH 7/9] setup: Report schema upgrade outcomes in setup output Expose canonical config schema upgrade and skip results as a leading `Config schema:` section, so setup users can see when stale files were upgraded or left unchanged. Render each upgraded or skipped outcome with actionable wording, keep skipped outcomes non-fatal, and cover the output with focused tests. Update the durable T06 plan record and related architecture/setup documentation. Plan: config-schema-series-compatibility (T06) Co-authored-by: SCE --- cli/src/services/setup/command.rs | 199 +++++++++++++++++- cli/src/services/setup/mod.rs | 4 - cli/src/services/setup/schema_upgrade.rs | 11 +- context/architecture.md | 2 +- context/context-map.md | 2 +- .../config-schema-series-compatibility.md | 41 +++- .../sce/setup-repo-local-config-bootstrap.md | 3 +- 7 files changed, 235 insertions(+), 27 deletions(-) diff --git a/cli/src/services/setup/command.rs b/cli/src/services/setup/command.rs index 2018def00..50daa0cff 100644 --- a/cli/src/services/setup/command.rs +++ b/cli/src/services/setup/command.rs @@ -4,7 +4,9 @@ use std::path::{Path, PathBuf}; use anyhow::Context; use crate::app::ContextWithRepoRoot; +use crate::services::agent_trace::sce_config_schema_url; use crate::services::auth_command::{self, AuthFormat, LoginReport}; +use crate::services::config::schema_declaration::current_schema_version; use crate::services::config::{AuthMode, ResolvedAccessPolicy, ValueSource}; use crate::services::default_paths::RepoPaths; use crate::services::error::{AuthPolicyConfigFile, CliError, UserError}; @@ -17,10 +19,13 @@ use crate::services::setup::access_preferences::{ RepositoryAccessPreferencesError, }; use crate::services::setup::claude_telemetry::CLAUDE_TELEMETRY_DISABLED_BY_CONFIG_LINE; -use crate::services::setup::schema_upgrade::{CanonicalSchemaUpgrades, SchemaUpgradeReport}; +use crate::services::setup::schema_upgrade::{ + CanonicalSchemaUpgrades, SchemaUpgradeOutcome, SchemaUpgradeReport, SchemaUpgradeSkip, +}; use crate::services::{config, managed_telemetry, setup}; const SCE_ACCESS_HEADING: &str = "SCE access:"; +const CONFIG_SCHEMA_HEADING: &str = "Config schema:"; const SAVED_ACCESS_SCOPE: &str = "applies only to this repository"; const AUTH_LOGOUT_HINT_LINE: &str = "- SCE login: stored SCE credentials remain on this machine; run 'sce auth logout' to remove them."; @@ -43,6 +48,7 @@ pub struct SetupCommand { } impl SetupCommand { + #[allow(clippy::too_many_lines)] pub async fn execute(&self, context: &C) -> Result { let setup_start_path = match &self.request.hooks_repo_path { Some(path) => path.clone(), @@ -53,9 +59,14 @@ impl SetupCommand { let repository = resolve_setup_repository(&setup_start_path, self.request.context_only)?; let repository_root = repository.root; + let mut sections = + Vec::from_iter(render_schema_upgrade_section(&repository.schema_upgrades)); let (setup_dispatch, interactive_access) = match self.prepare_dispatch(&repository_root)? { - PreparedSetup::Cancelled => return Ok(setup::setup_cancelled_text()), + PreparedSetup::Cancelled => { + sections.push(setup::setup_cancelled_text()); + return Ok(sections.join("\n\n")); + } PreparedSetup::Ready { dispatch, interactive_access, @@ -65,8 +76,6 @@ impl SetupCommand { let claude_telemetry_selected = dispatch_targets_claude(setup_dispatch.as_ref()); let authorization_check = RepositoryAuthorizationCheck::for_mode(self.request.config_mode); - let mut sections = Vec::new(); - let context_message = setup::bootstrap_context_baseline(&repository_root).map_err(unexpected_failure)?; sections.push(context_message); @@ -507,6 +516,54 @@ fn render_setup_access_section( lines.join("\n") } +fn render_schema_upgrade_section(reports: &[SchemaUpgradeReport]) -> Option { + let lines: Vec = reports + .iter() + .filter_map(|report| { + let path = report.path.display(); + let skip = match &report.outcome { + SchemaUpgradeOutcome::Unchanged => return None, + SchemaUpgradeOutcome::Upgraded { from } => { + return Some(format!( + "- upgraded '{path}' from v{from} to v{}", + current_schema_version() + )); + } + SchemaUpgradeOutcome::Skipped(skip) => skip, + }; + let reason = match skip { + SchemaUpgradeSkip::NewerSeries(version) => format!( + "it declares newer schema series {}; upgrade the SCE CLI", + version.series() + ), + SchemaUpgradeSkip::Unrecognized => format!( + "'$schema' is not a canonical SCE schema URL; set it to {}", + sce_config_schema_url() + ), + SchemaUpgradeSkip::CandidateInvalid { reason } => format!( + "upgrading '$schema' to v{} would fail validation: {reason}. Fix the listed fields, then rerun 'sce setup'.", + current_schema_version() + ), + SchemaUpgradeSkip::SymlinkManaged => "it is a symlink; SCE cannot modify a symlink-managed config. Update the file's source directly.".to_string(), + SchemaUpgradeSkip::ConcurrentModification => { + "it changed while setup was upgrading '$schema'. Rerun 'sce setup'.".to_string() + } + SchemaUpgradeSkip::Io(error) => { + format!("setup could not upgrade '$schema': {error:#}") + } + }; + Some(format!("- left '{path}' unchanged; {reason}")) + }) + .collect(); + + (!lines.is_empty()).then(|| { + std::iter::once(CONFIG_SCHEMA_HEADING.to_string()) + .chain(lines) + .collect::>() + .join("\n") + }) +} + fn format_policy_source(source: ValueSource) -> String { match source.config_source() { Some(config_source) => format!( @@ -734,13 +791,6 @@ where struct SetupRepository { root: PathBuf, - #[cfg_attr( - not(test), - expect( - dead_code, - reason = "the setup report renders the schema upgrade outcomes" - ) - )] schema_upgrades: Vec, } @@ -852,6 +902,8 @@ mod tests { use super::*; + use crate::services::config::schema_declaration::SchemaVersion; + use crate::services::config::AuthMode; use crate::services::setup::{ resolve_setup_dispatch, resolve_setup_request, ConcreteSetupTarget, @@ -1991,6 +2043,131 @@ mod tests { ); } + fn schema_report(path: &str, outcome: SchemaUpgradeOutcome) -> SchemaUpgradeReport { + SchemaUpgradeReport { + path: std::path::PathBuf::from(path), + outcome, + } + } + + fn schema_version(raw: &str) -> SchemaVersion { + SchemaVersion::parse(raw).unwrap() + } + + fn render_schema_skip(skip: SchemaUpgradeSkip) -> String { + render_schema_upgrade_section(&[schema_report( + "/repo/.sce/config.json", + SchemaUpgradeOutcome::Skipped(skip), + )]) + .unwrap() + } + + #[test] + fn schema_upgrade_section_is_omitted_when_every_file_is_unchanged() { + assert_eq!(render_schema_upgrade_section(&[]), None); + assert_eq!( + render_schema_upgrade_section(&[ + schema_report("/repo/.sce/config.json", SchemaUpgradeOutcome::Unchanged), + schema_report("/home/sce/config.json", SchemaUpgradeOutcome::Unchanged), + ]), + None + ); + } + + #[test] + fn schema_upgrade_section_renders_upgraded_line_and_skips_unchanged_files() { + let section = render_schema_upgrade_section(&[ + schema_report( + "/repo/.sce/config.json", + SchemaUpgradeOutcome::Upgraded { + from: schema_version("0.3.3"), + }, + ), + schema_report("/home/sce/config.json", SchemaUpgradeOutcome::Unchanged), + ]); + + assert_eq!( + section.unwrap(), + format!( + "Config schema:\n- upgraded '/repo/.sce/config.json' from v0.3.3 to v{}", + current_schema_version() + ) + ); + } + + #[test] + fn schema_upgrade_section_renders_newer_series_line() { + assert_eq!( + render_schema_skip(SchemaUpgradeSkip::NewerSeries(schema_version("1.4.2-rc.1"))), + "Config schema:\n- left '/repo/.sce/config.json' unchanged; it declares newer schema series 1.4; upgrade the SCE CLI" + ); + } + + #[test] + fn schema_upgrade_section_renders_unrecognized_line() { + assert_eq!( + render_schema_skip(SchemaUpgradeSkip::Unrecognized), + format!( + "Config schema:\n- left '/repo/.sce/config.json' unchanged; '$schema' is not a canonical SCE schema URL; set it to {}", + sce_config_schema_url() + ) + ); + } + + #[test] + fn schema_upgrade_section_renders_candidate_invalid_line() { + assert_eq!( + render_schema_skip(SchemaUpgradeSkip::CandidateInvalid { + reason: "unknown field 'unknown_key'".to_string(), + }), + format!( + "Config schema:\n- left '/repo/.sce/config.json' unchanged; upgrading '$schema' to v{} would fail validation: unknown field 'unknown_key'. Fix the listed fields, then rerun 'sce setup'.", + current_schema_version() + ) + ); + } + + #[test] + fn schema_upgrade_section_renders_symlink_concurrent_and_io_lines() { + assert_eq!( + render_schema_skip(SchemaUpgradeSkip::SymlinkManaged), + "Config schema:\n- left '/repo/.sce/config.json' unchanged; it is a symlink; SCE cannot modify a symlink-managed config. Update the file's source directly." + ); + assert_eq!( + render_schema_skip(SchemaUpgradeSkip::ConcurrentModification), + "Config schema:\n- left '/repo/.sce/config.json' unchanged; it changed while setup was upgrading '$schema'. Rerun 'sce setup'." + ); + assert_eq!( + render_schema_skip(SchemaUpgradeSkip::Io(anyhow::anyhow!("permission denied"))), + "Config schema:\n- left '/repo/.sce/config.json' unchanged; setup could not upgrade '$schema': permission denied" + ); + } + + #[test] + fn schema_upgrade_section_reports_each_canonical_file_from_setup_resolution() { + let fixture = AccessFixture::git_repository(&["origin"]); + fixture.write_repo_config(&stale(r#" "unknown_key": true"#)); + fs::write( + &fixture.global, + stale(r#" "auth": { "mode": "optional" }"#), + ) + .unwrap(); + + let repository = fixture.resolve_repository(false).unwrap(); + let section = render_schema_upgrade_section(&repository.schema_upgrades).unwrap(); + + let lines: Vec<&str> = section.lines().collect(); + assert_eq!(lines[0], "Config schema:"); + assert!(lines[1].starts_with(&format!( + "- left '{}' unchanged; upgrading '$schema' to v{} would fail validation: ", + fixture.repo_config_path().display(), + current_schema_version() + ))); + assert!(section.contains("unknown_key")); + assert!(lines[1].ends_with(". Fix the listed fields, then rerun 'sce setup'.")); + assert!(section.contains(&format!("- upgraded '{}' from v", fixture.global.display()))); + } + fn walk_files(root: &Path) -> Vec { let mut files = Vec::new(); for entry in fs::read_dir(root).unwrap() { diff --git a/cli/src/services/setup/mod.rs b/cli/src/services/setup/mod.rs index bc80389c6..687dbf974 100644 --- a/cli/src/services/setup/mod.rs +++ b/cli/src/services/setup/mod.rs @@ -17,10 +17,6 @@ pub mod command; mod config_file_write; pub(crate) mod config_merge; pub(crate) mod hook_merge; -#[expect( - dead_code, - reason = "setup ordering and the setup report consume the planner and its skip details" -)] mod schema_upgrade; #[derive(Debug)] diff --git a/cli/src/services/setup/schema_upgrade.rs b/cli/src/services/setup/schema_upgrade.rs index 12e9f4a6e..0145497f3 100644 --- a/cli/src/services/setup/schema_upgrade.rs +++ b/cli/src/services/setup/schema_upgrade.rs @@ -25,7 +25,6 @@ pub(crate) enum SchemaUpgradePlan { from: SchemaVersion, }, CandidateInvalid { - from: SchemaVersion, reason: String, }, } @@ -41,7 +40,7 @@ pub(crate) enum SchemaUpgradeOutcome { pub(crate) enum SchemaUpgradeSkip { NewerSeries(SchemaVersion), Unrecognized, - CandidateInvalid { from: SchemaVersion, reason: String }, + CandidateInvalid { reason: String }, SymlinkManaged, ConcurrentModification, Io(anyhow::Error), @@ -88,7 +87,6 @@ pub(crate) fn plan_schema_upgrade(path: &Path, layer: ConfigLayer) -> SchemaUpgr }; if let Err(error) = parse_file_config(&candidate, path, layer.path_source()) { return SchemaUpgradePlan::CandidateInvalid { - from, reason: format!("{error:#}"), }; } @@ -124,8 +122,8 @@ where SchemaUpgradePlan::NewerSeries(version) => SchemaUpgradeSkip::NewerSeries(version), SchemaUpgradePlan::Unrecognized => SchemaUpgradeSkip::Unrecognized, SchemaUpgradePlan::Unreadable(error) => SchemaUpgradeSkip::Io(error), - SchemaUpgradePlan::CandidateInvalid { from, reason } => { - SchemaUpgradeSkip::CandidateInvalid { from, reason } + SchemaUpgradePlan::CandidateInvalid { reason } => { + SchemaUpgradeSkip::CandidateInvalid { reason } } SchemaUpgradePlan::Ready { original, @@ -494,10 +492,9 @@ mod tests { write_config(&path, &contents); let plan = plan_schema_upgrade(&path, ConfigLayer::Repository); - let SchemaUpgradePlan::CandidateInvalid { from, reason } = &plan else { + let SchemaUpgradePlan::CandidateInvalid { reason } = &plan else { panic!("expected an invalid candidate, got {plan:?}"); }; - assert_eq!(from.to_string(), older); assert!(reason.contains("unknown_key"), "{reason}"); assert!(!reason.contains("/$schema"), "{reason}"); diff --git a/context/architecture.md b/context/architecture.md index edf30ea47..26ba801b2 100644 --- a/context/architecture.md +++ b/context/architecture.md @@ -72,7 +72,7 @@ The repository includes a new placeholder Rust binary crate at `cli/`. - `cli/src/services/setup/mod.rs` defines the setup command contract (`SetupMode`, `SetupTarget`, `SetupRequest`, CLI flag parser/validator), an `inquire`-backed interactive target prompter (`InquireSetupTargetPrompter`), setup dispatch outcomes (proceed/cancelled), additive durable-context bootstrap (`bootstrap_context_baseline` for standalone `--bootstrap-context` and every normal successful setup path), and compile-time embedded asset access (`EmbeddedAsset`, target-scoped iterators, required-hook asset iterators/lookups). It also owns the install-time optional-workflow seam: the `OptionalWorkflow` type plus the build-generated `OPTIONAL_WORKFLOWS` catalog, a per-target `WorkflowAssetLayout` built from the existing `default_paths` command/skill directory constants (`command`/`commands`/`prompts` plus `skills`), and `iter_embedded_assets_for_setup_target_with_selection(target, selection)`, which yields every embedded asset except the `{command_dir}/{command_slug}.md` file and `{skills_dir}/{skill_slug}/` subtree of each optional workflow the selection omits. Membership is derived from the catalog's slugs rather than an enumerated file list, so a new optional workflow needs no Rust change. This filtered iterator is the only way embedded assets are enumerated; setup installs through it and doctor inspects through it, so there is no unfiltered enumeration path that could reintroduce an unselected workflow. The non-interactive selection flows through the repeatable `--workflow ` flag into `SetupRequest.optional_workflows: Option>` (`None` meaning the flag was absent); `validate_optional_workflow_slugs` checks each slug against `OPTIONAL_WORKFLOWS` during request resolution, before any write, and `run_setup_for_mode` resolves `None` to the persisted `integrations.optional_workflows` (exported as `persisted_optional_workflows`) before installing through the filtered iterator and persisting the resolved selection. The interactive selection flows through the same seam: `SetupTargetPrompter` carries `prompt_targets(defaults)`, `prompt_optional_workflows(defaults)`, the two behavior confirmations `prompt_agent_trace_auto_sync(default)` / `prompt_attribution_hooks_enabled(default)` (dispatch passes `true`), and the access prompts `prompt_auth_mode(default)` (an `inquire::Select` with fixed Optional/Mandatory/Disabled rows), `prompt_telemetry_consent(default)`, and `prompt_login()` (Yes default), which dispatch does not call yet; every method returns `None` for a cancelled prompt and fails with the fixed TTY guidance on `NotTTY`, `SetupDispatch::Proceed { mode, optional_workflows }` carries a prompted selection alongside the resolved mode, and `resolve_setup_dispatch(mode, prompter, defaults)` runs the workflow prompt only after an interactive target prompt, mapping either cancellation to `SetupDispatch::Cancelled`. The prompt module builds its `inquire::MultiSelect` from `optional_workflow_prompt_inputs(catalog, defaults)`, which returns `None` for an empty catalog (skipping the prompt) and otherwise catalog-ordered rows plus the indices to pre-check, ignoring ids absent from the catalog. `setup/command.rs` therefore resolves the repository root before dispatch, mapping the setup domain's typed positive non-Git classification to `setup.not_git_repository` and other resolution failures to `general.unexpected_failure`, so a non-Git directory fails before any prompt without CLI-layer string matching. For repository builds, `cli/build.rs` validates the `SCE_CLI_GENERATED_INPUT_DIR` payload and canonical-input inventories, copies the payload into Cargo `OUT_DIR/pkl-generated`, stages `cli/assets/hooks/**` under `OUT_DIR/static`, requires the staged `config/optional-workflows.json`, and generates both the setup manifest and the optional-workflow catalog (`optional_workflows.rs`, rejecting a manifest whose `schemaVersion` is not 1 or whose entries lack a non-empty `id`/`title`/`description`/`commandSlug`/`skillSlug`) in `OUT_DIR`; focused internal seams separate install-flow from prompt-flow logic; `cli/src/services/setup/command.rs` owns the `SetupCommand` payload used by the static `RuntimeCommand` enum and executes against any context implementing repo-root scoping. Its install engine/orchestrator (`install_assets_for_concrete_target_with_rename`) installs each embedded asset individually: it stages the asset's content next to its final destination and swaps it into place by renaming the staging file directly over the destination — never unlinking the destination first, since `fs::rename` already replaces an existing file atomically — with deterministic recovery guidance naming the failing asset's path on swap failure (the pre-existing destination content, if any, is untouched) and no backup artifact creation; it never removes an integration target directory as a whole, so files a repository owns inside `.opencode`/`.claude`/`.pi` — including nested inside an SCE-owned subdirectory such as `skills/` or `commands/` — survive a setup run; Codex has no single target directory this way, since `CODEX_EMBEDDED_ASSETS` relative paths already carry their own `.agents/`/`.codex/` prefix and `InstallTargetPaths::codex_target_dir()` resolves to the repository root itself, but the same per-asset stage/atomic-swap and never-delete-what-it-did-not-author guarantees still apply file by file. For the three assets that are merge targets — the Claude target's `settings.json`, the OpenCode target's `opencode.json`, and Codex's user-owned `.codex/hooks.json` — the content staged is not always the embedded asset's bytes. Claude and OpenCode use `cli/src/services/setup/config_merge.rs`, while Codex uses the shared `cli/src/services/codex_hook_config.rs` service for structural validation and canonical-registration merging. For the Claude and OpenCode targets, the content staged is the result of `cli/src/services/setup/config_merge.rs::merge_or_create_claude_settings`/`merge_or_create_opencode_config(existing_bytes, generated_bytes, source_path)`: each returns the generated document verbatim when no file exists yet, otherwise parses the existing file as JSON (a parse failure is a hard error naming `source_path`, and nothing is written) and merges it with a pure per-shape function that copies `$schema` from the generated document and preserves every other key from the existing file untouched. `merge_claude_settings` replaces, per hook event key the generated document declares, only the entries whose command contains the ownership marker `run-sce-or-show-install-guidance.sh`, preserving every event key and non-SCE hook entry from the existing file. `merge_opencode_config` merges the `plugin` array as a set: existing entries whose path starts with the ownership marker `./plugins/sce-` are dropped structurally — so a stale plugin path an older or renamed catalog once installed is still recognized and pruned even after the current generated document stops declaring it — and the generated document's `plugin` entries are appended after the surviving entries. Codex's merge validates the document shape and recognizes ownership by the generated helper path plus one of two trailing command contracts — `sce hooks codex` or `sce hooks codex-mutation-scope` — attributing each handler to whichever it matches; it replaces stale or duplicate handlers with exactly one current handler per registration (the four `sce hooks codex` registrations plus six additive `sce hooks codex-mutation-scope` registrations, within that mutation-scope set, `PreToolUse` and `PostToolUse` use matcher `^(Bash|apply_patch)$`, while `Stop`, `Interrupt`, `SubagentStop`, and `SessionEnd` are appended in unmatched groups after the existing groups so an already-trusted handler keeps its `(event, matcher, group index, handler index)` identity and computed Codex trust key), touching only the matching command's handlers and preserving unrelated valid Codex fields, groups, and handlers. After that per-asset install loop, `prune_stale_assets_for_concrete_target` deletes every path the full embedded catalog for the concrete target claims but the resolved selection did not install (a deselected optional workflow, or an asset a newer catalog renamed or dropped), and `remove_empty_ancestor_directories` removes any parent directory left empty by that deletion, stopping at the target root or at a directory that still holds something such as a user file. It formats deterministic completion messaging; required-hook install orchestration (`install_required_git_hooks`, backed by the rename-injectable `install_required_git_hooks_with_rename`) is a third content-computation seam alongside the two JSON merge targets: `install_single_required_hook_with_rename` computes the bytes to stage with `cli/src/services/setup/hook_merge.rs::merge_or_create_hook(existing_bytes, canonical_bytes, hook_name)` rather than writing `hook_asset.bytes` verbatim — a foreign hook (no SCE managed block, no legacy guidance-URL marker) is kept as an exact byte prefix with the canonical block appended after it, an SCE-owned hook has only its block spliced in place or left unchanged, and a legacy pre-marker hook is replaced wholesale — then follows the same per-file stage/atomic-swap choreography as config-asset install: the staging file is renamed directly over the existing hook without unlinking it first, so a rename failure leaves the prior hook's bytes and executable bit intact, with deterministic recovery guidance on swap failure. `Installed`/`Updated`/`Skipped` are decided against the merged bytes plus the executable bit rather than the canonical asset's raw bytes, so an already-current foreign-plus-block hook reports `Skipped`; `RequiredHookInstallResult.unreachable_block_advisory` is set, and rendered as a named advisory line in setup's hook output, when an appended block follows a foreign hook's zero-indent `exec`/`exit` and so would never run. After the Git gate, setup always ensures the context baseline; context-only requests return there, while normal modes derive a repo-root-scoped context before aggregating static lifecycle provider `setup` dispatch across providers (config → local_db → auth_db → agent_trace_db → hooks when requested), so setup providers consume only repo-root access from the scoped context. - `cli/src/services/setup/mod.rs` keeps those responsibilities inside one file for now, but the current ownership split is explicit: the inline `install` module owns repository-path normalization, staging/swap install behavior, required-hook installation, and filesystem safety guards, while the inline `prompt` module owns interactive target selection and prompt styling. - `cli/src/services/setup/access_preferences.rs` owns setup's access preferences (`auth.mode`, `telemetry.enabled`): the repository-layer reader and atomic writer for `.sce/config.json` (`read_repository_access_preferences` / `persist_repository_access_preferences`, through `write_guarded` in `setup/config_file_write.rs`, which owns `ConfigLayer` and the setup config-file write guard: reread-compare, symlink refusal, then the shared temp-file-plus-rename helper in `setup/atomic_write.rs`), plus a read-only global reader used for the Claude telemetry removal check. `setup/command.rs` calls the writer after dispatch and before the target install, and re-resolves the policy for the `SCE access` report and for the setup login step (`setup_login_action` / `run_setup_login_step`, which reuses `auth_command::run_login_in` rather than a separate login path); precedence stays in `config/resolver.rs` (see `context/cli/access-policy.md#setup`). -- `cli/src/services/setup/schema_upgrade.rs` owns the `$schema`-only upgrade of one canonical config file (`ConfigLayer::Repository` or `ConfigLayer::Global`). `plan_schema_upgrade` is read-only. It classifies the declaration with `config/schema_declaration.rs`. For an older-series declaration, it builds a candidate that changes only `$schema`: it swaps the single URL literal when the result stays structurally identical, and otherwise falls back to pretty serialization. It then validates the candidate through `parse_file_config` with the layer's path source, yielding `Ready` or `CandidateInvalid`. `commit_schema_upgrade` writes only a `Ready` plan, through `write_guarded` against the planned original bytes. Every other plan becomes `Unchanged` or a typed `Skipped(SchemaUpgradeSkip)`. Newer-series, unrecognized, and invalid-candidate files are never rewritten, and a missing file is never created. `CanonicalSchemaUpgrades` plans the canonical repository and global files, exposes `Ready` candidates as a read overlay for the setup remote preflight, and commits them into per-path `SchemaUpgradeReport`s; `setup/command.rs` calls it between the Git-repo gate and the named-remote check and commits after the check passes. +- `cli/src/services/setup/schema_upgrade.rs` owns the `$schema`-only upgrade of one canonical config file (`ConfigLayer::Repository` or `ConfigLayer::Global`). `plan_schema_upgrade` is read-only. It classifies the declaration with `config/schema_declaration.rs`. For an older-series declaration, it builds a candidate that changes only `$schema`: it swaps the single URL literal when the result stays structurally identical, and otherwise falls back to pretty serialization. It then validates the candidate through `parse_file_config` with the layer's path source, yielding `Ready` or `CandidateInvalid`. `commit_schema_upgrade` writes only a `Ready` plan, through `write_guarded` against the planned original bytes. Every other plan becomes `Unchanged` or a typed `Skipped(SchemaUpgradeSkip)`. Newer-series, unrecognized, and invalid-candidate files are never rewritten, and a missing file is never created. `CanonicalSchemaUpgrades` plans the canonical repository and global files, exposes `Ready` candidates as a read overlay for the setup remote preflight, and commits them into per-path `SchemaUpgradeReport`s; `setup/command.rs` calls it between the Git-repo gate and the named-remote check, commits after the check passes, and renders the reports as the leading `Config schema:` setup section. - `cli/src/services/security.rs` provides shared security utilities for deterministic secret redaction (`redact_sensitive_text`) and directory write-permission probes (`ensure_directory_is_writable`) used by app/setup/observability surfaces. - `cli/src/services/doctor/mod.rs` owns the current doctor request/report surface while focused submodules (`doctor/inspect.rs`, `doctor/render.rs`, `doctor/fixes.rs`, `doctor/types.rs`) split report fact collection, rendering, manual fix reporting, and doctor-owned domain types into smaller seams; `cli/src/services/doctor/command.rs` owns the `DoctorCommand` payload used by the static `RuntimeCommand` enum and executes against any context implementing repo-root scoping. Runtime doctor execution resolves a repository root, derives a scoped context, requests the shared static lifecycle provider catalog with hooks included for service-owned `diagnose` and `fix` behavior, adapts lifecycle-owned health/fix records into doctor-owned problem/fix records, and then renders stable text/JSON problem records with category/severity/fixability/remediation fields plus deterministic fix-result reporting in fix mode. Agent Trace database inspection is no longer a doctor-adjacent command surface; doctor owns repository-scoped DB health and checkout identity facts, while `sce sync` owns control-plane synchronization. Report fact collection preserves environment/repository/hook/integration display data and adds a non-launching `post_commit_auto_sync` fact based on canonical post-commit managed-block currency plus resolved `agent_trace.auto_sync` source/value; this fact does not launch `sce sync`, and existing hook problem/remediation/readiness semantics remain authoritative. Service-owned lifecycle providers own config validation, local DB and repository-scoped Agent Trace DB readiness/bootstrap, and hook rollout diagnosis/repair. Integration inspection in `doctor/inspect.rs` is scoped twice over: `resolve_doctor_integration_targets` picks which targets to inspect, and `persisted_optional_workflows` (reused from setup) resolves which optional workflows the repository selected, which the OpenCode/Claude/Pi/Codex child collectors apply through `iter_embedded_assets_for_setup_target_with_selection`. An unselected optional workflow therefore contributes no expected children at all, so no row and no missing/mismatch problem can be produced for it, while a selected one keeps the unchanged presence and content-hash checks. Codex's collector (`collect_codex_integration_groups`) is the one exception to the single-target-directory shape the other three share: since `CODEX_EMBEDDED_ASSETS` relative paths already carry their own `.agents/`/`.codex/` prefix, it resolves its integration root through `InstallTargetPaths::codex_target_dir()` (the repository root itself) rather than a `RepoPaths` per-target subdirectory, and splits assets into `Skills` (`.agents/skills/` prefix) and `Hooks` (`.codex/` prefix) groups instead of Pi's `Extensions`/`Prompts`/`Skills` split. Its `.codex/hooks.json` reporting is per-registration rather than one whole-file child: `codex_hook_config::diagnose_document` classifies each required registration — the four `sce hooks codex` registrations plus the six `sce hooks codex-mutation-scope` ones, the latter reported under distinct `#(mutation-scope)` rows — as `PresentAndCurrent`/`Missing`/`Stale` (or the whole document as `Malformed` when it cannot be structurally validated) without writing anything, so unrelated user handlers never create a false whole-document mismatch. For a structurally current registration, `codex_hook_trust` separately reads (never writes) Codex's own durable `$CODEX_HOME/config.toml` hook-trust state — reproducing upstream's `hook_hash`/`hook_key`/`hook_trust_status` exactly — and reports `Trusted`/`Untrusted`/`Modified`/`Disabled`/`Unknown`, and `codex_hook_policy` probes Codex's effective hook-discovery policy once per invocation; only a structurally current registration that is trusted *and* policy-allowed renders healthy. `sce doctor --fix` repairs a structurally unhealthy `.codex/hooks.json` through the existing merge-install path, but a registration that is current yet not-yet-trusted or policy-blocked is never "fixed", since SCE cannot grant Codex hook trust or change managed policy. - `cli/src/services/version/mod.rs` defines the version command parser/rendering contract (`parse_version_request`, `render_version`) with deterministic text output and stable JSON runtime-identification fields; `cli/src/services/version/command.rs` owns the `VersionCommand` payload used by the static `RuntimeCommand` enum. diff --git a/context/context-map.md b/context/context-map.md index 24f7e368c..ddc4178b2 100644 --- a/context/context-map.md +++ b/context/context-map.md @@ -87,7 +87,7 @@ Feature/domain context: - `context/sce/setup-githooks-hook-asset-packaging.md` (compile-time `sce setup --hooks` required-hook template packaging contract, including all-hook non-blocking missing-`sce` install guidance, available-CLI argument forwarding, post-commit-only origin remote lookup plus remote-URL forwarding/fallback behavior, setup-service accessor surface, and current validation posture) - `context/sce/setup-githooks-install-flow.md` (setup-service required-hook install orchestration with git-truth hooks-path resolution, managed-block merge content computation that preserves foreign hook content, per-hook installed/updated/skipped outcomes decided against the merged content, the unreachable-block advisory, and atomic-swap replacement with recovery guidance) - `context/sce/setup-githooks-cli-ux.md` (T04 composable `sce setup` target+`--hooks` / `--repo` command-surface contract, option compatibility validation, and deterministic setup/hook output semantics) -- `context/sce/setup-repo-local-config-bootstrap.md` (setup local bootstrap behavior: Git-root and named-remote preflights before prompts/context/lifecycle/hooks/assets, canonical `$schema` upgrades planned before the named-remote check and committed after it, shared degraded handling for invalid default-discovered repo-local config with byte-preserving skip of integration persistence, explicit-config fatality, additive durable-context baseline via `sce setup --bootstrap-context` and every normal setup path, repo-local `.sce/config.json` create-if-missing via config lifecycle, additive `integrations.target` persistence after successful target installs, `integrations.optional_workflows` selection persistence with prompt-over-flag-over-persisted precedence, plus lifecycle-owned local DB initialization before hooks/config asset dispatch, and interactive global-only `auth.mode` / `telemetry.enabled` persistence before any repository write with the re-resolved `SCE access` report, the interactive setup login step decided by the effective auth mode with `auth.setup_login_required` on mandatory failure, and interactive-only `sce setup --reconfigure` re-asking every saved answer with the saved value as the default) +- `context/sce/setup-repo-local-config-bootstrap.md` (setup local bootstrap behavior: Git-root and named-remote preflights before prompts/context/lifecycle/hooks/assets, canonical `$schema` upgrades planned before the named-remote check, committed after it, and reported in the leading `Config schema:` section, shared degraded handling for invalid default-discovered repo-local config with byte-preserving skip of integration persistence, explicit-config fatality, additive durable-context baseline via `sce setup --bootstrap-context` and every normal setup path, repo-local `.sce/config.json` create-if-missing via config lifecycle, additive `integrations.target` persistence after successful target installs, `integrations.optional_workflows` selection persistence with prompt-over-flag-over-persisted precedence, plus lifecycle-owned local DB initialization before hooks/config asset dispatch, and interactive global-only `auth.mode` / `telemetry.enabled` persistence before any repository write with the re-resolved `SCE access` report, the interactive setup login step decided by the effective auth mode with `auth.setup_login_required` on mandatory failure, and interactive-only `sce setup --reconfigure` re-asking every saved answer with the saved value as the default) - `context/sce/cli-security-hardening-contract.md` (T06 CLI redaction contract, setup `--repo` canonicalization/validation, and setup write-permission probe behavior) - `context/sce/agent-trace-post-rewrite-local-remap-ingestion.md` (current post-rewrite no-op baseline plus historical remap-ingestion reference) - `context/sce/agent-trace-rewrite-trace-transformation.md` (current post-rewrite no-op baseline plus historical rewrite-transformation reference) diff --git a/context/plans/config-schema-series-compatibility.md b/context/plans/config-schema-series-compatibility.md index 7c281c80f..5d9e42eb0 100644 --- a/context/plans/config-schema-series-compatibility.md +++ b/context/plans/config-schema-series-compatibility.md @@ -435,7 +435,7 @@ Persist this field in every plan; this is durable plan state, not chat state: - Context impact: localized. The `sce setup` ordering now upgrades stale canonical `$schema` declarations before the remote preflight and before dispatch, which changes setup behavior for stale files. Setup output is unchanged until T06, and durable documentation is scheduled for T08. - Context synchronization: synced -- [ ] T06: `Report schema upgrade outcomes in setup output` (status:todo) +- [x] T06: `Report schema upgrade outcomes in setup output` (status:done) - Task ID: T06 - Scope: - In: @@ -454,7 +454,44 @@ Persist this field in every plan; this is durable plan state, not chat state: - Silent outcomes (absent file, no declaration, current series) render nothing. - Verify: - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml setup` - - Context synchronization: pending + - Completed: 2026-11-10 + - Files changed: + - `cli/src/services/setup/command.rs` + - `cli/src/services/setup/schema_upgrade.rs` + - `cli/src/services/setup/mod.rs` + - Result: + - `render_schema_upgrade_section(&[SchemaUpgradeReport]) -> Option` in `command.rs` renders a `Config schema:` heading and one `- `-prefixed line for each `Upgraded` or `Skipped` file, using the fixed wording. It returns `None` when every outcome is `Unchanged`, which covers an absent file, no declaration, and the current series. + - `SetupCommand::execute` puts the section first, before the context baseline message. Context-only mode plans no upgrades, so it never renders the section. + - Wording: + - Upgraded: `upgraded '' from v to v`. + - Newer series: `version.series()` gives the `X.Y` series. + - Unrecognized: the line names `sce_config_schema_url()`. + - Symlink-managed: `it is a symlink; SCE cannot modify a symlink-managed config. Update the file's source directly.` + - Concurrent modification: `it changed while setup was upgrading '$schema'. Rerun 'sce setup'.` + - The setup exit code is unchanged (successful). + - Removed the `dead_code` expects on `SetupRepository.schema_upgrades` and on `mod schema_upgrade`. + - 7 new tests in `setup::command::tests`: + - the section is omitted for no reports or only `Unchanged` reports; + - the upgraded line, with `Unchanged` files skipped; + - the newer-series line; + - the unrecognized line; + - the candidate-invalid line; + - the symlink, concurrent-modification, and I/O lines; + - real setup resolution with an invalid stale local file and a valid stale global file, which renders both lines in order. + - Approved assumptions: + - `Skipped(Io)`, which covers unreadable or non-JSON files and I/O write failures, renders `left '' unchanged; setup could not upgrade '$schema': `. The plan gave this case no fixed wording. + - Deviations: + - `from` was removed from `SchemaUpgradePlan::CandidateInvalid` and `SchemaUpgradeSkip::CandidateInvalid`. The fixed candidate-invalid wording does not use it, and once the expects were removed it was dead code that failed `clippy -D warnings`. The T04 test drops its `from` assertion. + - When the user cancels an interactive prompt, the section is also put before the cancellation text, because upgrades are committed before dispatch prompts. That text still reads "No files were changed." Its wording was left unchanged because other sections are out of scope. + - `execute` carries `#[allow(clippy::too_many_lines)]`, matching `config/resolver.rs` and `config/render.rs`. + - Verify: + - `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml setup`: passed (108 passed, 0 failed). + - Supplementary checks: + - `… test … schema_upgrade`: passed (23 passed, 0 failed). + - `clippy --all-targets -- -D warnings`: passed. + - `cargo fmt --check`: passed. + - Context impact: localized, user-visible. `sce setup` output now starts with a `Config schema:` section whenever a canonical file's `$schema` is upgraded or skipped. Durable documentation of the section is scheduled for T08. + - Context synchronization: synced - [ ] T07: `Add an end-to-end setup schema-upgrade scenario check` (status:todo) - Task ID: T07 diff --git a/context/sce/setup-repo-local-config-bootstrap.md b/context/sce/setup-repo-local-config-bootstrap.md index 7b575197e..c07d30a05 100644 --- a/context/sce/setup-repo-local-config-bootstrap.md +++ b/context/sce/setup-repo-local-config-bootstrap.md @@ -42,13 +42,14 @@ Task `setup-repo-gate-and-local-config-bootstrap` T02, `turso-local-db-sync` T04 - The setup flow also bootstraps the canonical local DB through `LocalDbLifecycle::setup` and the Agent Trace DB through `AgentTraceDbLifecycle::setup`; both use the shared `TursoDb` adapter. - After both repository preflights (`ensure_git_repository` and the effective named-remote URL check), setup consumes the shared resolver's degraded result for an invalid default-discovered repo-local `.sce/config.json`; the file remains untouched and the run continues through prompts, context baseline bootstrap, lifecycle providers, hooks, and target assets. An absent config continues through the normal bootstrap path, while explicit config selections remain fatal. - Between the Git-repo gate and the named-remote check, normal setup modes (not context-only) plan a `$schema`-only upgrade of the canonical repository `.sce/config.json` and, when its path resolves, the default global config (`cli/src/services/setup/schema_upgrade.rs`). An explicit `--config` / `SCE_CONFIG_FILE` file is never planned, and a missing file is never created. The named-remote check resolves `agent_trace.repository_remote` through a read overlay that serves each validated older-series candidate, so a stale but otherwise valid local file selects its configured remote. The planned upgrades are committed only after that check passes, so a failed Git or remote preflight writes nothing. Dispatch, access prompts and the access gate, the access writer, lifecycle bootstrap, and target persistence all read the committed files. Each file is upgraded or skipped independently: an invalid candidate leaves its file byte-identical and degraded as before, without blocking the other file. +- Setup reports these outcomes in a `Config schema:` section, which is the first section of the setup output and also comes before the cancellation text when a prompt is cancelled. It lists one line for each upgraded or skipped canonical file and is omitted when every file is missing, has no declaration, or already declares the current series. An upgrade prints `upgraded '' from v to v`. A skipped file prints `left '' unchanged; `, where the reason is one of: a newer series (upgrade the SCE CLI); a non-canonical `$schema` (set it to the current URL); an invalid candidate (the validation errors, then fix the fields and rerun `sce setup`); a symlink-managed file (update the file's source directly); a concurrent modification (rerun `sce setup`); or an I/O failure. Skips never fail setup; the exit code stays successful. - Config/DB bootstrap runs after those preflights and after context baseline bootstrap, before config/hooks dispatch, so it applies to all normal setup modes: config-only, hooks-only, combined, and interactive. If the repo-local config is invalid, target and optional-workflow persistence is skipped rather than rewriting it. ## Repository access preferences Setup saves `auth.mode` and `telemetry.enabled` only to the repository's committed `/.sce/config.json`, so each answer applies only to that repository ([ADR](../decisions/2026-10-22-setup-writes-access-answers-to-repository-config.md)). It never writes them to the global user config or an explicit config file, and it does not migrate access values that earlier setup runs left in the global config; those stay in effect. This applies to interactive runs only; non-interactive, hooks-only, and context-only runs never write access keys and print no `SCE access` section. -- **Order:** the write runs right after dispatch and before `bootstrap_context_baseline`, so it is setup's first write after any `$schema` upgrade. Cancelling a prompt still writes nothing. +- **Order:** the write runs right after dispatch and before `bootstrap_context_baseline`, so it is setup's first write after any `$schema` upgrade. Cancelling a prompt writes nothing beyond the `$schema` upgrades already committed after the preflights. - **Writer:** `persist_repository_access_preferences` (`cli/src/services/setup/access_preferences.rs`). It writes explicit answers only, preserves unrelated keys, validates the result with `parse_file_config`, and skips the write when the values are already current. A missing file starts from the repo-local bootstrap payload. The write is a sibling temp file plus a rename, keeping the existing mode. It is refused when the file's bytes changed after the first read. When a write is needed and the path is a symlink (valid or dangling, detected with `fs::symlink_metadata`), the writer refuses with `SymlinkManaged` and leaves the link and its target unchanged, because a rename would replace the link. A symlinked file whose values are already current is still a no-op. The access writer only reads the global config, for the [Claude telemetry removal](#setup-login) check; setup's only global write is the `$schema` upgrade in [Behavior](#behavior). - **Invalid repository config:** when the file is already invalid at read time, the access prompts are skipped and nothing is written; the file stays byte-identical. - **Write failure** (unwritable, became invalid, or concurrently modified): setup exits with a runtime error (`SCE-ERR-RUNTIME`) before the target install, with no other repository file changed. The error is three fixed lines: `Failed to save SCE access preferences to the repository SCE config ''.`, then the writer's `Repository SCE config '' ...` error, then `No repository files were changed. Run 'sce config validate' to check the file, then rerun 'sce setup'.` For a symlink-managed file, the writer's error says SCE cannot modify it and to update its source directly, and the last line is `No repository files were changed. Rerun 'sce setup' after updating the config source.` From 77507bb0e3fda288f6e5fc7a434a52d3b0f014a4 Mon Sep 17 00:00:00 2001 From: David Abram Date: Tue, 10 Nov 2026 13:56:27 +0100 Subject: [PATCH 8/9] tests: Add setup schema upgrade check Add an end-to-end scenario that verifies read-only compatibility handling, canonical schema upgrades, idempotency, invalid and newer declarations, and symlink protection against a built `sce` binary. Expose the scenario as a root flake check and document the covered setup contract, keeping the check isolated through temporary Git, HOME, and XDG state. Record completion of the config-schema-series-compatibility plan's T07 task. Plan: config-schema-series-compatibility (T07) Co-authored-by: SCE --- context/glossary.md | 2 +- context/overview.md | 2 +- .../config-schema-series-compatibility.md | 30 +- .../sce/setup-repo-local-config-bootstrap.md | 1 + flake.nix | 14 + scripts/test-setup-schema-upgrade.sh | 309 ++++++++++++++++++ 6 files changed, 354 insertions(+), 4 deletions(-) create mode 100755 scripts/test-setup-schema-upgrade.sh diff --git a/context/glossary.md b/context/glossary.md index 9fc90d620..942bece85 100644 --- a/context/glossary.md +++ b/context/glossary.md @@ -47,7 +47,7 @@ - `checkout registry` (removed): The central JSON registry at `/sce/checkout-registry.json` was removed in the `remove-checkout-registry` plan. The later trace inspection surface and checkout identity itself (`remove-checkout-id` plan) were also removed, so nothing is discovered as a per-checkout command target. Any pre-migration `/sce/agent-trace-*.db` files remain untouched. See `context/cli/checkout-identity.md`. - `generated OpenCode plugin registration contract`: Current generated-config contract where `config/.opencode/opencode.json` serializes the OpenCode `plugin` field from canonical Pkl sources for SCE-managed plugins only; the current registered paths, in order, are `./plugins/sce-bash-policy.ts`, `./plugins/sce-agent-trace.ts`, and `./plugins/sce-mutation-scope.ts`. `sce-mutation-scope` must stay last; the OpenCode config merge appends the generated SCE entries after arbitrary user plugins and `sce doctor` (`inspect_opencode_plugin_ordering_health`) flags a non-last position. Claude does not use an OpenCode-style plugin manifest; Claude bash-policy enforcement is registered through generated `.claude/settings.json` as a `PreToolUse` `Bash` command hook running `sce policy bash`. - `root Biome contract`: Repository-root formatting/linting contract owned by `biome.json`, currently scoped only to `npm/**` and the shared `config/lib/**` plugin package root with package-local `node_modules/**` excluded; the canonical execution path is the root Nix dev shell (`nix develop -c biome ...`). -- `cli flake checks`: Check derivations in root `flake.nix` (`checks..cli-tests`, `cli-clippy`, `cli-fmt`), plus `pkl-generated`, `workflow-actionlint`, split npm/config-lib JS checks, and Linux-only Flatpak checks; invoked via `nix flake check` at repo root. +- `cli flake checks`: Check derivations in root `flake.nix` (`checks..cli-tests`, `cli-clippy`, `cli-fmt`), plus `pkl-generated`, `codex-hook-command`, `setup-schema-upgrade`, `workflow-actionlint`, split npm/config-lib JS checks, and Linux-only Flatpak checks; invoked via `nix flake check` at repo root. - `npm JS flake checks`: The current `npm/` validation slice exposed by root `flake.nix`: `npm-bun-tests` runs only `bun test ./test/*.test.js`, `npm-biome-check` runs only Biome lint/check with formatter verification disabled, `npm-biome-format` runs only Biome format verification with linter checks disabled, and `npm-pack-smoke` runs `npm/test/pack-smoke.sh` (offline packed-package install against a loopback fixture release) in the build sandbox. - `config-lib JS flake checks`: The current shared `config/lib/` validation slice exposed by root `flake.nix`: `config-lib-bun-tests` runs Bun-discovered tests from the copied shared `config/lib/` package source (including bash-policy plugin wrapper tests and tracked agent-trace plugin tests), with dependencies resolved from `config/lib/package.json` and `config/lib/bun.lock`, while `config-lib-biome-check` and `config-lib-biome-format` run Biome lint/check and format verification over the copied shared package source with formatter/linter halves disabled respectively. - `config-lib repo-shaped test source`: Root-flake source-layout contract where `config-lib-bun-tests`, `config-lib-biome-check`, and `config-lib-biome-format` run from `config/lib/` while their copied Nix source preserves repo-relative shared fixtures, currently `cli/src/services/structured_patch/fixtures` for Claude agent-trace golden tests (fully Rust-owned; the Claude TypeScript Bun test was removed in T07). diff --git a/context/overview.md b/context/overview.md index da4dde842..653cf8ca4 100644 --- a/context/overview.md +++ b/context/overview.md @@ -94,7 +94,7 @@ The root flake splits native and release outputs:`packages.sce`and`packages.defa Git-commit embedding is **release-only**: `SCE_GIT_COMMIT`is injected via a`releaseCommitArgs` fragment applied only to the release derivations (`scePackageMusl`on Linux,`sceReleasePackageNative`on Darwin), not to`commonCargoArgs`. So native `.#sce`/`.#default`and every`nix flake check` derivation (`cli-tests`, `cli-clippy`, `cli-fmt`) build without the commit in their inputs and stay cache-reusable across commits (native `sce version`reports`unknown`), while `.#sce-release`still reports the real commit via`sce version`. `cli/build.rs` `emit_git_commit`emits`SCE_GIT_COMMIT`only when the env var is explicitly set — no`git rev-parse`fallback and no`.git/HEAD`/`.git/packed-refs`rerun watches. On Darwin the release now uses a distinct native-toolchain derivation (native toolchain + commit), so it diverges from`.#sce`to carry the commit while native stays commit-independent. The default development shell is slimmed for fast iteration:`devShells.default`no longer includes`scePackage`or`tursoPackage`, so `nix develop`compiles neither the CLI package nor the Turso CLI — it provides only the Rust toolchain and JS/pkl tooling for`cargo`/`biome`/`pkl`work. Turso stays available as`packages..turso`and through a new opt-in`devShells..database`shell (default tools +`tursoPackage`), entered via `nix develop .#database`. Both shells share `defaultDevShellPackages`/`defaultDevShellHook` `let`bindings so they cannot drift. The CLI Cargo package metadata now includes crates.io publication-ready fields with crate-local install guidance in`cli/README.md`; supported Cargo install paths are `cargo install shared-context-engineering --locked`and local checkout installation through`./scripts/run-cli-cargo.sh install --path cli --locked`. Direct `cargo install --git`is unsupported because it cannot run the repository pre-Cargo producer. The published crate installs the`sce`binary. The crate also keeps`cargo clippy --manifest-path cli/Cargo.toml`warnings-denied through`cli/Cargo.toml`lint configuration, so an extra`-- -D warnings`flag is redundant. -The repository-root flake is the single Nix entrypoint for repo tooling and CLI packaging/checks, so root-level`nix flake check` evaluates the Crane-backed CLI checks (`cli-tests`, `cli-clippy`, `cli-fmt`), the ephemeral `pkl-generated`inventory check, the generated Codex `codex-hook-command` invocation check, Linux-only Flatpak checks,`workflow-actionlint`, and the split npm/config-lib JavaScript checks plus the offline npm packed-package smoke (`npm-pack-smoke`) without nested-flake indirection. Repository Cargo builds copy a validated pre-Cargo generated payload into Cargo `OUT_DIR`; crates.io packaging prepares a self-contained Pkl-free fallback in a temporary clean workspace, and Flatpak helpers prepare the same payload beside generated manifests before the sandboxed source build. No general-purpose `cli/assets/generated/`mirror or committed generated target tree remains. +The repository-root flake is the single Nix entrypoint for repo tooling and CLI packaging/checks, so root-level`nix flake check` evaluates the Crane-backed CLI checks (`cli-tests`, `cli-clippy`, `cli-fmt`), the ephemeral `pkl-generated`inventory check, the generated Codex `codex-hook-command` invocation check, the end-to-end `setup-schema-upgrade` scenario check, Linux-only Flatpak checks,`workflow-actionlint`, and the split npm/config-lib JavaScript checks plus the offline npm packed-package smoke (`npm-pack-smoke`) without nested-flake indirection. Repository Cargo builds copy a validated pre-Cargo generated payload into Cargo `OUT_DIR`; crates.io packaging prepares a self-contained Pkl-free fallback in a temporary clean workspace, and Flatpak helpers prepare the same payload beside generated manifests before the sandboxed source build. No general-purpose `cli/assets/generated/`mirror or committed generated target tree remains. Config-lib JS flake checks execute from`config/lib/`, but the copied Nix check source is repo-shaped when tests require shared repo fixtures; the current Claude agent-trace golden tests are fully Rust-owned in `cli/src/services/structured_patch/fixtures`(Claude TypeScript plugin test removed in T07). Local developer Nix tuning guidance now lives in`AGENTS.md`, including optional user-level `~/.config/nix/nix.conf`recommendations for`max-jobs`and`cores`plus an explicit system-level-only note for`auto-optimise-store`. The Pkl authoring layer owns generated OpenCode plugin registration for SCE-managed plugins: `config/pkl/base/opencode.pkl`defines the canonical plugin entries,`config/pkl/renderers/common.pkl`re-exports the shared plugin list for renderer use, and generated`config/.opencode/opencode.json`registers`./plugins/sce-bash-policy.ts`and`./plugins/sce-agent-trace.ts`through OpenCode's`plugin`field. Claude does not use an OpenCode-style plugin manifest; Claude bash-policy enforcement is registered through generated`.claude/settings.json`as a`PreToolUse` `Bash`command hook routed through`.claude/hooks/run-sce-or-show-install-guidance.sh`before running`sce policy bash`. diff --git a/context/plans/config-schema-series-compatibility.md b/context/plans/config-schema-series-compatibility.md index 5d9e42eb0..d481e1fac 100644 --- a/context/plans/config-schema-series-compatibility.md +++ b/context/plans/config-schema-series-compatibility.md @@ -493,7 +493,7 @@ Persist this field in every plan; this is durable plan state, not chat state: - Context impact: localized, user-visible. `sce setup` output now starts with a `Config schema:` section whenever a canonical file's `$schema` is upgraded or skipped. Durable documentation of the section is scheduled for T08. - Context synchronization: synced -- [ ] T07: `Add an end-to-end setup schema-upgrade scenario check` (status:todo) +- [x] T07: `Add an end-to-end setup schema-upgrade scenario check` (status:done) - Task ID: T07 - Scope: - In: @@ -511,7 +511,33 @@ Persist this field in every plan; this is durable plan state, not chat state: - Verify: - `nix develop -c ./scripts/test-setup-schema-upgrade.sh` - `nix flake check` - - Context synchronization: pending + - Completed: 2026-11-10 + - Files changed: + - `scripts/test-setup-schema-upgrade.sh` (new, staged intent-to-add so the flake source includes it) + - `flake.nix` + - Result: + - The new script runs a built `sce` (`--sce-bin`, `$SCE_BIN`, or `sce` on `PATH`). It isolates `HOME`, `XDG_CONFIG_HOME`, `XDG_STATE_HOME`, `XDG_CACHE_HOME`, and the Git global config. Each phase uses a fresh temporary Git repo with an `https://example.com/acme/demo.git` `origin`. + - The global config path follows the CLI's platform resolution: `$XDG_CONFIG_HOME/sce/config.json`, or `~/Library/Application Support/sce/config.json` on Darwin. + - The current version, current URL, and newer series (`major.(minor+1).0`) are read from `sce version`. The only literal series in the script is the `0.3.3` fixture. + - Phases: + 1. **Read-only.** For older (`0.3.3`), newer, and foreign declarations in both files, `sce version`, `config validate`, `config show`, `doctor`, and `doctor --fix` leave both sha256 hashes unchanged. Every command except `version` prints the matching `$schema` message. + 2. **Upgrade.** `sce setup --hooks --repo` reports both `upgraded …` lines. Each file equals its original with only the URL swapped, with exactly one changed line, using a fixture with `auth.mode`, `telemetry.enabled`, `integrations.target`, custom `policies.bash`, and irregular whitespace. `sce config validate` then reports valid. + 3. **Idempotency.** A second run prints no `Config schema:` section and leaves both hashes unchanged. + 4. **Mixed.** With an unknown-key stale local file and a valid stale global file, the local file is byte-identical and its candidate-invalid line names `not_a_real_key` and `rerun 'sce setup'`. The global file is upgraded with only `$schema` changed. + 5. **Newer.** The local file is byte-identical, the newer-series line is reported, and no global config is created. + 6. **Symlink.** A global config symlinked to a `dotfiles` target stays a symlink with the same target and target bytes, and is reported with the symlink line. + - `flake.nix` adds a `setupSchemaUpgradeCheck` `runCommand`, which runs the script against `scePackage` with `bash`, `coreutils`, `diffutils`, `gnugrep`, `gnused`, and `git`, and exposes it as `checks..setup-schema-upgrade`. + - Approved assumptions: + - Read-only commands may exit non-zero on a stale config. The script asserts on their output and hashes, not their exit codes. Setup must exit 0. + - `nix develop -c ./scripts/test-setup-schema-upgrade.sh` needs a built `sce` on `PATH` or `SCE_BIN`. The dev shell does not provide one, so the local run put `cli/target/debug` on `PATH`. + - Verify: + - `nix develop -c ./scripts/test-setup-schema-upgrade.sh` (with `cli/target/debug` on `PATH`): passed (all 6 phases). + - `nix flake check`: passed ("all checks passed!", running on x86_64-linux only; Darwin and aarch64 systems were omitted by default). + - Supplementary checks: + - `nix build .#checks.x86_64-linux.setup-schema-upgrade -L`: passed in the Nix sandbox (all 6 phases). + - Negative control: against a stale pre-T02 `cli/target/release/sce` (exact-version `const`), the script failed at the first read-only assertion, as expected. + - Context impact: localized. This adds a test-only script and a flake check, with no runtime behavior change. A new check name now appears in `nix flake check`. + - Context synchronization: synced - [ ] T08: `Document series compatibility and setup schema upgrades` (status:todo) - Task ID: T08 diff --git a/context/sce/setup-repo-local-config-bootstrap.md b/context/sce/setup-repo-local-config-bootstrap.md index c07d30a05..b84823830 100644 --- a/context/sce/setup-repo-local-config-bootstrap.md +++ b/context/sce/setup-repo-local-config-bootstrap.md @@ -43,6 +43,7 @@ Task `setup-repo-gate-and-local-config-bootstrap` T02, `turso-local-db-sync` T04 - After both repository preflights (`ensure_git_repository` and the effective named-remote URL check), setup consumes the shared resolver's degraded result for an invalid default-discovered repo-local `.sce/config.json`; the file remains untouched and the run continues through prompts, context baseline bootstrap, lifecycle providers, hooks, and target assets. An absent config continues through the normal bootstrap path, while explicit config selections remain fatal. - Between the Git-repo gate and the named-remote check, normal setup modes (not context-only) plan a `$schema`-only upgrade of the canonical repository `.sce/config.json` and, when its path resolves, the default global config (`cli/src/services/setup/schema_upgrade.rs`). An explicit `--config` / `SCE_CONFIG_FILE` file is never planned, and a missing file is never created. The named-remote check resolves `agent_trace.repository_remote` through a read overlay that serves each validated older-series candidate, so a stale but otherwise valid local file selects its configured remote. The planned upgrades are committed only after that check passes, so a failed Git or remote preflight writes nothing. Dispatch, access prompts and the access gate, the access writer, lifecycle bootstrap, and target persistence all read the committed files. Each file is upgraded or skipped independently: an invalid candidate leaves its file byte-identical and degraded as before, without blocking the other file. - Setup reports these outcomes in a `Config schema:` section, which is the first section of the setup output and also comes before the cancellation text when a prompt is cancelled. It lists one line for each upgraded or skipped canonical file and is omitted when every file is missing, has no declaration, or already declares the current series. An upgrade prints `upgraded '' from v to v`. A skipped file prints `left '' unchanged; `, where the reason is one of: a newer series (upgrade the SCE CLI); a non-canonical `$schema` (set it to the current URL); an invalid candidate (the validation errors, then fix the fields and rerun `sce setup`); a symlink-managed file (update the file's source directly); a concurrent modification (rerun `sce setup`); or an I/O failure. Skips never fail setup; the exit code stays successful. +- `scripts/test-setup-schema-upgrade.sh` (root flake check `setup-schema-upgrade`) proves this contract end to end against a built `sce` with isolated `HOME`/XDG directories. It checks that `version`, `config validate`, `config show`, `doctor`, and `doctor --fix` leave older, newer, and foreign declarations byte-identical. It also covers setup's `$schema`-only upgrade of both canonical files, idempotency, an invalid local file next to an upgradable global file, the no-downgrade rule for newer declarations, and refusal of a symlinked global file. - Config/DB bootstrap runs after those preflights and after context baseline bootstrap, before config/hooks dispatch, so it applies to all normal setup modes: config-only, hooks-only, combined, and interactive. If the repo-local config is invalid, target and optional-workflow persistence is skipped rather than rewriting it. ## Repository access preferences diff --git a/flake.nix b/flake.nix index 59f4d506e..0cd6e3689 100644 --- a/flake.nix +++ b/flake.nix @@ -1438,6 +1438,19 @@ checkCommand = "bash ./scripts/test-codex-hook-command.sh"; }; + setupSchemaUpgradeCheck = + pkgs.runCommand "setup-schema-upgrade-check" + { + nativeBuildInputs = [ pkgs.bash pkgs.coreutils pkgs.diffutils pkgs.gnugrep pkgs.gnused pkgs.git ]; + } + '' + set -euo pipefail + + bash ${./scripts/test-setup-schema-upgrade.sh} --sce-bin ${scePackage}/bin/sce + + mkdir -p "$out" + ''; + configLibBunTests = mkBunCheck { name = "config-lib-bun-tests"; src = configLibBashPolicySrc; @@ -1723,6 +1736,7 @@ cli-generated-input = cliGeneratedInputCheck; pkl-generated = pklGeneratedCheck; codex-hook-command = codexHookCommandCheck; + setup-schema-upgrade = setupSchemaUpgradeCheck; npm-bun-tests = npmTests; npm-pack-smoke = npmPackSmoke; diff --git a/scripts/test-setup-schema-upgrade.sh b/scripts/test-setup-schema-upgrade.sh new file mode 100755 index 000000000..1c2019f41 --- /dev/null +++ b/scripts/test-setup-schema-upgrade.sh @@ -0,0 +1,309 @@ +#!/usr/bin/env bash +set -euo pipefail + +sce_bin="${SCE_BIN:-sce}" + +usage() { + cat <<'EOF' +Usage: + scripts/test-setup-schema-upgrade.sh [--sce-bin ] + +End-to-end check for config `$schema` series compatibility and the +`sce setup` schema upgrade. Runs a built `sce` binary against an isolated +temporary Git repository and isolated HOME/XDG directories, then asserts: + + 1. read-only version, config validate/show, doctor, and doctor --fix never + modify older, newer, or foreign declarations and print the + actionable `$schema` messages + 2. upgrade setup changes exactly the `$schema` line in both canonical files + 3. idempotency a second setup leaves both files byte-identical and prints no + `Config schema:` section + 4. mixed an invalid stale local file stays byte-identical and is + reported while a valid stale global file is upgraded + 5. newer a newer-series local file stays byte-identical + 6. symlink a symlinked global config is reported and left untouched + +Flags: + --sce-bin Binary to invoke. Default: $SCE_BIN or sce + -h, --help Show this help text +EOF +} + +fail() { + printf 'Setup schema upgrade test failed: %s\n' "$1" >&2 + exit 1 +} + +while [[ $# -gt 0 ]]; do + case "$1" in + --sce-bin) + [[ -n "${2:-}" ]] || fail "--sce-bin requires a value" + sce_bin="$2" + shift 2 + ;; + -h|--help) + usage + exit 0 + ;; + *) + fail "unknown flag '$1'" + ;; + esac +done + +if ! command -v "$sce_bin" >/dev/null 2>&1; then + fail "sce binary '$sce_bin' was not found; set SCE_BIN or pass --sce-bin" +fi + +if [[ "$sce_bin" == */* ]]; then + sce_bin="$(realpath "$sce_bin")" +else + sce_bin="$(command -v "$sce_bin")" +fi + +tmp_dir="$(mktemp -d)" +cleanup() { + rm -rf "$tmp_dir" +} +trap cleanup EXIT + +export HOME="$tmp_dir/home" +export XDG_CONFIG_HOME="$tmp_dir/xdg-config" +export XDG_STATE_HOME="$tmp_dir/xdg-state" +export XDG_CACHE_HOME="$tmp_dir/xdg-cache" +export GIT_CONFIG_NOSYSTEM=1 +export GIT_CONFIG_GLOBAL="$tmp_dir/gitconfig" +unset SCE_CONFIG_FILE +mkdir -p "$HOME" "$XDG_CONFIG_HOME" "$XDG_STATE_HOME" "$XDG_CACHE_HOME" +: > "$GIT_CONFIG_GLOBAL" + +case "$(uname -s)" in + Darwin) global_config="$HOME/Library/Application Support/sce/config.json" ;; + *) global_config="$XDG_CONFIG_HOME/sce/config.json" ;; +esac + +current_version="$("$sce_bin" version 2>/dev/null | sed -n 's/^shared-context-engineering \([^ ]*\).*/\1/p')" +[[ -n "$current_version" ]] || fail "could not read the CLI version from 'sce version'" +current_major="${current_version%%.*}" +current_minor="${current_version#*.}" +current_minor="${current_minor%%.*}" + +schema_url() { + printf 'https://sce.crocoder.dev/v%s/config.json' "$1" +} + +current_url="$(schema_url "$current_version")" +older_url="$(schema_url 0.3.3)" +newer_url="$(schema_url "${current_major}.$((current_minor + 1)).0")" +foreign_url='https://example.com/s.json' + +older_message="declares SCE config schema version 0.3.3" +newer_message="which was written by a newer SCE release" +foreign_message="is not a canonical SCE schema URL" + +stale_local_config() { + cat <&1) || true +} + +run_setup() { + local repo="$1" + local output + if ! output="$(cd "$repo" && "$sce_bin" setup --hooks --repo "$repo" &1)"; then + printf '%s\n' "$output" >&2 + fail "'sce setup --hooks' exited non-zero in '$repo'" + fi + printf '%s\n' "$output" +} + +assert_contains() { + local haystack="$1" needle="$2" label="$3" + [[ "$haystack" == *"$needle"* ]] || { + printf '%s\n' "$haystack" >&2 + fail "$label: expected output to contain '$needle'" + } +} + +assert_not_contains() { + local haystack="$1" needle="$2" label="$3" + [[ "$haystack" != *"$needle"* ]] || { + printf '%s\n' "$haystack" >&2 + fail "$label: expected output not to contain '$needle'" + } +} + +assert_unchanged() { + local path="$1" expected_hash="$2" label="$3" + [[ "$(hash_of "$path")" == "$expected_hash" ]] || fail "$label: '$path' was modified" +} + +assert_only_schema_changed() { + local original="$1" upgraded="$2" label="$3" + local expected + expected="$(mktemp "$tmp_dir/expected.XXXXXX")" + sed "s|$older_url|$current_url|" "$original" > "$expected" + cmp -s "$expected" "$upgraded" || { + diff "$original" "$upgraded" >&2 || true + fail "$label: '$upgraded' differs from the original by more than the \$schema value" + } + [[ "$(diff "$original" "$upgraded" | grep -c '^[<>]')" == 2 ]] \ + || fail "$label: expected exactly one changed line in '$upgraded'" +} + +printf 'Testing sce %s (accepts %s.%s.x) at %s\n' "$current_version" "$current_major" "$current_minor" "$sce_bin" + +# Phase 1: read-only commands never modify older, newer, or foreign declarations. +for variant in older newer foreign; do + case "$variant" in + older) url="$older_url"; message="$older_message" ;; + newer) url="$newer_url"; message="$newer_message" ;; + foreign) url="$foreign_url"; message="$foreign_message" ;; + esac + + repo="$(new_repo "readonly-$variant")" + reset_global + stale_local_config "$url" > "$repo/.sce/config.json" + stale_global_config "$url" > "$global_config" + local_hash="$(hash_of "$repo/.sce/config.json")" + global_hash="$(hash_of "$global_config")" + + for command in "version" "config validate" "config show" "doctor" "doctor --fix"; do + # shellcheck disable=SC2086 + output="$(run_sce "$repo" $command)" + label="read-only ($variant): sce $command" + assert_unchanged "$repo/.sce/config.json" "$local_hash" "$label" + assert_unchanged "$global_config" "$global_hash" "$label" + case "$command" in + "config validate"|"config show"|doctor*) assert_contains "$output" "$message" "$label" ;; + esac + done +done +printf 'ok: read-only commands left older, newer, and foreign declarations unchanged\n' + +# Phase 2: setup upgrades exactly the $schema line in both canonical files. +repo="$(new_repo upgrade)" +reset_global +stale_local_config "$older_url" > "$repo/.sce/config.json" +stale_global_config "$older_url" > "$global_config" +cp "$repo/.sce/config.json" "$tmp_dir/upgrade-local.orig" +cp "$global_config" "$tmp_dir/upgrade-global.orig" + +output="$(run_setup "$repo")" +assert_contains "$output" "Config schema:" "upgrade" +assert_contains "$output" "upgraded '$repo/.sce/config.json' from v0.3.3 to v$current_version" "upgrade" +assert_contains "$output" "upgraded '$global_config' from v0.3.3 to v$current_version" "upgrade" +assert_only_schema_changed "$tmp_dir/upgrade-local.orig" "$repo/.sce/config.json" "upgrade" +assert_only_schema_changed "$tmp_dir/upgrade-global.orig" "$global_config" "upgrade" + +output="$(run_sce "$repo" config validate)" +assert_contains "$output" "SCE config validation: valid" "upgrade: sce config validate" +printf 'ok: setup upgraded only the $schema line in both canonical files\n' + +# Phase 3: a second setup run is a no-op for both files. +local_hash="$(hash_of "$repo/.sce/config.json")" +global_hash="$(hash_of "$global_config")" +output="$(run_setup "$repo")" +assert_not_contains "$output" "Config schema:" "idempotency" +assert_unchanged "$repo/.sce/config.json" "$local_hash" "idempotency" +assert_unchanged "$global_config" "$global_hash" "idempotency" +printf 'ok: second setup run left both files byte-identical\n' + +# Phase 4: an invalid stale local file does not block upgrading a valid stale global file. +repo="$(new_repo mixed)" +reset_global +cat > "$repo/.sce/config.json" < "$global_config" +local_hash="$(hash_of "$repo/.sce/config.json")" +cp "$global_config" "$tmp_dir/mixed-global.orig" + +output="$(run_setup "$repo")" +assert_unchanged "$repo/.sce/config.json" "$local_hash" "mixed" +assert_only_schema_changed "$tmp_dir/mixed-global.orig" "$global_config" "mixed" +assert_contains "$output" "upgraded '$global_config' from v0.3.3 to v$current_version" "mixed" +assert_contains "$output" "left '$repo/.sce/config.json' unchanged; upgrading '\$schema' to v$current_version would fail validation:" "mixed" +assert_contains "$output" "not_a_real_key" "mixed" +assert_contains "$output" "rerun 'sce setup'" "mixed" +printf 'ok: invalid stale local file was reported and kept while the global file was upgraded\n' + +# Phase 5: setup never downgrades a newer-series declaration. +repo="$(new_repo newer)" +reset_global +stale_local_config "$newer_url" > "$repo/.sce/config.json" +local_hash="$(hash_of "$repo/.sce/config.json")" + +output="$(run_setup "$repo")" +assert_unchanged "$repo/.sce/config.json" "$local_hash" "newer" +assert_contains "$output" "left '$repo/.sce/config.json' unchanged; it declares newer schema series ${current_major}.$((current_minor + 1))" "newer" +[[ ! -e "$global_config" ]] || fail "newer: setup created a missing global config" +printf 'ok: newer-series local file was left byte-identical\n' + +# Phase 6: a symlinked global config is reported and its target left untouched. +repo="$(new_repo symlink)" +reset_global +mkdir -p "$tmp_dir/dotfiles" +stale_global_config "$older_url" > "$tmp_dir/dotfiles/sce-config.json" +ln -s "$tmp_dir/dotfiles/sce-config.json" "$global_config" +target_hash="$(hash_of "$tmp_dir/dotfiles/sce-config.json")" + +output="$(run_setup "$repo")" +[[ -L "$global_config" ]] || fail "symlink: global config is no longer a symlink" +[[ "$(readlink "$global_config")" == "$tmp_dir/dotfiles/sce-config.json" ]] \ + || fail "symlink: global config symlink target changed" +assert_unchanged "$tmp_dir/dotfiles/sce-config.json" "$target_hash" "symlink" +assert_contains "$output" "left '$global_config' unchanged; it is a symlink" "symlink" +printf 'ok: symlinked global config was reported and left untouched\n' + +printf 'Setup schema upgrade test passed.\n' From bc89e48bb0909d81973b5e8200627b51f362f46d Mon Sep 17 00:00:00 2001 From: David Abram Date: Tue, 10 Nov 2026 14:05:30 +0100 Subject: [PATCH 9/9] context: Document config schema series upgrades and complete T08 Document compatibility-series acceptance and setup-only upgrades for older-series config declarations, including canonical-file scope, validation, no-downgrade, and read-only safeguards. Record the decision and synchronize the related context, glossary, publication, and setup documentation. Plan: config-schema-series-compatibility.md Task: T08 Co-authored-by: SCE --- context/cli/config-precedence-contract.md | 4 +- context/context-map.md | 2 + ...tup-upgrades-config-schema-declarations.md | 60 +++++++++++++ context/glossary.md | 1 + .../config-schema-series-compatibility.md | 89 ++++++++++++++++--- context/sce/config-schema-publication.md | 10 ++- .../sce/setup-repo-local-config-bootstrap.md | 2 +- 7 files changed, 153 insertions(+), 15 deletions(-) create mode 100644 context/decisions/2026-11-10-setup-upgrades-config-schema-declarations.md diff --git a/context/cli/config-precedence-contract.md b/context/cli/config-precedence-contract.md index 05df9046e..8e4e16d8d 100644 --- a/context/cli/config-precedence-contract.md +++ b/context/cli/config-precedence-contract.md @@ -55,7 +55,7 @@ Supported auth-adjacent runtime keys can participate in one shared key-declared 2. config file value (`control_plane_base_url`) 3. baked default (`https://sce.crocoderlab.dev`) -The control-plane default is separate from the config schema declaration (`https://sce.crocoder.dev/config.json`) and from the SCE web URL owner used for Agent Trace links. +The control-plane default is separate from the config schema declaration (`https://sce.crocoder.dev/v/config.json`) and from the SCE web URL owner used for Agent Trace links. When a supported auth-adjacent key omits a baked default, the same resolver still reports `value: null` / `(unset)` with no resolved source when both env and config inputs are absent. @@ -84,7 +84,7 @@ When a default-discovered global or repo-local config file exists but fails JSON - Older series: `declares SCE config schema version 0.3.3; this CLI uses 0.4.0 and accepts 0.4.x; run 'sce setup' to upgrade the repository and global config, or set `$schema` to `. - Newer series: `declares SCE config schema version , which was written by a newer SCE release; …; upgrade the SCE CLI`. - Malformed, foreign, or non-string: `is not a canonical SCE schema URL; use `. -- When every collected discovered-config validation error is only an older-series `$schema` issue, the startup diagnostic says the local or discovered config "declares an older config schema series" and tells the user to run `sce setup`; any other failure keeps the generic "is invalid … run `sce config validate`" guidance. Validation itself never rewrites the file. +- When every collected discovered-config validation error is only an older-series `$schema` issue, the startup diagnostic says the local or discovered config "declares an older config schema series" and tells the user to run `sce setup`; any other failure keeps the generic "is invalid … run `sce config validate`" guidance. Validation itself never rewrites the file. `sce config validate`, `sce doctor` (including `--fix`), and command startup leave older, newer, and foreign declarations byte-identical. Only `sce setup` upgrades an older-series declaration, and it changes only `$schema` in the canonical repository and global files ([setup schema upgrade decision](../decisions/2026-11-10-setup-upgrades-config-schema-declarations.md)). - Startup/runtime config resolution degrades gracefully for default-discovered files across ordinary startup, setup, and Agent Trace storage: invalid discovered files are skipped and reported via collected `validation_errors`, while explicit `--config` / `SCE_CONFIG_FILE` targets still fail immediately on the same parse or validation errors. - Config file content must be valid JSON with a top-level object. diff --git a/context/context-map.md b/context/context-map.md index ddc4178b2..14fa28a40 100644 --- a/context/context-map.md +++ b/context/context-map.md @@ -148,6 +148,8 @@ Supporting repo docs: Recent decision records: +- `context/decisions/2026-11-10-setup-upgrades-config-schema-declarations.md` (`sce setup` is the only writer that upgrades an older-series config `$schema` declaration. It covers only the canonical repository and global files, changes only `$schema`, validates the candidate first, never downgrades, does no field migrations, and reports skips without failing setup. It amends the read-only global-config statement of `2026-10-22-setup-writes-access-answers-to-repository-config.md`) +- `context/decisions/2026-11-10-config-schema-accepts-release-series-declarations.md` (the config schema accepts canonical `$schema` declarations within the CLI's `major.minor` series, derived from `.version`; the `$id` stays the concrete versioned URL) - `context/decisions/2026-10-22-repository-telemetry-enabled-grants-consent.md` (a `telemetry.enabled: true` in any layer, including the repository `.sce/config.json`, grants telemetry consent unless some layer opts out; removes `no_user_consent` and its warning; supersedes only the user-side-only consent rule of `2026-10-20-consent-gated-access-policy-precedence.md`) - `context/decisions/2026-10-22-setup-writes-access-answers-to-repository-config.md` (interactive setup writes `auth.mode` / `telemetry.enabled` only to the repository `.sce/config.json`, atomically and before the target install; repository values are saved answers, not locks, while global/explicit `disabled` and `mandatory` lock the prompts and login offer; supersedes `2026-10-21-setup-writes-access-answers-to-user-layer-only.md`) - `context/decisions/2026-10-21-setup-writes-access-answers-to-user-layer-only.md` (superseded by `2026-10-22-setup-writes-access-answers-to-repository-config.md`; retained as history: interactive setup wrote `auth.mode` / `telemetry.enabled` only to the global user config, atomically and before any repository write, never to repository or explicit config files; resolver precedence unchanged) diff --git a/context/decisions/2026-11-10-setup-upgrades-config-schema-declarations.md b/context/decisions/2026-11-10-setup-upgrades-config-schema-declarations.md new file mode 100644 index 000000000..e10bb63f6 --- /dev/null +++ b/context/decisions/2026-11-10-setup-upgrades-config-schema-declarations.md @@ -0,0 +1,60 @@ +# Decision: Setup upgrades older-series config `$schema` declarations + +Date: 2026-11-10 +Status: Accepted +Plan: `context/plans/config-schema-series-compatibility.md` +Task: `T08` + +## Context + +The SCE config schema accepts `$schema` declarations within the CLI's `major.minor` release series ([series-acceptance decision](2026-11-10-config-schema-accepts-release-series-declarations.md)). A file that declares an older series is still rejected. In a default-discovered file, that rejection discards every setting in the file. Users need one supported way to move a stale declaration forward that does not hand-edit files or silently accept other series. + +Setup previously never revisited an existing declaration. It treated the global config as read-only ([repository access answers decision](2026-10-22-setup-writes-access-answers-to-repository-config.md)). + +## Decision + +`sce setup` is the only writer that upgrades a config `$schema` declaration. + +- **Acceptance stays by series.** Validation, `sce config validate`, `sce doctor` (including `--fix`), and command startup stay read-only. They report an actionable `$schema` message and never rewrite a file. +- **Canonical files only.** Setup considers only the existing repository `.sce/config.json` and the existing default global config. It never creates a missing file to upgrade it, and it never writes an explicit `--config` / `SCE_CONFIG_FILE` file at a non-canonical path. +- **`$schema` only.** For an older-series declaration, the candidate changes only the `$schema` value to the CLI's concrete URL. The global config write is limited to that value. This amends the read-only global-config statement in the repository access answers decision. Access answers still go only to the repository config. +- **Validate before saving.** The candidate is validated through `parse_file_config` with the file's own source. It is saved through the guarded reread-compare, symlink-refusing, atomic write. +- **No downgrade.** Same-series, newer-series, malformed, foreign, and absent declarations are never written. +- **No field migrations.** An older-series file whose other fields fail validation stays byte-identical. +- **Non-fatal reporting.** Setup reports each upgrade or skip in a `Config schema:` section and exits successfully. A still-invalid file keeps the existing degraded behavior. +- **Ordering.** Upgrades are planned after the Git-repo gate and committed only after the named-remote preflight passes. The remote check reads the validated candidates. + +## Rationale + +Setup already owns repository bootstrap writes, and the user runs it deliberately. Keeping validation and doctor read-only means no ordinary command mutates config as a side effect. Limiting the global write to `$schema` keeps user-owned settings untouched. Validating the full candidate keeps unrelated field changes from slipping in through a `$schema` bump. + +## Alternatives considered + +- **Upgrade from `sce doctor --fix` or `sce config`.** Not selected, because those paths stay read-only. +- **Leniently accept other series.** Not selected, because it would silently accept incompatible declarations. +- **A general field-migration framework.** Not selected, because it is out of scope for a declaration bump. +- **Fail setup when an upgrade is skipped.** Not selected, because a dotfile-managed (symlinked) global config should not block repository setup. + +## Compatibility and risks + +- Historical `schema/v*/config.json` snapshots are left unchanged. The next `nix run .#bump-version` publishes the first series-pattern snapshot. +- A symlinked config is refused and must be updated at its source. + +## Guardrails + +- The Git-repo gate and named-remote preflight run before every setup write. +- Context-only setup (`--bootstrap-context`) performs no config work. + +## Consequences + +- A series bump no longer requires hand-editing config files. Running `sce setup` upgrades both canonical files. +- Setup can now write the global config, but only its `$schema` value. + +## References + +- Plan: [`config-schema-series-compatibility`](../plans/config-schema-series-compatibility.md) +- Current-state context: [`setup-repo-local-config-bootstrap.md`](../sce/setup-repo-local-config-bootstrap.md) +- Current-state context: [`config-schema-publication.md`](../sce/config-schema-publication.md) +- Current-state context: [`config-precedence-contract.md`](../cli/config-precedence-contract.md) +- Evidence: [`schema_upgrade.rs`](../../cli/src/services/setup/schema_upgrade.rs) +- Evidence: [`test-setup-schema-upgrade.sh`](../../scripts/test-setup-schema-upgrade.sh) diff --git a/context/glossary.md b/context/glossary.md index 942bece85..2caef953a 100644 --- a/context/glossary.md +++ b/context/glossary.md @@ -163,6 +163,7 @@ - `config runtime resolver seam`: Canonical ownership in `cli/src/services/config/resolver.rs` for config-file discovery, file-layer merging, env/flag/default precedence resolution, shared auth-key resolution (`workos_client_id`, `control_plane_base_url`), observability runtime resolution, attribution-hooks gate resolution, and default-discovered invalid-file degradation; `control_plane_base_url` defaults to `https://sce.crocoderlab.dev` while the schema and SCE web URL remain `https://sce.crocoder.dev`; `cli/src/services/config/mod.rs` delegates `sce config show|validate` runtime resolution to this seam while facade re-exports preserve startup/auth/hooks callers through `services::config`. - `config render seam`: Canonical ownership in `cli/src/services/config/render.rs` for `sce config show` and `sce config validate` text/JSON output construction, including rendering-specific config-path formatting, resolved-value formatting, validation issue/warning rendering, and auth display-value redaction/abbreviation helpers; `cli/src/services/config/mod.rs` delegates rendering to this private submodule after resolver-owned runtime config resolution. - `sce config schema artifact`: Canonical JSON Schema for global and repo-local `sce/config.json` files, authored in `config/pkl/base/sce-config-schema.pkl`, generated to payload-relative `config/schema/sce-config.schema.json` beneath Cargo `OUT_DIR` or packaging fallback roots, and embedded by `cli/src/services/config/schema.rs` for shared `sce config validate` and doctor config validation. The current schema accepts any canonical `$schema` declaration in the CLI's `major.minor` config schema compatibility series (derived from `.version`), flat logging keys (`log_level`, `log_format`, `log_dir`, positive-integer `log_file_retention_limit`), and existing auth/config keys. +- `config schema compatibility series`: The `major.minor` part of the CLI version from `.version`. Any canonical `https://sce.crocoder.dev/v..[-pre][+build]/config.json` declaration in that series is accepted. `config/pkl/base/sce-config-schema-url.pkl` derives the schema's series `pattern`, and `cli/src/services/config/schema_declaration.rs` classifies declarations as absent, current, older, newer, or unrecognized. Only `sce setup` upgrades an older-series declaration, and it changes only `$schema` in the canonical repository and global config. See `context/decisions/2026-11-10-setup-upgrades-config-schema-declarations.md`. - `log_file_retention_limit`: Flat top-level `sce/config.json` key validated as an integer with minimum `1`. It has config-file > default precedence only, defaults to `10`, merges global before local, appears in `sce config show` with provenance, and is carried by `ResolvedObservabilityRuntimeConfig`; the concrete logger stores the resolved value and uses it for primary and v2 creation-triggered cleanup. - `bash tool policy config surface`: Nested repo config namespace under `.sce/config.json` at `policies.bash`, currently supporting unique built-in `presets` plus repo-owned `custom` argv-prefix rules with deterministic validation, merged global/local resolution, and first-class `sce config show|validate` reporting. - `attribution hooks gate`: Enabled-by-default local hook runtime gate resolved through shared config precedence in `cli/src/services/config/mod.rs` (with parsing in `schema.rs`): opt-out env `SCE_ATTRIBUTION_HOOKS_DISABLED` overrides repo/global config key `policies.attribution_hooks.enabled` with inverted semantics, and the current enabled path activates commit-msg-only attribution gated by the staged-diff AI-overlap preflight. diff --git a/context/plans/config-schema-series-compatibility.md b/context/plans/config-schema-series-compatibility.md index d481e1fac..973fc17c8 100644 --- a/context/plans/config-schema-series-compatibility.md +++ b/context/plans/config-schema-series-compatibility.md @@ -90,21 +90,21 @@ How this plan is proven complete. Each criterion is observable and names the check that proves it. `/validate` runs these checks; no task in the stack performs final validation. -- [ ] AC1: The embedded config schema accepts every canonical `https://sce.crocoder.dev/v..[-pre][+build]/config.json` declaration in the CLI's `major.minor` series. It rejects other series, malformed or foreign URLs, and non-string values. Every other field is still validated. +- [x] AC1: The embedded config schema accepts every canonical `https://sce.crocoder.dev/v..[-pre][+build]/config.json` declaration in the CLI's `major.minor` series. It rejects other series, malformed or foreign URLs, and non-string values. Every other field is still validated. - Validate: `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema_declaration`, which runs the embedded-schema compatibility matrix and the classifier parity tests, plus the `$schema`-valid/other-field-invalid case. -- [ ] AC2: The accepted series comes from `.version` alone. Changing the release series changes the generated `$schema` rule and the Rust classifier without editing any other constant. +- [x] AC2: The accepted series comes from `.version` alone. Changing the release series changes the generated `$schema` rule and the Rust classifier without editing any other constant. - Validate: `nix run .#pkl-check-generated`, which runs the Pkl series-pattern fixtures for `0.4.0-pre-alpha-v14`, `0.5.0`, and `1.4.2`. Also confirm that `nix shell nixpkgs#ripgrep -c rg -n '0\\.4' config/pkl/base/sce-config-schema*.pkl cli/src/services/config/schema_declaration.rs` finds no hard-coded series outside test fixtures. -- [ ] AC3: `sce config validate`, `sce doctor` (including `--fix`), and ordinary command startup never modify a config file with an older, newer, or foreign declaration. Each prints an actionable `$schema` message. +- [x] AC3: `sce config validate`, `sce doctor` (including `--fix`), and ordinary command startup never modify a config file with an older, newer, or foreign declaration. Each prints an actionable `$schema` message. - Validate: the T07 end-to-end scenario's read-only phase compares file hashes before and after each command. -- [ ] AC4: `sce setup` upgrades an older-series declaration in the existing repository `.sce/config.json` and the existing global config. Only the `$schema` value changes, and every other byte is preserved, including auth policy, telemetry consent, integrations, and custom policies. A second run leaves both files byte-identical. +- [x] AC4: `sce setup` upgrades an older-series declaration in the existing repository `.sce/config.json` and the existing global config. Only the `$schema` value changes, and every other byte is preserved, including auth policy, telemetry consent, integrations, and custom policies. A second run leaves both files byte-identical. - Validate: `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema_upgrade` plus the T07 scenario's upgrade and idempotency phases. -- [ ] AC5: Setup never writes a newer-series, malformed, foreign, absent, or same-series declaration. It never creates a missing global config for a schema upgrade, and it never writes an explicit `--config`/`SCE_CONFIG_FILE` file at a non-canonical path. +- [x] AC5: Setup never writes a newer-series, malformed, foreign, absent, or same-series declaration. It never creates a missing global config for a schema upgrade, and it never writes an explicit `--config`/`SCE_CONFIG_FILE` file at a non-canonical path. - Validate: `schema_upgrade` unit matrix plus the `setup` command tests from T05. -- [ ] AC6: When an older-series candidate fails validation, the file stays byte-identical and setup reports the path, the validation errors, and the remediation. Any other canonical file is still upgraded. +- [x] AC6: When an older-series candidate fails validation, the file stays byte-identical and setup reports the path, the validation errors, and the remediation. Any other canonical file is still upgraded. - Validate: `schema_upgrade` candidate-invalid test plus the T07 mixed-file phase. -- [ ] AC7: Upgrade writes keep the existing protections. A symlinked file, whether its target is valid or dangling, is refused and left untouched. A file whose bytes change between planning and commit is refused. The write is atomic and keeps the file mode. +- [x] AC7: Upgrade writes keep the existing protections. A symlinked file, whether its target is valid or dangling, is refused and left untouched. A file whose bytes change between planning and commit is refused. The write is atomic and keeps the file mode. - Validate: `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema_upgrade` (symlink, dangling-symlink, concurrent-modification, and mode cases) and `… test … access_preferences`, which must stay unchanged after the T03 refactor. -- [ ] AC8: An older-series declaration does not change setup's preflights. The remote preflight checks the remote configured in a stale but otherwise valid local file. Access prompts, the access gate, and saved selections come from the upgraded files. A failed Git or remote preflight writes nothing. +- [x] AC8: An older-series declaration does not change setup's preflights. The remote preflight checks the remote configured in a stale but otherwise valid local file. Access prompts, the access gate, and saved selections come from the upgraded files. A failed Git or remote preflight writes nothing. - Validate: `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml setup` (T05 ordering tests). ### Full validation @@ -539,7 +539,7 @@ Persist this field in every plan; this is durable plan state, not chat state: - Context impact: localized. This adds a test-only script and a flake check, with no runtime behavior change. A new check name now appears in `nix flake check`. - Context synchronization: synced -- [ ] T08: `Document series compatibility and setup schema upgrades` (status:todo) +- [x] T08: `Document series compatibility and setup schema upgrades` (status:done) - Task ID: T08 - Scope: - In: @@ -560,7 +560,38 @@ Persist this field in every plan; this is durable plan state, not chat state: - The ADR is linked from the setup bootstrap and schema publication files. - Verify: - `nix shell nixpkgs#ripgrep -c rg -n 'const.*config.json|global config is only read|sce.crocoder.dev/config.json' context --glob '!context/plans/**' --glob '!context/decisions/**'` returns no stale claims. - - Context synchronization: pending + - Completed: 2026-11-10 + - Files changed: + - `context/decisions/2026-11-10-setup-upgrades-config-schema-declarations.md` (new) + - `context/cli/config-precedence-contract.md` + - `context/sce/config-schema-publication.md` + - `context/sce/setup-repo-local-config-bootstrap.md` + - `context/glossary.md` + - Result: + - The new ADR records: + - series acceptance; + - `sce setup` as the only `$schema` writer, limited to the canonical repository and global files; + - the global write limited to `$schema`, which amends the read-only global statement in the 2026-10-22 access-answers ADR without changing where access answers go; + - candidate validation before the guarded atomic write; + - no downgrade; + - no field migrations; + - non-fatal reporting and preflight ordering; + - historical snapshots left unchanged. + - `config-precedence-contract.md`: + - The drifted unversioned URL becomes `https://sce.crocoder.dev/v/config.json`. + - It now states that validate, doctor (including `--fix`), and startup leave declarations byte-identical and that only setup upgrades them, with an ADR link. + - `config-schema-publication.md` documents setup's older-series `$schema`-only upgrade and no-downgrade rule, with an ADR link. It states that historical exact-`const` snapshots are never rewritten and that the next `bump-version` publishes the first series-pattern snapshot. + - `setup-repo-local-config-bootstrap.md` links the ADR from the schema-upgrade ordering bullet. The "global config is only read" claim was already replaced during T05/T06 synchronization. + - `glossary.md` adds the `config schema compatibility series` entry. + - Approved assumptions: + - The remaining verify-grep hits are false positives: `context/context-map.md:27` ("construction … sce/config.json") and `context/patterns.md:159` ("constructors … sce/config.json"). + - The publication file's statement that pre-rule snapshots keep their exact-version `const` is accurate history, not a current-schema claim, so it stays. + - Commit `77507bb0` (T07's staged changes) landed outside this task during execution. It is not part of this task's changes. + - Verify: + - The verify grep ran (exit 0). It found no stale exact-`const`, read-only-global, or unversioned-declaration claims. Only the two false positives above matched. + - Supplementary: every relative link in the new ADR resolves. The ADR is linked from `setup-repo-local-config-bootstrap.md`, `config-schema-publication.md`, `config-precedence-contract.md`, and `glossary.md`. + - Context impact: localized. This task changes documentation only, with no code or behavior change. It adds one ADR and one glossary term. + - Context synchronization: synced ## Open questions @@ -568,3 +599,41 @@ None. The one judgment call is that schema-upgrade failures are reported but not fatal. It is recorded under Assumptions with its rationale. It follows the documented degraded-invalid-config setup behavior. Revise the plan if setup should instead exit non-zero when a stale canonical file cannot be upgraded. + +## Validation Report + +**Status:** validated +**Date:** 2026-11-10 + +### Commands run + +- `nix flake check` -> exit 0 ("all checks passed!" on x86_64-linux, including `checks.x86_64-linux.setup-schema-upgrade`; other systems omitted by default) +- `nix run .#pkl-check-generated` -> exit 0 (ephemeral Pkl generation passed: 145 files) +- `nix develop -c ./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` -> exit 0 (no warnings) +- `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml` -> exit 0 (844 passed, 0 failed) +- `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema_declaration` -> exit 0 (13 passed, 0 failed) +- `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml schema_upgrade` -> exit 0 (23 passed, 0 failed) +- `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml access_preferences` -> exit 0 (7 passed, 0 failed) +- `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml setup` -> exit 0 (108 passed, 0 failed) +- `nix shell nixpkgs#ripgrep -c rg -n '0\.4' config/pkl/base/sce-config-schema*.pkl cli/src/services/config/schema_declaration.rs` -> exit 0 (3 hits, all inside the `#[cfg(test)] mod schema_declaration_tests` fixtures at lines 315, 342, 345; none in the Pkl files) +- `nix develop -c ./scripts/test-setup-schema-upgrade.sh` (with freshly built `cli/target/debug` on `PATH`) -> exit 0 (all 6 phases passed) + +### Success-criteria verification + +- [x] AC1: Embedded schema accepts same-series canonical declarations and rejects other series, malformed/foreign URLs, and non-strings while validating other fields -> `schema_declaration` tests: 13 passed +- [x] AC2: Accepted series derives from `.version` alone -> `pkl-check-generated` passed (series-pattern fixtures); ripgrep found no hard-coded `0.4` series outside test fixtures +- [x] AC3: validate, doctor (incl. `--fix`), and startup never modify stale/newer/foreign files and print actionable `$schema` messages -> T07 script read-only phase passed (hashes unchanged) +- [x] AC4: setup upgrades older-series `$schema` only in both canonical files; second run byte-identical -> `schema_upgrade` 23 passed; T07 upgrade and idempotency phases passed +- [x] AC5: setup never writes newer/malformed/foreign/absent/same-series declarations, never creates a global config, never writes explicit non-canonical files -> `schema_upgrade` matrix and `setup` tests (108) passed; T07 newer phase passed +- [x] AC6: invalid older-series candidate leaves file byte-identical and is reported with errors and remediation while the other file upgrades -> `schema_upgrade` candidate-invalid test passed; T07 mixed phase passed +- [x] AC7: symlink, dangling-symlink, concurrent-modification refusal; atomic, mode-preserving writes -> `schema_upgrade` cases passed; `access_preferences` 7 passed; T07 symlink phase passed +- [x] AC8: stale declaration does not change setup preflights; failed preflight writes nothing -> `setup` tests (T05 ordering) 108 passed + +### Failed checks and follow-ups + +- None. + +### Residual risks + +- `nix flake check` ran on x86_64-linux only; Darwin and aarch64 checks (including the Darwin global-config path in the T07 script) were not exercised. +- The branch stacks on PR #351's head; it still needs a rebase onto `main` after #351 merges. diff --git a/context/sce/config-schema-publication.md b/context/sce/config-schema-publication.md index 09d3a4c07..c7b2b9627 100644 --- a/context/sce/config-schema-publication.md +++ b/context/sce/config-schema-publication.md @@ -20,9 +20,15 @@ schema, not the separate Agent Trace schema. URL and the series pattern from `.version`. Other series, malformed or foreign URLs, and non-string values are rejected. See [the series-acceptance decision](../decisions/2026-11-10-config-schema-accepts-release-series-declarations.md). -- Snapshots published before this rule keep their exact-version `const`. +- Snapshots published before this rule keep their exact-version `const` and are + never rewritten. The next `nix run .#bump-version` publishes the first + series-pattern snapshot. - `sce setup` writes that same versioned declaration into a newly created - repo-local `.sce/config.json`. + repo-local `.sce/config.json`. It also upgrades an older-series declaration + in the existing canonical repository and global config, and changes only + `$schema`. It never downgrades a newer series or rewrites an unrecognized + value. See + [the setup schema upgrade decision](../decisions/2026-11-10-setup-upgrades-config-schema-declarations.md). - The generated working-tree schema remains ephemeral. Its version is read from the root `.version` file during Pkl generation. Repository builds and packaged builds consume the validated generated payload through Cargo diff --git a/context/sce/setup-repo-local-config-bootstrap.md b/context/sce/setup-repo-local-config-bootstrap.md index b84823830..248bf8108 100644 --- a/context/sce/setup-repo-local-config-bootstrap.md +++ b/context/sce/setup-repo-local-config-bootstrap.md @@ -41,7 +41,7 @@ Task `setup-repo-gate-and-local-config-bootstrap` T02, `turso-local-db-sync` T04 - The parent `.sce/` directory is created via `fs::create_dir_all` if missing. - The setup flow also bootstraps the canonical local DB through `LocalDbLifecycle::setup` and the Agent Trace DB through `AgentTraceDbLifecycle::setup`; both use the shared `TursoDb` adapter. - After both repository preflights (`ensure_git_repository` and the effective named-remote URL check), setup consumes the shared resolver's degraded result for an invalid default-discovered repo-local `.sce/config.json`; the file remains untouched and the run continues through prompts, context baseline bootstrap, lifecycle providers, hooks, and target assets. An absent config continues through the normal bootstrap path, while explicit config selections remain fatal. -- Between the Git-repo gate and the named-remote check, normal setup modes (not context-only) plan a `$schema`-only upgrade of the canonical repository `.sce/config.json` and, when its path resolves, the default global config (`cli/src/services/setup/schema_upgrade.rs`). An explicit `--config` / `SCE_CONFIG_FILE` file is never planned, and a missing file is never created. The named-remote check resolves `agent_trace.repository_remote` through a read overlay that serves each validated older-series candidate, so a stale but otherwise valid local file selects its configured remote. The planned upgrades are committed only after that check passes, so a failed Git or remote preflight writes nothing. Dispatch, access prompts and the access gate, the access writer, lifecycle bootstrap, and target persistence all read the committed files. Each file is upgraded or skipped independently: an invalid candidate leaves its file byte-identical and degraded as before, without blocking the other file. +- Between the Git-repo gate and the named-remote check, normal setup modes (not context-only) plan a `$schema`-only upgrade of the canonical repository `.sce/config.json` and, when its path resolves, the default global config (`cli/src/services/setup/schema_upgrade.rs`). An explicit `--config` / `SCE_CONFIG_FILE` file is never planned, and a missing file is never created. The named-remote check resolves `agent_trace.repository_remote` through a read overlay that serves each validated older-series candidate, so a stale but otherwise valid local file selects its configured remote. The planned upgrades are committed only after that check passes, so a failed Git or remote preflight writes nothing. Dispatch, access prompts and the access gate, the access writer, lifecycle bootstrap, and target persistence all read the committed files. Each file is upgraded or skipped independently: an invalid candidate leaves its file byte-identical and degraded as before, without blocking the other file. See [the setup schema upgrade decision](../decisions/2026-11-10-setup-upgrades-config-schema-declarations.md). - Setup reports these outcomes in a `Config schema:` section, which is the first section of the setup output and also comes before the cancellation text when a prompt is cancelled. It lists one line for each upgraded or skipped canonical file and is omitted when every file is missing, has no declaration, or already declares the current series. An upgrade prints `upgraded '' from v to v`. A skipped file prints `left '' unchanged; `, where the reason is one of: a newer series (upgrade the SCE CLI); a non-canonical `$schema` (set it to the current URL); an invalid candidate (the validation errors, then fix the fields and rerun `sce setup`); a symlink-managed file (update the file's source directly); a concurrent modification (rerun `sce setup`); or an I/O failure. Skips never fail setup; the exit code stays successful. - `scripts/test-setup-schema-upgrade.sh` (root flake check `setup-schema-upgrade`) proves this contract end to end against a built `sce` with isolated `HOME`/XDG directories. It checks that `version`, `config validate`, `config show`, `doctor`, and `doctor --fix` leave older, newer, and foreign declarations byte-identical. It also covers setup's `$schema`-only upgrade of both canonical files, idempotency, an invalid local file next to an upgradable global file, the no-downgrade rule for newer declarations, and refusal of a symlinked global file. - Config/DB bootstrap runs after those preflights and after context baseline bootstrap, before config/hooks dispatch, so it applies to all normal setup modes: config-only, hooks-only, combined, and interactive. If the repo-local config is invalid, target and optional-workflow persistence is skipped rather than rewriting it.