Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions docs/audits/metadata-closure-retirement-2026-11-13.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Retirement of the last hand-listed metadata check

Reviewed revision `a5e8bf7ee4f29ba14826df1ce363bef13fbe2963`. This is the review
record supplied to the Engine's `maintain_evidence` operation for the retirement
below. It records what was checked before the retirement; it is not a coverage
attestation.

## Decision

On 2026-11-13 the repository owner decided to retire
`release-maintenance-policy/metadata-closure` and accepted that the coverage
certificates it invalidates are reissued afterwards, each after its own review.
Fifty-five sibling checks were retired on 2026-11-12; this one was held back
only because of that certificate effect.

## What is retired

| Item | Disposition |
| --- | --- |
| Check `release-maintenance-policy/metadata-closure` | Retired. It ran `metadata_graph` over a hand-typed list of 12 module paths. |
| Suite `release-maintenance-policy` | Retired by the Engine because it has no other check. |
| Registry entry for that suite | Removed with the suite. |
| Catalogue node `release-maintenance-policy` (`enforcement-suite`) | Removed with the suite. |

The suite required `release-reference-closure`, which stays registered with its
own checks. No registered suite requires `release-maintenance-policy`.

## Why the check is redundant

The check `canonical-corpus-metadata/canonical-corpus` validates every member of
`evaluation/standards-effectiveness/canonical-module-corpus.toml` through the same
`validate_module_metadata` call the retired check used.

- All 12 listed paths are among the 74 corpus members.
- Mutation test on a scratch copy of the reviewed revision: four defects
(a `Requires` target that does not exist, a removed `Level` line, a wrong
`Canonical owner` path, a `Specializes` target that does not exist) were
introduced one at a time into each of the 12 modules, 48 mutations in all.
The retired check failed 48 times. The whole-corpus check failed the same 48
times with the same diagnostic code each time. No defect was reported by the
retired check alone.

The retired check therefore could not fail for a reason the whole-corpus check
does not also report.

## Graph and certificate effect

- The removed node is neither the source nor the consumer of any policy-impact
relationship, and no relationship names `suite:release-maintenance-policy` as
its evidence owner. No relationship is removed or re-owned.
- The node catalogue digest is part of every coverage requirement. Removing a
node changes all 53 requirement IDs, so the 16 certificates that were current
at the reviewed revision become stale. The 8 certificates that were already
stale stay stale.
- Stale certificates are left in place (`prune_stale_certificates` is `false`).
A stale certificate is ignored when coverage is read; its policy unit reports
`review-required` until a new certificate is published for it.

## Counts

| | Before | After |
| --- | ---: | ---: |
| Registered suites | 24 | 23 |
| Checks | 67 | 66 |
| Catalogue nodes | 122 | 121 |
| Certificates current / stale | 16 / 8 | 0 / 24 |

## Engine operation

`maintain_evidence` with this file as evidence, first with `apply: false` and
then with `apply: true`. Plan: `retire_checks` naming the one check; no suite,
input, description, relationship or consumer entries;
`replacement_evidence_owner` `review:consumer` (unused, because no relationship
names the retired suite). The Engine's candidate verification passed with 23
suites and 66 checks.
97 changes: 97 additions & 0 deletions docs/audits/plan-format-check-removal-2026-11-13.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Removal of the plan-format checks

Reviewed revision `a5e8bf7ee4f29ba14826df1ce363bef13fbe2963` with the
metadata-check retirement of the same date applied. This is the review record
supplied to the Engine operations named below. It records what was checked
before the removal; it is not a coverage attestation.

## Decision

On 2026-11-13 the repository owner decided that the plan template is a guide
and that the layout of a plan is not something to verify, so the plan-format
checks are removed. Under the owner's rule for weak checks (repair a check
that is at fault; remove a check for something there is no reason to check)
this is a removal: the checks work as written, but what they check is not a
requirement.

`templates/PLAN-TEMPLATE.md`, the planning prompts and the Planning policy
text are kept. Historical records under `docs/archive/`, `docs/audits/` and
`docs/proposals/` are not changed.

## What the checks observed

Suite `planning-consolidation` held 19 checks, all of type `plan_contract`.
Each ran the plan-layout validator on one sample plan under
`evaluation/standards-effectiveness/fixtures/plans/`: 4 samples had to be
accepted and 15 had to be rejected with a fixed message.

- No check ran the validator on a real plan. The validator had two callers:
this suite and its own unit test, which read the same 19 samples.
- Run by hand on this repository's own plans at the reviewed revision, the
validator rejects 25 of the 28 `plan.md` files under `docs/plans/` (16 of
them for having no `## Objective` heading) and 1 of the 20 under
`docs/archive/plans/`. Nothing reported this, because nothing ran it.

The suite therefore showed that the validator agreed with its samples. It did
not show that any plan in use followed the template.

## What is removed, and how

| Part | Items | Operation |
| --- | --- | --- |
| Registry | The 19 checks, suite `planning-consolidation`, its registry entry, its `enforcement-suite` catalogue node, and the 4 relationships that name it | Engine `maintain_evidence`, `retire_suites` |
| Validator | `checks/plan_contract.py`, its unit test, its registration and its README row in `tools/standards_verifier/` | Ordinary code commit |
| Samples | The 19 files under `fixtures/plans/` | Engine `maintain_evidence`, `retire_inputs` |

The samples are retired after the validator code so that the validator's unit
test, which reads them, never runs without them.

## Relationships

Four relationships named the suite as both consumer and evidence owner. They
are removed with its node. No other relationship names `suite:planning-consolidation`,
so no evidence owner is replaced.

| Source policy unit | Relationships before | After |
| --- | ---: | ---: |
| `workflow.planning.active-plan-fields` | 4 | 3 |
| `workflow.planning.lifecycle-authority` | 7 | 6 |
| `topic.architecture.composed-design-admission` | 9 | 8 |
| `workflow.commit.per-commit-boundary` | 4 | 3 |

Each unit keeps its template, prompt, documentation and normative consumers,
all owned by consumer review. Policy-impact relationships 356 to 352.

## Certificates

All 24 certificates are already stale after the metadata-check retirement.
Two of them, for `workflow.planning.active-plan-fields` and
`workflow.planning.lifecycle-authority`, cite the retired suite file as
evidence. A stale certificate is ignored when coverage is read and its evidence
is not resolved, so both stay loadable. They are left in place
(`prune_stale_certificates` is `false`) until a reissue replaces them.

## Statements this makes untrue

- `workflows/planning.md`, metadata line `Verification: Active-plan structure
fixtures and objective-level acceptance review.`
- `evaluation/standards-effectiveness/verification-guide.md`, which lists
`plan records` among what the checkpoint checks.
- `tools/standards_verifier/README.md`, the `plan_contract` row of the
check-kind table. This row is removed with the validator code.

The first two can only be corrected through the Engine's standards-authoring
workflow. That correction is prepared as a separate change so that it can be
accepted or rejected on its own. `workflows/planning.md` also says that
automated verification may inspect current plan structure. That sentence
grants a permission and does not say a check exists, so it stays true.

## Counts

| | Before | After |
| --- | ---: | ---: |
| Registered suites | 23 | 22 |
| Checks | 66 | 47 |
| Catalogue nodes | 121 | 120 |
| Policy-impact relationships | 356 | 352 |
| Sample plans | 19 | 0 |

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

Loading
Loading