Repository navigation
Add skill fallbacks and fixes from the first real audits - #2
Merged
Merged
Conversation
The first real /onboard and /audit run on examples/acme-demo found the
skills forcing guesses: unknown answers had nowhere to go, the audit
called modes that didn't exist, and output paths conflicted.
- AGENTS.md: what to do when nobody can answer, a schema has no value,
a hook blocks a write, a skill has no mode, or the only evidence is
an absence
- /onboard: record unknowns rather than false or 0, ask about RBI status
- /audit: define {scope}, law_as_at and the coverage count, name where
posture and the data map go, add a sector check, cap absence-only
findings at medium
- New modes: /dpdp-rights audit, /dpdp-breach readiness, /gdpr-draft review
- /dpdp-catalog: Rule 3 and Rule 6 sub-clauses, RBI Digital Lending
Directions 2025. /gdpr-catalog: last-verified date
- Personal data in logs now cites Rule 6(1)(a), not 6(1)(c), in sample-co
and the acme-demo key, which also accepts Rule 3(a) or 3(b) for bundled
consent
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The blind re-run of /audit on examples/acme-demo found 9 of 10 planted issues. It missed the realistic identifiers in the test fixture, because it trusted a README saying they were synthetic. - /pii-scan: file a finding for realistic-format fixture identifiers even when a README calls them synthetic, rated low on that claim alone - /dpdp-breach readiness: list Rule 7(2)(a) and 7(2)(b) separately - /gdpr-audit: name the Art. 32(1) limb for logged data and credentials Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two tests counted every report in assessments/, including gitignored local ones, so npm test went red after any real audit. dataCopy() now copies only the tracked demo data. company-details gains unknowns, and data-map gains notes and a none_identified basis. Their booleans no longer default to false, so a missing answer stops reading as "no". /onboard and /gdpr-data-flows use the new fields. AGENTS.md has humans edit .claude/scripts and .claude/schemas. The maintainer asked the agent to apply this patch, once. The write-guard itself is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Vigil's skills had never run end to end. Running
/onboardand/auditonexamples/acme-demoshowed them forcing guesses, so this adds fallbacks and fixes what two runs found.What changes
AGENTS.mdnow says what to do when nobody can answer, a schema has no value, a hook blocks a write, a skill has no mode for the step, or the only evidence is an absence./onboard: records unknowns rather thanfalseor0, and asks whether a financial company is RBI-regulated./auditalready assumed:/dpdp-rights audit,/dpdp-breach readiness,/gdpr-draft review./audit: defines the id's{scope},law_as_atand the coverage count, names where posture and the data map go, adds a sector check, and caps absence-only findings at medium./dpdp-catalog, and a last-verified date in/gdpr-catalog./pii-scanfiles a finding for realistic-format fixture identifiers even when a README calls them synthetic,/dpdp-breach readinesssplits Rule 7(2)(a) and (b), and/gdpr-auditnames the Art. 32(1) limb.dataCopy()copies only tracked demo data, so a local audit no longer turnsnpm testred.company-detailsgainsunknowns,data-mapgainsnotesandnone_identified, and their booleans stop defaulting tofalse.Please check in review
sample-coF-0008 moved with it. The catalog now states these mappings, so hits on P2 and P3 no longer test independent judgment..claude/scriptsand.claude/schemas.AGENTS.mdhas humans edit those. The maintainer asked for it, once, and the write-guard is unchanged./gdpr-catalogdate comes from the README, not a fresh check. The RBI paragraph numbers are from the May 2025 text, and later consolidation was not ruled out.How far the runs go
The first run found 10 of 10 planted issues but was not blind. The second found 9 of 10, missing the fixture identifiers, which this PR addresses. Neither flagged
0004_payments.sqlor leaked a fixture value. Both were self-scored, and the fixture's comments state most issues outright, so neither number measures accuracy on real code. Reports stay local and gitignored.No audit has been run with the final skills. A harder fixture,
kadamba-fit, sits on its own branch and is not in this PR.Checks
npm run validate -- --strict: 0 errors.npm test: 23 pass, 0 fail, 1 skipped, both on a clean branch and with local audits present.Compliance tooling, not legal advice.
🤖 Generated with Claude Code