From bc69fa65d6bbe82ce8ca6bda23cd8e35d45a1d01 Mon Sep 17 00:00:00 2001 From: Brady Gaster Date: Wed, 2 Sep 2026 11:52:17 -0700 Subject: [PATCH 1/2] gh-aw: replace gzip+Base64 Cast validator transport with plaintext resource Replace the fake-skill gzip+Base64 transport for the Cast validator (workflows/shared/squad-cast-validator.md, materialized at runtime via SKILL.md discovery and gunzip/base64 extraction) with a canonical plaintext gh-aw resources sidecar, workflows/shared/squad-cast-validator.mjs, installed verbatim at .github/workflows/shared/ in consumer repos. - workflows/squad.md: declare the validator as a resources entry (not imports); rewrite the validator runner step to execute the installed resource directly from GITHUB_WORKSPACE (never staged to or run from RUNNER_TEMP); drop the SKILL.md search/extraction logic; restore the pwd -P physical-path normalization the old script relied on (needed so import.meta.url resolution is not defeated by symlinked temp/workspace paths); reduce the cast-failure job stage taxonomy from 6 to 4 stages (discovery/integrity/syntax/validation); update Step 7 docs and stage table to match. - workflows/shared/squad-cast-validator.mjs: new plaintext resource, byte-identical to scripts/validate-gh-aw-cast.mjs, pinned by SHA-256 in the runner script. - Remove workflows/shared/squad-cast-validator.md (fake-skill payload) and workflows/shared/squad-gh-aw-resource-probe.txt (experimental probe-only resource from #1982/#1983), plus their .gitattributes entry and probe-only docs. - test/gh-aw-cast-validator.test.ts, test/gh-aw-quality.test.ts: rewrite delivery-mechanism coverage for the new 4-stage model; remove probe-only tests; update SOURCE_GROWTH_BUDGET_KB. Pinned SHA-256 authentication, node --check, the exact authorization output ("Cast validation passed."), and the typed Cast failure contract (fail-closed on missing/unreadable/modified/syntactically invalid/rejecting validators) are unchanged. The direct install path in .github/AGENTS.md (gh aw add bradygaster/squad/workflows/squad.md@dev) is untouched. Validation: - npx vitest run test/gh-aw-cast-validator.test.ts test/gh-aw-quality.test.ts -> 213/213 passed (includes dynamic gh-aw strict-mode compile via lockText()). - Manual gh aw compile squad --strict against a scratch workspace (repository-pinned gh-aw v0.87.10): compiled successfully; inspected generated squad.lock.yml runner script contains no SKILL.md search, base64, or gzip logic, reads validator_script directly from GITHUB_WORKSPACE, and preserves the 4-stage fail_cast taxonomy. - npm run build (packages/squad-sdk then packages/squad-cli): passed. Closes #1984 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .gitattributes | 3 - docs/src/content/docs/guide/gh-aw.md | 30 +- test/gh-aw-cast-validator.test.ts | 239 +++-------- test/gh-aw-quality.test.ts | 142 ++----- workflows/shared/squad-cast-validator.md | 61 --- workflows/shared/squad-cast-validator.mjs | 379 ++++++++++++++++++ .../shared/squad-gh-aw-resource-probe.txt | 3 - workflows/squad.md | 101 +---- 8 files changed, 487 insertions(+), 471 deletions(-) delete mode 100644 workflows/shared/squad-cast-validator.md create mode 100644 workflows/shared/squad-cast-validator.mjs delete mode 100644 workflows/shared/squad-gh-aw-resource-probe.txt diff --git a/.gitattributes b/.gitattributes index 6edb01a25..823971637 100644 --- a/.gitattributes +++ b/.gitattributes @@ -53,8 +53,5 @@ Dockerfile.* text eol=lf # perpetually dirty tree (#1788). *.snap text eol=lf -# Squad: the gh-aw resource probe is authenticated byte-for-byte before agent startup. -workflows/shared/squad-gh-aw-resource-probe.txt text eol=lf - # Squad: union merge for append-only team state files .squad/fact-checker/audit-trail.md merge=union \ No newline at end of file diff --git a/docs/src/content/docs/guide/gh-aw.md b/docs/src/content/docs/guide/gh-aw.md index 5b3d2c7dc..08b5cfa11 100644 --- a/docs/src/content/docs/guide/gh-aw.md +++ b/docs/src/content/docs/guide/gh-aw.md @@ -242,28 +242,6 @@ committed. If a `.gitignore` is missing from `.github/aw/logs/`, add one there: Once pushed, the `/squad` slash command is live on your repo. -### Experimental gh-aw resource probe (`dev` test zone) - -Issue #1982 adds a gh-aw-only plaintext probe to the `@dev` dispatcher. It tests -whether a direct install delivers top-level `resources:` beside the installed -workflow. The probe does not change Cast behavior or any CLI/SDK install path; -the existing Base64 Cast validator remains authoritative. - -In a disposable test repository, install or refresh only the dispatcher: - -```bash -gh aw add bradygaster/squad/workflows/squad.md@dev --force -gh aw compile squad --strict -test -r .github/workflows/shared/squad-gh-aw-resource-probe.txt -``` - -Review the source, generated lockfile, and plaintext probe before committing the -test-zone update. Then comment `/squad cast` on a test issue. A missing, -unreadable, or modified probe fails the pre-agent assertion explicitly; a valid -probe leaves the current Cast path unchanged. After updating `@dev`, rerun the -same Cast issue to confirm the installed resource is used without changing the -expected Cast PR contents. - ### Optional: pin a CLI version Activation downloads a self-contained GitHub Release bundle; it does not install @@ -1322,8 +1300,8 @@ gh aw add \ customizations first, then reapply them before the final compile. Never customize generated `.lock.yml` files. -Existing local imports are not guaranteed to refresh with the top-level files. -Fetch every Squad shared import at the same SHA: +Existing local imports and resources are not guaranteed to refresh with the +top-level files. Fetch every Squad shared import and resource at the same SHA: ```bash mkdir -p .github/workflows/shared @@ -1332,8 +1310,8 @@ curl --fail --silent --show-error --location \ "https://raw.githubusercontent.com/bradygaster/squad/${SQUAD_SHA}/workflows/shared/squad.md" \ --output .github/workflows/shared/squad.md curl --fail --silent --show-error --location \ - "https://raw.githubusercontent.com/bradygaster/squad/${SQUAD_SHA}/workflows/shared/squad-cast-validator.md" \ - --output .github/workflows/shared/squad-cast-validator.md + "https://raw.githubusercontent.com/bradygaster/squad/${SQUAD_SHA}/workflows/shared/squad-cast-validator.mjs" \ + --output .github/workflows/shared/squad-cast-validator.mjs curl --fail --silent --show-error --location \ "https://raw.githubusercontent.com/bradygaster/squad/${SQUAD_SHA}/workflows/shared/squad-planning-ontology.md" \ --output .github/workflows/shared/squad-planning-ontology.md diff --git a/test/gh-aw-cast-validator.test.ts b/test/gh-aw-cast-validator.test.ts index b67c7b87f..9c069fa2b 100644 --- a/test/gh-aw-cast-validator.test.ts +++ b/test/gh-aw-cast-validator.test.ts @@ -1,15 +1,15 @@ -import { chmodSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; import { createHash } from 'node:crypto'; import { tmpdir } from 'node:os'; import { dirname, join } from 'node:path'; import { spawnSync } from 'node:child_process'; import { compileFunction, constants as vmConstants } from 'node:vm'; -import { gunzipSync, gzipSync } from 'node:zlib'; import { afterEach, describe, expect, it } from 'vitest'; import { requirePosixShell } from './posix-shell'; const validator = join(process.cwd(), 'scripts', 'validate-gh-aw-cast.mjs'); -const helperPath = join(process.cwd(), 'workflows', 'shared', 'squad-cast-validator.md'); +const resourcePath = join(process.cwd(), 'workflows', 'shared', 'squad-cast-validator.mjs'); +const installedResourceRelativePath = '.github/workflows/shared/squad-cast-validator.mjs'; const workflowPath = join(process.cwd(), 'workflows', 'squad.md'); const workspaces: string[] = []; @@ -148,12 +148,8 @@ function validate(root: string, payload: string) { }); } -function helperSource(): string { - return readFileSync(helperPath, 'utf8'); -} - -function materializedSkillSource(): string { - return helperSource().replace(/^## skill: `squad-cast-validator`\r?\n/, ''); +function resourceSource(): string { + return readFileSync(resourcePath, 'utf8'); } function validatorCommand(): string { @@ -202,18 +198,11 @@ function sha256(content: string | Buffer): string { return createHash('sha256').update(content).digest('hex'); } -function replaceValidatorPayload(source: string, program: string): string { - const encoded = gzipSync(Buffer.from(program)).toString('base64'); - return source.replace( - /(\r?\n)[A-Za-z0-9+/\r\n=]+(\r?\n)/, - `$1${encoded}$2`, - ); -} - -function materializeSkill( +/** Install the validator resource at its fixed installed location under the fixture workspace. */ +function installResource( root: string, - path = '.github/skills/squad-cast-validator/SKILL.md', - content = materializedSkillSource(), + content = resourceSource(), + path = installedResourceRelativePath, ): void { write(root, path, content); } @@ -294,16 +283,10 @@ afterEach(() => { }); describe('GH-AW Cast final-tree validator', () => { - it('ships the reviewed validator byte-for-byte in the materialized inline skill', () => { - const helper = helperSource(); - const encoded = helper.match( - /\r?\n([A-Za-z0-9+/\r\n=]+)\r?\n/, - )?.[1]; - expect(encoded, 'embedded validator payload must remain extractable').toBeDefined(); - const embedded = gunzipSync(Buffer.from((encoded as string).replace(/\s/g, ''), 'base64')); + it('ships the reviewed validator byte-for-byte as the installed plaintext resource', () => { const canonical = readFileSync(validator); - const normalizedEmbedded = Buffer.from(embedded.toString('utf8').replace(/\r\n/g, '\n')); - expect(normalizedEmbedded).toEqual(canonical); + const installed = readFileSync(resourcePath); + expect(installed).toEqual(canonical); const commandDigest = validatorRunnerSource().match( /validator_expected_sha256="([a-f0-9]{64})"/, @@ -312,7 +295,15 @@ describe('GH-AW Cast final-tree validator', () => { expect(commandDigest).toBe(sha256(canonical)); }); - it('keeps validator bytes and extraction logic out of the agent command', () => { + it('declares the validator as a top-level gh-aw resource, not an imported skill', () => { + const workflow = readFileSync(workflowPath, 'utf8'); + const resourcesBlock = workflow.match(/^resources:\n((?: - .+\n)+)/m)?.[1] ?? ''; + expect(resourcesBlock).toContain('shared/squad-cast-validator.mjs'); + const importsBlock = workflow.match(/^imports:\n((?: - .+\n)+)/m)?.[1] ?? ''; + expect(importsBlock).not.toContain('squad-cast-validator'); + }); + + it('keeps validator bytes out of the agent command', () => { const command = validatorCommand(); const runner = validatorRunnerSource(); const workflow = readFileSync(workflowPath, 'utf8'); @@ -320,12 +311,12 @@ describe('GH-AW Cast final-tree validator', () => { expect(command).not.toMatch(/[A-Za-z0-9+/]{256}/); expect(command).not.toMatch(/cat\s+<<|base64|gzip|awk|validator_expected_sha256/); expect(command).not.toContain('H4sI'); - expect(helperSource()).not.toMatch(/cat\s+< { ); }); - it('finds the materialized skill and runs the exact validator successfully', () => { + it('finds the installed validator resource and runs it directly from GITHUB_WORKSPACE', () => { const fixture = createFixture(); - materializeSkill( - fixture.root, - undefined, - materializedSkillSource().replace(/\n/g, '\r\n'), - ); + installResource(fixture.root); const invocationDirectory = join(fixture.root, 'nested', 'invocation-directory'); mkdirSync(invocationDirectory, { recursive: true }); const result = runValidatorCommand(fixture, invocationDirectory); expect(result.status, result.stderr).toBe(0); expect(result.stdout).toBe('Cast validation passed.\n'); expect(authorizesPullRequest(result)).toBe(true); - const normalizeLf = (value: string) => value.replace(/\r\n/g, '\n'); - expect( - normalizeLf(readFileSync(join(fixture.runnerTemp, 'validate-gh-aw-cast.mjs'), 'utf8')), - ).toBe(normalizeLf(readFileSync(validator, 'utf8'))); - }); - - it('fails clearly when the materialized validator skill is missing', () => { - const fixture = createFixture(); - const result = runValidatorCommand(fixture); - expect(result.status).not.toBe(0); - expect(result.stderr).toContain('Cast validator skill not found under GITHUB_WORKSPACE.'); - expect(authorizesPullRequest(result)).toBe(false); + // Never copied or extracted into $RUNNER_TEMP -- executed in place from $GITHUB_WORKSPACE. + expect(existsSync(join(fixture.runnerTemp, 'validate-gh-aw-cast.mjs'))).toBe(false); }); - it('fails clearly when multiple materialized validator skills match', () => { + it('fails clearly when the installed validator resource is missing', () => { const fixture = createFixture(); - materializeSkill(fixture.root); - materializeSkill(fixture.root, '.claude/skills/squad-cast-validator/SKILL.md'); const result = runValidatorCommand(fixture); expect(result.status).not.toBe(0); - expect(result.stderr).toContain('Expected exactly one Cast validator skill, found 2:'); - expect(result.stderr).toContain('.github/skills/squad-cast-validator/SKILL.md'); - expect(result.stderr).toContain('.claude/skills/squad-cast-validator/SKILL.md'); + expect(result.stderr).toContain('Cast validator resource is missing'); expect(authorizesPullRequest(result)).toBe(false); }); - it('fails integrity checks when the materialized payload is corrupt', () => { + it.skipIf( + process.platform === 'win32' || + typeof process.getuid !== 'function' || + process.getuid() === 0, + )('fails clearly when the installed validator resource is unreadable', () => { const fixture = createFixture(); - const corrupt = materializedSkillSource().replace( - /(\r?\n)H/, - '$1!', - ); - materializeSkill(fixture.root, undefined, corrupt); - const result = runValidatorCommand(fixture); - expect(result.status).not.toBe(0); - expect(result.stderr).toMatch(/base64|gzip|invalid|error/i); - expect(result.stderr).toContain('Cast validator payload extraction failed'); - expect(authorizesPullRequest(result)).toBe(false); + installResource(fixture.root); + const installedPath = join(fixture.root, installedResourceRelativePath); + try { + chmodSync(installedPath, 0o000); + const result = runValidatorCommand(fixture); + expect(result.status).not.toBe(0); + expect(result.stderr).toContain('Cast validator resource is not readable'); + expect(authorizesPullRequest(result)).toBe(false); + } finally { + chmodSync(installedPath, 0o600); + } }); - it('fails clearly when the materialized payload markers are malformed', () => { + it('fails integrity checks when the installed resource is modified', () => { const fixture = createFixture(); - const corrupt = materializedSkillSource().replace( - '', - '', - ); - materializeSkill(fixture.root, undefined, corrupt); + installResource(fixture.root, "console.log('Cast validation passed.');\n"); const result = runValidatorCommand(fixture); expect(result.status).not.toBe(0); - expect(result.stderr).toContain( - 'Cast validator payload extraction failed; expected one marker pair with valid base64+gzip data.', + expect(result.stderr).toMatch( + /Cast validator SHA-256 mismatch: expected 82aa5620d81e26513658fbde210b0f8d2ac3bc7572e672b421aaa17a2832e8cc, got [a-f0-9]{64}\./, ); - expect(authorizesPullRequest(result)).toBe(false); - }); - - it('rejects a syntactically corrupt validator at the digest boundary before execution', () => { - const fixture = createFixture(); - const corrupt = replaceValidatorPayload(materializedSkillSource(), 'const = ;\n'); - materializeSkill(fixture.root, undefined, corrupt); - const result = runValidatorCommand(fixture); - expect(result.status).not.toBe(0); - expect(result.stderr).toContain('Cast validator SHA-256 mismatch'); expect(result.stdout).not.toContain('Cast validation passed.'); expect(authorizesPullRequest(result)).toBe(false); }); @@ -417,11 +381,7 @@ describe('GH-AW Cast final-tree validator', () => { it('rejects authenticated validator bytes that fail node --check', () => { const fixture = createFixture(); const invalidProgram = 'const = ;\n'; - materializeSkill( - fixture.root, - undefined, - replaceValidatorPayload(materializedSkillSource(), invalidProgram), - ); + installResource(fixture.root, invalidProgram); const authenticatedInvalidRunner = validatorRunnerSource().replace( /validator_expected_sha256="[a-f0-9]{64}"/, `validator_expected_sha256="${sha256(invalidProgram)}"`, @@ -430,30 +390,14 @@ describe('GH-AW Cast final-tree validator', () => { const result = runValidatorCommand(fixture, fixture.root, authenticatedInvalidRunner); expect(result.status).not.toBe(0); expect(result.stderr).toContain('SyntaxError'); - expect(result.stderr).toContain('validate-gh-aw-cast.mjs'); - expect(result.stdout).not.toContain('Cast validation passed.'); - expect(authorizesPullRequest(result)).toBe(false); - }); - - it('rejects a syntactically valid impostor that prints the exact success sentinel', () => { - const fixture = createFixture(); - const impostor = replaceValidatorPayload( - materializedSkillSource(), - "console.log('Cast validation passed.');\n", - ); - materializeSkill(fixture.root, undefined, impostor); - const result = runValidatorCommand(fixture); - expect(result.status).not.toBe(0); - expect(result.stderr).toMatch( - /Cast validator SHA-256 mismatch: expected 82aa5620d81e26513658fbde210b0f8d2ac3bc7572e672b421aaa17a2832e8cc, got [a-f0-9]{64}\./, - ); + expect(result.stderr).toContain('squad-cast-validator.mjs'); expect(result.stdout).not.toContain('Cast validation passed.'); expect(authorizesPullRequest(result)).toBe(false); }); it('preserves validator failures and cannot authorize an invalid Cast tree', () => { const fixture = createFixture(); - materializeSkill(fixture.root); + installResource(fixture.root); write( fixture.root, '.github/agents/squad.agent.md', @@ -470,37 +414,13 @@ describe('GH-AW Cast final-tree validator', () => { const missing = createFixture(); const missingResult = runValidatorCommand(missing); - const ambiguous = createFixture(); - materializeSkill(ambiguous.root); - materializeSkill(ambiguous.root, '.claude/skills/squad-cast-validator/SKILL.md'); - const ambiguousResult = runValidatorCommand(ambiguous); - - const extraction = createFixture(); - materializeSkill( - extraction.root, - undefined, - materializedSkillSource().replace( - '', - '', - ), - ); - const extractionResult = runValidatorCommand(extraction); - const integrity = createFixture(); - materializeSkill( - integrity.root, - undefined, - replaceValidatorPayload(materializedSkillSource(), "console.log('Cast validation passed.');\n"), - ); + installResource(integrity.root, "console.log('Cast validation passed.');\n"); const integrityResult = runValidatorCommand(integrity); const syntax = createFixture(); const invalidProgram = 'const = ;\n'; - materializeSkill( - syntax.root, - undefined, - replaceValidatorPayload(materializedSkillSource(), invalidProgram), - ); + installResource(syntax.root, invalidProgram); const syntaxResult = runValidatorCommand( syntax, syntax.root, @@ -511,7 +431,7 @@ describe('GH-AW Cast final-tree validator', () => { ); const validation = createFixture(); - materializeSkill(validation.root); + installResource(validation.root); write( validation.root, '.github/agents/squad.agent.md', @@ -520,9 +440,7 @@ describe('GH-AW Cast final-tree validator', () => { const validationResult = runValidatorCommand(validation); for (const { stage, commandCategory, result } of [ - { stage: 'discovery', commandCategory: 'validator skill discovery', result: missingResult }, - { stage: 'uniqueness', commandCategory: 'validator skill uniqueness', result: ambiguousResult }, - { stage: 'extraction', commandCategory: 'validator payload extraction', result: extractionResult }, + { stage: 'discovery', commandCategory: 'validator resource discovery', result: missingResult }, { stage: 'integrity', commandCategory: 'SHA-256 authentication', result: integrityResult }, { stage: 'syntax', commandCategory: 'node --check', result: syntaxResult }, { stage: 'validation', commandCategory: 'validator execution', result: validationResult }, @@ -560,7 +478,7 @@ describe('GH-AW Cast final-tree validator', () => { it('preserves empty stderr when a validator command is unavailable', async () => { const failures = await runCastFailureJob({ stage: 'discovery', - command_category: 'validator skill discovery', + command_category: 'validator resource discovery', exit_status: 'unavailable', stderr: '', }); @@ -568,48 +486,13 @@ describe('GH-AW Cast final-tree validator', () => { [ 'Cast did not complete.', 'Stage: discovery', - 'Command category: validator skill discovery', + 'Command category: validator resource discovery', 'Exit status: unavailable', 'Stderr:', '', ].join('\n'), ]); }); - - it('fails malformed agent output with a deterministic diagnostic', async () => { - const failures = await runCastFailureJobOutput('{not-json'); - expect(failures).toHaveLength(1); - expect(failures[0]).toMatch(/^Unable to read Cast failure output: /); - }); - - it('fails non-object output items with a deterministic diagnostic', async () => { - const failures = await runCastFailureJobOutput(JSON.stringify({ - items: [null, { type: 'cast_failure' }], - })); - expect(failures).toEqual(['Cast failure output item 0 is not an object.']); - }); - - it('diagnoses conflicting failure and pull-request outputs without claiming prevention', async () => { - const failures = await runCastFailureJobOutput(JSON.stringify({ - items: [ - { - type: 'cast_failure', - stage: 'validation', - command_category: 'validator execution', - exit_status: '1', - stderr: 'Cast validation failed.', - }, - { type: 'create_pull_request' }, - ], - })); - expect(failures).toEqual([ - [ - 'Conflicting Cast terminal outputs: found 1 create_pull_request item(s) with cast_failure.', - 'This post-agent diagnostic cannot prevent a concurrently materialized pull request.', - ].join('\n'), - ]); - }); - it('accepts a self-contained descriptive Cast tree', () => { const fixture = createFixture(); const result = validate(fixture.root, fixture.payload); diff --git a/test/gh-aw-quality.test.ts b/test/gh-aw-quality.test.ts index 4e1aacbeb..c39a8429e 100644 --- a/test/gh-aw-quality.test.ts +++ b/test/gh-aw-quality.test.ts @@ -193,15 +193,6 @@ function extractResources(frontmatter: string): string[] { return resources; } -function extractNamedStepScript(frontmatter: string, stepName: string): string { - const escapedName = stepName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); - const match = frontmatter.match( - new RegExp(`^ - name: ${escapedName}\\n shell: bash\\n run: \\|\\n([\\s\\S]*?)(?=^ - name:|\\n\\S)`, 'm'), - ); - if (!match) throw new Error(`No bash step named "${stepName}" found`); - return match[1].split('\n').map(line => line.replace(/^ {6}/, '')).join('\n').trim(); -} - function extractWorkflowDispatchInputs(frontmatter: string): Record> { const inputs: Record> = {}; const lines = frontmatter.split('\n'); @@ -603,109 +594,22 @@ describe('gh-aw: shared component imports', () => { expect(imports.length).toBeGreaterThan(0); }); - describe('#1982: experimental gh-aw resource delivery probe', () => { - const frontmatter = extractFrontmatter(SQUAD_WORKFLOW); - const probeResource = 'shared/squad-gh-aw-resource-probe.txt'; - const probePath = join(WORKFLOWS_DIR, probeResource); - const expectedDigest = 'dba0c331c0b0fda06539bd9245dda72bd9c36cca6962726147b03168c1d97a73'; - const stepName = 'Assert experimental gh-aw resource delivery'; - - it('declares the plaintext probe on the top-level direct-install workflow only', () => { - expect(extractResources(frontmatter)).toEqual([probeResource]); - expect(existsSync(probePath)).toBe(true); - expect(readText(probePath)).toContain('Experiment: bradygaster/squad#1982'); - expect(readText(probePath)).toContain('no executable or runtime behavior'); - - for (const sharedFile of [ - 'squad.md', - 'squad-cast-validator.md', - 'squad-planning-ontology.md', - 'squad-planning-policy.md', - ]) { - expect(readText(join(SHARED_DIR, sharedFile))).not.toMatch(/^resources:/m); - } - }); - - it('pins the exact plaintext bytes and checks missing, unreadable, and mismatched resources', () => { - const actualDigest = createHash('sha256').update(readFileSync(probePath)).digest('hex'); - const script = extractNamedStepScript(frontmatter, stepName); + it('declares the plaintext Cast validator resource, not an imported skill', () => { + const resource = 'shared/squad-cast-validator.mjs'; + const resourcePath = join(WORKFLOWS_DIR, resource); + expect(extractResources(frontmatter)).toEqual([resource]); + expect(imports).not.toContain('shared/squad-cast-validator.md'); + expect(existsSync(resourcePath)).toBe(true); + expect(existsSync(join(SHARED_DIR, 'squad-cast-validator.md'))).toBe(false); + expect(existsSync(join(SHARED_DIR, 'squad-gh-aw-resource-probe.txt'))).toBe(false); - expect(actualDigest).toBe(expectedDigest); - expect(script).toContain(`expected_sha256="${expectedDigest}"`); - expect(script).toContain('if [ ! -f "$probe" ]'); - expect(script).toContain('if [ ! -r "$probe" ]'); - expect(script).toContain('Experimental gh-aw resource probe SHA-256 mismatch'); - }); - - it('fails closed before the agent for a missing or modified installed probe', () => { - const script = extractNamedStepScript(frontmatter, stepName); - const bash = process.platform === 'win32' ? requirePosixShell() : 'bash'; - const workspace = createTestWorkspace('gh-aw-resource-probe-'); - const installedProbe = join(workspace, '.github', 'workflows', probeResource); - mkdirSync(dirname(installedProbe), { recursive: true }); - - const missing = spawnSync(bash, ['-c', script], { - cwd: workspace, - encoding: 'utf8', - env: { ...process.env, GITHUB_WORKSPACE: workspace, RUNNER_TEMP: workspace }, - }); - expect(missing.status).not.toBe(0); - expect(missing.stderr).toContain('Experimental gh-aw resource probe is missing'); - - writeFileSync(installedProbe, 'tampered\n'); - const modified = spawnSync(bash, ['-c', script], { - cwd: workspace, - encoding: 'utf8', - env: { ...process.env, GITHUB_WORKSPACE: workspace, RUNNER_TEMP: workspace }, - }); - expect(modified.status).not.toBe(0); - expect(modified.stderr).toContain('Experimental gh-aw resource probe SHA-256 mismatch'); - - cpSync(probePath, installedProbe); - const verified = spawnSync(bash, ['-c', script], { - cwd: workspace, - encoding: 'utf8', - env: { ...process.env, GITHUB_WORKSPACE: workspace, RUNNER_TEMP: workspace }, - }); - expect(verified.status).toBe(0); - expect(verified.stdout).toContain('Experimental gh-aw resource delivery verified'); - }); - - it.skipIf( - process.platform === 'win32' || - typeof process.getuid !== 'function' || - process.getuid() === 0, - )('fails closed before the agent for an unreadable installed probe', () => { - const script = extractNamedStepScript(frontmatter, stepName); - const workspace = createTestWorkspace('gh-aw-resource-probe-unreadable-'); - const installedProbe = join(workspace, '.github', 'workflows', probeResource); - const bash = 'bash'; - mkdirSync(dirname(installedProbe), { recursive: true }); - cpSync(probePath, installedProbe); - - try { - chmodSync(installedProbe, 0o000); - const permissionCheck = spawnSync( - bash, - ['-c', '[ ! -r "$1" ]', 'bash', installedProbe], - { encoding: 'utf8' }, - ); - expect( - permissionCheck.status, - 'fixture must be unreadable to Bash before exercising the workflow branch', - ).toBe(0); - - const unreadable = spawnSync(bash, ['-c', script], { - cwd: workspace, - encoding: 'utf8', - env: { ...process.env, GITHUB_WORKSPACE: workspace, RUNNER_TEMP: workspace }, - }); - expect(unreadable.status).not.toBe(0); - expect(unreadable.stderr).toContain('Experimental gh-aw resource probe is not readable'); - } finally { - chmodSync(installedProbe, 0o600); - } - }); + for (const sharedFile of [ + 'squad.md', + 'squad-planning-ontology.md', + 'squad-planning-policy.md', + ]) { + expect(readText(join(SHARED_DIR, sharedFile))).not.toMatch(/^resources:/m); + } }); it('each imported file exists in workflows/shared/', () => { @@ -1090,7 +994,15 @@ describe('gh-aw: prompt budget & planning import regression', () => { // Raised 194 -> 196 KB by #1982's gh-aw-only resource assertion. The deterministic // step runs before the agent and does not enter its prompt; the separate ambient // prompt budget remains the authoritative delivered-context guard. - const SOURCE_GROWTH_BUDGET_KB = 196; + // Lowered 196 -> 187 KB by #1984: replaced the gzip+Base64-encoded validator skill + // wrapper (`shared/squad-cast-validator.md`, imported) and the experimental probe + // resource with a canonical plaintext top-level `resources:` sidecar + // (`shared/squad-cast-validator.mjs`). Resources are installed verbatim by gh-aw but + // are not summed into this budget (they were never part of the imported/ambient + // source), so removing the large embedded payload from `imports:` drops the total. + // Combined authored source now measures ~186.1 KB; 187 KB leaves a similar margin to + // prior raises so the guard still bites on genuine growth. + const SOURCE_GROWTH_BUDGET_KB = 187; const SOURCE_GROWTH_BUDGET_BYTES = SOURCE_GROWTH_BUDGET_KB * 1024; it('squad-planning-ontology.md is in the imports list', () => { @@ -2759,7 +2671,7 @@ describe('gh-aw: Cast replaces disposable bootstrap state (#1909)', () => { /only exit status zero with stdout exactly\s+`Cast validation passed\.` authorizes exactly one `create-pull-request`/s, ); expect(candidateCast).toContain('"${RUNNER_TEMP:?}/run-squad-cast-validator"'); - expect(candidateCast).toContain('Do not transcribe validator bytes or extraction commands'); + expect(candidateCast).toMatch(/Do not transcribe validator bytes, and do not invoke or\s+load/); expect(candidateCast).toContain('emits one JSON\nrecord on stdout'); expect(candidateCast.match( /[\s\S]*?/, @@ -2784,9 +2696,7 @@ describe('gh-aw: Cast replaces disposable bootstrap state (#1909)', () => { expect(failureBranch).not.toMatch(/re-?materialize the (?:committed )?(?:validator )?payload/i); for (const [stage, category] of [ - ['discovery', 'validator skill discovery'], - ['uniqueness', 'validator skill uniqueness'], - ['extraction', 'validator payload extraction'], + ['discovery', 'validator resource discovery'], ['integrity', 'SHA-256 authentication'], ['syntax', 'node --check'], ['validation', 'validator execution'], diff --git a/workflows/shared/squad-cast-validator.md b/workflows/shared/squad-cast-validator.md deleted file mode 100644 index bbe2cdfe4..000000000 --- a/workflows/shared/squad-cast-validator.md +++ /dev/null @@ -1,61 +0,0 @@ -## skill: `squad-cast-validator` ---- -description: Store the reviewed GH-AW Cast final-tree validator as a disk-only payload. ---- - -# GH-AW Cast final-tree validator - -This inline skill is a disk-only validator container. Do not invoke or load it -into model context. Before the agent turn, the workflow prepares a runner that -locates this materialized `SKILL.md` and extracts the validator directly on disk. - - -H4sIAAAAAAACCrU723LbOJbvqco/II6rScYSlcy87MqtaB3HPe0px8na7u3dkeQIIiEJbYpQA6BtRdQ87gfsJ+6XbB1cSJAiHc90rV8k4XLuOBcc+PWrXiZ4 -b0bTHknvUcpi8vLFyxd0tWZcou3LFwiRRyqkuN6kUQd+coLjn2hCKgMx5cVvIbE0P3ZoztkKeQC2PxfecQkZ/cZo2kGcCJbcE1RZucZyWVkLAzcMsP5ydVFd -m/EElr58EbFUSHT6+ers65eT/7r4fPIRDdAICPJC8XuG454keBWuYq/jDnKWSZou9sYjLGC8x8mCCsk34W+Cpc1LllRI9uSKNUtoVF2woHKZzXp4QVIpemp5 -qH6UlKwIkV25JF09q8cnDrPXv3z58vnq5uvV5wtg+ubm7OoSDVBvPPOH/euI0xnJr3CyXuZXmOZXJ+f5TziS6HRJojvCg/GsR48trC8XJ6dnP3+++Hh2tQdq -TdKYpos8S4GhYDzL/WFfrElEcQK2kUss7m42ayLyJU2lCAZvK7B/+nz14fzjx7PLr1dnP51dnV2eFuReWyWNPCFxGuOEpQRJslonWBKvg3q3Yy3Mcc+OinGv -N+nsbRLZWpkL2F9tpz/sxySigrJU5BFL53SRLxm7EzmNSSqp3OSMR0siJMeSsrSbsEW+TrIFTUUuIk7XUuTijiaJyB8Yv5sn7EEE/rA/7uXjMD8MCnpSlnb/ -8nP35FcUJZSk0tDhD/tRgrOY5BFb04TJPMq4YDxfkBVNac7WJI1YTPJ7AR+BwyFNJeEpTpBgGY8UY/6wf5uPe0DAGkd3eEFELnhkd00KwX84uTr7en4Jgj65 -AJH/XOoWoGtIo7GYHnj+JPAdgOPe6LanJoLJ0bgneAQolCi1zebGuPMooblm1oo2YozHNMWScSNmnqWSrkguGUtEEABwCzvoLVbKpudZGoHw0RpzQU74QviY -L+4D7YU0Q5gvBBqglDygT3jtB8cwNWcc+QmRiKYxeUQD9PbYfP0RNtyHCUkXcmkHjwboTwaoBXtHNmig1o7UmsmxO3uPk4xU5tERemfX0DnyX92RzTAUEnMp -fqVy6XvdrhegPLd7BwOUpTGZ05TEBW6E5JKzB8XNGeeM+94vAi9IH3bRGEvSXSy7+KELkg5XvwnU7XLGJPoR/OF71O2u8SZhOEY/gmfpzmlC3nuBoWunP0Bi -oSDSvyObUCQ0Iv6fgo6mSy9VCxUXau0SC98DNJoBZ9Bg84KCg/8/+nc6ssiMpxYZbO3bgOErshZEWlqDjl5lQH7Bctm0uORBrd8dv3yxqxgfRLMb8ih9AAvx -KcGS3hOAZ9g2VLlx0IdgZnaEYehuCsU6odL3eoASeZmc/4sXhJysExwRvzfm47S36CBvnCrOq8SkjK9wQr+RLyVXvladpgXUZlSxWRM2N/b2ajBAnpCcpgtP -T1tTNGdBWeTb2pRrvz0vqM3SNEqymAjfG4/3JksmnXVh6Kzrjd4Mt7vReDLphZIIadiA+cKcjGTTLEn2jEDjKeR2Ow7HvV4HeQ1SI484kiCqa4llJto1CT4j -yjgnqUQDZV+lP9FHX5DFCmbZHLUp1RKvTlCZK/kGsD5ENicqRkMqPlJOIkgc/MBxCdbmTdrVR3OcCNJBMZGYJn00XWNFb2x3IyoQngkYm5E54wQdbg3Zu6ky -cMcbaK5IKjkl4Eid3K0gzfVrZmWpVAPZpVjDjLAgn6hYYRkt0cCiCOc0jX0ffm0CNHivxjehZBfsgfBTLIgfKFs0cKszlpS6J4C/qnjKcSsnl55yFqEhmsIU -Wpm5PiKPaxJJEjty66A5y1IYccHspi6gPpquqBA0XRRW4gi+oKiugMLclMswPzsWRKvzs+w2GJhBBd6zcR798ANqsj9wXr7dbcXWtH+IPHDYYGgpkwgrVB7q -l8Pa/rwWl0oeJceRvGARTq7InHCSRkT4K8zvYvaQVgI8L+YhMdQxVs+ssYRECJJSN2sZTQJ/HI5Oun/D3W9vu//6tQvpivM7BL/zb0d/73WLXKNyxLXFsjmy -9IRq5CRJfIMycByUpS5cZ2Lpq5Wjd5PSL43CznF/cnSoXdO+OkdhGEK8vCbSL6EFkz2hqSToShcnvilSOghH4IIu8YqIDiIQc0VFfEuCIVP/BHTpA27LGxjx -e7evXyMDFN3gWULG4s1hb7EK0HBoxQ0HvwrHBg0IK+8KYWj0WhBTg8c5TcoLJxsEqflBHe2BpbQ8aY0od9NajqI4+JXxOwT1xli88Yf9//3v/8m774OxeHMC -malhieowYwgLGsn2CrITssDRpoSMJEMKGhJEK4QKsJoZjWOSlklKKXq77nky90djMb6evBkG/nBw+/r1WOT+8JUeDILeKhiGo3cT0IrnOYcgoalSq0Fm4xAk -D+EKr30fFihHC19CyenKD4JwThNJuP+BsYTgNCjU/ErBc0J27kggV1QTdMNQjs4e8WqdEIFy73uiVIpX+iUcHTwD4gHIlpPfM8pJ3CRazh6AaUWsxmw5Kvnt -3Y7z8GicH5r0Qk0Ebavhi86SnsVxG5hXgHUs3vSH3aMmvA0q0Sn4uw7qvgus+nKrvYgkiVoKX6z26thhThSrhJvR/Tk41nID5YLUKuned/SGJUoIFlKdWDMK -ov++djJJYuWTTGkGzi2o+9kRFM4g5o7eccMmKq2CSrqSQ9l1KnEyS510o5l42KRyYITTWCNAEvMFkWiVCUiNEJTmZLWWG6/MKyKWSppmpBqkHaoNDEWoosTq -rGN1ZtLxwXuToracOYcDXWuVnlxVVxqRu6zNwxqSDg63+tvuoIjMqQkQyN5aoTXhggpJUvk1xSsyLVnfIZII4qJzFBniOLYklTscIRkLcCQF4EFODmNVvbrg -gWPYELQotuTWMLQiqxm4k8MtbNsdoCUGnjXN00rN64ZbsK6W0GokZKqDSiyFusBKUEEGSRYpgJHsAP31+vNlqID5tarx6UtEmxagSKUevsLdElbNxj6iqSqq -FVLkH27VsnBFBFTcu6AQgc0zJrXa3kIa6ltGAafL1IwFdWZG+UU2+41E0oNlJ5zjTUiF+vRrq9uiQUG4ZOtuQu5Jggx4DbnBqzQSb+59tBUM0Ge1OTTlxR4x -NVc5Mhcdk/J8qmsamQnlEj0N16v46xGNK9v8LYIRsLu+OeO1U4V2QRlTNcjnON5CRBXPWzX4Rt9bzV/1wuLkVQ9d73ak8+CJ+exO3tgQqXeGNA4cYzCDii33 -GkF5Y2fS+rm287tnuFW+Drfq9GjgdL4xxAS79rOsAeydZnvXdIrXeEYTKsEunBtImzR3bEy7Yg/NyfOMLGh6yjJVnjkQbCL346tuF13/+y8nH/s3Zyefuqcn -X04+nF+c35yfXfc/nP3l/BJ1u+97i2BotT8cordO9kbS+J8Ff3b5sRU4KNqhXWXpoK4C3RN5u0OHk7uDHUZWnhtVGRGO1phym68rdgeH2xLvDp1dfhwcbi3W -Xc0l1Y/0LGHRXaMk9K5nirtMpJ8lP1WitiTXKyJxjCVGA01cAznKrfcjx9SQiJZkhQfvkNMQGfjj+ChARVvE/FbNEfO9JKbMxi0Bzf6ioiqxSaMlZyn9RmLI -bL4Rzioqs7w0OJBK4DfJWcclv1NS3tFET9CgAKm85GWmjqtzV2TdgANHGV7FH9pF4G8sCrXKOZv7SxUJ31/mYnbuN2vY6jMauDPa4tLKPKl6aByhR2D3Qiec -5PcMJ4Z522TUrSnkXCMdoanv2s3htiKunaOJweF2n/ud0U/zZFBeQ+17VdBZQ9NPhwZl/8Ez7NC0BjtI9wY7iHG0Z4vKfQiEOWkqof+BaKaPZVGuOuGoNRC1 -6Uq7H7aiUuznmA7g3UFDRHpOWVNo40kW7G59TVybtMXPH+FNl0IHh1uLaXeAuu9hwEBvYFAFWPKouqkNcVbIG06Iv0U62XU6LmhXCaia2PIiD/Jqs/o/II+q -59bu3H5+XTRbHIRFR+XJlNqmlVOz8/vZdD17rqbALp3BHhavwFLEU6wRYdgN1hGxNOJEQp22ZoLCNX7X9hfUYwfhTfYLbdstK+TpmKHu+7B5RYa1LmfRTQIQ -T7SW3AZAuafNCOsy7ahC23KoHAIMfPl8ff6firf93E+jdVK/WmlqUGiEDkWNPs1eQJhNQSjoN53KWjDaOzY7t4KbOFsnNAJ3rfTR5rxqarChFjThPkWp+gBL -SHnQzbYgaJGt7TQU8I0g7W/nnrShaHqi4D2uL6/f45h4pCqjLTLVDpRF2uvu7T9dYi6JOvXVrTQ2G6emODZPXw63NN71Ir0tXMVTF6Y9QV8K07d0wUW6K2HV -eK2SMNm7g1JSKw9JvXdXwaXuKGBDq1KytDjgRh3wUbmybsBdQ/Mdy3iSAGsVRtHgmVsoeaYczE22KZBbGqmKJNdH6PWhbh0V7c4Mmn8JaXMbkhPSt4SimBF9 -haWAoAcql+Ye+YKm2SN084BR/3BrQOt+lXPvsds3f2VfKiyZblvlcsbrIE8v8coU3Gl/lduDmnxWWBJOtQ86j+ttVGefZby4jyj7oEVDThFW7unoDmlQbxAX -oATj9nK1evA0JS1njoIfdHcCs/UKvMpWoHxmbU2BKvh+lqw17ELVKila11A96VzZtIp0K65+hXn+UdRTZqdlW1AUKkmCVgOnf1tjyl30RHZskxV4MYgGtkpU -14Imk68MOnlYOeHkNQZMyyWhfZZYlMsFipYNzpPF0hBcCvb2PfnY8Pm3kVqjEcuSWB1WwIPmFB6nOQQohoqS5/tXlRqF66nAOJveNuraBMC3lCYwBYmIKbvs -i0DOEmtqQPaMIBzfEy6pIDGabZBcEqQf7SkXypmQhHu1zmM7QQ7vzymZWsirJBiIpi5FLJPrTDY1X4BjJ+S2tNqVyGp3o0X32bbP7GNJbSXq/R+00k1kHoer -2F4cllstUMe3NHgWl8hGtzLaj90GZLNA7bmdaoUbEt3HA8/wK7rIc3xLo2exBBl6mr2My2Cji2lpm5kKsUjRntPuV5D+2KXnsUuHtq2ryrsLo9UwfMqedJCp -mtFIHUGvU6piYl+YtENzz08zUGdFA+y9Ymikn8q6KyHbqXNamJa79Y6mccc+NlHbnnq9XOvqmW17p6S1qXe41aTu+s7ZP9wCEbuSepXom++7aXND7okktpGQ -NjISUI6Dunxdph7dg6skj1AaUWlcf5GOtlDZ3GN9VqpZUv6H881nsdiSd7aw1tIILazPPihSVjQqjgZ8TDp7Vu38mkyaTPOJh0rt77yfo3I36lTemlcs0L51 -qvHvPJKm8QWekcStHi1qk5bOEyw/2dS041aTo/KA0Lj6CtB53KcbT+6k8/Lq1jS6jtR72q77yva2m3cP1bBXgis8UzWbvqeCzhLyyQgZLr6tvAt40AuwfQfV -lbEPUdtUVgNaaq43numegg/9Oduk+9qdvIF/z1jQvXcDjpzV4S6eoFVfTj7P5VB1tYcTuKSxaQkS6t0LSgCHaXg42q+8g5o++Tig4cmbCUCTMmGoNvNWmKa+ -04VnaxgW9YTaDNu4qf5XYM1ZRIQI1Yt/+9D9ydwWdMQSEqpRf6oyLRNYgZY5pgmJ++O0i+o5bMG3RUoeqTxlMVy2vHuy61Vcie5dpxqenFpUa810+947beTn -Em7IFjqoGubhBqbgaDe1GQ08Ifun+QJiErbwvTopaywEicPykbaKko6moA/3ww9I/6dXCC2mMOOJasdU/umrvikIl5zMjUS01WgU/wfPs/TH0TYAAA== - diff --git a/workflows/shared/squad-cast-validator.mjs b/workflows/shared/squad-cast-validator.mjs new file mode 100644 index 000000000..f8b5df932 --- /dev/null +++ b/workflows/shared/squad-cast-validator.mjs @@ -0,0 +1,379 @@ +#!/usr/bin/env node + +import { + existsSync, + readFileSync, + readdirSync, + statSync, +} from 'node:fs'; +import { join, resolve } from 'node:path'; +import { pathToFileURL } from 'node:url'; + +const CORE_PAYLOAD = [ + '.squad/team.md', + '.squad/routing.md', + '.squad/casting/registry.json', + '.squad/casting/history.json', + '.squad/casting/policy.json', + '.github/agents/squad.agent.md', + 'meet-the-squad.md', +]; + +const SUPPORT_ROLE_PATTERN = /\b(?:Scribe|Ralph|Rai|RAI|Fact Checker)\b/i; +const PLACEHOLDER_PATTERN = /\b(?:pending|uncast)\b|(?:specialists|taskTypes|hints)=0\b/i; +const FORBIDDEN_REFERENCE_PATTERNS = [ + ['standalone template', /^\.squad\/templates\//], + ['standalone support state', /^\.squad\/(?:decisions|config|hooks|identity|orchestration-log|plugins|scripts|skills|workflows)(?:\/|\.|$)/], + ['non-GH-AW client', /^\.(?:claude|copilot|cursor|gemini|opencode|vscode)\//], + ['internal source', /(?:^|\/)(?:packages|src)\//], +]; +const BARE_INTERNAL_PATH_PATTERN = + /(?:^|[\s`"'(])((?:packages\/[^/\s`"')]+\/src|src\/(?:agents|casting|cli|client|config|coordinator|hooks|runtime|tools))\/[^\s`"')]+)/gm; + +function parseArgs(argv) { + const args = new Map(); + for (let index = 0; index < argv.length; index += 2) { + const key = argv[index]; + const value = argv[index + 1]; + if (!key?.startsWith('--') || value === undefined) { + throw new Error('Usage: validate-gh-aw-cast.mjs --root --payload '); + } + args.set(key.slice(2), value); + } + if (!args.has('root') || !args.has('payload')) { + throw new Error('Usage: validate-gh-aw-cast.mjs --root --payload '); + } + return { + root: resolve(args.get('root')), + payloadPath: resolve(args.get('payload')), + }; +} + +function readText(root, relativePath) { + return readFileSync(join(root, ...relativePath.split('/')), 'utf8').replace(/\r\n/g, '\n'); +} + +function normalizePayloadPath(value) { + if ( + typeof value !== 'string' + || value.length === 0 + || value.startsWith('/') + || value.includes('\\') + || value.split('/').includes('..') + || /[*?{}[\]]/.test(value) + ) { + return null; + } + return value.replace(/^\.\//, ''); +} + +function exactPathStatus(root, relativePath) { + let current = root; + for (const segment of relativePath.split('/')) { + if (!existsSync(current) || !statSync(current).isDirectory()) { + return { exists: false, detail: `parent directory is absent before ${segment}` }; + } + const entries = readdirSync(current); + if (!entries.includes(segment)) { + const caseMismatch = entries.find((entry) => entry.toLowerCase() === segment.toLowerCase()); + return { + exists: false, + detail: caseMismatch + ? `case mismatch: expected ${segment}, found ${caseMismatch}` + : `missing segment ${segment}`, + }; + } + current = join(current, segment); + } + return { + exists: existsSync(current), + file: existsSync(current) && statSync(current).isFile(), + detail: existsSync(current) ? 'path is not a file' : 'path is absent', + }; +} + +function extractLocalReferences(markdown) { + const references = []; + const pattern = /(?:^|[\s`"'([])(\.[A-Za-z0-9_-]+\/[A-Za-z0-9_.*?{}@+~/-]+)/gm; + for (const match of markdown.matchAll(pattern)) { + references.push(match[1].replace(/[.,;:]+$/, '')); + } + return [...new Set(references)]; +} + +function parseRouting(routing, activeNames, errors) { + const headingMatches = routing.match(/^## Routing Table\s*$/gm) ?? []; + if (headingMatches.length !== 1) { + errors.push(`routing: expected exactly one "## Routing Table" heading, found ${headingMatches.length}`); + } + if (/^## Work Type\s*(?:→|->)\s*Agent\s*$/gmi.test(routing)) { + errors.push('routing: legacy Work Type to Agent section is forbidden'); + } + + const section = routing.match(/^## Routing Table\s*$([\s\S]*?)(?=^##\s|(?![\s\S]))/m)?.[1] ?? ''; + const lines = section.split('\n').map((line) => line.trim()).filter(Boolean); + if (!lines.includes('| Work Type | Route To | Examples |')) { + errors.push('routing: exact header "| Work Type | Route To | Examples |" is required'); + } + + const rows = lines + .filter((line) => /^\|.+\|$/.test(line)) + .filter((line) => line !== '| Work Type | Route To | Examples |') + .filter((line) => !/^\|\s*:?-+/.test(line)) + .map((line) => line.slice(1, -1).split('|').map((cell) => cell.trim())) + .filter((cells) => cells.length === 3); + + if (rows.length === 0) { + errors.push('routing: at least one routing row is required'); + } + + const routedNames = new Set(); + for (const [workType, routeTo] of rows) { + if (!workType || !routeTo) { + errors.push('routing: work type and route target must be non-empty'); + continue; + } + for (const target of routeTo.split(',').map((value) => value.trim()).filter(Boolean)) { + if (!activeNames.has(target)) { + errors.push(`routing: target "${target}" is not an active registry persistent_name`); + } else { + routedNames.add(target); + } + } + } + + for (const name of activeNames) { + if (!routedNames.has(name)) { + errors.push(`routing: active member "${name}" has no route`); + } + } + return rows; +} + +function parseRegistry(root, errors) { + let registry; + try { + registry = JSON.parse(readText(root, '.squad/casting/registry.json')); + } catch (error) { + errors.push(`registry: invalid JSON (${error.message})`); + return []; + } + if (!registry?.agents || typeof registry.agents !== 'object' || Array.isArray(registry.agents)) { + errors.push('registry: top-level agents object is required'); + return []; + } + const active = Object.entries(registry.agents) + .filter(([, value]) => value?.status === 'active') + .map(([id, value]) => ({ id, name: value.persistent_name })); + if (active.length === 0) { + errors.push('registry: at least one active member is required'); + } + for (const member of active) { + if (!/^[a-z0-9][a-z0-9-]*$/.test(member.id) || typeof member.name !== 'string' || !member.name.trim()) { + errors.push(`registry: invalid active member ${JSON.stringify(member)}`); + } + } + return active; +} + +function validateCapabilities(coordinator, active, routingRows, errors) { + const beginCount = coordinator.match(//g)?.length ?? 0; + const endCount = coordinator.match(//g)?.length ?? 0; + if (beginCount !== 1 || endCount !== 1) { + errors.push(`coordinator: expected one capability marker pair, found BEGIN=${beginCount} END=${endCount}`); + return; + } + const block = coordinator.match( + /([\s\S]*?)/, + )?.[1] ?? ''; + const metadata = block.match( + //, + ); + if (!metadata) { + errors.push('coordinator: synchronized nonzero capability metadata is required'); + } else { + const [, specialists, taskTypes, hints] = metadata.map(Number); + if ( + specialists !== active.length + || taskTypes !== routingRows.length + || hints !== routingRows.length + || specialists === 0 + || taskTypes === 0 + || hints === 0 + ) { + errors.push( + `coordinator: capability counts must equal active/routing state ` + + `(specialists=${active.length}, taskTypes=${routingRows.length}, hints=${routingRows.length})`, + ); + } + } + if (PLACEHOLDER_PATTERN.test(block)) { + errors.push('coordinator: pending, uncast, or zero capability markers are forbidden'); + } + for (const member of active) { + if (!block.includes(member.name)) { + errors.push(`coordinator: capability block omits active member "${member.name}"`); + } + } + for (const [workType, routeTo] of routingRows) { + if (!block.includes(workType) || !block.includes(routeTo)) { + errors.push(`coordinator: capability block omits route "${workType}" -> "${routeTo}"`); + } + } +} + +export function validateCastTree({ root, payloadPath }) { + const errors = []; + let payloadValue; + try { + payloadValue = JSON.parse(readFileSync(payloadPath, 'utf8')); + } catch (error) { + return [`payload: invalid JSON (${error.message})`]; + } + if (!Array.isArray(payloadValue)) { + return ['payload: expected a JSON array of concrete repository-relative paths']; + } + + const payload = []; + for (const value of payloadValue) { + const normalized = normalizePayloadPath(value); + if (!normalized) { + errors.push(`payload: invalid, non-concrete, or non-POSIX path ${JSON.stringify(value)}`); + } else { + payload.push(normalized); + } + } + if (new Set(payload).size !== payload.length) { + errors.push('payload: duplicate paths are forbidden'); + } + + for (const required of CORE_PAYLOAD) { + if (!payload.includes(required)) errors.push(`payload: missing required path ${required}`); + } + + const active = parseRegistry(root, errors); + const activeNames = new Set(active.map(({ name }) => name)); + const activeCharters = active.map(({ id }) => `.squad/agents/${id}/charter.md`); + const expectedPayload = new Set([...CORE_PAYLOAD, ...activeCharters]); + for (const path of payload) { + if (!expectedPayload.has(path)) errors.push(`payload: unexpected path ${path}`); + } + for (const path of expectedPayload) { + if (!payload.includes(path)) errors.push(`payload: missing active Cast path ${path}`); + } + + for (const path of payload) { + const status = exactPathStatus(root, path); + if (!status.exists || !status.file) { + errors.push(`tree: ${path} does not exist with exact Linux casing (${status.detail})`); + } + } + + const agentsPath = join(root, '.squad', 'agents'); + if (existsSync(agentsPath)) { + const materializedIds = readdirSync(agentsPath) + .filter((entry) => statSync(join(agentsPath, entry)).isDirectory()) + .sort(); + const activeIds = active.map(({ id }) => id).sort(); + if (JSON.stringify(materializedIds) !== JSON.stringify(activeIds)) { + errors.push( + `tree: materialized agent directories must exactly match active registry IDs ` + + `(expected ${activeIds.join(', ')}, found ${materializedIds.join(', ')})`, + ); + } + } + + let team = ''; + let routing = ''; + let coordinator = ''; + try { + team = readText(root, '.squad/team.md'); + routing = readText(root, '.squad/routing.md'); + coordinator = readText(root, '.github/agents/squad.agent.md'); + } catch (error) { + errors.push(`tree: could not read final coordinator/team/routing (${error.message})`); + return errors; + } + + if (SUPPORT_ROLE_PATTERN.test(team)) { + errors.push('team: inactive/support roles must not be advertised by the GH-AW Cast roster'); + } + if (SUPPORT_ROLE_PATTERN.test(coordinator)) { + errors.push('coordinator: inactive/support roles are forbidden in GH-AW Cast output'); + } + + const teamCharters = extractLocalReferences(team) + .filter((reference) => /^\.squad\/agents\/[^/]+\/charter\.md$/.test(reference)) + .sort(); + if (JSON.stringify(teamCharters) !== JSON.stringify([...activeCharters].sort())) { + errors.push( + `team: charter references must exactly match active registry members ` + + `(expected ${activeCharters.sort().join(', ')}, found ${teamCharters.join(', ')})`, + ); + } + + const routingRows = parseRouting(routing, activeNames, errors); + validateCapabilities(coordinator, active, routingRows, errors); + + const outputReferences = [ + ...extractLocalReferences(team).map((reference) => ['team', reference]), + ...extractLocalReferences(coordinator).map((reference) => ['coordinator', reference]), + ]; + for (const [source, reference] of outputReferences) { + for (const [kind, pattern] of FORBIDDEN_REFERENCE_PATTERNS) { + if (pattern.test(reference)) { + errors.push(`${source}: forbidden ${kind} reference ${reference}`); + } + } + if (!expectedPayload.has(reference)) { + errors.push(`${source}: local reference is absent from the explicit final payload: ${reference}`); + continue; + } + const status = exactPathStatus(root, reference); + if (!status.exists || !status.file) { + errors.push(`${source}: local reference is absent with exact Linux casing: ${reference}`); + } + } + + for (const [source, markdown] of [['team', team], ['coordinator', coordinator]]) { + for (const match of markdown.matchAll(BARE_INTERNAL_PATH_PATTERN)) { + errors.push(`${source}: forbidden internal source reference ${match[1]}`); + } + const validLabels = new Set( + active.flatMap(({ id, name }) => [ + id.toLowerCase(), + name.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, ''), + ]), + ); + const visibleMarkdown = markdown.replace(//g, ''); + for (const match of visibleMarkdown.matchAll(/\bsquad:([a-z][a-z0-9_-]*)\b/gi)) { + if (!validLabels.has(match[1].toLowerCase())) { + errors.push(`${source}: fictional or inactive sample label squad:${match[1]} is forbidden`); + } + } + } + + return [...new Set(errors)].sort(); +} + +function main() { + let options; + try { + options = parseArgs(process.argv.slice(2)); + } catch (error) { + console.error(`Cast validation failed:\n- ${error.message}`); + process.exitCode = 1; + return; + } + const errors = validateCastTree(options); + if (errors.length > 0) { + console.error(`Cast validation failed:\n${errors.map((error) => `- ${error}`).join('\n')}`); + process.exitCode = 1; + return; + } + console.log('Cast validation passed.'); +} + +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { + main(); +} diff --git a/workflows/shared/squad-gh-aw-resource-probe.txt b/workflows/shared/squad-gh-aw-resource-probe.txt deleted file mode 100644 index 90041bae8..000000000 --- a/workflows/shared/squad-gh-aw-resource-probe.txt +++ /dev/null @@ -1,3 +0,0 @@ -Squad gh-aw resource delivery probe. -Experiment: bradygaster/squad#1982 -This plaintext file has no executable or runtime behavior. diff --git a/workflows/squad.md b/workflows/squad.md index f6259e254..a3998dc02 100644 --- a/workflows/squad.md +++ b/workflows/squad.md @@ -40,11 +40,10 @@ network: - defaults imports: - shared/squad.md - - shared/squad-cast-validator.md - shared/squad-planning-ontology.md - shared/squad-planning-policy.md resources: - - shared/squad-gh-aw-resource-probe.txt + - shared/squad-cast-validator.mjs tools: bash: true web-fetch: @@ -52,30 +51,6 @@ tools: mode: gh-proxy toolsets: [default] steps: - - name: Assert experimental gh-aw resource delivery - shell: bash - run: | - set -euo pipefail - probe="${GITHUB_WORKSPACE:?}/.github/workflows/shared/squad-gh-aw-resource-probe.txt" - expected_sha256="dba0c331c0b0fda06539bd9245dda72bd9c36cca6962726147b03168c1d97a73" - if [ ! -f "$probe" ]; then - printf 'Experimental gh-aw resource probe is missing: %s\n' "$probe" >&2 - exit 1 - fi - if [ ! -r "$probe" ]; then - printf 'Experimental gh-aw resource probe is not readable: %s\n' "$probe" >&2 - exit 1 - fi - actual_sha256="$( - node -e 'const c=require("node:crypto"),f=require("node:fs");process.stdout.write(c.createHash("sha256").update(f.readFileSync(process.argv[1])).digest("hex"))' \ - "$probe" - )" - if [ "$actual_sha256" != "$expected_sha256" ]; then - printf 'Experimental gh-aw resource probe SHA-256 mismatch: expected %s, got %s.\n' \ - "$expected_sha256" "$actual_sha256" >&2 - exit 1 - fi - printf 'Experimental gh-aw resource delivery verified.\n' - name: Prepare deterministic Cast validator runner shell: bash run: | @@ -87,8 +62,7 @@ steps: cd "${GITHUB_WORKSPACE:?}" stderr_file="${RUNNER_TEMP:?}/squad-cast-validator.stderr" - validator_matches="${RUNNER_TEMP:?}/squad-cast-validator-matches.txt" - validator_script="${RUNNER_TEMP:?}/validate-gh-aw-cast.mjs" + validator_script="${GITHUB_WORKSPACE:?}/.github/workflows/shared/squad-cast-validator.mjs" validator_output="${RUNNER_TEMP:?}/squad-cast-validator.stdout" expected_output="${RUNNER_TEMP:?}/squad-cast-validator.expected" @@ -112,54 +86,16 @@ steps: } : > "$stderr_file" - find "${GITHUB_WORKSPACE}" -maxdepth 6 -name "SKILL.md" \ - -path "*/squad-cast-validator/SKILL.md" -print \ - > "$validator_matches" 2> "$stderr_file" - status=$? - if [ "$status" -ne 0 ]; then - fail_cast "discovery" "validator skill discovery" "$status" + if [ ! -f "$validator_script" ]; then + printf 'Cast validator resource is missing: %s\n' "$validator_script" > "$stderr_file" + fail_cast "discovery" "validator resource discovery" "1" fi - - validator_count="$(wc -l < "$validator_matches" | tr -d '[:space:]')" - case "$validator_count" in - 0) - printf 'Cast validator skill not found under GITHUB_WORKSPACE.\n' > "$stderr_file" - fail_cast "discovery" "validator skill discovery" "1" - ;; - 1) ;; - *) - { - printf 'Expected exactly one Cast validator skill, found %s:\n' "$validator_count" - sed 's/^/ /' "$validator_matches" - } > "$stderr_file" - fail_cast "uniqueness" "validator skill uniqueness" "1" - ;; - esac - validator_skill="$(sed -n '1p' "$validator_matches")" - - : > "$stderr_file" - { - awk ' - { sub(/\r$/, "", $0) } - $0 == "" { if (inside || seen) exit 41; inside = seen = 1; next } - $0 == "" { if (!inside || ended) exit 42; inside = 0; ended = 1; next } - inside { print } - END { if (!seen || inside || !ended) exit 43 } - ' "$validator_skill" \ - | tr -d '\r\n' \ - | base64 --decode \ - | gzip --decompress \ - | sed 's/\r$//' - } > "$validator_script" 2> "$stderr_file" - status=$? - if [ "$status" -ne 0 ]; then - printf '%s\n' \ - 'Cast validator payload extraction failed; expected one marker pair with valid base64+gzip data.' \ - >> "$stderr_file" - fail_cast "extraction" "validator payload extraction" "$status" + if [ ! -r "$validator_script" ]; then + printf 'Cast validator resource is not readable: %s\n' "$validator_script" > "$stderr_file" + fail_cast "discovery" "validator resource discovery" "1" fi - validator_script="$(cd "$(dirname "$validator_script")" && pwd -P)/$(basename "$validator_script")" + validator_expected_sha256="82aa5620d81e26513658fbde210b0f8d2ac3bc7572e672b421aaa17a2832e8cc" : > "$stderr_file" validator_actual_sha256="$( @@ -276,9 +212,7 @@ safe-outputs: } const categories = new Map([ - ['discovery', 'validator skill discovery'], - ['uniqueness', 'validator skill uniqueness'], - ['extraction', 'validator payload extraction'], + ['discovery', 'validator resource discovery'], ['integrity', 'SHA-256 authentication'], ['syntax', 'node --check'], ['validation', 'validator execution'], @@ -1070,10 +1004,11 @@ its charter from scratch. Natural-language review is not the gate. Immediately before requesting safe output, create `$RUNNER_TEMP/squad-cast-payload.json` as a JSON array containing every concrete Step 6 payload path, then invoke the prepared runner with the -exact command below. Do not transcribe validator bytes or extraction commands, -and do not invoke or load `squad-cast-validator` into model context. The runner -was prepared before this turn and deterministically consumes the validator -skill that gh-aw materialized on disk. +exact command below. Do not transcribe validator bytes, and do not invoke or +load `squad-cast-validator` into model context. The runner was prepared before +this turn and deterministically executes the plaintext validator resource that +gh-aw installed at `.github/workflows/shared/squad-cast-validator.mjs` under +`$GITHUB_WORKSPACE`. ```bash @@ -1106,14 +1041,12 @@ Exactly one of these mutually exclusive outcomes is allowed: nonzero, do not request a PR. Classify the stage as `validation` and the command category as `validator execution`. 3. **Validator unavailable or pre-validation failure** — when discovery, - uniqueness, extraction, integrity, or syntax does not complete, do not + integrity, or syntax does not complete, do not request a PR. Classify only from the observed command boundary: | Stage | Command category | |---|---| - | `discovery` | `validator skill discovery` | - | `uniqueness` | `validator skill uniqueness` | - | `extraction` | `validator payload extraction` | + | `discovery` | `validator resource discovery` | | `integrity` | `SHA-256 authentication` | | `syntax` | `node --check` | From f6c5adf517d2604fede4f352d6297cf2c602394f Mon Sep 17 00:00:00 2001 From: Brady Gaster Date: Wed, 2 Sep 2026 11:59:17 -0700 Subject: [PATCH 2/2] test(gh-aw): restore cast-failure output diagnostic tests Restore three tests deleted during the plaintext resource transport migration for unchanged cast-failure safe-output handler behavior: - fails malformed agent output with a deterministic diagnostic - fails non-object output items with a deterministic diagnostic - diagnoses conflicting failure and pull-request outputs without claiming prevention Closes #1984 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- test/gh-aw-cast-validator.test.ts | 35 +++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/test/gh-aw-cast-validator.test.ts b/test/gh-aw-cast-validator.test.ts index 9c069fa2b..63f41ddb9 100644 --- a/test/gh-aw-cast-validator.test.ts +++ b/test/gh-aw-cast-validator.test.ts @@ -493,6 +493,41 @@ describe('GH-AW Cast final-tree validator', () => { ].join('\n'), ]); }); + + it('fails malformed agent output with a deterministic diagnostic', async () => { + const failures = await runCastFailureJobOutput('{not-json'); + expect(failures).toHaveLength(1); + expect(failures[0]).toMatch(/^Unable to read Cast failure output: /); + }); + + it('fails non-object output items with a deterministic diagnostic', async () => { + const failures = await runCastFailureJobOutput(JSON.stringify({ + items: [null, { type: 'cast_failure' }], + })); + expect(failures).toEqual(['Cast failure output item 0 is not an object.']); + }); + + it('diagnoses conflicting failure and pull-request outputs without claiming prevention', async () => { + const failures = await runCastFailureJobOutput(JSON.stringify({ + items: [ + { + type: 'cast_failure', + stage: 'validation', + command_category: 'validator execution', + exit_status: '1', + stderr: 'Cast validation failed.', + }, + { type: 'create_pull_request' }, + ], + })); + expect(failures).toEqual([ + [ + 'Conflicting Cast terminal outputs: found 1 create_pull_request item(s) with cast_failure.', + 'This post-agent diagnostic cannot prevent a concurrently materialized pull request.', + ].join('\n'), + ]); + }); + it('accepts a self-contained descriptive Cast tree', () => { const fixture = createFixture(); const result = validate(fixture.root, fixture.payload);