Repository navigation
Description
Classification / status
P3 / RESEARCH / REUSE-FIRST / KILL-FIRST / TEST-ONLY / NOT ADOPTED / NO IMPLEMENTATION AUTHORIZED.
Source: Orca AI Incident Archive · SCHEMA · data quality · dataset stats · full research Page.
Research question: Can independently corroborated, publicly documented failures of AI agents be translated into bounded, harmless, deterministic regression scenarios that reveal new gaps in existing 007 security/admission controls? The archive is a discovery index, not an authoritative incident oracle or a ready-made security benchmark.
Verified source facts and important caveats
- The published
dist/stats.jsonsnapshot hasversion=2026-11-08,count=417, including 177 incident, 93 research, 74 policy, 58 vulnerability, 15 report records.real_harm=trueoccurs in 153 records; 175 false and 89 null. Never report all 417 as actual successful attacks. - Each Markdown record has YAML frontmatter with
kind,type[],severity,confidence,real_harm,ai_involvement,disputed,sourcesand event date.confidence=Ais the editors' grade (359 records), not automatic proof of primary evidence. - The archive has a correction history and explicitly separates disputed/unsupported claims. Its CI checks schema/derived outputs and performs an advisory sample URL check; it does not validate that each cited source proves every claim.
- Sample caveat: GitSpawn has
kind=research,real_harm=falseand a secondary press URL in the record despite A-grade; do not claim verified in-the-wild exploitation. A normal clone does not copy the source repository's.git/configfrom the remote. Replit incident also lists secondary references despite grade A. WorkOS/ARTEX includes an agent-leaving-sandbox narrative, not an independently confirmed exploit of sandbox isolation. - Dataset content is CC BY 4.0; include attribution and the exact archived source commit if using its compilation or summaries. Original linked source materials have their own rights. No malware, exploit code, or customer data is needed.
Ownership / no duplication
- 007 #46: immutable source snapshots, evidence spans, claim/contradiction status and deterministic citations. This issue consumes those contracts; no parallel source/claim database.
- 007 #74: independent admission gates and policy authority.
- 007 #42, #43: action identities, verifier outcomes and replayable run evidence.
- 007 #156: untrusted-repository intake and execution surfaces.
- 007 #161: comment/tool-result contamination tests.
- 007 #160: false facts and dynamic planner.
- OwnAudit #45: supply-chain evidence adapters. No direct implementation of their adapters here.
This issue owns only: incident selection → independent source corroboration → one security invariant per case → candidate inert fixture and acceptance oracle → optional bounded regression evaluation.
Gate 0: desk review / STOP-NO-GAP first
- Pin a single archive commit and read
dist/incidents.json,SCHEMA.md, relevant source Markdown anddocs/data-quality.mdas untrusted research input. Do not auto-fetch all 795 source URLs or copy the whole corpus into 007. - Predeclare 10–15 cases selected across
incident/research/vulnerability,real_harm,ai_involvement,confidence, and distinct classes (rogue destructive action, prompt injection, repo preflight, credential access, supply-chain, sandbox policy, tool confusion). Include negative and disputed controls, not only sensational/critical records. - For each candidate: stable archive record ID, exact archive SHA, event/disclosure dates, link and content digest of genuinely first-hand evidence when available, supporting/contradicting exact spans, attribution and what remains UNKNOWN.
- Flag source-grade discrepancies, unsupported impact figures, evidence after corrections, ambiguous AI participation, duplicates and missing primary sources. Reject a record as a factual anchor when no adequate source exists, but retain the rejection in the research packet.
- Compare the hypothesized invariant to existing 007 tests/issues. Mark ALREADY-COVERED / UNIQUE-GAP / NON-REPRODUCIBLE / UNKNOWN before proposing implementation. STOP if no net-new cases survive.
Case-to-test candidate examples (not executable scripts)
| Archive case | Candidate independent invariant | Existing owner |
|---|---|---|
| Replit DB deletion | An ordinary coding task cannot authorize destructive production-like DB actions; simulate with inert fake DB | #74 |
| GitSpawn | Inspecting a crafted local repo must not execute unapproved pre-model hooks; use inert Git configuration fixtures only | #156 |
| WorkOS/ARTEX | Outbound network and API scopes remain bounded; test on fake endpoints and deny-by-default policy, do not assume a real sandbox escape was proven | #74/#156 |
| LiteLLM compromised releases | An unexpected artifact hash/provider version is rejected against pinned provenance; synthetic manifest only | OwnAudit #45 |
Gate 1: proposed inert regression harness, separate GO required
- Design a 1:1 trace:
record ID → verified first-party claim → trust boundary / required property → synthetic fixture → independent oracle → expected PASS/FAIL/INCONCLUSIVE → 007 action/verifier reference. A record'sseverityorconfidencedoes not supply the oracle. - Per scenario, add one failure fixture and one benign control; evaluate against existing mock/policy/sandbox APIs without real exploits, target hosts, production credentials or client code.
- Intentionally break an evidence span, swap source SHA, inject a disputed claim, and leave an unsupported source unavailable. The test MUST report UNKNOWN/INCONCLUSIVE, never silently accept a favorable result.
- Define independent evidence for actual effects vs simulated capability. A model's interpretation of a case or its self-certified success is not authoritative.
- Capture tool/model version, input/fixture/source digests, selected scope, measured wall-clock and resource cost, emitted actions, rejected policy attempts, verifier results, false positives, unknowns and reviewer effort. Never infer a zero cost from missing usage.
- Classify reproducibility:
VERIFIED-FIXTURE,DESIGN-ONLY,NON-COMPARABLE,REJECTED-EVIDENCEandALREADY-COVERED, with reason.
Decision / kill first
- GO-NARROW: adequate first-party evidence, at least one independently reproducible net-new gap beyond existing 007 tests, bounded runtime and no dangerous effects; authorize only a separate design-review / fixture PR if later explicitly approved.
- KEEP IDEAS / CHANGE: retain typed incident distinctions, correction log, incident-to-invariant trace and negative examples while using existing tests; avoid new framework or data pipeline.
- PARK / KILL: unreliable source attribution, no feasible independent oracle, duplicated baseline coverage, or more work than expected value.
Hard stop: this issue grants no permission to clone and execute untrusted projects, install malicious packages, run exploit proofs, scan third-party systems, auto-download source pages, embed a whole incident database, create CI gates, change policy, deploy, merge, or alter products. Keep proprietary/customer material and credentials out of fixtures and reports.
Activity