Skip to content

feat(taxonome): integrate taxonomies with Scope - #1597

Draft
Cedric Vidal (cedricvidal) wants to merge 23 commits into
microsoft:cedricvidal-microsoft/gepa-scope-adapterfrom
cedricvidal:cedricvidal-taxonome-integration
Draft

Cedric Vidal (cedricvidal) wants to merge 23 commits into
microsoft:cedricvidal-microsoft/gepa-scope-adapterfrom
cedricvidal:cedricvidal-taxonome-integration

Conversation

@cedricvidal

@cedricvidal Cedric Vidal (cedricvidal) commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Integrates Taxonome into Scope as a project-scoped taxonomy product and exact-run Experiment post-processor.

  • Adds mutable taxonomy/element roots plus immutable sequential revisions in exactly four Cosmos-compatible collections, canonical {id}@rN refs, built-in seeding/copy, YAML import/export, content-SHA integrity, and project isolation.
  • Ships the canonical ax-taxonomy portable identity at semantic version 1.0.0; Scope independently pins its first immutable database revision as ax-taxonomy@r1.
  • Pins one exact taxonomy revision to Experiment revisions/executions and evaluates exact-run ATIF through the Taxonome programmatic API. Full results are gzip-compressed in Blob Storage and published with run/claim CAS; no taxonomy evaluation collection is added.
  • Adds typed Experiment JSON/Markdown readouts, default-off fail-closed feature gating, and API/Portal/CLI parity.
  • Replaces serialized Ax runtime artifacts with standard optimized element YAML, provenance-only manifests, repeatable --element-override, and transactional --output-dir / --in-place optimization.
  • Supports exact optimize --from-candidate ... --in-place promotion so the reviewed YAML used for side-by-side Test evaluation is published atomically without rerunning sampling-managed model selection.
  • Adds separate Using Taxonome and Create your own taxonomy website guides plus generated CLI, format, Ax catalog, and deterministic-handler references.
  • Groups the two guides under Guides → Use and design taxonomies with tested desktop/mobile active navigation, and exposes real Ax/GEPA proposal, metric, best-score, Pareto, and stop signals without synthetic percentages.
  • Replaces the taxonomy card/catalog page with the shared searchable/filterable table model and adds a guarded taxonomy detail route with revision history, exact authored-order element resolution, built-in/user actions, and prompt edits that append both element and taxonomy revisions.
  • Adds one project-scoped bulk taxonomy detail projection used by API, Portal, and scope taxonomy show; no per-row element requests or unrelated global element catalog expansion.
  • Fixes Portal Experiment creation to translate run composition into canonical requests[].workerType JSON, including count/profile-variation expansion, while preserving legacy ordinary run submission.
  • Fixes Experiment Detail run hydration so current and historical attempt vectors render every exact member through bounded bulk request/run-history reads, including explicit deleted and missing states.

Demos

Two corrected headless recordings were produced locally with real commands and model-backed execution. They remain session artifacts pending separate user review and publication authorization.

Before

N/A — new integration.

After

Media publication is intentionally deferred. Automated Storybook interactions and the Docker/browser smoke cover the active Portal surfaces.

Testing

  • pnpm --filter taxonomy-engine test — passed, 326 tests.
  • pnpm --filter ax-taxonomy test — passed, 38 tests.
  • Full Taxonome app + engine source suites — passed, 408 tests.
  • Correction-focused engine/CLI/seeder tests — passed, 38 tests.
  • Portal/API/Experiment integration selection — passed, 95 tests.
  • pnpm --filter static-prompt-evals test:ts — passed, 41 tests.
  • All touched package builds/typechecks — passed.
  • Website Node tests — passed, 9 tests.
  • SITE=https://microsoft.github.io BASE_PATH=/scope pnpm build — passed.
  • Full website Playwright browser/accessibility matrix — passed, 92 tests across light/dark desktop/mobile.
  • Deterministic fake-transport GEPA regression — passed: naive Validation score 0.75, changed candidate 1.0, every final Validation row rescored, and candidate-change events observed.
  • Real authenticated development rehearsal — passed: naive prompt 1/2, two independent Ax-selected changed candidates each 2/2, with the first selected candidate improving Validation from 0.75 to 1.0 in 24 bounded metric calls. The sealed Test packet remained unused during development.
  • Final frozen tutorial comparison — naive held-out Test 3/4 with one reviewed confirmation false positive; Ax-selected optimized YAML 4/4 on the identical packet. Exact candidate promotion produced a focused real implementation.prompt diff and the final run/readout passed.
  • Taxonomy detail contract tests — passed, 6 shared/API tests including project mismatch, historical exact element ordering, and one-bulk-read list enrichment.
  • Taxonomy Portal page/story tests — passed, 8 focused interactions; shared UI Storybook play harness — passed, 40 tests.
  • Portal typecheck/production build and static Storybook build — passed.
  • OpenAPI tests — passed, 36 tests; generated website OpenAPI snapshot updated.
  • CLI command-tree snapshot and production build — passed; live taxonomy show returned the same six ordered exact element refs as the API detail projection.
  • Live authenticated browser smoke — taxonomy table, built-in detail, user prompt edit (element@r1 → @r2 plus taxonomy@r1 → @r2), and historical selection resolving back to exact element@r1 all passed.
  • Credential/runtime recovery — securely recreated API/post-processor/report/judge and Copilot worker with the active token (presence/length only), restored Portal AI generation HTTP 200, registered active coder-acp-copilot@copilot-dev, and verified Submit Run shows GitHub Copilot CLI.
  • Post-processor Copilot SDK auth required the OS CA bundle already used by report-generator; the running container was repaired and transport readiness passed. Dockerfile now installs ca-certificates in both stages. A durable image rebuild was attempted but blocked by registry ECONNRESET/ENOTCONN; no source/test failure remains.
  • CLI command-tree snapshot test — passed.
  • Copilot model scanner recovery — passed: the standard one-shot scanner synced 45 active models, set chamomile as the default, and restored Profile creation; a bounded smoke profile was created and soft-deleted.
  • Experiment canonical payload regression — passed: 12 focused Portal tests, Portal typecheck/build, and a live authenticated Copilot/chamomile Experiment create; the smoke Experiment deletion cascade was started afterward.
  • Experiment Detail exact-run hydration — passed: 13 route tests cover ordered current members, historical run resolution, deleted/missing members, and bounded request/history reads; 53 combined API/OpenAPI tests and 16 Portal Experiment Detail interactions passed. API and Portal builds passed. Live Experiment 2ae16ba4-6e5e-41fb-be0a-73839111877a now renders all six exact rows with status, task, worker, and model.
  • Repository-wide active-source searches found no shipped agent-behavior, 3.0.0, Default Taxonomy, serialized artifact runtime contract, or Taxonome saved-prompt “replay” wording. Remaining replay references are immutable recorded-spike evidence or unrelated SSE log replay.
  • Complete Docker stack smoke — passed:
    • migration 037 and all four taxonomy index sets;
    • exactly one active builtin ax-taxonomy@r1;
    • API list/detail/revision/export with portable version 1.0.0;
    • Portal taxonomy navigation and catalog;
    • Scope CLI list/export/copy;
    • direct taxonome output;
    • exact Experiment taxonomy pin;
    • typed JSON/Markdown readout;
    • pp-atif / pp-taxonomy, post-processor, gateway, and Copilot worker queues.

The root monorepo run completed 3,935 passing tests and 59 failures caused by the local Node 25 localStorage implementation lacking setItem across pre-existing Portal suites. Focused touched Portal suites pass; this environment-only failure was not broadened into unrelated changes.

Acceptance tests use injected fake executors and make no paid/live model calls. Recording acceptance used the authenticated Copilot SDK path: ten reviewed ATIF v1 inputs and five exact Scope-v2 archive roots completed the full six-element taxonomy with no failed roots.

Documentation and compatibility

Updated:

  • docs/architecture/db.md
  • docs/architecture/experiments.md
  • docs/architecture/post-processing.md
  • docs/architecture/taxonome.md
  • docs/architecture/trajectory-taxonomy-format.md
  • ENV_VARIABLES.md
  • Website guides and Taxonome reference pages

Local fresh-database startup ordering is API → register-agents with Copilot enabled → model-scanner-copilot; the standard docker:dev:copilot flow already encodes this, while manual selective restarts must preserve it.

Migration 037 is non-destructive and Cosmos-compatible. The identity replacement is development-only: no legacy active alias or compatibility migration is introduced. Immutable historical evidence remains unchanged. Optimized YAML is the only runtime optimization input; manifests contain bounded provenance and no complete ATIF, examples, demonstrations, or serialized runtime program.

Checklist

  • If Portal features changed, keep CLI capabilities in sync.
  • If Portal components changed, update their Storybook stories.
  • If database changes require a migration, include up() / down() and keep it CosmosDB-compatible.
  • If dependencies changed, update the lockfile and regenerate NOTICE / NOTICE-REVIEW.txt as needed. (Workspace/site dependency updates only; no new third-party notice content.)
  • Video showing the behavior before the suggested change (N/A — new feature)
  • Video showing the behavior after the suggested change (awaiting separate publication approval)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Bound GEPA minibatches to the actual Train corpus so candidate proposals can be evaluated within the metric budget. Surface measured proposal, score, Pareto, and stop signals while preserving plain and JSON output contracts.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Place the two task-oriented guides under Use and design taxonomies, verify desktop and mobile navigation, and make generated code samples keyboard reachable with unique labels.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Allow optimize --in-place to publish a previously selected optimized YAML and its matching provenance manifest without rerunning sampling-managed model selection.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Clip interactive dashboard fields to a fixed physical width so cursor rewrites do not leave stale wrapped rows in terminals and recordings.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep every interactive optimization row below 80 columns so cursor updates replace prior state instead of leaving wrapped stale blocks.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add project-scoped list enrichment and bulk taxonomy detail resolution with bounded revision history and exact ordered element revisions. Expose the same projection through the CLI.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use shared list/table primitives for taxonomy discovery and add guarded exact-revision detail, history, ordered elements, copy/delete actions, and prompt revision editing with Storybook coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Install system CA certificates in builder and runtime stages so the bundled Copilot CLI can validate token-backed SDK authentication.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Map Portal run composition to canonical Experiment request payloads with workerType, preserving legacy request submission behavior and exact profile variation expansion.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolve current and historical Experiment execution vectors into ordered request/run summaries with bounded bulk reads, preserving deleted and missing members.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant