Skip to content

Automation v2: ecosystem health, evidence, and guarded OpenAI review - #6

Merged
qt314wink merged 22 commits into
mainfrom
automation/ecosystem-health-v2
Jul 28, 2026
Merged

qt314wink merged 22 commits into
mainfrom
automation/ecosystem-health-v2

Conversation

@qt314wink

Copy link
Copy Markdown
Contributor

Purpose

Evolve the merged MelodicBloom automation control plane into a portfolio-wide v2 contract for reproducible verification, durable failure evidence, and optional OpenAI-assisted analysis of bounded evidence.

Evidence from the uploaded Actions exports

The three exports cover 37 workflows, 54 jobs, and 17 repositories. The highest-priority signals are:

  • Flore Orthography: build.yml failed 59/59 and currently combines non-frozen installation, lockfile mutation, commit, and push inside normal CI.
  • Wildfire Iridescent: Playwright VRT failed 83.33% across 18 runs; the workflow performs four separate npm install operations and no committed package-lock.json was found.
  • Nacre Orchid Atelier: 97.37% failure across 38 job runs.
  • Omni-Loom Case Study: 100% failure across five runs.
  • AETHER: 62.5% failure across 24 runs, with the reproducible-build job failing 75%.
  • Mochi UI: 75% failure across eight runs.
  • ChromaFlora Prime: 53.33% failure across 15 runs.

What changed

  • adds reusable-node-verify-v2.yml
    • frozen dependency contract
    • lockfile-hash cache identity
    • one deterministic vertical slice
    • concurrency cancellation and timeout
    • logs and machine-readable receipts retained on early failure
    • explicit missing-test semantics
    • deterministic artifact digest
    • no source mutation, deployment, or secret access
  • adds reusable-openai-evidence-review.yml
    • manual/caller-controlled evidence input
    • optional OPENAI_API_KEY; missing key produces an explicit skipped receipt
    • rejects pull_request_target
    • size-bounds and credential-scans evidence
    • checks out the analyzer from the immutable reusable-workflow SHA
    • calls the OpenAI Responses API with strict JSON Schema output and store: false
    • records model, prompt hash/version, source hash, OpenAI/client request IDs, limitations, and approval requirement
    • advisory only: no branches, source edits, deploys, merges, publishing, ad spend, or affiliate enrollment
  • adds governance for conversion hooks, affiliate marketing, ads, generative images/video, generated code, platforms, integrations, and design systems
  • adds caller examples that require replacement with an immutable merge SHA
  • records the July 28, 2026 Actions export audit and prioritized repair queue

Security and authority boundary

  • project-scoped key only, stored as OPENAI_API_KEY in GitHub Actions secrets
  • no secret is printed, committed, placed in issue text, or uploaded
  • no arbitrary URL expansion or command execution
  • issue comments are opt-in and contain only a compact summary
  • full structured analysis remains an evidence artifact
  • all recommendations require human disposition
  • Jennipher remains the promotion approval gate

QA performed before upload

  • JSON parsed successfully
  • analyzer passed node --check
  • all workflow and caller YAML files parsed successfully with Ruby's standard YAML parser
  • control-plane checkout boundary corrected so callers execute the analyzer from the pinned reusable-workflow revision rather than assuming the script exists in the caller repository

Not included

  • no key value
  • no repository secret write
  • no caller adoption yet
  • no branch-protection changes
  • no source mutation in product repositories
  • no deployments, merges, ad campaigns, affiliate links, or generated assets

Next gated phase

After review and merge, pin canary callers to the immutable merge SHA and repair repositories in this order: Flore Orthography, Wildfire Iridescent, Nacre Orchid Atelier, Omni-Loom Case Study, AETHER, Mochi UI, and ChromaFlora Prime. Require seven consecutive green deterministic runs before promoting a new check to required status.

Do not merge without Jennipher approval.

@qt314wink qt314wink added enhancement New feature or request automation labels Jul 28, 2026 — with ChatGPT Codex Connector
@qt314wink
qt314wink marked this pull request as ready for review July 28, 2026 17:14
Copilot AI review requested due to automatic review settings July 28, 2026 17:14
@qt314wink
qt314wink merged commit 51d2de7 into main Jul 28, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Not ready to approve

The OpenAI evidence workflow currently allows evidence-path access to .git/config (credential exfiltration risk) and the analyzer’s Responses API output parsing is brittle and likely to fail against valid responses.

Pull request overview

This PR introduces “Automation v2” for the MelodicBloom control plane: a portfolio-wide contract for deterministic Node verification (with durable evidence receipts) and an optional, guarded OpenAI evidence review workflow that operates only on bounded, caller-supplied evidence.

Changes:

  • Add reusable workflows for deterministic Node verification v2 and guarded OpenAI evidence review (with artifact receipts and concurrency/timeout).
  • Add the OpenAI analyzer script and a corresponding JSON schema file, plus caller examples.
  • Add documentation for the v2 automation model, OpenAI evidence envelope, monetization governance, and an Actions export audit record.
File summaries
File Description
scripts/automation/openai-evidence-review.mjs New analyzer that sends bounded evidence to OpenAI Responses API and writes analysis.json + provenance receipts.
schemas/intelligence/ecosystem-recommendation.schema.json JSON Schema describing the structured OpenAI output envelope.
examples/callers/opportunity-studio.yml Example caller for governed opportunity modes (placeholder SHA to be replaced).
examples/callers/openai-health-review.yml Example caller for health review mode (placeholder SHA to be replaced).
examples/callers/node-web-verify-v2.yml Example caller for deterministic Node verification v2 (placeholder SHA to be replaced).
docs/monetization/OPPORTUNITY_GOVERNANCE.md Governance rules for conversion/affiliate/ads/generative work and approval boundaries.
docs/intelligence/OPENAI_EVIDENCE_REVIEW_V2.md Documentation for evidence envelope, prompt-injection boundary, and prohibited actions.
docs/automation/ECOSYSTEM_AUTOMATION_V2.md Overall v2 contract and rollout sequence for ecosystem automation.
docs/automation/ACTIONS_EXPORT_AUDIT_2026-07-28.md Recorded audit results and prioritized repair queue from Actions exports.
.github/workflows/reusable-openai-evidence-review.yml Reusable workflow to validate evidence, optionally call the analyzer, and retain artifacts.
.github/workflows/reusable-node-verify-v2.yml Reusable workflow implementing frozen-install deterministic checks and evidence receipts.
.github/workflows/control-plane-qa.yml Control-plane QA workflow to validate syntax and enforce safety invariants.

Review details

Comments suppressed due to low confidence (1)

.github/workflows/reusable-openai-evidence-review.yml:99

  • evidence-path can currently reference any file under the checkout (including source/.git/config), and the credential scan only checks a few token patterns. This creates a realistic path to exfiltrate credentials (e.g., checkout tokens) into the evidence sent to OpenAI. Add a path allow/deny check (no .., no absolute paths, forbid .git/) and broaden the credential regex to catch common GitHub token formats and x-access-token:.
          set -euo pipefail
          mkdir -p "$OUTPUT_DIR"
          test -f "$EVIDENCE_PATH"
          bytes=$(wc -c < "$EVIDENCE_PATH")
          if (( bytes == 0 || bytes > MAX_INPUT_BYTES )); then
  • Files reviewed: 12/12 changed files
  • Comments generated: 4
  • Review effort level: Low

Note

Your feedback helps us improve the quality of this feature.
Please use 👍 or 👎 to tell us whether this assessment is correct.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +75 to +78
- name: Checkout caller source revision
uses: actions/checkout@v4
with:
path: source
Comment on lines +5 to +9
const redact = (value) => String(value ?? "Unknown failure")
.replace(/sk-[A-Za-z0-9_-]+/g, "[REDACTED_OPENAI_KEY]")
.replace(/github_pat_[A-Za-z0-9_]+/g, "[REDACTED_GITHUB_TOKEN]")
.replace(/Bearer\s+[^\s]+/gi, "Bearer [REDACTED]")
.slice(0, 2000);
Comment on lines +4 to +7
"title": "Ecosystem Evidence Review",
"type": "object",
"additionalProperties": false,
"required": [
Comment on lines +205 to +210
const outputText = payload.output
?.flatMap((item) => item.type === "message" ? item.content ?? [] : [])
.find((content) => content.type === "output_text")
?.text;
if (!outputText) throw new Error("OpenAI response did not contain output_text");
const analysis = JSON.parse(outputText);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants