Skip to content

feat(compliance): add RMF/FedRAMP/DoD ATO package automation skill - #241

Closed
ForeverYoung48 wants to merge 1 commit into
google:mainfrom
ForeverYoung48:feature/compliance-skill
Closed

ForeverYoung48 wants to merge 1 commit into
google:mainfrom
ForeverYoung48:feature/compliance-skill

Conversation

@ForeverYoung48

@ForeverYoung48 ForeverYoung48 commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds .gemini/skills/compliance/, a self-contained agent skill and compliance_engine Python package that generates draft NIST SP 800-53 Rev. 5 Authorization to Operate (ATO) packages from Stellar Engine Terraform blueprints and application manifests. It supports 11 accreditation regimes and fails closed on unsupported ones (GovRAMP / StateRAMP, FedRAMP Tailored / LI-SaaS, and non-NIST frameworks).

Building an ATO package by hand means re-deriving the same architecture facts (resources, encryption, ports and protocols, inventory) into the SSP, policies, matrices, and workbooks for every blueprint and regime. The skill extracts those facts from the Terraform that already defines each blueprint, fills the package templates for the selected regime, and validates the result, so teams start from a consistent draft instead of a blank template. Generated packages are drafts for ISSO and assessor review: anything the engine cannot derive from code is left as an explicit [CONFIG_REQUIRED: ...] marker instead of being invented.

This PR only adds .gemini/skills/compliance/ plus .gitignore entries for the skill's local virtualenv and the per-target files it generates (ato_artifacts/, system_inventory.json, compliance_config.yaml, and Terraform plan JSON). The repository-wide lint and tooling fixes that were previously bundled here are in #286.

Fixes: N/A (no tracking issue)

Supported regimes

Regime Baseline Catalog Controls SSP Template OSCAL v1.2.3 Profile
FedRAMP Low config/baselines/fedramp_rev5_low.json 156 SSP_FedRAMP_High_Template.md FedRAMP_rev5_LOW
FedRAMP Moderate config/baselines/fedramp_rev5_moderate.json 323 SSP_FedRAMP_High_Template.md FedRAMP_rev5_MODERATE
FedRAMP High config/baselines/fedramp_rev5_high.json 410 SSP_FedRAMP_High_Template.md FedRAMP_rev5_HIGH
FISMA Low config/baselines/nist_800_53b_low.json 149 SSP_FedRAMP_High_Template.md NIST_SP-800-53_rev5_LOW
FISMA Moderate config/baselines/nist_800_53b_moderate.json 287 SSP_FedRAMP_High_Template.md NIST_SP-800-53_rev5_MODERATE
FISMA High config/baselines/nist_800_53b_high.json 370 SSP_FedRAMP_High_Template.md NIST_SP-800-53_rev5_HIGH
DoD IL2 config/baselines/dod_il2.json 323 SSP_IL5_Template.md FedRAMP_rev5_MODERATE
DoD IL4 config/baselines/dod_il4.json 334 SSP_IL5_Template.md FedRAMP_rev5_MODERATE
DoD IL5 config/baselines/dod_il5.json 624 SSP_IL5_Template.md FedRAMP_rev5_HIGH
DoD IL6 config/baselines/dod_il6.json 627 SSP_IL5_Template.md FedRAMP_rev5_HIGH
CJIS config/baselines/cjis.json 323 SSP_FedRAMP_High_Template.md NIST_SP-800-53_rev5_MODERATE

What the skill does

  • Extract (scripts/extract_system_data.py): Parses Terraform HCL (.tf, .tfvars, .tf.json), optional Terraform state and plan JSON, local child modules (bounded to the enclosing git work tree or --module-root), Kubernetes manifests, Dockerfiles, dependency manifests, and CycloneDX / SPDX SBOMs into <TARGET_FOLDER>/system_inventory.json. Child-module count and for_each are evaluated so inactive resources are never added to the inventory. Symlinked target folders and the repository root are refused.
  • Generate (scripts/generate_compliance_artifacts.py): Writes 69 deliverables plus 2 logs (71 files) into <TARGET_FOLDER>/ato_artifacts/ with all formats enabled: the SSP (.md / .docx) filtered to the regime's controls and Organization-Defined Parameters, 20 policy manuals (AC through SR), the SCTM, PPSM, hardware/software inventory, and POA&M as .xlsm eMASS workbooks and YAML, a FIPS cryptographic matrix, 5 incident response runbooks, a NIST OSCAL v1.2.3 SSP and component definition, and a Path to Authorization. Control statuses, PPSM VPN fields, and cryptographic module attributions come from discovered resources and open POA&M items. Output is byte-identical across runs when SOURCE_DATE_EPOCH is set.
  • Validate (scripts/validate_compliance_artifacts.py): Checks every deliverable for unresolved {{ ... }} tokens, [CONFIG_REQUIRED: ...] and [Assignment: ...] placeholders, OpenXML integrity, workbook data validation, baseline control-set parity, and content substance. Exits 1 on CAT I findings, unresolved tokens, baseline mismatches, or failed --fix runs, and 2 on invalid flags. Resolves applicable DISA STIG and SRG benchmarks from config/stig_catalog.json.
  • Security boundaries: Live scanners (Checkov, Semgrep, Trivy) run only with the --run-scanners flag (target config and inventory files cannot enable them), against a dotfile-free temporary copy with an isolated HOME and working directory. The engine also enforces path-boundary and symlink checks, defusedxml and YAML alias-expansion budgets, linear-time regular expressions, Markdown escaping, spreadsheet formula-injection neutralization, and credential redaction.
  • Packaging: Ships an installable wheel (pyproject.toml) with compliance-extract, compliance-generate, and compliance-validate entry points, and marks .xlsm and .docx assets as binary in .gemini/skills/compliance/.gitattributes.

Usage

python3 -m venv .gemini/skills/compliance/.venv
.gemini/skills/compliance/.venv/bin/python -m pip install --require-virtualenv \
  -r .gemini/skills/compliance/requirements.txt

PY=.gemini/skills/compliance/.venv/bin/python
$PY .gemini/skills/compliance/scripts/extract_system_data.py blueprints/il5/bigquery
$PY .gemini/skills/compliance/scripts/generate_compliance_artifacts.py blueprints/il5/bigquery \
  --policy-format=both \
  --data-format=both \
  --oscal-format=both
$PY .gemini/skills/compliance/scripts/validate_compliance_artifacts.py blueprints/il5/bigquery --fix

CLI options

  • extract_system_data.py <target_dir>: --allow-terraform-plan, --allow-scanners, --module-root <dir>.
  • generate_compliance_artifacts.py <target_dir>: --policy-format {both,markdown,docx}, --data-format {both,yaml,excel}, --oscal-format {both,json,yaml,none}, --oscal-version <ver>, --run-scanners, --scanners <list>, --semgrep-config <path-or-ref>.
  • validate_compliance_artifacts.py <target_dir>: --fix, --policy-format, --data-format, --oscal-format, --oscal-version, --run-scanners, --fill-example-data, --no-fill-example-data, --update-stigs, --stigs-mode {auto,online,offline}, --stigs-catalog <source>, --no-linter, --no-linter-strict.

Notes for reviewers

  • The only file changed outside .gemini/skills/compliance/ is .gitignore.
  • Of the roughly 178,800 added lines, about 86,000 are vendored reference data in config/ (mainly the 11 baseline catalogs), 27,000 are templates, 31,000 are tests, and 33,000 are Python in src/ and scripts/.
  • No CI workflow changes are included. The skill's tests run locally with scripts/run_tests.py (see Testing Performed).

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

Deployment & Compliance Impact

  • Applicable Regimes:
    • US Region Restricted (e.g., Access Policy constraint)
    • FedRAMP Moderate
    • FedRAMP High
    • FedRAMP Moderate
    • DoD IL4
    • DoD IL5
    • General / All
    • Also supported (not listed above): FedRAMP Low, FISMA Low / Moderate / High, DoD IL2, DoD IL6, and CJIS.
  • NIST 800-53r5 Controls: No deployed control implementations change; the skill only reads Terraform and writes documentation. Generated packages document every control in the selected baseline (for example, 410 controls for FedRAMP High and 624 for DoD IL5) and produce the artifacts behind PL-2 (SSP), CA-5 (POA&M), CM-8 (hardware/software inventory), CM-7 (ports, protocols, and services), IR-8 (incident response runbooks), SC-13 (FIPS cryptographic matrix), and the -1 policy and procedure controls for all 20 families.

Checklist

Code Quality & Reusability

  • My code adheres to the Maximize Reusability principle. I have not redefined common elements and have reused existing base configurations and modules where possible.
  • I have checked that no existing module or configuration in modules/ or fast/ can be leveraged for this change.
  • My code follows the established naming conventions outlined in documentation/naming-convention.md.

Documentation

  • I have updated the README.md of the modified module or blueprint.
  • I have added/updated documentation for inputs (variables) and outputs.

Security

  • My change adheres to GCP security best practices and the principle of least privilege.
  • I have ensured compliance with the targeted regime (FedRAMP Moderate, FedRAMP High, IL5, etc.).

Testing

  • I have tested my changes locally.
  • I have included details of my testing in this PR.

Testing Performed

  • scripts/run_tests.py on Python 3.11 and Python 3.13: 772 tests ran, 771 passed, 1 skipped (Pandoc HTML), 0 failed.
  • End-to-end extract -> generate -> validate --fix runs on 6 repository blueprints (blueprints/il5/bigquery, blueprints/il5/postgresql, blueprints/il5/gke-hardened, blueprints/il5/compute-engine, blueprints/fedramp-high/cloud-run, blueprints/stand-alone/vpc-peering-project) and all 11 regimes: 71 files per run (69 deliverables plus 2 logs), 0 unresolved {{ ... }} tokens, 28/28 valid .docx files, 150/150 .xlsm data validation rules preserved, and schema-valid OSCAL v1.2.3 output. The validator's semantic linter returns REJECTED (exit 1) for blueprints/il5/postgresql and blueprints/stand-alone/vpc-peering-project; the other 4 blueprints validate with exit 0.
  • semgrep --test --config .gemini/skills/compliance/config/semgrep_rules .gemini/skills/compliance/tests/semgrep_fixtures: 14/14 rules pass.
  • flake8 --config=.gemini/skills/compliance/.flake8 --disable-noqa: 0 issues. vulture --min-confidence 80: 0 dead symbols. mypy --ignore-missing-imports: 0 errors in 31 source files. python3 tools/check_boilerplate.py .gemini/skills/compliance/: 0 missing license headers.
  • Upstream CI scope on this branch: pyflakes tools/ blueprints/gemini-enterprise/gem4gov-cli/ tests/ reports 0 issues and pytest passes (65 tests).
  • Wheel packaging: built the compliance_engine wheel with pip wheel --no-deps, installed it into a clean virtualenv, and verified the compliance-extract, compliance-generate, and compliance-validate entry points.

@Calvin-Cheng1
Calvin-Cheng1 self-requested a review September 14, 2026 15:26
aghassemlouei added a commit to ForeverYoung48/stellar-engine that referenced this pull request Sep 14, 2026
Implements the outcome of a full review of the ATO automation skill.
The changes fall into five groups.

Truth in extraction (the central issue)

The extractor defaulted unstated security attributes to their compliant
value, so a blueprint that never mentioned a control was documented in
the generated SSP as enforcing it. Several of those defaults also
contradicted the fabric modules being invoked: modules/gcs defaults
versioning to null and modules/cloudsql-instance defaults
backup_configuration.enabled to false, yet both were reported as
enabled. KMS keys with no version_template were reported as HSM, which
fabricates a FIPS 140-3 Level 3 claim at IL5.

Attributes now resolve to the real provider or module default, or to an
explicit None when the degraded text-scan path cannot determine them. A
new DATA_GAP POA&M rule turns those unknowns into tracked findings, and
deliberately ignores explicit False values so a deficiency is never
double-reported under two identifiers. Against blueprints/il5/postgresql
this correctly surfaces an unenforced TLS posture that was previously
reported as compliant.

Safety of the automation

terraform plan is gated behind --allow-terraform-plan and runs with
-lock=false -refresh=false; syft and trivy behind --allow-scanners.
Semgrep defaults to the bundled offline ruleset, always passes
--metrics=off, and refuses cleartext HTTP rulesets. Dependency
bootstrapping is fail-closed via COMPLIANCE_ALLOW_BORROWED_DEPS.
Mock-detection branches were removed from production code in favour of
dependency injection. Binary writers gained boundary enforcement,
symlink refusal, zip-bomb guards, and atomic replace.

Correctness of generated artifacts

allowed_boundary is now passed into export_document rather than only
asserted afterwards; artifact enumeration is symlink-safe; --fix takes a
timestamped backup before regenerating.

Provenance and privacy

Stripped third-party Microsoft 365 tenant GUIDs from two eMASS
templates and documented origin, hashes, and modifications in
templates/PROVENANCE.md. Sanitized vendor names from shipped defaults.

Repository hygiene

Removed duplicate compliance_skill.md, corrected unverifiable test-count
and efficacy claims, added Apache headers, and added a CI job so the
engine suite actually runs (pytest.ini's testpaths never reached it).

Suite: 407 tests passing on Python 3.9, the declared floor.
Verified end to end on blueprints/il5/bigquery: 71 deliverables,
0 unresolved tokens, all schemas valid.
@aghassemlouei

aghassemlouei commented Sep 14, 2026 •

Copy link
Copy Markdown
Member

Ran an in-depth review and hardening pass across the compliance skill to get the PR ready for production and open source use. Here is a summary of the improvements and cleanups applied across the branch:

Accurate extraction

Previously, the extractor defaulted missing or unstated Terraform settings to compliant values (like assuming shielded VMs, SSL, backups, or HSM protection were on). Updated these to reflect genuine provider defaults, or explicit null values when static analysis cannot determine them. Also introduced a new DATA_GAP finding in .gemini/skills/compliance/src/compliance_engine/poam_rules.py so unknown configurations surface as tracked assessment items instead of false positives. CMEK detection was tightened to check real key references instead of matching substrings, and status tracking was aligned across poam_rules.py, semantic_linter.py, and generate_compliance_artifacts.py to prevent discrepancies across deliverables.

Operational safety

External tooling executions are now strictly controlled. Terraform plan no longer runs automatically on initialized workspaces; it is gated behind the --allow-terraform-plan flag, executes in an isolated temporary location with credential environment variables scrubbed, and uses non-locking, non-refreshing flags. Security scanners like Syft and Trivy are gated behind --allow-scanners to avoid unexpected remote database updates. Semgrep is configured to use the local bundled ruleset in .gemini/skills/compliance/config/semgrep_rules/public_sector_baseline.yaml with metrics turned off and cleartext HTTP URLs blocked. Also removed mock-checking logic from production code in favor of standard dependency injection, and updated the artifact validator so running with --fix creates a timestamped backup before touching existing files.

Security and file protections

All binary document exporters and hydrators now enforce path boundary checks directly within .gemini/skills/compliance/src/compliance_engine/export_strategies.py before writing to disk. Directory traversals reject symlinks during recursive walks, Excel template handling includes decompression bounds to guard against zip bombs, and formula injection defenses remain active. Additionally, third-party Microsoft 365 tenant GUIDs were removed from the Excel templates, and their origin and checksums are now documented in .gemini/skills/compliance/templates/PROVENANCE.md.

Open source readiness

Replaced hardcoded author strings with a configurable PREPARED_BY parameter that defaults to "Platform Security & Compliance Engineering Team" across all 20 policy documents, configuration files, docx_generator.py, and oscal_generator.py. Verified that no customer names or sensitive IDs remain. All emojis were removed from tests, templates, and docstrings, and HTML mark tags were replaced with clean text markers like [WARNING] and [INFORMATIONAL]. Also removed the redundant duplicate skill file, brought documented test numbers in line with reality, added Apache 2.0 headers across 56 files to satisfy tools/check_boilerplate.py, and added a test runner job in .github/workflows/ci.yml.

Code cleanup and optimization

Fixed missing imports (such as Callable in stig_resolver.py and typing helpers in test_hardening_scanner_edges.py), removed dead variables and unreachable assignment branches across extract_system_data.py and generate_compliance_artifacts.py, cleaned up unnecessary f-strings, and ensured public helpers like parse_yaml_scalar remain cleanly re-exported for backwards compatibility. Flake8 now passes with zero errors.

Verification

All 407 tests pass on Python 3.9 in under 50 seconds (with 2 skipped and 0 failures). An end-to-end run against blueprints/il5/bigquery successfully generated 71 schema-valid compliance deliverables with zero unresolved placeholders. The skill runs completely self-contained and is compatible with Antigravity as well as standard command line workflows.

@aghassemlouei aghassemlouei added Enhancement New feature or request Security Something is insecure or can be secured Gemini - Government Gemini for Government (G4G) related labels Sep 14, 2026
@aghassemlouei

Copy link
Copy Markdown
Member

Adding @mgcarey-google since he mentioned that this of interest to him.

@aghassemlouei aghassemlouei added compliance Level of Effort - High Complex task taking a week or more; consider breaking this down into smaller issues. Priority - Medium Standard features and non-blocking bugs; important for the current milestone but not urgent Scope - Blueprint Related to solution blueprints in the blueprints directory labels Oct 13, 2026
@ForeverYoung48
ForeverYoung48 force-pushed the feature/compliance-skill branch from 1c3c4b9 to 96d05d4 Compare October 14, 2026 18:48
@github-actions github-actions Bot added Gemini - Enterprise Gemini Enterprise (GE) related Framework - IL5 DoD Impact Level 5 (IL5) compliance regime and controls Framework - FedRAMP High FedRAMP High compliance regime and controls Scope - Fast Related to Fabric FAST deployment stages in the fast directory Scope - Modules labels Oct 16, 2026
@ForeverYoung48
ForeverYoung48 force-pushed the feature/compliance-skill branch from fe29aea to c6313f0 Compare October 16, 2026 13:22
Add `.gemini/skills/compliance/`, a local read-only agent skill that
generates Authorization to Operate (ATO) packages from Terraform
blueprints and application configuration.

Supported regimes and baselines
- Supports 11 NIST SP 800-53 Rev. 5 accreditation baselines: FedRAMP Low,
  FedRAMP Moderate, FedRAMP High, FISMA Low, FISMA Moderate, FISMA High,
  DoD IL2, DoD IL4, DoD IL5, DoD IL6, and CJIS.
- Fails closed on unsupported or unrecognized regimes (GovRAMP/StateRAMP,
  FedRAMP Tailored/LI-SaaS, and non-NIST frameworks).

Deliverables and CLI workflow
- Extracts architecture facts via `scripts/extract_system_data.py` from
  `.tf`, `.tfvars`, `.tfstate`, and `tfplan.json`, including relative
  Cloud Foundation Fabric (`modules/*`) child modules within the
  enclosing repository boundary.
- Hydrates templates via `scripts/generate_compliance_artifacts.py` into
  Markdown, Word (`.docx`), YAML, JSON, and macro-enabled Excel (`.xlsm`)
  under `<target_folder>/ato_artifacts/`: System Security Plan (SSP), 20
  policy manuals, Security Control Traceability Matrix (SCTM), Ports,
  Protocols, and Services Management (PPSM), Hardware/Software inventory,
  Plan of Action and Milestones (POA&M), FIPS 140-3 matrix, Incident
  Response runbooks, NIST OSCAL packages, and Path to Authorization.
- Validates generated packages via
  `scripts/validate_compliance_artifacts.py` with structural, schema, and
  placeholder checks.

Security and operational guardrails
- Rejects repository-root execution and enforces target-folder containment
  with symlink and path-boundary checks.
- Enforces YAML alias-expansion budgets, linear-time regular expressions,
  credential scrubbing, and Markdown inline field escaping.
- Isolates optional external security scanners (Checkov, Semgrep, Trivy)
  behind an explicit `--run-scanners` flag using a sanitized private input
  copy and bounded subprocess execution.
- Ignores the skill's local virtualenv and generated per-target output
  paths in `.gitignore`, and marks binary `.xlsm`/`.docx` files in
  `.gemini/skills/compliance/.gitattributes`.

Verification
- Run `.gemini/skills/compliance/scripts/run_tests.py` inside the skill
  virtualenv (Python 3.11 and Python 3.13).
@aghassemlouei
aghassemlouei force-pushed the feature/compliance-skill branch from c6313f0 to 1499d8a Compare October 17, 2026 23:20
@ForeverYoung48

Copy link
Copy Markdown
Contributor Author

Closing this PR since the compliance skill has moved out of stellar-engine into a standalone, repo-agnostic tool. That lets teams generate and validate ATO packages across different codebases without writing outputs into the scanned repo.

The repo-wide Terraform and documentation fixes from this branch continue separately in #286. Appreciate everyone's review and feedback on this work!

@aghassemlouei aghassemlouei added Bug Something isn't working Documentation Improvements or additions to documentation Priority - High Critical issues blocking development or users; urgent bugs or core features for release and removed Priority - Medium Standard features and non-blocking bugs; important for the current milestone but not urgent Scope - Modules labels Oct 20, 2026
@aghassemlouei aghassemlouei added Scope - Module Related to reusable Terraform modules in the modules directory Framework - IL4 DoD Impact Level 4 (IL4) compliance regime and controls Framework - FedRAMP Moderate FedRAMP Moderate compliance regime and controls and removed Gemini - Government Gemini for Government (G4G) related labels Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug Something isn't working Documentation Improvements or additions to documentation Enhancement New feature or request Framework - FedRAMP High FedRAMP High compliance regime and controls Framework - FedRAMP Moderate FedRAMP Moderate compliance regime and controls Framework - IL4 DoD Impact Level 4 (IL4) compliance regime and controls Framework - IL5 DoD Impact Level 5 (IL5) compliance regime and controls Gemini - Enterprise Gemini Enterprise (GE) related Level of Effort - High Complex task taking a week or more; consider breaking this down into smaller issues. Priority - High Critical issues blocking development or users; urgent bugs or core features for release Scope - Blueprint Related to solution blueprints in the blueprints directory Scope - Fast Related to Fabric FAST deployment stages in the fast directory Scope - Module Related to reusable Terraform modules in the modules directory Security Something is insecure or can be secured

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants