Repository navigation
feat(compliance): add RMF/FedRAMP/DoD ATO package automation skill - #241
ForeverYoung48 wants to merge 1 commit into
Conversation
Implements the outcome of a full review of the ATO automation skill. The changes fall into five groups. Truth in extraction (the central issue) The extractor defaulted unstated security attributes to their compliant value, so a blueprint that never mentioned a control was documented in the generated SSP as enforcing it. Several of those defaults also contradicted the fabric modules being invoked: modules/gcs defaults versioning to null and modules/cloudsql-instance defaults backup_configuration.enabled to false, yet both were reported as enabled. KMS keys with no version_template were reported as HSM, which fabricates a FIPS 140-3 Level 3 claim at IL5. Attributes now resolve to the real provider or module default, or to an explicit None when the degraded text-scan path cannot determine them. A new DATA_GAP POA&M rule turns those unknowns into tracked findings, and deliberately ignores explicit False values so a deficiency is never double-reported under two identifiers. Against blueprints/il5/postgresql this correctly surfaces an unenforced TLS posture that was previously reported as compliant. Safety of the automation terraform plan is gated behind --allow-terraform-plan and runs with -lock=false -refresh=false; syft and trivy behind --allow-scanners. Semgrep defaults to the bundled offline ruleset, always passes --metrics=off, and refuses cleartext HTTP rulesets. Dependency bootstrapping is fail-closed via COMPLIANCE_ALLOW_BORROWED_DEPS. Mock-detection branches were removed from production code in favour of dependency injection. Binary writers gained boundary enforcement, symlink refusal, zip-bomb guards, and atomic replace. Correctness of generated artifacts allowed_boundary is now passed into export_document rather than only asserted afterwards; artifact enumeration is symlink-safe; --fix takes a timestamped backup before regenerating. Provenance and privacy Stripped third-party Microsoft 365 tenant GUIDs from two eMASS templates and documented origin, hashes, and modifications in templates/PROVENANCE.md. Sanitized vendor names from shipped defaults. Repository hygiene Removed duplicate compliance_skill.md, corrected unverifiable test-count and efficacy claims, added Apache headers, and added a CI job so the engine suite actually runs (pytest.ini's testpaths never reached it). Suite: 407 tests passing on Python 3.9, the declared floor. Verified end to end on blueprints/il5/bigquery: 71 deliverables, 0 unresolved tokens, all schemas valid.
|
Ran an in-depth review and hardening pass across the compliance skill to get the PR ready for production and open source use. Here is a summary of the improvements and cleanups applied across the branch: Accurate extractionPreviously, the extractor defaulted missing or unstated Terraform settings to compliant values (like assuming shielded VMs, SSL, backups, or HSM protection were on). Updated these to reflect genuine provider defaults, or explicit null values when static analysis cannot determine them. Also introduced a new DATA_GAP finding in Operational safetyExternal tooling executions are now strictly controlled. Terraform plan no longer runs automatically on initialized workspaces; it is gated behind the Security and file protectionsAll binary document exporters and hydrators now enforce path boundary checks directly within Open source readinessReplaced hardcoded author strings with a configurable Code cleanup and optimizationFixed missing imports (such as VerificationAll 407 tests pass on Python 3.9 in under 50 seconds (with 2 skipped and 0 failures). An end-to-end run against |
|
Adding @mgcarey-google since he mentioned that this of interest to him. |
1c3c4b9 to
96d05d4
Compare
fe29aea to
c6313f0
Compare
Add `.gemini/skills/compliance/`, a local read-only agent skill that generates Authorization to Operate (ATO) packages from Terraform blueprints and application configuration. Supported regimes and baselines - Supports 11 NIST SP 800-53 Rev. 5 accreditation baselines: FedRAMP Low, FedRAMP Moderate, FedRAMP High, FISMA Low, FISMA Moderate, FISMA High, DoD IL2, DoD IL4, DoD IL5, DoD IL6, and CJIS. - Fails closed on unsupported or unrecognized regimes (GovRAMP/StateRAMP, FedRAMP Tailored/LI-SaaS, and non-NIST frameworks). Deliverables and CLI workflow - Extracts architecture facts via `scripts/extract_system_data.py` from `.tf`, `.tfvars`, `.tfstate`, and `tfplan.json`, including relative Cloud Foundation Fabric (`modules/*`) child modules within the enclosing repository boundary. - Hydrates templates via `scripts/generate_compliance_artifacts.py` into Markdown, Word (`.docx`), YAML, JSON, and macro-enabled Excel (`.xlsm`) under `<target_folder>/ato_artifacts/`: System Security Plan (SSP), 20 policy manuals, Security Control Traceability Matrix (SCTM), Ports, Protocols, and Services Management (PPSM), Hardware/Software inventory, Plan of Action and Milestones (POA&M), FIPS 140-3 matrix, Incident Response runbooks, NIST OSCAL packages, and Path to Authorization. - Validates generated packages via `scripts/validate_compliance_artifacts.py` with structural, schema, and placeholder checks. Security and operational guardrails - Rejects repository-root execution and enforces target-folder containment with symlink and path-boundary checks. - Enforces YAML alias-expansion budgets, linear-time regular expressions, credential scrubbing, and Markdown inline field escaping. - Isolates optional external security scanners (Checkov, Semgrep, Trivy) behind an explicit `--run-scanners` flag using a sanitized private input copy and bounded subprocess execution. - Ignores the skill's local virtualenv and generated per-target output paths in `.gitignore`, and marks binary `.xlsm`/`.docx` files in `.gemini/skills/compliance/.gitattributes`. Verification - Run `.gemini/skills/compliance/scripts/run_tests.py` inside the skill virtualenv (Python 3.11 and Python 3.13).
c6313f0 to
1499d8a
Compare
|
Closing this PR since the compliance skill has moved out of stellar-engine into a standalone, repo-agnostic tool. That lets teams generate and validate ATO packages across different codebases without writing outputs into the scanned repo. The repo-wide Terraform and documentation fixes from this branch continue separately in #286. Appreciate everyone's review and feedback on this work! |
Description
Adds
.gemini/skills/compliance/, a self-contained agent skill andcompliance_enginePython package that generates draft NIST SP 800-53 Rev. 5 Authorization to Operate (ATO) packages from Stellar Engine Terraform blueprints and application manifests. It supports 11 accreditation regimes and fails closed on unsupported ones (GovRAMP / StateRAMP, FedRAMP Tailored / LI-SaaS, and non-NIST frameworks).Building an ATO package by hand means re-deriving the same architecture facts (resources, encryption, ports and protocols, inventory) into the SSP, policies, matrices, and workbooks for every blueprint and regime. The skill extracts those facts from the Terraform that already defines each blueprint, fills the package templates for the selected regime, and validates the result, so teams start from a consistent draft instead of a blank template. Generated packages are drafts for ISSO and assessor review: anything the engine cannot derive from code is left as an explicit
[CONFIG_REQUIRED: ...]marker instead of being invented.This PR only adds
.gemini/skills/compliance/plus.gitignoreentries for the skill's local virtualenv and the per-target files it generates (ato_artifacts/,system_inventory.json,compliance_config.yaml, and Terraform plan JSON). The repository-wide lint and tooling fixes that were previously bundled here are in #286.Fixes: N/A (no tracking issue)
Supported regimes
config/baselines/fedramp_rev5_low.jsonSSP_FedRAMP_High_Template.mdFedRAMP_rev5_LOWconfig/baselines/fedramp_rev5_moderate.jsonSSP_FedRAMP_High_Template.mdFedRAMP_rev5_MODERATEconfig/baselines/fedramp_rev5_high.jsonSSP_FedRAMP_High_Template.mdFedRAMP_rev5_HIGHconfig/baselines/nist_800_53b_low.jsonSSP_FedRAMP_High_Template.mdNIST_SP-800-53_rev5_LOWconfig/baselines/nist_800_53b_moderate.jsonSSP_FedRAMP_High_Template.mdNIST_SP-800-53_rev5_MODERATEconfig/baselines/nist_800_53b_high.jsonSSP_FedRAMP_High_Template.mdNIST_SP-800-53_rev5_HIGHconfig/baselines/dod_il2.jsonSSP_IL5_Template.mdFedRAMP_rev5_MODERATEconfig/baselines/dod_il4.jsonSSP_IL5_Template.mdFedRAMP_rev5_MODERATEconfig/baselines/dod_il5.jsonSSP_IL5_Template.mdFedRAMP_rev5_HIGHconfig/baselines/dod_il6.jsonSSP_IL5_Template.mdFedRAMP_rev5_HIGHconfig/baselines/cjis.jsonSSP_FedRAMP_High_Template.mdNIST_SP-800-53_rev5_MODERATEWhat the skill does
scripts/extract_system_data.py): Parses Terraform HCL (.tf,.tfvars,.tf.json), optional Terraform state and plan JSON, local child modules (bounded to the enclosing git work tree or--module-root), Kubernetes manifests, Dockerfiles, dependency manifests, and CycloneDX / SPDX SBOMs into<TARGET_FOLDER>/system_inventory.json. Child-modulecountandfor_eachare evaluated so inactive resources are never added to the inventory. Symlinked target folders and the repository root are refused.scripts/generate_compliance_artifacts.py): Writes 69 deliverables plus 2 logs (71 files) into<TARGET_FOLDER>/ato_artifacts/with all formats enabled: the SSP (.md/.docx) filtered to the regime's controls and Organization-Defined Parameters, 20 policy manuals (ACthroughSR), the SCTM, PPSM, hardware/software inventory, and POA&M as.xlsmeMASS workbooks and YAML, a FIPS cryptographic matrix, 5 incident response runbooks, a NIST OSCAL v1.2.3 SSP and component definition, and a Path to Authorization. Control statuses, PPSM VPN fields, and cryptographic module attributions come from discovered resources and open POA&M items. Output is byte-identical across runs whenSOURCE_DATE_EPOCHis set.scripts/validate_compliance_artifacts.py): Checks every deliverable for unresolved{{ ... }}tokens,[CONFIG_REQUIRED: ...]and[Assignment: ...]placeholders, OpenXML integrity, workbook data validation, baseline control-set parity, and content substance. Exits1on CAT I findings, unresolved tokens, baseline mismatches, or failed--fixruns, and2on invalid flags. Resolves applicable DISA STIG and SRG benchmarks fromconfig/stig_catalog.json.--run-scannersflag (target config and inventory files cannot enable them), against a dotfile-free temporary copy with an isolatedHOMEand working directory. The engine also enforces path-boundary and symlink checks,defusedxmland YAML alias-expansion budgets, linear-time regular expressions, Markdown escaping, spreadsheet formula-injection neutralization, and credential redaction.pyproject.toml) withcompliance-extract,compliance-generate, andcompliance-validateentry points, and marks.xlsmand.docxassets as binary in.gemini/skills/compliance/.gitattributes.Usage
CLI options
extract_system_data.py <target_dir>:--allow-terraform-plan,--allow-scanners,--module-root <dir>.generate_compliance_artifacts.py <target_dir>:--policy-format {both,markdown,docx},--data-format {both,yaml,excel},--oscal-format {both,json,yaml,none},--oscal-version <ver>,--run-scanners,--scanners <list>,--semgrep-config <path-or-ref>.validate_compliance_artifacts.py <target_dir>:--fix,--policy-format,--data-format,--oscal-format,--oscal-version,--run-scanners,--fill-example-data,--no-fill-example-data,--update-stigs,--stigs-mode {auto,online,offline},--stigs-catalog <source>,--no-linter,--no-linter-strict.Notes for reviewers
.gemini/skills/compliance/is.gitignore.config/(mainly the 11 baseline catalogs), 27,000 are templates, 31,000 are tests, and 33,000 are Python insrc/andscripts/.scripts/run_tests.py(see Testing Performed).Type of Change
Deployment & Compliance Impact
-1policy and procedure controls for all 20 families.Checklist
Code Quality & Reusability
modules/orfast/can be leveraged for this change.documentation/naming-convention.md.Documentation
README.mdof the modified module or blueprint.Security
Testing
Testing Performed
scripts/run_tests.pyon Python 3.11 and Python 3.13: 772 tests ran, 771 passed, 1 skipped (Pandoc HTML), 0 failed.extract->generate->validate --fixruns on 6 repository blueprints (blueprints/il5/bigquery,blueprints/il5/postgresql,blueprints/il5/gke-hardened,blueprints/il5/compute-engine,blueprints/fedramp-high/cloud-run,blueprints/stand-alone/vpc-peering-project) and all 11 regimes: 71 files per run (69 deliverables plus 2 logs), 0 unresolved{{ ... }}tokens, 28/28 valid.docxfiles, 150/150.xlsmdata validation rules preserved, and schema-valid OSCAL v1.2.3 output. The validator's semantic linter returns REJECTED (exit 1) forblueprints/il5/postgresqlandblueprints/stand-alone/vpc-peering-project; the other 4 blueprints validate with exit 0.semgrep --test --config .gemini/skills/compliance/config/semgrep_rules .gemini/skills/compliance/tests/semgrep_fixtures: 14/14 rules pass.flake8 --config=.gemini/skills/compliance/.flake8 --disable-noqa: 0 issues.vulture --min-confidence 80: 0 dead symbols.mypy --ignore-missing-imports: 0 errors in 31 source files.python3 tools/check_boilerplate.py .gemini/skills/compliance/: 0 missing license headers.pyflakes tools/ blueprints/gemini-enterprise/gem4gov-cli/ tests/reports 0 issues andpytestpasses (65 tests).compliance_enginewheel withpip wheel --no-deps, installed it into a clean virtualenv, and verified thecompliance-extract,compliance-generate, andcompliance-validateentry points.