Skip to content

chore(repo): fix repo-wide lint failures and make tools/lint.sh fail closed - #286

Merged
aghassemlouei merged 4 commits into
google:mainfrom
ForeverYoung48:chore/repo-lint-hygiene
Oct 22, 2026
Merged

aghassemlouei merged 4 commits into
google:mainfrom
ForeverYoung48:chore/repo-lint-hygiene

Conversation

@ForeverYoung48

@ForeverYoung48 ForeverYoung48 commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Description

Fixes existing repository-wide lint failures, makes tools/lint.sh fail closed so new failures are caught, and wires bash tools/lint.sh into .github/workflows/ci.yml (Repository Lint) so every pull request is linted prior to merge into main. Before this change, tools/lint.sh returned only the exit status of its last command, so failures from earlier checks (license headers, tfdoc README tables, variable ordering, and Python lint) did not fail the script and had accumulated across the repository.

These changes were previously bundled in #241 and are split out so they can be reviewed and merged on their own. The functional gem4gov-cli empty data store list fix originally pushed to this branch has been split out into #336.

Fixes: N/A (no tracking issue)

Changes

  • Lint tooling: tools/lint.sh records failing checks and exits non-zero if any check fails, and skips .git, .terraform, .venv, venv, node_modules, and __pycache__ in its versions.tf scan while running Python tooling through uv run --locked and ruff (style(repo): replace yapf with ruff and adopt the Google Python Style Guide's 4-space indent #302, build(tools): run the repository tooling on a locked uv Python 3.14 environment #323). tools/check_boilerplate.py skips the same virtualenv and cache directories. Also fixes root-relative link resolution in tools/check_links.py, missing-fixture handling in tools/tfdoc.py, and a null-doc guard in tools/check_documentation.py. Tests are in tests/tools/test_lint_sh.py and tests/tools/test_check_boilerplate.py.
  • Terraform and README tables: Sorts variable and output declarations in 15 variables.tf and 5 outputs.tf files and refreshes tfdoc tables in 63 README.md files. No resource, type, default, or value changes.
  • License headers: Adds or normalizes Apache 2.0 headers across 21 files (16 missing headers added and 5 gem4gov-cli headers updated from Google Inc. to Google LLC) across .github/, blueprints/, fast/, modules/, and scripts/.
  • CI: Adds a dedicated Repository Lint (repo-lint) job in .github/workflows/ci.yml executing bash tools/lint.sh on every push and pull_request to main, plus explicit timeout-minutes bounds on security-scan, shell-tests, and python-tests.

Notes for reviewers

  • tools/lint.sh now exits non-zero when any check fails and runs automatically in CI via the Repository Lint job.
  • The versions.tf check requires each file to declare required_version and the hashicorp/google provider instead of matching default-versions.tf byte for byte. All 90 versions.tf files on main differ from default-versions.tf, and the previous find ... -exec diff check never failed the script, so enforcing an exact match would have meant rewriting provider constraints across the repository.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

Deployment & Compliance Impact

  • Applicable Regimes:
    • US Region Restricted (e.g., Access Policy constraint)
    • FedRAMP Moderate
    • FedRAMP High
    • DoD IL4
    • DoD IL5
    • General / All
  • NIST 800-53r5 Controls: None. No control implementations are added or modified, and no deployed resources change.

Checklist

Code Quality & Reusability

  • My code adheres to the Maximize Reusability principle. I have not redefined common elements and have reused existing base configurations and modules where possible.
  • I have checked that no existing module or configuration in modules/ or fast/stages-aw/ can be leveraged for this change.
  • My code follows the established naming conventions outlined in docs/naming-convention.md.

Documentation

  • I have updated the README.md of the modified module or blueprint.
  • I have added/updated documentation for inputs (variables) and outputs.

Security

  • My change adheres to GCP security best practices and the principle of least privilege.
  • I have ensured compliance with the targeted regime (FedRAMP Moderate, FedRAMP High, IL4, IL5, etc.).

Testing

  • I have tested my changes locally.
  • I have included details of my testing in this PR.

Testing Performed

  • bash tools/lint.sh: all 10 checks pass (exit 0).
  • bash tools/run_tests.sh: all suites pass (uv run --locked pytest 95 passed, 29 shell syntax checks, 49 common-functions.sh unit tests, 6 tflint.sh unit tests, and 58 CodeQL Python AST checks).
  • uv run --locked python tools/check_boilerplate.py .: 0 missing license headers.
  • uv run --locked ruff format --check . and uv run --locked ruff check .: 251 files formatted, 0 issues.
  • Negative test for the pruned scan: an invalid versions.tf under .terraform/modules/ is ignored, and an invalid versions.tf in a normal blueprint folder still fails with Invalid versions.tf.
  • Line-level diff of every changed .tf file: only ordering, whitespace, license headers, and description text changed.

@github-actions github-actions Bot added Gemini - Enterprise Gemini Enterprise (GE) related Gemini - Government Gemini for Government (G4G) related Framework - IL5 DoD Impact Level 5 (IL5) compliance regime and controls Framework - FedRAMP High FedRAMP High compliance regime and controls Scope - Fast Related to Fabric FAST deployment stages in the fast directory Scope - Blueprint Related to solution blueprints in the blueprints directory Scope - Modules labels Oct 16, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request performs a comprehensive update across the repository, primarily focusing on standardizing copyright headers, updating documentation references (variable and output links) in numerous README files, and improving code quality and maintainability in various CLI tools and scripts. Notable changes include refactoring variable and output definitions in several modules, adding new unit tests for linting tools, and implementing defensive coding practices in Python scripts. The review feedback correctly identifies a potential runtime error in the analytics application, a logic error in the CLI data store validation, a path traversal issue in the linting script, and a potential attribute error in the data store validation logic, all of which are actionable and should be addressed.

Comment thread tools/lint.sh Resolved
@ForeverYoung48
ForeverYoung48 marked this pull request as ready for review October 16, 2026 13:51
@ForeverYoung48

Copy link
Copy Markdown
Contributor Author

@aghassemlouei @LanceWray @mgcarey-google @Calvin-Cheng1 can y'all take a look at this when you have a moment? It's the repo-wide lint and tooling cleanup split out of #241, and it stands on its own. I don't have permission to request reviewers on this repo, so feel free to add yourselves. Appreciate your help!

@digitalresistor

Copy link
Copy Markdown
Contributor

I would like to propose that instead of using 2 spaces for Python code we follow the Google style guide for Python code, and replace yapf with ruff.

See: #304

@aghassemlouei aghassemlouei added Bug Something isn't working Documentation Improvements or additions to documentation Priority - Medium Standard features and non-blocking bugs; important for the current milestone but not urgent Level of Effort - Medium Moderate task requiring thought and testing; typically takes a couple of days to a week Scope - Module Related to reusable Terraform modules in the modules directory and removed Scope - Modules labels Oct 20, 2026
@ForeverYoung48
ForeverYoung48 force-pushed the chore/repo-lint-hygiene branch from 9658863 to be200ad Compare October 21, 2026 13:49
@ForeverYoung48

Copy link
Copy Markdown
Contributor Author

@digitalresistor,

Rebased onto main to pick up #302 (ruff with 4-space indentation) and #323 (locked uv Python 3.14 environment). All 2-space yapf formatting diffs have been dropped in favor of ruff, and all 10 checks in tools/lint.sh and tools/run_tests.sh pass cleanly.

…check_boilerplate

- Update `tools/lint.sh` to track failing checks in `FAILED_CHECKS` and
  exit non-zero when any lint check fails instead of silently succeeding
  after individual command errors.
- Validate `versions.tf` files for `required_version` and
  `hashicorp/google` declarations rather than requiring an exact byte
  diff against `default-versions.tf`.
- Add `.venv`, `venv`, `__pycache__`, and `node_modules` to
  `_EXCLUDE_DIRS` in `tools/check_boilerplate.py` so local virtualenvs
  and tool caches are skipped when scanning `$PWD`.
- Add unit tests in `tests/tools/test_lint_sh.py` and
  `tests/tools/test_check_boilerplate.py` covering fail-closed behavior,
  `.yml` matching, and excluded directories.
…nd license headers

- Add missing Apache 2.0 license headers to 22 legacy files across
  `.github/`, `blueprints/`, `fast/`, `modules/`, and `scripts/`.
- Fix `tools/check_links.py` root-relative link resolution,
  `tools/tfdoc.py` missing-fixtures handling, and
  `tools/check_documentation.py` null-doc guard.
- Sort Terraform variable and output declarations and refresh `tfdoc`
  README tables across legacy modules and blueprints.
- Add explicit `timeout-minutes` bounds to existing jobs in
  `.github/workflows/ci.yml`.
…ons.tf check

- Prune `.git`, `.terraform`, `.venv`, `venv`, `node_modules`, and `__pycache__`
  when scanning for `versions.tf` in `tools/lint.sh`.
- Add unit tests in `tests/tools/test_lint_sh.py` verifying that
  mismatched `versions.tf` files inside excluded directories are ignored
  while invalid `versions.tf` files in tracked directories fail closed.

@aghassemlouei aghassemlouei left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; all 10 tools/lint.sh checks and CI suites pass.

@aghassemlouei
aghassemlouei merged commit e8c4616 into google:main Oct 22, 2026
17 checks passed
@aghassemlouei aghassemlouei mentioned this pull request Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug Something isn't working Documentation Improvements or additions to documentation Framework - FedRAMP High FedRAMP High compliance regime and controls Framework - IL5 DoD Impact Level 5 (IL5) compliance regime and controls Gemini - Enterprise Gemini Enterprise (GE) related Gemini - Government Gemini for Government (G4G) related Level of Effort - Medium Moderate task requiring thought and testing; typically takes a couple of days to a week Priority - Medium Standard features and non-blocking bugs; important for the current milestone but not urgent Scope - Blueprint Related to solution blueprints in the blueprints directory Scope - Fast Related to Fabric FAST deployment stages in the fast directory Scope - Module Related to reusable Terraform modules in the modules directory

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants