Repository navigation
chore(repo): fix repo-wide lint failures and make tools/lint.sh fail closed - #286
Conversation
There was a problem hiding this comment.
Code Review
This pull request performs a comprehensive update across the repository, primarily focusing on standardizing copyright headers, updating documentation references (variable and output links) in numerous README files, and improving code quality and maintainability in various CLI tools and scripts. Notable changes include refactoring variable and output definitions in several modules, adding new unit tests for linting tools, and implementing defensive coding practices in Python scripts. The review feedback correctly identifies a potential runtime error in the analytics application, a logic error in the CLI data store validation, a path traversal issue in the linting script, and a potential attribute error in the data store validation logic, all of which are actionable and should be addressed.
|
@aghassemlouei @LanceWray @mgcarey-google @Calvin-Cheng1 can y'all take a look at this when you have a moment? It's the repo-wide lint and tooling cleanup split out of #241, and it stands on its own. I don't have permission to request reviewers on this repo, so feel free to add yourselves. Appreciate your help! |
57dc13b to
9658863
Compare
|
I would like to propose that instead of using 2 spaces for Python code we follow the Google style guide for Python code, and replace See: #304 |
9658863 to
be200ad
Compare
…check_boilerplate - Update `tools/lint.sh` to track failing checks in `FAILED_CHECKS` and exit non-zero when any lint check fails instead of silently succeeding after individual command errors. - Validate `versions.tf` files for `required_version` and `hashicorp/google` declarations rather than requiring an exact byte diff against `default-versions.tf`. - Add `.venv`, `venv`, `__pycache__`, and `node_modules` to `_EXCLUDE_DIRS` in `tools/check_boilerplate.py` so local virtualenvs and tool caches are skipped when scanning `$PWD`. - Add unit tests in `tests/tools/test_lint_sh.py` and `tests/tools/test_check_boilerplate.py` covering fail-closed behavior, `.yml` matching, and excluded directories.
…nd license headers - Add missing Apache 2.0 license headers to 22 legacy files across `.github/`, `blueprints/`, `fast/`, `modules/`, and `scripts/`. - Fix `tools/check_links.py` root-relative link resolution, `tools/tfdoc.py` missing-fixtures handling, and `tools/check_documentation.py` null-doc guard. - Sort Terraform variable and output declarations and refresh `tfdoc` README tables across legacy modules and blueprints. - Add explicit `timeout-minutes` bounds to existing jobs in `.github/workflows/ci.yml`.
…ons.tf check - Prune `.git`, `.terraform`, `.venv`, `venv`, `node_modules`, and `__pycache__` when scanning for `versions.tf` in `tools/lint.sh`. - Add unit tests in `tests/tools/test_lint_sh.py` verifying that mismatched `versions.tf` files inside excluded directories are ignored while invalid `versions.tf` files in tracked directories fail closed.
be200ad to
48fe2d8
Compare
Description
Fixes existing repository-wide lint failures, makes
tools/lint.shfail closed so new failures are caught, and wiresbash tools/lint.shinto.github/workflows/ci.yml(Repository Lint) so every pull request is linted prior to merge intomain. Before this change,tools/lint.shreturned only the exit status of its last command, so failures from earlier checks (license headers,tfdocREADME tables, variable ordering, and Python lint) did not fail the script and had accumulated across the repository.These changes were previously bundled in #241 and are split out so they can be reviewed and merged on their own. The functional
gem4gov-cliempty data store list fix originally pushed to this branch has been split out into #336.Fixes: N/A (no tracking issue)
Changes
tools/lint.shrecords failing checks and exits non-zero if any check fails, and skips.git,.terraform,.venv,venv,node_modules, and__pycache__in itsversions.tfscan while running Python tooling throughuv run --lockedandruff(style(repo): replace yapf with ruff and adopt the Google Python Style Guide's 4-space indent #302, build(tools): run the repository tooling on a locked uv Python 3.14 environment #323).tools/check_boilerplate.pyskips the same virtualenv and cache directories. Also fixes root-relative link resolution intools/check_links.py, missing-fixture handling intools/tfdoc.py, and a null-doc guard intools/check_documentation.py. Tests are intests/tools/test_lint_sh.pyandtests/tools/test_check_boilerplate.py.variables.tfand 5outputs.tffiles and refreshestfdoctables in 63README.mdfiles. No resource, type, default, or value changes.16missing headers added and5gem4gov-cliheaders updated fromGoogle Inc.toGoogle LLC) across.github/,blueprints/,fast/,modules/, andscripts/.Repository Lint(repo-lint) job in.github/workflows/ci.ymlexecutingbash tools/lint.shon everypushandpull_requesttomain, plus explicittimeout-minutesbounds onsecurity-scan,shell-tests, andpython-tests.Notes for reviewers
tools/lint.shnow exits non-zero when any check fails and runs automatically in CI via theRepository Lintjob.versions.tfcheck requires each file to declarerequired_versionand thehashicorp/googleprovider instead of matchingdefault-versions.tfbyte for byte. All 90versions.tffiles onmaindiffer fromdefault-versions.tf, and the previousfind ... -exec diffcheck never failed the script, so enforcing an exact match would have meant rewriting provider constraints across the repository.Type of Change
Deployment & Compliance Impact
Checklist
Code Quality & Reusability
modules/orfast/stages-aw/can be leveraged for this change.docs/naming-convention.md.Documentation
README.mdof the modified module or blueprint.Security
Testing
Testing Performed
bash tools/lint.sh: all 10 checks pass (exit 0).bash tools/run_tests.sh: all suites pass (uv run --locked pytest95 passed, 29 shell syntax checks, 49common-functions.shunit tests, 6tflint.shunit tests, and 58 CodeQL Python AST checks).uv run --locked python tools/check_boilerplate.py .: 0 missing license headers.uv run --locked ruff format --check .anduv run --locked ruff check .: 251 files formatted, 0 issues.versions.tfunder.terraform/modules/is ignored, and an invalidversions.tfin a normal blueprint folder still fails withInvalid versions.tf..tffile: only ordering, whitespace, license headers, and description text changed.