Skip to content

fix(blueprints): apply fedramp-high/gitlab GKE control plane and node settings - #345

Merged
aghassemlouei merged 2 commits into
google:mainfrom
digitalresistor:fix/fedramp-gitlab-gke-attrs
Oct 27, 2026
Merged

aghassemlouei merged 2 commits into
google:mainfrom
digitalresistor:fix/fedramp-gitlab-gke-attrs

Conversation

@digitalresistor

@digitalresistor digitalresistor commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Description

The blueprints/fedramp-high/gitlab/ blueprint now authorizes the internal 10.203.0.0/16 range on the private GKE control plane endpoint and runs its node pool as Confidential GKE Nodes, as the cluster call always intended. The cluster call in main.tf passed attributes that the modules/gke-cluster-standard/ module does not declare, and Terraform discarded them without an error.

  • The vpc_config.master_authorized_ranges attribute never reached the cluster. The plan on the main branch enables authorized networks with no CIDR blocks, so only the node subnet primary range reaches the endpoint. The range now goes through the access_config.ip_access.authorized_ranges input, which the module renders as master_authorized_networks_config CIDR blocks.
  • The default_nodepool.remove_default_node_pool attribute duplicated the declared remove_pool attribute, which already removes the default node pool. The change drops the duplicate.
  • The node_config.machine_type attribute has no cluster input. The gke-node-pool module call already sets the same n2d-standard-2 machine type, so the change drops the duplicate.
  • The node_config.confidential_nodes attribute has no cluster input either. The change drops it from the cluster call and sets the node_config.confidential_nodes input from fix(blueprints): apply il5/gke-hardened CIS node settings #344 on the gke-node-pool module call instead.

The node pool also needed two more settings that a test against the GKE API found.

  • The GKE API rejects Confidential GKE Nodes on a boot disk of 15 GB or less, so the node pool disk_size_gb input rises from 10 to 30.
  • The node pool now sets the block-project-ssh-keys metadata key for CIS GKE 4.3, and the gke-nodepool module from fix(blueprints): apply il5/gke-hardened CIS node settings #344 adds disable-legacy-endpoints to it.

Confidential GKE Nodes need no other setting in this blueprint.

  • The n2d-standard-2 machine type supports AMD SEV, the default confidential instance type for the N2D series.
  • The node pool already enables Secure Boot and integrity monitoring through the shielded_instance_config input.
  • The CMEK boot disk and the default pd-balanced disk type work with Confidential VMs, once the disk is larger than 15 GB.
  • The node pool setting applies without the cluster-wide confidential_nodes setting, which would force every node pool in the cluster onto confidential machine types. The gke-cluster-standard module leaves that cluster setting unset.

Every other attribute that the blueprint passes to the gke-cluster-standard and gke-nodepool modules matches a declared input.

The draft PR #311 syncs upstream Fabric v59 and sets authorized_ranges = {} in the same block. After this PR merges, that draft PR keeps the 10.203.0.0/16 entry instead of the empty map.

#344 merged and added the node_config.confidential_nodes input and the metadata merge to the modules/gke-nodepool/ module, which this fix uses, so the branch now carries only the #314 commits.

Fixes #314

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

Deployment & Compliance Impact

  • Applicable Regimes:
    • US Region Restricted (e.g., Access Policy constraint)
    • FedRAMP Moderate
    • FedRAMP High
    • DoD IL4
    • DoD IL5
    • General / All
  • NIST 800-53r5 Controls: AC-3 access enforcement, AC-17 remote access, and SC-7 boundary protection, because the authorized networks list limits which internal sources reach the control plane endpoint. SC-39 process isolation and SC-28 protection of information, because Confidential GKE Nodes encrypt node memory with keys that the host and hypervisor cannot read, which extends the CMEK boot disk protection to data in use.

Breaking Changes

Existing deployments replace the nodes node pool on the next apply.

  • The disk_size_gb and metadata node pool settings are ForceNew on every supported google provider version, so Terraform destroys and recreates the nodes node pool, and GitLab workloads lose their nodes until the new pool is ready.
  • With a google provider version from 6.21.0 to 6.30.x, the confidential_nodes block is ForceNew as well.
  • The authorized networks change updates the cluster in place on every supported provider version.
blueprints/fedramp-high/gitlab: the gke-node-pool node pool now enables Confidential GKE Nodes
on a 30 GB boot disk and blocks project-wide SSH keys. Terraform replaces the node pool
on every supported google provider version. Plan a maintenance window for the GitLab workloads.

Checklist

Code Quality & Reusability

  • My code adheres to the Maximize Reusability principle. I have not redefined common elements and have reused existing base configurations and modules where possible.
  • I have checked that no existing module or configuration in modules/ or fast/stages-aw/ can be leveraged for this change.
  • My code follows the established naming conventions outlined in docs/naming-convention.md.

Documentation

  • I have updated the README.md of the modified module or blueprint.
  • I have added/updated documentation for inputs (variables) and outputs.

The blueprint README does not describe these settings, and the change adds no variables or outputs. #344 documented the new gke-nodepool module input in that module README.

Security

  • My change adheres to GCP security best practices and the principle of least privilege.
  • I have ensured compliance with the targeted regime (FedRAMP Moderate, FedRAMP High, IL4, IL5, etc.).

Testing

  • I have tested my changes locally.
  • I have included details of my testing in this PR.

Testing Performed

A temporary Terraform test with mocked google, google-beta and tls providers planned the blueprint without credentials, using placeholder inputs. Temporary outputs in scratch copies of the gke-cluster-standard and gke-nodepool modules exposed the planned resource attributes. The PR contains none of these temporary files.

  1. terraform fmt -check, terraform init -backend=false and terraform validate in the blueprint directory pass. The init resolves google provider 6.50.0.
  2. On this branch, terraform test with command = plan passes three assertions.
    • The master_authorized_networks_config block lists exactly the 10.203.0.0/16 CIDR block.
    • The remove_default_node_pool attribute is true.
    • The nodes node pool plans confidential_nodes with enabled = true.
  3. The same test against the main branch fails. The plan there has one master_authorized_networks_config block with zero CIDR blocks and no confidential_nodes block on the node pool.
  4. The google provider source settles the replacement impact. In v6.21.0 through v6.30.0, the provider's node_config.go file marks the confidential_nodes block and its enabled field as ForceNew. Since v6.31.0, the provider drops ForceNew and updates the field through the UpdateNodePool API, which the provider changelog lists for 6.31.0. Version 6.50.0 keeps that update path.
  5. bash tools/lint.sh passes every check except Terraform lint, which fails only because the local machine lacks the tflint binary. The CI tflint job runs that check.

@github-actions github-actions Bot added Scope - Module Related to reusable Terraform modules in the modules directory Framework - IL5 DoD Impact Level 5 (IL5) compliance regime and controls Framework - FedRAMP High FedRAMP High compliance regime and controls Scope - Blueprint Related to solution blueprints in the blueprints directory labels Oct 23, 2026
@digitalresistor
digitalresistor force-pushed the fix/fedramp-gitlab-gke-attrs branch from af49624 to 1422e5a Compare October 23, 2026 16:51

@aghassemlouei aghassemlouei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for moving 10.203.0.0/16 into access_config.ip_access.authorized_ranges, cleaning up default_nodepool, and wiring confidential_nodes on module.gke-node-pool. While testing blueprints/fedramp-high/gitlab against the GKE API and inspecting the planned attributes, we found two node pool configuration items to address alongside the rebase after #344.

Comment thread blueprints/fedramp-high/gitlab/main.tf Resolved
Comment thread blueprints/fedramp-high/gitlab/main.tf Resolved
@aghassemlouei aghassemlouei added Bug Something isn't working Level of Effort - Medium Moderate task requiring thought and testing; typically takes a couple of days to a week Priority - Medium Standard features and non-blocking bugs; important for the current milestone but not urgent Security Something is insecure or can be secured labels Oct 26, 2026
… settings

The cluster call passed attributes that the gke-cluster-standard module
does not declare, and Terraform discarded them without an error:

- vpc_config.master_authorized_ranges never reached the cluster, so the
  control plane had authorized networks enabled with no ranges. The
  10.203.0.0/16 range now goes through
  access_config.ip_access.authorized_ranges, which the module renders as
  master_authorized_networks_config cidr_blocks.
- default_nodepool.remove_default_node_pool duplicated remove_pool,
  which already removes the default node pool.
- node_config.machine_type and node_config.confidential_nodes have no
  cluster input. The gke-node-pool module call already sets the
  n2d-standard-2 machine type, which supports Confidential GKE Nodes.

The gke-node-pool module call now sets node_config.confidential_nodes,
the optional gke-nodepool module input that the il5/gke-hardened fix
for google#315 adds, so the GitLab nodes run as Confidential GKE Nodes. This
change needs that module input and merges after it.

Existing clusters see an in-place update of the authorized networks.
With google provider 6.31.0 or later, the provider updates the node pool
confidential_nodes setting in place and GKE recreates the nodes. With a
lock file on a 6.21.0 to 6.30.x provider, the setting is ForceNew and
Terraform replaces the node pool.

Fixes google#314.
… node pool

The GKE API rejects Confidential GKE Nodes on a 10 GB boot disk with
"Disk_size_gb must be greater than 15", and the node pool passed no
metadata, so it lacked block-project-ssh-keys (CIS GKE 4.3).

- Raise the node pool disk_size_gb to 30.
- Set block-project-ssh-keys on the node pool metadata; the gke-nodepool
  module now adds disable-legacy-endpoints to it.
- Extend the node hardening test to the gitlab node pool.
@digitalresistor
digitalresistor force-pushed the fix/fedramp-gitlab-gke-attrs branch from 55420a6 to dc474f7 Compare October 27, 2026 15:37
@digitalresistor
digitalresistor marked this pull request as ready for review October 27, 2026 15:37
@aghassemlouei aghassemlouei removed Scope - Module Related to reusable Terraform modules in the modules directory Framework - IL5 DoD Impact Level 5 (IL5) compliance regime and controls labels Oct 27, 2026

@aghassemlouei aghassemlouei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for updating disk_size_gb = 30, adding block-project-ssh-keys = "true", and adding unit coverage in tests/test_gke_node_hardening.py. Everything looks ready to merge.

@aghassemlouei
aghassemlouei merged commit 2be6eae into google:main Oct 27, 2026
17 checks passed
@digitalresistor
digitalresistor deleted the fix/fedramp-gitlab-gke-attrs branch October 27, 2026 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug Something isn't working Framework - FedRAMP High FedRAMP High compliance regime and controls Level of Effort - Medium Moderate task requiring thought and testing; typically takes a couple of days to a week Priority - Medium Standard features and non-blocking bugs; important for the current milestone but not urgent Scope - Blueprint Related to solution blueprints in the blueprints directory Security Something is insecure or can be secured

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] fedramp-high/gitlab: GKE control plane authorized ranges and node settings are silently ignored

2 participants