Skip to content

feat(networking)!: modernize FedRAMP stage 2 with NCC hub, NGFW policies, PSC APIs, and SWP egress - #404

Merged
aghassemlouei merged 11 commits into
mainfrom
feat/stage2-ncc-factory
Nov 10, 2026
Merged

aghassemlouei merged 11 commits into
mainfrom
feat/stage2-ncc-factory

Conversation

@Calvin-Cheng1

Copy link
Copy Markdown
Collaborator

Description

Modernizes fast/stages-aw/2-networking-a-fedramp to implement the 1st-Party Google Cloud Network Landing Zone architecture (leaving 2-networking-b-il5-ngfw untouched):

  • Central Hub + Default STAR NCC Hub (net-hub.tf, factory-ncc.tf, schemas/ncc-hub.schema.json): Replaces the DoD VDSS two-VPC + simple-nva layout (net-vdss.tf, nva.tf, module.peering-envs) with module.hub-project (<prefix>-net-hub-host), module.hub-vpc (hub-0), and a default STAR-topology Network Connectivity Center hub (main, enable_default_ncc_hub = true, export_psc = true) placing hub-0 in center and <env>-spoke-0 VPCs in edge, extensible via data/ncc-hubs/*.yaml and var.ncc_hubs for vpcs, tunnels, vlan_attachments, and router_appliances spokes.
  • Global Network Firewall Policies (modules/net-firewall-policy): Replaces per-VPC classic firewall rules (modules/net-vpc-firewall, var.firewall_rules) with per-VPC Global Network Firewall Policies (module.hub-firewall-policy, module.env-spoke-firewall-policy, module.egress-firewall-policy, firewall_policy_enforcement_order = "BEFORE_CLASSIC_FIREWALL") supporting Cloud NGFW Essentials (L4 + IAM secure tags), Standard (fqdns, region_codes, threat_intelligences), and Enterprise (apply_security_profile_group via var.security_profile_group_ids + tls_inspect) inline via var.firewall_policy_rules or YAML files under data/firewall-policies/.
  • Per-VPC Private Service Connect (PSC) Endpoints for Google APIs (psc-google-apis.tf, var.psc_google_apis): Provisions dedicated non-transitive PSC Google APIs endpoints (modules/net-address targeting all-apis or vpc-sc), private p.googleapis.com. DNS zones (modules/dns), and googleapis Cloud DNS response policies (modules/dns-response-policy) in hub-0 (accessible from on-premises via inbound DNS forwarding), each <env>-spoke-0 VPC, and the optional egress-0 VPC.
  • Distributed Cloud NAT or Centralized SWP Next-Hop Egress (egress.tf, var.enable_cloud_nat, var.centralized_egress): Defaults to per-VPC Cloud NAT gateways, with an opt-in egress-0 VPC spoke (centralized_egress.enabled = true) in the NCC main/center group running Secure Web Proxy (modules/net-swp) in NEXT_HOP_ROUTING_MODE paired with ENDPOINT_TYPE_SWG Cloud NAT and automatic 0.0.0.0/0 next-hop IP routes from hub-0 and <env>-spoke-0.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

Deployment & Compliance Impact

  • Applicable Regimes:
    • US Region Restricted (e.g., Access Policy constraint)
    • FedRAMP Moderate
    • FedRAMP High
    • DoD IL4
    • DoD IL5
    • General / All
  • NIST 800-53r5 Controls: Satisfies SC-7 (Boundary Protection) and AC-4 (Information Flow Enforcement) in 2-networking-a-fedramp via NCC STAR center/edge group isolation, folder hierarchical + per-VPC Global Network Firewall Policies (Cloud NGFW Essentials/Standard/Enterprise), per-VPC PSC Google APIs endpoints, and distributed Cloud NAT or centralized Secure Web Proxy L7 egress without unmanaged simple-nva VMs.

Checklist

Code Quality & Reusability

  • My code adheres to the Maximize Reusability principle. I have not redefined common elements and have reused existing base configurations and modules where possible.
  • I have checked that no existing module or configuration in modules/ or fast/stages-aw/ can be leveraged for this change.
  • My code follows the established naming conventions outlined in docs/naming-convention.md.

Documentation

  • I have updated the README.md of the modified module or blueprint.
  • I have added/updated documentation for inputs (variables) and outputs.

Security

  • My change adheres to GCP security best practices and the principle of least privilege.
  • I have ensured compliance with the targeted regime (FedRAMP Moderate, FedRAMP High, IL4, IL5, etc.).

Testing

  • I have tested my changes locally.
  • I have included details of my testing in this PR.

Testing Performed

  • terraform fmt -recursive -check & terraform validate on 2-networking-a-fedramp
  • terraform plan -refresh=false across 3 topologies:
    1. Default (psc_google_apis.enabled = true, enable_cloud_nat = true, Global Network Firewall Policies, main STAR NCC hub)
    2. Centralized SWP Next-Hop Egress (centralized_egress.enabled = true, NEXT_HOP_ROUTING_MODE, ENDPOINT_TYPE_SWG NAT, vpc-sc PSC bundle, Cloud NGFW Standard fqdns/threat_intelligences + Enterprise apply_security_profile_group rules)
    3. Legacy PGA / Air-Gapped Fallback (psc_google_apis.enabled = false, enable_cloud_nat = false)
  • tools/tflint.sh, tools/check_documentation.py, tools/check_links.py, tools/check_names.py, tools/check_boilerplate.py
  • pytest & ruff check

Breaking Changes

In `fast/stages-aw/2-networking-a-fedramp`, the VDSS DMZ VPC (`module.dmz-vpc`), `simple-nva` instances (`nva.tf`, `var.nva_spot_vms`), spoke VPC peerings (`module.peering-envs`), and classic per-VPC firewall rules (`var.firewall_rules`) are replaced by a central hub Shared VPC (`module.hub-vpc` in `module.hub-project`, `<prefix>-net-hub-host`), a default `STAR`-topology NCC hub (`var.enable_default_ncc_hub`), Global Network Firewall Policies (`var.firewall_policy_rules`), per-VPC PSC Google APIs endpoints (`var.psc_google_apis`), and per-VPC Cloud NAT (`var.enable_cloud_nat`) or optional centralized Secure Web Proxy next-hop egress (`var.centralized_egress`). Update `terraform.tfvars` to replace `subnets.landing` and `subnets.dmz` with `subnets.hub`, migrate `firewall_rules` entries to `firewall_policy_rules`, remove `nva_spot_vms`, and read `outputs.tfvars.hub` instead of `outputs.tfvars.vdss`.

…ess to FedRAMP stage 2

  * Replace per-VPC classic firewall rules in 2-networking-a-fedramp with Global Network Firewall Policies supporting Cloud NGFW Essentials, Standard, and Enterprise rules.
  * Provision per-VPC Private Service Connect endpoints for Google APIs (all-apis or vpc-sc), p.googleapis.com private DNS zones, and Cloud DNS response policies across hub, spoke, and optional egress VPCs.
  * Add an optional centralized egress-0 VPC spoke running Secure Web Proxy in NEXT_HOP_ROUTING_MODE with ENDPOINT_TYPE_SWG Cloud NAT and 0.0.0.0/0 next-hop IP routing.
@github-actions github-actions Bot added Documentation Improvements or additions to documentation Scope - Fast Related to Fabric FAST deployment stages in the fast directory Enhancement New feature or request Framework - FedRAMP Moderate FedRAMP Moderate compliance regime and controls Framework - FedRAMP High FedRAMP High compliance regime and controls Level of Effort - High Complex task taking a week or more; consider breaking this down into smaller issues. Priority - High Critical issues blocking development or users; urgent bugs or core features for release Security Something is insecure or can be secured labels Nov 9, 2026
@Calvin-Cheng1
Calvin-Cheng1 marked this pull request as draft November 9, 2026 18:25
…vor of PSC and opt-in psa_ranges

  * Remove the legacy psc.tf symlink from 2-networking-a-fedramp so spoke VPCs no longer create unconditional /22 Private Services Access VPC peerings by default.
  * Generalize psa_ranges to a map keyed by environment name so workloads that still require Private Services Access can opt in with explicit IPAM ranges.
  * Resolve merge conflicts in 2-networking-a-fedramp and tests/test_stage2_networking.py after syncing Cloud Foundation Fabric v60.0.0.
  * Align subnets[*].secondary_ip_ranges and create_googleapis_routes.directpath in 2-networking-a-fedramp with modules/net-vpc v60.0.0.
…ll context, and SWP TLS inspection

  * Replace raw google_compute_project_metadata with compute_metadata on module.hub-project and module.env-spoke-projects.
  * Replace raw google_compute_address resources in connectivity-tests.tf with modules/net-address internal_addresses.
  * Wire CFF v60 context and Cloud NGFW Enterprise packet-mirroring rules across Stage 2 firewall policies, plus optional SWP tls_inspection_config and PSC service_directory_registration.
…, and default SAs

  * Enable Cloud NAT audit logging (logging_filter = ALL) across hub, spoke, and centralized SWP egress NAT gateways for NIST 800-53r5 AU-2/AU-12/SC-7 compliance.
  * Enable full VPC Flow Logs on the centralized egress-swp-0 subnet (CIS 3.9) and deprivilege default Compute Engine service accounts on all network host projects (CIS 1.5, NIST AC-6).
  * Attach googleapis DNS response policy and org-domain DNS peering to egress-0 in both PSC and legacy PGA modes.

@aghassemlouei aghassemlouei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good Evening @Calvin-Cheng1!

Thank you for opening #404, modernizing FedRAMP Stage 2 around a central hub VPC, the Network Connectivity Center factory, global network firewall policies, and per-VPC Private Service Connect endpoints, and aligning the stage with the synced Cloud Foundation Fabric v60.0.0 modules. Separating the FedRAMP Moderate, FedRAMP High, and IL2 topology from the IL4 and IL5 NGFW appliance stack makes Stage 2 much cleaner to deploy and maintain.

All required GitHub Actions checks, check_documentation.py, Python unit tests, a six-scenario terraform plan matrix, and live Google Cloud smoke provisioning and teardown of the default hub, spoke, firewall policy, Network Connectivity Center STAR group, Private Service Connect all-apis, Cloud DNS response policy, Cloud NAT, and CIS alerting stack succeeded against commit 0e686cd. Testing centralized_egress, renamed hub subnets, standalone folder creation, Private Service Connect Service Directory DNS zones, and folder-scoped billing_override execution surfaced the apply, destroy, and plan-time items commented with inline suggestions below, plus two cross-file VDSS cleanup items outside the current diff:

  • scripts/deploy.sh at lines 98 to 126 and scripts/restore.sh at lines 25, 34, and 59 to 68 still reference ${PREFIX}-net-vdss-host, module.vdss-host-project, vdss-dmz-0, vdss-landing-0, and module.kms keyring vdss-keyring when Stage 2a is selected, which breaks brownfield import and project restoration on the new hub layout.

  • blueprints/gemini-enterprise/README.md at lines 163 to 174 and 652 to 687 plus docs/ddg.md at line 518 still describe 2-networking-a-fedramp as deploying net-vdss-host, vdss-dmz-0, vdss-landing-0, and NVA clusters through nva.tf and net-vdss.tf.

Comment thread docs/tdd.md Resolved
… factories

- Route centralized SWP egress via next_hop_ilb and rely on SWP managed NAT
- Drop manual p.googleapis.com zones that collide with auto-created Service Directory PSC zones
- Parent host projects on module.folder.id, support common_services/AW folder fallback, and move Shared VPC host activation onto net-vpc
- Avoid hardcoded hub-default subnet in connectivity tests and fix host_project_ids/host_project_numbers outputs
- Simplify factory-ncc.tf with deep-merged NCC groups and site_to_site_data_transfer = false default
- Restore built-in data/dns-policy-rules.yaml via dns-response-policy factory/context and remove unused dns.resolvers
- Refresh FedRAMP system boundary diagram, docs, and Stage 2a paths in deploy.sh and restore.sh
@github-actions github-actions Bot added Gemini - Enterprise Gemini Enterprise (GE) related Scope - Blueprint Related to solution blueprints in the blueprints directory labels Nov 10, 2026
@Calvin-Cheng1
Calvin-Cheng1 marked this pull request as ready for review November 10, 2026 06:33

@aghassemlouei aghassemlouei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Calvin-Cheng1 for resolving all 14 review threads and the cross-file VDSS references in commit 6324874. All GitHub Actions checks, Stage 2 unit tests, and five terraform plan topologies covering default Private Service Connect, vpc-sc, centralized Secure Web Proxy egress, air-gapped Private Google Access, and custom Network Connectivity Center hubs pass cleanly.

One non-blocking follow-up note on optional centralized_egress.route_tags in fast/stages-aw/2-networking-a-fedramp/egress.tf at lines 173 and 189 and fast/stages-aw/2-networking-a-fedramp/variables.tf at line 71: Network Connectivity Center static routes require cross-spoke next_hop_ilb routes to be untagged, so setting a non-empty route_tags list succeeds during terraform apply while dropping outbound packets from tagged hub and spoke VMs at runtime. Default centralized_egress.enabled deployments leave route_tags unset and route cleanly, so route_tags can be removed in a small follow-up PR after merge.

@aghassemlouei
aghassemlouei merged commit d87fdc3 into main Nov 10, 2026
23 checks passed
@aghassemlouei
aghassemlouei deleted the feat/stage2-ncc-factory branch November 10, 2026 17:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Documentation Improvements or additions to documentation Enhancement New feature or request Framework - FedRAMP High FedRAMP High compliance regime and controls Framework - FedRAMP Moderate FedRAMP Moderate compliance regime and controls Gemini - Enterprise Gemini Enterprise (GE) related Level of Effort - High Complex task taking a week or more; consider breaking this down into smaller issues. Priority - High Critical issues blocking development or users; urgent bugs or core features for release Scope - Blueprint Related to solution blueprints in the blueprints directory Scope - Fast Related to Fabric FAST deployment stages in the fast directory Security Something is insecure or can be secured

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants