[Nguyen Vuong (normanvuong)]

Description
Adds a ROCm 7.2.4 SuperBench Docker image targeting AMD MI300X (gfx942) and MI355X (gfx950).
This replaces the outdated draft #814. It is rebuilt on top of rocm6.4.x.dockerfile as merged
in #837 and does not depend on the ROCm 7.0 PR (#854), which has been dropped in favor of this one.

ROCm 7.2 is the newest ROCm userspace supported on the 6.4.x amdgpu driver used by the current
fleet (per AMD's compatibility matrix), so it can be validated on existing MI300X/MI355X SKUs
without a driver upgrade.

Supersedes #814.

Major Revision

• Add dockerfile/rocm7.2.x.dockerfile, a new ROCm 7.2.4 image (Ubuntu 24.04, Python 3.12,
  PyTorch 2.9.1). Base image: rocm/pytorch:rocm7.2.4_ubuntu24.04_py3.12_pytorch_release_2.9.1.
  • RCCL, rocBLAS and hipBLASLt sources are pinned to the rocm-7.2.4 tag that matches the base
    image, so rebuilds are reproducible and the hipblaslt-bench headers match the installed
    libhipblaslt.so.
  • AMDGPU_TARGETS defaults to "gfx942 gfx950" and can be changed with --build-arg. The
    value is cleaned up and passed to the micro-benchmark builds, rccl-tests, hipblaslt-bench
    and TransformerEngine. If it is empty, CMAKE_HIP_ARCHITECTURES is left out of the
    hipblaslt-bench build so CMake falls back to auto-detection.
  • Only hipblaslt-bench is built, against the hipBLASLt already in the base image. The
    upstream 7.2 build pulls in AMD-internal "origami" headers and a new tensilelite-host
    library even for client-only builds, so it is replaced by a small CMake file,
    dockerfile/etc/hipblaslt-bench-standalone.cmake. LAPACK/CBLAS come from hipBLASLt's
    deps project. The hipBLASLt source and build folders are deleted afterwards.
  • TransformerEngine is pinned to v2.10_rocm. AMD tests that release on ROCm 7.2, and it
    ships AOTriton 0.11.2b for gfx942 and gfx950. CK fused attention stays off, as in 6.4.
    AOTriton is on only when every target is gfx942 or gfx950. onnxscript==0.7.2 and
    onnx==1.23.0 are installed up front, pinned to the versions validated with this
    TE/torch combination.
  • CMake 3.25.3 is built when the base image's CMake is older. rocBLAS 7.2 needs at least
    3.24.4, and the ROCm 7.2 hipblaslt-config.cmake uses block(), which needs 3.25 or later.
• third_party/Makefile: add #include <cassert> to hipBusBandwidth.cpp when it is
  missing. The ROCm 7 clang rejects assert() without it. The step does nothing if the
  include is already there.
• Add rocm7.2 to .github/workflows/build-image.yml with NUM_MAKE_JOBS=8. 7.2 skips the
  memory-heavy Tensile library build that required 6.4's limit of 2.

Minor Revision

• Carried over from Dockerfile - Add ROCm6.4 dockerfile #837: DOCA-Host 3.2.3 user-space OFED, Docker 29.6.2 client only,
  flex, removal of the real OpenMPI clone directory, apt-get clean, rm -rf .git, and an
  apt-get update before the third_party ROCm build.
• Intel MLC v3.13 with SHA-256 verification, matching the other Dockerfiles on main.
• gfortran added to the apt list, because LAPACK in hipBLASLt's deps needs it.
• Build parallelism (OpenMPI, RCCL, hipblaslt-bench, TransformerEngine MAX_JOBS) follows
  NUM_MAKE_JOBS instead of $(nproc). CMAKE_POLICY_VERSION_MINIMUM=3.5 is set for
  CMake 4.x, and the deps project's reserved install target is renamed.
• Add tests/test_rocm72_dockerfile.py, a test that needs no Docker or GPU. It covers how
  AMDGPU_TARGETS is passed on (including the empty case), use of the standalone CMake file,
  the rocm-7.2.4 source pins, MLC checksum verification and the apt refresh.

Not included from #814

• hipBLASLt output parsing by column name: already on main (Enhancement - Fix hipblaslt-gemm result parsing for hipBLASLt v1500+ output format #791).
• gfx1251: out of scope.

Verification

• tests/test_rocm72_dockerfile.py and tests/test_rocm64_dockerfile.py pass.
• Image builds on the self-hosted ROCm runner (passed on fa2e0c0; pending re-run after review fixes).

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Copilot review overview

🟡 Changes recommended

The Dockerfile must honor the configured NUM_MAKE_JOBS limit for its OpenMPI build phases.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1

Open (1)

• Respect NUM_MAKE_JOBS for both OpenMPI build phases · New

What changed in this PR

Adds a ROCm 7.2.4 SuperBench image targeting MI300X and MI355X GPUs.

Changes:

• Adds the ROCm 7.2.4 Dockerfile and standalone hipBLASLt benchmark build.
• Adds architecture-routing tests and CI workflow integration.
• Adds the ROCm 7 HIP compilation compatibility fix.

File	Summary
third_party/​Makefile	Adds the required <cassert> workaround.
tests/​test_rocm72_dockerfile.py	Tests architecture routing and standalone CMake usage.
dockerfile/​rocm7.2.x.dockerfile	Defines the ROCm 7.2.4 image and dependency builds; OpenMPI phases do not honor NUM_MAKE_JOBS.
dockerfile/​etc/​hipblaslt-bench-standalone.cmake	Provides the standalone hipBLASLt benchmark build.
.github/​workflows/​build-image.yml	Registers the ROCm 7.2 image build.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🔵 Needs a closer look

Unresolved build-parallelism issues and incomplete image/runtime verification require human review.

Review effort: Lite
Findings: None

Resolved since last review (1)

• Respect NUM_MAKE_JOBS for both OpenMPI build phases

Previously missed (1)

In code that hasn't changed since last review

Nested RCCL build ignores configured make job limit

dockerfile/​rocm7.2.x.dockerfile:199

The workflow supplies NUM_MAKE_JOBS=8 to this build, but this top-level rocm target still reaches rocm_rccl_tests, whose recipe invokes a nested make ... -j without a job count (third_party/Makefile:148). That bypasses the configured cap and can spawn unbounded RCCL-tests compile jobs on the self-hosted runner, causing avoidable CPU/memory pressure; propagate NUM_MAKE_JOBS to that sub-build (and the other nested builds) or use the make jobserver.

[Copilot]

Copilot review overview

🔵 Needs a closer look

Three moderate build-configuration issues remain unresolved.

Review effort: Lite
Findings: None

Previously missed (1)

In code that hasn't changed since last review

Nested ROCm builds bypass NUM_MAKE_JOBS with unbounded parallelism

dockerfile/​rocm7.2.x.dockerfile:202

The workflow sets NUM_MAKE_JOBS=8, but this invokes the third_party rocm aggregate, whose rocm_perftest and rocm_rccl_tests recipes run nested make -j without a value. Those sub-builds therefore use unlimited parallelism, bypass the configured cap, and can exhaust CPU or memory on the self-hosted runner. Pass the limit through those nested recipes (or replace the bare -j) before relying on this build limit.

[Copilot]

Copilot review overview

🔵 Needs a closer look

Recursive ROCm builds still use unbounded make -j, so the configured job limit may not prevent resource oversubscription.

Review effort: Lite
Findings: None

Previously missed (1)

In code that hasn't changed since last review

Propagate NUM_MAKE_JOBS to nested ROCm builds

dockerfile/​rocm7.2.x.dockerfile:202

This top-level make rocm still invokes several ROCm recipes whose nested commands use unbounded make -j (for example rocm_perftest and rocm_rccl_tests), so the workflow's NUM_MAKE_JOBS=8 does not cap the main third-party build. On the shared self-hosted runner this can oversubscribe CPU and memory despite the explicit caps on OpenMPI, RCCL, and hipBLASLt; pass the job limit through the Makefile's recursive builds (and any relevant upstream install scripts) rather than relying only on the later Dockerfile commands.

[Nguyen Vuong (normanvuong)]

The remaining note about unbounded make -j in the nested third_party ROCm targets (rocm_perftest, rocm_rccl_tests, fio) applies to the shared third_party/Makefile used by every image, not to this Dockerfile. NUM_MAKE_JOBS is already visible to that make because Docker exposes ARG values to RUN. The targets just don't consume it. The same targets build fine in the ROCm 6.3, 6.4 and 7.0 images, so I'd rather cap them in a separate Makefile PR than widen this one.

[Nguyen Vuong (normanvuong)]

The Docker build rocm6.3 failure is a transient Ansible Galaxy timeout during make postinstall (The read operation timed out calling galaxy.ansible.com), not related to this PR. The 6.3 image isn't touched here and built successfully on earlier pushes; rocm6.4 and rocm7.2 pass on fa2e0c0. Could someone with write access re-run the failed job?

[Hongtao Zhang (polarG)]

Thanks for the rework. The image build and arch routing look good. Main asks: pin the hipBLASLt source to the rocm-7.2.4 tag so the bench headers match the installed library, and handle an empty AMDGPU_TARGETS in the hipBLASLt configure step. The rest are smaller hygiene suggestions.

[Hongtao Zhang (polarG)]

Nguyen Vuong (@normanvuong) pls also add 7.2 to the build pipeline follow ROCm6.4 pattern.

[Nguyen Vuong (normanvuong)]

Nguyen Vuong (Nguyen Vuong (@normanvuong)) pls also add 7.2 to the build pipeline follow ROCm6.4 pattern.

7.2 is already in build-image.yml, in the same format as the 6.4 entry, but with NUM_MAKE_JOBS=8 because 7.2 doesn't build the Tensile library that needed 6.4's limit of 2. Did you mean a different pipeline, or should I match 6.4 with NUM_MAKE_JOBS=2?

[Copilot]

Copilot review overview

🟡 Changes recommended

Parallel build steps can oversubscribe runner resources, and source provenance documentation needs reconciliation.

Review effort: Lite
Findings: 1

Open (1)

• Reconcile documented branches with pinned source tags · New

[Hongtao Zhang (polarG)]

Nguyen Vuong (@normanvuong) I meant the same image-build pipeline. I see ROCm 7.2 is already included, so please keep NUM_MAKE_JOBS=8.

I checked the ROCm 6.4 history: the gfx942-only hipBLASLt/Tensile build was still killed with exit code 137 at 8 workers, so c7ccd245 reduced it to 2 and narrowed the Tensile logic catalog as a build-memory workaround.

Since ROCm 7.2 builds only hipblaslt-bench against the preinstalled hipBLASLt library and skips Tensile library generation, it does not need to inherit that 2-job limit.

[Copilot]

🟡 Changes recommended

The uncapped nested RCCL build parallelism should be fixed before approval.

1 open finding

• Propagate NUM_MAKE_JOBS to rocm_rccl_tests nested make · New

1 resolved since last review

• Reconcile documented branches with pinned source tags

🧠 Review effort: Lite

---

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

[Hongtao Zhang (polarG)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.36%. Comparing base (9142bdf) to head (0a707dd).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #855   +/-   ##
=======================================
  Coverage   86.36%   86.36%           
=======================================
  Files         107      107           
  Lines        8258     8258           
=======================================
  Hits         7132     7132           
  Misses       1126     1126           

Flag	Coverage Δ
cpu-python3.10-unit-test	71.95% <ø> (ø)
cpu-python3.12-unit-test	71.95% <ø> (ø)
cpu-python3.7-unit-test	71.41% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

🔵 Needs a closer look

The extensive ROCm Docker, toolchain, and CI integration warrants final human review.

0 open findings

1 resolved since last review

• Propagate NUM_MAKE_JOBS to rocm_rccl_tests nested make

🧠 Review effort: Lite

[Hongtao Zhang (polarG)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).

[Copilot]

🔵 Needs a closer look

The image build, the standalone hipblaslt-bench CMake configure and compile, and the TransformerEngine AOTriton routing were not validated by execution, and the rocm7.2 build that passed on fa2e0c0 predates the review fixes and still needs a fresh run on the final commit.

No open issues 🎉

_{Review effort: Lite}