[gangdeng-intel]

Summary

This PR adds optional decompression code paths that use Intel® Advanced Performance Extensions (Intel® APX). The CPU is checked at runtime, and the APX paths are selected only on processors and operating systems that support APX. All other platforms continue to use the existing code paths unchanged.

The changes are expected to improve decompression performance on future APX-capable processors.

Intel APX architecture specification:
https://www.intel.com/content/www/us/en/content-details/861610/intel-advanced-performance-extensions-intel-apx-architecture-specification.html

Background

Intel APX extends the x86-64 instruction set with, among other features:

• 16 additional general-purpose registers (R16–R31, "EGPR"), which reduce register pressure and stack spills in hot loops
• New Data Destination (NDD) three-operand forms of common integer instructions, which remove extra register-to-register moves
• No Flags (NF) variants of arithmetic instructions, which reduce false dependencies on the flags register
• Conditional compare (CCMP), which lets multiple conditions be evaluated without branches
• PUSH2/POP2, which save and restore two registers per instruction

Zstd's decompression hot paths, especially the Huffman fast loop and sequence decoding, are register-heavy and branchy, so they can benefit from these features.

Changes

The PR is organized as five self-contained commits:

1. common: add APX detection and target attribute

   • Compile-time detection (STATIC_APXF, set when the compiler targets APX, i.e. __APX_F__ is defined, e.g. with -mapxf) and runtime dispatch control (DYNAMIC_APXF)
   • DYNAMIC_APXF is enabled only on x86-64 with clang >= 20 or gcc >= 15, since APX is available only in 64-bit mode
   • DYNAMIC_APXF is forced to 0 for Linux kernel builds (ZSTD_LINUX_KERNEL), so no APX code or extended-register usage is introduced into the kernel
   • Runtime detection checks CPUID.(EAX=7,ECX=1):EDX[21] (APX_F) and OS enablement of the APX extended register state via XCR0[19]
   • APX detection is implemented in a separate ZSTD_cpuSupportsApx() function. The shared ZSTD_cpuid() is left unchanged. The result is computed once per process and cached, so creating a ZSTD_CCtx/ZSTD_DCtx costs no additional CPUID/XGETBV instructions (cf. the per-context CPUID cost discussed in zstd: use x86 feature infrastructure for BMI2 dispatch #4767)
   • APXF_TARGET_ATTRIBUTE for functions compiled with APX code generation enabled

2. common: add APX-optimized XXH64_update

   • XXH64_update_apxf(), an APX-compiled variant used for frame checksum verification during decompression

3. common: add HUF_flags_apxf flag

   • New HUF_flags_e flag used by the decompressor to opt into the APX Huffman fast loop

4. decompress: add APX-optimized HUF_decompress4X1 fast asm loop

   • New assembly variant of the 4-stream Huffman X1 fast decoding loop in huf_decompress_amd64.S
   • Uses EGPR to remove the stack-spill area for ip[1..3]/olimit, PUSH2/POP2 in the prologue and epilogue, NDD/NF to reduce flag-write pressure, and CCMP to merge the three ip ordering checks into one chained compare
   • Existing BMI2 instructions are kept, so the APX loop is only selected where the BMI2 asm loop would be selected
   • With STATIC_APXF (e.g. -mapxf), the APX loop is used directly without runtime detection

5. decompress: add APX dynamic dispatch for sequence decoding

   • The cached APX capability is stored in the ZSTD_DCtx at initialization
   • Dispatches to APX-compiled variants of ZSTD_decompressSequences(), ZSTD_decompressSequencesSplitLitBuffer(), and ZSTD_decompressSequencesLong()
   • Selects the APX Huffman fast loop and the APX XXH64_update when APX is available

Compatibility

• No API or format changes. All changes are internal to the library.
• Non-APX CPUs: behavior is unchanged; the existing BMI2/default paths are used.
• Non-x86-64 targets, 32-bit x86, older compilers, MSVC, and Linux kernel builds: DYNAMIC_APXF evaluates to 0 and none of the APX code is compiled.
• APX runtime dispatch can be disabled at build time with -DDYNAMIC_APXF=0.
• Builds targeting APX (-mapxf): the whole library is already compiled for APX, so no runtime detection or duplicated _apxf C functions are generated; only the APX Huffman asm loop is added.
• Context creation cost: unchanged from upstream after the first APX check in a process. The existing BMI2 CPUID path is not modified.
• Builds with compilers lacking APX support (e.g. gcc < 15) produce code identical in size to upstream (verified with gcc 11).
• The existing BMI2 dispatch, including the recent ZSTD_USE_BMI2() / Linux kernel CPU-feature handling, is preserved. The APX check is evaluated first, and the code falls through to the existing logic when APX is unavailable.

Code size

With APX-capable compilers, the APX variants are compiled in addition to the existing code paths. Measured with gcc 16.2.1 on x86-64 Linux using the default Makefile settings (make -C programs zstd, make -C lib libzstd, i.e. -O3); text as reported by size:

Binary	upstream/dev	This PR	Delta
programs/zstd	1,296,417 B	1,320,313 B	+23,896 B (+1.8%)
lib/libzstd.so	1,093,524 B	1,117,388 B	+23,864 B (+2.2%)

Building this PR with -DDYNAMIC_APXF=0 produces the same sizes as upstream/dev.

Testing

• Correctness on real APX-capable hardware: built with clang 23.1.4 and passed make test (including fuzzer, zstreamtest, decodecorpus, legacy, invalidDictionaries, and the CLI tests), with the APX code paths active. With clang 23 this required MOREFLAGS=-Wno-unused-but-set-global, because of a new -Wunused-but-set-global warning in programs/zstdcli.c. The warning also occurs on upstream/dev and is unrelated to this PR.
• Non-APX hardware (x86-64, Alder Lake): built with gcc 11, gcc 15, gcc 16, and clang 21 with -Werror; make test passed.
• 32-bit x86: -m32 -fPIC builds of the CPU-detection code with gcc and clang (APX code is correctly excluded).
• Linux kernel import: make -C contrib/linux-kernel test passes; APX is compiled out of the imported sources.
• Each commit builds independently with -Werror.

Performance

The changes are expected to improve decompression performance on future APX-capable processors.

No regression on non-APX hardware. Decompression speed was compared between upstream/dev and this PR on an Intel Core i9-12900K (Alder Lake, P-core pinned, turbo disabled). Both were built with the default Makefile flags. The test used zstd -b -d -i30 on the Silesia corpus pre-compressed at levels 1–19 (plus one level-3 file without checksum), with 3 interleaved rounds per build:

Compiler	This PR vs upstream/dev (median per level)
gcc 16.2.1	within ±0.1% at every level
clang 21.1.7	−1.0% to +0.7%, average about −0.1% (within run-to-run variation)

Checklist

• I searched open pull requests and issues: this problem is not already being addressed.
• Bug fix: it comes with a reproducer or a test that fails without the fix.
• Performance change: numbers from repeated runs are included, with CPU, compiler, flags and data.
  A gain in one configuration is not a general gain: see the Performance section of CONTRIBUTING.md.
• I stated below whether AI or other automated tools were used.

Tools used

Copilot (AI) was used to help write the code changes, and run the benchmarks and tests. I reviewed the diff and the benchmark data before submitting.