Skip to content

ext/standard: Add SSE2/NEON fast path for bin2hex()/hex2bin() - #23789

Open
adapik wants to merge 1 commit into
php:masterfrom
adapik:binhex-simd
Open

adapik wants to merge 1 commit into
php:masterfrom
adapik:binhex-simd

Conversation

@adapik

@adapik adapik commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

bin2hex()/hex2bin() currently run a plain byte-by-byte scalar loop. This adds a SIMD fast path (zend_simd.h: SSE2 on x86-64, NEON on aarch64) that processes 16 bytes at a time, falling back to the existing scalar loop for tails and, in hex2bin, for any chunk containing an invalid hex character - so error handling and output stay byte-for-byte identical to before.

Who benefits: anything hex-encoding/decoding hashes, checksums, UUIDs, MAC addresses, tokens, or signatures — hash()/md5()/sha1() formatting, crypto/signature verification, logging, cache/session keys. This is the dominant real-world call shape for both functions.

Precedent: vectorizing hot byte/string functions via this header is established practice — stripslashes SSE2, bcmath ported to zend_simd.h, SSE2 mb_strlen, zend_simd.h itself.

Benchmarks

Linux x86-64 macOS arm64
CPU AMD Ryzen 5 7500F Apple M2 Silicon, 10 cores
SIMD path SSE2 NEON

Crypto/hash-sized inputs

Size bin2hex x86 bin2hex arm64 hex2bin x86 hex2bin arm64
4B (CRC32) ~1.0× ~1.0× ~1.0× ~1.0×
16B (MD5/UUID) 1.53× 1.34× 2.97× 2.15×
32B (SHA-256) 1.98× 1.76× 4.06× 2.92×
64B (SHA-512) 2.93× 2.38× 5.04× 3.65×
256B (RSA-2048 sig) 5.46× 4.56× 6.38× 4.87×
512B (RSA-4096 sig) 6.86× 5.46× 6.85× 4.98×

4B/6B show no win on either function — both are below the size where the SIMD path even engages (16 bytes for bin2hex, one full 32-char chunk for hex2bin), so those calls run the unmodified scalar loop. Not a regression, nothing changed for them.

Full corpus (throughput, branch/patched vs baseline/unpatched)

Size bin2hex x86 bin2hex arm64 hex2bin x86 hex2bin arm64
1KB–512KB up to 9.9× (20.6 GB/s) up to 6.8× up to 7.3× up to 5.7×
1MB–16MB 1.4–1.6× 2.6–3.0× 3.7–7.3× 3.6–5.1×

Allocator-boundary step-down, not a SIMD regression

strace on x86, 20 calls each: crossing the ~2MB output-buffer threshold takes bin2hex from 6 munmap/24 mmap calls to 65/63 — the allocator switches from heap reuse to a fresh mmap/munmap pair almost every call. That fixed syscall cost is the same for branch and baseline, but eats a much bigger share of the branch's now-tiny per-call time, which is why the throughput ratio compresses from ~10× down to 1.4–3× above 1MB rather than the branch getting slower in absolute terms.

@adapik adapik changed the title Zend/standard: Add SSE2/NEON fast path for bin2hex()/hex2bin() ext/standard: Add SSE2/NEON fast path for bin2hex()/hex2bin() Sep 20, 2026
@ArtUkrainskiy

Copy link
Copy Markdown
Contributor

I measured this on master c65321c (c4c2ad7 merges cleanly) with GCC 13.3, GCC 14.2, clang 18.1 and clang 20.1 on an i7-13700H, built with the flags of the official Docker image. On big-endian AArch64 hex2bin() returns wrong bytes; details and a two-line fix are in my review comments on string.c.

Change in CPU cycles per call with the PR, relative to master (negative = fewer cycles, faster); median of five alternating runs:

call master, CPU cycles per call (GCC 13.3) GCC 13.3 GCC 14.2 clang 18.1 clang 20.1
bin2hex(), 8 bytes 82 +4% +1% 0% +3%
bin2hex(), 16 bytes 97 −25% −27% −22% −31%
bin2hex(), 32 bytes 129 −41% −36% −37% −44%
bin2hex(), 256 bytes 618 −79% −78% −76% −76%
bin2hex(), 64 KB 133,748 −86% −85% −85% −85%
hex2bin(), 16 hex characters 124 +2% +2% +3% +2%
hex2bin(), 32 hex characters 182 −52% −53% −48% −55%
hex2bin(), 64 hex characters 307 −65% −67% −64% −67%
hex2bin(), 512 hex characters 1,985 −80% −77% −83% −83%
hex2bin(), 128 KB of hex 473,088 −84% −79% −89% −87%
md5() of 32 bytes 443 −9% −10% −11% −11%
hash('sha256') of 32 bytes 440 −14% −12% −17% −12%

Short hex2bin() inputs. Below 32 hex characters the SIMD loop never runs, but hex2bin() executes 3–5% more instructions there and takes 1–4% more cycles: the non-inlined call to php_hex2bin_simd_chunk() inside zif_hex2bin() makes the compiler save two more callee-saved registers and keep three values on the stack on every call. In builds with -falign-functions=64 -falign-loops=32 this showed up as 13–20% more cycles with clang 20.1. Putting the SIMD loop and the scalar tail after it into one non-inlined function, the way zend_bin2hex() already does it, brings these sizes to within 2% of master's instructions and 3% of its cycles, with the same gains from 32 hex characters up: tail-variant.diff.

Correctness on x86-64: output identical to master with all four compilers and under ASan/UBSan, on every length up to 300 bytes for bin2hex() and 600 characters for hex2bin() (odd lengths too), every byte value at every position around the chunk boundaries, all character pairs at the chunk edges, and 200,000 random strings with invalid bytes mixed in; the harness catches seven deliberate mutations of the PR. I found no problem in the restart of the scalar loop after a rejected chunk.

Tests: in hex2bin_simd_boundary.phpt the 2000 case tests 220 characters (str_repeat('0123456789abcdefABCDEF', 10) is 220 long), and no case puts an invalid character in the scalar tail after a completed SIMD loop, for example offsets 32 and 33 of a 34-character string or 64 and 65 of a 66-character one.

Full tables, scripts and the big-endian harness: https://github.com/ArtUkrainskiy/php-src-bench/tree/main/reports/pr-23789-bin2hex-hex2bin

Comment thread ext/standard/string.c Outdated
static MUTEX_T locale_mutex = NULL;
#endif

#ifdef XSSE2

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On big-endian AArch64 this helper returns true for valid input but writes wrong bytes. zend_simd.h enables the NEON path for every __aarch64__ target, and php-src's configure handles aarch64_be. I cut this function out of the PR into a freestanding program, built it for aarch64_be-none-linux-gnu with clang 18.1 and 20.1 (-O2 and -O0) and ran it under qemu-aarch64_be 8.2.2. For all 50,000 random valid 32-character chunks it returns true and writes 16 zero bytes, for example:

in   b25Db6b0507f311F7C5acBeBbD66FDe3
got  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Invalid chunks are still rejected, zend_bin2hex_simd() gives correct output there, and the same program built for little-endian AArch64 is correct. Harness and output: https://github.com/ArtUkrainskiy/php-src-bench/tree/main/reports/pr-23789-bin2hex-hex2bin#big-endian-aarch64

Excluding big-endian targets, as zend_inline_hash_func() does for its aarch64 code, avoids it:

Suggested change
#ifdef XSSE2
#if defined(XSSE2) && !defined(WORDS_BIGENDIAN)

Comment thread ext/standard/string.c Outdated

for (i = j = 0; i < target_length; i++) {
i = j = 0;
#ifdef XSSE2

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same guard here, so that the loop is not compiled where the helper is not:

Suggested change
#ifdef XSSE2
#if defined(XSSE2) && !defined(WORDS_BIGENDIAN)

}
echo implode(' ', $bytes), "\n";

$hex = str_repeat('0123456789abcdefABCDEF', 10);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This string is 22 characters, so str_repeat(..., 10) gives 220 and the 2000 case below tests 220 characters:

Suggested change
$hex = str_repeat('0123456789abcdefABCDEF', 10);
$hex = str_repeat('0123456789abcdefABCDEF', 100);

@adapik

adapik commented Oct 14, 2026

Copy link
Copy Markdown
Contributor Author

@ArtUkrainskiy thanks for the review, I found your comments reasonable and implemented the changes. Nice catch regarding endianness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants