str_starts_with slower than userland

Description

Firstly, apologies if this is the wrong place.

I was reviewing some code going into a project, and it used str_starts_with. Being new to using it myself wondered about its performance compared to other options and suspected that it should outperform any userland functions. Knowing that if it performed better, it would make a use case for a future micro-optimization PR in the project.

I shamelessly borrowed some Stack Overflow code and added str_starts_with into the benchmarking script and ran it
https://gist.github.com/jdarwood007/1a949424f8cf85ca1b5f66ce38527eb6

PHP 8.4 results

generating tests........................................done!
strncmp_startswith2: 49.7 ms
strncmp_startswith: 88.3 ms
substr_compare_startswith: 89.7 ms
str_starts_with: 133.4 ms
substr_startswith: 156.3 ms
strpos_startswith: 201.2 ms
preg_match_startswith: 15,547.4 ms

PHP 8.3 had similar results.
8.2 had improved results across the board on all tests except preg_match, but I only have a limited set of systems to test with. Results were in the same order.

Is this something that could be expected to be looked into, and see if performance improvements can be made? Seeing userland functions outperform the native functions seems to indicate to me that there could be some improvements

PHP Version

PHP 8.4.5 (cli) (built: Mar 13 2025 15:36:20) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.4.5, Copyright (c) Zend Technologies
with Zend OPcache v8.4.5, Copyright (c), by Zend Technologies
with Xdebug v3.4.2, Copyright (c) 2002-2025, by Derick Rethans

Operating System

Ubuntu 24.04, MacOS 12.7

Possibly related to #18204

Interesting that on PHP 8.2 the results for str_starts_with aren't that bad, it's on the 2nd position (loosing not much). But on PHP 8.4 (while all cases got much faster) str_starts_with fell to 4th position (loosing more). strncmp_startswith2 is the winner for both.

Fedora 42 x86_64 (64-bit).

Alright, let's have a look...

Possibly related to #18204

I don't think so, the implementation only relies on engine API and doesn't call into userland, it should be easy to beat a PHP implementation normally.

First of all, I can't fully reproduce the results you get.
I get the following:

strncmp_startswith2: 15.2 ms
str_starts_with: 17.5 ms
substr_compare_startswith: 20.1 ms
strncmp_startswith: 20.7 ms
substr_startswith: 28.6 ms
strpos_startswith: 97.2 ms
preg_match_startswith: 12,074.9 ms

If I then remove the pcre one, I get this

str_starts_with: 18.5 ms
strncmp_startswith2: 18.6 ms
strncmp_startswith: 19.5 ms
substr_compare_startswith: 21.4 ms
substr_startswith: 29.2 ms
strpos_startswith: 98.8 ms

Possibly caching/throttling related.
Anyway, the difference is marginal.

Second, the benchmarking code is likely flawed.
If you take a look at a profiler, you'll see that zflf_strpos_2 takes 5.36% of runtime, and the next one is zflf_substr_3 with 0.73%, ... etc. So you're barely measuring the execution time of these functions. What you're mostly measuring is the overhead of doing dynamic calls to various functions and also a bit constructs like argument unpacking.
When you write benchmarking code, make sure you only actually do the calls to these functions, no fancy constructs. And also that you do enough work that the function call overhead is less than the function execution.

Rewriting the code like this: https://gist.github.com/nielsdos/c5525659ecad22074b41b1acd2bccac1
Yields:

str_starts_with: 15.8 ms
strncmp_startswith: 17.6 ms
strncmp_startswith2: 18.0 ms
substr_compare_startswith: 18.3 ms
substr_startswith: 25.9 ms
strpos_startswith: 93.7 ms

Suggesting that the overhead of the call is still drowning the execution time of the funtion.
That said, by looking at the profile, str_starts_with takes the least amount of time and that's confirmed by the results here.

The original report appears to have xdebug enabled, I get similar results to the original report with this enabled using the updated bench version on MacOS.

When testing with xdebug not loaded I do see something interesting with timing.

PHP 8.4.6 (cli)
Zend Engine v4.4.6
    with Zend OPcache v8.4.6,

On Linux (compiled with gcc) I don't get much of a different between the calls, the same as the prev comment.

strncmp_startswith: 24.3 ms
strncmp_startswith2: 24.6 ms
str_starts_with: 25.0 ms
substr_compare_startswith: 29.4 ms
substr_startswith: 33.9 ms
strpos_startswith: 136.2 ms

On MacOS (compiled with clang) str_starts_with drops down the ladder for execution time and it always slower.

strncmp_startswith2: 21.8 ms
strncmp_startswith: 24.0 ms
substr_compare_startswith: 29.4 ms
substr_startswith: 32.7 ms
str_starts_with: 43.1 ms
strpos_startswith: 111.9 ms

I did notice that strncmp is defined with fastcall vs str_starts_with with zend_inline_always, I don't know if clang has different results than GCC for those two. Of course this is only a single call of each function, however the timing spread was the same on multiple calls of the bench script.

str_starts_with internally relies on the memcmp standard C library call, whereas our strncmp code relies on zend_memnstr. The implementation is different, but e.g. glibc I'd expect memcmp to outperform zend_memnstr.

Oh I misread the code then. I thought strncmp was defined ZEND_FUNCTION(strncmp) which calls zend_binary_strncmp which relies on memcmp too. I only noticed that this function is defined ZEND_FASTCALL zend_binary_strncmp vs zend_always_inline bool zend_string_starts_with_cstr

No actually I got it mixed up ;) Which makes the difference even weirder

If I expand the number of test cases to from 100,000 to 1,000,000 the differences almost disappear, changing the size of the test cases also sees a lot of the differences disappear. strncmp_startswith2 is consistently faster with it's first byte fast fail however.

strncmp_startswith2: 232.9 ms
str_starts_with: 245.2 ms
substr_startswith: 249.0 ms
substr_compare_startswith: 266.1 ms
strncmp_startswith: 280.2 ms
strpos_startswith: 698.5 ms

If I expand the number of test cases to from 100,000 to 1,000,000 the differences almost disappear

Probably makes sense as you're going to measure the overhead of the engine less and you're starting to more measure the actual comparison.

strncmp_startswith2 is consistently faster with it's first byte fast fail however.

Sure, but that check is only worth it if you know likely that the first byte is different (which is true for this benchmark, which is a flaw of the data used), whereas str_starts_with should cater to the general use case :)

Anyway, what these experiments seem to point out is mostly that the benchmark is flawed.

@nielsdos
I do have xdebug on the system I posted the results from, but I also tested with my MacBook, which doesn't, and those results were similar. I removed Xdebug extension call, confirmed with php -i/php -m and ran the updated benchmark. I am using packages and not compiling.

Ubuntu 24.04 (packages via //ppa.launchpadcontent.net/ondrej/php/ubuntu/) on a "AMD EPYC 7642 48-Core Processor"

generating tests........................................done!
strncmp_startswith2: 34.8 ms
strncmp_startswith: 36.2 ms
substr_compare_startswith: 51.5 ms
substr_startswith: 52.1 ms
str_starts_with: 65.1 ms
strpos_startswith: 140.8 ms

MacOS 12.7 (package via homebrew) on an Intel mac.

generating tests........................................done!
strncmp_startswith2: 38.2 ms
strncmp_startswith: 38.8 ms
substr_startswith: 49.7 ms
substr_compare_startswith: 54.8 ms
str_starts_with: 96.7 ms
strpos_startswith: 132.1 ms

@iluuu1994

@iluuu1994 I tried the one niels mentioned and still see the slower results.
I am using a package maintainer version and am not compiling it, which could result in some loss.
Could this be another library related issue that is outdated? I don't know what the underlying C needs if anything that could be showing this.

I am using a package maintainer version and am not compiling it, which could result in some loss.

I tried both with the packaged 8.4.6 version (from nix) and compiling master with similar results.

As mentioned, both strncmp and str_starts_with are implemented with memcmp. Performance can be weird and unpredictable, and change drastically between environments and data sets. I really don't know how we can help here, especially when our results are so drastically different.

Also: try without xdebug 🙂 (i.e. extension unloaded)

I think they did:

I removed Xdebug extension call, confirmed with php -i/php -m and ran the updated benchmark.

If others are not reproducing this, I will say it's my environment. Odd as it sounds, both the laptop and commercial VPS show similar results. We can close up if there isn't anything to be found with others testing.

@jdarwood007

@jdarwood007 can you run this https://gist.github.com/lucasnetau/8f134b59e1dc25f52744cb9fb6799685 benchmark, it just isolates strncmp vs str_starts_with comparing different haystack lengths and then having the needle matching, best case not matching at start and worst case not matching at end.

I get a very stable result locally. On https://3v4l.org timing results are stable except for whichever test is run first.

str_starts_with_short_match: 16 ns per call
str_starts_with_medium_match: 17 ns per call
str_starts_with_long_match: 17 ns per call
str_starts_with_xlong_match: 17 ns per call
str_starts_with_xlong_nomatch_tail: 17 ns per call
str_starts_with_xlong_nomatch_head: 18 ns per call
str_starts_with_short_nomatch_head: 18 ns per call
str_starts_with_medium_nomatch_tail: 18 ns per call
str_starts_with_long_nomatch_head: 18 ns per call
str_starts_with_long_nomatch_tail: 18 ns per call
str_starts_with_short_nomatch_tail: 19 ns per call
str_starts_with_medium_nomatch_head: 20 ns per call
strncmp_short_match: 24 ns per call
strncmp_long_match: 24 ns per call
strncmp_xlong_match: 24 ns per call
strncmp_medium_match: 25 ns per call
strncmp_short_nomatch_head: 25 ns per call
strncmp_xlong_nomatch_tail: 25 ns per call
strncmp_long_nomatch_tail: 25 ns per call
strncmp_xlong_nomatch_head: 25 ns per call
strncmp_long_nomatch_head: 26 ns per call
strncmp_medium_nomatch_head: 26 ns per call
strncmp_short_nomatch_tail: 26 ns per call
strncmp_medium_nomatch_tail: 28 ns per call

Mac os 12.7

str_starts_with_xlong_match: 44 ns per call
str_starts_with_xlong_nomatch_tail: 44 ns per call
str_starts_with_xlong_nomatch_head: 46 ns per call
str_starts_with_short_match: 46 ns per call
str_starts_with_long_match: 46 ns per call
str_starts_with_medium_match: 47 ns per call
str_starts_with_medium_nomatch_tail: 47 ns per call
str_starts_with_long_nomatch_tail: 48 ns per call
str_starts_with_short_nomatch_head: 48 ns per call
str_starts_with_long_nomatch_head: 48 ns per call
str_starts_with_short_nomatch_tail: 48 ns per call
str_starts_with_medium_nomatch_head: 49 ns per call
strncmp_xlong_match: 55 ns per call
strncmp_long_match: 56 ns per call
strncmp_xlong_nomatch_head: 56 ns per call
strncmp_xlong_nomatch_tail: 56 ns per call
strncmp_medium_match: 57 ns per call
strncmp_long_nomatch_tail: 58 ns per call
strncmp_short_match: 58 ns per call
strncmp_medium_nomatch_tail: 60 ns per call
strncmp_medium_nomatch_head: 61 ns per call
strncmp_short_nomatch_head: 62 ns per call
strncmp_short_nomatch_tail: 62 ns per call
strncmp_long_nomatch_head: 63 ns per call

Ubuntu 24.04

str_starts_with_short_match: 35 ns per call
str_starts_with_long_match: 35 ns per call
str_starts_with_xlong_match: 35 ns per call
str_starts_with_short_nomatch_head: 36 ns per call
str_starts_with_xlong_nomatch_tail: 36 ns per call
str_starts_with_medium_nomatch_tail: 36 ns per call
str_starts_with_medium_match: 36 ns per call
str_starts_with_xlong_nomatch_head: 36 ns per call
str_starts_with_long_nomatch_tail: 37 ns per call
str_starts_with_short_nomatch_tail: 38 ns per call
str_starts_with_medium_nomatch_head: 38 ns per call
str_starts_with_long_nomatch_head: 39 ns per call
strncmp_long_match: 42 ns per call
strncmp_xlong_nomatch_tail: 42 ns per call
strncmp_short_match: 42 ns per call
strncmp_xlong_match: 42 ns per call
strncmp_xlong_nomatch_head: 42 ns per call
strncmp_short_nomatch_head: 43 ns per call
strncmp_medium_nomatch_tail: 43 ns per call
strncmp_long_nomatch_tail: 43 ns per call
strncmp_long_nomatch_head: 43 ns per call
strncmp_medium_match: 44 ns per call
strncmp_short_nomatch_tail: 44 ns per call
strncmp_medium_nomatch_head: 45 ns per call

Thanks for the feedback. This seems to confirm that str_starts_with is not inherently slower. I'm closing this issue then, but feel free to let us know if you see any other peculiar performance behavior.

Navigation Menu

Repository navigation

str_starts_with slower than userland #18474

Description

PHP Version

Operating System

6 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

str_starts_with slower than userland #18474

Description

Description

PHP Version

Operating System

Activity

Girgias commented on May 1, 2025

alecpl commented on May 1, 2025

ndossche commented on May 1, 2025

ndossche commented on May 1, 2025

lucasnetau commented on May 1, 2025

ndossche commented on May 1, 2025

lucasnetau commented on May 1, 2025

ndossche commented on May 1, 2025

lucasnetau commented on May 1, 2025

ndossche commented on May 1, 2025

jdarwood007 commented on May 1, 2025

6 remaining items

jdarwood007 commented on May 1, 2025

iluuu1994 commented on May 2, 2025

ndossche commented on May 2, 2025

iluuu1994 commented on May 2, 2025

jdarwood007 commented on May 3, 2025

lucasnetau commented on May 3, 2025

jdarwood007 commented on May 3, 2025

iluuu1994 commented on May 3, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions