Skip to content

Performance of preg_match varies depending on how subject string is initialized #24380

Closed as not planned
Closed as not planned
Performance of preg_match varies depending on how subject string is initialized#24380
Labels
BugStatus: Invalid

Description

@rr-it
rr-it
opened on Oct 16, 2026

Description

The following code:

<?php

/*
 * Run pinned to specific CPU core:
 * taskset -c 3 php perf-regex-strings.php
 */

$pattern = '/[&]/u'; // This will not produce any match in test subject.

$textA = 'weengahhoghecho1eeSheeG5el9cie5b';
$textB = sprintf('%s', 'weengahhoghecho1eeSheeG5el9cie5b');
$textC = 'weengahhoghecho1eeSheeG5el9cie5' . 'b';

$textD_1 = 'weengahhoghecho1eeSheeG5el9cie5';
$textD = $textD_1 . 'b';

$tests = ['A', 'A', 'B', 'C', 'D', 'A', 'B', 'C', 'D'];

foreach ($tests as $test) {
    $subject = '';
    switch ($test) {
        case 'A':
            $subject = $textA;
            break;
        case 'B':
            $subject = $textB;
            break;
        case 'C':
            $subject = $textC;
            break;
        case 'D':
            $subject = $textD;
            break;
    }

    $start = microtime(true);

    for ($i = 0; $i < 1000000; $i++) {
        preg_match($pattern, $subject);
    }

    $diff = sprintf('%.6f sec.', microtime(true) - $start);
    echo "Execution time $test: $diff\n";

    gc_collect_cycles();
}

See:
https://3v4l.org/uFPrD#v8.5.11

Resulted in this output:

Execution time A: […]
Execution time A: 0.052051 sec.
Execution time B: 0.054563 sec.
Execution time C: 0.052694 sec.
Execution time D: 0.030555 sec.
Execution time A: 0.052350 sec.
Execution time B: 0.052309 sec.
Execution time C: 0.052223 sec.
Execution time D: 0.030175 sec.

But I expected this output instead:

All execution times should nearly be identical.

$textD is somehow fastest for PHP 8.4 and 8.5

$textB and $textD are fastest for PHP 8.2 and PHP 8.3

PHP Version

PHP 8.2, 8.3, 8.4, 8.5 on Debian.

Operating System

No response

Activity

  1. rr-it
    added
    Bug
    Status: Needs Triage
    on Oct 16, 2026

  2. lazerg commented on Oct 17, 2026

    @lazerg
    lazerg
    on Oct 17, 2026
    Contributor

    This is expected. With /u, preg_match() checks that the subject is valid UTF-8, then marks the string so later calls skip the check, but it can't mark interned strings (php_pcre.c). Only D is built at runtime; A and C are compile-time literals, and so is B since 8.4 (#14546).

  3. ndossche commented on Oct 17, 2026

    @ndossche
    ndossche
    on Oct 17, 2026
    Member

    Indeed, depending on how something is compiled or how something is initialized, you can get different performance characteristics.
    Passing fixed compile-time (and thus interned) strings to preg_match to check its performance isn't realistic anyway, microbenchmarks can be deceiving.
    Some common use cases have fast paths, so you can always find cases where functions behave differently because of what's commonly found in real-world code.
    Closing as not a bug.

  4. ndossche
    closed this as not plannedon Oct 17, 2026
  5. ndossche
    added
    Status: Invalid
    and removed
    Status: Needs Triage
    on Oct 17, 2026

  6. rr-it commented on Oct 18, 2026

    @rr-it
    rr-it
    on Oct 18, 2026
    Author

    @lazerg Thank you for pointing out the underlying mechanisms:
    Interned strings do not get a flag IS_STR_VALID_UTF8 after UTF-8 validation - and thereby need constant re-evaluation.

    @ndossche About common use cases:
    Might the general performance get improved by implementing these two measures:

    • String concatenation: On concat of two strings that both are already flagged IS_STR_VALID_UTF8, also flag the resulting string without further check.

      Edit: already implemented.

      • Concatenating two valid UTF-8 strings produces a valid UTF-8 string 64127b6
      • Introduce convenience macros for copying flags that hold when concatenating two strings 99b8614
      • Fix #97836 and #81705: Segfault / type confusion in concat_function 727e26f

    • Add op_cache optimization: check interned strings if they are ASCII-only - and then give them a lifetime flag IS_STR_VALID_UTF8.

  7. rr-it commented on Oct 18, 2026

    @rr-it
    rr-it
    on Oct 18, 2026
    Author

    This might be somehow related to: #20984

  8. ndossche commented on Oct 19, 2026

    @ndossche
    ndossche
    on Oct 19, 2026
    Member

    A general performance improvement idea, wrt the UTF8 flag, would be to find a way to update the flags of zend_strings in shm in a lockless and safe way. This should be doable.

    Add op_cache optimization: check interned strings if they are ASCII-only - and then give them a lifetime flag IS_STR_VALID_UTF8.

    @iluuu1994 prototyped a compile-time UTF-8 check at one point

  9. rr-it commented on Oct 20, 2026

    @rr-it
    rr-it
    on Oct 20, 2026
    Author

    With an implementation that updates the flags of zend_strings in shm, I see string concat as next turning point:

    If two or multiple strings are concatenated with at least one string not having the UTF-8 flag, the resulting string will not have the UTF-8 flag either.
    Thereby the UTF-8 flag is not passed on any further - and this goes on and on with every upcoming string concatination.

    So I think additionally string concat should be changed to implement these features:

    1. Ensure all input strings are UTF-8 checked - and thereby really UTF-8 check and also UTF-8 flag input strings that were not UTF-8 flagged before.
    2. In most cases the output string then also gets the UTF-8 flag without checking the string content itself - as long as all input strings actually are UTF-8 strings.

    This breaks the chain of not handing on the UTF-8 flag.
    And additionally it flags all proper UTF-8 input strings, in case they have not been flagged before.

  10. ndossche commented on Oct 21, 2026

    @ndossche
    ndossche
    on Oct 21, 2026
    Member

    Checking whether many strings are UTF-8 can also be expensive. If you want to do this in a performant way then we need 2 flags: the absence of "this is valid utf-8" right now either means "it was not checked" or "it's not utf-8".

  11. rr-it commented on Oct 30, 2026

    @rr-it
    rr-it
    on Oct 30, 2026
    Author

    My general assumption about strings:
    All regular strings are multibyte character sets which use chunks of multiples of 8bit - and thereby are UTF8 compatible.

    About ISO-8859-1 (Latin-1) and all similar character sets, they all use 8-bit for a single character:
    So I assume ISO-8859-1 is in this context handling-wise a subset of UTF-8. Meaning all ISO-8859-1 strings are already flagged IS_STR_VALID_UTF8 when mb_*-functions are applied - aren't they?

    Are there real use cases where the strings are not UTF-8 (compatible)?
    Like any 7-bit character sets still in use or so?

    As a result: I'm thinking of permitting a small performance penalty (in the form of constant repetitive UTF-8 checks) for unusual non-8-bit character sets.
    And thereby eliminate the need for an extra flag like IS_STR_UTF8_CHECKED.

  12. ndossche commented on Oct 30, 2026

    @ndossche
    ndossche
    on Oct 30, 2026
    Member

    About ISO-8859-1 (Latin-1) and all similar character sets, they all use 8-bit for a single character:
    So I assume ISO-8859-1 is in this context handling-wise a subset of UTF-8.

    No, ISO-8859-1 is not a subset of UTF-8

  13. rr-it commented on Oct 30, 2026

    @rr-it
    rr-it
    on Oct 30, 2026
    Author

    13. No, ISO-8859-1 is not a subset of UTF-8

    I know that.

    But in context of PHP internal IS_STR_VALID_UTF8 checks:
    Does a ISO-8859-1 string actually get the IS_STR_VALID_UTF8 here?

    Edit: The difference of a UTF-8 and an ISO-8859-1 string is defined by context - not "visible" in the byte-code itself.

    E.g. here mbfl_encoding is defined and used as context.

    php-src/ext/mbstring/libmbfl/mbfl/mbfl_encoding.h

    Lines 275 to 282 in 8ab2340

    static inline zend_string* mb_convert_buf_result(mb_convert_buf *buf, const mbfl_encoding *enc)
    {
    zend_string *ret = mb_convert_buf_result_raw(buf);
    if (enc == &mbfl_encoding_utf8 && buf->error_mode != MBFL_OUTPUTFILTER_ILLEGAL_MODE_BADUTF8) {
    GC_ADD_FLAGS(ret, IS_STR_VALID_UTF8);
    }
    return ret;
    }

  14. rr-it commented on Oct 30, 2026

    @rr-it
    rr-it
    on Oct 30, 2026
    Author

    @ndossche I assume while we are in UTF-8 context, all strings most usually are valid UTF-8 and thereby no extra flag like IS_STR_ALREADY_UTF8_CHECKED is needed.

    So the sticking point where I don't get my head around is:
    A simple $a . $b string concatenation without any context given and neither of the strings flagged UTF-8 before.

    Can any context be constructed here?

    How about flagging any string checked for proper UTF-8-possible byte-code as IS_STR_CANDIDATE_UTF8.
    As soon as a string like this is handled in UTF-8 context, it can be handled like any other IS_STR_VALID_UTF8 string without further checks.
    Assumption: Almost any string will get this IS_STR_CANDIDATE_UTF8 flag, as long as there are no super special-bytes and control characters in it like:

    \x{0000}-\x{0008}\x{000E}-\x{001F}\x{007F}
\x{FDD0}-\x{FDEF}
\x{FFFE}\x{FFFF}\x{1FFFE}\x{1FFFF}\x{2FFFE}\x{2FFFF}\x{3FFFE}\x{3FFFF}
\x{4FFFE}\x{4FFFF}\x{5FFFE}\x{5FFFF}\x{6FFFE}\x{6FFFF}\x{7FFFE}\x{7FFFF}
\x{8FFFE}\x{8FFFF}\x{9FFFE}\x{9FFFF}\x{AFFFE}\x{AFFFF}\x{BFFFE}\x{BFFFF}
\x{CFFFE}\x{CFFFF}\x{DFFFE}\x{DFFFF}\x{EFFFE}\x{EFFFF}\x{FFFFE}\x{FFFFF}
\x{10FFFE}\x{10FFFF}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugStatus: Invalid

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions