Skip to content

Apache / PHP segfaults (PHP 8.0 and 8.1, but also 7.4) #8159

Closed
Closed
Apache / PHP segfaults (PHP 8.0 and 8.1, but also 7.4)#8159
Labels
BugExtension: opcacheSAPI: apache2handlerStatus: Needs Triage

Description

@unix-world
unix-world
opened on Mar 1, 2022

Description

PHP 7.4.27 / 8.0.15 / 8.1.3
Apache 2.4.52
OpenBSD 7.0 / amd64

If running any of the above PHP versions under Apache (as module), I noticed that every time when apache rotate it's logs it crashes. To be sure of this issue I set the log rotation at a fixed hour (not by size).
In the last 3 days, each day at the exactly same hour apache crashed and did not crashed in the rest.
Finally I did not loaded the opcache.so and it did not crashed on log rotation.
After further investigation I noticed this:

The problem is somwehere in opcache.so.
All the above versions are affected.
The PHP 7.3.33 works OK, it does not crash.

How to reproduce the issue:

  1. load the opcache.so PHP extensions
  2. apachectl2 graceful OR kill -HUP 12345 (where 12345 is the process ID number)
  3. apache crashes

If I do not load opcache.so it does not crashes.

PHP Version

8.0.15

Operating System

OpenBSD 7.0

Activity

  1. unix-world
    added
    Bug
    Status: Needs Triage
    on Mar 1, 2022
  2. cmb69
    added
    Extension: opcache
    SAPI: apache2handler
    on Mar 1, 2022

  3. devnexen commented on Mar 2, 2022

    @devnexen
    devnexen
    on Mar 2, 2022
    Member

    a backtrace would help, if I recall correctly, openbsd tends to generate a coredump as <process name>.core (so php.core most likely here) where the said process had been called. What might help further in your case eventually is to write to ports@openbsd.org so they might help you generate a version with debug symbols so we can get more useful informations. And maybe opcache settings.

  4. unix-world commented on Mar 4, 2022

    @unix-world
    unix-world
    on Mar 4, 2022
    Author

    There is no php.core, only httpd2.core. I tried looking but there are not enough debug symbols within ... perhaps need a binary package with DEBUG=on ...
    The opcache settings I excluded since I can switch between PHP 7.3.33 / 7.4.27 / 8.0.15 / 8.1.2 and with the same opcache settings the PHP 7.3.33 is not crashing apache.

    This crash is very weird.
    If I do 'rcctl restart apache2' it does not crash ... (this is like in linux: systemctl restart apache2'
    If I do kill -HUP 12345 (where 12345 is the process ID number) it crashes with a core dump, but only with PHP 7.4 / 8.0 / 8.1 and not with 7.3 ! ... (and only if opcache is loaded as a PHP extension, not in the rest).

  5. unix-world commented on Mar 4, 2022

    @unix-world
    unix-world
    on Mar 4, 2022
    Author

    This is the output of GDB

    gdb /usr/local/sbin/httpd2 /httpd2.core

    GNU gdb 6.3
    Copyright 2004 Free Software Foundation, Inc.
    GDB is free software, covered by the GNU General Public License, and you are
    welcome to change it and/or distribute copies of it under certain conditions.
    Type "show copying" to see the conditions.
    There is absolutely no warranty for GDB. Type "show warranty" for details.
    This GDB was configured as "amd64-unknown-openbsd7.0"...(no debugging symbols found)

    Core was generated by `httpd2'.
    Program terminated with signal 11, Segmentation fault.
    Reading symbols from /usr/lib/libpthread.so.26.1...done.
    Loaded symbols for /usr/lib/libpthread.so.26.1
    Loaded symbols for /usr/local/sbin/httpd2
    Reading symbols from /usr/local/lib/libpcre.so.3.0...done.
    Loaded symbols for /usr/local/lib/libpcre.so.3.0
    Reading symbols from /usr/local/lib/libaprutil-1.so.6.1...done.
    Loaded symbols for /usr/local/lib/libaprutil-1.so.6.1
    Symbols already loaded for /usr/lib/libpthread.so.26.1
    Reading symbols from /usr/lib/libexpat.so.14.0...done.
    Loaded symbols for /usr/lib/libexpat.so.14.0
    Reading symbols from /usr/local/lib/libiconv.so.7.0...done.
    Loaded symbols for /usr/local/lib/libiconv.so.7.0
    Reading symbols from /usr/local/lib/libapr-1.so.7.0...done.
    Loaded symbols for /usr/local/lib/libapr-1.so.7.0
    Reading symbols from /usr/local/lib/libdb.so.5.0...done.
    Loaded symbols for /usr/local/lib/libdb.so.5.0
    Reading symbols from /usr/lib/libc.so.96.1...done.
    Loaded symbols for /usr/lib/libc.so.96.1
    Reading symbols from /usr/libexec/ld.so...Error while reading shared library symbols:
    Dwarf Error: wrong version in compilation unit header (is 4, should be 2) [in module /usr/libexec/ld.so]
    Reading symbols from /usr/local/lib/php-7.4/modules/opcache.so...done.
    Loaded symbols for /usr/local/lib/php-7.4/modules/opcache.so
    Reading symbols from /usr/local/lib/apache2/mod_authn_file.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_authn_file.so
    Reading symbols from /usr/local/lib/apache2/mod_authn_core.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_authn_core.so
    Reading symbols from /usr/local/lib/apache2/mod_authz_host.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_authz_host.so
    Reading symbols from /usr/local/lib/apache2/mod_authz_user.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_authz_user.so
    Reading symbols from /usr/local/lib/apache2/mod_authz_core.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_authz_core.so
    Reading symbols from /usr/local/lib/apache2/mod_access_compat.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_access_compat.so
    Reading symbols from /usr/local/lib/apache2/mod_auth_basic.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_auth_basic.so
    Reading symbols from /usr/local/lib/apache2/mod_auth_digest.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_auth_digest.so
    Reading symbols from /usr/local/lib/apache2/mod_allowmethods.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_allowmethods.so
    Reading symbols from /usr/local/lib/apache2/mod_file_cache.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_file_cache.so
    Reading symbols from /usr/local/lib/apache2/mod_cache.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_cache.so
    Reading symbols from /usr/local/lib/apache2/mod_cache_disk.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_cache_disk.so
    Reading symbols from /usr/local/lib/apache2/mod_cache_socache.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_cache_socache.so
    Reading symbols from /usr/local/lib/apache2/mod_socache_shmcb.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_socache_shmcb.so
    Reading symbols from /usr/local/lib/apache2/mod_socache_memcache.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_socache_memcache.so
    Reading symbols from /usr/local/lib/apache2/mod_buffer.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_buffer.so
    Reading symbols from /usr/local/lib/apache2/mod_reqtimeout.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_reqtimeout.so
    Reading symbols from /usr/local/lib/apache2/mod_request.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_request.so
    Reading symbols from /usr/local/lib/apache2/mod_include.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_include.so
    Reading symbols from /usr/local/lib/apache2/mod_filter.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_filter.so
    Reading symbols from /usr/local/lib/apache2/mod_deflate.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_deflate.so
    Reading symbols from /usr/lib/libz.so.6.0...done.
    Loaded symbols for /usr/lib/libz.so.6.0
    Reading symbols from /usr/local/lib/apache2/mod_mime.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_mime.so
    Reading symbols from /usr/local/lib/apache2/mod_log_config.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_log_config.so
    Reading symbols from /usr/local/lib/apache2/mod_env.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_env.so
    Reading symbols from /usr/local/lib/apache2/mod_mime_magic.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_mime_magic.so
    Reading symbols from /usr/local/lib/apache2/mod_expires.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_expires.so
    Reading symbols from /usr/local/lib/apache2/mod_headers.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_headers.so
    Reading symbols from /usr/local/lib/apache2/mod_unique_id.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_unique_id.so
    Reading symbols from /usr/local/lib/apache2/mod_setenvif.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_setenvif.so
    Reading symbols from /usr/local/lib/apache2/mod_version.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_version.so
    Reading symbols from /usr/local/lib/apache2/mod_session.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_session.so
    Reading symbols from /usr/local/lib/apache2/mod_ssl.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_ssl.so
    Reading symbols from /usr/lib/libssl.so.50.0...done.
    Loaded symbols for /usr/lib/libssl.so.50.0
    Reading symbols from /usr/lib/libcrypto.so.47.0...done.
    Loaded symbols for /usr/lib/libcrypto.so.47.0
    Reading symbols from /usr/local/lib/apache2/mod_mpm_prefork.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_mpm_prefork.so
    Reading symbols from /usr/local/lib/apache2/mod_unixd.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_unixd.so
    Reading symbols from /usr/local/lib/apache2/mod_dav.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_dav.so
    Reading symbols from /usr/local/lib/apache2/mod_status.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_status.so
    Reading symbols from /usr/local/lib/apache2/mod_autoindex.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_autoindex.so
    Reading symbols from /usr/local/lib/apache2/mod_dav_fs.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_dav_fs.so
    Reading symbols from /usr/local/lib/apache2/mod_dav_lock.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_dav_lock.so
    Reading symbols from /usr/local/lib/apache2/mod_vhost_alias.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_vhost_alias.so
    Reading symbols from /usr/local/lib/apache2/mod_negotiation.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_negotiation.so
    Reading symbols from /usr/local/lib/apache2/mod_dir.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_dir.so
    Reading symbols from /usr/local/lib/apache2/mod_alias.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_alias.so
    Reading symbols from /usr/local/lib/apache2/mod_rewrite.so...done.
    Loaded symbols for /usr/local/lib/apache2/mod_rewrite.so
    Reading symbols from /usr/local/lib/php-7.4/libphp7.so...done.
    Loaded symbols for /usr/local/lib/php-7.4/libphp7.so
    Reading symbols from /usr/local/lib/libargon2.so.0.0...done.
    Loaded symbols for /usr/local/lib/libargon2.so.0.0
    Reading symbols from /usr/lib/libreadline.so.4.0...done.
    Loaded symbols for /usr/lib/libreadline.so.4.0
    Reading symbols from /usr/lib/libcurses.so.14.0...done.
    Loaded symbols for /usr/lib/libcurses.so.14.0
    Reading symbols from /usr/local/lib/libintl.so.7.0...done.
    Loaded symbols for /usr/local/lib/libintl.so.7.0
    Reading symbols from /usr/lib/libm.so.10.1...done.
    Loaded symbols for /usr/lib/libm.so.10.1
    Reading symbols from /usr/local/lib/libonig.so.7.2...done.
    Loaded symbols for /usr/local/lib/libonig.so.7.2
    Reading symbols from /usr/local/lib/libsqlite3.so.37.15...done.
    Loaded symbols for /usr/local/lib/libsqlite3.so.37.15
    Reading symbols from /usr/local/lib/libxml2.so.17.0...done.
    Loaded symbols for /usr/local/lib/libxml2.so.17.0
    Reading symbols from /usr/local/lib/liblzma.so.2.1...done.
    Loaded symbols for /usr/local/lib/liblzma.so.2.1
    Reading symbols from /usr/local/lib/php-7.4/modules/bz2.so...done.
    Loaded symbols for /usr/local/lib/php-7.4/modules/bz2.so
    Reading symbols from /usr/local/lib/libbz2.so.10.4...done.
    Loaded symbols for /usr/local/lib/libbz2.so.10.4
    Reading symbols from /usr/local/lib/php-7.4/modules/curl.so...done.
    Loaded symbols for /usr/local/lib/php-7.4/modules/curl.so
    Reading symbols from /usr/local/lib/libcurl.so.26.13...done.
    Loaded symbols for /usr/local/lib/libcurl.so.26.13
    Reading symbols from /usr/local/lib/libnghttp2.so.0.18...done.
    Loaded symbols for /usr/local/lib/libnghttp2.so.0.18
    Reading symbols from /usr/local/lib/php-7.4/modules/dba.so...done.
    Loaded symbols for /usr/local/lib/php-7.4/modules/dba.so
    Reading symbols from /usr/local/lib/libgdbm.so.7.0...done.
    Loaded symbols for /usr/local/lib/libgdbm.so.7.0
    Reading symbols from /usr/local/lib/php-7.4/modules/gd.so...done.
    Loaded symbols for /usr/local/lib/php-7.4/modules/gd.so
    Reading symbols from /usr/local/lib/libpng.so.18.0...done.
    Loaded symbols for /usr/local/lib/libpng.so.18.0
    Reading symbols from /usr/local/lib/libwebp.so.4.3...done.
    Loaded symbols for /usr/local/lib/libwebp.so.4.3
    Reading symbols from /usr/local/lib/libjpeg.so.70.0...done.
    Loaded symbols for /usr/local/lib/libjpeg.so.70.0
    Reading symbols from /usr/X11R6/lib/libfreetype.so.30.0...done.
    Loaded symbols for /usr/X11R6/lib/libfreetype.so.30.0
    Reading symbols from /usr/local/lib/php-7.4/modules/gmp.so...done.
    Loaded symbols for /usr/local/lib/php-7.4/modules/gmp.so
    Reading symbols from /usr/local/lib/libgmp.so.11.0...done.
    Loaded symbols for /usr/local/lib/libgmp.so.11.0
    Reading symbols from /usr/local/lib/php-7.4/modules/intl.so...done.
    Loaded symbols for /usr/local/lib/php-7.4/modules/intl.so
    Reading symbols from /usr/local/lib/libicuio.so.20.0...done.
    Loaded symbols for /usr/local/lib/libicuio.so.20.0
    Reading symbols from /usr/local/lib/libicui18n.so.20.0...done.
    Loaded symbols for /usr/local/lib/libicui18n.so.20.0
    Reading symbols from /usr/local/lib/libicuuc.so.20.0...done.
    Loaded symbols for /usr/local/lib/libicuuc.so.20.0
    Reading symbols from /usr/local/lib/libicudata.so.20.0...done.
    Loaded symbols for /usr/local/lib/libicudata.so.20.0
    Reading symbols from /usr/lib/libc++.so.8.0...done.
    Loaded symbols for /usr/lib/libc++.so.8.0
    Reading symbols from /usr/lib/libc++abi.so.5.0...done.
    Loaded symbols for /usr/lib/libc++abi.so.5.0
    Reading symbols from /usr/local/lib/php-7.4/modules/mongodb.so...done.
    Loaded symbols for /usr/local/lib/php-7.4/modules/mongodb.so
    Reading symbols from /usr/lib/libtls.so.22.0...done.
    Loaded symbols for /usr/lib/libtls.so.22.0
    Reading symbols from /usr/local/lib/php-7.4/modules/mysqli.so...done.
    Loaded symbols for /usr/local/lib/php-7.4/modules/mysqli.so
    Reading symbols from /usr/local/lib/php-7.4/modules/pdo_mysql.so...done.
    Loaded symbols for /usr/local/lib/php-7.4/modules/pdo_mysql.so
    Reading symbols from /usr/local/lib/php-7.4/modules/pdo_pgsql.so...done.
    Loaded symbols for /usr/local/lib/php-7.4/modules/pdo_pgsql.so
    Reading symbols from /usr/local/lib/libpq.so.6.10...done.
    Loaded symbols for /usr/local/lib/libpq.so.6.10
    Reading symbols from /usr/local/lib/php-7.4/modules/pgsql.so...done.
    Loaded symbols for /usr/local/lib/php-7.4/modules/pgsql.so
    Reading symbols from /usr/local/lib/php-7.4/modules/zip.so...done.
    Loaded symbols for /usr/local/lib/php-7.4/modules/zip.so
    Reading symbols from /usr/local/lib/libzip.so.4.1...done.
    Loaded symbols for /usr/local/lib/libzip.so.4.1
    #0 0x00000ce72b51aeb0 in ?? ()

    There is nothing more ... in the GDB output !

  6. unix-world commented on Mar 4, 2022

    @unix-world
    unix-world
    on Mar 4, 2022
    Author

    Also maybe this can help you:

    (gdb) list
    1 /usr/src/lib/libtls/tls.c: No such file or directory.
    in /usr/src/lib/libtls/tls.c
    (gdb) info locals
    No symbol table info available.
    (gdb) bt
    #0 0x000008f23da53eb0 in ?? ()
    #1 0x000008f2ef2029e7 in accel_startup () from /usr/local/lib/php-7.4/modules/opcache.so
    #2 0x000008f2931d9126 in zend_extension_startup () from /usr/local/lib/php-7.4/libphp7.so
    #3 0x000008f2931bbf59 in zend_llist_apply_with_del () from /usr/local/lib/php-7.4/libphp7.so
    #4 0x000008f2931d90e5 in zend_startup_extensions () from /usr/local/lib/php-7.4/libphp7.so
    #5 0x000008f29315039e in php_module_startup () from /usr/local/lib/php-7.4/libphp7.so
    #6 0x000008f293275853 in php_apache2_startup () from /usr/local/lib/php-7.4/libphp7.so
    #7 0x000008f2932750e7 in php_apache_server_startup () from /usr/local/lib/php-7.4/libphp7.so
    #8 0x000008f02f5b4955 in ap_run_post_config () from /usr/local/sbin/httpd2
    #9 0x000008f02f5b36a5 in main () from /usr/local/sbin/httpd2
    (gdb)

  7. unix-world commented on Mar 4, 2022

    @unix-world
    unix-world
    on Mar 4, 2022
    Author

    (gdb) bt full
    #0 0x000008f23da53eb0 in ?? ()
    No symbol table info available.
    #1 0x000008f2ef2029e7 in accel_startup () from /usr/local/lib/php-7.4/modules/opcache.so
    No symbol table info available.
    #2 0x000008f2931d9126 in zend_extension_startup () from /usr/local/lib/php-7.4/libphp7.so
    No symbol table info available.
    #3 0x000008f2931bbf59 in zend_llist_apply_with_del () from /usr/local/lib/php-7.4/libphp7.so
    No symbol table info available.
    #4 0x000008f2931d90e5 in zend_startup_extensions () from /usr/local/lib/php-7.4/libphp7.so
    No symbol table info available.
    #5 0x000008f29315039e in php_module_startup () from /usr/local/lib/php-7.4/libphp7.so
    No symbol table info available.
    #6 0x000008f293275853 in php_apache2_startup () from /usr/local/lib/php-7.4/libphp7.so
    No symbol table info available.
    #7 0x000008f2932750e7 in php_apache_server_startup () from /usr/local/lib/php-7.4/libphp7.so
    No symbol table info available.
    #8 0x000008f02f5b4955 in ap_run_post_config () from /usr/local/sbin/httpd2
    No symbol table info available.
    #9 0x000008f02f5b36a5 in main () from /usr/local/sbin/httpd2
    No symbol table info available.

  8. unix-world commented on Mar 4, 2022

    @unix-world
    unix-world
    on Mar 4, 2022
    Author

    Even if i use just this in opcache.ini
    zend_extension=opcache.so

    or if I use:
    zend_extension=opcache.so
    opcache.enable = 1
    opcache.enable_cli = 0
    opcache.memory_consumption = 512
    opcache.interned_strings_buffer = 16
    opcache.max_accelerated_files = 5000
    opcache.max_wasted_percentage = 5
    opcache.use_cwd = 1
    opcache.validate_timestamps = 1
    opcache.revalidate_freq = 2
    opcache.revalidate_path = 0
    opcache.save_comments = 0
    opcache.load_comments = 0
    opcache.fast_shutdown = 0
    opcache.enable_file_override = 0
    ;opcache.optimization_level = 0x00000000
    opcache.inherited_hack = 1
    opcache.blacklist_filename = ""
    opcache.max_file_size = 0
    opcache.consistency_checks = 0
    opcache.force_restart_timeout = 180
    opcache.error_log = "/var/log/apache2/php-opcache-errors.log"
    opcache.log_verbosity_level = 1
    opcache.preferred_memory_model = ""
    opcache.protect_memory = 0

    It crashes with any of these configs (default or custom) for opcache.ini ...
    This opcache.ini I am using since many years and worked OK.
    I recently turned on PHP 7.4 and I noticed this crash.
    Since I am able to still test this crash also with PHP 7.3 it does not crashes but only crashes with 7.4 / 8.0 / 8.1 if opcache.so is loaded :-(

  9. unix-world commented on Mar 4, 2022

    @unix-world
    unix-world
    on Mar 4, 2022
    Author

    Core

    PHP Version 7.4.27

    Core
    PHP Version 7.4.27
    Directive Local Value Master Value
    allow_url_fopen Off Off
    allow_url_include Off Off
    arg_separator.input & &
    arg_separator.output & &
    auto_append_file no value no value
    auto_globals_jit On On
    auto_prepend_file no value no value
    browscap no value no value
    default_charset UTF-8 UTF-8
    default_mimetype text/html text/html
    disable_classes no value no value
    disable_functions no value no value
    display_errors Off Off
    display_startup_errors Off Off
    doc_root no value no value
    docref_ext no value no value
    docref_root no value no value
    enable_dl Off Off
    enable_post_data_reading On On
    error_append_string no value no value
    error_log no value no value
    error_prepend_string no value no value
    error_reporting 22527 22527
    expose_php On On
    extension_dir /usr/local/lib/php-7.4/modules /usr/local/lib/php-7.4/modules
    file_uploads On On
    hard_timeout 2 2
    highlight.comment #FF8000 #FF8000
    highlight.default #0000BB #0000BB
    highlight.html #000000 #000000
    highlight.keyword #7700 #7700
    highlight.string #DD0000 #DD0000
    html_errors On On
    ignore_repeated_errors Off Off
    ignore_repeated_source Off Off
    ignore_user_abort Off Off
    implicit_flush Off Off
    include_path .:/pear/lib:/var/www/pear/lib .:/pear/lib:/var/www/pear/lib
    input_encoding no value no value
    internal_encoding no value no value
    log_errors On On
    log_errors_max_len 1024 1024
    mail.add_x_header Off Off
    mail.force_extra_parameters no value no value
    mail.log no value no value
    max_execution_time 30 30
    max_file_uploads 20 20
    max_input_nesting_level 64 64
    max_input_time 60 60
    max_input_vars 1000 1000
    memory_limit 256M 256M
    open_basedir no value no value
    output_buffering 4096 4096
    output_encoding no value no value
    output_handler no value no value
    post_max_size 178M 178M
    precision 14 14
    realpath_cache_size 4096K 4096K
    realpath_cache_ttl 120 120
    register_argc_argv Off Off
    report_memleaks On On
    report_zend_debug On On
    request_order GP GP
    sendmail_from no value no value
    sendmail_path /usr/sbin/sendmail -t -i /usr/sbin/sendmail -t -i
    serialize_precision -1 -1
    short_open_tag Off Off
    SMTP localhost localhost
    smtp_port 25 25
    sys_temp_dir no value no value
    syslog.facility LOG_USER LOG_USER
    syslog.filter no-ctrl no-ctrl
    syslog.ident php php
    track_errors Off Off
    unserialize_callback_func no value no value
    upload_max_filesize 172M 172M
    upload_tmp_dir no value no value
    user_dir no value no value
    user_ini.cache_ttl 300 300
    user_ini.filename .user.ini .user.ini
    variables_order GPCS GPCS
    xmlrpc_error_number 0 0
    xmlrpc_errors Off Off
    zend.assertions -1 -1
    zend.detect_unicode On On
    zend.enable_gc On On
    zend.exception_ignore_args On On
    zend.multibyte Off Off
    zend.script_encoding no value no value
    zend.signal_check Off Off

  10. unix-world commented on Mar 4, 2022

    @unix-world
    unix-world
    on Mar 4, 2022
    Author

    pcre

    PCRE (Perl Compatible Regular Expressions) Support enabled
    PCRE Library Version 10.35 2020-05-09
    PCRE Unicode Version 13.0.0
    PCRE JIT Support not compiled in

    pcre
    PCRE (Perl Compatible Regular Expressions) Support enabled
    PCRE Library Version 10.35 2020-05-09
    PCRE Unicode Version 13.0.0
    PCRE JIT Support not compiled in

  11. unix-world commented on Mar 4, 2022

    @unix-world
    unix-world
    on Mar 4, 2022
    Author

    Zend OPcache

    Opcode Caching Up and Running
    Optimization Enabled
    SHM Cache Enabled
    File Cache Disabled
    Startup OK
    Shared memory model mmap
    Cache hits 0
    Cache misses 1
    Used memory 17160440
    Free memory 519710472
    Wasted memory 0
    Interned Strings Used memory 387632
    Interned Strings Free memory 12194832
    Cached scripts 1
    Cached keys 2
    Max keys 7963
    OOM restarts 0
    Hash keys restarts 0
    Manual restarts 0

    Zend OPcache
    Opcode Caching Up and Running
    Optimization Enabled
    SHM Cache Enabled
    File Cache Disabled
    Startup OK
    Shared memory model mmap
    Cache hits 0
    Cache misses 1
    Used memory 17160440
    Free memory 519710472
    Wasted memory 0
    Interned Strings Used memory 387632
    Interned Strings Free memory 12194832
    Cached scripts 1
    Cached keys 2
    Max keys 7963
    OOM restarts 0
    Hash keys restarts 0
    Manual restarts 0
    Directive Local Value Master Value
    opcache.blacklist_filename no value no value
    opcache.consistency_checks 0 0
    opcache.dups_fix Off Off
    opcache.enable On On
    opcache.enable_cli Off Off
    opcache.enable_file_override Off Off
    opcache.error_log /var/log/apache2/php-opcache-errors.log /var/log/apache2/php-opcache-errors.log
    opcache.file_cache no value no value
    opcache.file_cache_consistency_checks On On
    opcache.file_cache_only Off Off
    opcache.file_update_protection 2 2
    opcache.force_restart_timeout 180 180
    opcache.huge_code_pages Off Off
    opcache.interned_strings_buffer 16 16
    opcache.lockfile_path /tmp /tmp
    opcache.log_verbosity_level 1 1
    opcache.max_accelerated_files 5000 5000
    opcache.max_file_size 0 0
    opcache.max_wasted_percentage 5 5
    opcache.memory_consumption 512 512
    opcache.opt_debug_level 0 0
    opcache.optimization_level 0x7FFEBFFF 0x7FFEBFFF
    opcache.preferred_memory_model no value no value
    opcache.preload no value no value
    opcache.preload_user no value no value
    opcache.protect_memory Off Off
    opcache.restrict_api no value no value
    opcache.revalidate_freq 2 2
    opcache.revalidate_path Off Off
    opcache.save_comments Off Off
    opcache.use_cwd On On
    opcache.validate_permission Off Off
    opcache.validate_root Off Off
    opcache.validate_timestamps On On

  12. unix-world commented on Mar 4, 2022

    @unix-world
    unix-world
    on Mar 4, 2022
    Author

    The only difference I see between
    rcctl restart apache2 (not crashing)
    and
    kill -HUP 12345 (crashing)
    is in errors.log as:

    restarting via rcctl says in log: caught SIGWINCH, shutting down gracefully
    (ok, not crashing)

    but with kill -HUP says in log:
    SIGHUP received. Attempting to restart
    (in this situation is crashing all the time ...)

  13. devnexen commented on Mar 4, 2022

    @devnexen
    devnexen
    on Mar 4, 2022
    Member

    (gdb) bt full #0 0x000008f23da53eb0 in ?? () No symbol table info available. #1 0x000008f2ef2029e7 in accel_startup () from /usr/local/lib/php-7.4/modules/opcache.so No symbol table info available. #2 0x000008f2931d9126 in zend_extension_startup () from /usr/local/lib/php-7.4/libphp7.so No symbol table info available. #3 0x000008f2931bbf59 in zend_llist_apply_with_del () from /usr/local/lib/php-7.4/libphp7.so No symbol table info available. #4 0x000008f2931d90e5 in zend_startup_extensions () from /usr/local/lib/php-7.4/libphp7.so No symbol table info available. #5 0x000008f29315039e in php_module_startup () from /usr/local/lib/php-7.4/libphp7.so No symbol table info available. #6 0x000008f293275853 in php_apache2_startup () from /usr/local/lib/php-7.4/libphp7.so No symbol table info available. #7 0x000008f2932750e7 in php_apache_server_startup () from /usr/local/lib/php-7.4/libphp7.so No symbol table info available. #8 0x000008f02f5b4955 in ap_run_post_config () from /usr/local/sbin/httpd2 No symbol table info available. #9 0x000008f02f5b36a5 in main () from /usr/local/sbin/httpd2 No symbol table info available.

    thanks however would be best to focus on PHP 8.x, as said in another ticket PHP 7.4 is no longer supported.

  14. 36 remaining items

  15. unix-world commented on Mar 9, 2022

    @unix-world
    unix-world
    on Mar 9, 2022
    Author

    @devnexen

    I was trying any of the above scenarios.
    int signal = SIGTERM; -> int signal = SIGKILL;
    Then commenting out the } else if (errno != 0) { zend_accel_error(...) lines.
    Still the same result, crashing on SIGHUP or SIGUSR1, both scenarios.

    I was talking with an internal guy here that have some limited C skills and he told me that from what he can see in the debug log the crashing issue comes from accel_startup or start_accel_module methods, thus he presumes everything is fine when doing SIGTERM or SIGHUP, but on SIGHUP actually apache is shuting down then restart all childs, thus he thinks is an issue with restart after SIGHUP not with the closing process on SIGHUP.

    #1  0x00000cec80fed18e in start_accel_module () at ext/opcache/zend_accelerator_module.c:518
No locals.
#2  0x00000cec80fdf63f in accel_startup (extension=0xcec16ffe210) at ext/opcache/ZendAccelerator.c:3114
No locals.

    Thus I have noticed only one major difference in the logic between PHP 7.3.33 and 8.0.16:
    In 7.3.33 the accel_activate method is called in ext/opcache/ZendAccelerator.c, line # 3156
    https://github.com/php/php-src/blob/php-7.3.33/ext/opcache/ZendAccelerator.c#L3156

    In 8.0.16 the accel_activate method is called in a completely different place:
    ext/opcache/zend_accelerator_module.c, line # 507
    https://github.com/php/php-src/blob/php-8.0.16/ext/opcache/zend_accelerator_module.c#L507

    What is your opinion on this ? I am not aware too much of Zend internal code ...
    Should I try to create a patch to call accel_activate in the same place as it was in PHP 7.3.33 ? Does it make sense ?

  16. devnexen commented on Mar 10, 2022

    @devnexen
    devnexen
    on Mar 10, 2022
    Member

    I do not have opinion yet, I ll try to set up this weekend hopefully a separate openbsd VM to be able to "mess up" with packages and debugging unless someone find some clue in the meantime.

  17. unix-world commented on Mar 10, 2022

    @unix-world
    unix-world
    on Mar 10, 2022
    Author

    @devnexen

    OK.
    If you cannot reproduce the bug just let me know and I will send you the exact configs I use.
    I use OpenBSD since more than 10 years or so, if you need any support for installing, config or others just let me know too. I am very focused to see this bug solved ... :-)

    By the way, you can install the binary ports, but in order to compile it from source there are some things you have to know if you don't already know:

    1. the original src ports of OpenBSD 7.0 are too old and have not the latest updates to build from source PHP 8.0.16
    2. the current ports are already targeting OpenBSD 7.1 beta, thus perhaps will fail to build at some packages
    3. the original src ports of OpenBSD 7.0, with updates packages just targeting OpenBSD 7.0 you can find here
      https://codeberg.org/unixman/ports-openbsd-uxm
      (this repo is containing only the changes marked as stable from the ongoing ports and also extra several ports that are not found in the original openbsd ports: ex: unixman/php-mongodb or unixman/php-xdebug)
    4. or you can download and use original OpenBSD 7.0 src ports but you have to update from the current src ports some packages like: zstd, apr, apr-utils, apache-httpd, php and some others they depend on
    5. an alternative also would be you install all binary ports except PHP and use and updated src ports for PHP to start build 8.0.16 (but not sure exactly if all dependencies are met).

    I can confirm that also the original ports built by the package maintainers have this bug. I get the email messages with one of the maintainers. He told me that this is for sure something inside PHP opcache that changed.

    PS: also be sure to use the llvm 11, which is the default compiler (not gcc)

    R.

  18. dstogov commented on Mar 11, 2022

    @dstogov
    dstogov
    on Mar 11, 2022
    Member

    @unix-world I suspect, the crash is related to Apache graceful restart and dynamic loading of php and opcache. According to backtrace opcache can't call back to PHP from accel_startup(). I don't have OpenBSD installation and can't reproduce the problem,

  19. dstogov
    closed this as completedon Mar 11, 2022

  20. dstogov commented on Mar 11, 2022

    @dstogov
    dstogov
    on Mar 11, 2022
    Member

    @unix-world

    The best backtrace shows crash under start_accel_module(), but this function just forwards to zend_startup_module(), that is not in backtrace.

    Try to attach GDB/LLDB to the main Apache process, set breakpoint on start_accel_module() and then try to understand what is going wrong after restart. It's also possible to add some instrumentation code into start_accel_module() to write address of zend_startup_module and accel_module_entry into Apache error log.

  21. unix-world commented on Mar 11, 2022

    @unix-world
    unix-world
    on Mar 11, 2022
    Author

    @dstogov
    Based on your explanation, I am not sure this is all ...
    Why the PHP 7.3.33 does not crash with the exactly same apache build and exactly same configuration ?
    I currently have 4 versions of PHP all installed:
    7.3.33
    7.4.28
    8.0.16
    8.1.3
    Except PHP 7.3.33 all the rest are crashing (7.4.28 / 8.0.16 / 8.1.3).
    Apache is the same, 2.4.52. When switching from a PHP version to another I just change one thing - the PHP configuration for apache, in the apache configuration is 100% identical. More, to test I loaded just opcache.so extension for any of these versions.
    See the only configs I change when switching between the PHP versions:

    LoadModule php7_module /usr/local/lib/php-7.3/libphp7.so
<IfModule mod_php7.c>
	AddType application/x-httpd-php .php .phtml .php3
	AddType application/x-httpd-php-source .phps
# Most php configs require this
	DirectoryIndex index.php
</IfModule>


    LoadModule php7_module /usr/local/lib/php-7.4/libphp7.so
<IfModule mod_php7.c>
	AddType application/x-httpd-php .php .phtml .php3
	AddType application/x-httpd-php-source .phps
# Most php configs require this
	DirectoryIndex index.php
</IfModule>


    LoadModule php_module /usr/local/lib/php-8.0/libphp.so
<IfModule mod_php.c>
	AddType application/x-httpd-php .php .phtml .php3
	AddType application/x-httpd-php-source .phps
# Most php configs require this
	DirectoryIndex index.php
</IfModule>


    LoadModule php_module /usr/local/lib/php-8.1/libphp.so
<IfModule mod_php.c>
	AddType application/x-httpd-php .php .phtml .php3
	AddType application/x-httpd-php-source .phps
# Most php configs require this
	DirectoryIndex index.php
</IfModule>

    All 4 versions are built with exactly the same build options and same platform patches (of course adapted by version).
    How can you explain this ?
    I noticed all the crashing versions (7.4.x / 8.0.x / 8.1.x) have inside opcache code a major change:
    the accel_activate method is called in a completely different place: ext/opcache/zend_accelerator_module.c
    but in 7.3.33 which is not crashing
    the accel_activate method is called in ext/opcache/ZendAccelerator.c

    I suspect is something about this change ... because the opcache JIT is just in 8.0.x and 8.1.x but 7.4.x still crashes and does not have yet the opcache JIT...

  22. unix-world commented on Mar 11, 2022

    @unix-world
    unix-world
    on Mar 11, 2022
    Author

    PS:
    I will try to fill up with logging before each execution line and see where it stops ...
    Running apache prefork with GDB I am not sure if possible, there are many childs not only the main process running at the same time concurrently and the crashing is when doing SIGHUP which suppose will stop all childs except the main process.
    Do you have any idea how to run this under gdb ?

  23. unix-world commented on Mar 11, 2022

    @unix-world
    unix-world
    on Mar 11, 2022
    Author

    I am really confused. You closed this bug without any explanation.

    Look at the below bug, it was an older bug reporting approximately the same issue, perhaps still not solved:
    https://bugs.php.net/bug.php?id=73901

  24. unix-world commented on Mar 11, 2022

    @unix-world
    unix-world
    on Mar 11, 2022
    Author

    ** And I think the following FreeBSD bug is exactly the same after they switch from PHP 7.3 to 7.4 ... and still unsolved / unfixed in PHP 8.0 / 8.1 !: **
    https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247487

  25. unix-world commented on Mar 12, 2022

    @unix-world
    unix-world
    on Mar 12, 2022
    Author

    @devnexen I don't know why this ticket was closed without an answer.
    More, this looks like a serious security issue to me.

    In my opinion the bug I reported here is because of the new changes in the opcache, in PHP 7.4 / 8.0 and 8.1.
    PHP 7.3 is working correctly, without this kind bug.
    Somewhere in time it this kind of opcache bugs were fixed, but now appears (in another form) that have come back. Looks to me very similar with this one:
    https://bugzilla.redhat.com/show_bug.cgi?id=1314888

    I am not a C expert, but this clear looks to me like a "use-after-free error" ...
    And probabily is comming from this changed area:

    In 7.3.33 the accel_activate method is called in ext/opcache/ZendAccelerator.c, line # 3156
    https://github.com/php/php-src/blob/php-7.3.33/ext/opcache/ZendAccelerator.c#L3156

    In 8.0.16 the accel_activate method is called in a completely different place:
    ext/opcache/zend_accelerator_module.c, line # 507
    https://github.com/php/php-src/blob/php-8.0.16/ext/opcache/zend_accelerator_module.c#L507

    Please not that I included the PHP 7.4 in this discussion just to exclude the possibility this bug have nothing to do with the opcache JIT (introduced only since 8.0) !
    Thus, PHP 7.4 have no opcache JIT and still crashing because it uses a similar way of calling accel_activate in ext/opcache/zend_accelerator_module.c just like PHP 8.0 and 8.1.

  26. unix-world commented on Mar 12, 2022

    @unix-world
    unix-world
    on Mar 12, 2022
    Author

    @dstogov , as you said:

    The best backtrace shows crash under start_accel_module(), but this function just forwards to zend_startup_module(), that is not in backtrace.

    But please look at this part of the code and you see that accel_activate is called here !!:

    PHP 8.0 / 8.1 / 7.4

    static zend_module_entry accel_module_entry = {
	STANDARD_MODULE_HEADER,
	ACCELERATOR_PRODUCT_NAME,
	ext_functions,
	ZEND_MINIT(zend_accelerator),
	ZEND_MSHUTDOWN(zend_accelerator),
	accel_activate,  // !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! THIS I AM TALKING ABOUT
	NULL,
	zend_accel_info,
	PHP_VERSION,
	NO_MODULE_GLOBALS,
	accel_post_deactivate,
	STANDARD_MODULE_PROPERTIES_EX
};

int start_accel_module(void)
{
	return zend_startup_module(&accel_module_entry);
}

    But in PHP 7.3 (not crashing) is called in completely different place

    static zend_module_entry accel_module_entry = {
	STANDARD_MODULE_HEADER,
	ACCELERATOR_PRODUCT_NAME,
	accel_functions,
	ZEND_MINIT(zend_accelerator),
	ZEND_MSHUTDOWN(zend_accelerator),
	NULL, // !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! THIS I AM TALKING ABOUT
	NULL,
	zend_accel_info,
	PHP_VERSION,
	NO_MODULE_GLOBALS,
	accel_post_deactivate,
	STANDARD_MODULE_PROPERTIES_EX
};

int start_accel_module(void)
{
	return zend_startup_module(&accel_module_entry);
}

    In PHP 7.3 the accel_activate is called here

    ZEND_EXT_API zend_extension zend_extension_entry = {
	ACCELERATOR_PRODUCT_NAME,               /* name */
	PHP_VERSION,							/* version */
	"Zend Technologies",					/* author */
	"http://www.zend.com/",					/* URL */
	"Copyright (c) 1999-2018",				/* copyright */
	accel_startup,					   		/* startup */
	NULL,									/* shutdown */
	accel_activate,							/* per-script activation */
	accel_deactivate,						/* per-script deactivation */
	NULL,									/* message handler */
	NULL,									/* op_array handler */
	NULL,									/* extended statement handler */
	NULL,									/* extended fcall begin handler */
	NULL,									/* extended fcall end handler */
	NULL,									/* op_array ctor */
	NULL,									/* op_array dtor */
	STANDARD_ZEND_EXTENSION_PROPERTIES
};

  27. dstogov commented on Mar 14, 2022

    @dstogov
    dstogov
    on Mar 14, 2022
    Member

    I see accel_activate() is going to be called from different places, but I don't see it in the backtrace. So this is not a problem. The crash occurs in start_accel_module(), that is the same for all PHP versions. I can't guess what is going wrong. I suspect issues with dynamic loader.

  28. unix-world commented on Mar 14, 2022

    @unix-world
    unix-world
    on Mar 14, 2022
    Author

    I suspect calling the accel_activate() from the new place in PHP 7.4 / 8.0 / 8.1 which also occurs when calling start_accel_module() could be a cause ... not sure.

  29. dstogov commented on Mar 14, 2022

    @dstogov
    dstogov
    on Mar 14, 2022
    Member

    You may revert the the commit 34a491d1ed5 to check this.
    Anyway, there is nothing wrong with this commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugExtension: opcacheSAPI: apache2handlerStatus: Needs Triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions