Skip to content

Segmentation fault on exit when using ldap_bind() #8620

Open

Description

@uberbrady

Description

(I'd like to note that while I'm reporting this against PHP 7.4.28, I also saw the problem in PHP 8.0.16)

The following code:

<?php

$connection = ldap_connect("ldaps://ldap.google.com");

$bind_results = ldap_bind($connection, 'AnyUsername','AnyPassword');

Resulted in this output:

PHP Warning:  ldap_bind(): Unable to bind to server: Protocol error in /home/ec2-user/testcase.php on line 5
Segmentation fault

But I expected this output instead:

PHP Warning:  ldap_bind(): Unable to bind to server: Protocol error in /home/ec2-user/testcase.php on line 5

Removing the ldap_bind() statement allows the script to run without segfaulting.

When run under GDB, the following backtrace occurs (PHP 7.4):

#0  0x00007fffe8d1b2b0 in ?? ()
#1  0x00007ffff57fb358 in ENGINE_remove (e=0x555555ccb7b0) at eng_list.c:284
#2  0x00007ffff57fb4a5 in engine_list_cleanup () at eng_list.c:94
#3  0x00007ffff57faad6 in engine_cleanup_cb_free (item=0x555555ccbae0) at eng_lib.c:198
#4  0x00007ffff580a507 in sk_pop_free (st=0x555555ccba80, func=0x7ffff57faad0 <engine_cleanup_cb_free>) at stack.c:327
#5  0x00007ffff57fadbc in ENGINE_cleanup () at eng_lib.c:205
#6  0x00007fffec3c627e in ossl_cleanup () at ../../lib/vtls/openssl.c:1228
#7  0x00007fffec3cc087 in Curl_ssl_cleanup () at ../../lib/vtls/vtls.c:254
#8  0x00007fffec37cd42 in curl_global_cleanup () at ../../lib/easy.c:264
#9  0x00007fffec5feab8 in zm_shutdown_curl (type=<optimized out>, module_number=18) at /usr/src/debug/php-7.4.28/ext/curl/interface.c:1439
#10 0x00005555556e54fb in module_destructor (module=module@entry=0x555555c951b0) at /usr/src/debug/php-7.4.28/Zend/zend_API.c:2563
#11 0x00005555556dee8c in module_destructor_zval (zv=<optimized out>) at /usr/src/debug/php-7.4.28/Zend/zend.c:768
#12 0x00005555556f0272 in _zend_hash_del_el_ex (prev=<optimized out>, p=<optimized out>, idx=17, ht=<optimized out>) at /usr/src/debug/php-7.4.28/Zend/zend_hash.c:1305
#13 _zend_hash_del_el (p=0x555555c48140, idx=17, ht=0x555555c1b880 <module_registry>) at /usr/src/debug/php-7.4.28/Zend/zend_hash.c:1328
#14 zend_hash_graceful_reverse_destroy (ht=ht@entry=0x555555c1b880 <module_registry>) at /usr/src/debug/php-7.4.28/Zend/zend_hash.c:1782
#15 0x00005555556e389c in zend_destroy_modules () at /usr/src/debug/php-7.4.28/Zend/zend_API.c:1995
#16 0x00005555556dfda3 in zend_shutdown () at /usr/src/debug/php-7.4.28/Zend/zend.c:1055
#17 0x0000555555682a08 in php_module_shutdown () at /usr/src/debug/php-7.4.28/main/main.c:2534
#18 0x00005555554ec4c8 in main (argc=2, argv=0x555555c320a0) at /usr/src/debug/php-7.4.28/sapi/cli/php_cli.c:1375

Here's the backtrace from PHP 8.0:

#0  0x00007fffef7f62b0 in ?? ()
#1  0x00007ffff55f8358 in ENGINE_remove (e=0x555555c7c990) at eng_list.c:284
#2  0x00007ffff55f84a5 in engine_list_cleanup () at eng_list.c:94
#3  0x00007ffff55f7ad6 in engine_cleanup_cb_free (item=0x555555c7ccc0) at eng_lib.c:198
#4  0x00007ffff5607507 in sk_pop_free (st=0x555555c7cc60, func=0x7ffff55f7ad0 <engine_cleanup_cb_free>) at stack.c:327
#5  0x00007ffff55f7dbc in ENGINE_cleanup () at eng_lib.c:205
#6  0x00007ffff2d9527e in ossl_cleanup () at ../../lib/vtls/openssl.c:1228
#7  0x00007ffff2d9b087 in Curl_ssl_cleanup () at ../../lib/vtls/vtls.c:254
#8  0x00007ffff2d4bd42 in curl_global_cleanup () at ../../lib/easy.c:264
#9  0x00007ffff2fca438 in zm_shutdown_curl (type=<optimized out>, module_number=19) at /usr/src/debug/php-8.0.16/ext/curl/interface.c:1318
#10 0x0000555555702c7b in module_destructor (module=module@entry=0x555555cacc30) at /usr/src/debug/php-8.0.16/Zend/zend_API.c:2631
#11 0x00005555556fc6cc in module_destructor_zval (zv=<optimized out>) at /usr/src/debug/php-8.0.16/Zend/zend.c:782
#12 0x000055555570d942 in _zend_hash_del_el_ex (prev=<optimized out>, p=<optimized out>, idx=18, ht=<optimized out>) at /usr/src/debug/php-8.0.16/Zend/zend_hash.c:1330
#13 _zend_hash_del_el (p=0x555555cce310, idx=18, ht=0x555555c1c000 <module_registry>) at /usr/src/debug/php-8.0.16/Zend/zend_hash.c:1353
#14 zend_hash_graceful_reverse_destroy (ht=ht@entry=0x555555c1c000 <module_registry>) at /usr/src/debug/php-8.0.16/Zend/zend_hash.c:1807
#15 0x00005555557011cc in zend_destroy_modules () at /usr/src/debug/php-8.0.16/Zend/zend_API.c:2008
#16 0x00005555556fd643 in zend_shutdown () at /usr/src/debug/php-8.0.16/Zend/zend.c:1078
#17 0x000055555569e768 in php_module_shutdown () at /usr/src/debug/php-8.0.16/main/main.c:2410
#18 0x00005555554f8228 in main (argc=2, argv=0x555555c330a0) at /usr/src/debug/php-8.0.16/sapi/cli/php_cli.c:1351

AWS Linux 2 Hints (if you need them, of course!)

  • To install PHP 7.4 under Amazon Linux 2, try amazon-linux-extras install php7.4
  • You'll also have to install php-ldap by doing yum install php-ldap
  • You can uninstall one version of php by doing yum remove php-pdo php-ldap php-debuginfo php-common php-mysqlnd php-fpm php-json php-cli then amazon-linux-extras disable php7.4 - then you can enable php8.0.
  • You'll be prompted by gdb to install debuginfo; if you want symbols in your backtraces you'll want to do that.

Please don't hesitate to reach out if there are any further details I can get for you. Thank you!

PHP Version

PHP 7.4.28

Operating System

Amazon Linux 2

Activity

  1. adoy commented on May 24, 2022

    @adoy
    Member

    Looking at the backtrace, it looks like the process segfault during curl extension shutdown calling an openssl cleanup. Could you please provide us the libcurl version, and also the openssl version you're using ?

  2. uberbrady commented on May 24, 2022

    @uberbrady
    Author

    Sure thing!

    rpm -qa |grep openssl
    openssl-1.0.2k-24.amzn2.0.2.x86_64
    openssl-libs-1.0.2k-24.amzn2.0.2.x86_64
    openssl-debuginfo-1.0.2k-24.amzn2.0.2.x86_64
    

    and

    python-pycurl-7.19.0-19.amzn2.0.2.x86_64
    curl-debuginfo-7.79.1-1.amzn2.0.1.x86_64
    libcurl-7.79.1-1.amzn2.0.1.x86_64
    curl-7.79.1-1.amzn2.0.1.x86_64
    

    And, just for good measure (in case this is some kind of openldap library thing) -

    openldap-2.4.44-23.amzn2.0.3.x86_64
    openldap-debuginfo-2.4.44-23.amzn2.0.3.x86_64
    
  3. uberbrady commented on May 24, 2022

    @uberbrady
    Author

    Ooooh! Interesting - if I take the s off of the ldaps:// name, the segfault stops happening...And if I put in ldap_start_tls($connection); then it starts happening again. So I think you're onto something!

  4. cmb69 commented on May 24, 2022

    @cmb69
    Member

    This might not be a php-src issue at all, but rather a conflict between libcurl and openldap (libsasl?)

    Anyhow, please focus on PHP 8.0, since PHP 7.4 is no longer actively supported, so wouldn't receive regular bug fixes anyway.

    And a question: are you using a threaded SAPI?

  5. uberbrady commented on May 24, 2022

    @uberbrady
    Author

    If I add php_sapi_name() I get cli - I'm not sure if that's what you're asking though. This is a distribution-default Amazon Linux 2 build, (which is pretty much just CentOS) so I don't imagine there is too much funny going on there. I'll attach php -i just in case that helps though. Here's a gist of the output: https://gist.github.com/uberbrady/4e43f38e45ff3762e617715ca0996894

    As for my focus, I help to run a rather large open source project, so my users often trail on their PHP releases, often for reasons out of their own control. We're dragging them forward as fast as we can. We only just pulled our minimum PHP version up to 7.4 with our latest release :/

  6. cmb69 commented on May 24, 2022

    @cmb69
    Member

    Thanks for the further info. I was looking for

    Thread Safety => disabled

    Since this is a non thread-safe build, there shouldn't be any multi-threading issues.

    From looking at the stack backtrace, libcurl calls ENGINE_cleanup(), and some other library may already have done this (or something related). But even if that would be something caused by PHP, I'm not sure it's worth spending time on that, since the OpenSSL initialization/cleanup is reworked as of OpenSSL 1.1 where this happens automatically behind the scenes. Can't you use some newer OpenSSL version?

  7. uberbrady commented on May 24, 2022

    @uberbrady
    Author

    Our open-source users can't in most cases, and in the cases where they could, they wouldn't know how to do that. In probably quite a few other cases, they might actually be able to, and know how, but not be allowed by their management.

    Most of them are just going to use the distro's default version of PHP against their distro's default version of openssl.

  8. crrodriguez commented on Jun 7, 2022

    @crrodriguez
    Contributor

    libcurl does no longer call engine_cleanup on recentish versions given that you are not using arcane openssl versions. this is not a PHP bug..you just need to build libcurl against openssl 1.1 and this will go away.
    Bugs like this will continue to pop up though, as long as there is shared global state.

  9. uberbrady commented on Jul 7, 2022

    @uberbrady
    Author

    It's honestly something I talked about with my VP of tech - custom-building our own PHP and curl and various other extensions to avoid this. He's loathe to do it because it means every point release of PHP means yet another build. But if we need to, we can do that.

    But what do I do for the rest of my open-source users? We guesstimate (it's impossible to know) that we have probably 10x or 100x the number of free, self-hosted, open-source users as we do hosted-by-us users. I don't think they'll have the technical acumen to be able to compile custom builds. As I mentioned before, we also have people who aren't permitted to custom-compile stuff by policy, as well. And in some cases, we have people running on shared infrastructure where they simply don't have access to be able to modify local installations of PHP.

    Amazon Linux is basically just a slightly tweaked version of CentOS, so this is a pretty large install base. I totally get that openssl 1.0 seems to have been a bit of a nightmare (and a nightmare you folks are trying to forget, for good reason), but I still do believe that folks doing yum install php should end up with a version of PHP that doesn't segfault.

  10. cmb69 commented on Jul 8, 2022

    @cmb69
    Member

    Okay, I've checked our OpenSSL requirements: ≥ 1.0.1 for PHP-8.0, ≥ 1.0.2 for PHP-8.1 and master. As such, it is our liability to do something about this. The question is, though, whether we can do something about it (except for documenting this issue).

    @bukka, thoughts?

  11. uberbrady commented on Jul 8, 2022

    @uberbrady
    Author

    I mean, I have an idea - but I suspect you're not gonna like it :)

    Is there a way for PHP to not clean up SSL stuff as its tearing itself down? I mean, once the binary actually terminates, isn't everything effectively gone? If it weren't, it would mean everything you control-C a PHP executable it would be leaving stuff around, which I suspect doesn't happen.

    Anyways, I don't know the codebase and I don't know how the internals work, but just thought I'd throw a janky idea out there...

  12. bukka commented on Jul 8, 2022

    @bukka
    Member

    Previously we had a similar issue with locking callback but that was freed by PHP so we could fix it. However this seems to be issue with freeing engines which PHP does not do as you can see in https://github.com/php/php-src/blob/PHP-8.0.16/ext/openssl/openssl.c#L1346-L1352 . I checked your version of LDAP https://github.com/openldap/openldap/blob/1c9416493bd219b08d839cd9e93fc64daa89b752/libraries/libldap/tls_o.c#L226-L235 and it also does not free engines either so it is interesting that it is crashing there. I might need to do a bit of digging later to see what is going on. Currently busy with some other things so might take me a little bit of time to get to it.

  13. added a commit that references this issue on Oct 17, 2022
    bbeee51
  14. 20 remaining items

  15. remicollet commented on Nov 24, 2022

    @remicollet
    Member

    At least, for ex with apr

    On CentOS 7 have apr 1.4.8-7 which includes fixes related to RTLD_DEEPBIND (reported to fix this issue, at least for mod_php), while Amzn2 have 1.7.0-9.amzn2...

  16. crrodriguez commented on Nov 24, 2022

    @crrodriguez
    Contributor

    Problem is that libcurl is build with libldap while php ext are properly build with libldap_r

    Huh,, That's ugly..indeed that's gonna be a problem ..

  17. crrodriguez commented on Nov 24, 2022

    @crrodriguez
    Contributor

    Problem is that libcurl is build with libldap while php ext are properly build with libldap_r

    Huh,, That's ugly..indeed that's gonna be a problem ..

    On my system libldap_r is just a symlink to libldap.. isn't that the case of this problematic target ?

  18. remicollet commented on Nov 24, 2022

    @remicollet
    Member

    On my system libldap_r is just a symlink to libldap..

    Yes on recent distro

    isn't that the case of this problematic target ?

    Not in this 8 years old distro.

  19. bukka commented on Nov 24, 2022

    @bukka
    Member

    That CentOS 7 segfault should be fixed by 3d90a24 . It is not exactly the same thing because the order seems to be different there so it's not triggered by curl but openssl ext. Might be due to using different lib as you noted but not sure. I see segfaults for all versions there.

    I think Amazon Linux 2 should be still important for us to support as it's default for AWS EC2 and it's heavily used and people most likely want to run latest PHP there. At least it's default when you create EC2 instance in console so we might need to support it (meaning support OpenSSL 1.0.2) for some time - think they extended support to June 2024.

  20. remicollet commented on Nov 25, 2022

    @remicollet
    Member

    I confirm 8.2.0RC7 with 3d90a24 doesn't raise segfault anymore (build 22 in my testing repo) on CentOS 7

  21. remicollet commented on Nov 25, 2022

    @remicollet
    Member

    Trying on amzn2 using https://git.remirepo.net/cgit/tools/docker.git/tree/amzn2-php82.dockerfile

    • "remi-php82" repo have 8.2.0-rc7-8, without any patch => segfault
    • "remi-php82-test" repo have 8.2.0-rc7-9, WITH your patch => also segfault
    $ podman build -t foo -f amzn2-php82.dockerfile .
    ...
    STEP 6/9: RUN php /tmp/ldap.php
    object(LDAP\Connection)#1 (0) {
    }
    bool(false)
    PHP Warning:  ldap_bind(): Unable to bind to server: Protocol error in /tmp/ldap.php on line 4
    container exited on segmentation fault
    ...
    
    
  22. bukka commented on Nov 25, 2022

    @bukka
    Member

    Yes that's expected as I didn't expect to fix the amzn2 issue because the problem is not in OpenSSL extension but in OpenLDAP as I mentioned above - it basically doesn't offer any way how to clean up the locking callback that it introduces... 😞

  23. jerm commented on Jul 17, 2023

    @jerm

    For what it's worth, our amzn2 issue (running 8.0) was fixed with @bukka 's import re-ordering suggestion: #8620 (comment)

    Thanks for that!

  24. richardeaxon commented on Feb 25, 2024

    @richardeaxon

    @uberbrady I have been thinking about this a bit more and realised that you can also fix it by prioritising ldap extension in the load process. It means if it loads before curl, then it will unload after curl so the thing that you can do is to just rename PHP ldap ext ini file. For default distro (Amazon Linux 2 in this case) packages it would be:

    mv /etc/php.d/20-ldap.ini /etc/php.d/10-ldap.ini
    

    If you are going to use Remi repo, it would be (this is for PHP 8.2 but if you want PHP 8.1, just s/php82/php81/g) :

    mv /etc/opt/remi/php82/php.d/20-ldap.ini etc/opt/remi/php82/php.d/10-ldap.ini
    

    I just tested it and there's no segfault if you do that.

    It might be worth if @remicollet could change it in the distributed packages. Not sure however how to change it for the main distro package or where to ask for it. Btw this bug impacts all distributions using OpenSSL 1.0.2 and loading ldap after curl - there's nothing Amazon Linux 2 specific here.

    Can confirm that this resolved apache php segfaults for me too when using ldap.

  25. andypost commented on Mar 24, 2024

    @andypost
    Contributor
  26. imThief commented on Jul 13, 2024

    @imThief

    There is a way to affect loading order for built-in (static) extensions?
    In Gentoo all extensions are controlled by use flags and there is no way to make it dynamic without altering .ebuild file unfortunately, thus this bug still exists in such situation...
    The only workaround that I've found is to build php through portage without curl and ldap extensions and then compile and install these extensions manually and load these dynamically via php.ini in the proper order but this way doesn't feels right :(

  27. cmb69 commented on Nov 5, 2024

    @cmb69
    Member

    think they extended support to June 2024.

    Seems they have extended to June 2025 in the meantime.

    There is a way to affect loading order for built-in (static) extensions?

    No.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions