Repository navigation
test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI #88395
Description
Activity
See https://github.com/python/cpython/pull/26104/checks?check_run_id=2662511684
======================================================================
ERROR: test_get_server_certificate (test.test_ssl.SimpleBackgroundTests)
----------------------------------------------------------------------Traceback (most recent call last): File "/Users/runner/work/cpython/cpython/Lib/test/test_ssl.py", line 2132, in test_get_server_certificate _test_get_server_certificate(self, *self.server_addr, cert=SIGNING_CA) File "/Users/runner/work/cpython/cpython/Lib/test/test_ssl.py", line 2329, in _test_get_server_certificate pem = ssl.get_server_certificate((host, port), ca_certs=cert) File "/Users/runner/work/cpython/cpython/Lib/ssl.py", line 1520, in get_server_certificate with create_connection(addr, timeout=timeout) as sock: File "/Users/runner/work/cpython/cpython/Lib/socket.py", line 844, in create_connection raise err File "/Users/runner/work/cpython/cpython/Lib/socket.py", line 832, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 61] Connection refused
- changed the title
[-]test_get_server_certificate fails on macOS[/-][+]test_get_server_certificate fails intermittently on macOS[/+]on May 25, 2021 - changed the title
[-]test_get_server_certificate fails on macOS[/-][+]test_get_server_certificate fails intermittently on macOS[/+]on May 25, 2021 I neither have a macOS nor a Windows machine to reproduce and debug the issue.
Can you maybe use a VM for Windows?
I’ll see if I can provoke it on my Mac.
What’s the preferred solution?
- Catch the connection failure and skip the test?
- Retry connection?
- Find the root cause and make sure it never ever happens :)
Maybe 2. is acceptable :)
Is possible that these are flaky tests but I am afraid this may be something more important so o would prefer to go with 3 ;)
I feared that ;)
Getting there. The error message is similar:
$ ./python.exe -m test test_ssl -m test_get_server_certificate -u all -F ... 0:02:11 load avg: 2.10 [328] test_ssl test test_ssl failed -- Traceback (most recent call last): File "/Users/erlendaasland/src/cpython-ssl/Lib/test/test_ssl.py", line 2132, in test_get_server_certificate _test_get_server_certificate(self, *self.server_addr, cert=SIGNING_CA) File "/Users/erlendaasland/src/cpython-ssl/Lib/test/test_ssl.py", line 2329, in _test_get_server_certificate pem = ssl.get_server_certificate((host, port), ca_certs=cert) File "/Users/erlendaasland/src/cpython-ssl/Lib/ssl.py", line 1521, in get_server_certificate with context.wrap_socket(sock, server_hostname=host) as sslsock: File "/Users/erlendaasland/src/cpython-ssl/Lib/ssl.py", line 518, in wrap_socket return self.sslsocket_class._create( File "/Users/erlendaasland/src/cpython-ssl/Lib/ssl.py", line 1070, in _create self.do_handshake() File "/Users/erlendaasland/src/cpython-ssl/Lib/ssl.py", line 1339, in do_handshake self._sslobj.do_handshake() ConnectionResetError: [Errno 54] Connection reset by peer
test_ssl failed
== Tests result: FAILURE ==
Got it:
0:02:29 load avg: 2.81 [389] test_ssl test test_ssl failed -- Traceback (most recent call last): File "/Users/erlendaasland/src/cpython-ssl/Lib/test/test_ssl.py", line 2132, in test_get_server_certificate _test_get_server_certificate(self, *self.server_addr, cert=SIGNING_CA) File "/Users/erlendaasland/src/cpython-ssl/Lib/test/test_ssl.py", line 2329, in _test_get_server_certificate pem = ssl.get_server_certificate((host, port), ca_certs=cert) File "/Users/erlendaasland/src/cpython-ssl/Lib/ssl.py", line 1520, in get_server_certificate with create_connection(addr, timeout=timeout) as sock: File "/Users/erlendaasland/src/cpython-ssl/Lib/socket.py", line 844, in create_connection raise err File "/Users/erlendaasland/src/cpython-ssl/Lib/socket.py", line 832, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 61] Connection refused
test_ssl failed
== Tests result: FAILURE ==
$ git diff diff --git a/Lib/ssl.py b/Lib/ssl.py index 2b131de043..9c281d8028 100644 --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -257,8 +257,9 @@ class _TLSMessageType: if sys.platform == "win32": from _ssl import enum_certificates, enum_crls +from test.support import SHORT_TIMEOUT as _GLOBAL_DEFAULT_TIMEOUT from socket import socket, SOCK_STREAM, create_connection -from socket import SOL_SOCKET, SO_TYPE, _GLOBAL_DEFAULT_TIMEOUT +from socket import SOL_SOCKET, SO_TYPE import socket as _socket import base64 # for DER-to-PEM translation import errno
$ % ./python.exe -m test test_ssl -m test_get_server_certificate -u all -F ... 0:03:27 load avg: 2.24 [535] test_ssl test test_ssl failed -- Traceback (most recent call last): File "/Users/erlendaasland/src/cpython-ssl/Lib/test/test_ssl.py", line 2132, in test_get_server_certificate _test_get_server_certificate(self, *self.server_addr, cert=SIGNING_CA) File "/Users/erlendaasland/src/cpython-ssl/Lib/test/test_ssl.py", line 2329, in _test_get_server_certificate pem = ssl.get_server_certificate((host, port), ca_certs=cert) File "/Users/erlendaasland/src/cpython-ssl/Lib/ssl.py", line 1521, in get_server_certificate with create_connection(addr, timeout=timeout) as sock: File "/Users/erlendaasland/src/cpython-ssl/Lib/socket.py", line 844, in create_connection raise err File "/Users/erlendaasland/src/cpython-ssl/Lib/socket.py", line 832, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 61] Connection refused
test_ssl failed
== Tests result: FAILURE ==
29 remaining items
FYI, here's a CI failure against 3.9:
https://github.com/python/cpython/runs/2894813367?check_suite_focus=truehttp://erickt.github.io/blog/2014/11/19/adventures-in-debugging-a-potential-osx-kernel-bug/ says:
"If we trigger a send while the kernel is in the middle of tearing down the socket, it returns EPROTOTYPE." (instead of ECONNRESET)
Maybe send() could raise a ConnectionResetError exception on EPROTOTYPE?
But here, it is write() that returns EPROTOTYPE. See msg394798.
I'd be interested in hearing Ronald's opinion. (Added to nosy.)
test_ssl, ignore spurious EPROTOTYPE on macOS #26893Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields: