When sending binary file to a Microsoft FTP server over FTP TLS, the SSL unwind method hangs

@tiran

BPO	34557
Nosy	@tiran

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-08-31.18:00:53.069>
labels = ['3.10', 'library', '3.9', 'type-crash', '3.11']
title = 'When sending binary file to a Microsoft FTP server over FTP TLS, the SSL unwind method hangs'
updated_at = <Date 2021-12-11.19:06:37.095>
user = 'https://bugs.python.org/JamesCampbell2'

bugs.python.org fields:

activity = <Date 2021-12-11.19:06:37.095>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2018-08-31.18:00:53.069>
creator = 'James Campbell2'
dependencies = []
files = []
hgrepos = []
issue_num = 34557
keywords = []
message_count = 1.0
messages = ['324440']
nosy_count = 2.0
nosy_names = ['christian.heimes', 'James Campbell2']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue34557'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

When using the FTP library to transfer a binary file to a Microsoft FTP server using TLS, then the library will hang when unwinding the connection until it finally times out.

The storbinary method calls conn.unwind which seems to have an issue with SSL connections with a Microsoft server. If we terminate the connection early the file is successfully transferred so it's just the unwind procedure that crashes and hangs our server until it times out.

We are able to work around it by creating our own version of the storbinary method which just closes the connection and doesn't do the unwind step.

It's not clear why the library does this step since we never need to drop down to an unencrypted connection so it should be enough to just close it once done.

You can read more information on this by somebody else with Python 3.2
http://www.sami-lehtinen.net/blog/python-32-ms-ftps-ssl-tls-lockup-fix

Closing because of lack of a reproducer.

Just came across this against an Azure FTPS server, this issue still persists. Can I help somehow?

For now using the workaround mentioned here: https://stackoverflow.com/a/50129806

Python version: Python 3.11.2 (v3.11.2:878ead1ac1, Feb 7 2023, 10:02:41) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin

@kumaraditya303

@kumaraditya303, reopening because we seem to get some reproducer (in the StackOverflow question mentioned by @rikroe):

This appears to be an issue with Python's SSLSocket class, which is waiting for data from the server when running unwrap. Since it never receives this data from the server, it is unable to unwrap SSL from the socket and therefore times out.

This server in particular I have identified by the welcome message as some Microsoft FTP server, which fits in well with the issue written about in this blog

The "fix" (if you can call it that) was to stop the SSLSocket from attempting to unwrap the connection altogether by editing ftplib.py and amending the FTP_TLS.storbinary() method.
def storbinary(self, cmd, fp, blocksize=8192, callback=None, rest=None):
  self.voidcmd('TYPE I')
  with self.transfercmd(cmd, rest) as conn:
    while 1:
      buf = fp.read(blocksize)
      if not buf: break
      conn.sendall(buf)
      if callback: callback(buf)
    # shutdown ssl layer
    if isinstance(conn, ssl.SSLSocket):
      # HACK: Instead of attempting unwrap the connection, pass here
      pass
  return self.voidresp()

I'm able to reproduce this as well on Docker Image python:3.10-bullseye. Implementing the workaround above works.

The Microsoft FTP Service I used belongs to a third party.

I'm also able to reproduce, on 3.8.10, and the workaround above works. I've not dug into the nitty-gritty of unwrap.

I'm also running into this issue attempting to connect to a partner's server branded "Microsoft FTP Service".

As others have mentioned, why is the call to unwrap necessary? From what I understand, conn should be closed immediately after anyway when the context manager exits.

@kumaraditya303

@kumaraditya303 since there are now reproducers available, this issue is not stale anymore. This bug has been existent for several years now, AFAIK with no fix. What's the next steps?

Navigation Menu

Repository navigation

When sending binary file to a Microsoft FTP server over FTP TLS, the SSL unwind method hangs #78738

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

When sending binary file to a Microsoft FTP server over FTP TLS, the SSL unwind method hangs #78738

Description

Activity

JamesCampbell2 commented on Aug 31, 2018

kumaraditya303 commented on Jun 19, 2022

rikroe commented on Apr 16, 2023

arhadthedev commented on Apr 16, 2023

emkZero commented on Apr 20, 2023

bwinston-sdp commented on Aug 21, 2023

jvolkman commented on Jan 9, 2024

emkZero commented on Jan 31, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions