Skip to content

Fix download of compressed and chunked responses - #383

Open
iLLiCiTiT wants to merge 2 commits into
developfrom
bugfix/download-content-encoding
Open

iLLiCiTiT wants to merge 2 commits into
developfrom
bugfix/download-content-encoding

Conversation

@iLLiCiTiT

@iLLiCiTiT iLLiCiTiT commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

Changelog Description

download_file and download_file_to_stream work when the server (or the host it redirects to) sends compressed content (Content-Encoding: gzip) or no Content-Length (chunked transfer).

Additional info

Two problems in _download_file_to_stream:

  1. Compressed responses. requests asks for gzip by default. Content-Length is then the compressed size, but iter_content returns decoded bytes. The size check failed (Downloaded 16384 out of 420 bytes), and the retry sent a Range based on the decoded size, which the server answered with 416, or with a wrong part of the compressed content.
  2. No Content-Length. A chunked response failed with KeyError: 'content-length'.

Changes:

  • Downloads send Accept-Encoding: identity, so the server sends the file as it is, and size and Range match the written bytes.
  • If the server compresses content anyway, the size isn't checked against decoded content, and a failed attempt starts again from the beginning instead of resuming with Range. The Range header is removed at the start of every attempt, so a restart doesn't keep the previous attempt's Range.
  • A response without Content-Length leaves the content size unknown instead of failing. The size is also no longer taken from a 206 response, where it's only the size of the remaining part.
  • The raw bytes received are compared with Content-Length of every response. urllib3 1.26 accepts a response that ends early, which the decoded size check can't catch when the total size is unknown (compressed content, or a resumed download of a chunked response).

Testing notes:

  1. pytest tests/test_download_content_encoding.py, with a local server that compresses content when accepted, always compresses, cuts the first compressed response in half, sends chunked content, cuts a chunked response and then a resumed one, or answers a resumed request with compressed content that is cut. All tests pass on requests 2.27.1 + urllib3 1.26 and requests 2.32.3 + urllib3 2.6, and fail on develop.
  2. Other offline tests pass on both (download resume, token validation, EventHub lifecycle and ping, request retries, max retries kwarg, server version validation, GraphQL pagination).
  3. python automated_api.py makes no changes.

- Download asks for uncompressed content ('Accept-Encoding: identity').
  'Content-Length' and 'Range' of compressed response are related to
  compressed content, but 'iter_content' returns decoded content, so
  the size check failed and resume requested wrong range.
- When server compresses content anyway, size isn't checked against
  decoded content and failed download restarts instead of resuming.
  Incomplete compressed content is detected by received raw bytes,
  'urllib3' 1.x does not fail on it.
- Response without 'Content-Length' (chunked) doesn't fail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@iLLiCiTiT iLLiCiTiT added the type: bug Something isn't working label Nov 10, 2026
@iLLiCiTiT

Copy link
Copy Markdown
Member Author

The question is if we do actually use gzip anywhere, I'd say we can handle this when it becomes an issue.

@iLLiCiTiT
iLLiCiTiT marked this pull request as draft November 10, 2026 13:21
@BigRoy

BigRoy commented Nov 10, 2026

Copy link
Copy Markdown
Member

The question is if we do actually use gzip anywhere, I'd say we can handle this when it becomes an issue.

On "do we actually use gzip anywhere": the AYON backend doesn't compress, but reverse proxies do. With Caddy's common encode gzip line, develop fails immediately with KeyError: 'content-length' for download_file and download_file_to_stream, because Caddy sends the response chunked. A CDN that gzips (redirect to raw.githubusercontent.com) fails with 416, the bug from my first report.

And possibly relevant to note is: 🤔

Caddy's default encode only compresses text-like types, so zip and image downloads were never affected. JSON, text and similar files were.

But it's been untested against a real S3.

@iLLiCiTiT
iLLiCiTiT marked this pull request as ready for review November 10, 2026 15:24
@iLLiCiTiT iLLiCiTiT self-assigned this Nov 10, 2026
@iLLiCiTiT
iLLiCiTiT requested review from BigRoy and a balanced review from Copilot November 10, 2026 15:24

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Resumed downloads can silently report truncated files as successful, and encoded-response retries can retain a stale Range header that prevents a proper restart.

2 open issues
What changed in this PR

This PR updates the shared download implementation used by download_file and download_file_to_stream to handle compressed responses and responses without a known size.

Changes:

  • Requests uncompressed content and restarts failed compressed downloads rather than resuming them.
  • Handles missing Content-Length and checks compressed response completeness using raw bytes received.
  • Adds local HTTP-server tests for compression, interrupted compressed transfers, and chunked responses.
File Description
tests/​test_download_content_encoding.py Adds four regression tests for compression and chunked downloads.
ayon_api/​server_api.py Updates download headers, retry handling, and response-size validation.

Review effort: Balanced

Comment thread ayon_api/server_api.py Outdated
Comment on lines +2015 to +2027
if is_encoded and content_length:
# 'urllib3' 1.x does not fail when compressed
# content is incomplete, compare received bytes
received = getattr(response.raw, "tell", None)
if (
received is not None
and received() < int(content_length)
):
raise requests.exceptions.ConnectionError(
f"Downloaded {received()} out of"
f" {content_length} compressed bytes"
f" from '{url}'."
)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in c4a8ddb. Received raw bytes are now compared with Content-Length of every response, not only compressed ones. Covered by test_download_resumed_response_is_complete (chunked response cut, then a resumed 206 that ends early), which failed on urllib3 1.26 before the fix.

Comment thread ayon_api/server_api.py
stream.seek(0)
stream.truncate()
progress.reset_transferred()
offset = 0

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in c4a8ddb. Range is removed at the start of every attempt and set again only when resuming. Covered by test_download_restart_does_not_keep_range (resumed response comes back compressed and is cut, the next request has no Range).

- Received raw bytes are compared with 'Content-Length' of every
  response, not only compressed. A resumed download of chunked response
  has unknown total size, and 'urllib3' 1.x accepted a resumed response
  that ended early.
- 'Range' header is removed at start of every attempt, so a restart
  after compressed response does not keep 'Range' of previous attempt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants