Skip to content

A short read before end of data makes upload checksum and length cover only a prefix #654

Closed

Description

@sophiecarreras

What happens

Both the checksum helper and the hashing stream treat any read that returns fewer bytes than requested as end of data. A source whose read(n) returns fewer than n bytes before the real end (for example a pipe or socket-like stream) gets a SHA1 and length covering only the prefix, so a truncated object can be uploaded with a matching checksum.

Where

b2sdk/_internal/utils/__init__.py:183 (IncrementalHexDigester.update_from_stream, data_len < to_read stops the loop); b2sdk/_internal/stream/hashing.py:65 (StreamWithHash.read, len(data) < size is taken as end of stream).

Why it matters

Impact: Data, silent. Severity: Critical, because it needs a source that returns short reads before end of data, and I have no evidence of how common such sources are, so I kept the highest level.

How to check

Wrap a BytesIO in a stream whose read(n) returns at most 3 bytes and pass it to hex_sha1_of_unlimited_stream; the returned length is 3, not the full length. Found by reading the code, not reproduced.

Version

Seen at v2.13.4. Present on the default branch: present.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions