[sethmlarson]

Makes it so you can pass iterators, async iterators, open files and file-like objects, and raw strings as data. Do we need Request.is_streaming anymore?

[sethmlarson]

Another bigger question is if we move towards having Request.content be this interface, should we make it private?

[lovelydinosaur]

Something along these lines seems like a good tack, yup.

A thought... perhaps rather than a custom interface, our wrapper class should be an implementation of IOBase. (With UnsupportedOperation for various stuff that's not relevant.)

Ie.

class ByteStream(io.IOBase):
    """
    Wraps various text/bytes data types up into a single standard byte stream interface.
    """
    def __init__(self, content: typing.Union[
        str,
        bytes,
        typing.Iterator[typing.AnyStr],
        typing.IO[typing.AnyStr],
    ]):
        ...

(We'd also end up needing an equivelent async alternative.)

[sethmlarson]

I started off on the IOBase route and it's a lot more involved. :) I'll grit my teeth through it though.

[lovelydinosaur]

Ah okay, that's a shame. I'd figured there'd be a fair bit of functionality that's covered by the built-in mixin implementations. (Also, I think we should probably roll a streams.py module if we do go down that route, since it's a biggish chunk of stuff and we probably want to keep it isolated.)

[sethmlarson]

I implemented via IOBase and this is what I've got right now. I'm not happy with the only way to test both RequestDataStream and AsyncRequestDataStream in the same testcase is via request.content._wrapped.read() but I don't see how else aside from a combined interface as before. What are your thoughts on this patch @tomchristie?

[lovelydinosaur]

Presumably we need an else error case here?

[lovelydinosaur]

Okay, so this is quite large ATM.

There's also a few bits that I think we could seperate out more cleanly. Eg. I don't think RedirectBodyUnavailable should have any place in the stream class. Plus I think that the byte stream implementations are a distinct enough piece of work that they should live in an entirely seperate streams.py module.

So that we can get nice clean PRs onto all this I'd probably suggest:

• Farm out all the typo any cleanup stuff that's here into a seperate PR.
• We start with just a PR for a nicely seperate ByteStream class - without even hooking it in to anything else, or thinking about the async variant. Just a clean isolated interface, that's got test cases that exercise it.

[lovelydinosaur]

We could support this if we wanted, by reading and throwing away the data.