Repository navigation
Does HTTPX support single-connection HTTP/2 with unbounded concurrent requests? #281
Description
First sorry for opening an issue but I didn't find a better way to reach you. Thank you for developing a good looking library which supports HTTP/2.
We are tinking for switching to HTTPX. Before I start with the implementation, I have two questions about parallel requests. Those things were not clear from the documentation.
-
We have a server that supports HTTP/2 protocol and wants to send multiple parallel requests through one connection to the server. Does
httpx.Client()use of one connection for multiple parallel requests or it connects multiple times? What abouthttpx.parallel()? -
If it is possible to send multiple parallel requests through one connection, do we need to handle the number of requests manually when we send a huge number of requests (e.g. 1000) - that we do not reach the limit of requests per connection. What I mean is do we need to send let say 100 requests and then wait for responses or do httpx library do that for us?
Activity
lovelydinosaur commentedon Aug 27, 2019
Yes. HTTPX deals with connection pooling for you.
Right now it won't limit the number of requests available per HTTP/2 connection, no. I'm not clear what the preferred behavior would be here. (Eg. we could either open up multiple HTTP/2 connections per origin in that case if we wanted, or we could block until the requests-per-connection limit is hit.) Does HTTP/2 have any specific rules around limits here, or rules around servers informing clients about appropriate limits?
sethmlarson commentedon Aug 27, 2019
@tomchristie HTTP/2 has a limit on concurrent streams that are allowed to be open, I don't think HTTPX handles that case? (ie we don't wait for other streams to get closed before we fire off the next request) so that could cause some ProtocolErrors if you fire off a huge amount requests all at once.
sethmlarson commentedon Aug 27, 2019
Also @PrimozGodec thank you for giving HTTPX a whirl, please report back with any issues you're having with using it and we'll try to resolve them 🙇♂️
lovelydinosaur commentedon Aug 27, 2019
Correct - It doesn't yet handle that, no.
[-]One connetion multiple requests[/-][+]Does HTTPX support single-connection HTTP/2 with unbounded concurrent requests?[/+]sethmlarson commentedon Aug 27, 2019
I created #282 to track implementing that behavior.
lovelydinosaur commentedon Aug 27, 2019
Closing the question now that we have relevant issues to track it.
PrimozGodec commentedon Aug 27, 2019
@tomchristie and @sethmlarson thank you for great and fast response.