Skip to content

Consider URLLib3Transport interface, since it's becoming public API. #966

Closed

Description

@lovelydinosaur

Follow on from #963

There's a couple of things we could consider in the URLLib3Transport interface, now that it's becoming public API.

  • We don't really want to pass pool_limits, since it doesn't quite fit the parameters that urllib3 uses to configure it's PoolManager. We needed to pass it before, since we were using the class for our sync implementation, and needed the consistency between our async and sync cases, but now that we're providing it separately we should just mirror their native num_pools and maxsize.
  • With httpcore we provide different classes for the proxy and the plain-connection-pool transports. We could? choose to do the same with the urllib3 transport, rather than providing a single class with an optional proxy=... argument.

Activity

  1. added this to the v0.13 milestone on May 21, 2020
  2. lovelydinosaur commented on May 21, 2020

    @lovelydinosaur
    MemberAuthor

    Also we could just drop the URLLib3Dispatch compat shim, since it's only ever existed in the pre-releases.

    And in any case, never made it to the public interface even then.

  3. florimondmanca commented on May 21, 2020

    @florimondmanca
    Contributor

    The only reason I'd be in favor of keeping URLLib3Transport is to make it easier for folks that use Requests to migrate over without having to deal with slight behavior differences right away.

    I'm actually not certain that's what people would do in practice - there are already lots of behavior differences that come from switching from Requests anyway, so not sure if switching to our transport is a big leap.

    Another argument in favor of dropping it before 1.0 lands would be reducing maintenance burden - now that the transport interface is public it would be pretty easy for someone to maintain this as a 3rd party component (esp given that we have a working implementation in 0.12.x for people to take inspiration from). But at the same time, it's not very high cost.

    Edit: oh okay, I didn't get that you were talking about the Transport/Dispatch compatibility layer. Sure, let's drop it IMO. I guess what I wrote above^ is another discussion then. 😄

  4. yeraydiazdiaz commented on May 21, 2020

    @yeraydiazdiaz
    Contributor

    I'm 👍 on dropping URLLib3Transport completely tbh.

    I might be missing something but it seems atm it feels like the remnants of our old sync flow rather than a core piece of HTTPX that users would reach for. It already has our lowest test coverage percentage, feels like it will eventually break for someone since it's now in the public API.

  5. lovelydinosaur commented on May 21, 2020

    @lovelydinosaur
    MemberAuthor

    Okay, well let's drop the URLLib3Transport/URLLib3ProxyTransport convo from the 0.13 milestone so we can get this thing released, and keep the idea of URLLib3Transport as a third-party-package as an open option.

    (Personally I probably think it's a pretty useful thing to have available to us, but I'm not absolute on it.)

  6. modified the milestones: v0.13, v1.0 on May 21, 2020
  7. lovelydinosaur commented on May 21, 2020

    @lovelydinosaur
    MemberAuthor

    Remaining points for 1.0 consideration...

    • Do we want URLLib3Transport/URLLib3ProxyTransport for consistency with httpcore interface, and urllib3's PoolManager/ProxyManager?
    • Do we want to drop URLLib3Transport into a third party package instead?
  8. lovelydinosaur commented on Jul 25, 2020

    @lovelydinosaur
    MemberAuthor

    We can close this off now.

    Once #1090 is addressed, the signatures are...

    class URLLib3Transport(httpcore.SyncHTTPTransport):
        def __init__(
            self,
            *,
            verify: VerifyTypes = True,
            cert: CertTypes = None,
            trust_env: bool = None,
            pool_connections: int = 10,
            pool_maxsize: int = 10,
            pool_block: bool = False,
        ):

    And...

    class URLLib3ProxyTransport(URLLib3Transport):
        def __init__(
            self,
            *,
            proxy_url: str,
            proxy_headers: dict = None,
            verify: VerifyTypes = True,
            cert: CertTypes = None,
            trust_env: bool = None,
            pool_connections: int = 10,
            pool_maxsize: int = 10,
            pool_block: bool = False,
        ):
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions