Skip to content

Repository navigation

Potential Proxy refinements #405

Closed
Closed
Potential Proxy refinements#405
Labels
proxiesIssues related to HTTP and SOCKS proxies

Description

@lovelydinosaur
lovelydinosaur
opened on Sep 27, 2019

Having taken a good look over the new proxy functionality I'm really impressed!

There's a couple of things I'm interested in talking through...

  • Could httpx/dispatch/proxy_http.py simply be httpx/dispatch/proxy.py?
  • It would be nice if we could avoid exposing HTTPProxyMode. Alternatives could be...
    • tunnel=<bool>, forward=<bool> arguments (defaulting to None and doing sensible defaults on eg. tunnel=True|False.)
    • tunnel_only=False, forward_only=False arguments.
    • mode=<str>, and explicitly check for "forward" or "tunnel".
  • I'd quite like to see ProxyDispatch compose ConnectionPool rather than subclassing it, as that ends up with a better seperation.
  • Possibly have the docs link to some relevant places wrt. tunnelling vs. forwarding, since we're using some pretty technical terms here without neccessarily giving enough context to users.

I'm happy to take on any of these bits myself, but figure they're worth talking through first.

Also for reference, are proxies={"all:/...": ...} and proxies={"all": ...} conventions that are used in requests?

Activity

  1. sethmlarson commented on Sep 27, 2019

    @sethmlarson
    sethmlarson
    on Sep 27, 2019
    Contributor
    • SOCKS support is planned for the future where I would use httpx/dispatch/proxy_socks.py
    • That can be changed, those are the only two modes that have been around for a long time so I don't think extensibility is an issue. Mostly used the enum before for type-checkability. I think keeping proxy_mode but taking a string instead is best.
    • That's true. It'd also be an interesting idea to pass the client's connection pool instance to the proxy so that the # of connections has the same limit as the overall client.
    • Agree that tunneling and forwarding should use some external documentation, maybe MDN.
    • Requests does use the all and all://host: https://github.com/psf/requests/blob/master/requests/utils.py#L783

  2. lovelydinosaur commented on Sep 27, 2019

    @lovelydinosaur
    lovelydinosaur
    on Sep 27, 2019
    MemberAuthor

    SOCKS support is planned for the future where I would use httpx/dispatch/proxy_socks.py

    Okay. I'd probably tend towards proxy.py and socks.py? Not obvs. if socks would be third-party, optional, or always available perhaps? Not entirely sure, but proxy_http.py feels a bit blunt?

    I think keeping proxy_mode but taking a string instead is best.

    That seems okay, yeah.

    It'd also be an interesting idea to pass the client's connection pool instance to the proxy so that the # of connections has the same limit as the overall client.

    Ah yeah - we do want to share the same connection pool - good point.

    Agree that tunneling and forwarding should use some external documentation, maybe MDN.

    Yeah not a bad option. I still think even the MDN docs are a bit light on actually giving motivation focused documentation on the difference between the two, but ho-hum.

  3. sethmlarson commented on Sep 27, 2019

    @sethmlarson
    sethmlarson
    on Sep 27, 2019
    Contributor

    Why would you tend towards removing the proxy_ prefix? They're both proxies, just different methods of proxying. Users won't see the modules anyways as the members are exposed top-level, the names are for our benefit.

    And speaking of SOCKS, we're implementing a lightweight sans-I/O SOCKS library without any dependencies unlike all current solutions so it can be marked as a dependency and SOCKS proxies will be first-class just like HTTP. :)

  4. nwalsh1995 commented on Sep 30, 2019

    @nwalsh1995
    nwalsh1995
    on Sep 30, 2019
    Contributor

    Should connecting to HTTPS proxy be added onto this issue? Tunneling or forwarding mode does not work for me and I get the same ssl.SSLError: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:1045) error mentioned in #259.

    It was recommended to connect to the proxy over HTTP and the HTTP CONNECT tunnel works well, but initially connecting over the proxy over HTTP is insecure and should be avoided since a MiTM could act like the server you are trying to connect to. This problem is avoided when using HTTPS.

  5. sethmlarson
    added
    proxiesIssues related to HTTP and SOCKS proxies
    on Oct 17, 2019

  6. yeraydiazdiaz commented on May 13, 2020

    @yeraydiazdiaz
    yeraydiazdiaz
    on May 13, 2020
    Contributor

    Closing this issue as the points have either been already addressed or are reflected in other issues in httpx and httpcore.

  7. yeraydiazdiaz
    closed this as completedon May 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    proxiesIssues related to HTTP and SOCKS proxies

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions