http2 not work when proxies apply to the AsyncClient

import asyncio
import httpx
proxies = {
    "https": "http://127.0.0.1:12590",
    "http": "http://127.0.0.1:12590"
}
async def main():
    async with httpx.AsyncClient(http2=True, verify=False, proxies=proxies) as client:
        response = await client.get('https://www.google.com/')
        print(response.status_code)
        print(response.http_version)

asyncio.run(main())

200
HTTP/1.1

import asyncio
import httpx
async def main():
    async with httpx.AsyncClient(http2=True, verify=False) as client:
        response = await client.get('https://www.google.com/')
        print(response.status_code)
        print(response.http_version)

asyncio.run(main())

200
HTTP/2

@houoop

Hi @houoop,

I went ahead and fixed the code formatting.

I'm wondering if the reason you don't get HTTP/2 back is due to verify=False. AFAIK HTTPS is required by us for HTTP/2 to work (we don't support Upgrade: h2c yet, see #503).

In general, what really helps is to be able to reproduce against a 100% local setup. Can you clarify what your localhost proxy is?

Eg I tried to spin up a local HTTP/2 + HTTPS setup using trustme-cli, hypercorn and mitmproxy. I'm running into certificate issues between the server and the proxy, but here's what I've got:

Click to expand

Dependencies:

$ python -m venv venv
$ . venv/bin/activate
$ pip install hypercorn starlette trustme-cli mitmproxy

Generate localhost certificates:

$ trustme-cli
$ cat server.key server.pem > cert.pem

ASGI app:

# app.py
from starlette.applications import Starlette
from starlette.routing import Route
from starlette.responses import PlainTextResponse


async def home(request):
    return PlainTextResponse("Hello, world!")


app = Starlette(routes=[Route("/", home)])

Start hypercorn:

$ hypercorn app:app --keyfile server.key --certfile server.pem

Start the proxy:

$ mitmdump --cert "*=cert.pem"

HTTPX script:

import asyncio
import httpx

proxies = {"http": "http://localhost:8080", "https": "http://localhost:8080"}


async def main():
    async with httpx.AsyncClient(
        http2=True, verify="client.pem", proxies=proxies
    ) as client:
        r = await client.get("https://localhost:8000")
        print(r, r.http_version)
        print(r.text)


asyncio.run(main())

HTTPX output:

<Response [502 Bad Gateway]> HTTP/1.1
<Response [502 Bad Gateway]> HTTP/1.1
<html>
            <head>
                <title>502 Bad Gateway</title>
            </head>
            <body>
            <h1>502 Bad Gateway</h1>
            <p>Certificate verification error for localhost: unable to get local issuer certificate (errno: 20, depth: 0)</p>
            </body>
        </html>

mitmproxy output:

127.0.0.1:61974: clientconnect
127.0.0.1:61974: CONNECT localhost:8000
 << Certificate verification error for localhost: unable to get local issuer certificate (errno: 20, depth: 0)
127.0.0.1:61974: clientdisconnect

HTTP/2 cleartext can be achieved in 2 ways:

First the client sends an HTTP/1.1 request requesting to the server an upgrade to HTTP/2 by providing some headers like: Connection: Upgrade, HTT2Settings, Upgrade: h2c, HTTP2Settings: ... . If the server can speak HTTP/2 then it answers with a response status 101 (Switching Protocol), and then the actual response to the request in HTTP/2 protocol.
Second is known as "with prior knowledge". The client does not send a first HTTP/1.1 request but directly send the request in HTTP/2 format (with frames...) assuming "with prior knowledge" that the server understands the HTTP/2 protocol. All of this without TLS.

Implementing the second option "with prior knowledge" would be very useful to query HTTP API without having to generate certificates (even self-signed).

@florimondmanca

I'm wondering if the reason you don't get HTTP/2 back is due to verify=False. AFAIK HTTPS is required by us for HTTP/2 to work (we don't support Upgrade: h2c yet, see #503).

@florimondmanca I can verify that setting verify to anything, not just False, seems to break the HTTP/2 support. See example:

import httpx
import asyncio
import ssl
async def main():
	ctx = ssl.SSLContext()
	ctx.set_ciphers("ECDHE-ECDSA-AES128-GCM-SHA256")
	async with httpx.AsyncClient(http2=True, verify=ctx) as client:
		response = await client.get('https://www.google.com')
		print(response.http_version)

asyncio.run(main())

Is there a better way to specify SSL cipher suites that does not break HTTP/2 support?

im having these exact problem, using verify and http2 and proxy #905

is there any solution?

@gnubibored No, there doesn’t seem to be an obvious solution yet; looks like something is simply not working on our side.

There is work being done in the HTTPCore repo to improve our proxy implementation and properly support HTTPS there, hopefully it will help us resolve this.

the issue is this wasnt happening when 4 months ago i was using httpx, so im gonna try using a previous release for see in what change works and in what no

Thanks, would be useful to pinpoint at which release this started occurring, indeed. :-)

Someone found something about this? I been trying different versions but im not able to reproduce the issue, maybe i was wrong, in the ones i tested i get errors with ssl or i get http/1 when using proxy

I just merged a change in HTTPCore dealing with this issue so this should be resolved from httpx 0.13.

@yeraydiazdiaz

@yeraydiazdiaz Thanks, I'm looking forward to the latest release.

I just merged a change in HTTPCore dealing with this issue so this should be resolved from httpx 0.13.

I can confirm the issue is resolved in 0.13.0.dev1 release. HTTP/2 is preserved when specifying a custom verify SSL context.

Navigation Menu

Repository navigation

http2 not work when proxies apply to the AsyncClient #873

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

http2 not work when proxies apply to the AsyncClient #873

Description

Activity

florimondmanca commented on Mar 24, 2020

laurentvaills commented on Apr 2, 2020

valiant1x commented on Apr 10, 2020

qwerty32123 commented on Apr 12, 2020

florimondmanca commented on Apr 13, 2020

qwerty32123 commented on Apr 13, 2020

florimondmanca commented on Apr 13, 2020

qwerty32123 commented on Apr 19, 2020

yeraydiazdiaz commented on May 4, 2020

houoop commented on May 4, 2020

valiant1x commented on May 10, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions