Host header always overwritten in HTTP/2

I have a use-case where I want to use an http proxy and resolve the IP address of the target on my side of the proxy.

url = "https://1.2.3.4/"
client.headers.update({"Host": "domain.com"})
client.get(url, verify=False)

However in HTTP/2 mode, the host header is always removed. See

httpcore/httpcore/_sync/http2.py

Line 327 in 84c7738

] + [(k, v) for k, v in headers if k not in (b"host", b"transfer-encoding")]

It works as expected in HTTP/1.1

Right, yup. We drop the host header, because HTTP/2 doesn't use it. Instead it has the :authority pseudo header...

https://tools.ietf.org/html/rfc7540

Clients that generate HTTP/2 requests directly SHOULD use the ":authority" pseudo-header field instead of the Host header field.

However, we probably ought to give precedence to the value of headers["Host"] when setting the :authority pseudo header. Eg...

httpcore/httpcore/_sync/http2.py

Lines 317 to 320 in 84c7738

    
           if port is None or port == default_port: 
        
               authority = hostname 
        
           else: 
        
               authority = b"%s:%d" % (hostname, port)

Should instead perhaps be, this?...

if port is None or port == default_port:
    authority = hostname
else:
    authority = b"%s:%d" % (hostname, port)

for k, v in headers:
    if k == b"host":
        authority = v

That would be great. That would make it possible to always use Host and not worry about 1.1 or 2.

Btw I tried setting the ":authority" header as well, but ended up with two of them.

Do you need me to submit a pull request?

Do you need me to submit a pull request?

It'd be great if you could, yup.

I submitted a pull request, but wanted some input before fixing for async as well.

My problem with this is that Host is always set, so it will always override the hostname. I don't know if that is intended.

@chripede

@chripede I just saw your comment here:

My problem with this is that Host is always set, so it will always override the hostname. I don't know if that is intended.

Do you mean that HTTPX always passes a Host header, and so with the changes from #241 we will actually end up always using that instead of the URL hostname? I think that's actually beneficial, since I see Host and :authority as closer in goal than the URL hostname and :authority. But reviewing the definition of :authority might help…

Well, it seems Host, :authority and hostname+port all basically contain the same information:

Host: https://tools.ietf.org/html/rfc7230#section-5.4

The "Host" header field in a request provides the host and port
information from the target URI, enabling the origin server to
distinguish among resources while servicing requests for multiple
host names on a single IP address.

:authority: https://tools.ietf.org/html/rfc7540#section-8.1.2.3

The ":authority" pseudo-header field includes the authority portion of the target URI ([RFC3986], Section 3.2).

…So, it looks to me like the way you got it in #241 is correct: if Host is present we can use it as :authority, based on the assumption that it's meant to represent the same information than hostname+port. In practice for HTTPX we'll always go this route, but if you were to use HTTPcore itself then you probably wouldn't, so that's not a problem imo.

@florimondmanca

@florimondmanca Yes maybe I should have mentioned that this was via HTTPX.

I've updated my pull request but are facing some linting problems. Moving the conversation to the pull request

I've merged the PR (#241 (comment)).

But it's reasonable to add new tests to check newly added functionality

Closing this since a fix was merged, but I filed #250 to track adding a test case. :-)

Allow Host to override :authority in HTTP2 (encode#241)

Navigation Menu

Repository navigation

Host header always overwritten in HTTP/2 #240

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if port is None or port == default_port:
	authority = hostname
	else:
	authority = b"%s:%d" % (hostname, port)

Host header always overwritten in HTTP/2 #240

Description

Activity

lovelydinosaur commented on Nov 25, 2020

chripede commented on Nov 25, 2020

lovelydinosaur commented on Nov 25, 2020

chripede commented on Nov 25, 2020

florimondmanca commented on Nov 26, 2020

florimondmanca commented on Nov 26, 2020

chripede commented on Nov 26, 2020

cdeler commented on Nov 27, 2020

florimondmanca commented on Dec 7, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions