Repository navigation
Path encoding broken for non-ASCII in WSGI mode #1365
Description
Describe the bug
When using httpx to call into a WSGI application (my preferred way to test Flask app nowadays), the web view receives urlencoded paths.
I'm not sure where this really belongs to (is it a bug in httpx or Flask!?), but I've been working around it for months, so I guess I'll make myself unpopular and try it here first. If it turns out to be Flask's fault, I'l headl over and bother the Pallets folks.
To reproduce
from flask import Flask, request
import httpx
app = Flask(__name__)
@app.route('/<path>')
def hello_world(path):
return f"path: { path }, query: { request.args['a'] }, url: { request.url }"
if __name__ == "__main__":
with httpx.Client(app=app, base_url="http://testserver") as client:
resp = client.get("/ä", params={"a": "ä"})
print("httpx", resp.text)
with app.test_client() as client:
resp = client.get("/ä?a=%C3%A4")
print("flask", resp.get_data().decode("utf-8"))Expected behavior
httpx path: ä, query: ä, url: http://testserver/ä?a=ä
flask path: ä, query: ä, url: http://localhost/ä?a=ä
Actual behavior
httpx path: %C3%A4, query: ä, url: http://testserver/%25C3%25A4?a=ä
flask path: ä, query: ä, url: http://localhost/ä?a=ä
NB
- it seems to handle query parameters just fine.
%25is the ASCII code of the percent sign
Environment
- OS: macOS
- Python version: 3.8.6
- HTTPX version: 0.16.1
- Async environment: n/a
- HTTP proxy: n/a
- Custom certificates: no
Activity
[-]Path encoding in WSGI mode.[/-][+]Path encoding broken for non-ASCII in WSGI mode[/+]JayH5 commentedon Oct 21, 2020
This does look like a bug in httpx. Although I can't see where in PEP-3333 it's specified that paths should be unquoted by WSGI servers, it does seem like a thing that is widely done. You can see some discussion of a sorta related issue to this here: pallets/flask#900
There are probably also some areas in the WSGI integration where we should be using latin1 encoding instead of ascii.
lovelydinosaur commentedon Nov 12, 2020
So the implication of https://www.python.org/dev/peps/pep-3333/#url-reconstruction is that
PATH_INFOshould be unquoted.Which, yeah, we're not doing...
httpx/httpx/_transports/wsgi.py
Line 86 in f8f5430
Although we are doing so for ASGI...
httpx/httpx/_transports/asgi.py
Line 92 in f8f5430