ASGITransport does not correctly simulate raw_path in the scope

I'm trying to switch Datasette's internal tests over to using httpx with AsyncClient.

This has almost worked perfectly, but I've run into one problem: it looks like the ASGITransport class used by the AsyncClient(app=asgi_app) mechanism does not correctly simulate the raw_path and path keys.

Here's the code in question:

httpx/httpx/_transports/asgi.py

Lines 82 to 97 in 92ca4d0

    
           # ASGI scope. 
        
           scheme, host, port, full_path = url 
        
           path, _, query = full_path.partition(b"?") 
        
           scope = { 
        
               "type": "http", 
        
               "asgi": {"version": "3.0"}, 
        
               "http_version": "1.1", 
        
               "method": method.decode(), 
        
               "headers": [(k.lower(), v) for (k, v) in headers], 
        
               "scheme": scheme.decode("ascii"), 
        
               "path": unquote(path.decode("ascii")), 
        
               "query_string": query, 
        
               "server": (host.decode("ascii"), port), 
        
               "client": self.client, 
        
               "root_path": self.root_path, 
        
           }

As you can see, it's not populating raw_path even though that's part of the ASGI spec.

This matters for Datasette because it supports this URL: https://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv - which refers to a SQLite database table called table/with/slashes.csv (a weird table name but that's test cases for you). The way it does this is through careful decoding of the raw_path ASGI scope variable.

Here are my notes when I first ran into this limitation of ASGITransport: simonw/datasette#1000 (comment)

The fix here may be as straight-forward as adding this line to the code that constructs the scope:

    "raw_path": path,

Since we're here - one related issue.

I'm interested in cases where you'd like to use the client for making the request, but need some information that is lost by the time you've sent an HTTP response. In particular eg. making a request and then verifying that the correct HTML template was used, and contains some particular context.

Have discussed some of this provisionally against the ASGI spec here... django/asgiref#135

Curious if any of that has relevance to any of your Datasette tests?

Add raw_path to scope in ASGITransport (#1357)

Extra debug/test information in ASGI messages is an interesting idea.

I've been working around that using the ASGI equivalent of threadlocals (context vars etc) - I usually avoid threadlocal-equivalents like the plague, but I've come to accept them for debugging/observability style mechanisms where you need to know what happened for informational puproses without having to re-engineer your entire stack to pass tracking values around.

Likewise, with ASGI I've thought about doing this using request IDs (also known as correlation IDs) that are set in an HTTP header and can be used to establish a key/value store that has been populated with data during the request.

These workarounds are working OK for me at the moment, but I'd certainly be keen to try out a less global-variable approach if one was available.

Navigation Menu

Repository navigation

ASGITransport does not correctly simulate raw_path in the scope #1356

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	# ASGI scope.
	scheme, host, port, full_path = url
	path, _, query = full_path.partition(b"?")
	scope = {
	"type": "http",
	"asgi": {"version": "3.0"},
	"http_version": "1.1",
	"method": method.decode(),
	"headers": [(k.lower(), v) for (k, v) in headers],
	"scheme": scheme.decode("ascii"),
	"path": unquote(path.decode("ascii")),
	"query_string": query,
	"server": (host.decode("ascii"), port),
	"client": self.client,
	"root_path": self.root_path,
	}

ASGITransport does not correctly simulate raw_path in the scope #1356

Description

Activity

simonw commented on Oct 9, 2020

lovelydinosaur commented on Oct 9, 2020

simonw commented on Oct 9, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions