Repository navigation
Never unpickle binary responses unless explicitly asked (binary_loads= / binary_dumps= seams) - #20
Merged
Merged
Conversation
A binary (octet-stream) response used to be unpickled, which lets a spoofed or compromised server run code on the client. It now raises BinaryResponseNotDecoded unless a decoder is given: - handle_binary_resp(func, loads=...); - mk_request_function / HttpClient / mk_request_func_from_openapi_spec / openapi_to_py(..., binary_loads=...): bytes for the raw content, a decoder of your own, or the new http2py.decorators.unsafe_pickle_loads for a fully trusted server (loads=pickle.loads also works but warns). Binary request bodies get a binary_dumps= seam on the same functions; the default stays pickle.dumps (safe for the client, and the server has to opt into unpickling it). Closes #19 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…caller, Py2Request accepts a partial Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nt/ju#6) tests/test_dependency_workarounds.py was the tripwire for this: it failed once ju stopped importing dill, and asked for this removal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #19. Client-side counterpart of i2mint/py2http#20.
What changes
application/octet-stream) responses are no longer unpickled. Without a decoder, a binary response raiseshttp2py.decorators.BinaryResponseNotDecoded(aValueError), whose message names the options. Unpickling a response lets a spoofed or compromised server, or anyone on a plain-http path, run code on the client.binary_loads=(keyword-only) onmk_request_function,HttpClient,mk_request_func_from_openapi_spec,openapi_to_py, andloads=onhandle_binary_resp. It accepts:bytesfor the raw content (e.g. images);bytes -> objectdecoder;unsafe_pickle_loads(new, inhttp2py.decorators) for a fully trusted server.loads=pickle.loadsstill works but warns, mirroring py2http'shandle_binary_req. A non-callableloadsgives aTypeErrorat construction.binary_dumps=on the same functions, for binary request bodies. The default stayspickle.dumps. Pickling the client's own arguments is no risk to the client, and the server already has to opt into unpickling them (loads=unsafe_pickle_loadson py2http's side). Raising instead would only break clients of such opted-in servers, with no security gain. Passbinary_dumps=to match a server that uses another decoder.default_configs.default_binary_output_transis kept for importers. It now raises on binary responses as well.Opt-in for the old behaviour
HttpClient(spec, binary_loads=unsafe_pickle_loads)(or the same keyword on the other factories). Only when the server is trusted.Tests
tests/test_binary_payloads.py(8 tests). It uses a fake dispatch, so no network is involved. It goes in the roottests/becausehttp2py/testsis excluded from CI collection. The tests cover:bytesandunsafe_pickle_loadsboth work;pickle.loadswarns, and a non-callableloadsis refused;binary_dumpsis used, and the default is still pickle;HttpClientpassesbinary_loadsthrough.Full suite with doctests: all pass (the dill tripwire is removed, see below).
Also: dill workaround removed
CI was red on
tests/test_dependency_workarounds.py::test_dill_workaround_is_still_needed. That failure happens on master too. The test is a deliberate tripwire: it fails once the releasedjustops importingdill(i2mint/ju#6), and it asks for exactly this fix. So this PR dropsdillfrom[project].dependenciesand deletes that test module, in a separate commit.Adversarial review
Fresh Opus reviewer, two passes.
yaml.load(the CLI usessafe_load). A server that lies about its Content-Type gets nothing either, because the decoder comes from the spec'sresponse_type, not from the header. PositionalHttpClientcallers and importers ofdefault_binary_output_transstill work. Small fixes, all done:binary_loadsis now validated up front inmk_request_functionandHttpClient, so a non-callable is rejected even when the spec has no binary endpoints.pickle.loadswarning now points at the caller's line instead of library code.Py2Requestacceptspartial(mk_request_function, binary_loads=...), which used to crash on string specs.mk_cli/dispatch_cliandapi_pkg_maker's generated module have nobinary_loads=passthrough. Binary endpoints raise there, which is safe but loses capability.Dependents gate (
priv test-dependents http2py --transitive): http2py and extrude pass, and the reviewer found no dependent relying on unpickled responses (meshed, extrude and the plunk demos use JSON services). py2http's own suite passes against this branch (74 passed). Its gate entry failed at collection because pytest imported the repo-rootdeploy.py, which runs a build and atwine uploadat import time. Nothing ran, because the machine has nopythonon PATH. That is unrelated to this change and is filed as i2mint/py2http#26.🤖 Generated with Claude Code