Repository navigation
Follow redirects whose Location holds non-ASCII bytes - #95
Open
ihamzatarar wants to merge 1 commit into
Open
ihamzatarar wants to merge 1 commit into
ihamzatarar wants to merge 1 commit into
Conversation
URI.parse rejects a Location field containing raw non-ASCII bytes, so open-uri gave up on the redirect even though browsers percent encode those bytes and follow it. Servers that send them are not following RFC 9110, but they are common enough to matter. Escape the non-ASCII bytes and parse once more when the first attempt fails. A Location that parses today is unaffected, and one that is malformed for any other reason still raises.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #46.
A
Locationfield holding raw non-ASCII bytes cannot be parsed byURI.parse, so the redirect is abandoned:Servers that do this are not following RFC 9110, which requires the field to be a URI reference, but they are common, and browsers percent encode the bytes and follow the redirect rather than fail.
This does the same: if
URI.parserejects the field, the non-ASCII bytes are percent encoded and parsed once more. The first attempt is unchanged, so aLocationthat parses today takes exactly the path it takes now, and one that is malformed for any other reason still raises.test_redirect_invalid, which sendsLocation: ::, still passes.Tests
test_redirect_non_ascii_locationintest/open-uri/test_open-uri.rb, modelled ontest_redirect_relative: the server answers301withLocation: /caf\xC3\xA9, and the test asserts the follow-up request isGET /caf%C3%A9.On master it fails with:
Suite on Ruby 3.4.6, run standalone:
The 30 errors are the same before and after: they need
assert_join_threadsfrom ruby/ruby's test helpers, which is not available when the suite runs outside the Ruby tree.