Skip to content
This repository was archived by the owner on Nov 6, 2023. It is now read-only.
This repository was archived by the owner on Nov 6, 2023. It is now read-only.

Target gets different HTTP code in test depending on whether another specific target is present #7711

Closed

Description

@jeremyn

When fetch-test.sh runs against a ruleset with only target forum, forum gets a 200. If I add a second target www either before or after forum, forum gets a 502. If instead I add target ^, forum gets a 200.

In other words, one target passes the test or doesn't based on whether another specific target is in the ruleset. This happens consistently. I have no explanation. This comes from pull request #5390.

good.xml:

<ruleset name="glavbukh.ru">
	<target host="forum.glavbukh.ru" />

	<rule from="^http:" to="https:" />
</ruleset>

fetch-test.sh against good.xml:

INFO Finished comparing http://forum.glavbukh.ru/ -> https://forum.glavbukh.ru/. Rulefile: good.xml.
INFO Finished in 4.10 seconds. Loaded rulesets: 1, URL pairs: 1.

also_good.xml:

<ruleset name="glavbukh.ru">
	<target host="glavbukh.ru" />
	<target host="forum.glavbukh.ru" />

	<rule from="^http:" to="https:" />
</ruleset>

fetch-test.sh against also_good.xml:

INFO Finished comparing http://glavbukh.ru/ -> https://glavbukh.ru/. Rulefile: also_good.xml.
INFO Finished comparing http://forum.glavbukh.ru/ -> https://forum.glavbukh.ru/. Rulefile: also_good.xml.
INFO Finished in 9.55 seconds. Loaded rulesets: 1, URL pairs: 2.

bad.xml:

<ruleset name="glavbukh.ru">
	<target host="www.glavbukh.ru" />
	<target host="forum.glavbukh.ru" />

	<rule from="^http:" to="https:" />
</ruleset>

fetch-test.sh against bad.xml:

INFO Finished comparing http://www.glavbukh.ru/ -> https://www.glavbukh.ru/. Rulefile: bad.xml.
INFO Finished comparing http://forum.glavbukh.ru/ -> https://forum.glavbukh.ru/. Rulefile: bad.xml.
ERROR bad.xml: Non-2xx HTTP code: http://forum.glavbukh.ru/ (200) => https://forum.glavbukh.ru/ (502)
INFO Finished in 7.75 seconds. Loaded rulesets: 1, URL pairs: 2.

also_bad.xml:

<ruleset name="glavbukh.ru">
	<target host="forum.glavbukh.ru" />
	<target host="www.glavbukh.ru" />

	<rule from="^http:" to="https:" />
</ruleset>

fetch-test.sh against also_bad.xml:

INFO Finished comparing http://forum.glavbukh.ru/ -> https://forum.glavbukh.ru/. Rulefile: also_bad.xml.
INFO Finished comparing http://www.glavbukh.ru/ -> https://www.glavbukh.ru/. Rulefile: also_bad.xml.
ERROR also_bad.xml: Non-2xx HTTP code: http://forum.glavbukh.ru/ (200) => https://forum.glavbukh.ru/ (502)
INFO Finished in 7.84 seconds. Loaded rulesets: 1, URL pairs: 2.

Activity

  1. jeremyn commented on Nov 29, 2016

    @jeremyn
    ContributorAuthor
  2. Hainish commented on Nov 29, 2016

    @Hainish
    Member

    @jeremyn looking into this now

  3. self-assigned this
    on Nov 29, 2016
  4. Hainish commented on Nov 30, 2016

    @Hainish
    Member

    If we set the loglevel in test/rules/http.checker.config to debug, we see the following with good.xml:

    (.venv) legind@EFF-420 ~/workspace/https-everywhere (master) $ python test/rules/src/https_everywhere_checker/check_rules.py test/rules/http.checker.config src/chrome/content/rules/good.xml                      
    DEBUG Loaded certificate platforms: default,cacert,chromium,firefox
    DEBUG Parsing src/chrome/content/rules/good.xml
    DEBUG =**= Start http://forum.glavbukh.ru/ => https://forum.glavbukh.ru/ ****
    DEBUG Fetching transformed page https://forum.glavbukh.ru/
    DEBUG Following redirect https://forum.glavbukh.ru/ => http://www.glavbukh.ru/forum
    DEBUG Fetching plain page http://forum.glavbukh.ru/
    DEBUG Following redirect http://forum.glavbukh.ru/ => http://www.glavbukh.ru/forum
    DEBUG ==== D: 0.0000; http://forum.glavbukh.ru/ (95274) -> https://forum.glavbukh.ru/ (101880) =====
    INFO Finished comparing http://forum.glavbukh.ru/ -> https://forum.glavbukh.ru/. Rulefile: src/chrome/content/rules/good.xml.
    INFO Finished in 3.19 seconds. Loaded rulesets: 1, URL pairs: 1.

    And with bad.xml:

    DEBUG Loaded certificate platforms: default,cacert,chromium,firefox
    DEBUG Parsing src/chrome/content/rules/bad.xml
    DEBUG =**= Start http://www.glavbukh.ru/ => https://www.glavbukh.ru/ ****
    DEBUG Fetching transformed page https://www.glavbukh.ru/
    DEBUG Fetching plain page http://www.glavbukh.ru/
    DEBUG ==== D: 0.0000; http://www.glavbukh.ru/ (140638) -> https://www.glavbukh.ru/ (140519) =====
    INFO Finished comparing http://www.glavbukh.ru/ -> https://www.glavbukh.ru/. Rulefile: src/chrome/content/rules/bad.xml.
    DEBUG =**= Start http://forum.glavbukh.ru/ => https://forum.glavbukh.ru/ ****
    DEBUG Fetching transformed page https://forum.glavbukh.ru/
    DEBUG Following redirect https://forum.glavbukh.ru/ => http://www.glavbukh.ru/forum
    DEBUG Redirect rewritten: http://www.glavbukh.ru/forum => https://www.glavbukh.ru/forum
    DEBUG Fetching plain page http://forum.glavbukh.ru/
    DEBUG Following redirect http://forum.glavbukh.ru/ => http://www.glavbukh.ru/forum
    DEBUG Non-2xx HTTP code: http://forum.glavbukh.ru/ (200) => https://forum.glavbukh.ru/ (502)
    INFO Finished comparing http://forum.glavbukh.ru/ -> https://forum.glavbukh.ru/. Rulefile: src/chrome/content/rules/bad.xml.
    ERROR src/chrome/content/rules/bad.xml: Non-2xx HTTP code: http://forum.glavbukh.ru/ (200) => https://forum.glavbukh.ru/ (502)

    So here we can see that https://forum.glavbukh.ru is issued a normal redirect to http://www.glavbukh.ru/forum, which exists and returns a 200 response. But because of the other rule, this is in turn redirected via HTTPSE to https://www.glavbukh.ru/forum, which does not exist, and returns a 502 response. Thus the error.

  5. Hainish commented on Nov 30, 2016

    @Hainish
    Member

    What's interesting here is that the non-parity between http://www.glavbukh.ru and https://www.glavbukh.ru is only discovered through the existence of a separate target, because of the fact that it generates a new test url which is transformed by the original target.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions