Skip to content
This repository was archived by the owner on Nov 6, 2023. It is now read-only.
This repository was archived by the owner on Nov 6, 2023. It is now read-only.

Evaluate ssl-yeah as a testing tool #1192

Closed

Description

@jsha

In #1015, @guiweber links to a tool they created, https://github.com/guiweber/ssl-yeah. The goal of this tool is to automate creation of rulesets.

@guiweber, based on your latest comments it sounds like SSL-Yeah is pretty sophisticated at detecting mixed content and crawling multiple pages on a site, which are two things the https-everywhere-checker currently lacks. Can you modify it to do crawling and mixed content detection on existing rulesets, while performing rewrites? That would be hugely useful.

To answer your question about automatically generating test URLs: Yes, the https-everywhere-checker takes any non-wildcard target host tag as an implicit test URL. However, to get good test coverage we ideally want multiple URLs per hostname. For instance, on a news site we'd like to have the root page (implicit), plus a feature article page, plus a photo gallery page, plus a video page, etc. As a secondary goal, if your tool can generate a sampling of test URLs in addition to the implicit ones, that would be helpful.

Activity

  1. guiweber commented on Mar 15, 2015

    @guiweber
    Contributor

    Yes, this would definitely be possible, although I'll be finishing the rule writing functionality and do some polishing work before getting started on this.

    So just to make sure I understand, here are the basic steps it would need to do:

    1. Read ruleset file
    2. Perform rewrite for each host and load it, detecting mixed content
    3. Load a few more pages for each host, while performing the rewrites and detecting mixed content

    What kind of reporting/messaging would be needed? How would you be invoking the script? Do you want to test all existing rules or just new rules are they are submitted?

    I already plan on adding the creation of a varied sample of test URLs (with images, video, article) in the rule-writing functionality, but doing this while testing has limitations. You would need to either use the first pages crawled no matter what they contain, or to load more pages to get some variety, which would increase the testing time (I suggest keeping the rate at which pages are loaded at 1 per second, as some hosts don't play well if requesting pages faster).

  2. jsha commented on Mar 16, 2015

    @jsha
    MemberAuthor

    Yep, you have the right idea with your list of actions.

    The output should include, for each ruleset, which pages failed and why, similar to the 'Disabled because' reasons the https-everywhere-checker currently puts into failing rulesets.

    I would want to run it the same way I currently run the https-everywhere-checker: Occasionally running it manually for all rules. Also, for both https-everywhere-checker, and potentially ssl-yeah, I want to run it on two other occasions:

    • Automatically, once per day, for all rulesets
    • Automatically, on any pull request, for any rulesets that changed in that pull request.

    Neither of these is implemented yet but I plan to.

    For mixed content you should have some heuristics, e.g.:

    • If 100% of CSS resources or 100% of JS resources on a page are mixed content, fail.
      • If some JS or some CSS fails, but not 100%, it may be JS ad tracking requests or auxiliary stylesheets, so we'll assume they're fine.
    • If any object resource on a page is mixed content, fail.
    • Otherwise pass.

    Probably we want to ignore iframe mixed content as well, because those are likely to be from ad trackers as well. But we might change that depending on what testing shows in practice.

    Does it make sense to use PhantomJS? It might do a better job of trying to render a page.

  3. guiweber commented on Mar 23, 2015

    @guiweber
    Contributor

    Sounds good. I will get started with this and will let you know if I need more details or have other questions along the way.

    I was planning on implementing some sort of headless browser at some point in the future, as a way of detecting MCB triggered by XMLHttpRequests. It looks like Phantom JS would do perfectly, but I'll have to give it a try. There seem to be good libraries that allow controlling it from node.js, so I'm hopeful it won't be too complicated.

  4. jsha commented on Mar 23, 2015

    @jsha
    MemberAuthor

    The other possibility, BTW, is to just run Firefox headless using Xvfb, and have a driver extension that loads URLs, crawls, and watches for MCB. There's already some ~2 year old code that does this inside the HTTPS Everywhere extension, called the "ruleset tests." You could pull that out and revitalize that. The advantage would be that you'd be running against the actual extension code, with all its quirks and bug fixes, so you'd minimize false positives and false negatives.

  5. pipboy96 commented on May 9, 2019

    @pipboy96
    Contributor

    Project seems to no longer be developed, last update in 2016. Last update of issue in 2015. If this should be re-opened, tell me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions