Skip to content
This repository was archived by the owner on Nov 6, 2023. It is now read-only.
This repository was archived by the owner on Nov 6, 2023. It is now read-only.

Test CA repository does not reflect Firefox state #6569

Closed

Description

@fuglede

PR #6102 gives an example of a host which whose SSL configuration will cause breakage in Firefox (due to an invalid chain) but which passes the fetch test.

cc @Hainish.

@jeremyn, @J0WI: You may want to be aware of this when reviewing; I just came across a second false positive.

Activity

  1. J0WI commented on Aug 23, 2016

    @J0WI
    Contributor

    When you visited a site with the same cert chain before, the browser caches the chain and doesn't show any errors.
    As far as I know we integrated some cert chains in our test infrastructure that tests don't fail. We may reconsider this.

  2. J0WI commented on Aug 23, 2016

    @J0WI
    Contributor

    They are located in "test/rules/platform_certs/", but they might only be used on rules with platform CAcert or incomplete (do we have a platform for that?) specified.

  3. jeremyn commented on Aug 23, 2016

    @jeremyn
    Contributor

    This is a useful link for anyone trying to understand this problem: https://wiki.mozilla.org/CA:FAQ .

  4. fuglede commented on Aug 23, 2016

    @fuglede
    ContributorAuthor

    @J0WI: The ones in test/rules/platform_certs/default are exactly the ones that fetch-test.sh and friends use, so those are the ones that would appear to be out of sync.

  5. jeremyn commented on Aug 23, 2016

    @jeremyn
    Contributor

    Considering @J0WI 's "We may reconsider this.":

    My rule of thumb has been that a rewrite works if the https and http version of a page do the same thing: return the same content, have the same 4xx error, whatever. If a site is misconfigured to not correctly fetch an intermediate certificate, the two versions don't do the same thing and the rule is bad. So by this rule of thumb, I don't think we should include prefetched intermediate certificates in our tests.

  6. Hainish commented on Aug 23, 2016

    @Hainish
    Member

    Looking into this now

  7. Hainish commented on Sep 1, 2016

    @Hainish
    Member

    I wonder how this list of intermediates was generated in the first place.

    The Mozilla CA Certificate Store lists this source file with all of the included root certificates. We could possibly parse this file and generate a new test/rules/platform_certs/defaults that is more current.

    @jsha, do you know how this was generated in the first place?

  8. fuglede commented on Sep 1, 2016

    @fuglede
    ContributorAuthor

    Pinging @hiviah as he added them in the first place. Having an automated fetch-and-parser sounds like it would be great.

  9. jsha commented on Sep 6, 2016

    @jsha
    Member

    There are two questions:

    • How do we add new roots when they are added to Mozilla's root program? (we don't, or do it ad-hoc)
    • How do we add new intermediates?

    We include intermediates because some sites fail to include those intermediates, which will break curl, but not necessarily Firefox (since Firefox has different certificate caching logic). I'm not sure how @hiviah generated the original set, but for similar work in the SSL Observatory we've, in the past, generated the transitive closure of all intermediates from trusted roots. To keep the list up-to-date, I've occasionally used the ad-hoc script in test/rules/platform_certs/get-issuer.sh.

    Perhaps a better long-term solution would be to use the public CT logs. Not guaranteed to be comprehensive, but given that Google pushes certs from their crawls into those logs, most likely all the intermediates we care about would show up there.

  10. fuglede commented on Sep 6, 2016

    @fuglede
    ContributorAuthor

    We include intermediates because some sites fail to include those intermediates, which will break curl, but not necessarily Firefox (since Firefox has different certificate caching logic). I'm not sure how @hiviah generated the original set, but for similar work in the SSL Observatory we've, in the past, generated the transitive closure of all intermediates from trusted roots. To keep the list up-to-date, I've occasionally used the ad-hoc script in test/rules/platform_certs/get-issuer.sh.

    I'm actually a bit surprised by this part, as I've typically been able to use curl to catch faulty Firefox behavior. Do we have an example of a host which will break curl but not Firefox?

  11. jsha commented on Sep 6, 2016

    @jsha
    Member

    I don't have one off-hand, but the case where it happens is when the site doesn't provide an intermediate and the Firefox user has previously visited another site that did provide the same intermediate, so the intermediate is cached by Firefox.

    You could make an argument against including such sites, because they will break when visited in a completely fresh FF profile, but so far we have included them.

  12. fuglede commented on Sep 6, 2016

    @fuglede
    ContributorAuthor

    Ah, right, then we're on the same note. But actually, I've avoided including such rulesets exactly because they potentially cause issues; in fact #3107 was more or less motivated by this very phenomenon.

  13. Hainish commented on Sep 6, 2016

    @Hainish
    Member

    I'd also air on the side of not including hosts which do not provide their intermediates, since this would cause us to force users with a fresh FF profile on to a site they can't access. If these sites are submitted in a PR and we spot this, it may also be worth it to try to reach out to the site maintainers to notify them of the problem.

  14. hiviah commented on Sep 6, 2016

    @hiviah

    I generated the original certificate sets from Firefox exported roots and generating transitive closure - with this script.

    The intermediates were taken from my own observatories (Perspectives notary fork storing complete chains - notary1.qabs.cz/notary2.qabs.cz, CA certs have separate table which makes export easy)

    Using CT would certainly work as well, but you'd have to sieve the certs for intermediate certs.

  15. jsha commented on Sep 6, 2016

    @jsha
    Member

    Also worth noting: Often Chrome will successfully load such misconfigured sites due to something called "AIA chasing:" i.e., the leaf certificate includes metadata telling you where to fetch its parent. Firefox doesn't implement AIA chasing. So that brings additional pressure to include such sites. All that said, I agree that it makes sense to leave them out in general.

  16. J0WI commented on Sep 6, 2016

    @J0WI
    Contributor

    I think we should not check with intermediates, except with a platform specified (#6697 (comment)). This would also allow us to not default_off all the affected rules.

    To catch all the intermediates you may have a look at https://crt.sh/mozilla-disclosures

  17. added a commit that references this issue on Sep 7, 2016
    aa90f4f
  18. Hainish commented on Sep 7, 2016

    @Hainish
    Member

    PR #6790 updates the certificate store to reflect the latest Firefox export of roots, as of 48.0.2, and removes all intermediates.

    This is still a work in progress, which requires more testing.

    Testing with #6789, a PR which passes the fetch test currently on travis, I see a number of new failures:

    ERROR src/chrome/content/rules/Zalando.xml: Fetch error: http://help.zalando.at/ => https://help.zalando.at/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/Zalando.xml: Fetch error: http://help.zalando.be/ => https://help.zalando.be/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/Zalando.xml: Fetch error: http://help.zalando.ch/ => https://help.zalando.ch/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/Zalando.xml: Fetch error: http://help.zalando.de/ => https://help.zalando.de/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/Zalando.xml: Fetch error: http://help.zalando.dk/ => https://help.zalando.dk/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/Zalando.xml: Fetch error: http://help.zalando.es/ => https://help.zalando.es/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/Zalando.xml: Fetch error: http://help.zalando.fi/ => https://help.zalando.fi/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/Zalando.xml: Fetch error: http://help.zalando.fr/ => https://help.zalando.fr/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/Zalando.xml: Fetch error: http://help.zalando.it/ => https://help.zalando.it/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/Zalando.xml: Fetch error: http://help.zalando.nl/ => https://help.zalando.nl/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/Zalando.xml: Fetch error: http://help.zalando.no/ => https://help.zalando.no/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/Zalando.xml: Fetch error: http://help.zalando.pl/ => https://help.zalando.pl/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/Zalando.xml: Fetch error: http://help.zalando.se/ => https://help.zalando.se/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/Zalando.xml: Fetch error: http://help.zalando.co.uk/ => https://help.zalando.co.uk/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    

    And indeed, these domains do give a certificate error when accessed from a fresh profile on Firefox.

  19. Hainish commented on Sep 7, 2016

    @Hainish
    Member

    For #6102 with #6790 applied, I get the following, which seems to correspond to what @fuglede saw in the browser for this issue:

    ERROR src/chrome/content/rules/octopuspop.com.xml: Fetch error: http://admin.octopuspop.com/ => https://admin.octopuspop.com/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/octopuspop.com.xml: Fetch error: http://ap.octopuspop.com/ => https://ap.octopuspop.com/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/octopuspop.com.xml: Fetch error: http://octopuspop.com/ => https://octopuspop.com/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    ERROR src/chrome/content/rules/octopuspop.com.xml: Fetch error: http://www.octopuspop.com/ => https://www.octopuspop.com/: (60, 'SSL certificate problem: unable to get local issuer certificate')
    
  20. Hainish commented on Sep 7, 2016

    @Hainish
    Member

    Is there an example of a ruleset which failed tests due to its CA not yet being in our store of certs, but may pass with the newest Firefox store of CAs?

  21. added a commit that references this issue on Sep 7, 2016
    82a57c3
  22. added a commit that references this issue on Sep 9, 2016
    66f0021
  23. fuglede commented on Sep 11, 2016

    @fuglede
    ContributorAuthor

    Is there an example of a ruleset which failed tests due to its CA not yet being in our store of certs, but may pass with the newest Firefox store of CAs?

    Hard to say for sure, but I do not recall ever seeing such an example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions