Skip to content
This repository was archived by the owner on Nov 6, 2023. It is now read-only.
This repository was archived by the owner on Nov 6, 2023. It is now read-only.

How can I find out a Git commit corresponding to the out-of-band ruleset update? #17641

Closed

Description

@pipboy96

E.g. how can I know which state of the tree the update is based on? I would like to create a script that automatically picks up new updates and tries to reproduce them from corresponding Git commits, and if they do not match, notify me so I can contact someone at EFF.

Activity

  1. pipboy96 commented on Mar 29, 2019

    @pipboy96
    ContributorAuthor

    Pinging @Hainish as a developer of updater and @zoracon in case she knows.

  2. added
    EFFSomeone at EFF needs to look into
    on Mar 31, 2019
  3. zoracon commented on Apr 9, 2019

    @zoracon
    Contributor

    Our out of band updates are done in a separate private repo. There would be no git commit in this repo that would have a corresponding git commit #.

  4. pipboy96 commented on Apr 9, 2019

    @pipboy96
    ContributorAuthor

    @zoracon I want to build a server/serverless task that tries to reproduce the updates and if they don't match, alert the owner and show the exact diff between expected and real update file.

  5. pipboy96 commented on Apr 9, 2019

    @pipboy96
    ContributorAuthor

    tbh I always hate when things are done in a non-transparent way, but I can understand why EFF maintains its own closed mirror of the repo.

  6. zoracon commented on Apr 9, 2019

    @zoracon
    Contributor

    tbh I always hate when things are done in a non-transparent way, but I can understand why EFF maintains its own closed mirror of the repo.

    It's not a mirror of the repo, but rather a separate repo for ruleset releases. This was previous work done to enable frequent ruleset releases without doing a full extension update. This was heavily discussed in the open with #15096 and we now do releases for rulesets from this work.

  7. pipboy96 commented on Apr 9, 2019

    @pipboy96
    ContributorAuthor

    @zoracon Does this repo only contain rulesets? Moving rulesets into a separate repo on Github too would be amazing.

  8. Hainish commented on Apr 9, 2019

    @Hainish
    Member

    @pipboy96 yes that repo only contains the rulesets and ruleset signatures. It's basically a repo that stores the results of running the async-request.sh and async-airgap.sh scripts.

  9. pipboy96 commented on Apr 9, 2019

    @pipboy96
    ContributorAuthor

    @Hainish I wonder what's the point of air gap if the hash to be signed is created on an online machine? I think this was asked here before. This is also the main reason why I want to make software that will try to independently reproduce these files.

  10. Hainish commented on Apr 9, 2019

    @Hainish
    Member

    @pipboy96 for the same reason we use an airgap for the regular release process - it means that malware or a hacker with remote access on our machines is not able to make a release in an automated fashion. Ideally we would verify that the build results in the same hash across multiple machines before signing it. However, even without that process in place using an airgap helps mitigate against attacks. Some malware is unable to persist after a reboot (this is particularly true in Qubes). Some malware will be detected by the developer. Requiring explicit human interaction to make a release via the airgap gives us a critical opportunity to notice something is awry, and rectify the situation before something goes wrong. Will it prevent an advanced attack in every case? No. But it adds a layer of protection which disallows automated malicious releases from being made.

  11. pipboy96 commented on Apr 9, 2019

    @pipboy96
    ContributorAuthor

    @Hainish To be honest, I would have simply used a YubiKey, since it would provide the same security guarantees. I would probably keep a backup copy of the key in case it breaks though.

  12. Hainish commented on Apr 10, 2019

    @Hainish
    Member

    @pipboy96 last time I checked Chrome did not support using a GPG smartcard for signing extensions. Also, for the rulesets we're using raw RSA signatures, since that signature type is supported by WebCrypto in the browser without the need for external libraries.

  13. pipboy96 commented on Apr 10, 2019

    @pipboy96
    ContributorAuthor

    The second case can be solved using PKCS#11.

  14. zoracon commented on Apr 10, 2019

    @zoracon
    Contributor

    The second case can be solved using PKCS#11.

    Do you mean the PKCS#11 API? If so I only saw that available in Firefox.

  15. pipboy96 commented on Apr 10, 2019

    @pipboy96
    ContributorAuthor

    @zoracon no, not in the browser, but through the use of OpenSSL Engine API.

  16. pipboy96 commented on Apr 10, 2019

    @pipboy96
    ContributorAuthor

    I would probably not use a regular YubiKey, but would use YubiHSM2 instead, since it supports Ed25519, which we should probably migrate to at some point of time.

  17. zoracon commented on Apr 10, 2019

    @zoracon
    Contributor

    @pipboy96 while using a YubiKey for signing extensions would potentially streamline processes on Chrome releases, I hope the above explanations solidifies as to why there can't be external automated probes on the release process for rulesets. As of right now, we have quite a few checkpoints if files, hashes, etc. do not match up properly. For the same reasons security comes into question are the same reasons we can't have automated detection from outside scripts.

  18. pipboy96 commented on Apr 10, 2019

    @pipboy96
    ContributorAuthor

    @zoracon I believe source files should be available for each update, be it ruleset or extension. I also believe GPL requires that.

  19. pipboy96 commented on Apr 10, 2019

    @pipboy96
    ContributorAuthor

    @zoracon The second case I mentioned is signing rulesets themselves using OpenSSL CLI, not Chrome updates.

  20. zoracon commented on Apr 10, 2019

    @zoracon
    Contributor

    @pipboy96 We have rulesets hosted and available on the https://atlas.eff.org for viewing exact XML files involved in a release.

  21. pipboy96 commented on Apr 10, 2019

    @pipboy96
    ContributorAuthor

    Those are for extension releases though, not for out-of-band updates.

  22. zoracon commented on Apr 10, 2019

    @zoracon
    Contributor

    @pipboy96 Sorry not atlas but https://www.https-rulesets.org (and it's JSON, not XML there).

  23. pipboy96 commented on Apr 10, 2019

    @pipboy96
    ContributorAuthor

    Well, I am looking for a way to replicate this JSON (sans timestamps, compression and signatures of course) from the public GitHub repository state.

  24. cschanaj commented on Apr 10, 2019

    @cschanaj
    Collaborator

    I agree with @pipboy96 that without knowing the exact commit (for the out-of-band rulesets update) made it difficult to reproduce the releases. This sometime makes it more difficult to debug some site breakage.

    IMHO, it would be better to have more reproducibility on the OOB releases, e.g. by including a reference to the latest commit when a release is made. It should be fairly trivial to include git rev-parse HEAD in the releases without breaking backward compatibility.

    In this way, security researchers/ interested parties can extract the gzipped releases and compare with their own build in case they have some concerns.

    echo "{ \"timestamp\": $TIMESTAMP, \"rulesets\":" "`cat $RULESETS_FILE`" "}" | tr -d '\n' | gzip -nc > $2/default.rulesets.$TIMESTAMP.gz

  25. Hainish commented on Apr 10, 2019

    @Hainish
    Member

    I don't think it should be a problem to publish which commit from HTTPS Everywhere we're deriving the out-of-band rulesets from. We usually just do it from HEAD at the point of OOB ruleset publication, but I can see how that isn't necessarily helpful and the exact HTTPSE commit would allow you to debug more efficiently.

    Relatedly, currently the Atlas tracks the latest bundled rulesets, rather than ruleset releases when, for instance, in https://atlas.eff.org/domains/0xbadc0de.be.html it says Current Release. Perhaps it would be better to track the latest ruleset releases in the Atlas instead.

  26. pipboy96 commented on Apr 11, 2019

    @pipboy96
    ContributorAuthor

    @Hainish Maybe make another branch for latest rulesets file?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

EFFSomeone at EFF needs to look into

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions