You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Nov 6, 2023. It is now read-only.
Repository navigation
This repository was archived by the owner on Nov 6, 2023. It is now read-only.
@jeremynsign-rulesets only has full ruleset delivery. I'd love to get a functioning branch for intermediate updates in the near future. I don't see why we couldn't gracefully move to intermediate updates, so I'm leaving that for a later iteration.
By intermediate ruleset delivery I mean that it downloads rulesets, whether all 20k+ or just what's changed, from some EFF URL. In other words "intermediate" meaning "between full extension releases", not "intermediate" meaning "less than the full number of rulesets", in case that wasn't clear.
@Hainish So we're on the same page: my understanding of sign-rulesets is that it downloads a ruleset collection from some URL, say https://eff.org/rulesets.zip , confirms its integrity somehow, and then replaces/updates the existing set of rulesets with what's in the collection.
I'm interested in downloading rulesets.zip manually and taking a look at it, and I asked you for the URL for it, and you said you're going to create one.
Is that right? If so, how have you been testing so far? An internal/local server?
@jeremyn yes, that's all correct (though it's not a zip file). I've been signing the rulesets via utils/sign-rulesets-standalone.sh and serving the generated ruleset files via python -m SimpleHTTPServer, modifying chromium/background-scripts/update_channels.js to the appropriate local values.
@J0WI the Tor Browser transparently proxies any connections made from extensions. The downloads will go over Tor. You're not able to test this with a localhost connection, however. You have to post the rulesets on a publicly accessible endpoint.
@J0WI add-ons have always been able to be updated from within Tor Browser. Can you link me to a specific design document specification where this is blocked?
@J0WI this is for updates to the extension itself, rather than the rulesets. A lot of the motivation for pinning the version comes from a lack of trust in the AMO updates process, see https://hackernoon.com/postmortem-of-the-firefox-and-tor-certificate-pinning-vulnerability-rabbit-hole-bd507c1403b4. Delivery of rulesets out-of-band from the extension itself makes it so that if the Tor Browser pins the version of the extension, as it looks like they might do, users will still have a recent version of the rulesets. The ruleset updates would not be blocked by Tor Browser.
Regarding HPKP, Chrome is planning on deprecating and removing it
I think Tor have different priorities... Also Mozilla itself didn't announced to drop this feature too, but they started to support it on their own services.
I think that the rule should be that the sign-rulesets updates should be at least as secure as however users download the full extension. So if AMO or the Tor Project uses HPKP, then the sign-rulesets updates should too. At least, the burden of argument for using HPKP shouldn't fall on @J0WI or anyone else asking for HPKP.
I've prevented the downgrade attack scenario in fcf9f10:
Protect against downgrade attacks. A sophisticated attacker would
previously be able to switch default.rulesets.gz with an old signed
ruleset file. We now combine the fact that the JavaScript checks that latest-rulesets-timestamp has a greater value than what is currently
stored and, vitally, that it matches the timestamp bundled with the
signed ruleset file.
I've added timestamped filenames with this as well. I'd like to keep our schema simple. The advantage of having a latest-rulesets-timestamp file as the heartbeat is that it's a very small file. With 2M+ weekly chrome users and close to a million daily Firefox users (plus who knows how many Tor Browser users) I'd like to keep the bandwidth strain low when we have no new updates.
Monthly download bandwidth for latest-rulesets-timestamp is about 1GB/month, and for instructions.json it's about 18GB/month. Current AWS S3 download bandwidth pricing is $0.01/GB.
So, latest-rulesets-timestamp saves about 17 cents per month relative to instructions.json, at least with these assumptions. I don't think saving bandwidth is a strong argument for using one approach instead of the other.
It seems to me that downstream projects will not be able to utilize this endpoint if they would like to have deterministic builds/ pinned versions. Not sure if EFF should entertain them or not in this endpoint or another. Besides, IMHO it might also worth to include the ruleset version inside default.rulesets itself.
@cschanaj As far as I know, downstream projects should be able to use and verify these rulesets if they want, or they can disable these automatic updates entirely or by default.
@cschanaj the extension itself will still be reproducible, and the rulesets can either be verified directly using the HTTPS Everywhere signing key, or they can modify the update_channels.js file to provide their own endpoint with their own rulesets (possibly just an audited version of ours) and verify ruleset delivery with their own keys, if they don't trust ours.
Pushing this to tomorrow to finish up a few loose ends (https-rulesets.org home page, ensuring updating does not occur if the extension version > the rulesets version for the "EFF (Full)" update channel.
Georg from Tor Browser has looked through our signing code, and had some feedback which I incorporated into sign-rulesets.
@Hainish Congratulations on getting the sign-rulesets feature out! I've argued against it before, but I want to acknowledge that it has taken a lot of work and planning from you and others to get to this point, and that you and others should feel proud for launching it. Now that the project has committed to it, I wish it well.
Activity
jeremyn commentedon Mar 14, 2018
Where can I download an example of an intermediate ruleset update, if that's active yet?
Hainish commentedon Mar 14, 2018
@jeremyn
sign-rulesetsonly has full ruleset delivery. I'd love to get a functioning branch for intermediate updates in the near future. I don't see why we couldn't gracefully move to intermediate updates, so I'm leaving that for a later iteration.jeremyn commentedon Mar 15, 2018
By intermediate ruleset delivery I mean that it downloads rulesets, whether all 20k+ or just what's changed, from some EFF URL. In other words "intermediate" meaning "between full extension releases", not "intermediate" meaning "less than the full number of rulesets", in case that wasn't clear.
Hainish commentedon Mar 15, 2018
@jeremyn I don't have this available just yet, but I should have the actual URL we'll be using within the extension available in the next few days.
jeremyn commentedon Mar 16, 2018
@Hainish So we're on the same page: my understanding of
sign-rulesetsis that it downloads a ruleset collection from some URL, sayhttps://eff.org/rulesets.zip, confirms its integrity somehow, and then replaces/updates the existing set of rulesets with what's in the collection.I'm interested in downloading
rulesets.zipmanually and taking a look at it, and I asked you for the URL for it, and you said you're going to create one.Is that right? If so, how have you been testing so far? An internal/local server?
Hainish commentedon Mar 16, 2018
@jeremyn yes, that's all correct (though it's not a zip file). I've been signing the rulesets via
utils/sign-rulesets-standalone.shand serving the generated ruleset files viapython -m SimpleHTTPServer, modifyingchromium/background-scripts/update_channels.jsto the appropriate local values.J0WI commentedon Mar 16, 2018
How will this work in the Tor browser? They don't allow any connection like this.
Hainish commentedon Mar 16, 2018
@J0WI the Tor Browser transparently proxies any connections made from extensions. The downloads will go over Tor. You're not able to test this with a
localhostconnection, however. You have to post the rulesets on a publicly accessible endpoint.J0WI commentedon Mar 17, 2018
It's not a technical limitation, it's a design decision by the Tor Project.
Add-On updates, Safe search, telemetry etc. are all disabled.
[-]Release 2018.3.27 forthcoming[/-][+]Release 2018.4.2 forthcoming[/+]Hainish commentedon Mar 19, 2018
@J0WI add-ons have always been able to be updated from within Tor Browser. Can you link me to a specific design document specification where this is blocked?
Hainish commentedon Mar 19, 2018
I've changed the estimated release date to 2018.4.2 to give enough lead-up time to coordinate with our press team at EFF.
J0WI commentedon Mar 19, 2018
@Hainish it's hard to find this again, but I think https://trac.torproject.org/projects/tor/ticket/10394 is the right one.
First attempt to disable updates was here but this has been reverted due a regression.
Hainish commentedon Mar 19, 2018
@J0WI this is for updates to the extension itself, rather than the rulesets. A lot of the motivation for pinning the version comes from a lack of trust in the AMO updates process, see https://hackernoon.com/postmortem-of-the-firefox-and-tor-certificate-pinning-vulnerability-rabbit-hole-bd507c1403b4. Delivery of rulesets out-of-band from the extension itself makes it so that if the Tor Browser pins the version of the extension, as it looks like they might do, users will still have a recent version of the rulesets. The ruleset updates would not be blocked by Tor Browser.
20 remaining items
J0WI commentedon Mar 23, 2018
I think Tor have different priorities... Also Mozilla itself didn't announced to drop this feature too, but they started to support it on their own services.
jeremyn commentedon Mar 23, 2018
I think that the rule should be that the
sign-rulesetsupdates should be at least as secure as however users download the full extension. So if AMO or the Tor Project uses HPKP, then thesign-rulesetsupdates should too. At least, the burden of argument for using HPKP shouldn't fall on @J0WI or anyone else asking for HPKP.So, I agree with @J0WI here.
Hainish commentedon Mar 24, 2018
I've prevented the downgrade attack scenario in fcf9f10:
I've added timestamped filenames with this as well. I'd like to keep our schema simple. The advantage of having a
latest-rulesets-timestampfile as the heartbeat is that it's a very small file. With 2M+ weekly chrome users and close to a million daily Firefox users (plus who knows how many Tor Browser users) I'd like to keep the bandwidth strain low when we have no new updates.jeremyn commentedon Mar 24, 2018
@Hainish I'm not sure if your "small file" comment was against my
instructions.jsonrecommendation, but if so:Assume we have three million daily users and are comparing
latest-rulesets-timestamp(11 bytes):1234567890with
instructions.json(about 200 bytes):{ "timestamp": 1234567890, "url": "https://www.https-rulesets.org/v1/default.rulesets.1234567890.gz", "signature": "https://www.https-rulesets.org/v1/rulesets-signature.1234567890.sha256" }Monthly download bandwidth for
latest-rulesets-timestampis about 1GB/month, and forinstructions.jsonit's about 18GB/month. Current AWS S3 download bandwidth pricing is $0.01/GB.So,
latest-rulesets-timestampsaves about 17 cents per month relative toinstructions.json, at least with these assumptions. I don't think saving bandwidth is a strong argument for using one approach instead of the other.cschanaj commentedon Mar 26, 2018
It seems to me that downstream projects will not be able to utilize this endpoint if they would like to have deterministic builds/ pinned versions. Not sure if EFF should entertain them or not in this endpoint or another. Besides, IMHO it might also worth to include the ruleset version inside
default.rulesetsitself.jeremyn commentedon Mar 26, 2018
@cschanaj As far as I know, downstream projects should be able to use and verify these rulesets if they want, or they can disable these automatic updates entirely or by default.
Hainish commentedon Mar 27, 2018
@cschanaj the extension itself will still be reproducible, and the rulesets can either be verified directly using the HTTPS Everywhere signing key, or they can modify the
update_channels.jsfile to provide their own endpoint with their own rulesets (possibly just an audited version of ours) and verify ruleset delivery with their own keys, if they don't trust ours.[-]Release 2018.4.2 forthcoming[/-][+]Release 2018.4.3 forthcoming[/+]Hainish commentedon Apr 2, 2018
Pushing this to tomorrow to finish up a few loose ends (https-rulesets.org home page, ensuring updating does not occur if the extension version > the rulesets version for the "EFF (Full)" update channel.
Georg from Tor Browser has looked through our signing code, and had some feedback which I incorporated into
sign-rulesets.Sign rulesets (#15063)
Hainish commentedon Apr 3, 2018
Release made!
Hainish commentedon Apr 3, 2018
Release blog post: https://www.eff.org/deeplinks/2018/04/https-everywhere-introduces-new-feature-continual-ruleset-updates
jeremyn commentedon Apr 3, 2018
@Hainish Congratulations on getting the
sign-rulesetsfeature out! I've argued against it before, but I want to acknowledge that it has taken a lot of work and planning from you and others to get to this point, and that you and others should feel proud for launching it. Now that the project has committed to it, I wish it well.Hainish commentedon Apr 3, 2018
Thank you @jeremyn, I really appreciate the well-wishes!