Skip to content
This repository was archived by the owner on Nov 6, 2023. It is now read-only.
This repository was archived by the owner on Nov 6, 2023. It is now read-only.

Discussion about out-of-band ruleset updates #12606

Closed

Description

@jeremyn

(@Hainish, @cowlicks:)

There's a code branch https://github.com/EFForg/https-everywhere/tree/sign-rulesets that is intended to allow ruleset updates out-of-band from regular version updates. The idea is that HTTPS Everywhere will request ruleset updates from a central EFF server periodically with the effect that rulesets will update from time to time without the version year.month.day changing.

Privacy Badger does something similar for a whitelist it uses. Recently there were some problems with the updates which lead to a frustrating experience for me, see EFForg/privacybadger#1466. After I understood what was happening, I argued against it, see for example EFForg/privacybadger#1466 (comment) and the related discussion.

I think this is also a bad idea for HTTPS Everywhere and I'd like to understand why this change is being made. I would at least like to avoid the user experience of things cycling between breaking and working for no apparent reason that I had with Privacy Badger.

Activity

  1. jeremyn commented on Sep 17, 2017

    @jeremyn
    ContributorAuthor

    @koops76 My concerns aren't about the crypto, it's about the surrounding user and maintainer experience. You can read through EFForg/privacybadger#1466 to get an idea of what I mean.

    In any case I would specifically like to hear from the EFF why they are doing this so I understand the motivation.

  2. Bisaloo commented on Sep 18, 2017

    @Bisaloo
    Collaborator

    @jeremyn, I know you are mainly waiting about the EFF answer but if I may chime in:

    I understand and I agree with your concerns. On the other hand, separating the rulesets updates from the core updates has several upsides in my opinion, which would still make this change a positive move for our users.

    Right now, when a ruleset fix is pushed on master, we have to either:

    • rush an update because it concerns a high traffic website (Urgent: fix redirect loops for meta.*.stackexchange.com #9110)
    • tell the users to wait a month for this change to reach them (and we know this is problematic because it's common to have reports saying "X is broken" when a fix has been pushed for weeks but hasn't been released yet)

    In short, I think this could be a net gain depending on how frequent the rulesets updates would be. So I guess my follow-up question is: how often do you plan to push the rulesets updates @Hainish?

    In any case, this change and this behaviour should be made very clear at different places (changelog, add-on description, FAQ, etc.) to alleviate some of the concerns raised by @jeremyn.

    (As a broader point, I think it would be really helpful if you kept the community in the loop when you are making big changes like this one. Not necessarily to argue but at least to inform so we can prepare for what's coming. This was already an issue in #10538.)

  3. cschanaj commented on Sep 18, 2017

    @cschanaj
    Collaborator

    So I guess my follow-up question is: how often do you plan to push the rulesets updates

    AFAIK, we make a release every 2 weeks with exceptions when there is blocking PRs. The benefit of out of band updates is not obvious unless we are pushing much more frequent than that.

    Unlike uBlock, HTTPSE and PB are privacy focused extensions. We should avoid phoning home while leaving the users unnoticed (as they are privacy/ security conscious). IMHO, out of band ruleset updates should be made on request (require user interactions) or on a opt-in basis.

    Besides, we rarely make core code changes. @Hainish How often are we going to make core releases when we have out of band ruleset updates? We should make the (ruleset & core code) release policies clearly documented.

    @jeremyn thanks for bringing this issue to the discussion!!

  4. Hainish commented on Sep 19, 2017

    @Hainish
    Member

    There are a number of reasons for out-of-band ruleset updates.

    1. Lax security of AMO updates
      A year ago today, Ryan Duff posted about a vulnerability discovered accidentally in the way Mozilla does key pinning in Firefox. This would allow anyone granted a misissued certificate for addons.mozilla.org to update extensions arbitrarily. Given the broad permissions traditional extensions are granted, this is a serious issue that is mitigated in part with the XPCOM deprecation. But it's a troubling sign that the security of AMO was circumvented by pure happenstance, and does not engender confidence in the Mozilla addon update infrastructure.
    2. Waiting for code review
      On the topic of AMO, we have in the past had to wait up to a couple of weeks before a new release is reviewed and published. In communication with Mozilla, we've been able to have them prioritize review, and this time span has shortened. But occasionally we'll observe that a new version will again be taking a long time for review. Of course, this is not a problem for the self-hosted version. But if there is a critical site that is in need of an update, the process of fixing it is made more difficult.
    3. Reliance on AMO, Chrome Webstore, Opera Addons
      This is related to but not the same as the above. The release process is simply made more difficult and cumbersome by having to upload to the various browser addon stores every time a release is made. This is also the case for the self-hosted Firefox version, since a signature is required by Mozilla before posting. It makes the release process much more laborious.
    4. Keys we control
      Given (1), Tor Browser is readying itself to disable the update channel for extensions.
      In order to ensure ruleset updates are reliably delivered, we have to move to an out-of-band delivery channel. With the sign-rulesets branch merged, updating the rulesets will be possible using keys that we control. This will rely on our own infrastructure and keys that are pinned in the extension itself.
    5. Multiple update channels
      Moving in this direction also makes the extension more modular. With update_channels.js, we allow the extension to specify multiple update channels. This will facilitate, for example, an intranet to deliver another rulset bundle to its constituents without publicly divulging secret FQDNs. This will also allow the Tor Browser to deliver its own ruleset bundle, with its own pinned key, via HTTPS Everywhere. This possibility is powerful for the reasons Paul Syversion, Tor creator, describes in this whitepaper.
    6. Possibility of granular updates in the future
      Currently, the rulesets comprise the vast majority of the extension download size. Of the 1.7M compressed xpi, a mere 421K is left to everything else - mostly translations. This means that delivery of the rulesets will be roughly equivalent to delivery of the extension itself, currently. But this transition opens up the possibility of delivering granular ruleset updates to extensions, which will vastly lower the total size of downloads in a unit time.

    In the future we may also expose the update channel to users, so that they themselves can compile ruleset bundles. But this will require us weighing the costs and benefits of adding an additional maintenance burden.

  5. jeremyn commented on Sep 19, 2017

    @jeremyn
    ContributorAuthor

    @Hainish Thanks for the detailed response.

    I would say that my concerns could be met by adding the following:

    • Inform users of the auto-update mechanism and allow them to enable/disable it. The Observatory UI is an example of what this might look like.
    • Allow users to reset the rulesets to the "stock" set that came with their current version of the add-on.
    • Version the ruleset updates and display this version to the user somewhere.

    Optionally:

    • Publish information somewhere about the EFF infrastructure responsible for serving updates, to give confidence about its reliability. Maybe a status page/dashboard would be good here.

    The motivations for all of these are, I hope, obvious, but I can elaborate if needed. None of these should interfere with the reasons you've given.

    Also, to repeat, my experience with similar functionality in PB was that the update target (the whitelist) was repeatedly and quietly wiped. If we move to this update model for HTTPS Everywhere then I consider it a more-than-theoretical possibility that the rulesets could be similarly wiped or broken. The above items give users the ability to lock down or reset HTTPS Everywhere to a known-good version.

  6. Hainish commented on Sep 20, 2017

    @Hainish
    Member

    @jeremyn

    Also, to repeat, my experience with similar functionality in PB was that the update target (the whitelist) was repeatedly and quietly wiped. If we move to this update model for HTTPS Everywhere then I consider it a more-than-theoretical possibility that the rulesets could be similarly wiped or broken. The above items give users the ability to lock down or reset HTTPS Everywhere to a known-good version.

    This bug was due to the fact that the whitelisted resources in Privacy Badger were not being verified before being applied. When the website had a bug, the whitelist would return a blank result. In this unfortunate edge-case, Privacy Badger happily applied this whitelist, thus causing the extension to malfunction.

    This will not happen with the ruleset updates, since every ruleset update is cryptographically signed by the extension developers, then verified in each users' extension before being applied.

    Given this, in what case would you envision the auto-update mechanism causing a fail case so large that we should expose extra options to the user? If privacy is the issue, we should keep in mind that already the users browser pings the extension store every day to check for new updates. EFF has a more stringent privacy policy than AMO or Google, so that shouldn't be a concern.

  7. Hainish commented on Sep 20, 2017

    @Hainish
    Member

    This being said, I do think you're right in this:

    Version the ruleset updates and display this version to the user somewhere.

  8. Hainish commented on Sep 20, 2017

    @Hainish
    Member

    The ruleset updates will already have a timestamp associated with them, when applied. This is to keep track of whether we need to update the rulesets upon subsequent checks. We can simply use this timestamp, formatted as YYYY-MM-DD, as the version that is displayed to the user.

  9. jeremyn commented on Sep 20, 2017

    @jeremyn
    ContributorAuthor

    The problem isn't with a specific bug, it's about the process. It's like your criticism of Mozilla in "Lax security of AMO updates": a bug was found, Mozilla fixed it, but now you're uncomfortable with their update process. Same thing here, right? A bug in the EFF update process was found, the EFF fixed it, but now a reasonable person could be uncomfortable with the EFF's update process. This reasonable person should have the ability to disable or revert EFF updates.

    For privacy, it's not about the privacy policy. It's about giving users control over outbound network requests and software updates. To quote myself from EFForg/privacybadger#1466 (comment):

    ...there is an endless list of software creators who want to build in quiet, uncontrollable update processes to their software. They always think they are justified. Users complain because they feel a loss of control over their system and because it causes unexpected breakage. This is no different. The EFF heavily criticized Microsoft for this sort of thing during the Windows 10 rollout.

  10. HTTPSNowhere commented on Sep 20, 2017

    @HTTPSNowhere
    Contributor

    @Hainish

    There's still the issue of the need of a fallback mirror; For example, in China EFF infrastructure is blocked there, so people with HTTPS Everywhere installed there can't benefit from this feature. Is there anyway to add some fallback mirror to for example directly from Github (which is thankfully not blocked there)?

  11. 70 remaining items

  12. jeremyn commented on Jan 25, 2018

    @jeremyn
    ContributorAuthor

    Some of my points were not addressed, such as those in #12606 (comment) (versioning, reset-to-stock, status dashboard) and my concern in #12606 (comment) about what happens if updates are permanently unavailable for a particular user.

  13. Hainish commented on Jan 25, 2018

    @Hainish
    Member

    @jeremyn updates will not cease to be available through extension updates. These will continue to be delivered for the foreseeable future. I suggest opening separate issues for feature specifics.

  14. jeremyn commented on Jan 25, 2018

    @jeremyn
    ContributorAuthor

    @Hainish I opened this issue to have a discussion here, and the comments I linked are over three months old now. I understand wanting to wrap up an open-ended discussion but I brought those points up very early on. It would be great if you could give at least high level answers to those concerns here. If you implement those features and I have questions/suggestions for the implementation, I can make new issues then.

  15. Hainish commented on Jan 25, 2018

    @Hainish
    Member

    @jeremyn I have addressed these issues, repeatedly in comments.

    See #12606 (comment), #12606 (comment), as well as having implemented the ruleset toggle you mentioned in https://github.com/EFForg/https-everywhere/tree/sign-rulesets. Again, if you have specific requests on the development of this branch, let's move that to individual issues.

  16. jeremyn commented on Jan 26, 2018

    @jeremyn
    ContributorAuthor

    @Hainish You did mention your support for versioning in #12606 (comment), thanks for pointing that out. Also I suppose that for the user whose updates are permanently broken, the implied fix is to wait until the next full extension update. You haven't, as far as I know, addressed the reset-to-stock or status dashboard ideas, but those aren't really that important.

    You keep mentioning the https://github.com/EFForg/https-everywhere/tree/sign-rulesets branch. Are you expecting contributors to monitor that branch and jump in if they have questions? Is that instead or, or will there also be, an announced alpha/beta release with enough time allowed to get and incorporate feedback?

  17. Hainish commented on Jan 26, 2018

    @Hainish
    Member

    @jeremyn For reset to stock, I think that can be built out as part of a more comrpehensive UX that I mention in #12606 (comment). I envision this as being dashboard-like, if I take your meaning correctly.

    Incorporating the sign-rulesets into a beta release seems like a good idea, and relates to the email sent by @brainwane over the list. It does open broader issues of beta channel release overhead, which I'm hesitant to introduce. For now, the sign-rulesets branch should be considered on a track for feature inclusion, and I would encourage anyone interested in this feature to check the branch out and play with it. I've also created a new label, sign-rulesets Issues and PRs related to out-of-band delivery of signed ruleset updates , and I'm adding it to this issue now.

  18. added
    sign-rulesetsIssues and PRs related to out-of-band delivery of signed ruleset updates
    on Jan 26, 2018
  19. jeremyn commented on Jan 26, 2018

    @jeremyn
    ContributorAuthor

    @Hainish So, my main concern with the functionality of this feature is to avoid the negative experience I had with the similar feature in Privacy Badger, which I discussed in my earliest comments in this discussion. I didn't know the feature existed in Privacy Badger, was surprised to find out it was contacting the EFF for updates, things cycled through breaking and working, and I couldn't stop, reset, or even know when these updates were happening. I'm not really concerned about the inclusion of any one specific subfeature, but rather that the unified whole is robust to failure, even in hostile networks, and that problems can be avoided, or fixed automatically or fixed by very low technical users. There's no mock/whiteboard/design document for what you are implementing so it's difficult to discuss it at that level, so instead we've been talking about individual subfeatures, which misses the point.

    (I'm not saying you need to produce a design document or anything, I'm just trying to explain one reason why this conversation has been perhaps a little awkward.)

    I probably won't track an active development branch in order to give comments. Maybe other contributors will do that. I think you would find it irritating for people to do that, most developers don't want to get bug reports on stuff they are actively coding. I understand what you mean about the burden of making a beta release, particularly in this case where you have no guarantee that anyone will review it even if the beta release is packaged beautifully. I'm not sure what to suggest about that. Maybe you could build a week or two gap in the middle of the development cycle for sign-rulesets, where you'll have something stable-ish for people to review if they want to, but also plan to work on something else during that gap so the time isn't wasted if no one responds.

  20. Hainish commented on Jan 26, 2018

    @Hainish
    Member

    @Hainish So, my main concern with the functionality of this feature is to avoid the negative experience I had with the similar feature in Privacy Badger, which I discussed in my earliest comments in this discussion. I didn't know the feature existed in Privacy Badger, was surprised to find out it was contacting the EFF for updates, things cycled through breaking and working, and I couldn't stop, reset, or even know when these updates were happening. I'm not really concerned about the inclusion of any one specific subfeature, but rather that the unified whole is robust to failure, even in hostile networks, and that problems can be avoided, or fixed automatically or fixed by very low technical users.

    The reason for that failure, as well as why it doesn't affect this branch, was explained over four months ago: #12606 (comment). sign-rulesets was never affected, since we have had a robust process for ruleset verification since day one. In Privacy Badger, no verification was present at all, afaik. I did not work on that feature, and the underlying cause of that bug is absent here. So a bug in another project I do not work on seems hardly relevant here. It is an EFF project, but there are a different set of people working on the security-focused HTTPS Everywhere and the privacy-focused Privacy Badger.

    After implementing the sub-features in #12606 (comment), I think the most sensible thing to do is to solicit reviews, with instructions on how to build and test from sign-rulesests. If no one responds, that will be that - all I can do is extend the invitation. I wouldn't be blocking my time on this feature - in any case, I'd be working on other things.

  21. jeremyn commented on Jan 26, 2018

    @jeremyn
    ContributorAuthor

    @Hainish Where Privacy Badger failed, for me, wasn't in whether it verified downloads, but in that it didn't tell the user what was happening and gave them no tools to fix problems. If you want to talk about how you're avoiding the Privacy Badger problem, the discussion should be about UX/UI improvements, not about cryptographic signing, which is good but not really the issue.

    If you had a phone, and one day you discovered the phone has been bricked, and you learn it's because some update failed, would you be upset at the vendor because of the update bug, or would you be upset because there was a way for your phone to be bricked without warning and with no way to recover it? Probably the second, and you wouldn't really care about the update bug at that point.

    That sounds reasonable about announcing a review. If no one responds, there's not much you can do. Is there, or will there be, a ruleset endpoint available for testing?

  22. wonderchook commented on Jan 27, 2018

    @wonderchook
    Contributor

    @jeremyn from a user perspective you are suggesting it be clear how to turn the feature on and off, plus having some sort of indication of an error if things fail?

  23. jeremyn commented on Jan 27, 2018

    @jeremyn
    ContributorAuthor

    @wonderchook Yes, pretty much. Turn on or off, show the current version, report errors in a friendly way, and allow users to roll back updates.

  24. Hainish commented on Jan 30, 2018

    @Hainish
    Member

    @jeremyn I'll consider the best way to balance user controls and clear design for this PR. Notably, I've mentioned that certain UX controls will have to be added later, as we build out this feature. Others (such as display of the ruleset version and source) may make sense for initial rollout.

  25. Hainish commented on Mar 8, 2018

    @Hainish
    Member

    https://github.com/EFForg/https-everywhere/tree/sign-rulesets now includes versioning information for rulesets in the popup.

  26. added a commit that references this issue on Apr 3, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    sign-rulesetsIssues and PRs related to out-of-band delivery of signed ruleset updates

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions