Skip to content
This repository was archived by the owner on Nov 6, 2023. It is now read-only.
This repository was archived by the owner on Nov 6, 2023. It is now read-only.

Ruleset repository #2697

Closed
Closed

Description

@chaoticsmol

I was wondering if it'd be possible to create a separate repository to contain only the rulesets?
With a tiny bit of modification to the setup scripts that already exist, it would be fairly trivial to include that repository as a sub-repository exactly where it already is in https-everywhere.

The reason we might want to do this is to make the rules more available to other applications that might want to use them. In fact, I'm hoping to do exactly that in a project I'm working on.

Activity

  1. jsha commented on Aug 27, 2015

    @jsha
    Member

    The problem with splitting out into a different repository is that we'd lose all ruleset history. Although maybe we could clone the whole repository history. A less invasive approach would be to just move the rules directory within HTTPS Everywhere to the top level, which makes logical sense since both Chrome and Firefox use the rules, and would save some typing.

    I've considered the latter a bunch of times, and would like to do one or the other. But my experience is that seemingly small changes like this generate surprisingly disproportionate maintenance overhead, as unexpected things break. So I'd like to postpone either such move until after crunch time for Let's Encrypt is over and I can dedicate more time to fixing issues.

    For the project you're working on, can you embed the https everywhere repo as a whole? The JS files shouldn't add too much size relative to the rules.

    Thanks,
    Jacob

  2. chaoticsmol commented on Sep 4, 2015

    @chaoticsmol
    Author

    I ended up writing a simple bash script to clone https-everywhere, run the make script that compiles the rulesets into a single XML file and performs verification etc., and then just removes the unneeded parts of the rest of the codebase. It's far from elegant, but it works.

    Your concerns are very valid, but I think I can suggest a method that might sit well with you and work out seamlessly.

    1. Create a new repository, just call it rules
    2. On your own filesystem, copy all of the contents of the https-everywhere directory to rules/, including the .git/ directory and related files/directories.
    3. In rules/, directly modify the .git/[...] files to change the repository settings (e.g. name) appropriately
    4. Push rules/ to origin master (github.com/EFForg/rules) - You now have the entire HTTPS Everywhere history copied
    5. Move the contents of rules/src/chrome/content/rules to the top-level of rules/
    6. git rm -rf everything but the ruleset files and commit + push. rules/ now contains only rulesets.
    7. In https-everywhere, git rm -r src/chrome/content/rules
    8. Make EFForg/rules a sub-repository of EFForg/https-everywhere in src/chrome/content

    With all this, you should now retain the exact same structure in https-everywhere as before, and would only have to modify build scripts ever so slightly so that they perform the appropriate sub-repository management to handle the rules sub-repo.

    A lot of this is speculative and I'm not sure it's guaranteed to work. I'll try the idea out in a proof-of-concept setting sometime and report back. It's certainly not completely effortless, but it should be fairly straightforward and achieve the desired effect. I understand not being able to do it immediately.

  3. strugee commented on May 22, 2017

    @strugee
    Contributor

    @jsha just FYI it's pretty easy ("easy") to split the rulesets out to a different repository while still maintaining a clean history. Similar to what @zsck said, but with a bit less sketchiness: https://help.github.com/articles/splitting-a-subfolder-out-into-a-new-repository/

    While I'm generally in favor of this one concern I have is that it makes the contributor experience worse, because you have to grapple with two repos instead of one. How is a ruleset contributor supposed to get work done? Clone the main repo and work in the submodule? How do they send PRs? (I know it's technically possible but it's beyond many people's git abilities since the mental model is too complex.) Clone just the rules repo? How do they test? Etc.

    One potential compromise might be to provide a read-only (i.e. no PRs) repo that provided just the rules. It'd be built using a (modified version of) the filter-branch process on a cronjob. This would still provide a repo with just the rules, and even provide the benefit of ruleset issues being separate from codebase issues. But that in turn would cause some awkwardness with people being expected to send issues to one repo and PRs to another.

    Or, third idea: maybe we could eliminate the requirement to have the source checked out in order to test rulesets? I.e. the ruleset contributor experience would be: install the extension from AMO, clone the ruleset repo, maybe run a build step (though hopefully we could eliminate this step too), and point the production extension at the local clone.

  4. migueldemoura commented on Jun 20, 2017

    @migueldemoura
    Contributor

    As I mentioned on #10320, splitting these two into separate repositories would be helpful in improving the speed at which the actual extension core is developed. It also fosters external implementations of the rewrite engine.

    From what I've read, the main issue it the tight coupling between the core and the rulesets due to tests. Why is this?

  5. cowlicks commented on Oct 18, 2017

    @cowlicks
    Contributor

    Looking over this. It seems like the main priority would be to ensure the process is smooth for the ruleset maintainers.

    So we should probably move the extension source to another repo, and leave the rulesets here. This way the github issue & PR history and git history is preserved and easily accessible for rs maintainers.

    We should also provide a way for rulesets to be imported directly into the extension. This way rs maintainers would not need to mess with the development version of the HE extension.

    Moving forward, I think we should start to develop an import function for maintainers to use. This could be built on top of @Hainish work in #12863

    If that goes well, we could start asking rs maintainers to use the feature. If we are able ween them off of manually rebuilding the extension this way, then we can start to think about moving the extension source into another directory.

  6. pipboy96 commented on Aug 10, 2019

    @pipboy96
    Contributor

    This is planned for 2020.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions