Skip to content
This repository was archived by the owner on Nov 6, 2023. It is now read-only.

Embedded webextension isolated - #11792

Closed
Hainish wants to merge 10 commits into
EFForg:firefox-webextensions-corefrom
Hainish:embedded-webextension-isolated
Closed

Hainish wants to merge 10 commits into
EFForg:firefox-webextensions-corefrom
Hainish:embedded-webextension-isolated

Conversation

@Hainish

@Hainish Hainish commented Aug 9, 2017

Copy link
Copy Markdown
Member

No description provided.

@Hainish
Hainish force-pushed the embedded-webextension-isolated branch from 2ef14ca to ca9b6d3 Compare August 9, 2017 19:42
@Hainish
Hainish requested a review from jsha August 9, 2017 19:44
@Hainish

Hainish commented Aug 9, 2017

Copy link
Copy Markdown
Member Author

A failure to the firefox tests is expected, resolution depends on #11789. This is rebased on top of #11791

@Hainish Hainish mentioned this pull request Aug 9, 2017
Comment thread chromium/background.js Outdated
});

// Load in the legacy custom rulesets, if any
storage.get({legacy_custom_rulesets: false}, item => {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be simpler with a default value of [] so you can skip the if, and just iterate straight through.

Comment thread chromium/background.js Outdated
// Load custom rulesets and add to storage
await new Promise(resolve => {
for(let ruleset of settings.custom_rulesets){
all_rules.addFromXml((new DOMParser()).parseFromString(ruleset, 'text/xml'));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please factor out a function load_legacy_custom_rulesets, so that you can call it here after you do storage.set.

Comment thread chromium/background.js Outdated
all_rules.ruleCache = new Map();

// Set/store globals
storage.set({'httpNowhere': settings.prefs.http_nowhere_enabled});

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason you provide a callback when calling set above, but not here? It seems like we should use the callback on both, to detect success or failure.

Comment thread chromium/background.js Outdated
}
await Promise.all(rule_toggle_promises);

all_rules.ruleCache = new Map();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment explaining why this is needed?

Comment thread chromium/background.js Outdated

rule_toggle_promises.push((ruleset_name, ruleset_active => {
return new Promise(resolve => {
for(let host in all_rules.targets){

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of comments here:

  • This doesn't seem to do any async work, so why is it wrapped in a Promise?
  • This seems to re-do the work of marking rulesets active or inactive that should happen at startup. Similar to the loading of legacy rulesets I mentioned above, can you just store the settings, and then re-load the rulesets? In fact, since loading the rulesets should include loading legacy_custom_rulesets, you can just do that and don't have to separately invoke the loader for legacy_custom_rulesets.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This task can take quite a while, since it's iterating through all ruleset targets and checking all rules they contain until finding a match. Creating a new promise for each of these isn't to resolve callbacks, but to start asynchronous program flow for each of these potentially many toggles.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is javascript and not actually dedicating separate cores to the problem, it does not necessarily give us any efficiency gain to split this up into multiple iterations of the event loop. I can refactor this to be synchronous.

Comment thread chromium/background.js Outdated
// Set/store globals
storage.set({'httpNowhere': settings.prefs.http_nowhere_enabled});
storage.set({'showCounter': settings.prefs.show_counter});
isExtensionEnabled = settings.prefs.global_enabled;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This setting doesn't seem to get stored?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's true, the extension enabled toggle is not stored (and has never been). We should fix that but I figured it was outside of the scope of this PR.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, interesting! I did not know that. It seems weird to apply this toggle only on first run after an upgrade, though. If storing it doesn't work, I'd suggest just removing this line.

@ghost

ghost commented Aug 9, 2017

Copy link
Copy Markdown

@Hainish Also, can you already code this in StandardJS style or the style of your choice (I recommend Standard)?

Comment thread makexpi.sh Outdated
rsync -aL --delete --delete-excluded --exclude /chrome/content/rules src/ pkg/xpi-eff
cp -a translations/* pkg/xpi-eff/chrome/locale/

# The next lines are for embedded WebExtensions, and can be deleted after the full transition to WebExtensions

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should probably include a footer indicating the end of the lines that can be deleted after the full transition.

Comment thread src/bootstrap.js Outdated
if (msg == "import-legacy-data") {
let globals = get_prefs("extensions.https_everywhere.");
let prefs_array = globals.getChildList("", {});
if(~prefs_array.indexOf("webextensions-migrated") ? !globals.getBoolPref("webextensions-migrated") : true){

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This if statement is overly complicated. In particular, :? operators are ideally avoided because they are hard to read and error prone. Embedding a :? inside an if makes that especially so. :-) Can you break it out for clarity? Also, the ~ operator is a new idiom in this codebase. I think it would be better not to introduce it in such a large change without strong motivation.

Comment thread src/bootstrap.js Outdated
let changed = false;

let prefs = {
http_nowhere_enabled: ~prefs_array.indexOf("http_nowhere.enabled") ? globals.getBoolPref("http_nowhere.enabled") : false,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pattern of "check if pref exists, return default value if not" should be a small helper function.

Comment thread src/bootstrap.js Outdated
let globals = get_prefs("extensions.https_everywhere.");
let prefs_array = globals.getChildList("", {});
if(~prefs_array.indexOf("webextensions-migrated") ? !globals.getBoolPref("webextensions-migrated") : true){
let rules = get_prefs("extensions.https_everywhere.rule_toggle.");

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move let rules and let changed down to right before they are used.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree for rules, but for changed it seems this is a var which is featured prominently throughout this scope. I'd argue that it's appropriate to keep it at the top.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds reasonable.

Comment thread chromium/background.js
updateState();
} else if (message.type == "remove_rule") {
removeRule(message.object);
} else if (message.type == "import_settings") {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment explaining when this is used? I.e., importing from the Options page rather than automatically from the parent extension?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added.

@Hainish
Hainish force-pushed the embedded-webextension-isolated branch from ca9b6d3 to aa68e92 Compare August 9, 2017 21:18
@J0WI J0WI added this to the Before the Next Release milestone Aug 9, 2017
@Hainish
Hainish force-pushed the embedded-webextension-isolated branch from aa68e92 to e70a968 Compare August 9, 2017 21:44
@Hainish
Hainish force-pushed the embedded-webextension-isolated branch from e70a968 to 100fdde Compare August 9, 2017 22:48
Comment thread chromium/background.js Outdated
for(let ruleset_name in settings.rule_toggle){
localStorage[ruleset_name] = settings.rule_toggle[ruleset_name];

(ruleset_name, ruleset_active => {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you see my earlier comment about just storing the data and reloading? E.g. all_rules = new RuleSets(localStorage)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! Didn't catch that. Fixing now.

Comment thread chromium/background.js Outdated
})(ruleset_name, settings.rule_toggle[ruleset_name]);
}

// This is necessary in order to clear the cache for any previous ruleset lookups

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you do the reload, this won't be necessary anymore.

Comment thread chromium/background.js Outdated
await new Promise(resolve => {
storage.set({'showCounter': settings.prefs.show_counter}, resolve);
});
isExtensionEnabled = settings.prefs.global_enabled;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you see my comment about removing this since it will only work once, and is therefore inconsistent/confusing?

@Hainish
Hainish force-pushed the embedded-webextension-isolated branch from 100fdde to a674977 Compare August 9, 2017 22:55
@Hainish
Hainish force-pushed the embedded-webextension-isolated branch from a674977 to bd23706 Compare August 9, 2017 23:11
@Hainish

Hainish commented Aug 9, 2017

Copy link
Copy Markdown
Member Author

Ready for review again.

@Hainish
Hainish force-pushed the embedded-webextension-isolated branch from a44f7f1 to 3386b94 Compare August 10, 2017 00:20
@Hainish

Hainish commented Aug 10, 2017

Copy link
Copy Markdown
Member Author

Hold on, for some reason the default rulesets are absent when the second call of new RuleSets(localStorage) is made.

@ghost

ghost commented Aug 10, 2017

Copy link
Copy Markdown

Do you mutate the argument in the RuleSets constructor?

@Hainish

Hainish commented Aug 10, 2017

Copy link
Copy Markdown
Member Author

@koops76 not in a way that would cause targets to be lost

@ghost

ghost commented Aug 10, 2017

Copy link
Copy Markdown

Try making a copy of everything in localStorage and passing that copy into the RuleSets constructor.

@Hainish

Hainish commented Aug 10, 2017

Copy link
Copy Markdown
Member Author

I figured it out....

@ghost ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good overall. 👍🏻
Sorry for such a large amount of comments.

Comment thread chromium/background.js
all_rules = new RuleSets(localStorage);

// Set/store globals
await new Promise(resolve => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DRY

Comment thread chromium/options.css
@@ -0,0 +1,17 @@
.section-header{

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a space between the { and the selector.

Comment thread chromium/options.html
<script src="send-message.js"></script>
</head>
<body>
<div id='import-confirmed' i18n="options_imported"></div>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use data-i18n for valid HTML5.

Comment thread chromium/options.html
<head>
<meta charset="utf-8">
<link href="options.css" rel="stylesheet" type="text/css"/>
<script src="options.js"></script>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move the scripts to the end of body tag.

Comment thread chromium/options.js
document.addEventListener("DOMContentLoaded", () => {

let json_data;
let import_button = document.querySelector("#import");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const?

Comment thread src/bootstrap.js
if(!pref_retrieve_bool_with_default(globals, "webextension-migrated", false)){
let changed = false;

let prefs = {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const?

Comment thread src/bootstrap.js
changed = true;
}

let rule_toggle = {}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const?

Comment thread src/bootstrap.js
}

let rule_toggle = {}
let rules = get_prefs("extensions.https_everywhere.rule_toggle.");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const?

Comment thread src/bootstrap.js

let rule_toggle = {}
let rules = get_prefs("extensions.https_everywhere.rule_toggle.");
for(let rule_toggle_key of rules.getChildList("", {})){

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const?

Comment thread src/bootstrap.js
changed = true;
}

let custom_rulesets = get_custom_rulesets_array();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const?

@Hainish

Hainish commented Aug 10, 2017 •

Copy link
Copy Markdown
Member Author

@jsha There's a problem with loading the XML for legacy rulesets in the constructor. The rules themselves aren't stored in localStorage and are only loaded on what's currently line 31 of this branch in background.js, directly by reading the file. So in order to just do new RuleSets(localStorage) any time we want to reload rules, we'd have to move that into the constructor as well (along with the lookup for mixed content rules). I'm uncomfortable with this, because it makes the RuleSets object less agnostic, and we will have to rip this out anyway when we merge the sign-rulesets branch, which loads rules and stores them via the network.

I therefore propose that we keep the previous workflow for loading rulesets.

@jsha

jsha commented Aug 10, 2017

Copy link
Copy Markdown
Member

That sounds fine for now, although I think the localStorage distinction is a bit of a red herring, since the custom rules get stored in the chrome.storage API, which nearly the same thing but a little fancier. I would rephrase it as: The current "load custom rulesets" code expects to be able to call .addFromXml on a fully constructed RuleSets object. Calling it from the middle of the constructor winds up being a bit weird. Might be worth refactoring, but not on the blocking path for this PR IMO.

LGTM, modulo @koops76's feedback, and fixing the tests.

@Hainish

Hainish commented Aug 10, 2017 •

Copy link
Copy Markdown
Member Author

@jsha, great, I'll change it back to the way it was and merge master into this branch (so the tests pass). Can you give me a final LGTM before I pull the trigger?

@jsha

jsha commented Aug 10, 2017

Copy link
Copy Markdown
Member

Sure, will check back later tonight.

@Hainish

Hainish commented Aug 10, 2017

Copy link
Copy Markdown
Member Author

@koops76 thanks for the feedback. Since we are trying to get this out the door, I'd like to save that for a further refactor.

@ghost

ghost commented Aug 10, 2017

Copy link
Copy Markdown

@Hainish No problem.

@Hainish
Hainish force-pushed the firefox-webextensions-core branch from 957b3ce to 0d1954a Compare August 10, 2017 01:43
Comment thread chromium/background.js Outdated
})(ruleset_name, settings.rule_toggle[ruleset_name]);

// This line is necessary to clear cached lookups of rulesets already loaded
all_rules.ruleCache = new Map();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you reverted too far. You had an intermediate branch that looked good where you skipped the multiply-nested for loops above, and called all_rules = new RuleSet(localStorage). That will work fine if you just much load_legacy_custom_rulesets(...) below new RuleSet

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsha we'd have to copy lines 27-32 as well...

// Allow users to enable `platform="mixedcontent"` rulesets                
var enableMixedRulesets = false;                                           
storage.get({enableMixedRulesets: false}, function(item) {                 
  enableMixedRulesets = item.enableMixedRulesets;                          
  all_rules.addFromXml(loadExtensionFile('rules/default.rulesets', 'xml'));
});                                                                        

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise we'll have the problem I mentioned in #11792 (comment).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could do

all_rules = new RuleSet(localStorage);
all_rules.addFromXml(loadExtensionFile('rules/default.rulesets', 'xml'));
load_legacy_custom_rulesets(settings.custom_rulesets);

Which is probably fine for now, but seems like we may run into complications when we are working on merging sign-rulesets. Though I can take care of that problem when we come to it, I suppose.

@Hainish
Hainish force-pushed the embedded-webextension-isolated branch from 575f4d7 to 32bd342 Compare August 10, 2017 02:15
@Hainish

Hainish commented Aug 10, 2017

Copy link
Copy Markdown
Member Author

@jsha ready for final review

Comment thread chromium/background.js
async function import_settings(settings){
if(settings.changed){
// Load custom rulesets and add to storage
await new Promise(resolve => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need to wait until the settings are stored?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@koops76 we do for the options ui. We only want to display the Settings imported string once the settings have actually been imported.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then use Promise.all to await all four promises.

@ghost

ghost commented Aug 10, 2017 •

Copy link
Copy Markdown

@Hainish My opinion is: it's OK, but has repetitive code that must be moved into a function and you probably do not need to wait until the settings are stored, or instead you could use Promise.all.

Comment thread chromium/background.js
return null;
},

addHostnameToTab: function(tabId, ruleset_name, hostname) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the "addHostnameToTab" work snuck back in by accident.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsha this is just because this was forked from the first iteration of firefox-webextensions-core - should I rebase it?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could rebase from master, now that firefox-webextensions-core is merged.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Hainish Rebase it I guess.

@Hainish

Hainish commented Aug 10, 2017

Copy link
Copy Markdown
Member Author

New PR, rebased from master: #11800

@Hainish Hainish closed this Aug 10, 2017
@jsha

jsha commented Aug 10, 2017

Copy link
Copy Markdown
Member

I'll review that, but it probably would have been easier to merge rather than rebase.

@Hainish

Hainish commented Aug 10, 2017 via email

Copy link
Copy Markdown
Member Author

@Hainish
Hainish deleted the embedded-webextension-isolated branch February 23, 2018 01:50
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants