Skip to content
This repository was archived by the owner on Nov 6, 2023. It is now read-only.

Add ability to disable HTTPS Everywhere for specific domains without first visiting the site (#17434) - #17890

Closed
bershanskiy wants to merge 13 commits into
EFForg:masterfrom
bershanskiy:master
Closed

bershanskiy wants to merge 13 commits into
EFForg:masterfrom
bershanskiy:master

Conversation

@bershanskiy

Copy link
Copy Markdown

WORK IN PROGRESS! DO NOT MERGE!

This allows user to add a domain name to the whitelist from within Options menu (without visiting the site). No domain validation is performed.

This is an incredibly simplistic implementation (e.g., no domain validation or Punycode encoding or any CSS work) so please tell me which things I should change.

Preview

eff

I decided to put the form under the disabled list title on "General" page because it seemed more natural than putting it on "Advanced" page.

Domain names validation

What is the preferred validation method? Internally, HTTPS Everywhere seems to prefer IDN in Punycode and does not allow wild cards. Should I enforce this and display error messages? What

Preferred code style

  1. There are a few variables declared as let even though they can be declared as const. Is this intentional?
  2. Variable names are not always consistent (some use underscores, some are camel-cased). What's preferred?
  3. There are many tiny refactoring things that can be done in the Options UI. I omitted them from this commit for now for simplicity sake.

CC @raman325 @pipboy96

@pipboy96

pipboy96 commented May 4, 2019

Copy link
Copy Markdown
Contributor

@bershanskiy Use camelCase and const where possible.

@pipboy96

pipboy96 commented May 4, 2019

Copy link
Copy Markdown
Contributor

It would make sense to allow left-side wildcards, such as *.github.io.

@bershanskiy

Copy link
Copy Markdown
Author

@pipboy96 Thanks for the prompt reply.

Use camelCase and const where possible.

Will do.

It would make sense to allow left-side wildcards, such as *.github.io.

Currently it allows wildcards, as well as anything else you put in the box. In fact, there is no validation performed at all (which is definitely bad). Could you write-up exact criteria for input validation?

I understand it as follows:

  • fully-qualified domain names like github.com; IDN should be converted to Punycode, but invalid Punycode (domain labels that start with xn-- but are valid domain name and not valid Punycode) are allowed.
  • left-hand side wildcards like *.github.io
  • all mentioned above but with a port number, e.g. github.io:1000 or *.github.io:1000

I did not make sure the background scripts support wildcards yet. But I'll definitely do that before merging.

@pipboy96

pipboy96 commented May 4, 2019

Copy link
Copy Markdown
Contributor

@bershanskiy I will write a JS function to validate domains.

@pipboy96

pipboy96 commented May 4, 2019

Copy link
Copy Markdown
Contributor

I think port numbers are not allowed.

@pipboy96 pipboy96 added the ui label May 4, 2019
@bershanskiy

Copy link
Copy Markdown
Author

@pipboy96

I will write a JS function to validate domains.

Great. Will you also implement the error message display for invalid entries or should I do that?

I think port numbers are not allowed.

Could you double-check? The popup UI disableOnSite() and enableOnSite() explicitly use url.host and not url.hostname. Also, when I open a site with a non-default port and disable HTTPS-E on it, the Popup does display the port.

Is there anything I should do to advance this PR besides changing to camelCase and const?

@pipboy96

pipboy96 commented May 4, 2019

Copy link
Copy Markdown
Contributor

@bershanskiy Generally, HTTPS Everywhere doesn't seem to handle custom ports well, e.g. https://example.com and https://example.com:443 are treated like different origins.

@pipboy96

pipboy96 commented May 4, 2019 •

Copy link
Copy Markdown
Contributor

You would also need to .trim() and .toLowerCase() the domain.

@pipboy96

pipboy96 commented May 4, 2019

Copy link
Copy Markdown
Contributor

Probably some kind of punycode encoder/decoder would also be needed.

Anton Bershanskiy added 2 commits May 4, 2019 16:56
…17434)

This allows user to add a domain name to the whitelist from within
Options menu (without visiting the site). No domain validation is
performed.
@pipboy96

pipboy96 commented May 4, 2019

Copy link
Copy Markdown
Contributor

This is code to encode/decode punycode: https://gist.github.com/pipboy96/cd3b89bf9598e90c0c2dda4e9fe9c286

@pipboy96

pipboy96 commented May 5, 2019

Copy link
Copy Markdown
Contributor

Regex: /^(\*|[a-z0-9_-]+)(\.[a-z0-9_-]+)*$/.

@pipboy96

pipboy96 commented May 5, 2019 •

Copy link
Copy Markdown
Contributor

I recommend you to show IDNs in this way: "xn--80acde.xn--j1aef (абвг.ком)" but obviously store in encoded form.

@bershanskiy

Copy link
Copy Markdown
Author

@pipboy96

I recommend you to show IDNs in this way: xn--80acde.xn--j1aef (абвг.ком) but obviously store in encoded form.

Will do.

Also, could you go over the validateDomain:

  function validateDomain(domain) {
    // Checks whether input is a valid domain and returns it in a canonocal form
    // TODO: should apply Punycode before or after toLowerCase()?
    // TODO: what if toASCII errors out?
    domain = domain.trim();
    try {
      domain = toASCII(domain);
    } catch(e) {
      // This domain is not representable as Punycode
      return null
    }
    domain = domain.toLowerCase();
    const pattern = /^(\*|[a-z0-9_-]+)(\.[a-z0-9_-]+)*$/
    const match = pattern.test(domain);
    if (match !== true){
      domain = null;
    }
    return domain;
  }

What is the proper order of trim(), toLowerCase(), and toASCII(...)? I guess trim() goes first to remove the obviously unneeded spaces, but what's next toLowerCase() or toASCII(...). I believe the IDN codepoint tables are designed so that toLowerCase() is a noop, but I do not have a solid source for that.

The linter does not see the definition of Punycode functions:

  291:25  error  'toUnicode' is not defined          no-undef
  328:16  error  'toASCII' is not defined            no-undef

May be you have an idea where to add the proper includes/ exclusions?

@pipboy96

pipboy96 commented May 5, 2019

Copy link
Copy Markdown
Contributor

toLowerCase may do weird things for non-ASCII chars depending on OS locale. Do it after toASCII.

@bershanskiy

Copy link
Copy Markdown
Author

@pipboy96

toLowerCase may do weird things for non-ASCII chars depending on OS locale. Do it after toASCII.

That's what I was thinking too.
But is it possible that Punycode is different depending on capitalization of the letters in the domain? I think Punycode automatically lowers the case, e.g. Punycode of абвг.ком and АБвг.ком is xn--80acde.xn--j1aef, but Unicode of xn--80acde.xn--j1aef is абвг.ком. I don'thave time to work on that today, so I'll look it up tomorrow.

@pipboy96

pipboy96 commented May 5, 2019 •

Copy link
Copy Markdown
Contributor

This (6293476, 0cda424) should remove linter error.

@pipboy96

pipboy96 commented May 5, 2019

Copy link
Copy Markdown
Contributor

I was going to lint anyway, but thanks!

Comment thread chromium/pages/options/index.html Outdated
<script src="../util.js" defer></script>
<script src="../punycode.js" defer></script>
<script src="ux.js" defer></script>
<script src="../translation.js" defer></script>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

async?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also move to end of <body>.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not use async because scripts have dependencies on each other. ux.js and translation.js can be made into async because they wait for "DOMContentLoaded" anyway, but they do need util.js and punycode.js to be loaded in advance. I think I can make translation.js into async, but not the others.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved those to the end of <body> and made translation.js async, but ux.js still needs util.js and punycode.js to run before it so I mad them defer to preserve order.

@bershanskiy

Copy link
Copy Markdown
Author

This (6293476, 0cda424) should remove linter error.

Linter was also complaining about indentation, so I changed that. However, Travis still complains about "preload", which should not be impacted by this PR, since it does not change any rules...

@pipboy96

pipboy96 commented May 5, 2019

Copy link
Copy Markdown
Contributor

@bershanskiy hsts-prune needs npm audit to be run.

@pipboy96

Copy link
Copy Markdown
Contributor

@bershanskiy Can you fix merge conflicts?

@pipboy96 pipboy96 self-assigned this Sep 10, 2019
@bershanskiy

Copy link
Copy Markdown
Author

@pipboy96 I resolved conflicts on this one via the GitHub online interface as well, but I'll test it for real too. I'll comment when I'm done.

@pipboy96 pipboy96 removed the conflict label Sep 11, 2019
@pipboy96 pipboy96 added the inactive Label for PRs/issues which stay inactive for a long time label Oct 4, 2019
@pipboy96

pipboy96 commented Oct 4, 2019

Copy link
Copy Markdown
Contributor

@bershanskiy Do you have any plans to continue work on this?

@zoracon

zoracon commented Feb 21, 2020

Copy link
Copy Markdown
Contributor

Closing out for now, can reopen at request of the author.

@zoracon zoracon closed this Feb 21, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

inactive Label for PRs/issues which stay inactive for a long time ui

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants