You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Nov 6, 2023. It is now read-only.
Repository navigation
This repository was archived by the owner on Nov 6, 2023. It is now read-only.
install-dev-dependencies.sh and test.sh fail on Debian Jessie #2801
Currently, install-dev-dependencies.sh attempts to install chromium-chromedriver which is not part of the standard repositories. There is a package called chromedriver which perhaps(?) does the same thing.
The same script attempts to access test/chrome/ which does not exist. The folder test/chromium/ does, though.
Even with those two changes, test.sh still gives the following error:
ChromeDriver isn't installed. Check test/chrome/README.mdfor instructions on how to install ChromeDriver
I don't think chromedriver is a standard package and therefore might not exist in Debian's repositories. If there exists a package called chromedriver then that should be the same thing indeed, but we need to add a condition to install-dev-dependencies to either install the chromedriver package or download and install chromedriver manually.
I'm on a train right now and I don't have enough connectivity or space, so it's a bit too difficult for me to check Debian's repos right now. Sorry :(
Sorry, was testing this on a master from a couple of hours ago; hadn't seen the latest changes. Anyway, I'm still getting the same error off of the current master. Performing the 'chrome' -> 'chromium' change to https-everywhere-tests/tests/test-httpse-installed in #2803, the test stalls at
Full Message: Error: Module: https-everywhere-tests/tests/test-httpse-installed has no authority to load: chromium
Full Stack: @https-everywhere-tests/tests/test-httpse-installed:58:undefined
resolve@resource://gre/modules/commonjs/sdk/loader/cuddlefish.js:58:1
@resource://jid1-we5bqofc6sksmg-at-jetpack/https-everywhere-tests/tests/test-httpse-installed.js:3:20
CuddlefishLoader/options<.load@resource://gre/modules/commonjs/sdk/loader/cuddlefish.js:79:18
findTests/</getNextTest@resource://gre/modules/commonjs/sdk/deprecated/unit-test-finder.js:164:25
findTests/</<.getNext@resource://gre/modules/commonjs/sdk/deprecated/unit-test-finder.js:197:32
runNextTest@resource://gre/modules/commonjs/sdk/deprecated/unit-test.js:516:14
startMany@resource://gre/modules/commonjs/sdk/deprecated/unit-test.js:530:12
findAndRunTests/<@resource://gre/modules/commonjs/sdk/deprecated/unit-test.js:28:1
Handler.prototype.process@resource://gre/modules/Promise-backend.js:867:23
this.PromiseWalker.walkerLoop@resource://gre/modules/Promise-backend.js:746:7
this.PromiseWalker.scheduleWalkerLoop/<@resource://gre/modules/Promise-backend.js:688:37
" {file: "https-everywhere-tests/tests/test-httpse-installed" line: 58 column: 0 source: "58"}]
And no, that error is different. It doesn't actually require Chrome or Chromium, it requires Chrome library, which is a completely different thing AFAIK (I have some experience developing Firefox addons). It's a library available for Firefox's SDK. Google something like "Firefox Chrome CC", if you'd like more details.
This seems weird for me because that shouldn't be related to whatever we've changed.
Can you try reverting to an old commit and running the file again? Something older than the whole restructuring thing.
We might blacklist Debian for now, but again, that makes no sense. It's possibly some small thing that broke it all, but it doesn't make sense that it's connected to our restructuring.
One thing that " might" be possible is if Chromium package doesn't install the libraries this file requires. Did you try installing Google Chrome before running that script?
Yeah, it's actually pretty strange. I'll try to spin up a Debian VM later tonight and take a look.
Maybe we're specifying some Google Chrome dependencies, when we're installing chromium? (Although you'd think that would fail on Travis, since it doesn't have Chrome, just chromium...)
Ah, I think I understand it a bit more right now. That's simple, yet too confusing.
When you changed require Chrome to require Chromium you didn't push it forward, you crashed it before Chromium testing even had a chance to start. Therefore the Chromedriver doesn't exist message didn't even appear. You shouldn't change this file, it's perfectly correct.
The Chromedriver is missing message, on the other hand, is different and unrelated (although might seem so, confusingly). To fix that, we need to install chromedriver properly. Apparently we're not doing that when it comes to Debian.
Tl;Dr:
Revert the httpse-whatever file. It's original version is correct, and it's unrelated to the "Chromedriver is missing" message.
Write code to install Chromedriver correctly for Debian.
And then you'll be good to go.
Probably I should have been more precise, but the error about ChromeDriver not being installed was obtained after the Debian package chromedriver was installed.
I'll be home in an hour, I'll take a look at this, it'll be nice if you guys would be available on the IRC by then.
Can you try running Chromedriver manually through the terminal and confirming that it runs correctly? Can you check where was it installed? (I.e. chromedriver's path)
It's not critical now -- and the EFF folks are pretty busy with the "Let's Encrypt" push.
You could play with Docker a bit more -- and see if this is a Firefox issue. I see a bunch of threads on "Loop is not enabled" and "modules/loop/MozLoopService.jsm"
Yea, that was the first thing I guessed when I saw jsha's feed :/
On the other hand, I shared somewhere Travis CI's team reply regarding how difficult it was to install Chromium. You mentioned that sudo: false is supposed to enable docker mode, apparently, it doesn't.
I say that because of two things. That's what one of Travis CI guys said.
Our recently announced Docker infrastructure (http://docs.travis-ci.com/user/docker/) runs on Trusty. It can use "sudo", too, so you can check that out. Do note that some languages (PHP, for example), is not yet supported.
Here it mentions that we can actually use sudo there.
And second, I don't think we're running on Trusty.
Maybe if we added the services: - docker part (as mentioned in the link above) we'll be blessed by the holy light of Shuttleworth's Ubuntu Trusty?
I don't think it makes a ton of sense for us to make a Docker image for HTTPSe right now — it's a bunch of overhead and complexity to replace a few install lines.
Adding services: - docker would require us to build a docker image, add some docker pull/start/run lines, etc.
Just to chime in on the loop.enabled bit (since your issue here helped me in tackling an issue with mine)
The main issue is that while Firefox has loop.enabled set to true be default in the config, for reasons unknown, Iceweasel defaults to false (at least on jessie)
As far as why npm test checks the status of that config key (it will even do it with the default empty test) I have no clue.
Ran into a new problem in install-dev-dependencies.sh, apparently due to an issue with requests:
$ sudo bash install-dev-dependencies.sh
+ type apt-get
+ BROWSERS='firefox chromium-browser'
+ CHROMEDRIVER=chromium-chromedriver
++ lsb_release -is
+ [[ Debian == \D\e\b\i\a\n ]]
+ BROWSERS='iceweasel chromium'
+ CHROMEDRIVER=chromedriver
+ sudo apt-get install libxml2-dev libxml2-utils libxslt1-dev python-dev iceweasel chromium zip sqlite3 python-pip libcurl4-openssl-dev libssl-dev chromedriver
Indlæser pakkelisterne... Færdig
Opbygger afhængighedstræ
Læser tilstandsoplysninger... Færdig
libcurl4-openssl-dev er allerede den nyeste version.
libxml2-dev er allerede den nyeste version.
libxml2-utils er allerede den nyeste version.
libxslt1-dev er allerede den nyeste version.
libssl-dev er allerede den nyeste version.
python-dev er allerede den nyeste version.
python-pip er allerede den nyeste version.
sqlite3 er allerede den nyeste version.
zip er allerede den nyeste version.
chromedriver er allerede den nyeste version.
chromium er allerede den nyeste version.
iceweasel er allerede den nyeste version.
Følgende pakker blev installeret automatisk, og behøves ikke længere:
libfglrx:i386 libjs-sphinxdoc libxinerama1:i386 python-concurrent.futures python-livestreamer python-singledispatch rtmpdump
Brug »apt-get autoremove« til at fjerne dem.
0 opgraderes, 0 nyinstalleres, 0 afinstalleres og 1 opgraderes ikke.
+ git submodule init
+ git submodule update
+ pip install --user --no-allow-insecure --no-allow-external -r requirements.txt
Traceback (most recent call last):
File "/usr/bin/pip", line 9, in <module>
load_entry_point('pip==1.5.6', 'console_scripts', 'pip')()
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 356, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2476, in load_entry_point
return ep.load()
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2190, in load
['__name__'])
File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 74, in <module>
from pip.vcs import git, mercurial, subversion, bazaar # noqa
File "/usr/lib/python2.7/dist-packages/pip/vcs/mercurial.py", line 9, in <module>
from pip.download import path_to_url
File "/usr/lib/python2.7/dist-packages/pip/download.py", line 25, in <module>
from requests.compat import IncompleteRead
ImportError: cannot import name IncompleteRead
Yeah, came across the same one. It's not super local if it affects all Debian installations though; and python-pip is installed as part of the same bash script. I would try the fix to see if it could be modified to cover that, but that does not appear to be working at the moment:
$ sudo easy_install pip
Searching for pip
Reading https://pypi.python.org/simple/pip/
Best match: pip 7.1.2
Downloading https://pypi.python.org/packages/source/p/pip/pip-7.1.2.tar.gz#md5=3823d2343d9f3aaab21cf9c917710196
error: [Errno 104] Connection reset by peer
@TheNavigat: Nah, this was really unrelated to the old Debian related issue which persists (but which has a nonsensical fix). I'm not entirely happy with the solution to the new issue either: having people use a bunch of different package managers for rather simple tasks seems undesirable.
Regarding the new issue, I'm not sure how can we fix that on our own side. Not even sure whether it's a bug in some pip version, where if you install another version using easy_install it'd work.
Not sure if it's worth researching, since as I mentioned above, I'm not sure how can we fix it on our own side. Maybe require a specific version of pip? A little bit too much, right?
I haven't tested their interoperation but it seems like all required Python modules are in the Jessie repos, except from bsdiff4. A bit unfortunate. But yes, it is not the most pressing issue.
Activity
semenko commentedon Sep 6, 2015
Can you try rebasing to master? A bunch of this was fixed in #2779 -- though it won't fix the bug with
chromium-chromedriverin Debian(FYI @TheNavigat )
TheNavigat commentedon Sep 6, 2015
Can you possibly assign this issue to me?
I don't think chromedriver is a standard package and therefore might not exist in Debian's repositories. If there exists a package called chromedriver then that should be the same thing indeed, but we need to add a condition to install-dev-dependencies to either install the chromedriver package or download and install chromedriver manually.
I'm on a train right now and I don't have enough connectivity or space, so it's a bit too difficult for me to check Debian's repos right now. Sorry :(
semenko commentedon Sep 6, 2015
Sure -- chromium-chromedriver isn't in Debian -- it's just in Ubuntu it seems.
Perhaps do the same thing we've done in Travis (copy to testing path) and add
chromedriverto .gitignore?TheNavigat commentedon Sep 6, 2015
That is possible indeed.
In this specific case though we can use sudo and move the thing to /usr/bin which I believe is much better an option.
semenko commentedon Sep 6, 2015
Hm, I think some users are likely to be unhappy with us adding directly to their /usr/bin without a package manager.
fuglede commentedon Sep 6, 2015
Sorry, was testing this on a master from a couple of hours ago; hadn't seen the latest changes. Anyway, I'm still getting the same error off of the current master. Performing the 'chrome' -> 'chromium' change to
https-everywhere-tests/tests/test-httpse-installedin #2803, the test stalls atsemenko commentedon Sep 6, 2015
Gosh, that's a cool error!
has no authority to load: chromium@TheNavigat want to just blacklist Debian (or whitelist Ubuntu) for the Chromium tests until we get this sorted out?
TheNavigat commentedon Sep 6, 2015
@semenko Mmmm... Alright then, copying it is.
And no, that error is different. It doesn't actually require Chrome or Chromium, it requires Chrome library, which is a completely different thing AFAIK (I have some experience developing Firefox addons). It's a library available for Firefox's SDK. Google something like "Firefox Chrome CC", if you'd like more details.
This seems weird for me because that shouldn't be related to whatever we've changed.
Can you try reverting to an old commit and running the file again? Something older than the whole restructuring thing.
We might blacklist Debian for now, but again, that makes no sense. It's possibly some small thing that broke it all, but it doesn't make sense that it's connected to our restructuring.
One thing that " might" be possible is if Chromium package doesn't install the libraries this file requires. Did you try installing Google Chrome before running that script?
semenko commentedon Sep 6, 2015
Yeah, it's actually pretty strange. I'll try to spin up a Debian VM later tonight and take a look.
Maybe we're specifying some Google Chrome dependencies, when we're installing chromium? (Although you'd think that would fail on Travis, since it doesn't have Chrome, just chromium...)
TheNavigat commentedon Sep 6, 2015
Ah, I think I understand it a bit more right now. That's simple, yet too confusing.
When you changed require Chrome to require Chromium you didn't push it forward, you crashed it before Chromium testing even had a chance to start. Therefore the Chromedriver doesn't exist message didn't even appear. You shouldn't change this file, it's perfectly correct.
The Chromedriver is missing message, on the other hand, is different and unrelated (although might seem so, confusingly). To fix that, we need to install chromedriver properly. Apparently we're not doing that when it comes to Debian.
Tl;Dr:
Revert the httpse-whatever file. It's original version is correct, and it's unrelated to the "Chromedriver is missing" message.
Write code to install Chromedriver correctly for Debian.
And then you'll be good to go.
fuglede commentedon Sep 6, 2015
Probably I should have been more precise, but the error about ChromeDriver not being installed was obtained after the Debian package chromedriver was installed.
TheNavigat commentedon Sep 6, 2015
I'll be home in an hour, I'll take a look at this, it'll be nice if you guys would be available on the IRC by then.
Can you try running Chromedriver manually through the terminal and confirming that it runs correctly? Can you check where was it installed? (I.e. chromedriver's path)
fuglede commentedon Sep 6, 2015
The package produces an executable called
chromedriverin/usr/lib/chromium:semenko commentedon Sep 6, 2015
Different than: https://github.com/EFForg/https-everywhere/blob/master/test/chromium/script.py#L37
We should clean up those path tests.
16 remaining items
semenko commentedon Sep 11, 2015
It's not critical now -- and the EFF folks are pretty busy with the "Let's Encrypt" push.
You could play with Docker a bit more -- and see if this is a Firefox issue. I see a bunch of threads on "Loop is not enabled" and "modules/loop/MozLoopService.jsm"
TheNavigat commentedon Sep 11, 2015
Yea, that was the first thing I guessed when I saw jsha's feed :/
On the other hand, I shared somewhere Travis CI's team reply regarding how difficult it was to install Chromium. You mentioned that
sudo: falseis supposed to enable docker mode, apparently, it doesn't.I say that because of two things. That's what one of Travis CI guys said.
Our recently announced Docker infrastructure (http://docs.travis-ci.com/user/docker/) runs on Trusty. It can use "sudo", too, so you can check that out. Do note that some languages (PHP, for example), is not yet supported.Here it mentions that we can actually use sudo there.
And second, I don't think we're running on Trusty.
Maybe if we added the
services: - dockerpart (as mentioned in the link above) we'll be blessed by the holy light of Shuttleworth's Ubuntu Trusty?semenko commentedon Sep 11, 2015
I don't think that's correct -- I think you might be confused about the Docker phrasing.
We don't have a Docker image that we're testing on. We are, however, running on their containerized infrastructure, which runs on Docker. See the start of every log:
https://travis-ci.org/EFForg/https-everywhere#L165
and this post:
http://blog.travis-ci.com/2014-12-17-faster-builds-with-container-based-infrastructure/
I don't think it makes a ton of sense for us to make a Docker image for HTTPSe right now — it's a bunch of overhead and complexity to replace a few install lines.
Adding
services: - dockerwould require us to build a docker image, add some docker pull/start/run lines, etc.TheNavigat commentedon Sep 11, 2015
Oh, I see. I guess you're right, then.
drazisil commentedon Oct 10, 2015
Just to chime in on the loop.enabled bit (since your issue here helped me in tackling an issue with mine)
The main issue is that while Firefox has loop.enabled set to true be default in the config, for reasons unknown, Iceweasel defaults to false (at least on jessie)
As far as why npm test checks the status of that config key (it will even do it with the default empty test) I have no clue.
fuglede commentedon Oct 19, 2015
Ran into a new problem in
install-dev-dependencies.sh, apparently due to an issue withrequests:semenko commentedon Oct 19, 2015
Hm, any chance this is a local pip problem?
See: https://stackoverflow.com/questions/27341064/how-do-i-fix-importerror-cannot-import-name-incompleteread/27341847#27341847
& proposed solution:
fuglede commentedon Oct 19, 2015
Yeah, came across the same one. It's not super local if it affects all Debian installations though; and
python-pipis installed as part of the same bash script. I would try the fix to see if it could be modified to cover that, but that does not appear to be working at the moment:Edit: Alright, it works now, and does the job.
TheNavigat commentedon Oct 21, 2015
So, everything works now?
fuglede commentedon Oct 21, 2015
@TheNavigat: Nah, this was really unrelated to the old Debian related issue which persists (but which has a nonsensical fix). I'm not entirely happy with the solution to the new issue either: having people use a bunch of different package managers for rather simple tasks seems undesirable.
TheNavigat commentedon Oct 21, 2015
Regarding the new issue, I'm not sure how can we fix that on our own side. Not even sure whether it's a bug in some pip version, where if you install another version using easy_install it'd work.
Not sure if it's worth researching, since as I mentioned above, I'm not sure how can we fix it on our own side. Maybe require a specific version of pip? A little bit too much, right?
fuglede commentedon Oct 21, 2015
I haven't tested their interoperation but it seems like all required Python modules are in the Jessie repos, except from
bsdiff4. A bit unfortunate. But yes, it is not the most pressing issue.Hainish commentedon May 24, 2017
This appears to be fixed in the latest, closing