Skip to content
This repository was archived by the owner on Nov 6, 2023. It is now read-only.
This repository was archived by the owner on Nov 6, 2023. It is now read-only.

install-dev-dependencies.sh and test.sh fail on Debian Jessie #2801

Closed

Description

@fuglede

Currently, install-dev-dependencies.sh attempts to install chromium-chromedriver which is not part of the standard repositories. There is a package called chromedriver which perhaps(?) does the same thing.

The same script attempts to access test/chrome/ which does not exist. The folder test/chromium/ does, though.

Even with those two changes, test.sh still gives the following error:

ChromeDriver isn't installed. Check test/chrome/README.mdfor instructions on how to install ChromeDriver

Activity

  1. semenko commented on Sep 6, 2015

    @semenko
    Contributor

    Can you try rebasing to master? A bunch of this was fixed in #2779 -- though it won't fix the bug with chromium-chromedriver in Debian

    (FYI @TheNavigat )

  2. TheNavigat commented on Sep 6, 2015

    @TheNavigat
    Contributor

    Can you possibly assign this issue to me?

    I don't think chromedriver is a standard package and therefore might not exist in Debian's repositories. If there exists a package called chromedriver then that should be the same thing indeed, but we need to add a condition to install-dev-dependencies to either install the chromedriver package or download and install chromedriver manually.

    I'm on a train right now and I don't have enough connectivity or space, so it's a bit too difficult for me to check Debian's repos right now. Sorry :(

  3. semenko commented on Sep 6, 2015

    @semenko
    Contributor

    Sure -- chromium-chromedriver isn't in Debian -- it's just in Ubuntu it seems.

    Perhaps do the same thing we've done in Travis (copy to testing path) and add chromedriver to .gitignore?

  4. TheNavigat commented on Sep 6, 2015

    @TheNavigat
    Contributor

    That is possible indeed.

    In this specific case though we can use sudo and move the thing to /usr/bin which I believe is much better an option.

  5. semenko commented on Sep 6, 2015

    @semenko
    Contributor

    Hm, I think some users are likely to be unhappy with us adding directly to their /usr/bin without a package manager.

  6. fuglede commented on Sep 6, 2015

    @fuglede
    ContributorAuthor

    Sorry, was testing this on a master from a couple of hours ago; hadn't seen the latest changes. Anyway, I'm still getting the same error off of the current master. Performing the 'chrome' -> 'chromium' change to https-everywhere-tests/tests/test-httpse-installed in #2803, the test stalls at

    Full Message: Error: Module: https-everywhere-tests/tests/test-httpse-installed has no authority to load: chromium
    Full Stack: @https-everywhere-tests/tests/test-httpse-installed:58:undefined
    resolve@resource://gre/modules/commonjs/sdk/loader/cuddlefish.js:58:1
    @resource://jid1-we5bqofc6sksmg-at-jetpack/https-everywhere-tests/tests/test-httpse-installed.js:3:20
    CuddlefishLoader/options<.load@resource://gre/modules/commonjs/sdk/loader/cuddlefish.js:79:18
    findTests/</getNextTest@resource://gre/modules/commonjs/sdk/deprecated/unit-test-finder.js:164:25
    findTests/</<.getNext@resource://gre/modules/commonjs/sdk/deprecated/unit-test-finder.js:197:32
    runNextTest@resource://gre/modules/commonjs/sdk/deprecated/unit-test.js:516:14
    startMany@resource://gre/modules/commonjs/sdk/deprecated/unit-test.js:530:12
    findAndRunTests/<@resource://gre/modules/commonjs/sdk/deprecated/unit-test.js:28:1
    Handler.prototype.process@resource://gre/modules/Promise-backend.js:867:23
    this.PromiseWalker.walkerLoop@resource://gre/modules/Promise-backend.js:746:7
    this.PromiseWalker.scheduleWalkerLoop/<@resource://gre/modules/Promise-backend.js:688:37
    " {file: "https-everywhere-tests/tests/test-httpse-installed" line: 58 column: 0 source: "58"}]
    
  7. semenko commented on Sep 6, 2015

    @semenko
    Contributor

    Gosh, that's a cool error! has no authority to load: chromium

    @TheNavigat want to just blacklist Debian (or whitelist Ubuntu) for the Chromium tests until we get this sorted out?

  8. TheNavigat commented on Sep 6, 2015

    @TheNavigat
    Contributor

    @semenko Mmmm... Alright then, copying it is.

    And no, that error is different. It doesn't actually require Chrome or Chromium, it requires Chrome library, which is a completely different thing AFAIK (I have some experience developing Firefox addons). It's a library available for Firefox's SDK. Google something like "Firefox Chrome CC", if you'd like more details.

    This seems weird for me because that shouldn't be related to whatever we've changed.

    Can you try reverting to an old commit and running the file again? Something older than the whole restructuring thing.

    We might blacklist Debian for now, but again, that makes no sense. It's possibly some small thing that broke it all, but it doesn't make sense that it's connected to our restructuring.

    One thing that " might" be possible is if Chromium package doesn't install the libraries this file requires. Did you try installing Google Chrome before running that script?

  9. semenko commented on Sep 6, 2015

    @semenko
    Contributor

    Yeah, it's actually pretty strange. I'll try to spin up a Debian VM later tonight and take a look.

    Maybe we're specifying some Google Chrome dependencies, when we're installing chromium? (Although you'd think that would fail on Travis, since it doesn't have Chrome, just chromium...)

  10. TheNavigat commented on Sep 6, 2015

    @TheNavigat
    Contributor

    Ah, I think I understand it a bit more right now. That's simple, yet too confusing.

    When you changed require Chrome to require Chromium you didn't push it forward, you crashed it before Chromium testing even had a chance to start. Therefore the Chromedriver doesn't exist message didn't even appear. You shouldn't change this file, it's perfectly correct.

    The Chromedriver is missing message, on the other hand, is different and unrelated (although might seem so, confusingly). To fix that, we need to install chromedriver properly. Apparently we're not doing that when it comes to Debian.

    Tl;Dr:

    Revert the httpse-whatever file. It's original version is correct, and it's unrelated to the "Chromedriver is missing" message.
    Write code to install Chromedriver correctly for Debian.
    And then you'll be good to go.

  11. fuglede commented on Sep 6, 2015

    @fuglede
    ContributorAuthor

    Probably I should have been more precise, but the error about ChromeDriver not being installed was obtained after the Debian package chromedriver was installed.

  12. TheNavigat commented on Sep 6, 2015

    @TheNavigat
    Contributor

    I'll be home in an hour, I'll take a look at this, it'll be nice if you guys would be available on the IRC by then.

    Can you try running Chromedriver manually through the terminal and confirming that it runs correctly? Can you check where was it installed? (I.e. chromedriver's path)

  13. fuglede commented on Sep 6, 2015

    @fuglede
    ContributorAuthor

    The package produces an executable called chromedriverin /usr/lib/chromium:

    $ sudo find / -name 'chromedriver'
    /usr/share/doc/chromedriver
    /usr/lib/chromium/chromedriver
    $ apt-file search /usr/lib/chromium/chromedriver 
    chromedriver: /usr/lib/chromium/chromedriver
    $ /usr/lib/chromium/chromedriver
    Starting ChromeDriver 2.16 (undefined) on port 9515
    Only local connections are allowed.
    
  14. semenko commented on Sep 6, 2015

    @semenko
    Contributor
  15. 16 remaining items

  16. semenko commented on Sep 11, 2015

    @semenko
    Contributor

    It's not critical now -- and the EFF folks are pretty busy with the "Let's Encrypt" push.

    You could play with Docker a bit more -- and see if this is a Firefox issue. I see a bunch of threads on "Loop is not enabled" and "modules/loop/MozLoopService.jsm"

  17. TheNavigat commented on Sep 11, 2015

    @TheNavigat
    Contributor

    Yea, that was the first thing I guessed when I saw jsha's feed :/

    On the other hand, I shared somewhere Travis CI's team reply regarding how difficult it was to install Chromium. You mentioned that sudo: false is supposed to enable docker mode, apparently, it doesn't.

    I say that because of two things. That's what one of Travis CI guys said.

    Our recently announced Docker infrastructure (http://docs.travis-ci.com/user/docker/) runs on Trusty. It can use "sudo", too, so you can check that out. Do note that some languages (PHP, for example), is not yet supported.

    Here it mentions that we can actually use sudo there.

    And second, I don't think we're running on Trusty.

    Maybe if we added the services: - docker part (as mentioned in the link above) we'll be blessed by the holy light of Shuttleworth's Ubuntu Trusty?

  18. semenko commented on Sep 11, 2015

    @semenko
    Contributor

    I don't think that's correct -- I think you might be confused about the Docker phrasing.

    We don't have a Docker image that we're testing on. We are, however, running on their containerized infrastructure, which runs on Docker. See the start of every log:
    https://travis-ci.org/EFForg/https-everywhere#L165
    and this post:
    http://blog.travis-ci.com/2014-12-17-faster-builds-with-container-based-infrastructure/

    I don't think it makes a ton of sense for us to make a Docker image for HTTPSe right now — it's a bunch of overhead and complexity to replace a few install lines.

    Adding services: - docker would require us to build a docker image, add some docker pull/start/run lines, etc.

  19. TheNavigat commented on Sep 11, 2015

    @TheNavigat
    Contributor

    Oh, I see. I guess you're right, then.

  20. drazisil commented on Oct 10, 2015

    @drazisil

    Just to chime in on the loop.enabled bit (since your issue here helped me in tackling an issue with mine)

    The main issue is that while Firefox has loop.enabled set to true be default in the config, for reasons unknown, Iceweasel defaults to false (at least on jessie)

    As far as why npm test checks the status of that config key (it will even do it with the default empty test) I have no clue.

  21. fuglede commented on Oct 19, 2015

    @fuglede
    ContributorAuthor

    Ran into a new problem in install-dev-dependencies.sh, apparently due to an issue with requests:

    $ sudo bash install-dev-dependencies.sh 
    + type apt-get
    + BROWSERS='firefox chromium-browser'
    + CHROMEDRIVER=chromium-chromedriver
    ++ lsb_release -is
    + [[ Debian == \D\e\b\i\a\n ]]
    + BROWSERS='iceweasel chromium'
    + CHROMEDRIVER=chromedriver
    + sudo apt-get install libxml2-dev libxml2-utils libxslt1-dev python-dev iceweasel chromium zip sqlite3 python-pip libcurl4-openssl-dev libssl-dev chromedriver
    Indlæser pakkelisterne... Færdig
    Opbygger afhængighedstræ        
    Læser tilstandsoplysninger... Færdig
    libcurl4-openssl-dev er allerede den nyeste version.
    libxml2-dev er allerede den nyeste version.
    libxml2-utils er allerede den nyeste version.
    libxslt1-dev er allerede den nyeste version.
    libssl-dev er allerede den nyeste version.
    python-dev er allerede den nyeste version.
    python-pip er allerede den nyeste version.
    sqlite3 er allerede den nyeste version.
    zip er allerede den nyeste version.
    chromedriver er allerede den nyeste version.
    chromium er allerede den nyeste version.
    iceweasel er allerede den nyeste version.
    Følgende pakker blev installeret automatisk, og behøves ikke længere:
      libfglrx:i386 libjs-sphinxdoc libxinerama1:i386 python-concurrent.futures python-livestreamer python-singledispatch rtmpdump
    Brug »apt-get autoremove« til at fjerne dem.
    0 opgraderes, 0 nyinstalleres, 0 afinstalleres og 1 opgraderes ikke.
    + git submodule init
    + git submodule update
    + pip install --user --no-allow-insecure --no-allow-external -r requirements.txt
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        load_entry_point('pip==1.5.6', 'console_scripts', 'pip')()
      File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 356, in load_entry_point
        return get_distribution(dist).load_entry_point(group, name)
      File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2476, in load_entry_point
        return ep.load()
      File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2190, in load
        ['__name__'])
      File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 74, in <module>
        from pip.vcs import git, mercurial, subversion, bazaar  # noqa
      File "/usr/lib/python2.7/dist-packages/pip/vcs/mercurial.py", line 9, in <module>
        from pip.download import path_to_url
      File "/usr/lib/python2.7/dist-packages/pip/download.py", line 25, in <module>
        from requests.compat import IncompleteRead
    ImportError: cannot import name IncompleteRead
    
  22. semenko commented on Oct 19, 2015

    @semenko
    Contributor

    Hm, any chance this is a local pip problem?

    See: https://stackoverflow.com/questions/27341064/how-do-i-fix-importerror-cannot-import-name-incompleteread/27341847#27341847

    & proposed solution:

    apt-get remove python-pip
    easy_install pip
    
  23. fuglede commented on Oct 19, 2015

    @fuglede
    ContributorAuthor

    Yeah, came across the same one. It's not super local if it affects all Debian installations though; and python-pip is installed as part of the same bash script. I would try the fix to see if it could be modified to cover that, but that does not appear to be working at the moment:

    $ sudo easy_install pip
    Searching for pip
    Reading https://pypi.python.org/simple/pip/
    Best match: pip 7.1.2
    Downloading https://pypi.python.org/packages/source/p/pip/pip-7.1.2.tar.gz#md5=3823d2343d9f3aaab21cf9c917710196
    error: [Errno 104] Connection reset by peer
    

    Edit: Alright, it works now, and does the job.

  24. TheNavigat commented on Oct 21, 2015

    @TheNavigat
    Contributor

    So, everything works now?

  25. fuglede commented on Oct 21, 2015

    @fuglede
    ContributorAuthor

    @TheNavigat: Nah, this was really unrelated to the old Debian related issue which persists (but which has a nonsensical fix). I'm not entirely happy with the solution to the new issue either: having people use a bunch of different package managers for rather simple tasks seems undesirable.

  26. TheNavigat commented on Oct 21, 2015

    @TheNavigat
    Contributor

    Regarding the new issue, I'm not sure how can we fix that on our own side. Not even sure whether it's a bug in some pip version, where if you install another version using easy_install it'd work.

    Not sure if it's worth researching, since as I mentioned above, I'm not sure how can we fix it on our own side. Maybe require a specific version of pip? A little bit too much, right?

  27. fuglede commented on Oct 21, 2015

    @fuglede
    ContributorAuthor

    I haven't tested their interoperation but it seems like all required Python modules are in the Jessie repos, except from bsdiff4. A bit unfortunate. But yes, it is not the most pressing issue.

  28. Hainish commented on May 24, 2017

    @Hainish
    Member

    This appears to be fixed in the latest, closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions