You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Nov 6, 2023. It is now read-only.
Repository navigation
This repository was archived by the owner on Nov 6, 2023. It is now read-only.
Dev install: "ChromeDriver isn't installed" error on Debian 9.2 #13597
I'm on Debian 9.2 and successfully ran the install-dev-dependencies.sh script. But running the test script gave me this error:
ChromeDriver isn't installed. Check test/chromium/README.md for instructions on how to install ChromeDriver
I had installed chromedriver successfully:
$ which chromedriver
/usr/bin/chromedriver
$ /usr/bin/chromedriver --version
ChromeDriver 2.32 (undefined)
$ dpkg -l | grep chromedriver
ii chromedriver 62.0.3202.89-1~deb9u1 amd64 web browser - WebDriver support transitional package
I dug around in the commit log and found that we used to have clearer instructions about ChromeDriver in that README.md file, in b3965a1. Based on that, I created a symbolic link:
and after that I was able to successfully run the tests.
So I think we need to improve test/chromium/README.md to put those troubleshooting directions back, and potentially to fix install-dev-dependencies.sh to put that dependency in the right place in the filesystem.
It's possible the expected location of chromedriver on Debian changed from v8 to v9.
The only build platforms that see any regular use I think are Ubuntu and OS X. A better approach for now would be to only support those two platforms and remove references to anything else from the documentation and build process. A long-term fix would be to remake the build process to use Node for everything as @koops76 suggests in #13512.
I can confirm based on testing using the debian:8 and debian:9 docker images that the location of chromedriver has indeed changed between 8 and 9 from /usr/lib/chromium/chromedriver to /usr/bin/chromedriver
@jeremyn:
I do develop from Debian 8 within Qubes, and I think we should continue support for all the typical flavors of Linux (for instance, in install-dev-dependencies.sh we reference dnf for Fedora support). In my experience, it creates a more welcoming atmosphere for developers if we try to support a wide range of typical development environments. Acknowledging that "wide" and "typical" are terms at ends with each other, I feel there is a balance to be struck here.
@Hainish The commit b3965a1 that @brainwane refers to is from PR #5584 that I made trying to get the build to work on Fedora last year. Some of my earliest PRs to this project (#5584, #6215) were to fix weird Fedora-related problems. Eventually by #5584 (comment) you can see that I settled into using Ubuntu. So in my personal experience, having a half-supported Fedora option was the opposite of welcoming, except insofar as it made me dive into the deep end almost immediately. I probably made the problem worse by trying to fix the build process on Fedora, because it's probably out of date by now, and someone else will come along thinking that it works when it doesn't and go through it all again.
@koops76 do you know of anyone setting up the dev environment on Windows 10? I set it up using Ubuntu run natively on Windows 10. I was also planning to try the set-up process on Mac.
Is there any interest in making a statement somewhere saying what platforms are most supported, which are sort-of supported, and which are not at all supported? For example Ubuntu is most supported since it's the base of all the Travis tests; Fedora is sort-of supported in that it (with #13686) technically installs but the tests might not work; Windows 10 (without WSL) is not supported at all because it simply doesn't support needed functionality like Bash-created file symlinks.
Come to think of it, we should probably create a list of platforms we wish to be supported, and then use that as a guide for which platforms we should actively work on to support.
In other words, there's a distinction that should be made between platforms we support but are broken, and platforms we don't wish to support at all. Fedora is one I'd like to support, since it's not an uncommon platform for developers. macOS is another platform I believe we should support, but I'm not sure what the current status is.
I'd like to wait until we do a little more work on coming up with this list and then actively working on getting each platform in the list supported before we affirmatively say (in CONTRIBUTING.md, for instance) that these are the platforms supported. The reason is that we may find along the way that there is an impasse that makes it impossible to support certain distros, or makes the maintenance cost preventatively high. It would be unfortunate, in those cases, to have to backtrack our statements.
My HTTPS Everywhere dev environment is an Ubuntu 16.04 VM without a graphical interface. (It comes from https://github.com/jeremyn/https-everywhere-dev-environment but that is not important here.) I ssh into it and do stuff there. It could be Debian, Fedora, anything *nix-based and it would be about the same experience for me.
I edit with vim. The HTTPS Everywhere code is not very complicated and doesn't require IDE support, though if it did or someone wanted it, the mainstream IDEs can edit remote files. Nowadays someone could even do ruleset development using just the new CodeMirror-based debugging tool and creating PRs in the browser.
It's unclear to me what people are doing such that having e.g. Fedora support or not is a dealbreaker. If someone wants to put a bunch of time into Fedora support as a personal project, okay, sure, go for it. It has to work with Ubuntu in the end anyway or it won't pass the tests. But I don't think building out Debian, Fedora, or macOS support is a good use of official time or will have a good "return on investment".
An advantage of reducing our number of supported platforms is that we could use system-provided libraries instead of stuff from pip. I do this in https://github.com/jeremyn/ansible-role-https-everywhere (used in my dev environment) for everything except two packages. In our current top-level requirements.txt we don't do hash-checking or even pin versions.
@jeremyn even within a specific distribution, different releases will have different versions for system-level libraries. So unless we're very strict about the development environment, we couldn't rely on users having the exact same dependency versions.
For whatever reason, developers often find it annoying, cumbersome, or downright impossible to set up a new VM for every project they develop for. This could be because of hard disk space or RAM limitations. If you're developing on an ARM Chromebook, most hypervisors will simply not run. These are especially important issues for developers facing low-resource situations. I don't want to discourage them from contributing to the project. And if they are discouraged and leave, we may never know - in effect we'd be unwittingly subjecting ourselves to a selection bias.
Additionally, facilitating a quick path from project discovery to contribution encourages contributions from developers who may not be familiar with VM or container technologies, or don't have the time to set those environments up. And while I sympathize with having a unified ecosystem and don't wish to do debugging for various different platforms, I do think it's important to have support for at least the most common developer use cases.
These issues make it all the more important to cast a wide net, so that a diversity of development environments can get up and running quickly.
I support making the development environment accessible to as many people as possible. I think there are multiple ways to do this of varying levels of effectiveness.
The new CodeMirror tool is a big step forward, because as I said in #13597 (comment), now people can create and submit rulesets entirely in the browser. Rulesets are what most people who contribute, do. We could improve that experience.
By far, most people in the world use Windows. This is I think even more true in the low-computing-power developing world than in the developed world. Right now you can't develop HTTPS Everywhere in Windows without using WSL. We could improve this experience. There's some low-hanging Windows-hostile fruit that we could fix, such as the file symlink from rules/ to src/chrome/content/rules/. Long-term we could use Node for the build environment to make things cross-platform.
I don't know what macOS support is like but there are a lot of developers who use it, of course, so it could use some attention.
For people with very low computing resources such that creating a VM with 10GB disk space with 1GB RAM is out of reach, we could put together instructions for developing in free-tier AWS/GCP/Azure. This is probably the best option for your hypothetical Chromebook user. Another option would be improving Raspberry Pi (Raspbian) support.
Way down on the list is putting time into supporting Linux distros with six-month release cycles like Fedora. We might get some contributors that way but that time would be, I think, better spent elsewhere.
I tried the dev set-up on a borrowed Mac and got it to work but the tests fail #13794 . I get the feeling that issues I'm running into might not be difficult for someone familiar with Mac dev to get working. The set-up did make some assumptions about the environment such as Homebrew being installed, so to make it really easy we'd need to think about listing some prerequisites prior to installing or possibly installing Homebrew in the script.
I like @Hainish's suggestion of creating a list of environments that are supported and those we wish to support. We might be able to enlist some help from those familiar with those environments to get the support in place.
@wonderchook recently in 9e48776 I did add a warning that the installation script may alter one's system. We may want to add an additional dialogue for macOS users, asking them if it's okay to install brew. Currently, we use brew to determine if it is a macOS system at all, I think this is a pretty weak condition. Stronger would be to cat /etc/issue and inspect the contents there.
Activity
brainwane commentedon Nov 14, 2017
(A note about who I am & what you'll see from @wonderchook and me in the next several weeks in this repo.)
jeremyn commentedon Nov 15, 2017
Welcome @brainwane and @wonderchook!
It's possible the expected location of
chromedriveron Debian changed from v8 to v9.The only build platforms that see any regular use I think are Ubuntu and OS X. A better approach for now would be to only support those two platforms and remove references to anything else from the documentation and build process. A long-term fix would be to remake the build process to use Node for everything as @koops76 suggests in #13512.
Hainish commentedon Nov 15, 2017
I can confirm based on testing using the
debian:8anddebian:9docker images that the location ofchromedriverhas indeed changed between8and9from/usr/lib/chromium/chromedriverto/usr/bin/chromedriver@jeremyn:
I do develop from Debian 8 within Qubes, and I think we should continue support for all the typical flavors of Linux (for instance, in
install-dev-dependencies.shwe referencednffor Fedora support). In my experience, it creates a more welcoming atmosphere for developers if we try to support a wide range of typical development environments. Acknowledging that "wide" and "typical" are terms at ends with each other, I feel there is a balance to be struck here.jeremyn commentedon Nov 15, 2017
@Hainish The commit b3965a1 that @brainwane refers to is from PR #5584 that I made trying to get the build to work on Fedora last year. Some of my earliest PRs to this project (#5584, #6215) were to fix weird Fedora-related problems. Eventually by #5584 (comment) you can see that I settled into using Ubuntu. So in my personal experience, having a half-supported Fedora option was the opposite of welcoming, except insofar as it made me dive into the deep end almost immediately. I probably made the problem worse by trying to fix the build process on Fedora, because it's probably out of date by now, and someone else will come along thinking that it works when it doesn't and go through it all again.
wonderchook commentedon Nov 20, 2017
@koops76 do you know of anyone setting up the dev environment on Windows 10? I set it up using Ubuntu run natively on Windows 10. I was also planning to try the set-up process on Mac.
jeremyn commentedon Nov 20, 2017
@wonderchook By "Ubuntu run natively on Windows 10" do you mean with the Windows Subsystem for Linux?
wonderchook commentedon Nov 20, 2017
@jeremyn yes correct.
jeremyn commentedon Nov 20, 2017
@wonderchook Thanks for confirming, that's pretty neat.
Fixing some dev dependency requirements for Debian 9 as well as Fedor…
Fixing some dev dependency requirements for Debian 9 as well as Fedor…
jeremyn commentedon Nov 21, 2017
Is there any interest in making a statement somewhere saying what platforms are most supported, which are sort-of supported, and which are not at all supported? For example Ubuntu is most supported since it's the base of all the Travis tests; Fedora is sort-of supported in that it (with #13686) technically installs but the tests might not work; Windows 10 (without WSL) is not supported at all because it simply doesn't support needed functionality like Bash-created file symlinks.
Hainish commentedon Nov 21, 2017
I think that's sensible, I'll draft something up.
Fixing some dev dependency requirements for Debian 9 as well as Fedor…
Hainish commentedon Nov 22, 2017
Come to think of it, we should probably create a list of platforms we wish to be supported, and then use that as a guide for which platforms we should actively work on to support.
In other words, there's a distinction that should be made between platforms we support but are broken, and platforms we don't wish to support at all. Fedora is one I'd like to support, since it's not an uncommon platform for developers. macOS is another platform I believe we should support, but I'm not sure what the current status is.
I'd like to wait until we do a little more work on coming up with this list and then actively working on getting each platform in the list supported before we affirmatively say (in
CONTRIBUTING.md, for instance) that these are the platforms supported. The reason is that we may find along the way that there is an impasse that makes it impossible to support certain distros, or makes the maintenance cost preventatively high. It would be unfortunate, in those cases, to have to backtrack our statements.jeremyn commentedon Nov 22, 2017
My HTTPS Everywhere dev environment is an Ubuntu 16.04 VM without a graphical interface. (It comes from https://github.com/jeremyn/https-everywhere-dev-environment but that is not important here.) I ssh into it and do stuff there. It could be Debian, Fedora, anything *nix-based and it would be about the same experience for me.
I edit with vim. The HTTPS Everywhere code is not very complicated and doesn't require IDE support, though if it did or someone wanted it, the mainstream IDEs can edit remote files. Nowadays someone could even do ruleset development using just the new CodeMirror-based debugging tool and creating PRs in the browser.
It's unclear to me what people are doing such that having e.g. Fedora support or not is a dealbreaker. If someone wants to put a bunch of time into Fedora support as a personal project, okay, sure, go for it. It has to work with Ubuntu in the end anyway or it won't pass the tests. But I don't think building out Debian, Fedora, or macOS support is a good use of official time or will have a good "return on investment".
jeremyn commentedon Nov 22, 2017
An advantage of reducing our number of supported platforms is that we could use system-provided libraries instead of stuff from
pip. I do this in https://github.com/jeremyn/ansible-role-https-everywhere (used in my dev environment) for everything except two packages. In our current top-levelrequirements.txtwe don't do hash-checking or even pin versions.Hainish commentedon Nov 28, 2017
@jeremyn even within a specific distribution, different releases will have different versions for system-level libraries. So unless we're very strict about the development environment, we couldn't rely on users having the exact same dependency versions.
For whatever reason, developers often find it annoying, cumbersome, or downright impossible to set up a new VM for every project they develop for. This could be because of hard disk space or RAM limitations. If you're developing on an ARM Chromebook, most hypervisors will simply not run. These are especially important issues for developers facing low-resource situations. I don't want to discourage them from contributing to the project. And if they are discouraged and leave, we may never know - in effect we'd be unwittingly subjecting ourselves to a selection bias.
Additionally, facilitating a quick path from project discovery to contribution encourages contributions from developers who may not be familiar with VM or container technologies, or don't have the time to set those environments up. And while I sympathize with having a unified ecosystem and don't wish to do debugging for various different platforms, I do think it's important to have support for at least the most common developer use cases.
These issues make it all the more important to cast a wide net, so that a diversity of development environments can get up and running quickly.
jeremyn commentedon Nov 28, 2017
I support making the development environment accessible to as many people as possible. I think there are multiple ways to do this of varying levels of effectiveness.
The new CodeMirror tool is a big step forward, because as I said in #13597 (comment), now people can create and submit rulesets entirely in the browser. Rulesets are what most people who contribute, do. We could improve that experience.
By far, most people in the world use Windows. This is I think even more true in the low-computing-power developing world than in the developed world. Right now you can't develop HTTPS Everywhere in Windows without using WSL. We could improve this experience. There's some low-hanging Windows-hostile fruit that we could fix, such as the file symlink from
rules/tosrc/chrome/content/rules/. Long-term we could use Node for the build environment to make things cross-platform.I don't know what macOS support is like but there are a lot of developers who use it, of course, so it could use some attention.
For people with very low computing resources such that creating a VM with 10GB disk space with 1GB RAM is out of reach, we could put together instructions for developing in free-tier AWS/GCP/Azure. This is probably the best option for your hypothetical Chromebook user. Another option would be improving Raspberry Pi (Raspbian) support.
We could improve the VM creation process. My https://github.com/jeremyn/https-everywhere-dev-environment tries to do this -- you create a VM, set up some Ansible configuration and then run an Ansible playbook on the VM -- but it's not perfect.
Way down on the list is putting time into supporting Linux distros with six-month release cycles like Fedora. We might get some contributors that way but that time would be, I think, better spent elsewhere.
wonderchook commentedon Nov 28, 2017
I tried the dev set-up on a borrowed Mac and got it to work but the tests fail #13794 . I get the feeling that issues I'm running into might not be difficult for someone familiar with Mac dev to get working. The set-up did make some assumptions about the environment such as Homebrew being installed, so to make it really easy we'd need to think about listing some prerequisites prior to installing or possibly installing Homebrew in the script.
I like @Hainish's suggestion of creating a list of environments that are supported and those we wish to support. We might be able to enlist some help from those familiar with those environments to get the support in place.
Hainish commentedon Nov 28, 2017
@wonderchook recently in 9e48776 I did add a warning that the installation script may alter one's system. We may want to add an additional dialogue for macOS users, asking them if it's okay to install
brew. Currently, we usebrewto determine if it is a macOS system at all, I think this is a pretty weak condition. Stronger would be to cat/etc/issueand inspect the contents there.brainwane commentedon Nov 29, 2017
Hey, since this particular issue's closed, could we continue the question of what platforms to support in #13815?
Hainish commentedon Nov 29, 2017
@brainwane thank you, that makes sense to me.