Skip to content

[BUG] dash_duo does not set remote thus fails when get_webdriver is called #829

New issue
New issue
Closed
#894
Closed
[BUG] dash_duo does not set remote thus fails when get_webdriver is called#829
#894
Assignees
byronz
Milestone
Dash v1.3.0

Description

@Oronis
Oronis
opened on Jul 19, 2019

Describe your context

dash                                           1.0.2
dash-core-components                           1.0.0
dash-html-components                           1.0.0
dash-renderer                                  1.0.0
dash-table                                     4.0.2
  • OS: Windows 10
  • Browser Chrome
  • Version 75.0.3770.142

Describe the bug

Trying to use new dash.testing. Running the example code copied from the documentation (https://dash.plot.ly/testing) in a headless mode (with a Chrome webdriver running).

An error gets thrown in

dash\testing\browser.py:47: in __init__
    self._driver = until(lambda: self.get_webdriver(remote), timeout=1)

It appears that the variable remote is never set in the call stack of dash_duo and thus the call to get_webdriver always fails.

Expected behavior

It should work. I don't know what else to say beyond the basic example doesn't appear to work. If there is some setup requirement that is missing, then a clarification on the documentation needs to be added, otherwise at some point in the setup of dash_duo the value of remote needs to be set such that the get_webdriver call doesn't fail and it correctly fetches the webdriver when initialising the browser.

Activity

  1. Oronis
    added
    Type: Bug
    on Jul 19, 2019

  2. byronz commented on Jul 19, 2019

    @byronz
    byronz
    on Jul 19, 2019
    Contributor

    @Oronis thank you for reporting the dash.testing issue in details. It looks like an extra option --disable-gpu is needed on Windows OS.

    can you give it a quick try before I can find a windows machine for verification?

    1. upgrade to latest chrome and webdriver

    2. configure the option as follow if Refactor #1 doesn't work

    define this in your top conftest.py

    from selenium.webdriver.chrome.options import Options

def pytest_setup_options():
    """called before webdriver is initialized"""
    options = Options()
    options.add_argument('--disable-gpu')
    return options

    I will also update the document once the fix is confirmed.

  3. Oronis commented on Jul 22, 2019

    @Oronis
    Oronis
    on Jul 22, 2019
    Author

    @byronz I'm afraid I'm on a managed PC and haven't been able to force an update to version 76. I've done everything else and I'm afraid this doesn't seem to affect the issue. It still fails when calling get_webdriver as the value of remote is never set in the code.

    As a quick check, how or when is this value supposed to be set? I've tried to sense check the code and, as far as I can see from debugging it, the value is always None when called from the dash_duo initialisation and thus will always cause the call to until to fail.

  4. byronz commented on Jul 22, 2019

    @byronz
    byronz
    on Jul 22, 2019
    Contributor

    @Oronis the remote is like a reserved parameter in case we might need selenium grid, it's normal that it's always None in our current usage. it calls (in this case) the private getattr(self, "_get_{}".format(self._browser))() to initialize the corresponding browser per browser name. so the actual call for Chrome case is _get_chrome.

        def get_webdriver(self, remote):
        try:
            return (
                getattr(self, "_get_{}".format(self._browser))()
                if remote is None
                else webdriver.Remote(
                    command_executor=remote,
                    desired_capabilities=getattr(
                        DesiredCapabilities, self._browser.upper()
                    ),
                )
            )
        except WebDriverException:
            logger.exception("<<<Webdriver not initialized correctly>>>")
            return None

  5. byronz commented on Jul 22, 2019

    @byronz
    byronz
    on Jul 22, 2019
    Contributor

    @Oronis
    can you paste the structure of your folder? the location of the conftest and tests
    can you increase the log level by pytest --headless --log-cli-level DEBUG and past the log here?

    thanks

  6. byronz commented on Jul 24, 2019

    @byronz
    byronz
    on Jul 24, 2019
    Contributor

    @Oronis I cannot reproduce the problem using

    Windows 10 Home
    Chrome version 75.0.3770.100 (Official Build) (64-bit)

    pip list | grep dash
dash                          1.0.2
dash-core-components          1.0.0
dash-cytoscape                0.1.1
dash-html-components          1.0.0
dash-renderer                 1.0.0
dash-table                    4.0.2
WARNING: You are using pip version 19.1.1, however version 19.2.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

    python -m pytest --log-cli-level DEBUG --headless test_x.py
Test session starts (platform: win32, Python 2.7.15, pytest 4.6.4, pytest-sugar 0.9.2)
cachedir: .pytest_cache
rootdir: c:\Users\xingh\code\dash, inifile: pytest.ini
plugins: dash-1.0.2, mock-1.10.4, sugar-0.9.2
collecting ...
------------------------------- live log setup --------------------------------
11:47:37 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session {"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"prefs": {"safebrowsing.enabled": false, "download.default_directory": "c:\\users\\xingh\\appdata\\local\\temp\\pytest-of-xingh\\pytest-3\\test_bsly001_falsy_child0\\download", "download.directory_upgrade": true, "safebrowsing.disable_download_protection": true, "download.prompt_for_download": false}, "extensions": [], "args": ["--headless", "--disable-gpu"]}, "browserName": "chrome", "platformName": "any"}, "firstMatch": [{}]}, "desiredCapabilities": {"goog:chromeOptions": {"prefs": {"safebrowsing.enabled": false, "download.default_directory": "c:\\users\\xingh\\appdata\\local\\temp\\pytest-of-xingh\\pytest-3\\test_bsly001_falsy_child0\\download", "download.directory_upgrade": true, "safebrowsing.disable_download_protection": true, "download.prompt_for_download": false}, "extensions": [], "args": ["--headless", "--disable-gpu"]}, "platform": "ANY", "browserName": "chrome", "version": "", "loggingPrefs": {"browser": "SEVERE"}}}
11:47:37 | DEBUG | urllib3.connectionpool:206 | Starting new HTTP connection (1): 127.0.0.1:52395

DevTools listening on ws://127.0.0.1:52401/devtools/browser/33c2c4e8-d246-4020-955a-d712b2eb7164
11:47:39 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session HTTP/1.1" 200 678
11:47:39 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:39 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/chromium/send_command {"cmd": "Page.setDownloadBehavior", "params": {"downloadPath": "c:\\users\\xingh\\appdata\\local\\temp\\pytest-of-xingh\\pytest-3\\test_bsly001_falsy_child0\\download", "behavior": "allow"}}
11:47:39 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/chromium/send_command HTTP/1.1" 200 14
11:47:39 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:39 | DEBUG | dash.testing.browser:321 | enabled headless download returns {u'value': None}
11:47:39 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/window/rect {"y": 0, "x": 0, "width": null, "height": null}
11:47:40 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/window/rect HTTP/1.1" 200 48
11:47:40 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:40 | DEBUG | dash.testing.wait:23 | start wait.until with method, timeout, poll => <function <lambda> at 0x0000000018793748> 1 0.1
11:47:40 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/timeouts {"implicit": 2000}
11:47:40 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/timeouts HTTP/1.1" 200 14
11:47:40 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:40 | DEBUG | dash.testing.browser:65 | initialize browser with arguments
11:47:40 | DEBUG | dash.testing.browser:66 |   headless => True
11:47:40 | DEBUG | dash.testing.browser:67 |   download_path => c:\users\xingh\appdata\local\temp\pytest-of-xingh\pytest-3\test_bsly001_falsy_child0\download
-------------------------------- live log call --------------------------------
11:47:40 | DEBUG | urllib3.connectionpool:206 | Starting new HTTP connection (1): localhost:8050
11:47:40 | INFO | werkzeug:122 |  * Running on http://127.0.0.1:8050/ (Press CTRL+C to quit)
11:47:41 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:41] "GET / HTTP/1.1" 200 -
11:47:41 | DEBUG | urllib3.connectionpool:396 | http://localhost:8050 "GET / HTTP/1.1" 200 515
11:47:41 | DEBUG | dash.testing.wait:23 | start wait.until with method, timeout, poll => <function accessible at 0x0000000018793D68> 1 0.1
11:47:41 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/url {"url": "http://localhost:8050"}
11:47:41 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:41] "GET / HTTP/1.1" 200 -
11:47:42 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:42] "GET /_dash-component-suites/dash_renderer/react@16.8.6.min.js?v=1.0.0&m=1563993858 HTTP/1.1" 200 -
11:47:42 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:42] "GET /_dash-component-suites/dash_renderer/react-dom@16.8.6.min.js?v=1.0.0&m=1563993858 HTTP/1.1" 200 -
11:47:42 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:42] "GET /_dash-component-suites/dash_renderer/prop-types@15.7.2.min.js?v=1.0.0&m=1563993858 HTTP/1.1" 200 -
11:47:42 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:42] "GET /_dash-component-suites/dash_html_components/dash_html_components.min.js?v=1.0.0&m=1563993862 HTTP/1.1" 200 -
11:47:42 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:42] "GET /_dash-component-suites/dash_renderer/dash_renderer.min.js?v=1.0.0&m=1563993858 HTTP/1.1" 200 -
11:47:42 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:42] "GET /_dash-layout HTTP/1.1" 200 -
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/url HTTP/1.1" 200 14
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | DEBUG | dash.testing.browser:137 | method, timeout, poll => <class 'selenium.webdriver.support.expected_conditions.presence_of_element_located'> 10 0.5
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/element {"using": "css selector", "value": "#react-entry-point"}
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/element HTTP/1.1" 200 88
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | DEBUG | dash.testing.browser:137 | method, timeout, poll => <class 'dash.testing.wait.text_to_equal'> 4 0.5
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/element {"using": "css selector", "value": "#nully-wrapper"}
11:47:42 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:42] "GET /_dash-dependencies HTTP/1.1" 200 -
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/element HTTP/1.1" 200 88
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | GET http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/element/2b90207c-f7ac-4bd2-8734-d4fac4facfc6/text {"id": "2b90207c-f7ac-4bd2-8734-d4fac4facfc6"}
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "GET /session/a1880316ff15cef70ba554f94f88d50c/element/2b90207c-f7ac-4bd2-8734-d4fac4facfc6/text HTTP/1.1" 200 13
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | DEBUG | dash.testing.wait:85 | text to equal {0} => expected 0
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | GET http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/element/2b90207c-f7ac-4bd2-8734-d4fac4facfc6/text {"id": "2b90207c-f7ac-4bd2-8734-d4fac4facfc6"}
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "GET /session/a1880316ff15cef70ba554f94f88d50c/element/2b90207c-f7ac-4bd2-8734-d4fac4facfc6/text HTTP/1.1" 200 13
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/element {"using": "css selector", "value": "#nully-wrapper"}
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/element HTTP/1.1" 200 88
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | GET http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/element/2b90207c-f7ac-4bd2-8734-d4fac4facfc6/text {"id": "2b90207c-f7ac-4bd2-8734-d4fac4facfc6"}
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "GET /session/a1880316ff15cef70ba554f94f88d50c/element/2b90207c-f7ac-4bd2-8734-d4fac4facfc6/text HTTP/1.1" 200 13
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/log {"type": "browser"}
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/log HTTP/1.1" 200 12
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | INFO | dash.testing.browser:88 | taking snapshot name => bsly001-layout - py2.7


------------------------------ live log teardown ------------------------------
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | DELETE http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c {}
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "DELETE /session/a1880316ff15cef70ba554f94f88d50c HTTP/1.1" 200 14
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:44 | INFO | dash.testing.application_runners:80 | killing the app runner
11:47:44 | DEBUG | urllib3.connectionpool:206 | Starting new HTTP connection (1): localhost:8050
11:47:45 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:45] "GET /_stop-267a1537e9a4404ba2dbbc5c9840cc11 HTTP/1.1" 200 -
11:47:45 | DEBUG | urllib3.connectionpool:396 | http://localhost:8050 "GET /_stop-267a1537e9a4404ba2dbbc5c9840cc11 HTTP/1.1" 200 29
11:47:45 | DEBUG | dash.testing.wait:44 | start wait.until_not method, timeout, poll => <bound method Thread.isAlive of <Thread(Thread-1, started daemon 7788)>> 3 0.1
11:47:45 | DEBUG | dash.testing.wait:52 | poll => 1563994066.0
11:47:46 | DEBUG | dash.testing.wait:52 | poll => 1563994066.11
11:47:46 | DEBUG | dash.testing.wait:52 | poll => 1563994066.22
11:47:46 | DEBUG | dash.testing.wait:52 | poll => 1563994066.33
11:47:46 | DEBUG | dash.testing.wait:52 | poll => 1563994066.43
 tests\test_x.py::test_bsly001_falsy_child \r \x1b[36mtests\\test_x.py\x1b[0m::test_bsly001_falsy_child\x1b[0m \x1b[32m\u2713\x1b[0m                     \x1b[32m100% \x1b[0m\x1b[40m\x1b[32m\u2588\x1b[0m\x1b[40m\x1b[32m\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\x1b[0m

  7. Oronis commented on Jul 24, 2019

    @Oronis
    Oronis
    on Jul 24, 2019 via email
    Author

  8. byronz commented on Jul 24, 2019

    @byronz
    byronz
    on Jul 24, 2019
    Contributor

    no worries, get well soon.

  9. cormacc commented on Aug 7, 2019

    @cormacc
    cormacc
    on Aug 7, 2019

    I've run into what appears to be the same issue -- not on windows, but trying to setup a CI build using gitlab-ci / docker.
    The same sequence of build steps runs fine on my own linux installation and on another test machine running OSX.
    I've tried using a variety of base docker images (python:3.7, some variants with chromedriver installed), all to no effect.

    See below -- may be helpful in replicating the issue without a windows box?
    Two variants included. The simplest installs chromium and chromedriver. The second (commented out) uses pytest-selenium and a separate selenium/chromedriver docker image as a service. Both exhibit the same issue described by @Oronis

    # Official framework image. Look for the different tagged releases at:
# https://hub.docker.com/r/library/python
image: python:3.7
# Alternate image with added google-chrome/chromedriver
# image: joyzoursky/python-chromedriver:3.7

#See useful docs here:
#https://devconfusa18.readthedocs.io/en/latest/gitlab.html

# variables:
#   PERCY_ENABLE: 0

# This folder is cached between builds
# http://docs.gitlab.com/ce/ci/yaml/README.html#cache
cache:
  paths:
  - ~/.cache/pip/


stages:
  - test

Unit tests:
  stage: test
  before_script:
    - pip install pipenv
  script:
    - cp Pipfile.lock.linux Pipfile.lock
    - pipenv install --dev
    - pipenv run coverage run -m pytest -vvv -m "not webtest"

# This installs chromedriver locally....
UI tests:
  stage: test
  before_script:
    #Comment out these next two lines if using a base with embedded chromedriver
    - apt-get -y update
    - apt-get -y install chromium chromium-driver
    - pip install pipenv
  script:
    - cp Pipfile.lock.linux Pipfile.lock
    - pipenv install --dev
    - pipenv run coverage run -m pytest -vvv -m webtest --headless

# This uses a separate docker image for chrome
# UI tests:
#   stage: test
#   services:
#     - name: selenium/standalone-chrome
#       alias: selenium
#   before_script:
#     - pip install pipenv
#   script:
#     - cp Pipfile.lock.linux Pipfile.lock
#     - pipenv install --dev
#     - pipenv run coverage run -m pytest -vvv --driver Remote --capability browserName chrome --host selenium --port 4444 -m webtest
#     #N.B. the --driver Remote --capability... bit requires pytest-selenium to be installed
  10. byronz
    added this to the Dash v1.3.0 milestone on Aug 9, 2019
  11. byronz
    added
    size: 2
    Stage: Backlog
    on Aug 26, 2019
  12. byronz
    self-assigned this
    on Aug 26, 2019
  13. byronz
    added
    Stage: In Progress
    and removed
    Stage: Backlog
    on Aug 28, 2019
  14. byronz
    closed this as completedin #894on Sep 3, 2019
  15. byronz
    added
    Stage: Docs Needed
    and removed
    Stage: In Progress
    on Sep 3, 2019
  16. Marc-Andre-Rivet
    removed
    Stage: Docs Needed
    on Sep 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions