Skip to content

Is playwright supported? Tests fail but show as passed #16

Open
Open
Is playwright supported? Tests fail but show as passed#16

Description

@i-love-code
i-love-code
opened on Dec 13, 2023

The sdk branch appears to use framework: playwright, and this is GitHub repository is linked to in the setup documentation for Playwright and BrowserStack.

In the sdk branch, when you set retries: 3 in the playwright configuration file for re-running test failures and then run npm run sample-test command, (with your own user/access key) it marks the tests in BrowserStack as passed, but throws these errors in the console and the SDK marks the tests as failed, leading to it retrying over and over.

Running 4 tests using 2 workers

1) [playwright-firefox@latest:Windows 11@browserstack] › tests\bstack_test_add_to_cart.js:3:1 › BStackDemo test add to cart

    Error: : expected object, got undefined

    Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
    Error: : expected object, got undefined

    Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
    Error: : expected object, got undefined

    Retry #3 ───────────────────────────────────────────────────────────────────────────────────────
    Error: : expected object, got undefined

But in BrowserStack, it shows as passed sessions:
image

Activity

  1. i-love-code
    changed the title [-]Is playwright supported?[/-] [+]Is playwright supported? Tests fail but show as passed[/+] on Dec 13, 2023

  2. i-love-code commented on Dec 13, 2023

    @i-love-code
    i-love-code
    on Dec 13, 2023
    Author

    It would appear that the browserstack-node-sdk that is wrapping the npx playwright test command does not correctly infer the outcome of the test, and marks all tests as having failed.

  3. i-love-code commented on Dec 13, 2023

    @i-love-code
    i-love-code
    on Dec 13, 2023
    Author

    I have active BrowserStack support issues open for this, but I have not seen a helpful response/resolution. Trying all available paths for support.

  4. rohit-y180 commented on Dec 14, 2023

    @rohit-y180
    rohit-y180
    on Dec 14, 2023

    Hi @i-love-code , could you please let us know the support ticket ID that you have created on BrowserStack?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions