Skip to content

Flaky test sequential/test-http2-settings-flood #31107

Closed
Closed
Flaky test sequential/test-http2-settings-flood#31107
Labels
flaky-testIssues and PRs involving tests that fail intermittently in CI.http2Issues and PRs related to the http2 subsystem.v8 engineIssues and PRs related to the V8 dependency.

Description

@addaleax
addaleax
opened on Dec 27, 2019
  • Version: master
  • Platform: AIX p158a05 1 7 00CCB1C24C00
  • Subsystem: v8?

The test in sequential/test-http2-settings-flood sometimes fails on AIX PPC64 with SIGILL and no other output (about 4/100 times). Bisecting points to @ronag’s cd6b00d, which obviously shouldn’t be causing that issue. It doesn’t reproduce when running the test with --jitless, so my best guess would be that it’s a V8 issue on PPC.

I can’t really debug this any further, the issue doesn’t seem to reproduce under gdb and core dumps don’t seem helpful, so it would be great if somebody in @nodejs/platform-aix (@miladfarca?) could take a look. In the short term, I guess we could also revert the commit above, although that’s obviously not ideal.

Activity

  1. addaleax
    added
    aixIssues and PRs related to the AIX platform.
    flaky-testIssues and PRs involving tests that fail intermittently in CI.
    on Dec 27, 2019

  2. miladfarca commented on Jan 6, 2020

    @miladfarca
    miladfarca
    on Jan 6, 2020
    Contributor

    I have tested the above on AIX 6.1 as well as pLinux on a loop (~400 times) and it always passes.
    Could you please add my key to the above machine:

    ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQZihjEXcY52UZo09CEb29HwOWwXcmwbwZFR4rsACQQyGUexL4fkVrFfwuG8eV1vg5KECsO8LiGY/MHkNIpABaJJoip0Qxgv0pAFtAukIDjLXXOV/VNJjfIto16vOAehRZkmI+BtQP8TjoT2CSyJgvVQcay8BhH52in1LQQsyCi2crHLYzDrrCgY/rAmuVb1MzMnT8mFOdJ8E5RBhjnmc1K4YBKmNTf6yefgbOJssI0lLzp7Q2uytzp3pipg7AO/VqmRn8953UTJS/cOQeBi3nCYGpz4I7kOKHgwbdW1IP/XFfm0KO5daulHQeToRGIE85ntxF314wsYE3ZyeJwKgH

    Edit:
    Seems like #31146 is causing it

  3. sam-github commented on Jan 6, 2020

    @sam-github
    sam-github
    on Jan 6, 2020
    Contributor

    Gave access, permission already granted in nodejs/build#1706

  4. addaleax commented on Jan 7, 2020

    @addaleax
    addaleax
    on Jan 7, 2020
    MemberAuthor

    Seems like #31146 is causing it

    @miladfarca Can you figure out why it is crashing?

  5. miladfarca commented on Jan 7, 2020

    @miladfarca
    miladfarca
    on Jan 7, 2020
    Contributor

    @addaleax Yes I will be looking into it, might be related to V8 deoptimization

  6. miladfarca commented on Jan 7, 2020

    @miladfarca
    miladfarca
    on Jan 7, 2020
    Contributor

    This issue doesn't seem to be specific to PPC or AIX. It is also failing on x64-freebsd on the CI: https://ci.nodejs.org/job/node-test-commit-freebsd/nodes=freebsd11-x64/30698/console

    I have done a debug compile on both pLinux and x64 Linux and this test periodically fails on the same DCHEK:

    #
# Fatal error in ../deps/v8/src/deoptimizer/deoptimizer.cc, line 253
# Debug check failed: code == topmost_ implies safe_to_deopt_.
#
#
#
#FailureMessage Object: 0x7ffc4970cda0

    There seems to be an issue with V8 de-optimizer.

  7. ronag commented on Jan 7, 2020

    @ronag
    ronag
    on Jan 7, 2020
    Member

    @miladfarca: good job! would you mind creating a V8 ticket?

  8. miladfarca commented on Jan 7, 2020

    @miladfarca
    miladfarca
    on Jan 7, 2020
    Contributor

    @ronag I've created this ticket but not sure if we need to reproduce it under V8 (and not nodejs), will wait for their reply:
    https://bugs.chromium.org/p/v8/issues/detail?id=10101

  9. Trott commented on Jan 8, 2020

    @Trott
    Trott
    on Jan 8, 2020
    Member

    @miladfarca Same results with a debug build on MacOS Mojave:

    $ tools/test.py --mode=debug --repeat 1000 test/sequential/test-http2-ping-flood.js
=== debug test-http2-ping-flood ===                   
Path: sequential/test-http2-ping-flood
#
# Fatal error in ../deps/v8/src/deoptimizer/deoptimizer.cc, line 253
# Debug check failed: code == topmost_ implies safe_to_deopt_.
#
#
#
  10. Trott
    changed the title [-]Flaky test sequential/test-http2-settings-flood on AIX[/-] [+]Flaky test sequential/test-http2-settings-flood[/+] on Jan 8, 2020

  11. miladfarca commented on Jan 8, 2020

    @miladfarca
    miladfarca
    on Jan 8, 2020
    Contributor

    Thanks @Trott , just to confirm your mac is x64 correct? I'll add to the V8 ticket.

  12. Trott commented on Jan 10, 2020

    @Trott
    Trott
    on Jan 10, 2020
    Member

    Thanks @Trott , just to confirm your mac is x64 correct? I'll add to the V8 ticket.

    Yes, x64.

  13. ronag commented on Jan 11, 2020

    @ronag
    ronag
    on Jan 11, 2020
    Member

    ping @nodejs/v8

  14. vtjnash commented on Jan 13, 2020

    @vtjnash
    vtjnash
    on Jan 13, 2020
    Contributor

    It appears this is already known to be platform agnostic, but just for additional info, it also has been observed on debian9-docker-armv7 (https://ci.nodejs.org/job/node-test-commit-arm/28686/)

  15. Trott
    added
    v8 engineIssues and PRs related to the V8 dependency.
    http2Issues and PRs related to the http2 subsystem.
    and removed
    aixIssues and PRs related to the AIX platform.
    on Jan 26, 2020

  16. mscdex commented on Feb 3, 2020

    @mscdex
    mscdex
    on Feb 3, 2020
    Contributor

    FWIW this is also occurring in Ubuntu 18.04 containers.

  17. miladfarca commented on Feb 5, 2020

    @miladfarca
    miladfarca
    on Feb 5, 2020
    Contributor

    @addaleax V8 ticket (https://bugs.chromium.org/p/v8/issues/detail?id=10101) is now closed with a comment suggesting an API misuse in node. Issue might be related to this commit/line which might need to be refactored: 7e6104f#diff-9e96b9359319a317a5a1ee45f11eea67R126

  18. addaleax commented on Feb 5, 2020

    @addaleax
    addaleax
    on Feb 5, 2020
    MemberAuthor

    @miladfarca Thanks for the info, I’ll try to come up with a fix. They are right here, but I am surprised that ArrayBuffer::Detach() is unsafe for calling here.

    Issue might be related to this commit/line which might need to be refactored: 7e6104f#diff-9e96b9359319a317a5a1ee45f11eea67R126

    I’d like to avoid that, and instead try to not perform the invalid V8 call here.

  19. addaleax
    added a commit that references this issue on Feb 5, 2020

    src: give Http2Session JS fields their own backing store

    cca0ba2
  20. addaleax
    closed this as completedin f368210on Feb 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    flaky-testIssues and PRs involving tests that fail intermittently in CI.http2Issues and PRs related to the http2 subsystem.v8 engineIssues and PRs related to the V8 dependency.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions