Investigate flaky test eval_messages

https://ci.nodejs.org/job/node-test-commit-arm/434/nodes=armv7-wheezy/tapTestReport/

not ok 838 - eval_messages.js
# [eval]
# [eval]:1
# with(this){__filename}
# ^^^^
# 
# SyntaxError: Strict mode code may not include a with statement
# at Object.exports.runInThisContext (vm.js:53:16)
# at Object.<anonymous> ([eval]-wrapper:6:22)
# at Module._compile (module.js:430:26)
# at node.js:566:27
# at doNTCallback0 (node.js:407:9)
# at process._tickCallback (node.js:336:13)
#42
#42

In the six and a half weeks since the build that this test failed on armv7-wheezy, it has not failed again on armv7-wheezy. And it had apparently not failed before that either. Any objections to removing its flaky status? It seems to me that either one of two things are happening:

This was caused by some weird momentary external thing, perhaps with the CI infrastructure, that we will likely never be able to reproduce.
Technically, the test is flaky, but it's flaky with such unbelievable infrequency that it is in fact indistinguishable from not flaky.

EDIT: There's also the possibility that it was a problem with the build (e.g., a configuration or OS bug) that has since been corrected. But whatever the case, it seems that there's perhaps not much point in having it marked flaky. If the test fails again, it seems that it is more likely the result of an actual bug in the commit and I'd rather have it flagged than marked as "oh yeah, that test failed, but it fails a lot so you can ignore it".

test: remove flaky status from eval_messages test

Navigation Menu

Repository navigation

Investigate flaky test eval_messages #2672

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Investigate flaky test eval_messages #2672

Description

Activity

Trott commented on Oct 18, 2015

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions