Investigate flaky tests on AIX #28517

Activity

1. Trott

   added

   flaky-testIssues and PRs involving tests that fail intermittently in CI.Issues and PRs involving tests that fail intermittently in CI.

   on Jul 2, 2019
2. bnoordhuis commented on Jul 16, 2019

   

   bnoordhuis

   on Jul 16, 2019

   Member

   More actions

   I re-enabled the addons/stringbytes-external-exceed-max tests and ran them repeatedly on test-osuosl-aix61-ppc64_be-2 but I can't get them to fail, not even with python tools/test.py -J.

   I hacked test/common/index.js so it always says there is enough memory:

   Details

   diff --git a/test/addons/addon.status b/test/addons/addon.status
   index ee094f4..ec2d213 100644
   --- a/test/addons/addon.status
   +++ b/test/addons/addon.status
   @@ -1,19 +1,3 @@
    prefix addons
    
    [true] # This section applies to all platforms
   -
   -[$system==aix]
   -# https://github.com/nodejs/build/issues/1820#issuecomment-505998851
   -# https://github.com/nodejs/node/pull/28469
   -# https://github.com/nodejs/node/pull/28516
   -stringbytes-external-exceed-max/test-stringbytes-external-exceed-max.js: SKIP
   -
   -# https://github.com/nodejs/node/pull/28516
   -stringbytes-external-exceed-max/test-stringbytes-external-at-max: SKIP
   -stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-ascii: SKIP
   -stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-base64: SKIP
   -stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-binary: SKIP
   -stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-hex: SKIP
   -stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-utf8: SKIP
   -stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-2: SKIP
   -stringbytes-external-exceed-max/test-stringbytes-external-exceed-max: SKIP
   diff --git a/test/common/index.js b/test/common/index.js
   index 13604d0..a627922 100644
   --- a/test/common/index.js
   +++ b/test/common/index.js
   @@ -111,7 +111,7 @@ const isOpenBSD = process.platform === 'openbsd';
    const isLinux = process.platform === 'linux';
    const isOSX = process.platform === 'darwin';
    
   -const enoughTestMem = os.totalmem() > 0x70000000; /* 1.75 Gb */
   +const enoughTestMem = true;
    const cpus = os.cpus();
    const enoughTestCpu = Array.isArray(cpus) &&
                          (cpus.length > 1 || cpus[0].speed > 999);

   It makes no difference however because the tests have additional checks that creating a big buffer is possible:
   

   node/test/addons/stringbytes-external-exceed-max/test-stringbytes-external-at-max.js

   Lines 15 to 22 in 462f438

   	let buf;
   	try {
   	buf = Buffer.allocUnsafe(kStringMaxLength);
   	} catch (e) {
   	// If the exception is not due to memory confinement then rethrow it.
   	if (e.message !== 'Array buffer allocation failed') throw (e);
   	common.skip(skipMessage);
   	}

   
   And that causes the tests to bail out with a "Skipped:" message.

   Not sure where to go from here. Maybe just re-enable the tests and see what happens?
3. mhdawson commented on Jul 16, 2019

   

   mhdawson

   on Jul 16, 2019

   Member

   More actions

   If you create a PR that re-enables them we can run the CI on that PR a bunch of times to see if we see failures. It was very common before they were diabled.
4. bnoordhuis commented on Jul 17, 2019

   

   bnoordhuis

   on Jul 17, 2019

   Member

   More actions

   Here's a trial run: https://ci.nodejs.org/job/node-test-commit/30163/
5. bnoordhuis commented on Jul 17, 2019

   

   bnoordhuis

   on Jul 17, 2019

   Last edited by bnoordhuis

   Member

   More actions

   Another run with just AIX: https://ci.nodejs.org/job/node-test-commit-aix/24447/

   edit: and another: https://ci.nodejs.org/job/node-test-commit-aix/24448/
6. sam-github commented on Jul 22, 2019

   

   sam-github

   on Jul 22, 2019

   ContributorAuthor

   More actions

   24448 failed:

   04:35:29 ok 2424 addons/stringbytes-external-exceed-max/test-stringbytes-external-at-max
   04:35:29   ---
   04:35:29   duration_ms: 3.59
   04:35:29   ...
   04:38:07 FATAL: command execution failed
   04:38:07 java.nio.channels.ClosedChannelException
   

   This is an instance of nodejs/build#1849
7. targos commented on Nov 20, 2021

   

   targos

   on Nov 20, 2021

   Member

   More actions

   Closing as there was no recent activity
8. targos

   closed this as completedon Nov 20, 2021