Fix test step for EESSI 2023.06 and 2025.06 and fix picking latest EESSI testsuite version by casparvl · Pull Request #336 · EESSI/software-layer-scripts

casparvl · 2026-11-02T08:30:05Z

Since the 1.0 release, this logic was stuck at returning the 0.9 version of the test suite. This PR fixes that and correctly returns the latest version. E.g. one would see:

Starting Gentoo Prefix shell /cvmfs/software.eessi.io/versions/2025.06/compat/linux/aarch64/bin/bash
[exit the shell to deactivate Gentoo Prefix]
Cloning into 'EESSI-test-suite'...
Note: switching to 'v0.9.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

In the output of the test step.

Also fixes EESSI/software-layer#1788

…ion of the test suite. This commit fixes that and correctly returns the latest version

casparvl · 2026-11-02T10:05:57Z

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2

eessi-bot-aws · 2026-11-02T10:06:05Z

New job on instance eessi-bot-mc-aws for repository eessi.io-2025.06-software
Building on: amd-zen2
Building for: x86_64/amd/zen2
Job dir: /project/def-users/SHARED/jobs/2026.11/pr_336/221211

date	job status	comment
Nov 02 10:06:03 UTC 2026	submitted	job id `221211` awaits release by job manager
Nov 02 10:06:33 UTC 2026	released	job awaits launch by Slurm scheduler
Nov 02 10:07:39 UTC 2026	running	job `221211` is running
Nov 02 10:08:42 UTC 2026	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-221211.out` ✅ no message matching `FATAL:` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.* created!` Artefacts `eessi-2025.06-software-linux-x86_64-amd-zen2-17936140910.tar.zst` size: 0 MiB (215734 bytes) entries: 66 modules under 2025.06/software/linux/x86_64/amd/zen2/modules/all `CMSeq/1.0.4-foss-2025b.lua` software under 2025.06/software/linux/x86_64/amd/zen2/software `CMSeq/1.0.4-foss-2025b` reprod directories under 2025.06/software/linux/x86_64/amd/zen2/reprod `CMSeq/1.0.4-foss-2025b/20261102_100757UTC` other under 2025.06/software/linux/x86_64/amd/zen2 no other files in tarball
Nov 02 10:08:42 UTC 2026	test result	😢 FAILURE (click triangle for details) Reason EESSI test suite was not run, test step itself failed to execute. Details ✅ job output file `slurm-221211.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

… cannot (and should not) get it outside the compat layer env

casparvl · 2026-11-02T10:38:39Z

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2

eessi-bot-aws · 2026-11-02T10:38:46Z

New job on instance eessi-bot-mc-aws for repository eessi.io-2025.06-software
Building on: amd-zen2
Building for: x86_64/amd/zen2
Job dir: /project/def-users/SHARED/jobs/2026.11/pr_336/221212

date	job status	comment
Nov 02 10:38:44 UTC 2026	submitted	job id `221212` awaits release by job manager
Nov 02 10:39:29 UTC 2026	released	job awaits launch by Slurm scheduler
Nov 02 10:40:34 UTC 2026	running	job `221212` is running
Nov 02 10:48:58 UTC 2026	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-221212.out` ✅ no message matching `FATAL:` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.* created!` Artefacts `eessi-2025.06-software-linux-x86_64-amd-zen2-17936164790.tar.zst` size: 0 MiB (216041 bytes) entries: 66 modules under 2025.06/software/linux/x86_64/amd/zen2/modules/all `CMSeq/1.0.4-foss-2025b.lua` software under 2025.06/software/linux/x86_64/amd/zen2/software `CMSeq/1.0.4-foss-2025b` reprod directories under 2025.06/software/linux/x86_64/amd/zen2/reprod `CMSeq/1.0.4-foss-2025b/20261102_104736UTC` other under 2025.06/software/linux/x86_64/amd/zen2 no other files in tarball
Nov 02 10:48:58 UTC 2026	test result	😢 FAILURE (click triangle for details) Reason EESSI test suite was not run, test step itself failed to execute. Details ✅ job output file `slurm-221212.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

casparvl · 2026-11-02T10:55:55Z

Bam, that works:

Ignoring invalid max threads value 4294967295 > max (100000).
Running 'export EESSI_INIT_PREFIX=/cvmfs/software.eessi.io/versions/2025.06/init; export EESSI_COMPAT_LAYER_DIR=/cvmfs/software.eessi.io/versions/2025.06/compat/linux/x86_64; export EESSI_VERSION_OVERRIDE=2025.06; export EESSI_CVMFS_REPO_OVERRIDE=/cvmfs/software.eessi.io; export EESSI_
SOFTWARE_SUBDIR_OVERRIDE=x86_64/amd/zen2; export SLURM_JOB_ID=221212; git clone https://github.com/EESSI/test-suite EESSI-test-suite && cd EESSI-test-suite && VERSION=$(git tag --list 'v*' | sort -V | tail -n1) && git checkout ${VERSION}' in EESSI (/cvmfs/software.eessi.io) 2025.06 com
patibility layer environment...
Starting Gentoo Prefix shell /cvmfs/software.eessi.io/versions/2025.06/compat/linux/x86_64/bin/bash
[exit the shell to deactivate Gentoo Prefix]
Cloning into 'EESSI-test-suite'...
Note: switching to 'v1.1.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

I'm now getting v1.1.0, which is the latest testuite.

…posed to import it directly, as the reframe executable would add the /externals dir to the sys.path - it won't work without it. It works for 4.10 onwards, but... there's no point in specifically test this if we can already run the ReFrame command itself

casparvl · 2026-11-02T11:06:13Z

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2

eessi-bot-aws · 2026-11-02T11:06:21Z

New job on instance eessi-bot-mc-aws for repository eessi.io-2025.06-software
Building on: amd-zen2
Building for: x86_64/amd/zen2
Job dir: /project/def-users/SHARED/jobs/2026.11/pr_336/221223

date	job status	comment
Nov 02 11:06:19 UTC 2026	submitted	job id `221223` awaits release by job manager
Nov 02 11:06:24 UTC 2026	released	job awaits launch by Slurm scheduler
Nov 02 11:07:40 UTC 2026	running	job `221223` is running
Nov 02 11:18:01 UTC 2026	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-221223.out` ✅ no message matching `FATAL:` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.* created!` Artefacts `eessi-2025.06-software-linux-x86_64-amd-zen2-17936180980.tar.zst` size: 0 MiB (216108 bytes) entries: 66 modules under 2025.06/software/linux/x86_64/amd/zen2/modules/all `CMSeq/1.0.4-foss-2025b.lua` software under 2025.06/software/linux/x86_64/amd/zen2/software `CMSeq/1.0.4-foss-2025b` reprod directories under 2025.06/software/linux/x86_64/amd/zen2/reprod `CMSeq/1.0.4-foss-2025b/20261102_111435UTC` other under 2025.06/software/linux/x86_64/amd/zen2 no other files in tarball
Nov 02 11:18:01 UTC 2026	test result	😁 SUCCESS (click triangle for details) ReFrame Summary WARNING: PROCESS BINDING ERROR: wrong number of cpus per process: expected 2, found Counter({16: 8}) WARNING: PROCESS BINDING WARNING: processes with cores shared by processing units, indicating hyperthreading: Counter({2: 64}), [ OK ] (1/6) EESSI_LAMMPS_lj %device_type=cpu %module_info=('BotBuildTests:x86-64-zen2', 'default', 'LAMMPS/22Jul2025_update4-foss-2025b-kokkos') %scale=1_node /40138b25 @BotBuildTests:x86-64-zen2+default P: perf: 417.85 timesteps/s (r:0, l:None, u:None) WARNING: PROCESS BINDING ERROR: wrong number of cpus per process: expected 2, found Counter({16: 8}) WARNING: PROCESS BINDING WARNING: processes with cores shared by processing units, indicating hyperthreading: Counter({2: 64}), [ OK ] (2/6) EESSI_LAMMPS_lj %device_type=cpu %module_info=('BotBuildTests:x86-64-zen2', 'default', 'LAMMPS/22Jul2025-foss-2024a-kokkos') %scale=1_node /3214c348 @BotBuildTests:x86-64-zen2+default P: perf: 403.28 timesteps/s (r:0, l:None, u:None) WARNING: PROCESS BINDING ERROR: wrong number of cpus per process: expected 1, found Counter({16: 8}) WARNING: PROCESS BINDING WARNING: processes with cores shared by processing units, indicating hyperthreading: Counter({2: 64}), [ OK ] (3/6) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_info=('BotBuildTests:x86-64-zen2', 'default', 'OSU-Micro-Benchmarks/7.5-gompi-2025a') %scale=1_node %device_type=cpu /e6c1bb98 @BotBuildTests:x86-64-zen2+default P: latency: 1.79 us (r:0, l:None, u:None) WARNING: PROCESS BINDING ERROR: wrong number of cpus per process: expected 1, found Counter({16: 8}) WARNING: PROCESS BINDING WARNING: processes with cores shared by processing units, indicating hyperthreading: Counter({2: 64}), [ OK ] (4/6) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_info=('BotBuildTests:x86-64-zen2', 'default', 'OSU-Micro-Benchmarks/7.5-gompi-2025a') %scale=1_node %device_type=cpu /a527c6fe @BotBuildTests:x86-64-zen2+default P: latency: 2.2 us (r:0, l:None, u:None) WARNING: PROCESS BINDING ERROR: wrong number of cpus per process: expected 1, found Counter({16: 2}) WARNING: PROCESS BINDING WARNING: processes with cores shared by processing units, indicating hyperthreading: Counter({2: 16}), [ OK ] (5/6) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_info=('BotBuildTests:x86-64-zen2', 'default', 'OSU-Micro-Benchmarks/7.5-gompi-2025a') %scale=1_node /95b023d0 @BotBuildTests:x86-64-zen2+default P: latency: 0.39 us (r:0, l:None, u:None) WARNING: PROCESS BINDING ERROR: wrong number of cpus per process: expected 1, found Counter({16: 2}) WARNING: PROCESS BINDING WARNING: processes with cores shared by processing units, indicating hyperthreading: Counter({2: 16}), [ OK ] (6/6) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_info=('BotBuildTests:x86-64-zen2', 'default', 'OSU-Micro-Benchmarks/7.5-gompi-2025a') %scale=1_node /2b54440b @BotBuildTests:x86-64-zen2+default P: bandwidth: 7435.2 MB/s (r:0, l:None, u:None) [ PASSED ] Ran 6/6 test case(s) from 6 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-221223.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

Caspar van Leeuwen added 2 commits November 2, 2026 09:28

Since the 1.0 release, this logic was stuck at returning the 0.9 vers…

9822337

…ion of the test suite. This commit fixes that and correctly returns the latest version

Add some easystack file to test with

6b9cae1

Of course, we should specify the COMMAND that defines the version, we…

0a35ba2

… cannot (and should not) get it outside the compat layer env

casparvl changed the title ~~Fix picking the latest EESSI testsuite version in test step~~ Fix test step for EESSI 2023.06 and 2025.06 and fix picking latest EESSI testsuite version Nov 2, 2026

casparvl mentioned this pull request Nov 2, 2026

ReFrame test step failing for 2023.06 and 2025.06 EESSI/software-layer#1788

Closed

Remove easystack file, it's proven to work now

3f74345

casparvl marked this pull request as ready for review November 2, 2026 11:19

bedroge approved these changes Nov 2, 2026

View reviewed changes

bedroge merged commit 43df6f4 into EESSI:main Nov 2, 2026
118 checks passed

Navigation Menu

Repository navigation

Fix test step for EESSI 2023.06 and 2025.06 and fix picking latest EESSI testsuite version - #336

Fix test step for EESSI 2023.06 and 2025.06 and fix picking latest EESSI testsuite version#336
bedroge merged 5 commits into
EESSI:mainfrom
casparvl:fix_use_latest_testsuite_version

casparvl commented Nov 2, 2026 •

edited

Loading

Uh oh!

casparvl commented Nov 2, 2026

Uh oh!

eessi-bot-aws Bot commented Nov 2, 2026 •

edited

Loading

Uh oh!

casparvl commented Nov 2, 2026

Uh oh!

eessi-bot-aws Bot commented Nov 2, 2026 •

edited

Loading

Uh oh!

casparvl commented Nov 2, 2026

Uh oh!

casparvl commented Nov 2, 2026

Uh oh!

eessi-bot-aws Bot commented Nov 2, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

casparvl commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

casparvl commented Nov 2, 2026

Uh oh!

eessi-bot-aws Bot commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

casparvl commented Nov 2, 2026

Uh oh!

eessi-bot-aws Bot commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

casparvl commented Nov 2, 2026

Uh oh!

casparvl commented Nov 2, 2026

Uh oh!

eessi-bot-aws Bot commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

casparvl commented Nov 2, 2026 •

edited

Loading

eessi-bot-aws Bot commented Nov 2, 2026 •

edited

Loading

eessi-bot-aws Bot commented Nov 2, 2026 •

edited

Loading

eessi-bot-aws Bot commented Nov 2, 2026 •

edited

Loading