Repository navigation
extended_test (with memory limit tracking) are commented out #14680
Description
Describe the bug
- CI tests were failing on main: Extended tests are (still) failing on main #14576
I commented out the extended_test in
This test was added by @2010YOUY01 in
It started failing after
There is more detail in this comment
To Reproduce
No response
Expected behavior
I would like the extended_test to be fixed so it runs regularly or is removed
Having a commented out / unrun test in the codebase is not adding any value
Additional context
No response
Activity
2010YOUY01 commentedon Feb 16, 2025
This test is failing because it runs out of disk space, I've checked the remaining disk space after Github's CI runner has setup the rust toolchain and before running any test: It's 12GB disk space left.
The failing
extended_testrequires 11G, and the normal test requires 7G according to my local experiments. This disk space budget is quite tight IMO.I'm trying to find is there any good way to free up more disk space, It's quite hard since now CI jobs are running inside the container.
datafusion/.github/workflows/extended.yml
Lines 62 to 63 in 50c3547
I'm wondering is it possible not to run inside docker, and manually setup rust dependency, and this way we can delete some unused files and free up disk space.
alamb commentedon Feb 16, 2025
Yeah, and figuring out how to reduce the disk space usage in general might be a good idea too (maybe we can make a separate binary for extended tests, for example 🤔 )
Omega359 commentedon Feb 16, 2025
We already run with the CI profile which reduces binary size iirc.
2010YOUY01 commentedon Feb 18, 2025
This is a good idea to let extended tests run in a separate integration test binary.
I did a PR to fix the CI by freeing up disk spaces, which I think can also help in the long run, so perhaps this separating binary can be done as a follow up PR.