You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Some SOF tests modify machine state by setting controls to some arbitrary values and not resetting them after.
Tests I've detected on a quick look-through:
check-keyword-detection.sh sets the volume at 40 and does not seem to reset it after.
check-volume-levels.sh modifies a lot of amixer controls and should be looked at when handling this issue.
volume-basic-test.sh calls the reset_sof_volume function rather than the full set_alsa_settings.
Note that this issue directly had its hand in the FAILs mentioned in this PR.
verify-bootsequence.sh can theoretically mess up the settings if the test crashes, but I'd like to mention it as an example for a possible new approach of a common test runner. Were we to alsactl store -f FILE the machine state before each test and then alsactl restore -f FILE after, we would be able to avoid such issues.
To Reproduce
For example, run volume-basic-test.sh and then a check-alsabat.sh test on a RT712. The latter will currently fail.
Expected behavior
Test leaves the machine in the same state as it was before the test.
Detail Info (for the aforementioned example)
Branch name and commit hash of the 3 repositories: sof (firmware/topology), linux (kernel driver) and sof-test (test case)
In my humble experience, you want a "belt and braces" approach here:
Passing tests should try to clean up after themselves.
At start time, every test should ALSO make an effort to cleanup and start from a good state. Because you want to try to recover from an earlier, failing test.
It's absolutely impossible to guarantee that a failing test will not mess up the system. In fact, most of the time you do NOT want cleaners to comes and mess with the "crime scene", it makes forensics and debugging MUCH harder.
So, everything and everyone should always try to clean things up when things go well. Never an exact science, always best effort.
BTW Jenkins has some logic to automatically reboot a device after each test failure and after "dangerous" tests. I forgot all the details but there are relevant links in internal issue sof-framework#623
Activity
marc-hb commentedon May 30, 2025
Good catch!
In my humble experience, you want a "belt and braces" approach here:
It's absolutely impossible to guarantee that a failing test will not mess up the system. In fact, most of the time you do NOT want cleaners to comes and mess with the "crime scene", it makes forensics and debugging MUCH harder.
So, everything and everyone should always try to clean things up when things go well. Never an exact science, always best effort.
BTW Jenkins has some logic to automatically reboot a device after each test failure and after "dangerous" tests. I forgot all the details but there are relevant links in internal issue sof-framework#623
marc-hb commentedon Jun 20, 2025
Progress in:
marc-hb commentedon Sep 4, 2025
New status needed after #1284 and #1294 were merged.