[CML][TGLH][ICL] suspend/resume TIMEOUT, no test starting log

This issue is transferred from https://gitlab.devtools.intel.com/sof-ci/sofcitest/issues/670 and https://gitlab.devtools.intel.com/sof-ci/sofcitest/issues/667

TIMEOUT errors without any test log.
Found in jf-cml-hel-rt5682-01 and jf-cml-hel-rt5682-02. Looks this started in JF's Helios after replacing USB hub.
cmd: TPLG=sof-cml-rt1011-rt5682.tplg ~/sof-test/test-case/check-suspend-resume.sh -l 5

framework load: ssh ubuntu@jf-cml-hel-rt5682-01.jf.intel.com 'nohup bash -c "TPLG=sof-cml-rt1011-rt5682.tplg ~/sof-test/test-case/check-suspend-resume.sh -l 5 2>&1 | nc -N 10.239.156.45 56947" >/dev/null &'

framework run: 'netcat -l -k 56947'

The latest update is that we found this issue only happens on JF DUTs. We suspect this issue is network or usb hub related.

check-suspend-resume: sleep 1s before rtcwake iteration

Still happening even after the sleep 1 of #651

https://sof-ci.sh.intel.com/#/result/planresultdetail/3401?model=TGLH_RVP_HDA&testcase=check-suspend-resume-50 and the next test experienced a timeout
So did https://sof-ci.01.org/sofpr/PR4084/build8802/devicetest/?model=TGLH_RVP_HDA&testcase=check-suspend-resume-with-playback

@keqiaozhang

We suspect this issue is network or usb hub related.

Thanks to @keqiaozhang's large testing efforts, we also know old enough kernels don't reproduce this (while keeping everything the same)

Duplicate of https://github.com/intel-innersource/drivers.audio.ci.sof-framework/issues/9. Fixed and closed.

This is less frequent but still happening regularly, see for instance all the links above automatically created by github.

I just spotted this, mentioning just in case:

https://sof-ci.01.org/sofpr/PR4105/build8867/devicetest/?model=APL_UP2_NOCODEC&testcase=check-suspend-resume-with-capture

rtcwake: assuming RTC uses UTC ...
rtcwake: wakeup from "mem" using /dev/rtc0 at Wed Apr 28 22:14:48 2021
2021-04-28 22:14:50 UTC Sub-Test: [REMOTE_COMMAND] sleep for 5
2021-04-28 22:14:55 UTC Sub-Test: [REMOTE_INFO] Check for the kernel log status
declare -- cmd="journalctl_cmd --since=@1619648077"
2021-04-28 22:14:55 UTC [ERROR] Caught kernel log error
===========================>>
[ 1218.444372] kernel: rtc_cmos 00:02: Alarms can be up to one month in the future

EDIT: thesofproject/linux#3011

This issue is fixed already. close.

This is the start of the only known "fix" reference above: #651

After some kernel update, we start to observe a timeout issue on several JF devices. As we are not going to fix the kernel, this workaround is applied.

If there is some other, actual fix then please link to it and let's remove the workaround.

Inserting a delay without any understanding of why it makes any difference is not a "fix".

Hopefully caused by a combination of

@aiChaoSONG

@aiChaoSONG @marc-hb this doesn't seem like a current issue, can we close?

This is not really a sof-test issue, but sometimes caused by the network traffic between SH and JF.

Navigation Menu

Repository navigation

[CML][TGLH][ICL] suspend/resume TIMEOUT, no test starting log #650

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[CML][TGLH][ICL] suspend/resume TIMEOUT, no test starting log #650

Description

Activity

marc-hb commented on Apr 23, 2021

marc-hb commented on Apr 26, 2021

mengdonglin commented on Apr 28, 2021

marc-hb commented on Apr 28, 2021

marc-hb commented on Apr 29, 2021

aiChaoSONG commented on Jun 21, 2021

marc-hb commented on Jun 21, 2021

marc-hb commented on Jan 21, 2022

plbossart commented on Mar 18, 2022

aiChaoSONG commented on Mar 20, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions