Repository navigation
[BUG][MTL] Firmware boot failure due to timeout during suspend/resume stress test (ROM status 0x50000005, ROM error 0x0) #8148
Description
Describe the bug
This issue happened when testing suspend/resume with audio. It only happened once with stress test.
dmesg
[51369.325401] kernel: snd_sof_intel_hda_common:mtl_dsp_cl_init: sof-audio-pci-intel-mtl 0000:00:1f.3: Primary core power up successful
[51369.325408] kernel: snd_sof_intel_hda_common:mtl_dsp_cl_init: sof-audio-pci-intel-mtl 0000:00:1f.3: FW Poll Status: reg[0x73214]=0x80000000 successful
[51369.325420] kernel: snd_sof_intel_hda_common:mtl_enable_interrupts: sof-audio-pci-intel-mtl 0000:00:1f.3: FW Poll Status: reg[0x1800]=0x41 successful
[51369.325429] kernel: snd_sof_intel_hda_common:mtl_enable_interrupts: sof-audio-pci-intel-mtl 0000:00:1f.3: FW Poll Status: reg[0x1140]=0x1 successful
[51369.337199] kernel: nvme nvme0: 8/0/0 default/read/poll queues
[51371.343143] kernel: sof-audio-pci-intel-mtl 0000:00:1f.3: ------------[ DSP dump start ]------------
[51371.343150] kernel: sof-audio-pci-intel-mtl 0000:00:1f.3: Firmware boot failure due to timeout
[51371.343153] kernel: sof-audio-pci-intel-mtl 0000:00:1f.3: fw_state: SOF_FW_BOOT_IN_PROGRESS (3)
[51371.343197] kernel: sof-audio-pci-intel-mtl 0000:00:1f.3: ROM status: 0x0, ROM error: 0x0
[51371.343200] kernel: sof-audio-pci-intel-mtl 0000:00:1f.3: ROM debug status: 0x50000005, ROM debug error: 0x0
[51371.343205] kernel: sof-audio-pci-intel-mtl 0000:00:1f.3: ROM feature bit enabled
[51371.343207] kernel: sof-audio-pci-intel-mtl 0000:00:1f.3: ------------[ DSP dump end ]------------
[51371.343209] kernel: sof-audio-pci-intel-mtl 0000:00:1f.3: error: failed to boot DSP firmware after resume -5
[51371.343213] kernel: snd_sof:sof_set_fw_state: sof-audio-pci-intel-mtl 0000:00:1f.3: fw_state change: 3 -> 4
[51371.343216] kernel: sof-audio-pci-intel-mtl 0000:00:1f.3: PM: dpm_run_callback(): pci_pm_resume+0x0/0xe0 returns -5
[51371.343230] kernel: sof-audio-pci-intel-mtl 0000:00:1f.3: PM: failed to resume async: error -5
[51371.348200] kernel: OOM killer enabled.
[51371.348203] kernel: Restarting tasks ... done.
To Reproduce
~/sof-test/test-case/check-suspend-resume-with-audio.sh -l 100 -m playback
Reproduction Rate
TBD, need more tests to confirm it.
Environment
- Branch name and commit hash of the 2 repositories: sof (firmware/topology) and linux (kernel driver).
- Kernel: sof-dev, commit e327169bf752 + {PR4566}
- SOF: {stable-v2.7} commit 760d58485131
- Name of the topology file
- Topology: {development/sof-mtl-nocodec.tplg}
- Name of the platform(s) on which the bug is observed.
- Platform: {MTLP_RVP_NOCODEC}
Report ID: https://sof-ci.ostc.intel.com/#/result/planresultdetail/31196
- Platform: {MTLP_RVP_NOCODEC}
Activity
keqiaozhang commentedon Sep 5, 2023
A similar boot failure like #7866 (comment)
mengdonglin commentedon Sep 13, 2023
@kv2019i This issue cannot be reproduced with v2.7-rc1 on both MTL and cavs2.5 platforms, confirmed by @keqiaozhang by running more stress test on MTLP_RVP_NOCODEC and ADLP_RVP_NOCODEC.
v2.7-rc1 disabled IMR context save by default.
@keqiaozhang If we disable IMR context save on main branch, can we reproduce this issue?
keqiaozhang commentedon Sep 15, 2023
Confirmed that this issue cannot be reproduced on main branch when IMR is disabled.
fredoh9 commentedon Oct 5, 2023
Today daily test has same issue with MTLP_RVP_NOCODEC multicore.
Internal Intel Daily test link:
planresultdetail/32700?model=MTLP_RVP_NOCODEC-ace1_0-multicore-3cores&testcase=check-suspend-resume-with-playback-5
[-][BUG][MTL] Firmware boot failure due to timeout during suspend/resume stress test[/-][+][BUG][MTL] Firmware boot failure due to timeout during suspend/resume stress test (ROM status 0x50000005, ROM error 0x0)[/+]lrudyX commentedon Nov 24, 2023
@tmleman Please check
tmleman commentedon Nov 24, 2023
@keqiaozhang it looks very similar to the #7866 and probably have the same root cause. Why we tract it in separate issue?
keqiaozhang commentedon Dec 25, 2023
This issue cannot be reproduced on main branch. Closing this bug.
tmleman commentedon Jan 19, 2024
@keqiaozhang Is there any update? The latest comment mentions that the problem is not reproducing, but the issue is still open.