Repository navigation
[BUG] FW reported error: 13 - Requested power transition failed to complete #7969
Description
Describe the bug
Observed this issue in CI daily test, this issue only happens on MTL-NOCODEC so far. The reproduce rate is 100% when doing the suspend/resume test w/ audio. It should be a FW regression. Will do further checks.
[ 5405.125019] snd_sof:sof_ipc4_log_header: sof-audio-pci-intel-mtl 0000:00:1f.3: ipc tx : 0x47000000|0x0: MOD_SET_DX [data size: 8]
[ 5405.125608] snd_sof:sof_ipc4_log_header: sof-audio-pci-intel-mtl 0000:00:1f.3: ipc tx reply: 0x6700000d|0x0: MOD_SET_DX
[ 5405.125616] sof-audio-pci-intel-mtl 0000:00:1f.3: FW reported error: 13 - Requested power transition failed to complete
[ 5405.125798] sof-audio-pci-intel-mtl 0000:00:1f.3: ipc error for msg 0x47000000|0x0
[ 5405.125806] sof-audio-pci-intel-mtl 0000:00:1f.3: ctx_save IPC error: -22, proceeding with suspend
To Reproduce
check-suspend-resume-with-audio.sh -l 50 -m playback
Reproduction Rate
100%
Environment
- Branch name and commit hash of the 2 repositories: sof (firmware/topology) and linux (kernel driver).
- Kernel: {70d2c68e48d0}
- SOF: {011de4a09497}
- Name of the topology file
- Topology: {sof-ace-tplg/sof-mtl-nocodec.tplg}
- Name of the platform(s) on which the bug is observed.
- Platform: {mtl-nocodec}
Activity
keqiaozhang commentedon Jul 20, 2023
This is a FW regression and caused by #7325.
The commit ID should be 965e1c1.
mengdonglin commentedon Jul 31, 2023
Commit series on this D3 /IMR context save support:
Commit 53d471b andrey.borisovich@intel.com ipc4: notify Host on the failed IPC Device power transition
Commit 965e1c1 andrey.borisovich@intel.com board: intel_adsp_ace15_mtpm: Enabled MTL IMR context save
Commit 015488b andrey.borisovich@intel.com ipc4: added D3 support using Zephyr Power Manager API
Commit 80f27b2 andrey.borisovich@intel.com board: intel_adsp_ace15_mtpm: disabled CONFIG_PM_DEVICE_RUNTIME_EXCLUSIVE
Commit 4e8040c marcin.szkudlinski@intel.com platform: ace: Add pm notifiers to support Zephyr's D3 transition
mengdonglin commentedon Jul 31, 2023
@keqiaozhang If we remove
CONFIG_ADSP_IMR_CONTEXT_SAVE=yto disable IMR context save, can we reproduce this issue with tip of SOF main branch? commit 53d471b is kept and can still report device power transition failure.5 remaining items
mengdonglin commentedon Aug 1, 2023
@lgirdwood @mwasko @tmleman @wszypelt @keqiaozhang @fredoh9 @alex-cri
Summary of this issue in ww31:
Recent occurrences:
mengdonglin commentedon Aug 1, 2023
@lgirdwood @ujfalusi @mwasko @abonislawski @keqiaozhang @tmleman @wszypelt
It seems PR #7995 fixed this issue.
From latest PR test result https://sof-ci.ostc.intel.com/#/result/planresultdetail/29601
Let's wait for new daily test result.
keqiaozhang commentedon Aug 1, 2023
This issue still exists on MTL platforms. I still can reproduce this issue with tip main branch on ww31.2 (daily test report ID 29617), on MTLP_RVP_NOCODEC_MULTICORE. @lgirdwood @tmleman
lgirdwood commentedon Aug 1, 2023
@keqiaozhang did the repro rate change after #7995 was merged ? Any other difference to before ?
keqiaozhang commentedon Aug 2, 2023
@lgirdwood the repro rate is the same, no differences. This issue happened on 3 different platforms in today's daily test.
tmleman commentedon Aug 3, 2023
23ww31.5: I don't see any error in mtrace but I guess the problem comes from ipc device suspension:
https://github.com/thesofproject/sof/blob/main/src/ipc/ipc4/handler.c#L1120-L1123
This could easily be confirmed by finding one of these logs:
https://github.com/thesofproject/sof/blob/main/src/ipc/ipc-zephyr.c#L102-L117
I need to reproduce this behavior in a local environment.
tmleman commentedon Aug 8, 2023
23ww32.3: I'm unable to reproduce this issue, work in progress.
tmleman commentedon Aug 11, 2023
23ww32.6: root cause:
SOF_IPC4_NOTIFY_LOG_BUFFER_STATUSnotification added to the IPC queue. FW did not manage to send the message before the D3 entry procedure.@keqiaozhang thanks for help with confirmation.
I see two possible solutions:
SOF_IPC4_NOTIFY_LOG_BUFFER_STATUSis in IPC queue, we continue the D3 transition procedure. The message will be sent when the DSP wakes up (PR8029).Initially, I would go with version A. In version B, I do not know how HOST will behave when waiting for a SET_DX response when it will receive notification.
kv2019i commentedon Aug 14, 2023
Option A merged via #8029
fredoh9 commentedon Aug 15, 2023
This problem is not found for two consecutive days
lgirdwood commentedon Aug 16, 2023
@fredoh9 pls close when happy resolved.
fredoh9 commentedon Aug 16, 2023
no issue found in today's daily test too. Closing now