Repository navigation
[BUG] "IPC timed out" on multiple ACE platforms with xrun warnings in mtrace #9914
Description
Describe the bug
multiple-pause-resume-50 and multiple-pipeline-playback tests fail on PTL and LNL with xrun floods in mtrace logs, leading to IPC timeouts.
Example 1:
https://sof-ci.01.org/linuxpr/PR5369/build6689/devicetest/index.html?model=PTLP_RVP_SDW&testcase=multiple-pipeline-all
platform: PTL
test: multiple-pipeline-all
mtrace:
[00:00:05.803,305] <inf> ipc: ipc_cmd: rx : 0x12020000|0x0
[00:00:05.803,328] <inf> dma: sof_dma_put: dma_put(), dma = 0x4009e2e0, sref = 2
[00:00:05.803,335] <inf> dai_intel_dmic: dai_dmic_remove: dmic_remove(), dmic_active_fifos_mask = 0x0, dmic_pause_mask = 0x0
[00:00:05.803,443] <inf> host_comp: host_get_copy_bytes_normal: comp:5 0x70002 no bytes to copy, available samples: 0, free_samples: 384
[00:00:05.804,441] <inf> host_comp: host_get_copy_bytes_normal: comp:5 0x70002 no bytes to copy, available samples: 0, free_samples: 384
[00:00:05.805,441] <inf> host_comp: host_get_copy_bytes_normal: comp:5 0x70002 no bytes to copy, available samples: 0, free_samples: 384
Linux kernel:
[ 1605.125092] kernel: snd_sof:sof_ipc4_log_header: sof-audio-pci-intel-ptl 0000:00:1f.3: ipc tx : 0x12020000|0x0: GLB_DELETE_PIPELINE
[ 1605.125189] kernel: snd_sof:sof_pcm_trigger: sof-audio-pci-intel-ptl 0000:00:1f.3: pcm1 (Jack In), dir 1: Entry: trigger (cmd: 0)
[ 1605.125194] kernel: snd_sof:sof_ipc4_trigger_pipelines: sof-audio-pci-intel-ptl 0000:00:1f.3: pcm1 (Jack In), dir 1: cmd: 0, state: 3
[ 1605.125476] kernel: soundwire_bus:sdw_slave_get_scale_index: rt722-sdca sdw:0:3:025d:0722:01: Configured bus base 1, scale 3, mclk 19200000, curr_freq 4800000
[ 1605.129152] kernel: snd_sof:sof_pcm_trigger: sof-audio-pci-intel-ptl 0000:00:1f.3: pcm4 (Microphone), dir 1: Entry: trigger (cmd: 0)
[ 1605.129159] kernel: snd_sof:sof_ipc4_trigger_pipelines: sof-audio-pci-intel-ptl 0000:00:1f.3: pcm4 (Microphone), dir 1: cmd: 0, state: 3
[ 1605.135919] kernel: snd_sof:sof_pcm_trigger: sof-audio-pci-intel-ptl 0000:00:1f.3: pcm0 (Jack Out), dir 0: Entry: trigger (cmd: 0)
[ 1605.135937] kernel: snd_sof:sof_ipc4_trigger_pipelines: sof-audio-pci-intel-ptl 0000:00:1f.3: pcm0 (Jack Out), dir 0: cmd: 0, state: 3
[ 1605.136063] kernel: snd_sof:sof_pcm_trigger: sof-audio-pci-intel-ptl 0000:00:1f.3: pcm31 (Deepbuffer Jack Out), dir 0: Entry: trigger (cmd: 0)
[ 1605.136069] kernel: snd_sof:sof_ipc4_trigger_pipelines: sof-audio-pci-intel-ptl 0000:00:1f.3: pcm31 (Deepbuffer Jack Out), dir 0: cmd: 0, state: 3
[ 1605.136094] kernel: snd_sof:sof_pcm_trigger: sof-audio-pci-intel-ptl 0000:00:1f.3: pcm2 (Speaker), dir 0: Entry: trigger (cmd: 0)
[ 1605.136100] kernel: snd_sof:sof_ipc4_trigger_pipelines: sof-audio-pci-intel-ptl 0000:00:1f.3: pcm2 (Speaker), dir 0: cmd: 0, state: 3
[ 1605.625967] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: ipc timed out for 0x12020000|0x0
Example 2:
https://sof-ci.01.org/sofpr/PR9860/build11121/devicetest/index.html?model=LNLM_RVP_HDA&testcase=multiple-pause-resume-50
platform: LNL
test: multiple-pause-resume-50
mtrace:
[ 763.205285] \<inf\> host_comp: host_get_copy_bytes_normal: comp:3 0x40004 no bytes to copy, available samples: 0, free_samples: 768
Linux kernel:
[ 762.318154] kernel: snd_sof:sof_ipc4_log_header: sof-audio-pci-intel-lnl 0000:00:1f.3: ipc tx : 0x12010000|0x0: GLB_DELETE_PIPELINE
[ 762.824476] kernel: sof-audio-pci-intel-lnl 0000:00:1f.3: ipc timed out for 0x12010000|0x0
Example 3 (uncertain):
platform: LNL
test: multiple-pause-resume-50
mtrace:
[ 1439.250870] <wrn> dai_comp: dai_common_copy: comp:0 0x4 nothing to copy, src_frames: 0, sink_frames: 96
Linux kernel:
(none)
Uncertain because the firmware flooding error is different, configuration is different (SDW vs. HDA), no kernel logs to check IPC timeouts
To Reproduce
So far observed with multiple-pause-resume-50 and multiple-pipeline-playback tests on LNL and PTL platforms in HDA configurations
Reproduction Rate
Rare
Impact
Must be fixed
Environment
The oldest of the above examples is from Feb 26 2025
Activity
lgirdwood commentedon Mar 24, 2025
@lyakh can you confirm if the host no bytes errors are for playback or capture ? If playback have we seen this with capture ?
lyakh commentedon Mar 24, 2025
another instance of this bug https://sof-ci.01.org/sofpr/PR9921/build11752/devicetest/index.html?model=PTLP_RVP_SDW&testcase=multiple-pipeline-all
lyakh commentedon Mar 24, 2025
@lgirdwood here's #9914 (comment) a good capture - with synchronised timestamps, both kernel and firmware logs available:
That seems to be capture, but I'm not 100% certain that DMIC messages above the timeout error are related. The corresponding firmware part:
The test was trying to run 6 playback and 3 capture streams simultaneously So, hard to be sure...
lyakh commentedon Mar 25, 2025
Currently it seems that this bug is almost all PTL SDW jenkins test runs fail the multiple-pipeline-all.sh test, however, some do pass, e.g. https://sof-ci.01.org/linuxpr/PR5366/build6770/devicetest/index.html?model=PTLP_RVP_SDW&testcase=multiple-pipeline-all
lyakh commentedon Apr 22, 2025
no reproduction found since #9926 and #9929 got merged, propose to close