Repository navigation
[BUG][CML] IPC timed out for '0x60040000: GLB_STREAM_MSG: TRIG_START due to pipeline copy errors' #1396
Copy link
Copy link
Closed
Labels
CMLApplies to Comet Lake platformApplies to Comet Lake platformHDAApplies to HD-Audio bus for codec connectionApplies to HD-Audio bus for codec connectionIPC timeoutIPC timeout observedIPC timeout observedbugSomething isn't working as expectedSomething isn't working as expected
Description
Summary:
'ipc timed out for '0x60040000: GLB_STREAM_MSG: TRIG_START' after system reboot.
To reproduce
Do system reboot stress test.
Actual result
error: ipc timed out for '0x60040000: GLB_STREAM_MSG: TRIG_START'
Reproduce rate: 1%
Environment
Rander UCM https://github.com/RanderWang/intel-UCM
Kernel repo:c99938c https://github.com/thesofproject/linux/tree/release/sof-v5.0
Firmware repo: 05d4e19 https://github.com/thesofproject/sof/commits/master |
Topology: https://github.com/thesofproject/sof/commits/master . Same as above SOF repo
file: tools/topology/sof-hda-generic.m4, sof-hda-generic-2ch.tplg
error sof-logger
CORE LEVEL COMP_ID TIMESTAMP DELTA FILE_NAME CONTENT
0 1 VOLUME 5334495.520833 5334495.500000 src/audio/volume.c:474 volume_copy() error: sink component buffer has not enough free bytes for copy
0 1 PIPE 5334543.020833 47.500000 src/audio/pipeline.c:738 pipeline_copy() error: ret = -5, start->comp.id = 2, dir = 1
0 1 PIPE 1.6 5334549.687500 6.666667 src/audio/pipeline.c:846 pipeline_xrun_recover()
0 1 VOLUME 5338245.000000 3695.312500 src/audio/volume.c:474 volume_copy() error: sink component buffer has not enough free bytes for copy
0 1 PIPE 5338291.718750 46.718750 src/audio/pipeline.c:738 pipeline_copy() error: ret = -5, start->comp.id = 2, dir = 1
0 1 PIPE 1.6 5338298.489583 6.770833 src/audio/pipeline.c:846 pipeline_xrun_recover()
0 1 VOLUME 5341994.687500 3696.197998 src/audio/volume.c:474 volume_copy() error: sink component buffer has not enough free bytes for copy
0 1 PIPE 5342041.406250 46.718750 src/audio/pipeline.c:738 pipeline_copy() error: ret = -5, start->comp.id = 2, dir = 1
0 1 PIPE 1.6 5342048.020833 6.614583 src/audio/pipeline.c:846 pipeline_xrun_recover()
0 1 VOLUME 5345744.791667 3696.770752 src/audio/volume.c:474 volume_copy() error: sink component buffer has not enough free bytes for copy
0 1 PIPE 5345791.250000 46.458332 src/audio/pipeline.c:738 pipeline_copy() error: ret = -5, start->comp.id = 2, dir = 1
0 1 PIPE 1.6 5345797.864583 6.614583 src/audio/pipeline.c:846 pipeline_xrun_recover()
0 1 VOLUME 5349494.583333 3696.718750 src/audio/volume.c:474 volume_copy() error: sink component buffer has not enough free bytes for copy
0 1 PIPE 5349540.989583 46.406250 src/audio/pipeline.c:738 pipeline_copy() error: ret = -5, start->comp.id = 2, dir = 1
0 1 PIPE 1.6 5349547.864583 6.875000 src/audio/pipeline.c:846 pipeline_xrun_recover()
0 1 VOLUME 5353245.052083 3697.187500 src/audio/volume.
dmesg
[ 6.368975] sof-audio-pci 0000:00:1f.3: pcm: hw params stream 0 dir 0
[ 6.368985] sof-audio-pci 0000:00:1f.3: generating page table for 00000000fc8aaece size 0xffc0 pages 16
[ 6.369035] sof-audio-pci 0000:00:1f.3: period_bytes:0x3fc0
[ 6.369037] sof-audio-pci 0000:00:1f.3: periods:4
[ 6.369066] sof-audio-pci 0000:00:1f.3: stream_tag 1
[ 6.369075] sof-audio-pci 0000:00:1f.3: ipc tx: 0x60010000: GLB_STREAM_MSG: PCM_PARAMS
[ 6.369319] sof-audio-pci 0000:00:1f.3: ipc tx succeeded: 0x60010000: GLB_STREAM_MSG: PCM_PARAMS
[ 6.369328] sof-audio-pci 0000:00:1f.3: pcm: stream dir 0, posn mailbox offset is 790528
[ 6.568048] sof-audio-pci 0000:00:1f.3: pcm: trigger stream 0 dir 0 cmd 1
[ 6.568080] sof-audio-pci 0000:00:1f.3: ipc tx: 0x60040000: GLB_STREAM_MSG: TRIG_START
[ 6.871913] sof-audio-pci 0000:00:1f.3: error: ipc timed out for 0x60040000 size 12
[ 6.871953] sof-audio-pci 0000:00:1f.3: status: fw entered - code 00000005
[ 6.872155] sof-audio-pci 0000:00:1f.3: error: unexpected fault 0x00000000 trace 0x00004000
[ 6.872159] sof-audio-pci 0000:00:1f.3: error: waking up any trace sleepers
[ 6.872174] HDA Analog: ASoC: trigger FE failed -110
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
CMLApplies to Comet Lake platformApplies to Comet Lake platformHDAApplies to HD-Audio bus for codec connectionApplies to HD-Audio bus for codec connectionIPC timeoutIPC timeout observedIPC timeout observedbugSomething isn't working as expectedSomething isn't working as expected
Activity
tlauda commentedon May 10, 2019
FW logs indicate that HDA Link pointer is stuck after trigger start, so there is something wrong with the Link DMA on the kernel side.
ClarexZhou commentedon May 14, 2019
0x60040000: GLB_STREAM_MSG: TRIG_START timed out occurred on CFL-s during playback stress test after running for about 210 times
Error Sof-logger:
Dmesg
Environment
Rander UCM https://github.com/RanderWang/intel-UCM
Kernel repo:9494607 https://github.com/thesofproject/linux/tree/release/sof-v5.0
Firmware repo: b73b9a6 https://github.com/thesofproject/sof/commits/master +PR1354
Topology: https://github.com/thesofproject/sof/commits/master . Same as above SOF repo
file: tools/topology/sof-hda-generic.m4, sof-hda-generic-2ch.tplg
ipctimedoutAplay_dmesg.log
logger.log
errorlogger.log
tlauda commentedon May 14, 2019
HDA Link problem. Please verify with the same changes as mentioned in #1413.
wenqingfu commentedon May 14, 2019
with 1354 and 895 integrated (aka, the fixes mentioned in #1413), there is still occurrence in Playabck_aplay -d 1_10000 times stressing
YvonneYang2 commentedon May 17, 2019
Issue occurred once after do steps below:
1.System reboot with headphone connected.
2.Paplay 1.wav
3.Connect the external DP monitor, and choose to HDMI2/DP output setting in Sound settings -Output.
4.Switch the setting between headphone and HDMI2/DP2 in Sound settings -Output.
Environment
Rander UCM https://github.com/RanderWang/intel-UCM/tree/hdmi
Kernel repo:df06e42 https://github.com/thesofproject/linux/tree/release/sof-v5.0
Firmware repo: b73b9a6 https://github.com/thesofproject/sof/commits/debug-ipc-timeout
Topology: https://github.com/thesofproject/sof/commits/debug-ipc-timeout . Same as above SOF repo
file: tools/topology/sof-hda-generic.m4, sof-hda-generic-2ch.tplg
platform: CML laptop with ALC3204 codec
dmesg
sof-logger
ipctimeout_dmesg.log
logger_ipctimeout.log
YvonneYang2 commentedon May 22, 2019
Issues occurred during 1000 times playback(aplay -d 1) stress test .
Environment
Firmware: 5fa2426 https://github.com/thesofproject/sof/tree/master
Kernel: b58f448 https://github.com/thesofproject/linux/tree/release/sof-v5.0
Topology: file: tools/topology/sof-hda-generic.tplg, sof-hda-generic-2ch.tplg same as firmware
Kconfig: https://sof-ci-storage.sh.intel.com/daily/20190521/release/linux/topic/sof-dev/901cd2a5-2/config-daily-hda-20190521.txt
platform :CML laptop
dmesg
error logger
playback_failed_353th_error_logger.log
playback_failed_353th_logger.log
playback_dmesg.log
ClarexZhou commentedon May 24, 2019
Error occurred during S3 with playback stress test.
Environment
Kernel: 9453da1 https://github.com/lyakh/linux/tree/ipc-timeout (with PR895)
FW:b527fd7 https://github.com/lyakh/sof/tree/ipc-timeout (with PR1354)
Topology:Same as above FW repo, file: tools/topology/sof-hda-generic.m4, sof-hda-generic-2ch.tplg
kconfig:
unset CONFIG_SND_SOC_SOF_DEBUG_ENABLE_FIRMWARE_TRACE
unset CONFIG_SND_SOC_SOF_DEBUG_ENABLE_DEBUGFS_CACHE
s3_test_14_dmesg.log
s3_test_14_logger_error.log
keyonjie commentedon May 24, 2019
@ClarexZhou can you paste the log of 13th iteration also? I doubt error happened earlier from the log you pasted(14th one).
ClarexZhou commentedon May 24, 2019
@keyonjie previous logs are already cleaned. will past once reproduced again.
And ipc error for 0x60040000 size 12 is occurred during S3 with playback test in 13th. Also add previous logs.
s3_test_12_dmesg.log
s3_test_12_logger_error.log
s3_test_13_dmesg.log
s3_test_13_logger_error.log
keyonjie commentedon May 24, 2019
@ClarexZhou yes, better to paste logs for iteration:
[n-1]: normal iteration(no any errors in dmesg and sof-logger) right before the the error happen, and
[n]: the first error iteration.
BTW, better to apply thesofproject/linux#888 to survive the sof-logger error logs.
ClarexZhou commentedon May 27, 2019
Track ipc error in #1477
ClarexZhou commentedon May 28, 2019
Error occurred during s3 with playback stress test at 29th.
Environment
Kernel: https://github.com/lyakh/linux/tree/ipc-timeout commit 9453da1 (with PR895)
FW:https://github.com/lyakh/sof/tree/ipc-timeout commit 47dc14d (with PR1354 and PR1470)
Kconfig: disable CONFIG_SND_SOC_SOF_DEBUG_ENABLE_DEBUGFS_CACHE in kernel config.
s3_test_28_logger_all.log
s3_test_28_logger_error.log
s3_test_29_logger_all.log
s3_test_29_logger_error.log
test_28.log
test_29.log
[-][BUG][CML]IPC timed out for '0x60040000: GLB_STREAM_MSG: TRIG_START'[/-][+][BUG][CML] IPC timed out for '0x60040000: GLB_STREAM_MSG: TRIG_START due to pipeline copy errors'[/+]lgirdwood commentedon May 28, 2019
@ClarexZhou your last comment is a different bug, can you open a new issue for it. Your last comment looks like a FW crash as I can see a scheduler error message which puts pipeline into an inconsistent state..
ClarexZhou commentedon May 28, 2019
keep monitor, and will report new issue once reproduce the scheduler error message caused ipc timed out issue again.
ClarexZhou commentedon May 28, 2019
Error occurred during aplay stress test at 25th.
Environment
Kernel: https://github.com/lyakh/linux/tree/ipc-timeout commit 9453da1 (with PR895)
FW:https://github.com/lyakh/sof/tree/ipc-timeout commit 47dc14d (with PR1354 and PR1470)
Kconfig: disable CONFIG_SND_SOC_SOF_DEBUG_ENABLE_DEBUGFS_CACHE in kernel config.
aplay_error_25_dmesg.log
aplay_error_25_logger_all.log
aplay_error_25_logger_error.log
emilchudzik commentedon May 31, 2019
@ClarexZhou reporting issue on non-official branch (official is master or alpha/beta/pv branches) is only debug, so please report it directly to owner od code.
Please double check issue on master or cml-hda... branch.
jajanusz commentedon Jun 17, 2019
@emilchudzik @ClarexZhou Can we close it? Looks like it turned out to be kernel-side issue.
ClarexZhou commentedon Jun 18, 2019
Close for not reproduced