Repository navigation
[BUG] DSP panic during the suspend/resume test w/ audio on TGL-NOCODEC #8721
Copy link
Copy link
Closed
Labels
DSP panicDSP panic observedDSP panic observedI2SApplies to I2S bus for codec connectionApplies to I2S bus for codec connectionIPC4Issues observed with IPC4 (same IPC as Windows)Issues observed with IPC4 (same IPC as Windows)Intel Linux Daily testsThis issue can be found in internal Linux daily testsThis issue can be found in internal Linux daily testsP1Blocker bugs or important featuresBlocker bugs or important featuresbugSomething isn't working as expectedSomething isn't working as expectedsuspend-resumeIssues observed when doing system suspend and resumeIssues observed when doing system suspend and resume
Description
Describe the bug
Observed this issue in CI daily test, it happened during the suspend/resume test. the reproduction rate is high. Maybe it's a regression, I will check it later.
dmesg
[12543.439854] kernel: snd_sof:sof_ipc4_trigger_pipelines: sof-audio-pci-intel-tgl 0000:00:1f.3: trigger cmd: 0 state: 2
[12543.439867] kernel: snd_sof:sof_ipc4_log_header: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc tx : 0x13000002|0x1: GLB_SET_PIPELINE_STATE [data size: 16]
[12543.443817] kernel: snd_sof:sof_ipc4_log_header: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc rx : 0x1b0a0000|0x0: GLB_NOTIFICATION|EXCEPTION_CAUGHT
[12543.443825] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump start ]------------
[12543.443828] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: DSP panic!
[12543.443830] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: fw_state: SOF_FW_BOOT_COMPLETE (7)
[12543.443839] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0x00000005: module: ROM, state: FW_ENTERED, running
[12543.443947] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: FW is built with XCC toolchain
[12543.443951] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: error: DSP Firmware Oops
[12543.443953] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: EXCCAUSE 0x0000003f EXCVADDR 0x00000000 PS 0x00060e20 SAR 0x0000001b
[12543.443957] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: EPC1 0xbe059a37 EPC2 0x00000000 EPC3 0x00000000 EPC4 0x00000000
[12543.443961] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: EPC5 0x00000000 EPC6 0x00000000 EPC7 0x00000000 DEPC 0x00000000
[12543.443963] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: EPS2 0x00000000 EPS3 0x00000000 EPS4 0x00000000 EPS5 0x00000000
[12543.443966] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: EPS6 0x00000000 EPS7 0x00000000 INTENABL 0x00000000 INTERRU 0x00000000
[12543.443968] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: stack dump from 0x00000000
[12543.443971] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: AR registers:
[12543.443974] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0x0: be014fb8 be0b0000 00000004 be0b0010
[12543.443977] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0x10: be0afff0 00000004 00000020 00045147
[12543.443980] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0x20: be014179 be0aff70 be07bb30 be0b0010
[12543.443983] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0x30: be014179 be0aff70 be07bb30 be0b0010
[12543.443986] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump end ]------------
[12543.443988] kernel: snd_sof:sof_set_fw_state: sof-audio-pci-intel-tgl 0000:00:1f.3: fw_state change: 7 -> 8
[12543.444015] kernel: snd_sof:sof_ipc4_log_header: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc rx done : 0x1b0a0000|0x0: GLB_NOTIFICATION|EXCEPTION_CAUGHT
[12543.943887] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc timed out for 0x13000002|0x1
[12543.943897] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: Attempting to prevent DSP from entering D3 state to preserve context
[12543.943901] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ IPC dump start ]------------
[12543.943920] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: hda irq intsts 0x00000000 intlctl 0xc0000000 rirb 00
[12543.943925] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: dsp irq ppsts 0x00000000 adspis 0x00000000
[12543.943947] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: Host IPC initiator: 0x93000002|0x1|0x0, target: 0x0|0x0|0x80000000, ctl: 0x3
[12543.943952] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ IPC dump end ]------------
[12543.943955] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump start ]------------
[12543.943957] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: IPC timeout
[12543.943960] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: fw_state: SOF_FW_CRASHED (8)
[12543.943970] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0x00000005: module: ROM, state: FW_ENTERED, running
[12543.944080] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: FW is built with XCC toolchain
[12543.944084] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: error: DSP Firmware Oops
[12543.944087] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: EXCCAUSE 0x0000003f EXCVADDR 0x00000000 PS 0x00060e20 SAR 0x0000001b
[12543.944091] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: EPC1 0xbe059a37 EPC2 0x00000000 EPC3 0x00000000 EPC4 0x00000000
[12543.944095] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: EPC5 0x00000000 EPC6 0x00000000 EPC7 0x00000000 DEPC 0x00000000
[12543.944098] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: EPS2 0x00000000 EPS3 0x00000000 EPS4 0x00000000 EPS5 0x00000000
[12543.944101] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: EPS6 0x00000000 EPS7 0x00000000 INTENABL 0x00000000 INTERRU 0x00000000
[12543.944104] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: stack dump from 0x00000000
[12543.944106] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: AR registers:
[12543.944110] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0x0: be014fb8 be0b0000 00000004 be0b0010
[12543.944114] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0x10: be0afff0 00000004 00000020 00045147
[12543.944117] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0x20: be014179 be0aff70 be07bb30 be0b0010
[12543.944121] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0x30: be014179 be0aff70 be07bb30 be0b0010
[12543.944124] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump end ]------------
To Reproduce
~/sof-test/test-case/check-suspend-resume-with-audio.sh -l 5 -m playback
Reproduction Rate
High, >50%
Environment
- Branch name and commit hash of the 2 repositories: sof (firmware/topology) and linux (kernel driver).
- Kernel: {623c00b5e977}
- SOF: {b07d6e9161a6}
- Name of the topology file
- Topology: {sof-tgl-nocodec.tplg}
- Name of the platform(s) on which the bug is observed.
- Platform: {TGLU-NOCODEC-IPC4}
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
DSP panicDSP panic observedDSP panic observedI2SApplies to I2S bus for codec connectionApplies to I2S bus for codec connectionIPC4Issues observed with IPC4 (same IPC as Windows)Issues observed with IPC4 (same IPC as Windows)Intel Linux Daily testsThis issue can be found in internal Linux daily testsThis issue can be found in internal Linux daily testsP1Blocker bugs or important featuresBlocker bugs or important featuresbugSomething isn't working as expectedSomething isn't working as expectedsuspend-resumeIssues observed when doing system suspend and resumeIssues observed when doing system suspend and resume
Activity
keqiaozhang commentedon Jan 12, 2024
Confirmed that this is a sof regression and caused by:4bafdee
keqiaozhang commentedon Jan 15, 2024
@abonislawski @andyross @lgirdwood @ranj063 FYI
ASoC: SOF: ipc4-pcm: Workaround for firmware crash on stream stop
keqiaozhang commentedon Jan 16, 2024
kernel log with IPC payload enabled:
ujfalusi commentedon Jan 16, 2024
@keqiaozhang, do you have the full dmesg, the fragment is missing the DSP panic.
keqiaozhang commentedon Jan 16, 2024
@ujfalusi full dmesg attached.
dmesg.txt
lyakh commentedon Jan 16, 2024
Apparently hitting one of
__ASSERT()statements insys_heap_free(), which might either indicate a heap corruption or trying to free a wrong addressASoC: SOF: ipc4-pcm: Workaround for crashed firmware on system suspend
29 remaining items
ASoC: SOF: ipc4-pcm: Workaround for crashed firmware on system suspend
ASoC: SOF: ipc4-pcm: Workaround for crashed firmware on system suspend
ASoC: SOF: ipc4-pcm: Workaround for crashed firmware on system suspend
ASoC: SOF: ipc4-pcm: Workaround for crashed firmware on system suspend
ASoC: SOF: ipc4-pcm: Workaround for crashed firmware on system suspend
ASoC: SOF: ipc4-pcm: Workaround for crashed firmware on system suspend
ASoC: SOF: ipc4-pcm: Workaround for crashed firmware on system suspend
ASoC: SOF: ipc4-pcm: Workaround for crashed firmware on system suspend
ASoC: SOF: ipc4-pcm: Workaround for crashed firmware on system suspend
UPSTREAM: ASoC: SOF: ipc4-pcm: Workaround for crashed firmware on sys…
UPSTREAM: ASoC: SOF: ipc4-pcm: Workaround for crashed firmware on sys…
ASoC: SOF: ipc4-pcm: Workaround for crashed firmware on system suspend
UPSTREAM: ASoC: SOF: ipc4-pcm: Workaround for crashed firmware on sys…
UPSTREAM: ASoC: SOF: ipc4-pcm: Workaround for crashed firmware on sys…
UPSTREAM: ASoC: SOF: ipc4-pcm: Workaround for crashed firmware on sys…