Repository navigation
SOF: Multicore support failed when _widget_free introduced. #2824
Description
We call sof_pipeline_core_enable() in sof_widget_setup() to power up the corresponding DSP cores,
linux/sound/soc/sof/sof-audio.c
Line 160 in 9f92f3c
but not do the corresponding powering off Cores at sof_widget_free(), e.g. at runtime suspending,
linux/sound/soc/sof/sof-audio.c
Line 94 in 9f92f3c
this will lead to the subsequent run of the pipeline/widget after the runtime suspend/resume cycle on non-0 DSP core fail as the driver will keep think core 1 is already enabled according to the sdev->enabled_cores_mask,
which is not aligned with the FW side, the FW will complain with "ipc_process_on_core(): core 1 is disable" and reject to run it.
https://github.com/thesofproject/sof/blob/0b0733749c55b39c9e8699538b77c65f0c322397/src/ipc/ipc-common.c#L44
[ 272.220926] sof-audio-pci-intel-tgl 0000:00:1f.3: ipc tx succeeded: 0x30100000: GLB_TPLG_MSG: PIPE_NEW
[ 272.220929] sof-audio-pci-intel-tgl 0000:00:1f.3: widget PIPELINE.4.SSP1.IN setup complete
[ 272.220931] sof-audio-pci-intel-tgl 0000:00:1f.3: core_enable widget: PCM2P swidget core: 1
[ 272.220933] sof-audio-pci-intel-tgl 0000:00:1f.3: sdev->enabled_cores_mask: 0x3
[ 272.220935] sof-audio-pci-intel-tgl 0000:00:1f.3: sdev->enabled_cores_mask: 0x3
[ 272.220941] sof-audio-pci-intel-tgl 0000:00:1f.3: ipc tx: 0x30010000: GLB_TPLG_MSG: COMP_NEW
[ 272.221036] sof-audio-pci-intel-tgl 0000:00:1f.3: error: ipc error for 0x30010000 size 20
[ 272.221041] sof-audio-pci-intel-tgl 0000:00:1f.3: error: failed to load widget PCM2P
[ 272.221044] sof-audio-pci-intel-tgl 0000:00:1f.3: error: failed to restore pipeline after resume -13
[ 272.221050] sof-audio-pci-intel-tgl 0000:00:1f.3: ASoC: error at snd_soc_pcm_component_pm_runtime_get on 0000:00:1f.3: -5
[ 272.221113] NoCodec-2: soc_pcm_open() failed (-5)
[ 272.221118] smart-nocodec: ASoC: dpcm_be_dai_startup() failed at NoCodec-2 (-5)
[ 272.221122] smart-nocodec: dpcm_fe_dai_startup() failed (-5)
[ 272.233943] sof-audio-pci-intel-tgl 0000:00:1f.3: ipc rx: 0x90020000: GLB_TRACE_MSG
This is actually SOF common issue, not TGL specific, and happen in all scenarios where widgets free is used, including runtime suspend/resume, not dedicated for dynamic pipelines only.
Proposed solution:
add refcount to each DSP core, increase/decrease the refcount when a widget/pipeline using the Core is created/freed, and do the core power up/down and core enable/disable IPC only when needed.
Activity
plbossart commentedon Apr 7, 2021
@keyonjie with all due respect, this is a terrible bug description...
Can you please reword it with code pointers so that we don't have to reverse-engineer your line of thought, or why it's TGL-specific, and whether this is for dynamic pipelines or not.
[-]TGL: Multicore support failed[/-][+]SOF: Multicore support failed when _widget_free introduced.[/+]keyonjie commentedon Apr 8, 2021
Thanks for feedback, only had 3 minutes to heads up this on github last night, now revised.
ranj063 commentedon Apr 8, 2021
@keyonjie I agree we should power off cores properly depending on refcount with multicore. But we must be missing something in our suspend sequence too. We must not be updating the enabled cores mask when poering up secondary cores via IPC. Something to look into
keyonjie commentedon Apr 8, 2021
It looks to me that the powering up of the secondary cores works fine (the enabled cores mask set correctly), but the widget free or runtime suspending not clearing this mask for us. Sorry I assigned this to you as I think you know this part best.
keyonjie commentedon Apr 14, 2021
@plbossart since @ranj063 is in leave, I may take my old PR #888 back to address this issue.
keyonjie commentedon Apr 16, 2021
Hi @plbossart @ranj063 I managed to figure out solution for this in my PR #2850, please help to review.
kv2019i commentedon Apr 28, 2021
This is impacting a key feature in sof-dev, so applying P1.
ASoC: SOF: reset enabled_cores state at suspend
ASoC: SOF: reset enabled_cores state at suspend
ASoC: SOF: reset enabled_cores state at suspend
ASoC: SOF: reset enabled_cores state at suspend
plbossart commentedon May 6, 2021
@keyonjie isn't this issue solved by PR #2880 already? Please close if this is the case, thank you.
keyonjie commentedon May 7, 2021
Yes, let's close it, and file new multi-core issues if observed by CI after topology change thesofproject/sof#4153 merged.
ASoC: SOF: reset enabled_cores state at suspend
ASoC: SOF: reset enabled_cores state at suspend
ASoC: SOF: reset enabled_cores state at suspend
ASoC: SOF: reset enabled_cores state at suspend
ASoC: SOF: reset enabled_cores state at suspend
ASoC: SOF: reset enabled_cores state at suspend
UPSTREAM: ASoC: SOF: reset enabled_cores state at suspend