Skip to content

[HACK] soundwire: add traces to debug bank switch issues - #5042

Closed
bardliao wants to merge 12 commits into
thesofproject:topic/sof-devfrom
bardliao:debug-bank-switch-error
Closed

bardliao wants to merge 12 commits into
thesofproject:topic/sof-devfrom
bardliao:debug-bank-switch-error

Conversation

@bardliao

@bardliao bardliao commented Jun 6, 2024

Copy link
Copy Markdown
Collaborator

No description provided.

@plbossart

Copy link
Copy Markdown
Member

Build is stuck for over 3 hours now. I can't even see it in the internal build results as having started or failed.

@plbossart

Copy link
Copy Markdown
Member

SOFCI TEST

@plbossart

plbossart commented Jun 6, 2024 •

Copy link
Copy Markdown
Member

Not sure what's going on, now all Linux PRs seem to be building with a Start Time: 2024-06-06 09:16:09 UTC or later

image

Looks like a server restarted @fredoh9 @marc-hb

Comment thread drivers/soundwire/cadence_master.c Outdated
Comment thread drivers/soundwire/bus.c Outdated
@bardliao
bardliao force-pushed the debug-bank-switch-error branch from b20e0f1 to 6aa76f5 Compare June 6, 2024 10:50
@bardliao

bardliao commented Jun 6, 2024

Copy link
Copy Markdown
Collaborator Author

SOFCI TEST

@bardliao

bardliao commented Jun 6, 2024

Copy link
Copy Markdown
Collaborator Author
[ 1002.420696] sdw_acquire_bus_lock: acquiring locks for stream subdevice #0-Playback
[ 1002.420698] sdw_acquire_bus_lock: acquired locks for stream subdevice #0-Playback
[ 1002.420699] _sdw_prepare_stream: start for stream subdevice #0-Playback
[ 1002.420702] soundwire sdw-master-0-2: sdw_transfer
[ 1002.420756] soundwire_intel soundwire_intel.link.2: sdw_cdns_irq tx complete
[ 1002.420763] soundwire sdw-master-0-2: sdw_transfer
[ 1002.420791] soundwire_intel soundwire_intel.link.2: sdw_cdns_irq tx complete
[ 1002.420794] soundwire sdw-master-0-2: sdw_transfer
[ 1002.420825] soundwire_intel soundwire_intel.link.2: sdw_cdns_irq tx complete
[ 1002.420828] soundwire sdw-master-0-2: sdw_transfer
[ 1002.420857] soundwire_intel soundwire_intel.link.2: sdw_cdns_irq tx complete
[ 1002.420860] soundwire sdw-master-0-2: sdw_transfer
[ 1002.420895] soundwire_intel soundwire_intel.link.2: sdw_cdns_irq tx complete
[ 1002.420897] soundwire sdw-master-0-2: sdw_transfer
[ 1002.420929] soundwire_intel soundwire_intel.link.2: sdw_cdns_irq tx complete
[ 1002.420932] soundwire sdw-master-0-2: sdw_transfer
[ 1002.420957] soundwire_intel soundwire_intel.link.2: sdw_cdns_irq tx complete
[ 1002.420960] soundwire sdw-master-0-2: sdw_transfer
[ 1002.420989] soundwire_intel soundwire_intel.link.2: sdw_cdns_irq tx complete
[ 1002.420991] soundwire sdw-master-0-2: sdw_transfer
[ 1002.421022] soundwire_intel soundwire_intel.link.2: sdw_cdns_irq tx complete
[ 1002.421034] soundwire sdw-master-0-3: sdw_transfer
[ 1002.421062] soundwire_intel soundwire_intel.link.3: sdw_cdns_irq tx complete
[ 1002.421065] soundwire sdw-master-0-3: sdw_transfer
[ 1002.421094] soundwire_intel soundwire_intel.link.3: sdw_cdns_irq tx complete
[ 1002.421096] soundwire sdw-master-0-3: sdw_transfer
[ 1002.421129] soundwire_intel soundwire_intel.link.3: sdw_cdns_irq tx complete
[ 1002.421132] soundwire sdw-master-0-3: sdw_transfer
[ 1002.421159] soundwire_intel soundwire_intel.link.3: sdw_cdns_irq tx complete
[ 1002.421162] soundwire sdw-master-0-3: sdw_transfer
[ 1002.421198] soundwire_intel soundwire_intel.link.3: sdw_cdns_irq tx complete
[ 1002.421200] soundwire sdw-master-0-3: sdw_transfer
[ 1002.421229] soundwire_intel soundwire_intel.link.3: sdw_cdns_irq tx complete
[ 1002.421232] soundwire sdw-master-0-3: sdw_transfer
[ 1002.421260] soundwire_intel soundwire_intel.link.3: sdw_cdns_irq tx complete
[ 1002.421263] soundwire sdw-master-0-3: sdw_transfer
[ 1002.421295] soundwire_intel soundwire_intel.link.3: sdw_cdns_irq tx complete
[ 1002.421297] soundwire sdw-master-0-3: sdw_transfer
[ 1002.421323] soundwire_intel soundwire_intel.link.3: sdw_cdns_irq tx complete
[ 1002.421333] _sdw_prepare_stream: before bank switch for stream subdevice #0-Playback
[ 1002.421337] soundwire sdw-master-0-2: do_transfer_defer
[ 1002.421344] soundwire sdw-master-0-3: do_transfer_defer
[ 1005.439423] soundwire sdw-master-0-2: Controller Timed out on bank switch
[ 1005.439641] soundwire sdw-master-0-2: multi link bank switch failed: -110
[ 1005.439763] _sdw_prepare_stream: do_bank_switch failed for stream subdevice #0-Playback: -110
[ 1005.439863] sdw_release_bus_lock: releasing locks for stream subdevice #0-Playback
[ 1005.439873] sdw_release_bus_lock: released locks for stream subdevice #0-Playback
[ 1005.439880]  SDW2-Playback: ASoC: error at snd_soc_link_prepare on SDW2-Playback: -110
[ 1005.439913]  SDW2-Playback: ASoC: error at __soc_pcm_prepare on SDW2-Playback: -110
[ 1005.439945]  Speaker: ASoC: error at dpcm_be_dai_prepare on Speaker: -110
[ 1005.439983]  Speaker: ASoC: error at dpcm_fe_dai_prepare on Speaker: -110
[ 1005.440013] sdw_acquire_bus_lock: acquiring locks for stream subdevice #0-Playback
[ 1005.440024] sdw_acquire_bus_lock: acquired locks for stream subdevice #0-Playback
[ 1005.440028] _sdw_prepare_stream: start for stream subdevice #0-Playback

Still can't get the reason of "Controller Timed out on bank switch". Just didn't get the CDNS_MCP_INT_RX_WL irq.

@bardliao
bardliao force-pushed the debug-bank-switch-error branch from 6aa76f5 to b167bbc Compare June 6, 2024 14:19
@plbossart

Copy link
Copy Markdown
Member

well this is progress in my book @bardliao. At least we know it's not the jack interrupt, it's not a race condition between bank switch and other writes, it's squarely a missed interrupt issue.

Do you mind adding a log when we disable the SoundWire interrupts? I don't think it's the reason, because we had multiple transfers active before the bank switch, but it'd prove that the interrupts are on.

@bardliao

bardliao commented Jun 7, 2024

Copy link
Copy Markdown
Collaborator Author

SOFCI TEST

2 similar comments
@bardliao

bardliao commented Jun 7, 2024

Copy link
Copy Markdown
Collaborator Author

SOFCI TEST

@bardliao

bardliao commented Jun 7, 2024

Copy link
Copy Markdown
Collaborator Author

SOFCI TEST

@plbossart

Copy link
Copy Markdown
Member

Something's not right in the results, I can't see the traces in the logs.
dmesg6.log

@plbossart

Copy link
Copy Markdown
Member

SOFCI TEST

@plbossart plbossart mentioned this pull request Jun 7, 2024
plbossart and others added 12 commits June 12, 2024 13:45
Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Testing the theory that if the interrupt happens during a bank switch
the first read will be blocked and since the interrupts are now
disabled the bank switch will fail as well.

Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Signed-off-by: Bard Liao <yung-chuan.liao@linux.intel.com>
Signed-off-by: Bard Liao <yung-chuan.liao@linux.intel.com>
We want to make sure the sync_go does happen after the last write

Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Follow the recommended programming flows.

Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
For some reason, we don't have a symmetry between clock stop and clock
start. We should disable the interrupts first on clock stop, and
re-enable them last on clock restart.

Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Follow recommended programming flows.

Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Test hypothesis that a delay might help with rate conditions

Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
The exclusion between regular and deferred messages doesn't seem quite
right.

Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
These registers come straight from platform firmware properties, for
integration/debug it's useful to know the values directly from the
kernel logs.

Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Signed-off-by: Bard Liao <yung-chuan.liao@linux.intel.com>
@bardliao
bardliao force-pushed the debug-bank-switch-error branch from d2ab3e1 to 0e7211a Compare June 12, 2024 05:45
@bardliao bardliao closed this Jun 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants