Skip to content

Add IPC regs dump when IPC timeout happen - #872

Merged
plbossart merged 3 commits into
thesofproject:topic/sof-devfrom
xiulipan:pr/ipcdump
Apr 29, 2019
Merged

plbossart merged 3 commits into
thesofproject:topic/sof-devfrom
xiulipan:pr/ipcdump

Conversation

@xiulipan

@xiulipan xiulipan commented Apr 25, 2019 •

Copy link
Copy Markdown

Add new platform specific ops to dump IPC regs when IPC timeout happen.

This is only a framework RFC, please help to review and will add more detail for the logs.

Only add APL and CNL+ platforms with raw data.

@xiulipan
xiulipan marked this pull request as ready for review April 25, 2019 21:01

@plbossart plbossart left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xiulipan thanks for starting this, it's a good idea. Couple of suggested fixes below

Comment thread sound/soc/sof/intel/hda.c Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get what the %s does here?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see the comment I made below.

Comment thread sound/soc/sof/intel/hda.c Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parameter alignment seems inconsistent?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if we really need them. Since not all of them are useful

Comment thread sound/soc/sof/intel/hda.c Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same, why do you need the %s for?

Comment thread sound/soc/sof/ipc.c Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

snd_sof_ipc_dump? we have IPC registers but others may have more stuff to log?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also thinks about snd_sof_dsp_dbg_dump(ipc->sdev, SOF_DBG_REGS | SOF_DBG_MBOX | SOF_DBG_IPC);
but I found that we never used the flag in snd_sof_dsp_dbg_dump(struct snd_sof_dev *sdev, u32 flags)
So I just create a new function for this.

Comment thread sound/soc/sof/intel/hda.c Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we use rate_limited versions?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is a rate_limited version here?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dev_err_ratelimited()

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not think we need here. Since IPC timed out will be a blocker for the FW, the FW will be dead if an timed out happen. I think there will not be a flooding.

@xiulipan

Copy link
Copy Markdown
Author

@lgirdwood @ranj063
Could you have a look if this as you expected. I am not sure which regs I need to check and parse.

@xiulipan

Copy link
Copy Markdown
Author

@plbossart

This actually some proposal from @lgirdwood. The %s is some reservation for future parse to string to help understand.
But when I try to do the implement, I found that it seems we did not have so many bits for this in one reg.

@lgirdwood @ranj063 Any idea

#846 (comment)

IPC timeouts are often difficult to debug as we have two state machines running, one within the driver and the other within the FW.

This feature is to add useful debug output to the kernel when it detects and IPC timeout. i.e.

<< dmesg after current IPC timeout is printed >>
error: IPC host state: HOST_TO_DSP_CMD, DSP_TO_HOST_NOTIFICATION
error: IPC host mask: DSP_TO_HOST_NOTIFICATION
error: IPC host raw: status 0x00000003 mask 0x0000001
error: IPC dsp state: DSP_TO_HOST_NOTIFICATION
error: IPC dsp mask: DSP_TO_HOST_NOTIFICATION
error: IPC dsp raw: status 0x00000001 mask 0x0000001

The intention here is to show the IPC state machines for both DSP and host and show the pending operations in a high level manner alongside the raw status and mask.

@ranj063

ranj063 commented Apr 26, 2019

Copy link
Copy Markdown
Collaborator

This actually some proposal from @lgirdwood. The %s is some reservation for future parse to string to help understand.
But when I try to do the implement, I found that it seems we did not have so many bits for this in one reg.

@xiulipan the host/dsp state is something we will have to infer from the IPC registers I think. For example, hipct & HDA_DSP_REG_HIPCT_BUSY indicates a new message from DSP, so this means the dsp state is DSP_TO_HOST_CMD.

@lgirdwood please correct me if I am wrong

@plbossart

Copy link
Copy Markdown
Member

This actually some proposal from @lgirdwood. The %s is some reservation for future parse to string to help understand.
But when I try to do the implement, I found that it seems we did not have so many bits for this in one reg.

I don't want 'reservation for future stuff', since these kind of things tend to never come and are left as placeholder until a janitor (me in most cases) comes in. Please submit what exists today and an update if and when an extension is needed.

@xiulipan

Copy link
Copy Markdown
Author

@plbossart @ranj063
I have test with IPC timeout issue. And will update the PR withe the RAW register now.
I think the IPC state can be added later. I will remove the placeholder here.

Dump IPC status when IPC timed out. IPC status is platform specific and
need bind to plaform.

Signed-off-by: Pan Xiuli <xiuli.pan@linux.intel.com>
Add IPC dump function for APL plaform

Signed-off-by: Pan Xiuli <xiuli.pan@linux.intel.com>
Add ipc dump function to CNL+ platforms.

Signed-off-by: Pan Xiuli <xiuli.pan@linux.intel.com>

@plbossart plbossart left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but someone else must approve.

@plbossart plbossart changed the title [RFC][WIP]Add IPC regs dump when IPC timeout happen Add IPC regs dump when IPC timeout happen Apr 29, 2019
@plbossart

Copy link
Copy Markdown
Member

edited title to remove the RFC/WIP tags

@plbossart
plbossart merged commit a5ea42b into thesofproject:topic/sof-dev Apr 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants