Skip to content

Stress: Failed to restore the playback from S3 after more than 80 suspend/resume. #637

Closed

Description

@markyang

Summary:
Failed to restore the playback from S3 after more than 80 times suspend/resume.
sof-logger is empty. sof-logger -t log is attached below.

dmesg:

[ 7944.714311] Restarting tasks ... done.
[ 7944.721317] video LNXVIDEO:00: Restoring backlight state
[ 7944.721334] PM: suspend exit
[ 7944.722676] sof-audio sof-audio: pcm: trigger stream 0 dir 0 cmd 6
[ 7944.722681] sof-audio sof-audio: PCM: resume stream 0 dir 0 DMA position 0
[ 7944.722683] sof-audio sof-audio: pcm: hw params stream 0 dir 0
[ 7944.722688] sof-audio sof-audio: generating page table for 00000000f58accdc size 0xff00 pages 16
[ 7944.722710] sof-audio sof-audio: period_bytes:0xfc0
[ 7944.722712] sof-audio sof-audio: periods:16
[ 7944.722721] sof-audio sof-audio: stream_tag 1
[ 7944.726751] sof-audio sof-audio: ipc tx: 0x60010000: GLB_STREAM_MSG: PCM_PARAMS
[ 7944.726916] sof-audio sof-audio: ipc tx succeeded: 0x60010000: GLB_STREAM_MSG: PCM_PARAMS
[ 7944.756250] sof-audio sof-audio: ipc tx: 0x60040000: GLB_STREAM_MSG: TRIG_START
[ 7944.756419] sof-audio sof-audio: ipc tx succeeded: 0x60040000: GLB_STREAM_MSG: TRIG_START
[ 7944.756506] sof-audio sof-audio: pcm: trigger stream 0 dir 0 cmd 0
[ 7944.756785] sof-audio sof-audio: ipc tx: 0x60050000: GLB_STREAM_MSG: TRIG_STOP
[ 7945.061694] sof-audio sof-audio: error: ipc timed out for 0x60050000 size 0xc
[ 7945.061715] sof-audio sof-audio: status: fw entered - code 00000005
[ 7945.061743] sof-audio sof-audio: error: unexpected fault 0x00000000 trace 0x00004000
[ 7945.061750] sof-audio sof-audio: error: waking up any trace sleepers
[ 7945.061779]  Port5: ASoC: trigger FE failed -110
[ 7945.063124] sof-audio sof-audio: pcm: trigger stream 0 dir 0 cmd 1
[ 7945.365692] sof-audio sof-audio: error: ipc timed out for 0x60040000 size 0xc
[ 7945.365712] sof-audio sof-audio: status: fw entered - code 00000005
[ 7945.365741] sof-audio sof-audio: error: unexpected fault 0x00000000 trace 0x00004000
[ 7945.365748] sof-audio sof-audio: error: waking up any trace sleepers
[ 7945.365777]  Port5: ASoC: trigger FE failed -110
[ 7945.365792] sof-audio sof-audio: pcm: trigger stream 0 dir 0 cmd 0
[ 7945.669689] sof-audio sof-audio: error: ipc timed out for 0x60050000 size 0xc
[ 7945.669709] sof-audio sof-audio: status: fw entered - code 00000005
[ 7945.669737] sof-audio sof-audio: error: unexpected fault 0x00000000 trace 0x00004000
[ 7945.669744] sof-audio sof-audio: error: waking up any trace sleepers
[ 7945.669773]  Port5: ASoC: trigger FE failed -110
[ 7946.278296] sof-audio sof-audio: pcm: free stream 0 dir 0
[ 7946.581705] sof-audio sof-audio: error: ipc timed out for 0x60030000 size 0xc
[ 7946.581725] sof-audio sof-audio: status: fw entered - code 00000005
[ 7946.581754] sof-audio sof-audio: error: unexpected fault 0x00000000 trace 0x00004000
[ 7946.581762] sof-audio sof-audio: error: waking up any trace sleepers
[ 7946.581992] sof-audio sof-audio: pcm: close stream 0 dir 0
[ 7948.885712] sof-audio sof-audio: error: ipc timed out for 0x40010000 size 0x40
[ 7948.885730] sof-audio sof-audio: status: fw entered - code 00000005
[ 7948.885757] sof-audio sof-audio: error: unexpected fault 0x00000000 trace 0x00004000
[ 7948.885770] sof-audio sof-audio: error: ctx_save ipc error during suspend -110
[ 7948.885790] sof-audio-pci 0000:00:0e.0: can't suspend (snd_sof_runtime_suspend [snd_sof] returned -110)
[ 7949.904352] PM: suspend entry (deep)
[ 7949.904356] PM: Syncing filesystems ... done.
[ 7949.908107] Freezing user space processes ... (elapsed 0.001 seconds) done.
[ 7949.909743] OOM killer disabled.
[ 7949.909745] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
[ 7949.911313] Suspending console(s) (use no_console_suspend to debug)
[ 7950.349719] sof-audio sof-audio: error: ipc timed out for 0x40010000 size 0x40
[ 7950.349730] sof-audio sof-audio: status: fw entered - code 00000005
[ 7950.349755] sof-audio sof-audio: error: unexpected fault 0x00000000 trace 0x00004000
[ 7950.349780] sof-audio sof-audio: error: ctx_save ipc error during suspend -110
[ 7950.349795] dpm_run_callback(): pci_pm_suspend_late+0x0/0x30 returns -110
[ 7950.349819] PM: Device 0000:00:0e.0 failed to suspend late: error -110
[ 7950.380452] i2c_designware i2c_designware.0: timeout in disabling adapter
[ 7950.409063] i2c_designware i2c_designware.1: timeout in disabling adapter
[ 7950.437791] i2c_designware i2c_designware.2: timeout in disabling adapter
[ 7950.466780] i2c_designware i2c_designware.3: timeout in disabling adapter
[ 7950.495459] i2c_designware i2c_designware.4: timeout in disabling adapter
[ 7950.524421] i2c_designware i2c_designware.5: timeout in disabling adapter
[ 7950.553421] i2c_designware i2c_designware.6: timeout in disabling adapter
[ 7950.582374] i2c_designware i2c_designware.7: timeout in disabling adapter
[ 7950.582434] PM: late suspend of devices failed
[ 7950.582774] sof-audio-pci 0000:00:0e.0: Turning i915 HDAC power on
[ 7950.582814] sof-audio sof-audio: Debug PCIR: 00000000 at  00000044
[ 7950.582824] sof-audio sof-audio: Debug PCIR: 00000000 at  00000048
[ 7950.582831] sof-audio sof-audio: Debug PCIW: 00000000 at  00000048
[ 7950.582847] sof-audio sof-audio: Debug PCIR: 00000000 at  00000048
[ 7950.582855] sof-audio sof-audio: Debug PCIW: 00000040 at  00000048
[ 7950.582869] sof-audio sof-audio: Debug PCIR: 00000000 at  00000048
[ 7950.582877] sof-audio sof-audio: Debug PCIW: 00000000 at  00000048
[ 7950.582914] sof-audio sof-audio: Debug PCIR: 00000000 at  00000048
[ 7950.582921] sof-audio sof-audio: Debug PCIW: 00000040 at  00000048
[ 7950.584549] sof-audio sof-audio: loading firmware
[ 7950.584574] sof-audio sof-audio: Debug PCIR: 00000000 at  00000048
[ 7950.584581] sof-audio sof-audio: Debug PCIW: 00000000 at  00000048
[ 7950.584591] sof-audio sof-audio: Debug PCIR: 00000000 at  00000044
[ 7950.584597] sof-audio sof-audio: Debug PCIW: 00000004 at  00000044
[ 7950.584601] sof-audio sof-audio: booting DSP firmware
[ 7950.584814] sof-audio sof-audio: period_bytes:0x0
[ 7950.584819] sof-audio sof-audio: periods:1
[ 7950.584894] sof-audio sof-audio: unstall/run core: core_mask = 1
[ 7950.584901] sof-audio sof-audio: DSP core(s) enabled? 1 : core_mask 1
[ 7950.892926] ata2: SATA link down (SStatus 4 SControl 300)
[ 7950.892954] ata1: SATA link down (SStatus 4 SControl 300)
[ 7951.085727] sof-audio sof-audio: error: waiting for HIPCIE done, reg: 0x0
[ 7951.085731] sof-audio sof-audio: status: fw entered - code 00000005
[ 7951.085735] sof-audio sof-audio: error: status = 0x00000000 panic = 0x00004000
[ 7951.085738] sof-audio sof-audio: status: fw entered - code 00000005
[ 7951.107723] sof-audio sof-audio: DSP core(s) enabled? 0 : core_mask 3
[ 7951.107733] sof-audio sof-audio: error: could not get stream with stream tag0
[ 7951.107738] sof-audio sof-audio: error: load fw failed err: -19
[ 7951.107745] sof-audio sof-audio: unknown ROM status value ffffffff
[ 7951.107752] sof-audio sof-audio: error: status = 0xffffffff panic = 0xffffffff
[ 7951.107757] sof-audio sof-audio: unknown ROM status value ffffffff
[ 7951.107764] sof-audio sof-audio: error: load fw failed err: -19
[ 7951.107768] sof-audio sof-audio: error: failed to reset DSP
[ 7951.107773] sof-audio sof-audio: error: failed to boot DSP firmware after resume -19
[ 7951.107786] dpm_run_callback(): pci_pm_resume+0x0/0xa0 returns -19
[ 7951.107804] PM: Device 0000:00:0e.0 failed to resume: error -19

Test steps:

  1. aplay -Dhw:0,0 -c2 -fS16_LE -vv -i sine-16k-1h-stereo-1k.wav #playback a long sine
  2. sudo ./stress_pm_S3nocheck.sh

Test env:
sof master: 6e14892
sof tool: ad3f3e4
kernel sof-dev: 4a3aa5f
tplg: sof-apl-pcm512x.tplg-ad3f3e4

Log:
dmesg-apl.log
logger-t.log
stress_pm_S3nocheck.sh.txt

Activity

  1. added
    APLApplies to Apollolake platform
    bugSomething isn't working as expected
    on Nov 27, 2018
  2. changed the title [-]Failed to restore the playback from S3 after more than 80 suspend/resume.[/-] [+]Stress: Failed to restore the playback from S3 after more than 80 suspend/resume.[/+] on Nov 27, 2018
  3. lgirdwood commented on Nov 29, 2018

    @lgirdwood
    Member

    Two points

    1. Driver sends STOP after resuming stream ???
    [ 7944.722676] sof-audio sof-audio: pcm: trigger stream 0 dir 0 cmd 6
    [ 7944.722681] sof-audio sof-audio: PCM: resume stream 0 dir 0 DMA position 0
    [ 7944.722683] sof-audio sof-audio: pcm: hw params stream 0 dir 0
    [ 7944.722688] sof-audio sof-audio: generating page table for 00000000f58accdc size 0xff00 pages 16
    [ 7944.722710] sof-audio sof-audio: period_bytes:0xfc0
    [ 7944.722712] sof-audio sof-audio: periods:16
    [ 7944.722721] sof-audio sof-audio: stream_tag 1
    [ 7944.726751] sof-audio sof-audio: ipc tx: 0x60010000: GLB_STREAM_MSG: PCM_PARAMS
    [ 7944.726916] sof-audio sof-audio: ipc tx succeeded: 0x60010000: GLB_STREAM_MSG: PCM_PARAMS
    [ 7944.756250] sof-audio sof-audio: ipc tx: 0x60040000: GLB_STREAM_MSG: TRIG_START
    [ 7944.756419] sof-audio sof-audio: ipc tx succeeded: 0x60040000: GLB_STREAM_MSG: TRIG_START
    [ 7944.756506] sof-audio sof-audio: pcm: trigger stream 0 dir 0 cmd 0
    [ 7944.756785] sof-audio sof-audio: ipc tx: 0x60050000: GLB_STREAM_MSG: TRIG_STOP
    [ 7945.061694] sof-audio sof-audio: error: ipc timed out for 0x60050000 size 0xc
    
    1. FW timeout cause by STOP (should either return error or stop).
  4. wenqingfu commented on Dec 20, 2018

    @wenqingfu

    dup of thesofproject/linux#370? Can you please re-test on the GLK PV release or the tip of master?

  5. lgirdwood commented on Dec 21, 2018

    @lgirdwood
    Member

    Closed as duplicate

  6. markyang commented on Dec 28, 2018

    @markyang
    Author

    Summary:
    This issue can still be reproduced on APL with PCM512x after 47 times suspend/resume.

    Test steps:
    aplay -Dhw:0,0 -c2 -fS16_LE -vv -i sine-16k-1h-stereo-1k.wav #playback a long sine
    sudo ./stress_pm_S3nocheck.sh

    Test env:
    sof master: a2e6f82
    kernel sof-dev: 62aa569
    tplg: sof-apl-pcm512x.tplg

    Log:
    logger-t.log
    dmesg-apl.txt

  7. libinyang commented on Jan 7, 2019

    @libinyang
    Contributor

    @markyang @stevyan
    updated to the latest kernel, sof. It seems this bug can't be reproduced.

    BTW: Not sure it is exactly duplicated with thesofproject/linux#370, as 370 doesn't run with aplay. The test cases are different.

  8. markyang commented on Jan 7, 2019

    @markyang
    Author
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

APLApplies to Apollolake platformP2Critical bugs or normal featuresbugSomething isn't working as expectedduplicateThis issue or pull request already exists

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions