Skip to content

DSP panic after booting up #274

Closed
#424
Closed
DSP panic after booting up#274
#424
Assignees
xiulipan
Labels
BYTApplies to Baytrail platformP1Blocker bugs or important featuresbugSomething isn't working as expectedduplicateThis issue or pull request already exists
Milestone
1.2

Description

@ZhendanYang
ZhendanYang
opened on Aug 29, 2018

Summary:
DSP panic when baytrail boot up, but this doesn't happen every time. When I disable pulseaudio, this issue disappears. I git revert commit df0d701 in sof-master, after that, either I enable or disable pulseaudio, this issue can't be repreduced.

Reproduce steps:

  1. boot
  2. dmesg

Environment:
on byt
sof-dev : f9c923e5ca9b38df7ddc71f26b55eb2bd1ffed46
sof stable 1.2 : 7346243
soft : 41861aa
tplg: test-ssp2-mclk-0-I2S-volume-s16le-s16le-48k-19200k-codec.tplg

Log:
dmesg-panic.log
rmbox-panic.log
rmbox-t-panic.log

Activity

  1. ZhendanYang
    added
    bugSomething isn't working as expected
    BYTApplies to Baytrail platform
    regression
    P1Blocker bugs or important features
    on Aug 29, 2018

  2. lgirdwood commented on Aug 29, 2018

    @lgirdwood
    lgirdwood
    on Aug 29, 2018
    Member

    duplicate of #272

  3. lgirdwood
    added
    duplicateThis issue or pull request already exists
    P3Low-impact bugs or features
    and removed
    P1Blocker bugs or important features
    on Aug 29, 2018

  4. lgirdwood commented on Aug 30, 2018

    @lgirdwood
    lgirdwood
    on Aug 30, 2018
    Member

    duplicate of #210

  5. keqiaozhang
    added this to the 1.2 milestone on Aug 31, 2018
  6. mengdonglin
    modified the milestones: 1.2, TBD on Aug 31, 2018

  7. ZhendanYang commented on Sep 3, 2018

    @ZhendanYang
    ZhendanYang
    on Sep 3, 2018
    Author

    DSP panic can also happen after booting on APL using glk-003 branch sof, either I enable or disable pulseaudio. But kernel panic can happen only after booting when pulseaudio is enabled.
    sof : glk-003-drop-stable 0ead9e7
    kernel : bafb59032343459cc4019cb65e515595451357e7
    soft : d40087f
    log:
    dmesg-bootpanic.log
    rmbox-bootpanic.log
    rmbox-t-bootpanic.log
    dmesg-kernelpanic.log
    rmbox-kernelpanic.log
    rmbox-t-kernelpanic.log

    Also reproduced on APL and CNL with sof stable 1.2 branch. commit : 7346243

  8. ZhendanYang
    changed the title [-]DSP panic after booting up on BYT.[/-] [+]DSP panic after booting up on APL and BYT.[/+] on Sep 3, 2018
  9. ZhendanYang
    added
    APLApplies to Apollolake platform
    on Sep 3, 2018

  10. 25 remaining items

  11. ZhendanYang
    changed the title [-]DSP panic after booting up on APL, BYT and CNL.[/-] [+]DSP panic after booting up[/+] on Sep 12, 2018

  12. ZhendanYang commented on Sep 14, 2018

    @ZhendanYang
    ZhendanYang
    on Sep 14, 2018
    Author

    Latest status:
    Test with stable1.2 branch(f7ea2e6), enable and disable pulseaudio both will cause DSP panic, but seems the reproducibity is much less than before.
    Test with master branch(f17abe6), DSP panic occurs every time whether enable pulseaudio or not.

  13. lgirdwood commented on Sep 14, 2018

    @lgirdwood
    lgirdwood
    on Sep 14, 2018
    Member

    @ZhendanYang is the reason still [ 8.249425] sof-audio sof-audio: error: unknown reason 0dead00a ??

    This is the wait checker we have to detect incorrect wait transition states

  14. mengdonglin
    modified the milestones: TBD, 1.2 on Sep 15, 2018

  15. ZhendanYang commented on Sep 17, 2018

    @ZhendanYang
    ZhendanYang
    on Sep 17, 2018
    Author

    @lgirdwood
    No, the reason changed.
    With stable-1.2 branch, when pulseaudio enabled :
    [ 8.214583] sof-audio sof-audio: error : DSP panic!
    [ 8.214594] sof-audio sof-audio: panic: dsp_oops_offset 1329152 offset 1329152
    [ 8.214624] sof-audio sof-audio: error: invalid wait state
    [ 8.214630] sof-audio sof-audio: error: trace point 90010000
    [ 8.214636] sof-audio sof-audio: error: DSP Firmware Oops
    [ 8.214641] sof-audio sof-audio: error: Exception Cause: IllegalInstructionCause, Illegal instruction
    [ 8.214649] sof-audio sof-audio: EXCCAUSE 0x00000000 EXCVADDR 0x00000000 PS 0x00060125 SAR 0x00000011
    [ 8.214657] sof-audio sof-audio: EPC1 0xff2c29f9 EPC2 0xff2c0f1b EPC3 0xff2c0f1b EPC4 0xff2c0f1b
    [ 8.214665] sof-audio sof-audio: EPC5 0x00000000 EPC6 0x00000000 EPC7 0x00000000 DEPC 0x00000000
    [ 8.214672] sof-audio sof-audio: EPS2 0x00060520 EPS3 0x00060520 EPS4 0x00060520 EPS5 0x00000000
    [ 8.214680] sof-audio sof-audio: EPS6 0x00000000 EPS7 0x00000000 INTENABL 0x0007ef48 INTERRU 0x000020a2
    [ 8.214687] sof-audio sof-audio: stack dump from 0xff327d30
    dmesg-1.2-enable.log

    WIth master branch, when pulseaudio disabled :
    [ 7.898558] sof-audio sof-audio: error : DSP panic!
    [ 7.898596] sof-audio sof-audio: error: out of memory
    [ 7.898603] sof-audio sof-audio: error: trace point 00004170
    [ 7.898608] sof-audio sof-audio: error: DSP Firmware Oops
    [ 7.898614] sof-audio sof-audio: error: Exception Cause: IllegalInstructionCause, Illegal instruction
    [ 7.898622] sof-audio sof-audio: EXCCAUSE 0x00000000 EXCVADDR 0x00000000 PS 0x00060525 SAR 0x00000011
    [ 7.898630] sof-audio sof-audio: EPC1 0xff2ce8ed EPC2 0x00000000 EPC3 0x00000000 EPC4 0xff2cce8f
    [ 7.898637] sof-audio sof-audio: EPC5 0x00000000 EPC6 0x00000000 EPC7 0x00000000 DEPC 0x00000000
    [ 7.898645] sof-audio sof-audio: EPS2 0x00000000 EPS3 0x00000000 EPS4 0x00060720 EPS5 0x00000000
    [ 7.898653] sof-audio sof-audio: EPS6 0x00000000 EPS7 0x00000000 INTENABL 0x00008b48 INTERRU 0x000000a2
    [ 7.898659] sof-audio sof-audio: stack dump from 0xff327f20
    dmesg-disable-master.log

  16. xiulipan commented on Sep 17, 2018

    @xiulipan
    xiulipan
    on Sep 17, 2018
    Contributor

    @ZhendanYang

    [ 8.214624] sof-audio sof-audio: error: invalid wait state

    is same as

    [ 8.249425] sof-audio sof-audio: error: unknown reason 0dead00a

    @lgirdwood
    We still missing one PR #335
    to remove all waiti on BYT

    And we need to enable another DMA stop just like #342

    With updated #335 and #342, I think BYT could be good.

  17. xiulipan commented on Sep 18, 2018

    @xiulipan
    xiulipan
    on Sep 18, 2018
    Contributor

    @ZhendanYang
    #335 and #342 is merged on stable-1.2
    Please verify.

    I will send merge PR from stabe-1.2 to master.

  18. ZhendanYang commented on Sep 18, 2018

    @ZhendanYang
    ZhendanYang
    on Sep 18, 2018
    Author

    @xiulipan
    Verified. This issue can't reproduce on stable-1.2, but still exists on master.

  19. ZhendanYang
    removed
    br-1.2
    on Sep 18, 2018

  20. lgirdwood commented on Sep 20, 2018

    @lgirdwood
    lgirdwood
    on Sep 20, 2018
    Member

    @xiulipan the master branch out of memory may need heap size adjustment.. Btw it would be nice if panic() also included line number and filename in the dump it gives to the kernel.

  21. xiulipan commented on Sep 21, 2018

    @xiulipan
    xiulipan
    on Sep 21, 2018
    Contributor

    @lgirdwood
    OK, I create a issues #403 to track this

  22. xiulipan commented on Sep 26, 2018

    @xiulipan
    xiulipan
    on Sep 26, 2018
    Contributor

    @lgirdwood
    Now panic info patch set is test with BYT and APL, here is the result on BYT

    [ 5.685890] sof-audio sof-audio: error : DSP panic!
    [ 5.686049] sof-audio sof-audio: error: out of memory
    [ 5.686132] sof-audio sof-audio: error: trace point 00004170
    [ 5.686220] sof-audio sof-audio: error: panic happen at alloc.c:131

    128         /* always succeeds or panics */
129         if (alignment + bytes > cpu_heap->info.free) {
130                 trace_mem_error("eM1");
131                 panic(SOF_IPC_PANIC_MEM);
132         }

    Will send the PR first and start debut this issue.

  23. xiulipan
    added a commit that references this issue on Sep 26, 2018

    byt: enlarge system heap size

    a0b169d
  24. lgirdwood
    closed this as completedin #424on Sep 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

BYTApplies to Baytrail platformP1Blocker bugs or important featuresbugSomething isn't working as expectedduplicateThis issue or pull request already exists

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions