Skip to content

Kernel panic after unloading and reloading driver #144

Closed
Closed
Kernel panic after unloading and reloading driver#144
Assignees
bardliao
Labels
CNLApplies to CannonlakeP1Blocker bugs or important featuresbugSomething isn't working

Description

@ZhendanYang
ZhendanYang
opened on Sep 18, 2018

Summary:
Run the "soft/kmod_scripts/sof_bootone.sh" script, which means to do the module reload once, but after unloading and reloading driver, kernel panic happens.

[ 31.980316] sof-audio sof-audio: warn: can't find PCM with DAI ID 0
[ 31.980776] sof-audio sof-audio: tag 1 not opened!
[ 31.980817] sof-audio sof-audio: DSP core(s) enabled? 0 : core_mask f
[ 32.301721] sof-audio-pci 0000:00:1f.3: PCI DSP detected
[ 32.301952] sof-audio-pci 0000:00:1f.3: No matching ASoC machine driver found - using nocodec

[ 32.306463] Call Trace:
[ 32.306468] hda_codec_i915_init+0x19/0x60 [snd_sof_intel_hda]
[ 32.306472] hda_dsp_probe+0x458/0x8f0 [snd_sof_intel_hda_common]
[ 32.306477] sof_probe+0x145/0x320 [snd_sof]
[ 32.306483] platform_drv_probe+0x3a/0x90
[ 32.306485] ? driver_sysfs_add+0x75/0xd0
[ 32.306487] driver_probe_device+0x2fa/0x470
[ 32.306489] ? __driver_attach+0x100/0x100
[ 32.306491] bus_for_each_drv+0x66/0xb0
[ 32.306494] __device_attach+0xd9/0x150
[ 32.306496] bus_probe_device+0x8a/0xa0
[ 32.306497] device_add+0x3f7/0x620
[ 32.306500] ? _cond_resched+0x15/0x40
[ 32.306503] platform_device_add+0x10e/0x260
[ 32.306506] platform_device_register_full+0xf0/0x100
[ 32.306509] sof_create_platform_device+0x5d/0xa0 [snd_sof]
[ 32.306513] sof_pci_probe+0x232/0x346 [sof_pci_dev]
[ 32.306518] local_pci_probe+0x42/0xa0
[ 32.306520] ? pci_assign_irq+0x27/0x130
[ 32.306522] pci_device_probe+0x146/0x1b0
[ 32.306524] driver_probe_device+0x2fa/0x470
[ 32.306526] __driver_attach+0xdc/0x100
[ 32.306528] ? driver_probe_device+0x470/0x470
[ 32.306529] bus_for_each_dev+0x76/0xc0
[ 32.306531] bus_add_driver+0x161/0x260
[ 32.306533] ? 0xffffffffc05c9000
[ 32.306535] driver_register+0x5b/0xe0
[ 32.306536] ? 0xffffffffc05c9000
[ 32.306539] do_one_initcall+0x46/0x1c8
[ 32.306542] ? kobject_uevent_env+0x117/0x810
[ 32.306544] ? _cond_resched+0x15/0x40
[ 32.306546] ? kmem_cache_alloc_trace+0x3e/0x1c0
[ 32.306549] do_init_module+0x5b/0x201
[ 32.306552] load_module+0x2689/0x2e00
[ 32.306556] ? __do_sys_finit_module+0xe9/0x110
[ 32.306558] __do_sys_finit_module+0xe9/0x110
[ 32.306560] do_syscall_64+0x55/0x110
[ 32.306563] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 32.306565] RIP: 0033:0x7f6ae6e7da49
[ 32.306565] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 0f b4 2c 00 f7 d8 64 89 01 48
[ 32.306592] RSP: 002b:00007ffdaeec0b08 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 32.306593] RAX: ffffffffffffffda RBX: 000055842d93ce90 RCX: 00007f6ae6e7da49
[ 32.306594] RDX: 0000000000000000 RSI: 000055842d1a6cee RDI: 0000000000000004
[ 32.306595] RBP: 000055842d1a6cee R08: 0000000000000000 R09: 0000000000000000
[ 32.306596] R10: 0000000000000004 R11: 0000000000000246 R12: 0000000000000000
[ 32.306597] R13: 000055842d93d3a0 R14: 0000000000040000 R15: 000055842d93ce90
[ 32.306598] ---[ end trace 37c92cd1f0a66c62 ]---

[ 32.306646] sof-audio sof-audio: error: failed to probe DSP -16
[ 32.306654] sof-audio: probe of sof-audio failed with error -16
[ 32.306933] sdhci-pci 0000:00:14.5: SDHCI controller found [8086:9df5] (rev 11)
[ 32.307753] sof-nocodec sof-nocodec: ASoC: CPU DAI SSP0 Pin not registered

Test env:
kernel: c3ba91f
sof stable-1.2: 0875f0c
soft: d77beaf
tplg: sof-apl-nocodec.tplg on APL and test-ssp0-mclk-0-I2S-volume-s16le-s16le-48k-24000k-nocodec.tplg on CNL

Log:
dmesg-apl.log
dmesg-cnl.log

Activity

  1. ZhendanYang
    added
    bugSomething isn't working
    P1Blocker bugs or important features
    APLApplies to ApolloLake platform
    CNLApplies to Cannonlake
    br-sof-dev
    on Sep 18, 2018
  2. mengdonglin
    added
    reload
    on Sep 18, 2018
  3. mengdonglin
    assigned
    lyakh
    on Sep 18, 2018

  4. lyakh commented on Sep 18, 2018

    @lyakh
    lyakh
    on Sep 18, 2018
    Collaborator

    First of all I think the topic of this bug is not quite correct: this isn't a kernel panic, this is a kernel WARNING. Still, it has to be fixed of course.

    I've tried to reproduce the bug but failed so far. The difference is, that my Up^2 has no codec. I tried both topology variants - with a codec and a "nocodec" one. In both cases kernel was reporting some errors, so, apparently the drivers didn't complete the registration, which might then explain, why a different path was taken when unloading and reloading modules, so no warnings were generated. I also noticed, that not all snd-soc modules were unloaded by the "remove" script. These modules were kept:

    snd_soc_sst_bxt_pcm512x    16384  0
snd_soc_pcm512x_i2c    16384  0
snd_soc_pcm512x        32768  1 snd_soc_pcm512x_i2c
snd_seq_midi           16384  0
snd_seq_midi_event     16384  1 snd_seq_midi
snd_rawmidi            36864  1 snd_seq_midi
snd_sof                98304  0
snd_sof_xtensa_dsp     20480  0
snd_soc_core          258048  3 snd_soc_pcm512x,snd_soc_sst_bxt_pcm512x,snd_sof

    I modified the script to also unload those, and still got no warning. @ZhendanYang could you maybe try to reproduce the problem with the codec board unplugged? To check, whether I've done something wrong or indeed it only occurs with a codec?

  5. ZhendanYang commented on Sep 19, 2018

    @ZhendanYang
    ZhendanYang
    on Sep 19, 2018
    Author

    @lyakh
    I modified the "remove" script to unload those snd-soc modules.
    I add some lines:
    remove_module snd_soc_hdac_hdmi
    remove_module snd_soc_hdac_hda
    remove_module snd_soc_core
    remove_module snd_seq_midi
    remove_module snd_seq_midi_event
    remove_module snd_rawmidi
    And when I run sof_remove.sh, system crashed. I had to reboot through the power button.

  6. lyakh commented on Sep 19, 2018

    @lyakh
    lyakh
    on Sep 19, 2018
    Collaborator

    @ZhendanYang do you have a kernel log from that crash? Logs, attached to this issue "only" contain a kernel warning. Have you tried with no codec?

  7. lyakh commented on Sep 19, 2018

    @lyakh
    lyakh
    on Sep 19, 2018
    Collaborator

    I've got a "nocodec" Up^2 configuration that seems to work - at least aplay and arecord return after some time and I'm seeing interrupts. Although the time is larger than it should be. And I've got an Oops! Investigating...

  8. lyakh commented on Sep 20, 2018

    @lyakh
    lyakh
    on Sep 20, 2018
    Collaborator

    I think I've found the reason for this Oops. It seems to be a case of a use after free. First the DAI driver gets freed along the lines of

    sof_pcm_free()
snd_sof_free_topology()
snd_soc_tplg_component_remove()
remove_dai()

    And then as resources are freed the driver's .remove method is verified, initially it was NULL, but since the memory was released, it seems to be often reused in our use-cases and the .remove pointer then contains random data, if that memory is again accessible, of course. That happens as

    devm_card_release()
snd_soc_unregister_card()
soc_cleanup_card_resources()
soc_remove_dai_links()
soc_remove_link_dais()
soc_remove_dai()
dai->driver->remove()

    I'm looking at ways to fix this.

  9. lyakh
    added a commit that references this issue on Sep 20, 2018

    soc: fix a use after free case

    270273e
  10. lgirdwood
    added a commit that references this issue on Sep 20, 2018

    soc: fix a use after free case

    1b07494
  11. mengdonglin
    added
    fixed
    on Sep 26, 2018

  12. 62 remaining items

  13. plbossart
    added 2 commits that reference this issue on Mar 5, 2019

    crypto: sha256/arm - fix crash bug in Thumb2 build

    69216a5

    crypto: sha512/arm - fix crash bug in Thumb2 build

    c643165
  14. bardliao
    added 2 commits that reference this issue on Apr 26, 2019

    crypto: sha256/arm - fix crash bug in Thumb2 build

    789b981

    crypto: sha512/arm - fix crash bug in Thumb2 build

    cec6b7b
  15. paulstelian97
    added a commit that references this issue on May 4, 2020

    Merge pull request thesofproject#144 from valx90/vghita/teaching/labs…

    6c013fa
  16. aiChaoSONG
    added a commit that references this issue on May 24, 2021

    rust/kernel: Move PointerWrapper trait to types.rs

    ab9da39
  17. plbossart
    added a commit that references this issue on Jun 24, 2024

    bcachefs: Fix snapshot_create_lock lock ordering

    e0cb572
  18. bardliao
    added a commit that references this issue on Dec 20, 2024

    usb: dwc3: imx8mp: fix software node kernel dump

    a4faee0
  19. oder-chiou
    added 4 commits that reference this issue on May 5, 2026

    ARM: 9170/1: fix panic when kasan and kprobe are enabled

    8b59b0a

    wifi: rtlwifi: Fix global-out-of-bounds bug in _rtl8812ae_phy_set_txp…

    117dbed

    Bluetooth: btinel: Check ACPI handle for NULL before accessing

    902160c

    mm, slub: avoid zeroing kmalloc redzone

    59090e4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

CNLApplies to CannonlakeP1Blocker bugs or important featuresbugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions