Skip to content

[BUG]Sof-logger ABI version is mismatch with .ldc file built by xcc #2507

Closed

Description

@dengyangchao

Describe the bug
Error: abi version in /etc/sof/sof-apl.ldc file does not coincide with abi version used by logger.
logger ABI Version is 4:2:0
ldc_file ABI Version is 3:14:0

To Reproduce

  1. Reboot via "sudo reboot"
  2. Check with command "sudo sof-logger -l /etc/sof/sof-apl.ldc -t"

Reproduction Rate
100%

Expected behavior
Can see sof trace print normal

Impact
No sof trace because sof-logger ABI mismatch with .ldc file

Environment

  1. Branch name and commit hash of the 2 repositories: sof (firmware/topology) and linux (kernel driver).
  2. Name of the topology file
    • Topology: sof-apl-pcm512x.tplg
  3. Name of the platform(s) on which the bug is observed.
    • Platform: APL-UP2 with codec PCM512X in I2S mode

check-sof-logger.sh.txt
dmesg.txt

Activity

  1. added
    bugSomething isn't working as expected
    APLApplies to Apollolake platform
    BYTApplies to Baytrail platform
    P1Blocker bugs or important features
    on Mar 9, 2020
  2. Liviali155 commented on Mar 9, 2020

    @Liviali155

    on below platforms also has this paoblem
    GLK Chrome with onboard codec DA7219 in I2S mode
    CML laptop with onboard codec ALC3204 in HDA mode
    ICL-U RVP with onboard codec ALC700 in HDA mode
    TGL-U RVP with onboard codec ALC711 in HDA mode
    CFL-S RVP with onboard codec ALC700 in HDA mode

  3. added
    CMLApplies to Comet Lake platform
    TGLApplies to Tiger Lake
    ICLApplies to IceLake platform
    CFLApplies to Coffee Lake platform
    GLKApplies to Gemini Lake platform
    on Mar 9, 2020
  4. dengyangchao commented on Mar 9, 2020

    @dengyangchao
    Author

    Bi-sect result: caused by master 332e150 , @mmaka1 please help to check this

  5. mmaka1 commented on Mar 9, 2020

    @mmaka1

    Rebuild and upgrade the logger when you see a version mismatch.

  6. 6 remaining items

  7. mmaka1 commented on Mar 10, 2020

    @mmaka1

    @xiulipan Are you sure you did a clean build? I am using xcc all the time. This is from my system:

    $ hexdump -C build_fw_apl/sof-apl.ldc |head
    00000000  4c 6f 67 73 00 00 00 20  54 27 01 00 4c 00 00 00  |Logs... T'..L...|
    00000010  3c 00 00 00 01 00 04 00  01 00 00 00 00 00 00 00  |<...............|
    00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    00000030  00 00 30 34 36 38 61 00  00 20 00 04 00 00 00 00  |..0468a.. ......|
    

    You should also see the current dbg abi version when running rimage as Liam mentioned:

    Firmware manifest and signing completed !
    fw abi main version: 3:14:0
    fw abi dbg version: 4:2:0
    
  8. xiulipan commented on Mar 11, 2020

    @xiulipan
    Contributor

    @mmaka1 @lgirdwood I did some dig in and find this is related to maybe compiler. Here is my analysis:

    First to easily dump info from ELF file I add some debug patch to change the ENUM

    +       SOF_IPC_EXT_WINDOW              = 0x1f,
    +       SOF_IPC_EXT_CC_INFO             = 0x2e,
    +       SOF_IPC_EXT_PROBE_INFO          = 0x3d,
    +       SOF_IPC_EXT_USER_ABI_INFO       = 0x4c,
     };

    Then I add some logs into rimage:

    diff --git a/rimage/file_simple.c b/rimage/file_simple.c
    index b68483f2ebab..cb0c4024dcd5 100644
    --- a/rimage/file_simple.c
    +++ b/rimage/file_simple.c
    @@ -119,15 +119,21 @@ static int fw_version_copy(struct snd_sof_logs_header *header,
             */
            ext_hdr = buffer + ((struct sof_ipc_fw_ready *)buffer)->hdr.size;
            while ((uintptr_t)ext_hdr < (uintptr_t)buffer + section->size) {
    +               fprintf(stdout, "PXL: fw abi dbg type: %x, %x\n", ext_hdr->type, ext_hdr->hdr.size);
                    if (ext_hdr->type == SOF_IPC_EXT_USER_ABI_INFO) {
    +                       fprintf(stdout, "PXL: fw abi dbg version: %x\n",
    +                               ((struct sof_ipc_user_abi_version *)
    +                                                       ext_hdr)->abi_dbg_version);
                            header->version.abi_version =

    And here is what I got

    Firmware manifest and signing completed !
    fw abi main version: 3:14:0
    PXL: fw abi dbg type: 1f, b8
    PXL: fw abi dbg type: 2e, 50
    PXL: fw abi dbg type: 0, 70000000
    fw abi dbg version: 3:14:0
    logs dictionary: size 75680
    

    Dump from elf file
    Note the size for the SOF_IPC_EXT_CC_INFO cc_versionis is 0x50, but actually the size is 0x4c. Not sure if this issue is that my CC_DESC here is RG-2017.8-linux and this is 16-char length (a space at the beginning)

    00043c60  00 **50** 00 00 00 00 00 00  70 2e 00 00 00 0c 00 00  |.P......p.......|
    00043c70  00 00 00 00 00 08 00 00  00 00 00 00 00 00 00 00  |................|
    00043c80  00 00 00 00 00 00 00 00  00 58 43 43 00 00 00 00  |.........XCC....|
    00043c90  00 00 00 00 00 00 00 00  00 4f 32 00 00 20 52 47  |.........O2.. RG|
    00043ca0  2d 32 30 31 37 2e 38 2d  6c 69 6e 75 78 1c 00 00  |-2017.8-linux...| // here 1c is the start of the SOF_IPC_EXT_PROBE_INFO          
    00043cb0  00 00 00 00 70 3d 00 00  00 00 00 00 00 00 00 00  |....p=..........|
    00043cc0  00 00 00 00 00 00 00 00  00 10 00 00 00 00 00 00  |................|
    00043cd0  70 4c 00 00 00 00 20 00  04 00 00 00 00 00 00 00  |pL.... .........|
    
  9. xiulipan commented on Mar 11, 2020

    @xiulipan
    Contributor

    @macchian @lgirdwood @ktrzcinx I have a fix here to change flex array into static one to avoid issues between different compiler. #2522
    I can also think solution like, move the flex array to the end of the fw_ready_metadata section.

    diff --git a/src/ipc/CMakeLists.txt b/src/ipc/CMakeLists.txt
    index 7423a2eac72b..cf579da81854 100644
    --- a/src/ipc/CMakeLists.txt
    +++ b/src/ipc/CMakeLists.txt
    @@ -36,9 +36,9 @@ set_property(TARGET data_structs APPEND
                 CC_OPTIMIZE_FLAGS="${optimization_flag}")
    
     add_local_sources(data_structs
    -       cc_version.c
            probe_support.c
    -       user_abi_version.c)
    +       user_abi_version.c
    +       cc_version.c)
    
     target_link_libraries(data_structs sof_options)
     target_link_libraries(sof_static_libraries INTERFACE data_structs)
    

    rimage logs will becomes to

    fw abi main version: 3:14:0
    PXL: fw abi dbg type: 1f, b8
    PXL: fw abi dbg type: 3d, 1c
    PXL: fw abi dbg type: 4c, 10
    PXL: fw abi dbg version: 4002000
    fw abi dbg version: 4:2:0
    
  10. ktrzcinx commented on Mar 11, 2020

    @ktrzcinx
    Member

    As I see, there is a some limitation in XCC because RG-2017.8-linux string should be terminated with '\0' but I can't see it. Moreover this terminating zero is included in sizeof(CC_DESC), so there is also a solution like this:

    const struct sof_ipc_cc_version cc_version
    	__section(".fw_ready_metadata") = {
    	.ext_hdr = {
    		.hdr.cmd = SOF_IPC_FW_READY,
    		.hdr.size = ALIGN_UP(sizeof(struct sof_ipc_cc_version)
    				     + sizeof(CC_DESC) - 1, 4),
    

    After this change:

    fw abi main version: 3:14:0
    PXL: fw abi dbg type: 1, b8
    PXL: fw abi dbg type: 2, 4c
    PXL: fw abi dbg type: 3, 1c
    PXL: fw abi dbg type: 4, 10
    PXL: fw abi dbg version: 4002000
    fw abi dbg version: 4:2:0
    

    Another possible issue in future is related with lack of memory alignment, SOF_IPC_EXT_PROBE_INFO starts just after SOF_IPC_EXT_CC_INFO, but there should be padding to '4

  11. xiulipan commented on Mar 11, 2020

    @xiulipan
    Contributor

    @kakulesza Yep, I think this is because that the 0 at the end maybe merged by the all four sections in __section(".fw_ready_metadata") is merged in ELF.
    Your solution will result a overflow as the CC_DESC STRING has no termination. It will result in issues in the later use.

  12. mmaka1 commented on Mar 11, 2020

    @mmaka1

    @xiulipan

    Not sure if this issue is that my CC_DESC here is RG-2017.8-linux

    It seems to work on my system since the beginning because I am compiling with the Windows version of the tools and this section looks like (note 4c and two extra zeroes because of shorter description):

     be0561b4 00000000 4c000000 00000070 02000000  ....L......p....
     be0561c4 0c000000 00000000 08000000 00000000  ................
     be0561d4 00000000 00000000 00000000 58434300  ............XCC.
     be0561e4 00000000 00000000 00000000 4f320000  ............O2..
     be0561f4 2052472d 32303137 2e382d77 696e0000   RG-2017.8-win..
    
  13. xiulipan commented on Mar 12, 2020

    @xiulipan
    Contributor

    @mmaka1 Oh, that explain why we have different result. The issue seems to be more related to the cc_version.c .
    @jajanusz @tlauda @lgirdwood Do you know any issue like this here?
    It seems the flex length structure in ELF may not have the right layout as we expected. So I think we should avoid flex length structure in ELF file #2522

  14. jajanusz commented on Mar 13, 2020

    @jajanusz
    Contributor

    @jajanusz @tlauda @lgirdwood Do you know any issue like this here?

    Do you mean on linux version of compiler? For me it's weird that same version of compiler gives different output just cos of os, but anyway, it's better to w/a issue with having fixed sizes on problematic fields.

  15. xiulipan commented on Mar 16, 2020

    @xiulipan
    Contributor

    @jajanusz After debug, it seems to be related to the flex length structure when assemble to section in ELF. The appending length will disappear.

  16. xiulipan commented on Mar 16, 2020

    @xiulipan
    Contributor

    Update with compare test between GCC and XCC, it seems to be some issue related to compiler linker.

    diff --git a/src/arch/xtensa/include/arch/compiler_info.h b/src/arch/xtensa/include/arch/compiler_info.h
    index d59da46a7a1f..fc4a3a8b6ce0 100644
    --- a/src/arch/xtensa/include/arch/compiler_info.h
    +++ b/src/arch/xtensa/include/arch/compiler_info.h
    @@ -23,6 +23,6 @@
     #define CC_MINOR ((XTHAL_RELEASE_MAJOR % 1000) / 10)
     #define CC_MICRO XTHAL_RELEASE_MINOR
     #define CC_NAME "XCC"
    -#define CC_DESC " " XCC_TOOLS_VERSION
    +#define CC_DESC "0123456789ABCDEF"
    
     #endif /* __ARCH_COMPILER_INFO_H__ */
    

    XCC got header size 0x50, actual binary size 0x4c

    00043c60  00 50 00 00 00 00 00 00  70 02 00 00 00 0c 00 00  |.P......p.......|
    00043c70  00 00 00 00 00 08 00 00  00 00 00 00 00 00 00 00  |................|
    00043c80  00 00 00 00 00 00 00 00  00 58 43 43 00 00 00 00  |.........XCC....|
    00043c90  00 00 00 00 00 00 00 00  00 4f 32 00 00 30 31 32  |.........O2..012|
    00043ca0  33 34 35 36 37 38 39 41  42 43 44 45 46 1c 00 00  |3456789ABCDEF...|
    

    GCC got both size is 0x50

    0002a5e0  00 00 00 00 50 00 00 00  00 00 00 70 02 00 00 00  |....P......p....|
    0002a5f0  0c 00 00 00 00 00 00 00  08 00 00 00 00 00 00 00  |................|
    0002a600  00 00 00 00 00 00 00 00  00 00 00 00 58 43 43 00  |............XCC.|
    0002a610  00 00 00 00 00 00 00 00  00 00 00 00 4f 32 00 00  |............O2..|
    0002a620  30 31 32 33 34 35 36 37  38 39 41 42 43 44 45 46  |0123456789ABCDEF|
    0002a630  00 00 00 00 1c 00 00 00  00 00 00 70 03 00 00 00  |...........p....|
    
  17. mengdonglin commented on Mar 31, 2020

    @mengdonglin
    Collaborator

    worked around by #2647

  18. Liviali155 commented on Mar 31, 2020

    @Liviali155

    Error not seen because PR #2647 merged,closed it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    APLApplies to Apollolake platformBYTApplies to Baytrail platformCFLApplies to Coffee Lake platformCMLApplies to Comet Lake platformGLKApplies to Gemini Lake platformICLApplies to IceLake platformP1Blocker bugs or important featuresTGLApplies to Tiger LakebugSomething isn't working as expected

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions