Skip to content

Start decentralizing zephyr/CMakeLists.txt - #8422

Merged
lgirdwood merged 2 commits into
thesofproject:mainfrom
marc-hb:decentralize1
Nov 8, 2023
Merged

lgirdwood merged 2 commits into
thesofproject:mainfrom
marc-hb:decentralize1

Conversation

@marc-hb

@marc-hb marc-hb commented Oct 31, 2023

Copy link
Copy Markdown
Collaborator

@lgirdwood lgirdwood left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice start !

@andyross andyross left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very much needed. The existing Zephyr kconfig CMakeLists.txt is indeed a giant merge collision factory as it stands. Some drive-by notes.

Comment thread src/init/CMakeLists.txt Outdated
@@ -1,5 +1,17 @@
# SPDX-License-Identifier: BSD-3-Clause

if(CONFIG_SOF) ### Zephyr ###

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be worth creating a little bit of new API here. Now, I personally know perfectly well what this means ("CONFIG_SOF" is defined in $ZEPHYR_BASE/modules/Kconfig.sof and declares that the "SOF" module is active, therefore we must be a Zephyr build).

But anyone looking at this for the first time is going to balk: "Wait, what? SOF means ZEPHYR? I thought it was all SOF?"

And if they were a little more sophisticated and knew that we had Kconfigs used in both modes, they'd probably assume that "CONFIG_SOF" means not Zephyr.

Maybe set a variable somewhere so you can write if(__ZEPHYR__) instead? (By analogy to the same symbol present in the C preprocessor).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, I didn't like that too. I added a comment but...

Maybe set a variable somewhere so you can write if(ZEPHYR) instead?

The problem is "somewhere" and as usual racing to be first before users. Enough bootstrap issues in the real code already, the fewer in CMake the better :-) CONFIG_XXX is great because CMake code uses it almost everywhere so I'm not the only one "racing it".

I can do two things:

  1. Switch to the much longer CONFIG_ZEPHYR_SOF_MODULE that I just discovered. It seems generated by the build system. Not sure how but it's already being used elsewhere for the very same purpose.

  2. Add an indirection layer on top of CONFIG_ZEPHYR_SOF_MODULE but with a well-named is_zephyr() CMake function instead. So the build fails if it's not defined somewhere unlike an ambiguous and dangerous ifdef. Also, the indirection layer will provide room for verbose comments if needed and a single place to patch if/when things change. Now the question is still: where do I define this function?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you do it in the upstream zephyr module, you'll be before any of the cmake code in SOF proper for sure. But yeah, if there's a clearer kconfig that's probably even better (I'm not much of an expert on the module layer, but if it's exposed in kconfig it's surely valid to rely on).

Comment thread src/init/CMakeLists.txt
if(CONFIG_SOF) ### Zephyr ###


zephyr_library_sources(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a spot where Zephyr's promiscuous cmake layer hurts pretty badly, because you can't add files to the project in the same way in both modes. Again, maybe work around this by adding yet another layer of indirection and have a "sof_sources()".

Also, FWIW: the "_library_sources" variant really doesn't do anything useful beyond make the build slower by taking the time to create a static library. There is (AFAIK) exactly one spot in Zephyr where we have to do this for obscure linkage reasons[1], but for general C code it's almost always faster and cleaner to link all the object files at once.

I've tried to make that point upstream and basically failed: "zephyr_library_sources()" is a pervasive paradigm. But really it's needless except in specific situations.

(For reference: the reason to use it is that a static library only presents internal files to the link if one of their symbols is accessed, so you can build it and include static stuff like e.g. SYS_INIT() or interrupt declarations, and avoid having those end up in the final output unless something is using the file. But DTS and kconfig are the right mechanisms for that kind of dependency tracking, doing it with the linker is dangerous and surprising.)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, maybe work around this by adding yet another layer of indirection and have a "sof_sources()".

I thought about this and decided against it. There are already layers of indirection-obfuscation in both XTOS and Zephyr, don't want to stack yet another one on top. A dead simple set(common_files ...) works great and avoids enough duplication.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried to make that point upstream and basically failed: "zephyr_library_sources()" is a pervasive paradigm. But really it's needless except in specific situations.

Would you care to change this in sof/zephyr/CMakeLists.txt right now? Don't mind conflicting with this, I'll happily rebase[*] It's a good time now because:

  • we just started the SOF release cycle (and the Zephyr one for that matter)
  • They are all still in one file for now! Quick :-)
  • There is no rush to "de-centralize" yet.

[*] As much as one can be happy resolving git conflicts :-D

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heh, I've spent my bikeshed quota for the month already.

The way this goes is that someone says "libraries make linking faster!" and the response is "only after the ranlib stage builds the table, and in zephyr most symbols are actually needed by the final link", then someone else demands benchmarks, etc...

Really I was mostly just whining about a personal peeve. I've made my peace with it: per-directory/per-app fine grained static libraries are the way zephyr links itself.

@marc-hb
marc-hb marked this pull request as draft October 31, 2023 17:57
Comment thread zephyr/CMakeLists.txt Outdated
Adding all source files in a single, giant zephyr/CMakeLists.txt is
inconvenient and does not scale.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Adding all source files in a single, giant zephyr/CMakeLists.txt is
inconvenient and does not scale.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
@marc-hb
marc-hb marked this pull request as ready for review November 1, 2023 22:26

@btian1 btian1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. can you add details for how decentralize zephyr/CMakeList.txt in PR comments?
  2. anyway, two patches share exact same comments is not good.
  3. is this all for decentralize? or just a beginning?

@marc-hb

marc-hb commented Nov 2, 2023

Copy link
Copy Markdown
Collaborator Author

can you add details for how decentralize zephyr/CMakeList.txt in PR comments?

I don't understand the question, can you be more specific? This PR is just moving CMake code from one giant file to multiple CMake files in multiple subdirectories. What about this move don't you understand?

anyway, two patches share exact same comments is not good.

The two commits do the same thing in two different directories. So the only difference between the two commit messages is the directory name.

is this all for decentralize? or just a beginning?

Did you look at the title of the PR?

@lgirdwood

Copy link
Copy Markdown
Member

@andyross good for you as a starting point to merge, we can factor in more review comments incrementally ?

@andyross andyross left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, realized I never had a +1 on this

@kv2019i kv2019i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @marc-hb . This is a great start.

Comment thread src/ipc/CMakeLists.txt
dma-copy.c
)

if (CONFIG_SOC_SERIES_INTEL_CAVS_V25 OR CONFIG_SOC_SERIES_INTEL_ACE)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks strange, but it matches the existing logic, so great to have this become more visible in the cmake code. The long per-product sections in one-giant-zephyr-cmakefile easily hide these things (when we have per-platform compilation of stuff that really shouldn't be per-platform).

@marc-hb

marc-hb commented Nov 6, 2023 •

Copy link
Copy Markdown
Collaborator Author

@marc-hb
marc-hb requested a review from pjdobrowolski November 6, 2023 18:41
@lgirdwood
lgirdwood merged commit 1bd9e0d into thesofproject:main Nov 8, 2023
@marc-hb
marc-hb deleted the decentralize1 branch November 10, 2023 21:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants