Repository navigation
[FEATURE] Move logger/trace (and debug tools?) to other repository #2480
Description
Actually, the SOF repository is "all in one" now. While we have the sof-bin repository now, the debug tools which are supposed to be used by the end users for the problem analysis like sof-logger should be moved to a separate repository. The code should be compatible with all SOF firmware/driver releases.
Activity
jajanusz commentedon Mar 4, 2020
We had it like that previously but TBH I don't remember what was the issue - probably something with keeping headers shared by tools and FW aligned (ABI changes).
jajanusz commentedon Mar 4, 2020
Anyway splitting is always good and we should do this.
It could be problematic because of how tools and FW code was coupled, but with new build-system and some refactor it shouldn't be that big issue.
FW doesn't have hard dependency on these tools, the tools depend on some headers from FW and in case of f.e. testbench some runtime code - but it can be exported as .so lib that this tool can use.
perexg commentedon Mar 4, 2020
As I wrote, those tools must support all ABI versions for the released (tagged) firmware files.
lgirdwood commentedon Mar 4, 2020
@xiulipan fyi. This will need to be added into your release scripting.
lgirdwood commentedon Aug 18, 2020
@xiulipan any update here ? I think we have all the topologies and LDCs now but not the tools ?
xiulipan commentedon Aug 19, 2020
@lgirdwood Good point here for sof-bin to include the tools., but I am not sure which version tools should I put into the repo. @jajanusz Can we include the tools binary with release also?
lgirdwood commentedon Aug 20, 2020
@xiulipan ideally we would have our makefile
See
https://askubuntu.com/questions/1130558/how-to-build-deb-package-for-ubuntu-18-04
https://packaging.ubuntu.com/html/packaging-new-software.html
and for rpms
https://docs.fedoraproject.org/en-US/quick-docs/creating-rpm-packages/
It looks like for both we need to have a config file that the packaging tools will use to build and pull in the binaries. Btw, this could be done in two containers (one ubuntu and the other fedora) if you have the time (as it means anyone could then build the packages).
perexg commentedon Aug 20, 2020
@lgirdwood : It diverges from my request to put the logger tool code to another repository and maintain the compatibility with all known (previous) SOF releases (one binary should work with all released SOF firmware files). In this case, distributions can maintain this tool in a package themselves. Distributions have already a package for SOF firmware (sof-bin) and I would not mix the tools with firmware.
lgirdwood commentedon Aug 20, 2020
@perexg ok, your right -
@mengdonglin this may be a few days effort for @xiulipan . Do you think we can do for v1,6 ?
xiulipan commentedon Aug 21, 2020
@lgirdwood Here is my understanding for the request here:
lgirdwood commentedon Aug 21, 2020
@xiulipan I think @perexg is asking that the sof tooling (rimage, sof-logger etc - everything that builds a binary) goes in the rimage repo and is then distrubuted indipendently from the FW. This way distro folks can pick up the new repo and build for thier targets.
@perexg how do you recommend dealing with the header dependcy the tools have on the FW repo ? The FW headers are used by most of the tools, so anyone building tools would also need the FW headers (note FW does not need to be built to generate any of the headers).
31 remaining items
[-][FEATURE] Move logger (and debug tools?) to other repository[/-][+][FEATURE] Move logger/trace (and debug tools?) to other repository[/+]perexg commentedon Feb 16, 2023
The name of the tool is not relevant. The point is that there should be a split.
marc-hb commentedon Feb 16, 2023
The discussion was about who's maintaining what and not. Testing and maintaining more combinations is significant work. A separate (and on-going) validation effort is required for each tool; logging and not.
This by the way can be done independently of the number of source repos. I mean you can release separately and implement backwards-compatibility across multiple components all hosted in a unique repo. You can also have components hosted in separate repos that are tested only in very specific version combinations; example:
sof/west.yml. This must be kept in mind in discussions like these. This is a complex topic https://medium.com/@sdboyer/so-you-want-to-write-a-package-manager-4ae9c17d9527lgirdwood commentedon Feb 20, 2023
@perexg I think most of the bulk code changes are mostly in place for Zephyr now, next steps are some repo reconfiguration to remove sub repos and use west as a repo manifest. This would enable tools to be split out of the SOF repo. Fwiw some tools will also come from Zephyr too.
lgirdwood commentedon May 10, 2023
Moving to v2.7 west now integrated, some tools in Zephyr, but other tools, topologies and configs still pending.
marc-hb commentedon Oct 26, 2023
October 2023 update:
- sof-logger is dead for Intel and not maintained by anyone else. It's Zephyr logging / mtrace now.
- After many long discussions, we realized that a separate rimage was orders more maintenance work that we could handle, so rimage was just moved back into sof.git: #8178 . There are some generic, non-rimage specific "talking points" about the number of git repos in 8178
- No one seems to use sof-ctl and sof-probes; we never hear anything about them. In any case their code never moves.
So I think the only question left is topologies. This issue was not filed about topologies and the long discussion above barely ever mentioned topologies so I'm closing this issue as "wontfix". Please open a new issue about topologies if desired (referencing this one).
A separate repo is indeed a strong "compatibility matrix" statement - and a lot of validation work is required to achieve such support. I don't know whether separate topologies would be a realistic validation and support work commitment. For sure it wasn't for the tools above.
(A compability matrix can also be implemented inside a single repo - but that's less intuitive and less assertive)
https://github.com/thesofproject/sof-bin/releases has been implementing incremental releases of topologies just fine