Repository navigation
[BUG] scripts/build-tools.sh race can corrupt development topologies #5067
Description
SOF main branch, commit 7c6186a
Run these:
NO_PROCESSORS=1 ./scripts/build-tools.sh
NO_PROCESSORS=2 ./scripts/build-tools.sh
NO_PROCESSORS=400 ./scripts/build-tools.sh
Save and compare the output sof/tools/build_tools directories each time and compare them. They don't have the same content. In one case some development topologies files are likely to be corrupted.
I learned this the really hard way while testing a totally unrelated CMake fix.
Finer-grained reproduction:
./scripts/build-tools.sh -C # really clean
cd sof/build_tools
make dev_topologies1 # builds dev topologies in topology/topology1/develoment/
make topologies1 # "installs" topology/topology1/* files one level up, INCLUDING development topologies
=> dev topologies are part of the "installed" topologies.
Compare with:
./scripts/build-tools.sh -C # really clean
cd sof/build_tools
# "installs" topology/topology1/* files one level up while MISSING development topologies
# that are not built yet
make topologies1
# builds dev topologies in topology/topology1/develoment/
make dev_topologies1
=> dev topologies are NOT part of the "installed" topologies.
Build systems are highly parallel, so a sequencing problem like this is a red flag: a race condition red flag.
The simple problem is topologies and (optional) dev_topologies1 run concurrently. This would be great if the installer of the former didn't accidentally grab whatever the latter is still building. That makes the number of development topologies "installed" VARIABLE.
Much WORSE and how I actually found this in the first place, it can happen that some topology files are TRUNCATED because they're installed before they're finished.
Activity
lgirdwood commentedon Dec 10, 2021
@marc-hb can you add a cmake dependency between them or fix the script to work around this until a proper solution is found ?
marc-hb commentedon Dec 10, 2021
I can think of a number of "quick fixes" but I could first use some idea of what purpose development topologies serve. Maybe a quick fix is not to copy / "install" them one level up if no one cares about them.
plbossart commentedon Dec 10, 2021
'Development' topologies are typically added for a specific purpose such as
a) testing an interface that requires custom hardware not widely available
b) adding paths for testing (e.g. nocodec-ci), which extends a baseline
c) added modules that are not or not yet part of the baseline.
The intent is that they are developer-maintained, but not suitable for release or use by non-developers. there are no plans for any backwards compatibility with e.g. UCM files.
As such these topologies do not belong in sof-bin. Whoever needs those topologies should use the git tree and know what they are doing.
Does this help?
topology1: build the production topologies one level down
marc-hb commentedon Dec 10, 2021
It does thank you!
This took a while (always with CMake...) but I think I have a pretty good fix: #5071
topology1: build the production topologies one level down
lgirdwood commentedon Dec 13, 2021
@marc-hb can we close ?
[-][BUG] scripts/build-tools.sh can corrupt development topologies[/-][+][BUG] scripts/build-tools.sh race can corrupt development topologies[/+]