Repository navigation
google/oss-fuzz/infra/cifuzz/actions downloads about 200 extra packages from http://archive.ubuntu.com/ubuntu #7629
Description
Question for @cujomalainey and @andyross
The (older) IPC fuzzer check added by commit 39551b0 uses the Github Action google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@master
This action runs in Docker image gcr.io/oss-fuzz-base/base-builder:latest
Then, the action uses apt-get to download ~ 200 extra Ubuntu packages from http://archive.ubuntu.com/ubuntu on top of oss-fuzz-base/base-builder
One of the main purposes of Docker is to prepackage a complete, specialized image for the job. So it's strange to see that an image named "oss-fuzz-base" is short of 200 Ubuntu packages to... perform fuzzing.
This is not just inefficient; it may also hit some rate limits. If I were http://archive.ubuntu.com/ubuntu , I would not accept that much traffic from Github Actions. In fact this run timed out, http://archive.ubuntu.com/ubuntu
stopped responding: https://github.com/thesofproject/sof/actions/runs/4993519332/jobs/8942765323
To avoid rate limits, most Github Actions download images and packages from internal mirrors like ghcr.io and http://azure.archive.ubuntu.com/
Again it would be even better if the gcr.io/oss-fuzz-base/base-builder:latest image was better self-contained and didn't need that many extra packages in the first place.
EDIT: timing out again in https://github.com/thesofproject/sof/actions/runs/5000127070/jobs/8957163851?pr=7546 / PR #7546
Activity
andyross commentedon May 16, 2023
This code is still in use in the oss-fuzz integration, so needs to stay there until it's replaced with the new version. I should have that submitted today. Once that merges in upstream, this becomes dead code and can be removed.
cujomalainey commentedon May 16, 2023
@marc-hb this sounds related google/oss-fuzz#7130
andyross commentedon May 16, 2023
New integration submitted at google/oss-fuzz#10342
It's somewhat lighter-weight, relying on the built-in clang and tooling more. Though it now has to pull the Zephyr SDK for the build image (but not every time it builds the fuzzers!), which is 1G+ and constantly growing. (Actually there is a variant with only the host tools somewhere, which is all we need for x86 fuzzing, which would be a big win.)
andyross commentedon May 16, 2023
Oh, but to be clear: none of the docker stuff is required for the SOF CI anymore (again, once that merges), which works with the host clang and builds directly out of the tree. oss-fuzz is docker based for some good reasons, but there's no point to us paying that kind of overhead when it's not needed.
lgirdwood commentedon Jul 4, 2023
@andyross @marc-hb can we close now ?
marc-hb commentedon Jul 5, 2023
The old fuzzing framework is gone (except in sof-docs) and the new one starts up quickly, example: https://github.com/thesofproject/sof/actions/runs/5454148796/jobs/9923882503?pr=7891
Closing, thanks @lgirdwood for the reminder.