Repository navigation
Reduce CI duration #167
Description
Since we currently run a complete HTML build process in CI, the CI has an unacceptable duration (~16minutes).
Possible solutions:
- Cache CPython directory with actions/cache.
- Avoid the full building process in CI.
Activity
cschan1828 commentedon Jul 15, 2023
Now even a single CI workflow takes around 27 minutes.
In my opinion, we can consider checking
.rstfilesusingwhen running CI. On the other hand, CD should be kept as unchanged as it requires full html build.sphinx-build -b gettextcschan1828 commentedon Jul 15, 2023
After investigating the
Makefile, I found it is more practical to change the build target to "text". It is not elegant because it occupies extra space for text files. However, it effectively reduces the validation time. In the future we can further optimizations.ezio-melotti commentedon Jul 15, 2023
Is this issue specific to the zh-tw translations? Have you checked if other Python translation projects have the same issue, and if/how they fixed it?
mattwang44 commentedon Jul 15, 2023
CPython repo has a Makefile for doc build, and all translation repos that run doc build with GitHub Action depend on it. It provides different build modes and we currently setup our CI action to run a full doc build with the
autobuild-dev-htmlmode, which is the sphinx-nativehtmlmode with-E(always rebuild completely) and-a(write all output files) options. We've tried outtext&htmlmode locally and they do much faster thanautobuild-dev-html, and that's why we believe the-Eaoption increase the duration and changing the build mode would probably be a decent solution for us.Here are some metrics from those translation repos that have adopted GitHub Action. French repo also builds with
autobuild-dev-htmlbut I have no idea why their actions are still blazing fast like the regularhtmlmode...cschan1828 commentedon Jul 16, 2023
I think we can have a simple but effective solution first, then keep working on optimizations.
mattwang44 commentedon Jul 22, 2024
@ken71301
As we mentioned at the sprint, please refer to the discussion above.
My untested thoughts:
make all), and utilize the cache action to cache the build artifacts.for filename in <find-updated-files>; do make build $$filename; done) -> expected to be much faster than full buildken71301 commentedon Aug 22, 2024
@mattwang44
It seems like FR also use html mode now.
python/python-docs-fr#426
ken71301 commentedon Aug 22, 2024
In latest sprint Josix and me just tried load of setting and it all seems not effective at all.
So I start believing that cache might be our next option, though I still need to check other's test flow to make sure how we can do it.
mattwang44 commentedon Oct 11, 2024
With this patch (issue) released along with Sphinx 8.1, the build duration in CI has significantly dropped from ~25 mins to ~4 mins 🚀. I believe doc build time would then not be an issue for us. I'll close this issue if no objections in the following days.