What are unity builds (and how they help) #40

Activity

1. lefticus

   added

   episode idea

   C++ Weekly

   on Jun 23, 2022
2. lefticus

   added

   ready to record

   on May 24, 2023
3. lefticus commented on Jan 8, 2024

   

   lefticus

   on Jan 8, 2024

   Last edited by lefticus

   OwnerAuthor

   More actions

   • the process: Many .cpp files are merged into one file before compilation
   • the theory: this improves compile time (less re-parsing of headers) and runtime (greater cross-module visibility, like LTO)
   • the downside (or is it an upside?): all ODR violations become compile time errors

   The process in cmake is relatively easy:

   One way: turn on CMAKE_UNITY_BUILD globally, but make sure any targets that are broken with Unity builds have it explicitly disabled.

   Gotcha: "warnings as errors" is more important than ever. If one .cpp file defines a macro and a subsequent file redefines the macro, this is normally a warning, not an error, but you have no idea what the impact might be on the compiled code.

   Conclusion: on this project we saw 0 perf differences, but we did get a 2x faster build!

   ---

   Original mold build (Release)

   [658/658] Creating executable symlink Products/energyplus
   8283.41user 283.24system 14:48.95elapsed 963%CPU (avgtext+avgdata 2933712maxresident)k
   1424inputs+1589072outputs (17major+67988849minor)pagefaults 0swaps

   perf

   40.24user 0.17system 0:40.61elapsed 99%CPU (avgtext+avgdata 268032maxresident)k
   0inputs+43120outputs (0major+64250minor)pagefaults 0swaps
   

   69967000 Jan  8 14:27 libenergyplusapi.so.22.1.0
   

   original mold lto build

   [658/658] Creating executable symlink Products/energyplus
   3936.56user 256.16system 11:51.28elapsed 589%CPU (avgtext+avgdata 2426960maxresident)k
   0inputs+3914232outputs (17major+59543380minor)pagefaults 0swaps

   mold unity build

   Note that there are fewer targets because the unity build merged targets together

   [219/219] Creating executable symlink Products/energyplus
   4916.14user 125.28system 9:37.50elapsed 872%CPU (avgtext+avgdata 3074240maxresident)k
   511488inputs+1263136outputs (97744major+23875086minor)pagefaults 0swaps

   perf run for unity mold build

   EnergyPlus Run Time=00hr 00min 40.00sec
   EnergyPlus Completed Successfully.
   39.90user 0.14system 0:40.20elapsed 99%CPU (avgtext+avgdata 270004maxresident)k
   0inputs+43120outputs (0major+64415minor)pagefaults 0swaps

   File size:

   68976032 Jan  8 13:39 libenergyplusapi.so.22.1.0
   

   mold unity lto build

   [219/219] Creating executable symlink Products/energyplus
   1503.43user 77.32system 7:26.81elapsed 353%CPU (avgtext+avgdata 2491416maxresident)k
   143904inputs+24238000outputs (195major+18620542minor)pagefaults 0swaps

   Perf run for unity lto build, with mold

   EnergyPlus Run Time=00hr 00min 40.26sec
   EnergyPlus Completed Successfully.
   40.09user 0.17system 0:40.46elapsed 99%CPU (avgtext+avgdata 265884maxresident)k
   3280inputs+43200outputs (0major+64171minor)pagefaults 0swaps

   62300800 Jan  8 13:58 Products/libenergyplusapi.so.22.1.0
   
4. lefticus commented on Jan 9, 2024

   

   lefticus

   on Jan 9, 2024

   OwnerAuthor

   More actions

   Best Practice:

   Test your project regularly with Unity Builds to know if it has sneaky UB because of ODR violations and decide as a team if you want Unity Builds to be the default on your CI
5. lefticus

   added

   Recorded

   and removed

   ready to record

   on Jan 9, 2024
6. lefticus commented on Jan 13, 2024

   

   lefticus

   on Jan 13, 2024

   OwnerAuthor

   More actions

   Coming in Ep413
7. lefticus

   closed this as completedon Jan 13, 2024
8. github-project-automation

   moved this to Done in "C++ Weekly" Episode Planson Jan 13, 2024
9. Dharmesh946 commented on Sep 27, 2024

   

   Dharmesh946

   on Sep 27, 2024

   Last edited by Dharmesh946

   More actions

   Hi, I've juste seen this episode and played with that. One of the downside I felt upon is the undesired "merging" of unnamed namespace.
   A practical use-case: I've got a bunch of unit-tests separated in different source files. In each source file, I declare a global std::mt19937 variable, within an unnamed namespace, in order to have a single random generator in the translation unit. I gave the same name for the generator in each TU.
   But now the unity build decided to merge all my unit tests within a single TU and I've got now a redefinition issue...
   Of course I can give a different name to my generator in each file but it's a bit of an overhead for the developer...
   Same issue for static global constants and "local" functions...
   I can't recall you mentioned that in the video (may be, in this case I apologize) and I felt it was worth mentioning.