Skip to content

TSan races and a lost 32-bit CFFI string view on ubuntu-26.04 runners #430

Open

Description

@dimensionscape

GitHub moved ubuntu-latest to ubuntu-26.04, and two Linux jobs have failed on main on every run since (73b4e23, 3089d85, fb7f804). On ubuntu-24.04 the same commits pass, in CI and locally. Both jobs are pinned to ubuntu-24.04 for now, so this tracks making them pass on 26.04.

Check with TSan

https://github.com/openfl/libhxcpp/actions/runs/41885587875/job/126769049078 (image ubuntu-26.04, 20261021.618)

  • 1509 reports in three processes: the default-collector gc stress run (541, bin/Stress 10, 4 threads, seed 673031576), the incremental gc stress run (942) and LargeAllocThreads (26).
  • Nearly all of them share one pattern: a worker reads a BlockDataInfo in LocalAllocator::DoAlloc (mostly a 4-byte field, at offset 0x110), and the other side of the race is the main thread allocating that BlockDataInfo with operator new in GlobalAllocator::AllocMoreBlocks. TSan sees nothing ordering the read after the allocation. The worker gets the block through GetFreeBlock, whose reads are suppressed as hints, so the pointer may really be published without a release.
  • The TSan runtime is a different build from 24.04's (libtsan.so.2 BuildId 9e0607059cf1... against 2a13a7710e36...). The 26.04 image also has a newer GCC.

On ubuntu-24.04 the TSan job was green at 346c7e5 (https://github.com/openfl/libhxcpp/actions/runs/41375157695/job/125053682292). In WSL with Ubuntu 24.04 (GCC 13.3, the 2a13... libtsan), the default and incremental stress runs report nothing at fb7f804, including with CI's thread count and seed.

Linux 32: native.TestCffiStringView.testViewsSurviveCollection

https://github.com/openfl/libhxcpp/actions/runs/41885587875/job/126779901591 and /job/126779901360 (ubuntu-26.04, 20261015.610)

native.TestCffiStringView
  testViewsSurviveCollection: FAILURE F
    line: 97, view 7 has 0 characters, null first: ""

The test keeps the UTF-8 views of 64 boxed UTF-16 strings (#322), runs three full collections and a compaction, and reads them again. One comes back empty, so its buffer was freed or moved without the box being updated. It is the only failure in those jobs, and it fails in every 32-bit build they make: generational, default collector, moving, a larger block pool and C++20. On ubuntu-24.04 it passed at 346c7e5 (job 125055393380), and in WSL with Ubuntu 24.04 the 32-bit suite passes it at 346c7e5, 166b57b, ac1a5fa and 73b4e23.

A related lead: a standalone copy of the test (the same loop, run 20 times, with one extra view taken before it) loses a view in every round when built 32-bit in that same WSL setup, at every commit from merge 3b6ddf3 on, though both of that merge's parents pass. Small changes to the program make it pass, so it depends on allocation layout. It may or may not be the same bug.

Next

Reproduce both on Ubuntu 26.04 (its GCC and libtsan), fix them, and move the two jobs back to ubuntu-latest.

Activity

  1. dimensionscape commented on Nov 6, 2026

    @dimensionscape
    MemberAuthor

    The TSan job that was green at 346c7e5 on ubuntu-24.04 (https://github.com/openfl/libhxcpp/actions/runs/41375157695/job/125053682292) fails when rerun today on ubuntu-26.04 (image 20261101.186.1): https://github.com/openfl/libhxcpp/actions/runs/41375157695/job/127893821336

    It has 1094 reports, in four processes (1, 39, 544 and 510), with the same pattern as above: LocalAllocator::DoAlloc reading a BlockDataInfo that the other thread allocated with operator new, under libtsan.so.2 BuildId 9e0607059cf1.... Same commit, different image, so the change is in the runner image, not in a commit since 346c7e5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions