Bug report

Bug description:

On CPython 3.12.0rc3 I am reliably seeing a new apparent race condition in the build system.

On prior CPython releases, make -jN install would reliably work. On 3.12, it no longer does.

Here is example build output:

To find the necessary bits, look in configure.ac and config.log.

Checked 111 modules (31 built-in, 73 shared, 4 n/a on macosx-13.5-x86_64, 0 disabled, 3 missing, 0 failed on import)
Creating directory /tools/host/lib/python3.12
Creating directory /tools/host/lib/python3.12/config-3.12-darwin
Creating directory /tools/host/lib/python3.12
install: mkdir /private/var/folders/dd/xb3jz0tj133_hgnvdttctwxc0000gn/T/tmpsgqb2dup/out/tools/host/lib/python3.12: File exists
Creating directory /tools/host/lib/python3.12/lib-dynload
make: *** [libainstall] Error 71
make: *** Waiting for unfinished jobs....
Creating directory /tools/host/lib/python3.12/asyncio

The Creating directory %s log message comes from CPython's Makefile.pre.in and we can see from these logs that there appear to be 2 callers racing to create lib/python3.12 around the same time.

Steps to reproduce are just a simple ./configure + make -jN install, where N is say 8.

I can reliably reproduce this failure on macOS 13.5.2 and it might be due to a buggy /usr/bin/install -d implementation on macOS. I think the problem is /usr/bin/install -d doesn't swallow/ignore EEXIST like mkdir -p does.

I want to say I encountered limitations/bugs in macOS /usr/bin/install a few years back when I was a maintainer of Firefox's build system.

CPython versions tested on:

3.12

Operating systems tested on:

macOS