Dramatic slowdown with random.random on free-threading build.

This slowdown was found with one of my favorite benchmarks, which is calculating the pi value with the Monte Carlo method.

import os
import random
import time
from threading import Thread

def monte_carlo_pi_part(n: int, idx: int, results: list[int]) -> None:
    count = 0
    for i in range(n):
        x = random.random()
        y = random.random()

        if x*x + y*y <= 1:
            count += 1
    results[idx] = count


n = 10000
threads = []
num_threads = 100
results = [0] * num_threads
a = time.time()
for i in range(num_threads):
    t = Thread(target=monte_carlo_pi_part, args=(n, i, results))
    t.start()
    threads.append(t)

while threads:
    t = threads.pop()
    t.join()

b = time.time()
print(sum(results) / (n * num_threads) * 4)
print(b-a)

Acquiring critical sections for random methods causes this slowdown.
Removing @critical_section from the method, which uses genrand_uint32 and then updating genrand_uint32 to use atomic operation makes the performance acceptable.

Build	Elapsed	PI
Default (with specialization)	0.16528010368347168	3.144508
Free-threading (with no specialization)	0.548654317855835	3.1421
Free-threading with my patch (with no specialization)	0.2606849670410156	3.141108

Linked PRs

@colesbury

cc @colesbury

pythongh-118392: Update genrand_uint32 to use atomic operation

It'd be great to have the default random number generator be safe and efficient when called from multiple threads, but I think that's a bigger change that requires more discussion and maybe a PEP.

In the meantime, I think the advice should be to use individual Random objects per-thread. In other words, the program should be written as:

def monte_carlo_pi_part(n: int, idx: int, results: list[int]) -> None:
    r = random.Random()
    count = 0
    for i in range(n):
        x = r.random()
        y = r.random()

        if x*x + y*y <= 1:
            count += 1
    results[idx] = count

I think that you also would need #118112 (or deferred reference counting) for the program to be efficient, or otherwise the reference count contention is going to be a bottleneck.

In the meantime, I think the advice should be to use individual Random objects per-thread. In other words, the program should be written as:

Yeah, it works. so it should be included somewhere in the migration guide.

#118393 (comment)

pythongh-118392: Add warning about random.random for multi thread app

We could also consider adopting a strategy like Go does, at least for the free-threaded build:

If seed() has not been called, use a fast per-thread RNG for the global calls
Once seed() is called, switch to the shared state RNG to preserve old behavior.

gh-118392: Add note about random.random for multi thread app (gh-118396)

pythongh-118392: Add note about random.random for multi thread app (p…

Navigation Menu

Repository navigation

Dramatic slowdown with random.random on free-threading build. #118392

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Dramatic slowdown with random.random on free-threading build. #118392

Description

Linked PRs

Activity

corona10 commented on Apr 29, 2024

colesbury commented on Apr 29, 2024

corona10 commented on Apr 29, 2024

corona10 commented on Apr 29, 2024

colesbury commented on Apr 29, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions