Repository navigation
Make queue.SimpleQueue thread-safe in --disable-gil builds #113884
Description
Feature or enhancement
Proposal:
Instances of queue.SimpleQueue have mutable state that is not safe to mutate without the GIL. The previous implementation (in 3.12) was colesbury/nogil-3.12@7e60a01aee.
@colesbury - You mentioned in chat that porting the 3.12 approach wouldn't be straightforward. Can you say more?
Has this already been discussed elsewhere?
This is a minor feature, which does not need previous discussion elsewhere
Links to previous discussion of this feature:
No response
Activity
colesbury commentedon Jan 9, 2024
@mpage, after looking it over again, I think the nogil-3.12 approach is mostly the right direction with a few modifications:
_PyMutex_lock()/unlock()calls with critical sections and remove the_PyMutex mutexfield. Since, the code already uses Argument Clinic (AC), this is mostly just a matter of adding the@critical_sectionto the AC input. This will avoid the extra lock operations in the default build, which can rely on the GIL.This means we will go from a SimpleQueue implementation built around
listto run that more explicitly manages a buffer. I think that's worthwhile here; otherwise I think handling the synchronization across both thelistoperations and the containingSimpleQueuewill be too messy.gh-113884: Refactor `queue.SimpleQueue` to use a ring buffer to store…
pythongh-113884: Refactor `queue.SimpleQueue` to use a ring buffer to…
gh-113884: Make queue.SimpleQueue thread-safe when the GIL is disabled (
erlend-aasland commentedon Jan 23, 2024
Thank you, @mpage and @colesbury 🙏
pythongh-113884: Refactor `queue.SimpleQueue` to use a ring buffer to…
pythongh-113884: Make queue.SimpleQueue thread-safe when the GIL is d…
pythongh-113884: Refactor `queue.SimpleQueue` to use a ring buffer to…
pythongh-113884: Make queue.SimpleQueue thread-safe when the GIL is d…