Activity

1. brijkapadia

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Jun 23, 2026
2. brijkapadia

   changed the title [-]'deque_clear' clears `MemoryError`[/-] [+]`deque_clear` clears `MemoryError`[/+] on Jun 23, 2026
3. StanFromIreland

   added

   extension-modulesC modules in the Modules dirC modules in the Modules dir

   on Jun 23, 2026
4. StanFromIreland

   added a parent issue on Jun 23, 2026

   • Umbrella issue: bugs found using cpython-review-toolkit #146102
5. rhettinger commented on Jun 26, 2026

   

   rhettinger

   on Jun 26, 2026

   Contributor

   More actions

   What is wrong with the current code? If an allocation fails due to lack of memory (a very rare almost non-existent event), then an alternate path is taken that does not require additional memory.

   AFAICT this is a false positive from the umbrella issue.
6. rhettinger

   self-assigned this

   on Jun 26, 2026
7. brijkapadia commented on Jun 27, 2026

   

   brijkapadia

   on Jun 27, 2026

   ContributorAuthor

   More actions

   AFAICT this is a false positive from the umbrella issue.

   There is a real error here that the reproducer catches. However, the umbrella issue gives the wrong explanation. The problem has nothing to do with deque.copy but deque.copy is one execution path for the error.

   Reproducer (for reference)

   import _testcapi
   from collections import deque
   
   for n in range(1, 80):
       d = deque(range(200))
       _testcapi.set_nomemory(n, 0)
       try:
           d2 = d.copy()
           _testcapi.remove_mem_hooks()
           break
       except MemoryError:
           _testcapi.remove_mem_hooks()
       except:
           _testcapi.remove_mem_hooks()
   # Fatal: _Py_Dealloc: Deallocator of type 'collections.deque' cleared

   The real problem is that this debug check in _Py_Dealloc fails:

   Debug Check

   cpython/Objects/object.c

   Lines 3321 to 3342 in 1812162

   	#ifdef Py_DEBUG
   	// gh-89373: The tp_dealloc function must leave the current exception
   	// unchanged.
   	if (tstate != NULL && tstate->current_exception != old_exc) {
   	const char *err;
   	if (old_exc == NULL) {
   	err = "Deallocator of type '%s' raised an exception";
   	}
   	else if (tstate->current_exception == NULL) {
   	err = "Deallocator of type '%s' cleared the current exception";
   	}
   	else {
   	// It can happen if dealloc() normalized the current exception.
   	// A deallocator function must not change the current exception,
   	// not even normalize it.
   	err = "Deallocator of type '%s' overrode the current exception";
   	}
   	_Py_FatalErrorFormat(__func__, err, type->tp_name);
   	}
   	Py_XDECREF(old_exc);
   	Py_DECREF(type);
   	#endif

   ---

   For the specific case of the reproducer, deque_copy_impl calls deque_append_lock_held and in the case of an error, calls Py_DECREF(new_deque):
   

   cpython/Modules/_collectionsmodule.c

   Lines 634 to 638 in 1812162

   	if (deque_append_lock_held(new_deque, Py_NewRef(item),
   	new_deque->maxlen) < 0) {
   	Py_DECREF(new_deque);
   	return NULL;
   	}

   Furthermore, deque_append_lock_held calls newblock which raises a PyErr_NoMemory() in the case of OOM:
   

   cpython/Modules/_collectionsmodule.c

   Line 187 in 1812162

   PyErr_NoMemory();

   Thus, in the case of OOM, Py_DECREF(new_deque) is called with an exception already set.

   In order to free the memory, eventually deque_clear is called. deque_clear calls newblock, which in the case of OOM, replaces the current exception with a new PyErr_NoMemory(). Then, deque_clear calls PyErr_Clear(), meaning that there are now no exceptions despite deque_append_lock_held originally setting one:
   

   cpython/Modules/_collectionsmodule.c

   Lines 749 to 753 in 1812162

   	b = newblock(deque);
   	if (b == NULL) {
   	PyErr_Clear();
   	goto alternate_method;
   	}

   In debug builds, this is caught in _Py_Dealloc,
   

   cpython/Objects/object.c

   Lines 3329 to 3331 in 1812162

   	else if (tstate->current_exception == NULL) {
   	err = "Deallocator of type '%s' cleared the current exception";
   	}

   which gives the error Fatal: _Py_Dealloc: Deallocator of type 'collections.deque' cleared the current exception.
8. stestagg commented on Jul 2, 2026

   

   stestagg

   on Jul 2, 2026

   Last edited by stestagg

   Contributor

   More actions

   EDIT: ignore my previous, it was something else.

   I just came across this in fuzzing, the following code reliably aborts for me:

   import collections
   d =33333333333* collections.deque([10])

   gives (in linux with 2gb ram)

   Quitting LLDB will kill one or more processes. Do you really want to proceed: [Y/n] y
   ~ # /pfm/tools/mem_limit_exec /pfm/py/bin/python3 /pfm/scratch/reproducers/t1-1.
   min.py
   Fatal Python error: _Py_Dealloc: Deallocator of type 'collections.deque' cleared the current exception
   Python runtime state: initialized
   
   Current thread 0x0000fffff7fee020 [python3] (most recent call first):
     File "<input-53>", line 2 in <module>
     File "/pfm/scratch/reproducers/t1-1.min.py", line 10 in <module>
   Aborted
   ~ # /pfm/tools/mem_limit_exec /pfm/py/bin/python3 /pfm/scratch/reproducers/t1-1.
   min.py
   Fatal Python error: _Py_Dealloc: Deallocator of type 'collections.deque' cleared the current exception
   Python runtime state: initialized
   
   Current thread 0x0000fffff7fee020 [python3] (most recent call first):
     File "<input-53>", line 2 in <module>
     File "/pfm/scratch/reproducers/t1-1.min.py", line 10 in <module>
   Aborted
   ~ #
   Python 3.16.0a0 (heads/main-dirty:f8514dc, Jul  2 2026, 09:14:40) [Clang 22.1.6 ]