You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
zipfile extractall is not multi-threading safe #112998
I ran into the following traceback when extracting multiple zipfiles into the same directory with concurrent.futures:
Traceback (most recent call last):
File "/home/pbsds/ntnu/ifield/ifield/mldata/processing.py", line 252, in run
res, start, end = future.result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result
return self.__get_result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/pbsds/ntnu/ifield/ifield/mldata/processing.py", line 290, in _run_job
retval = job.do(pre_args=a)
File "/home/pbsds/ntnu/ifield/ifield/mldata/processing.py", line 41, in do
f(*a, **kw)
File "/home/pbsds/ntnu/ifield/ifield/mldata/extract_file.py", line 21, in extract
f.extractall(path=dest)
File "/usr/lib/python3.10/zipfile.py", line 1647, in extractall
self._extract_member(zipinfo, path, pwd)
File "/usr/lib/python3.10/zipfile.py", line 1693, in _extract_member
os.makedirs(upperdirs)
File "/usr/lib/python3.10/os.py", line 225, in makedirs
mkdir(name, mode)
Activity
pbsds commentedon Dec 12, 2023
related: #67837
pythongh-112998: zipfile: fix extractall makedirs race condition
pythongh-112998: zipfile: fix extractall makedirs race condition
pythongh-112998: zipfile: fix extractall makedirs race condition
pythongh-112998: zipfile: fix extractall makedirs race condition
pythongh-67837, pythongh-112998: Fix dirs creation in concurrent extr…
gh-67837, gh-112998: Fix dirs creation in concurrent extraction (GH-1…
pythongh-67837, pythongh-112998: Fix dirs creation in concurrent extr…