Repository navigation

bpo-33671: efficient zero-copy for shutil.copy* functions (Linux, OSX and Win) - #7160

Merged

giampaolo merged 114 commits into

python:masterfrom

giampaolo:shutil-zero-copy

Jun 12, 2018

Merged

bpo-33671: efficient zero-copy for shutil.copy* functions (Linux, OSX and Win)#7160
giampaolo merged 114 commits into
python:masterfrom
giampaolo:shutil-zero-copy

Commits on May 22, 2018
- have shutil.copyfileobj use sendfile() if possible
  giampaolo
  committed
  1a72c01
- refactoring: use ctx manager
  giampaolo
  committed
  77c4bfa
- add test with non-regular file obj
  giampaolo
  committed
  2afa04a
- emulate case where file size can't be determined
  giampaolo
  committed
  542cd17
- reference _copyfileobj_sendfile directly
  giampaolo
  committed
  3520c6c
- add test for offset() at certain position
  giampaolo
  committed
  050a722
- add test for empty file
  giampaolo
  committed
  c1fd38a
- add test for non regular file dst
  giampaolo
  committed
  2ab6317
- small refactoring
  giampaolo
  committed
  dacc3b6
Commits on May 24, 2018
- leave copyfileobj() alone in order to not introduce any incompatibility
  giampaolo
  committed
  29d5881
- minor refactoring
  giampaolo
  committed
  114c4de
- remove old test
  giampaolo
  committed
  501c0dd
- update docstring
  giampaolo
  committed
  41b4506
- update docstring; rename exception class
  giampaolo
  committed
  fdb0973
- detect platforms which only support file to socket zero copy
  giampaolo
  committed
  64d2bc5
- don't run test on platforms where file-to-file zero copy is not supported
  giampaolo
  committed
  3a3c8ef
- use tempfiles
  giampaolo
  committed
  7861737
- reset verbosity
  giampaolo
  committed
  f3eecfd
- add test for smaller chunks
  giampaolo
  committed
  f67ce57
- add big file size test
  giampaolo
  committed
  d457254
- add comment
  giampaolo
  committed
  8eb211d
- update doc
  giampaolo
  committed
  a0fe703
- update whatsnew doc
  giampaolo
  committed
  7296147
- update doc
  giampaolo
  committed
  d0c3bba
- catch Exception
  giampaolo
  committed
  2cafd80
- remove unused import
  giampaolo
  committed
  bb2a75f
- add test case for error on second sendfile() call
  giampaolo
  committed
  e5025dc
- turn docstring into comment
  giampaolo
  committed
  a36a534
- add one more test
  giampaolo
  committed
  e9da3fa
- update comment
  giampaolo
  committed
  9fcc2e7
- add Misc/NEWS entry
  giampaolo
  committed
  4f32242
Commits on May 25, 2018
- get rid of COPY_BUFSIZE; it belongs to another PR
  giampaolo
  committed
  24ad25a
- update doc
  giampaolo
  committed
  24d20e6
Commits on May 27, 2018
- expose posix._fcopyfile() for OSX
  giampaolo
  committed
  7b6e576
- Merge branch 'master' into shutil-osx-copyfile
  giampaolo
  committed
  b82ddc9
- merge from linux branch
  giampaolo
  committed
  b62b61e
- merge from linux branch
  giampaolo
  committed
  34e9618
- expose fcopyfile
  giampaolo
  committed
  6b20902
Commits on May 28, 2018
- arg clinic for the win implementation
  giampaolo
  committed
  abf3ecb
- convert path type to path_t
  giampaolo
  committed
  91e492c
- expose CopyFileW
  giampaolo
  committed
  e02c69d
- fix windows tests
  giampaolo
  committed
  73837e2
- release GIL
  giampaolo
  committed
  28be4c1
- minor refactoring
  giampaolo
  committed
  6c59adf
- update doc
  giampaolo
  committed
  700629d
- update comment
  giampaolo
  committed
  077912e
- update docstrings
  giampaolo
  committed
  62c6568
- rename functions
  giampaolo
  committed
  a40a755
- rename test classes
  giampaolo
  committed
  7ba0085
- update doc
  giampaolo
  committed
  6c96d97
- update doc
  giampaolo
  committed
  80fbe6e
- update docstrings and comments
  giampaolo
  committed
  fdf4bcb
- avoid do import nt|posix modules if unnecessary
  giampaolo
  committed
  185f130
- set nt|posix modules to None if not available
  giampaolo
  committed
  c8c98ae
- micro speedup
  giampaolo
  committed
  17bb5e6
- update description
  giampaolo
  committed
  d8b9bf9
- add doc note
  giampaolo
  committed
  b59ac57
Commits on May 29, 2018
- use better wording in doc
  giampaolo
  committed
  8eefce7
Commits on May 30, 2018
- Merge branch 'master' into shutil-zero-copy
  giampaolo
  committed
  4fc8c6b
- rename function using 'fastcopy' prefix instead of 'zerocopy'
  giampaolo
  committed
  3048e3d
- use :ref: in rst doc
  giampaolo
  committed
  11102e1
- change wording in doc
  giampaolo
  committed
  7545273
- add test to make sure sendfile() doesn't get called aymore in case it doesn't support file to file copies
  giampaolo
  committed
  3261b74
- move CopyFileW in _winapi and actually expose CopyFileExW instead
  giampaolo
  committed
  51c476d
- fix line endings
  giampaolo
  committed
  729dd23
- add tests for mode bits
  giampaolo
  committed
  1823828
- add docstring
  giampaolo
  committed
  a9d6a07
- remove test file mode class; let's keep it for later when Istart addressing OSX fcopyfile() specific copies
  giampaolo
  committed
  e3ce917
- update doc to reflect new changes
  giampaolo
  committed
  f81a0ec
- update doc
  giampaolo
  committed
  3e7475b
Commits on May 31, 2018
- adjust tests on win
  giampaolo
  committed
  05dd3cf
- fix argument clinic error
  giampaolo
  committed
  9b54930
- update doc
  giampaolo
  committed
  2bec11c
- OSX: expose copyfile(3) instead of fcopyfile(3); also expose flags arg to python
  giampaolo
  committed
  c87648f
- osx / copyfile: use path_t instead of char
  giampaolo
  committed
  941f740
- do not set dst name in the OSError exception in order to remain consistent with platforms which cannot do that (e.g. linux)
  giampaolo
  committed
  4d28c12
- add same file test
  giampaolo
  committed
  2149b8b
- add test for same file
  giampaolo
  committed
  6a02a2a
- have osx copyfile() pre-emptively check if src and dst are the same, otherwise it will return immedialtey and src file content gets deleted
  giampaolo
  committed
  2287508
- turn PermissionError into appropriate SameFileError
  giampaolo
  committed
  b9da5d5
- expose ERROR_SHARING_VIOLATION in order to raise more appropriate SameFileError
  giampaolo
  committed
  c921f46
- honour follow_symlinks arg when using CopyFileEx
  giampaolo
  committed
  bb24490
- update Misc/NEWS
  giampaolo
  committed
  fef8b32
Commits on Jun 5, 2018
- expose CreateDirectoryEx mock
  giampaolo
  committed
  71be453
Commits on Jun 6, 2018
- change C type
  giampaolo
  committed
  6035fe2
- CreateDirectoryExW actual implementation
  giampaolo
  committed
  8dc651e
- provide specific makedirs() implementation for win
  giampaolo
  committed
  5d0eada
- Merge branch 'shutil-zero-copy-8' of https://github.com/giampaolo/cpython into shutil-zero-copy-8
  giampaolo
  committed
  d67cdc5
- fix typo
  giampaolo
  committed
  f65c8ae
- skeleton for SetNamedSecurityInfo
  giampaolo
  committed
  9c4508e
- get security info for src path
  giampaolo
  committed
  bb1fee6
- finally set security attrs
  giampaolo
  committed
  566898a
- add unit tests
  giampaolo
  committed
  f435053
- mimick os.makedirs() behavior and raise if dst dir exists
  giampaolo
  committed
  30c9a57
- set 2 paths for OSError object
  giampaolo
  committed
  33f362f
- set 2 paths for OSError object
  giampaolo
  committed
  e17e729
- expand windows test
  giampaolo
  committed
  bc46f75
- in case of exception on os.sendfile() set filename and filename2 exception attributes
  giampaolo
  committed
  cabbc02
- set 2 filenames (src, dst) for OSError in case copyfile() fails on OSX
  giampaolo
  committed
  d22ee08
Commits on Jun 7, 2018
Commits on Jun 8, 2018
- refactoring + introduce a new _fastcopy_binfileobj() fun
  giampaolo
  committed
  42a597e
- remove CopyFileEx and other C wrappers
  giampaolo
  committed
  5008a8d
- remove code related to CopyFileEx
  giampaolo
  committed
  e89dd20
- Recognize binary files in copyfileobj()
  giampaolo
  committed
  c0dc4b8
- set 1MB copy bufsize on win; also add a global _COPY_BUFSIZE variable
  giampaolo
  committed
  29b9730
- use ctx manager for memoryview()
  giampaolo
  committed
  a1bed32
Commits on Jun 9, 2018
- update doc
  giampaolo
  committed
  d9d27a7
- remove outdated doc
  giampaolo
  committed
  17bd78b
- remove last CopyFileEx remnants
  giampaolo
  committed
  b1d4917
Commits on Jun 12, 2018
- OSX - use fcopyfile(3) instead of copyfile(3)
  giampaolo
  committed
  5ce94e4
- update doc
  giampaolo
  committed
  07bcef5

Commits

Commits on May 22, 2018

Commits on May 24, 2018

Commits on May 25, 2018

Commits on May 27, 2018

Commits on May 28, 2018

Commits on May 29, 2018

Commits on May 30, 2018

Commits on May 31, 2018

Commits on Jun 5, 2018

Commits on Jun 6, 2018

Commits on Jun 7, 2018

Commits on Jun 8, 2018

Commits on Jun 9, 2018

Commits on Jun 12, 2018