Repository navigation
Add fs.copyFile for copying files, using uv_fs_copyfile #14906
Description
Activity
- changed the title
[-]Add fs.copyFile for copying files[/-][+]Add fs.copyFile for copying files, using uv_fs_copyfile[/+]on Aug 17, 2017 I'll be working on this once libuv can be updated. Right now, it's blocked on a broken test.
Reacted by Valentin Semirulnik, Daniel Lo Nigro and Eric AndersonCould someone detail the advantages of
uv_fs_copyfile?Do note that we make no guarentees to expose every libuv API. 😉
Could someone detail the advantages of uv_fs_copyfile?
Two main advantages:
- It's significantly faster than a JS implementation, as it uses native system calls where available (eg. CopyFile on Windows, copyfile(3) on Mac OS). See some benchmark results on use fcopy for faster file copying yarnpkg/yarn#3290 and [WIP] Use the windows CopyFile API for copying files yarnpkg/yarn#2960 for example.
- It allows us to take advantage of copy-on-write semantics on file systems that support it, such as ZFS, BTRFS, ReFS, APFS. Copy-on-write means that creating a copy of a file reuses the same data on disk, similar to a hardlink. It uses very very little additional disk space, until the file is modified (at which point it's actually physically copied). This is hugely beneficial for apps that deal with lots of file copies, like Yarn and npm. It allows the benefits of hardlinks/symlinks (very fast installation, reusing the same data on the disk) without any of the disadvantages (changing one of the copies does not modify any of the other ones).
Copy-on-write needs native system calls to function, as the hacky mechanism used in Node.js projects at the moment (read the input file into a buffer and write it to the new location) is not recognised as a file copy by the OS. The file system just sees it as creation of a brand new file.
Reacted by Colin Ihrig, Jeremiah Senkpiel, Sindre Sorhus, Joran Dirk Greef, Eric Anderson, Seth Holladay and Yaroslav AdminOne thing to keep an eye out for. Replacing a JS implementation with a native API for file system related stuff will end up changing the underlying syscall failures if anything goes wrong. We saw this happen with realpath, and things got weird quickly.
This is in no way saying we shouldn't do this, but we should be pretty thorough with smoke testing if we do decide to change things. Either way it will likely be semver major, even if not obviously so
We aren't replacing anything here though. This would be purely semver minor for core. The applications using it can pick their own level of semver.
. Replacing a JS implementation with a native API
There is no core JS implementation at the moment. This would add a brand new function, so there's no breaking change. Currently, apps are implementing their own JS versions of it.
Existing #12902 requests the same feature.
I am in favor of implementing this on core 👍
- added 4 commits that reference this issue
on Sep 10, 2017
A new
uv_fs_copyfilefunction recently landed in libuv in order to allow more efficient copying of files (in the future, it could allow copy-on-write semantics on file systems that support it). A copyFile function that uses this should be added to Node.js. 😃Depends on upgrade to libuv 1.14.0 (#14866)