Repository navigation
Description
Hi! I'm looking for help solving an issue we're starting to see more often on Windows, when npm install builds our native node modules.
The issue is that because npm install runs native module builds in parallel, and each needs exclusive access to some files, they end up failing due to not being able to get that exclusive access. We believe we're seeing this issue more often because we recently standardized on using node-addon-api for all five of our native modules. Thus, we have five parallel builds, all trying to access a lot of the same files. I've provided a series of example errors below (if someone wants full logs, I can provide those as well).
To be clear, I don't think this is necessarialy node-gyp's fault, however it may be part of the solution. For now, I am here to see if anyone in the node-gyp community has any ideas/thoughts/suggestions on how to proceed next. So far here's some ideas I've had:
- Use
--install-strategy=nestedto de-deduplicatenode-addon-api, thus reducing (maybe even eliminating) the chance of conflicting file access. Our project is ultimately bundled, so this will probably work out ok, but there are likely going to be other complications. Would be nice if we could only de-deduplicatenode-addon-api. - Combine some of the five native modules, thus reducing the total number. This won't fix the problem, but it'll reduce the chance of it happening.
- Solicit
npmto not do parallel native node module installs. - Add a retry mechanism to node-gyp.
- Prebuild the binaries for as many of our native node_modules as we can, thus avoiding having to rebuild them for the main project.
- Write our own build-time code to de-deduplicate
node-addon-api. - Use
--ignore-scripts, and then usenpm rebuildfor any package which has an installScript: true in package-lock. Currently using this. - Use
--foreground-scripts. Need to test this.
Details
The specified task executable "CL.exe" could not be run. System.IO.IOException: The process cannot access the file 'PATH_TO_NODE_MODULES\node-addon-api\Release\obj\nothing\nothing.tlog\CL.command.1.tlog' because it is being used by another process. [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at System.IO.StreamReader..ctor(String path, Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize, Boolean checkHost) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at System.IO.StreamReader..ctor(String path) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at System.IO.File.OpenText(String path) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at Microsoft.Build.CPPTasks.TrackedVCToolTask.MapSourcesToCommandLines() [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at Microsoft.Build.CPPTasks.TrackedVCToolTask.GenerateSourcesOutOfDateDueToCommandLine() [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at Microsoft.Build.CPPTasks.CL.ComputeOutOfDateSources() [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at Microsoft.Build.CPPTasks.TrackedVCToolTask.SkipTaskExecution() [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at Microsoft.Build.Utilities.ToolTask.Execute() [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
Details
node-gyp\src\win_delay_load_hook.cc(1,1): error C1083: Cannot open compiler generated file: 'PATH_TO_NODE_MODULES\node-addon-api\Release\obj\nothing\win_delay_load_hook.obj': Permission denied [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
Details
npm error LINK : fatal error LNK1181: cannot open input file 'Release\obj\nothing\\node-addon-api\nothing.obj' [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
Details
The specified task executable "CL.exe" could not be run. System.IO.IOException: The process cannot access the file 'PATH_TO_NODE_MODULES\node-addon-api\Release\obj\nothing\nothing.tlog\CL.write.1.tlog' because it is being used by another process. [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at System.IO.StreamWriter.CreateFile(String path, Boolean append, Boolean checkHost) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding, Int32 bufferSize, Boolean checkHost) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at System.IO.File.InternalWriteAllText(String path, String contents, Encoding encoding, Boolean checkHost) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at System.IO.File.WriteAllText(String path, String contents, Encoding encoding) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at Microsoft.Build.Utilities.CanonicalTrackedOutputFiles.SaveTlog(DependencyFilter includeInTLog) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at Microsoft.Build.CPPTasks.CL.PostExecuteTool(Int32 exitCode) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at Microsoft.Build.CPPTasks.TrackedVCToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at Microsoft.Build.CPPTasks.CL.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
at Microsoft.Build.Utilities.ToolTask.Execute() [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
Details
npm error LINK : fatal error LNK1181: cannot open input file 'Release\obj\nothing\win_delay_load_hook.obj' [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
Details
npm error C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft\VC\v170\Microsoft.CppBuild.targets(386,5): error MSB3374: The last access/last write time on file "Release\obj\nothing\nothing.tlog\unsuccessfulbuild" cannot be set. Could not find file 'PATH_TO_NODE_MODULES\node-addon-api\Release\obj\nothing\nothing.tlog\unsuccessfulbuild'. [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
Details
npm error FileTracker : error FTK1011: could not create the new file tracking log file: PATH_TO_NODE_MODULES\node-addon-api\Release\obj\nothing\nothing.tlog\CL.write.1.tlog. The file exists. [PATH_TO_NODE_MODULES\node-addon-api\nothing.vcxproj]
Activity
lukekarrys commentedon Dec 2, 2024
@MRayermannMSFT I think
3is likely the best option sincenpmis in charge of how and when those scripts get run.There is also a hack/workaround that uses the npm flag
--foreground-scriptswhich forces npm scripts to run sequentially: npm/cli#3034 (comment). Can you try and see if this flag does the same to native module builds?MRayermannMSFT commentedon Dec 2, 2024
Oh nice! I had actually just implemented a workaround using
--ignore-scripts, where we use that flag, and then we usenpm rebuildfor any package which has aninstallScript: trueinpackage-lock. I'll give--foreground-scriptsa try as well though.tmm1 commentedon Dec 29, 2024
Does this manifest as follows?
Does the
--foreground-scriptsworkaround help?EDIT: Missed the detailed logs above. I can confirm
--foreground-scriptsruns in serial and works around the issue.fix: disable msbuild.exe nodeReuse
tmm1 commentedon Dec 30, 2024
An alternative to
--foreground-scriptsis to use a simple retry loop aroundnpm ci. However, this requires ataskkill /im vctip.exe /fwithin the loop to work (or the patch from #3112)fix: disable msbuild.exe nodeReuse (#3112)
lukekarrys commentedon Apr 1, 2025
I think with #3112 landing and any other fix being outside of what
node-gypis likely responsible for, I'm going to close this issue. Please let me know if this should be reopened if there's anything here I'm missing.MRayermannMSFT commentedon Apr 1, 2025
Thanks for making an effort at fixing this! I know it's a bit of a power user scenario. Will let you know if it comes up again.
tmm1 commentedon Jun 23, 2025
@MRayermannMSFT I'm curious if you noticed any difference after #3112?
I had a hard time upgrading to the latest node-gyp, but figured it out in #3172
However I still see some errors like:
as well as warnings:
These go away by adding
--foreground-scripts, but that makes execution unbearably slow.tmm1 commentedon Jun 23, 2025
Yea my logs clearly show
/nodeReuse:falseadded, but the errors continue.It seems that flag will stop lingering processes, but currently tlog files are always generated and node-gyp running in parallel will cause failures. See dotnet/msbuild#10814
MRayermannMSFT commentedon Jun 23, 2025
Unfortunately I no longer work on the project that was haivng these issues. However, I do see that the project is still using the strategy of "Use --ignore-scripts, and then use npm rebuild for any package which has an installScript: true in package-lock". Guess they haven't gotten around to trying out the fixes made in April.
tmm1 commentedon Jun 27, 2025
@lukekarrys @cclauss can you re-open this issue?
Seems like we would need some type of retry at the msbuild invocation layer to solve this.
cclauss commentedon Jun 28, 2025
#3172 (comment)
tmm1 commentedon Jul 18, 2025
I'm still working on trying to make a standalone repro here. It has been surprisingly difficult, mostly because the problems are so intermittent. The standard github runners have low cpu counts so its hard to achieve much concurrency.
Here's another type of recent failure:
tmm1 commentedon Jul 18, 2025
Interesting, here I was still using MSVC 2019, but since then the windows-2019 images have been deprecated. It seems like the race condition specific to msbuild holding files open (or perhaps respecting
/nodeReuse:falsecorrectly) is much improved in newer versions of MSVC.tmm1 commentedon Jul 29, 2025
Here is a failing run: https://github.com/anysphere/node-gyp-win32-test/actions/runs/16587954733/job/46916898348
tmm1 commentedon Jul 29, 2025
Another failure here: https://github.com/anysphere/node-gyp-win32-test/actions/runs/16587962641/job/46916919457
And another: https://github.com/anysphere/node-gyp-win32-test/actions/runs/16587963490/job/46916922118
In order to reproduce, I had to create a larger github runner with more CPU cores:
tmm1 commentedon Jul 29, 2025
One thing I noticed is gyp sets
TrackerLogDirectory. Its possible using a distinct dir here would help.node-gyp/gyp/pylib/gyp/MSVSSettings.py
Line 800 in 3df8789
There is also another option
TrackFileAccessthat might help.tmm1 commentedon Jul 29, 2025
It appears that even after
npm ciexits due to error, there are many node/MSBuild processes still running. This explains why retry loops often don't work.https://github.com/anysphere/node-gyp-win32-test/actions/runs/16590860920/job/46925908613
In the build above I used
npm ci || npm ci, but the second invocation fails right away with:tmm1 commentedon Jul 29, 2025
FWIW if you use
pnpmit works fine. So the issue here may be thatnpmitself is not waiting on or cleaning up parallel postinstalls when one fails. There also seems to be some difference in how the two buildnode-addon-api, which multiple packages depend on and cause the race in thenpmcase.https://github.com/anysphere/node-gyp-win32-test/actions/runs/16591354158/job/46927553476
tmm1 commentedon Jul 29, 2025
Yep, it looks like
pnpmis building them all in separate dirs so the file race doesn't occur.