Repository navigation
Node.js[5483]: ../src/node.cc:701:void node::ResetStdio(): Assertion `!(err != 0) || (err == -1 && (*__errno_location ()) == 1)' failed. #51519
Description
Activity
Can you provide a minimal repro?
This is the assertion that fails// Normally we expect err == 0. But if macOS App Sandbox is enabled, // tcsetattr will fail with err == -1 and errno == EPERM. CHECK_IMPLIES(err != 0, err == -1 && errno == EPERM);
Try making a simple express app and running it in linux using this command
Then do some other simple linux commands and then log off the server using
exityou will notice (if it happens) that it will go down, sometimes it takes a few days or hours, but if you visit the express site throughout the day, and use the linux server too, eventually this error comes up and you will notice the app has crashed.Without a reproducible snippet of code, it seems very hard to me to find out what is causing this
This issue/PR was marked as stalled, it will be automatically closed in 30 days. If it should remain open, please leave a comment explaining why it should remain open.
Hi! I've marked this issue as stalled, as @marco-ippolito asked for a minimal reproduction back in January, and one is yet to be provided.
(The stalled bot doesn't appear to be working, so I've manually closed. Feel free to reopen if this is no longer stalled)
I just experienced this:
Node.js[843246]: ../src/node.cc:701:void node::ResetStdio(): Assertion `!(err != 0) || (err == -1 && (*__errno_location ()) == 1)' failed.node --version v18.20.4My script is running for a long period of time (hours) and producing a lot of output on stdout and a small amount on stderr. It is not reading anything from stdin.
I don't really know how to get from here to a minimal repro because I have tons of nodejs scripts, and no others have ever done this. My script spends a lot of time in
child_process.execSynccalls, I don't know if that could be relevant or if it's entirely irrelevant (e.g. perhaps this part of src/node.cc is concerned entirely with the stdout of the main process itself, which is not involved, since I'm not passing interesting options like stdio: 'inherit').For the last hour-plus before it fails, this script has done nothing but orchestrate a series
cp.execSynccalls to the AWS CLI, with no fancy options beyond{ encoding: 'utf8' }. I do pay attention to the return value and parse it as JSON on each call. I can't see anything that would cause memory pressure because it works in small batches and retains nothing between batches. Stdout and stderr are both being piped to a file (at the bash shell level).I've restarted it with much less output to stdout and will report on whether that seems to matter at all.
Unfortunately this happens consistently even with much less output.
I'm going to try zero output just to see if it really has anything to do with normal use of standard output at all.
I neglected to mention I'm running Ubuntu 22.04 LTS.
If you can provide a minimal reproduction, it'd be easier for someone to reproduce.
I realize that. The difficulty is that it's probably dependent on specific responses I'm getting from an external service (AWS) and they contain information that isn't mine to share. But if I can nail it down and it's not something sensitive I will share it.
I just came here because we are experiencing the same / a similar issue in our pipeline from time to time. It seems like it appears whenever our build runner has high utilization. So it might have something to do with the available system resources which makes it hard to reproduce. It sometimes appears in our pipeline, sometimes it does not. We are running in qemu VMs with ubuntu-22.04 inside:
# /nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node[16614]: void node::ResetStdio() at ../../src/node.cc:729 # Assertion failed: !(err != 0) || (err == -1 && (*__errno_location ()) == 1) ----- Native stack trace ----- 1: 0xc1c005 node::Assert(node::AssertionInfo const&) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 2: 0xbc4e0e [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 3: 0xbc6562 node::TearDownOncePerProcess() [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 4: 0xbcd2f0 node::Start(int, char**) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 5: 0x7fbc834b914e [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6] 6: 0x7fbc834b9209 __libc_start_main [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6] 7: 0xaeb455 _start [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] ELIFECYCLE Command failed with exit code 134. # Node.js[16557]: void node::ResetStdio() at ../../src/node.cc:729 # Assertion failed: !(err != 0) || (err == -1 && (*__errno_location ()) == 1) ----- Native stack trace ----- 1: 0xc1c005 node::Assert(node::AssertionInfo const&) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 2: 0xbc4e0e [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 3: 0x7f9e97f15b56 [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6] 4: 0x7f9e97f15c8e [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6] 5: 0xb21029 [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 6: 0xb21099 node::DefaultProcessExitHandlerInternal(node::Environment*, node::ExitCode) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 7: 0xb85a95 node::Environment::Exit(node::ExitCode) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 8: 0xed03f2 v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 9: 0xed09c8 [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 10: 0xed1238 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 11: 0x18dcdf6 [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] # Node.js[16557]: void node::ResetStdio() at ../../src/node.cc:729 # Assertion failed: !(err != 0) || (err == -1 && (*__errno_location ()) == 1) ----- Native stack trace ----- 1: 0xc1c005 node::Assert(node::AssertionInfo const&) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 2: 0xbc4e0e [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 3: 0xbc4ffc node::TrapWebAssemblyOrContinue(int, siginfo_t*, void*) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 4: 0x7f9e97f135c0 [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6] 5: 0x[154](https://github.com/PionixPro/summiteer-cloud/actions/runs/11857584699/job/33046303079#step:20:155)0450 v8::internal::tracing::TraceEventHelper::GetTracingController() [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 6: 0xff7ca5 v8::internal::Isolate::CaptureDetailedStackTrace(int, v8::StackTrace::StackTraceOptions) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 7: 0xe6c790 v8::StackTrace::CurrentStackTrace(v8::Isolate*, int, v8::StackTrace::StackTraceOptions) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 8: 0xc1bf36 node::GetCurrentStackTrace(v8::Isolate*, int) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 9: 0xb67b1a node::DumpJavaScriptBacktrace(_IO_FILE*) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 10: 0xc1c00d node::Assert(node::AssertionInfo const&) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 11: 0xbc4e0e [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 12: 0x7f9e97f15b56 [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6] 13: 0x7f9e97f15c8e [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6] 14: 0xb21029 [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 15: 0xb21099 node::DefaultProcessExitHandlerInternal(node::Environment*, node::ExitCode) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 16: 0xb85a95 node::Environment::Exit(node::ExitCode) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 17: 0xed03f2 v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 18: 0xed09c8 [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 19: 0xed1238 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 20: 0x18dcdf6 [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] ELIFECYCLE Command failed with exit code 134. # Node.js[[164](https://github.com/PionixPro/summiteer-cloud/actions/runs/11857584699/job/33046303079#step:20:165)71]: void node::ResetStdio() at ../../src/node.cc:729 # Assertion failed: !(err != 0) || (err == -1 && (*__errno_location ()) == 1) ----- Native stack trace ----- 1: 0xc1c005 node::Assert(node::AssertionInfo const&) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 2: 0xbc4e0e [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 3: 0x7f097afdcb56 [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6] 4: 0x7f097afdcc8e [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6] 5: 0xb21029 [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 6: 0xb21099 node::DefaultProcessExitHandlerInternal(node::Environment*, node::ExitCode) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 7: 0xb85a95 node::Environment::Exit(node::ExitCode) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 8: 0xed03f2 v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 9: 0xed09c8 [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 10: 0xed1238 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 11: 0x18dcdf6 [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] # Node.js[16471]: void node::ResetStdio() at ../../src/node.cc:729 # Assertion failed: !(err != 0) || (err == -1 && (*__errno_location ()) == 1) ----- Native stack trace ----- 1: 0xc1c005 node::Assert(node::AssertionInfo const&) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 2: 0xbc4e0e [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 3: 0xbc4ffc node::TrapWebAssemblyOrContinue(int, siginfo_t*, void*) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node] 4: 0x7f097afda5c0 [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6]I just had this happen once on Node v22.14.0. I've no idea if this extra information is useful but here it is anyway:
Project is BrickBot at this commit, which I launched with
npm run start > brickbot.log 2>&1 &. It's running on the cheapest possible Hetzner virtual machine running Fedora 41, but even so it's the only thing running, and it's hardly using any resources.The only clue I have based on the logs is that I'm throwing an error from inside an async function.
Logs:
> discord-getting-started@1.0.0 start > node app.js Loaded 477 themes. My favourite is Spybotics Listening on port 3000 404 file:///root/brickbot/utils.js:22 throw new Error(JSON.stringify(data)); ^ Error: {"message":"Unknown Webhook","code":10015} at DiscordRequest (file:///root/brickbot/utils.js:22:11) at process.processTicksAndRejections (node:internal/process/task_queues:105:5) Node.js v22.14.0 # Node.js[96795]: void node::ResetStdio() at ../../src/node.cc:760 # Assertion failed: !(err != 0) || (err == -1 && (*__errno_location ()) == 1) ----- Native stack trace ----- 1: 0x7fc4d25e4d47 node::Assert(node::AssertionInfo const&) [/lib64/libnode.so.127] 2: 0x7fc4d259a436 [/lib64/libnode.so.127] 3: 0x7fc4d22296c1 [/lib64/libc.so.6] 4: 0x7fc4d222978e [/lib64/libc.so.6] 5: 0x7fc4d24fa33d [/lib64/libnode.so.127] 6: 0x7fc4d25169ad node::DefaultProcessExitHandlerInternal(node::Environment*, node::ExitCode) [/lib64/libnode.so.127] 7: 0x7fc4d257a9b9 node::Environment::Exit(node::ExitCode) [/lib64/libnode.so.127] 8: 0x7fc4d25f38c7 node::errors::TriggerUncaughtException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>, bool) [/lib64/libnode.so.127] 9: 0x7fc4d432c5e2 [/lib64/libnode.so.127] # Node.js[96795]: void node::ResetStdio() at ../../src/node.cc:760 # Assertion failed: !(err != 0) || (err == -1 && (*__errno_location ()) == 1) ----- Native stack trace ----- 1: 0x7fc4d25e4d47 node::Assert(node::AssertionInfo const&) [/lib64/libnode.so.127] 2: 0x7fc4d259a436 [/lib64/libnode.so.127] 3: 0x7fc4d259a614 node::TrapWebAssemblyOrContinue(int, siginfo_t*, void*) [/lib64/libnode.so.127] 4: 0x7fc4d2227050 [/lib64/libc.so.6] 5: 0x7fc4d4cb87e4 v8::internal::tracing::TraceEventHelper::GetTracingController() [/lib64/libnode.so.127] 6: 0x7fc4d46c7e35 v8::internal::Isolate::CaptureDetailedStackTrace(int, v8::StackTrace::StackTraceOptions) [/lib64/libnode.so.127] 7: 0x7fc4d4540c67 v8::StackTrace::CurrentStackTrace(v8::Isolate*, int, v8::StackTrace::StackTraceOptions) [/lib64/libnode.so.127] 8: 0x7fc4d25e1f3c node::GetCurrentStackTrace(v8::Isolate*, int) [/lib64/libnode.so.127] 9: 0x7fc4d254179e node::DumpJavaScriptBacktrace(_IO_FILE*) [/lib64/libnode.so.127] 10: 0x7fc4d25e4d4f node::Assert(node::AssertionInfo const&) [/lib64/libnode.so.127] 11: 0x7fc4d259a436 [/lib64/libnode.so.127] 12: 0x7fc4d22296c1 [/lib64/libc.so.6] 13: 0x7fc4d222978e [/lib64/libc.so.6] 14: 0x7fc4d24fa33d [/lib64/libnode.so.127] 15: 0x7fc4d25169ad node::DefaultProcessExitHandlerInternal(node::Environment*, node::ExitCode) [/lib64/libnode.so.127] 16: 0x7fc4d257a9b9 node::Environment::Exit(node::ExitCode) [/lib64/libnode.so.127] 17: 0x7fc4d25f38c7 node::errors::TriggerUncaughtException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>, bool) [/lib64/libnode.so.127] 18: 0x7fc4d432c5e2 [/lib64/libnode.so.127]Going by #51519 (comment), I expect it's caused by node's process group having been orphaned and the SIGHUP signal being ignored or caught by the script.
The kernel sends SIGHUP when the process group is orphaned and that normally kills node but not, of course, when you handle the signal. Then, when node tries to exit and calls tcsetattr to restore the tty, the system call fails with EIO.
Long story short, we should relax the error check. I'll open a pull request.
- added a commit that references this issue
on May 8, 2025 - added a commit that references this issue
on May 16, 2025 - added a commit that references this issue
on Jun 10, 2025
Version
18.19.0
Platform
Linux localhost 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Subsystem
No response
What steps will reproduce the bug?
Running nodejs with logging enabled
How often does it reproduce? Is there a required condition?
It reproduces periodically at weird intervals. Doesn't appear to have an actual condition
What is the expected behavior? Why is that the expected behavior?
The expected behavior is that it should be running without issues or at least posting some genuine error. This is expected because this error originates out of a NodeJS core lib, meaning we are dealing with some core error which is usually not expected.
What do you see instead?
Additional information
No response