Skip to content

parallel/test-domain-error-types can crash if GC timings get unlucky #28275

Description

@LeszekSwirski
  • Version: v13.0.0-pre
  • Platform: Linux (debian)
  • Subsystem: Domain

It appears that the TickObject used for process.nextTick can die between the tick callback and the tick's after hook (possibly only in the case where the callback throws an exception), in which case there is no longer a strong reference to the tick's domain and the WeakReference to the domain can die (on an unlucky GC run). Then, the exit() call in domain.js will be called on undefined, and will crash.

Depending on the desired semantics, either current.get() in the after() call should check the WeakReference, or the TickObject resource/domain should be kept strongly alive until after the after hook.

Reproducible by manually setting a tighter GC interval and stressing compaction on a debug node build:

$ node --gc-interval=100 --stress-compaction test/parallel/test-domain-error-types.js
TypeError: Cannot read property 'exit' of undefined
    at AsyncHook.after (domain.js:82:20)
    at emitHook (internal/async_hooks.js:164:38)
    at emitAfterScript (internal/async_hooks.js:363:5)
    at process._fatalException (internal/process/execution.js:180:9)

Activity

added
domainIssues and PRs related to the domain subsystem.
confirmed-bugIssues and PRs for confirmed bugs.
on Jun 19, 2019

addaleax commented on Jun 19, 2019

@addaleax
Member

Thanks for the bug report, this is something I definitely missed.

I can’t reproduce the issue without a debug build, so testing this out is going to take a while, but something like this might help:

diff --git a/lib/domain.js b/lib/domain.js
index b2ffed2741bc..3dc56c179aca 100644
--- a/lib/domain.js
+++ b/lib/domain.js
@@ -73,13 +73,18 @@ const asyncHook = createHook({
     if (current !== undefined) { // Enter domain for this cb
       // We will get the domain through current.get(), because the resource
       // object's .domain property makes sure it is not garbage collected.
+      // However, we do need to make the reference to the domain non-weak,
+      // so that it cannot be garbage collected before the after() hook.
+      current.incRef();
       current.get().enter();
     }
   },
   after(asyncId) {
     const current = pairing.get(asyncId);
     if (current !== undefined) { // Exit domain for this cb
-      current.get().exit();
+      const domain = current.get();
+      current.decRef();
+      domain.exit();
     }
   },
   destroy(asyncId) {
diff --git a/src/node_util.cc b/src/node_util.cc
index 518865fe5368..fa39583b04ba 100644
--- a/src/node_util.cc
+++ b/src/node_util.cc
@@ -189,12 +189,26 @@ class WeakReference : public BaseObject {
       args.GetReturnValue().Set(weak_ref->target_.Get(isolate));
   }
 
+  static void IncRef(const FunctionCallbackInfo<Value>& args) {
+    WeakReference* weak_ref = Unwrap<WeakReference>(args.Holder());
+    if (weak_ref->reference_count_ == 0) weak_ref->target_.ClearWeak();
+    weak_ref->reference_count_++;
+  }
+
+  static void DecRef(const FunctionCallbackInfo<Value>& args) {
+    WeakReference* weak_ref = Unwrap<WeakReference>(args.Holder());
+    CHECK_GE(weak_ref->reference_count_, 1);
+    weak_ref->reference_count_--;
+    if (weak_ref->reference_count_ == 0) weak_ref->target_.SetWeak();
+  }
+
   SET_MEMORY_INFO_NAME(WeakReference)
   SET_SELF_SIZE(WeakReference)
   SET_NO_MEMORY_INFO()
 
  private:
   Global<Object> target_;
+  uint64_t reference_count_ = 0;
 };
 
 static void GuessHandleType(const FunctionCallbackInfo<Value>& args) {
@@ -294,6 +308,8 @@ void Initialize(Local<Object> target,
   weak_ref->InstanceTemplate()->SetInternalFieldCount(1);
   weak_ref->SetClassName(weak_ref_string);
   env->SetProtoMethod(weak_ref, "get", WeakReference::Get);
+  env->SetProtoMethod(weak_ref, "incRef", WeakReference::IncRef);
+  env->SetProtoMethod(weak_ref, "decRef", WeakReference::DecRef);
   target->Set(context, weak_ref_string,
               weak_ref->GetFunction(context).ToLocalChecked()).Check();

addaleax commented on Jun 20, 2019

@addaleax
Member

Actually, I think this may have been caused by #26211 because the domain is now no longer kept alive through the domain stack. I’ll open a PR with the above suggestion, it does seem to resolve the issue.

added a commit that references this issue on Aug 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    confirmed-bugIssues and PRs for confirmed bugs.domainIssues and PRs related to the domain subsystem.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions