Skip to content

Assertion `parser->current_buffer_.IsEmpty()' Exception #34016

Description

@omkadiri

Version: 12.0.0
Platform: UNIX AWS EC2

Processing lot of json file from S3 using worker_thread node module in an EC2(m4.xlarge)

Process step
(1)Got message for SQS
(2)Use message info to list bucket using listObjectV2
(3)Get each json from the list api call and process it(merge the data into one file).
(4)write back to S3.

This error only occurs when the number of json file increase to hundred of thousand

Exception message from run
/.nvm/versions/node/v12.0.0/bin/node[3884]: ../src/node_http_parser_impl.h:500:static void node::{anonymous}::Parser::Finish(const v8::FunctionCallbackInfov8::Value&): Assertion `parser->current_buffer_.IsEmpty()' failed.
1: 0x97a1c0 node::Abort() [/.nvm/versions/node/v12.0.0/bin/node]
2: 0x97a247 [/.nvm/versions/node/v12.0.0/bin/node]
3: 0x991492 [/.nvm/versions/node/v12.0.0/bin/node]
4: 0xb7a2b6 [/.nvm/versions/node/v12.0.0/bin/node]
5: 0xb7c1c9 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/.nvm/versions/node/v12.0.0/bin/node]
6: 0x1a5c1a2 [/.nvm/versions/node/v12.0.0/bin/node]

This file processing run successfully 99% of the time.

Node should catch this exception and print a detail message for why this exception occurs.

Activity

  1. added
    httpIssues and PRs related to the http subsystem.
    http_parserIssues and PRs related to the HTTP parser implementation or http_parser binding.
    on Jun 22, 2020
  2. addaleax commented on Jun 22, 2020

    @addaleax
    Member

    Is there any chance you can verify that this also happens on the latest v12.x release? v12.0.0 was over a year ago, and lots of bug fixes have gone into it since then.

  3. omkadiri commented on Jun 23, 2020

    @omkadiri
    Author

    I got the same exception running with version 12.18.1

    /.nvm/versions/node/v12.18.1/bin/node[3921]: ../src/node_http_parser_impl.h:509:static void node::{anonymous}::Parser::Finish(const v8::FunctionCallbackInfov8::Value&): Assertion `parser->current_buffer_.IsEmpty()' failed.
    1: 0xa08900 node::Abort() [/.nvm/versions/node/v12.18.1/bin/node]
    2: 0xa0897e [/.nvm/versions/node/v12.18.1/bin/node]
    3: 0xa20302 [/.nvm/versions/node/v12.18.1/bin/node]
    4: 0xbeaee9 [/.nvm/versions/node/v12.18.1/bin/node]
    5: 0xbeccd7 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/.nvm/versions/node/v12.18.1/bin/node]
    6: 0x13c9919 [/.nvm/versions/node/v12.18.1/bin/node]

  4. bnoordhuis commented on Jul 6, 2020

    @bnoordhuis
    Member

    I haven't been able to reproduce but I can see the general shape of the bug. The call chain probably looks like this:

    • Parser::Execute() -> llhttp_execute() -> Parser::on_body() -> parserOnBody() -> ... -> parser.finish()

    I think it should be safe to remove the CHECK and reset the current_buffer_ fields like this:

    diff --git a/src/node_http_parser.cc b/src/node_http_parser.cc
    index c7a3df8d06..f9729f3f51 100644
    --- a/src/node_http_parser.cc
    +++ b/src/node_http_parser.cc
    @@ -514,7 +514,9 @@ class Parser : public AsyncWrap, public StreamListener {
         Parser* parser;
         ASSIGN_OR_RETURN_UNWRAP(&parser, args.Holder());
     
    -    CHECK(parser->current_buffer_.IsEmpty());
    +    parser->current_buffer_.Clear();
    +    parser->current_buffer_len_ = 0;
    +    parser->current_buffer_data_ = nullptr;
         Local<Value> ret = parser->Execute(nullptr, 0);
     
         if (!ret.IsEmpty())

    Opinions? Parser::Execute() performs the same reset after calling llhttp_execute().

    @omkadiri If you want you can compile Node from source with that patch applied until this issue is resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    httpIssues and PRs related to the http subsystem.http_parserIssues and PRs related to the HTTP parser implementation or http_parser binding.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions