Skip to content

rocksdb coredump when graph load #833

Closed

Description

@kidleaf-jiang

Actual behavior 实际表现

image

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007fa733a22a03, pid=13467, tid=140356624496384
#
# JRE version: Java(TM) SE Runtime Environment (8.0_45-b14) (build 1.8.0_45-b14)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.45-b02 mixed mode linux-amd64 )
# Problematic frame:
# C  [librocksdbjni3759649349396566752.so+0x2f4a03]  rocksdb::MemTable::ApproximateMemoryUsage()+0xb3
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

---------------  T H R E A D  ---------------

Current thread (0x00007fafc80c5000):  JavaThread "Finalizer" daemon [_thread_in_native, id=13852, stack(0x00007fa752aeb000,0x00007fa752bec000)]

siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: 0x0000000000000000

Registers:
RAX=0x0000000000000001, RBX=0x00007fa4a45070f0, RCX=0x0010000000000000, RDX=0x3237373531353137
RSP=0x00007fa752bea4d0, RBP=0x00007fa752bea590, RSI=0x00000000026e0918, RDI=0x0000000000000000
R8 =0x3237373531353137, R9 =0x000000000000361c, R10=0x0000000000000158, R11=0x0000000000000001
R12=0x00007fa752bea4f0, R13=0x0000000000000000, R14=0x0000000000000001, R15=0x00007fa4a4507a80
RIP=0x00007fa733a22a03, EFLAGS=0x0000000000010202, CSGSFS=0x0000000000000033, ERR=0x0000000000000000
  TRAPNO=0x000000000000000d

Top of Stack: (sp=0x00007fa752bea4d0)
0x00007fa752bea4d0:   0000000000000100 00007fafcf520cd0
0x00007fa752bea4e0:   0000000000000000 0000000000000000
0x00007fa752bea4f0:   00007fa7382d6100 00007fafc80c5000
0x00007fa752bea500:   0000000000000000 00007fafc80681e0
0x00007fa752bea510:   00007fa752bea560 00007fafcf520ac2
0x00007fa752bea520:   00007fa7382d6128 00007fafc80c5000
0x00007fa752bea530:   00007fafcf520b50 00007fafc808b100
0x00007fa752bea540:   00007fa7382d6128 00007fa7382d6128
0x00007fa752bea550:   00007fa3880cb368 00007fa733b13b09
0x00007fa752bea560:   00007fa752bea5c0 00007fa4a449c680
0x00007fa752bea570:   00007fa550246df0 00007fa4a0314780
0x00007fa752bea580:   00007fa752bea8e8 00007fafc80c5000
0x00007fa752bea590:   00007fa752bea5c0 00007fa733940f51
0x00007fa752bea5a0:   00007fa4a0453800 00007fa4a45070f0
0x00007fa752bea5b0:   00007fa4a0453800 00007fa752bea5d0
0x00007fa752bea5c0:   00007fa752bea7c0 00007fa733986009
0x00007fa752bea5d0:   0000000000000000 0000000000000000
0x00007fa752bea5e0:   0000000000000000 0000000000000000
0x00007fa752bea5f0:   0000000000000000 0000000000000000
0x00007fa752bea600:   0000000000000000 0000000000000000
0x00007fa752bea610:   0000000000000000 0000000000000000
0x00007fa752bea620:   0000000000000000 0000000000000000
0x00007fa752bea630:   0000000000000000 0000000000000000
0x00007fa752bea640:   0000000000000000 0000000000000000
0x00007fa752bea650:   0000000000000000 0000000000000000
0x00007fa752bea660:   0000000000000000 0000000000000000
x00007fa752bea670:   00007fa7382d6128 00007fafc80c5000
0x00007fa752bea680:   0000000000000000 0000ffff00001fa0
0x00007fa752bea690:   00007fa752bea6b0 00000000cfb1b4e9
0x00007fa752bea6a0:   00007fafc80c6228 0000000000000000
0x00007fa752bea6b0:   00007fa752bea670 0000000000000000
0x00007fa752bea6c0:   0000000000000000 0000000000000000

Instructions: (pc=0x00007fa733a22a03)
0x00007fa733a229e3:   85 c0 48 89 c1 74 28 4c 8b 83 f8 00 00 00 31 ff
0x00007fa733a229f3:   31 c0 48 89 c2 48 83 c0 01 48 c1 e2 06 4c 01 c2
0x00007fa733a22a03:   48 8b 52 38 48 01 d7 48 39 c1 77 e6 48 29 fe c6
0x00007fa733a22a13:   83 90 09 00 00 00 48 8b bb f0 09 00 00 48 89 b5

Register to memory mapping:

RAX=0x0000000000000001 is an unknown value
RBX=0x00007fa4a45070f0 is an unknown value
RCX=0x0010000000000000 is an unknown value
RDX=0x3237373531353137 is an unknown value
RSP=0x00007fa752bea4d0 is pointing into the stack for thread: 0x00007fafc80c5000
RBP=0x00007fa752bea590 is pointing into the stack for thread: 0x00007fafc80c5000
RSI=0x00000000026e0918 is an unknown value
RDI=0x0000000000000000 is an unknown value
R8 =0x3237373531353137 is an unknown value
R9 =0x000000000000361c is an unknown value
R10=0x0000000000000158 is an unknown value
R11=0x0000000000000001 is an unknown value
R12=0x00007fa752bea4f0 is pointing into the stack for thread: 0x00007fafc80c5000
R13=0x0000000000000000 is an unknown value
R14=0x0000000000000001 is an unknown value
R15=0x00007fa4a4507a80 is an unknown value

Stack: [0x00007fa752aeb000,0x00007fa752bec000],  sp=0x00007fa752bea4d0,  free space=1021k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [librocksdbjni3759649349396566752.so+0x2f4a03]  rocksdb::MemTable::ApproximateMemoryUsage()+0xb3
C  [librocksdbjni3759649349396566752.so+0x212f51]  rocksdb::SuperVersion::Cleanup()+0x61
C  [librocksdbjni3759649349396566752.so+0x258009]
C  [librocksdbjni3759649349396566752.so+0x434eac]  rocksdb::Cleanable::~Cleanable()+0x1c
C  [librocksdbjni3759649349396566752.so+0x2c55e2]  rocksdb::DBIter::~DBIter()+0x4a2
C  [librocksdbjni3759649349396566752.so+0x2beab3]  rocksdb::ArenaWrappedDBIter::~ArenaWrappedDBIter()+0x23
C  [librocksdbjni3759649349396566752.so+0x2beb11]  rocksdb::ArenaWrappedDBIter::~ArenaWrappedDBIter()+0x11
J 6658  org.rocksdb.RocksIterator.disposeInternal(J)V (0 bytes) @ 0x00007fafba3d3df0 [0x00007fafba3d3d40+0xb0]
J 24055 C2 org.rocksdb.AbstractRocksIterator.disposeInternal()V (19 bytes) @ 0x00007fafbf96acd8 [0x00007fafbf96aca0+0x38]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 6658  org.rocksdb.RocksIterator.disposeInternal(J)V (0 bytes) @ 0x00007fafba3d3d78 [0x00007fafba3d3d40+0x38]
J 24055 C2 org.rocksdb.AbstractRocksIterator.disposeInternal()V (19 bytes) @ 0x00007fafbf96acd8 [0x00007fafbf96aca0+0x38]
J 7249 C2 org.rocksdb.AbstractImmutableNativeReference.close()V (17 bytes) @ 0x00007fafba5f3570 [0x00007fafba5f3520+0x50]
J 7274 C2 org.rocksdb.AbstractNativeReference.finalize()V (16 bytes) @ 0x00007fafba5f3864 [0x00007fafba5f3820+0x44]
J 7662% C2 java.lang.ref.Finalizer$FinalizerThread.run()V (55 bytes) @ 0x00007fafba6fdcc0 [0x00007fafba6fd9e0+0x2e0]
v  ~StubRoutines::call_stub

Steps to reproduce the problem 复现步骤

  1. graph clear
  2. graph loader
  3. if restart hugegraph and loader data, it will ok

Vertex/Edge summary 数据量

edge:5350235
node:3346651

Specifications of environment 环境信息

hugegraph version: 0.10.4
operating system: centos 6, 32CPUs, 180G RAM
hugegraph backend: rocksdb

Activity

  1. changed the title [-]whenloader[/-] [+]rocksdb core when graph loader[/+] on Jan 15, 2020
  2. javeme commented on Jan 15, 2020

    @javeme
    Contributor

    This is a rocksdb bug, caused by doing clear(truncate) and query at the same time, and fixed rocksdb/issues/5982.

  3. changed the title [-]rocksdb core when graph loader[/-] [+]rocksdb coredump when graph loader[/+] on Jan 15, 2020
  4. changed the title [-]rocksdb coredump when graph loader[/-] [+]rocksdb coredump when graph load[/+] on Jan 15, 2020
  5. javeme commented on Feb 24, 2023

    @javeme
    Contributor

    fixed since 0.12

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions