Repository navigation
configuration option --gdb not working properly #2076
Description
Activity
Running the same script with current master I get the following stack trace:
$ gdb --args ./iojs --gdbjit --gdbjit-full /tmp/crash.js ... (gdb) bt #0 0x00007ffff6bed267 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55 #1 0x00007ffff6beeeca in __GI_abort () at abort.c:89 #2 0x0000000000c0e304 in node::OnFatalError(char const*, char const*) () #3 0x00000000007a9b12 in v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) () #4 0x00000000008f1490 in v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) () #5 0x0000000000afa62e in v8::internal::Runtime_AllocateInTargetSpace(int, v8::internal::Object**, v8::internal::Isolate*) () #6 0x00001f5e12f062b5 in ?? () #7 0x3030393334382e30 in ?? () #8 0x1c7b3b0900003a91 in ?? () #9 0x7b3400000032550e in ?? ()So even though the breakpoint is hit on master, it doesn't give us any information. Though d8 on the head of 4.2 branch gives:
$ gdb --args ./out/x64.release/d8 --gdbjit --gdbjit-full /tmp/crash.js ... (gdb) bt #0 0x000000000086c16f in v8::base::OS::Abort() () #1 0x000000000040f931 in v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) () #2 0x00000000004b41ed in v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) () #3 0x00000000007d623b in v8::internal::Runtime_AllocateInTargetSpace(int, v8::internal::Object**, v8::internal::Isolate*) () #4 0x00000eb7bf807d95 in Stub:CEntryStub () #5 0x00000eb7bf90ac53 in Stub:StringAddStub_Tenured () #6 0x00000eb7bf9077d2 in zzz () at /tmp/crash.js:1 #7 0x00000eb7bf858cae in yyy () at /tmp/crash.js:3 #8 0x00000eb7bf858dae in xxx () at /tmp/crash.js:2 #9 0x00000eb7bf858861 in /tmp/crash.js () at /tmp/crash.js:15Not sure how this could be happening.
@bnoordhuis Sorry to pull you in on this one, but cannot figure out how a build of iojs with
--gdbwould show no output, but a build of same patch version of d8 with it enabled would show the JS output. Any thoughts where I could dig to find the answer?EDIT: Interesting note that d8 shows similar output to iojs if the
--gdbjit-fullis removed. Wonder if additional information isn't being emitted to gdb.@trevnorris I'll try to reproduce. In the meanwhile there is another trick you can try. After you have hit the out-of-memory abort, run this in gdb:
(gdb) source $path_to_node/deps/v8/tools/gdbinit (gdb) jstThis should give you the javascript stacktrace (with a lot more verbose information). Note that you don't really need to reconfigure with
--gdbfor this to work.@trevnorris Do you know the last time --gdbjit and --gdbjit-full worked for you? I can go all the way back to 1.0.0 and it still doesn't work. Whatever is broken, hasn't broken recently.
Looking at the V8 code in gdb-jit.cc, I don't see anything obviously wrong, and as you mention d8 seems to work okay. Also, I have verified, using
--gdbjit-dumpthat the object files being produced by gdb-jit.cc have decent symbols.My suspicion is that there is something different about memory layout/map for the node process that confuses gdb somehow.
Removed the build tag as the
--gdbconfigure option does correctly enable gdb support in V8.I'm not 100% sure but I believe gdbjit was broken until 4.3. The commit that fixes it (if I'm right) is v8/v8@652061e and is in next.
Maybe worth installing gdbinit to $PREFIX/share/io.js, documenting it's use? It's not impossible to crash iojs / nodejs so this seems useful, especially if it can be used without recompiling iojs.
@coreyfarrell 👍. It is much easier to use gdbinit than recompiling with
--gdb. Jit support in gdb happens to be buggy and slow anyway. PRs are welcome, or I will try to do it once I get a chance.@bnoordhuis gdbjit still doesn't seem to work for me with node in next, or any release of io.js (that I tried) going back to 1.0. Note that it does work with d8 in atleast 4.2. I think this is something node-specific, but I am not entirely sure what yet.
@ofrobots The issue with current
masterandnextare different. InmasterI can hit the__jit_debug_register_codebreakpoint, but nobtoutput. Innextbranch the breakpoint is never reached.node v0.11.9 would show the first JS point in the backtrace output with
gdb. Back at that time I would uselldbbecause the output was more comprehensive. Thenlldbfor linux x64 got borked and I stopped using the feature.Recently gave it a shot against d8 and found that it mostly worked. Though both options must be passed. But even with both options passed to io.js the end of the stack looks like so:
#8 0x000027ddf1006001 in ?? () #9 0x00007fffffffd530 in ?? () #10 0x00007fffffffd598 in ?? () #11 0x000027ddf10e864e in zzz () Backtrace stopped: previous frame inner to this frame (corrupt stack?)I think this is a gdb-related issue. This is how far I got:
V8 is reporting the code-compile events to gdb by calling
__jit_debug_register_code. If you enable internal gdb debugging using theset debug jit 1command, you can see that gdb acknowledges the receipt of these symbols. However, for some reason it doesn't use them during the backtrace. I am not sure why this works with d8 but not node.lemongrab :: ~/src/io.js ‹master› % gdb --args ./iojs --gdbjit --gdbjit-full /tmp/crash.js ... (gdb) set debug jit 10 (gdb) r Starting program: /ofrobots/src/io.js/out/Release/iojs --gdbjit --gdbjit-full /tmp/crash.js jit_breakpoint_re_set_internal, breakpoint_addr = 0x9b9a30 jit_breakpoint_re_set_internal, breakpoint_addr = 0x9b9a30 warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff7ffa000 jit_breakpoint_re_set_internal, breakpoint_addr = 0x9b9a30 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". jit_breakpoint_re_set_internal, breakpoint_addr = 0x9b9a30 [New Thread 0x7ffff6bca700 (LWP 7994)] [New Thread 0x7ffff63c9700 (LWP 7995)] [New Thread 0x7ffff5bc8700 (LWP 7996)] [New Thread 0x7ffff53c7700 (LWP 7997)] [New Thread 0x7ffff4bc6700 (LWP 7998)] jit_read_descriptor, descriptor_addr = 0x1343cc0 jit_register_code, symfile_addr = 0x13a2fe0, symfile_size = 688 jit_register_code, symfile_addr = 0x13a2fe0, symfile_size = 688 jit_breakpoint_re_set_internal, breakpoint_addr = 0x9b9a30 jit_read_descriptor, descriptor_addr = 0x1343cc0 jit_register_code, symfile_addr = 0x139e490, symfile_size = 699 jit_register_code, symfile_addr = 0x139e490, symfile_size = 699 jit_breakpoint_re_set_internal, breakpoint_addr = 0x9b9a30 jit_read_descriptor, descriptor_addr = 0x1343cc0 ...The above was with io.js @ master. To compare I used a d8 from branch-heads/4.2.
Is this still not working correctly?
3 remaining items
As for
gdbjit, IIRC it was useful to be able to step through the JS source in gdb.gdbinitdefinitely gives more information, but being able to step through the JS itself in gdb was also useful.Okay, that is a reasonable use-case. I don't have bandwidth at the moment, but I will try to take another look when I get some spare cycles. Others are welcome to investigate this too.
Self-assigning so that I don't forget, but others are welcome to investigate it too.
Thanks. Seems it has to be something like build configuration. Since it works perfectly in d8. I'll also try to take a peek there.
Any chance someone knows if this is resolved or still an issue?
@Trott This unfortunately is still an issue. Building d8@4.9.385.27 (same as current master) and running the simple script:
performance.now();
and executing with:
> lldb-3.6 -- ./out/x64.release/d8 --gdbjit --gdbjit-full script.jswe can see a significant amount of information. For example:
* frame #0: 0x0000000000406300 d8`v8::Shell::PerformanceNow(v8::FunctionCallbackInfo<v8::Value> const&) frame #1: 0x0000000000838719 d8`v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) + 153 frame #2: 0x00000000004613d0 d8`v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::(anonymous namespace)::BuiltinArguments<(v8::internal::BuiltinExtraArguments)1>) + 960 frame #3: 0x000000000046f654 d8`v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) + 52 frame #4: 0x0000118c6f60c17b JIT(0xbc3f30)`Stub:CEntryStub + 91 frame #5: 0x0000118c6f63cf93 JIT(0xc31050)`Script:script.js + 115 at script.js:3 frame #6: 0x0000118c6f636904 JIT(0xc1feb0)`Builtin:JSEntryTrampoline + 132 (lldb) f 5 frame #5: 0x0000118c6f63cf93 JIT(0xc31050)`Script:script.js + 115 at script.js:1 -> 1 performance.now(); (lldb) di -f 0x118c6f63cf20 <Script:script.js>: pushq %rbp 0x118c6f63cf21 <Script:script.js+1>: movq %rsp, %rbp 0x118c6f63cf24 <Script:script.js+4>: pushq %rsi 0x118c6f63cf25 <Script:script.js+5>: pushq %rdi 0x118c6f63cf26 <Script:script.js+6>: pushq -0x58(%r13) 0x118c6f63cf2a <Script:script.js+10>: cmpq 0xa78(%r13), %rsp 0x118c6f63cf31 <Script:script.js+17>: jae 0x118c6f63cf38 ; Script:script.js + 24 at script.js:1 0x118c6f63cf33 <Script:script.js+19>: callq 0x118c6f6380a0 ; Builtin:StackCheck ...This is excellent we can walk through the JS on a release build.
Unfortunately on master building with
./configure --gdb --debugand addingCXXFLAGS="-g"we still can't get any information even running the debug build. I've attempted to fix this several times, since this functionality could greatly improve certain debugging cases, but haven't figured out what's going on. I'd love if someone else could pick this up and figure out what's wrong.- added a commit that references this issue
on Mar 23, 2016 @bnoordhuis Awesome. Thanks much for taking care of this.
- added 2 commits that reference this issue
on Mar 30, 2016 - added a commit that references this issue
on Apr 8, 2016
After running
$ ./configure --gdb; makeit should be possible to do this from gdb:And it should break like so:
But it doesn't. The above was captured from d8 setting
ENABLE_GDB_JIT_INTERFACE.I'll look into this later. Just filing a proper issue.
EDIT: Missing breakpoint is only exhibited on current
nextbranch.