DWARF and unwind support for AOT assembly output.

DWARF and unwind support for AOT assembly output.

Emit CFI information to teach gdb how to unwind Dart frames. Fixes unwinding for Mac x64 and Linux x64/ARM/ARM64. Unwinding already worked for iOS ARM64. Fixes unwinding on Android 6+ ARM.

Emit DWARF information about function inlining and line number mappings. Works with gdb, addr2line and llvm-symbolizer.

With --dwarf-stack-traces, also remove CodeSourceMaps from snapshots and change stack traces to a non-compliant form resembling Android's debuggerd output, which can be passed to ndk-stack.

Issue #28785

R=fschneider@google.com, johnmccutchan@google.com, vegorov@google.com

Committed: https://github.com/dart-lang/sdk/commit/ca20aab38465ce5d5990987337c57650350af1f6

[rmacnak, 2017-03-08 22:28:03]

Description was changed from

==========
--dwarf-stack-traces
==========

to

==========
DWARF support for AOT.

When precompiling as assembly, emit CFI information to teach gdb how to unwind
Dart frames. Fixes unwinding for Mac x64 and Linux x64/ARM/ARM64. Unwinding
already worked for iOS ARM64. Unwinding still broken on Android.

With --dwarf-stack-traces, also emit DWARF information about function inlining
and line number mappings. Works with gdb, addr2line and llvm-symbolizer. This
flag also removes CodeSourceMaps from snapshots and changes stack traces to a
non-compilant form resembling Android's debuggerd output.
==========

[rmacnak, 2017-03-09 19:16:39]

Description was changed from

==========
DWARF support for AOT.

When precompiling as assembly, emit CFI information to teach gdb how to unwind
Dart frames. Fixes unwinding for Mac x64 and Linux x64/ARM/ARM64. Unwinding
already worked for iOS ARM64. Unwinding still broken on Android.

With --dwarf-stack-traces, also emit DWARF information about function inlining
and line number mappings. Works with gdb, addr2line and llvm-symbolizer. This
flag also removes CodeSourceMaps from snapshots and changes stack traces to a
non-compilant form resembling Android's debuggerd output.
==========

to

==========
DWARF and unwind support for AOT assembly output.

Emit CFI information to teach gdb how to unwind Dart frames. Fixes unwinding for
Mac x64 and Linux x64/ARM/ARM64. Unwinding already worked for iOS ARM64.
Unwinding still broken on Android.

Emit DWARF information about function inlining and line number mappings. Works
with gdb, addr2line and llvm-symbolizer.

With --dwarf-stack-traces, also remove CodeSourceMaps from snapshots and changes
stack traces to a non-compilant form resembling Android's debuggerd output.
==========

[rmacnak, 2017-03-09 20:56:56]

Description was changed from

==========
DWARF and unwind support for AOT assembly output.

Emit CFI information to teach gdb how to unwind Dart frames. Fixes unwinding for
Mac x64 and Linux x64/ARM/ARM64. Unwinding already worked for iOS ARM64.
Unwinding still broken on Android.

Emit DWARF information about function inlining and line number mappings. Works
with gdb, addr2line and llvm-symbolizer.

With --dwarf-stack-traces, also remove CodeSourceMaps from snapshots and changes
stack traces to a non-compilant form resembling Android's debuggerd output.
==========

to

==========
DWARF and unwind support for AOT assembly output.

Emit CFI information to teach gdb how to unwind Dart frames. Fixes unwinding for
Mac x64 and Linux x64/ARM/ARM64. Unwinding already worked for iOS ARM64.

Emit DWARF information about function inlining and line number mappings. Works
with gdb, addr2line and llvm-symbolizer.

With --dwarf-stack-traces, also remove CodeSourceMaps from snapshots and changes
stack traces to a non-compilant form resembling Android's debuggerd output.

BUG=
==========

[rmacnak, 2017-03-14 21:16:21]

Description was changed from

==========
DWARF and unwind support for AOT assembly output.

Emit CFI information to teach gdb how to unwind Dart frames. Fixes unwinding for
Mac x64 and Linux x64/ARM/ARM64. Unwinding already worked for iOS ARM64.

Emit DWARF information about function inlining and line number mappings. Works
with gdb, addr2line and llvm-symbolizer.

With --dwarf-stack-traces, also remove CodeSourceMaps from snapshots and changes
stack traces to a non-compilant form resembling Android's debuggerd output.

BUG=
==========

to

==========
DWARF and unwind support for AOT assembly output.

Emit CFI information to teach gdb how to unwind Dart frames. Fixes unwinding for
Mac x64 and Linux x64/ARM/ARM64. Unwinding already worked for iOS ARM64. Fixes
unwinding on Android 6+ ARM.

Emit DWARF information about function inlining and line number mappings. Works
with gdb, addr2line and llvm-symbolizer.

With --dwarf-stack-traces, also remove CodeSourceMaps from snapshots and changes
stack traces to a non-compilant form resembling Android's debuggerd output.

BUG=
==========

[rmacnak, 2017-03-15 00:51:51]

Description was changed from

==========
DWARF and unwind support for AOT assembly output.

Emit CFI information to teach gdb how to unwind Dart frames. Fixes unwinding for
Mac x64 and Linux x64/ARM/ARM64. Unwinding already worked for iOS ARM64. Fixes
unwinding on Android 6+ ARM.

Emit DWARF information about function inlining and line number mappings. Works
with gdb, addr2line and llvm-symbolizer.

With --dwarf-stack-traces, also remove CodeSourceMaps from snapshots and changes
stack traces to a non-compilant form resembling Android's debuggerd output.

BUG=
==========

to

==========
DWARF and unwind support for AOT assembly output.

Emit CFI information to teach gdb how to unwind Dart frames. Fixes unwinding for
Mac x64 and Linux x64/ARM/ARM64. Unwinding already worked for iOS ARM64. Fixes
unwinding on Android 6+ ARM.

Emit DWARF information about function inlining and line number mappings. Works
with gdb, addr2line and llvm-symbolizer.

With --dwarf-stack-traces, also remove CodeSourceMaps from snapshots and change
stack traces to a non-compliant form resembling Android's debuggerd output,
which can be passed to ndk-stack.

Issue #28785
==========

[rmacnak, 2017-03-15 00:56:48]

rmacnak@google.com changed reviewers:
+ fschneider@google.com, johnmccutchan@google.com, vegorov@google.com

[rmacnak, 2017-03-15 00:56:49]

On Flutter iOS, using --dwarf-stack-traces shrinks a stripped app.dylib by

11222104 -> 10337376 (-7.8%)


On Android 5, debuggerd only gives the top two frames.
On Android 6 and 7, debuggerd unwinds through all Dart frames, but gets stuck at
DartEntry::InvokeFunction.


Stack traces with --dwarf-stack-traces looks like

E/flutter (24138): exception message
E/flutter (24138): *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
***
E/flutter (24138): pid: 24138, tid: 24165, name Dart_Initialize
E/flutter (24138):     #00 pc 005a16b7  vm_snapshot_instr
E/flutter (24138):     #01 pc 005a160b  vm_snapshot_instr
E/flutter (24138):     #02 pc 000afeb7  vm_snapshot_instr
E/flutter (24138):     #03 pc 000afe47  vm_snapshot_instr
E/flutter (24138):     #04 pc 00657c1b  vm_snapshot_instr
E/flutter (24138):     #05 pc 000c817f  vm_snapshot_instr
E/flutter (24138):     #06 pc 00657e53  vm_snapshot_instr
E/flutter (24138):     #07 pc 000c30f3  vm_snapshot_instr

which ndk-stack symbolizes as

********** Crash dump: **********
pid: 24138, tid: 24165, name Dart_Initialize
Stack frame #00 pc 005a16b7  vm_snapshot_instr: Routine baz at
file:///usr/local/google/home/rmacnak/Flutter/flutter/examples/flutter_gallery/lib/main.dart:1
Stack frame #01 pc 005a160b  vm_snapshot_instr: Routine foo at
file:///usr/local/google/home/rmacnak/Flutter/flutter/examples/flutter_gallery/lib/main.dart:20
Stack frame #02 pc 000afeb7  vm_snapshot_instr: Routine main at
file:///usr/local/google/home/rmacnak/Flutter/flutter/examples/flutter_gallery/lib/main.dart:24
Stack frame #03 pc 000afe47  vm_snapshot_instr: Routine main at
file:///usr/local/google/home/rmacnak/Flutter/flutter/examples/flutter_gallery/lib/main.dart:1
Stack frame #04 pc 00657c1b  vm_snapshot_instr: Routine _Closure.call at
dart:core-patch/function.dart:1
Stack frame #05 pc 000c817f  vm_snapshot_instr: Routine _startIsolate.<anonymous
closure> at dart:isolate-patch/isolate_patch.dart:1
Stack frame #06 pc 00657e53  vm_snapshot_instr: Routine _Closure.call at
dart:core-patch/function.dart:1

[Cutch, 2017-03-15 14:54:13]

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc
File runtime/vm/dwarf.cc (right):

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:152: Print(".uleb128 %" Pd "\n", kCompilationUnit);
Might be a good idea to document what this all means

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:235: // Compilation Unit DIE
Maybe add a comment that we only have a single compilation unit for the entire
Dart program.

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:237: Print(".string \"hello.dart\"\n");  // DW_AT_name
shouldn't this be the script name?

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:241: // DW_AT_low_pc
This is encoding the lowest PC that this compilation unit is at?

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:251: #if defined(ARCH_IS_32_BIT)
More comments:

This is encoding the highest PC that this compilation unit is at?

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.h
File runtime/vm/dwarf.h (right):

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.h#new...
runtime/vm/dwarf.h:60: static inline intptr_t Hashcode(Key key) { return
key->token_pos().value(); }
maybe use a better hash code?

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/flags.cc
File runtime/vm/flags.cc (right):

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/flags.cc#ne...
runtime/vm/flags.cc:448: "Warning: This VM has been configured to produce stack
traces "
Is this necessary?

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/object.cc
File runtime/vm/object.cc (right):

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/object.cc#n...
runtime/vm/object.cc:22604: uword call_addr = return_addr - 1;
This function is getting too complex. Refactor this into a function for printing
stack traces in the Android format.

[Florian Schneider, 2017-03-15 18:15:06]

Lgtm

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc
File runtime/vm/dwarf.cc (right):

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:377: int32_t func = stream.Read<int32_t>();
s/func/func_index/

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:584: // This varient is relocatable.
s/varient/variant/

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.h
File runtime/vm/dwarf.h (right):

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.h#new...
runtime/vm/dwarf.h:89: static inline intptr_t Hashcode(Key key) { return
key->PayloadStart(); }
Maybe add a comment that code object don't move.

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.h#new...
runtime/vm/dwarf.h:95: CodeIndexPair(const Code* f, intptr_t index) : code_(f),
index_(index) {}
Code* c

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/snapshot.cc
File runtime/vm/snapshot.cc (right):

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/snapshot.cc...
runtime/vm/snapshot.cc:913: dwarf_->Write();
Maybe move this outside of WriteText instead of having it conditionally here.

[Vyacheslav Egorov (Google), 2017-03-15 21:36:43]

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc
File runtime/vm/dwarf.cc (right):

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:153: Print(".uleb128 %" Pd "\n", DW_TAG_compile_unit);
Maybe you can make helpers like:

u1(intptr_t x)
u4(intptr_t x)
u8(intptr_t x)
uleb128(intptr_t x)

that encapsulate printing.

Also Abbreviations can be written like this:

WriteAbbrev(kCompilationUnit, 
                      { DW_AT_name,  DW_FORM_string, 
                        DW_AT_producer, DW_FORM_string});

Which could be a macro:

#define WriteAbbrev(id, attributes) do {
  intptr_t seq[] = attributes;
  uleb128(id); u1(DW_CHILDREN_yes);
  for (intptr_t i = 0; i < ARRAYSIZE(seq); i++) uleb128(seq[i]); uleb128(0);
uleb128(0);
} while (0);

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:252: Print(".4byte .Lcode%" Pd " + %" Pd "\n",
last_code_index, last_code.Size());
Maybe define PTRSZ and do 

Print( PTR_SZ " .Lcode...")

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:584: // This varient is relocatable.
variant

[rmacnak, 2017-03-16 17:16:43]

Patchset #12 (id:220001) has been deleted

[rmacnak, 2017-03-16 17:46:06]

Patchset #12 (id:230001) has been deleted

[rmacnak, 2017-03-16 17:50:11]

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc
File runtime/vm/dwarf.cc (right):

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:152: Print(".uleb128 %" Pd "\n", kCompilationUnit);
On 2017/03/15 14:54:13, Cutch wrote:
> Might be a good idea to document what this all means

Done.

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:153: Print(".uleb128 %" Pd "\n", DW_TAG_compile_unit);
On 2017/03/15 21:36:43, Vyacheslav Egorov (Google) wrote:
> Maybe you can make helpers like:
> 
> u1(intptr_t x)
> u4(intptr_t x)
> u8(intptr_t x)
> uleb128(intptr_t x)
> 
> that encapsulate printing.


Done.

> Also Abbreviations can be written like this:
> 
> WriteAbbrev(kCompilationUnit, 
>                       { DW_AT_name,  DW_FORM_string, 
>                         DW_AT_producer, DW_FORM_string});
> 
> Which could be a macro:
> 
> #define WriteAbbrev(id, attributes) do {
>   intptr_t seq[] = attributes;
>   uleb128(id); u1(DW_CHILDREN_yes);
>   for (intptr_t i = 0; i < ARRAYSIZE(seq); i++) uleb128(seq[i]); uleb128(0);
> uleb128(0);
> } while (0);

Tried this but I couldn't get the literal array to play nice with the macro, and
I had to locally disable clang-format to make it readable. I don't think it's
worth it.

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:235: // Compilation Unit DIE
On 2017/03/15 14:54:13, Cutch wrote:
> Maybe add a comment that we only have a single compilation unit for the entire
> Dart program.

Done.

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:237: Print(".string \"hello.dart\"\n");  // DW_AT_name
On 2017/03/15 14:54:13, Cutch wrote:
> shouldn't this be the script name?

Yes, done.

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:241: // DW_AT_low_pc
On 2017/03/15 14:54:13, Cutch wrote:
> This is encoding the lowest PC that this compilation unit is at?

Done.

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:251: #if defined(ARCH_IS_32_BIT)
On 2017/03/15 14:54:13, Cutch wrote:
> More comments:
> 
> This is encoding the highest PC that this compilation unit is at?

Done.

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:252: Print(".4byte .Lcode%" Pd " + %" Pd "\n",
last_code_index, last_code.Size());
On 2017/03/15 21:36:43, Vyacheslav Egorov (Google) wrote:
> Maybe define PTRSZ and do 
> 
> Print( PTR_SZ " .Lcode...")
> 

Done as FORM_ADDR

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:377: int32_t func = stream.Read<int32_t>();
On 2017/03/15 18:15:06, Florian Schneider wrote:
> s/func/func_index/

Done.

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.cc#ne...
runtime/vm/dwarf.cc:584: // This varient is relocatable.
On 2017/03/15 18:15:06, Florian Schneider wrote:
> s/varient/variant/

Done.

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.h
File runtime/vm/dwarf.h (right):

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.h#new...
runtime/vm/dwarf.h:89: static inline intptr_t Hashcode(Key key) { return
key->PayloadStart(); }
On 2017/03/15 18:15:06, Florian Schneider wrote:
> Maybe add a comment that code object don't move.

Done.

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/dwarf.h#new...
runtime/vm/dwarf.h:95: CodeIndexPair(const Code* f, intptr_t index) : code_(f),
index_(index) {}
On 2017/03/15 18:15:06, Florian Schneider wrote:
> Code* c

Done.

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/flags.cc
File runtime/vm/flags.cc (right):

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/flags.cc#ne...
runtime/vm/flags.cc:448: "Warning: This VM has been configured to produce stack
traces "
On 2017/03/15 14:54:13, Cutch wrote:
> Is this necessary?

Maybe in the stack trace instead, but we should be clear the VM isn't running
Dart with this flag.

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/object.cc
File runtime/vm/object.cc (right):

https://codereview.chromium.org/2723213002/diff/200001/runtime/vm/object.cc#n...
runtime/vm/object.cc:22604: uword call_addr = return_addr - 1;
On 2017/03/15 14:54:13, Cutch wrote:
> This function is getting too complex. Refactor this into a function for
printing
> stack traces in the Android format.

Split into Dart and Android print functions. Also removed max_frames, which was
always kMaxInt32.

[Cutch, 2017-03-16 17:54:46]

lgtm

[Vyacheslav Egorov (Google), 2017-03-16 18:20:02]

lgtm

[rmacnak, 2017-03-16 20:09:00]

Description was changed from

==========
DWARF and unwind support for AOT assembly output.

Emit CFI information to teach gdb how to unwind Dart frames. Fixes unwinding for
Mac x64 and Linux x64/ARM/ARM64. Unwinding already worked for iOS ARM64. Fixes
unwinding on Android 6+ ARM.

Emit DWARF information about function inlining and line number mappings. Works
with gdb, addr2line and llvm-symbolizer.

With --dwarf-stack-traces, also remove CodeSourceMaps from snapshots and change
stack traces to a non-compliant form resembling Android's debuggerd output,
which can be passed to ndk-stack.

Issue #28785
==========

to

==========
DWARF and unwind support for AOT assembly output.

Emit CFI information to teach gdb how to unwind Dart frames. Fixes unwinding for
Mac x64 and Linux x64/ARM/ARM64. Unwinding already worked for iOS ARM64. Fixes
unwinding on Android 6+ ARM.

Emit DWARF information about function inlining and line number mappings. Works
with gdb, addr2line and llvm-symbolizer.

With --dwarf-stack-traces, also remove CodeSourceMaps from snapshots and change
stack traces to a non-compliant form resembling Android's debuggerd output,
which can be passed to ndk-stack.

Issue #28785

R=fschneider@google.com, johnmccutchan@google.com, vegorov@google.com

Committed:
https://github.com/dart-lang/sdk/commit/ca20aab38465ce5d5990987337c57650350af1f6
==========

[rmacnak, 2017-03-16 20:09:01]

Committed patchset #13 (id:260001) manually as
ca20aab38465ce5d5990987337c57650350af1f6 (presubmit successful).