|
|
DescriptionBuild addr2line-pdb from tcmalloc for use in memory-infra symbolization
The memory-infra tools were first written on linux where the
symbolization was done using addr2line. Turns out tcmalloc has some old
code (with a few small bugs) that wraps the windows dbghelp.dll
symbolization APIs in an addr2line-like interface called addr2line-pdb.
This CL
* fixes the small format string bugs
* makes the windows library configuration #defines more friendly
for chromium build
* modifies addr2line to expect addresses relative to DllBase to better
simulate how addr2line works with modules in linux.
The last point is important. Windows DLL have a concept of "default load
address" which hints to the OS where to load the binary image after
relocation. The dbghelp.dll symbolization library will load the module
at this location in the virtual address space meaning the caller of
these functions would need to be aware of the base address. This makes
things unnecessarily complex in the face of ASLR and also diverges from
the behavior of addr2line when used with linux-style DSOs. This CL
simply adds the module base address to the incoming addresses thereby
making the input relative addresses for the module which both is easier
to use and lines up better with linux's addr2line behavior.
BUG=694792
Review-Url: https://codereview.chromium.org/2730473002
Cr-Commit-Position: refs/heads/master@{#457271}
Committed: https://chromium.googlesource.com/chromium/src/+/bf193279edce86f900ae5b222dc93ea427ae1ff0
Patch Set 1 #Patch Set 2 : make it all work #
Total comments: 4
Patch Set 3 : address comments #Patch Set 4 : reformated #
Total comments: 3
Patch Set 5 : move GN target #Patch Set 6 : Move target correctly #
Messages
Total messages: 46 (25 generated)
Description was changed from ========== Fix format string errors. Add addr2line-pdb to build. BUG= ========== to ========== The memory-infra tools were first written on linux where the symbolization was done using addr2line. Turns out tcmalloc has some old code (with a few small bugs) that wraps the windows dbghelp.dll symbolization APIs in an addr2line-like interface called addr2line-pdb. This CL * fixes the small format string bugs * makes the windows library configuration #defines more friendly for chromium build * modifies addr2line to expect addresses relative to DllBase to better simulate how addr2line works with modules in linux. The last point is important. Windows DSO have a concept of "default load address" which hints to the OS where to load the binary image after relocation. The dbghelp.dll symbolization library will load the module at this location in the virtual address space meaning the caller of these functions would need to be aware of the base address. This makes things unnecessarily complex in the face of ASLR and also diverges from the behavior of addr2line when used with linux DSO. This CL simply adds the module base address to the incoming addresses thereby making the input relative addresses for the module which both is easier to use and lines up better with linux's addr2line behavior. BUG= ==========
Description was changed from ========== The memory-infra tools were first written on linux where the symbolization was done using addr2line. Turns out tcmalloc has some old code (with a few small bugs) that wraps the windows dbghelp.dll symbolization APIs in an addr2line-like interface called addr2line-pdb. This CL * fixes the small format string bugs * makes the windows library configuration #defines more friendly for chromium build * modifies addr2line to expect addresses relative to DllBase to better simulate how addr2line works with modules in linux. The last point is important. Windows DSO have a concept of "default load address" which hints to the OS where to load the binary image after relocation. The dbghelp.dll symbolization library will load the module at this location in the virtual address space meaning the caller of these functions would need to be aware of the base address. This makes things unnecessarily complex in the face of ASLR and also diverges from the behavior of addr2line when used with linux DSO. This CL simply adds the module base address to the incoming addresses thereby making the input relative addresses for the module which both is easier to use and lines up better with linux's addr2line behavior. BUG= ========== to ========== The memory-infra tools were first written on linux where the symbolization was done using addr2line. Turns out tcmalloc has some old code (with a few small bugs) that wraps the windows dbghelp.dll symbolization APIs in an addr2line-like interface called addr2line-pdb. This CL * fixes the small format string bugs * makes the windows library configuration #defines more friendly for chromium build * modifies addr2line to expect addresses relative to DllBase to better simulate how addr2line works with modules in linux. The last point is important. Windows DSO have a concept of "default load address" which hints to the OS where to load the binary image after relocation. The dbghelp.dll symbolization library will load the module at this location in the virtual address space meaning the caller of these functions would need to be aware of the base address. This makes things unnecessarily complex in the face of ASLR and also diverges from the behavior of addr2line when used with linux DSO. This CL simply adds the module base address to the incoming addresses thereby making the input relative addresses for the module which both is easier to use and lines up better with linux's addr2line behavior. BUG=694792 ==========
The CQ bit was checked by ajwong@google.com to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
ajwong@chromium.org changed reviewers: + ajwong@chromium.org, erikchen@chromium.org
Thanks, lgtm with nits. Please wrap your CL description to 80 columns. I wasn't familiar with the acronym DSO - it looks like you're referring to dynamic libraries? Searching for "windows dso" gives unrelated results. https://codereview.chromium.org/2730473002/diff/20001/BUILD.gn File BUILD.gn (right): https://codereview.chromium.org/2730473002/diff/20001/BUILD.gn#newcode676 BUILD.gn:676: "//third_party/tcmalloc:addr2line-pdb", gyp is deprecated, maybe move this to gn_only? https://codereview.chromium.org/2730473002/diff/20001/third_party/tcmalloc/BU... File third_party/tcmalloc/BUILD.gn (right): https://codereview.chromium.org/2730473002/diff/20001/third_party/tcmalloc/BU... third_party/tcmalloc/BUILD.gn:7: executable("addr2line-pdb") { Code search seems to show that most other third_parties use relative paths and don't require tcmaloc_dir. https://cs.chromium.org/search/?q=include_dir+file:third_party.*gn&type=cs https://cs.chromium.org/search/?q=executable+file:third_party.*gn+package:%5E... Can we try to match their style? If that doesn't work, it looks like v8 has a slighty less fragile way of getting the current path: https://cs.chromium.org/chromium/src/v8/gni/v8.gni?type=cs&q=no_chromium_code...
Description was changed from ========== The memory-infra tools were first written on linux where the symbolization was done using addr2line. Turns out tcmalloc has some old code (with a few small bugs) that wraps the windows dbghelp.dll symbolization APIs in an addr2line-like interface called addr2line-pdb. This CL * fixes the small format string bugs * makes the windows library configuration #defines more friendly for chromium build * modifies addr2line to expect addresses relative to DllBase to better simulate how addr2line works with modules in linux. The last point is important. Windows DSO have a concept of "default load address" which hints to the OS where to load the binary image after relocation. The dbghelp.dll symbolization library will load the module at this location in the virtual address space meaning the caller of these functions would need to be aware of the base address. This makes things unnecessarily complex in the face of ASLR and also diverges from the behavior of addr2line when used with linux DSO. This CL simply adds the module base address to the incoming addresses thereby making the input relative addresses for the module which both is easier to use and lines up better with linux's addr2line behavior. BUG=694792 ========== to ========== The memory-infra tools were first written on linux where the symbolization was done using addr2line. Turns out tcmalloc has some old code (with a few small bugs) that wraps the windows dbghelp.dll symbolization APIs in an addr2line-like interface called addr2line-pdb. This CL * fixes the small format string bugs * makes the windows library configuration #defines more friendly for chromium build * modifies addr2line to expect addresses relative to DllBase to better simulate how addr2line works with modules in linux. The last point is important. Windows DSO have a concept of "default load address" which hints to the OS where to load the binary image after relocation. The dbghelp.dll symbolization library will load the module at this location in the virtual address space meaning the caller of these functions would need to be aware of the base address. This makes things unnecessarily complex in the face of ASLR and also diverges from the behavior of addr2line when used with linux DSO. This CL simply adds the module base address to the incoming addresses thereby making the input relative addresses for the module which both is easier to use and lines up better with linux's addr2line behavior. BUG=694792 ==========
comment wrapped. https://codereview.chromium.org/2730473002/diff/20001/BUILD.gn File BUILD.gn (right): https://codereview.chromium.org/2730473002/diff/20001/BUILD.gn#newcode676 BUILD.gn:676: "//third_party/tcmalloc:addr2line-pdb", On 2017/03/01 22:42:51, erikchen wrote: > gyp is deprecated, maybe move this to gn_only? Done. https://codereview.chromium.org/2730473002/diff/20001/third_party/tcmalloc/BU... File third_party/tcmalloc/BUILD.gn (right): https://codereview.chromium.org/2730473002/diff/20001/third_party/tcmalloc/BU... third_party/tcmalloc/BUILD.gn:7: executable("addr2line-pdb") { On 2017/03/01 22:42:51, erikchen wrote: > Code search seems to show that most other third_parties use relative paths and > don't require tcmaloc_dir. > > https://cs.chromium.org/search/?q=include_dir+file:third_party.*gn&type=cs > https://cs.chromium.org/search/?q=executable+file:third_party.*gn+package:%5E... > > Can we try to match their style? > > If that doesn't work, it looks like v8 has a slighty less fragile way of getting > the current path: > https://cs.chromium.org/chromium/src/v8/gni/v8.gni?type=cs&q=no_chromium_code... Done. Infact, reduced it even further. FYI, original code was copied of the tcmalloc target which seems to be following some outdated conventions perhaps?
Description was changed from ========== The memory-infra tools were first written on linux where the symbolization was done using addr2line. Turns out tcmalloc has some old code (with a few small bugs) that wraps the windows dbghelp.dll symbolization APIs in an addr2line-like interface called addr2line-pdb. This CL * fixes the small format string bugs * makes the windows library configuration #defines more friendly for chromium build * modifies addr2line to expect addresses relative to DllBase to better simulate how addr2line works with modules in linux. The last point is important. Windows DSO have a concept of "default load address" which hints to the OS where to load the binary image after relocation. The dbghelp.dll symbolization library will load the module at this location in the virtual address space meaning the caller of these functions would need to be aware of the base address. This makes things unnecessarily complex in the face of ASLR and also diverges from the behavior of addr2line when used with linux DSO. This CL simply adds the module base address to the incoming addresses thereby making the input relative addresses for the module which both is easier to use and lines up better with linux's addr2line behavior. BUG=694792 ========== to ========== The memory-infra tools were first written on linux where the symbolization was done using addr2line. Turns out tcmalloc has some old code (with a few small bugs) that wraps the windows dbghelp.dll symbolization APIs in an addr2line-like interface called addr2line-pdb. This CL * fixes the small format string bugs * makes the windows library configuration #defines more friendly for chromium build * modifies addr2line to expect addresses relative to DllBase to better simulate how addr2line works with modules in linux. The last point is important. Windows DLL have a concept of "default load address" which hints to the OS where to load the binary image after relocation. The dbghelp.dll symbolization library will load the module at this location in the virtual address space meaning the caller of these functions would need to be aware of the base address. This makes things unnecessarily complex in the face of ASLR and also diverges from the behavior of addr2line when used with linux-style DSOs. This CL simply adds the module base address to the incoming addresses thereby making the input relative addresses for the module which both is easier to use and lines up better with linux's addr2line behavior. BUG=694792 ==========
The CQ bit was checked by ajwong@google.com to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: This issue passed the CQ dry run.
The CQ bit was checked by ajwong@chromium.org
The patchset sent to the CQ was uploaded after l-g-t-m from erikchen@chromium.org Link to the patchset: https://codereview.chromium.org/2730473002/#ps60001 (title: "reformated")
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: chromium_presubmit on master.tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)
ajwong@chromium.org changed reviewers: + wfh@chromium.org
Heya will...i can has an OWNERS review?
Heya will...i can has an OWNERS review?
On 2017/03/02 22:52:32, awong wrote: > Heya will...i can has an OWNERS review? hmm - can't you use breakpad .sym files here (the output of running dumpsyms) and/or some part of existing crashpad/breakpad symbolization? Where will this tool actually be running? Maybe you can use code in pdb_source_line_writer.h instead? Can you also make the first line of your description match the CL description. yes it's crazy but otherwise your git log --format=oneline will be "The memory-infra tools were first written on linux where the" which might be confusing to people. +scottmg as he might have views on this I'm not opposed to an l-g-t-m but just want some clarifications.
On 2017/03/02 23:02:33, Will Harris wrote: > On 2017/03/02 22:52:32, awong wrote: > > Heya will...i can has an OWNERS review? > > hmm - can't you use breakpad .sym files here (the output of running dumpsyms) > and/or some part of existing crashpad/breakpad symbolization? Where will this > tool actually be running? Maybe you can use code in pdb_source_line_writer.h > instead? I'm trying to avoid adding even more steps into what's already a multi-step process. Hence asking people to dumpsyms first seems like bad UX so-to-speak. As for pdb_source_line_writer.h.... sure... but not sure I see a benefit to just fixing the existing binary by writing yet-another-binary-that-invokes-DIA? The long term solution should likely be to fix llvm_symbolizer to talk to DIA (which is a WIP) and shift the calling scripts to that. > Can you also make the first line of your description match the CL description. > yes it's crazy but otherwise your git log --format=oneline will be "The > memory-infra tools were first written on linux where the" which might be > confusing to people. Ah. fun. Done. > > +scottmg as he might have views on this > > I'm not opposed to an l-g-t-m but just want some clarifications.
Description was changed from ========== The memory-infra tools were first written on linux where the symbolization was done using addr2line. Turns out tcmalloc has some old code (with a few small bugs) that wraps the windows dbghelp.dll symbolization APIs in an addr2line-like interface called addr2line-pdb. This CL * fixes the small format string bugs * makes the windows library configuration #defines more friendly for chromium build * modifies addr2line to expect addresses relative to DllBase to better simulate how addr2line works with modules in linux. The last point is important. Windows DLL have a concept of "default load address" which hints to the OS where to load the binary image after relocation. The dbghelp.dll symbolization library will load the module at this location in the virtual address space meaning the caller of these functions would need to be aware of the base address. This makes things unnecessarily complex in the face of ASLR and also diverges from the behavior of addr2line when used with linux-style DSOs. This CL simply adds the module base address to the incoming addresses thereby making the input relative addresses for the module which both is easier to use and lines up better with linux's addr2line behavior. BUG=694792 ========== to ========== Build addr2line-pdb from tcmalloc for use in memory-infra symbolization The memory-infra tools were first written on linux where the symbolization was done using addr2line. Turns out tcmalloc has some old code (with a few small bugs) that wraps the windows dbghelp.dll symbolization APIs in an addr2line-like interface called addr2line-pdb. This CL * fixes the small format string bugs * makes the windows library configuration #defines more friendly for chromium build * modifies addr2line to expect addresses relative to DllBase to better simulate how addr2line works with modules in linux. The last point is important. Windows DLL have a concept of "default load address" which hints to the OS where to load the binary image after relocation. The dbghelp.dll symbolization library will load the module at this location in the virtual address space meaning the caller of these functions would need to be aware of the base address. This makes things unnecessarily complex in the face of ASLR and also diverges from the behavior of addr2line when used with linux-style DSOs. This CL simply adds the module base address to the incoming addresses thereby making the input relative addresses for the module which both is easier to use and lines up better with linux's addr2line behavior. BUG=694792 ==========
okay I trust your judgement so lgtm
On 2017/03/02 23:08:28, Will Harris wrote: > okay I trust your judgement so lgtm <3
The CQ bit was checked by ajwong@chromium.org
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: chromium_presubmit on master.tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)
scottmg@chromium.org changed reviewers: + scottmg@chromium.org
lgtm for BUILD.gn after moving. As far as general approach, either way seems fine, but someday we would like to remove Breakpad (and dump_syms as a result), so this way is probably nicer from that perspective. https://codereview.chromium.org/2730473002/diff/60001/BUILD.gn File BUILD.gn (right): https://codereview.chromium.org/2730473002/diff/60001/BUILD.gn#newcode758 BUILD.gn:758: deps += [ "//third_party/tcmalloc:addr2line-pdb" ] There's no such thing as gyp any more so this should go up near line 660, I think.
https://codereview.chromium.org/2730473002/diff/60001/BUILD.gn File BUILD.gn (right): https://codereview.chromium.org/2730473002/diff/60001/BUILD.gn#newcode758 BUILD.gn:758: deps += [ "//third_party/tcmalloc:addr2line-pdb" ] On 2017/03/03 00:20:24, scottmg wrote: > There's no such thing as gyp any more so this should go up near line 660, I > think. I was told to move it down hear in an earlier review... since I'm a know-nothing noogler at this point...who should I listen to? :)
https://codereview.chromium.org/2730473002/diff/60001/BUILD.gn File BUILD.gn (right): https://codereview.chromium.org/2730473002/diff/60001/BUILD.gn#newcode758 BUILD.gn:758: deps += [ "//third_party/tcmalloc:addr2line-pdb" ] On 2017/03/06 18:45:52, awong wrote: > On 2017/03/03 00:20:24, scottmg wrote: > > There's no such thing as gyp any more so this should go up near line 660, I > > think. > > I was told to move it down hear in an earlier review... since I'm a know-nothing > noogler at this point...who should I listen to? :) I'm the one who asked for this move. This group is called gn_only and the group above is both_gn_and_gyp.
On 2017/03/06 19:31:18, erikchen wrote: > https://codereview.chromium.org/2730473002/diff/60001/BUILD.gn > File BUILD.gn (right): > > https://codereview.chromium.org/2730473002/diff/60001/BUILD.gn#newcode758 > BUILD.gn:758: deps += [ "//third_party/tcmalloc:addr2line-pdb" ] > On 2017/03/06 18:45:52, awong wrote: > > On 2017/03/03 00:20:24, scottmg wrote: > > > There's no such thing as gyp any more so this should go up near line 660, I > > > think. > > > > I was told to move it down hear in an earlier review... since I'm a > know-nothing > > noogler at this point...who should I listen to? :) > > I'm the one who asked for this move. This group is called gn_only and the group > above is both_gn_and_gyp. There's no gyp any more, so those groups should all be merged, I think. With that in mind, it just didn't seem worth making a new if win block. But doesn't really matter, either way is fine for this CL.
The CQ bit was checked by ajwong@chromium.org
The patchset sent to the CQ was uploaded after l-g-t-m from erikchen@chromium.org, wfh@chromium.org, scottmg@chromium.org Link to the patchset: https://codereview.chromium.org/2730473002/#ps100001 (title: "Move target correctly")
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: linux_chromium_chromeos_rel_ng on master.tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
The CQ bit was checked by ajwong@chromium.org
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
CQ is committing da patch. Bot data: {"patchset_id": 100001, "attempt_start_ts": 1489617166914170, "parent_rev": "5217bf718aa6da446b8acee61341b1f4ceab2248", "commit_rev": "bf193279edce86f900ae5b222dc93ea427ae1ff0"}
Message was sent while issue was closed.
Description was changed from ========== Build addr2line-pdb from tcmalloc for use in memory-infra symbolization The memory-infra tools were first written on linux where the symbolization was done using addr2line. Turns out tcmalloc has some old code (with a few small bugs) that wraps the windows dbghelp.dll symbolization APIs in an addr2line-like interface called addr2line-pdb. This CL * fixes the small format string bugs * makes the windows library configuration #defines more friendly for chromium build * modifies addr2line to expect addresses relative to DllBase to better simulate how addr2line works with modules in linux. The last point is important. Windows DLL have a concept of "default load address" which hints to the OS where to load the binary image after relocation. The dbghelp.dll symbolization library will load the module at this location in the virtual address space meaning the caller of these functions would need to be aware of the base address. This makes things unnecessarily complex in the face of ASLR and also diverges from the behavior of addr2line when used with linux-style DSOs. This CL simply adds the module base address to the incoming addresses thereby making the input relative addresses for the module which both is easier to use and lines up better with linux's addr2line behavior. BUG=694792 ========== to ========== Build addr2line-pdb from tcmalloc for use in memory-infra symbolization The memory-infra tools were first written on linux where the symbolization was done using addr2line. Turns out tcmalloc has some old code (with a few small bugs) that wraps the windows dbghelp.dll symbolization APIs in an addr2line-like interface called addr2line-pdb. This CL * fixes the small format string bugs * makes the windows library configuration #defines more friendly for chromium build * modifies addr2line to expect addresses relative to DllBase to better simulate how addr2line works with modules in linux. The last point is important. Windows DLL have a concept of "default load address" which hints to the OS where to load the binary image after relocation. The dbghelp.dll symbolization library will load the module at this location in the virtual address space meaning the caller of these functions would need to be aware of the base address. This makes things unnecessarily complex in the face of ASLR and also diverges from the behavior of addr2line when used with linux-style DSOs. This CL simply adds the module base address to the incoming addresses thereby making the input relative addresses for the module which both is easier to use and lines up better with linux's addr2line behavior. BUG=694792 Review-Url: https://codereview.chromium.org/2730473002 Cr-Commit-Position: refs/heads/master@{#457271} Committed: https://chromium.googlesource.com/chromium/src/+/bf193279edce86f900ae5b222dc9... ==========
Message was sent while issue was closed.
Committed patchset #6 (id:100001) as https://chromium.googlesource.com/chromium/src/+/bf193279edce86f900ae5b222dc9... |