Script for parsing of verbose linker output

tools/win/linker_verbose_tracking.py

Index: tools/win/linker_verbose_tracking.py
diff --git a/tools/win/linker_verbose_tracking.py b/tools/win/linker_verbose_tracking.py
new file mode 100644
index 0000000000000000000000000000000000000000..585d7f0577db30ccf9e4246eae1bf9564ae4ffe2
--- /dev/null
+++ b/tools/win/linker_verbose_tracking.py
@@ -0,0 +1,138 @@
+# Copyright (c) 2016 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+"""
+This script parses the /verbose output from the VC++ linker and uses it to
+explain why a particular object file is being linked in. It parses records
+like these:
+
+      Found "public: static void * __cdecl SkTLS::Get(void * (__cdecl*)(void),void (__cdecl*)(void *))" (?Get@SkTLS@@SAPAXP6APAXXZP6AXPAX@Z@Z)
+        Referenced in chrome_crash_reporter_client_win.obj
+        Referenced in skia.lib(SkError.obj)
+        Loaded skia.lib(SkTLS.obj)
+
+and then uses the information to answer questions such as "why is SkTLS.obj
+being linked in. In this case it was requested by SkError.obj, and the process
+is then repeated for SkError.obj. It traces the dependency tree back to a file
+that was specified on the command line. Typically that file is part of a
+source_set, and if that source_set is causing unnecessary code and data to be
+pulled in then changing it to a static_library may reduce the binary size. See
+crrev.com/2556603002 for an example of a ~900 KB savings from such a change.
+
+In other cases the source_set to static_library fix does not work because some
+of the symbols are required, while others are pulling in unwanted object files.
+In these cases it can be necessary to see what symbol is causing one object file
+to reference another. Removing or moving the problematic symbol can fix the
+problem. See crrev.com/2559063002 for an example of such a change.
+
+One complication is that there are sometimes multiple source files with the
+same name, such as crc.c, which can make analysis more difficult or
+ambiguous. If this becomes a blocking issue they it may be necessary to
+temporarily rename the source file.
+
+Object file name matching is case sensitive.
+
+Typical output when run on chrome.dll verbose link output is:
+
+>python linker_verbose_tracking.py chrome_dll_verbose_01.txt chrome_browser_main
+Database loaded - 11286 xrefs found
+chrome_browser_main.obj referenced by
+        chrome_browser_main_win.obj for symbol public: virtual __thiscall ChromeBrowserMainParts::~ChromeBrowserMainParts(void)
+
+chrome_browser_main_win.obj referenced by
+        chrome_content_browser_client.obj for symbol public: __thiscall ChromeBrowserMainPartsWin::ChromeBrowserMainPartsWin(struct content::MainFunctionParams const &)
+
+chrome_content_browser_client.obj referenced by
+        Command-line obj file: chrome_main_delegate.obj for symbol public: __thiscall ChromeContentBrowserClient::ChromeContentBrowserClient(void)
+"""
+
+import pdb
+import re
+import sys
+
+def ParseVerbose(input_file):
+  obj_match = re.compile('.*\((.*)\.obj\)')

[stanisc, 2016/12/10 00:13:11]

It would help to see a comment with an example of matching string (just because
my regular expression skills are a bit rusty and all the escaping makes it
harder to read the pattern). I guess it is any sequence of characters followed
by an obj file name in parenthesis.

[brucedawson, 2016/12/10 00:59:57]

Done.

+
+  cross_refs = {}
+  cross_refed_symbols = {}
+
+  references = None
+  for line in open(input_file):
+    if line.startswith('      Found'):
+      references = []
+      # Grab the symbol name
+      symbol = line[13:line.find('"', 13)]
+      continue
+    if type(references) == type([]):
+      sub_line = line.strip()
+      match = obj_match.match(sub_line)
+      # See if the line is part of the list of places where this symbol was
+      # referenced
+      if sub_line.count('Referenced ') > 0:
+        if match:
+          # This indicates a match that is xxx.lib(yyy.obj), so a referencing
+          # .obj file that was itself
+          reference = match.groups()[0]
+        else:
+          # This indicates a match that is just a pure .obj file name
+          # I think this means that the .obj file was specified on the linker
+          # command line.
+          reference = ('Command-line obj file: ' +
+                       sub_line[len('Referenced in '): -len('.obj')])
+        references.append(reference)
+      elif sub_line.count('Loaded ') > 0:
+        if match:
+          loaded = match.groups()[0]
+          cross_refs[loaded] = references
+          cross_refed_symbols[loaded] = symbol
+        references = None
+    if line.startswith('Finished pass 1'):
+      # Stop now because the remaining 90% of the verbose output is
+      # not of interest. Could probably use /VERBOSE:REF to trim out
+      # boring information.
+      break
+  return cross_refs, cross_refed_symbols
+
+
+def TrackObj(cross_refs, cross_refed_symbols, obj_name):
+  if obj_name.lower().endswith('.obj'):
+    obj_name = obj_name[:-len('.obj')]
+
+  # Keep track of which references we've already followed.
+  tracked = {}
+
+  # Initial set of object files that we are tracking.
+  targets = [obj_name]
+  printed = False
+  for i in range(100):
+    new_targets = {}
+    for target in targets:
+      if not target in tracked:
+        tracked[target] = True
+        if target in cross_refs.keys():
+          printed = True
+          print '%s.obj referenced by' % target
+          for ref in cross_refs[target]:
+            symbol = cross_refed_symbols[target]

[stanisc, 2016/12/10 00:13:11]

Does this need to be inside the inner for loop?

[brucedawson, 2016/12/10 00:59:57]

Good point. And since it is invariant for the target it should actually only be
printed once, so I've changed the output format slightly.

+            print '\t%s.obj for symbol %s' % (ref, symbol)
+            new_targets[ref] = True
+    if len(new_targets) == 0:
+      break
+    print
+    targets = new_targets.keys()
+  if not printed:
+    print 'No references to %s.obj found.' % obj_name
+
+
+def main():
+  if len(sys.argv) < 3:
+    print r'Usage: %s <verbose_output_file> <objfile>' % sys.argv[0]
+    print r'Sample: %s chrome_dll_verbose.txt SkTLS'

[stanisc, 2016/12/10 00:13:11]

It looks like this also needs % sys.argv[0] argument

[brucedawson, 2016/12/10 00:59:57]

Good catch. Done.

+    return 0
+  cross_refs, cross_refed_symbols = ParseVerbose(sys.argv[1])
+  print 'Database loaded - %d xrefs found' % len(cross_refs)
+  TrackObj(cross_refs, cross_refed_symbols, sys.argv[2])
+
+if __name__ == '__main__':
+  sys.exit(main())