Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(459)

Side by Side Diff: tools/clang/scripts/run_tool.py

Issue 754433003: Update from https://crrev.com/305340 (Closed) Base URL: git@github.com:domokit/mojo.git@master
Patch Set: Created 6 years ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « tools/clang/scripts/generate_win_compdb.py ('k') | tools/clang/scripts/update.sh » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 #!/usr/bin/env python 1 #!/usr/bin/env python
2 # Copyright (c) 2013 The Chromium Authors. All rights reserved. 2 # Copyright (c) 2013 The Chromium Authors. All rights reserved.
3 # Use of this source code is governed by a BSD-style license that can be 3 # Use of this source code is governed by a BSD-style license that can be
4 # found in the LICENSE file. 4 # found in the LICENSE file.
5 5
6 """Wrapper script to help run clang tools across Chromium code. 6 """Wrapper script to help run clang tools across Chromium code.
7 7
8 How to use this tool: 8 How to use this tool:
9 If you want to run the tool across all Chromium code: 9 If you want to run the tool across all Chromium code:
10 run_tool.py <tool> <path/to/compiledb> 10 run_tool.py <tool> <path/to/compiledb>
11 11
12 If you want to include all files mentioned in the compilation database:
13 run_tool.py <tool> <path/to/compiledb> --all
14
12 If you only want to run the tool across just chrome/browser and content/browser: 15 If you only want to run the tool across just chrome/browser and content/browser:
13 run_tool.py <tool> <path/to/compiledb> chrome/browser content/browser 16 run_tool.py <tool> <path/to/compiledb> chrome/browser content/browser
14 17
15 Please see https://code.google.com/p/chromium/wiki/ClangToolRefactoring for more 18 Please see https://code.google.com/p/chromium/wiki/ClangToolRefactoring for more
16 information, which documents the entire automated refactoring flow in Chromium. 19 information, which documents the entire automated refactoring flow in Chromium.
17 20
18 Why use this tool: 21 Why use this tool:
19 The clang tool implementation doesn't take advantage of multiple cores, and if 22 The clang tool implementation doesn't take advantage of multiple cores, and if
20 it fails mysteriously in the middle, all the generated replacements will be 23 it fails mysteriously in the middle, all the generated replacements will be
21 lost. 24 lost.
22 25
23 Unfortunately, if the work is simply sharded across multiple cores by running 26 Unfortunately, if the work is simply sharded across multiple cores by running
24 multiple RefactoringTools, problems arise when they attempt to rewrite a file at 27 multiple RefactoringTools, problems arise when they attempt to rewrite a file at
25 the same time. To work around that, clang tools that are run using this tool 28 the same time. To work around that, clang tools that are run using this tool
26 should output edits to stdout in the following format: 29 should output edits to stdout in the following format:
27 30
28 ==== BEGIN EDITS ==== 31 ==== BEGIN EDITS ====
29 r:<file path>:<offset>:<length>:<replacement text> 32 r:<file path>:<offset>:<length>:<replacement text>
30 r:<file path>:<offset>:<length>:<replacement text> 33 r:<file path>:<offset>:<length>:<replacement text>
31 ...etc... 34 ...etc...
32 ==== END EDITS ==== 35 ==== END EDITS ====
33 36
34 Any generated edits are applied once the clang tool has finished running 37 Any generated edits are applied once the clang tool has finished running
35 across Chromium, regardless of whether some instances failed or not. 38 across Chromium, regardless of whether some instances failed or not.
36 """ 39 """
37 40
38 import collections 41 import collections
39 import functools 42 import functools
43 import json
40 import multiprocessing 44 import multiprocessing
41 import os.path 45 import os.path
42 import pipes 46 import pipes
43 import subprocess 47 import subprocess
44 import sys 48 import sys
45 49
46 50
47 Edit = collections.namedtuple( 51 Edit = collections.namedtuple(
48 'Edit', ('edit_type', 'offset', 'length', 'replacement')) 52 'Edit', ('edit_type', 'offset', 'length', 'replacement'))
49 53
(...skipping 10 matching lines...) Expand all
60 else: 64 else:
61 args.append('git') 65 args.append('git')
62 args.append('ls-files') 66 args.append('ls-files')
63 if paths: 67 if paths:
64 args.extend(paths) 68 args.extend(paths)
65 command = subprocess.Popen(args, stdout=subprocess.PIPE) 69 command = subprocess.Popen(args, stdout=subprocess.PIPE)
66 output, _ = command.communicate() 70 output, _ = command.communicate()
67 return [os.path.realpath(p) for p in output.splitlines()] 71 return [os.path.realpath(p) for p in output.splitlines()]
68 72
69 73
74 def _GetFilesFromCompileDB(build_directory):
75 """ Gets the list of files mentioned in the compilation database.
76
77 Args:
78 build_directory: Directory that contains the compile database.
79 """
80 compiledb_path = os.path.join(build_directory, 'compile_commands.json')
81 with open(compiledb_path, 'rb') as compiledb_file:
82 json_commands = json.load(compiledb_file)
83
84 return [os.path.join(entry['directory'], entry['file'])
85 for entry in json_commands]
86
87
70 def _ExtractEditsFromStdout(build_directory, stdout): 88 def _ExtractEditsFromStdout(build_directory, stdout):
71 """Extracts generated list of edits from the tool's stdout. 89 """Extracts generated list of edits from the tool's stdout.
72 90
73 The expected format is documented at the top of this file. 91 The expected format is documented at the top of this file.
74 92
75 Args: 93 Args:
76 build_directory: Directory that contains the compile database. Used to 94 build_directory: Directory that contains the compile database. Used to
77 normalize the filenames. 95 normalize the filenames.
78 stdout: The stdout from running the clang tool. 96 stdout: The stdout from running the clang tool.
79 97
(...skipping 208 matching lines...) Expand 10 before | Expand all | Expand 10 after
288 clang_format_diff_path = os.path.join( 306 clang_format_diff_path = os.path.join(
289 os.path.dirname(os.path.realpath(__file__)), 307 os.path.dirname(os.path.realpath(__file__)),
290 '../../../third_party/llvm/tools/clang/tools/clang-format', 308 '../../../third_party/llvm/tools/clang/tools/clang-format',
291 'clang-format-diff.py') 309 'clang-format-diff.py')
292 # TODO(dcheng): Allow this to be controlled with a flag as well. 310 # TODO(dcheng): Allow this to be controlled with a flag as well.
293 # TODO(dcheng): Shell escaping of args to git diff to clang-format is broken 311 # TODO(dcheng): Shell escaping of args to git diff to clang-format is broken
294 # on Windows. 312 # on Windows.
295 if not os.path.isfile(clang_format_diff_path) or sys.platform == 'win32': 313 if not os.path.isfile(clang_format_diff_path) or sys.platform == 'win32':
296 clang_format_diff_path = None 314 clang_format_diff_path = None
297 315
298 filenames = frozenset(_GetFilesFromGit(argv[2:])) 316 if len(argv) == 3 and argv[2] == '--all':
317 filenames = frozenset(_GetFilesFromCompileDB(argv[1]))
318 else:
319 filenames = frozenset(_GetFilesFromGit(argv[2:]))
299 # Filter out files that aren't C/C++/Obj-C/Obj-C++. 320 # Filter out files that aren't C/C++/Obj-C/Obj-C++.
300 extensions = frozenset(('.c', '.cc', '.m', '.mm')) 321 extensions = frozenset(('.c', '.cc', '.m', '.mm'))
301 dispatcher = _CompilerDispatcher(argv[0], argv[1], 322 dispatcher = _CompilerDispatcher(argv[0], argv[1],
302 [f for f in filenames 323 [f for f in filenames
303 if os.path.splitext(f)[1] in extensions]) 324 if os.path.splitext(f)[1] in extensions])
304 dispatcher.Run() 325 dispatcher.Run()
305 # Filter out edits to files that aren't in the git repository, since it's not 326 # Filter out edits to files that aren't in the git repository, since it's not
306 # useful to modify files that aren't under source control--typically, these 327 # useful to modify files that aren't under source control--typically, these
307 # are generated files or files in a git submodule that's not part of Chromium. 328 # are generated files or files in a git submodule that's not part of Chromium.
308 _ApplyEdits({k : v for k, v in dispatcher.edits.iteritems() 329 _ApplyEdits({k : v for k, v in dispatcher.edits.iteritems()
309 if os.path.realpath(k) in filenames}, 330 if os.path.realpath(k) in filenames},
310 clang_format_diff_path) 331 clang_format_diff_path)
311 if dispatcher.failed_count != 0: 332 if dispatcher.failed_count != 0:
312 return 2 333 return 2
313 return 0 334 return 0
314 335
315 336
316 if __name__ == '__main__': 337 if __name__ == '__main__':
317 sys.exit(main(sys.argv[1:])) 338 sys.exit(main(sys.argv[1:]))
OLDNEW
« no previous file with comments | « tools/clang/scripts/generate_win_compdb.py ('k') | tools/clang/scripts/update.sh » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698