Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(41)

Side by Side Diff: tools/checkdeps/checkdeps.py

Issue 10805042: Implement ability to specify temporarily-allowed dependencies in DEPS (Closed) Base URL: svn://svn.chromium.org/chrome/trunk/src
Patch Set: Merge to LKGR. Created 8 years, 4 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
« no previous file with comments | « tools/checkdeps/PRESUBMIT.py ('k') | tools/checkdeps/checkdeps_test.py » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 #!/usr/bin/env python 1 #!/usr/bin/env python
2 # Copyright (c) 2012 The Chromium Authors. All rights reserved. 2 # Copyright (c) 2012 The Chromium Authors. All rights reserved.
3 # Use of this source code is governed by a BSD-style license that can be 3 # Use of this source code is governed by a BSD-style license that can be
4 # found in the LICENSE file. 4 # found in the LICENSE file.
5 5
6 """Makes sure that files include headers from allowed directories. 6 """Makes sure that files include headers from allowed directories.
7 7
8 Checks DEPS files in the source tree for rules, and applies those rules to 8 Checks DEPS files in the source tree for rules, and applies those rules to
9 "#include" commands in source files. Any source file including something not 9 "#include" commands in source files. Any source file including something not
10 permitted by the DEPS files will fail. 10 permitted by the DEPS files will fail.
11 11
12 The format of the deps file: 12 The format of the deps file:
13 13
14 First you have the normal module-level deps. These are the ones used by 14 First you have the normal module-level deps. These are the ones used by
15 gclient. An example would be: 15 gclient. An example would be:
16 16
17 deps = { 17 deps = {
18 "base":"http://foo.bar/trunk/base" 18 "base":"http://foo.bar/trunk/base"
19 } 19 }
20 20
21 DEPS files not in the top-level of a module won't need this. Then you have 21 DEPS files not in the top-level of a module won't need this. Then you
22 any additional include rules. You can add (using "+") or subtract (using "-") 22 have any additional include rules. You can add (using "+") or subtract
23 from the previously specified rules (including module-level deps). 23 (using "-") from the previously specified rules (including
24 module-level deps). You can also specify a path that is allowed for
25 now but that we intend to remove, using "!"; this is treated the same
26 as "+" when check_deps is run by our bots, but a presubmit step will
27 show a warning if you add a new include of a file that is only allowed
28 by "!".
29
30 Note that for .java files, there is currently no difference between
31 "+" and "!", even in the presubmit step.
24 32
25 include_rules = { 33 include_rules = {
26 # Code should be able to use base (it's specified in the module-level 34 # Code should be able to use base (it's specified in the module-level
27 # deps above), but nothing in "base/evil" because it's evil. 35 # deps above), but nothing in "base/evil" because it's evil.
28 "-base/evil", 36 "-base/evil",
29 37
30 # But this one subdirectory of evil is OK. 38 # But this one subdirectory of evil is OK.
31 "+base/evil/not", 39 "+base/evil/not",
32 40
33 # And it can include files from this other directory even though there is 41 # And it can include files from this other directory even though there is
34 # no deps rule for it. 42 # no deps rule for it.
35 "+tools/crime_fighter" 43 "+tools/crime_fighter",
44
45 # This dependency is allowed for now but work is ongoing to remove it,
46 # so you shouldn't add further dependencies on it.
47 "!base/evil/ok_for_now.h",
36 } 48 }
37 49
38 DEPS files may be placed anywhere in the tree. Each one applies to all 50 DEPS files may be placed anywhere in the tree. Each one applies to all
39 subdirectories, where there may be more DEPS files that provide additions or 51 subdirectories, where there may be more DEPS files that provide additions or
40 subtractions for their own sub-trees. 52 subtractions for their own sub-trees.
41 53
42 There is an implicit rule for the current directory (where the DEPS file lives) 54 There is an implicit rule for the current directory (where the DEPS file lives)
43 and all of its subdirectories. This prevents you from having to explicitly 55 and all of its subdirectories. This prevents you from having to explicitly
44 allow the current directory everywhere. This implicit rule is applied first, 56 allow the current directory everywhere. This implicit rule is applied first,
45 so you can modify or remove it using the normal include rules. 57 so you can modify or remove it using the normal include rules.
46 58
47 The rules are processed in order. This means you can explicitly allow a higher 59 The rules are processed in order. This means you can explicitly allow a higher
48 directory and then take away permissions from sub-parts, or the reverse. 60 directory and then take away permissions from sub-parts, or the reverse.
49 61
50 Note that all directory separators must be slashes (Unix-style) and not 62 Note that all directory separators must be slashes (Unix-style) and not
51 backslashes. All directories should be relative to the source root and use 63 backslashes. All directories should be relative to the source root and use
52 only lowercase. 64 only lowercase.
53 """ 65 """
54 66
55 import os 67 import os
56 import optparse 68 import optparse
57 import pipes 69 import subprocess
58 import sys 70 import sys
59 import copy 71 import copy
60 72
61 import cpp_checker 73 import cpp_checker
62 import java_checker 74 import java_checker
75 from rules import Rule, Rules
63 76
64 77
65 # Variable name used in the DEPS file to add or subtract include files from 78 # Variable name used in the DEPS file to add or subtract include files from
66 # the module-level deps. 79 # the module-level deps.
67 INCLUDE_RULES_VAR_NAME = "include_rules" 80 INCLUDE_RULES_VAR_NAME = "include_rules"
68 81
69 # Optionally present in the DEPS file to list subdirectories which should not 82 # Optionally present in the DEPS file to list subdirectories which should not
70 # be checked. This allows us to skip third party code, for example. 83 # be checked. This allows us to skip third party code, for example.
71 SKIP_SUBDIRS_VAR_NAME = "skip_child_includes" 84 SKIP_SUBDIRS_VAR_NAME = "skip_child_includes"
72 85
73 # Set to true for more output. This is set by the command line options. 86
74 VERBOSE = False 87 def NormalizePath(path):
75 88 """Returns a path normalized to how we write DEPS rules and compare paths.
76 # In lowercase, using forward slashes as directory separators, ending in a
77 # forward slash. Set by the command line options.
78 BASE_DIRECTORY = ""
79
80 # The directories which contain the sources managed by git.
81 GIT_SOURCE_DIRECTORY = set()
82
83
84 # Specifies a single rule for an include, which can be either allow or disallow.
85 class Rule(object):
86 def __init__(self, allow, directory, source):
87 self.allow = allow
88 self._dir = directory
89 self._source = source
90
91 def __str__(self):
92 if (self.allow):
93 return '"+%s" from %s.' % (self._dir, self._source)
94 return '"-%s" from %s.' % (self._dir, self._source)
95
96 def ParentOrMatch(self, other):
97 """Returns true if the input string is an exact match or is a parent
98 of the current rule. For example, the input "foo" would match "foo/bar"."""
99 return self._dir == other or self._dir.startswith(other + "/")
100
101 def ChildOrMatch(self, other):
102 """Returns true if the input string would be covered by this rule. For
103 example, the input "foo/bar" would match the rule "foo"."""
104 return self._dir == other or other.startswith(self._dir + "/")
105
106
107 def ParseRuleString(rule_string, source):
108 """Returns a tuple of a boolean indicating whether the directory is an allow
109 rule, and a string holding the directory name.
110 """ 89 """
111 if len(rule_string) < 1: 90 return path.lower().replace('\\', '/')
112 raise Exception('The rule string "%s" is too short\nin %s' % 91
113 (rule_string, source)) 92
114 93 class DepsChecker(object):
115 if rule_string[0] == "+": 94 """Parses include_rules from DEPS files and can verify files in the
116 return (True, rule_string[1:]) 95 source tree against them.
117 if rule_string[0] == "-":
118 return (False, rule_string[1:])
119 raise Exception('The rule string "%s" does not begin with a "+" or a "-"' %
120 rule_string)
121
122
123 class Rules:
124 def __init__(self):
125 """Initializes the current rules with an empty rule list."""
126 self._rules = []
127
128 def __str__(self):
129 ret = "Rules = [\n"
130 ret += "\n".join([" %s" % x for x in self._rules])
131 ret += "]\n"
132 return ret
133
134 def AddRule(self, rule_string, source):
135 """Adds a rule for the given rule string.
136
137 Args:
138 rule_string: The include_rule string read from the DEPS file to apply.
139 source: A string representing the location of that string (filename, etc.)
140 so that we can give meaningful errors.
141 """
142 (add_rule, rule_dir) = ParseRuleString(rule_string, source)
143 # Remove any existing rules or sub-rules that apply. For example, if we're
144 # passed "foo", we should remove "foo", "foo/bar", but not "foobar".
145 self._rules = [x for x in self._rules if not x.ParentOrMatch(rule_dir)]
146 self._rules.insert(0, Rule(add_rule, rule_dir, source))
147
148 def DirAllowed(self, allowed_dir):
149 """Returns a tuple (success, message), where success indicates if the given
150 directory is allowed given the current set of rules, and the message tells
151 why if the comparison failed."""
152 for rule in self._rules:
153 if rule.ChildOrMatch(allowed_dir):
154 # This rule applies.
155 if rule.allow:
156 return (True, "")
157 return (False, rule.__str__())
158 # No rules apply, fail.
159 return (False, "no rule applying")
160
161
162 def ApplyRules(existing_rules, includes, cur_dir):
163 """Applies the given include rules, returning the new rules.
164
165 Args:
166 existing_rules: A set of existing rules that will be combined.
167 include: The list of rules from the "include_rules" section of DEPS.
168 cur_dir: The current directory. We will create an implicit rule that
169 allows inclusion from this directory.
170
171 Returns: A new set of rules combining the existing_rules with the other
172 arguments.
173 """ 96 """
174 rules = copy.copy(existing_rules) 97
175 98 def __init__(self, base_directory=None, verbose=False, being_tested=False):
176 # First apply the implicit "allow" rule for the current directory. 99 """Creates a new DepsChecker.
177 if cur_dir.lower().startswith(BASE_DIRECTORY): 100
178 relative_dir = cur_dir[len(BASE_DIRECTORY) + 1:] 101 Args:
179 # Normalize path separators to slashes. 102 base_directory: OS-compatible path to root of checkout, e.g. C:\chr\src.
180 relative_dir = relative_dir.replace("\\", "/") 103 verbose: Set to true for debug output.
181 source = relative_dir 104 being_tested: Set to true to ignore the DEPS file at tools/checkdeps/DEPS.
182 if len(source) == 0: 105 """
183 source = "top level" # Make the help string a little more meaningful. 106 self.base_directory = base_directory
184 rules.AddRule("+" + relative_dir, "Default rule for " + source) 107 if not base_directory:
185 else: 108 self.base_directory = os.path.abspath(
186 raise Exception("Internal error: base directory is not at the beginning" + 109 os.path.join(os.path.abspath(os.path.dirname(__file__)), '..', '..'))
187 " for\n %s and base dir\n %s" % 110
188 (cur_dir, BASE_DIRECTORY)) 111 self.verbose = verbose
189 112
190 # Last, apply the additional explicit rules. 113 self.git_source_directories = set()
191 for (_, rule_str) in enumerate(includes): 114 self._AddGitSourceDirectories()
192 if not len(relative_dir): 115
193 rule_description = "the top level include_rules" 116 self._under_test = being_tested
117
118 def _ApplyRules(self, existing_rules, includes, cur_dir):
119 """Applies the given include rules, returning the new rules.
120
121 Args:
122 existing_rules: A set of existing rules that will be combined.
123 include: The list of rules from the "include_rules" section of DEPS.
124 cur_dir: The current directory, normalized path. We will create an
125 implicit rule that allows inclusion from this directory.
126
127 Returns: A new set of rules combining the existing_rules with the other
128 arguments.
129 """
130 rules = copy.copy(existing_rules)
131
132 # First apply the implicit "allow" rule for the current directory.
133 if cur_dir.startswith(
134 NormalizePath(os.path.normpath(self.base_directory))):
135 relative_dir = cur_dir[len(self.base_directory) + 1:]
136
137 source = relative_dir
138 if len(source) == 0:
139 source = "top level" # Make the help string a little more meaningful.
140 rules.AddRule("+" + relative_dir, "Default rule for " + source)
194 else: 141 else:
195 rule_description = relative_dir + "'s include_rules" 142 raise Exception("Internal error: base directory is not at the beginning" +
196 rules.AddRule(rule_str, rule_description) 143 " for\n %s and base dir\n %s" %
197 144 (cur_dir, self.base_directory))
198 return rules 145
199 146 # Last, apply the additional explicit rules.
200 147 for (_, rule_str) in enumerate(includes):
201 def ApplyDirectoryRules(existing_rules, dir_name): 148 if not relative_dir:
202 """Combines rules from the existing rules and the new directory. 149 rule_description = "the top level include_rules"
203 150 else:
204 Any directory can contain a DEPS file. Toplevel DEPS files can contain 151 rule_description = relative_dir + "'s include_rules"
205 module dependencies which are used by gclient. We use these, along with 152 rules.AddRule(rule_str, rule_description)
206 additional include rules and implicit rules for the given directory, to 153
207 come up with a combined set of rules to apply for the directory. 154 return rules
208 155
209 Args: 156 def _ApplyDirectoryRules(self, existing_rules, dir_name):
210 existing_rules: The rules for the parent directory. We'll add-on to these. 157 """Combines rules from the existing rules and the new directory.
211 dir_name: The directory name that the deps file may live in (if it exists). 158
212 This will also be used to generate the implicit rules. 159 Any directory can contain a DEPS file. Toplevel DEPS files can contain
213 160 module dependencies which are used by gclient. We use these, along with
214 Returns: A tuple containing: (1) the combined set of rules to apply to the 161 additional include rules and implicit rules for the given directory, to
215 sub-tree, and (2) a list of all subdirectories that should NOT be 162 come up with a combined set of rules to apply for the directory.
216 checked, as specified in the DEPS file (if any). 163
217 """ 164 Args:
218 # Check for a .svn directory in this directory or check this directory is 165 existing_rules: The rules for the parent directory. We'll add-on to these.
219 # contained in git source direcotries. This will tell us if it's a source 166 dir_name: The directory name that the deps file may live in (if
220 # directory and should be checked. 167 it exists). This will also be used to generate the
221 if not (os.path.exists(os.path.join(dir_name, ".svn")) or 168 implicit rules. This is a non-normalized path.
222 (dir_name.lower() in GIT_SOURCE_DIRECTORY)): 169
223 return (None, []) 170 Returns: A tuple containing: (1) the combined set of rules to apply to the
224 171 sub-tree, and (2) a list of all subdirectories that should NOT be
225 # Check the DEPS file in this directory. 172 checked, as specified in the DEPS file (if any).
226 if VERBOSE: 173 """
227 print "Applying rules from", dir_name 174 norm_dir_name = NormalizePath(dir_name)
228 def FromImpl(_unused, _unused2): 175
229 pass # NOP function so "From" doesn't fail. 176 # Check for a .svn directory in this directory or check this directory is
230 177 # contained in git source direcotries. This will tell us if it's a source
231 def FileImpl(_unused): 178 # directory and should be checked.
232 pass # NOP function so "File" doesn't fail. 179 if not (os.path.exists(os.path.join(dir_name, ".svn")) or
233 180 (norm_dir_name in self.git_source_directories)):
234 class _VarImpl: 181 return (None, [])
235 def __init__(self, local_scope): 182
236 self._local_scope = local_scope 183 # Check the DEPS file in this directory.
237 184 if self.verbose:
238 def Lookup(self, var_name): 185 print "Applying rules from", dir_name
239 """Implements the Var syntax.""" 186 def FromImpl(_unused, _unused2):
240 if var_name in self._local_scope.get("vars", {}): 187 pass # NOP function so "From" doesn't fail.
241 return self._local_scope["vars"][var_name] 188
242 raise Exception("Var is not defined: %s" % var_name) 189 def FileImpl(_unused):
243 190 pass # NOP function so "File" doesn't fail.
244 local_scope = {} 191
245 global_scope = { 192 class _VarImpl:
246 "File": FileImpl, 193 def __init__(self, local_scope):
247 "From": FromImpl, 194 self._local_scope = local_scope
248 "Var": _VarImpl(local_scope).Lookup, 195
249 } 196 def Lookup(self, var_name):
250 deps_file = os.path.join(dir_name, "DEPS") 197 """Implements the Var syntax."""
251 198 if var_name in self._local_scope.get("vars", {}):
252 if os.path.isfile(deps_file): 199 return self._local_scope["vars"][var_name]
253 execfile(deps_file, global_scope, local_scope) 200 raise Exception("Var is not defined: %s" % var_name)
254 elif VERBOSE: 201
255 print " No deps file found in", dir_name 202 local_scope = {}
256 203 global_scope = {
257 # Even if a DEPS file does not exist we still invoke ApplyRules 204 "File": FileImpl,
258 # to apply the implicit "allow" rule for the current directory 205 "From": FromImpl,
259 include_rules = local_scope.get(INCLUDE_RULES_VAR_NAME, []) 206 "Var": _VarImpl(local_scope).Lookup,
260 skip_subdirs = local_scope.get(SKIP_SUBDIRS_VAR_NAME, []) 207 }
261 208 deps_file = os.path.join(dir_name, "DEPS")
262 return (ApplyRules(existing_rules, include_rules, dir_name), skip_subdirs) 209
263 210 # The second conditional here is to disregard the
264 211 # tools/checkdeps/DEPS file while running tests. This DEPS file
265 def CheckDirectory(parent_rules, checkers, dir_name): 212 # has a skip_child_includes for 'testdata' which is necessary for
266 (rules, skip_subdirs) = ApplyDirectoryRules(parent_rules, dir_name) 213 # running production tests, since there are intentional DEPS
267 if rules == None: 214 # violations under the testdata directory. On the other hand when
268 return True 215 # running tests, we absolutely need to verify the contents of that
269 216 # directory to trigger those intended violations and see that they
270 # Collect a list of all files and directories to check. 217 # are handled correctly.
271 files_to_check = [] 218 if os.path.isfile(deps_file) and (
272 dirs_to_check = [] 219 not self._under_test or not os.path.split(dir_name)[1] == 'checkdeps'):
273 success = True 220 execfile(deps_file, global_scope, local_scope)
274 contents = os.listdir(dir_name) 221 elif self.verbose:
275 for cur in contents: 222 print " No deps file found in", dir_name
276 if cur in skip_subdirs: 223
277 continue # Don't check children that DEPS has asked us to skip. 224 # Even if a DEPS file does not exist we still invoke ApplyRules
278 full_name = os.path.join(dir_name, cur) 225 # to apply the implicit "allow" rule for the current directory
279 if os.path.isdir(full_name): 226 include_rules = local_scope.get(INCLUDE_RULES_VAR_NAME, [])
280 dirs_to_check.append(full_name) 227 skip_subdirs = local_scope.get(SKIP_SUBDIRS_VAR_NAME, [])
281 elif os.path.splitext(full_name)[1] in checkers: 228
282 files_to_check.append(full_name) 229 return (self._ApplyRules(existing_rules, include_rules, norm_dir_name),
283 230 skip_subdirs)
284 # First check all files in this directory. 231
285 for cur in files_to_check: 232 def CheckDirectory(self, start_dir):
286 checker = checkers[os.path.splitext(cur)[1]] 233 """Checks all relevant source files in the specified directory and
287 file_status = checker.CheckFile(rules, cur) 234 its subdirectories for compliance with DEPS rules throughout the
288 if file_status: 235 tree (starting at |self.base_directory|). |start_dir| must be a
289 print "ERROR in " + cur + "\n" + file_status 236 subdirectory of |self.base_directory|.
290 success = False 237
291 238 Returns an empty array on success. On failure, the array contains
292 # Next recurse into the subdirectories. 239 strings that can be printed as human-readable error messages.
293 for cur in dirs_to_check: 240 """
294 if not CheckDirectory(rules, checkers, cur): 241 # TODO(joi): Make this work for start_dir != base_dir (I have a
295 success = False 242 # subsequent change in flight to do this).
296 243 base_rules = Rules()
297 return success 244 java = java_checker.JavaChecker(self.base_directory, self.verbose)
298 245 cpp = cpp_checker.CppChecker(self.verbose)
299 246 checkers = dict(
300 def GetGitSourceDirectory(root): 247 (extension, checker)
301 """Returns a set of the directories to be checked. 248 for checker in [java, cpp] for extension in checker.EXTENSIONS)
302 249 return self._CheckDirectoryImpl(base_rules, checkers, start_dir)
303 Args: 250
304 root: The repository root where .git directory exists. 251 def _CheckDirectoryImpl(self, parent_rules, checkers, dir_name):
305 252 (rules, skip_subdirs) = self._ApplyDirectoryRules(parent_rules, dir_name)
306 Returns: 253 if rules == None:
307 A set of directories which contain sources managed by git. 254 return []
308 """ 255
309 git_source_directory = set() 256 # Collect a list of all files and directories to check.
310 popen_out = os.popen("cd %s && git ls-files --full-name ." % 257 files_to_check = []
311 pipes.quote(root)) 258 dirs_to_check = []
312 for line in popen_out.readlines(): 259 results = []
313 dir_name = os.path.join(root, os.path.dirname(line)) 260 contents = os.listdir(dir_name)
314 # Add the directory as well as all the parent directories. 261 for cur in contents:
315 while dir_name != root: 262 if cur in skip_subdirs:
316 git_source_directory.add(dir_name) 263 continue # Don't check children that DEPS has asked us to skip.
317 dir_name = os.path.dirname(dir_name) 264 full_name = os.path.join(dir_name, cur)
318 git_source_directory.add(root) 265 if os.path.isdir(full_name):
319 return git_source_directory 266 dirs_to_check.append(full_name)
267 elif os.path.splitext(full_name)[1] in checkers:
268 files_to_check.append(full_name)
269
270 # First check all files in this directory.
271 for cur in files_to_check:
272 checker = checkers[os.path.splitext(cur)[1]]
273 file_status = checker.CheckFile(rules, cur)
274 if file_status:
275 results.append("ERROR in " + cur + "\n" + file_status)
276
277 # Next recurse into the subdirectories.
278 for cur in dirs_to_check:
279 results.extend(self._CheckDirectoryImpl(rules, checkers, cur))
280 return results
281
282 def CheckAddedCppIncludes(self, added_includes):
283 """This is used from PRESUBMIT.py to check new #include statements added in
284 the change being presubmit checked.
285
286 Args:
287 added_includes: ((file_path, (include_line, include_line, ...), ...)
288
289 Return:
290 A list of tuples, (bad_file_path, rule_type, rule_description)
291 where rule_type is one of Rule.DISALLOW or Rule.TEMP_ALLOW and
292 rule_description is human-readable. Empty if no problems.
293 """
294 # Map of normalized directory paths to rules to use for those
295 # directories, or None for directories that should be skipped.
296 directory_rules = {}
297
298 def ApplyDirectoryRulesAndSkipSubdirs(parent_rules, dir_path):
299 rules_tuple = self._ApplyDirectoryRules(parent_rules, dir_path)
300 directory_rules[NormalizePath(dir_path)] = rules_tuple[0]
301 for subdir in rules_tuple[1]:
302 # We skip this one case for running tests.
303 directory_rules[NormalizePath(
304 os.path.normpath(os.path.join(dir_path, subdir)))] = None
305
306 ApplyDirectoryRulesAndSkipSubdirs(Rules(), self.base_directory)
307
308 def GetDirectoryRules(dir_path):
309 """Returns a Rules object to use for the given directory, or None
310 if the given directory should be skipped.
311 """
312 norm_dir_path = NormalizePath(dir_path)
313
314 if not dir_path.startswith(
315 NormalizePath(os.path.normpath(self.base_directory))):
316 dir_path = os.path.join(self.base_directory, dir_path)
317 norm_dir_path = NormalizePath(dir_path)
318
319 parent_dir = os.path.dirname(dir_path)
320 parent_rules = None
321 if not norm_dir_path in directory_rules:
322 parent_rules = GetDirectoryRules(parent_dir)
323
324 # We need to check for an entry for our dir_path again, in case we
325 # are at a path e.g. A/B/C where A/B/DEPS specifies the C
326 # subdirectory to be skipped; in this case, the invocation to
327 # GetDirectoryRules(parent_dir) has already filled in an entry for
328 # A/B/C.
329 if not norm_dir_path in directory_rules:
330 if not parent_rules:
331 # If the parent directory should be skipped, then the current
332 # directory should also be skipped.
333 directory_rules[norm_dir_path] = None
334 else:
335 ApplyDirectoryRulesAndSkipSubdirs(parent_rules, dir_path)
336 return directory_rules[norm_dir_path]
337
338 cpp = cpp_checker.CppChecker(self.verbose)
339
340 problems = []
341 for file_path, include_lines in added_includes:
342 # TODO(joi): Make this cover Java as well.
343 if not os.path.splitext(file_path)[1] in cpp.EXTENSIONS:
344 pass
345 rules_for_file = GetDirectoryRules(os.path.dirname(file_path))
346 if rules_for_file:
347 for line in include_lines:
348 is_include, line_status, rule_type = cpp.CheckLine(
349 rules_for_file, line, True)
350 if rule_type != Rule.ALLOW:
351 problems.append((file_path, rule_type, line_status))
352 return problems
353
354 def _AddGitSourceDirectories(self):
355 """Adds any directories containing sources managed by git to
356 self.git_source_directories.
357 """
358 if not os.path.exists(os.path.join(self.base_directory, ".git")):
359 return
360
361 popen_out = os.popen("cd %s && git ls-files --full-name ." %
362 subprocess.list2cmdline([self.base_directory]))
363 for line in popen_out.readlines():
364 dir_name = os.path.join(self.base_directory, os.path.dirname(line))
365 # Add the directory as well as all the parent directories. Use
366 # forward slashes and lower case to normalize paths.
367 while dir_name != self.base_directory:
368 self.git_source_directories.add(NormalizePath(dir_name))
369 dir_name = os.path.dirname(dir_name)
370 self.git_source_directories.add(NormalizePath(self.base_directory))
320 371
321 372
322 def PrintUsage(): 373 def PrintUsage():
323 print """Usage: python checkdeps.py [--root <root>] [tocheck] 374 print """Usage: python checkdeps.py [--root <root>] [tocheck]
375
324 --root Specifies the repository root. This defaults to "../../.." relative 376 --root Specifies the repository root. This defaults to "../../.." relative
325 to the script file. This will be correct given the normal location 377 to the script file. This will be correct given the normal location
326 of the script in "<root>/tools/checkdeps". 378 of the script in "<root>/tools/checkdeps".
327 379
328 tocheck Specifies the directory, relative to root, to check. This defaults 380 tocheck Specifies the directory, relative to root, to check. This defaults
329 to "." so it checks everything. Only one level deep is currently 381 to "." so it checks everything. Only one level deep is currently
330 supported, so you can say "chrome" but not "chrome/browser". 382 supported, so you can say "chrome" but not "chrome/browser".
331 383
332 Examples: 384 Examples:
333 python checkdeps.py 385 python checkdeps.py
334 python checkdeps.py --root c:\\source chrome""" 386 python checkdeps.py --root c:\\source chrome"""
335 387
336 388
337 def checkdeps(options, args):
338 global VERBOSE
339 if options.verbose:
340 VERBOSE = True
341
342 # Optional base directory of the repository.
343 global BASE_DIRECTORY
344 if not options.base_directory:
345 BASE_DIRECTORY = os.path.abspath(
346 os.path.join(os.path.abspath(os.path.dirname(__file__)), "../.."))
347 else:
348 BASE_DIRECTORY = os.path.abspath(options.base_directory)
349
350 # Figure out which directory we have to check.
351 if len(args) == 0:
352 # No directory to check specified, use the repository root.
353 start_dir = BASE_DIRECTORY
354 elif len(args) == 1:
355 # Directory specified. Start here. It's supposed to be relative to the
356 # base directory.
357 start_dir = os.path.abspath(os.path.join(BASE_DIRECTORY, args[0]))
358 else:
359 # More than one argument, we don't handle this.
360 PrintUsage()
361 return 1
362
363 print "Using base directory:", BASE_DIRECTORY
364 print "Checking:", start_dir
365
366 base_rules = Rules()
367
368 # The base directory should be lower case from here on since it will be used
369 # for substring matching on the includes, and we compile on case-insensitive
370 # systems. Plus, we always use slashes here since the include parsing code
371 # will also normalize to slashes.
372 BASE_DIRECTORY = BASE_DIRECTORY.lower()
373 BASE_DIRECTORY = BASE_DIRECTORY.replace("\\", "/")
374 start_dir = start_dir.replace("\\", "/")
375
376 if os.path.exists(os.path.join(BASE_DIRECTORY, ".git")):
377 global GIT_SOURCE_DIRECTORY
378 GIT_SOURCE_DIRECTORY = GetGitSourceDirectory(BASE_DIRECTORY)
379
380 java = java_checker.JavaChecker(BASE_DIRECTORY, VERBOSE)
381 cpp = cpp_checker.CppChecker(VERBOSE)
382 checkers = dict(
383 (extension, checker)
384 for checker in [java, cpp] for extension in checker.EXTENSIONS)
385 success = CheckDirectory(base_rules, checkers, start_dir)
386 if not success:
387 print "\nFAILED\n"
388 return 1
389 print "\nSUCCESS\n"
390 return 0
391
392
393 def main(): 389 def main():
394 option_parser = optparse.OptionParser() 390 option_parser = optparse.OptionParser()
395 option_parser.add_option("", "--root", default="", dest="base_directory", 391 option_parser.add_option("", "--root", default="", dest="base_directory",
396 help='Specifies the repository root. This defaults ' 392 help='Specifies the repository root. This defaults '
397 'to "../../.." relative to the script file, which ' 393 'to "../../.." relative to the script file, which '
398 'will normally be the repository root.') 394 'will normally be the repository root.')
399 option_parser.add_option("-v", "--verbose", action="store_true", 395 option_parser.add_option("-v", "--verbose", action="store_true",
400 default=False, help="Print debug logging") 396 default=False, help="Print debug logging")
401 options, args = option_parser.parse_args() 397 options, args = option_parser.parse_args()
402 return checkdeps(options, args) 398
399 deps_checker = DepsChecker(options.base_directory, options.verbose)
400
401 # Figure out which directory we have to check.
402 start_dir = deps_checker.base_directory
403 if len(args) == 1:
404 # Directory specified. Start here. It's supposed to be relative to the
405 # base directory.
406 start_dir = os.path.abspath(
407 os.path.join(deps_checker.base_directory, args[0]))
408 elif len(args) >= 2:
409 # More than one argument, we don't handle this.
410 PrintUsage()
411 return 1
412
413 print "Using base directory:", deps_checker.base_directory
414 print "Checking:", start_dir
415
416 results = deps_checker.CheckDirectory(start_dir)
417 if results:
418 for result in results:
419 print result
420 print "\nFAILED\n"
421 return 1
422 print "\nSUCCESS\n"
423 return 0
403 424
404 425
405 if '__main__' == __name__: 426 if '__main__' == __name__:
406 sys.exit(main()) 427 sys.exit(main())
OLDNEW
« no previous file with comments | « tools/checkdeps/PRESUBMIT.py ('k') | tools/checkdeps/checkdeps_test.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698