Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(4)

Side by Side Diff: third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/layout_tests_mover.py

Issue 1783073002: Run auto-formatter on files in webkitpy/layout_tests/. (Closed) Base URL: https://chromium.googlesource.com/chromium/src.git@master
Patch Set: Ran yapf -i --style '{based_on_style: pep8, column_limit: 132}' then did manual fix-up Created 4 years, 9 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
OLDNEW
1 # Copyright (C) 2013 Google Inc. All rights reserved. 1 # Copyright (C) 2013 Google Inc. All rights reserved.
2 # 2 #
3 # Redistribution and use in source and binary forms, with or without 3 # Redistribution and use in source and binary forms, with or without
4 # modification, are permitted provided that the following conditions are 4 # modification, are permitted provided that the following conditions are
5 # met: 5 # met:
6 # 6 #
7 # * Redistributions of source code must retain the above copyright 7 # * Redistributions of source code must retain the above copyright
8 # notice, this list of conditions and the following disclaimer. 8 # notice, this list of conditions and the following disclaimer.
9 # * Redistributions in binary form must reproduce the above 9 # * Redistributions in binary form must reproduce the above
10 # copyright notice, this list of conditions and the following disclaimer 10 # copyright notice, this list of conditions and the following disclaimer
11 # in the documentation and/or other materials provided with the 11 # in the documentation and/or other materials provided with the
12 # distribution. 12 # distribution.
13 # * Neither the name of Google Inc. nor the names of its 13 # * Neither the name of Google Inc. nor the names of its
14 # contributors may be used to endorse or promote products derived from 14 # contributors may be used to endorse or promote products derived from
15 # this software without specific prior written permission. 15 # this software without specific prior written permission.
16 # 16 #
17 # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 17 # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
18 # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 18 # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
19 # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 19 # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
20 # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 20 # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
21 # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 21 # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
22 # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 22 # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
23 # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 23 # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
24 # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 24 # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
25 # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 25 # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
26 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 26 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
27 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 27 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
28
29 """Moves a directory of LayoutTests. 28 """Moves a directory of LayoutTests.
30 29
31 Given a path to a directory of LayoutTests, moves that directory, including all recursive children, 30 Given a path to a directory of LayoutTests, moves that directory, including all recursive children,
32 to the specified destination path. Updates all references in tests and resources to reflect the new 31 to the specified destination path. Updates all references in tests and resources to reflect the new
33 location. Also moves any corresponding platform-specific expected results and up dates the test 32 location. Also moves any corresponding platform-specific expected results and up dates the test
34 expectations to reflect the move. 33 expectations to reflect the move.
35 34
36 If the destination directory does not exist, it and any missing parent directori es are created. If 35 If the destination directory does not exist, it and any missing parent directori es are created. If
37 the destination directory already exists, the child members of the origin direct ory are added to the 36 the destination directory already exists, the child members of the origin direct ory are added to the
38 destination directory. If any of the child members clash with existing members o f the destination 37 destination directory. If any of the child members clash with existing members o f the destination
(...skipping 10 matching lines...) Expand all
49 import re 48 import re
50 import urlparse 49 import urlparse
51 50
52 from webkitpy.common.checkout.scm.detection import SCMDetector 51 from webkitpy.common.checkout.scm.detection import SCMDetector
53 from webkitpy.common.host import Host 52 from webkitpy.common.host import Host
54 from webkitpy.common.system.executive import Executive 53 from webkitpy.common.system.executive import Executive
55 from webkitpy.common.system.filesystem import FileSystem 54 from webkitpy.common.system.filesystem import FileSystem
56 from webkitpy.layout_tests.port.base import Port 55 from webkitpy.layout_tests.port.base import Port
57 from webkitpy.layout_tests.models.test_expectations import TestExpectations 56 from webkitpy.layout_tests.models.test_expectations import TestExpectations
58 57
59
60 logging.basicConfig() 58 logging.basicConfig()
61 _log = logging.getLogger(__name__) 59 _log = logging.getLogger(__name__)
62 _log.setLevel(logging.INFO) 60 _log.setLevel(logging.INFO)
63 61
64 PLATFORM_DIRECTORY = 'platform' 62 PLATFORM_DIRECTORY = 'platform'
65 63
64
66 class LayoutTestsMover(object): 65 class LayoutTestsMover(object):
67
68 def __init__(self, port=None): 66 def __init__(self, port=None):
69 self._port = port 67 self._port = port
70 if not self._port: 68 if not self._port:
71 host = Host() 69 host = Host()
72 # Given that we use include_overrides=False and model_all_expectatio ns=True when 70 # Given that we use include_overrides=False and model_all_expectatio ns=True when
73 # constructing the TestExpectations object, it doesn't matter which Port object we use. 71 # constructing the TestExpectations object, it doesn't matter which Port object we use.
74 self._port = host.port_factory.get() 72 self._port = host.port_factory.get()
75 self._port.host.initialize_scm() 73 self._port.host.initialize_scm()
76 self._filesystem = self._port.host.filesystem 74 self._filesystem = self._port.host.filesystem
77 self._scm = self._port.host.scm() 75 self._scm = self._port.host.scm()
(...skipping 24 matching lines...) Expand all
102 if not self._is_child_path(self._layout_tests_root, self._absolute_desti nation): 100 if not self._is_child_path(self._layout_tests_root, self._absolute_desti nation):
103 raise Exception('Destination path %s is not in LayoutTests directory ' % self._destination) 101 raise Exception('Destination path %s is not in LayoutTests directory ' % self._destination)
104 102
105 # If destination is an existing directory, we move the children of origi n into destination. 103 # If destination is an existing directory, we move the children of origi n into destination.
106 # However, if any of the children of origin would clash with existing ch ildren of 104 # However, if any of the children of origin would clash with existing ch ildren of
107 # destination, we fail. 105 # destination, we fail.
108 # FIXME: Consider adding support for recursively moving into an existing directory. 106 # FIXME: Consider adding support for recursively moving into an existing directory.
109 if self._filesystem.isdir(self._absolute_destination): 107 if self._filesystem.isdir(self._absolute_destination):
110 for file_path in self._filesystem.listdir(self._absolute_origin): 108 for file_path in self._filesystem.listdir(self._absolute_origin):
111 if self._filesystem.exists(self._filesystem.join(self._absolute_ destination, file_path)): 109 if self._filesystem.exists(self._filesystem.join(self._absolute_ destination, file_path)):
112 raise Exception('Origin path %s clashes with existing destin ation path %s' % 110 raise Exception(
113 (self._filesystem.join(self._origin, file_path), sel f._filesystem.join(self._destination, file_path))) 111 'Origin path %s clashes with existing destination path % s' %
112 (self._filesystem.join(self._origin, file_path), self._f ilesystem.join(self._destination, file_path)))
114 113
115 def _get_expectations_for_test(self, model, test_path): 114 def _get_expectations_for_test(self, model, test_path):
116 """Given a TestExpectationsModel object, finds all expectations that mat ch the specified 115 """Given a TestExpectationsModel object, finds all expectations that mat ch the specified
117 test, specified as a relative path. Handles the fact that expectations m ay be keyed by 116 test, specified as a relative path. Handles the fact that expectations m ay be keyed by
118 directory. 117 directory.
119 """ 118 """
120 expectations = set() 119 expectations = set()
121 if model.has_test(test_path): 120 if model.has_test(test_path):
122 expectations.add(model.get_expectation_line(test_path)) 121 expectations.add(model.get_expectation_line(test_path))
123 test_path = self._filesystem.dirname(test_path) 122 test_path = self._filesystem.dirname(test_path)
124 while not test_path == '': 123 while not test_path == '':
125 # The model requires a trailing slash for directories. 124 # The model requires a trailing slash for directories.
126 test_path_for_model = test_path + '/' 125 test_path_for_model = test_path + '/'
127 if model.has_test(test_path_for_model): 126 if model.has_test(test_path_for_model):
128 expectations.add(model.get_expectation_line(test_path_for_model) ) 127 expectations.add(model.get_expectation_line(test_path_for_model) )
129 test_path = self._filesystem.dirname(test_path) 128 test_path = self._filesystem.dirname(test_path)
130 return expectations 129 return expectations
131 130
132 def _get_expectations(self, model, path): 131 def _get_expectations(self, model, path):
133 """Given a TestExpectationsModel object, finds all expectations for all tests under the 132 """Given a TestExpectationsModel object, finds all expectations for all tests under the
134 specified relative path. 133 specified relative path.
135 """ 134 """
136 expectations = set() 135 expectations = set()
137 for test in self._filesystem.files_under(self._filesystem.join(self._lay out_tests_root, path), dirs_to_skip=['script-tests', 'resources'], 136 for test in self._filesystem.files_under(
138 file_filter=Port.is_test_file): 137 self._filesystem.join(self._layout_tests_root, path),
139 expectations = expectations.union(self._get_expectations_for_test(mo del, self._filesystem.relpath(test, self._layout_tests_root))) 138 dirs_to_skip=['script-tests', 'resources'],
139 file_filter=Port.is_test_file):
140 expectations = expectations.union(self._get_expectations_for_test(mo del, self._filesystem.relpath(
141 test, self._layout_tests_root)))
140 return expectations 142 return expectations
141 143
142 @staticmethod 144 @staticmethod
143 def _clone_expectation_line_for_path(expectation_line, path): 145 def _clone_expectation_line_for_path(expectation_line, path):
144 """Clones a TestExpectationLine object and updates the clone to apply to the specified 146 """Clones a TestExpectationLine object and updates the clone to apply to the specified
145 relative path. 147 relative path.
146 """ 148 """
147 clone = copy.copy(expectation_line) 149 clone = copy.copy(expectation_line)
148 clone.original_string = re.compile(expectation_line.name).sub(path, expe ctation_line.original_string) 150 clone.original_string = re.compile(expectation_line.name).sub(path, expe ctation_line.original_string)
149 clone.name = path 151 clone.name = path
(...skipping 21 matching lines...) Expand all
171 else: 173 else:
172 # If the existing expectation is not a child of the moved path, we have to leave it 174 # If the existing expectation is not a child of the moved path, we have to leave it
173 # in place. But we also add a new expectation for the destinatio n path. 175 # in place. But we also add a new expectation for the destinatio n path.
174 new_path = self._destination 176 new_path = self._destination
175 _log.warning('Copying expectation for %s to %s. You should check that these expectations are still correct.' % 177 _log.warning('Copying expectation for %s to %s. You should check that these expectations are still correct.' %
176 (path, new_path)) 178 (path, new_path))
177 test_expectations.add_expectation_line(LayoutTestsMover._clone_e xpectation_line_for_path(expectation, new_path)) 179 test_expectations.add_expectation_line(LayoutTestsMover._clone_e xpectation_line_for_path(expectation, new_path))
178 180
179 expectations_file = self._port.path_to_generic_test_expectations_file() 181 expectations_file = self._port.path_to_generic_test_expectations_file()
180 self._filesystem.write_text_file(expectations_file, 182 self._filesystem.write_text_file(expectations_file,
181 TestExpectations.list_to_string(test_ex pectations._expectations, reconstitute_only_these=[])) 183 TestExpectations.list_to_string(test_ex pectations._expectations,
184 reconst itute_only_these=[]))
182 self._scm.add(self._filesystem.relpath(expectations_file, self._scm.chec kout_root)) 185 self._scm.add(self._filesystem.relpath(expectations_file, self._scm.chec kout_root))
183 186
184 def _find_references(self, input_files): 187 def _find_references(self, input_files):
185 """Attempts to find all references to other files in the supplied list o f files. Returns a 188 """Attempts to find all references to other files in the supplied list o f files. Returns a
186 dictionary that maps from an absolute file path to an array of reference strings. 189 dictionary that maps from an absolute file path to an array of reference strings.
187 """ 190 """
188 reference_regex = re.compile(r'(?:(?:src=|href=|importScripts\(|url\()(? :"([^"]+)"|\'([^\']+)\')|url\(([^\)\'"]+)\))') 191 reference_regex = re.compile(r'(?:(?:src=|href=|importScripts\(|url\()(? :"([^"]+)"|\'([^\']+)\')|url\(([^\)\'"]+)\))')
189 references = {} 192 references = {}
190 for input_file in input_files: 193 for input_file in input_files:
191 matches = reference_regex.findall(self._filesystem.read_binary_file( input_file)) 194 matches = reference_regex.findall(self._filesystem.read_binary_file( input_file))
(...skipping 12 matching lines...) Expand all
204 # Both the root path and the target of the reference my be subject to th e move, so there are 207 # Both the root path and the target of the reference my be subject to th e move, so there are
205 # four cases to consider. In the case where both or neither are subject to the move, the 208 # four cases to consider. In the case where both or neither are subject to the move, the
206 # reference doesn't need updating. 209 # reference doesn't need updating.
207 # 210 #
208 # This is true even if the reference includes superfluous dot segments w hich mention a moved 211 # This is true even if the reference includes superfluous dot segments w hich mention a moved
209 # directory, as dot segments are collapsed during URL normalization. For example, if 212 # directory, as dot segments are collapsed during URL normalization. For example, if
210 # foo.html contains a reference 'bar/../script.js', this remains valid ( though ugly) even if 213 # foo.html contains a reference 'bar/../script.js', this remains valid ( though ugly) even if
211 # bar/ is moved to baz/, because the reference is always normalized to ' script.js'. 214 # bar/ is moved to baz/, because the reference is always normalized to ' script.js'.
212 absolute_reference = self._filesystem.normpath(self._filesystem.join(roo t, reference)) 215 absolute_reference = self._filesystem.normpath(self._filesystem.join(roo t, reference))
213 if self._is_child_path(self._absolute_origin, root) == self._is_child_pa th(self._absolute_origin, absolute_reference): 216 if self._is_child_path(self._absolute_origin, root) == self._is_child_pa th(self._absolute_origin, absolute_reference):
214 return None; 217 return None
215 218
216 new_root = self._move_path(root, self._absolute_origin, self._absolute_d estination) 219 new_root = self._move_path(root, self._absolute_origin, self._absolute_d estination)
217 new_absolute_reference = self._move_path(absolute_reference, self._absol ute_origin, self._absolute_destination) 220 new_absolute_reference = self._move_path(absolute_reference, self._absol ute_origin, self._absolute_destination)
218 return self._filesystem.relpath(new_absolute_reference, new_root) 221 return self._filesystem.relpath(new_absolute_reference, new_root)
219 222
220 def _get_all_updated_references(self, references): 223 def _get_all_updated_references(self, references):
221 """Determines the updated references due to the move. Returns a dictiona ry that maps from an 224 """Determines the updated references due to the move. Returns a dictiona ry that maps from an
222 absolute file path to a dictionary that maps from a reference string to the corresponding 225 absolute file path to a dictionary that maps from a reference string to the corresponding
223 updated reference. 226 updated reference.
224 """ 227 """
(...skipping 21 matching lines...) Expand all
246 for target in updates.keys(): 249 for target in updates.keys():
247 regex = re.compile(r'((?:src=|href=|importScripts\(|url\()["\']?)%s( ["\']?)' % target) 250 regex = re.compile(r'((?:src=|href=|importScripts\(|url\()["\']?)%s( ["\']?)' % target)
248 contents = regex.sub(r'\1%s\2' % updates[target], contents) 251 contents = regex.sub(r'\1%s\2' % updates[target], contents)
249 self._filesystem.write_binary_file(path, contents) 252 self._filesystem.write_binary_file(path, contents)
250 self._scm.add(path) 253 self._scm.add(path)
251 254
252 def _update_test_source_files(self): 255 def _update_test_source_files(self):
253 def is_test_source_file(filesystem, dirname, basename): 256 def is_test_source_file(filesystem, dirname, basename):
254 pass_regex = re.compile(r'\.(css|js)$') 257 pass_regex = re.compile(r'\.(css|js)$')
255 fail_regex = re.compile(r'-expected\.') 258 fail_regex = re.compile(r'-expected\.')
256 return (Port.is_test_file(filesystem, dirname, basename) or pass_reg ex.search(basename)) and not fail_regex.search(basename) 259 return (Port.is_test_file(filesystem, dirname, basename) or
260 pass_regex.search(basename)) and not fail_regex.search(basen ame)
257 261
258 test_source_files = self._filesystem.files_under(self._layout_tests_root , file_filter=is_test_source_file) 262 test_source_files = self._filesystem.files_under(self._layout_tests_root , file_filter=is_test_source_file)
259 _log.info('Considering %s test source files for references' % len(test_s ource_files)) 263 _log.info('Considering %s test source files for references' % len(test_s ource_files))
260 references = self._find_references(test_source_files) 264 references = self._find_references(test_source_files)
261 _log.info('Considering references in %s files' % len(references)) 265 _log.info('Considering references in %s files' % len(references))
262 updates = self._get_all_updated_references(references) 266 updates = self._get_all_updated_references(references)
263 _log.info('Updating references in %s files' % len(updates)) 267 _log.info('Updating references in %s files' % len(updates))
264 count = 0 268 count = 0
265 for file_path in updates.keys(): 269 for file_path in updates.keys():
266 self._update_file(file_path, updates[file_path]) 270 self._update_file(file_path, updates[file_path])
(...skipping 16 matching lines...) Expand all
283 for directory in self._filesystem.listdir(absolute_origin): 287 for directory in self._filesystem.listdir(absolute_origin):
284 self._scm.move(self._scm_path(origin, directory), self._scm_path(des tination, directory)) 288 self._scm.move(self._scm_path(origin, directory), self._scm_path(des tination, directory))
285 self._filesystem.rmtree(absolute_origin) 289 self._filesystem.rmtree(absolute_origin)
286 290
287 def _move_files(self): 291 def _move_files(self):
288 """Moves the all files that correspond to the move, including platform-s pecific expected 292 """Moves the all files that correspond to the move, including platform-s pecific expected
289 results. 293 results.
290 """ 294 """
291 self._move_directory(self._origin, self._destination) 295 self._move_directory(self._origin, self._destination)
292 for directory in self._filesystem.listdir(self._filesystem.join(self._la yout_tests_root, PLATFORM_DIRECTORY)): 296 for directory in self._filesystem.listdir(self._filesystem.join(self._la yout_tests_root, PLATFORM_DIRECTORY)):
293 self._move_directory(self._filesystem.join(PLATFORM_DIRECTORY, direc tory, self._origin), 297 self._move_directory(
294 self._filesystem.join(PLATFORM_DIRECTORY, directory, self._destination)) 298 self._filesystem.join(PLATFORM_DIRECTORY, directory, self._origi n),
299 self._filesystem.join(PLATFORM_DIRECTORY, directory, self._desti nation))
295 300
296 def _commit_changes(self): 301 def _commit_changes(self):
297 if not self._scm.supports_local_commits(): 302 if not self._scm.supports_local_commits():
298 return 303 return
299 title = 'Move LayoutTests directory %s to %s' % (self._origin, self._des tination) 304 title = 'Move LayoutTests directory %s to %s' % (self._origin, self._des tination)
300 _log.info('Committing change \'%s\'' % title) 305 _log.info('Committing change \'%s\'' % title)
301 self._scm.commit_locally_with_message('%s\n\nThis commit was automatical ly generated by move-layout-tests.' % title, 306 self._scm.commit_locally_with_message('%s\n\nThis commit was automatical ly generated by move-layout-tests.' % title,
302 commit_all_working_directory_chang es=False) 307 commit_all_working_directory_chang es=False)
303 308
304 def move(self, origin, destination): 309 def move(self, origin, destination):
305 self._origin = origin 310 self._origin = origin
306 self._destination = destination 311 self._destination = destination
307 self._absolute_origin = self._filesystem.join(self._layout_tests_root, s elf._origin) 312 self._absolute_origin = self._filesystem.join(self._layout_tests_root, s elf._origin)
308 self._absolute_destination = self._filesystem.join(self._layout_tests_ro ot, self._destination) 313 self._absolute_destination = self._filesystem.join(self._layout_tests_ro ot, self._destination)
309 self._validate_input() 314 self._validate_input()
310 self._update_expectations() 315 self._update_expectations()
311 self._update_test_source_files() 316 self._update_test_source_files()
312 self._move_files() 317 self._move_files()
313 # FIXME: Handle virtual test suites. 318 # FIXME: Handle virtual test suites.
314 self._commit_changes() 319 self._commit_changes()
315 320
321
316 def main(argv): 322 def main(argv):
317 parser = optparse.OptionParser(description=__doc__) 323 parser = optparse.OptionParser(description=__doc__)
318 parser.add_option('--origin', 324 parser.add_option('--origin', help=('The directory of tests to move, as a re lative path from the LayoutTests directory.'))
319 help=('The directory of tests to move, as a relative path from the LayoutTests directory.'))
320 parser.add_option('--destination', 325 parser.add_option('--destination',
321 help=('The new path for the directory of tests, as a relat ive path from the LayoutTests directory.')) 326 help=('The new path for the directory of tests, as a relat ive path from the LayoutTests directory.'))
322 options, _ = parser.parse_args() 327 options, _ = parser.parse_args()
323 LayoutTestsMover().move(options.origin, options.destination) 328 LayoutTestsMover().move(options.origin, options.destination)
OLDNEW

Powered by Google App Engine
This is Rietveld 408576698