Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(11)

Side by Side Diff: third_party/WebKit/Tools/Scripts/webkitpy/w3c/update_w3c_test_expectations.py

Issue 2604213002: When auto-importing, don't limit rebaselining to directly modified tests. (Closed)
Patch Set: Update test Created 3 years, 11 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | third_party/WebKit/Tools/Scripts/webkitpy/w3c/update_w3c_test_expectations_unittest.py » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # Copyright 2016 The Chromium Authors. All rights reserved. 1 # Copyright 2016 The Chromium Authors. All rights reserved.
2 # Use of this source code is governed by a BSD-style license that can be 2 # Use of this source code is governed by a BSD-style license that can be
3 # found in the LICENSE file. 3 # found in the LICENSE file.
4 4
5 """A class for updating layout test expectations when updating w3c tests. 5 """A class for updating layout test expectations when updating w3c tests.
6 6
7 Specifically, this class fetches results from try bots for the current CL, and: 7 Specifically, this class fetches results from try bots for the current CL, and:
8 1. Downloads new baseline files for any tests that can be rebaselined. 8 1. Downloads new baseline files for any tests that can be rebaselined.
9 2. Updates the generic TestExpectations file for any other failing tests. 9 2. Updates the generic TestExpectations file for any other failing tests.
10 10
(...skipping 311 matching lines...) Expand 10 before | Expand all | Expand 10 after
322 failure test dictionary. 322 failure test dictionary.
323 323
324 Args: 324 Args:
325 tests_results: A dict mapping test name to platform to test results. 325 tests_results: A dict mapping test name to platform to test results.
326 326
327 Returns: 327 Returns:
328 An updated tests_results dictionary without the platform-specific 328 An updated tests_results dictionary without the platform-specific
329 testharness.js tests that required new baselines to be downloaded 329 testharness.js tests that required new baselines to be downloaded
330 from `webkit-patch rebaseline-cl`. 330 from `webkit-patch rebaseline-cl`.
331 """ 331 """
332 modified_tests = self.get_modified_existing_tests() 332 tests_to_rebaseline, tests_results = self.get_tests_to_rebaseline(tests_ results)
333 tests_to_rebaseline, tests_results = self.get_tests_to_rebaseline(modifi ed_tests, tests_results)
334 _log.debug('Tests to rebaseline: %r', tests_to_rebaseline) 333 _log.debug('Tests to rebaseline: %r', tests_to_rebaseline)
335 if tests_to_rebaseline: 334 if tests_to_rebaseline:
336 webkit_patch = self.host.filesystem.join( 335 webkit_patch = self.host.filesystem.join(
337 self.finder.chromium_base(), self.finder.webkit_base(), self.fin der.path_to_script('webkit-patch')) 336 self.finder.chromium_base(), self.finder.webkit_base(), self.fin der.path_to_script('webkit-patch'))
338 self.host.executive.run_command([ 337 self.host.executive.run_command([
339 'python', 338 'python',
340 webkit_patch, 339 webkit_patch,
341 'rebaseline-cl', 340 'rebaseline-cl',
342 '--verbose', 341 '--verbose',
343 '--no-trigger-jobs', 342 '--no-trigger-jobs',
344 ] + tests_to_rebaseline) 343 ] + tests_to_rebaseline)
345 return tests_results 344 return tests_results
346 345
347 def get_modified_existing_tests(self): 346 def get_tests_to_rebaseline(self, test_results):
348 """Returns a list of layout test names for layout tests that have been m odified."""
349 diff_output = self.host.executive.run_command(
350 ['git', 'diff', 'origin/master', '--name-only', '--diff-filter=AMR'] ) # Added, modified, and renamed files.
351 paths_from_chromium_root = diff_output.splitlines()
352 modified_tests = []
353 for path in paths_from_chromium_root:
354 absolute_path = self.host.filesystem.join(self.finder.chromium_base( ), path)
355 if not self.host.filesystem.exists(absolute_path):
356 _log.warning('File does not exist: %s', absolute_path)
357 continue
358 test_path = self.finder.layout_test_name(path)
359 if test_path:
360 modified_tests.append(test_path)
361 return modified_tests
362
363 def get_tests_to_rebaseline(self, modified_tests, test_results):
364 """Returns a list of tests to download new baselines for. 347 """Returns a list of tests to download new baselines for.
365 348
366 Creates a list of tests to rebaseline depending on the tests' platform- 349 Creates a list of tests to rebaseline depending on the tests' platform-
367 specific results. In general, this will be non-ref tests that failed 350 specific results. In general, this will be non-ref tests that failed
368 due to a baseline mismatch (rather than crash or timeout). 351 due to a baseline mismatch (rather than crash or timeout).
369 352
370 Args: 353 Args:
371 modified_tests: A list of paths to modified files (which should 354 test_results: A dictionary of failing test results, mapping tests
372 be added, removed or modified files in the imported w3c 355 to platforms to result dicts.
373 directory), relative to the LayoutTests directory.
374 test_results: A dictionary of failing tests results.
375 356
376 Returns: 357 Returns:
377 A pair: A set of tests to be rebaselined, and a modified copy of 358 A pair: A set of tests to be rebaselined, and a modified copy of
378 the test results dictionary. The tests to be rebaselined should 359 the test results dictionary. The tests to be rebaselined should
379 include testharness.js tests that failed due to a baseline mismatch. 360 include testharness.js tests that failed due to a baseline mismatch.
380 """ 361 """
381 test_results = copy.deepcopy(test_results) 362 test_results = copy.deepcopy(test_results)
382 tests_to_rebaseline = set() 363 tests_to_rebaseline = set()
383 for test_path in modified_tests: 364 for test_path in test_results:
384 if not (self.is_js_test(test_path) and test_results.get(test_path)): 365 if not (self.is_js_test(test_path) and test_results.get(test_path)):
385 continue 366 continue
386 for platform in test_results[test_path].keys(): 367 for platform in test_results[test_path].keys():
387 if test_results[test_path][platform]['actual'] not in ['CRASH', 'TIMEOUT']: 368 if test_results[test_path][platform]['actual'] not in ['CRASH', 'TIMEOUT']:
388 del test_results[test_path][platform] 369 del test_results[test_path][platform]
389 tests_to_rebaseline.add(test_path) 370 tests_to_rebaseline.add(test_path)
390 return sorted(tests_to_rebaseline), test_results 371 return sorted(tests_to_rebaseline), test_results
391 372
392 def is_js_test(self, test_path): 373 def is_js_test(self, test_path):
393 """Checks whether a given file is a testharness.js test. 374 """Checks whether a given file is a testharness.js test.
394 375
395 Args: 376 Args:
396 test_path: A file path relative to the layout tests directory. 377 test_path: A file path relative to the layout tests directory.
397 This might correspond to a deleted file or a non-test. 378 This might correspond to a deleted file or a non-test.
398 """ 379 """
399 absolute_path = self.host.filesystem.join(self.finder.layout_tests_dir() , test_path) 380 absolute_path = self.host.filesystem.join(self.finder.layout_tests_dir() , test_path)
400 test_parser = TestParser(absolute_path, self.host) 381 test_parser = TestParser(absolute_path, self.host)
401 if not test_parser.test_doc: 382 if not test_parser.test_doc:
402 return False 383 return False
403 return test_parser.is_jstest() 384 return test_parser.is_jstest()
OLDNEW
« no previous file with comments | « no previous file | third_party/WebKit/Tools/Scripts/webkitpy/w3c/update_w3c_test_expectations_unittest.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698