Call update-w3c-test-expectations in deps_updater.py

third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py

 # Copyright 2014 The Chromium Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 
 """Pull latest revisions of a W3C test repo and make a local commit."""
 
 import argparse
 import time
 
 from webkitpy.common.webkit_finder import WebKitFinder
@@ skipping 45 matching lines @@
                 self.run(['git', 'add', destination])
 
         elif self.target == 'css':
             import_commitish = self.update(
                 CSS_DEST_NAME,
                 'https://chromium.googlesource.com/external/w3c/csswg-test.git')
         else:
             raise AssertionError("Unsupported target %s" % self.target)
 
         self.commit_changes_if_needed(chromium_commitish, import_commitish)
-
+        # TODO(dcampb): Change try bots to run when new try bots are added.
         if self.auto_update:
-            self.check_run(['git', 'cl', 'upload', '-f'])

[dcampb, 2016/07/25 17:09:27]

note,

this flag uploads a cl even if no new test were imported. However, some
dependencies could be removed from the update. Is that okay?

-            self.run(['git', 'cl', 'set-commit', '--dry-run'])
+            self.print_('## Uploading change list.')
+            self.check_run(['git', 'cl', 'upload', '-f', '-m', 'W3C auto test importer'])
+            self.print_('## Triggering try jobs.')
+            self.run(['git', 'cl', 'try', '-b', 'linux_chromium_rel_ng'])
+            self.run(['git', 'cl', 'try', '-b', 'mac_chromium_rel_ng'])
+            self.run(['git', 'cl', 'try', '-b', 'win_chromium_rel_ng'])

[qyearsley, 2016/07/25 21:57:34]

As mentioned in person, we will want to get the list of try bots from a
BuilderList object.

[dcampb, 2016/07/26 21:18:10]

done

+            self.print_('## Waiting for Try Job Results')
+            has_failing_results = False
             while True:
                 time.sleep(POLL_DELAY_SECONDS)
                 _, out = self.run(['git', 'cl', 'try-results'])
                 results = self.parse_try_job_results(out)
-                if results['Started'] or results['Scheduled']:
+                if results.get('Started') or results.get('Scheduled'):
                     continue
-                if results['Failures']:
-                    return 1
+                if results.get('Failures'):
+                    has_failing_results = True
                 break
+            if has_failing_results:
+                self.print_('## Adding test expectations lines to LayoutTests/TestExpectations')
+                script_path = self.path_from_webkit_base('Tools', 'Scripts', 'update-w3c-test-expectations')
+                self.run([self.host.executable, script_path])

[dcampb, 2016/07/25 17:09:27]

This will add lines to Test Expectations IF the test does not already exist in
Test Expectations file. 
Also, It does not only add test files that were imported, but also test that
failed during try-jobs. 
What do you all think about this?

[qyearsley, 2016/07/25 21:57:34]

Seems like it should only ever add tests that were imported.

Additionally, for testharness.js tests that fail (but not crash/timeout), I
believe in the future we probably always want to add the new -expected.txt
baseline files rather than adding lines to TestExpectations -- this could either
be done in update_w3c_test_expectations.py (using the downloaded results from
the try bots), or potentially this could be done by `webkit-patch
rebaseline-from-try-jobs.

[dcampb, 2016/07/26 21:18:09]

Fixed only adding new imported test. See issue #:2183913002. 
As for the testharness.js test, I am still looking into that. Where
files/directory should I start to investigate in?

+                self.check_run(['git', 'commit', '-a', '-m', '\'Modified Test Expectations from W3C_Test_AutoRoller\''])

[qyearsley, 2016/07/25 21:57:34]

W3C_Test_Autoroller -> W3C Test Auto-roller?

[dcampb, 2016/07/26 21:18:09]

done

+                self.check_run(['git', 'cl', 'upload', '-m', '\'Wrote lines to Test Expectations\''])

[qyearsley, 2016/07/25 21:57:34]

Test Expectations -> TestExpectations

[dcampb, 2016/07/26 21:18:09]

done

+            else:
+                self.print_('No Failures, committing patch.')
             self.run(['git', 'cl', 'land', '-f'])
         return 0
 
     def parse_args(self, argv):
         parser = argparse.ArgumentParser()
         parser.description = __doc__
         parser.add_argument('-v', '--verbose', action='store_true',
                             help='log what we are doing')
         parser.add_argument('--allow-local-commits', action='store_true',
                             help='allow script to run even if we have local commits')
@@ skipping 185 matching lines @@
         sets = {}
         for line in results.splitlines():
             line = line.strip()
             if line[-1] == ':':
                 result_type = line[:-1]
                 sets[result_type] = set()
             elif line.split()[0] == 'Total:':
                 break
             else:
                 sets[result_type].add(line.split()[0])
-        sets['Failures'] -= sets['Successes']
-        sets['Started'] -= sets['Successes']
-        sets['Started'] -= sets['Failures']
         return sets
 
     def check_run(self, command):
         return_code, out = self.run(command)
         if return_code:
             raise Exception('%s failed with exit code %d.' % ' '.join(command), return_code)
         return out