WebRTC: Bot for autorolling WebRTC+libjingle in Chromium DEPS.

scripts/slave/recipes/webrtc/auto_roll_chromium_deps.py

-# Copyright 2014 The Chromium Authors. All rights reserved.
+# Copyright 2015 The Chromium Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 
 DEPS = [
   'bot_update',
   'chromium',
   'gclient',
   'json',
   'path',
   'properties',
   'python',
   'raw_io',
   'step',
   'url',
 ]
 
 
 def ContainsChromiumRoll(changes):
   for change in changes:
-    if change['subject'].startswith('Update V8 to'):
+    if change['subject'].startswith('Roll WebRTC '):
       return True
   return False
 
 
 def RunSteps(api):
   api.chromium.cleanup_temp()
   api.gclient.set_config('chromium')
-  api.gclient.apply_config('v8_bleeding_edge_git')

[kjellander_chromium, 2015/10/05 19:35:47]

Our script at
https://code.google.com/p/chromium/codesearch#chromium/src/tools/roll_webrtc.py
ensures the directories to run roll-dep-svn in are up to date using 'git fetch
origin'.

 
   step_result = api.python(
       'check roll status',
       api.path['build'].join('scripts', 'tools', 'runit.py'),
       [api.path['build'].join('scripts', 'tools', 'pycurl.py'),
-       'https://v8-roll.appspot.com/status'],
+       'https://webrtc-roll.appspot.com/status'],
       stdout=api.raw_io.output(),
       step_test_data=lambda: api.raw_io.test_api.stream_output(
           '1', stream='stdout')
   )
   step_result.presentation.logs['stdout'] = step_result.stdout.splitlines()
   if step_result.stdout.strip() != '1':
     step_result.presentation.step_text = 'Rolling deactivated'
     return
   else:
     step_result.presentation.step_text = 'Rolling activated'
 
   params = {
     'closed': 3,
-    'owner': 'v8-autoroll@chromium.org',
+    'owner': 'webrtc-autoroll@chromium.org',

[Michael Achenbach, 2015/10/06 09:31:55]

You have an account now?

[kjellander_chromium, 2015/10/06 09:58:09]

No, not yet. I might try this with my own account while waiting for that one.

[kjellander_chromium, 2015/10/26 08:40:29]

Now I have an account (crbug.com/539527).

[Michael Achenbach, 2015/10/26 08:54:38]

Acknowledged. FYI: You might need to log on to the bot once and log in to
rietveld with this account so that credentials are stored.

You might also want to add a git config for that account on the bot. Otherwise
your CL author might end up being called chrome-bot.

     'limit': 30,
     'format': 'json',
   }
 
   params = api.url.urlencode(params)
   search_url = 'https://codereview.chromium.org/search?' + params
 
   result = api.url.fetch(
       search_url,
       'check active roll',
       step_test_data=lambda: api.raw_io.test_api.output('{"results": []}')
   )
   if ContainsChromiumRoll(api.json.loads(result)['results']):
     api.step.active_result.presentation.step_text = 'Active rolls found.'
     return
 
   # Prevent race with gnumbd by waiting.
   api.python.inline(
       'wait for gnumbd',
       'import time; time.sleep(20)',

[Michael Achenbach, 2015/10/06 09:31:55]

Not sure if you need that, but it doesn't hurt much. In our case without waiting
we can run into:

1. CQ landing a roll (and closing the issue).
2. gnumbd not processing it yet
3. Auto roller checking for active roll and trying to roll the same again.
4. gnumbd processes the first roll finally.
5. The new roll creates patch failures on the CQ

But even with waiting, this can happen in a gnumbd outage.

[kjellander_chromium, 2015/10/06 09:58:09]

As you say it doesn't hurt so I'll keep it for initial tests.

[Michael Achenbach, 2015/10/26 08:54:38]

Actually you probably do need it. Also here a race can happen when CQ closes the
issue and this script passes the open-issue check, but gnumbd is slow and does
not yet stamp the last roll.

In case of gnumbd outages or occasional hiccups you might still get the problem.
The result would be a duped roll attempt that causes patch failures on the
trybots. Easy to fix by just closing the CL, but still annoying.

I wanted to work on a real fix at some point, which would be checking the
pending/heads/master branch for the currently rolled version.

   )
 
   api.bot_update.ensure_checkout(force=True, no_shallow=True)
 
   api.python(
       'roll deps',
-      api.path['checkout'].join(
-          'v8', 'tools', 'release', 'auto_roll.py'),
-      ['--chromium', api.path['checkout'],
-       '--author', 'v8-autoroll@chromium.org',
-       '--reviewer',
-       'hablich@chromium.org,machenbach@chromium.org,'
-       'yangguo@chromium.org,vogelheim@chromium.org',
-       '--roll',
-       '--work-dir', api.path['slave_build'].join('workdir')],
-      cwd=api.path['checkout'].join('v8'),
-    )
+      api.path['checkout'].join('tools', 'roll_webrtc.py'),
+      ['--wait-for-trybots', '--close-previous-roll', '--verbose'],
+      cwd=api.path['checkout'],

[Michael Achenbach, 2015/10/06 09:31:55]

Sure this works? This is from a chromium checkout.

[kjellander_chromium, 2015/10/06 09:58:09]

Yeah, we moved the script from WebRTC into Chromium a while back:
https://code.google.com/p/chromium/codesearch#chromium/src/tools/roll_webrtc.py

I'm not entirely sure about the --wait-for-trybots thing since I don't think
anyone is using it but I wrote it with a bot like this in mind a while back, so
I'll give it a try. With that the build would go on until the CL is landed
instead of checking back over and over (resulting in many no-op builds, which
makes the overview harder to read).

[kjellander_chromium, 2015/10/26 08:40:29]

It's worth mentioning that this will wait for max 1 hour, and that will output
to stdio every 30 seconds so the build step shouldn't die. I might take this
away if it doesn't work but I like the idea about having less no-op builds that
do nothing, which is why I wanted to at least give it a try.

[Michael Achenbach, 2015/10/26 08:54:38]

Acknowledged.

+  )
 
 
 def GenTests(api):
-  yield api.test('standard') + api.properties.generic(
-      mastername='client.v8.fyi')
+  yield (api.test('standard') +
+      api.properties.generic(mastername='client.webrtc.fyi')
+  )
   yield (api.test('rolling_deactivated') +
-      api.properties.generic(mastername='client.v8') +
+      api.properties.generic(mastername='client.webrtc.fyi') +
       api.override_step_data(
           'check roll status', api.raw_io.stream_output('0', stream='stdout'))
-    )
+  )
   yield (api.test('active_roll') +
-      api.properties.generic(mastername='client.v8') +
+      api.properties.generic(mastername='client.webrtc.fyi') +
       api.override_step_data(
           'check active roll', api.raw_io.output(
-              '{"results": [{"subject": "Update V8 to foo"}]}'))
-    )
-
+              '{"results": [{"subject": "Roll WebRTC to 123"}]}'))
+  )