Use script in depot_tools to retrieve builders from new cq.cfg

PRESUBMIT.py

Index: PRESUBMIT.py
diff --git a/PRESUBMIT.py b/PRESUBMIT.py
index 751ae525751be38445e4bca2fd0a02c0ac7dd03e..ad8f3753fffa9c41d3cd1f3845bfccfee215a7e4 100644
--- a/PRESUBMIT.py
+++ b/PRESUBMIT.py
@@ -1784,29 +1784,28 @@ def CheckChangeOnCommit(input_api, output_api):
 
 
 def GetPreferredTryMasters(project, change):
-  import re
-  files = change.LocalPaths()
-
-  import os
   import json
-  with open(os.path.join(
-      change.RepositoryRoot(), 'testing', 'commit_queue', 'config.json')) as f:
-    cq_config = json.load(f)
-    cq_verifiers = cq_config.get('verifiers_no_patch', {})
-    cq_try_jobs = cq_verifiers.get('try_job_verifier', {})
-    builders = cq_try_jobs.get('launched', {})
-
-    for master, master_config in cq_try_jobs.get('triggered', {}).iteritems():
-      for triggered_bot in master_config:
-        builders.get(master, {}).pop(triggered_bot, None)
-
-    # Explicitly iterate over copies of dicts since we mutate them.
-    for master in builders.keys():
-      for builder in builders[master].keys():
-        # Do not trigger presubmit builders, since they're likely to fail
-        # (e.g. OWNERS checks before finished code review), and we're
-        # running local presubmit anyway.
-        if 'presubmit' in builder:
-          builders[master].pop(builder)
-
-  return builders
+  import os.path
+  import subprocess
+
+  cq_config_path = os.path.join(
+      change.RepositoryRoot(), 'infra', 'config', 'cq.cfg')

[jam, 2015/05/29 18:07:27]

how does this work? i.e. if i do fetch chromium, I don't have infra/config?

[pgervais, 2015/05/29 18:14:45]

Yes you do :-) CQ configuration now lives in project repositories, not in an
hidden infra repository anymore.

[jam, 2015/05/29 18:19:46]

ah, looks like the checkout i checked was a day old and it didn't have this file
:)

I just synced, applied this patch, and ran git cl try and got the following
crash on Windows:

F:\src\chrome2\src>git cl try
Traceback (most recent call last):
  File "f:\src\depot_tools\git_cl.py", line 3349, in <module>
    sys.exit(main(sys.argv[1:]))
  File "f:\src\depot_tools\git_cl.py", line 3331, in main
    return dispatcher.execute(OptionParser(), argv)
  File "f:\src\depot_tools\subcommand.py", line 252, in execute
    return command(parser, args[1:])
  File "f:\src\depot_tools\git_cl.py", line 2973, in CMDtry
    masters = GetMasterMap()
  File "f:\src\depot_tools\git_cl.py", line 2935, in GetMasterMap
    sys.stdout)
  File "f:\src\depot_tools\presubmit_support.py", line 1286, in DoGetTryMasters
    presubmit_script, filename, project, change))
  File "f:\src\depot_tools\presubmit_support.py", line 1144, in
ExecPresubmitScript
    return get_preferred_try_masters(project, change)
  File "<string>", line 1796, in GetPreferredTryMasters
  File "f:\src\depot_tools\python276_bin\lib\subprocess.py", line 566, in
check_output
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
  File "f:\src\depot_tools\python276_bin\lib\subprocess.py", line 709, in
__init__
    errread, errwrite)
  File "f:\src\depot_tools\python276_bin\lib\subprocess.py", line 957, in
_execute_child
    startupinfo)
WindowsError: [Error 193] %1 is not a valid Win32 application
Sending crash report ...
  args: ['f:\\src\\depot_tools\\git_cl.py', 'try']
  cwd: F:\src\chrome2\src
  exception: [Error 193] %1 is not a valid Win32 application
  host: JABDELMALEK3-W.ad.corp.google.com
  stack:   File "f:\src\depot_tools\git_cl.py", line 3349,
  user: jabdelmalek
  version: 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500
A stack trace has been sent to the maintainers.

[pgervais, 2015/05/29 18:23:32]

sys.executable is missing on line 1796

[Sergiy Byelozyorov, 2015/05/29 18:57:55]

This would help, because python doesn't use $PATH or $PYTHONPATH to find
scripts. Instead I've removed the extension and will add commit_queue.bat and
commit_queue scripts to the depot_tools.

[Sergiy Byelozyorov, 2015/05/29 19:04:20]

Added binaries in https://codereview.chromium.org/1154623004

+  # commit_queue.py below is a script in depot_tools directory, which has a
+  # 'builders' command to retrieve a list of CQ builders from the CQ config.
+  masters = json.loads(subprocess.check_output(
+      ['commit_queue.py', 'builders', cq_config_path]))
+
+  # Explicitly iterate over copies of keys since we mutate them.
+  for master in masters.keys():
+    for builder in masters[master].keys():
+      # Do not trigger presubmit builders, since they're likely to fail
+      # (e.g. OWNERS checks before finished code review), and we're
+      # running local presubmit anyway.
+      if 'presubmit' in builder:
+        masters[master].pop(builder)
+      else:
+        # Convert testfilter format to the one expected by git-cl-try.
+        testfilter = masters[master][builder].get('testfilter', 'defaulttests')
+        masters[master][builder] = [testfilter]
+
+  return masters