[Telemetry] Add PRESUBMIT check to prevent .sha file uploaded without archive file uploaded

tools/perf/PRESUBMIT.py

Index: tools/perf/PRESUBMIT.py
diff --git a/tools/perf/PRESUBMIT.py b/tools/perf/PRESUBMIT.py
index 7ac8a1831d5c7f738601cc403023e41a1b73fac2..993ac4f14cf9f975098633056095f7c031dacfea 100644
--- a/tools/perf/PRESUBMIT.py
+++ b/tools/perf/PRESUBMIT.py
@@ -11,6 +11,10 @@ for more details about the presubmit API built into depot_tools.
 import os
 import sys
 
+sys.path = [os.path.join(os.pardir, 'telemetry')] + sys.path

[aiolos (Not reviewing), 2015/02/24 18:53:07]

PRESUBMIT scripts aren't supposed to have side effects, so this should go back
in the try block format that is was in earlier.

[nednguyen, 2015/02/24 19:28:33]

Done.

+
+from telemetry.util import cloud_storage
+
 PYLINT_BLACKLIST = []
 PYLINT_DISABLED_WARNINGS = [
     'R0923',  # Interface not implemented
@@ -22,17 +26,37 @@ PYLINT_DISABLED_WARNINGS = [
 def _CommonChecks(input_api, output_api):
   """Performs common checks, which includes running pylint."""
   results = []
-  old_sys_path = sys.path
-  try:
-    # Modules in tools/perf depend on telemetry.
-    sys.path = [os.path.join(os.pardir, 'telemetry')] + sys.path
-    results.extend(input_api.canned_checks.RunPylint(
-        input_api, output_api,
-        black_list=PYLINT_BLACKLIST,
-        disabled_warnings=PYLINT_DISABLED_WARNINGS))
-    results.extend(_CheckJson(input_api, output_api))
-  finally:
-    sys.path = old_sys_path
+  results.extend(input_api.canned_checks.RunPylint(
+      input_api, output_api,
+      black_list=PYLINT_BLACKLIST,
+      disabled_warnings=PYLINT_DISABLED_WARNINGS))
+  results.extend(_CheckJson(input_api, output_api))
+  results.extend(_CheckWprShaFiles(input_api, output_api))
+  return results
+
+
+def _CheckWprShaFiles(input_api, output_api):
+  """ Check whether the wpr sha files have matching URLs """

[slamm, 2015/02/24 19:22:59]

No spaces immediately inside triple quotes.

"""Check whether the wpr sha files have matching URLs."""

[nednguyen, 2015/02/24 19:28:33]

Done.

+  results = []
+  for affected_file in input_api.AffectedFiles(include_deletes=False):
+    filename = affected_file.AbsoluteLocalPath()
+    if not filename.endswith('wpr.sha1'):
+      continue
+    wpr_file_exist_on_cloud = False

[dtu, 2015/02/24 19:12:47]

nit: wpr_file_exists_in_cloud

[slamm, 2015/02/24 19:22:59]

How about the following?

is_wpr_file_uploaded

or 

is_wpr_file_in_cloud

[nednguyen, 2015/02/24 19:28:33]

Done.

[nednguyen, 2015/02/24 19:28:33]

Done.

+    expected_hash = cloud_storage.ReadHash(filename)
+    for bucket in cloud_storage.SUPPORTED_BUCKETS:
+      wpr_file_exist_on_cloud = (wpr_file_exist_on_cloud or
+                                 cloud_storage.Exists(bucket, expected_hash))

[slamm, 2015/02/24 19:22:59]

is_wpr_file_uploaded = any(cloud_storage.Exists(bucket, expected_hash)
                           for bucket in cloud_storage.SUPPORTED_BUCKETS))

[nednguyen, 2015/02/24 19:28:33]

Nice, I like this.

+    if not wpr_file_exist_on_cloud:
+      wpr_filename = filename[:-5]
+      results.append(output_api.PresubmitError(

[aiolos (Not reviewing), 2015/02/24 18:19:45]

Should this be a warning instead of an error? People outside of Google don't
have permission to access all of the buckets, and will need to record their own
wpr files to run the page_sets in Internal, and sometimes Partner. They
definitely shouldn't be uploading or committing those recordings.

[nednguyen, 2015/02/24 18:24:40]

If they don't commit or upload, then this PRESUBMIT check will not run. It
doesn't run on all .wpr.sha1 file, only the one that included in your commit.
And it only runs when you do "git cl upload" or "git cl presubmit". 

[dtu, 2015/02/24 19:12:47]

I think this is fine. I guess you have to be careful that the buckets are
ordered from most open to most restrictive.

+          'No URLs matched for wpr sha file %s.\n'
+          'You can upload your new wpr archive file with the command:\n'
+          'depot_tools/upload_to_google_storage.py --bucket '
+          '<Your pageset\'s bucket> %s.\nFor more info: see '
+          'http://www.chromium.org/developers/telemetry/'
+          'record_a_page_set#TOC-Upload-the-recording-to-Cloud-Storage' %
+          (filename, wpr_filename)))
   return results