[WPT Export] Require GitHub API token in WPT remote origin URL

third_party/WebKit/Tools/Scripts/webkitpy/w3c/local_wpt.py

Index: third_party/WebKit/Tools/Scripts/webkitpy/w3c/local_wpt.py
diff --git a/third_party/WebKit/Tools/Scripts/webkitpy/w3c/local_wpt.py b/third_party/WebKit/Tools/Scripts/webkitpy/w3c/local_wpt.py
index 6c9b0ecebf62a48f2f777851101cf72e20e00921..e218fe07fecf9a929284d136ed8bdaddcfcaf857 100644
--- a/third_party/WebKit/Tools/Scripts/webkitpy/w3c/local_wpt.py
+++ b/third_party/WebKit/Tools/Scripts/webkitpy/w3c/local_wpt.py
@@ -8,7 +8,7 @@ import logging
 
 from webkitpy.common.system.executive import ScriptError
 from webkitpy.w3c.chromium_commit import ChromiumCommit
-from webkitpy.w3c.common import WPT_GH_REPO_URL, CHROMIUM_WPT_DIR
+from webkitpy.w3c.common import WPT_GH_REPO_URL_TEMPLATE, CHROMIUM_WPT_DIR
 
 
 _log = logging.getLogger(__name__)
@@ -16,7 +16,7 @@ _log = logging.getLogger(__name__)
 
 class LocalWPT(object):
 
-    def __init__(self, host, path='/tmp/wpt'):
+    def __init__(self, host, gh_token, path='/tmp/wpt'):

[qyearsley, 2017/02/28 18:30:04]

LocalWPT is used in test_importer, which I believe doesn't need a token -- so
here, gh_token should perhaps be a named optional argument with a default value
of None?

[jeffcarp, 2017/02/28 18:53:38]

sgtm

         """
         Args:
             host: A Host object.
@@ -26,6 +26,7 @@ class LocalWPT(object):
         """
         self.host = host
         self.path = path
+        self.gh_token = gh_token
         self.branch_name = 'chromium-export-try'
 
     def fetch(self):
@@ -34,8 +35,9 @@ class LocalWPT(object):
             self.run(['git', 'fetch', 'origin'])
             self.run(['git', 'checkout', 'origin/master'])
         else:
-            _log.info('Cloning %s into %s', WPT_GH_REPO_URL, self.path)
-            self.host.executive.run_command(['git', 'clone', WPT_GH_REPO_URL, self.path])
+            _log.info('Cloning GitHub w3c/web-platform-tests into %s', self.path)
+            remote_url = WPT_GH_REPO_URL_TEMPLATE.format(self.gh_token)
+            self.host.executive.run_command(['git', 'clone', '--depth=10', remote_url, self.path])

[qyearsley, 2017/02/28 18:30:04]

Why depth = 10? What if the test that we want to upstream is nested really
deeply, like 11 or 12 directories deep?

[jeffcarp, 2017/02/28 18:53:38]

--depth in this case refers to how many commits back we want to clone (e.g. this
only clones the repository and its most recent 10 commits), but your comment
made me realize that this won't work for a similar reason, because the commit
containing Cr-Commit-Position might be more than 10 commits back.

I added --depth originally to lighten the load since we're cloning the GitHub
repo from scratch every 5-10 minutes or so (since the bot gets torn down every
time). I'll remove that flag and think of other ways to lighten the load.

[qyearsley, 2017/02/28 18:58:04]

Ah, I see, that's what depth means :-)

Maybe another depth might be reasonable, as an optimization? Like, there should
be no need to go back further than a few months of commits, probably?

 
     def run(self, command, **kwargs):
         """Runs a command in the local WPT directory."""