Docserver: Gitiles auth and cron refactoring.

Docserver: Gitiles auth and cron refactoring.

Pardon the dust. This is a large, huge, massive CL. This puts gitiles
in a mostly working state with auth. A hard requirement for this is
to make the periodic content update process work, and because gitiles
is so drastically slow compared to SVN, this meant breaking up the
Cron tasks into many small subtasks.

More work to follow, but this should allow us to get a live server
up and running ASAP.

BUG=404239
NOTRY=True
TBR=ahernandez.miralles@gmail.com

Committed: https://crrev.com/deaf2f7e812fd2bb8e0387491d0d080d1e1fe61e
Cr-Commit-Position: refs/heads/master@{#295079}

[Ken Rockot(use gerrit already), 2014-09-15 22:13:02]

Here's my patch. Mostly just looking for a rubber stamp rather than a thorough
review, but feel free to go in as much depth as you like.

Note that this temporarily breaks integration_tests.

This CL mirrors the code that is currently running on GAE under version 3-43-1.

[ahernandez, 2014-09-15 23:58:03]

Awesome patch! I've been excited to see these changes. Here's a rubbery lgtm,
with a couple comments.

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
File chrome/common/extensions/docs/server2/appengine_wrappers.py (right):

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
chrome/common/extensions/docs/server2/appengine_wrappers.py:22: import
google.appengine.ext.blobstore as blobstore
Is there a reason why this import was moved up?

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
File chrome/common/extensions/docs/server2/gitiles_file_system.py (right):

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
chrome/common/extensions/docs/server2/gitiles_file_system.py:10: import logging
Double import.

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
File chrome/common/extensions/docs/server2/instance_servlet.py (right):

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
chrome/common/extensions/docs/server2/instance_servlet.py:22: Why? SVN is slow
and a bit flaky. Refresh jobs failing is annoying but
Could you please changes instances of SVN to Gitiles or something similar while
you're here?

[Ken Rockot(use gerrit already), 2014-09-16 00:24:31]

Thanks!

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
File chrome/common/extensions/docs/server2/appengine_wrappers.py (right):

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
chrome/common/extensions/docs/server2/appengine_wrappers.py:22: import
google.appengine.ext.blobstore as blobstore
On 2014/09/15 23:58:02, ahernandez wrote:
> Is there a reason why this import was moved up?

Alphabetical ordering makes me happy

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
File chrome/common/extensions/docs/server2/gitiles_file_system.py (right):

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
chrome/common/extensions/docs/server2/gitiles_file_system.py:10: import logging
On 2014/09/15 23:58:02, ahernandez wrote:
> Double import.

Done.

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
File chrome/common/extensions/docs/server2/instance_servlet.py (right):

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
chrome/common/extensions/docs/server2/instance_servlet.py:22: Why? SVN is slow
and a bit flaky. Refresh jobs failing is annoying but
On 2014/09/15 23:58:02, ahernandez wrote:
> Could you please changes instances of SVN to Gitiles or something similar
while
> you're here?

Done.

[ahernandez, 2014-09-16 00:28:54]

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
File chrome/common/extensions/docs/server2/appengine_wrappers.py (right):

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
chrome/common/extensions/docs/server2/appengine_wrappers.py:22: import
google.appengine.ext.blobstore as blobstore
On 2014/09/16 00:24:31, Ken Rockot wrote:
> On 2014/09/15 23:58:02, ahernandez wrote:
> > Is there a reason why this import was moved up?
> 
> Alphabetical ordering makes me happy

Shouldn't any imports prefixed with google.appengine.ext be after imports
prefixed with google.appengine.api?

[Ken Rockot(use gerrit already), 2014-09-16 00:31:20]

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
File chrome/common/extensions/docs/server2/appengine_wrappers.py (right):

https://codereview.chromium.org/575613003/diff/1/chrome/common/extensions/doc...
chrome/common/extensions/docs/server2/appengine_wrappers.py:22: import
google.appengine.ext.blobstore as blobstore
On 2014/09/16 00:28:53, ahernandez wrote:
> On 2014/09/16 00:24:31, Ken Rockot wrote:
> > On 2014/09/15 23:58:02, ahernandez wrote:
> > > Is there a reason why this import was moved up?
> > 
> > Alphabetical ordering makes me happy
> 
> Shouldn't any imports prefixed with google.appengine.ext be after imports
> prefixed with google.appengine.api?

Ludicrous! Yeah, you're right. I the "ext/api" bit was invisible to me.
Reverted.

[Ken Rockot(use gerrit already), 2014-09-16 16:25:10]

The CQ bit was checked by rockot@chromium.org

[Ken Rockot(use gerrit already), 2014-09-16 16:25:46]

Gonna commit this. I've done enough testing that I'm confident in its contents.
If only we could solve the throttling issue, I think we'd be live right now.

[commit-bot: I haz the power, 2014-09-16 16:26:43]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patchset/575613003/40001

[commit-bot: I haz the power, 2014-09-16 16:29:32]

Committed patchset #3 (id:40001) as 56073c8bc8440374191afb7837a8ceee833cae99

[commit-bot: I haz the power, 2014-09-16 16:30:55]

Patchset 3 (id:??) landed as
https://crrev.com/deaf2f7e812fd2bb8e0387491d0d080d1e1fe61e
Cr-Commit-Position: refs/heads/master@{#295079}