Create Tasks.Tags endpoint and Cron job

Create Tasks.Tags endpoint and Cron job

BUG=631047
Committed: https://github.com/luci/luci-py/commit/83683a555eeecebca47baf6bfb59916fb4cd5731

[kjlubick, 2016-08-16 01:48:47]

Description was changed from

==========
First part of getting tags endpoint working

BUG=631047
==========

to

==========
Create Tasks.Tags endpoint and Cron job

BUG=631047
==========

[kjlubick, 2016-08-16 01:48:47]

kjlubick@google.com changed reviewers:
+ maruel@chromium.org

[M-A Ruel, 2016-08-16 15:37:22]

lgtm, let's see if it actually work on production, maybe not.

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
File appengine/swarming/handlers_backend.py (right):

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
appengine/swarming/handlers_backend.py:142: tasks, cursor =
datastore_utils.fetch_page(q, 1000, cursor)
fetching 1000 at a time is going to be very slow. If you iterate one at a time,
the ndb.Query object will actually do batches, so I recommend just doing that.
Then no need to use a cursor.

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
appengine/swarming/handlers_backend.py:147: if k != 'cron_invocation_id':
what's that?

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
appengine/swarming/handlers_backend.py:153:
task_result.TagValues(tag=k,values=sorted(values))
task_result.TagValues(tag=k, values=sorted(values))

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
File appengine/swarming/handlers_endpoints.py (right):

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
appengine/swarming/handlers_endpoints.py:425: 
remove one line

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
File appengine/swarming/handlers_endpoints_test.py (right):

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
appengine/swarming/handlers_endpoints_test.py:870: tag='foo', values=['alpha',
'beta']),
align at +4

[kjlubick, 2016-08-16 17:20:45]

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
File appengine/swarming/handlers_backend.py (right):

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
appengine/swarming/handlers_backend.py:142: tasks, cursor =
datastore_utils.fetch_page(q, 1000, cursor)
On 2016/08/16 at 15:37:21, M-A Ruel wrote:
> fetching 1000 at a time is going to be very slow. If you iterate one at a
time, the ndb.Query object will actually do batches, so I recommend just doing
that. Then no need to use a cursor.

I tried iterating one at a time, even with a filter and it timed out after 1
minute. 

I will have it only do the last 12 hours, until we see how it does under load.

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
appengine/swarming/handlers_backend.py:147: if k != 'cron_invocation_id':
On 2016/08/16 at 15:37:21, M-A Ruel wrote:
> what's that?

It appeared to be something that was different for all, or at least many of the
heartbeat tasks.  I figured we need not aggregate on it, right?

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
appengine/swarming/handlers_backend.py:153:
task_result.TagValues(tag=k,values=sorted(values))
On 2016/08/16 at 15:37:21, M-A Ruel wrote:
> task_result.TagValues(tag=k, values=sorted(values))

Done.

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
File appengine/swarming/handlers_endpoints.py (right):

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
appengine/swarming/handlers_endpoints.py:425: 
On 2016/08/16 at 15:37:21, M-A Ruel wrote:
> remove one line

Done.

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
File appengine/swarming/handlers_endpoints_test.py (right):

https://codereview.chromium.org/2249803003/diff/20001/appengine/swarming/hand...
appengine/swarming/handlers_endpoints_test.py:870: tag='foo', values=['alpha',
'beta']),
On 2016/08/16 at 15:37:21, M-A Ruel wrote:
> align at +4

Done.

[kjlubick, 2016-08-16 17:20:47]

The CQ bit was checked by kjlubick@google.com

[kjlubick, 2016-08-16 17:20:48]

The patchset sent to the CQ was uploaded after l-g-t-m from maruel@chromium.org
Link to the patchset: https://codereview.chromium.org/2249803003/#ps60001
(title: "Docs")

[commit-bot: I haz the power, 2016-08-16 17:20:52]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-08-16 17:24:06]

Description was changed from

==========
Create Tasks.Tags endpoint and Cron job

BUG=631047
==========

to

==========
Create Tasks.Tags endpoint and Cron job

BUG=631047

Committed:
https://github.com/luci/luci-py/commit/83683a555eeecebca47baf6bfb59916fb4cd5731
==========

[commit-bot: I haz the power, 2016-08-16 17:24:07]

Committed patchset #4 (id:60001) as
https://github.com/luci/luci-py/commit/83683a555eeecebca47baf6bfb59916fb4cd5731