Milo: Clean up builds that are no longer current.

Milo: Clean up builds that are no longer current.

Sometimes when masters restart, they don't flush builds, so that current builds
continue to linger in Milo despite being expired (and finished).

This uses the signal that if a build exists in Milo but isn't in the master
JSON, then it should get marked as expired.

BUG=654623
Committed: https://github.com/luci/luci-go/commit/a94e9e33c00d7ecc2acd4a8489732f026c768a35

[Ryan Tseng, 2016-10-12 06:56:38]

The CQ bit was checked by hinoka@google.com to run a CQ dry run

[commit-bot: I haz the power, 2016-10-12 06:56:40]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[hinoka, 2016-10-12 06:58:30]

hinoka@chromium.org changed reviewers:
+ hinoka@chromium.org, nodir@chromium.org

[hinoka, 2016-10-12 06:58:31]

nodir would you mind taking a look at this, since estaab is OOO?

https://codereview.chromium.org/2406163003/diff/1/milo/appengine/buildbot/pub...
File milo/appengine/buildbot/pubsub_test.go (right):

https://codereview.chromium.org/2406163003/diff/1/milo/appengine/buildbot/pub...
milo/appengine/buildbot/pubsub_test.go:318: ds.GetTestable(c).Consistent(true)
This sort of cheats, because the pubsub handler stores all the builds first, and
then does the master functions.  But it works.

[Ryan Tseng, 2016-10-12 06:58:31]

The CQ bit was checked by hinoka@google.com to run a CQ dry run

[commit-bot: I haz the power, 2016-10-12 06:58:40]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-10-12 07:07:36]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-10-12 07:07:37]

Dry run: This issue passed the CQ dry run.

[hinoka, 2016-10-17 22:00:13]

hinoka@chromium.org changed reviewers:
+ estaab@chromium.org, hinoka@chromium.org, martiniss@chromium.org

[hinoka, 2016-10-17 22:00:13]

+estaab,martiniss reviewer

This is necessary for getting the "current build" view on milo

[estaab, 2016-10-20 03:30:39]

https://codereview.chromium.org/2406163003/diff/20001/milo/appengine/buildbot...
File milo/appengine/buildbot/pubsub.go (right):

https://codereview.chromium.org/2406163003/diff/20001/milo/appengine/buildbot...
milo/appengine/buildbot/pubsub.go:309: // Extract current builds data out of the
master json, and use it to
This function is pretty long by this point. Is there another logical block you
can take out of it?

[Ryan Tseng, 2016-10-26 01:27:57]

The CQ bit was checked by hinoka@google.com to run a CQ dry run

[hinoka, 2016-10-26 01:28:04]

https://codereview.chromium.org/2406163003/diff/20001/milo/appengine/buildbot...
File milo/appengine/buildbot/pubsub.go (right):

https://codereview.chromium.org/2406163003/diff/20001/milo/appengine/buildbot...
milo/appengine/buildbot/pubsub.go:309: // Extract current builds data out of the
master json, and use it to
On 2016/10/20 03:30:39, estaab wrote:
> This function is pretty long by this point. Is there another logical block you
> can take out of it?

Done.

[commit-bot: I haz the power, 2016-10-26 01:28:07]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-10-26 01:34:44]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-10-26 01:34:44]

Dry run: This issue passed the CQ dry run.

[estaab, 2016-10-26 21:31:16]

lgtm

[hinoka, 2016-10-26 21:36:29]

The CQ bit was checked by hinoka@chromium.org

[commit-bot: I haz the power, 2016-10-26 21:36:33]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-10-26 21:39:49]

Description was changed from

==========
Milo: Clean up builds that are no longer current.

Sometimes when masters restart, they don't flush builds, so that current builds
continue to linger in Milo despite being expired (and finished).

This uses the signal that if a build exists in Milo but isn't in the master
JSON, then it should get marked as expired.

BUG=654623
==========

to

==========
Milo: Clean up builds that are no longer current.

Sometimes when masters restart, they don't flush builds, so that current builds
continue to linger in Milo despite being expired (and finished).

This uses the signal that if a build exists in Milo but isn't in the master
JSON, then it should get marked as expired.

BUG=654623

Committed:
https://github.com/luci/luci-go/commit/a94e9e33c00d7ecc2acd4a8489732f026c768a35
==========

[commit-bot: I haz the power, 2016-10-26 21:39:50]

Committed patchset #3 (id:40001) as
https://github.com/luci/luci-go/commit/a94e9e33c00d7ecc2acd4a8489732f026c768a35