Milo: Handle missing / transient LogDog failures.

Milo: Handle missing / transient LogDog failures.

If a build is labelled as having a LogDog stream, Milo will
unconditionally try and load that stream. This falls apart if the build
has either not started yet, or if the LogDog annotation stream is not
available due to natural ingest delay.

This will prevent stream loading if the build hasn't started yet, and
will retry LogDog transient and not-found errors prior to failing
absolutely.

BUG=chromium:692245, chromium:694913
TEST=None
R=hinoka@chromium.org, nodir@chromium.org

Review-Url: https://codereview.chromium.org/2717623002
Committed: https://github.com/luci/luci-go/commit/887de917e6e7d48a151293fd298d22519c86b6af

[hinoka, 2017-02-24 01:30:49]

lgtm + comment.  I have a preference for 404's from logdog not being retried and
to have the build remain in a "pending" state until we're past the 30s deadline.

ie. Build has started, but the swarming bot is taking a while to initialize the
stack.  That is still a "pending" mode of sorts.

https://codereview.chromium.org/2717623002/diff/1/milo/appengine/logdog/build.go
File milo/appengine/logdog/build.go (right):

https://codereview.chromium.org/2717623002/diff/1/milo/appengine/logdog/build...
milo/appengine/logdog/build.go:130: case coordinator.ErrNoSuchStream:
Consider making this just a non-transient error.  If we do so, Milo should still
display the build as "pending" immediately, which is probably better than
waiting 5 seconds and seeing maybe the same thing or something different.

[dnj, 2017-02-24 03:24:09]

OK, removed the retries.

https://codereview.chromium.org/2717623002/diff/1/milo/appengine/logdog/build.go
File milo/appengine/logdog/build.go (right):

https://codereview.chromium.org/2717623002/diff/1/milo/appengine/logdog/build...
milo/appengine/logdog/build.go:130: case coordinator.ErrNoSuchStream:
On 2017/02/24 01:30:49, hinoka wrote:
> Consider making this just a non-transient error.  If we do so, Milo should
still
> display the build as "pending" immediately, which is probably better than
> waiting 5 seconds and seeing maybe the same thing or something different.

Good idea, Done.

[dnj, 2017-02-24 03:29:31]

The CQ bit was checked by dnj@chromium.org

[dnj, 2017-02-24 03:29:31]

The patchset sent to the CQ was uploaded after l-g-t-m from hinoka@chromium.org
Link to the patchset: https://codereview.chromium.org/2717623002/#ps40001
(title: "remote unnecessary code")

[commit-bot: I haz the power, 2017-02-24 03:29:35]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-02-24 03:38:41]

CQ is committing da patch.

Bot data: {"patchset_id": 40001, "attempt_start_ts": 1487906971737710,
"parent_rev": "611ffa1c688962e21b7bdca1e9780576c69df94d", "commit_rev":
"887de917e6e7d48a151293fd298d22519c86b6af"}

[commit-bot: I haz the power, 2017-02-24 03:38:44]

Description was changed from

==========
Milo: Handle missing / transient LogDog failures.

If a build is labelled as having a LogDog stream, Milo will
unconditionally try and load that stream. This falls apart if the build
has either not started yet, or if the LogDog annotation stream is not
available due to natural ingest delay.

This will prevent stream loading if the build hasn't started yet, and
will retry LogDog transient and not-found errors prior to failing
absolutely.

BUG=chromium:692245, chromium:694913
TEST=None
R=hinoka@chromium.org, nodir@chromium.org
==========

to

==========
Milo: Handle missing / transient LogDog failures.

If a build is labelled as having a LogDog stream, Milo will
unconditionally try and load that stream. This falls apart if the build
has either not started yet, or if the LogDog annotation stream is not
available due to natural ingest delay.

This will prevent stream loading if the build hasn't started yet, and
will retry LogDog transient and not-found errors prior to failing
absolutely.

BUG=chromium:692245, chromium:694913
TEST=None
R=hinoka@chromium.org, nodir@chromium.org

Review-Url: https://codereview.chromium.org/2717623002
Committed:
https://github.com/luci/luci-go/commit/887de917e6e7d48a151293fd298d22519c86b6af
==========

[commit-bot: I haz the power, 2017-02-24 03:38:45]

Committed patchset #3 (id:40001) as
https://github.com/luci/luci-go/commit/887de917e6e7d48a151293fd298d22519c86b6af