Domain Reliability: Measure backoff/retry time to collectors.

Domain Reliability: Measure backoff/retry time to collectors.

Add a histogram to see, on an individual collector level, how long the
scheduler is backing off for, in failure or retry-after cases.

(In the future we will use this to make sure retry-after works.)

BUG=

Committed: https://crrev.com/4f147a69f7f4be66f601ce3ae070af2e971fe8a5
Cr-Commit-Position: refs/heads/master@{#316265}

[Deprecated (see juliatuttle), 2015-02-10 16:26:46]

ttuttle@chromium.org changed reviewers:
+ asvitkine@chromium.org, davidben@chromium.org

[Deprecated (see juliatuttle), 2015-02-10 16:26:46]

ttuttle@chromium.org changed required reviewers:
+ asvitkine@chromium.org, davidben@chromium.org

[Deprecated (see juliatuttle), 2015-02-10 16:26:47]

PTAL, davidben and asvitkine.

[davidben, 2015-02-10 19:00:00]

https://codereview.chromium.org/907993003/diff/1/components/domain_reliabilit...
File components/domain_reliability/scheduler.cc (right):

https://codereview.chromium.org/907993003/diff/1/components/domain_reliabilit...
components/domain_reliability/scheduler.cc:143: last_collector_retry_delay_ =
backoff->GetTimeUntilRelease();
This will also record in the success case, right? What does BackoffEntry do
there? Will it give you a spike of zeros?

[Deprecated (see juliatuttle), 2015-02-10 19:03:58]

On 2015/02/10 19:00:00, David Benjamin wrote:
>
https://codereview.chromium.org/907993003/diff/1/components/domain_reliabilit...
> File components/domain_reliability/scheduler.cc (right):
> 
>
https://codereview.chromium.org/907993003/diff/1/components/domain_reliabilit...
> components/domain_reliability/scheduler.cc:143: last_collector_retry_delay_ =
> backoff->GetTimeUntilRelease();
> This will also record in the success case, right? What does BackoffEntry do
> there? Will it give you a spike of zeros?

Yes, it will give us a spike of zeros. This is expected and desired.

[davidben, 2015-02-10 19:10:36]

https://codereview.chromium.org/907993003/diff/1/components/domain_reliabilit...
File components/domain_reliability/scheduler.cc (right):

https://codereview.chromium.org/907993003/diff/1/components/domain_reliabilit...
components/domain_reliability/scheduler.cc:143: last_collector_retry_delay_ =
backoff->GetTimeUntilRelease();
On 2015/02/10 19:00:00, David Benjamin wrote:
> This will also record in the success case, right? What does BackoffEntry do
> there? Will it give you a spike of zeros?

Ah, I see you even mentioned it in histograms.xml. Missed that on my first pass
somehow.

https://codereview.chromium.org/907993003/diff/1/tools/metrics/histograms/his...
File tools/metrics/histograms/histograms.xml (right):

https://codereview.chromium.org/907993003/diff/1/tools/metrics/histograms/his...
tools/metrics/histograms/histograms.xml:5412: +    upload was successful, this
will probably be 0; if not, it will be based on
Strike the word "probably"? Or are there cases where it isn't zero?

[Alexei Svitkine (slow), 2015-02-11 20:18:55]

lgtm

[Deprecated (see juliatuttle), 2015-02-11 21:10:28]

New patchsets have been uploaded after l-g-t-m from asvitkine@chromium.org

[davidben, 2015-02-11 21:15:17]

lgtm

[Deprecated (see juliatuttle), 2015-02-11 21:21:54]

The CQ bit was checked by ttuttle@chromium.org

[commit-bot: I haz the power, 2015-02-11 21:22:46]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/907993003/20001

[commit-bot: I haz the power, 2015-02-11 22:21:52]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2015-02-11 22:21:54]

Failed to apply patch for components/domain_reliability/context.cc:
While running git apply --index -3 -p1;
  error: patch failed: components/domain_reliability/context.cc:256
  Falling back to three-way merge...
  Applied patch to 'components/domain_reliability/context.cc' with conflicts.
  U components/domain_reliability/context.cc

Patch:       components/domain_reliability/context.cc
Index: components/domain_reliability/context.cc
diff --git a/components/domain_reliability/context.cc
b/components/domain_reliability/context.cc
index
730d344a47a7bbab1453ceea349d971517aea29f..ff200446f731fe457250560a46d61c02c41a8217
100644
--- a/components/domain_reliability/context.cc
+++ b/components/domain_reliability/context.cc
@@ -256,6 +256,8 @@ void DomainReliabilityContext::OnUploadComplete(
   DCHECK(!upload_time_.is_null());
   UMA_HISTOGRAM_MEDIUM_TIMES("DomainReliability.UploadDuration",
                              time_->NowTicks() - upload_time_);
+  UMA_HISTOGRAM_LONG_TIMES("DomainReliability.UploadCollectorRetryDelay",
+                           scheduler_.last_collector_retry_delay());
   last_upload_time_ = upload_time_;
   upload_time_ = base::TimeTicks();
 }

[commit-bot: I haz the power, 2015-02-11 22:22:02]

The CQ bit was unchecked by commit-bot@chromium.org

[Deprecated (see juliatuttle), 2015-02-12 20:13:50]

New patchsets have been uploaded after l-g-t-m from davidben@chromium.org

[Deprecated (see juliatuttle), 2015-02-12 20:15:25]

The CQ bit was checked by ttuttle@chromium.org

[commit-bot: I haz the power, 2015-02-12 20:16:34]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/907993003/40001

[commit-bot: I haz the power, 2015-02-12 21:31:37]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2015-02-12 21:31:38]

Try jobs failed on following builders:
  linux_chromium_rel_ng on tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)

[Deprecated (see juliatuttle), 2015-02-13 18:50:53]

The CQ bit was checked by ttuttle@chromium.org

[commit-bot: I haz the power, 2015-02-13 18:51:47]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/907993003/40001

[commit-bot: I haz the power, 2015-02-13 19:54:03]

Committed patchset #3 (id:40001)

[commit-bot: I haz the power, 2015-02-13 19:54:31]

Patchset 3 (id:??) landed as
https://crrev.com/4f147a69f7f4be66f601ce3ae070af2e971fe8a5
Cr-Commit-Position: refs/heads/master@{#316265}