[sql] Annotate sql::Recovery failure cases.

[sql] Annotate sql::Recovery failure cases.

A large number of thumbnail_database.cc recovery failures are in
building the scoped recovery instance.  Break down failure cases for
sql::Recovery to see if there is any action to be taken.

Additionally, track failure calling sql::Recovery::Recovered().

BUG=109482
TBR=sky@chromium.org

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=237680

[Scott Hess - ex-Googler, 2013-11-22 23:22:23]

Of the outstanding failure cases, failing to create the scoper is the biggest,
and failing when accessing the meta table is next.  My suspicion is that both
cases will turn out to be some systemic problem, like the filesystem is entirely
messed up, in which case an appropriate combination of "give up" or "blow
everything away" can be applied.

There is a very small number of Sqlite.RecoveryStep items logged, so I don't
think that sql::Recovery::Recovered() is probably actually failing for many
people, but I added that for completeness.

[erikwright (departed), 2013-11-25 17:59:31]

LGTM.

[Scott Hess - ex-Googler, 2013-11-25 19:15:30]

+Ilya for histograms.xml OWNERS.

[Ilya Sherman, 2013-11-25 22:52:25]

LGTM % suggestions below:

https://codereview.chromium.org/83323005/diff/50001/tools/metrics/histograms/...
File tools/metrics/histograms/histograms.xml (right):

https://codereview.chromium.org/83323005/diff/50001/tools/metrics/histograms/...
tools/metrics/histograms/histograms.xml:18998: +  <summary>Failure cases noted
in recovery.cc.</summary>
Please elaborate a bit more on what this histogram is measuring.  I'm assuming
it records an event every time that DB recovery is attempted and fails.  Is that
accurate?  If so, please document that in the summary.

https://codereview.chromium.org/83323005/diff/50001/tools/metrics/histograms/...
tools/metrics/histograms/histograms.xml:28985: +  </int>
It's often useful to have the first bucket measure the success rate, so that you
can tell how common failures are overall.

[Scott Hess - ex-Googler, 2013-11-26 20:17:58]

Damn presubmit.

+sky@ for OWNERS review on the minor thumbnails_database.cc change.  I'm only
adding that for completeness, because the existing code treated that failure as
success in the histograms.  Based on other histograms, I believe it is only
failing a tiny percentage of the time.

https://codereview.chromium.org/83323005/diff/50001/tools/metrics/histograms/...
File tools/metrics/histograms/histograms.xml (right):

https://codereview.chromium.org/83323005/diff/50001/tools/metrics/histograms/...
tools/metrics/histograms/histograms.xml:18998: +  <summary>Failure cases noted
in recovery.cc.</summary>
On 2013/11/25 22:52:25, Ilya Sherman wrote:
> Please elaborate a bit more on what this histogram is measuring.  I'm assuming
> it records an event every time that DB recovery is attempted and fails.  Is
that
> accurate?  If so, please document that in the summary.

I tried to augment it, but found that I was mostly writing filler.  It tracks
all of the failures in the class' implementation, for purposes of deciding which
(if any) are worth researching further and possibly developing additional
automated responses.  AFAICT, failures should always be due to system issues,
not issues with our code, so if the issue looks like "System is entirely fubar",
additional code can probably not solve the problem, while cases which could be
solved might not be worth the complexity if too few users hit them.

https://codereview.chromium.org/83323005/diff/50001/tools/metrics/histograms/...
tools/metrics/histograms/histograms.xml:28985: +  </int>
On 2013/11/25 22:52:25, Ilya Sherman wrote:
> It's often useful to have the first bucket measure the success rate, so that
you
> can tell how common failures are overall.

I'll layer in success rates, because they don't detract from anything.  Mostly
I'm not really interested in success rates - I'm rather looking to see if an
absolute number of current cases can be addressed by a constant development
cost.  I suspect not, but I want to see data before closing it off.

Additionally, in case of success, the code will never be called again, while in
case of failure it will probably always be called for that user.  The success
and failure buckets are not necessarily comparable.

[Scott Hess - ex-Googler, 2013-11-27 22:37:18]

I appear to have forgotten to loop sky@ in as an actual reviewer.  I'm going to
TBR, the thumbnail_database.cc change is trivial.

[commit-bot: I haz the power, 2013-11-27 23:39:17]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/shess@chromium.org/83323005/140001

[commit-bot: I haz the power, 2013-11-28 01:24:13]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/shess@chromium.org/83323005/140001

[commit-bot: I haz the power, 2013-11-28 02:04:51]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/shess@chromium.org/83323005/140001

[commit-bot: I haz the power, 2013-11-28 02:33:44]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/shess@chromium.org/83323005/140001

[commit-bot: I haz the power, 2013-11-28 03:03:34]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/shess@chromium.org/83323005/140001

[commit-bot: I haz the power, 2013-11-28 03:32:20]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/shess@chromium.org/83323005/140001

[commit-bot: I haz the power, 2013-11-28 05:27:49]

Change committed as 237680

[sky, 2013-12-02 16:13:09]

LGTM