Cherry-pick buildbot 95deef27d7c531ead19e0ac86a9aa1546d4ee7f9:

Cherry-pick buildbot 95deef27d7c531ead19e0ac86a9aa1546d4ee7f9:

Re-run queries after certain OperationalErrors

Sometimes malformed SQL can generate an OperationalError, so this looks
at the message itself.  It's unclear how this will work with
localization.

The code uses an exponential backoff algorithm (with a relatively small
multiplier), and will retry for a day, which seems a reasonable
get-the-db-server-fixed time.

Fixes #2005.  Hoepfully for good!

BUG=620572
Committed: https://chromium.googlesource.com/chromium/tools/build/+/8d24f5663861a7ae00e544f96e5c2e35787ecf4d

[dsansome, 2016-06-28 08:04:52]

Description was changed from

==========
Cherry-pick buildbot 95deef27d7c531ead19e0ac86a9aa1546d4ee7f9:

Re-run queries after certain OperationalErrors

Sometimes malformed SQL can generate an OperationalError, so this looks
at the message itself.  It's unclear how this will work with
localization.

The code uses an exponential backoff algorithm (with a relatively small
multiplier), and will retry for a day, which seems a reasonable
get-the-db-server-fixed time.

Fixes #2005.  Hoepfully for good!

BUG=
==========

to

==========
Cherry-pick buildbot 95deef27d7c531ead19e0ac86a9aa1546d4ee7f9:

Re-run queries after certain OperationalErrors

Sometimes malformed SQL can generate an OperationalError, so this looks
at the message itself.  It's unclear how this will work with
localization.

The code uses an exponential backoff algorithm (with a relatively small
multiplier), and will retry for a day, which seems a reasonable
get-the-db-server-fixed time.

Fixes #2005.  Hoepfully for good!

BUG=620572
==========

[dsansome, 2016-06-28 08:08:08]

dsansome@chromium.org changed reviewers:
+ stip@chromium.org

[dsansome, 2016-06-28 08:08:08]

I ran this Stress test with and without the patch to pool.py, and confirmed the
"database is locked" error goes away with the patch.

[ghost stip (do not use), 2016-07-01 01:30:16]

lgtm

https://codereview.chromium.org/2103053002/diff/20001/third_party/buildbot_8_...
File third_party/buildbot_8_4p1/buildbot/db/pool.py (right):

https://codereview.chromium.org/2103053002/diff/20001/third_party/buildbot_8_...
third_party/buildbot_8_4p1/buildbot/db/pool.py:92: # Try about 170 times over
the space of a day, with the last few tries
uh

[dsansome, 2016-07-01 04:42:04]

The CQ bit was checked by dsansome@chromium.org

[commit-bot: I haz the power, 2016-07-01 04:42:12]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-07-01 04:46:05]

Description was changed from

==========
Cherry-pick buildbot 95deef27d7c531ead19e0ac86a9aa1546d4ee7f9:

Re-run queries after certain OperationalErrors

Sometimes malformed SQL can generate an OperationalError, so this looks
at the message itself.  It's unclear how this will work with
localization.

The code uses an exponential backoff algorithm (with a relatively small
multiplier), and will retry for a day, which seems a reasonable
get-the-db-server-fixed time.

Fixes #2005.  Hoepfully for good!

BUG=620572
==========

to

==========
Cherry-pick buildbot 95deef27d7c531ead19e0ac86a9aa1546d4ee7f9:

Re-run queries after certain OperationalErrors

Sometimes malformed SQL can generate an OperationalError, so this looks
at the message itself.  It's unclear how this will work with
localization.

The code uses an exponential backoff algorithm (with a relatively small
multiplier), and will retry for a day, which seems a reasonable
get-the-db-server-fixed time.

Fixes #2005.  Hoepfully for good!

BUG=620572

Committed:
https://chromium.googlesource.com/chromium/tools/build/+/8d24f5663861a7ae00e5...
==========

[commit-bot: I haz the power, 2016-07-01 04:46:06]

Committed patchset #2 (id:20001) as
https://chromium.googlesource.com/chromium/tools/build/+/8d24f5663861a7ae00e5...

[ghost stip (do not use), 2016-07-07 20:13:30]

A revert of this CL (patchset #2 id:20001) has been created in
https://codereview.chromium.org/2128953002/ by stip@chromium.org.

The reason for reverting is: possible cause of
https://bugs.chromium.org/p/chromium/issues/detail?id=626354..