Disable history tests with flaky timeout

Disable history tests with flaky timeout

These have all flaked and timed out at least once in the last week.

R=dmazzoni@chromium.org
BUG=336845, 375910
Committed: https://crrev.com/c12345f315ad642ebbd0bba87b26ad3642025e6d
Cr-Commit-Position: refs/heads/master@{#320141}

[enne (OOO), 2015-03-04 19:22:36]

enne@chromium.org changed reviewers:
+ dbeam@chromium.org, hcarmona@chromium.org

[enne (OOO), 2015-03-04 19:24:00]

This is a big hammer, but I spotted quite a few other tests in this file which
had been similarly disabled, so this is clearly an ongoing issue.

It seems like somebody needs to look into why these tests are timing out, and
the ongoing costs of investigating flaky tests shouldn't be externalized to
sheriffs and gardeners by letting these keep running.

[enne (OOO), 2015-03-06 18:52:07]

enne@chromium.org changed reviewers:
+ sergiu@chromium.org

[hcarmona, 2015-03-06 19:01:14]

https://codereview.chromium.org/975143002/diff/1/chrome/test/data/webui/histo...
File chrome/test/data/webui/history_browsertest.js (right):

https://codereview.chromium.org/975143002/diff/1/chrome/test/data/webui/histo...
chrome/test/data/webui/history_browsertest.js:369: // Times out on Mac:
http://crbug.com/336845
Why not disable only on the failing platform?

GEN('#if defined(OS_MACOSX)');
GEN('#define MAYBE_emptyHistory DISABLED_emptyHistory');
GEN('#else');
GEN('#define MAYBE_emptyHistory emptyHistory');
GEN('#endif  // defined(OS_MACOSX)');
TEST_F('HistoryWebUIFakeBackendTest', 'MAYBE_emptyHistory', function() {

https://codereview.chromium.org/975143002/diff/1/chrome/test/data/webui/histo...
chrome/test/data/webui/history_browsertest.js:707: // Times out on Mac and Win:
http://crbug.com/336845
Similarly here:

GEN('#if defined(OS_MACOSX) || defined(OS_WIN)');
...

[enne (OOO), 2015-03-06 20:29:43]

PTAL

[Dan Beam, 2015-03-06 20:34:18]

lgtm, but i highly suspect:
- webui is slow to load because it runs a lot (renderer, browser, etc.)
 - whenever the bot load gets high, these'll be the first to timeout
- the issue is not platform dependent
 - up to you whether #ifdef spam is worth it

[enne (OOO), 2015-03-09 17:05:10]

Are these really timeouts? There some errors in the logs (messages too big to be
sent, if I remember correctly), which naively seem like a legit failure rather
than a timeout.

I'm happy to land with or without #ifdefs.  I'd prefer without, as I agree that
the issue does not seem platform dependent.

[hcarmona, 2015-03-09 17:15:17]

On 2015/03/09 17:05:10, enne wrote:
> Are these really timeouts? There some errors in the logs (messages too big to
be
> sent, if I remember correctly), which naively seem like a legit failure rather
> than a timeout.
> 
> I'm happy to land with or without #ifdefs.  I'd prefer without, as I agree
that
> the issue does not seem platform dependent.

It makes sense to commit without the #ifdefs if the issue is platform
independent.

[dmazzoni, 2015-03-09 17:22:42]

There are a lot of warnings in the logs, I have no idea what they could be
- and I'm not entirely sure they're related.

I agree that they don't look like the test is just taking too long. Either
one of the errors/warnings is the real issue, or there's a race condition
somewhere and the tests sometimes don't finish.

Hector, can you look into trying to reproduce and fix the issue?


On Mon, Mar 9, 2015 at 10:15 AM, <hcarmona@chromium.org> wrote:

> On 2015/03/09 17:05:10, enne wrote:
>
>> Are these really timeouts? There some errors in the logs (messages too
>> big to
>>
> be
>
>> sent, if I remember correctly), which naively seem like a legit failure
>> rather
>> than a timeout.
>>
>
>  I'm happy to land with or without #ifdefs.  I'd prefer without, as I agree
>>
> that
>
>> the issue does not seem platform dependent.
>>
>
> It makes sense to commit without the #ifdefs if the issue is platform
> independent.
>
>
> https://codereview.chromium.org/975143002/
>
> To unsubscribe from this group and stop receiving emails from it, send an
> email to chromium-reviews+unsubscribe@chromium.org.
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[Dan Beam, 2015-03-09 17:50:42]

i see no compelling reason to think these tests are overly flakey or failing
much from:
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=...

what logs are you talking about?

[dmazzoni, 2015-03-09 18:00:11]

The flakiness dashboard seems to have pretty bad coverage these days. This
is far more comprehensive (find it from UsefulURLs
<https://code.google.com/p/chromium/wiki/UsefulURLs>):

http://chromium-build-logs.appspot.com/gtest_query?gtest_query=HistoryWebUIFa...


On Mon, Mar 9, 2015 at 10:50 AM, <dbeam@chromium.org> wrote:

> i see no compelling reason to think these tests are overly flakey or
> failing
> much from:
> http://test-results.appspot.com/dashboards/flakiness_
> dashboard.html#testType=browser_tests&tests=HistoryWebUIFakeBackendTest.
> emptyHistory
>
> what logs are you talking about?
>
>
> https://codereview.chromium.org/975143002/
>
> To unsubscribe from this group and stop receiving emails from it, send an
> email to chromium-reviews+unsubscribe@chromium.org.
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[enne (OOO), 2015-03-09 18:34:01]

I uploaded a patch without the #ifdefs.

[Dan Beam, 2015-03-09 18:42:06]

On 2015/03/09 18:00:11, dmazzoni wrote:
> The flakiness dashboard seems to have pretty bad coverage these days. This
> is far more comprehensive (find it from UsefulURLs
> <https://code.google.com/p/chromium/wiki/UsefulURLs>):
> 
>
http://chromium-build-logs.appspot.com/gtest_query?gtest_query=HistoryWebUIFa...

enne@: are you looking at these same logs?  nothing here looks like a useful
stack and most are time outs (exactly as I predicted).

[enne (OOO), 2015-03-09 18:46:26]

On 2015/03/09 at 18:42:06, dbeam wrote:
> enne@: are you looking at these same logs?  nothing here looks like a useful
stack and most are time outs (exactly as I predicted).

Maybe this is a common warning, but this is what I was referring to:

[1021:28931:0309/065242:WARNING:raw_channel_posix.cc(281)] sendmsg/write/writev:
Bad file descriptor
[1021:28931:0309/065242:WARNING:channel.cc(314)] RawChannel write error
[1024:12551:0309/065242:WARNING:channel.cc(547)] Failed to send message to ack
remove remote endpoint (local ID 1, remot (truncated by parser)
[1024:12551:0309/065242:WARNING:channel.cc(547)] Failed to send message to ack
remove remote endpoint (local ID 21474836 (truncated by parser)
[1024:12551:0309/065242:WARNING:channel.cc(314)] RawChannel write error

Most of the stacks are a timeout, but I was trying to clarify that the warnings
made me suspicious that it was a bug as opposed to just a slowness timeout.

[Dan Beam, 2015-03-09 18:54:02]

On 2015/03/09 18:46:26, enne wrote:
> On 2015/03/09 at 18:42:06, dbeam wrote:
> > enne@: are you looking at these same logs?  nothing here looks like a useful
> stack and most are time outs (exactly as I predicted).
> 
> Maybe this is a common warning, but this is what I was referring to:
> 
> [1021:28931:0309/065242:WARNING:raw_channel_posix.cc(281)]
sendmsg/write/writev:
> Bad file descriptor
> [1021:28931:0309/065242:WARNING:channel.cc(314)] RawChannel write error
> [1024:12551:0309/065242:WARNING:channel.cc(547)] Failed to send message to ack
> remove remote endpoint (local ID 1, remot (truncated by parser)
> [1024:12551:0309/065242:WARNING:channel.cc(547)] Failed to send message to ack
> remove remote endpoint (local ID 21474836 (truncated by parser)
> [1024:12551:0309/065242:WARNING:channel.cc(314)] RawChannel write error
> 
> Most of the stacks are a timeout, but I was trying to clarify that the
warnings
> made me suspicious that it was a bug as opposed to just a slowness timeout.

none of those are useful/related to these tests, as far as I know

[enne (OOO), 2015-03-10 18:36:36]

There's a lot of discussion, but I'd like to commit this without the #ifdefs if
there aren't objections.  Maybe the timeouts aren't the fault of these tests,
but they're still flaking.

[Dan Beam, 2015-03-10 18:59:49]

On 2015/03/09 17:05:10, enne wrote:
> Are these really timeouts? There some errors in the logs (messages too big to
be
> sent, if I remember correctly), which naively seem like a legit failure rather
> than a timeout.

the messages you sent in #c16 are unrelated.  which other "legit failures" can
you show me?

[Dan Beam, 2015-03-10 19:00:42]

On 2015/03/09 17:22:42, dmazzoni wrote:
> There are a lot of warnings in the logs, I have no idea what they could be
> - and I'm not entirely sure they're related.
> 
> I agree that they don't look like the test is just taking too long. Either
> one of the errors/warnings is the real issue, or there's a race condition
> somewhere and the tests sometimes don't finish.

which of these looks like a race or real issue?  i looked through ~30 and didn't
see anything useful.

[Dan Beam, 2015-03-10 19:02:18]

BUG=357910 seems unrelated

[enne (OOO), 2015-03-10 19:59:30]

On 2015/03/10 at 19:02:18, dbeam wrote:
> BUG=357910 seems unrelated

Fixed to 375910.

[enne (OOO), 2015-03-10 20:03:32]

On 2015/03/10 at 19:00:42, dbeam wrote:
> which of these looks like a race or real issue?  i looked through ~30 and
didn't see anything useful.

I looked through a number and the failures all seemed similar to the above logs
I linked.  So, maybe there's no indication of what the underlying problem is
from the logs.

However, these tests are still flaking and timing out, so they should be
disabled and somebody can the investigate and fix whatever the underlying cause
is.

[Dan Beam, 2015-03-10 20:04:16]

On 2015/03/10 20:03:32, enne wrote:
> On 2015/03/10 at 19:00:42, dbeam wrote:
> > which of these looks like a race or real issue?  i looked through ~30 and
> didn't see anything useful.
> 
> I looked through a number and the failures all seemed similar to the above
logs
> I linked.  So, maybe there's no indication of what the underlying problem is
> from the logs.
> 
> However, these tests are still flaking and timing out, so they should be
> disabled and somebody can the investigate and fix whatever the underlying
cause
> is.

that's fine, you have my lgtm

[enne (OOO), 2015-03-11 18:33:01]

The CQ bit was checked by enne@chromium.org

[commit-bot: I haz the power, 2015-03-11 18:33:26]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/975143002/40001

[commit-bot: I haz the power, 2015-03-11 20:45:45]

Committed patchset #3 (id:40001)

[commit-bot: I haz the power, 2015-03-11 20:46:41]

Patchset 3 (id:??) landed as
https://crrev.com/c12345f315ad642ebbd0bba87b26ad3642025e6d
Cr-Commit-Position: refs/heads/master@{#320141}