MD Settings: deflake back/forward navigation test via session restore

MD Settings: deflake back/forward navigation test via session restore

R=creis@chromium.org
BUG=620370

[Dan Beam, 2016-06-22 02:09:41]

creis: how bad of an idea is this?

[Charlie Reis, 2016-06-22 18:32:29]

creis@chromium.org changed reviewers:
+ nick@chromium.org

[Charlie Reis, 2016-06-22 18:32:29]

[+nick]

On 2016/06/22 02:09:41, Dan Beam wrote:
> creis: how bad of an idea is this?

Yeah, fabricating a session history in a browser test (to avoid flakiness from
just navigating and going back) does seem like the wrong thing to do.  Like a
unit test, it could miss all kinds of realistic state, like the contents of
PageState or bindings.

Nick has been looking at how to make waiting for navigations more reliable in
tests.  Maybe he has some ideas about what might be going wrong in the original
test?

[Dan Beam, 2016-06-23 21:12:44]

On 2016/06/22 18:32:29, Charlie Reis wrote:
> [+nick]
> 
> On 2016/06/22 02:09:41, Dan Beam wrote:
> > creis: how bad of an idea is this?
> 
> Yeah, fabricating a session history in a browser test (to avoid flakiness from
> just navigating and going back) does seem like the wrong thing to do.  Like a
> unit test, it could miss all kinds of realistic state, like the contents of
> PageState or bindings.
> 
> Nick has been looking at how to make waiting for navigations more reliable in
> tests.  Maybe he has some ideas about what might be going wrong in the
original
> test?

it's not really about reliability.  it's more about speed.  loading md-settings
-> newtab -> md-settings is just too slow on memory bots :(

[Charlie Reis, 2016-06-24 23:36:54]

On 2016/06/23 21:12:44, Dan Beam wrote:
> On 2016/06/22 18:32:29, Charlie Reis wrote:
> > [+nick]
> > 
> > On 2016/06/22 02:09:41, Dan Beam wrote:
> > > creis: how bad of an idea is this?
> > 
> > Yeah, fabricating a session history in a browser test (to avoid flakiness
from
> > just navigating and going back) does seem like the wrong thing to do.  Like
a
> > unit test, it could miss all kinds of realistic state, like the contents of
> > PageState or bindings.
> > 
> > Nick has been looking at how to make waiting for navigations more reliable
in
> > tests.  Maybe he has some ideas about what might be going wrong in the
> original
> > test?
> 
> it's not really about reliability.  it's more about speed.  loading
md-settings
> -> newtab -> md-settings is just too slow on memory bots :(

Oh, that's unfortunate.  Does the middle page need to be the NTP for the test to
be meaningful, or could it be about:blank?

The fake history approach could probably be made to work, but it seems likely to
be brittle.  Any time we change what's expected on a NavigationEntry, we might
have to come update this test as well.

[Dan Beam, 2016-06-24 23:46:15]

On 2016/06/24 23:36:54, Charlie Reis wrote:
> On 2016/06/23 21:12:44, Dan Beam wrote:
> > On 2016/06/22 18:32:29, Charlie Reis wrote:
> > > [+nick]
> > > 
> > > On 2016/06/22 02:09:41, Dan Beam wrote:
> > > > creis: how bad of an idea is this?
> > > 
> > > Yeah, fabricating a session history in a browser test (to avoid flakiness
> from
> > > just navigating and going back) does seem like the wrong thing to do. 
Like
> a
> > > unit test, it could miss all kinds of realistic state, like the contents
of
> > > PageState or bindings.
> > > 
> > > Nick has been looking at how to make waiting for navigations more reliable
> in
> > > tests.  Maybe he has some ideas about what might be going wrong in the
> > original
> > > test?
> > 
> > it's not really about reliability.  it's more about speed.  loading
> md-settings
> > -> newtab -> md-settings is just too slow on memory bots :(
> 
> Oh, that's unfortunate.  Does the middle page need to be the NTP for the test
to
> be meaningful, or could it be about:blank?

about:blank is a better choice, but I highly doubt that's the real issue.  the
real issue is loading md-settings twice.

> 
> The fake history approach could probably be made to work, but it seems likely
to
> be brittle.  Any time we change what's expected on a NavigationEntry, we might
> have to come update this test as well.

yep, don't really know a way around this

[ncarter (slow), 2016-06-27 17:33:01]

On 2016/06/24 23:46:15, Dan Beam wrote:
> On 2016/06/24 23:36:54, Charlie Reis wrote:
> > On 2016/06/23 21:12:44, Dan Beam wrote:
> > > On 2016/06/22 18:32:29, Charlie Reis wrote:
> > > > [+nick]
> > > > 
> > > > On 2016/06/22 02:09:41, Dan Beam wrote:
> > > > > creis: how bad of an idea is this?
> > > > 
> > > > Yeah, fabricating a session history in a browser test (to avoid
flakiness
> > from
> > > > just navigating and going back) does seem like the wrong thing to do. 
> Like
> > a
> > > > unit test, it could miss all kinds of realistic state, like the contents
> of
> > > > PageState or bindings.
> > > > 
> > > > Nick has been looking at how to make waiting for navigations more
reliable
> > in
> > > > tests.  Maybe he has some ideas about what might be going wrong in the
> > > original
> > > > test?
> > > 
> > > it's not really about reliability.  it's more about speed.  loading
> > md-settings
> > > -> newtab -> md-settings is just too slow on memory bots :(
> > 
> > Oh, that's unfortunate.  Does the middle page need to be the NTP for the
test
> to
> > be meaningful, or could it be about:blank?
> 
> about:blank is a better choice, but I highly doubt that's the real issue.  the
> real issue is loading md-settings twice.
> 
> > 
> > The fake history approach could probably be made to work, but it seems
likely
> to
> > be brittle.  Any time we change what's expected on a NavigationEntry, we
might
> > have to come update this test as well.
> 
> yep, don't really know a way around this

What if we don't wait for load stop, just for commit, for the first navigation
to md-settings?

[Dan Beam, 2016-09-15 21:27:06]

Description was changed from

==========
MD Settings: deflake back/forward navigation test via session restore

R=creis@chromium.org
BUG=620370
==========

to

==========
MD Settings: deflake back/forward navigation test via session restore

R=creis@chromium.org
BUG=620370
==========