Omnibox: Create Disable-Inlining Experiment

Omnibox: Create Disable-Inlining Experiment

Sets up a experiment that, when active, demotes all suggestions with
inline autocompletions so that a suggestion without one (or, more precisely,
a suggestion with an empty inline autocompletion) appears first / within
the omnibox.

BUG=

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=273966

[Mark P, 2014-05-29 00:07:36]

Peter,

Are you comfortable with where I put this code?

If so, I'll add a test for it.

--mark

[Peter Kasting, 2014-05-29 00:12:46]

I don't know if this will have the kind of effect you want, because if the
providers are told that they can inline autocomplete, then for example UNKNOWN
inputs generally won't generate a history what-you-typed match.  I worry that
maybe there might be enough cases where there's no apparent legal match as to
skew the results we get.

I wonder if it would be better to simply try to make the experiment force
|prevent_inline_autocomplete| on except for users with no default search engine.

[Mark P, 2014-05-29 02:54:13]

On Wed, May 28, 2014 at 5:12 PM, <pkasting@chromium.org> wrote:

> I don't know if this will have the kind of effect you want, because if the
> providers are told that they can inline autocomplete, then for example
> UNKNOWN
> inputs generally won't generate a history what-you-typed match.  I worry
> that
> maybe there might be enough cases where there's no apparent legal match as
> to
> skew the results we get.


This is intentional.  I want to make this change have as little impact on
what suggestions are displayed and their ranking as possible under the
constraint that I would like to disable inlining (when feasible).  That why
I don't want to create more, say, UWYT you matches than we previously do.

In addition, SearchProvider always returns a legal default match.  It also
generally reserves extra space for the what-you-typed suggestion.
 Admittedly, sometimes these can be beaten out by local search history.  In
reality, I'm not particularly concerned.  Almost always the SWYT match gets
returned.  Hence, I believe this change behaves close enough to as I want
it behave (i.e., will almost always disable inlining) so that the metrics
will reflect a good approximation of the importance of this feature.

--mark


>


> I wonder if it would be better to simply try to make the experiment force
> |prevent_inline_autocomplete| on except for users with no default search
> engine.
>
> https://codereview.chromium.org/303013002/
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[Mark P, 2014-05-29 03:18:05]

In realized I never answered your second paragraph.  Good idea.  I forgot
to consider the prevent_inline_autocomplete solution.

However, now that I consider it, I don't like it.
 prevent_inline_autocomplete effectively kills HistoryURL scoring, which
means this parameter changes the ranking.  Try it with about:omnibox.  As I
said, I'd prefer to make this change cause the omnibox to display
suggestions in as close an order as possible to that without this change,
just making sure a non-inline-autocomplete match is first.  This allows
testing as close as possible purely the effect of inlining.

--mark



On Wed, May 28, 2014 at 5:12 PM, <pkasting@chromium.org> wrote:

> I don't know if this will have the kind of effect you want, because if the
> providers are told that they can inline autocomplete, then for example
> UNKNOWN
> inputs generally won't generate a history what-you-typed match.  I worry
> that
> maybe there might be enough cases where there's no apparent legal match as
> to
> skew the results we get.
>
> I wonder if it would be better to simply try to make the experiment force
> |prevent_inline_autocomplete| on except for users with no default search
> engine.
>
> https://codereview.chromium.org/303013002/
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[Peter Kasting, 2014-05-29 18:25:46]

Will the HUP still return a WYT match for input type==URL when there are many
better, inline-autocompletable matches?  For example, say I have many really
good matches deep inside foo.com, and I type foo.com, and the HUP wants to score
all these deep links more highly.  Will there still be a "foo.com" match to
promote?  If so, then I think you're right that this change will probably
disable inline autocompletion most of the time.  Otherwise, I'm worried.

Regardless, I think you should also add code that can log the number of times
when this experiment is on but we inline autocomplete something anyway.  That
will tell us whether these assumptions are correct.

[Mark P, 2014-05-29 18:35:18]

On Thu, May 29, 2014 at 11:25 AM, <pkasting@chromium.org> wrote:

> Will the HUP still return a WYT match for input type==URL when there are
> many
> better, inline-autocompletable matches?  For example, say I have many
> really
> good matches deep inside foo.com, and I type foo.com, and the HUP wants
> to score
> all these deep links more highly.  Will there still be a "foo.com" match
> to
> promote?


No.


> If so, then I think you're right that this change will probably
> disable inline autocompletion most of the time.  Otherwise, I'm worried.
>
> Regardless, I think you should also add code that can log the number of
> times
> when this experiment is on but we inline autocomplete something anyway.
>  That
> will tell us whether these assumptions are correct.
>

Let's try this as-is, with the logging you suggest.  We have many weeks to
run this on dev before the next branch point so we can see if this is close
enough or if we need to improve it.

-mark


>
> https://codereview.chromium.org/303013002/
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[Mark P, 2014-05-29 18:48:15]

On 2014/05/29 18:35:18, Mark P wrote:
> On Thu, May 29, 2014 at 11:25 AM, <mailto:pkasting@chromium.org> wrote:
> 
> > Will the HUP still return a WYT match for input type==URL when there are
> > many
> > better, inline-autocompletable matches?  For example, say I have many
> > really
> > good matches deep inside http://foo.com, and I type http://foo.com, and the
HUP wants
> > to score
> > all these deep links more highly.  Will there still be a "foo.com" match
> > to
> > promote?
> 
> 
> No.
> 
> 
> > If so, then I think you're right that this change will probably
> > disable inline autocompletion most of the time.  Otherwise, I'm worried.
> >
> > Regardless, I think you should also add code that can log the number of
> > times
> > when this experiment is on but we inline autocomplete something anyway.
> >  That
> > will tell us whether these assumptions are correct.
> >
> 
> Let's try this as-is, with the logging you suggest.  We have many weeks to
> run this on dev before the next branch point so we can see if this is close
> enough or if we need to improve it.

I added the histogram.

It's this changelist I meant to say:
  next step: add a test.

--mark

[Mark P, 2014-05-29 19:38:43]

On 2014/05/29 18:48:15, Mark P wrote:
> On 2014/05/29 18:35:18, Mark P wrote:
> > On Thu, May 29, 2014 at 11:25 AM, <mailto:pkasting@chromium.org> wrote:
> > 
> > > Will the HUP still return a WYT match for input type==URL when there are
> > > many
> > > better, inline-autocompletable matches?  For example, say I have many
> > > really
> > > good matches deep inside http://foo.com, and I type http://foo.com, and
the
> HUP wants
> > > to score
> > > all these deep links more highly.  Will there still be a "foo.com" match
> > > to
> > > promote?
> > 
> > 
> > No.
> > 
> > 
> > > If so, then I think you're right that this change will probably
> > > disable inline autocompletion most of the time.  Otherwise, I'm worried.
> > >
> > > Regardless, I think you should also add code that can log the number of
> > > times
> > > when this experiment is on but we inline autocomplete something anyway.
> > >  That
> > > will tell us whether these assumptions are correct.
> > >
> > 
> > Let's try this as-is, with the logging you suggest.  We have many weeks to
> > run this on dev before the next branch point so we can see if this is close
> > enough or if we need to improve it.
> 
> I added the histogram.
> 
> It's this changelist I meant to say:
>   next step: add a test.
> 
> --mark

Added a test.

PTAL.

--mark

[Peter Kasting, 2014-05-29 21:30:49]

LGTM

https://codereview.chromium.org/303013002/diff/40001/chrome/browser/autocompl...
File chrome/browser/autocomplete/autocomplete_result.cc (right):

https://codereview.chromium.org/303013002/diff/40001/chrome/browser/autocompl...
chrome/browser/autocomplete/autocomplete_result.cc:182: // inline
autocompletion, then in the disable inlining experiment it's
Nit: it's -> it

https://codereview.chromium.org/303013002/diff/40001/chrome/browser/autocompl...
chrome/browser/autocomplete/autocomplete_result.cc:204:
(!matches_.begin()->allowed_to_be_default_match ||
I might create a small helper functions which does:

  return it->allowed_to_be_default_match &&
      (!OmniboxFieldTrial::DisableInlining() ||
       !has_legal_default_match_without_completion ||
       it->inline_autocompletion.empty());

This could be used (negated) here and below.

[Mark P, 2014-05-30 00:03:58]

https://codereview.chromium.org/303013002/diff/40001/chrome/browser/autocompl...
File chrome/browser/autocomplete/autocomplete_result.cc (right):

https://codereview.chromium.org/303013002/diff/40001/chrome/browser/autocompl...
chrome/browser/autocomplete/autocomplete_result.cc:182: // inline
autocompletion, then in the disable inlining experiment it's
On 2014/05/29 21:30:49, Peter Kasting wrote:
> Nit: it's -> it

Done.  Also adjusted wrapping.

https://codereview.chromium.org/303013002/diff/40001/chrome/browser/autocompl...
chrome/browser/autocomplete/autocomplete_result.cc:204:
(!matches_.begin()->allowed_to_be_default_match ||
On 2014/05/29 21:30:49, Peter Kasting wrote:
> I might create a small helper functions which does:
> 
>   return it->allowed_to_be_default_match &&
>       (!OmniboxFieldTrial::DisableInlining() ||
>        !has_legal_default_match_without_completion ||
>        it->inline_autocompletion.empty());
> 
> This could be used (negated) here and below.

Done.  The function name is awkward though.  If you have a better suggestion I'm
all ears.

[Mark P, 2014-05-30 19:13:35]

The CQ bit was checked by mpearson@chromium.org

[commit-bot: I haz the power, 2014-05-30 19:15:30]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/mpearson@chromium.org/303013002/60001

[commit-bot: I haz the power, 2014-05-31 00:52:22]

Change committed as 273966