Flag mouse messages received when cursor is hidden.

Flag mouse messages received when cursor is hidden.

<Begin preamble to explain the problem:>
Many Windows messages get generated after a touch event, including a
number of mouse messages. Some messages are flagged from touch, while
others are not flagged by the system. For example, WM_MOUSEMOVE is sent
long after the touch event finishes, without flags.

Previously, we tested to see if WM_MOUSEMOVE is fired when the cursor
has not moved (this is done in IsSynthesizedMouseMessage), within a
fixed timeout period from the last touch event. This strategy captures
some synthesized events, but some are sent long after the timeout,
while others don't match this test [failure cases WM_MOUSEEXIT that
does move the cursor.]

A further failure case arose when touches happen very quickly, causing
apparent mouse movement on unflagged mouse events. Experimentally, we
found that tracking a number of recent touch locations would let us
flag all (most?) synthesized mouse events, but the number of locations
we needed to track was sometimes very large. This touch tracking seems
very hacky, and I'm not confident that it would ever work reliably or
efficiently. [This case degnerates badly with multitouch.]
</end preamble.>

Independently, Windows hides the mouse cursor when touch is active,
and restores it when the mouse is active. This patch flags mouse
events that are fired when the cursor is hidden. Chromium ui elements
(such as hover states or tooltips) can choose to ignore such events.

BUG=699668

[girard, 2017-06-06 21:05:04]

girard@chromium.org changed reviewers:
+ ananta@chromium.org, sadrul@chromium.org

[girard, 2017-06-06 21:05:05]

Ananta, please review this change to windows touch logic - it removes some of
your previous code.

Sadrul, please review as owner.

[sadrul, 2017-06-07 22:09:00]

> Chromium ui elements (such as hover states or tooltips) can
> choose to ignore such events.

Can we just not dispatch these mouse events generated while the cursor is
hidden? (i.e. filter these out in HWNDMessageHandler)

[girard, 2017-06-08 15:57:24]

On 2017/06/07 22:09:00, sadrul wrote:
> > Chromium ui elements (such as hover states or tooltips) can
> > choose to ignore such events.
> 
> Can we just not dispatch these mouse events generated while the cursor is
> hidden? (i.e. filter these out in HWNDMessageHandler)

When I tried that the mouse disappeared forever (cursor gets restored as
a side effect of a WM_MOUSEENTER or WM_MOUSEMOVE message - and we can't 
distinguish mouse-generated events from touch-generated events).

[ananta, 2017-07-12 18:40:55]

I am a touch worried that the list of controls choosing not to handle these
messages is going to grow. +1 to sadrul's suggestion of potentially not
dispatching the mouse messages  if the cursor is hidden

[girard, 2017-08-17 19:49:23]

On 2017/07/12 18:40:55, ananta wrote:
> I am a touch worried that the list of controls choosing not to handle these
> messages is going to grow. +1 to sadrul's suggestion of potentially not
> dispatching the mouse messages  if the cursor is hidden

Not sure how to accomplish this. As I stated above, we lose the cursor if we
drop these messages. 

I've written a detailed explanation at
https://docs.google.com/document/d/1N3CR0RQYGCJpA8ddaPytyj9ibDvS_FS5DVcf0UQuaRI/

[sadrul, 2017-08-23 06:34:20]

sadrul@chromium.org changed reviewers:
+ sky@chromium.org

[sadrul, 2017-08-23 06:34:21]

I am going to hand this off to sky@

(sorry about sitting on it for so long)

[sadrul, 2017-08-23 06:34:37]

Description was changed from

==========
Flag mouse messages received when cursor is hidden.

<Begin preamble to explain the problem:>
Many Windows messages get generated after a touch event, including a
number of mouse messages. Some messages are flagged from touch, while
others are not flagged by the system. For example, WM_MOUSEMOVE is sent
long after the touch event finishes, without flags.

Previously, we tested to see if WM_MOUSEMOVE is fired when the cursor
has not moved (this is done in IsSynthesizedMouseMessage), within a
fixed timeout period from the last touch event. This strategy captures
some synthesized events, but some are sent long after the timeout,
while others don't match this test [failure cases WM_MOUSEEXIT that
does move the cursor.]

A further failure case arose when touches happen very quickly, causing
apparent mouse movement on unflagged mouse events. Experimentally, we
found that tracking a number of recent touch locations would let us
flag all (most?) synthesized mouse events, but the number of locations
we needed to track was sometimes very large. This touch tracking seems
very hacky, and I'm not confident that it would ever work reliably or
efficiently. [This case degnerates badly with multitouch.]
</end preamble.>

Independently, Windows hides the mouse cursor when touch is active,
and restores it when the mouse is active. This patch flags mouse
events that are fired when the cursor is hidden. Chromium ui elements
(such as hover states or tooltips) can choose to ignore such events.

BUG=699668
==========

to

==========
Flag mouse messages received when cursor is hidden.

<Begin preamble to explain the problem:>
Many Windows messages get generated after a touch event, including a
number of mouse messages. Some messages are flagged from touch, while
others are not flagged by the system. For example, WM_MOUSEMOVE is sent
long after the touch event finishes, without flags.

Previously, we tested to see if WM_MOUSEMOVE is fired when the cursor
has not moved (this is done in IsSynthesizedMouseMessage), within a
fixed timeout period from the last touch event. This strategy captures
some synthesized events, but some are sent long after the timeout,
while others don't match this test [failure cases WM_MOUSEEXIT that
does move the cursor.]

A further failure case arose when touches happen very quickly, causing
apparent mouse movement on unflagged mouse events. Experimentally, we
found that tracking a number of recent touch locations would let us
flag all (most?) synthesized mouse events, but the number of locations
we needed to track was sometimes very large. This touch tracking seems
very hacky, and I'm not confident that it would ever work reliably or
efficiently. [This case degnerates badly with multitouch.]
</end preamble.>

Independently, Windows hides the mouse cursor when touch is active,
and restores it when the mouse is active. This patch flags mouse
events that are fired when the cursor is hidden. Chromium ui elements
(such as hover states or tooltips) can choose to ignore such events.

BUG=699668
==========

[sadrul, 2017-08-23 06:34:37]

sadrul@chromium.org changed reviewers:
- sadrul@chromium.org

[sky, 2017-08-23 15:44:16]

+1 to some how eating the fake mouse messages. I get that missing one may
mean the mouse stays hidden, but if the user is really moving the mouse
won't we end up with a slew of mouse events that will trigger showing the
cursor? I know if it were me, I would end up shaking the mouse in such a
situation.

  -Scott

On Thu, Aug 17, 2017 at 12:49 PM, <girard@chromium.org> wrote:

> On 2017/07/12 18:40:55, ananta wrote:
> > I am a touch worried that the list of controls choosing not to handle
> these
> > messages is going to grow. +1 to sadrul's suggestion of potentially not
> > dispatching the mouse messages if the cursor is hidden
>
> Not sure how to accomplish this. As I stated above, we lose the cursor if
> we
> drop these messages.
>
> I've written a detailed explanation at
> https://docs.google.com/document/d/1N3CR0RQYGCJpA8ddaPytyj9ibDvS_
> FS5DVcf0UQuaRI/
>
> https://codereview.chromium.org/2925853002/
>

-- 
You received this message because you are subscribed to the Google Groups
"Chromium-reviews" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[girard, 2017-08-24 21:04:10]

On 2017/08/23 15:44:16, sky wrote:
> +1 to some how eating the fake mouse messages. I get that missing one may
> mean the mouse stays hidden, but if the user is really moving the mouse
> won't we end up with a slew of mouse events that will trigger showing the
> cursor? I know if it were me, I would end up shaking the mouse in such a
> situation.

Unfortunately, no amount of mouse shaking would work, since we'd be
discarding all of them. At the point where we're deciding to eat the fakes,
we have no way to tell if they're fake. And therefore we'd eat them all,
including the one that restores the cursor. 

Unless you're suggesting that we only eat the first n WM_MOUSEMOVE messages?

[sky, 2017-08-24 23:34:10]

I was hoping there was some heuristic we could use that is mostly right.
Say location is same as recent touch event?

  -Scott

On Thu, Aug 24, 2017 at 2:04 PM, <girard@chromium.org> wrote:

> On 2017/08/23 15:44:16, sky wrote:
> > +1 to some how eating the fake mouse messages. I get that missing one may
> > mean the mouse stays hidden, but if the user is really moving the mouse
> > won't we end up with a slew of mouse events that will trigger showing the
> > cursor? I know if it were me, I would end up shaking the mouse in such a
> > situation.
>
> Unfortunately, no amount of mouse shaking would work, since we'd be
> discarding all of them. At the point where we're deciding to eat the fakes,
> we have no way to tell if they're fake. And therefore we'd eat them all,
> including the one that restores the cursor.
>
> Unless you're suggesting that we only eat the first n WM_MOUSEMOVE
> messages?
>
> https://codereview.chromium.org/2925853002/
>

-- 
You received this message because you are subscribed to the Google Groups
"Chromium-reviews" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[girard, 2017-08-25 18:44:27]

On 2017/08/24 23:34:10, sky wrote:
> I was hoping there was some heuristic we could use that is mostly right.
> Say location is same as recent touch event?

I worked with that for a while, and found a few troublesome elements
 1) Sometimes WM_MOUSEMOVE is sent for an earlier touch location, such as
the third in a series of five for a touch move. I worked around this by
keeping a large buffer of touch locations.
 2) Things get more confusing when multitouch gets involved, but the work-
around in (1) kind of addresses this.
 3) Sometimes the mouse event gets sent at a point that is near the touch
event - off by up to 3-5 pixels. I think this might be related to a
center-of-finger calculation that is different for touch than for mouse? 

I ended up with a rolling buffer of 20 recent touch points, with a "fuzz 
factor" of 5 pixels, and discarded any WM_MOUSEMOVE that got matched. Then
I put a breakpoint on the OnMouseMove and hammered away with my fingers. I 
was able to get a touch to mis-fire every five seconds or so (meaning I
was catching something like 99% of the fake mouse events).

I could code this up again if you want to see it, but the rolling buffer 
and the fuzzing seemed pretty hacky. (Ick!)

> 
>   -Scott
> 
> On Thu, Aug 24, 2017 at 2:04 PM, <mailto:girard@chromium.org> wrote:
> 
> > On 2017/08/23 15:44:16, sky wrote:
> > > +1 to some how eating the fake mouse messages. I get that missing one may
> > > mean the mouse stays hidden, but if the user is really moving the mouse
> > > won't we end up with a slew of mouse events that will trigger showing the
> > > cursor? I know if it were me, I would end up shaking the mouse in such a
> > > situation.
> >
> > Unfortunately, no amount of mouse shaking would work, since we'd be
> > discarding all of them. At the point where we're deciding to eat the fakes,
> > we have no way to tell if they're fake. And therefore we'd eat them all,
> > including the one that restores the cursor.
> >
> > Unless you're suggesting that we only eat the first n WM_MOUSEMOVE
> > messages?
> >
> > https://codereview.chromium.org/2925853002/
> >
> 
> -- 
> You received this message because you are subscribed to the Google Groups
> "Chromium-reviews" group.
> To unsubscribe from this group and stop receiving emails from it, send an
email
> to mailto:chromium-reviews+unsubscribe@chromium.org.

[sky, 2017-08-25 21:51:10]

I'm still confused as to how we can make controls deal correctly with
EF_CURSOR_HIDDEN, but can't centrally ignore the event. By that I mean if
we know the mouse is generated when the cursor is hidden, then don't
dispatch it. Your concern there is that results in the cursor permanently
hidden. DesktopWindowTreeHostWin can force the cursor to be visible when
necessary.

  -Scott

On Fri, Aug 25, 2017 at 11:44 AM, <girard@chromium.org> wrote:

> On 2017/08/24 23:34:10, sky wrote:
> > I was hoping there was some heuristic we could use that is mostly right.
> > Say location is same as recent touch event?
>
> I worked with that for a while, and found a few troublesome elements
> 1) Sometimes WM_MOUSEMOVE is sent for an earlier touch location, such as
> the third in a series of five for a touch move. I worked around this by
> keeping a large buffer of touch locations.
> 2) Things get more confusing when multitouch gets involved, but the work-
> around in (1) kind of addresses this.
> 3) Sometimes the mouse event gets sent at a point that is near the touch
> event - off by up to 3-5 pixels. I think this might be related to a
> center-of-finger calculation that is different for touch than for mouse?
>
> I ended up with a rolling buffer of 20 recent touch points, with a "fuzz
> factor" of 5 pixels, and discarded any WM_MOUSEMOVE that got matched. Then
> I put a breakpoint on the OnMouseMove and hammered away with my fingers. I
> was able to get a touch to mis-fire every five seconds or so (meaning I
> was catching something like 99% of the fake mouse events).
>
> I could code this up again if you want to see it, but the rolling buffer
> and the fuzzing seemed pretty hacky. (Ick!)
>
> >
> > -Scott
> >
> > On Thu, Aug 24, 2017 at 2:04 PM, <mailto:girard@chromium.org> wrote:
> >
> > > On 2017/08/23 15:44:16, sky wrote:
> > > > +1 to some how eating the fake mouse messages. I get that missing
> one may
> > > > mean the mouse stays hidden, but if the user is really moving the
> mouse
> > > > won't we end up with a slew of mouse events that will trigger
> showing the
> > > > cursor? I know if it were me, I would end up shaking the mouse in
> such a
> > > > situation.
> > >
> > > Unfortunately, no amount of mouse shaking would work, since we'd be
> > > discarding all of them. At the point where we're deciding to eat the
> fakes,
> > > we have no way to tell if they're fake. And therefore we'd eat them
> all,
> > > including the one that restores the cursor.
> > >
> > > Unless you're suggesting that we only eat the first n WM_MOUSEMOVE
> > > messages?
> > >
> > > https://codereview.chromium.org/2925853002/
> > >
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Chromium-reviews" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> email
> > to mailto:chromium-reviews+unsubscribe@chromium.org.
>
>
>
> https://codereview.chromium.org/2925853002/
>

-- 
You received this message because you are subscribed to the Google Groups
"Chromium-reviews" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.