Implement autoplay gesture override experiment.

Source/core/html/HTMLMediaElement.h

 /*
  * Copyright (C) 2007, 2008, 2009, 2010, 2011, 2012, 2013 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
  * are met:
  * 1. Redistributions of source code must retain the above copyright
  *    notice, this list of conditions and the following disclaimer.
  * 2. Redistributions in binary form must reproduce the above copyright
  *    notice, this list of conditions and the following disclaimer in the
@@ skipping 268 matching lines @@
     void didMoveToNewDocument(Document& oldDocument) override;
     virtual KURL posterImageURL() const { return KURL(); }
 
     enum DisplayMode { Unknown, Poster, Video };
     DisplayMode displayMode() const { return m_displayMode; }
     virtual void setDisplayMode(DisplayMode mode) { m_displayMode = mode; }
 
     void setControllerInternal(MediaController*);
 
 private:
+    // These values are used for a histogram. Do not reorder.
+    enum AutoplayMetrics {
+        // Media element with autoplay seen.

[philipj_slow, 2015/08/05 10:03:12]

I'm not sure what this broad metric is for, but the documentation isn't
accurate, this counts both autoplay and play(). If it's possible to change the
enum names that could be more clear too.

[liberato (no reviews please), 2015/08/06 06:37:58]

i've added some comments and renamed some of the new ones to match.  i don't
want to fiddle with the existing ones, since they match up with descriptions
(stored elsewhere), that the original experiment owner might care about.

[philipj_slow, 2015/08/13 09:27:53]

I see, so entries 0-5 should be left as they are.

+        AutoplayMediaFound = 0,
+        // Autoplay enabled and user stopped media play at any point.
+        AutoplayStopped = 1,

[philipj_slow, 2015/08/05 10:03:12]

Isn't it necessary to have one stopped entry for each start entry in order to
compare how users react, including measuring the stop and bailout metrics for
when the experiment is disabled entirely. (Maybe people bailout a lot even when
starting a play deliberately?)

[liberato (no reviews please), 2015/08/06 06:37:58]

i gave this more thought today, but still believe that we don't need all the
cases.  I added one that counts any play/stop/bailout, whether user-initiated or
not.  that seems very useful, thanks!

as for the remainder, if this were the only round of measurement, then i would
agree.  however, we will be able to see which buckets have potentially
interesting things in them, and can design additional experiments more carefully
to target those buckets specifically.

[philipj_slow, 2015/08/13 09:27:53]

I don't think I understand, if we don't have any per-experiment data on bailout,
what can we learn about how users react to different kinds of restrictions?

[philipj_slow, 2015/09/02 09:35:37]

Ping. If separate buckets are needed, as it seems to me that they should be,
then not having them will influence the whole experiment. It's only if there's
some external mechanism not visibile inside Blink that pairs up the experiment
mode with these results and maps them into separate buckets elsewhere that I
would understand how it works.

+        // Autoplay enabled but user bailed out on media play early.
+        AutoplayBailout = 2,
+        // Autoplay disabled but user manually started media.
+        AutoplayManualStart = 3,
+        // Autoplay was (re)enabled through a user-gesture triggered load()
+        AutoplayEnabledThroughLoad = 4,
+        // Autoplay disabled by sandbox flags.
+        AutoplayDisabledBySandbox = 5,
+        // Autoplay started by experiment when media scrolled into view.  We don't
+        // record whether it was a javascript or attribute autoplay request.
+        AutoplayExperimentStartedByScroll = 6,
+        // Autoplay started by experiment during initial load.
+        AutoplayExperimentStartedByLoad = 7,

[philipj_slow, 2015/08/05 10:03:12]

I think s/Load/AutoplayAttribute/, it's not really the load that causes the
play, it's the autoplay attribute which causes the load, and reaching a certain
point in loading then plays. It's probably worth testing if this actually works
at all on Android, or if it's only actually calling play() that will start the
load and eventually play.

[liberato (no reviews please), 2015/08/06 06:37:58]

true, but this is from a previous experiment.  the text here currently lines up
with the description (elsewhere).  i'd prefer to keep them roughly in sync, and
i don't want to change the other description without the previous experiment
owner's agreement.

+        // Autoplay started by experiment in play() call.
+        AutoplayExperimentStartedByPlay = 8,
+        // play() failed to play due to gesture requirement.
+        AutoplayPlayFailed = 9,
+        // This enum value must be last.
+        NumberOfAutoplayMetrics,
+    };
+
     void resetMediaPlayerAndMediaSource();
 
     bool alwaysCreateUserAgentShadowRoot() const final { return true; }
     bool areAuthorShadowsAllowed() const final { return false; }
 
     bool supportsFocus() const final;
     bool isMouseFocusable() const final;
     bool layoutObjectIsNeeded(const ComputedStyle&) override;
     LayoutObject* createLayoutObject(const ComputedStyle&) override;
     InsertionNotificationRequest insertedInto(ContainerNode*) final;
@@ skipping 120 matching lines @@
 
     void changeNetworkStateFromLoadingToIdle();
 
     const AtomicString& mediaGroup() const;
     void setMediaGroup(const AtomicString&);
     void updateMediaController();
     bool isBlocked() const;
     bool isBlockedOnMediaController() const;
     bool isAutoplaying() const { return m_autoplaying; }
 
+    void recordAutoplayMetric(AutoplayMetrics);
+
+    // vvvv Helpers for clank autoplay investigation vvvv
+
+    // Install an event listener to check for changes in visibility.  If a

[philipj_slow, 2015/08/05 10:03:12]

Instead of the touchend/touchcancel+timeout solution, can you investigate
hooking into FrameView::scrollPositionChanged()?
setTextTrackKindUserPreferenceForAllMediaElements is another method that takes a
Document* and does something for all media elements within.

I don't think there's really a good way to know that a scroll has ended, so
perhaps just continually resetting a 200ms (?) timeout and doing the check when
it finally runs would work? Rick Byers would be a good person to ask about
better ways here.

[liberato (no reviews please), 2015/08/06 06:37:58]

based on all the comments around this, i've merged the two approaches.  we now
check for scroll events only, but use then to reset a 300msec timer. when the
timer fires, we declare that scrolling has stopped.  it does no work in the
event handler other than to set the timer, and seems to work quite well in
practice.

[Rick Byers, 2015/08/06 15:04:26]

This sounds like a good plan to me.  I'm sure websites do this sort of thing in
JS all the time.  You could of course debate over the timer value (it's not
uncommon for someone to pause briefly while scrolling with touch), but that's a
UI decision that should be entirely up to you I think.

+    // listener is already installed, then this does nothing.
+    void autoplayExperimentInstallEventListenerIfNeeded();
+
+    // Remove any event listener.  It's okay to call this if one isn't
+    // installed already.
+    void autoplayExperimentClearEventListenerIfNeeded();
+
+    // Return true if any only if this player meets (most) of the eligibility
+    // requirements for the experiment to override the need for a user
+    // gesture.  This includes everything except the visibility test.
+    bool autoplayExperimentIsEligible() const;
+
+    // Return true if and only if the player is visible.
+    bool autoplayExperimentIsVisible();
+
+    // Set the mute flag on the media if we're in an experiment mode that
+    // requires it, else do nothing.
+    void autoplayExperimentMuteIfNeeded();
+
+    // Maybe override the requirement for a user gesture, and start playing
+    // autoplay media.  Returns true if only if it starts playback.
+    bool autoplayExperimentMaybeStartPlaying();
+
+    // Configure internal state to record that the autoplay experiment is
+    // going to start playback.  This doesn't actually start playback, since
+    // there are several different cases.
+    void autoplayExperimentPrepareToPlay(AutoplayMetrics);
+
+    // Begin (or start over) a periodic check for visibility.  We will poll
+    // during this check to see if the video is in view.
+    void beginPeriodicVisibilityCheck();
+
+    // Process a timer for checking visibility.
+    void visibilityTimerFired(Timer<HTMLMediaElement>*);
+    // ^^^^ Helpers for clank autoplay investigation ^^^^
+
     WebMediaPlayer::CORSMode corsMode() const;
 
     // Returns the "direction of playback" value as specified in the HTML5 spec.
     enum DirectionOfPlayback { Backward, Forward };
     DirectionOfPlayback directionOfPlayback() const;
 
     // Returns the "effective playback rate" value as specified in the HTML5 spec.
     double effectivePlaybackRate() const;
 
     // Creates placeholder AudioTrack and/or VideoTrack objects when WebMemediaPlayer objects
@@ skipping 108 matching lines @@
     bool m_playingRemotely : 1;
     bool m_isFinalizing : 1;
     bool m_initialPlayWithoutUserGestures : 1;
     bool m_autoplayMediaCounted : 1;
 
     PersistentWillBeMember<AudioTrackList> m_audioTracks;
     PersistentWillBeMember<VideoTrackList> m_videoTracks;
     PersistentWillBeMember<TextTrackList> m_textTracks;
     PersistentHeapVectorWillBeHeapVector<Member<TextTrack>> m_textTracksWhenResourceSelectionBegan;
 
+    // Autoplay experiment state.
+    // True if we've received a play() without a pause().
+    bool m_autoplayExperimentPlayPending : 1;

[philipj_slow, 2015/08/05 10:03:12]

It looks like the delayed play is only done for explicit play() calls and not
for autoplay?

[liberato (no reviews please), 2015/08/06 06:37:58]

the delayed play will happen for autoplay attribute as well.  this just records
whether a play() has been issued without a subsequent pause().  the attr would
also cause a play in that case (...IsEligible())

[philipj_slow, 2015/08/13 09:27:53]

OK, so the mechanism for delayed play is different for autoplay and play().

+
+    // Autoplay experiment state.
+    // True if and only if we initiated playback because of the autoplay
+    // experiment.  Once set, this is never unset.
+    bool m_autoplayExperimentStartedByExperiment : 1;
+
+    // Autoplay experiment state.
+    // Touch listener for the autoplay experiment.
+    class AutoplayExperimentTouchListener;
+    friend class AutoplayExperimentTouchListener;
+    RefPtrWillBeMember<EventListener> m_autoplayExperimentTouchListener;
+
+    enum AutoplayExperimentMode {
+        ExperimentOff       = 0,
+        ExperimentEnabled   = 1,
+        ExperimentIfVisible = 2,
+        ExperimentIfMuted   = 4,
+        ExperimentIfMobile  = 8,
+        ExperimentPlayMuted = 16

[philipj_slow, 2015/08/05 10:03:12]

Bit fields are usually defined using 1 << n in Blink, see e.g. DelayedActionType
in this header.

[liberato (no reviews please), 2015/08/06 06:37:58]

Done.

+    };
+    int m_autoplayExperimentMode; // Bitwise-or of AutoplayExperimentMode
+
     OwnPtrWillBeMember<CueTimeline> m_cueTimeline;
 
 #if ENABLE(WEB_AUDIO)
     // This is a weak reference, since m_audioSourceNode holds a reference to us.
     // FIXME: Oilpan: Consider making this a strongly traced pointer with oilpan where strong cycles are not a problem.
     GC_PLUGIN_IGNORE("http://crbug.com/404577")
     RawPtrWillBeWeakMember<AudioSourceProviderClient> m_audioSourceNode;
 
     // AudioClientImpl wraps an AudioSourceProviderClient.
     // When the audio format is known, Chromium calls setFormat().
@@ skipping 38 matching lines @@
 
     private:
         WebAudioSourceProvider* m_webAudioSourceProvider;
         PersistentWillBeMember<AudioClientImpl> m_client;
         Mutex provideInputLock;
     };
 
     AudioSourceProviderImpl m_audioSourceProvider;
 #endif
 
+    Timer<HTMLMediaElement> m_autoplayVisibilityTimer;
+    double m_autoplayLastScrollX;
+    double m_autoplayLastScrollY;
+    double m_autoplayVisibilityTimerSpan;
+
     friend class MediaController;
     PersistentWillBeMember<MediaController> m_mediaController;
 
     friend class Internals;
     friend class TrackDisplayUpdateScope;
 
     static URLRegistry* s_mediaStreamRegistry;
 };
 
 inline bool isHTMLMediaElement(const HTMLElement& element)
 {
     return isHTMLAudioElement(element) || isHTMLVideoElement(element);
 }
 
 DEFINE_HTMLELEMENT_TYPE_CASTS_WITH_FUNCTION(HTMLMediaElement);
 
 } // namespace blink
 
 #endif // HTMLMediaElement_h