[WIP] Re-implement MediaControls in Blink-in-JS

Source/core/html/shadow/MediaControls.idl

+// Copyright 2014 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+interface MediaControls : HTMLDivElement {
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void createdCallback(HTMLElement mediaElement);
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void reset();
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void show();
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void mediaElementFocused();
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void hide();
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void playbackStarted();
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void playbackProgressed();
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void playbackStopped();
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void beginScrubbing();
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void endScrubbing();
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void updateCurrentTimeDisplay();
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void updateVolume();
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void changedClosedCaptionsVisibility();
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void refreshClosedCaptionsButtonVisibility();
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void closedCaptionTracksChanged();
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void enteredFullscreen();
+    [ImplementedInPrivateScript, OnlyExposedToPrivateScript] void exitedFullscreen();
+
+    [OnlyExposedToPrivateScript] DOMString getResourceDataURL(DOMString name);
+    [OnlyExposedToPrivateScript] boolean isOverlayPlayButtonEnabled();
+    [OnlyExposedToPrivateScript] boolean shouldShowControls();
+    [OnlyExposedToPrivateScript] void setDivDisplayType(HTMLElement element, unsigned long type);
+    [OnlyExposedToPrivateScript] void setInputDisplayType(HTMLElement element, unsigned long type);
+    [OnlyExposedToPrivateScript] boolean fullscreenIsSupported();
+    [OnlyExposedToPrivateScript] boolean hasAudio();
+    [OnlyExposedToPrivateScript] boolean hasVideo();
+    [OnlyExposedToPrivateScript] boolean hasClosedCaptions();
+    [OnlyExposedToPrivateScript] boolean togglePlayStateWillPlay();
+    [OnlyExposedToPrivateScript] void togglePlayState();
+    [OnlyExposedToPrivateScript] void setCurrentTime(double time);
+    [OnlyExposedToPrivateScript] boolean isClosedCaptionsVisible();
+    [OnlyExposedToPrivateScript] void toggleClosedCaptionsVisible();
+    [OnlyExposedToPrivateScript] void toggleFullscreen();
+    [OnlyExposedToPrivateScript] double getTextTrackContainerFontSize();
+    [OnlyExposedToPrivateScript] void updateTextTrackContainerDisplay(HTMLElement textTrackContainer);

[abarth-chromium, 2014/08/23 05:48:33]

This is way too many private APIs.  Why do you need private APIs?  Aren't there
sufficient public APIs on the video element?

[haraken, 2014/08/25 03:59:28]

As far as I look at the implementation, it seems hard to avoid adding these
private APIs if you keep HTMLMediaElement in C++ and move MediaControlsElement
to JavaScript. HTMLMediaElement and MediaControlsElement need to interact with
each other, and we need a private API to proxy the interaction.

One way to address the issue would be to move (a part of) HTMLMediaElement to
JavaScript as well and decrease the interaction that crosses the boundary
between C++ and JavaScript. How hard would it be?

[hajimehoshi, 2014/08/25 06:34:01]

That would be quite hard. Now MediaElement uses WebMediaPlayer and
MediaController. For example, if I tried to implement HTMLMediaElement::hasAudio
to Blink-in-JS, another backdoor would be needed since hasAudio uses
WebMediaPlayer. Another example is togglePlayState. This accesses
MediaController, so how can I re-implement this in Blink-in-JS?

I basically agree to haraken-san's idea. In the long term, some (or all?) of
implementation of HTMLMediaElement should be moved to Blink-in-JS and the above
backdoors should be disappeared.

In the short term, can we allow those backdoors temporarily? Of course I'll
continue to work on this.

[haraken, 2014/08/25 06:53:49]

I'm not really happy to introduce ad-hoc backdoors without having a big picture.
Shall we spend more time investigating where we should cut the boundary between
Blink-in-JS and C++?

For example, I think MediaController is something that should be moved to
Blink-in-JS with MediaControlsElement. That way you can avoid adding backdoors
to MediaController.

In my ideal world, we want to move the entire MediaControlsElement,
RenderMediaControls, MediaController and a part of HTMLMediaElement associated
with MediaControlsElement. That way, the only backdoors we'll need would just
be:

- Lifecycle callbacks of HTMLMediaElement (i.e., createdCallback,
attachedCallback etc).
- Interaction between HTMLMediaElement and WebMediaPlayer.
- createUserAgentShadowRoot().

Either way, let's spend more time investigating the ideal boundary and clarify
what backdoors are needed (i.e., what APIs are missing from the web platform). A
part of the goals of this project is to understand missing APIs of the web
platform. Thus it's more important to understand it rather than trying to make
it workable somehow.

[hajimehoshi, 2014/08/25 07:11:27]

Sure, MediaController seems to be relatively independent from other C++ classes
and worth moving to Blink-in-JS, so let me start with MediaController.

[philipj_slow, 2014/08/25 08:16:07]

Moving MediaController to Blink-in-JS doesn't sound like a good idea to me, to
be implemented properly that is eventually going to need some bits in C++. See
https://groups.google.com/a/chromium.org/d/msg/blink-dev/MVcoNSPs1UQ/LIF-fvu2...

As for all of the private APIs, e.g. playbackStarted, these bits should be
rewritten to use the standard events fired on HTMLMediaElement, play and/or
playing.

[hajimehoshi, 2014/08/25 09:11:18]

Thank you for the information!
Even though I rewrite these private API to use standard events, they just will
be replaced with methods like onplaybackstarted, right? It seems that the number
of methods won't change.

[philipj_slow, 2014/08/25 09:47:54]

If MediaControls.js does mediaElement.addEventListener("play", /* update the
play button, etc */) no additional API should be needed. I haven't done anything
with Blink-in-JS so possibly I'm not understanding something.

[hajimehoshi, 2014/08/26 04:33:51]

I see, but I'm afraid your suggested way doesn't work because these events would
be exposed to not only Blink-in-JS but users. I think API with
OnlyExposedToPrivateScript will be inevitable.

Seeing the current situation, it seems impossible to implement
MediaControleElement in Blink-in-JS without a number of backdoors. MediaControls
can't be separated off the other classes. As for the other classes,
HTMLMediaElement uses WebMediaPlayer, and MediaController is being developed in
C++ as Phillip said, so both are also difficult to implement in Blink-in-JS.

Adam, what do you think?

+};