Fix media stream playback event as media element.

Fix media stream playback event as media element.

When media stream is played with <video>, it needs to emit events as media element (defined at http://dev.w3.org/html5/spec-author-view/video.html#mediaevents). This patch fixes events: canplay, play, playing and timeupdate.

VideoFrameProvider will be started when player is loaded. HaveMetaData and HaveEnoughData will be sent out when the first frame is received in player.

In autoplay off mode, <video> for media stream will display black window till play is requrested.

BUG=156310, 142988, 110938
Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=163872

[scherkus (not reviewing), 2012-10-18 05:19:58]

OOC are there any automated tests to back this up? The bug links to what appears
to be a manual test.

[wjia(left Chromium), 2012-10-19 00:14:17]

On 2012/10/18 05:19:58, scherkus wrote:
> OOC are there any automated tests to back this up? The bug links to what
appears
> to be a manual test.

For now, there is no automated test for this timing issue. The layout test
video-capture-preview.html covers the events and their order, but not timing.
Need to figure out how to measure that in a test. If you have some idea, please
let me know.

I have a different question though: the previous code was designed so that when
autoplay is off, the camera won't start till play() is called. But when I check
regular <video> playback, it somehow has different behavior. I'd like to discuss
it offline.

[scherkus (not reviewing), 2012-10-19 02:07:34]

On 2012/10/19 00:14:17, wjia wrote:
> On 2012/10/18 05:19:58, scherkus wrote:
> > OOC are there any automated tests to back this up? The bug links to what
> appears
> > to be a manual test.
> 
> For now, there is no automated test for this timing issue. The layout test
> video-capture-preview.html covers the events and their order, but not timing.
> Need to figure out how to measure that in a test. If you have some idea,
please
> let me know.
> 
> I have a different question though: the previous code was designed so that
when
> autoplay is off, the camera won't start till play() is called. But when I
check
> regular <video> playback, it somehow has different behavior. I'd like to
discuss
> it offline.

Sure thing -- want to set a time on my calendar and we can VC?

[scherkus (not reviewing), 2012-10-19 22:16:37]

http://codereview.chromium.org/11198055/diff/1/webkit/media/webmediaplayer_ms.cc
File webkit/media/webmediaplayer_ms.cc (right):

http://codereview.chromium.org/11198055/diff/1/webkit/media/webmediaplayer_ms...
webkit/media/webmediaplayer_ms.cc:346: paused_ = false;
as discussed offline... you shouldn't set this

[wjia(left Chromium), 2012-10-21 22:11:12]

Thanks! PTAL. Now most of events will be fired after first frame is received.

http://codereview.chromium.org/11198055/diff/1/webkit/media/webmediaplayer_ms.cc
File webkit/media/webmediaplayer_ms.cc (right):

http://codereview.chromium.org/11198055/diff/1/webkit/media/webmediaplayer_ms...
webkit/media/webmediaplayer_ms.cc:346: paused_ = false;
On 2012/10/19 22:16:37, scherkus wrote:
> as discussed offline... you shouldn't set this

Done.

[scherkus (not reviewing), 2012-10-22 17:40:05]

thanks wei!

one more question then we can close this out

http://codereview.chromium.org/11198055/diff/13001/webkit/media/webmediaplaye...
File webkit/media/webmediaplayer_ms.cc (right):

http://codereview.chromium.org/11198055/diff/13001/webkit/media/webmediaplaye...
webkit/media/webmediaplayer_ms.cc:357: start_time_ = frame->GetTimestamp();
when we play, then pause, then play again won't time jump forward?

is that expected or will the frame provider adjust timestamps as necessary when
pausing/playing?

[wjia(left Chromium), 2012-10-22 18:26:21]

On 2012/10/22 17:40:05, scherkus wrote:
> thanks wei!
> 
> one more question then we can close this out
> 
>
http://codereview.chromium.org/11198055/diff/13001/webkit/media/webmediaplaye...
> File webkit/media/webmediaplayer_ms.cc (right):
> 
>
http://codereview.chromium.org/11198055/diff/13001/webkit/media/webmediaplaye...
> webkit/media/webmediaplayer_ms.cc:357: start_time_ = frame->GetTimestamp();
> when we play, then pause, then play again won't time jump forward?
> 
> is that expected or will the frame provider adjust timestamps as necessary
when
> pausing/playing?

Yes, the time will jump forward after resuming from pause for current
implementation.

It's not defined in spec. "Pause" is kind of weird for real-time communication
anyway. It might be mapped to "mute". The key is: what's the media time? Is it
tied with wall clock or something else such as audio? I think both are ok.

If "pause" also pauses master clock (i.e., media time), then VFP should adjust
timestamp after being paused. Otherwise, VFP doesn't need to change timestamp.
Current implementation takes the latter approach. This is ok with WebKit since
they don't do scheduling for video rendering. Another benefit is <video>
controls can show correct time for the meeting.

WDYT?

[scherkus (not reviewing), 2012-10-22 18:29:49]

On 2012/10/22 18:26:21, wjia wrote:
> On 2012/10/22 17:40:05, scherkus wrote:
> > thanks wei!
> > 
> > one more question then we can close this out
> > 
> >
>
http://codereview.chromium.org/11198055/diff/13001/webkit/media/webmediaplaye...
> > File webkit/media/webmediaplayer_ms.cc (right):
> > 
> >
>
http://codereview.chromium.org/11198055/diff/13001/webkit/media/webmediaplaye...
> > webkit/media/webmediaplayer_ms.cc:357: start_time_ = frame->GetTimestamp();
> > when we play, then pause, then play again won't time jump forward?
> > 
> > is that expected or will the frame provider adjust timestamps as necessary
> when
> > pausing/playing?
> 
> Yes, the time will jump forward after resuming from pause for current
> implementation.
> 
> It's not defined in spec. "Pause" is kind of weird for real-time communication
> anyway. It might be mapped to "mute". The key is: what's the media time? Is it
> tied with wall clock or something else such as audio? I think both are ok.
> 
> If "pause" also pauses master clock (i.e., media time), then VFP should adjust
> timestamp after being paused. Otherwise, VFP doesn't need to change timestamp.
> Current implementation takes the latter approach. This is ok with WebKit since
> they don't do scheduling for video rendering. Another benefit is <video>
> controls can show correct time for the meeting.
> 
> WDYT?

It sounds like it should be clarified in the spec but we don't need to hold up
this CL for now.

LGTM and can you look into clarifying the spec? (i.e., ask hta@?)

[wjia(left Chromium), 2012-10-22 20:56:39]

On Mon, Oct 22, 2012 at 11:29 AM, <scherkus@chromium.org> wrote:

> On 2012/10/22 18:26:21, wjia wrote:
>
>> On 2012/10/22 17:40:05, scherkus wrote:
>> > thanks wei!
>> >
>> > one more question then we can close this out
>> >
>> >
>>
>
> http://codereview.chromium.**org/11198055/diff/13001/**
>
webkit/media/webmediaplayer_**ms.cc<http://codereview.chromium.org/11198055/diff/13001/webkit/media/webmediaplayer_ms.cc>
>
>> > File webkit/media/webmediaplayer_**ms.cc (right):
>> >
>> >
>>
>
> http://codereview.chromium.**org/11198055/diff/13001/**
>
webkit/media/webmediaplayer_**ms.cc#newcode357<http://codereview.chromium.org/11198055/diff/13001/webkit/media/webmediaplayer_ms.cc#newcode357>
>
>> > webkit/media/webmediaplayer_**ms.cc:357: start_time_ =
>> frame->GetTimestamp();
>> > when we play, then pause, then play again won't time jump forward?
>> >
>> > is that expected or will the frame provider adjust timestamps as
>> necessary
>> when
>> > pausing/playing?
>>
>
>  Yes, the time will jump forward after resuming from pause for current
>> implementation.
>>
>
>  It's not defined in spec. "Pause" is kind of weird for real-time
>> communication
>> anyway. It might be mapped to "mute". The key is: what's the media time?
>> Is it
>> tied with wall clock or something else such as audio? I think both are ok.
>>
>
>  If "pause" also pauses master clock (i.e., media time), then VFP should
>> adjust
>> timestamp after being paused. Otherwise, VFP doesn't need to change
>> timestamp.
>> Current implementation takes the latter approach. This is ok with WebKit
>> since
>> they don't do scheduling for video rendering. Another benefit is <video>
>> controls can show correct time for the meeting.
>>
>
>  WDYT?
>>
>
> It sounds like it should be clarified in the spec but we don't need to
> hold up
> this CL for now.
>
> LGTM and can you look into clarifying the spec? (i.e., ask hta@?)
>

Thanks! I will follow up with hta@ about this.



>
>
http://codereview.chromium.**org/11198055/<http://codereview.chromium.org/111...
>