Decouple capture timestamp and reference time

Decouple capture timestamp and reference time

In this CL, we changed the signature of
VideoCaptureDevice::OnIncomingCapturedData
to take one more parameter called reference_time.

Background:
Previously, we have only timestamp parameter, and it actually
plays the role as timestamp and reference_time.
Here we define timestamp as the ideal time span between
the current frame and the first frame in the stream.
Define reference time as the system clock time when the frame
gets generated.

Ideally, timestamp can be derived from reference_time.
However, in reality, especially in a multi process application
like Chrome, system clock read has noise on the order of 5ms.
To overcome that, we found the original timestamp from camera
driver is much smoother. But the defect is that the camera
clock would have a drift with respect to the system clock.

Thus a perfect solution would be that we take system time for
reference time, which is used for AV sync; we stream timestamp
to the remote side, which will make the rendering smooth.

This CL is preliminary for the complete fix.

BUG=601657
Committed: https://crrev.com/ed732c9c81df6d8fd4cfb4105ad154f52d458cd9
Cr-Commit-Position: refs/heads/master@{#396021}

[qiangchen, 2016-05-17 17:54:25]

Description was changed from

==========
OnIncomingCapturedData Signature Change

BUG=
==========

to

==========
OnIncomingCapturedData Signature Change

BUG=601657
==========

[qiangchen, 2016-05-17 21:14:11]

Description was changed from

==========
OnIncomingCapturedData Signature Change

BUG=601657
==========

to

==========
Decouple capture timestamp and reference time

In this CL, we changed the signature of
VideoCaptureDevice::OnIncomingCapturedData
to take one more parameter called reference_time.

Previously, we have only timestamp parameter, and it actually 
plays the role as timestamp and reference_time. 
Here we define timestamp as the ideal time span between
the current frame and the first frame in the stream.
Define reference time as the system clock time when the frame
gets generated.

Ideally, timestamp can be derived from reference_time. 
However, in reality, especially in a multi process application
like Chrome, system clock read has noise on the order of 5ms.
To overcome that, we found the original timestamp from camera 
driver is much smoother. But the defect is that the camera
clock
would have a drift with respect to the system clock.

Thus a perfect solution would be that we take system time for 
reference time, which is used for AV sync; we stream timestamp 
to the remote side, which will make the rendering smooth.

BUG=601657
==========

[qiangchen, 2016-05-17 21:14:33]

Description was changed from

==========
Decouple capture timestamp and reference time

In this CL, we changed the signature of
VideoCaptureDevice::OnIncomingCapturedData
to take one more parameter called reference_time.

Previously, we have only timestamp parameter, and it actually 
plays the role as timestamp and reference_time. 
Here we define timestamp as the ideal time span between
the current frame and the first frame in the stream.
Define reference time as the system clock time when the frame
gets generated.

Ideally, timestamp can be derived from reference_time. 
However, in reality, especially in a multi process application
like Chrome, system clock read has noise on the order of 5ms.
To overcome that, we found the original timestamp from camera 
driver is much smoother. But the defect is that the camera
clock
would have a drift with respect to the system clock.

Thus a perfect solution would be that we take system time for 
reference time, which is used for AV sync; we stream timestamp 
to the remote side, which will make the rendering smooth.

BUG=601657
==========

to

==========
Decouple capture timestamp and reference time

In this CL, we changed the signature of
VideoCaptureDevice::OnIncomingCapturedData
to take one more parameter called reference_time.

Background:
Previously, we have only timestamp parameter, and it actually 
plays the role as timestamp and reference_time. 
Here we define timestamp as the ideal time span between
the current frame and the first frame in the stream.
Define reference time as the system clock time when the frame
gets generated.

Ideally, timestamp can be derived from reference_time. 
However, in reality, especially in a multi process application
like Chrome, system clock read has noise on the order of 5ms.
To overcome that, we found the original timestamp from camera 
driver is much smoother. But the defect is that the camera
clock
would have a drift with respect to the system clock.

Thus a perfect solution would be that we take system time for 
reference time, which is used for AV sync; we stream timestamp 
to the remote side, which will make the rendering smooth.

BUG=601657
==========

[qiangchen, 2016-05-17 21:15:22]

Description was changed from

==========
Decouple capture timestamp and reference time

In this CL, we changed the signature of
VideoCaptureDevice::OnIncomingCapturedData
to take one more parameter called reference_time.

Background:
Previously, we have only timestamp parameter, and it actually 
plays the role as timestamp and reference_time. 
Here we define timestamp as the ideal time span between
the current frame and the first frame in the stream.
Define reference time as the system clock time when the frame
gets generated.

Ideally, timestamp can be derived from reference_time. 
However, in reality, especially in a multi process application
like Chrome, system clock read has noise on the order of 5ms.
To overcome that, we found the original timestamp from camera 
driver is much smoother. But the defect is that the camera
clock
would have a drift with respect to the system clock.

Thus a perfect solution would be that we take system time for 
reference time, which is used for AV sync; we stream timestamp 
to the remote side, which will make the rendering smooth.

BUG=601657
==========

to

==========
Decouple capture timestamp and reference time

In this CL, we changed the signature of
VideoCaptureDevice::OnIncomingCapturedData
to take one more parameter called reference_time.

Background:
Previously, we have only timestamp parameter, and it actually 
plays the role as timestamp and reference_time. 
Here we define timestamp as the ideal time span between
the current frame and the first frame in the stream.
Define reference time as the system clock time when the frame
gets generated.

Ideally, timestamp can be derived from reference_time. 
However, in reality, especially in a multi process application
like Chrome, system clock read has noise on the order of 5ms.
To overcome that, we found the original timestamp from camera 
driver is much smoother. But the defect is that the camera
clock would have a drift with respect to the system clock.

Thus a perfect solution would be that we take system time for 
reference time, which is used for AV sync; we stream timestamp 
to the remote side, which will make the rendering smooth.

BUG=601657
==========

[qiangchen, 2016-05-17 23:25:27]

Patchset #4 (id:60001) has been deleted

[qiangchen, 2016-05-18 15:43:16]

Patchset #3 (id:40001) has been deleted

[qiangchen, 2016-05-18 20:22:58]

Patchset #1 (id:1) has been deleted

[qiangchen, 2016-05-18 20:23:06]

Patchset #1 (id:20001) has been deleted

[qiangchen, 2016-05-18 20:23:13]

Patchset #1 (id:80001) has been deleted

[qiangchen, 2016-05-18 20:33:45]

Description was changed from

==========
Decouple capture timestamp and reference time

In this CL, we changed the signature of
VideoCaptureDevice::OnIncomingCapturedData
to take one more parameter called reference_time.

Background:
Previously, we have only timestamp parameter, and it actually 
plays the role as timestamp and reference_time. 
Here we define timestamp as the ideal time span between
the current frame and the first frame in the stream.
Define reference time as the system clock time when the frame
gets generated.

Ideally, timestamp can be derived from reference_time. 
However, in reality, especially in a multi process application
like Chrome, system clock read has noise on the order of 5ms.
To overcome that, we found the original timestamp from camera 
driver is much smoother. But the defect is that the camera
clock would have a drift with respect to the system clock.

Thus a perfect solution would be that we take system time for 
reference time, which is used for AV sync; we stream timestamp 
to the remote side, which will make the rendering smooth.

BUG=601657
==========

to

==========
Decouple capture timestamp and reference time

In this CL, we changed the signature of
VideoCaptureDevice::OnIncomingCapturedData
to take one more parameter called reference_time.

Background:
Previously, we have only timestamp parameter, and it actually 
plays the role as timestamp and reference_time. 
Here we define timestamp as the ideal time span between
the current frame and the first frame in the stream.
Define reference time as the system clock time when the frame
gets generated.

Ideally, timestamp can be derived from reference_time. 
However, in reality, especially in a multi process application
like Chrome, system clock read has noise on the order of 5ms.
To overcome that, we found the original timestamp from camera 
driver is much smoother. But the defect is that the camera
clock would have a drift with respect to the system clock.

Thus a perfect solution would be that we take system time for 
reference time, which is used for AV sync; we stream timestamp 
to the remote side, which will make the rendering smooth.

This CL is preliminary for the complete fix.

BUG=601657
==========

[qiangchen, 2016-05-18 20:33:46]

qiangchen@chromium.org changed reviewers:
+ dalecurtis@chromium.org, miu@chromium.org

[qiangchen, 2016-05-18 20:41:45]

This CL is preliminary for fix of crbug/601657.

In this CL, we essentially changed the signature of OnIncomingCapturedData, and
no functional change in this CL.

PTAL.

[miu, 2016-05-18 22:35:41]

Thanks for doing this! :)

Comments on Patch Set 1:

https://codereview.chromium.org/1983193002/diff/100001/content/browser/render...
File content/browser/renderer_host/media/video_capture_device_client.cc (right):

https://codereview.chromium.org/1983193002/diff/100001/content/browser/render...
content/browser/renderer_host/media/video_capture_device_client.cc:258:
OnIncomingCapturedBuffer(std::move(buffer), output_format, reference_time);
See comment I made in video_capture_device.h. You'll want to continue passing
the frame |timestamp| through here and into OnIncomingCapturedBuffer().

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/an...
File media/capture/video/android/video_capture_device_android.cc (right):

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/an...
media/capture/video/android/video_capture_device_android.cc:187: first_ref_time_
= current_time;
There's probably a way to pass the video capture timestamp from the Java code to
this native code. If you don't want to fix this in this change, then please at
least add a TODO comment like "This is a place-holder for the video capture
frame timestamp. The actual timestamp (provided by the platform) should be
plumbed-through to here."

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/li...
File media/capture/video/linux/v4l2_capture_delegate.cc (right):

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/li...
media/capture/video/linux/v4l2_capture_delegate.cc:395: first_ref_time_ = now;
The |buffer| has a |timestamp| field you can translate:

  base::TimeDelta timestamp =
      base::TimeDelta::FromSeconds(buffer.timestamp.tv_sec) +
      base::TimeDelta::FromMicroseconds(buffer.timestamp.tv_usec);

...so you don't need the |first_ref_time_| mechanism here. :)

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/ma...
File media/capture/video/mac/video_capture_device_decklink_mac.mm (right):

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/ma...
media/capture/video/mac/video_capture_device_decklink_mac.mm:263:
base::TimeTicks now = base::TimeTicks::Now();
The call to TimeTicks::Now() should be moved to the top of the method (the
current thread might block waiting to acquire the |lock_|).

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/ma...
media/capture/video/mac/video_capture_device_decklink_mac.mm:265:
first_ref_time_ = now;
BTW--You can get the timestamp from the |video_frame| argument:

    const BMDTimeScale micros_time_scale = base::Time::kMicrosecondsPerSecond;
    BMDTimeValue frame_time;
    BMDTimeValue frame_duration;
    base::TimeDelta timestamp;
    if (SUCCEEDED(video_frame->GetStreamTime(&frame_time, &frame_duration,
micros_time_scale))) {
      timestamp = base::TimeDelta::FromMicroseconds(frame_time);
    } else {
      timestamp = now - first_ref_time_;
    }

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/vi...
File media/capture/video/video_capture_device.h (right):

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/vi...
media/capture/video/video_capture_device.h:203: // available, the frame will be
silently dropped. |referernce_time| is
typo: s/referernce_time/reference_time/

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/vi...
media/capture/video/video_capture_device.h:204: // estimated system clock time
when the capture happened. |timestamp|
Remove "estimated" since we really *are* taking a snapshot of system clock time.
;-)

IMO, it's also worth mentioning: "The reference time is used to achieve A/V
sync, and not meant to be used to derive an exact presentation time for
playout."

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/vi...
media/capture/video/video_capture_device.h:206: // the current frame; however,
the time source is not necessarily the
nit: End of this sentence should be: "...however, the time source is determined
by the platform's device driver and is often not the system clock."

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/vi...
media/capture/video/video_capture_device.h:239: const base::TimeTicks&
timestamp) = 0;
These two methods need similar changes. Do you intend to do that in a future
change?

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/vi...
media/capture/video/video_capture_device.h:243: const base::TimeTicks&
timestamp) = 0;
BTW--For this method, the TimeTicks argument would be reference_time, and
instead of a separate |timestamp| TimeDelta argument,
VideoFrame::set_timestamp() should be used. Then, client code should
DCHECK(!frame->timestamp().is_null());

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/wi...
File media/capture/video/win/video_capture_device_win.cc (right):

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/wi...
media/capture/video/win/video_capture_device_win.cc:458: if (timestamp ==
media::kNoTimestamp())
Please add a comment that sometimes the platform does not provide a timestamp
from the video capture hardware, and so this code is a workaround to derive a
timestamp from the system clock.

BTW--Do we know why/when the platform would not provide a timestamp? If so, it's
probably worth documenting that in a comment here or in sink_input_pin_win.cc.

[qiangchen, 2016-05-20 16:43:23]

Patchset #2 (id:120001) has been deleted

[qiangchen, 2016-05-20 17:55:14]

Fixed, thanks.

https://codereview.chromium.org/1983193002/diff/100001/content/browser/render...
File content/browser/renderer_host/media/video_capture_device_client.cc (right):

https://codereview.chromium.org/1983193002/diff/100001/content/browser/render...
content/browser/renderer_host/media/video_capture_device_client.cc:258:
OnIncomingCapturedBuffer(std::move(buffer), output_format, reference_time);
On 2016/05/18 22:35:40, miu wrote:
> See comment I made in video_capture_device.h. You'll want to continue passing
> the frame |timestamp| through here and into OnIncomingCapturedBuffer().

Done.

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/an...
File media/capture/video/android/video_capture_device_android.cc (right):

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/an...
media/capture/video/android/video_capture_device_android.cc:187: first_ref_time_
= current_time;
On 2016/05/18 22:35:40, miu wrote:
> There's probably a way to pass the video capture timestamp from the Java code
to
> this native code. If you don't want to fix this in this change, then please at
> least add a TODO comment like "This is a place-holder for the video capture
> frame timestamp. The actual timestamp (provided by the platform) should be
> plumbed-through to here."

Done.

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/li...
File media/capture/video/linux/v4l2_capture_delegate.cc (right):

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/li...
media/capture/video/linux/v4l2_capture_delegate.cc:395: first_ref_time_ = now;
On 2016/05/18 22:35:40, miu wrote:
> The |buffer| has a |timestamp| field you can translate:
> 
>   base::TimeDelta timestamp =
>       base::TimeDelta::FromSeconds(buffer.timestamp.tv_sec) +
>       base::TimeDelta::FromMicroseconds(buffer.timestamp.tv_usec);
> 
> ...so you don't need the |first_ref_time_| mechanism here. :)

Done.

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/ma...
File media/capture/video/mac/video_capture_device_decklink_mac.mm (right):

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/ma...
media/capture/video/mac/video_capture_device_decklink_mac.mm:263:
base::TimeTicks now = base::TimeTicks::Now();
On 2016/05/18 22:35:40, miu wrote:
> The call to TimeTicks::Now() should be moved to the top of the method (the
> current thread might block waiting to acquire the |lock_|).

Done.

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/ma...
media/capture/video/mac/video_capture_device_decklink_mac.mm:265:
first_ref_time_ = now;
On 2016/05/18 22:35:40, miu wrote:
> BTW--You can get the timestamp from the |video_frame| argument:
> 
>     const BMDTimeScale micros_time_scale = base::Time::kMicrosecondsPerSecond;
>     BMDTimeValue frame_time;
>     BMDTimeValue frame_duration;
>     base::TimeDelta timestamp;
>     if (SUCCEEDED(video_frame->GetStreamTime(&frame_time, &frame_duration,
> micros_time_scale))) {
>       timestamp = base::TimeDelta::FromMicroseconds(frame_time);
>     } else {
>       timestamp = now - first_ref_time_;
>     }

Done.

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/vi...
File media/capture/video/video_capture_device.h (right):

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/vi...
media/capture/video/video_capture_device.h:203: // available, the frame will be
silently dropped. |referernce_time| is
On 2016/05/18 22:35:40, miu wrote:
> typo: s/referernce_time/reference_time/

Done.

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/vi...
media/capture/video/video_capture_device.h:204: // estimated system clock time
when the capture happened. |timestamp|
On 2016/05/18 22:35:40, miu wrote:
> Remove "estimated" since we really *are* taking a snapshot of system clock
time.
> ;-)
> 
> IMO, it's also worth mentioning: "The reference time is used to achieve A/V
> sync, and not meant to be used to derive an exact presentation time for
> playout."

Done.

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/vi...
media/capture/video/video_capture_device.h:206: // the current frame; however,
the time source is not necessarily the
On 2016/05/18 22:35:40, miu wrote:
> nit: End of this sentence should be: "...however, the time source is
determined
> by the platform's device driver and is often not the system clock."

Done.

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/vi...
media/capture/video/video_capture_device.h:239: const base::TimeTicks&
timestamp) = 0;
On 2016/05/18 22:35:40, miu wrote:
> These two methods need similar changes. Do you intend to do that in a future
> change?

I intended to do it in next CL, but now I think it is better to do it in this
CL, because it touches same subclasses and thus same files with the change for
OnIncomingCapturedData.

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/vi...
media/capture/video/video_capture_device.h:243: const base::TimeTicks&
timestamp) = 0;
On 2016/05/18 22:35:40, miu wrote:
> BTW--For this method, the TimeTicks argument would be reference_time, and
> instead of a separate |timestamp| TimeDelta argument,
> VideoFrame::set_timestamp() should be used. Then, client code should
> DCHECK(!frame->timestamp().is_null());

Done.

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/wi...
File media/capture/video/win/video_capture_device_win.cc (right):

https://codereview.chromium.org/1983193002/diff/100001/media/capture/video/wi...
media/capture/video/win/video_capture_device_win.cc:458: if (timestamp ==
media::kNoTimestamp())
On 2016/05/18 22:35:40, miu wrote:
> Please add a comment that sometimes the platform does not provide a timestamp
> from the video capture hardware, and so this code is a workaround to derive a
> timestamp from the system clock.
> 
> BTW--Do we know why/when the platform would not provide a timestamp? If so,
it's
> probably worth documenting that in a comment here or in sink_input_pin_win.cc.

In my testing, I did not see a case that timestamp is not provided by OS API,
but that API do have a return value to indicate whether a timestamp is provided
or not. So we need to handle it in case a timestamp is not provided. 

I think the API's implementation could differ from camera to camera, and thus it
is safe we do have a fall back solution here.

[miu, 2016-05-25 01:51:30]

Nice work! Thanks for plumbing this in! :-)

lgtm % a few minor things:

https://codereview.chromium.org/1983193002/diff/140001/content/browser/render...
File content/browser/renderer_host/media/video_capture_device_client.cc (right):

https://codereview.chromium.org/1983193002/diff/140001/content/browser/render...
content/browser/renderer_host/media/video_capture_device_client.cc:326:
frame->set_timestamp(timestamp);
nit: Instead of calling set_timestamp() here, you should pass it as the last
argument (i.e., instead of base::TimeDelta()) to the VideoFrame::WrapXXX() calls
above.

https://codereview.chromium.org/1983193002/diff/140001/media/capture/video/ma...
File media/capture/video/mac/video_capture_device_decklink_mac.mm (right):

https://codereview.chromium.org/1983193002/diff/140001/media/capture/video/ma...
media/capture/video/mac/video_capture_device_decklink_mac.mm:272: timestamp =
now - first_ref_time_;
You're still going to need this code for this fall-back case:

  if (first_ref_time_.is_null())
    first_ref_time_ = now;

https://codereview.chromium.org/1983193002/diff/140001/media/capture/video/vi...
File media/capture/video/video_capture_device.h (right):

https://codereview.chromium.org/1983193002/diff/140001/media/capture/video/vi...
media/capture/video/video_capture_device.h:207: // first frame in the stream and
the current frame; however, however, the
s/however, however,/however,/

[qiangchen, 2016-05-25 16:46:17]

fixed.

https://codereview.chromium.org/1983193002/diff/140001/content/browser/render...
File content/browser/renderer_host/media/video_capture_device_client.cc (right):

https://codereview.chromium.org/1983193002/diff/140001/content/browser/render...
content/browser/renderer_host/media/video_capture_device_client.cc:326:
frame->set_timestamp(timestamp);
On 2016/05/25 01:51:30, miu wrote:
> nit: Instead of calling set_timestamp() here, you should pass it as the last
> argument (i.e., instead of base::TimeDelta()) to the VideoFrame::WrapXXX()
calls
> above.

Done.

https://codereview.chromium.org/1983193002/diff/140001/media/capture/video/ma...
File media/capture/video/mac/video_capture_device_decklink_mac.mm (right):

https://codereview.chromium.org/1983193002/diff/140001/media/capture/video/ma...
media/capture/video/mac/video_capture_device_decklink_mac.mm:272: timestamp =
now - first_ref_time_;
On 2016/05/25 01:51:30, miu wrote:
> You're still going to need this code for this fall-back case:
> 
>   if (first_ref_time_.is_null())
>     first_ref_time_ = now;

Done.

https://codereview.chromium.org/1983193002/diff/140001/media/capture/video/vi...
File media/capture/video/video_capture_device.h (right):

https://codereview.chromium.org/1983193002/diff/140001/media/capture/video/vi...
media/capture/video/video_capture_device.h:207: // first frame in the stream and
the current frame; however, however, the
On 2016/05/25 01:51:30, miu wrote:
> s/however, however,/however,/

Done.

[DaleCurtis, 2016-05-25 20:00:12]

skimmed and rs lgtm, defer to yuri's review.

[qiangchen, 2016-05-25 22:43:43]

The CQ bit was checked by qiangchen@chromium.org

[qiangchen, 2016-05-25 22:43:43]

The patchset sent to the CQ was uploaded after l-g-t-m from miu@chromium.org
Link to the patchset: https://codereview.chromium.org/1983193002/#ps160001
(title: "Resolve Comments")

[commit-bot: I haz the power, 2016-05-25 22:44:41]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1983193002/160001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1983193002/160001

[commit-bot: I haz the power, 2016-05-25 22:51:56]

Description was changed from

==========
Decouple capture timestamp and reference time

In this CL, we changed the signature of
VideoCaptureDevice::OnIncomingCapturedData
to take one more parameter called reference_time.

Background:
Previously, we have only timestamp parameter, and it actually 
plays the role as timestamp and reference_time. 
Here we define timestamp as the ideal time span between
the current frame and the first frame in the stream.
Define reference time as the system clock time when the frame
gets generated.

Ideally, timestamp can be derived from reference_time. 
However, in reality, especially in a multi process application
like Chrome, system clock read has noise on the order of 5ms.
To overcome that, we found the original timestamp from camera 
driver is much smoother. But the defect is that the camera
clock would have a drift with respect to the system clock.

Thus a perfect solution would be that we take system time for 
reference time, which is used for AV sync; we stream timestamp 
to the remote side, which will make the rendering smooth.

This CL is preliminary for the complete fix.

BUG=601657
==========

to

==========
Decouple capture timestamp and reference time

In this CL, we changed the signature of
VideoCaptureDevice::OnIncomingCapturedData
to take one more parameter called reference_time.

Background:
Previously, we have only timestamp parameter, and it actually 
plays the role as timestamp and reference_time. 
Here we define timestamp as the ideal time span between
the current frame and the first frame in the stream.
Define reference time as the system clock time when the frame
gets generated.

Ideally, timestamp can be derived from reference_time. 
However, in reality, especially in a multi process application
like Chrome, system clock read has noise on the order of 5ms.
To overcome that, we found the original timestamp from camera 
driver is much smoother. But the defect is that the camera
clock would have a drift with respect to the system clock.

Thus a perfect solution would be that we take system time for 
reference time, which is used for AV sync; we stream timestamp 
to the remote side, which will make the rendering smooth.

This CL is preliminary for the complete fix.

BUG=601657
==========

[commit-bot: I haz the power, 2016-05-25 22:51:57]

Committed patchset #3 (id:160001)

[commit-bot: I haz the power, 2016-05-25 22:53:29]

Description was changed from

==========
Decouple capture timestamp and reference time

In this CL, we changed the signature of
VideoCaptureDevice::OnIncomingCapturedData
to take one more parameter called reference_time.

Background:
Previously, we have only timestamp parameter, and it actually 
plays the role as timestamp and reference_time. 
Here we define timestamp as the ideal time span between
the current frame and the first frame in the stream.
Define reference time as the system clock time when the frame
gets generated.

Ideally, timestamp can be derived from reference_time. 
However, in reality, especially in a multi process application
like Chrome, system clock read has noise on the order of 5ms.
To overcome that, we found the original timestamp from camera 
driver is much smoother. But the defect is that the camera
clock would have a drift with respect to the system clock.

Thus a perfect solution would be that we take system time for 
reference time, which is used for AV sync; we stream timestamp 
to the remote side, which will make the rendering smooth.

This CL is preliminary for the complete fix.

BUG=601657
==========

to

==========
Decouple capture timestamp and reference time

In this CL, we changed the signature of
VideoCaptureDevice::OnIncomingCapturedData
to take one more parameter called reference_time.

Background:
Previously, we have only timestamp parameter, and it actually
plays the role as timestamp and reference_time.
Here we define timestamp as the ideal time span between
the current frame and the first frame in the stream.
Define reference time as the system clock time when the frame
gets generated.

Ideally, timestamp can be derived from reference_time.
However, in reality, especially in a multi process application
like Chrome, system clock read has noise on the order of 5ms.
To overcome that, we found the original timestamp from camera
driver is much smoother. But the defect is that the camera
clock would have a drift with respect to the system clock.

Thus a perfect solution would be that we take system time for
reference time, which is used for AV sync; we stream timestamp
to the remote side, which will make the rendering smooth.

This CL is preliminary for the complete fix.

BUG=601657

Committed: https://crrev.com/ed732c9c81df6d8fd4cfb4105ad154f52d458cd9
Cr-Commit-Position: refs/heads/master@{#396021}
==========

[commit-bot: I haz the power, 2016-05-25 22:53:31]

Patchset 3 (id:??) landed as
https://crrev.com/ed732c9c81df6d8fd4cfb4105ad154f52d458cd9
Cr-Commit-Position: refs/heads/master@{#396021}