Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(921)

Unified Diff: media/cast/sender/video_sender.cc

Issue 1484403002: cast: Support for low-latency interactive mode (Closed) Base URL: https://chromium.googlesource.com/chromium/src.git@master
Patch Set: Created 5 years ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View side-by-side diff with in-line comments
Download patch
« media/cast/sender/frame_sender.cc ('K') | « media/cast/sender/video_sender.h ('k') | no next file » | no next file with comments »
Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
Index: media/cast/sender/video_sender.cc
diff --git a/media/cast/sender/video_sender.cc b/media/cast/sender/video_sender.cc
index ff65423d49ad6ba5523e0fbe792ce636f40cdfb8..63b5a798e303fcbe159c1200c8bf60107c42410e 100644
--- a/media/cast/sender/video_sender.cc
+++ b/media/cast/sender/video_sender.cc
@@ -39,6 +39,11 @@ const int kConstantTimeMs = 75;
// available to handle the occasional more-complex frames).
const int kTargetUtilizationPercentage = 75;
+// Minimum number of user interactions before we consider the user to be in
+// interactive mode. The goal is to prevent user interactions to launch
+// animated content from causing target playout time flip-flop.
+const int kMinUserInteractionsForInteractiveMode = 5;
+
// Extract capture begin/end timestamps from |video_frame|'s metadata and log
// it.
void LogVideoCaptureTimestamps(CastEnvironment* cast_environment,
@@ -105,6 +110,10 @@ VideoSender::VideoSender(
frames_in_encoder_(0),
last_bitrate_(0),
playout_delay_change_cb_(playout_delay_change_cb),
+ animation_content_(true),
+ user_interaction_(false),
+ user_interaction_count_(0),
+ interactive_mode_(false),
last_reported_deadline_utilization_(-1.0),
last_reported_lossy_utilization_(-1.0),
weak_factory_(this) {
@@ -161,6 +170,28 @@ void VideoSender::InsertRawVideoFrame(
"timestamp", reference_time.ToInternalValue(),
"rtp_timestamp", rtp_timestamp);
+ bool prev_user_interaction = user_interaction_;
+ if (video_frame->metadata()->GetBoolean(VideoFrameMetadata::ANIMATION_CONTENT,
+ &animation_content_) &&
+ video_frame->metadata()->GetBoolean(VideoFrameMetadata::USER_INTERACTION,
+ &user_interaction_)) {
+ if (user_interaction_ && !prev_user_interaction) {
+ user_interaction_count_++;
miu 2015/12/01 21:15:27 IIUC, this is counting the number of frames where
Irfan 2015/12/02 22:32:44 This has moved to capture side now.
+ }
+ if (user_interaction_count_ > kMinUserInteractionsForInteractiveMode &&
+ !animation_content_ && !interactive_mode_) {
miu 2015/12/01 21:15:27 Perhaps we should only consider the "user interact
Irfan 2015/12/02 22:37:55 As discussed f2f, I think we should be more conser
+ interactive_mode_ = true;
+ VLOG(1) << "Interactive mode playout time "
+ << media::cast::kInteractiveModeStartPlayoutTimeMs;
+ playout_delay_change_cb_.Run(base::TimeDelta::FromMilliseconds(
+ media::cast::kInteractiveModeStartPlayoutTimeMs));
+ } else if (animation_content_) {
+ // Reset user interactions when animating content is playing.
+ user_interaction_count_ = 0;
+ interactive_mode_ = false;
+ }
+ }
+
miu 2015/12/01 21:15:27 BTW--What about audio? If we change video's playo
Irfan 2015/12/02 22:32:44 This already works as you describe :-)
// Drop the frame if either its RTP or reference timestamp is not an increase
// over the last frame's. This protects: 1) the duration calculations that
// assume timestamps are monotonically non-decreasing, and 2) assumptions made
@@ -192,7 +223,21 @@ void VideoSender::InsertRawVideoFrame(
current_round_trip_time_ * kRoundTripsNeeded +
base::TimeDelta::FromMilliseconds(kConstantTimeMs),
max_playout_delay_);
- if (new_target_delay > target_playout_delay_) {
+ // In case of interactive mode, we prefer frame drops over increasing
+ // playout time.
+ if (!interactive_mode_ && new_target_delay > target_playout_delay_) {
+ // In case we detect user is no more in an interactive mode and there is
+ // a need to drop a frame, we ensure the playout delay is at-least the
+ // the starting value that is known to work well.
+ // This is intentended to minimize freeze when moving from an interactive
+ // session to watching animating content while being limited by end-to-end
+ // delay.
+ VLOG(1) << "Ensure playout time is at least "
+ << media::cast::kNonInteractiveModeStartPlayoutTimeMs;
+ if (new_target_delay.InMilliseconds() <
+ media::cast::kNonInteractiveModeStartPlayoutTimeMs)
+ new_target_delay = base::TimeDelta::FromMilliseconds(
+ kNonInteractiveModeStartPlayoutTimeMs);
VLOG(1) << "New target delay: " << new_target_delay.InMilliseconds();
playout_delay_change_cb_.Run(new_target_delay);
}
« media/cast/sender/frame_sender.cc ('K') | « media/cast/sender/video_sender.h ('k') | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698