Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(47)

Side by Side Diff: content/browser/media/capture/web_contents_video_capture_device.cc

Issue 1865283003: Revert of Tab/Desktop Capture: Use requests instead of timer-based refreshing. (Closed) Base URL: https://chromium.googlesource.com/chromium/src.git@video_refresh_from_sinks
Patch Set: Created 4 years, 8 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
OLDNEW
1 // Copyright (c) 2012 The Chromium Authors. All rights reserved. 1 // Copyright (c) 2012 The Chromium Authors. All rights reserved.
2 // Use of this source code is governed by a BSD-style license that can be 2 // Use of this source code is governed by a BSD-style license that can be
3 // found in the LICENSE file. 3 // found in the LICENSE file.
4 // 4 //
5 // Implementation notes: This needs to work on a variety of hardware 5 // Implementation notes: This needs to work on a variety of hardware
6 // configurations where the speed of the CPU and GPU greatly affect overall 6 // configurations where the speed of the CPU and GPU greatly affect overall
7 // performance. Spanning several threads, the process of capturing has been 7 // performance. Spanning several threads, the process of capturing has been
8 // split up into four conceptual stages: 8 // split up into four conceptual stages:
9 // 9 //
10 // 1. Reserve Buffer: Before a frame can be captured, a slot in the client's 10 // 1. Reserve Buffer: Before a frame can be captured, a slot in the client's
(...skipping 165 matching lines...) Expand 10 before | Expand all | Expand 10 after
176 base::WeakPtr<WindowActivityTracker> window_activity_tracker_; 176 base::WeakPtr<WindowActivityTracker> window_activity_tracker_;
177 base::WeakPtrFactory<FrameSubscriber> weak_ptr_factory_; 177 base::WeakPtrFactory<FrameSubscriber> weak_ptr_factory_;
178 }; 178 };
179 179
180 // ContentCaptureSubscription is the relationship between a RenderWidgetHost 180 // ContentCaptureSubscription is the relationship between a RenderWidgetHost
181 // whose content is updating, a subscriber that is deciding which of these 181 // whose content is updating, a subscriber that is deciding which of these
182 // updates to capture (and where to deliver them to), and a callback that 182 // updates to capture (and where to deliver them to), and a callback that
183 // knows how to do the capture and prepare the result for delivery. 183 // knows how to do the capture and prepare the result for delivery.
184 // 184 //
185 // In practice, this means (a) installing a RenderWidgetHostFrameSubscriber in 185 // In practice, this means (a) installing a RenderWidgetHostFrameSubscriber in
186 // the RenderWidgetHostView, to process compositor updates, and (b) occasionally 186 // the RenderWidgetHostView, to process compositor updates, and (b) running a
187 // initiating forced, non-event-driven captures needed by downstream consumers 187 // timer to possibly initiate forced, non-event-driven captures needed by
188 // that request "refresh frames" of unchanged content. 188 // downstream consumers that require frame repeats of unchanged content.
189 // 189 //
190 // All of this happens on the UI thread, although the 190 // All of this happens on the UI thread, although the
191 // RenderWidgetHostViewFrameSubscriber we install may be dispatching updates 191 // RenderWidgetHostViewFrameSubscriber we install may be dispatching updates
192 // autonomously on some other thread. 192 // autonomously on some other thread.
193 class ContentCaptureSubscription { 193 class ContentCaptureSubscription {
194 public: 194 public:
195 typedef base::Callback<void( 195 typedef base::Callback<void(
196 const base::TimeTicks&, 196 const base::TimeTicks&,
197 const scoped_refptr<media::VideoFrame>&, 197 const scoped_refptr<media::VideoFrame>&,
198 const RenderWidgetHostViewFrameSubscriber::DeliverFrameCallback&)> 198 const RenderWidgetHostViewFrameSubscriber::DeliverFrameCallback&)>
199 CaptureCallback; 199 CaptureCallback;
200 200
201 // Create a subscription. Whenever a manual capture is required, the 201 // Create a subscription. Whenever a manual capture is required, the
202 // subscription will invoke |capture_callback| on the UI thread to do the 202 // subscription will invoke |capture_callback| on the UI thread to do the
203 // work. 203 // work.
204 ContentCaptureSubscription( 204 ContentCaptureSubscription(
205 const RenderWidgetHost& source, 205 const RenderWidgetHost& source,
206 const scoped_refptr<media::ThreadSafeCaptureOracle>& oracle_proxy, 206 const scoped_refptr<media::ThreadSafeCaptureOracle>& oracle_proxy,
207 const CaptureCallback& capture_callback); 207 const CaptureCallback& capture_callback);
208 ~ContentCaptureSubscription(); 208 ~ContentCaptureSubscription();
209 209
210 void MaybeCaptureForRefresh();
211
212 private: 210 private:
213 // Called for active frame refresh requests, or mouse activity events. 211 // Called on timer or mouse activity events.
214 void OnEvent(FrameSubscriber* subscriber); 212 void OnEvent(FrameSubscriber* subscriber);
215 213
216 // Maintain a weak reference to the RenderWidgetHost (via its routing ID), 214 // Maintain a weak reference to the RenderWidgetHost (via its routing ID),
217 // since the instance could be destroyed externally during the lifetime of 215 // since the instance could be destroyed externally during the lifetime of
218 // |this|. 216 // |this|.
219 const int render_process_id_; 217 const int render_process_id_;
220 const int render_widget_id_; 218 const int render_widget_id_;
221 219
222 VideoFrameDeliveryLog delivery_log_; 220 VideoFrameDeliveryLog delivery_log_;
223 scoped_ptr<FrameSubscriber> refresh_subscriber_; 221 scoped_ptr<FrameSubscriber> timer_subscriber_;
224 scoped_ptr<FrameSubscriber> mouse_activity_subscriber_; 222 scoped_ptr<FrameSubscriber> mouse_activity_subscriber_;
225 CaptureCallback capture_callback_; 223 CaptureCallback capture_callback_;
224 base::Timer timer_;
226 225
227 // Responsible for tracking the cursor state and input events to make 226 // Responsible for tracking the cursor state and input events to make
228 // decisions and then render the mouse cursor on the video frame after 227 // decisions and then render the mouse cursor on the video frame after
229 // capture is completed. 228 // capture is completed.
230 scoped_ptr<content::CursorRenderer> cursor_renderer_; 229 scoped_ptr<content::CursorRenderer> cursor_renderer_;
231 230
232 // Responsible for tracking the UI events and making a decision on whether 231 // Responsible for tracking the UI events and making a decision on whether
233 // user is actively interacting with content. 232 // user is actively interacting with content.
234 scoped_ptr<content::WindowActivityTracker> window_activity_tracker_; 233 scoped_ptr<content::WindowActivityTracker> window_activity_tracker_;
235 234
(...skipping 24 matching lines...) Expand all
260 ~WebContentsCaptureMachine() override; 259 ~WebContentsCaptureMachine() override;
261 260
262 // VideoCaptureMachine overrides. 261 // VideoCaptureMachine overrides.
263 void Start(const scoped_refptr<media::ThreadSafeCaptureOracle>& oracle_proxy, 262 void Start(const scoped_refptr<media::ThreadSafeCaptureOracle>& oracle_proxy,
264 const media::VideoCaptureParams& params, 263 const media::VideoCaptureParams& params,
265 const base::Callback<void(bool)> callback) override; 264 const base::Callback<void(bool)> callback) override;
266 void Stop(const base::Closure& callback) override; 265 void Stop(const base::Closure& callback) override;
267 bool IsAutoThrottlingEnabled() const override { 266 bool IsAutoThrottlingEnabled() const override {
268 return auto_throttling_enabled_; 267 return auto_throttling_enabled_;
269 } 268 }
270 void MaybeCaptureForRefresh() override;
271 269
272 // Starts a copy from the backing store or the composited surface. Must be run 270 // Starts a copy from the backing store or the composited surface. Must be run
273 // on the UI BrowserThread. |deliver_frame_cb| will be run when the operation 271 // on the UI BrowserThread. |deliver_frame_cb| will be run when the operation
274 // completes. The copy will occur to |target|. 272 // completes. The copy will occur to |target|.
275 // 273 //
276 // This may be used as a ContentCaptureSubscription::CaptureCallback. 274 // This may be used as a ContentCaptureSubscription::CaptureCallback.
277 void Capture(const base::TimeTicks& start_time, 275 void Capture(const base::TimeTicks& start_time,
278 const scoped_refptr<media::VideoFrame>& target, 276 const scoped_refptr<media::VideoFrame>& target,
279 const RenderWidgetHostViewFrameSubscriber::DeliverFrameCallback& 277 const RenderWidgetHostViewFrameSubscriber::DeliverFrameCallback&
280 deliver_frame_cb); 278 deliver_frame_cb);
281 279
282 private: 280 private:
283 bool InternalStart( 281 bool InternalStart(
284 const scoped_refptr<media::ThreadSafeCaptureOracle>& oracle_proxy, 282 const scoped_refptr<media::ThreadSafeCaptureOracle>& oracle_proxy,
285 const media::VideoCaptureParams& params); 283 const media::VideoCaptureParams& params);
286 void InternalStop(const base::Closure& callback); 284 void InternalStop(const base::Closure& callback);
287 void InternalMaybeCaptureForRefresh();
288 bool IsStarted() const; 285 bool IsStarted() const;
289 286
290 // Computes the preferred size of the target RenderWidget for optimal capture. 287 // Computes the preferred size of the target RenderWidget for optimal capture.
291 gfx::Size ComputeOptimalViewSize() const; 288 gfx::Size ComputeOptimalViewSize() const;
292 289
293 // Response callback for RenderWidgetHost::CopyFromBackingStore(). 290 // Response callback for RenderWidgetHost::CopyFromBackingStore().
294 void DidCopyFromBackingStore( 291 void DidCopyFromBackingStore(
295 const base::TimeTicks& start_time, 292 const base::TimeTicks& start_time,
296 const scoped_refptr<media::VideoFrame>& target, 293 const scoped_refptr<media::VideoFrame>& target,
297 const RenderWidgetHostViewFrameSubscriber::DeliverFrameCallback& 294 const RenderWidgetHostViewFrameSubscriber::DeliverFrameCallback&
(...skipping 124 matching lines...) Expand 10 before | Expand all | Expand 10 after
422 return interactive_mode; 419 return interactive_mode;
423 } 420 }
424 421
425 ContentCaptureSubscription::ContentCaptureSubscription( 422 ContentCaptureSubscription::ContentCaptureSubscription(
426 const RenderWidgetHost& source, 423 const RenderWidgetHost& source,
427 const scoped_refptr<media::ThreadSafeCaptureOracle>& oracle_proxy, 424 const scoped_refptr<media::ThreadSafeCaptureOracle>& oracle_proxy,
428 const CaptureCallback& capture_callback) 425 const CaptureCallback& capture_callback)
429 : render_process_id_(source.GetProcess()->GetID()), 426 : render_process_id_(source.GetProcess()->GetID()),
430 render_widget_id_(source.GetRoutingID()), 427 render_widget_id_(source.GetRoutingID()),
431 delivery_log_(), 428 delivery_log_(),
432 capture_callback_(capture_callback) { 429 capture_callback_(capture_callback),
430 timer_(true, true) {
433 DCHECK_CURRENTLY_ON(BrowserThread::UI); 431 DCHECK_CURRENTLY_ON(BrowserThread::UI);
434 432
435 RenderWidgetHostView* const view = source.GetView(); 433 RenderWidgetHostView* const view = source.GetView();
436 #if defined(USE_AURA) || defined(OS_MACOSX) 434 #if defined(USE_AURA) || defined(OS_MACOSX)
437 if (view) { 435 if (view) {
438 cursor_renderer_ = CursorRenderer::Create(view->GetNativeView()); 436 cursor_renderer_ = CursorRenderer::Create(view->GetNativeView());
439 window_activity_tracker_ = 437 window_activity_tracker_ =
440 WindowActivityTracker::Create(view->GetNativeView()); 438 WindowActivityTracker::Create(view->GetNativeView());
441 } 439 }
442 #endif 440 #endif
443 refresh_subscriber_.reset(new FrameSubscriber( 441 timer_subscriber_.reset(new FrameSubscriber(
444 media::VideoCaptureOracle::kActiveRefreshRequest, oracle_proxy, 442 media::VideoCaptureOracle::kTimerPoll, oracle_proxy, &delivery_log_,
445 &delivery_log_,
446 cursor_renderer_ ? cursor_renderer_->GetWeakPtr() 443 cursor_renderer_ ? cursor_renderer_->GetWeakPtr()
447 : base::WeakPtr<CursorRenderer>(), 444 : base::WeakPtr<CursorRenderer>(),
448 window_activity_tracker_ ? window_activity_tracker_->GetWeakPtr() 445 window_activity_tracker_ ? window_activity_tracker_->GetWeakPtr()
449 : base::WeakPtr<WindowActivityTracker>())); 446 : base::WeakPtr<WindowActivityTracker>()));
450 mouse_activity_subscriber_.reset(new FrameSubscriber( 447 mouse_activity_subscriber_.reset(new FrameSubscriber(
451 media::VideoCaptureOracle::kMouseCursorUpdate, oracle_proxy, 448 media::VideoCaptureOracle::kMouseCursorUpdate, oracle_proxy,
452 &delivery_log_, cursor_renderer_ ? cursor_renderer_->GetWeakPtr() 449 &delivery_log_, cursor_renderer_ ? cursor_renderer_->GetWeakPtr()
453 : base::WeakPtr<CursorRenderer>(), 450 : base::WeakPtr<CursorRenderer>(),
454 window_activity_tracker_ ? window_activity_tracker_->GetWeakPtr() 451 window_activity_tracker_ ? window_activity_tracker_->GetWeakPtr()
455 : base::WeakPtr<WindowActivityTracker>())); 452 : base::WeakPtr<WindowActivityTracker>()));
456 453
457 // Subscribe to compositor updates. These will be serviced directly by the 454 // Subscribe to compositor updates. These will be serviced directly by the
458 // oracle. 455 // oracle.
459 if (view) { 456 if (view) {
460 scoped_ptr<RenderWidgetHostViewFrameSubscriber> subscriber( 457 scoped_ptr<RenderWidgetHostViewFrameSubscriber> subscriber(
461 new FrameSubscriber( 458 new FrameSubscriber(
462 media::VideoCaptureOracle::kCompositorUpdate, oracle_proxy, 459 media::VideoCaptureOracle::kCompositorUpdate, oracle_proxy,
463 &delivery_log_, cursor_renderer_ ? cursor_renderer_->GetWeakPtr() 460 &delivery_log_, cursor_renderer_ ? cursor_renderer_->GetWeakPtr()
464 : base::WeakPtr<CursorRenderer>(), 461 : base::WeakPtr<CursorRenderer>(),
465 window_activity_tracker_ ? window_activity_tracker_->GetWeakPtr() 462 window_activity_tracker_ ? window_activity_tracker_->GetWeakPtr()
466 : base::WeakPtr<WindowActivityTracker>())); 463 : base::WeakPtr<WindowActivityTracker>()));
467 view->BeginFrameSubscription(std::move(subscriber)); 464 view->BeginFrameSubscription(std::move(subscriber));
468 } 465 }
469 466
467 // Subscribe to timer events. This instance will service these as well.
468 timer_.Start(
469 FROM_HERE,
470 std::max(oracle_proxy->min_capture_period(),
471 base::TimeDelta::FromMilliseconds(
472 media::VideoCaptureOracle::kMinTimerPollPeriodMillis)),
473 base::Bind(&ContentCaptureSubscription::OnEvent, base::Unretained(this),
474 timer_subscriber_.get()));
470 // Subscribe to mouse movement and mouse cursor update events. 475 // Subscribe to mouse movement and mouse cursor update events.
471 if (window_activity_tracker_) { 476 if (window_activity_tracker_) {
472 window_activity_tracker_->RegisterMouseInteractionObserver( 477 window_activity_tracker_->RegisterMouseInteractionObserver(
473 base::Bind(&ContentCaptureSubscription::OnEvent, base::Unretained(this), 478 base::Bind(&ContentCaptureSubscription::OnEvent, base::Unretained(this),
474 mouse_activity_subscriber_.get())); 479 mouse_activity_subscriber_.get()));
475 } 480 }
476 } 481 }
477 482
478 ContentCaptureSubscription::~ContentCaptureSubscription() { 483 ContentCaptureSubscription::~ContentCaptureSubscription() {
479 // If the BrowserThreads have been torn down, then the browser is in the final 484 // If the BrowserThreads have been torn down, then the browser is in the final
480 // stages of exiting and it is dangerous to take any further action. We must 485 // stages of exiting and it is dangerous to take any further action. We must
481 // return early. http://crbug.com/396413 486 // return early. http://crbug.com/396413
482 if (!BrowserThread::IsMessageLoopValid(BrowserThread::UI)) 487 if (!BrowserThread::IsMessageLoopValid(BrowserThread::UI))
483 return; 488 return;
484 489
485 DCHECK_CURRENTLY_ON(BrowserThread::UI); 490 DCHECK_CURRENTLY_ON(BrowserThread::UI);
486 RenderWidgetHost* const source = 491 RenderWidgetHost* const source =
487 RenderWidgetHost::FromID(render_process_id_, render_widget_id_); 492 RenderWidgetHost::FromID(render_process_id_, render_widget_id_);
488 RenderWidgetHostView* const view = source ? source->GetView() : NULL; 493 RenderWidgetHostView* const view = source ? source->GetView() : NULL;
489 if (view) 494 if (view)
490 view->EndFrameSubscription(); 495 view->EndFrameSubscription();
491 } 496 }
492 497
493 void ContentCaptureSubscription::MaybeCaptureForRefresh() {
494 DCHECK_CURRENTLY_ON(BrowserThread::UI);
495 OnEvent(refresh_subscriber_.get());
496 }
497
498 void ContentCaptureSubscription::OnEvent(FrameSubscriber* subscriber) { 498 void ContentCaptureSubscription::OnEvent(FrameSubscriber* subscriber) {
499 DCHECK_CURRENTLY_ON(BrowserThread::UI); 499 DCHECK_CURRENTLY_ON(BrowserThread::UI);
500 TRACE_EVENT0("gpu.capture", "ContentCaptureSubscription::OnEvent"); 500 TRACE_EVENT0("gpu.capture", "ContentCaptureSubscription::OnEvent");
501 501
502 scoped_refptr<media::VideoFrame> frame; 502 scoped_refptr<media::VideoFrame> frame;
503 RenderWidgetHostViewFrameSubscriber::DeliverFrameCallback deliver_frame_cb; 503 RenderWidgetHostViewFrameSubscriber::DeliverFrameCallback deliver_frame_cb;
504 504
505 const base::TimeTicks start_time = base::TimeTicks::Now(); 505 const base::TimeTicks start_time = base::TimeTicks::Now();
506 DCHECK(subscriber == refresh_subscriber_.get() || 506 DCHECK(subscriber == timer_subscriber_.get() ||
507 subscriber == mouse_activity_subscriber_.get()); 507 subscriber == mouse_activity_subscriber_.get());
508 if (subscriber->ShouldCaptureFrame(gfx::Rect(), start_time, &frame, 508 if (subscriber->ShouldCaptureFrame(gfx::Rect(), start_time, &frame,
509 &deliver_frame_cb)) { 509 &deliver_frame_cb)) {
510 capture_callback_.Run(start_time, frame, deliver_frame_cb); 510 capture_callback_.Run(start_time, frame, deliver_frame_cb);
511 } 511 }
512 } 512 }
513 513
514 void RenderVideoFrame( 514 void RenderVideoFrame(
515 const SkBitmap& input, 515 const SkBitmap& input,
516 const scoped_refptr<media::VideoFrame>& output, 516 const scoped_refptr<media::VideoFrame>& output,
(...skipping 180 matching lines...) Expand 10 before | Expand all | Expand 10 after
697 697
698 // The render thread cannot be stopped on the UI thread, so post a message 698 // The render thread cannot be stopped on the UI thread, so post a message
699 // to the thread pool used for blocking operations. 699 // to the thread pool used for blocking operations.
700 if (render_thread_) { 700 if (render_thread_) {
701 BrowserThread::PostBlockingPoolTask( 701 BrowserThread::PostBlockingPoolTask(
702 FROM_HERE, base::Bind(&DeleteOnWorkerThread, 702 FROM_HERE, base::Bind(&DeleteOnWorkerThread,
703 base::Passed(&render_thread_), callback)); 703 base::Passed(&render_thread_), callback));
704 } 704 }
705 } 705 }
706 706
707 void WebContentsCaptureMachine::MaybeCaptureForRefresh() {
708 BrowserThread::PostTask(
709 BrowserThread::UI, FROM_HERE,
710 base::Bind(&WebContentsCaptureMachine::InternalMaybeCaptureForRefresh,
711 // Use of Unretained() is safe here since this task must run
712 // before InternalStop().
713 base::Unretained(this)));
714 }
715
716 void WebContentsCaptureMachine::InternalMaybeCaptureForRefresh() {
717 DCHECK_CURRENTLY_ON(BrowserThread::UI);
718 if (IsStarted() && subscription_)
719 subscription_->MaybeCaptureForRefresh();
720 }
721
722 void WebContentsCaptureMachine::Capture( 707 void WebContentsCaptureMachine::Capture(
723 const base::TimeTicks& start_time, 708 const base::TimeTicks& start_time,
724 const scoped_refptr<media::VideoFrame>& target, 709 const scoped_refptr<media::VideoFrame>& target,
725 const RenderWidgetHostViewFrameSubscriber::DeliverFrameCallback& 710 const RenderWidgetHostViewFrameSubscriber::DeliverFrameCallback&
726 deliver_frame_cb) { 711 deliver_frame_cb) {
727 DCHECK_CURRENTLY_ON(BrowserThread::UI); 712 DCHECK_CURRENTLY_ON(BrowserThread::UI);
728 713
729 RenderWidgetHost* rwh = tracker_->GetTargetRenderWidgetHost(); 714 RenderWidgetHost* rwh = tracker_->GetTargetRenderWidgetHost();
730 RenderWidgetHostViewBase* view = 715 RenderWidgetHostViewBase* view =
731 rwh ? static_cast<RenderWidgetHostViewBase*>(rwh->GetView()) : NULL; 716 rwh ? static_cast<RenderWidgetHostViewBase*>(rwh->GetView()) : NULL;
(...skipping 232 matching lines...) Expand 10 before | Expand all | Expand 10 after
964 WebContentsMediaCaptureId::IsAutoThrottlingOptionSet(device_id)); 949 WebContentsMediaCaptureId::IsAutoThrottlingOptionSet(device_id));
965 } 950 }
966 951
967 void WebContentsVideoCaptureDevice::AllocateAndStart( 952 void WebContentsVideoCaptureDevice::AllocateAndStart(
968 const media::VideoCaptureParams& params, 953 const media::VideoCaptureParams& params,
969 scoped_ptr<Client> client) { 954 scoped_ptr<Client> client) {
970 DVLOG(1) << "Allocating " << params.requested_format.frame_size.ToString(); 955 DVLOG(1) << "Allocating " << params.requested_format.frame_size.ToString();
971 core_->AllocateAndStart(params, std::move(client)); 956 core_->AllocateAndStart(params, std::move(client));
972 } 957 }
973 958
974 void WebContentsVideoCaptureDevice::RequestRefreshFrame() {
975 core_->RequestRefreshFrame();
976 }
977
978 void WebContentsVideoCaptureDevice::StopAndDeAllocate() { 959 void WebContentsVideoCaptureDevice::StopAndDeAllocate() {
979 core_->StopAndDeAllocate(); 960 core_->StopAndDeAllocate();
980 } 961 }
981 962
982 } // namespace content 963 } // namespace content
OLDNEW

Powered by Google App Engine
This is Rietveld 408576698