Pass ffmpeg media track info to HTMLMediaElement

media/blink/webmediaplayer_impl.cc

 // Copyright 2013 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 #include "media/blink/webmediaplayer_impl.h"
 
 #include <algorithm>
 #include <cmath>
 #include <limits>
 #include <string>
@@ skipping 836 matching lines @@
       ConvertToWebInitDataType(init_data_type), init_data.data(),
       base::saturated_cast<unsigned int>(init_data.size()));
 }
 
 void WebMediaPlayerImpl::OnFFmpegMediaTracksUpdated(
     scoped_ptr<MediaTracks> tracks) {
   // For MSE/chunk_demuxer case the media track updates are handled by
   // WebSourceBufferImpl.
   DCHECK(demuxer_.get());
   DCHECK(!chunk_demuxer_);
+  DCHECK(client_);

[wolenetz, 2016/03/11 23:19:54]

nit: are you aware of any path, in any WMPI method, where client_ might be null?
I think the basic assumption of all of WMPI is a non-null |client_|. Do we need
this line? If we do, maybe put it in ctor instead of here?

[servolk, 2016/03/18 18:36:42]

Good point, indeed the client_ is set only once in ctor, so I've moved the
DCHECK there.

+
+  // Remove the old tracks first, if any were present

[wolenetz, 2016/03/11 23:19:54]

I'm not aware of any code path that allows multiple calls to
OnFFmpegMediaTracksUpdated per WMPI instance. OnFindStreamInfoDone is done only
as part of FFmpegDemuxer::Initialize() processing, which is once per player
IIUC.

Are midstream track changes expected in any FFmpegDemuxer-based playback? I
don't think so now, and if this ever changes, we'll need to also adjust usage of
the media_tracks_updated_ callback. All this makes me think that the callback's
name and parameter needs some clarification, at least like "called exactly once
as part of initialization" (documented in code comments of FFmpegDemuxer::ctor
and FFmpegDemuxer::Initialize()).

I'm also growing concerned that, due to route this set of CLs is taking (this
one, https://codereview.chromium.org/1659653002/, and
https://codereview.chromium.org/1658033002/), where *initial* knowledge of the
various tracks is known at different times on the chromium side of the media
engine for MSE playbacks, that there could be some problems. For instance, as
part of compliant initialization segment received processing, if there are new
tracks, at what point will WMPI become aware of those blink track ids? We'll
need to have some hopefully common-to-at-least-WMPI across src= and MSE handling
of Blink signals like "enable audio track X" and "select video track Y". If X
and Y are known only to ChunkDemuxer, WebSourceBufferImpl, and Blink
SourceBuffer/HTMLMediaElement/etc, but not to WMPI, how will WMPI be able to
operate on the Blink enable/select signals? Maybe we should chat about this to
make sure we're on the same page. I'd be happy to hangout today, if you have
time.

[servolk, 2016/03/18 18:36:42]

Yes, currently FFmpeg tracks are only going to be reported once, I was just
curious to try and make this code a bit more future-proof, and as you can see
it's not that difficult to be future-proof here and allow the media track info
to be reported multiple times per WMPI lifetime (which would allow mid-stream
track configuration changes). But I agree for now this is not strictly necessary
and we can simplify things a bit by removing this, so I'll remove it for now.

Re the second part, I've put together another CL to demonstrate how we can
achieve handling of blink enable/disable/select track signals. All we need for
that is to be able to translate blink track ids into DemuxerStreams on demuxer
level. Then we can propagate those signals through the rest of media stack as a
set of DemuxStream* objects and that works just fine, see
https://codereview.chromium.org/1812543003/.

+  while (!blink_audio_tracks_.empty()) {
+    client_->removeAudioTrack(blink_audio_tracks_[0]);
+    blink_audio_tracks_.erase(blink_audio_tracks_.begin());
+  }
+  while (!blink_video_tracks_.empty()) {
+    client_->removeVideoTrack(blink_video_tracks_[0]);
+    blink_audio_tracks_.erase(blink_audio_tracks_.begin());
+  }
+
+  // Now report the new media tracks to blink
+  for (const auto& track : tracks->tracks()) {
+    if (track->type() == MediaTrack::Audio) {
+      auto track_id = client_->addAudioTrack(
+          blink::WebString::fromUTF8(track->id()),
+          blink::WebMediaPlayerClient::AudioTrackKindMain,
+          blink::WebString::fromUTF8(track->label()),
+          blink::WebString::fromUTF8(track->language()),
+          /*enabled*/ true);
+      blink_audio_tracks_.push_back(track_id);
+    } else if (track->type() == MediaTrack::Video) {
+      auto track_id = client_->addVideoTrack(
+          blink::WebString::fromUTF8(track->id()),
+          blink::WebMediaPlayerClient::VideoTrackKindMain,
+          blink::WebString::fromUTF8(track->label()),
+          blink::WebString::fromUTF8(track->language()),
+          /*selected*/ true);
+      blink_video_tracks_.push_back(track_id);
+    } else {
+      // Text tracks are not supported yet.
+      NOTREACHED();
+    }
+  }
 }
 
 void WebMediaPlayerImpl::OnWaitingForDecryptionKey() {
   encrypted_client_->didBlockPlaybackWaitingForKey();
 
   // TODO(jrummell): didResumePlaybackBlockedForKey() should only be called
   // when a key has been successfully added (e.g. OnSessionKeysChange() with
   // |has_additional_usable_key| = true). http://crbug.com/461903
   encrypted_client_->didResumePlaybackBlockedForKey();
 }
@@ skipping 743 matching lines @@
            << ", Video: " << stats.video_memory_usage << ", DataSource: "
            << (data_source_ ? data_source_->GetMemoryUsage() : 0)
            << ", Demuxer: " << demuxer_memory_usage;
 
   const int64_t delta = current_memory_usage - last_reported_memory_usage_;
   last_reported_memory_usage_ = current_memory_usage;
   adjust_allocated_memory_cb_.Run(delta);
 }
 
 }  // namespace media