Render inband text tracks in the media pipeline

media/base/demuxer.h

 // Copyright (c) 2012 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 #ifndef MEDIA_BASE_DEMUXER_H_
 #define MEDIA_BASE_DEMUXER_H_
 
 #include <vector>
 
 #include "base/time/time.h"
@@ skipping 51 matching lines @@
   virtual void Stop(const base::Closure& callback);
 
   // This method is called from the pipeline when the audio renderer
   // is disabled. Demuxers can ignore the notification if they do not
   // need to react to this event.
   //
   // TODO(acolwell): Change to generic DisableStream(DemuxerStream::Type).
   // TODO(scherkus): this might not be needed http://crbug.com/234708
   virtual void OnAudioRendererDisabled();
 
-  // Returns the given stream type, or NULL if that type is not present.
+  // Returns the first stream of the given stream type, or NULL if that type
+  // of stream is not present.
   virtual DemuxerStream* GetStream(DemuxerStream::Type type) = 0;
 
+  // Returns the total number of streams in the media.
+  virtual int GetStreamCount() const;
+
+  // Returns the media stream with the given |index|.
+  virtual DemuxerStream* GetStreamByIndex(int index);

[acolwell GONE FROM CHROMIUM, 2013/09/12 00:15:15]

I think this API can be problematic for the MSE case where tracks can come and
go based on SourceBuffers coming and going. In general, I'd like to move away
from the GetStream() style method and have the demuxer add/remove streams via
the DemuxerHost with something along these lines.

bool AddStream(DemuxerStream* stream);
void RemoveStream(DemuxerStream* stream);

This would remove the need for the index because the Pipeline could pass the
DemuxerStream directly to the appropriate renderer. For now I think we should
try this for TextTracks and if we like it, then I can migrate the audio & video
code to use the same mechanism.

[Matthew Heaney (Chromium), 2013/09/13 19:51:54]

It looks like the only class that inherits DemuxerHost is Pipeline.  I think
what you're suggesting here is that the demuxer should hold a pointer to the
Pipeline (perhaps via the DemuxerHost interface), and that as it finds text
streams, it should call its DemuxerHost with AddStream.

The Pipeline, which implements AddStream, will forward that request to the text
renderer, and the text renderer will adjust its container-of-streams
accordingly.

Is my understanding correct?

[acolwell GONE FROM CHROMIUM, 2013/09/13 20:57:30]

Yes. At this point I think signatures along the lines of

bool AddTextStream(DemuxerTextStream*);
void RemoveTextStream(DemuxerTextStream*); 

should work. I am suggesting DemuxerTextStream instead of DemuxerStream because
then you could avoid having to cram the TextCue data into a DecoderBuffer. You
could just have the demuxer return the data in a more natural form and not have
to do the delimited side-data like you have now.
Exactly.
Yes.

[Matthew Heaney (Chromium), 2013/09/20 23:53:54]

I made this change, but haven't introduced a separate DemuxerTextStream type
just yet.  Such a type would complicate the implementation of the demuxer.  (Not
that that's a bad thing, I just wanted to get all the big issues such as the
threading model working first.)

+
   // Returns the starting time for the media file.
   virtual base::TimeDelta GetStartTime() const = 0;
 
  private:
   DISALLOW_COPY_AND_ASSIGN(Demuxer);
 };
 
 }  // namespace media
 
 #endif  // MEDIA_BASE_DEMUXER_H_