Implemented HEVC video demuxing and parsing

net/base/mime_util.cc

Index: net/base/mime_util.cc
diff --git a/net/base/mime_util.cc b/net/base/mime_util.cc
index a7fe9f3eea57b099becd42f0b45ad61bbe883738..c4aab5a3e26879bc8675452102462934a275d139 100644
--- a/net/base/mime_util.cc
+++ b/net/base/mime_util.cc
@@ -45,6 +45,7 @@ class MimeUtil : public PlatformMimeUtil {
     H264_BASELINE,
     H264_MAIN,
     H264_HIGH,
+    HEVC_MAIN,
     VP8,
     VP9,
     THEORA
@@ -464,6 +465,9 @@ static bool IsCodecSupportedOnAndroid(MimeUtil::Codec codec) {
     case MimeUtil::VORBIS:
       return true;
 
+    case MimeUtil::HEVC_MAIN:
+      return false;
+
     case MimeUtil::MPEG2_AAC_LC:
     case MimeUtil::MPEG2_AAC_MAIN:
     case MimeUtil::MPEG2_AAC_SSR:
@@ -523,6 +527,9 @@ static const char kMP4AudioCodecsExpression[] =
     "mp4a.40.05,mp4a.40.29";
 static const char kMP4VideoCodecsExpression[] =
     "avc1.42E00A,avc1.4D400A,avc1.64000A,"
+#if defined(ENABLE_HEVC_DEMUXING)

[ddorwin, 2015/02/21 02:09:21]

"DEMUXING" is not really accurate. This enables support in general.

[servolk, 2015/02/23 19:16:16]

No, I believe this is accurate. What exactly do you mean by "support in
general"? This change doesn't contain anything to support h265 decoding (which
would require us to deal with h265 licensing), it only enables demuxing and
parsing (which doesn't require any licenses) and this will allow us to support
hevc/h265 on platforms which have "external" h265 decoder, e.g. hardware h265
decoder, for which license costs are going to be paid by hardware manufacturer,
or OS-provided h265 decoder, for which license costs are going to be paid by OS
vendor. So, again, I believe the term "demuxing" is correct in this context, and
in fact we should be very careful about this, since changing it to something
more generic, like "enable_hevc" could get us in trouble by not being clear
enough that all we do is demuxing. Please let me know if you disagree and we can
discuss further.

[ddorwin, 2015/02/23 20:00:49]

The primary "support" it enables is advertising support via canPlayType().

Regardless of what this is called, we're clearly not enabling a software
decoder, but it seems reasonable to reuse this value if one needed to use it to
enable code around their decoder.

[servolk, 2015/02/24 02:15:13]

Yes, this does enable hevc support via canPlayType, but my understanding was
that only platforms that actually support hevc would explicitly set
enable_hevc_demuxing, so it's probably ok to gate mime type checks with this
flag as well.

[ddorwin, 2015/02/24 18:07:33]

Agreed. My point is that this flag enables HEVC, not just HEVC deumuxing. Since
it would only be anbled on platforms that support HEVC, I don't think we need to
worry about the other issues you mentioned and can just call it ENABLE_HEVC.

+    "hev1,hvc1,"

[ddorwin, 2015/02/21 02:09:21]

See below.

[ddorwin, 2015/02/23 20:00:49]

See some of the comments about how avc1 is handled in
https://codereview.chromium.org/952673002/. We should only have a single
unambiguous HEVC_MAIN string here.

[ddorwin, 2015/02/23 20:12:43]

Note that you must have an unambiguous string or you'll hit a DCHECK when
format_codec_mappings is processed.

[servolk, 2015/02/24 02:15:13]

Wait, I don't understand - even in that CL we have both avc1 and avc3 - so
there's two strings for hevc as well. Are you saying that there should be only
avc3+hvc1?

[ddorwin, 2015/02/24 18:07:33]

I didn't change anything in that CL, I just explained why things are the way
they are. (One of the things we need to do is replace these strings with
H264_BASELINE, etc.)
Look at line 529 here. We only use "avc1..." (not "avc3...). The whole point of
this is to add H264_BASELINE, etc. to a map.

In addition, these strings are unambiguous. You need an actual "hev1.xyzabc" or
you will hit a DCHECK.

+#endif
     "mp4a.66,mp4a.67,mp4a.68,mp4a.69,mp4a.6B,mp4a.40.2,mp4a.40.02,mp4a.40.5,"
     "mp4a.40.05,mp4a.40.29";
 
@@ -556,6 +563,10 @@ struct CodecIDMappings {
 // The "mp4a" strings come from RFC 6381.
 static const CodecIDMappings kUnambiguousCodecIDs[] = {
     {"1", MimeUtil::PCM},  // We only allow this for WAV so it isn't ambiguous.
+#if defined(ENABLE_HEVC_DEMUXING)
+    {"hev1", MimeUtil::HEVC_MAIN},

[ddorwin, 2015/02/21 02:09:21]

I don't think we should support these extension-less versions. It was a mistake
to support "avc1", which is not spec-compliant, and now we are stuck supporting
it forever.

Are there really two separate variants? Why? Can we pick one and lead the
industry in that direction?

[ddorwin, 2015/02/21 02:44:40]

To clarify, this may not have been so much a mistake as needing to be compatible
with other browsers that supported it. But as the first (?) UA to support HEVC,
please try to avoid setting such a precedent.

[servolk, 2015/02/23 19:16:16]

As far as I know the difference between hev1 and hvc1 is analogous to that
between avc1 and avc3. It's defined in ISO/IEC 14496-15 (for avc3/hev1 decoder
params PPS/SPS/VPS are embedded in elementary stream, whereas avc1/hvc1 contain
all params in avcC/hevC record). Chrome actually supports both avc1 and avc3, so
I think we should do the same with hev1/hvc1 - i.e. we should support both, they
are both standard but cover slightly different use cases, which gives some
flexibility to encoders. My understanding is that people do really need to have
both, one is typically used e.g. in live streaming scenarios, where it's
essential to be able to change decoder parameters on the fly to accommodate
bandwidth and content bitrate changes. Whereas another one is typically used by
offline encoders, which can afford to preprocess the whole media stream
beforehand, collect all the necessary parameter sets and write them down in the
decoder config box of the MP4 file. It doesn't really matter if we are the first
UA to support HEVC (I doubt it), we don't get to pick and choose. There is an
international standard ISO/IEC 14496-15 and we need to try and implement/support
as much of it as possible. I don't see a good reason to go against the standard
and support only one type, when the standard clearly says there's two and
clearly specifies the difference between the two.
Btw, other media stacks/libraries are also going to support both, for example
take a look at FFmpeg's libavformat/isom.c - they also support both hev1 and
hvc1.

[ddorwin, 2015/02/23 20:00:50]

Thanks for the exmplanation of "hev" vs. "hvc". I thought maybe there was no
clear decision in this area (vs. being defined to be different by a spec). I
agree that supporting both makes sense if it is like avc1 vs. avc3.

However, I still don't think we should support the extensionless strings (unless
that is also specified to be valid by some spec). (For avc1, it is not
spec-compliant.) This should be in the Ambiguous array like avc1/avc3.

[ddorwin, 2015/02/23 20:12:43]

Oops, strike that. Because we do not want to support the extensionless strings,
this should not appear in either list. ParseHEVCCodecID will handle this like we
do for avc1.

+    {"hvc1", MimeUtil::HEVC_MAIN},
+#endif
     {"mp3", MimeUtil::MP3},
     {"mp4a.66", MimeUtil::MPEG2_AAC_MAIN},
     {"mp4a.67", MimeUtil::MPEG2_AAC_LC},
@@ -1008,6 +1019,18 @@ bool MimeUtil::StringToCodec(const std::string& codec_id,
     return true;
   }
 
+#if defined(ENABLE_HEVC_DEMUXING)

[ddorwin, 2015/02/23 20:00:49]

Move this code to ParseHEVCCodecID or something like that.
We should try to keep this function as simple as possible.

+  if (StartsWithASCII(codec_id, "hev1.", true) ||

[ddorwin, 2015/02/21 02:09:21]

Once the other issues are addressed, you'll need to update
media_canplaytype_browsertest.cc. (Wait for
https://codereview.chromium.org/918463002/.)

[servolk, 2015/02/23 19:16:16]

Ok, yeah, once we have more clarity here we'll try to add more unit tests,
including mime type tests.

[ddorwin, 2015/02/23 20:00:49]

You should add at least some basic tests soon. This code is complex and
difficult to get right.

[servolk, 2015/02/24 02:15:13]

Sure, I will add some tests. In fact I've already added some unit tests for
ffmpeg and chunk demuxer. Do we have any end-to-end MSE tests for Chromium?

[ddorwin, 2015/02/24 18:07:32]

I think we have pipeline tests and content and chrome browser tests, both for
clear and encrypted content.

I was specifically referring to canPlayType() tests, though. The CL I referenced
has landed, so if you sync, you'll see examples of the detailed coverage that is
necessary for these codec strings.

+      StartsWithASCII(codec_id, "hvc1.", true)) {
+    // TODO(servolk): Implement parsing of hevc codec ids as described in

[ddorwin, 2015/02/21 02:09:21]

Our avc1 string handling is a mess. Similar to my comments above, we should be
accurate and restrictive from the beginning to avoid similar problems.

[servolk, 2015/02/23 19:16:16]

I understand your concerns, but we are in the early stages of implementation
here. We just don't know yet which profiles our hardware partners are going to
need/support. So for now we'd prefer to allow them all, to facilitate testing.

[ddorwin, 2015/02/23 20:00:49]

Can we at least check for valid values, similar to IsValidH264Level()?

[servolk, 2015/02/24 02:15:13]

Yep, will do

+    // ETSI TS 126 244 standard section A.2.2, but for now allow any
+    // HEVC tiers/profiles and let decoder decide if it can handle that.
+    *codec = MimeUtil::HEVC_MAIN;
+    *is_ambiguous = false;

[ddorwin, 2015/02/21 02:09:21]

This should be true since you do no verification.

[servolk, 2015/02/23 19:16:16]

yep, that's a good point, I'll change this.

+    return true;
+  }
+#endif
+
   // If |codec_id| is not in |string_to_codec_map_|, then we assume that it is
   // an H.264 codec ID because currently those are the only ones that can't be
   // stored in the |string_to_codec_map_| and require parsing.
@@ -1038,6 +1061,7 @@ bool MimeUtil::IsCodecProprietary(Codec codec) const {
     case H264_BASELINE:
     case H264_MAIN:
     case H264_HIGH:
+    case HEVC_MAIN:
       return true;
 
     case PCM: