Android implementation of WebAudio audio file decoder

media/base/android/java/src/org/chromium/media/WebAudioMediaCodecBridge.java

+// Copyright (c) 2013 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package org.chromium.media;
+
+import android.content.Context;
+import android.media.AudioFormat;
+import android.media.MediaCodec;
+import android.media.MediaCodec.BufferInfo;
+import android.media.MediaExtractor;
+import android.media.MediaFormat;
+import android.util.Log;
+
+import java.nio.ByteBuffer;
+import android.os.ParcelFileDescriptor;
+
+import org.chromium.base.CalledByNative;
+import org.chromium.base.JNINamespace;
+
+@JNINamespace("media")
+class WebAudioMediaCodecBridge {
+    static final String LOG_TAG = "WebAudioMediaCodec";
+    // TODO(rtoy): What is the correct timeout value for reading
+    // from a file in memory?
+    static final long TIMEOUT_MICROSECONDS = 500;
+    @CalledByNative
+    private static boolean decodeAudioFile(Context ctx, int nativeMediaCodecBridge, int inputFD) {
+        MediaExtractor extractor;
+        extractor = new MediaExtractor();
+
+        ParcelFileDescriptor encodedFD;
+        try {
+            encodedFD = ParcelFileDescriptor.adoptFd(inputFD);
+            extractor.setDataSource(encodedFD.getFileDescriptor());
+        } catch (Exception e) {
+            e.printStackTrace();

[digit1, 2013/04/04 15:56:47]

I assume you would want to call encodedFD.detachFd() here too.

+            return false;
+        }
+
+        if (extractor.getTrackCount() <= 0) {
+            encodedFD.detachFd();
+            return false;
+        }
+
+        MediaFormat format = extractor.getTrackFormat(0);
+
+        int channelCount = format.getInteger(MediaFormat.KEY_CHANNEL_COUNT);
+        int sampleRate = format.getInteger(MediaFormat.KEY_SAMPLE_RATE);
+        String mime = format.getString(MediaFormat.KEY_MIME);
+
+        long duration_microseconds = 0;
+        if (format.containsKey(MediaFormat.KEY_DURATION)) {
+            try {
+                duration_microseconds = format.getLong(MediaFormat.KEY_DURATION);
+            } catch (Exception e) {
+                Log.d(LOG_TAG, "Cannot get duration");

[digit1, 2013/04/04 15:56:47]

should you exit here, or is it ok to keep duration_microseconds at 0?

[Raymond Toy (Google), 2013/04/04 17:47:35]

I believe it's ok to keep going.  MediaCodec will decode the correct number of
samples, even if MediaFormat could not determine the duration. 

I don't know under what conditions this happens, but I have seen this on some
mp3 files on some demos.

+            }
+        }
+
+        Log.d(LOG_TAG, "Tracks: " + extractor.getTrackCount()

[digit1, 2013/04/04 15:56:47]

This Log.d() call is really slow: it will create a StringBuilder and perform
multiple appends on it before calling a function that does absolutely nothing
unless you run on a non-production device.

For this typical use case, it is recommended to guard this with an if (DEBUG) {
Log.i() }, where DEBUG is a static final boolean that is defined to false by
default. E.g.:

  private static final DEBUG = false;

  ...

  if (DEBUG) {
    Log.i(LOG_TAG, "Tracks:" + ....);
  }

It will get optimized out on production builds, but you can easily toogle this
during debugging.

[Raymond Toy (Google), 2013/04/04 17:47:35]

Ok.  I'll add this, but I do hope to remove this particular log after letting
webaudio bake for a bit on Android.  I'd like to make the DEBUG default to true
for now.

+              + " Rate: " + sampleRate
+              + " Channels: " + channelCount
+              + " Mime: " + mime
+              + " Duration: " + duration_microseconds + " microsec");
+
+        // For audio/vorbis files, MediaFormat returns a really huge
+        // (multi-year) duration value even for short files.  Tell
+        // nativeInitializeDestination that this is a vorbis file so
+        // it can handle it properly.
+        //
+        // See b/8528051
+        nativeInitializeDestination(nativeMediaCodecBridge,
+                                    channelCount,
+                                    sampleRate,
+                                    duration_microseconds,
+                                    mime.equals("audio/vorbis"));
+
+        // Create decoder
+        MediaCodec codec = MediaCodec.createDecoderByType(mime);
+        codec.configure(format, null /* surface */, null /* crypto */, 0 /* flags */);
+        codec.start();
+
+        ByteBuffer[] codecInputBuffers = codec.getInputBuffers();
+        ByteBuffer[] codecOutputBuffers = codec.getOutputBuffers();
+
+        // A track must be selected and will be used to read samples.
+        extractor.selectTrack(0);
+
+        boolean sawInputEOS = false;
+        boolean sawOutputEOS = false;
+
+        // Keep processing until the output is done.
+        while (!sawOutputEOS) {
+            int sampleSize = 0;
+            if (!sawInputEOS) {
+                // Input side
+                int inputBufIndex = codec.dequeueInputBuffer(TIMEOUT_MICROSECONDS);
+
+                if (inputBufIndex >= 0) {
+                    ByteBuffer dstBuf = codecInputBuffers[inputBufIndex];
+                    sampleSize = extractor.readSampleData(dstBuf, 0);
+                    long presentationTimeMicroSec = 0;
+
+                    if (sampleSize < 0) {
+                        sawInputEOS = true;
+                        sampleSize = 0;
+                    } else {
+                        presentationTimeMicroSec = extractor.getSampleTime();
+                    }
+
+                    codec.queueInputBuffer(inputBufIndex,
+                                           0, /* offset */
+                                           sampleSize,
+                                           presentationTimeMicroSec,
+                                           sawInputEOS ? MediaCodec.BUFFER_FLAG_END_OF_STREAM : 0);
+
+                    if (!sawInputEOS) {
+                        extractor.advance();
+                    }
+                }
+            }
+
+            // Output side
+            MediaCodec.BufferInfo info = new BufferInfo();
+            final int outputBufIndex = codec.dequeueOutputBuffer(info, TIMEOUT_MICROSECONDS);
+
+            if (outputBufIndex >= 0) {

[digit1, 2013/04/04 15:56:47]

Shouldn't you also handle INFO_OUTPUT_BUFFERS_CHANGED /
INFO_OUTPUT_FORMAT_CHANGED ? I admit the documentation isn't very clear about
this.

[Raymond Toy (Google), 2013/04/04 17:47:35]

Oops.  The case for INFO_OUTPUT_BUFFERS_CHANGED was deleted in an earlier patch.
 As far as I know, INFO_OUTPUT_FORMAT_CHANGED doesn't need any special handling,
and the previous code just printed out the new format.

+                ByteBuffer buf = codecOutputBuffers[outputBufIndex];
+
+                if (info.size > 0) {
+                    nativeOnChunkDecoded(nativeMediaCodecBridge, buf, info.size);
+                }
+
+                buf.clear();
+                codec.releaseOutputBuffer(outputBufIndex, false /* render */);
+
+                if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
+                    sawOutputEOS = true;
+                }
+            }
+        }
+
+        encodedFD.detachFd();
+
+        codec.stop();
+        codec.release();
+        codec = null;
+
+        return true;
+    }
+
+    private static native void nativeOnChunkDecoded(
+        int nativeWebAudioMediaCodecBridge, ByteBuffer buf, int size);
+
+    private static native void nativeInitializeDestination(
+        int nativeWebAudioMediaCodecBridge,
+        int channelCount,
+        int sampleRate,
+        long duration_microseconds,
+        boolean is_vorbis);
+}