Android implementation of text-to-speech code for Web Speech Synthesis API

chrome/android/java/src/org/chromium/chrome/browser/TtsPlatformImpl.java

+// Copyright (c) 2013 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package org.chromium.chrome.browser;
+
+import android.content.Context;
+import android.speech.tts.TextToSpeech;
+import android.speech.tts.UtteranceProgressListener;
+import java.lang.Double;
+import java.lang.Integer;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Locale;
+import org.chromium.base.CalledByNative;
+import org.chromium.base.ThreadUtils;
+
+/**
+ * This class is the Java counterpart to the C++ TtsPlatformImplAndroid class.
+ * It implements the Android-native text-to-speech code to support the web
+ * speech synthesis API.
+ */
+public class TtsPlatformImpl {

[bulach, 2013/05/16 11:10:28]

nit: if this class is only used as a bridge to c++, make it private.. in
general, we tend to use java visibility rule for java code, so I guess all
methods in this class could be private as well, since they're only
@CalledByNative or by this class itself..

[dmazzoni, 2013/05/16 16:36:40]

I'm not sure I understand - I thought a top-level class can't be private.

I made all of the methods private.

[bulach, 2013/05/17 10:09:04]

sorry, my bad. :) I meant the class would be "package private", i.e., without
the "public" modifier.
just to clarify:
"public" would mean that _any_ other java class could use it, which is not the
case.
"package private" restricts a bit more.
the methods themselves can all be "private", since they're only used by native..
not sure if it's any clearer :) but please let me know if it makes any sense..

+    private class TtsVoice {

[bulach, 2013/05/16 11:10:28]

nit: static

[dmazzoni, 2013/05/16 16:36:40]

Done.

+        public String name;

[bulach, 2013/05/16 11:10:28]

nit: mName, mLanguage
also, they don't need to be public, but they could be final

[dmazzoni, 2013/05/16 16:36:40]

Done.

+        public String language;
+    };
+
+    private int mNativeTtsPlatformImplAndroid;
+    private TextToSpeech mTextToSpeech;
+    private boolean mInitialized;
+    private ArrayList<TtsVoice> mVoices;
+    private String mCurrentLanguage;
+
+    private TtsPlatformImpl(int nativeTtsPlatformImplAndroid, Context context) {
+        mInitialized = false;
+        mNativeTtsPlatformImplAndroid = nativeTtsPlatformImplAndroid;
+        mTextToSpeech = new TextToSpeech(context, new TextToSpeech.OnInitListener() {
+                public void onInit(int status) {
+                    if (status == TextToSpeech.SUCCESS) {
+                        initialize();

[bulach, 2013/05/16 11:10:28]

not clear to me what is the threading model here.
is it initialize() called in the UI thread? should it be? :)
my question here is that the UtteranceProgressListener is yielding (back?) to
the UI thread, but this one is not, so probably worth a comment.

[dmazzoni, 2013/05/16 16:36:40]

I added a class comment to clarify: all C++ calls must happen on the UI
thread. Android callbacks may occur on a different thread, so we have to
go back to the UI thread when calling back to C++.

I think you're right about initialize, though - to be safe, I changed it
so that it calls initialized on the UI thread. Now there's no need for
anything to be synchronized because anything that changes the
state of this class happens on the UI thread.

+                    }
+                }
+            });
+        mTextToSpeech.setOnUtteranceProgressListener(new UtteranceProgressListener() {
+                public void onDone(final String utteranceId) {
+                    ThreadUtils.runOnUiThread(new Runnable() {
+                        @Override
+                        public void run() {
+                            if (mNativeTtsPlatformImplAndroid != 0) {
+                                nativeOnEndEvent(mNativeTtsPlatformImplAndroid,
+                                                 Integer.parseInt(utteranceId));
+                            }
+                        }
+                    });
+                }
+
+                public void onError(final String utteranceId) {
+                    ThreadUtils.runOnUiThread(new Runnable() {
+                        @Override
+                        public void run() {
+                            if (mNativeTtsPlatformImplAndroid != 0) {
+                                nativeOnErrorEvent(mNativeTtsPlatformImplAndroid,
+                                                   Integer.parseInt(utteranceId));
+                            }
+                        }
+                    });
+                }
+
+                public void onStart(final String utteranceId) {
+                    ThreadUtils.runOnUiThread(new Runnable() {
+                        @Override
+                        public void run() {
+                            if (mNativeTtsPlatformImplAndroid != 0) {
+                                nativeOnStartEvent(mNativeTtsPlatformImplAndroid,
+                                                   Integer.parseInt(utteranceId));
+                            }
+                        }
+                    });
+                }
+            });
+    };
+
+    /**
+     * Create a TtsPlatformImpl object, which is owned by TtsPlatformImplAndroid
+     * on the C++ side.
+     *
+     * @param nativeTtsPlatformImplAndroid The C++ object that owns us.
+     * @param context The app context.
+     */
+    @CalledByNative
+    public static TtsPlatformImpl create(int nativeTtsPlatformImplAndroid,
+                                         Context context) {
+        return new TtsPlatformImpl(nativeTtsPlatformImplAndroid, context);
+    }
+
+    /**
+     * Called when our C++ counterpoint is deleted. Clear the handle to our
+     * native C++ object, ensuring it's never called.
+     */
+    @CalledByNative
+    public void destroy() {
+        mNativeTtsPlatformImplAndroid = 0;
+    }
+
+    /**
+     * @return true if our TextToSpeech object is initialized and we've
+     * finished scanning the list of voices.
+     */
+    @CalledByNative
+    public boolean isInitialized() {
+        return mInitialized;
+    }
+
+    /**
+     * @return the number of voices.
+     */
+    @CalledByNative
+    public int getVoiceCount() {
+        assert mInitialized == true;
+        return mVoices.size();
+    }
+
+    /**
+     * @return the name of the voice at a given index.
+     */
+    @CalledByNative
+    public String getVoiceName(int voiceIndex) {
+        assert mInitialized == true;
+        return mVoices.get(voiceIndex).name;
+    }
+
+    /**
+     * @return the language of the voice at a given index.
+     */
+    @CalledByNative
+    public String getVoiceLanguage(int voiceIndex) {
+        assert mInitialized == true;
+        return mVoices.get(voiceIndex).language;
+    }
+
+    /**
+     * Attempt to start speaking an utterance. If it returns true, will call back on
+     * start and end.
+     *
+     * @param utteranceId A unique id for this utterance so that callbacks can be tied
+     *     to a particular utterance.
+     * @param text The text to speak.
+     * @param lang The language code for the text (e.g., "en-US").
+     * @param rate The speech rate, in the units expected by Android TextToSpeech.
+     * @param pitch The speech pitch, in the units expected by Android TextToSpeech.
+     * @param volume The speech volume, in the units expected by Android TextToSpeech.
+     * @return true on success.
+     */
+    @CalledByNative
+    public boolean speak(int utteranceId, String text, String lang,
+                         float rate, float pitch, float volume) {
+        assert mInitialized == true;
+        if (lang != mCurrentLanguage) {
+            mTextToSpeech.setLanguage(new Locale(lang));
+            mCurrentLanguage = lang;
+        }
+
+        mTextToSpeech.setSpeechRate(rate);
+        mTextToSpeech.setPitch(pitch);
+        HashMap<String, String> params = new HashMap<String, String>();
+        if (volume != 1.0) {
+            params.put(TextToSpeech.Engine.KEY_PARAM_VOLUME, Double.toString(volume));
+        }
+        params.put(TextToSpeech.Engine.KEY_PARAM_UTTERANCE_ID, Integer.toString(utteranceId));
+        int result = mTextToSpeech.speak(text, TextToSpeech.QUEUE_FLUSH, params);
+        return (result == TextToSpeech.SUCCESS);
+    }
+
+    /**
+     * Stop the current utterance.
+     */
+    @CalledByNative
+    public void stop() {
+        assert mInitialized == true;
+        mTextToSpeech.stop();
+    }
+
+    private void initialize() {
+        // Note: Android supports multiple speech engines, but querying the
+        // metadata about all of them is expensive. So we deliberately only
+        // support the default speech engine, and expose the different
+        // supported languages for the default engine as different voices.
+        String defaultEngineName = mTextToSpeech.getDefaultEngine();
+        String engineLabel = defaultEngineName;
+        for (TextToSpeech.EngineInfo info : mTextToSpeech.getEngines()) {
+            if (info.name.equals(defaultEngineName)) engineLabel = info.label;
+        }
+        Locale[] locales = Locale.getAvailableLocales();
+        mVoices = new ArrayList<TtsVoice>();
+        for (int i = 0; i < locales.length; ++i) {
+            if (!locales[i].getVariant().isEmpty()) continue;
+            if (mTextToSpeech.isLanguageAvailable(locales[i]) > 0) {
+                TtsVoice voice = new TtsVoice();
+                voice.name = locales[i].getDisplayLanguage();
+                if (!locales[i].getCountry().isEmpty()) {
+                    voice.name += " " + locales[i].getDisplayCountry();
+                }
+                voice.language = locales[i].toString();
+                mVoices.add(voice);
+            }
+        }
+
+        mInitialized = true;
+
+        ThreadUtils.runOnUiThread(new Runnable() {
+            @Override
+            public void run() {
+                if (mNativeTtsPlatformImplAndroid != 0) {
+                    nativeVoicesChanged(mNativeTtsPlatformImplAndroid);
+                }
+            }
+        });
+    }
+
+    private native void nativeVoicesChanged(int nativeTtsPlatformImplAndroid);
+    private native void nativeOnEndEvent(int nativeTtsPlatformImplAndroid, int utteranceId);
+    private native void nativeOnStartEvent(int nativeTtsPlatformImplAndroid, int utteranceId);
+    private native void nativeOnErrorEvent(int nativeTtsPlatformImplAndroid, int utteranceId);
+}