Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(178)

Side by Side Diff: content/renderer/media/audio_input_device.h

Issue 10790121: First step towards moving AudioDevice from content/ to media/audio. (Closed) Base URL: svn://svn.chromium.org/chrome/trunk/src
Patch Set: Update comments after closing ppapi bug Created 8 years, 4 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
OLDNEW
1 // Copyright (c) 2012 The Chromium Authors. All rights reserved. 1 // Copyright (c) 2012 The Chromium Authors. All rights reserved.
2 // Use of this source code is governed by a BSD-style license that can be 2 // Use of this source code is governed by a BSD-style license that can be
3 // found in the LICENSE file. 3 // found in the LICENSE file.
4 4
5 // Low-latency audio capturing unit utilizing audio input stream provided 5 // Low-latency audio capturing class utilizing audio input stream provided
6 // by browser process through IPC. 6 // by a server (browser) process by use of an IPC interface.
7 // 7 //
8 // Relationship of classes: 8 // Relationship of classes:
9 // 9 //
10 // AudioInputController AudioInputDevice 10 // AudioInputController AudioInputDevice
11 // ^ ^ 11 // ^ ^
12 // | | 12 // | |
13 // v IPC v 13 // v IPC v
14 // AudioInputRendererHost <---------> AudioInputMessageFilter 14 // AudioInputRendererHost <---------> media::AudioInputIPCDelegate
15 // ^ 15 // ^ (impl in AudioInputMessageFilter)
16 // | 16 // |
17 // v 17 // v
18 // AudioInputDeviceManager 18 // AudioInputDeviceManager
19 // 19 //
20 // Transportation of audio samples from the browser to the render process 20 // Transportation of audio samples from the browser to the render process
21 // is done by using shared memory in combination with a sync socket pair 21 // is done by using shared memory in combination with a SyncSocket.
22 // to generate a low latency transport. The AudioInputDevice user registers 22 // The AudioInputDevice user registers an AudioInputDevice::CaptureCallback by
23 // an AudioInputDevice::CaptureCallback at construction and will be called 23 // calling Initialize(). The callback will be called with recorded audio from
24 // by the AudioInputDevice with recorded audio from the underlying audio layers. 24 // the underlying audio layers.
25 // The session ID is used by the AudioInputRendererHost to start the device 25 // The session ID is used by the AudioInputRendererHost to start the device
26 // referenced by this ID. 26 // referenced by this ID.
27 // 27 //
28 // State sequences: 28 // State sequences:
29 // 29 //
30 // Task [IO thread] IPC [IO thread]
31 //
32 // Sequence where session_id has not been set using SetDevice(): 30 // Sequence where session_id has not been set using SetDevice():
33 // Start -> InitializeOnIOThread -----> AudioInputHostMsg_CreateStream -------> 31 // ('<-' signifies callbacks, -> signifies calls made by AudioInputDevice)
34 // <- OnLowLatencyCreated <- AudioInputMsg_NotifyLowLatencyStreamCreated <- 32 // Start -> InitializeOnIOThread -> CreateStream ->
35 // ---> StartOnIOThread ---------> AudioInputHostMsg_PlayStream --------> 33 // <- OnStreamCreated <-
34 // -> StartOnIOThread -> PlayStream ->
36 // 35 //
37 // Sequence where session_id has been set using SetDevice(): 36 // Sequence where session_id has been set using SetDevice():
38 // Start -> InitializeOnIOThread --> AudioInputHostMsg_StartDevice ---> 37 // Start -> InitializeOnIOThread -> StartDevice ->
39 // <---- OnStarted <-------------- AudioInputMsg_NotifyDeviceStarted <---- 38 // <- OnDeviceReady <-
40 // -> OnDeviceReady ------------> AudioInputHostMsg_CreateStream -------> 39 // -> CreateStream ->
41 // <- OnLowLatencyCreated <- AudioInputMsg_NotifyLowLatencyStreamCreated <- 40 // <- OnStreamCreated <-
42 // ---> StartOnIOThread ---------> AudioInputHostMsg_PlayStream --------> 41 // -> StartOnIOThread -> PlayStream ->
43 // 42 //
44 // AudioInputDevice::Capture => low latency audio transport on audio thread => 43 // AudioInputDevice::Capture => low latency audio transport on audio thread =>
45 // | 44 // |
46 // Stop --> ShutDownOnIOThread ------> AudioInputHostMsg_CloseStream -> Close 45 // Stop --> ShutDownOnIOThread ------> CloseStream -> Close
47 // 46 //
48 // This class utilizes three threads during its lifetime, namely: 47 // This class depends on two threads to function:
49 // 48 //
50 // 1. Creating thread. 49 // 1. An IO thread.
51 // Must be the main render thread. Start and Stop should be called on 50 // This thread is used to asynchronously process Start/Stop etc operations
52 // this thread. 51 // that are available via the public interface. The public methods are
53 // 2. IO thread. 52 // asynchronous and simply post a task to the IO thread to actually perform
54 // The thread within which this class receives all the IPC messages and 53 // the work.
55 // IPC communications can only happen in this thread. 54 // 2. Audio transport thread.
56 // 3. Audio transport thread. 55 // Responsible for calling the CaptureCallback and feed audio samples from
57 // Responsible for calling the CaptrureCallback and feed audio samples from 56 // the server side audio layer using a socket and shared memory.
58 // the audio layer in the browser process using sync sockets and shared
59 // memory.
60 // 57 //
61 // Implementation notes: 58 // Implementation notes:
62 //
63 // - Start() is asynchronous/non-blocking.
64 // - Stop() is synchronous/blocking.
65 // - SetDevice() is asynchronous/non-blocking.
66 // - The user must call Stop() before deleting the class instance. 59 // - The user must call Stop() before deleting the class instance.
67 60
68 #ifndef CONTENT_RENDERER_MEDIA_AUDIO_INPUT_DEVICE_H_ 61 #ifndef CONTENT_RENDERER_MEDIA_AUDIO_INPUT_DEVICE_H_
69 #define CONTENT_RENDERER_MEDIA_AUDIO_INPUT_DEVICE_H_ 62 #define CONTENT_RENDERER_MEDIA_AUDIO_INPUT_DEVICE_H_
70 63
71 #include <string> 64 #include <string>
72 #include <vector> 65 #include <vector>
73 66
74 #include "base/basictypes.h" 67 #include "base/basictypes.h"
75 #include "base/compiler_specific.h" 68 #include "base/compiler_specific.h"
76 #include "base/memory/scoped_ptr.h" 69 #include "base/memory/scoped_ptr.h"
77 #include "base/shared_memory.h" 70 #include "base/shared_memory.h"
78 #include "content/common/content_export.h" 71 #include "content/common/content_export.h"
79 #include "content/renderer/media/audio_device_thread.h" 72 #include "content/renderer/media/audio_device_thread.h"
80 #include "content/renderer/media/audio_input_message_filter.h" 73 #include "content/renderer/media/audio_input_message_filter.h"
81 #include "content/renderer/media/scoped_loop_observer.h" 74 #include "content/renderer/media/scoped_loop_observer.h"
82 #include "media/audio/audio_parameters.h" 75 #include "media/audio/audio_parameters.h"
83 76
84 // TODO(henrika): This class is based on the AudioDevice class and it has 77 // TODO(henrika): This class is based on the AudioDevice class and it has
85 // many components in common. Investigate potential for re-factoring. 78 // many components in common. Investigate potential for re-factoring.
86 // TODO(henrika): Add support for event handling (e.g. OnStateChanged, 79 // TODO(henrika): Add support for event handling (e.g. OnStateChanged,
87 // OnCaptureStopped etc.) and ensure that we can deliver these notifications 80 // OnCaptureStopped etc.) and ensure that we can deliver these notifications
88 // to any clients using this class. 81 // to any clients using this class.
89 class CONTENT_EXPORT AudioInputDevice 82 class CONTENT_EXPORT AudioInputDevice
90 : public AudioInputMessageFilter::Delegate, 83 : NON_EXPORTED_BASE(public media::AudioInputIPCDelegate),
91 NON_EXPORTED_BASE(public ScopedLoopObserver), 84 NON_EXPORTED_BASE(public ScopedLoopObserver),
92 public base::RefCountedThreadSafe<AudioInputDevice> { 85 public base::RefCountedThreadSafe<AudioInputDevice> {
93 public: 86 public:
94 class CONTENT_EXPORT CaptureCallback { 87 class CONTENT_EXPORT CaptureCallback {
95 public: 88 public:
96 virtual void Capture(const std::vector<float*>& audio_data, 89 virtual void Capture(const std::vector<float*>& audio_data,
97 int number_of_frames, 90 int number_of_frames,
98 int audio_delay_milliseconds, 91 int audio_delay_milliseconds,
99 double volume) = 0; 92 double volume) = 0;
100 virtual void OnCaptureError() = 0; 93 virtual void OnCaptureError() = 0;
101 protected: 94 protected:
102 virtual ~CaptureCallback() {} 95 virtual ~CaptureCallback() {}
103 }; 96 };
104 97
105 class CONTENT_EXPORT CaptureEventHandler { 98 class CONTENT_EXPORT CaptureEventHandler {
106 public: 99 public:
107 // Notification to the client that the device with the specific |device_id| 100 // Notification to the client that the device with the specific |device_id|
108 // has been started. 101 // has been started.
109 // This callback is triggered as a result of StartDevice(). 102 // This callback is triggered as a result of StartDevice().
110 virtual void OnDeviceStarted(const std::string& device_id) = 0; 103 virtual void OnDeviceStarted(const std::string& device_id) = 0;
111 104
112 // Notification to the client that the device has been stopped. 105 // Notification to the client that the device has been stopped.
113 virtual void OnDeviceStopped() = 0; 106 virtual void OnDeviceStopped() = 0;
114 107
115 protected: 108 protected:
116 virtual ~CaptureEventHandler() {} 109 virtual ~CaptureEventHandler() {}
117 }; 110 };
118 111
119 // Methods called on main render thread ------------------------------------- 112 AudioInputDevice(media::AudioInputIPC* ipc,
120 AudioInputDevice(const media::AudioParameters& params, 113 const scoped_refptr<base::MessageLoopProxy>& io_loop);
121 CaptureCallback* callback, 114
122 CaptureEventHandler* event_handler); 115 // Initializes the AudioInputDevice. This method must be called before
116 // any other methods can be used.
117 void Initialize(const media::AudioParameters& params,
118 CaptureCallback* callback,
119 CaptureEventHandler* event_handler);
123 120
124 // Specify the |session_id| to query which device to use. This method is 121 // Specify the |session_id| to query which device to use. This method is
125 // asynchronous/non-blocking. 122 // asynchronous/non-blocking.
126 // Start() will use the second sequence if this method is called before. 123 // Start() will use the second sequence if this method is called before.
127 void SetDevice(int session_id); 124 void SetDevice(int session_id);
128 125
129 // Starts audio capturing. This method is asynchronous/non-blocking. 126 // Starts audio capturing. This method is asynchronous/non-blocking.
130 // TODO(henrika): add support for notification when recording has started. 127 // TODO(henrika): add support for notification when recording has started.
131 void Start(); 128 void Start();
132 129
133 // Stops audio capturing. This method is synchronous/blocking. 130 // Stops audio capturing. This method is synchronous/blocking.
134 // TODO(henrika): add support for notification when recording has stopped. 131 // TODO(henrika): add support for notification when recording has stopped.
135 void Stop(); 132 void Stop();
136 133
137 // Sets the capture volume scaling, with range [0.0, 1.0] inclusive. 134 // Sets the capture volume scaling, with range [0.0, 1.0] inclusive.
138 // Returns |true| on success. 135 // Returns |true| on success.
139 bool SetVolume(double volume); 136 bool SetVolume(double volume);
140 137
141 // Gets the capture volume scaling, with range [0.0, 1.0] inclusive.
142 // Returns |true| on success.
143 bool GetVolume(double* volume);
144
145 double sample_rate() const {
146 return audio_parameters_.sample_rate();
147 }
148
149 int buffer_size() const {
150 return audio_parameters_.frames_per_buffer();
151 }
152
153 // Sets the Automatic Gain Control state to on or off. 138 // Sets the Automatic Gain Control state to on or off.
154 // This method must be called before Start(). It will not have any effect 139 // This method must be called before Start(). It will not have any effect
155 // if it is called while capturing has already started. 140 // if it is called while capturing has already started.
156 void SetAutomaticGainControl(bool enabled); 141 void SetAutomaticGainControl(bool enabled);
157 142
143 protected:
158 // Methods called on IO thread ---------------------------------------------- 144 // Methods called on IO thread ----------------------------------------------
159 // AudioInputMessageFilter::Delegate impl., called by AudioInputMessageFilter. 145 // media::AudioInputIPCDelegate implementation.
160 virtual void OnStreamCreated(base::SharedMemoryHandle handle, 146 virtual void OnStreamCreated(base::SharedMemoryHandle handle,
161 base::SyncSocket::Handle socket_handle, 147 base::SyncSocket::Handle socket_handle,
162 uint32 length) OVERRIDE; 148 int length) OVERRIDE;
163 virtual void OnVolume(double volume) OVERRIDE; 149 virtual void OnVolume(double volume) OVERRIDE;
164 virtual void OnStateChanged(AudioStreamState state) OVERRIDE; 150 virtual void OnStateChanged(
151 media::AudioInputIPCDelegate::State state) OVERRIDE;
165 virtual void OnDeviceReady(const std::string& device_id) OVERRIDE; 152 virtual void OnDeviceReady(const std::string& device_id) OVERRIDE;
153 virtual void OnIPCClosed() OVERRIDE;
166 154
167 protected: 155 friend class base::RefCountedThreadSafe<AudioInputDevice>;
168 virtual ~AudioInputDevice(); 156 virtual ~AudioInputDevice();
169 157
170 private: 158 private:
171 friend class base::RefCountedThreadSafe<AudioInputDevice>;
172
173 // Methods called on IO thread ---------------------------------------------- 159 // Methods called on IO thread ----------------------------------------------
174 // The following methods are tasks posted on the IO thread that needs to 160 // The following methods are tasks posted on the IO thread that needs to
175 // be executed on that thread. They interact with AudioInputMessageFilter and 161 // be executed on that thread. They interact with AudioInputMessageFilter and
176 // sends IPC messages on that thread. 162 // sends IPC messages on that thread.
177 void InitializeOnIOThread(); 163 void InitializeOnIOThread();
178 void SetSessionIdOnIOThread(int session_id); 164 void SetSessionIdOnIOThread(int session_id);
179 void StartOnIOThread(); 165 void StartOnIOThread();
180 void ShutDownOnIOThread(); 166 void ShutDownOnIOThread();
181 void SetVolumeOnIOThread(double volume); 167 void SetVolumeOnIOThread(double volume);
182 void SetAutomaticGainControlOnIOThread(bool enabled); 168 void SetAutomaticGainControlOnIOThread(bool enabled);
183 169
184 void Send(IPC::Message* message);
185
186 // MessageLoop::DestructionObserver implementation for the IO loop. 170 // MessageLoop::DestructionObserver implementation for the IO loop.
187 // If the IO loop dies before we do, we shut down the audio thread from here. 171 // If the IO loop dies before we do, we shut down the audio thread from here.
188 virtual void WillDestroyCurrentMessageLoop() OVERRIDE; 172 virtual void WillDestroyCurrentMessageLoop() OVERRIDE;
189 173
190 // Format
191 media::AudioParameters audio_parameters_; 174 media::AudioParameters audio_parameters_;
192 175
193 CaptureCallback* callback_; 176 CaptureCallback* callback_;
194 CaptureEventHandler* event_handler_; 177 CaptureEventHandler* event_handler_;
195 178
196 // The current volume scaling [0.0, 1.0] of the audio stream. 179 media::AudioInputIPC* ipc_;
197 double volume_;
198
199 // Cached audio input message filter (lives on the main render thread).
200 scoped_refptr<AudioInputMessageFilter> filter_;
201 180
202 // Our stream ID on the message filter. Only modified on the IO thread. 181 // Our stream ID on the message filter. Only modified on the IO thread.
203 int32 stream_id_; 182 int stream_id_;
204 183
205 // The media session ID used to identify which input device to be started. 184 // The media session ID used to identify which input device to be started.
206 // Only modified on the IO thread. 185 // Only modified on the IO thread.
207 int session_id_; 186 int session_id_;
208 187
209 // State variable used to indicate it is waiting for a OnDeviceReady() 188 // State variable used to indicate it is waiting for a OnDeviceReady()
210 // callback. Only modified on the IO thread. 189 // callback. Only modified on the IO thread.
211 bool pending_device_ready_; 190 bool pending_device_ready_;
212 191
213 // Stores the Automatic Gain Control state. Default is false. 192 // Stores the Automatic Gain Control state. Default is false.
214 // Only modified on the IO thread. 193 // Only modified on the IO thread.
215 bool agc_is_enabled_; 194 bool agc_is_enabled_;
216 195
217 // Our audio thread callback class. See source file for details. 196 // Our audio thread callback class. See source file for details.
218 class AudioThreadCallback; 197 class AudioThreadCallback;
219 198
220 // In order to avoid a race between OnStreamCreated and Stop(), we use this 199 // In order to avoid a race between OnStreamCreated and Stop(), we use this
221 // guard to control stopping and starting the audio thread. 200 // guard to control stopping and starting the audio thread.
222 base::Lock audio_thread_lock_; 201 base::Lock audio_thread_lock_;
223 AudioDeviceThread audio_thread_; 202 AudioDeviceThread audio_thread_;
224 scoped_ptr<AudioInputDevice::AudioThreadCallback> audio_callback_; 203 scoped_ptr<AudioInputDevice::AudioThreadCallback> audio_callback_;
225 204
226 DISALLOW_IMPLICIT_CONSTRUCTORS(AudioInputDevice); 205 DISALLOW_IMPLICIT_CONSTRUCTORS(AudioInputDevice);
227 }; 206 };
228 207
229 #endif // CONTENT_RENDERER_MEDIA_AUDIO_INPUT_DEVICE_H_ 208 #endif // CONTENT_RENDERER_MEDIA_AUDIO_INPUT_DEVICE_H_
OLDNEW
« no previous file with comments | « content/renderer/media/audio_device_unittest.cc ('k') | content/renderer/media/audio_input_device.cc » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698