Letting Flash join MediaSession (stack implementaion)

docs/audio_focus.md

Index: docs/audio_focus.md
diff --git a/docs/audio_focus.md b/docs/audio_focus.md
new file mode 100644
index 0000000000000000000000000000000000000000..dba9738f6d6c5917e75bfcb816438b5735a3460e
--- /dev/null
+++ b/docs/audio_focus.md
@@ -0,0 +1,86 @@
+# Audio Focus Handling
+
+Audio focus handling is part of the default media session on desktop, which
+manages the mixing of MediaSessions.

[whywhat, 2016/09/27 17:54:17]

Expand on what MediaSession is unless we have a doc about that? E.g. all
audio-producing media elements, WebAudio AudioContexts (?) and audible Pepper
plugins from the same page form one MediaSession. Each page can only have one
session until the MediaSession API allows them to form custom sets of audible
objects?

[Zhiqiang Zhang (Slow), 2016/09/28 16:12:18]

Done.

+
+[TOC]
+
+## Processing model
+
+### Audio focus types
+
+There are "persistent" and "transient" audio focus types. Persistent audios
+should not mix with each other and transient audio should play on top of

[whywhat, 2016/09/27 17:54:16]

Audio playback with persistent audio focus stops other playbacks with persistent
audio focus and ducks playbacks with transient audio focus. Playback with
transient audio focus ducks all other playing media no matter what audio focus
they have. MediaSession's with Pepper plugins (and/or WebAudio?) behave like
they have transient audio focus since Pepper plugins are unstoppable!

[Zhiqiang Zhang (Slow), 2016/09/28 16:12:18]

Elaborated a bit here. But we treat Pepper as Persistent. I have a section
Pepper in the very end. WebAudio is currently not managed by MediaSession I
think?

+persistent audio.
+
+### `MediaSession`
+
+Audio-producing objects should join `MediaSession` when they want to produce
+sound.  `MediaSession` has the following states:
+
+* ACTIVE: the `MediaSession` has audio focus and should be play normally.

[whywhat, 2016/09/27 17:54:16]

s/be play/be played

What does "normally" mean? Maybe "has audio focus and is playing"?

[Zhiqiang Zhang (Slow), 2016/09/28 16:12:18]

Done.

+* SUSPENDED: the MediaSession does not has audio focus. All audio-producing

[whywhat, 2016/09/27 17:54:16]

s/has/have

[Zhiqiang Zhang (Slow), 2016/09/28 16:12:18]

Done.

+  objects are paused and can be resumed when the session resumes.

[whywhat, 2016/09/27 17:54:16]

s/resumes/gains audio focus or is resumed by user?

[Zhiqiang Zhang (Slow), 2016/09/28 16:12:18]

Done.

[Zhiqiang Zhang (Slow), 2016/09/28 16:12:18]

Done.

+* INACTIVE: the MediaSession does not has audio focus, and there is no

[whywhat, 2016/09/27 17:54:16]

s/has/have

[Zhiqiang Zhang (Slow), 2016/09/28 16:12:18]

Done.

+  audio-producing objects in this `MediaSession`.
+
+Besides, `MediaSession` has a `DUCKING` flag, which means its managed
+audio-producing objects has lowered volume. The flag is orthogonal with
+`MediaSession` state.
+
+### `AudioFocusManager`
+
+`AudioFocusManager` is a global instance which manages the state of
+`MediaSession`s.
+
+When an audio-producing object wants to play audio, it should join `MediaSession`
+and tell which kind of audio focus type it requires. `MediaSession` will then
+request audio focus from `AudioFocusManager`, and will allow the object to play
+sound if successful. `AudioFocusManager` will notify other `MediaSession`s if
+their states are changed.
+
+When an audio-producing object stops playing audio, it should be removed from
+its `MediaSession`, and `MediaSession` should abandon its audio focus if there
+are no audio-producing object is managed by it. `AudioFocusManager` will notify

[whywhat, 2016/09/27 17:54:16]

s/is// or maybe just say if it's empty?

[Zhiqiang Zhang (Slow), 2016/09/28 16:12:17]

Done.

+other `MediaSession`s of state change if necessary.
+
+## The algorithm for handling audio focus
+
+`AudioFocusManager` uses a stack implementation. It keeps track of all
+ACTIVE/SUSPENDED `MediaSession`s. When a `MediaSession` requests audio focus, it
+will be put at the top of the stack, and will be removed from the stack when it
+abandons audio focus.
+
+The algorithm is as follows:
+
+* When a `MediaSession` requests audio focus:
+
+  * Remove it from the audio focus stack, and place it at the top of audio

[whywhat, 2016/09/27 17:54:16]

"Remove it from the audio focus stack if it's already there" otherwise it's
unclear why you're removing MS from the stack before putting it in it.

[Zhiqiang Zhang (Slow), 2016/09/28 16:12:18]

Done.

+    focus stack, grant focus to the session and let it play.
+  * If the session is persistent, suspend all sessions on the stack

[whywhat, 2016/09/27 17:54:16]

s/all/all the other

[Zhiqiang Zhang (Slow), 2016/09/28 16:12:17]

Done.

+  * If the session is transient:
+
+    * If the next top entry is transient, do nothing.
+    * If the next top entry is persistent, let the next top entry start ducking.

[whywhat, 2016/09/27 17:54:16]

Maybe clarify we took care of the remaining session when putting the previous
one on top?

[Zhiqiang Zhang (Slow), 2016/09/28 16:12:18]

Sorry, I'm not very sure of what you mean here. Can you explain a bit more?

[whywhat, 2016/09/29 22:40:47]

I meant, clarify why we only duck the top session, not all sessions (like we
should ideally, right?)
The clarification would be that we would duck the remaining sessions before when
adding the "next top session", so only the top session needs to be ducked at
this point.

+
+* When a `MediaSession` abandons audio focus:
+
+  * If the session is not on the top, just remove it from the stack.
+  * If the session is on the top, remove it from the stack.
+
+    * If the stack becomes empty, do nothing.
+    * If the next top session is transient, do nothing.
+    * If the next top session is persistent, stop ducking it.
+
+### Handling Pepper
+
+Pepper is different from media elements since it has a different model. Pepper
+cannot be paused, but its volume can be changed. When considering Pepper, the
+above algorithm must be modified.
+
+When Pepper joins `MediaSession`, it should request persistent focus type. When
+AudioFocusManager wants to suspend a `MediaSession`, it must check whether the
+session has Pepper instance, and if yes, it should duck the session instead.
+
+Also, whenever a session abandons focus, and the next top session is INACTIVE,
+`AudioFocusManager` should find the next session having Pepper and unduck it.