|
|
Chromium Code Reviews|
Created:
3 years, 12 months ago by ke.he Modified:
3 years, 7 months ago CC:
chromium-reviews, creis+watch_chromium.org, qsr+mojo_chromium.org, viettrungluu+watch_chromium.org, nasko+codewatch_chromium.org, jam, yzshen+watch_chromium.org, abarth-chromium, haraken, Aaron Boodman, feature-vr-reviews_chromium.org, darin-cc_chromium.org, blink-reviews, darin (slow to review) Target Ref:
refs/heads/master Project:
chromium Visibility:
Public. |
DescriptionPorts VRService to be hosted in the device service.
Ports the VRService to be hosted by te Device Service instead of by
//content/browser.
BUG=612342
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_site_isolation
Patch Set 1 #Patch Set 2 : Ports VRService to be hosted in the device service. #Patch Set 3 : Ports VRService to be hosted in the device service. #
Total comments: 1
Patch Set 4 : Ports VRService to be hosted in the device service. #Patch Set 5 : Ports VRService to be hosted in the device service. #Patch Set 6 : make changes after code rebase. #Patch Set 7 : mock-vr-service.js #Patch Set 8 : Ports VRService to be hosted in the device service. #
Total comments: 2
Patch Set 9 : code rebase #Patch Set 10 : Ports VRService to be hosted in the device service. #Patch Set 11 : Ports VRService to be hosted in the device service. #Messages
Total messages: 83 (64 generated)
Description was changed from ========== Ports VRService to be hosted in the device service. BUG= ========== to ========== Ports VRService to be hosted in the device service. BUG= CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_site_isolation ==========
The CQ bit was checked by ke.he@intel.com to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: Try jobs failed on following builders: linux_chromium_asan_rel_ng on master.tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
The CQ bit was checked by ke.he@intel.com to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: Try jobs failed on following builders: linux_chromium_asan_rel_ng on master.tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...) linux_chromium_chromeos_rel_ng on master.tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
The CQ bit was checked by ke.he@intel.com to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: Try jobs failed on following builders: chromeos_daisy_chromium_compile_only_ng on master.tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/chromeos_daisy_...)
The CQ bit was checked by ke.he@intel.com to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: Try jobs failed on following builders: android_n5x_swarming_rel on master.tryserver.chromium.android (JOB_FAILED, https://build.chromium.org/p/tryserver.chromium.android/builders/android_n5x_...)
Patchset #2 (id:20001) has been deleted
Patchset #3 (id:60001) has been deleted
Patchset #2 (id:40001) has been deleted
The CQ bit was checked by ke.he@intel.com to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: Try jobs failed on following builders: android_n5x_swarming_rel on master.tryserver.chromium.android (JOB_FAILED, https://build.chromium.org/p/tryserver.chromium.android/builders/android_n5x_...)
The CQ bit was checked by ke.he@intel.com to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: This issue passed the CQ dry run.
Description was changed from ========== Ports VRService to be hosted in the device service. BUG= CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_site_isolation ========== to ========== Ports VRService to be hosted in the device service. Ports the VRService to be hosted by te Device Service instead of by //content/browser. BUG= CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_site_isolation ==========
ke.he@intel.com changed reviewers: + blundell@chromium.org
Hi,Colin, This patch is very similar as the porting of time_zone_monitor. PTAL. Thanks. By the way, what's the BUG ID?
This looks great, but it also re-raises the question in my mind of whether there should be one Device Service or many; when I see it laid out in code, it seems odd to have VR buried underneath the "Device Service." I'm going to re-raise this issue on chromium-mojo@. Let's hold off on this CL until resolving that question.
On 2016/12/29 16:44:04, blundell wrote: > This looks great, but it also re-raises the question in my mind of whether there > should be one Device Service or many; when I see it laid out in code, it seems > odd to have VR buried underneath the "Device Service." I'm going to re-raise > this issue on chromium-mojo@. Let's hold off on this CL until resolving that > question. Could you clarify something for me? Where does the Device Service run? Is it still in the browser process, or is it it's own process (or could it become it's own process some day?) Currently the VR service must run in the browser process on Android. I think we'll have more flexibility with desktop implementations.
On 2016/12/31 05:27:06, bajones (OOO till January) wrote: > On 2016/12/29 16:44:04, blundell wrote: > > This looks great, but it also re-raises the question in my mind of whether > there > > should be one Device Service or many; when I see it laid out in code, it seems > > odd to have VR buried underneath the "Device Service." I'm going to re-raise > > this issue on chromium-mojo@. Let's hold off on this CL until resolving that > > question. > > Could you clarify something for me? Where does the Device Service run? Is it > still in the browser process, or is it it's own process (or could it become it's > own process some day?) Currently the VR service must run in the browser process > on Android. I think we'll have more flexibility with desktop implementations. Hi Brandon, This is one of the things we're feeling out. The vision is that the Device Service would be able to run in its own process, but we're hitting limitations on other device features as well on Android. What causes the coupling of the VRService to the browser process on Android?
On 2017/01/03 15:34:40, blundell wrote: > Hi Brandon, > > This is one of the things we're feeling out. The vision is that the Device > Service would be able to run in its own process, but we're hitting limitations > on other device features as well on Android. What causes the coupling of the > VRService to the browser process on Android? I imagine it's similar to the other limitations you're encountering. The Daydream API requires that we create and manage the main API object from a Java object (GvrLayout: https://developers.google.com/vr/android/reference/com/google/vr/ndk/base/Gvr...) that must exist in the view hierarchy. This keeps us tied to the browser process, since that's the only process that has an android view hierarchy (and seems to be the only process that uses Java at all.) We've asked the Daydream team about how hard of a dependency this is, and it seems like it's here to stay for a variety of reasons.
On 2017/01/04 19:16:07, bajones (OOO till January) wrote: > On 2017/01/03 15:34:40, blundell wrote: > > Hi Brandon, > > > > This is one of the things we're feeling out. The vision is that the Device > > Service would be able to run in its own process, but we're hitting limitations > > on other device features as well on Android. What causes the coupling of the > > VRService to the browser process on Android? > > I imagine it's similar to the other limitations you're encountering. The > Daydream API requires that we create and manage the main API object from a Java > object (GvrLayout: > https://developers.google.com/vr/android/reference/com/google/vr/ndk/base/Gvr...) > that must exist in the view hierarchy. This keeps us tied to the browser > process, since that's the only process that has an android view hierarchy (and > seems to be the only process that uses Java at all.) We've asked the Daydream > team about how hard of a dependency this is, and it seems like it's here to stay > for a variety of reasons. I'm not sure what you're referring to Brandon, but only some parts of the GVR API can only be used from the browser process, like initial API context creation. After that the GVR API can be initialized and used from any single thread, but it's not threadsafe. crbug.com/674594 tracks the issue of using the GVR API from multiple threads simultaneously. VR Shell uses the GVR API from its GL Thread, while using the GvrLayout only from the UI (browser) thread, as is the intended usage pattern. However, the gamepad API and VRService use the API from the browser process while VR Shell uses it from its GL Thread. Ideally the VRService should either only asynchronously use GVR APIs, or should be hosted in VR Shell's GL Thread. This is complicated by the fact that the GL Thread doesn't exist currently while in magic window mode, so we would have to move (rebind?) the service between the browser process and GL Thread when entering/exiting presentation.
On 2017/01/04 19:40:20, mthiesse wrote: > On 2017/01/04 19:16:07, bajones (OOO till January) wrote: > > On 2017/01/03 15:34:40, blundell wrote: > > > Hi Brandon, > > > > > > This is one of the things we're feeling out. The vision is that the Device > > > Service would be able to run in its own process, but we're hitting > limitations > > > on other device features as well on Android. What causes the coupling of the > > > VRService to the browser process on Android? > > > > I imagine it's similar to the other limitations you're encountering. The > > Daydream API requires that we create and manage the main API object from a > Java > > object (GvrLayout: > > > https://developers.google.com/vr/android/reference/com/google/vr/ndk/base/Gvr...) > > that must exist in the view hierarchy. This keeps us tied to the browser > > process, since that's the only process that has an android view hierarchy (and > > seems to be the only process that uses Java at all.) We've asked the Daydream > > team about how hard of a dependency this is, and it seems like it's here to > stay > > for a variety of reasons. > > I'm not sure what you're referring to Brandon, but only some parts of the GVR > API can only be used from the browser process, like initial API context > creation. After that the GVR API can be initialized and used from any single > thread, but it's not threadsafe. > > crbug.com/674594 tracks the issue of using the GVR API from multiple threads > simultaneously. VR Shell uses the GVR API from its GL Thread, while using the > GvrLayout only from the UI (browser) thread, as is the intended usage pattern. > However, the gamepad API and VRService use the API from the browser process > while VR Shell uses it from its GL Thread. > > Ideally the VRService should either only asynchronously use GVR APIs, or should > be hosted in VR Shell's GL Thread. This is complicated by the fact that the GL > Thread doesn't exist currently while in magic window mode, so we would have to > move (rebind?) the service between the browser process and GL Thread when > entering/exiting presentation. I'm having trouble finding the Java side of the VR impl in Chromium. Could someone give me a pointer? Thanks!
On 2017/01/05 16:36:10, blundell wrote: > On 2017/01/04 19:40:20, mthiesse wrote: > > On 2017/01/04 19:16:07, bajones (OOO till January) wrote: > > > On 2017/01/03 15:34:40, blundell wrote: > > > > Hi Brandon, > > > > > > > > This is one of the things we're feeling out. The vision is that the Device > > > > Service would be able to run in its own process, but we're hitting > > limitations > > > > on other device features as well on Android. What causes the coupling of > the > > > > VRService to the browser process on Android? > > > > > > I imagine it's similar to the other limitations you're encountering. The > > > Daydream API requires that we create and manage the main API object from a > > Java > > > object (GvrLayout: > > > > > > https://developers.google.com/vr/android/reference/com/google/vr/ndk/base/Gvr...) > > > that must exist in the view hierarchy. This keeps us tied to the browser > > > process, since that's the only process that has an android view hierarchy > (and > > > seems to be the only process that uses Java at all.) We've asked the > Daydream > > > team about how hard of a dependency this is, and it seems like it's here to > > stay > > > for a variety of reasons. > > > > I'm not sure what you're referring to Brandon, but only some parts of the GVR > > API can only be used from the browser process, like initial API context > > creation. After that the GVR API can be initialized and used from any single > > thread, but it's not threadsafe. > > > > crbug.com/674594 tracks the issue of using the GVR API from multiple threads > > simultaneously. VR Shell uses the GVR API from its GL Thread, while using the > > GvrLayout only from the UI (browser) thread, as is the intended usage pattern. > > However, the gamepad API and VRService use the API from the browser process > > while VR Shell uses it from its GL Thread. > > > > Ideally the VRService should either only asynchronously use GVR APIs, or > should > > be hosted in VR Shell's GL Thread. This is complicated by the fact that the GL > > Thread doesn't exist currently while in magic window mode, so we would have to > > move (rebind?) the service between the browser process and GL Thread when > > entering/exiting presentation. > > I'm having trouble finding the Java side of the VR impl in Chromium. Could > someone give me a pointer? Thanks! It's at chrome/android/java/src/org/chromium/chrome/browser/vr_shell/VrShellImpl.java
blundell@chromium.org changed reviewers: + bajones@chromium.org, mthiesse@chromium.org
Apologies, it took a little while for me to analyze this code and process the implications here. (1) It's a non-goal to have the Device Service OOP on Android at the current time, so that's not a blocker. (2) //device/vr has code coupling to //chrome. In the long term, we should analyze that as well, but that shouldn't block us from further modularization of //content at this time. Ke He, I suggest that you add a README to //services/device noting that the Device Service is currently coupled to the browser process on certain platforms and giving a little context about VR in particular (//device/vr is used directly by //chrome/browser/android/vr_shell). I'll also file an umbrella bug for analyzing interdependencies between the Device Service and higher layers of Chromium (i.e., above //content), and then will file a bug under that for analyzing the VR coupling. I also have a technical question for Brandon below. https://codereview.chromium.org/2603553002/diff/100001/third_party/WebKit/Sou... File third_party/WebKit/Source/modules/vr/VRController.cpp (left): https://codereview.chromium.org/2603553002/diff/100001/third_party/WebKit/Sou... third_party/WebKit/Source/modules/vr/VRController.cpp:24: navigatorVR->document()->frame()->interfaceProvider()->getInterface( Brandon, do you know this interface was exposed at the per-frame level rather than the per-process level? I don't see that VRServiceImpl has any per-frame context.
The CQ bit was checked by ke.he@intel.com to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: Try jobs failed on following builders: win_chromium_rel_ng on master.tryserver.chromium.win (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_...)
Patchset #4 (id:120001) has been deleted
The CQ bit was checked by ke.he@intel.com to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: Try jobs failed on following builders: cast_shell_android on master.tryserver.chromium.android (JOB_FAILED, https://build.chromium.org/p/tryserver.chromium.android/builders/cast_shell_a...)
On 2017/01/12 09:59:56, blundell wrote: > https://codereview.chromium.org/2603553002/diff/100001/third_party/WebKit/Sou... > third_party/WebKit/Source/modules/vr/VRController.cpp:24: > navigatorVR->document()->frame()->interfaceProvider()->getInterface( > Brandon, do you know this interface was exposed at the per-frame level rather > than the per-process level? I don't see that VRServiceImpl has any per-frame > context. We should be differentiating these connections at a per-frame level because only one frame at a time is allowed to present to a VRDisplay. There are multiple checks for this in https://cs.chromium.org/chromium/src/device/vr/vr_display_impl.cc (look for "IsAccessAllowed"). If we were handling this connection on a per-process level it may be possible for one frame to preempt another's VR presentation unexpectedly.
The CQ bit was checked by ke.he@intel.com to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: Try jobs failed on following builders: win_chromium_rel_ng on master.tryserver.chromium.win (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_...)
Hi, Colin and Brandon, My understanding is Per-frame level or Per-process level has no difference because it uses the mojo connection. Each frame will create its own connection(by blink::ServiceConnector), we didn't change the "IsAccessAllowed" logic so there is no preempt, right? The real blocker is we broke the layout test after we replaced the original "frame-level mojo connection" by "device-service mojo connection". I found the VR layout test create a "mock-vr-service.js" which implements the VR mojo interface in javascript. The Javascript-mojo is enabled in "interface_registry_js_wrapper.cc", "interface_provider_js_wrapper.cc" and "mojo-helpers.js". The logic is to pass the interface-registry and interface-provider of frame-level or process-level into *js_wrapper, so Javascript code can also use mojo. But now the device-service hides the "interface_registry" and "interface_provider". I cannot find a way to enable the mojo of javascript. Sorry I'm not quite sure my understanding is correct:( One choice is to remove all the VR layouttest html files temporarily, and add them back after javascript mojo is ready to device service. Another choice is to pending this CL till javascript mojo is ready. Could you please give some comments, Thanks.
On 2017/01/18 09:53:45, ke.he wrote:
> Hi, Colin and Brandon,
> My understanding is Per-frame level or Per-process level has no difference
> because it uses the mojo connection. Each frame will create its own
> connection(by blink::ServiceConnector), we didn't change the "IsAccessAllowed"
> logic so there is no preempt, right?
Agreed. Thanks for the info, Brandon! This should still work fine.
>
> The real blocker is we broke the layout test after we replaced the original
> "frame-level mojo connection" by "device-service mojo connection".
>
> I found the VR layout test create a "mock-vr-service.js" which implements the
VR
> mojo interface in javascript. The Javascript-mojo is enabled in
> "interface_registry_js_wrapper.cc", "interface_provider_js_wrapper.cc" and
> "mojo-helpers.js". The logic is to pass the interface-registry and
> interface-provider of frame-level or process-level into *js_wrapper, so
> Javascript code can also use mojo.
>
> But now the device-service hides the "interface_registry" and
> "interface_provider". I cannot find a way to enable the mojo of javascript.
> Sorry I'm not quite sure my understanding is correct:(
>
> One choice is to remove all the VR layouttest html files temporarily, and add
> them back after javascript mojo is ready to device service.
> Another choice is to pending this CL till javascript mojo is ready.
>
> Could you please give some comments, Thanks.
Funny, I was examining this problem yesterday because I hit it with
BatteryMonitor. Your analysis of the problem is correct. I just sent an email to
{chromium-mojo, platform-architecture-dev}@ presenting two approaches for
solving it. Once I determine which approach people prefer, I anticipate landing
the solution in a short period. So I think we could just wait to land this CL
until I solve that issue.
The CQ bit was checked by ke.he@intel.com to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: Try jobs failed on following builders: mac_chromium_rel_ng on master.tryserver.chromium.mac (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel_...)
The CQ bit was checked by ke.he@intel.com to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: Try jobs failed on following builders: linux_chromium_tsan_rel_ng on master.tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
Patchset #10 (id:260001) has been deleted
Patchset #10 (id:280001) has been deleted
The CQ bit was checked by ke.he@intel.com to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: This issue passed the CQ dry run.
Hi, Colin, PTAL, thanks! https://codereview.chromium.org/2603553002/diff/220001/services/device/manife... File services/device/manifest.json (right): https://codereview.chromium.org/2603553002/diff/220001/services/device/manife... services/device/manifest.json:11: "device::mojom::VRDisplay" only VRService is enough. https://codereview.chromium.org/2603553002/diff/220001/third_party/WebKit/Sou... File third_party/WebKit/Source/modules/vr/VRController.cpp (left): https://codereview.chromium.org/2603553002/diff/220001/third_party/WebKit/Sou... third_party/WebKit/Source/modules/vr/VRController.cpp:26: m_service.set_connection_error_handler(convertToBaseCallback( del this by mistake.
The CQ bit was checked by ke.he@intel.com to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
Description was changed from ========== Ports VRService to be hosted in the device service. Ports the VRService to be hosted by te Device Service instead of by //content/browser. BUG= CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_site_isolation ========== to ========== Ports VRService to be hosted in the device service. Ports the VRService to be hosted by te Device Service instead of by //content/browser. BUG=612342 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_site_isolation ==========
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: This issue passed the CQ dry run.
On 2017/04/17 05:38:44, ke.he wrote: > Hi, Colin, PTAL, thanks! > > https://codereview.chromium.org/2603553002/diff/220001/services/device/manife... > File services/device/manifest.json (right): > > https://codereview.chromium.org/2603553002/diff/220001/services/device/manife... > services/device/manifest.json:11: "device::mojom::VRDisplay" > only VRService is enough. > > https://codereview.chromium.org/2603553002/diff/220001/third_party/WebKit/Sou... > File third_party/WebKit/Source/modules/vr/VRController.cpp (left): > > https://codereview.chromium.org/2603553002/diff/220001/third_party/WebKit/Sou... > third_party/WebKit/Source/modules/vr/VRController.cpp:26: > m_service.set_connection_error_handler(convertToBaseCallback( > del this by mistake. Oh, Sorry, please ignore the draft comments. Apologize for my careless :(
What process/thread does the device service run in? The VRService still has to run in the browser UI process currently.
What process/thread does the device service run in? The VRService still has to run in the browser UI process currently.
On 2017/04/17 14:41:45, mthiesse wrote: > What process/thread does the device service run in? The VRService still has to > run in the browser UI process currently. The Device Service run in Browser process, VRService still runs in UI thread after this porting.
Hi, Ke He, thanks for following up here! Michael, I, and others are in the middle of a discussion about servicification of VR, and it looks like there are more wide-ranging design issues here than I had been aware of. Let's hold off on this for now pending some followup design discussions. I apologize for the churn here.
On 2017/04/18 13:53:41, blundell wrote: > Hi, > > Ke He, thanks for following up here! Michael, I, and others are in the middle of > a discussion about servicification of VR, and it looks like there are more > wide-ranging design issues here than I had been aware of. Let's hold off on this > for now pending some followup design discussions. I apologize for the churn > here. There is nothing to apologize for, totally understand:)
Message was sent while issue was closed.
Description was changed from ========== Ports VRService to be hosted in the device service. Ports the VRService to be hosted by te Device Service instead of by //content/browser. BUG=612342 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_site_isolation ========== to ========== Ports VRService to be hosted in the device service. Ports the VRService to be hosted by te Device Service instead of by //content/browser. BUG=612342 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_site_isolation ========== |
