Chromium Code Reviews
|
|
Chromium Code Reviews
Issue
2810813002:
Add VR e2e tests to main waterfall (Closed)
DescriptionAdd VR e2e tests to main waterfall
The swarmed KLM end-to-end VR tests have been looking pretty stable, see
https://build.chromium.org/p/chromium.fyi/builders/Android%20VR%20Tests/builds/6532.
(The failing "chrome_public_test_vr_apk" test is being run on a Pixel w/ N,
so it isn't applicable to this change)
So, it would be helpful to get these tests running on builders with more
visibility than our FYI bot so breaking changes can get reverted quickly.
This CL adds the tests to the following bots:
- Lollipop Phone Tester
- Marshmallow 64 bit Tester
- Android N5X Swarm Builder
So, the tests won't be running on any K devices - I didn't see any builders
that are clearly running tests on K devices except for KitKat Tablet Tester.
BUG=709620
Review-Url: https://codereview.chromium.org/2810813002
Cr-Commit-Position: refs/heads/master@{#464829}
Committed: https://chromium.googlesource.com/chromium/src/+/ab15adbe306f64451bb0debb09f67d54f19d597f
Patch Set 1 #Patch Set 2 : Remove --strict-mode off #Patch Set 3 : Rebase to use 64-bit APK #Messages
Total messages: 27 (17 generated)
The CQ bit was checked by bsheedy@chromium.org to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
bsheedy@chromium.org changed reviewers: + bpastene@chromium.org
+bpastene for testing/buildbot OWNERS
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: Try jobs failed on following builders: android_n5x_swarming_rel on master.tryserver.chromium.android (JOB_FAILED, https://build.chromium.org/p/tryserver.chromium.android/builders/android_n5x_...)
On 2017/04/10 19:27:43, commit-bot: I haz the power wrote: > Dry run: Try jobs failed on following builders: > android_n5x_swarming_rel on master.tryserver.chromium.android (JOB_FAILED, > https://build.chromium.org/p/tryserver.chromium.android/builders/android_n5x_...) Adding it to "Android N5X Swarm Builder" also puts it on the "android_n5x_swarming_rel" CQ bot. Looks like that bot failed your CQ run. Any clue why?
On 2017/04/10 21:10:18, bpastene wrote: > On 2017/04/10 19:27:43, commit-bot: I haz the power wrote: > > Dry run: Try jobs failed on following builders: > > android_n5x_swarming_rel on master.tryserver.chromium.android (JOB_FAILED, > > > https://build.chromium.org/p/tryserver.chromium.android/builders/android_n5x_...) > > Adding it to "Android N5X Swarm Builder" also puts it on the > "android_n5x_swarming_rel" CQ bot. Looks like that bot failed your CQ run. Any > clue why? Getting this in the logcat output: Device(00ffcfd027c7d207) 04-10 19:11:18.270 20314 20314 E VrCoreLibraryLoader: Failed to load native GVR library from VrCore: Device(00ffcfd027c7d207) 04-10 19:11:18.270 20314 20314 E VrCoreLibraryLoader: java.lang.UnsatisfiedLinkError: dlopen failed: "/data/app/com.google.vr.vrcore-1/lib/arm/libvrcore_native.so" is 32-bit instead of 64-bit I haven't had issues with using 32-bit VrCore on 64-bit OSes before, but it seems to be having issues now? Not sure if there's some other issue that needs to be fixed, or if I just need to start including the 64-bit APK alongside the 32-bit one.
The CQ bit was checked by bsheedy@chromium.org to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
On 2017/04/10 21:34:33, bsheedy wrote: > On 2017/04/10 21:10:18, bpastene wrote: > > On 2017/04/10 19:27:43, commit-bot: I haz the power wrote: > > > Dry run: Try jobs failed on following builders: > > > android_n5x_swarming_rel on master.tryserver.chromium.android (JOB_FAILED, > > > > > > https://build.chromium.org/p/tryserver.chromium.android/builders/android_n5x_...) > > > > Adding it to "Android N5X Swarm Builder" also puts it on the > > "android_n5x_swarming_rel" CQ bot. Looks like that bot failed your CQ run. Any > > clue why? > > Getting this in the logcat output: > Device(00ffcfd027c7d207) 04-10 19:11:18.270 20314 20314 E VrCoreLibraryLoader: > Failed to load native GVR library from VrCore: > Device(00ffcfd027c7d207) 04-10 19:11:18.270 20314 20314 E VrCoreLibraryLoader: > java.lang.UnsatisfiedLinkError: dlopen failed: > "/data/app/com.google.vr.vrcore-1/lib/arm/libvrcore_native.so" is 32-bit instead > of 64-bit > > I haven't had issues with using 32-bit VrCore on 64-bit OSes before, but it > seems to be having issues now? Not sure if there's some other issue that needs > to be fixed, or if I just need to start including the 64-bit APK alongside the > 32-bit one. I switched the VrCore APK to be 64-bit , but it's still having issues. Looking at the gclient runhooks output, it looks like it isn't downloading the current APK because the DOWNLOAD_VR_TEST_APKS environment variable isn't set even though I submitted a change to set that on Android arm Builder (dbg) and Android arm64 Builder (dbg), which should cover the three non-trybots this would run on. My guess is that I have to set it on Android N5X Swarm Builder as well since android_n5x_swarming_rel/dbg mirrors that, but compiles/isolates the tests itself instead of having another builder do it. Does that seem correct to you?
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: Try jobs failed on following builders: android_n5x_swarming_rel on master.tryserver.chromium.android (JOB_FAILED, https://build.chromium.org/p/tryserver.chromium.android/builders/android_n5x_...)
The CQ bit was checked by bsheedy@chromium.org to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: This issue passed the CQ dry run.
Setting the environment variable on the bot did fix the issue (https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...). However, I'm less sure of the stability of the tests now, as https://bugs.chromium.org/p/chromium/issues/detail?id=710534 and https://bugs.chromium.org/p/chromium/issues/detail?id=710624 are causing failures about once a day. The latter is less of a problem since it only seems to happen on K and we won't be adding tests on K in this patch, but I'd like to fix the former if possible so that we don't add even more flaky tests to the main waterfall. If only it would actually reproduce locally.
On 2017/04/11 21:44:39, bsheedy wrote: > Setting the environment variable on the bot did fix the issue > (https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...). > > However, I'm less sure of the stability of the tests now, as > https://bugs.chromium.org/p/chromium/issues/detail?id=710534 and > https://bugs.chromium.org/p/chromium/issues/detail?id=710624 are causing > failures about once a day. > > The latter is less of a problem since it only seems to happen on K and we won't > be adding tests on K in this patch, > but I'd like to fix the former if possible so that we don't add even more flaky > tests to the main waterfall. > > If only it would actually reproduce locally. Resolving the flakiness sgtm. You can also see that test failing intermittently on the flakiness dashboard: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType...
On 2017/04/12 01:31:24, bpastene wrote: > On 2017/04/11 21:44:39, bsheedy wrote: > > Setting the environment variable on the bot did fix the issue > > > (https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...). > > > > However, I'm less sure of the stability of the tests now, as > > https://bugs.chromium.org/p/chromium/issues/detail?id=710534 and > > https://bugs.chromium.org/p/chromium/issues/detail?id=710624 are causing > > failures about once a day. > > > > The latter is less of a problem since it only seems to happen on K and we > won't > > be adding tests on K in this patch, > > but I'd like to fix the former if possible so that we don't add even more > flaky > > tests to the main waterfall. > > > > If only it would actually reproduce locally. > > Resolving the flakiness sgtm. You can also see that test failing intermittently > on the flakiness dashboard: > https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType... The KitKat issue has been fixed, and the Lollipop issue hasn't shown up in a few days. I've been unable to reproduce it locally or on the swarming bots. Looking at the logcat logs of failed tests, it looks like it may have been some device issue. The log was getting spammed with: <rb_adjust_internal_multisample_count:3892>: Reducing MSAA sample count from 4 to 2 and was also getting thermal warnings: ThermalEngine: TM Id 'SKIN_THERMAL_management_1' Sensor 'xo_therm_pu2' - alarm raised 1 at 40.0 degC The reason the test was failing was was that for whatever reason, the InputManager was unable to send the screen taps, and so the test was timing out waiting for the input to be registered: InputManager: Input event injection from pid 31521 timed out. Committing this now would probably be pretty safe, or we could wait until next week to make sure the issue doesn't pop up again over the weekend.
On 2017/04/14 16:46:15, bsheedy wrote: > Committing this now would probably be pretty safe, or we could wait until next > week to make sure the issue doesn't pop up again over the weekend. lgtm; thanks for being diligent I'm fine with committing whenever, up to you.
The CQ bit was checked by bsheedy@chromium.org
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
CQ is committing da patch.
Bot data: {"patchset_id": 40001, "attempt_start_ts": 1492205569378510,
"parent_rev": "755670d02cf3c0bfbfdedcaaf7a34d67accaea93", "commit_rev":
"bfab237d5a80b5094fd0038a615f7c6007b5794a"}
CQ is committing da patch.
Bot data: {"patchset_id": 40001, "attempt_start_ts": 1492205569378510,
"parent_rev": "c878a9ffb8f179cd7ab5f8e054d96e399d28d514", "commit_rev":
"ab15adbe306f64451bb0debb09f67d54f19d597f"}
Message was sent while issue was closed.
Description was changed from ========== Add VR e2e tests to main waterfall The swarmed KLM end-to-end VR tests have been looking pretty stable, see https://build.chromium.org/p/chromium.fyi/builders/Android%20VR%20Tests/build.... (The failing "chrome_public_test_vr_apk" test is being run on a Pixel w/ N, so it isn't applicable to this change) So, it would be helpful to get these tests running on builders with more visibility than our FYI bot so breaking changes can get reverted quickly. This CL adds the tests to the following bots: - Lollipop Phone Tester - Marshmallow 64 bit Tester - Android N5X Swarm Builder So, the tests won't be running on any K devices - I didn't see any builders that are clearly running tests on K devices except for KitKat Tablet Tester. BUG=709620 ========== to ========== Add VR e2e tests to main waterfall The swarmed KLM end-to-end VR tests have been looking pretty stable, see https://build.chromium.org/p/chromium.fyi/builders/Android%20VR%20Tests/build.... (The failing "chrome_public_test_vr_apk" test is being run on a Pixel w/ N, so it isn't applicable to this change) So, it would be helpful to get these tests running on builders with more visibility than our FYI bot so breaking changes can get reverted quickly. This CL adds the tests to the following bots: - Lollipop Phone Tester - Marshmallow 64 bit Tester - Android N5X Swarm Builder So, the tests won't be running on any K devices - I didn't see any builders that are clearly running tests on K devices except for KitKat Tablet Tester. BUG=709620 Review-Url: https://codereview.chromium.org/2810813002 Cr-Commit-Position: refs/heads/master@{#464829} Committed: https://chromium.googlesource.com/chromium/src/+/ab15adbe306f64451bb0debb09f6... ==========
Message was sent while issue was closed.
Committed patchset #3 (id:40001) as https://chromium.googlesource.com/chromium/src/+/ab15adbe306f64451bb0debb09f6... |
