Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(312)

Issue 1288043002: Fix ChromeVox next tests that passed without actually running. (Closed)

Created:
5 years, 4 months ago by Peter Lundblad
Modified:
5 years, 4 months ago
Reviewers:
dmazzoni, David Tseng
CC:
chromium-reviews, oshima+watch_chromium.org, plundblad+watch_chromium.org, aboxhall+watch_chromium.org, nektar+watch_chromium.org, yuzo+watch_chromium.org, je_julie, arv+watch_chromium.org, dtseng+watch_chromium.org, dmazzoni+watch_chromium.org
Base URL:
https://chromium.googlesource.com/chromium/src.git@master
Target Ref:
refs/pending/heads/master
Project:
chromium
Visibility:
Public.

Description

Fix 2 ChromeVox next tests that passed without actually running. The way to create callbacks for async functions in these e2e tests is to invoke this.newCallback(). See https://codereview.chromium.org/938623003 for more background. If this is not done, the tests will pass silently before the asyc callbacks get called which means much of the tests don't run. This CL fixes two of the tests and marks the rest with TODOs because they fail and need more debugging when this problem is fixed. The live region tests are fixed by making paragraph nodes output their descendants instead of their value property. BUG=None Committed: https://crrev.com/3d95fe156c4ebc4bec9c476a02761a102a7316b8 Cr-Commit-Position: refs/heads/master@{#343217}

Patch Set 1 #

Patch Set 2 : Leave some tests broken, makring with TODOs. They are actually broken. #

Unified diffs Side-by-side diffs Delta from patch set Stats (+19 lines, -8 lines) Patch
M chrome/browser/resources/chromeos/chromevox/cvox2/background/background_test.extjs View 1 4 chunks +10 lines, -4 lines 0 comments Download
M chrome/browser/resources/chromeos/chromevox/cvox2/background/output.js View 1 1 chunk +1 line, -1 line 0 comments Download
M chrome/browser/resources/chromeos/chromevox/testing/chromevox_next_e2e_test_base.js View 1 1 chunk +8 lines, -3 lines 0 comments Download

Dependent Patchsets:

Messages

Total messages: 8 (2 generated)
Peter Lundblad
I haven't figured out yet if the other tests that fail are caused by actual ...
5 years, 4 months ago (2015-08-13 15:56:52 UTC) #2
dmazzoni
lgtm Thank you! Good timing, I started writing a test this morning and discovered the ...
5 years, 4 months ago (2015-08-13 16:04:25 UTC) #4
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1288043002/20001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1288043002/20001
5 years, 4 months ago (2015-08-13 16:04:59 UTC) #5
commit-bot: I haz the power
Committed patchset #2 (id:20001)
5 years, 4 months ago (2015-08-13 16:35:49 UTC) #6
commit-bot: I haz the power
Patchset 2 (id:??) landed as https://crrev.com/3d95fe156c4ebc4bec9c476a02761a102a7316b8 Cr-Commit-Position: refs/heads/master@{#343217}
5 years, 4 months ago (2015-08-13 16:36:26 UTC) #7
Peter Lundblad
5 years, 4 months ago (2015-08-14 13:37:23 UTC) #8
Message was sent while issue was closed.
dmazzoni@chromium.org writes:
> Any ideas for how to make this more foolproof so it's harder to write (or
> modify) a test in a way that cause it to succeed without actually doing
> anything?

In general, I think that's hard.  What I usually do when writing tests,
especially async ones, is to introduce something that causes them to fail
(often a typo in the expected output) and make sure they fail.
Of course that's manual and error-prone.

For the particular case, I killed testDone in our tests totally.  See
https://codereview.chromium.org/1295773002.
Hopefully it will prevent us from falling back into old habbits when writing
new tests.

Thanks,
//peter

> https://codereview.chromium.org/1288043002/

-- 

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

Powered by Google App Engine
This is Rietveld 408576698