GM: run mode comparisons for all raster configs.

GM: run mode comparisons for all raster configs.

GM works like this today:
  - Test all enabled configs directly
  - Render into 8888
  - Test all enabled modes against that 8888.

This change makes it do this:
  - Test all enabled configs directly
    - for each config, if it's raster, test all enabled modes against its output.

Upshot is, we now check that picture draws identically to raster for 565 too, not just 8888.  optimizations-565 is correctly failing.

BUG=skia:1994
Committed: http://code.google.com/p/skia/source/detail?r=13050

[mtklein, 2014-01-08 20:05:53]

https://codereview.chromium.org/129203002/diff/1/gm/gmmain.cpp
File gm/gmmain.cpp (right):

https://codereview.chromium.org/129203002/diff/1/gm/gmmain.cpp#newcode1777
gm/gmmain.cpp:1777: // TODO: run only if gmmain.test_drawing succeeded.
The diff is weird, but what's happened here is:
  - run_multiple_modes moves up the file verbatim to be visible to
run_multiple_configs
  - Only these next three lines (plus passing tileGridReplayScales) are new in
run_multiple_configs.  Nothing else changed.

[mtklein, 2014-01-08 20:09:12]

On 2014/01/08 20:05:53, mtklein wrote:
> https://codereview.chromium.org/129203002/diff/1/gm/gmmain.cpp
> File gm/gmmain.cpp (right):
> 
> https://codereview.chromium.org/129203002/diff/1/gm/gmmain.cpp#newcode1777
> gm/gmmain.cpp:1777: // TODO: run only if gmmain.test_drawing succeeded.
> The diff is weird, but what's happened here is:
>   - run_multiple_modes moves up the file verbatim to be visible to
> run_multiple_configs
>   - Only these next three lines (plus passing tileGridReplayScales) are new in
> run_multiple_configs.  Nothing else changed.

Ran out/Release/gm --replay --rtree --serialize --pipe: only optimizations is
failing, as expected.

GM: Ran 4956 tests: NoGpuContext=0 IntentionallySkipped=928 RenderModeMismatch=3
GeneratePdfFailed=0 ExpectationsMismatch=0 MissingExpectations=1096
WritingReferenceImage=0
GM: [*] 0 NoGpuContext:
GM: [ ] 928 IntentionallySkipped
GM: [*] 3 RenderModeMismatch: optimizations_565-replay
optimizations_565-serialize optimizations_565-rtree
GM: [*] 0 GeneratePdfFailed:
GM: [*] 0 ExpectationsMismatch:
GM: [ ] 1096 MissingExpectations
GM: [*] 0 WritingReferenceImage:
GM: (results marked with [*] will cause nonzero return value)
GM: expected 3348 tests, but ran or skipped 4956 tests

[epoger, 2014-01-08 20:13:18]

looking now

[epoger, 2014-01-08 20:48:54]

I like how you made the change.  Questions/concerns below...

https://codereview.chromium.org/129203002/diff/1/gm/gmmain.cpp
File gm/gmmain.cpp (right):

https://codereview.chromium.org/129203002/diff/1/gm/gmmain.cpp#newcode1777
gm/gmmain.cpp:1777: // TODO: run only if gmmain.test_drawing succeeded.
On 2014/01/08 20:05:53, mtklein wrote:
> The diff is weird, but what's happened here is:
>   - run_multiple_modes moves up the file verbatim to be visible to
> run_multiple_configs
>   - Only these next three lines (plus passing tileGridReplayScales) are new in
> run_multiple_configs.  Nothing else changed.

Thanks for the explanation.

One way of avoiding the diff weirdness (at the expense of some time/effort) is
to upload the move as patchset 1, and the modifications as patchset 2.

https://codereview.chromium.org/129203002/diff/1/gm/gmmain.cpp#newcode2320
gm/gmmain.cpp:2320: int expectedNumberOfTests = gmsRun * (configs.count() +
modes.count());
expectedNumberOfTests should now be calculated differently; if not, the
GenerateGMs step will start failing with a message like the one you saw in your
test run:

GM: expected 3348 tests, but ran or skipped 4956 tests

I think the number of expected tests is now something like:
gmsRun * (numRasterConfigs * modes.count()) +
gmsRun * numNonRasterConfigs

... where numRasterConfigs and numNonRasterConfigs are not-yet-existing
variables, such that
numRasterConfigs + numNonRasterConfigs = configs.count()

(I guess numRasterConfigs is 2, for 8888 and 565)

https://codereview.chromium.org/129203002/diff/1/gm/gmmain.cpp#newcode2322
gm/gmmain.cpp:2322: // Output summary to stdout.
As expectedNumberOfTests indicates, this change will cause GM to take
significantly longer.  Do we feel that the benefits outweigh the costs?  (I'm
not sure how to make that determination; it probably has a lot to do with how
important we think 565 is)

https://codereview.chromium.org/129203002/diff/1/gm/gmmain.cpp#newcode2324
gm/gmmain.cpp:2324: gm_fprintf(stdout, "Ran %d GMs\n", gmsRun);
This change will modify the gm self-test results.  mtklein, are you developing
this change on a Linux machine?  If so, you should be able to run
gm/tests/run.sh (before and after your change) to see for yourself, and
gm/tests/rebaseline.sh to correct the self-test results within this same CL.

If you have trouble with that, or are NOT developing this CL on a Linux machine,
let's coordinate once this CL is otherwise ready-to-go and I will perform the gm
self-test rebaseline for you.

[reed1, 2014-01-08 20:49:43]

defers to epoger

[epoger, 2014-01-08 20:51:06]

On 2014/01/08 20:49:43, reed1 wrote:
> defers to epoger

Nice try.  Defers to reed on the benefits-vs-costs question I raised above. :-)

[reed1, 2014-01-08 20:53:51]

I think we either drop 565 from our tests, or we need this sort of fix, so we
can catch bugs, thus I'm ok with the extra time. lgtm

[epoger, 2014-01-08 20:55:53]

On 2014/01/08 20:53:51, reed1 wrote:
> I think we either drop 565 from our tests, or we need this sort of fix, so we
> can catch bugs, thus I'm ok with the extra time. lgtm

Cool, thanks for the clear direction.

In that case, mtklein, please address the remaining issues
(expectedNumberOfTests calculation, self-test rebaseline) before committing.

As I wrote before, I'll be happy to help with any of it.  Until I leave to pick
up my kid at 4:30. :-)  I'll be in the office tomorrow.

[mtklein, 2014-01-08 21:07:12]

On 2014/01/08 20:53:51, reed1 wrote:
> I think we either drop 565 from our tests, or we need this sort of fix, so we
> can catch bugs, thus I'm ok with the extra time. lgtm

Right, I'd rather test 565 fully or not at all.

[mtklein, 2014-01-08 21:53:39]

On 2014/01/08 21:07:12, mtklein wrote:
> On 2014/01/08 20:53:51, reed1 wrote:
> > I think we either drop 565 from our tests, or we need this sort of fix, so
we
> > can catch bugs, thus I'm ok with the extra time. lgtm
> 
> Right, I'd rather test 565 fully or not at all.

I feel a little silly, but I can't see how my new expectedNumberOfTests math is
wrong.  Can I get you to take a look Elliot?  The self test and full GM run
aren't adding up: 3 vs. 4 and 4464 vs 4398.  For short runs (gm --replay --rtree
--match xfer) it works correctly for me.

Does the selftest rebaseline otherwise look sensible?

[epoger, 2014-01-09 16:40:36]

Uploaded patchset 3, after running gm/tests/rebaseline.sh on my Ubiquity
instance.

I'm not sure why expectedNumberOfTests matches when I run it but not when you
run it.  Perhaps it has to do with the fact that my Linux machine is "headless"
(no GPU), and you're running on a machine with GPU?

I will pick a couple of trybots and run this patchset thru them.

https://codereview.chromium.org/129203002/diff/190001/gm/tests/outputs/pipe-p...
File gm/tests/outputs/pipe-playback-failure/output-expected/stderr (right):

https://codereview.chromium.org/129203002/diff/190001/gm/tests/outputs/pipe-p...
gm/tests/outputs/pipe-playback-failure/output-expected/stderr:1: GM: ----
8888/selftest1-pipe.png: 60000 (of 60000) differing pixels, max per-channel
mismatch R=135 G=246 B=135 A=0
We don't see the "FAILED to write" message anymore because the path is now
8888/blah instead of comparison/blah , and there is an "8888" dir available but
not a "comparison" dir available.

[mtklein, 2014-01-09 16:56:13]

On 2014/01/09 16:40:36, epoger wrote:
> Uploaded patchset 3, after running gm/tests/rebaseline.sh on my Ubiquity
> instance.
> 
> I'm not sure why expectedNumberOfTests matches when I run it but not when you
> run it.  Perhaps it has to do with the fact that my Linux machine is
"headless"
> (no GPU), and you're running on a machine with GPU?
> 
> I will pick a couple of trybots and run this patchset thru them.
> 
>
https://codereview.chromium.org/129203002/diff/190001/gm/tests/outputs/pipe-p...
> File gm/tests/outputs/pipe-playback-failure/output-expected/stderr (right):
> 
>
https://codereview.chromium.org/129203002/diff/190001/gm/tests/outputs/pipe-p...
> gm/tests/outputs/pipe-playback-failure/output-expected/stderr:1: GM: ----
> 8888/selftest1-pipe.png: 60000 (of 60000) differing pixels, max per-channel
> mismatch R=135 G=246 B=135 A=0
> We don't see the "FAILED to write" message anymore because the path is now
> 8888/blah instead of comparison/blah , and there is an "8888" dir available
but
> not a "comparison" dir available.

Thanks for working at this.

[epoger, 2014-01-09 18:20:09]

As of patchset 4, RunGmSelfTests passes on
http://108.170.217.252:10117/builders/Housekeeper-PerCommit-Trybot/builds/17

Still waiting for
http://108.170.217.252:10117/builders/Test-Ubuntu12-ShuttleA-ATI5770-x86-Debu...
(tried at patchset 3, which should also vouch for patchset 4) to see if
GenerateGMs step will pass on a real buildbot machine.

[epoger, 2014-01-09 20:23:09]

On 2014/01/09 18:20:09, epoger wrote:
> As of patchset 4, RunGmSelfTests passes on
> http://108.170.217.252:10117/builders/Housekeeper-PerCommit-Trybot/builds/17
> 
> Still waiting for
>
http://108.170.217.252:10117/builders/Test-Ubuntu12-ShuttleA-ATI5770-x86-Debu...
> (tried at patchset 3, which should also vouch for patchset 4) to see if
> GenerateGMs step will pass on a real buildbot machine.

Rats, the expectedNumberOfTests didn't match when it ran on
http://108.170.217.252:10117/builders/Test-Ubuntu12-ShuttleA-ATI5770-x86-Debu...

Here's the command it ran:

out/Debug/gm --verbose --writeChecksumBasedFilenames --mismatchPath
../../gm/actual/Test-Ubuntu12-ShuttleA-ATI5770-x86-Debug-Trybot
--missingExpectationsPath
../../gm/actual/Test-Ubuntu12-ShuttleA-ATI5770-x86-Debug-Trybot
--writeJsonSummaryPath
../../gm/actual/Test-Ubuntu12-ShuttleA-ATI5770-x86-Debug-Trybot/actual-results.json
--ignoreErrorTypes IntentionallySkipped MissingExpectations ExpectationsMismatch
--resourcePath resources --serialize --readPath
../../gm/expected/expected-results.json --ignoreFailuresFile
../../gm/expected/ignored-tests.txt --deferred --pipe --replay --rtree
--serialize --tileGrid --config defaults msaa16

And here's an excerpt of the output:

[11:58:14.236096] GM: Ran 279 GMs
[11:58:14.236118] GM: ... over  6 configs ["8888", "565", "gpu", "gpudebug",
"gpunull", "pdf"]
[11:58:14.236141] GM: ...  and  7 modes   ["pipe", "pipe cross-process", "pipe
cross-process, shared address", "replay", "rtree", "serialize", "tilegrid"]
[11:58:14.236171] GM: ... so there should be a total of 5580 tests.
[11:58:14.236194] GM: Ran 5496 tests: NoGpuContext=0 IntentionallySkipped=945
RenderModeMismatch=0 GeneratePdfFailed=0 ExpectationsMismatch=9
MissingExpectations=46 WritingReferenceImage=0
[11:58:14.236219] GM: [*] 0 NoGpuContext:
[11:58:14.236243] GM: [ ] 945 IntentionallySkipped: xfermodes3_gpudebug […]
aaclip_gpunull
[11:58:14.737413] GM: expected 5580 tests, but ran or skipped 5496 tests
[11:58:14.737538] GM: [*] 0 RenderModeMismatch:
[11:58:14.737595] GM: [*] 0 GeneratePdfFailed:
[11:58:14.737667] GM: [ ] 9 ExpectationsMismatch: rects_8888 […]
[11:58:14.737724] GM: [ ] 46 MissingExpectations: vertices_8888 […] bigblurs_pdf
[11:58:14.737786] GM: [*] 0 WritingReferenceImage:
[11:58:14.737833] GM: (results marked with [*] will cause nonzero return value)

I will continue to investigate.

[epoger, 2014-01-09 22:12:35]

When I patched this CL onto my Mac laptop, I was able to reproduce the error:

epoger-macbookpro2:skia epoger$ out/Debug/gm --verbose
--writeChecksumBasedFilenames --ignoreErrorTypes IntentionallySkipped
MissingExpectations ExpectationsMismatch --resourcePath resources --serialize
--deferred --pipe --replay --rtree --serialize --tileGrid --config defaults
msaa16 

Non-default runtime configuration options:
GM: Sample count (16) of config msaa16 is not supported. Config will be skipped.
GM: These configs will be run: 8888 565 gpu gpudebug gpunull pdf
GM: These PDF rasterizers will be run: mac poppler
GM: reading resources from resources
GM: drawing... xfermodes3 [630 1215]
GM: drawing... xfermodes2 [455 475]
GM: drawing... xfermodes [1990 640]
GM: drawing... xfermodeimagefilter [600 600]
[…]
GM: drawing... arcofzorro [1000 1000]
GM: drawing... alphagradients [640 480]
GM: drawing... aarectmodes [640 480]
GM: drawing... aaclip [640 480]
GM: Ran 279 GMs
GM: ... over  6 configs ["8888", "565", "gpu", "gpudebug", "gpunull", "pdf"]
GM: ...  and  7 modes   ["pipe", "pipe cross-process", "pipe cross-process,
shared address", "replay", "rtree", "serialize", "tilegrid"]
GM: ... so there should be a total of 5580 tests.
GM: Ran 5496 tests: NoGpuContext=0 IntentionallySkipped=945 RenderModeMismatch=0
GeneratePdfFailed=0 ExpectationsMismatch=0 MissingExpectations=1095
WritingReferenceImage=0
GM: [*] 0 NoGpuContext:
GM: [ ] 945 IntentionallySkipped: xfermodes3_gpudebug xfermodes3_gpunull
xfermodes2_gpudebug […] aaclip_565-pipe aaclip_565-pipe cross-process
aaclip_565-pipe cross-process, shared address aaclip_gpudebug aaclip_gpunull
GM: [*] 0 RenderModeMismatch:
GM: [*] 0 GeneratePdfFailed:
GM: [ ] 0 ExpectationsMismatch:
GM: [ ] 1095 MissingExpectations: xfermodes3_8888 xfermodes3_565 xfermodes3_gpu
xfermodes3_pdf xfermodes2_8888 xfermodes2_565 […] aarectmodes_pdf aaclip_8888
aaclip_565 aaclip_gpu aaclip_pdf
GM: [*] 0 WritingReferenceImage:
GM: (results marked with [*] will cause nonzero return value)
GM: expected 5580 tests, but ran or skipped 5496 tests

[epoger, 2014-01-09 22:15:51]

On 2014/01/09 22:12:35, epoger wrote:
> GM: expected 5580 tests, but ran or skipped 5496 tests

Note that this difference is 84 tests
84 = 6 * 7 * 2 (where 6 is the number of configs, and 7 is the number of modes)

Perhaps there are 2 GMs for which we skipped over all configs/modes, without
accounting for the skips.

[epoger, 2014-01-09 22:27:56]

I was able to narrow it down:

epoger-macbookpro2:skia epoger$ out/Debug/gm --verbose
--writeChecksumBasedFilenames --ignoreErrorTypes IntentionallySkipped
MissingExpectations ExpectationsMismatch --resourcePath resources --serialize
--deferred --pipe --replay --rtree --serialize --tileGrid --config defaults
msaa16 --match conic

Non-default runtime configuration options:
GM: Sample count (16) of config msaa16 is not supported. Config will be skipped.
GM: These configs will be run: 8888 565 gpu gpudebug gpunull pdf
GM: These PDF rasterizers will be run: mac poppler
GM: reading resources from resources
GM: drawing... twopointconical [480 780]
GM: drawing... shallow_gradient_conical [800 800]
GM: drawing... bezier_conic_effects [800 800]
GM: Ran 3 GMs
GM: ... over  6 configs ["8888", "565", "gpu", "gpudebug", "gpunull", "pdf"]
GM: ...  and  7 modes   ["pipe", "pipe cross-process", "pipe cross-process,
shared address", "replay", "rtree", "serialize", "tilegrid"]
GM: ... so there should be a total of 60 tests.
GM: Ran 46 tests: NoGpuContext=0 IntentionallySkipped=9 RenderModeMismatch=0
GeneratePdfFailed=0 ExpectationsMismatch=0 MissingExpectations=9
WritingReferenceImage=0
GM: [*] 0 NoGpuContext:
GM: [ ] 9 IntentionallySkipped: twopointconical_gpudebug twopointconical_gpunull
shallow_gradient_conical_gpudebug shallow_gradient_conical_gpunull
bezier_conic_effects_8888 bezier_conic_effects_565 bezier_conic_effects_gpudebug
bezier_conic_effects_gpunull bezier_conic_effects_pdf
GM: [*] 0 RenderModeMismatch:
GM: [*] 0 GeneratePdfFailed:
GM: [ ] 0 ExpectationsMismatch:
GM: [ ] 9 MissingExpectations: twopointconical_8888 twopointconical_565
twopointconical_gpu twopointconical_pdf shallow_gradient_conical_8888
shallow_gradient_conical_565 shallow_gradient_conical_gpu
shallow_gradient_conical_pdf bezier_conic_effects_gpu
GM: [*] 0 WritingReferenceImage:
GM: (results marked with [*] will cause nonzero return value)
GM: expected 60 tests, but ran or skipped 46 tests


If I run with "--match twopointconical" or "--match shallow_gradient_conical",
it runs 20 tests (and reports that it would expect to run 20 tests).

If I run with "--match bezier_conic_effects", it runs 6 tests (and reports that
it would expect to run 6 tests).

But when running all 3, it expects a total of 60 instead of 46!

[epoger, 2014-01-09 22:32:23]

> If I run with "--match twopointconical" or "--match shallow_gradient_conical",
> it runs 20 tests (and reports that it would expect to run 20 tests).
> 
> If I run with "--match bezier_conic_effects", it runs 6 tests (and reports
that
> it would expect to run 6 tests).
> 
> But when running all 3, it expects a total of 60 instead of 46!

bezier_conic_effects sets the kGPUOnly_Flag.

I think we get the wrong total when we run a combination of tests with and
without the kGPUOnly_Flag set.

[epoger, 2014-01-10 19:36:38]

https://codereview.chromium.org/129203002/diff/440001/gm/gmmain.cpp
File gm/gmmain.cpp (right):

https://codereview.chromium.org/129203002/diff/440001/gm/gmmain.cpp#newcode1167
gm/gmmain.cpp:1167: RecordSkippedTest(shortNamePlusConfig,
Patchset 5: created a new RecordSkippedTest() method, to record that we skipped
a test for a single renderMode.

In my next patchset, I will add a variant that records skipping a test for ALL
renderModes.

To be continued...

[epoger, 2014-01-10 20:53:26]

Patchset 6 fixes the count mismatch.

I have to make one more patchset to rebaseline GM self-test results, so hold on
for that.

https://codereview.chromium.org/129203002/diff/490001/gm/gmmain.cpp
File gm/gmmain.cpp (right):

https://codereview.chromium.org/129203002/diff/490001/gm/gmmain.cpp#newcode321
gm/gmmain.cpp:321: }
There's one renderMode whose name doesn't start with a "-"; that's the default
renderMode.  Let's start reporting it along with the other renderModes, for
completeness' sake.

Besides, when we call RecordTestResults() for all of the
fTestsSkippedOnAllRenderModes entries, we need to report that we skipped the
default renderMode along with the others.  Rather than handle that as a special
case, I think it's better to include the default renderMode in the list.

[epoger, 2014-01-10 21:04:20]

mtklein, this should finally be finished at patchset 7.  PTAL.

I will run it through a couple of trybots, but in the meanwhile, here's what I
recommend you look at:

1. the code change from patchset 4 to patchset 7, which is just one file:
https://codereview.chromium.org/129203002/diff2/290001:540001/gm/gmmain.cpp?c...

2. just one of the modifications to the self-test results, to get a taste:
https://codereview.chromium.org/129203002/diff2/290001:540001/gm/tests/output...

[mtklein, 2014-01-10 22:11:24]

On 2014/01/10 21:04:20, epoger wrote:
> mtklein, this should finally be finished at patchset 7.  PTAL.
> 
> I will run it through a couple of trybots, but in the meanwhile, here's what I
> recommend you look at:
> 
> 1. the code change from patchset 4 to patchset 7, which is just one file:
>
https://codereview.chromium.org/129203002/diff2/290001:540001/gm/gmmain.cpp?c...
> 
> 2. just one of the modifications to the self-test results, to get a taste:
>
https://codereview.chromium.org/129203002/diff2/290001:540001/gm/tests/output...

lgtm, with just one spot in the code where I stumbled over types a bit.

[mtklein, 2014-01-10 22:11:42]

oops, forgot to send the stumbling point :)

https://codereview.chromium.org/129203002/diff/540001/gm/gmmain.cpp
File gm/gmmain.cpp (right):

https://codereview.chromium.org/129203002/diff/540001/gm/gmmain.cpp#newcode2335
gm/gmmain.cpp:2335: SkString *shortNamePlusConfig =
&gmmain.fTestsSkippedOnAllRenderModes[testNum];
Why not const SkString& shortNamePlusConfig here, then pass it in later with no
*?

[epoger, 2014-01-12 21:13:23]

https://codereview.chromium.org/129203002/diff/540001/gm/gmmain.cpp
File gm/gmmain.cpp (right):

https://codereview.chromium.org/129203002/diff/540001/gm/gmmain.cpp#newcode2335
gm/gmmain.cpp:2335: SkString *shortNamePlusConfig =
&gmmain.fTestsSkippedOnAllRenderModes[testNum];
On 2014/01/10 22:11:43, mtklein wrote:
> Why not const SkString& shortNamePlusConfig here, then pass it in later with
no
> *?

Good idea, thanks!

[epoger, 2014-01-13 18:07:23]

verbal LGTM from mtklein

[commit-bot: I haz the power, 2014-01-13 18:07:42]

CQ is trying da patch. Follow status at
https://skia-tree-status.appspot.com/cq/mtklein@google.com/129203002/620001

[commit-bot: I haz the power, 2014-01-13 18:28:09]

Change committed as 13050

[epoger, 2014-01-13 21:08:53]

Followup on performance difference after this change went in.  It’s definitely
slower, but the build times are noisy enough that it’s hard to say how MUCH
slower.


Android-NexusS-SGX540-Arm7-Debug : I don’t know what to make of this one
  before:
http://108.170.217.252:10117/builders/Test-Android-NexusS-SGX540-Arm7-Debug/b...
          kept reporting Out of Memory, took 1433 sec
 after:
http://108.170.217.252:10117/builders/Test-Android-NexusS-SGX540-Arm7-Debug/b...
        kept reporting Out of Memory, took 1611 sec

ChromeOS-Link-HD4000-x86_64-Debug : got 13% FASTER (?!)
  before:
http://108.170.217.252:10117/builders/Test-ChromeOS-Link-HD4000-x86_64-Debug/...
          ran 1092 tests in 171 sec
  after:
http://108.170.217.252:10117/builders/Test-ChromeOS-Link-HD4000-x86_64-Debug/...
         ran 1365 tests in 149 sec

Mac10.6-MacMini4.1-GeForce320M-x86_64-Debug : got 30% slower
  before:
http://108.170.217.252:10117/builders/Test-Mac10.6-MacMini4.1-GeForce320M-x86...
          ran 3601 tests in 765 sec
  after:
http://108.170.217.252:10117/builders/Test-Mac10.6-MacMini4.1-GeForce320M-x86...
         ran 5540 tests in 988 sec

Ubuntu12-ShuttleA-ATI5770-x86_64-Debug : got 31% slower
  before:
http://108.170.217.252:10117/builders/Test-Ubuntu12-ShuttleA-ATI5770-x86_64-D...
          ran 3601 tests in 413 sec
  after:
http://108.170.217.252:10117/builders/Test-Ubuntu12-ShuttleA-ATI5770-x86_64-D...
         ran 5540 tests in 542 sec

Win7-ShuttleA-HD2000-x86_64-Debug : got 8% slower
  before:
http://108.170.217.252:10117/builders/Test-Win7-ShuttleA-HD2000-x86_64-Debug/...
          ran 1946 tests in 213 sec
  after:
http://108.170.217.252:10117/builders/Test-Win7-ShuttleA-HD2000-x86_64-Debug/...
         ran 2224 tests in 231 sec