GM self-tests: always run all tests (don't stop at first failure)

GM self-tests: always run all tests (don't stop at first failure)
and make rebaseline.sh rebaseline all tests in one go.
(RunBuilders:Skia_PerCommit_House_Keeping)

Committed: https://code.google.com/p/skia/source/detail?r=8881

[epoger, 2013-04-26 15:59:42]

Ready for review.

[borenet, 2013-04-26 16:04:09]

LGTM with optional suggestions.

https://codereview.chromium.org/13979017/diff/1/gm/tests/run.sh
File gm/tests/run.sh (right):

https://codereview.chromium.org/13979017/diff/1/gm/tests/run.sh#newcode43
gm/tests/run.sh:43: ENCOUNTERED_ANY_ERRORS=1
Alternatively, you could add error messages to this variable:
ERRORS="${ERRORS}\n${NEW_ERROR_MSG}"

And test if it's empty at the end.

https://codereview.chromium.org/13979017/diff/1/gm/tests/run.sh#newcode56
gm/tests/run.sh:56: exit 1
What about here?

https://codereview.chromium.org/13979017/diff/1/gm/tests/run.sh#newcode93
gm/tests/run.sh:93: exit 1
And here?

[epoger, 2013-04-26 16:10:54]

I'd like to leave it as-is (details below).  Is that OK with you?

https://codereview.chromium.org/13979017/diff/1/gm/tests/run.sh
File gm/tests/run.sh (right):

https://codereview.chromium.org/13979017/diff/1/gm/tests/run.sh#newcode43
gm/tests/run.sh:43: ENCOUNTERED_ANY_ERRORS=1
On 2013/04/26 16:04:09, borenet wrote:
> Alternatively, you could add error messages to this variable:
> ERRORS="${ERRORS}\n${NEW_ERROR_MSG}"
> 
> And test if it's empty at the end.

The question is, what would I put into $ERRORS that would be a useful summary?

I suppose I could pipe (or tee) the output of diff to $ERRORS, but I'm not sure
whether that output would make sense without the context provided by the output
generated as it goes.

I think it's best to just tell the human it failed, and trust her to scroll up
and see what happened, for now.  Maybe we can improve that later...

https://codereview.chromium.org/13979017/diff/1/gm/tests/run.sh#newcode56
gm/tests/run.sh:56: exit 1
On 2013/04/26 16:04:09, borenet wrote:
> What about here?

In this case, there was an error within the run.sh script, and I think we should
Stop The Presses.

https://codereview.chromium.org/13979017/diff/1/gm/tests/run.sh#newcode93
gm/tests/run.sh:93: exit 1
On 2013/04/26 16:04:09, borenet wrote:
> And here?

In this case, there was an error within the run.sh script, and I think we should
Stop The Presses.

[borenet, 2013-04-26 16:12:43]

LGTM

https://codereview.chromium.org/13979017/diff/1/gm/tests/run.sh
File gm/tests/run.sh (right):

https://codereview.chromium.org/13979017/diff/1/gm/tests/run.sh#newcode43
gm/tests/run.sh:43: ENCOUNTERED_ANY_ERRORS=1
On 2013/04/26 16:10:55, epoger wrote:
> On 2013/04/26 16:04:09, borenet wrote:
> > Alternatively, you could add error messages to this variable:
> > ERRORS="${ERRORS}\n${NEW_ERROR_MSG}"
> > 
> > And test if it's empty at the end.
> 
> The question is, what would I put into $ERRORS that would be a useful summary?
> 
> I suppose I could pipe (or tee) the output of diff to $ERRORS, but I'm not
sure
> whether that output would make sense without the context provided by the
output
> generated as it goes.
> 
> I think it's best to just tell the human it failed, and trust her to scroll up
> and see what happened, for now.  Maybe we can improve that later...

Fair enough.