run_cmd: Fix CommandResults for results from multiple sources

run_cmd: Fix CommandResults for results from multiple sources

BUG=skia:1329
R=epoger@google.com

Committed: https://skia.googlesource.com/buildbot/+/a7a1f87

[borenet, 2014-03-14 18:53:51]

As of patch set 6, we can run arbitrary commands on every slave on every remote
host, using a single command.

[epoger, 2014-03-14 19:45:46]

Looks decent, glad to hear it's working!

A couple of requests about exception handling is all I've got.

https://codereview.chromium.org/196423010/diff/90001/scripts/run_cmd.py
File scripts/run_cmd.py (right):

https://codereview.chromium.org/196423010/diff/90001/scripts/run_cmd.py#newco...
scripts/run_cmd.py:55: pass
Please add comments describing why we try SingleCommandResults(), then
MultiCommandResults if there is *any* exception.  Seems dangerous/fragile, I'd
like to know more.

If nothing else, please log the exception that was seen here, in case it's
something unusual.

https://codereview.chromium.org/196423010/diff/90001/scripts/run_cmd.py#newco...
scripts/run_cmd.py:62: pass
You're completely eating the underlying exception here!  How are we gonna be
able to debug problems?

Maybe, instead:

try:
  ...
except Exception:
  traceback.print_exc()
  raise Exception('blah')

I used this recently in
https://github.com/google/skia-buildbot/blob/master/compute_engine_scripts/te...

[borenet, 2014-03-14 20:19:40]

https://codereview.chromium.org/196423010/diff/90001/scripts/run_cmd.py
File scripts/run_cmd.py (right):

https://codereview.chromium.org/196423010/diff/90001/scripts/run_cmd.py#newco...
scripts/run_cmd.py:55: pass
On 2014/03/14 19:45:46, epoger wrote:
> Please add comments describing why we try SingleCommandResults(), then
> MultiCommandResults if there is *any* exception.  Seems dangerous/fragile, I'd
> like to know more.
> 
> If nothing else, please log the exception that was seen here, in case it's
> something unusual.

I added some comments.  I'd really rather not log anything, since this failure
is expected a lot of the time.

https://codereview.chromium.org/196423010/diff/90001/scripts/run_cmd.py#newco...
scripts/run_cmd.py:62: pass
On 2014/03/14 19:45:46, epoger wrote:
> You're completely eating the underlying exception here!  How are we gonna be
> able to debug problems?
> 
> Maybe, instead:
> 
> try:
>   ...
> except Exception:
>   traceback.print_exc()
>   raise Exception('blah')
> 
> I used this recently in
>
https://github.com/google/skia-buildbot/blob/master/compute_engine_scripts/te...

The dict which failed to decode is printed out in the exception below.  I added
code to collect the stack traces and add them to the message as well.  Since the
output of the script is supposed to be machine readable, I really want to avoid
printing anything that isn't absolutely necessary.

[epoger, 2014-03-14 20:25:17]

lgtm

[borenet, 2014-03-14 20:36:31]

The CQ bit was checked by borenet@google.com

[commit-bot: I haz the power, 2014-03-14 20:36:39]

CQ is trying da patch. Follow status at
https://skia-tree-status.appspot.com/cq/borenet@google.com/196423010/110001

[borenet, 2014-03-14 20:38:22]

Committed patchset #7 manually as ra7a1f87 (presubmit successful).