Report all swarming and isolate fatal errors in a consistent way.

Report all swarming and isolate fatal errors in a consistent way.

It makes them easier to extract with regular expressions in log parsing tools.

BUG=309661
Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=231704

[Vadim Sh., 2013-10-29 22:15:35]

I have mixed feelings about this CL... 

infra-monitor app uses regexps to figure out bad conditions it alerts on. I
thought instead of nailing errors one by one, lets wrap them all into a common
frame that we can easily detect via regexp. Also that way various retries (that
eventually succeed) won't trigger alerts -> they are not fatal, and not reported
via toole.report_error.

[M-A Ruel, 2013-10-29 22:42:50]

I feel a catch all like how it's done in breakpad.py would be more reliable than
a whitelist of exceptions to handle. Then if an exception is surfaced, it'll be
printed accordingly. You could still call manually in specific cases.

That said, I don't mind much.

https://codereview.chromium.org/51383003/diff/40001/isolate.py
File isolate.py (right):

https://codereview.chromium.org/51383003/diff/40001/isolate.py#newcode2457
isolate.py:2457: tools.report_error(str(e), include_traceback=True)
You may want to handle anything then? E.g. "except Exception as e:" If so, you
"may" want to use breakpad.py's technique. I don't have much opinion.

[Vadim Sh., 2013-10-30 00:38:35]

Catching generic Exception in outer blocks. I prefer to keep it explicit (as
opposite to breakpad.py atexit hooks).

Also removed 'include_traceback' in favor of checking first argument's type, I
think it's cleaner... ptal

[M-A Ruel, 2013-10-30 01:08:09]

lgtm but see warning below.

https://codereview.chromium.org/51383003/diff/170001/utils/tools.py
File utils/tools.py (right):

https://codereview.chromium.org/51383003/diff/170001/utils/tools.py#newcode132
utils/tools.py:132: print >> sys.stderr, str(error)
Note that this will crash if it is an unicode string that contains non-ascii
characters if the default encoding is not utf-8. In general, it is ascii.

[commit-bot: I haz the power, 2013-10-30 01:11:40]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/vadimsh@chromium.org/51383003/170001

[commit-bot: I haz the power, 2013-10-30 01:13:09]

Presubmit check for 51383003-170001 failed and returned exit status 1.

Running presubmit commit checks ...

** Presubmit ERRORS **
/b/commit-queue/workdir/tools/swarm_client/tests/isolate_smoke_test.py failed
..........F.F........F.F.......F.F.......F.F.......FFF.....
======================================================================
FAIL: test_missing_trailing_slash (__main__.Isolate_archive)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/b/commit-queue/workdir/tools/swarm_client/tests/isolate_smoke_test.py",
line 537, in test_missing_trailing_slash
    self._test_missing_trailing_slash('archive')
  File "/b/commit-queue/workdir/tools/swarm_client/tests/isolate_smoke_test.py",
line 371, in _test_missing_trailing_slash
    self.assertEqual(expected, out)
AssertionError: u'\nError: Input directory
/b/commit-queue/workdir/tools/swarm_client/tests/isolate/files1 must have a
trailing slash\n' != '[------ Swarming Error ------]\nInput directory
/b/commit-queue/workdir/tools/swarm_client/tests/isolate/files1 must have a
trailing slash\nTraceback (most recent call last):\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 2451, in main\n   
return dispatcher.execute(OptionParserIsolate(version=__version__), argv)\n 
File
"/b/commit-queue/workdir/tools/swarm_client/third_party/depot_tools/subcommand.py",
line 242, in execute\n    return command(parser, args[1:])\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 1988, in
CMDarchive\n    options, os.getcwd(), options.subdir, False)\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 1889, in
load_complete_state\n    cwd, isolate, options.variables,
options.ignore_broken_items)\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 1742, in
load_isolate\n    ignore_broken_items)\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 332, in
expand_directories_and_symlinks\n    follow_symlinks))\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 312, in
expand_directory_and_symlink\n    \'Input directory %s must have a trailing
slash\' % infile)\nMappingError: Input directory
/b/commit-queue/workdir/tools/swarm_client/tests/isolate/files1 must have a
trailing slash\n[----------------------------]\n'

======================================================================
FAIL: test_non_existent (__main__.Isolate_archive)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/b/commit-queue/workdir/tools/swarm_client/tests/isolate_smoke_test.py",
line 540, in test_non_existent
    self._test_non_existent('archive')
  File "/b/commit-queue/workdir/tools/swarm_client/tests/isolate_smoke_test.py",
line 388, in _test_non_existent
    self.assertEqual(expected, out)
AssertionError: u"\nError: Input file
/b/commit-queue/workdir/tools/swarm_client/tests/isolate/A_file_that_do_not_exist
doesn't exist\n" != '[------ Swarming Error ------]\nInput file
/b/commit-queue/workdir/tools/swarm_client/tests/isolate/A_file_that_do_not_exist
doesn\'t exist\nTraceback (most recent call last):\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 2451, in main\n   
return dispatcher.execute(OptionParserIsolate(version=__version__), argv)\n 
File
"/b/commit-queue/workdir/tools/swarm_client/third_party/depot_tools/subcommand.py",
line 242, in execute\n    return command(parser, args[1:])\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 1988, in
CMDarchive\n    options, os.getcwd(), options.subdir, False)\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 1889, in
load_complete_state\n    cwd, isolate, options.variables,
options.ignore_broken_items)\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 1742, in
load_isolate\n    ignore_broken_items)\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 332, in
expand_directories_and_symlinks\n    follow_symlinks))\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 316, in
expand_directory_and_symlink\n    \'Input file %s doesn\\\'t exist\' %
infile)\nMappingError: Input file
/b/commit-queue/workdir/tools/swarm_client/tests/isolate/A_file_that_do_not_exist
doesn\'t exist\n[----------------------------]\n'

======================================================================
FAIL: test_missing_trailing_slash (__main__.Isolate_check)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/b/commit-queue/workdir/tools/swarm_client/tests/isolate_smoke_test.py",
line 465, in test_missing_trailing_slash
    self._test_missing_trailing_slash('check')
  File "/b/commit-queue/workdir/tools/swarm_client/tests/isolate_smoke_test.py",
line 371, in _test_missing_trailing_slash
    self.assertEqual(expected, out)
AssertionError: u'\nError: Input directory
/b/commit-queue/workdir/tools/swarm_client/tests/isolate/files1 must have a
trailing slash\n' != '[------ Swarming Error ------]\nInput directory
/b/commit-queue/workdir/tools/swarm_client/tests/isolate/files1 must have a
trailing slash\nTraceback (most recent call last):\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 2451, in main\n   
return dispatcher.execute(OptionParserIsolate(version=__version__), argv)\n 
File
"/b/commit-queue/workdir/tools/swarm_client/third_party/depot_tools/subcommand.py",
line 242, in execute\n    return command(parser, args[1:])\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 2053, in
CMDcheck\n    options, os.getcwd(), options.subdir, False)\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 1889, in
load_complete_state\n    cwd, isolate, options.variables,
options.ignore_broken_items)\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 1742, in
load_isolate\n    ignore_broken_items)\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 332, in
expand_directories_and_symlinks\n    follow_symlinks))\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 312, in
expand_directory_and_symlink\n    \'Input directory %s must have a trailing
slash\' % infile)\nMappingError: Input directory
/b/commit-queue/workdir/tools/swarm_client/tests/isolate/files1 must have a
trailing slash\n[----------------------------]\n'

======================================================================
FAIL: test_non_existent (__main__.Isolate_check)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/b/commit-queue/workdir/tools/swarm_client/tests/isolate_smoke_test.py",
line 468, in test_non_existent
    self._test_non_existent('check')
  File "/b/commit-queue/workdir/tools/swarm_client/tests/isolate_smoke_test.py",
line 388, in _test_non_existent
    self.assertEqual(expected, out)
AssertionError: u"\nError: Input file
/b/commit-queue/workdir/tools/swarm_client/tests/isolate/A_file_that_do_not_exist
doesn't exist\n" != '[------ Swarming Error ------]\nInput file
/b/commit-queue/workdir/tools/swarm_client/tests/isolate/A_file_that_do_not_exist
doesn\'t exist\nTraceback (most recent call last):\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 2451, in main\n   
return dispatcher.execute(OptionParserIsolate(version=__version__), argv)\n 
File
"/b/commit-queue/workdir/tools/swarm_client/third_party/depot_tools/subcommand.py",
line 242, in execute\n    return command(parser, args[1:])\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 2053, in
CMDcheck\n    options, os.getcwd(), options.subdir, False)\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 1889, in
load_complete_state\n    cwd, isolate, options.variables,
options.ignore_broken_items)\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 1742, in
load_isolate\n    ignore_broken_items)\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 332, in
expand_directories_and_symlinks\n    follow_symlinks))\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 316, in
expand_directory_and_symlink\n    \'Input file %s doesn\\\'t exist\' %
infile)\nMappingError: Input file
/b/commit-queue/workdir/tools/swarm_client/tests/isolate/A_file_that_do_not_exist
doesn\'t exist\n[----------------------------]\n'

======================================================================
FAIL: test_missing_trailing_slash (__main__.Isolate_remap)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/b/commit-queue/workdir/tools/swarm_client/tests/isolate_smoke_test.py",
line 669, in test_missing_trailing_slash
    self._test_missing_trailing_slash('remap')
  File "/b/commit-queue/workdir/tools/swarm_client/tests/isolate_smoke_test.py",
line 371, in _test_missing_trailing_slash
    self.assertEqual(expected, out)
AssertionError: u'\nError: Input directory
/b/commit-queue/workdir/tools/swarm_client/tests/isolate/files1 must have a
trailing slash\n' != '[------ Swarming Error ------]\nInput directory
/b/commit-queue/workdir/tools/swarm_client/tests/isolate/files1 must have a
trailing slash\nTraceback (most recent call last):\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 2451, in main\n   
return dispatcher.execute(OptionParserIsolate(version=__version__), argv)\n 
File
"/b/commit-queue/workdir/tools/swarm_client/third_party/depot_tools/subcommand.py",
line 242, in execute\n    return command(parser, args[1:])\n  File
"/b/commit-queue/workdir/tools/swarm_client/isolate.py", line 2126, in
CMDremap\n    complete_state = load_complete_state(options, os.getcwd(), None,
False)\n  File "/b/commit-queue/workdir/tools/swarm_client/isolate.py", line
1889, in load_complete_state\n    cwd, isolate, options.…
(message too large)

[commit-bot: I haz the power, 2013-10-30 01:34:37]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/vadimsh@chromium.org/51383003/230001

[commit-bot: I haz the power, 2013-10-30 01:36:25]

Change committed as 231704