Fix report symbolization on swarming bots.

Fix report symbolization on swarming bots.
According to issue 444835 there were at least two problems with debug info on
the swarming bots:
 - the .dSYM bundles for the test binaries and other applications weren't
uploaded to swarming servers;
 - for some binaries (e.g. Chromium Framework) their names were replaced with
SHA-1 hashes that confused llvm-symbolizer and made finding the .dSYM bundles
impossible.

This CL adds the necessary .dSYM paths to .isolate files for the tests
currently being ran on swarming bots (for other tests their .dSYM paths will be
added as needed).
It also extends tools/valgrind/asan/asan_symbolize.py with a binary name filter
that replaces the hashes with the corresponding file names in the product
directory.

BUG=444835
R=earthdok@chromium.org,maruel@chromium.org,csharp@chromium.org

Committed: https://crrev.com/9d9193404a307ba7cf79db460a0f988bc349bc4e
Cr-Commit-Position: refs/heads/master@{#315059}

[Alexander Potapenko, 2015-01-23 12:09:32]

Please take a look.

[earthdok, 2015-01-23 15:41:50]

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
File tools/valgrind/asan/asan_symbolize.py (right):

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
tools/valgrind/asan/asan_symbolize.py:54: match = re.match('[0-9a-f]+', name)
Note that re.match() matches at the beginning of the string, but allows
unmatched characters at the end. You probably want '[0-9a-f]+$'.

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
tools/valgrind/asan/asan_symbolize.py:61: path_parts =
exe_path.split(os.path.sep)
why not os.path.split()/os.path.join()?

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
tools/valgrind/asan/asan_symbolize.py:62: if len(path_parts) == 1:
the logic is not entirely clear here. a comment would be helpful.

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
tools/valgrind/asan/asan_symbolize.py:102: binary_path = new_binary_path
return new_binary_path ?

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
tools/valgrind/asan/asan_symbolize.py:228:
parser.add_argument('--executable_path',
dash, not underscore

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
tools/valgrind/asan/asan_symbolize.py:229: help='Path to program executable.
Will be used on OSX swarming bots to '
Please drop the future tense.

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
tools/valgrind/asan/asan_symbolize.py:237: maybe_binary_name_filter = None
"maybe_" is kind of unnecessary

[Alexander Potapenko, 2015-01-23 16:37:55]

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
File tools/valgrind/asan/asan_symbolize.py (right):

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
tools/valgrind/asan/asan_symbolize.py:54: match = re.match('[0-9a-f]+', name)
On 2015/01/23 15:41:50, earthdok wrote:
> Note that re.match() matches at the beginning of the string, but allows
> unmatched characters at the end. You probably want '[0-9a-f]+$'.

Done.

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
tools/valgrind/asan/asan_symbolize.py:61: path_parts =
exe_path.split(os.path.sep)
On 2015/01/23 15:41:50, earthdok wrote:
> why not os.path.split()/os.path.join()?

Done

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
tools/valgrind/asan/asan_symbolize.py:62: if len(path_parts) == 1:
On 2015/01/23 15:41:50, earthdok wrote:
> the logic is not entirely clear here. a comment would be helpful.

Done.

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
tools/valgrind/asan/asan_symbolize.py:102: binary_path = new_binary_path
On 2015/01/23 15:41:50, earthdok wrote:
> return new_binary_path ?

Done.

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
tools/valgrind/asan/asan_symbolize.py:228:
parser.add_argument('--executable_path',
On 2015/01/23 15:41:50, earthdok wrote:
> dash, not underscore

Done.

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
tools/valgrind/asan/asan_symbolize.py:229: help='Path to program executable.
Will be used on OSX swarming bots to '
On 2015/01/23 15:41:50, earthdok wrote:
> Please drop the future tense.

Done.

https://codereview.chromium.org/859293002/diff/1/tools/valgrind/asan/asan_sym...
tools/valgrind/asan/asan_symbolize.py:237: maybe_binary_name_filter = None
On 2015/01/23 15:41:50, earthdok wrote:
> "maybe_" is kind of unnecessary

Done.

[Alexander Potapenko, 2015-01-27 14:21:08]

This CL depends on https://codereview.chromium.org/878923002/

[Alexander Potapenko, 2015-02-05 09:15:51]

Sergey, can you please take another look? Looks like everything works now.

[Alexander Potapenko, 2015-02-05 09:18:09]

glider@chromium.org changed reviewers:
+ maruel@chromium.org

[Alexander Potapenko, 2015-02-05 09:18:37]

On 2015/02/05 09:15:51, Alexander Potapenko wrote:
> Sergey, can you please take another look? Looks like everything works now.

Adding M-A for .isolate file ownership.

[earthdok, 2015-02-05 17:55:50]

On 2015/02/05 09:18:37, Alexander Potapenko wrote:
> On 2015/02/05 09:15:51, Alexander Potapenko wrote:
> > Sergey, can you please take another look? Looks like everything works now.
> 
> Adding M-A for .isolate file ownership.

Protip: maruel@ doesn't actually own those.

[earthdok, 2015-02-05 17:56:00]

https://codereview.chromium.org/859293002/diff/240001/testing/test_env.py
File testing/test_env.py (right):

https://codereview.chromium.org/859293002/diff/240001/testing/test_env.py#new...
testing/test_env.py:194:
get_sanitizer_symbolize_command(executable_path=cmd[0]),
cmd[0] is not guaranteed to be the test binary
take a look at fix_python_path()

[Alexander Potapenko, 2015-02-06 09:35:11]

On 2015/02/05 17:56:00, earthdok wrote:
> https://codereview.chromium.org/859293002/diff/240001/testing/test_env.py
> File testing/test_env.py (right):
> 
>
https://codereview.chromium.org/859293002/diff/240001/testing/test_env.py#new...
> testing/test_env.py:194:
> get_sanitizer_symbolize_command(executable_path=cmd[0]),
> cmd[0] is not guaranteed to be the test binary
> take a look at fix_python_path()

This is a best-effort attempt to obtain the path to the test binary.
In the case the command line starts with 'python', there's no other mean to
obtain the path.

Also, look at get_asan_env():

 98   if sys.platform == 'darwin':
 99     isolate_output_dir = os.path.abspath(os.path.dirname(cmd[0]))
100     # This is needed because the test binary has @executable_path embedded
in it
101     # it that the OS tries to resolve to the cache directory and not the
mapped
102     #  directory.
103     extra_env['DYLD_LIBRARY_PATH'] = str(isolate_output_dir)

[Alexander Potapenko, 2015-02-06 09:38:13]

+csharp
Chris, in the case you're really the person in charge of .isolate files, can you
please review my CL?

[earthdok, 2015-02-06 10:17:04]

On 2015/02/06 09:38:13, Alexander Potapenko wrote:
> +csharp
> Chris, in the case you're really the person in charge of .isolate files, can
you
> please review my CL?

Actually, I'm sorry, there's a subset of .isolate files that aren't owned by
maruel/csharp:

    extensions/extensions_browsertests.isolate
    gpu/gpu_unittests.isolate
    ui/app_list/app_list_unittests.isolate
    ui/base/ui_base_unittests.isolate
    ui/events/events_unittests.isolate
    ui/message_center/message_center_unittests.isolate
    ui/touch_selection/ui_touch_selection_unittests.isolate

It looks like you don't touch any of those, though you probably want to.

[Alexander Potapenko, 2015-02-06 14:04:19]

On 2015/02/06 10:17:04, earthdok wrote:
> On 2015/02/06 09:38:13, Alexander Potapenko wrote:
> > +csharp
> > Chris, in the case you're really the person in charge of .isolate files, can
> you
> > please review my CL?
> 
> Actually, I'm sorry, there's a subset of .isolate files that aren't owned by
> maruel/csharp:
> 
>     extensions/extensions_browsertests.isolate
>     gpu/gpu_unittests.isolate
>     ui/app_list/app_list_unittests.isolate
>     ui/base/ui_base_unittests.isolate
>     ui/events/events_unittests.isolate
>     ui/message_center/message_center_unittests.isolate
>     ui/touch_selection/ui_touch_selection_unittests.isolate
> 
> It looks like you don't touch any of those, though you probably want to.

I don't, at least for now - we're not running those tests on the swarming bots
so we won't be able to check them.

[csharp, 2015-02-06 14:34:13]

csharp@chromium.org changed reviewers:
+ csharp@chromium.org

[csharp, 2015-02-06 14:34:14]

Current set of *.isolate files lgtm

[M-A Ruel, 2015-02-06 14:35:56]

On 2015/02/06 14:34:14, csharp wrote:
> Current set of *.isolate files lgtm

Thanks Chris. I was travelling two days ago, yesterday I was recovering and it
passed under my radar today. lgtm

M-A

[Alexander Potapenko, 2015-02-06 14:43:57]

The CQ bit was checked by glider@chromium.org

[commit-bot: I haz the power, 2015-02-06 14:44:51]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/859293002/240001

[commit-bot: I haz the power, 2015-02-06 16:18:55]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2015-02-06 16:18:57]

Try jobs failed on following builders:
  mac_chromium_rel_ng on tryserver.chromium.mac (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel_...)

[Alexander Potapenko, 2015-02-06 16:26:52]

New patchsets have been uploaded after l-g-t-m from
csharp@chromium.org,maruel@chromium.org

[Alexander Potapenko, 2015-02-06 16:27:33]

The CQ bit was checked by glider@chromium.org

[commit-bot: I haz the power, 2015-02-06 16:28:04]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/859293002/260001

[commit-bot: I haz the power, 2015-02-06 17:42:38]

Committed patchset #14 (id:260001)

[commit-bot: I haz the power, 2015-02-06 17:43:38]

Patchset 14 (id:??) landed as
https://crrev.com/9d9193404a307ba7cf79db460a0f988bc349bc4e
Cr-Commit-Position: refs/heads/master@{#315059}

[Timur Iskhodzhanov, 2015-02-11 21:36:15]

timurrrr@chromium.org changed reviewers:
+ timurrrr@chromium.org

[Timur Iskhodzhanov, 2015-02-11 21:36:16]

https://codereview.chromium.org/859293002/diff/260001/tools/valgrind/asan/asa...
File tools/valgrind/asan/asan_symbolize.py (right):

https://codereview.chromium.org/859293002/diff/260001/tools/valgrind/asan/asa...
tools/valgrind/asan/asan_symbolize.py:251: if os.uname()[0] == 'Darwin':
this asserts on Windows, again.