Make --gdb work for android

Make --gdb work for android

This mostly works.  I haven't yet set up pulling down the
system binaries from the device to the host so that symbols
appear correctly, but I'll do that in the next patch.

One of the crazy things this patch adds it a script
to watch for loads on adb logcat and set up mappings
from the cache library names to the symboled binaries
in the out directory.  Presumably other scripts may
want to share this functionality so I've made it its
own script.

Better would be to have mojo_shell spit out a file
including the cache mapping information and we could
watch that file instead of logcat, but this works
for now.

R=qsr@chromium.org
BUG=

Committed: https://chromium.googlesource.com/external/mojo/+/57cd1992a2ecbb4103715d0969ce74c45d4b2847

[qsr, 2015-01-16 00:04:19]

https://codereview.chromium.org/848013004/diff/20001/shell/context.cc
File shell/context.cc (right):

https://codereview.chromium.org/848013004/diff/20001/shell/context.cc#newcode155
shell/context.cc:155: if (command_line->HasSwitch(switches::kWaitForDebugger))
You should be able to do this at the very start of this method. I can
definitively see EnsureEmbedderIsInitialized(); needing debugging.

https://codereview.chromium.org/848013004/diff/20001/sky/tools/mojo_cache_lin...
File sky/tools/mojo_cache_linker.py (right):

https://codereview.chromium.org/848013004/diff/20001/sky/tools/mojo_cache_lin...
sky/tools/mojo_cache_linker.py:38: logging.info('%s does not match base %s' %
(url, args.base_url))
This is kind of expected if you run the shell multiple time, maybe a debug
logging is enough?

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb
File sky/tools/skydb (right):

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb#newcode157
sky/tools/skydb:157: print fields[-1]
Is that some debugging leftover?

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb#newcode215
sky/tools/skydb:215: shell_command = ['gdbserver', ':%s' % GDB_PORT] +
shell_command
%d? Here and everywhere you format an integer

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb#newcode233
sky/tools/skydb:233: if not self.pids['mojo_shell_pid']:
Any reason not to write this:
if not pid:
  ...

 and do that before updating the dict?

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb#newcode254
sky/tools/skydb:254: subprocess.Popen(gdbserver_cmd).pid
I am missing something here. Why are you transforming gdbserver_cmd into a
string and using the string version of Popen instead of using the array version?

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb#newcode390
sky/tools/skydb:390: os.makedirs(links_path)
You do not seem to ever be cleaning this directory. Should you delete it on
startup?

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb#newcode432
sky/tools/skydb:432: gdb_path = os.path.join(SRC_ROOT,
'third_party/android_tools/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gdb')
This should be 80 chars long... You also might want to separate all the
directory:
os.path.join(SRC_ROOT, a, b, c, d)...

[eseidel, 2015-01-16 00:14:06]

https://codereview.chromium.org/848013004/diff/20001/shell/context.cc
File shell/context.cc (right):

https://codereview.chromium.org/848013004/diff/20001/shell/context.cc#newcode155
shell/context.cc:155: if (command_line->HasSwitch(switches::kWaitForDebugger))
On 2015/01/16 00:04:18, qsr wrote:
> You should be able to do this at the very start of this method. I can
> definitively see EnsureEmbedderIsInitialized(); needing debugging.

Assuming it's safe to call the command line before the embedder is initialized,
sure.  Done.

https://codereview.chromium.org/848013004/diff/20001/sky/tools/mojo_cache_lin...
File sky/tools/mojo_cache_linker.py (right):

https://codereview.chromium.org/848013004/diff/20001/sky/tools/mojo_cache_lin...
sky/tools/mojo_cache_linker.py:38: logging.info('%s does not match base %s' %
(url, args.base_url))
On 2015/01/16 00:04:18, qsr wrote:
> This is kind of expected if you run the shell multiple time, maybe a debug
> logging is enough?

Done.

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb
File sky/tools/skydb (right):

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb#newcode157
sky/tools/skydb:157: print fields[-1]
On 2015/01/16 00:04:19, qsr wrote:
> Is that some debugging leftover?

Yes!  Removed.

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb#newcode215
sky/tools/skydb:215: shell_command = ['gdbserver', ':%s' % GDB_PORT] +
shell_command
On 2015/01/16 00:04:19, qsr wrote:
> %d? Here and everywhere you format an integer

OK. What's the benefit?

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb#newcode233
sky/tools/skydb:233: if not self.pids['mojo_shell_pid']:
On 2015/01/16 00:04:18, qsr wrote:
> Any reason not to write this:
> if not pid:
>   ...
> 
>  and do that before updating the dict?

Done.

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb#newcode254
sky/tools/skydb:254: subprocess.Popen(gdbserver_cmd).pid
On 2015/01/16 00:04:19, qsr wrote:
> I am missing something here. Why are you transforming gdbserver_cmd into a
> string and using the string version of Popen instead of using the array
version?

I'm only transforming it into a string for the print command (which I could
remove).  I'm passing the array to popen.

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb#newcode390
sky/tools/skydb:390: os.makedirs(links_path)
On 2015/01/16 00:04:18, qsr wrote:
> You do not seem to ever be cleaning this directory. Should you delete it on
> startup?

Unclear.  I could?  Originally I had the new script manage this directory, but
that seemed error-prone.  I certainly could have this script delete it and
create it fresh every time...

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb#newcode432
sky/tools/skydb:432: gdb_path = os.path.join(SRC_ROOT,
'third_party/android_tools/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gdb')
On 2015/01/16 00:04:19, qsr wrote:
> This should be 80 chars long... You also might want to separate all the
> directory:
> os.path.join(SRC_ROOT, a, b, c, d)...

Eek!  I need to fix this lookup before landing. Thanks.

[eseidel, 2015-01-16 00:16:40]

ptal

[qsr, 2015-01-16 00:46:40]

https://codereview.chromium.org/848013004/diff/20001/sky/tools/mojo_cache_lin...
File sky/tools/mojo_cache_linker.py (right):

https://codereview.chromium.org/848013004/diff/20001/sky/tools/mojo_cache_lin...
sky/tools/mojo_cache_linker.py:38: logging.info('%s does not match base %s' %
(url, args.base_url))
On 2015/01/16 00:14:06, eseidel wrote:
> On 2015/01/16 00:04:18, qsr wrote:
> > This is kind of expected if you run the shell multiple time, maybe a debug
> > logging is enough?
> 
> Done.

Doesn't see to have changed...

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb
File sky/tools/skydb (right):

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb#newcode215
sky/tools/skydb:215: shell_command = ['gdbserver', ':%s' % GDB_PORT] +
shell_command
On 2015/01/16 00:14:06, eseidel wrote:
> On 2015/01/16 00:04:19, qsr wrote:
> > %d? Here and everywhere you format an integer
> 
> OK. What's the benefit?

It will crash if this is not a number. Not much, but can catch some stupid
things.

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb#newcode254
sky/tools/skydb:254: subprocess.Popen(gdbserver_cmd).pid
On 2015/01/16 00:14:06, eseidel wrote:
> On 2015/01/16 00:04:19, qsr wrote:
> > I am missing something here. Why are you transforming gdbserver_cmd into a
> > string and using the string version of Popen instead of using the array
> version?
> 
> I'm only transforming it into a string for the print command (which I could
> remove).  I'm passing the array to popen.

 That's what I was missing. Sorry.

https://codereview.chromium.org/848013004/diff/20001/sky/tools/skydb#newcode390
sky/tools/skydb:390: os.makedirs(links_path)
On 2015/01/16 00:14:06, eseidel wrote:
> On 2015/01/16 00:04:18, qsr wrote:
> > You do not seem to ever be cleaning this directory. Should you delete it on
> > startup?
> 
> Unclear.  I could?  Originally I had the new script manage this directory, but
> that seemed error-prone.  I certainly could have this script delete it and
> create it fresh every time...

I fear this will quickly be full on unused files. Clearing it each time should
help.

https://codereview.chromium.org/848013004/diff/40001/sky/tools/mojo_cache_lin...
File sky/tools/mojo_cache_linker.py (right):

https://codereview.chromium.org/848013004/diff/40001/sky/tools/mojo_cache_lin...
sky/tools/mojo_cache_linker.py:15: def main():
Indentation should be 2 spaces, no?

https://codereview.chromium.org/848013004/diff/40001/sky/tools/mojo_cache_lin...
sky/tools/mojo_cache_linker.py:19: 'of symlinks to symboled binaries for seen
cache names.')
I think you are missing a space.

https://codereview.chromium.org/848013004/diff/40001/sky/tools/mojo_cache_lin...
sky/tools/mojo_cache_linker.py:29: sys.exit(2)
Just a question, why 2?

https://codereview.chromium.org/848013004/diff/40001/sky/tools/skydb
File sky/tools/skydb (right):

https://codereview.chromium.org/848013004/diff/40001/sky/tools/skydb#newcode247
sky/tools/skydb:247: ':%s' % GDB_PORT,
%d ?

[eseidel, 2015-01-16 19:17:18]

https://codereview.chromium.org/848013004/diff/40001/sky/tools/mojo_cache_lin...
File sky/tools/mojo_cache_linker.py (right):

https://codereview.chromium.org/848013004/diff/40001/sky/tools/mojo_cache_lin...
sky/tools/mojo_cache_linker.py:15: def main():
On 2015/01/16 00:46:40, qsr wrote:
> Indentation should be 2 spaces, no?

I was just trying to follow python-standard pep8, which I think uses 4?  No
pre-submit complained to me.

https://codereview.chromium.org/848013004/diff/40001/sky/tools/mojo_cache_lin...
sky/tools/mojo_cache_linker.py:19: 'of symlinks to symboled binaries for seen
cache names.')
On 2015/01/16 00:46:40, qsr wrote:
> I think you are missing a space.

Done.

https://codereview.chromium.org/848013004/diff/40001/sky/tools/mojo_cache_lin...
sky/tools/mojo_cache_linker.py:29: sys.exit(2)
On 2015/01/16 00:46:40, qsr wrote:
> Just a question, why 2? 

No good reason.  I'll change it to 1.

https://codereview.chromium.org/848013004/diff/40001/sky/tools/skydb
File sky/tools/skydb (right):

https://codereview.chromium.org/848013004/diff/40001/sky/tools/skydb#newcode247
sky/tools/skydb:247: ':%s' % GDB_PORT,
On 2015/01/16 00:46:40, qsr wrote:
> %d ?

I think this makes the code needlessly fragile for no real benefit.  But I've
changed this one since it happens to be OK.  I agree it's nice to know the types
of things, but python doesn't have types, so what I"m asking here is for the
string value of the thing and if the thing quacks like a string, it's gonna be
OK. :)

I don't have any unit-testing for this code which is probably the real reason
for any fragility... :(  (and well over due).

[eseidel, 2015-01-16 19:17:46]

ptal

[qsr, 2015-01-16 19:22:46]

lgtm

https://codereview.chromium.org/848013004/diff/60001/sky/tools/mojo_cache_lin...
File sky/tools/mojo_cache_linker.py (right):

https://codereview.chromium.org/848013004/diff/60001/sky/tools/mojo_cache_lin...
sky/tools/mojo_cache_linker.py:10: import logging
alphabetize

https://codereview.chromium.org/848013004/diff/60001/sky/tools/skydb
File sky/tools/skydb (right):

https://codereview.chromium.org/848013004/diff/60001/sky/tools/skydb#newcode20
sky/tools/skydb:20: import re
Alphabetize.

[eseidel, 2015-01-16 19:28:35]

Committed patchset #4 (id:60001) manually as
57cd1992a2ecbb4103715d0969ce74c45d4b2847 (presubmit successful).