Fix subtle dynamic library loading problem and deflake the tree.

Fix subtle dynamic library loading problem and deflake the tree.

Dynamic libraries were given a name on disk determined by the sha256 of their   
content.  If the same library was loaded from multiple origins, it would be  
given the same path on disk.  Because it had the same path, this dynamic library
would be initialized once. mojo_shell would invoke MojoMain multiple times,
however, to create instances for different URLs.  This caused the TLS of
RunLoop::current() to be allocated multiple times, and prior instances would
suddenly see RunLoop::current() == nullptr when the existing TLS was clobbered.

This CL mixes the URL into the sha256 to give different apps different names.

Fixes #63

[Nick Bray (chromium), 2015-03-14 00:19:23]

ncbray@chromium.org changed reviewers:
+ viettrungluu@chromium.org

[Nick Bray (chromium), 2015-03-14 00:19:24]

Note: I am not entirely happy with this solution as it will fold the _entire_
URL into the AppId, including query parameters, anchors, usernames.  I think
there needs to be a centralized function for deriving the "canonical app URL"
from the URL... ?

[qsr, 2015-03-16 08:40:12]

qsr@chromium.org changed reviewers:
+ eseidel@chromium.org

[qsr, 2015-03-16 08:40:31]

On 2015/03/14 00:19:24, Nick Bray (chromium) wrote:
> Note: I am not entirely happy with this solution as it will fold the _entire_
> URL into the AppId, including query parameters, anchors, usernames.  I think
> there needs to be a centralized function for deriving the "canonical app URL"
> from the URL... ?

+eseidel

NOT LGTM. The only reason we rename application to the SHA of their content is
to have predictable name so that we can have gdb find the file in the host to
debug. If we cannot do this, then we should not renamed anything and just use
the temporary name.

[eseidel, 2015-03-16 14:11:01]

eseidel@chromium.org changed reviewers:
+ qsr@chromium.org

[eseidel, 2015-03-16 14:11:01]

qsr is right, that the reason we rename to sha256 is so we have a predictacble
name for the binary and can explain to pprof/gdb how to find the symbols for
that name (using a directory of similarly named symlinks).

We also record a url -> load name mapping in a tmp file.

[eseidel, 2015-03-16 14:17:19]

https://github.com/domokit/mojo/blob/master/sky/tools/mojo_cache_linker.py is
the script we use to build the directory of symlinks for gdb.

pprop (when called from skydb profile) reads the url -> loaded file mapping off
of the device to do the mapping.  Unfortunately we can't easily do that for gdb.
 I'm happy to show you the relevant discussions on gdb-discuss@google, but the
TL;DR is that we have to somehow provide GDB an apriori mapping of load names to
symbols.  We could load these mojo's from tmp directories based on their origin
to avoid conflicts?  We could also just keep their original download names/paths
reflected under our load directory.  The problem I was trying to fix originally
was that we would use a *random* load name.

[qsr, 2015-03-16 14:19:18]

On 2015/03/16 14:17:19, eseidel wrote:
> https://github.com/domokit/mojo/blob/master/sky/tools/mojo_cache_linker.py is
> the script we use to build the directory of symlinks for gdb.
> 
> pprop (when called from skydb profile) reads the url -> loaded file mapping
off
> of the device to do the mapping.  Unfortunately we can't easily do that for
gdb.
>  I'm happy to show you the relevant discussions on gdb-discuss@google, but the
> TL;DR is that we have to somehow provide GDB an apriori mapping of load names
to
> symbols.  We could load these mojo's from tmp directories based on their
origin
> to avoid conflicts?  We could also just keep their original download
names/paths
> reflected under our load directory.  The problem I was trying to fix
originally
> was that we would use a *random* load name.

 Origin will not be enough. The case in question is http://XXXX/app.mojo?foo vs
http://XXXX/app.mojo?bar

 Creating a temporary directory with the hash of the URL would work. But we
probably would need some cleaning up afterwards...

[eseidel, 2015-03-16 14:21:55]

fwiw, gdb only cares about the basename, not the full path.  So we could do
something like:

/tmp/{hash of everything else}/{app_id / contents hash}

[qsr, 2015-03-16 14:22:41]

On 2015/03/16 14:21:55, eseidel wrote:
> fwiw, gdb only cares about the basename, not the full path.  So we could do
> something like:
> 
> /tmp/{hash of everything else}/{app_id / contents hash}

 Sorry, was not clear. That's what I suggested using the URL for the temporary
hash.

[Nick Bray (chromium), 2015-03-16 18:29:31]

Do one of you want to pick this up and run with it?  It sounds like this is a
design issue, and it also sounds like there's lots of undocumented stuff.

Two points, however:

1) I don't see how giving the so a name based on its hash buys you anything. 
(Was there a design doc for this?)  In fact, this is a questionable solution
because sha256 is slow, compared to load times.  If you really want to do this,
the hash really needs to be as the file is streaming in form the network.

2) the question of app?foo vs app?bar being separate apps is rather important. 
A normal caching scheme would consider those different responses.  If we
consider them the same app, then what happens when we reverse the order
connecting to the app?  It seems to me that it might be safest to align app
"identity" with normal http caching heuristics.

[eseidel, 2015-03-16 19:55:12]

There was no design doc.

Here is the relevant thread on gdb-discuss:
https://groups.google.com/a/google.com/d/msg/gdb-discuss/Fd0R-gFaqXk/c9SN_xlF...

You should feel welcome to change any and all of this.  The only question this
answers is telling gdb where to find the binaries.

When gdbserve connects to mojo_shell on Android, it sees we've loaded
/tmp/name.mojo.  It sends that information over to the gdb instance on the host.
 The host then looks for libraries named 'name.mojo' in its search path.

GDB does not give you any hooks to do the lookup for it (as far as I know),
meaning we have to have provided gdb files in its search paths of all the
library names we might expect to load.  Hence the current solution.

I filed a bug against GDB about using build-id lookup on the remote host to
solve this problem in a nicer way:
https://sourceware.org/bugzilla/show_bug.cgi?id=17917

[qsr, 2015-03-19 11:09:53]

> 2) the question of app?foo vs app?bar being separate apps is rather important.

> A normal caching scheme would consider those different responses.  If we
> consider them the same app, then what happens when we reverse the order
> connecting to the app?  It seems to me that it might be safest to align app
> "identity" with normal http caching heuristics.


 Except that we do want to have the state of the application in the URL.

 Everything starting with the '?' is not the identity of the app. It is the
responsibility of whoever serving the bits to make sure that either the bytes
received are exactly the same, or that it doesn't matter.

[qsr, 2015-03-19 11:10:15]

https://codereview.chromium.org/1011333003/