Make the command cache use the directory name as part of the cache key

Make the command cache use the directory name as part of the cache key.

Some commands are sensitive to the directory they're executed from. Chrome's
Mojo is trying to use such a command.

This makes GYP about 10% slower on my Mac laptop when running on Chrome, which
is to say that it's not actually noticeably worse, because it was already so
bad.  Out of 2,413 command executions, there were formerly 475 cache misses;
this change adds another 1,596. I looked at the sorts of commands that are now
missing and the majority of them (879) are dir_exists.py followed by a
relative path, and those expansions are very incorrect to look up from the
cache. The next biggest buckets are 252 "echo -n ${HOME}/goma", 220
find_sdk.py, and another 220 plugin_flags.sh with an absolute directory
argument, and none of these are sensitive to the working directory. Finally,
there are 9 version.py invocations that are sensitive to the working
directory. The numbers don't add up because I examined different statistics
during different runs, and parallelism means that caching behavior is slightly
different on successive runs.

Considering that over half of the existing cache hits are dangerous, I'm
inclined to make this change from a correctness perspective over performance
concerns. Meanwhile, we should find a way to limit the time impact of the big
callees here. dir_exists.py, find_sdk.py, plugin_flags.sh, and the goma finder
might all perform better if migrated to pymod_do_main.

BUG=112, 418
TEST=gyptest.py test/variables/commands/gyptest-commands-repeated-multidir.py

Committed: https://code.google.com/p/gyp/source/detail?r=1913

[Mark Mentovai, 2014-04-08 18:54:37]

Any thoughts here?

[Mark Mentovai, 2014-05-02 13:15:29]

Nico, are you planning on reviewing this? If not, I’ll find someone else to look
at it.

[Daniel Bratell, 2014-05-05 12:31:23]

On 2014/05/02 13:15:29, Mark Mentovai wrote:
> Nico, are you planning on reviewing this? If not, I’ll find someone else to
look
> at it.

Didn't I write a comment here? If it's not here, then where is it...

Anyway, shorter version: So I managed to trigger the bug this patches by chance
last week (copy_ios_test_data.py is called with the same argument in different
directories and depend on the different invocations happening in different
processes so that not a cached result is returned). I approve this patch fully
and think it's the right thing to do since so many paths in gyp are releative.

The performance hit exists but it's smaller than when the patch was new since
dir_exists has moved to DoMain. The performance hit is also much smaller than
the recent gains so it will still be faster than a few weeks ago. And we would
get much of the same performance hit by fixing the broken cache in other ways.
So it looks good to me and I hope it looks good to the gyp owners as well.

[Mark Mentovai, 2014-05-05 16:23:45]

Committed patchset #1 manually as r1913.