Enable downloading Google Play services via deps hook

Enable downloading Google Play services via deps hook

Call build/android/play_services/update.py from DEPS and
remove now unused scripts.

BUG=541727
Committed: https://crrev.com/ebe5d99c6d713d6f1be12e9a0f4a678e31be9aa0
Cr-Commit-Position: refs/heads/master@{#358326}

[dgn, 2015-10-28 13:54:33]

dgn@chromium.org changed reviewers:
+ jbudorick@chromium.org

[dgn, 2015-10-28 13:54:34]

PTAL. This is the CL that will turn hooks updating on. Currently it's still an
8.1x version that it would download.

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
File build/android/play_services/update.py (left):

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
build/android/play_services/update.py:397: return yes_no.YesNo('Do you accept
the license? [y/n]: ')
Didn't work very well from inside the hook runner. It completely messed up the
formatting of the license, and I'm not really sure why. raw_input works as
expected there.

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
File build/android/play_services/utils.py (right):

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
build/android/play_services/utils.py:45: def EmphasizeMessage(message):
Should I make a proper stream handler for the logger instead? I'll move it to a
separate CL then.

https://codereview.chromium.org/1418573010/diff/1/build/check_sdk_extras_vers...
File build/check_sdk_extras_version.py (right):

https://codereview.chromium.org/1418573010/diff/1/build/check_sdk_extras_vers...
build/check_sdk_extras_version.py:9: This file is now a placeholder until
removal after 2 sided patches.
This one is used in third_party/android_tools/android.gyp

[jbudorick, 2015-10-28 14:08:25]

I'm excited to see this, but beware that this won't be properly tested by the
bots since they run a different code path than devs.

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
File build/android/play_services/update.py (left):

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
build/android/play_services/update.py:397: return yes_no.YesNo('Do you accept
the license? [y/n]: ')
On 2015/10/28 13:54:33, dgn wrote:
> Didn't work very well from inside the hook runner. It completely messed up the
> formatting of the license, and I'm not really sure why. raw_input works as
> expected there.

That's odd. Does that still happen when printing line-by-line?

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
File build/android/play_services/update.py (right):

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
build/android/play_services/update.py:393: # the text here and flush is
explicitely to avoid having part of it dropped
nit: explicitly

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
build/android/play_services/update.py:395: for license_part in
license_file.read().split('\\n'):
nit: splitlines()

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
File build/android/play_services/utils.py (right):

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
build/android/play_services/utils.py:45: def EmphasizeMessage(message):
On 2015/10/28 13:54:33, dgn wrote:
> Should I make a proper stream handler for the logger instead? I'll move it to
a
> separate CL then.

Have warnings print yellow and errors print red? I might actually be interested
in that elsewhere.

[dgn, 2015-10-28 14:27:13]

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
File build/android/play_services/update.py (left):

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
build/android/play_services/update.py:397: return yes_no.YesNo('Do you accept
the license? [y/n]: ')
On 2015/10/28 14:08:25, jbudorick wrote:
> On 2015/10/28 13:54:33, dgn wrote:
> > Didn't work very well from inside the hook runner. It completely messed up
the
> > formatting of the license, and I'm not really sure why. raw_input works as
> > expected there.
> 
> That's odd. Does that still happen when printing line-by-line?

Yes. In both cases it outputs something like:

```
Lorem ipsum dolor sit amet, consectetur adipisicing elit,
sed do eiusmod tempor incididunt ut labore et dolore magna
aliqua. 

       Ut enim ad minim veniam, quis nostrud exercitation
ullamco laboris nisi ut aliquip ex ea commodo consequat.                        
       

Duis aute irure dolor in reprehenderit in voluptate velit
esse cillum dolore eu fugiat nulla pariatur.

                                            Excepteur sint
occaecat cupidatat non proident, sunt in culpa qui officia
deserunt mollit anim id est laborum.
```

(after a new line the column counter does not go back to column 0)

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
File build/android/play_services/utils.py (right):

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
build/android/play_services/utils.py:45: def EmphasizeMessage(message):
On 2015/10/28 14:08:25, jbudorick wrote:
> On 2015/10/28 13:54:33, dgn wrote:
> > Should I make a proper stream handler for the logger instead? I'll move it
to
> a
> > separate CL then.
> 
> Have warnings print yellow and errors print red? I might actually be
interested
> in that elsewhere.

Should I put that in pylib?

[jbudorick, 2015-10-28 14:28:54]

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
File build/android/play_services/update.py (left):

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
build/android/play_services/update.py:397: return yes_no.YesNo('Do you accept
the license? [y/n]: ')
On 2015/10/28 14:27:13, dgn wrote:
> On 2015/10/28 14:08:25, jbudorick wrote:
> > On 2015/10/28 13:54:33, dgn wrote:
> > > Didn't work very well from inside the hook runner. It completely messed up
> the
> > > formatting of the license, and I'm not really sure why. raw_input works as
> > > expected there.
> > 
> > That's odd. Does that still happen when printing line-by-line?
> 
> Yes. In both cases it outputs something like:
> 
> ```
> Lorem ipsum dolor sit amet, consectetur adipisicing elit,
> sed do eiusmod tempor incididunt ut labore et dolore magna
> aliqua. 
> 
>        Ut enim ad minim veniam, quis nostrud exercitation
> ullamco laboris nisi ut aliquip ex ea commodo consequat.                      
 
>        
> 
> Duis aute irure dolor in reprehenderit in voluptate velit
> esse cillum dolore eu fugiat nulla pariatur.
> 
>                                             Excepteur sint
> occaecat cupidatat non proident, sunt in culpa qui officia
> deserunt mollit anim id est laborum.
> ```
> 
> (after a new line the column counter does not go back to column 0)

yikes, that seems like a bug.

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
File build/android/play_services/utils.py (right):

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
build/android/play_services/utils.py:45: def EmphasizeMessage(message):
On 2015/10/28 14:27:13, dgn wrote:
> On 2015/10/28 14:08:25, jbudorick wrote:
> > On 2015/10/28 13:54:33, dgn wrote:
> > > Should I make a proper stream handler for the logger instead? I'll move it
> to
> > a
> > > separate CL then.
> > 
> > Have warnings print yellow and errors print red? I might actually be
> interested
> > in that elsewhere.
> 
> Should I put that in pylib?

Yeah, that'd be great.

You may see it in the test runner in the not-too-distant future :)

[jbudorick, 2015-10-28 14:29:31]

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
File build/android/play_services/utils.py (right):

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
build/android/play_services/utils.py:45: def EmphasizeMessage(message):
On 2015/10/28 14:28:53, jbudorick wrote:
> On 2015/10/28 14:27:13, dgn wrote:
> > On 2015/10/28 14:08:25, jbudorick wrote:
> > > On 2015/10/28 13:54:33, dgn wrote:
> > > > Should I make a proper stream handler for the logger instead? I'll move
it
> > to
> > > a
> > > > separate CL then.
> > > 
> > > Have warnings print yellow and errors print red? I might actually be
> > interested
> > > in that elsewhere.
> > 
> > Should I put that in pylib?
> 
> Yeah, that'd be great.

(specifically, pylib/utils/)

> 
> You may see it in the test runner in the not-too-distant future :)

[dgn, 2015-10-28 14:29:31]

Testing: for bots, I expect the trybots to be good enough.
I asked some people around here to try the CL too. I'll make sure
it works for them before submitting.

[dgn, 2015-10-28 16:32:58]

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
File build/android/play_services/update.py (right):

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
build/android/play_services/update.py:393: # the text here and flush is
explicitely to avoid having part of it dropped
On 2015/10/28 14:08:25, jbudorick wrote:
> nit: explicitly

Addressed in https://codereview.chromium.org/1418573010/

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
build/android/play_services/update.py:395: for license_part in
license_file.read().split('\\n'):
On 2015/10/28 14:08:25, jbudorick wrote:
> nit: splitlines()

Addressed in https://codereview.chromium.org/1418573010/

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
File build/android/play_services/utils.py (right):

https://codereview.chromium.org/1418573010/diff/1/build/android/play_services...
build/android/play_services/utils.py:45: def EmphasizeMessage(message):
On 2015/10/28 14:29:31, jbudorick wrote:
> On 2015/10/28 14:28:53, jbudorick wrote:
> > On 2015/10/28 14:27:13, dgn wrote:
> > > On 2015/10/28 14:08:25, jbudorick wrote:
> > > > On 2015/10/28 13:54:33, dgn wrote:
> > > > > Should I make a proper stream handler for the logger instead? I'll
move
> it
> > > to
> > > > a
> > > > > separate CL then.
> > > > 
> > > > Have warnings print yellow and errors print red? I might actually be
> > > interested
> > > > in that elsewhere.
> > > 
> > > Should I put that in pylib?
> > 
> > Yeah, that'd be great.
> 
> (specifically, pylib/utils/)
> 
> > 
> > You may see it in the test runner in the not-too-distant future :)
> 

Addressed in https://codereview.chromium.org/1418573010/

[dgn, 2015-11-03 19:17:05]

aberent@ (cc) successfully ran the hooks, but using gclient runhooks messed up
his output directory. He usually uses cr. Any idea what could cause that? Maybe
just using cr runhooks would have avoided the issue.

[aberent, 2015-11-04 14:30:52]

I still don't fully understand what happened here, but it appears to be
nothing to do with updating Google Play Services. My checkout was corrupt.

On Tue, 3 Nov 2015 at 19:17 <dgn@chromium.org> wrote:

> aberent@ (cc) successfully ran the hooks, but using gclient runhooks
> messed
> up
> his output directory. He usually uses cr. Any idea what could cause that?
> Maybe
> just using cr runhooks would have avoided the issue.
>
> https://codereview.chromium.org/1418573010/
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[dgn, 2015-11-04 16:35:12]

@jbudorick: PTAL. I believe it is now ready.

[jbudorick, 2015-11-04 17:15:31]

The code looks fine, so I tried running this and your other patch locally. In
doing so, a few things came up:

1. We dump this big license file without saying exactly what we're updating. I
think we should do so -- something along the lines of 'updating play services to
XXX' -- just so that a developer that hits this can see what's going on.
2. WAI on no, but on accepting the license, I hit this:

Do you accept the license? [y/n]: 
y
> 0> Downloading /tmp/tmpf7bNjO/google_play_services_library.zip...
Traceback (most recent call last):
  File "src/build/android/play_services/update.py", line 482, in <module>
    sys.exit(Main(sys.argv[1:]))
  File "src/build/android/play_services/update.py", line 97, in Main
    return args.func(args)
  File "src/build/android/play_services/update.py", line 206, in Download
    os.makedirs(paths.lib)
  File "/usr/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 17] File exists:
'[redacted]/src/third_party/android_tools/sdk/extras/google/google_play_services/libproject/google-play-services_lib'
Sending crash report ...
  args: ['src/build/android/play_services/update.py', 'download']
  cwd: [redacted]
  exception: [Errno 17] File exists: '/usr/local/google/code/cl
  host: jbudorick.[redacted]
  stack:   File "src/build/android/play_services/update.py"
  user: jbudorick
  version: 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2
A stack trace has been sent to the maintainers.
Error: Command '/usr/bin/python src/build/android/play_services/update.py
download' returned non-zero exit status 1 in [redacted]
Hook '/usr/bin/python src/build/android/play_services/update.py download' took
34.25 secs

[jbudorick, 2015-11-04 17:20:01]

On 2015/11/04 17:15:31, jbudorick wrote:
> The code looks fine, so I tried running this and your other patch locally. In
> doing so, a few things came up:
> 
> 1. We dump this big license file without saying exactly what we're updating. I
> think we should do so -- something along the lines of 'updating play services
to
> XXX' -- just so that a developer that hits this can see what's going on.
> 2. WAI on no, but on accepting the license, I hit this:
> 
> Do you accept the license? [y/n]: 
> y
> > 0> Downloading /tmp/tmpf7bNjO/google_play_services_library.zip...
> Traceback (most recent call last):
>   File "src/build/android/play_services/update.py", line 482, in <module>
>     sys.exit(Main(sys.argv[1:]))
>   File "src/build/android/play_services/update.py", line 97, in Main
>     return args.func(args)
>   File "src/build/android/play_services/update.py", line 206, in Download
>     os.makedirs(paths.lib)
>   File "/usr/lib/python2.7/os.py", line 157, in makedirs
>     mkdir(name, mode)
> OSError: [Errno 17] File exists:
>
'[redacted]/src/third_party/android_tools/sdk/extras/google/google_play_services/libproject/google-play-services_lib'
> Sending crash report ...
>   args: ['src/build/android/play_services/update.py', 'download']
>   cwd: [redacted]
>   exception: [Errno 17] File exists: '/usr/local/google/code/cl
>   host: jbudorick.[redacted]
>   stack:   File "src/build/android/play_services/update.py"
>   user: jbudorick
>   version: 2.7.6 (default, Jun 22 2015, 17:58:13) 
> [GCC 4.8.2
> A stack trace has been sent to the maintainers.
> Error: Command '/usr/bin/python src/build/android/play_services/update.py
> download' returned non-zero exit status 1 in [redacted]
> Hook '/usr/bin/python src/build/android/play_services/update.py download' took
> 34.25 secs

... which seems to happen despite the shutil.rmtree call because I have a lot of
files with 644 permissions in that directory.

[jbudorick, 2015-11-04 17:36:41]

On 2015/11/04 17:20:01, jbudorick wrote:
> On 2015/11/04 17:15:31, jbudorick wrote:
> > The code looks fine, so I tried running this and your other patch locally.
In
> > doing so, a few things came up:
> > 
> > 1. We dump this big license file without saying exactly what we're updating.
I
> > think we should do so -- something along the lines of 'updating play
services
> to
> > XXX' -- just so that a developer that hits this can see what's going on.
> > 2. WAI on no, but on accepting the license, I hit this:
> > 
> > Do you accept the license? [y/n]: 
> > y
> > > 0> Downloading /tmp/tmpf7bNjO/google_play_services_library.zip...
> > Traceback (most recent call last):
> >   File "src/build/android/play_services/update.py", line 482, in <module>
> >     sys.exit(Main(sys.argv[1:]))
> >   File "src/build/android/play_services/update.py", line 97, in Main
> >     return args.func(args)
> >   File "src/build/android/play_services/update.py", line 206, in Download
> >     os.makedirs(paths.lib)
> >   File "/usr/lib/python2.7/os.py", line 157, in makedirs
> >     mkdir(name, mode)
> > OSError: [Errno 17] File exists:
> >
>
'[redacted]/src/third_party/android_tools/sdk/extras/google/google_play_services/libproject/google-play-services_lib'
> > Sending crash report ...
> >   args: ['src/build/android/play_services/update.py', 'download']
> >   cwd: [redacted]
> >   exception: [Errno 17] File exists: '/usr/local/google/code/cl
> >   host: jbudorick.[redacted]
> >   stack:   File "src/build/android/play_services/update.py"
> >   user: jbudorick
> >   version: 2.7.6 (default, Jun 22 2015, 17:58:13) 
> > [GCC 4.8.2
> > A stack trace has been sent to the maintainers.
> > Error: Command '/usr/bin/python src/build/android/play_services/update.py
> > download' returned non-zero exit status 1 in [redacted]
> > Hook '/usr/bin/python src/build/android/play_services/update.py download'
took
> > 34.25 secs
> 
> ... which seems to happen despite the shutil.rmtree call because I have a lot
of
> files with 644 permissions in that directory.

after nuking the directory manually:

Do you accept the license? [y/n]: 
y
> 0> Downloading /tmp/tmpPqSvAY/google_play_services_library.zip...
Traceback (most recent call last):
  File "src/build/android/play_services/update.py", line 482, in <module>
    sys.exit(Main(sys.argv[1:]))
  File "src/build/android/play_services/update.py", line 97, in Main
    return args.func(args)
  File "src/build/android/play_services/update.py", line 206, in Download
    os.makedirs(paths.lib)
  File "/usr/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied:
'[redacted]/src/third_party/android_tools/sdk/extras/google/google_play_services/libproject/google-play-services_lib'
Sending crash report ...
  args: ['src/build/android/play_services/update.py', 'download']
  cwd: [redacted]
  exception: [Errno 13] Permission denied: '/usr/local/google/c
  host: jbudorick.[redacted]
  stack:   File "src/build/android/play_services/update.py"
  user: jbudorick
  version: 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2
A stack trace has been sent to the maintainers.
Error: Command '/usr/bin/python src/build/android/play_services/update.py
download' returned non-zero exit status 1 in [redacted]
Hook '/usr/bin/python src/build/android/play_services/update.py download' took
16.10 secs

[dgn, 2015-11-04 17:37:15]

On 2015/11/04 17:20:01, jbudorick wrote:
> On 2015/11/04 17:15:31, jbudorick wrote:
> > The code looks fine, so I tried running this and your other patch locally.
In
> > doing so, a few things came up:
> > 
> > 1. We dump this big license file without saying exactly what we're updating.
I
> > think we should do so -- something along the lines of 'updating play
services
> to
> > XXX' -- just so that a developer that hits this can see what's going on.

> 
> ... which seems to happen despite the shutil.rmtree call because I have a lot
of
> files with 644 permissions in that directory.

Ah right, I forgot that some people downloaded gmscore by running the script
with sudo. 
I'll update the other CL to check the permissions, and be more explicit when
printing the license.

[jbudorick, 2015-11-06 15:21:14]

lgtm

[dgn, 2015-11-06 15:21:38]

The CQ bit was checked by dgn@chromium.org

[commit-bot: I haz the power, 2015-11-06 15:22:02]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1418573010/300001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1418573010/300001

[commit-bot: I haz the power, 2015-11-06 15:28:58]

Committed patchset #16 (id:300001)

[commit-bot: I haz the power, 2015-11-06 15:29:57]

Patchset 16 (id:??) landed as
https://crrev.com/ebe5d99c6d713d6f1be12e9a0f4a678e31be9aa0
Cr-Commit-Position: refs/heads/master@{#358326}