tool: add script that uploads go tool binaries

tool: add script that uploads go tool binaries

Script that downloads linux go binaries from golang.org,
build android binaries and uploads the stuff to GCS.
Script also creates VERSION file which will be used to check
that the current go tool installation is up to date.

R=jamesr@chromium.org, viettrungluu@chromium.org

Committed: https://chromium.googlesource.com/external/mojo/+/b98eda95ce18042a6e1db22e8867e14380510383

[rogulenko, 2015-04-08 20:05:14]

Patchset #1 (id:1) has been deleted

[rogulenko, 2015-04-08 20:06:45]

rogulenko@google.com changed reviewers:
+ jamesr@chromium.org, vtl@google.com

[viettrungluu, 2015-04-08 20:42:28]

viettrungluu@chromium.org changed reviewers:
+ viettrungluu@chromium.org

[viettrungluu, 2015-04-08 20:42:29]

https://codereview.chromium.org/1065333003/diff/20001/tools/go/upload.py
File tools/go/upload.py (right):

https://codereview.chromium.org/1065333003/diff/20001/tools/go/upload.py#newc...
tools/go/upload.py:23: This script doesn't check if current version is already
up to date, as produced
"as produced" -> "as the produced"

https://codereview.chromium.org/1065333003/diff/20001/tools/go/upload.py#newc...
tools/go/upload.py:24: tar.gz archive is slightly different every time, as
archive includes time stamp.
", as archive includes time stamp" -> "since it includes timestamps".

https://codereview.chromium.org/1065333003/diff/20001/tools/go/upload.py#newc...
tools/go/upload.py:36: NDK_PLATFORM = 'android-19'
Where do these NDK_... values come from? Don't they have to agree with what we
use elsewhere?

https://codereview.chromium.org/1065333003/diff/20001/tools/go/upload.py#newc...
tools/go/upload.py:105: os.environ["GOPATH"] = go_linux
Rather than modifying your own environment (which could conceivably have side
effects elsewhere), you could provide a suitable env=... to subprocess.call().

(You could either start with a copy of os.environ and modify it suitably, or you
could provide a clean, fully-specified environment. The latter is better for
reproducibility, but possibly more annoying to figure out everything you need to
set.)

https://codereview.chromium.org/1065333003/diff/20001/tools/go/upload.py#newc...
tools/go/upload.py:137: and writes sha1 to THID_DIR/VERSION."""
"THID" -> "THIS"

https://codereview.chromium.org/1065333003/diff/20001/tools/go/upload.py#newc...
tools/go/upload.py:158: sha1 = Compress(tmpdir)
Would it be useful to print out the sha1 somewhere? (I guess it ends up in the
VERSION file....)

[rogulenko, 2015-04-08 21:31:34]

https://codereview.chromium.org/1065333003/diff/20001/tools/go/upload.py
File tools/go/upload.py (right):

https://codereview.chromium.org/1065333003/diff/20001/tools/go/upload.py#newc...
tools/go/upload.py:23: This script doesn't check if current version is already
up to date, as produced
On 2015/04/08 20:42:29, viettrungluu wrote:
> "as produced" -> "as the produced"

Done.

https://codereview.chromium.org/1065333003/diff/20001/tools/go/upload.py#newc...
tools/go/upload.py:24: tar.gz archive is slightly different every time, as
archive includes time stamp.
On 2015/04/08 20:42:29, viettrungluu wrote:
> ", as archive includes time stamp" -> "since it includes timestamps".

Done.

https://codereview.chromium.org/1065333003/diff/20001/tools/go/upload.py#newc...
tools/go/upload.py:36: NDK_PLATFORM = 'android-19'
On 2015/04/08 20:42:29, viettrungluu wrote:
> Where do these NDK_... values come from? Don't they have to agree with what we
> use elsewhere?

Changed android-19 -> android-14.

Actually I didn't find a place where to get them.
Now at least tools/clang/scripts/update.sh thinks the same.

To have an idea of how often these values are used in mojo
I have compared grep -r "%s" for android-14 vs android-19 and
arm-linux-androideabi-4.9 vs arm-linux-androideabi-4.8.
The results are ~4800 vs ~48 matches for both tests.

https://codereview.chromium.org/1065333003/diff/20001/tools/go/upload.py#newc...
tools/go/upload.py:105: os.environ["GOPATH"] = go_linux
On 2015/04/08 20:42:29, viettrungluu wrote:
> Rather than modifying your own environment (which could conceivably have side
> effects elsewhere), you could provide a suitable env=... to subprocess.call().
> 
> (You could either start with a copy of os.environ and modify it suitably, or
you
> could provide a clean, fully-specified environment. The latter is better for
> reproducibility, but possibly more annoying to figure out everything you need
to
> set.)

Replaced with os.environ.copy().

https://codereview.chromium.org/1065333003/diff/20001/tools/go/upload.py#newc...
tools/go/upload.py:137: and writes sha1 to THID_DIR/VERSION."""
On 2015/04/08 20:42:29, viettrungluu wrote:
> "THID" -> "THIS"

Done.

https://codereview.chromium.org/1065333003/diff/20001/tools/go/upload.py#newc...
tools/go/upload.py:158: sha1 = Compress(tmpdir)
On 2015/04/08 20:42:29, viettrungluu wrote:
> Would it be useful to print out the sha1 somewhere? (I guess it ends up in the
> VERSION file....)

I think writing the sha1 code to VERSION is sufficient. For debug
purposes you can comment out line shutil.rmtree(tmpdir) and look into
the tmpdir, path to which is printed to stdout in the beginning of the
script execution.

[viettrungluu, 2015-04-08 21:54:19]

OK, LGTM w/change in default argument.

(You could also have **kwargs as an argument, and pass **kwargs on to
subprocess.call() if you want.)

https://codereview.chromium.org/1065333003/diff/40001/tools/go/upload.py
File tools/go/upload.py (right):

https://codereview.chromium.org/1065333003/diff/40001/tools/go/upload.py#newc...
tools/go/upload.py:45: def RunCommand(command, env=os.environ):
The default value for subprocess.call/Popen's env argument is None, so you
should probably do that too.

Doing env=os.environ binds to the object named by os.environ "here", so it'd be
surprising if anyone ever did os.environ = ... elsewhere and then called this
(the default argument would be the old os.environ object, not the one at the
time of the call).

[rogulenko, 2015-04-08 22:33:19]

https://codereview.chromium.org/1065333003/diff/40001/tools/go/upload.py
File tools/go/upload.py (right):

https://codereview.chromium.org/1065333003/diff/40001/tools/go/upload.py#newc...
tools/go/upload.py:45: def RunCommand(command, env=os.environ):
On 2015/04/08 21:54:19, viettrungluu wrote:
> The default value for subprocess.call/Popen's env argument is None, so you
> should probably do that too.
> 
> Doing env=os.environ binds to the object named by os.environ "here", so it'd
be
> surprising if anyone ever did os.environ = ... elsewhere and then called this
> (the default argument would be the old os.environ object, not the one at the
> time of the call).

Done.

[viettrungluu, 2015-04-09 02:30:12]

(Still LGTM, in case you were waiting[*].)

[*] Which to be clear and for future reference, you shouldn't.

[rogulenko, 2015-04-09 02:45:46]

On 2015/04/09 02:30:12, viettrungluu wrote:
> (Still LGTM, in case you were waiting[*].)
> 
> [*] Which to be clear and for future reference, you shouldn't.

Actually I am waiting for James, maybe he has some thoughts that
he would like to share.

[jamesr, 2015-04-09 22:00:33]

lgtm as well

https://codereview.chromium.org/1065333003/diff/60001/tools/go/upload.py
File tools/go/upload.py (right):

https://codereview.chromium.org/1065333003/diff/60001/tools/go/upload.py#newc...
tools/go/upload.py:37: NDK_PLATFORM = 'android-14'
i think we'll probably want to bump this to a newer rev (we're very unlikely to
want to build things targeting Ice Cream Sandwich) but it's not important to do
so now. most things reference android-14 as that's the oldest API level the
chrome repo targetted when it was first being brought up and most things just
haven't changed.

[rogulenko, 2015-04-09 22:41:14]

Committed patchset #3 (id:60001) manually as
b98eda95ce18042a6e1db22e8867e14380510383 (presubmit successful).