Sysroot: Add build_and_upload.py

Sysroot: Add build_and_upload.py

This CL adds a new script that automates building and uploading new
sysroot images.  Previously, you had to run UpdatePackageListsAll,
BuildSysrootAll, and UploadSysrootAll for each of the 4
sysroot-creators.  Then, you had to manually update the revision and
sha1sum in install_sysroot.py.  Now, simply run build_and_upload.py
and your CL should be ready to go.  In addition, the script
concurrently updates each sysroot image (there are currently 11 of
them), greatly reducing the time it took to update the sysroots.

In addition to adding the script, this CL also rolls all sysroots.

R=dpranke@chromium.org

Committed: https://crrev.com/4c8b67057de3ee3f6c4105a0dceebc0228d9832e
Cr-Commit-Position: refs/heads/master@{#438862}

[Tom (Use chromium acct), 2016-12-12 18:49:58]

+dpranke@ since we were discussing this the other day.  I know you're busing
making tryserver.chromium.linux great again, so feel free to punt over to mmoss@
if necessary.

+sbc@ FYI, you might have to rebase your change, but it should be as simple as
reverting SYSROOTS in install-sysroot.py and then running build_and_upload.py

https://codereview.chromium.org/2567123002/diff/1/build/linux/sysroot_scripts...
File build/linux/sysroot_scripts/build_and_upload.py (right):

https://codereview.chromium.org/2567123002/diff/1/build/linux/sysroot_scripts...
build/linux/sysroot_scripts/build_and_upload.py:38: revision = rand_string_32()
The revision can really be any string, but maybe it should be an increasing
integer?  Something like distro_release_arch_integer?  Or just the commit hash
on ToT?

I probably don't want to leave it random because that means if you run the
script multiple times, you'd be wasting google storage space.

[Sam Clegg, 2016-12-12 19:17:59]

sbc@chromium.org changed reviewers:
+ sbc@chromium.org

[Sam Clegg, 2016-12-12 19:18:00]

I like some parts of this CL.  Is there some way we could keep it as as single
json file rather than splitting into all those json fragments.  That way a
sysroot update will just touch a single file.

I would also drop the multi-processing.

https://codereview.chromium.org/2567123002/diff/1/build/linux/sysroot_scripts...
File build/linux/sysroot_scripts/build_and_upload.py (right):

https://codereview.chromium.org/2567123002/diff/1/build/linux/sysroot_scripts...
build/linux/sysroot_scripts/build_and_upload.py:38: revision = rand_string_32()
On 2016/12/12 18:49:57, Tom Anderson wrote:
> The revision can really be any string, but maybe it should be an increasing
> integer?  Something like distro_release_arch_integer?  Or just the commit hash
> on ToT?
> 
> I probably don't want to leave it random because that means if you run the
> script multiple times, you'd be wasting google storage space.

The revision we've been using previously has been the git SHA of the version of
chrome that was used to build in the image.

In theory it should possible to checkout the same chrome revisions and generate
the exact same tarballs.

Perhaps the best options is use `git rev-parse HEAD`.  In theory one should only
upload from the clean checkout and a revision that is on master, but I don't
think writing such as check is worth it.  Only certain people have upload access
anyway.

https://codereview.chromium.org/2567123002/diff/1/build/linux/sysroot_scripts...
build/linux/sysroot_scripts/build_and_upload.py:65: return stdout.strip()
You probably don't want to throw away stderr do you?

How about just using `subprocess.check_output().strip()` here?

https://codereview.chromium.org/2567123002/diff/1/build/linux/sysroot_scripts...
build/linux/sysroot_scripts/build_and_upload.py:84: proc.join()
Is multiprocessing really helpful here?  I imagine a lot of the stuff is going
to be network bound, and since this is rare operation I think maybe just doing
things synchronously (and showing the output as it happens) might be better.

[Tom (Use chromium acct), 2016-12-12 21:19:34]

On 2016/12/12 19:18:00, Sam Clegg wrote:
> I like some parts of this CL.  Is there some way we could keep it as as single
> json file rather than splitting into all those json fragments.  That way a
> sysroot update will just touch a single file.
> 

Added sysroots.json and removed all sysroot.*

> I would also drop the multi-processing.
> 

I disagree, please see my below comment for the reasoning

https://codereview.chromium.org/2567123002/diff/1/build/linux/sysroot_scripts...
File build/linux/sysroot_scripts/build_and_upload.py (right):

https://codereview.chromium.org/2567123002/diff/1/build/linux/sysroot_scripts...
build/linux/sysroot_scripts/build_and_upload.py:38: revision = rand_string_32()
On 2016/12/12 19:18:00, Sam Clegg wrote:
> On 2016/12/12 18:49:57, Tom Anderson wrote:
> > The revision can really be any string, but maybe it should be an increasing
> > integer?  Something like distro_release_arch_integer?  Or just the commit
hash
> > on ToT?
> > 
> > I probably don't want to leave it random because that means if you run the
> > script multiple times, you'd be wasting google storage space.
> 
> The revision we've been using previously has been the git SHA of the version
of
> chrome that was used to build in the image.
> 
> In theory it should possible to checkout the same chrome revisions and
generate
> the exact same tarballs.
> 
> Perhaps the best options is use `git rev-parse HEAD`.  In theory one should
only
> upload from the clean checkout and a revision that is on master, but I don't
> think writing such as check is worth it.  Only certain people have upload
access
> anyway.

Done.

https://codereview.chromium.org/2567123002/diff/1/build/linux/sysroot_scripts...
build/linux/sysroot_scripts/build_and_upload.py:65: return stdout.strip()
On 2016/12/12 19:18:00, Sam Clegg wrote:
> You probably don't want to throw away stderr do you?
> 
> How about just using `subprocess.check_output().strip()` here?

Done.

https://codereview.chromium.org/2567123002/diff/1/build/linux/sysroot_scripts...
build/linux/sysroot_scripts/build_and_upload.py:84: proc.join()
On 2016/12/12 19:17:59, Sam Clegg wrote:
> Is multiprocessing really helpful here?  I imagine a lot of the stuff is going
> to be network bound, and since this is rare operation I think maybe just doing
> things synchronously (and showing the output as it happens) might be better.

It is helpful.  I created this test script:

#! /bin/bash
set -o errexit
./sysroot-creator-precise.sh UpdatePackageListsAll >/dev/null 2>/dev/null
./sysroot-creator-precise.sh BuildSysrootAll >/dev/null 2>/dev/null
./sysroot-creator-precise.sh UploadSysrootAll test_revision_please_delete
>/dev/null 2>/dev/null
./sysroot-creator-trusty.sh UpdatePackageListsAll >/dev/null 2>/dev/null
./sysroot-creator-trusty.sh BuildSysrootAll >/dev/null 2>/dev/null
./sysroot-creator-trusty.sh UploadSysrootAll test_revision_please_delete
>/dev/null 2>/dev/null
./sysroot-creator-wheezy.sh UpdatePackageListsAll >/dev/null 2>/dev/null
./sysroot-creator-wheezy.sh BuildSysrootAll >/dev/null 2>/dev/null
./sysroot-creator-wheezy.sh UploadSysrootAll test_revision_please_delete
>/dev/null 2>/dev/null
./sysroot-creator-jessie.sh UpdatePackageListsAll >/dev/null 2>/dev/null
./sysroot-creator-jessie.sh BuildSysrootAll >/dev/null 2>/dev/null
./sysroot-creator-jessie.sh UploadSysrootAll test_revision_please_delete
>/dev/null 2>/dev/null

I deleted sysroot-build and ran it, and it took 17m42s to finish.
For build_and_upload.py, I also deleted sysroot-build and ran it, but it only
took 2m20s.

If any script fails, a stack trace will be printed which shows the script and
args that failed and you can run that manually to see the output.  Maybe in the
sysroot creation summary, we can just give a message saying to run the failing
scripts manually to see the output (or even save the log to a file)?  The
intended use-case for this script is just before you're ready to upload, which
is why I didn't add an option to disable upload.  I expect if you're making
package changes, you'll be running just the script/step you need manually anyway

[Tom (Use chromium acct), 2016-12-12 21:21:37]

The CQ bit was checked by thomasanderson@google.com to run a CQ dry run

[commit-bot: I haz the power, 2016-12-12 21:22:41]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[Sam Clegg, 2016-12-12 22:11:52]

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
File build/linux/sysroot_scripts/build_and_upload.py (right):

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/build_and_upload.py:60: try:
I seems like "with lock:" should work here.

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/build_and_upload.py:68: f.close()
No need with close here right?

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/build_and_upload.py:79: continue

How about just:

for filename in glob.glob(os.path.join(script_dir, 'sysroot-creator-*.sh')):

Or even just hardcode the list at the top of the file, since there should really
only be one at a given time (hopefully we will be dropping both trust and wheezy
ASAP)?

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/build_and_upload.py:94: for (name, proc) in procs:
nit: no need to braces when unpacking tuples like this (and 3 lines above)

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/build_and_upload.py:99: build_and_upload_all()
The pattern I've prefer is "sys.exit(main())" here, and return the status code
from main.

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
File build/linux/sysroot_scripts/install-sysroot.py (right):

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/install-sysroot.py:184: sysroots_file =
os.path.join(os.path.dirname(os.path.realpath(__file__)),
Use SCRIPT_DIR here?

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/install-sysroot.py:187: sysroot_dict =
sysroots['%s_%s' % (target_platform.lower(), target_arch)]
Should see keep the error for the case when the key is not in the sysroots dict?

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
File build/linux/sysroot_scripts/sysroot-creator.sh (right):

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/sysroot-creator.sh:110: mv "${2}.partial.$$" $2
Under what circumstances might that happen?   Shouldn't every instance doesn't
distinct files?

If two instances are racing to create $2 isn't that still a problem?

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/sysroot-creator.sh:862: }
Are any of these new Dump* functions used anywhere?

[commit-bot: I haz the power, 2016-12-12 22:38:55]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-12-12 22:38:56]

Dry run: This issue passed the CQ dry run.

[Tom (Use chromium acct), 2016-12-13 00:06:19]

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
File build/linux/sysroot_scripts/build_and_upload.py (right):

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/build_and_upload.py:60: try:
On 2016/12/12 22:11:52, Sam Clegg wrote:
> I seems like "with lock:" should work here.

Done.

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/build_and_upload.py:68: f.close()
On 2016/12/12 22:11:52, Sam Clegg wrote:
> No need with close here right?

Done.

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/build_and_upload.py:79: continue
On 2016/12/12 22:11:52, Sam Clegg wrote:
> 
> How about just:
> 
> for filename in glob.glob(os.path.join(script_dir, 'sysroot-creator-*.sh')):
> 

Done

> Or even just hardcode the list at the top of the file, since there should
really
> only be one at a given time (hopefully we will be dropping both trust and
wheezy
> ASAP)?

Yeah, it seems with the switch to jessie some of the sysroots are going to
become obsolete.  I didn't hardcode the list so that we don't have to maintain
it

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/build_and_upload.py:94: for (name, proc) in procs:
On 2016/12/12 22:11:51, Sam Clegg wrote:
> nit: no need to braces when unpacking tuples like this (and 3 lines above)

Done.

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/build_and_upload.py:99: build_and_upload_all()
On 2016/12/12 22:11:52, Sam Clegg wrote:
> The pattern I've prefer is "sys.exit(main())" here, and return the status code
> from main.

Done.

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
File build/linux/sysroot_scripts/install-sysroot.py (right):

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/install-sysroot.py:184: sysroots_file =
os.path.join(os.path.dirname(os.path.realpath(__file__)),
On 2016/12/12 22:11:52, Sam Clegg wrote:
> Use SCRIPT_DIR here?

Done.

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/install-sysroot.py:187: sysroot_dict =
sysroots['%s_%s' % (target_platform.lower(), target_arch)]
On 2016/12/12 22:11:52, Sam Clegg wrote:
> Should see keep the error for the case when the key is not in the sysroots
dict?

Oops, forgot about that.  Done

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
File build/linux/sysroot_scripts/sysroot-creator.sh (right):

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/sysroot-creator.sh:110: mv "${2}.partial.$$" $2
On 2016/12/12 22:11:52, Sam Clegg wrote:
> Under what circumstances might that happen?   Shouldn't every instance doesn't
> distinct files?
> 

It happens when downloading eg jessie-Release and jessie-Release.gpg, and when
packages are the same across architectures, like
debian-packages/x11proto-composite-dev_0.4.2-2_all.deb

> If two instances are racing to create $2 isn't that still a problem?

No because mv is atomic

https://codereview.chromium.org/2567123002/diff/20001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/sysroot-creator.sh:862: }
On 2016/12/12 22:11:52, Sam Clegg wrote:
> Are any of these new Dump* functions used anywhere?

No, forgot to remove them :)

[Sam Clegg, 2016-12-13 00:30:36]

sbc@chromium.org changed reviewers:
+ thestig@chromium.org

[Sam Clegg, 2016-12-13 00:30:37]

OK, this mostly lgtm now.  I'm still not a huge fan of the use of
multiprocessing or the piping of subprocess output to /dev/null.

+thestig who has done work on the sysroots before and might have opinions.

https://codereview.chromium.org/2567123002/diff/40001/build/linux/sysroot_scr...
File build/linux/sysroot_scripts/build_and_upload.py (right):

https://codereview.chromium.org/2567123002/diff/40001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/build_and_upload.py:91: print "%s sysroot
creation\t%s" % (name, status)
Shouldn't function return 1 if any proc.exitcode is non-zero? (and maybe just be
called main?)

https://codereview.chromium.org/2567123002/diff/40001/build/linux/sysroot_scr...
File build/linux/sysroot_scripts/sysroots.json (right):

https://codereview.chromium.org/2567123002/diff/40001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/sysroots.json:63: "Revision":
"8a2cf659459c3615093c4c0890270b0c8a72e987",
nit: Is it worth keeping these revisions the same for this CL so that the
sysroots themselves don't actually change, and then creating a seperate CL that
that actually updates them?

[Tom (Use chromium acct), 2016-12-13 02:33:09]

https://codereview.chromium.org/2567123002/diff/40001/build/linux/sysroot_scr...
File build/linux/sysroot_scripts/build_and_upload.py (right):

https://codereview.chromium.org/2567123002/diff/40001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/build_and_upload.py:91: print "%s sysroot
creation\t%s" % (name, status)
On 2016/12/13 00:30:37, Sam Clegg wrote:
> Shouldn't function return 1 if any proc.exitcode is non-zero? (and maybe just
be
> called main?)

Done.

https://codereview.chromium.org/2567123002/diff/40001/build/linux/sysroot_scr...
File build/linux/sysroot_scripts/sysroots.json (right):

https://codereview.chromium.org/2567123002/diff/40001/build/linux/sysroot_scr...
build/linux/sysroot_scripts/sysroots.json:63: "Revision":
"8a2cf659459c3615093c4c0890270b0c8a72e987",
On 2016/12/13 00:30:37, Sam Clegg wrote:
> nit: Is it worth keeping these revisions the same for this CL so that the
> sysroots themselves don't actually change, and then creating a seperate CL
that
> that actually updates them?

no imo.  Maybe dpranke@ will have a stronger opinion

[Sam Clegg, 2016-12-13 16:45:11]

On 2016/12/13 02:33:09, Tom Anderson wrote:
>
https://codereview.chromium.org/2567123002/diff/40001/build/linux/sysroot_scr...
> File build/linux/sysroot_scripts/build_and_upload.py (right):
> 
>
https://codereview.chromium.org/2567123002/diff/40001/build/linux/sysroot_scr...
> build/linux/sysroot_scripts/build_and_upload.py:91: print "%s sysroot
> creation\t%s" % (name, status)
> On 2016/12/13 00:30:37, Sam Clegg wrote:
> > Shouldn't function return 1 if any proc.exitcode is non-zero? (and maybe
just
> be
> > called main?)
> 
> Done.
> 
>
https://codereview.chromium.org/2567123002/diff/40001/build/linux/sysroot_scr...
> File build/linux/sysroot_scripts/sysroots.json (right):
> 
>
https://codereview.chromium.org/2567123002/diff/40001/build/linux/sysroot_scr...
> build/linux/sysroot_scripts/sysroots.json:63: "Revision":
> "8a2cf659459c3615093c4c0890270b0c8a72e987",
> On 2016/12/13 00:30:37, Sam Clegg wrote:
> > nit: Is it worth keeping these revisions the same for this CL so that the
> > sysroots themselves don't actually change, and then creating a seperate CL
> that
> > that actually updates them?
> 
> no imo.  Maybe dpranke@ will have a stronger opinion

In that case you might want to mention that this CL rolls all the sysroots as
well creating the script to roll them.

I don't have OWNERS powers here so lets wait for the thestig@ for approval.

[Tom (Use chromium acct), 2016-12-13 18:29:16]

Description was changed from

==========
Sysroot: Add build_and_upload.py

This CL adds a new script that automates building and uploading new
sysroot images.  Previously, you had to run UpdatePackageListsAll,
BuildSysrootAll, and UploadSysrootAll for each of the 4
sysroot-creators.  Then, you had to manually update the revision and
sha1sum in install_sysroot.py.  Now, simply run build_and_upload.py
and your CL should be ready to go.  In addition, the script
concurrently updates each sysroot image (there are currently 11 of
them), greatly reducing the time it took to update the sysroots.

R=dpranke@chromium.org
==========

to

==========
Sysroot: Add build_and_upload.py

This CL adds a new script that automates building and uploading new
sysroot images.  Previously, you had to run UpdatePackageListsAll,
BuildSysrootAll, and UploadSysrootAll for each of the 4
sysroot-creators.  Then, you had to manually update the revision and
sha1sum in install_sysroot.py.  Now, simply run build_and_upload.py
and your CL should be ready to go.  In addition, the script
concurrently updates each sysroot image (there are currently 11 of
them), greatly reducing the time it took to update the sysroots.

In addition to adding the script, this CL also rolls all sysroots.

R=dpranke@chromium.org
==========

[Tom (Use chromium acct), 2016-12-14 03:39:08]

pinging dpranke@ (thestig@ is OOO)

On 2016/12/13 16:45:11, Sam Clegg wrote:
> On 2016/12/13 02:33:09, Tom Anderson wrote:
>
https://codereview.chromium.org/2567123002/diff/40001/build/linux/sysroot_scr...
> > build/linux/sysroot_scripts/sysroots.json:63: "Revision":
> > "8a2cf659459c3615093c4c0890270b0c8a72e987",
> > On 2016/12/13 00:30:37, Sam Clegg wrote:
> > > nit: Is it worth keeping these revisions the same for this CL so that the
> > > sysroots themselves don't actually change, and then creating a seperate CL
> > that
> > > that actually updates them?
> > 
> > no imo.  Maybe dpranke@ will have a stronger opinion
> 
> In that case you might want to mention that this CL rolls all the sysroots as
> well creating the script to roll them.
> 

done

> I don't have OWNERS powers here so lets wait for the thestig@ for approval.

[Dirk Pranke, 2016-12-15 01:55:23]

lgtm

[Tom (Use chromium acct), 2016-12-15 01:56:04]

The CQ bit was checked by thomasanderson@google.com

[Tom (Use chromium acct), 2016-12-15 01:56:04]

The patchset sent to the CQ was uploaded after l-g-t-m from sbc@chromium.org
Link to the patchset: https://codereview.chromium.org/2567123002/#ps60001
(title: "Return non-zero exit code if any sysroot builds failed")

[commit-bot: I haz the power, 2016-12-15 01:56:13]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-12-15 02:03:25]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-12-15 02:03:26]

Try jobs failed on following builders:
  win_chromium_x64_rel_ng on master.tryserver.chromium.win (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_...)
  win_clang on master.tryserver.chromium.win (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.win/builders/win_clang/builds/...)

[Tom (Use chromium acct), 2016-12-15 02:26:15]

The CQ bit was checked by thomasanderson@google.com to run a CQ dry run

[commit-bot: I haz the power, 2016-12-15 02:26:25]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-12-15 02:32:35]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-12-15 02:32:36]

Dry run: Try jobs failed on following builders:
  win_chromium_x64_rel_ng on master.tryserver.chromium.win (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_...)
  win_clang on master.tryserver.chromium.win (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.win/builders/win_clang/builds/...)

[Tom (Use chromium acct), 2016-12-15 17:22:06]

The CQ bit was checked by thomasanderson@google.com

[commit-bot: I haz the power, 2016-12-15 17:22:29]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-12-15 17:34:47]

CQ is committing da patch.

Bot data: {"patchset_id": 60001, "attempt_start_ts": 1481822526456310,
"parent_rev": "e1ae02632ce12e98b89b74dbf4319ba166eadc13", "commit_rev":
"e6756d140e7f520cf63465ab3ba5b12e1278dd6b"}

[commit-bot: I haz the power, 2016-12-15 17:35:12]

Description was changed from

==========
Sysroot: Add build_and_upload.py

This CL adds a new script that automates building and uploading new
sysroot images.  Previously, you had to run UpdatePackageListsAll,
BuildSysrootAll, and UploadSysrootAll for each of the 4
sysroot-creators.  Then, you had to manually update the revision and
sha1sum in install_sysroot.py.  Now, simply run build_and_upload.py
and your CL should be ready to go.  In addition, the script
concurrently updates each sysroot image (there are currently 11 of
them), greatly reducing the time it took to update the sysroots.

In addition to adding the script, this CL also rolls all sysroots.

R=dpranke@chromium.org
==========

to

==========
Sysroot: Add build_and_upload.py

This CL adds a new script that automates building and uploading new
sysroot images.  Previously, you had to run UpdatePackageListsAll,
BuildSysrootAll, and UploadSysrootAll for each of the 4
sysroot-creators.  Then, you had to manually update the revision and
sha1sum in install_sysroot.py.  Now, simply run build_and_upload.py
and your CL should be ready to go.  In addition, the script
concurrently updates each sysroot image (there are currently 11 of
them), greatly reducing the time it took to update the sysroots.

In addition to adding the script, this CL also rolls all sysroots.

R=dpranke@chromium.org

Review-Url: https://codereview.chromium.org/2567123002
==========

[commit-bot: I haz the power, 2016-12-15 17:35:15]

Committed patchset #4 (id:60001)

[commit-bot: I haz the power, 2016-12-15 17:38:52]

Description was changed from

==========
Sysroot: Add build_and_upload.py

This CL adds a new script that automates building and uploading new
sysroot images.  Previously, you had to run UpdatePackageListsAll,
BuildSysrootAll, and UploadSysrootAll for each of the 4
sysroot-creators.  Then, you had to manually update the revision and
sha1sum in install_sysroot.py.  Now, simply run build_and_upload.py
and your CL should be ready to go.  In addition, the script
concurrently updates each sysroot image (there are currently 11 of
them), greatly reducing the time it took to update the sysroots.

In addition to adding the script, this CL also rolls all sysroots.

R=dpranke@chromium.org

Review-Url: https://codereview.chromium.org/2567123002
==========

to

==========
Sysroot: Add build_and_upload.py

This CL adds a new script that automates building and uploading new
sysroot images.  Previously, you had to run UpdatePackageListsAll,
BuildSysrootAll, and UploadSysrootAll for each of the 4
sysroot-creators.  Then, you had to manually update the revision and
sha1sum in install_sysroot.py.  Now, simply run build_and_upload.py
and your CL should be ready to go.  In addition, the script
concurrently updates each sysroot image (there are currently 11 of
them), greatly reducing the time it took to update the sysroots.

In addition to adding the script, this CL also rolls all sysroots.

R=dpranke@chromium.org

Committed: https://crrev.com/4c8b67057de3ee3f6c4105a0dceebc0228d9832e
Cr-Commit-Position: refs/heads/master@{#438862}
==========

[commit-bot: I haz the power, 2016-12-15 17:38:53]

Patchset 4 (id:??) landed as
https://crrev.com/4c8b67057de3ee3f6c4105a0dceebc0228d9832e
Cr-Commit-Position: refs/heads/master@{#438862}