Add repo sync all helper for non-pfq buildbots.

Add repo sync all helper for non-pfq buildbots.

TEST=Ran script and re-ran cbuildbot to completion of sync.

Change-Id: I31fddca6f720fecb461a96c1ffa54d4923b3c1c4

[sosa, 2010-08-18 22:50:01]

Buildbot needs update step to be one command line

[sosa, 2010-08-19 02:41:18]

Adding scott

[scottz-goog, 2010-08-19 03:08:09]

I understand time crunch but I made a few suggestions/comments.

http://codereview.chromium.org/3166019/diff/1/2
File bin/cbuildbot.py (right):

http://codereview.chromium.org/3166019/diff/1/2#newcode57
bin/cbuildbot.py:57: RunCommand(['sudo', 'rm', '-rf', buildroot])
The only thing that worries me here is that buildroot is user defined and we are
whacking it with sudo...I assume there is no way around this?

http://codereview.chromium.org/3166019/diff/1/3
File bin/cros_repo_sync_all (right):

http://codereview.chromium.org/3166019/diff/1/3#newcode1
bin/cros_repo_sync_all:1: cros_repo_sync_all.py
When this is called we will have cwd set to this dir right? 

./cros_repo_sync_all.py

http://codereview.chromium.org/3166019/diff/1/4
File bin/cros_repo_sync_all.py (right):

http://codereview.chromium.org/3166019/diff/1/4#newcode19
bin/cros_repo_sync_all.py:19: cbuildbot._FullCheckout(options.buildroot,
rw_checkout=False)
can we catch the exception and do a few retries? I am not asking for anything
fancy but the errors I have seen with repo basically go away if you just try
syncing again.

[sosa, 2010-08-19 03:33:42]

PTAL.  In addition to retries, added a clobber option that is passed from the
buildbot.

http://codereview.chromium.org/3166019/diff/1/2
File bin/cbuildbot.py (right):

http://codereview.chromium.org/3166019/diff/1/2#newcode57
bin/cbuildbot.py:57: RunCommand(['sudo', 'rm', '-rf', buildroot])
No not really.  If there is an old checkout and it built anything (i.e. has a
chroot) you need sudo.  On the other hand you can't be guaranteed you have
anything (repo sync could have failed immediately on the last run ... so I can't
really check reliably that this was an old checkout directory)


On 2010/08/19 03:08:10, scottz wrote:
> The only thing that worries me here is that buildroot is user defined and we
are
> whacking it with sudo...I assume there is no way around this?

http://codereview.chromium.org/3166019/diff/1/3
File bin/cros_repo_sync_all (right):

http://codereview.chromium.org/3166019/diff/1/3#newcode1
bin/cros_repo_sync_all:1: cros_repo_sync_all.py
This is just a symlink. 

On 2010/08/19 03:08:10, scottz wrote:
> When this is called we will have cwd set to this dir right? 
> 
> ./cros_repo_sync_all.py

http://codereview.chromium.org/3166019/diff/1/4
File bin/cros_repo_sync_all.py (right):

http://codereview.chromium.org/3166019/diff/1/4#newcode19
bin/cros_repo_sync_all.py:19: cbuildbot._FullCheckout(options.buildroot,
rw_checkout=False)
Great idea :)

On 2010/08/19 03:08:10, scottz wrote:
> can we catch the exception and do a few retries? I am not asking for anything
> fancy but the errors I have seen with repo basically go away if you just try
> syncing again.

[sosa, 2010-08-19 03:46:18]

Sorry PTAL.  Somehow re-committed without re-uploading before my last PTAL

[scottz-goog, 2010-08-19 04:00:03]

LGTM.

We have 3 defaults for retries. I suggest picking on place and have either the
other functions require it or just use the default one of them sets.

other than that great thanks for adding that :)

http://codereview.chromium.org/3166019/diff/1/2
File bin/cbuildbot.py (right):

http://codereview.chromium.org/3166019/diff/1/2#newcode57
bin/cbuildbot.py:57: RunCommand(['sudo', 'rm', '-rf', buildroot])
Yeah that is true, it is a bad assumption we have everywhere. So be it. 

On 2010/08/19 03:33:42, sosa wrote:
> No not really.  If there is an old checkout and it built anything (i.e. has a
> chroot) you need sudo.  On the other hand you can't be guaranteed you have
> anything (repo sync could have failed immediately on the last run ... so I
can't
> really check reliably that this was an old checkout directory)
> 
> 
> On 2010/08/19 03:08:10, scottz wrote:
> > The only thing that worries me here is that buildroot is user defined and we
> are
> > whacking it with sudo...I assume there is no way around this?
> 
>

http://codereview.chromium.org/3166019/diff/1/3
File bin/cros_repo_sync_all (right):

http://codereview.chromium.org/3166019/diff/1/3#newcode1
bin/cros_repo_sync_all:1: cros_repo_sync_all.py
Ahh hah sorry I am not used to seeing those in this interface. That makes sense.

On 2010/08/19 03:33:42, sosa wrote:
> This is just a symlink. 
> 
> On 2010/08/19 03:08:10, scottz wrote:
> > When this is called we will have cwd set to this dir right? 
> > 
> > ./cros_repo_sync_all.py 
> 
>