Add ability to run smoke_Suite on x86-pre-flight-queue.

Add methods to run vm tests on x86-pre-flight-queue.

Right now we're leaving error_ok=True until the tests are shown to work on builder.

Change-Id: I6b8c690a0da30948389fd4312032c78d87115364

BUG=6906
TEST=Ran through it locally.

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=c2db26c

[rtc, 2010-09-30 03:32:07]

http://codereview.chromium.org/3591002/diff/2001/3001
File bin/cbuildbot.py (right):

http://codereview.chromium.org/3591002/diff/2001/3001#newcode292
bin/cbuildbot.py:292: '--test_case',
I think you need to pass --image_path as well.

[sosa, 2010-09-30 04:09:32]

http://codereview.chromium.org/3591002/diff/2001/3001
File bin/cbuildbot.py (right):

http://codereview.chromium.org/3591002/diff/2001/3001#newcode292
bin/cbuildbot.py:292: '--test_case',
I modified the default behavior without image_path to do the same as the other
scripts (use the last) in http://codereview.chromium.org/3597001/show.

I can be explicit, but I was worried that adding logic to add the latest here
would unnecessarily complicate this call

On 2010/09/30 03:32:07, rtc wrote:
> I think you need to pass --image_path as well.

[rtc, 2010-09-30 04:39:23]

LGTM on the tests. I can't speak for the buildbot config since I don't know
how they work. Maybe ask Scottz to take a quick look.

On Wed, Sep 29, 2010 at 9:09 PM, <sosa@chromium.org> wrote:

>
> http://codereview.chromium.org/3591002/diff/2001/3001
> File bin/cbuildbot.py (right):
>
> http://codereview.chromium.org/3591002/diff/2001/3001#newcode292
> bin/cbuildbot.py:292: '--test_case',
> I modified the default behavior without image_path to do the same as the
> other scripts (use the last) in
> http://codereview.chromium.org/3597001/show.
>
> I can be explicit, but I was worried that adding logic to add the latest
> here would unnecessarily complicate this call
>
>
> On 2010/09/30 03:32:07, rtc wrote:
>
>> I think you need to pass --image_path as well.
>>
>
> http://codereview.chromium.org/3591002/show
>

[sosa, 2010-09-30 07:27:59]

+scottz for the config.  This is a follow up for getting kvm on the builders.

[scottz, 2010-09-30 16:04:50]

Thanks for picking this up off of the floor!

Just a few questions inline. 

I thought we wanted to run this on all targets? It takes ~5 minutes right? It is
possible this changed since we last talked about it.

http://codereview.chromium.org/3591002/diff/2001/3001
File bin/cbuildbot.py (right):

http://codereview.chromium.org/3591002/diff/2001/3001#newcode218
bin/cbuildbot.py:218: redirect_stdout=True)
There is no better way to do this? 

Also shell=True won't handle this properly? you have to call bash -c? 

What sets the cros_vm_constants.sh?

http://codereview.chromium.org/3591002/diff/2001/3001#newcode423
bin/cbuildbot.py:423: if buildconfig['smoke']:
smoke_bvt or smoke_kvm? Something that is a bit more descriptive.

http://codereview.chromium.org/3591002/diff/2001/3002
File bin/cbuildbot_config.py (right):

http://codereview.chromium.org/3591002/diff/2001/3002#newcode21
bin/cbuildbot_config.py:21: smoke -- Runs the test smoke suite in a qemu-based
VM using KVM.
As per the other file, something more descriptive than smoke? smoke_qemu,
smoke_kvm, smoke_bvt. Any one of those.

http://codereview.chromium.org/3591002/diff/2001/3003
File lib/cros_build_lib.py (right):

http://codereview.chromium.org/3591002/diff/2001/3003#newcode56
lib/cros_build_lib.py:56: (GetCallerName(), ' '.join(cmd), cwd))
We get all this for free with the python logging module. We really need to just
switch to that. A TODO should be added.

[sosa, 2010-09-30 18:21:06]

We do want to run this on all images BUT autotest doesn't run on all image types
(all arm) yet.  We'll have to block on that until that's ready for the other
platforms.  Also, are all the builders on Lucid yet?  This will only work on
Lucid (which I know chromeosbuild3 is on)

http://codereview.chromium.org/3591002/diff/2001/3001
File bin/cbuildbot.py (right):

http://codereview.chromium.org/3591002/diff/2001/3001#newcode218
bin/cbuildbot.py:218: redirect_stdout=True)
This run command doesn't have shell= ability so I decided not to add it since
this is the only one use.

The shell command also just uses your shell so it may not be bash.

Given that i thought this was the best way.  cros_vm_constants.sh is a shared
file between vm scripts and now this script.

On 2010/09/30 16:04:51, scottz wrote:
> There is no better way to do this? 
> 
> Also shell=True won't handle this properly? you have to call bash -c? 
> 
> What sets the cros_vm_constants.sh?

http://codereview.chromium.org/3591002/diff/2001/3001#newcode423
bin/cbuildbot.py:423: if buildconfig['smoke']:
On 2010/09/30 16:04:51, scottz wrote:
> smoke_bvt or smoke_kvm? Something that is a bit more descriptive. 

Done.

http://codereview.chromium.org/3591002/diff/2001/3002
File bin/cbuildbot_config.py (right):

http://codereview.chromium.org/3591002/diff/2001/3002#newcode21
bin/cbuildbot_config.py:21: smoke -- Runs the test smoke suite in a qemu-based
VM using KVM.
On 2010/09/30 16:04:51, scottz wrote:
> As per the other file, something more descriptive than smoke? smoke_qemu,
> smoke_kvm, smoke_bvt. Any one of those. 

Done.

http://codereview.chromium.org/3591002/diff/2001/3003
File lib/cros_build_lib.py (right):

http://codereview.chromium.org/3591002/diff/2001/3003#newcode56
lib/cros_build_lib.py:56: (GetCallerName(), ' '.join(cmd), cwd))
On 2010/09/30 16:04:51, scottz wrote:
> We get all this for free with the python logging module. We really need to
just
> switch to that. A TODO should be added. 

Done.

[rtc, 2010-09-30 18:23:50]

On Thu, Sep 30, 2010 at 11:21 AM, <sosa@chromium.org> wrote:

> We do want to run this on all images BUT autotest doesn't run on all image
> types
> (all arm) yet.  We'll have to block on that until that's ready for the
> other
> platforms.  Also, are all the builders on Lucid yet?  This will only work
> on
> Lucid (which I know chromeosbuild3 is on)


We can still put it on all x86 targets though, right? Can we enable this
without breaking the build on test failures for a few days so we can gain
some confidence that it is working as intended?

-Ryan



>
>
>
> http://codereview.chromium.org/3591002/diff/2001/3001
> File bin/cbuildbot.py (right):
>
> http://codereview.chromium.org/3591002/diff/2001/3001#newcode218
> bin/cbuildbot.py:218: redirect_stdout=True)
> This run command doesn't have shell= ability so I decided not to add it
> since this is the only one use.
>
> The shell command also just uses your shell so it may not be bash.
>
> Given that i thought this was the best way.  cros_vm_constants.sh is a
> shared file between vm scripts and now this script.
>
>
> On 2010/09/30 16:04:51, scottz wrote:
>
>> There is no better way to do this?
>>
>
>  Also shell=True won't handle this properly? you have to call bash -c?
>>
>
>  What sets the cros_vm_constants.sh?
>>
>
> http://codereview.chromium.org/3591002/diff/2001/3001#newcode423
> bin/cbuildbot.py:423: if buildconfig['smoke']:
> On 2010/09/30 16:04:51, scottz wrote:
>
>> smoke_bvt or smoke_kvm? Something that is a bit more descriptive.
>>
>
> Done.
>
>
> http://codereview.chromium.org/3591002/diff/2001/3002
> File bin/cbuildbot_config.py (right):
>
> http://codereview.chromium.org/3591002/diff/2001/3002#newcode21
> bin/cbuildbot_config.py:21: smoke -- Runs the test smoke suite in a
> qemu-based VM using KVM.
> On 2010/09/30 16:04:51, scottz wrote:
>
>> As per the other file, something more descriptive than smoke?
>>
> smoke_qemu,
>
>> smoke_kvm, smoke_bvt. Any one of those.
>>
>
> Done.
>
>
> http://codereview.chromium.org/3591002/diff/2001/3003
> File lib/cros_build_lib.py (right):
>
> http://codereview.chromium.org/3591002/diff/2001/3003#newcode56
> lib/cros_build_lib.py:56: (GetCallerName(), ' '.join(cmd), cwd))
> On 2010/09/30 16:04:51, scottz wrote:
>
>> We get all this for free with the python logging module. We really
>>
> need to just
>
>> switch to that. A TODO should be added.
>>
>
> Done.
>
>
> http://codereview.chromium.org/3591002/show
>

[sosa, 2010-09-30 18:27:06]

Yes this is true.  I'll update the config.

I set error_ok=True in RunCommand for the suite_Smoke to let us bypass errors.

On Thu, Sep 30, 2010 at 11:23 AM, Ryan Cairns <rtc@chromium.org> wrote:
>
>
> On Thu, Sep 30, 2010 at 11:21 AM, <sosa@chromium.org> wrote:
>>
>> We do want to run this on all images BUT autotest doesn't run on all image
>> types
>> (all arm) yet.  We'll have to block on that until that's ready for the
>> other
>> platforms.  Also, are all the builders on Lucid yet?  This will only work
>> on
>> Lucid (which I know chromeosbuild3 is on)
>
> We can still put it on all x86 targets though, right? Can we enable this
> without breaking the build on test failures for a few days so we can gain
> some confidence that it is working as intended?
> -Ryan
>
>>
>>
>> http://codereview.chromium.org/3591002/diff/2001/3001
>> File bin/cbuildbot.py (right):
>>
>> http://codereview.chromium.org/3591002/diff/2001/3001#newcode218
>> bin/cbuildbot.py:218: redirect_stdout=True)
>> This run command doesn't have shell= ability so I decided not to add it
>> since this is the only one use.
>>
>> The shell command also just uses your shell so it may not be bash.
>>
>> Given that i thought this was the best way.  cros_vm_constants.sh is a
>> shared file between vm scripts and now this script.
>>
>> On 2010/09/30 16:04:51, scottz wrote:
>>>
>>> There is no better way to do this?
>>
>>> Also shell=True won't handle this properly? you have to call bash -c?
>>
>>> What sets the cros_vm_constants.sh?
>>
>> http://codereview.chromium.org/3591002/diff/2001/3001#newcode423
>> bin/cbuildbot.py:423: if buildconfig['smoke']:
>> On 2010/09/30 16:04:51, scottz wrote:
>>>
>>> smoke_bvt or smoke_kvm? Something that is a bit more descriptive.
>>
>> Done.
>>
>> http://codereview.chromium.org/3591002/diff/2001/3002
>> File bin/cbuildbot_config.py (right):
>>
>> http://codereview.chromium.org/3591002/diff/2001/3002#newcode21
>> bin/cbuildbot_config.py:21: smoke -- Runs the test smoke suite in a
>> qemu-based VM using KVM.
>> On 2010/09/30 16:04:51, scottz wrote:
>>>
>>> As per the other file, something more descriptive than smoke?
>>
>> smoke_qemu,
>>>
>>> smoke_kvm, smoke_bvt. Any one of those.
>>
>> Done.
>>
>> http://codereview.chromium.org/3591002/diff/2001/3003
>> File lib/cros_build_lib.py (right):
>>
>> http://codereview.chromium.org/3591002/diff/2001/3003#newcode56
>> lib/cros_build_lib.py:56: (GetCallerName(), ' '.join(cmd), cwd))
>> On 2010/09/30 16:04:51, scottz wrote:
>>>
>>> We get all this for free with the python logging module. We really
>>
>> need to just
>>>
>>> switch to that. A TODO should be added.
>>
>> Done.
>>
>> http://codereview.chromium.org/3591002/show
>
>

[sosa, 2010-09-30 18:32:46]

Done. PTAL

On Thu, Sep 30, 2010 at 11:27 AM, Chris Sosa <sosa@chromium.org> wrote:
> Yes this is true.  I'll update the config.
>
> I set error_ok=True in RunCommand for the suite_Smoke to let us bypass errors.
>
> On Thu, Sep 30, 2010 at 11:23 AM, Ryan Cairns <rtc@chromium.org> wrote:
>>
>>
>> On Thu, Sep 30, 2010 at 11:21 AM, <sosa@chromium.org> wrote:
>>>
>>> We do want to run this on all images BUT autotest doesn't run on all image
>>> types
>>> (all arm) yet.  We'll have to block on that until that's ready for the
>>> other
>>> platforms.  Also, are all the builders on Lucid yet?  This will only work
>>> on
>>> Lucid (which I know chromeosbuild3 is on)
>>
>> We can still put it on all x86 targets though, right? Can we enable this
>> without breaking the build on test failures for a few days so we can gain
>> some confidence that it is working as intended?
>> -Ryan
>>
>>>
>>>
>>> http://codereview.chromium.org/3591002/diff/2001/3001
>>> File bin/cbuildbot.py (right):
>>>
>>> http://codereview.chromium.org/3591002/diff/2001/3001#newcode218
>>> bin/cbuildbot.py:218: redirect_stdout=True)
>>> This run command doesn't have shell= ability so I decided not to add it
>>> since this is the only one use.
>>>
>>> The shell command also just uses your shell so it may not be bash.
>>>
>>> Given that i thought this was the best way.  cros_vm_constants.sh is a
>>> shared file between vm scripts and now this script.
>>>
>>> On 2010/09/30 16:04:51, scottz wrote:
>>>>
>>>> There is no better way to do this?
>>>
>>>> Also shell=True won't handle this properly? you have to call bash -c?
>>>
>>>> What sets the cros_vm_constants.sh?
>>>
>>> http://codereview.chromium.org/3591002/diff/2001/3001#newcode423
>>> bin/cbuildbot.py:423: if buildconfig['smoke']:
>>> On 2010/09/30 16:04:51, scottz wrote:
>>>>
>>>> smoke_bvt or smoke_kvm? Something that is a bit more descriptive.
>>>
>>> Done.
>>>
>>> http://codereview.chromium.org/3591002/diff/2001/3002
>>> File bin/cbuildbot_config.py (right):
>>>
>>> http://codereview.chromium.org/3591002/diff/2001/3002#newcode21
>>> bin/cbuildbot_config.py:21: smoke -- Runs the test smoke suite in a
>>> qemu-based VM using KVM.
>>> On 2010/09/30 16:04:51, scottz wrote:
>>>>
>>>> As per the other file, something more descriptive than smoke?
>>>
>>> smoke_qemu,
>>>>
>>>> smoke_kvm, smoke_bvt. Any one of those.
>>>
>>> Done.
>>>
>>> http://codereview.chromium.org/3591002/diff/2001/3003
>>> File lib/cros_build_lib.py (right):
>>>
>>> http://codereview.chromium.org/3591002/diff/2001/3003#newcode56
>>> lib/cros_build_lib.py:56: (GetCallerName(), ' '.join(cmd), cwd))
>>> On 2010/09/30 16:04:51, scottz wrote:
>>>>
>>>> We get all this for free with the python logging module. We really
>>>
>>> need to just
>>>>
>>>> switch to that. A TODO should be added.
>>>
>>> Done.
>>>
>>> http://codereview.chromium.org/3591002/show
>>
>>
>

[scottz, 2010-09-30 18:32:49]

LGTM.

Any system that isn't Lucid I can update as needed. I was waiting to move to the
new lab to bother doing this but we keep getting hit with delays. I say enable
it on all targets that should run it. If that particular slave isn't lucid I
will commit to making it Lucid. 



On 2010/09/30 18:27:06, sosa wrote:
> Yes this is true.  I'll update the config.
> 
> I set error_ok=True in RunCommand for the suite_Smoke to let us bypass errors.
> 
> On Thu, Sep 30, 2010 at 11:23 AM, Ryan Cairns <mailto:rtc@chromium.org> wrote:
> >
> >
> > On Thu, Sep 30, 2010 at 11:21 AM, <mailto:sosa@chromium.org> wrote:
> >>
> >> We do want to run this on all images BUT autotest doesn't run on all image
> >> types
> >> (all arm) yet. &nbsp;We'll have to block on that until that's ready for the
> >> other
> >> platforms. &nbsp;Also, are all the builders on Lucid yet? &nbsp;This will
only work
> >> on
> >> Lucid (which I know chromeosbuild3 is on)
> >
> > We can still put it on all x86 targets though, right? Can we enable this
> > without breaking the build on test failures for a few days so we can gain
> > some confidence that it is working as intended?
> > -Ryan
> >
> >>
> >>
> >> http://codereview.chromium.org/3591002/diff/2001/3001
> >> File bin/cbuildbot.py (right):
> >>
> >> http://codereview.chromium.org/3591002/diff/2001/3001#newcode218
> >> bin/cbuildbot.py:218: redirect_stdout=True)
> >> This run command doesn't have shell= ability so I decided not to add it
> >> since this is the only one use.
> >>
> >> The shell command also just uses your shell so it may not be bash.
> >>
> >> Given that i thought this was the best way. &nbsp;cros_vm_constants.sh is a
> >> shared file between vm scripts and now this script.
> >>
> >> On 2010/09/30 16:04:51, scottz wrote:
> >>>
> >>> There is no better way to do this?
> >>
> >>> Also shell=True won't handle this properly? you have to call bash -c?
> >>
> >>> What sets the cros_vm_constants.sh?
> >>
> >> http://codereview.chromium.org/3591002/diff/2001/3001#newcode423
> >> bin/cbuildbot.py:423: if buildconfig['smoke']:
> >> On 2010/09/30 16:04:51, scottz wrote:
> >>>
> >>> smoke_bvt or smoke_kvm? Something that is a bit more descriptive.
> >>
> >> Done.
> >>
> >> http://codereview.chromium.org/3591002/diff/2001/3002
> >> File bin/cbuildbot_config.py (right):
> >>
> >> http://codereview.chromium.org/3591002/diff/2001/3002#newcode21
> >> bin/cbuildbot_config.py:21: smoke -- Runs the test smoke suite in a
> >> qemu-based VM using KVM.
> >> On 2010/09/30 16:04:51, scottz wrote:
> >>>
> >>> As per the other file, something more descriptive than smoke?
> >>
> >> smoke_qemu,
> >>>
> >>> smoke_kvm, smoke_bvt. Any one of those.
> >>
> >> Done.
> >>
> >> http://codereview.chromium.org/3591002/diff/2001/3003
> >> File lib/cros_build_lib.py (right):
> >>
> >> http://codereview.chromium.org/3591002/diff/2001/3003#newcode56
> >> lib/cros_build_lib.py:56: (GetCallerName(), ' '.join(cmd), cwd))
> >> On 2010/09/30 16:04:51, scottz wrote:
> >>>
> >>> We get all this for free with the python logging module. We really
> >>
> >> need to just
> >>>
> >>> switch to that. A TODO should be added.
> >>
> >> Done.
> >>
> >> http://codereview.chromium.org/3591002/show
> >
> >
>

[sosa, 2010-09-30 21:34:20]

Pushed.  Will keep eye on pineview and x86 bin's