Add a blacklist to disable autotests that don't build

Add a blacklist to disable autotests that don't build

TEST=built on arm-generic, x86-generic
BUG=none

[robotboy, 2010-03-17 05:06:12]

http://codereview.chromium.org/1004003/diff/1/2
File src/scripts/build_autotest.sh (right):

http://codereview.chromium.org/1004003/diff/1/2#newcode45
src/scripts/build_autotest.sh:45:
ALL_TESTS="${ALL_TESTS},${SITE_TEST##${CLIENT_TEST_PATH}/}"
This is certainly an improvement.  :)

http://codereview.chromium.org/1004003/diff/1/2#newcode50
src/scripts/build_autotest.sh:50: BLACKLISTED_TESTS="ltp netperf2 gpu_SanAngeles
audiovideo_PlaybackRecordSemiAuto"
Let's add a file in the primary board overlays.  It will contain this list.  For
now you can check for that file here and load it if it exists.  Then in the next
CL's we can add those files so that the builds go green.

http://codereview.chromium.org/1004003/diff/1/2#newcode52
src/scripts/build_autotest.sh:52: # Surely there is a better way to do this, but
my bash-fu is weak.
I looked into this a bit and there isn't a simple split/filter/join solution
that I can see.  I think the correct way to deal with this eventually is to have
the list be space instead of comma separated and then generate the comma
separated list at the end.

For now add a comment describing what's going on and It'll be good.

[Aaron Kemp, 2010-03-17 17:24:15]

On 2010/03/17 05:06:12, robotboy wrote:
> http://codereview.chromium.org/1004003/diff/1/2#newcode50
> src/scripts/build_autotest.sh:50: BLACKLISTED_TESTS="ltp netperf2
gpu_SanAngeles
> audiovideo_PlaybackRecordSemiAuto"
> Let's add a file in the primary board overlays.  It will contain this list. 
For
> now you can check for that file here and load it if it exists.  Then in the
next
> CL's we can add those files so that the builds go green.

OK - I added 'autotest-blacklist'.  If you prefer another name or style (eg, a
.conf file we can source with BLACKLIST=... in it) let me know.

> I looked into this a bit and there isn't a simple split/filter/join solution
> that I can see.  I think the correct way to deal with this eventually is to
have
> the list be space instead of comma separated and then generate the comma
> separated list at the end.

OK, I tried that too, actually, and wasn't convinced it was necessarily more
readable :)  Certainly happy to clean this up in another CL if you've got a nice
way to do it

PTAL

[petkov, 2010-03-17 17:27:49]

http://codereview.chromium.org/1004003/diff/4001/5001
File src/scripts/build_autotest.sh (right):

http://codereview.chromium.org/1004003/diff/4001/5001#newcode56
src/scripts/build_autotest.sh:56: for TEST in ${BLACKLISTED_TESTS}
Another way to do this, not necessarily better or more readable...

ALL_TESTS=,${ALL_TESTS},
for TEST in ${BLACKLISTED_TESTS}
do
  ALL_TESTS=${ALL_TESTS/,${TEST},/,}
done
ALL_TESTS=${ALL_TESTS/#,/}
ALL_TESTS=${ALL_TESTS/%,/}

[Aaron Kemp, 2010-03-17 17:30:45]

On 2010/03/17 17:27:49, petkov wrote:
> http://codereview.chromium.org/1004003/diff/4001/5001
> File src/scripts/build_autotest.sh (right):
> 
> http://codereview.chromium.org/1004003/diff/4001/5001#newcode56
> src/scripts/build_autotest.sh:56: for TEST in ${BLACKLISTED_TESTS}
> Another way to do this, not necessarily better or more readable...
> 
> ALL_TESTS=,${ALL_TESTS},
> for TEST in ${BLACKLISTED_TESTS}
> do
>   ALL_TESTS=${ALL_TESTS/,${TEST},/,}
> done
> ALL_TESTS=${ALL_TESTS/#,/}
> ALL_TESTS=${ALL_TESTS/%,/}

Doesn't this miss the case where a blacklisted test is the first or last item in
the ALL_TESTS list?

[Aaron Kemp, 2010-03-17 17:31:27]

On 2010/03/17 17:30:45, Aaron Kemp wrote:
> On 2010/03/17 17:27:49, petkov wrote:
> > http://codereview.chromium.org/1004003/diff/4001/5001
> > File src/scripts/build_autotest.sh (right):
> > 
> > http://codereview.chromium.org/1004003/diff/4001/5001#newcode56
> > src/scripts/build_autotest.sh:56: for TEST in ${BLACKLISTED_TESTS}
> > Another way to do this, not necessarily better or more readable...
> > 
> > ALL_TESTS=,${ALL_TESTS},
> > for TEST in ${BLACKLISTED_TESTS}
> > do
> >   ALL_TESTS=${ALL_TESTS/,${TEST},/,}
> > done
> > ALL_TESTS=${ALL_TESTS/#,/}
> > ALL_TESTS=${ALL_TESTS/%,/}
> 
> Doesn't this miss the case where a blacklisted test is the first or last item
in
> the ALL_TESTS list?

Woops.  I'm sorry, I missed the commas in the ALL_TESTS assignment at the top :)

[petkov, 2010-03-17 17:39:53]

On 2010/03/17 17:31:27, Aaron Kemp wrote:
> On 2010/03/17 17:30:45, Aaron Kemp wrote:
> > On 2010/03/17 17:27:49, petkov wrote:
> > > http://codereview.chromium.org/1004003/diff/4001/5001
> > > File src/scripts/build_autotest.sh (right):
> > > 
> > > http://codereview.chromium.org/1004003/diff/4001/5001#newcode56
> > > src/scripts/build_autotest.sh:56: for TEST in ${BLACKLISTED_TESTS}
> > > Another way to do this, not necessarily better or more readable...
> > > 
> > > ALL_TESTS=,${ALL_TESTS},
> > > for TEST in ${BLACKLISTED_TESTS}
> > > do
> > >   ALL_TESTS=${ALL_TESTS/,${TEST},/,}
> > > done
> > > ALL_TESTS=${ALL_TESTS/#,/}
> > > ALL_TESTS=${ALL_TESTS/%,/}
> > 
> > Doesn't this miss the case where a blacklisted test is the first or last
item
> in
> > the ALL_TESTS list?
> 
> Woops.  I'm sorry, I missed the commas in the ALL_TESTS assignment at the top
:)

I.e., it's not clear that it's more readable :-)

[robotboy, 2010-03-18 20:03:38]

LGTM, thanks for waiting, and thanks for doing this.  I've got a couple CL's
soon to submit that fix problems with building autotest from a clean setup_board
state.  But that isn't blocking this.