Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(457)

Side by Side Diff: mozilla-tests/runtests.sh

Issue 2865028: Update the mozilla tests to new version (as of 2010-06-29). (Closed) Base URL: svn://chrome-svn/chrome/trunk/deps/third_party/
Patch Set: Created 10 years, 5 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
« no previous file with comments | « mozilla-tests/process-logs.sh ('k') | mozilla-tests/shell.js » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 #!/bin/bash -e 1 #!/bin/bash -e
2 # -*- Mode: Shell-script; tab-width: 4; indent-tabs-mode: nil; -*- 2 # -*- Mode: Shell-script; tab-width: 4; indent-tabs-mode: nil; -*-
3 3
4 # ***** BEGIN LICENSE BLOCK ***** 4 # ***** BEGIN LICENSE BLOCK *****
5 # Version: MPL 1.1/GPL 2.0/LGPL 2.1 5 # Version: MPL 1.1/GPL 2.0/LGPL 2.1
6 # 6 #
7 # The contents of this file are subject to the Mozilla Public License Version 7 # The contents of this file are subject to the Mozilla Public License Version
8 # 1.1 (the "License"); you may not use this file except in compliance with 8 # 1.1 (the "License"); you may not use this file except in compliance with
9 # the License. You may obtain a copy of the License at 9 # the License. You may obtain a copy of the License at
10 # http://www.mozilla.org/MPL/ 10 # http://www.mozilla.org/MPL/
(...skipping 23 matching lines...) Expand all
34 # and other provisions required by the GPL or the LGPL. If you do not delete 34 # and other provisions required by the GPL or the LGPL. If you do not delete
35 # the provisions above, a recipient may use your version of this file under 35 # the provisions above, a recipient may use your version of this file under
36 # the terms of any one of the MPL, the GPL or the LGPL. 36 # the terms of any one of the MPL, the GPL or the LGPL.
37 # 37 #
38 # ***** END LICENSE BLOCK ***** 38 # ***** END LICENSE BLOCK *****
39 39
40 if [[ -z "$TEST_DIR" ]]; then 40 if [[ -z "$TEST_DIR" ]]; then
41 cat <<EOF 41 cat <<EOF
42 `basename $0`: error 42 `basename $0`: error
43 43
44 TEST_DIR, the location of the Sisyphus framework, 44 TEST_DIR, the location of the Sisyphus framework,
45 is required to be set prior to calling this script. 45 is required to be set prior to calling this script.
46 EOF 46 EOF
47 exit 2 47 exit 2
48 fi 48 fi
49 49
50 if [[ ! -e $TEST_DIR/bin/library.sh ]]; then 50 if [[ ! -e $TEST_DIR/bin/library.sh ]]; then
51 echo "TEST_DIR=$TEST_DIR" 51 echo "TEST_DIR=$TEST_DIR"
52 echo "" 52 echo ""
53 echo "This script requires the Sisyphus testing framework. Please " 53 echo "This script requires the Sisyphus testing framework. Please "
54 echo "cvs check out the Sisyphys framework from mozilla/testing/sisyphus" 54 echo "cvs check out the Sisyphys framework from mozilla/testing/sisyphus"
55 echo "and set the environment variable TEST_DIR to the directory where it" 55 echo "and set the environment variable TEST_DIR to the directory where it"
56 echo "located." 56 echo "located."
57 echo "" 57 echo ""
58 58
59 exit 2 59 exit 2
60 fi 60 fi
61 61
62 source $TEST_DIR/bin/library.sh 62 source $TEST_DIR/bin/library.sh
63 63
64 TEST_JSDIR=${TEST_JSDIR:-$TEST_DIR/tests/mozilla.org/js} 64 TEST_JSDIR=${TEST_JSDIR:-$TEST_DIR/tests/mozilla.org/js}
65 65
66 usage() 66 usage()
67 { 67 {
68 cat <<EOF 68 cat <<EOF
69 usage: runtests.sh -p products -b branches -e extra\\ 69 usage: runtests.sh -p products -b branches -e extra\\
70 -T buildtypes -B buildcommands \\ 70 -T buildtypes -B buildcommands \\
71 [-v] [-S] [-X excludetests] [-I includetests] [-c] [-t] \\ 71 [-v] [-S] [-X excludetests] [-I includetests] [-c] [-t] \\
72 [-Z n] 72 [-J javascriptoptions]
73 73
74 variable description 74 variable description
75 =============== ============================================================ 75 =============== ============================================================
76 -p products space separated list of js, firefox 76 -p products space separated list of js, firefox, fennec
77 -b branches space separated list of branches 1.8.0, 1.8.1, 1.9.0, 1.9.1 77 -b branches space separated list of supported branches. see library.sh
78 -e extra optional. extra qualifier to pick build tree and mozconfig. 78 -e extra optional. extra qualifier to pick build tree and mozconfig.
79 -T buildtypes space separated list of build types opt debug 79 -T buildtypes space separated list of build types opt debug
80 -B buildcommands optional space separated list of build commands 80 -B buildcommands optional space separated list of build commands
81 clean, checkout, build. If not specified, defaults to 81 clean, checkout, build. If not specified, defaults to
82 'clean checkout build'. 82 'clean checkout build'.
83 If you wish to skip any build steps, simply specify a value 83 If you wish to skip any build steps, simply specify a value
84 not containing any of the build commands, e.g. 'none'. 84 not containing any of the build commands, e.g. 'none'.
85 -v optional. verbose - copies log file output to stdout. 85 -v optional. verbose - copies log file output to stdout.
86 -S optional. summary - output tailered for use with 86 -S optional. summary - output tailered for use with
87 Buildbot|Tinderbox 87 Buildbot|Tinderbox
88 -X excludetests optional. By default the test will exclude the 88 -X excludetests optional. By default the test will exclude the
89 tests listed in spidermonkey-n-\$branch.tests, 89 tests listed in spidermonkey-n-\$branch.tests,
90 performance-\$branch.tests. excludetests is a list of either 90 performance-\$branch.tests. excludetests is a list of either
91 individual tests, manifest files or sub-directories which 91 individual tests, manifest files or sub-directories which
92 will override the default exclusion list. 92 will override the default exclusion list.
93 -I includetests optional. By default the test will include the 93 -I includetests optional. By default the test will include the
94 JavaScript tests appropriate for the branch. includetests is a 94 JavaScript tests appropriate for the branch. includetests is a
95 list of either individual tests, manifest files or 95 list of either individual tests, manifest files or
96 sub-directories which will override the default inclusion 96 sub-directories which will override the default inclusion
97 list. 97 list.
98 -c optional. By default the test will exclude tests 98 -c optional. By default the test will exclude tests
99 which crash on this branch, test type, build type and 99 which crash on this branch, test type, build type and
100 operating system. -c will include tests which crash. 100 operating system. -c will include tests which crash.
101 Typically this should only be used in combination with -R. 101 Typically this should only be used in combination with -R.
102 This has no effect on shell based tests which execute crash 102 This has no effect on shell based tests which execute crash
103 tests regardless. 103 tests regardless.
104 -t optional. By default the test will exclude tests 104 -t optional. By default the test will exclude tests
105 which time out on this branch, test type, build type and 105 which time out on this branch, test type, build type and
106 operating system. -t will include tests which timeout. 106 operating system. -t will include tests which timeout.
107 -J jsoptions optional. Set JavaScript options: 107 -J jsoptions optional. Set JavaScript options:
108 -Z n Set gczeal to n. Currently, only valid for 108 -Z n Set gczeal to n. Currently, only valid for
109 debug builds of Gecko 1.8.1.15, 1.9.0 and later. 109 debug builds of Gecko 1.8.1.15, 1.9.0 and later.
110 -z optional. use split objects in the shell. 110 -z optional. use split objects in the shell.
111 -j optional. use JIT in the shell. Only available on 1.9.1 a nd later 111 -j optional. use JIT in the shell. Only available on 1.9.1 a nd later
112 112
113 if an argument contains more than one value, it must be quoted. 113 if an argument contains more than one value, it must be quoted.
114 EOF 114 EOF
115 exit 2 115 exit 2
116 } 116 }
117 117
118 verbose=0 118 verbose=0
(...skipping 38 matching lines...) Expand 10 before | Expand all | Expand 10 after
157 fi 157 fi
158 ;; 158 ;;
159 esac 159 esac
160 160
161 testlogfilelist=`mktemp /tmp/TESTLOGFILES.XXXX` 161 testlogfilelist=`mktemp /tmp/TESTLOGFILES.XXXX`
162 trap "_exit; rm -f $testlogfilelist" EXIT 162 trap "_exit; rm -f $testlogfilelist" EXIT
163 163
164 export testlogfiles 164 export testlogfiles
165 export testlogfile 165 export testlogfile
166 166
167 # because without pipefail, the pipe will not return a non-zero 167 # because without pipefail, the pipe will not return a non-zero
168 # exit code, we must pipe stderr from tester.sh to stdout and then 168 # exit code, we must pipe stderr from tester.sh to stdout and then
169 # look into the testlogfilelist for the error 169 # look into the testlogfilelist for the error
170 170
171 branchesextra=`combo.sh -d - "$branches" "$extra"` 171 branchesextra=`combo.sh -d - "$branches" "$extra"`
172 172
173 # can't test tester.sh's exit code to see if there was 173 # can't test tester.sh's exit code to see if there was
174 # an error since we are piping it and can't count on pipefail 174 # an error since we are piping it and can't count on pipefail
175 tester.sh -t "$TEST_JSDIR/test.sh" $verboseflag "$products" "$branchesextra" "$b uildtypes" 2>&1 | tee -a $testlogfilelist 175 tester.sh -t "$TEST_JSDIR/test.sh" $verboseflag "$products" "$branchesextra" "$b uildtypes" 2>&1 | tee -a $testlogfilelist
176 testlogfiles="`grep '^log:' $testlogfilelist|sed 's|^log: ||'`" 176 testlogfiles="`grep '^log:' $testlogfilelist|sed 's|^log: ||'`"
177 177
178 fatalerrors=`grep 'FATAL ERROR' $testlogfiles | cat` 178 fatalerrors=`grep 'FATAL ERROR' $testlogfiles | cat`
179 if [[ -n "$fatalerrors" ]]; then 179 if [[ -n "$fatalerrors" ]]; then
180 testlogarray=( $testlogfiles ) 180 testlogarray=( $testlogfiles )
181 let itestlog=${#testlogarray[*]}-1 181 let itestlog=${#testlogarray[*]}-1
182 error "`tail -n 20 ${testlogarray[$itestlog]}`" $LINENO 182 error "`tail -n 20 ${testlogarray[$itestlog]}`" $LINENO
183 fi 183 fi
184 184
185 for testlogfile in $testlogfiles; do 185 for testlogfile in $testlogfiles; do
186 186
187 if [[ -n "$DEBUG" ]]; then 187 if [[ -n "$DEBUG" ]]; then
188 dumpvars testlogfile 188 dumpvars testlogfile
189 fi 189 fi
190 190
191 case "$testlogfile" in 191 case "$testlogfile" in
192 *,js,*) testtype=shell;; 192 *,js,*) testtype=shell;;
193 *,firefox,*) testtype=browser;; 193 *,firefox,*) testtype=browser;;
194 *,thunderbird,*) testtype=browser;;
195 *,fennec,*) testtype=browser;;
194 *) error "unknown testtype in logfile $testlogfile" $LINENO;; 196 *) error "unknown testtype in logfile $testlogfile" $LINENO;;
195 esac 197 esac
196 198
197 case "$testlogfile" in 199 case "$testlogfile" in
198 *,opt,*) buildtype=opt;; 200 *,opt,*) buildtype=opt;;
199 *,debug,*) buildtype=debug;; 201 *,debug,*) buildtype=debug;;
200 *,nightly,*) buildtype=opt;; 202 *,nightly*) buildtype=opt;;
201 *) error "unknown buildtype in logfile $testlogfile" $LINENO;; 203 *) error "unknown buildtype in logfile $testlogfile" $LINENO;;
202 esac 204 esac
203 205
204 case "$testlogfile" in 206 branch=`echo $testlogfile | sed 's|.*,\([0-9]\.[0-9]*\.[0-9]*\).*|\1|'`
205 *,1.8.0*) branch=1.8.0;;
206 *,1.8.1*) branch=1.8.1;;
207 *,1.9.0*) branch=1.9.0;;
208 *,1.9.1*) branch=1.9.1;;
209 *) error "unknown branch in logfile $testlogfile" $LINENO;;
210 esac
211 207
212 repo=`grep -m 1 '^environment: TEST_MOZILLA_HG=' $testlogfile | sed 's|.*TES T_MOZILLA_HG=http://hg.mozilla.org/\(.*\)|\1|'` 208
209 repo=`grep -m 1 '^environment: TEST_MOZILLA_HG=' $testlogfile | sed 's|.*TES T_MOZILLA_HG=http://hg.mozilla.org.*/\([^\/]*\)|\1|'`
213 if [[ -z "$repo" ]]; then 210 if [[ -z "$repo" ]]; then
214 repo=CVS 211 repo=CVS
215 fi 212 fi
216 debug "repo=$repo" 213 debug "repo=$repo"
217 214
218 outputprefix=$testlogfile 215 outputprefix=$testlogfile
219 216
220 if [[ -n "$DEBUG" ]]; then 217 if [[ -n "$DEBUG" ]]; then
221 dumpvars branch buildtype testtype OSID testlogfile TEST_PROCESSORTYPE T EST_KERNEL outputprefix 218 dumpvars branch buildtype testtype OSID testlogfile TEST_PROCESSORTYPE T EST_KERNEL outputprefix
222 fi 219 fi
223 220
224 if ! $TEST_DIR/tests/mozilla.org/js/known-failures.pl \ 221 if ! $TEST_DIR/tests/mozilla.org/js/known-failures.pl \
225 -b $branch \ 222 -b $branch \
226 -T $buildtype \ 223 -T $buildtype \
227 -R $repo \ 224 -R $repo \
228 -t $testtype \ 225 -t $testtype \
226 -J "$javascriptoptions" \
229 -o "$OSID" \ 227 -o "$OSID" \
230 -K "$TEST_KERNEL" \ 228 -K "$TEST_KERNEL" \
231 -A "$TEST_PROCESSORTYPE" \ 229 -A "$TEST_PROCESSORTYPE" \
232 -M "$TEST_MEMORY" \ 230 -M "$TEST_MEMORY" \
233 -S "$TEST_CPUSPEED" \
234 -z `date +%z` \ 231 -z `date +%z` \
235 -l $testlogfile \ 232 -l $testlogfile \
236 -r $TEST_JSDIR/failures.txt \ 233 -r $TEST_JSDIR/failures.txt \
237 -O $outputprefix; then 234 -O $outputprefix; then
238 error "known-failures.pl" $LINENO 235 error "known-failures.pl" $LINENO
239 fi 236 fi
240 237
241 if [[ -n "$summary" ]]; then 238 if [[ -n "$summary" ]]; then
242 239
243 # use let to work around mac problem where numbers were 240 # use let to work around mac problem where numbers were
(...skipping 12 matching lines...) Expand all
256 echo -e "\nPossible Fixes:\n" 253 echo -e "\nPossible Fixes:\n"
257 cat "${outputprefix}-results-possible-fixes.log" 254 cat "${outputprefix}-results-possible-fixes.log"
258 echo -e "\nPossible Regressions:\n" 255 echo -e "\nPossible Regressions:\n"
259 cat "${outputprefix}-results-possible-regressions.log" 256 cat "${outputprefix}-results-possible-regressions.log"
260 echo -e "\nTinderboxPrint:<div title=\"$testlogfile\">\n" 257 echo -e "\nTinderboxPrint:<div title=\"$testlogfile\">\n"
261 echo -e "\nTinderboxPrint:js tests<br/>$branch $buildtype $testtype<br/> $npass/$nfail<br/>F:$nfixes R:$nregressions" 258 echo -e "\nTinderboxPrint:js tests<br/>$branch $buildtype $testtype<br/> $npass/$nfail<br/>F:$nfixes R:$nregressions"
262 echo -e "\nTinderboxPrint:</div>\n" 259 echo -e "\nTinderboxPrint:</div>\n"
263 fi 260 fi
264 261
265 done 262 done
OLDNEW
« no previous file with comments | « mozilla-tests/process-logs.sh ('k') | mozilla-tests/shell.js » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698