Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(151)

Side by Side Diff: build/android/run_tests.py

Issue 11575019: Remove --performance_test from Android gtest runner. (Closed) Base URL: svn://svn.chromium.org/chrome/trunk/src
Patch Set: Created 8 years ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
« no previous file with comments | « build/android/pylib/test_package_executable.py ('k') | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 #!/usr/bin/env python 1 #!/usr/bin/env python
2 # 2 #
3 # Copyright (c) 2012 The Chromium Authors. All rights reserved. 3 # Copyright (c) 2012 The Chromium Authors. All rights reserved.
4 # Use of this source code is governed by a BSD-style license that can be 4 # Use of this source code is governed by a BSD-style license that can be
5 # found in the LICENSE file. 5 # found in the LICENSE file.
6 6
7 """Runs all the native unit tests. 7 """Runs all the native unit tests.
8 8
9 1. Copy over test binary to /data/local on device. 9 1. Copy over test binary to /data/local on device.
10 2. Resources: chrome/unit_tests requires resources (chrome.pak and en-US.pak) 10 2. Resources: chrome/unit_tests requires resources (chrome.pak and en-US.pak)
11 to be deployed to the device. We use the device's $EXTERNAL_STORAGE as the 11 to be deployed to the device. We use the device's $EXTERNAL_STORAGE as the
12 base dir (which maps to Context.getExternalFilesDir()). 12 base dir (which maps to Context.getExternalFilesDir()).
13 3. Environment: 13 3. Environment:
14 3.1. chrome/unit_tests requires (via chrome_paths.cc) a directory named: 14 3.1. chrome/unit_tests requires (via chrome_paths.cc) a directory named:
15 $EXTERNAL_STORAGE + /chrome/test/data 15 $EXTERNAL_STORAGE + /chrome/test/data
16 3.2. page_cycler_tests have following requirements,
17 3.2.1 the following data on host:
18 <chrome_src_dir>/tools/page_cycler
19 <chrome_src_dir>/data/page_cycler
20 3.2.2. two data directories to store above test data on device named:
21 $EXTERNAL_STORAGE + /tools/ (for database perf test)
22 $EXTERNAL_STORAGE + /data/ (for other perf tests)
23 3.2.3. a http server to serve http perf tests.
24 The http root is host's <chrome_src_dir>/data/page_cycler/, port 8000.
25 3.2.4 a tool named forwarder is also required to run on device to
26 forward the http request/response between host and device.
27 3.2.5 Chrome is installed on device.
28 4. Run the binary in the device and stream the log to the host. 16 4. Run the binary in the device and stream the log to the host.
29 4.1. Optionally, filter specific tests. 17 4.1. Optionally, filter specific tests.
30 4.2. If we're running a single test suite and we have multiple devices 18 4.2. If we're running a single test suite and we have multiple devices
31 connected, we'll shard the tests. 19 connected, we'll shard the tests.
32 5. Clean up the device. 20 5. Clean up the device.
33 21
34 Suppressions: 22 Suppressions:
35 23
36 Individual tests in a test binary can be suppressed by listing it in 24 Individual tests in a test binary can be suppressed by listing it in
37 the gtest_filter directory in a file of the same name as the test binary, 25 the gtest_filter directory in a file of the same name as the test binary,
(...skipping 140 matching lines...) Expand 10 before | Expand all | Expand 10 after
178 except: 166 except:
179 pass 167 pass
180 del os.environ['DISPLAY'] 168 del os.environ['DISPLAY']
181 self._pid = 0 169 self._pid = 0
182 170
183 171
184 class TestSharder(BaseTestSharder): 172 class TestSharder(BaseTestSharder):
185 """Responsible for sharding the tests on the connected devices.""" 173 """Responsible for sharding the tests on the connected devices."""
186 174
187 def __init__(self, attached_devices, test_suite, gtest_filter, 175 def __init__(self, attached_devices, test_suite, gtest_filter,
188 test_arguments, timeout, performance_test, 176 test_arguments, timeout, cleanup_test_files, tool,
189 cleanup_test_files, tool, log_dump_name, fast_and_loose, 177 log_dump_name, fast_and_loose, build_type, in_webkit_checkout):
190 build_type, in_webkit_checkout):
191 BaseTestSharder.__init__(self, attached_devices, build_type) 178 BaseTestSharder.__init__(self, attached_devices, build_type)
192 self.test_suite = test_suite 179 self.test_suite = test_suite
193 self.test_suite_basename = os.path.basename(test_suite) 180 self.test_suite_basename = os.path.basename(test_suite)
194 self.gtest_filter = gtest_filter or '' 181 self.gtest_filter = gtest_filter or ''
195 self.test_arguments = test_arguments 182 self.test_arguments = test_arguments
196 self.timeout = timeout 183 self.timeout = timeout
197 self.performance_test = performance_test
198 self.cleanup_test_files = cleanup_test_files 184 self.cleanup_test_files = cleanup_test_files
199 self.tool = tool 185 self.tool = tool
200 self.log_dump_name = log_dump_name 186 self.log_dump_name = log_dump_name
201 self.fast_and_loose = fast_and_loose 187 self.fast_and_loose = fast_and_loose
202 self.in_webkit_checkout = in_webkit_checkout 188 self.in_webkit_checkout = in_webkit_checkout
203 self.all_tests = [] 189 self.all_tests = []
204 if not self.gtest_filter: 190 if not self.gtest_filter:
205 # No filter has been specified, let's add all tests then. 191 # No filter has been specified, let's add all tests then.
206 self.all_tests = self._GetAllEnabledTests() 192 self.all_tests = self._GetAllEnabledTests()
207 self.tests = self.all_tests 193 self.tests = self.all_tests
(...skipping 18 matching lines...) Expand all
226 raise Exception('No device available to get the list of tests.') 212 raise Exception('No device available to get the list of tests.')
227 213
228 def _GetTestsFromDevice(self, device): 214 def _GetTestsFromDevice(self, device):
229 logging.info('Obtaining tests from %s', device) 215 logging.info('Obtaining tests from %s', device)
230 test_runner = SingleTestRunner( 216 test_runner = SingleTestRunner(
231 device, 217 device,
232 self.test_suite, 218 self.test_suite,
233 self.gtest_filter, 219 self.gtest_filter,
234 self.test_arguments, 220 self.test_arguments,
235 self.timeout, 221 self.timeout,
236 self.performance_test,
237 self.cleanup_test_files, 222 self.cleanup_test_files,
238 self.tool, 223 self.tool,
239 0, 224 0,
240 not not self.log_dump_name, 225 not not self.log_dump_name,
241 self.fast_and_loose, 226 self.fast_and_loose,
242 self.build_type, 227 self.build_type,
243 self.in_webkit_checkout) 228 self.in_webkit_checkout)
244 # The executable/apk needs to be copied before we can call GetAllTests. 229 # The executable/apk needs to be copied before we can call GetAllTests.
245 test_runner.test_package.StripAndCopyExecutable() 230 test_runner.test_package.StripAndCopyExecutable()
246 all_tests = test_runner.test_package.GetAllTests() 231 all_tests = test_runner.test_package.GetAllTests()
(...skipping 18 matching lines...) Expand all
265 device_num = len(self.attached_devices) 250 device_num = len(self.attached_devices)
266 shard_size = (len(self.tests) + device_num - 1) / device_num 251 shard_size = (len(self.tests) + device_num - 1) / device_num
267 shard_test_list = self.tests[index * shard_size : (index + 1) * shard_size] 252 shard_test_list = self.tests[index * shard_size : (index + 1) * shard_size]
268 test_filter = ':'.join(shard_test_list) + self.gtest_filter 253 test_filter = ':'.join(shard_test_list) + self.gtest_filter
269 return SingleTestRunner( 254 return SingleTestRunner(
270 device, 255 device,
271 self.test_suite, 256 self.test_suite,
272 test_filter, 257 test_filter,
273 self.test_arguments, 258 self.test_arguments,
274 self.timeout, 259 self.timeout,
275 self.performance_test,
276 self.cleanup_test_files, self.tool, index, 260 self.cleanup_test_files, self.tool, index,
277 not not self.log_dump_name, 261 not not self.log_dump_name,
278 self.fast_and_loose, 262 self.fast_and_loose,
279 self.build_type, 263 self.build_type,
280 self.in_webkit_checkout) 264 self.in_webkit_checkout)
281 265
282 def OnTestsCompleted(self, test_runners, test_results): 266 def OnTestsCompleted(self, test_runners, test_results):
283 """Notifies that we completed the tests.""" 267 """Notifies that we completed the tests."""
284 test_results.LogFull('Unit test', os.path.basename(self.test_suite), 268 test_results.LogFull('Unit test', os.path.basename(self.test_suite),
285 self.build_type, self.all_tests) 269 self.build_type, self.all_tests)
(...skipping 48 matching lines...) Expand 10 before | Expand all | Expand 10 after
334 if not attached_devices: 318 if not attached_devices:
335 logging.critical('A device must be attached and online.') 319 logging.critical('A device must be attached and online.')
336 buildbot_report.PrintError() 320 buildbot_report.PrintError()
337 return 1 321 return 1
338 322
339 # Reset the test port allocation. It's important to do it before starting 323 # Reset the test port allocation. It's important to do it before starting
340 # to dispatch any tests. 324 # to dispatch any tests.
341 if not ports.ResetTestServerPortAllocation(): 325 if not ports.ResetTestServerPortAllocation():
342 raise Exception('Failed to reset test server port.') 326 raise Exception('Failed to reset test server port.')
343 327
344 if options.performance_test or options.gtest_filter: 328 if options.gtest_filter:
345 logging.warning('Sharding is not possible with these configurations.') 329 logging.warning('Sharding is not possible with these configurations.')
346 attached_devices = [attached_devices[0]] 330 attached_devices = [attached_devices[0]]
347 331
348 sharder = TestSharder( 332 sharder = TestSharder(
349 attached_devices, 333 attached_devices,
350 options.test_suite, 334 options.test_suite,
351 options.gtest_filter, 335 options.gtest_filter,
352 options.test_arguments, 336 options.test_arguments,
353 options.timeout, 337 options.timeout,
354 options.performance_test,
355 options.cleanup_test_files, 338 options.cleanup_test_files,
356 options.tool, 339 options.tool,
357 options.log_dump, 340 options.log_dump,
358 options.fast_and_loose, 341 options.fast_and_loose,
359 options.build_type, 342 options.build_type,
360 options.webkit) 343 options.webkit)
361 test_results = sharder.RunShardedTests() 344 test_results = sharder.RunShardedTests()
362 345
363 for buildbot_emulator in buildbot_emulators: 346 for buildbot_emulator in buildbot_emulators:
364 buildbot_emulator.Shutdown() 347 buildbot_emulator.Shutdown()
(...skipping 50 matching lines...) Expand 10 before | Expand all | Expand 10 after
415 '(use -s help to list them)') 398 '(use -s help to list them)')
416 option_parser.add_option('--out-directory', dest='out_directory', 399 option_parser.add_option('--out-directory', dest='out_directory',
417 help='Path to the out/ directory, irrespective of ' 400 help='Path to the out/ directory, irrespective of '
418 'the build type. Only for non-Chromium uses.') 401 'the build type. Only for non-Chromium uses.')
419 option_parser.add_option('-d', '--device', dest='test_device', 402 option_parser.add_option('-d', '--device', dest='test_device',
420 help='Target device the test suite to run ') 403 help='Target device the test suite to run ')
421 option_parser.add_option('-f', '--gtest_filter', dest='gtest_filter', 404 option_parser.add_option('-f', '--gtest_filter', dest='gtest_filter',
422 help='gtest filter') 405 help='gtest filter')
423 option_parser.add_option('-a', '--test_arguments', dest='test_arguments', 406 option_parser.add_option('-a', '--test_arguments', dest='test_arguments',
424 help='Additional arguments to pass to the test') 407 help='Additional arguments to pass to the test')
425 option_parser.add_option('-p', dest='performance_test',
426 help='Indicator of performance test',
427 action='store_true')
428 option_parser.add_option('-L', dest='log_dump', 408 option_parser.add_option('-L', dest='log_dump',
429 help='file name of log dump, which will be put in ' 409 help='file name of log dump, which will be put in '
430 'subfolder debug_info_dumps under the same ' 410 'subfolder debug_info_dumps under the same '
431 'directory in where the test_suite exists.') 411 'directory in where the test_suite exists.')
432 option_parser.add_option('-e', '--emulator', dest='use_emulator', 412 option_parser.add_option('-e', '--emulator', dest='use_emulator',
433 action='store_true', 413 action='store_true',
434 help='Run tests in a new instance of emulator') 414 help='Run tests in a new instance of emulator')
435 option_parser.add_option('-n', '--emulator_count', 415 option_parser.add_option('-n', '--emulator_count',
436 type='int', default=1, 416 type='int', default=1,
437 help='Number of emulators to launch for running the ' 417 help='Number of emulators to launch for running the '
(...skipping 45 matching lines...) Expand 10 before | Expand all | Expand 10 after
483 # the batch (this happens because the exit status is a sum of all failures 463 # the batch (this happens because the exit status is a sum of all failures
484 # from all suites, but the buildbot associates the exit status only with the 464 # from all suites, but the buildbot associates the exit status only with the
485 # most recent step). 465 # most recent step).
486 if options.exit_code: 466 if options.exit_code:
487 return failed_tests_count 467 return failed_tests_count
488 return 0 468 return 0
489 469
490 470
491 if __name__ == '__main__': 471 if __name__ == '__main__':
492 sys.exit(main(sys.argv)) 472 sys.exit(main(sys.argv))
OLDNEW
« no previous file with comments | « build/android/pylib/test_package_executable.py ('k') | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698