Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(21)

Side by Side Diff: chrome/test/functional/perf.py

Issue 10871038: Dump data for stacked graphs in Chrome Endure. (Closed) Base URL: svn://svn.chromium.org/chrome/trunk/src
Patch Set: removed an empty line Created 8 years, 3 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
OLDNEW
1 #!/usr/bin/env python 1 #!/usr/bin/env python
2 # Copyright (c) 2012 The Chromium Authors. All rights reserved. 2 # Copyright (c) 2012 The Chromium Authors. All rights reserved.
3 # Use of this source code is governed by a BSD-style license that can be 3 # Use of this source code is governed by a BSD-style license that can be
4 # found in the LICENSE file. 4 # found in the LICENSE file.
5 5
6 """Basic pyauto performance tests. 6 """Basic pyauto performance tests.
7 7
8 For tests that need to be run for multiple iterations (e.g., so that average 8 For tests that need to be run for multiple iterations (e.g., so that average
9 and standard deviation values can be reported), the default number of iterations 9 and standard deviation values can be reported), the default number of iterations
10 run for each of these tests is specified by |_DEFAULT_NUM_ITERATIONS|. 10 run for each of these tests is specified by |_DEFAULT_NUM_ITERATIONS|.
(...skipping 238 matching lines...) Expand 10 before | Expand all | Expand 10 after
249 times, in milliseconds as a float. 249 times, in milliseconds as a float.
250 """ 250 """
251 assert callable(python_command) 251 assert callable(python_command)
252 def RunCommand(): 252 def RunCommand():
253 for _ in range(num_invocations): 253 for _ in range(num_invocations):
254 python_command() 254 python_command()
255 timer = timeit.Timer(stmt=RunCommand) 255 timer = timeit.Timer(stmt=RunCommand)
256 return timer.timeit(number=1) * 1000 # Convert seconds to milliseconds. 256 return timer.timeit(number=1) * 1000 # Convert seconds to milliseconds.
257 257
258 def _OutputPerfForStandaloneGraphing(self, graph_name, description, value, 258 def _OutputPerfForStandaloneGraphing(self, graph_name, description, value,
259 units, units_x): 259 units, units_x, is_stacked):
260 """Outputs perf measurement data to a local folder to be graphed. 260 """Outputs perf measurement data to a local folder to be graphed.
261 261
262 This function only applies to Chrome desktop, and assumes that environment 262 This function only applies to Chrome desktop, and assumes that environment
263 variable 'LOCAL_PERF_DIR' has been specified and refers to a valid directory 263 variable 'LOCAL_PERF_DIR' has been specified and refers to a valid directory
264 on the local machine. 264 on the local machine.
265 265
266 Args: 266 Args:
267 graph_name: A string name for the graph associated with this performance 267 graph_name: A string name for the graph associated with this performance
268 value. 268 value.
269 description: A string description of the performance value. Should not 269 description: A string description of the performance value. Should not
270 include spaces. 270 include spaces.
271 value: Either a single numeric value representing a performance 271 value: Either a single numeric value representing a performance
272 measurement, or else a list of (x, y) tuples representing one or more 272 measurement, or else a list of (x, y) tuples representing one or more
273 long-running performance measurements, where 'x' is an x-axis value 273 long-running performance measurements, where 'x' is an x-axis value
274 (such as an iteration number) and 'y' is the corresponding performance 274 (such as an iteration number) and 'y' is the corresponding performance
275 measurement. If a list of tuples is given, then the |units_x| 275 measurement. If a list of tuples is given, then the |units_x|
276 argument must also be specified. 276 argument must also be specified.
277 units: A string representing the units of the performance measurement(s). 277 units: A string representing the units of the performance measurement(s).
278 Should not include spaces. 278 Should not include spaces.
279 units_x: A string representing the units of the x-axis values associated 279 units_x: A string representing the units of the x-axis values associated
280 with the performance measurements, such as 'iteration' if the x values 280 with the performance measurements, such as 'iteration' if the x values
281 are iteration numbers. If this argument is specified, then the 281 are iteration numbers. If this argument is specified, then the
282 |value| argument must be a list of (x, y) tuples. 282 |value| argument must be a list of (x, y) tuples.
283 is_stacked: True to draw a "stacked" graph. First-come values are
284 stacked at bottom by default.
283 """ 285 """
284 revision_num_file = os.path.join(self._local_perf_dir, 'last_revision.dat') 286 revision_num_file = os.path.join(self._local_perf_dir, 'last_revision.dat')
285 if os.path.exists(revision_num_file): 287 if os.path.exists(revision_num_file):
286 with open(revision_num_file) as f: 288 with open(revision_num_file) as f:
287 revision = int(f.read()) 289 revision = int(f.read())
288 else: 290 else:
289 revision = 0 291 revision = 0
290 292
291 if not self._seen_graph_lines: 293 if not self._seen_graph_lines:
292 # We're about to output data for a new test run. 294 # We're about to output data for a new test run.
(...skipping 47 matching lines...) Expand 10 before | Expand all | Expand 10 after
340 if not description in new_line['traces']: 342 if not description in new_line['traces']:
341 new_line['traces'][description] = [] 343 new_line['traces'][description] = []
342 for x_value, y_value in value: 344 for x_value, y_value in value:
343 new_line['traces'][description].append([str(x_value), str(y_value)]) 345 new_line['traces'][description].append([str(x_value), str(y_value)])
344 else: 346 else:
345 new_line = { 347 new_line = {
346 'rev': revision, 348 'rev': revision,
347 'traces': { description: [str(value), str(0.0)] } 349 'traces': { description: [str(value), str(0.0)] }
348 } 350 }
349 351
352 if is_stacked:
353 new_line['stack'] = True
354 if 'stack_order' not in new_line:
355 new_line['stack_order'] = []
356 if description not in new_line['stack_order']:
357 new_line['stack_order'].append(description)
358
350 if seen_key in self._seen_graph_lines: 359 if seen_key in self._seen_graph_lines:
351 # Update results for the most recent revision. 360 # Update results for the most recent revision.
352 existing_lines[0] = new_line 361 existing_lines[0] = new_line
353 else: 362 else:
354 # New results for a new revision. 363 # New results for a new revision.
355 existing_lines.insert(0, new_line) 364 existing_lines.insert(0, new_line)
356 self._seen_graph_lines[seen_key] = True 365 self._seen_graph_lines[seen_key] = True
357 366
358 existing_lines = map(simplejson.dumps, existing_lines) 367 existing_lines = map(simplejson.dumps, existing_lines)
359 with open(data_file, 'w') as f: 368 with open(data_file, 'w') as f:
360 f.write('\n'.join(existing_lines)) 369 f.write('\n'.join(existing_lines))
361 os.chmod(data_file, 0755) 370 os.chmod(data_file, 0755)
362 371
363 with open(revision_num_file, 'w') as f: 372 with open(revision_num_file, 'w') as f:
364 f.write(str(revision)) 373 f.write(str(revision))
365 374
366 def _OutputPerfGraphValue(self, description, value, units, 375 def _OutputPerfGraphValue(self, description, value, units,
367 graph_name, units_x=None): 376 graph_name, units_x=None, is_stacked=False):
368 """Outputs a performance value to have it graphed on the performance bots. 377 """Outputs a performance value to have it graphed on the performance bots.
369 378
370 The output format differs, depending on whether the current platform is 379 The output format differs, depending on whether the current platform is
371 Chrome desktop or ChromeOS. 380 Chrome desktop or ChromeOS.
372 381
373 For ChromeOS, the performance bots have a 30-character limit on the length 382 For ChromeOS, the performance bots have a 30-character limit on the length
374 of the key associated with a performance value. A key on ChromeOS is 383 of the key associated with a performance value. A key on ChromeOS is
375 considered to be of the form "units_description" (for example, 384 considered to be of the form "units_description" (for example,
376 "milliseconds_NewTabPage"), and is created from the |units| and 385 "milliseconds_NewTabPage"), and is created from the |units| and
377 |description| passed as input to this function. Any characters beyond the 386 |description| passed as input to this function. Any characters beyond the
(...skipping 10 matching lines...) Expand all
388 corresponding performance measurement. If a list of tuples is given, 397 corresponding performance measurement. If a list of tuples is given,
389 the |units_x| argument must also be specified. 398 the |units_x| argument must also be specified.
390 units: A string representing the units of the performance measurement(s). 399 units: A string representing the units of the performance measurement(s).
391 Should not include spaces. 400 Should not include spaces.
392 graph_name: A string name for the graph associated with this performance 401 graph_name: A string name for the graph associated with this performance
393 value. Only used on Chrome desktop. 402 value. Only used on Chrome desktop.
394 units_x: A string representing the units of the x-axis values associated 403 units_x: A string representing the units of the x-axis values associated
395 with the performance measurements, such as 'iteration' if the x values 404 with the performance measurements, such as 'iteration' if the x values
396 are iteration numbers. If this argument is specified, then the 405 are iteration numbers. If this argument is specified, then the
397 |value| argument must be a list of (x, y) tuples. 406 |value| argument must be a list of (x, y) tuples.
407 is_stacked: True to draw a "stacked" graph. First-come values are
408 stacked at bottom by default.
398 """ 409 """
399 if (isinstance(value, list) and value[0] is not None and 410 if (isinstance(value, list) and value[0] is not None and
400 isinstance(value[0], tuple)): 411 isinstance(value[0], tuple)):
401 assert units_x 412 assert units_x
402 if units_x: 413 if units_x:
403 assert isinstance(value, list) 414 assert isinstance(value, list)
404 415
405 if self.IsChromeOS(): 416 if self.IsChromeOS():
406 # ChromeOS results don't support lists. 417 # ChromeOS results don't support lists.
407 if (isinstance(value, list) and value[0] is not None and 418 if (isinstance(value, list) and value[0] is not None and
(...skipping 11 matching lines...) Expand all
419 perf_key = '%s_%s' % (units, description) 430 perf_key = '%s_%s' % (units, description)
420 if len(perf_key) > 30: 431 if len(perf_key) > 30:
421 logging.warning('The description "%s" will be truncated to "%s" ' 432 logging.warning('The description "%s" will be truncated to "%s" '
422 '(length 30) when added to the autotest database.', 433 '(length 30) when added to the autotest database.',
423 perf_key, perf_key[:30]) 434 perf_key, perf_key[:30])
424 print '\n%s(\'%s\', %f)%s' % (self._PERF_OUTPUT_MARKER_PRE, 435 print '\n%s(\'%s\', %f)%s' % (self._PERF_OUTPUT_MARKER_PRE,
425 perf_key, value, 436 perf_key, value,
426 self._PERF_OUTPUT_MARKER_POST) 437 self._PERF_OUTPUT_MARKER_POST)
427 sys.stdout.flush() 438 sys.stdout.flush()
428 else: 439 else:
440 # TODO(dmikurube): Support stacked graphs in PrintPerfResult.
441 # See http://crbug.com/122119.
429 if units_x: 442 if units_x:
430 pyauto_utils.PrintPerfResult(graph_name, description, value, 443 pyauto_utils.PrintPerfResult(graph_name, description, value,
431 units + ' ' + units_x) 444 units + ' ' + units_x)
432 else: 445 else:
433 pyauto_utils.PrintPerfResult(graph_name, description, value, units) 446 pyauto_utils.PrintPerfResult(graph_name, description, value, units)
434 447
435 if self._local_perf_dir: 448 if self._local_perf_dir:
436 self._OutputPerfForStandaloneGraphing( 449 self._OutputPerfForStandaloneGraphing(
437 graph_name, description, value, units, units_x) 450 graph_name, description, value, units, units_x, is_stacked)
438 451
439 def _OutputEventForStandaloneGraphing(self, description, event_list): 452 def _OutputEventForStandaloneGraphing(self, description, event_list):
440 """Outputs event information to a local folder to be graphed. 453 """Outputs event information to a local folder to be graphed.
441 454
442 See function _OutputEventGraphValue below for a description of an event. 455 See function _OutputEventGraphValue below for a description of an event.
443 456
444 This function only applies to Chrome Endure tests running on Chrome desktop, 457 This function only applies to Chrome Endure tests running on Chrome desktop,
445 and assumes that environment variable 'LOCAL_PERF_DIR' has been specified 458 and assumes that environment variable 'LOCAL_PERF_DIR' has been specified
446 and refers to a valid directory on the local machine. 459 and refers to a valid directory on the local machine.
447 460
(...skipping 2206 matching lines...) Expand 10 before | Expand all | Expand 10 after
2654 """Identifies the port number to which the server is currently bound. 2667 """Identifies the port number to which the server is currently bound.
2655 2668
2656 Returns: 2669 Returns:
2657 The numeric port number to which the server is currently bound. 2670 The numeric port number to which the server is currently bound.
2658 """ 2671 """
2659 return self._server.server_address[1] 2672 return self._server.server_address[1]
2660 2673
2661 2674
2662 if __name__ == '__main__': 2675 if __name__ == '__main__':
2663 pyauto_functional.Main() 2676 pyauto_functional.Main()
OLDNEW
« no previous file with comments | « no previous file | chrome/test/functional/perf_endure.py » ('j') | chrome/test/functional/perf_endure.py » ('J')

Powered by Google App Engine
This is Rietveld 408576698