Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(20)

Side by Side Diff: media/tools/layout_tests/README

Issue 7778048: Add README file and parametrize result directory in layout test analzyer tool. (Closed) Base URL: svn://svn.chromium.org/chrome/trunk/src
Patch Set: Correct mistake in the last upload. Created 9 years, 3 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
OLDNEW
(Empty)
1 Layout test analyzer script.
2
3 * Prerequisite:
4
5 This script requires PySVN (http://pysvn.tigris.org/) to be installed on the
6 machine.
7
8 * How to execute
9
10 python layouttest_analyzer.py (Please run with '-h' for available command-line
11 options)
12
13 Example: python layouttest_analyzer.py -r imasaki@chromium.org
14 -t /var/www/analyzer/graph.html
15
16 * File/Directory structure
17
18 layout_tests/
19 README: this file.
20 *.py: python scripts.
21 tmp/: temp files.
22 test_data/: data for unit tests.
23 graph/: the default location for graph files.
24 anno/: the default location for bug annotation files.
25 result/: the default location for the analyzer results.
26
27 * Execution overview.
28
29 1) using PySVN, the script collects layout test of interest
30 (specified in command-line parameter and testnames/*.csv file) from
31 Webkit SVN repository as well as test description. Then, it gets the
32 latest test expectation file and does the simple parsing. The
33 parsed test expectation is joined with the layout tests using test name
34 as its join key. The tests are classified into 'whole', 'skip',
35 'nonskip' test groups. 'Whole' contains all tests, 'skip' contains
36 tests that are skipped as specified in the test expectation file, 'nonskip'
37 tests are in the test expectation file and not skipped (these are tests
38 that need attention).
39
40 2) the script reads the previous latest analyzer results, then,
41 compares it with current results for each test group ('whole', 'skip',
42 and 'nonskip'). It also looks into the SVN test expectation history
43 diff for the time period for the test names of interest.
44
45 3) all obtained information sent out to a email list specified
dennis_jeffrey 2011/09/01 00:29:13 'sent out' --> 'is sent out'
imasaki1 2011/09/01 03:08:23 Done.
46 in the parameters in pretty HTML format.
47
48 4) the trend graph is updated using the current results.
49
50 * Components
51
52 ** LayoutTests:
53
54 A class to store test names in layout tests. The test names (including
55 regular expression patterns) are read from a CSV file and used for
56 getting layout test names from Webkit SVN as well as test description.
57
58 ** TestExpectations:
59
60 A class to model the content of test expectation file for analysis.
61 The raw test expectation file can be found in
62 |DEFAULT_TEST_EXPECTATION_LOCATION|. It is necessary to parse this
63 file and store meaningful information for the analysis (joining with
64 existing layout tests using a test name). Instance variable
65 |all_test_expectation_info| is used. A test name such as
66 'media/video-source-type.html' is used for the key to store
67 information. However, a test name can appear multiple times in the
68 test expectation file. So, the map should keep all the occurrence
69 information. For example, the current test expectation file has the
70 following two entries:
71
72 BUGWK58587 LINUX DEBUG GPU : media/video-zoom.html = IMAGE
73 BUGCR86714 MAC GPU : media/video-zoom.html = CRASH IMAGE
74
75 In this case, all_test_expectation_info['media/video-zoom.html'] will
76 have a list with two elements, each of which is the map of the test
77 expectation information.
78
79 ** TestExpectationsHistory:
80
81 A class to represent history of the test expectation file. The history
82 is obtained by calling PySVN.log()/diff() APIs using the specified time
83 period.
84
85 ** TrendGraph:
86
87 A class to manage trend graph which is using Google Visualization
88 APIs. Google Visualization API
89 (http://code.google.com/apis/chart/interactive/docs/gallery/annotatedtimeline. html)
90 is used to present the historical analyzer result. Currently, data
91 is directly written to JavaScript file using file in-place
92 replacement for simplicity.
OLDNEW
« no previous file with comments | « no previous file | media/tools/layout_tests/layouttest_analyzer.py » ('j') | media/tools/layout_tests/trend_graph.py » ('J')

Powered by Google App Engine
This is Rietveld 408576698