Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(56)

Side by Side Diff: media/tools/layout_tests/README

Issue 7778048: Add README file and parametrize result directory in layout test analzyer tool. (Closed) Base URL: svn://svn.chromium.org/chrome/trunk/src
Patch Set: Created 9 years, 3 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
« no previous file with comments | « no previous file | media/tools/layout_tests/layouttest_analyzer.py » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
(Empty)
1 Layout test analyzer script.
2
3 * Prerequisite:
4
5 This script requires PySVN (http://pysvn.tigris.org/) to be installed on the
6 machine.
7
8 * How to execute
9
10 python layouttest_analyzer.py (Please run with '-h' for available command-line
11 options)
dennis_jeffrey 2011/08/31 17:49:40 (optional) You may want to also give a full exampl
imasaki1 2011/08/31 19:08:19 Done.
12
13 * Directory structure
dennis_jeffrey 2011/08/31 17:49:40 The description of the directory structure seems t
imasaki1 2011/08/31 19:08:19 Done.
14
15 Design overview:
16
17 * Execution overflow overview.
dennis_jeffrey 2011/08/31 17:49:40 I'm not sure what you mean by 'overflow' here; cou
imasaki1 2011/08/31 19:08:19 Done.
18
19 First, using PySVN, the script collects layout test of interest
20 (specified in command-line parameter and testnames/*.csv file) from
21 Webkit SVN repository as well as test description. Then, it gets the
22 the latest test expectation file and does the simple parsing. The
dennis_jeffrey 2011/08/31 17:49:40 remove 'the' at the start of this line (it's dupli
imasaki1 2011/08/31 19:08:19 Done.
23 parsed test expectation is joined with the layout tests using test name
24 as its join key. The tests are classified into 'whole', 'skip',
25 'nonskip' test groups. 'Whole' contains all tests, 'skip' contains
26 tests are skipped as specified in the test expectation file, 'nonskip'
dennis_jeffrey 2011/08/31 17:49:40 'tests are' --> 'tests that are'
imasaki1 2011/08/31 19:08:19 Done.
27 tests are in the test expectation file and not skipped (these are tests
28 that needs attention).
dennis_jeffrey 2011/08/31 17:49:40 'needs' --> 'need'
imasaki1 2011/08/31 19:08:19 Done.
29
30 Next, the script reads the previous latest analyzer results, then,
31 compares it with current results for each test group ('whole', 'skip',
32 and 'nonskip'). It also looks into the SVN test expectation history
33 diff for the time period for the test names of interest.
34
35 Thirdly, all obtained information sent out to a email list specified
36 in the parameters in pretty HTML format.
37
38 Lastly, the trend graph is updated using the current results.
dennis_jeffrey 2011/08/31 17:49:40 (optional) For lines 19-38, it might be a little e
imasaki1 2011/08/31 19:08:19 Done.
39
40 * Components
41
42 ** LayoutTests:
43
44 A class to store test names in layout tests. The test names (including
45 regular expression patterns) are read from a CSV file and used for
46 getting layout test names from Webkit SVN as well as test description.
47
48 ** TestExpectations:
49
50 A class to model the content of test expectation file for analysis.
51 The raw test expectation file can be found in
52 |DEFAULT_TEST_EXPECTATION_LOCATION|. It is necessary to parse this
53 file and store meaningful information for the analysis (joining with
54 existing layout tests using a test name). Instance variable
55 |all_test_expectation_info| is used. A test name such as
56 'media/video-source-type.html' is used for the key to store
57 information. However, a test name can appear multiple times in the
58 test expectation file. So, the map should keep all the occurrence
59 information. For example, the current test expectation file has the
60 following two entries:
61
62 BUGWK58587 LINUX DEBUG GPU : media/video-zoom.html = IMAGE
63 BUGCR86714 MAC GPU : media/video-zoom.html = CRASH IMAGE
64
65 In this case, all_test_expectation_info['media/video-zoom.html'] will
66 have a list with two elements, each of which is the map of the test
67 expectation information.
68
69 ** TestExpectationsHistory:
70
71 A class to represent history of the test expectation file. The history
72 is obtained by calling PySVN.log()/diff() APIs using the specified time
73 period.
74
75 ** TrendGraph:
76
77 A class to manage trend graph which is using Google Visualization
78 APIs. Google Visualization API
79 (http://code.google.com/apis/chart/interactive/docs/gallery/annotatedtimeline. html)
80 is used to present the historical analyzer result. Currently, data
81 is directly written to JavaScript file using file in-place
82 replacement for simplicity.
OLDNEW
« no previous file with comments | « no previous file | media/tools/layout_tests/layouttest_analyzer.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698