Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(877)

Unified Diff: media/tools/layout_tests/README

Issue 7778048: Add README file and parametrize result directory in layout test analzyer tool. (Closed) Base URL: svn://svn.chromium.org/chrome/trunk/src
Patch Set: Modify comments in trend_graph.py based on CR comments. Created 9 years, 3 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View side-by-side diff with in-line comments
Download patch
« no previous file with comments | « no previous file | media/tools/layout_tests/layouttest_analyzer.py » ('j') | no next file with comments »
Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
Index: media/tools/layout_tests/README
diff --git a/media/tools/layout_tests/README b/media/tools/layout_tests/README
new file mode 100644
index 0000000000000000000000000000000000000000..c13785ca86b1f00d1bd6d76a8f2b75b846d4e059
--- /dev/null
+++ b/media/tools/layout_tests/README
@@ -0,0 +1,93 @@
+Layout test analyzer script.
+
+* Prerequisite:
+
+This script requires PySVN (http://pysvn.tigris.org/) to be installed on the
+machine.
+
+* How to execute
+
+python layouttest_analyzer.py (Please run with '-h' for available command-line
+ options)
+
+Example: python layouttest_analyzer.py -r imasaki@chromium.org
+-t /var/www/analyzer/graph.html
+
+* File/Directory structure
+
+layout_tests/
+ README: this file.
+ *.py: python scripts.
+ tmp/: temp files.
+ test_data/: data for unit tests.
+ graph/: the default location for graph files.
+ anno/: the default location for bug annotation files.
+ result/: the default location for the analyzer results.
+ testname/: the default location for the test group definition files in CSV.
+
+* Execution overview.
+
+1) using PySVN, the script collects layout test of interest
+(specified in command-line parameter and testnames/*.csv file) from
+Webkit SVN repository as well as test description. Then, it gets the
+latest test expectation file and does the simple parsing. The
+parsed test expectation is joined with the layout tests using test name
+as its join key. The tests are classified into 'whole', 'skip',
+'nonskip' test groups. 'Whole' contains all tests, 'skip' contains
+tests that are skipped as specified in the test expectation file, 'nonskip'
+tests are in the test expectation file and not skipped (these are tests
+that need attention).
+
+2) the script reads the previous latest analyzer results, then,
+compares it with current results for each test group ('whole', 'skip',
+and 'nonskip'). It also looks into the SVN test expectation history
+diff for the time period for the test names of interest.
+
+3) all obtained information is sent out to a email list specified
+in the parameters in pretty HTML format.
+
+4) the trend graph is updated using the current results.
+
+* Components
+
+** LayoutTests:
+
+A class to store test names in layout tests. The test names (including
+regular expression patterns) are read from a CSV file and used for
+getting layout test names from Webkit SVN as well as test description.
+
+** TestExpectations:
+
+A class to model the content of test expectation file for analysis.
+The raw test expectation file can be found in
+|DEFAULT_TEST_EXPECTATION_LOCATION|. It is necessary to parse this
+file and store meaningful information for the analysis (joining with
+existing layout tests using a test name). Instance variable
+|all_test_expectation_info| is used. A test name such as
+'media/video-source-type.html' is used for the key to store
+information. However, a test name can appear multiple times in the
+test expectation file. So, the map should keep all the occurrence
+information. For example, the current test expectation file has the
+following two entries:
+
+BUGWK58587 LINUX DEBUG GPU : media/video-zoom.html = IMAGE
+BUGCR86714 MAC GPU : media/video-zoom.html = CRASH IMAGE
+
+In this case, all_test_expectation_info['media/video-zoom.html'] will
+have a list with two elements, each of which is the map of the test
+expectation information.
+
+** TestExpectationsHistory:
+
+A class to represent history of the test expectation file. The history
+is obtained by calling PySVN.log()/diff() APIs using the specified time
+period.
+
+** TrendGraph:
+
+A class to manage trend graph which is using Google Visualization
+ APIs. Google Visualization API
+ (http://code.google.com/apis/chart/interactive/docs/gallery/annotatedtimeline.html)
+ is used to present the historical analyzer result. Currently, data
+ is directly written to JavaScript file using file in-place
+ replacement for simplicity.
« no previous file with comments | « no previous file | media/tools/layout_tests/layouttest_analyzer.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698