Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(50)

Unified Diff: tools/testrunner/README

Issue 10919265: First commit of new tools/run-tests.py (Closed) Base URL: https://v8.googlecode.com/svn/branches/bleeding_edge
Patch Set: s/server.py/test-server.py/ in README Created 8 years, 3 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View side-by-side diff with in-line comments
Download patch
« no previous file with comments | « tools/test-server.py ('k') | tools/testrunner/__init__.py » ('j') | no next file with comments »
Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
Index: tools/testrunner/README
diff --git a/tools/testrunner/README b/tools/testrunner/README
new file mode 100644
index 0000000000000000000000000000000000000000..8f0c01f52ab3352dd5e324e9d6ac964f0d64bc36
--- /dev/null
+++ b/tools/testrunner/README
@@ -0,0 +1,174 @@
+Test suite runner for V8, including support for distributed running.
+====================================================================
+
+
+Local usage instructions:
+=========================
+
+Run the main script with --help to get detailed usage instructions:
+
+$ tools/run-tests.py --help
+
+The interface is mostly the same as it was for the old test runner.
+You'll likely want something like this:
+
+$ tools/run-tests.py --nonetwork --arch ia32 --mode release
+
+--nonetwork is the default on Mac and Windows. If you don't specify --arch
+and/or --mode, all available values will be used and run in turn (e.g.,
+omitting --mode from the above example will run ia32 in both Release and Debug
+modes).
+
+
+Networked usage instructions:
+=============================
+
+Networked running is only supported on Linux currently. Make sure that all
+machines participating in the cluster are binary-compatible (e.g. mixing
+Ubuntu Lucid and Precise doesn't work).
+
+Setup:
+------
+
+1.) Copy tools/test-server.py to a new empty directory anywhere on your hard
+ drive (preferably not inside your V8 checkout just to keep things clean).
+ Please do create a copy, not just a symlink.
+
+2.) Navigate to the new directory and let the server setup itself:
+
+$ ./test-server.py setup
+
+ This will install PIP and UltraJSON, create a V8 working directory, and
+ generate a keypair.
+
+3.) Swap public keys with someone who's already part of the networked cluster.
+
+$ cp trusted/`cat data/mypubkey`.pem /where/peers/can/see/it/myname.pem
+$ ./test-server.py approve /wherever/they/put/it/yourname.pem
+
+
+Usage:
+------
+
+1.) Start your server:
+
+$ ./test-server.py start
+
+2.) (Optionally) inspect the server's status:
+
+$ ./test-server.py status
+
+3.) From your regular V8 working directory, run tests:
+
+$ tool/run-tests.py --arch ia32 --mode debug
+
+4.) (Optionally) enjoy the speeeeeeeeeeeeeeeed
+
+
+Architecture overview:
+======================
+
+Code organization:
+------------------
+
+This section is written from the point of view of the tools/ directory.
+
+./run-tests.py:
+ Main script. Parses command-line options and drives the test execution
+ procedure from a high level. Imports the actual implementation of all
+ steps from the testrunner/ directory.
+
+./test-server.py:
+ Interface to interact with the server. Contains code to setup the server's
+ working environment and can start and stop server daemon processes.
+ Imports some stuff from the testrunner/server/ directory.
+
+./testrunner/local/*:
+ Implementation needed to run tests locally. Used by run-tests.py. Inspired by
+ (and partly copied verbatim from) the original test.py script.
+
+./testrunner/local/old_statusfile.py:
+ Provides functionality to read an old-style <testsuite>.status file and
+ convert it to new-style syntax. This can be removed once the new-style
+ syntax becomes authoritative (and old-style syntax is no longer supported).
+ ./status-file-converter.py provides a stand-alone interface to this.
+
+./testrunner/objects/*:
+ A bunch of data container classes, used by the scripts in the various other
+ directories; serializable for transmission over the network.
+
+./testrunner/network/*:
+ Equivalents and extensions of some of the functionality in ./testrunner/local/
+ as required when dispatching tests to peers on the network.
+
+./testrunner/network/network_execution.py:
+ Drop-in replacement for ./testrunner/local/execution that distributes
+ test jobs to network peers instead of running them locally.
+
+./testrunner/network/endpoint.py:
+ Receiving end of a network distributed job, uses the implementation
+ in ./testrunner/local/execution.py for actually running the tests.
+
+./testrunner/server/*:
+ Implementation of the daemon that accepts and runs test execution jobs from
+ peers on the network. Should ideally have no dependencies on any of the other
+ directories, but that turned out to be impractical, so there are a few
+ exceptions.
+
+./testrunner/server/compression.py:
+ Defines a wrapper around Python TCP sockets that provides JSON based
+ serialization, gzip based compression, and ensures message completeness.
+
+
+Networking architecture:
+------------------------
+
+The distribution stuff is designed to be a layer between deciding which tests
+to run on the one side, and actually running them on the other. The frontend
+that the user interacts with is the same for local and networked execution,
+and the actual test execution and result gathering code is the same too.
+
+The server daemon starts four separate servers, each listening on another port:
+- "Local": Communication with a run-tests.py script running on the same host.
+ The test driving script e.g. needs to ask for available peers. It then talks
+ to those peers directly (one of them will be the locally running server).
+- "Work": Listens for test job requests from run-tests.py scripts on the network
+ (including localhost). Accepts an arbitrary number of connections at the
+ same time, but only works on them in a serialized fashion.
+- "Status": Used for communication with other servers on the network, e.g. for
+ exchanging trusted public keys to create the transitive trust closure.
+- "Discovery": Used to detect presence of other peers on the network.
+ In contrast to the other three, this uses UDP (as opposed to TCP).
+
+
+Give us a diagram! We love diagrams!
+------------------------------------
+ .
+ Machine A . Machine B
+ .
++------------------------------+ .
+| run-tests.py | .
+| with flag: | .
+|--nonetwork --network | .
+| | / | | .
+| | / | | .
+| v / v | .
+|BACKEND / distribution | .
++--------- / --------| \ ------+ .
+ / | \_____________________
+ / | . \
+ / | . \
++----- v ----------- v --------+ . +---- v -----------------------+
+| LocalHandler | WorkHandler | . | WorkHandler | LocalHandler |
+| | | | . | | | |
+| | v | . | v | |
+| | BACKEND | . | BACKEND | |
+|------------- +---------------| . |---------------+--------------|
+| Discovery | StatusHandler <----------> StatusHandler | Discovery |
++---- ^ -----------------------+ . +-------------------- ^ -------+
+ | . |
+ +---------------------------------------------------------+
+
+Note that the three occurrences of "BACKEND" are the same code
+(testrunner/local/execution.py and its imports), but running from three
+distinct directories (and on two different machines).
« no previous file with comments | « tools/test-server.py ('k') | tools/testrunner/__init__.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698