OLD | NEW |
| (Empty) |
1 Test suite runner for V8, including support for distributed running. | |
2 ==================================================================== | |
3 | |
4 | |
5 Local usage instructions: | |
6 ========================= | |
7 | |
8 Run the main script with --help to get detailed usage instructions: | |
9 | |
10 $ tools/run-tests.py --help | |
11 | |
12 The interface is mostly the same as it was for the old test runner. | |
13 You'll likely want something like this: | |
14 | |
15 $ tools/run-tests.py --nonetwork --arch ia32 --mode release | |
16 | |
17 --nonetwork is the default on Mac and Windows. If you don't specify --arch | |
18 and/or --mode, all available values will be used and run in turn (e.g., | |
19 omitting --mode from the above example will run ia32 in both Release and Debug | |
20 modes). | |
21 | |
22 | |
23 Networked usage instructions: | |
24 ============================= | |
25 | |
26 Networked running is only supported on Linux currently. Make sure that all | |
27 machines participating in the cluster are binary-compatible (e.g. mixing | |
28 Ubuntu Lucid and Precise doesn't work). | |
29 | |
30 Setup: | |
31 ------ | |
32 | |
33 1.) Copy tools/test-server.py to a new empty directory anywhere on your hard | |
34 drive (preferably not inside your V8 checkout just to keep things clean). | |
35 Please do create a copy, not just a symlink. | |
36 | |
37 2.) Navigate to the new directory and let the server setup itself: | |
38 | |
39 $ ./test-server.py setup | |
40 | |
41 This will install PIP and UltraJSON, create a V8 working directory, and | |
42 generate a keypair. | |
43 | |
44 3.) Swap public keys with someone who's already part of the networked cluster. | |
45 | |
46 $ cp trusted/`cat data/mypubkey`.pem /where/peers/can/see/it/myname.pem | |
47 $ ./test-server.py approve /wherever/they/put/it/yourname.pem | |
48 | |
49 | |
50 Usage: | |
51 ------ | |
52 | |
53 1.) Start your server: | |
54 | |
55 $ ./test-server.py start | |
56 | |
57 2.) (Optionally) inspect the server's status: | |
58 | |
59 $ ./test-server.py status | |
60 | |
61 3.) From your regular V8 working directory, run tests: | |
62 | |
63 $ tool/run-tests.py --arch ia32 --mode debug | |
64 | |
65 4.) (Optionally) enjoy the speeeeeeeeeeeeeeeed | |
66 | |
67 | |
68 Architecture overview: | |
69 ====================== | |
70 | |
71 Code organization: | |
72 ------------------ | |
73 | |
74 This section is written from the point of view of the tools/ directory. | |
75 | |
76 ./run-tests.py: | |
77 Main script. Parses command-line options and drives the test execution | |
78 procedure from a high level. Imports the actual implementation of all | |
79 steps from the testrunner/ directory. | |
80 | |
81 ./test-server.py: | |
82 Interface to interact with the server. Contains code to setup the server's | |
83 working environment and can start and stop server daemon processes. | |
84 Imports some stuff from the testrunner/server/ directory. | |
85 | |
86 ./testrunner/local/*: | |
87 Implementation needed to run tests locally. Used by run-tests.py. Inspired by | |
88 (and partly copied verbatim from) the original test.py script. | |
89 | |
90 ./testrunner/local/old_statusfile.py: | |
91 Provides functionality to read an old-style <testsuite>.status file and | |
92 convert it to new-style syntax. This can be removed once the new-style | |
93 syntax becomes authoritative (and old-style syntax is no longer supported). | |
94 ./status-file-converter.py provides a stand-alone interface to this. | |
95 | |
96 ./testrunner/objects/*: | |
97 A bunch of data container classes, used by the scripts in the various other | |
98 directories; serializable for transmission over the network. | |
99 | |
100 ./testrunner/network/*: | |
101 Equivalents and extensions of some of the functionality in ./testrunner/local/ | |
102 as required when dispatching tests to peers on the network. | |
103 | |
104 ./testrunner/network/network_execution.py: | |
105 Drop-in replacement for ./testrunner/local/execution that distributes | |
106 test jobs to network peers instead of running them locally. | |
107 | |
108 ./testrunner/network/endpoint.py: | |
109 Receiving end of a network distributed job, uses the implementation | |
110 in ./testrunner/local/execution.py for actually running the tests. | |
111 | |
112 ./testrunner/server/*: | |
113 Implementation of the daemon that accepts and runs test execution jobs from | |
114 peers on the network. Should ideally have no dependencies on any of the other | |
115 directories, but that turned out to be impractical, so there are a few | |
116 exceptions. | |
117 | |
118 ./testrunner/server/compression.py: | |
119 Defines a wrapper around Python TCP sockets that provides JSON based | |
120 serialization, gzip based compression, and ensures message completeness. | |
121 | |
122 | |
123 Networking architecture: | |
124 ------------------------ | |
125 | |
126 The distribution stuff is designed to be a layer between deciding which tests | |
127 to run on the one side, and actually running them on the other. The frontend | |
128 that the user interacts with is the same for local and networked execution, | |
129 and the actual test execution and result gathering code is the same too. | |
130 | |
131 The server daemon starts four separate servers, each listening on another port: | |
132 - "Local": Communication with a run-tests.py script running on the same host. | |
133 The test driving script e.g. needs to ask for available peers. It then talks | |
134 to those peers directly (one of them will be the locally running server). | |
135 - "Work": Listens for test job requests from run-tests.py scripts on the network | |
136 (including localhost). Accepts an arbitrary number of connections at the | |
137 same time, but only works on them in a serialized fashion. | |
138 - "Status": Used for communication with other servers on the network, e.g. for | |
139 exchanging trusted public keys to create the transitive trust closure. | |
140 - "Discovery": Used to detect presence of other peers on the network. | |
141 In contrast to the other three, this uses UDP (as opposed to TCP). | |
142 | |
143 | |
144 Give us a diagram! We love diagrams! | |
145 ------------------------------------ | |
146 . | |
147 Machine A . Machine B | |
148 . | |
149 +------------------------------+ . | |
150 | run-tests.py | . | |
151 | with flag: | . | |
152 |--nonetwork --network | . | |
153 | | / | | . | |
154 | | / | | . | |
155 | v / v | . | |
156 |BACKEND / distribution | . | |
157 +--------- / --------| \ ------+ . | |
158 / | \_____________________ | |
159 / | . \ | |
160 / | . \ | |
161 +----- v ----------- v --------+ . +---- v -----------------------+ | |
162 | LocalHandler | WorkHandler | . | WorkHandler | LocalHandler | | |
163 | | | | . | | | | | |
164 | | v | . | v | | | |
165 | | BACKEND | . | BACKEND | | | |
166 |------------- +---------------| . |---------------+--------------| | |
167 | Discovery | StatusHandler <----------> StatusHandler | Discovery | | |
168 +---- ^ -----------------------+ . +-------------------- ^ -------+ | |
169 | . | | |
170 +---------------------------------------------------------+ | |
171 | |
172 Note that the three occurrences of "BACKEND" are the same code | |
173 (testrunner/local/execution.py and its imports), but running from three | |
174 distinct directories (and on two different machines). | |
OLD | NEW |