OLD | NEW |
(Empty) | |
| 1 Test suite runner for V8, including support for distributed running. |
| 2 ==================================================================== |
| 3 |
| 4 |
| 5 Local usage instructions: |
| 6 ========================= |
| 7 |
| 8 Run the main script with --help to get detailed usage instructions: |
| 9 |
| 10 $ tools/run-tests.py --help |
| 11 |
| 12 The interface is mostly the same as it was for the old test runner. |
| 13 You'll likely want something like this: |
| 14 |
| 15 $ tools/run-tests.py --nonetwork --arch ia32 --mode release |
| 16 |
| 17 --nonetwork is the default on Mac and Windows. If you don't specify --arch |
| 18 and/or --mode, all available values will be used and run in turn (e.g., |
| 19 omitting --mode from the above example will run ia32 in both Release and Debug |
| 20 modes). |
| 21 |
| 22 |
| 23 Networked usage instructions: |
| 24 ============================= |
| 25 |
| 26 Networked running is only supported on Linux currently. Make sure that all |
| 27 machines participating in the cluster are binary-compatible (e.g. mixing |
| 28 Ubuntu Lucid and Precise doesn't work). |
| 29 |
| 30 Setup: |
| 31 ------ |
| 32 |
| 33 1.) Copy tools/test-server.py to a new empty directory anywhere on your hard |
| 34 drive (preferably not inside your V8 checkout just to keep things clean). |
| 35 Please do create a copy, not just a symlink. |
| 36 |
| 37 2.) Navigate to the new directory and let the server setup itself: |
| 38 |
| 39 $ ./test-server.py setup |
| 40 |
| 41 This will install PIP and UltraJSON, create a V8 working directory, and |
| 42 generate a keypair. |
| 43 |
| 44 3.) Swap public keys with someone who's already part of the networked cluster. |
| 45 |
| 46 $ cp trusted/`cat data/mypubkey`.pem /where/peers/can/see/it/myname.pem |
| 47 $ ./test-server.py approve /wherever/they/put/it/yourname.pem |
| 48 |
| 49 |
| 50 Usage: |
| 51 ------ |
| 52 |
| 53 1.) Start your server: |
| 54 |
| 55 $ ./test-server.py start |
| 56 |
| 57 2.) (Optionally) inspect the server's status: |
| 58 |
| 59 $ ./test-server.py status |
| 60 |
| 61 3.) From your regular V8 working directory, run tests: |
| 62 |
| 63 $ tool/run-tests.py --arch ia32 --mode debug |
| 64 |
| 65 4.) (Optionally) enjoy the speeeeeeeeeeeeeeeed |
| 66 |
| 67 |
| 68 Architecture overview: |
| 69 ====================== |
| 70 |
| 71 Code organization: |
| 72 ------------------ |
| 73 |
| 74 This section is written from the point of view of the tools/ directory. |
| 75 |
| 76 ./run-tests.py: |
| 77 Main script. Parses command-line options and drives the test execution |
| 78 procedure from a high level. Imports the actual implementation of all |
| 79 steps from the testrunner/ directory. |
| 80 |
| 81 ./test-server.py: |
| 82 Interface to interact with the server. Contains code to setup the server's |
| 83 working environment and can start and stop server daemon processes. |
| 84 Imports some stuff from the testrunner/server/ directory. |
| 85 |
| 86 ./testrunner/local/*: |
| 87 Implementation needed to run tests locally. Used by run-tests.py. Inspired by |
| 88 (and partly copied verbatim from) the original test.py script. |
| 89 |
| 90 ./testrunner/local/old_statusfile.py: |
| 91 Provides functionality to read an old-style <testsuite>.status file and |
| 92 convert it to new-style syntax. This can be removed once the new-style |
| 93 syntax becomes authoritative (and old-style syntax is no longer supported). |
| 94 ./status-file-converter.py provides a stand-alone interface to this. |
| 95 |
| 96 ./testrunner/objects/*: |
| 97 A bunch of data container classes, used by the scripts in the various other |
| 98 directories; serializable for transmission over the network. |
| 99 |
| 100 ./testrunner/network/*: |
| 101 Equivalents and extensions of some of the functionality in ./testrunner/local/ |
| 102 as required when dispatching tests to peers on the network. |
| 103 |
| 104 ./testrunner/network/network_execution.py: |
| 105 Drop-in replacement for ./testrunner/local/execution that distributes |
| 106 test jobs to network peers instead of running them locally. |
| 107 |
| 108 ./testrunner/network/endpoint.py: |
| 109 Receiving end of a network distributed job, uses the implementation |
| 110 in ./testrunner/local/execution.py for actually running the tests. |
| 111 |
| 112 ./testrunner/server/*: |
| 113 Implementation of the daemon that accepts and runs test execution jobs from |
| 114 peers on the network. Should ideally have no dependencies on any of the other |
| 115 directories, but that turned out to be impractical, so there are a few |
| 116 exceptions. |
| 117 |
| 118 ./testrunner/server/compression.py: |
| 119 Defines a wrapper around Python TCP sockets that provides JSON based |
| 120 serialization, gzip based compression, and ensures message completeness. |
| 121 |
| 122 |
| 123 Networking architecture: |
| 124 ------------------------ |
| 125 |
| 126 The distribution stuff is designed to be a layer between deciding which tests |
| 127 to run on the one side, and actually running them on the other. The frontend |
| 128 that the user interacts with is the same for local and networked execution, |
| 129 and the actual test execution and result gathering code is the same too. |
| 130 |
| 131 The server daemon starts four separate servers, each listening on another port: |
| 132 - "Local": Communication with a run-tests.py script running on the same host. |
| 133 The test driving script e.g. needs to ask for available peers. It then talks |
| 134 to those peers directly (one of them will be the locally running server). |
| 135 - "Work": Listens for test job requests from run-tests.py scripts on the network |
| 136 (including localhost). Accepts an arbitrary number of connections at the |
| 137 same time, but only works on them in a serialized fashion. |
| 138 - "Status": Used for communication with other servers on the network, e.g. for |
| 139 exchanging trusted public keys to create the transitive trust closure. |
| 140 - "Discovery": Used to detect presence of other peers on the network. |
| 141 In contrast to the other three, this uses UDP (as opposed to TCP). |
| 142 |
| 143 |
| 144 Give us a diagram! We love diagrams! |
| 145 ------------------------------------ |
| 146 . |
| 147 Machine A . Machine B |
| 148 . |
| 149 +------------------------------+ . |
| 150 | run-tests.py | . |
| 151 | with flag: | . |
| 152 |--nonetwork --network | . |
| 153 | | / | | . |
| 154 | | / | | . |
| 155 | v / v | . |
| 156 |BACKEND / distribution | . |
| 157 +--------- / --------| \ ------+ . |
| 158 / | \_____________________ |
| 159 / | . \ |
| 160 / | . \ |
| 161 +----- v ----------- v --------+ . +---- v -----------------------+ |
| 162 | LocalHandler | WorkHandler | . | WorkHandler | LocalHandler | |
| 163 | | | | . | | | | |
| 164 | | v | . | v | | |
| 165 | | BACKEND | . | BACKEND | | |
| 166 |------------- +---------------| . |---------------+--------------| |
| 167 | Discovery | StatusHandler <----------> StatusHandler | Discovery | |
| 168 +---- ^ -----------------------+ . +-------------------- ^ -------+ |
| 169 | . | |
| 170 +---------------------------------------------------------+ |
| 171 |
| 172 Note that the three occurrences of "BACKEND" are the same code |
| 173 (testrunner/local/execution.py and its imports), but running from three |
| 174 distinct directories (and on two different machines). |
OLD | NEW |