OLD | NEW |
| (Empty) |
1 # Writing Complex Tests # | |
2 | |
3 For many tests, writing one or more static HTML files is | |
4 sufficient. However there are a large class of tests for which this | |
5 approach is insufficient, including: | |
6 | |
7 * Tests that require cross-domain access | |
8 | |
9 * Tests that depend on setting specific headers or status codes | |
10 | |
11 * Tests that need to inspect the browser sent request | |
12 | |
13 * Tests that require state to be stored on the server | |
14 | |
15 * Tests that require precise timing of the response. | |
16 | |
17 To make writing such tests possible, we are using a number of | |
18 server-side components designed to make it easy to manipulate the | |
19 precise details of the response: | |
20 | |
21 * *wptserve*, a custom python HTTP server. | |
22 | |
23 * *pywebsocket*, an existing websockets server | |
24 | |
25 This document will concentrate on the features of wptserve available | |
26 to test authors. | |
27 | |
28 ## Introduction to wptserve ## | |
29 | |
30 wptserve is a python-based web server. By default it serves static | |
31 files in the testsuite. For more sophisticated requirements, several | |
32 mechanisms are available to take control of the response. These are | |
33 outlined below. | |
34 | |
35 ## Pipes ## | |
36 | |
37 Suitable for: | |
38 | |
39 * Cross domain requests | |
40 * Adding headers or status codes to static files | |
41 * Controlling the sending of static file bodies | |
42 | |
43 Pipes are designed to allow simple manipulation of the way that | |
44 static files are sent without requiring any custom code. They are also | |
45 useful for cross-origin tests because they can be used to activate a | |
46 substitution mechanism which can fill in details of ports and server | |
47 names in the setup on which the tests are being run. | |
48 | |
49 Pipes are indicated by adding a query string to a request for a static | |
50 resource, with the parameter name `pipe`. The value of the query | |
51 should be a `|` serperated list of pipe functions. For example to | |
52 return a `.html` file with the status code 410 and a Content-Type of | |
53 text/plain, one might use: | |
54 | |
55 /resources/example.html?pipe=status(410)|header(Content-Type,text/plain) | |
56 | |
57 There are a selection of pipe functions provided with wptserve and | |
58 more may be added if there are good use cases. | |
59 | |
60 ### sub ### | |
61 | |
62 Used to subsitute variables from the server environment, or from the | |
63 request into the response. A typical use case is for testing | |
64 cross-domain since the exact domain name and ports of the servers are | |
65 generally unknown. | |
66 | |
67 Substitutions are marked in a file using a block delimited by `{{` | |
68 and `}}`. Inside the block the following variables are avalible: | |
69 | |
70 * `{{host}}` - the host name of the server exclusing any subdomain part. | |
71 * `{{domains[]}}` - the domain name of a particular subdomain | |
72 e.g. `{{domains[www]}}` for the `www` subdomain. | |
73 * `{{ports[][]}}` - The port number of servers, by protocol | |
74 e.g. `{{ports[http][1]}}` for the second (i.e. non-default) http | |
75 server. | |
76 * `{{headers[]}}` - The HTTP headers in the request | |
77 e.g. `{{headers[X-Test]}}` for a hypothetical `X-Test` header. | |
78 * `{{GET[]}}` - The query parameters for the request | |
79 e.g. `{{GET[id]}}` for an id parameter sent with the request. | |
80 | |
81 So, for example, to write a javascript file called `xhr.js` that does a | |
82 cross domain XHR test to a different subdomain and port, one would | |
83 write in the file: | |
84 | |
85 var server_url = "http://{{domains[www]}}:{{ports[http][1]}}/path/to/resourc
e"; | |
86 //Create the actual XHR and so on | |
87 | |
88 The file would then be included as: | |
89 | |
90 <script src="xhr.js?pipe=sub"></script> | |
91 | |
92 ### status ### | |
93 | |
94 Used to set the HTTP status of the response, for example: | |
95 | |
96 example.js?pipe=status(410) | |
97 | |
98 ### headers ### | |
99 | |
100 Used to add or replace http headers in the response. Takes two or | |
101 three arguments; the header name, the header value and whether to | |
102 append the header rather than replace an existing header (default: | |
103 False). So, for example, a request for: | |
104 | |
105 example.html?pipe=header(Content-Type,text/plain) | |
106 | |
107 causes example.html to be returned with a text/plain content type | |
108 whereas: | |
109 | |
110 example.html?pipe=header(Content-Type,text/plain,True) | |
111 | |
112 Will cause example.html to be returned with both text/html and | |
113 text/plain content-type headers. | |
114 | |
115 ### slice ### | |
116 | |
117 Used to send only part of a response body. Takes the start and, | |
118 optionally, end bytes as arguments, although either can be null to | |
119 indicate the start or end of the file, respectively. So for example: | |
120 | |
121 example.txt?pipe=slice(10,20) | |
122 | |
123 Would result in a response with a body containing 10 bytes of | |
124 example.txt including byte 10 but excluding byte 20. | |
125 | |
126 example.txt?pipe=slice(10) | |
127 | |
128 Would cause all bytes from byte 10 of example.txt to be sent, but: | |
129 | |
130 example.txt?pipe=slice(null,20) | |
131 | |
132 Would send the first 20 bytes of example.txt. | |
133 | |
134 ### trickle ### | |
135 | |
136 Used to send the body of a response in chunks with delays. Takes a | |
137 single argument that is a microsyntax consisting of colon-separated | |
138 commands. There are three types of commands: | |
139 | |
140 * Bare numbers represent a number of bytes to send | |
141 | |
142 * Numbers prefixed `d` indicate a delay in seconds | |
143 | |
144 * Numbers prefixed `r` must only appear at the end of the command, and | |
145 indicate that the preceding N items must be repeated until there is | |
146 no more content to send. | |
147 | |
148 In the absence of a repetition command, the entire remainder of the content is | |
149 sent at once when the command list is exhausted. So for example: | |
150 | |
151 example.txt?pipe=trickle(d1) | |
152 | |
153 causes a 1s delay before sending the entirety of example.txt. | |
154 | |
155 example.txt?pipe=trickle(100:d1) | |
156 | |
157 causes 100 bytes of example.txt to be sent, followed by a 1s delay, | |
158 and then the remainder of the file to be sent. On the other hand: | |
159 | |
160 example.txt?pipe=trickle(100:d1:r2) | |
161 | |
162 Will cause the file to be sent in 100 byte chunks separated by a 1s | |
163 delay until the whole content has been sent. | |
164 | |
165 ## asis files ## | |
166 | |
167 Suitable for: | |
168 | |
169 * Static, HTTP-non-compliant responses | |
170 | |
171 asis files are simply files with the extension `.asis`. They are sent | |
172 byte for byte to the server without adding a HTTP status line, | |
173 headers, or anything else. This makes them suitable for testing | |
174 situations where the precise bytes on the wire are static, and control | |
175 over the timing is unnecessary, but the response does not conform to | |
176 HTTP requirements. | |
177 | |
178 ## py files ## | |
179 | |
180 Suitable for: | |
181 | |
182 * All tests requiring dynamic responses | |
183 * Tests that need to store server side state. | |
184 | |
185 The most flexible mechanism for writing tests is to use `.py` | |
186 files. These are interpreted as code and are suitable for the same | |
187 kinds of tasks that one might achieve using cgi, PHP or a similar | |
188 technology. Unlike cgi or PHP, the file is not executed directly and | |
189 does not produce output by writing to `stdout`. Instead files must | |
190 contain (at least) a function named `main`, with the signature: | |
191 | |
192 def main(request, response): | |
193 pass | |
194 | |
195 Here `request` is a `Request` object that contains details of the | |
196 request, and `response` is a `Response` object that can be used to set | |
197 properties of the response. Full details of these objects is | |
198 provided in the [wptserve documentation](http://wptserve.readthedocs.org/en/late
st/). | |
199 | |
200 In many cases tests will not need to work with the `response` object | |
201 directly. Instead they can set the status, headers and body simply by | |
202 returning values from the `main` function. If any value is returned, | |
203 it is interpreted as the response body. If two values are returned | |
204 they are interpreted as headers and body, and three values are | |
205 interpreted as status, headers, body. So, for example: | |
206 | |
207 def main(request, response): | |
208 return "TEST" | |
209 | |
210 creates a response with no non-default headers and the body | |
211 `TEST`. Headers can be added as follows: | |
212 | |
213 def main(request, response): | |
214 return ([("Content-Type", "text/plain"), ("X-Test", "test")], | |
215 "TEST") | |
216 | |
217 And a status code as: | |
218 | |
219 def main(request, response): | |
220 return (410, | |
221 [("Content-Type", "text/plain"), ("X-Test", "test")], | |
222 "TEST") | |
223 | |
224 A custom status string may be returned by using a tuple `code, string` | |
225 in place of the code alone. | |
226 | |
227 At the other end of the scale, some tests require precision over the | |
228 exact bytes sent over the wire and their timing. This can be achieved | |
229 using the `writer` property of the response, which exposes a | |
230 `ResponseWriter` object that allows wither writing specific parts of | |
231 the request or direct access to the underlying socket. | |
232 | |
233 For full documentation on the facilities available in `.py` files, see | |
234 the [wptserve documentation](http://wptserve.readthedocs.org/en/latest/). | |
OLD | NEW |