OLD | NEW |
1 # Continuous build and test infrastructure for Chromium for iOS | 1 # This document has moved |
2 | 2 |
3 See the [instructions] for how to check out and build Chromium for iOS. | 3 NOTE: Please update your link to this file! |
4 | 4 |
5 The Chromium projects use buildbot for continuous integration. This doc starts | 5 The new file location is [//docs/ios/infra.md](ios/infra.md) |
6 with an overview of the system, then gives detailed explanations about each | |
7 part. | |
8 | |
9 [TOC] | |
10 | |
11 ## Overview | |
12 | |
13 Commits are made using the [commit queue], which triggers a series of try jobs | |
14 to compile and test the proposed patch against Chromium tip of tree before | |
15 actually making the commit. If the try jobs succeed the patch is committed. A | |
16 newly committed change triggers the builders (or "bots") to compile and test | |
17 the change again. | |
18 | |
19 ## Bots | |
20 | |
21 Bots are slaves attached to a buildbot master (or "waterfall"). A buildbot | |
22 master is a server which polls for commits to a repository and triggers workers | |
23 to compile and test new commits whenever they are detected. [chromium.mac] is | |
24 the main waterfall for Mac desktop and iOS. [tryserver.chromium.mac] serves | |
25 as the try server for Mac desktop and iOS. | |
26 | |
27 The bots know how to check out a given revision of Chromium, compile, and test. | |
28 | |
29 ### Code location | |
30 | |
31 #### Master configs | |
32 | |
33 The masters are configured in [tools/build], a separate repository which | |
34 contains various infra-related scripts. | |
35 | |
36 #### Pollers | |
37 | |
38 [chromium.mac] uses a `GitilesPoller` which polls the Chromium repository for | |
39 new commits using the [gitiles] interface. When a new commit is detected, the | |
40 bots are triggered. | |
41 | |
42 #### Recipes | |
43 | |
44 The bots run [recipes], which are scripts that specify their sequence of steps | |
45 located in [tools/build]. An iOS-specific [recipe module] contains common | |
46 functionality that the various [iOS recipes] use. | |
47 | |
48 #### Configs | |
49 | |
50 Because the recipes live in another repository, changes to the recipes don't | |
51 go through the Chromium [commit queue] and aren't tested on the [try server]. | |
52 In order to allow bot changes to be tested by the commit queue, the recipes | |
53 for iOS are generic instead of bot-specific, and rely on configuration files | |
54 which live in master-specific JSON config files located in [src/ios/build/bots]. | |
55 These configs define the `gn_args` to use during compilation as well as the | |
56 tests to run. | |
57 | |
58 #### Scripts | |
59 | |
60 The [test runner] is the script which installs and runs the tests, interprets | |
61 the results, and collects any files emitted by the test ("test data"). It can | |
62 be found in [src/ios/build/bots/scripts], which means changes to the test runner | |
63 can be tested on the [try server]. | |
64 | |
65 ### Compiling with goma | |
66 | |
67 Goma is the distributed build system used by Chromium. It reduces compilation | |
68 time by avoiding recompilation of objects which have already been compiled | |
69 elsewhere. | |
70 | |
71 ### Testing with swarming | |
72 | |
73 Tests run on [swarming], a distributed test system used by Chromium. After | |
74 compilation, configured tests will be zipped up along with their necessary | |
75 dependencies ("isolated") and sent to the [swarming server] for execution. The | |
76 server issues tasks to its attached workers for execution. The bots themselves | |
77 don't run any tests, they trigger tests to be run remotely on the swarming | |
78 server, then wait and display the results. This allows multiple tests to be | |
79 executed in parallel. | |
80 | |
81 ## Try bots | |
82 | |
83 Try bots are bots which test proposed patches which are not yet committed. | |
84 | |
85 Request [try job access] in order to trigger try jobs against your patch. The | |
86 relevant try bots for an iOS patch are `ios-device`, `ios-device-xcode-clang`, | |
87 `ios-simulator`, and `ios-simulator-xcode-clang`. These bots can be found on | |
88 the Mac-specific [try server]. A try job is said to succeed when the build | |
89 passes (i.e. when the bot successfully compiles and tests the patch). | |
90 | |
91 `ios-device` and `ios-device-xcode-clang` both compile for the iOS device | |
92 architecture (ARM), and neither run any tests. A build is considered successful | |
93 so long as compilation is successful. | |
94 | |
95 `ios-simulator` and `ios-simulator-xcode-clang` both compile for the iOS | |
96 simulator architecture (x86), and run tests in the iOS [simulator]. A build is | |
97 considered successful when both compilation and all configured test succeed. | |
98 | |
99 `ios-device` and `ios-simulator` both compile using the version of [clang] | |
100 defined by the `CLANG_REVISION` in the Chromium tree. | |
101 | |
102 `ios-device-xcode-clang` and `ios-simulator-xcode-clang` both compile using the | |
103 version of clang that ships with Xcode. | |
104 | |
105 ### Scheduling try jobs using buildbucket | |
106 | |
107 Triggering a try job and collecting its results is accomplished using | |
108 [buildbucket]. The service allows for build requests to be put into buckets. A | |
109 request in this context is a set of properties indicating things such as where | |
110 to get the patch. The try bots are set up to poll a particular bucket for build | |
111 requests which they execute and post the results of. | |
112 | |
113 ### Compiling with the analyzer | |
114 | |
115 In addition to goma, the try bots use another time-saving mechanism called the | |
116 [analyzer] to determine the subset of compilation targets affected by the patch | |
117 that need to be compiled in order to run the affected tests. If a patch is | |
118 determined not to affect a certain test target, compilation and execution of the | |
119 test target will be skipped. | |
120 | |
121 ## Configuring the bots | |
122 | |
123 See the [configs code location](#Configs) for where to find the config files for | |
124 the bots. The config files are JSON which describe how the bot should compile | |
125 and which tests it should run. The config files are located in the configs | |
126 directory. The configs directory contains a named directory for each master. For | |
127 example: | |
128 ```shell | |
129 $ ls ios/build/bots | |
130 OWNERS scripts tests chromium.fyi chromium.mac | |
131 ``` | |
132 In this case, configs are defined for iOS bots on [chromium.fyi] and | |
133 [chromium.mac]. Inside each master-specific directory are JSON config files | |
134 named after each bot. For example: | |
135 ```shell | |
136 $ ls ios/build/bots/chromium.mac | |
137 ios-device.json ios-simulator.json | |
138 ``` | |
139 The `ios-device` bot on [chromium.mac] will read its configuration from | |
140 `chromium.mac/ios-device.json` in the configs directory. | |
141 | |
142 ### Example | |
143 | |
144 ```json | |
145 { | |
146 "comments": [ | |
147 "Sample config for a bot." | |
148 ], | |
149 "gn_args": [ | |
150 "is_debug=true", | |
151 "target_cpu=\"x64\"" | |
152 ], | |
153 "tests": [ | |
154 { | |
155 "app": "ios_chrome_unittests", | |
156 "device type": "iPhone 5s", | |
157 "os": "10.0", | |
158 "xcode version": "8.0" | |
159 } | |
160 ] | |
161 } | |
162 ``` | |
163 The `comments` key is optional and defines a list of strings which can be used | |
164 to annotate the config. You may want to explain why the bot exists and what it's | |
165 doing, particularly if there are extensive and atypical `gn_args`. | |
166 | |
167 The `gn_args` key is a required list of arguments to pass to [GN] to generate | |
168 the build files. Two GN args are required, `is_debug` and `target_cpu`. Use | |
169 `is_debug` to define whether to compile for Debug or Release, and `target_cpu` | |
170 to define whether to compile for x86, x64, arm, or arm64. The iOS bots typically | |
171 perform Debug builds for x86 and x64, and Release builds for arm and arm64. An | |
172 x86/x64 build can only be tested on the [iOS simulator], while an arm/arm64 | |
173 build can only be tested on a physical device. | |
174 | |
175 Since Chromium for iOS is shipped as a [universal binary], it's also fairly | |
176 common to set `additional_target_cpus`. For simulator builds, we typically set: | |
177 ```json | |
178 "gn_args": [ | |
179 "additional_target_cpus=[\"x86\"]", | |
180 "is_debug=true", | |
181 "target_cpu=\"x64\"" | |
182 ] | |
183 ``` | |
184 This builds universal binaries which run in 32-bit mode on 32-bit simulators and | |
185 64-bit mode on 64-bit simulators. For device builds we typically set: | |
186 ```json | |
187 "gn_args": [ | |
188 "additional_target_cpus=[\"arm\"]", | |
189 "is_debug=false", | |
190 "target_cpu=\"arm64\"" | |
191 ] | |
192 ``` | |
193 In order to build universal binaries which run in 32-bit mode on 32-bit devices | |
194 and 64-bit mode on 64-bit devices. | |
195 | |
196 The `tests` key is an optional list of dictionaries defining tests to run. There | |
197 are two types of test dictionary, `app` and `include`. An `app` dict defines a | |
198 specific compiled app to run, for example: | |
199 ```json | |
200 "tests": [ | |
201 { | |
202 "app": "ios_chrome_unittests", | |
203 "device type": "iPhone 5s", | |
204 "os": "10.0", | |
205 "xcode version": "8.0" | |
206 } | |
207 ] | |
208 ``` | |
209 This dict says to run `ios_chrome_unittests` on an `iPhone 5s` running iOS | |
210 `10.0` using Xcode `8.0`. A test dict may optionally define a list of `test | |
211 args`, which are arguments to pass directly to the test on the command line, | |
212 and it may define a boolean value `xctest` to indicate whether the test is an | |
213 [xctest] \(default if unspecified is `false`\). For example: | |
214 ```json | |
215 "tests": [ | |
216 { | |
217 "app": "ios_chrome_unittests", | |
218 "device type": "iPhone 5s", | |
219 "os": "10.0", | |
220 "test args": [ | |
221 "--foo", | |
222 "--bar" | |
223 ], | |
224 "xcode version": "8.0" | |
225 }, | |
226 { | |
227 "app": "ios_chrome_integration_egtests", | |
228 "device type": "iPhone 5s", | |
229 "os": "10.0", | |
230 "xcode version": "8.0", | |
231 "xctest": true | |
232 } | |
233 ] | |
234 ``` | |
235 This defines two tests to run, first `ios_chrome_unittests` will be run with | |
236 `--foo` and `--bar` passed directly to the test on the command line. Next, | |
237 `ios_chrome_integration_egtests` will be run as an xctest. `"xctest": true` | |
238 must be specified for all xctests, it is an error to try and launch an xctest as | |
239 a regular test. | |
240 | |
241 An `include` dict defines a list of tests to import from the `tests` | |
242 subdirectory in the configs directory. For example: | |
243 ```json | |
244 "tests": [ | |
245 { | |
246 "include": "common_tests.json", | |
247 "device type": "iPhone 5s", | |
248 "os": "10.0", | |
249 "xcode version": "8.0" | |
250 } | |
251 ] | |
252 ``` | |
253 This dict says to import the list of tests from the `tests` subdirectory and run | |
254 each one on an `iPhone 5s` running iOS `10.0` using Xcode `8.0`. Here's what | |
255 `common_tests.json` might look like: | |
256 ```json | |
257 "tests": [ | |
258 { | |
259 "app": "ios_chrome_unittests" | |
260 }, | |
261 { | |
262 "app": "ios_net_unittests" | |
263 }, | |
264 { | |
265 "app": "ios_web_unittests" | |
266 }, | |
267 ] | |
268 ``` | |
269 Includes may contain other keys besides `app` which can then be omitted in the | |
270 bot config. For example if `common_tests.json` specifies: | |
271 ```json | |
272 "tests": [ | |
273 { | |
274 "app": "ios_chrome_integration_egtests", | |
275 "xctest": true, | |
276 "xcode version": "8.0" | |
277 } | |
278 ] | |
279 ``` | |
280 Then the bot config may omit the `xctest` or `xcode version` keys, for example: | |
281 ```json | |
282 { | |
283 "comments": [ | |
284 "Sample config for a bot." | |
285 ], | |
286 "gn_args": [ | |
287 "is_debug=true", | |
288 "target_cpu=\"x64\"" | |
289 ], | |
290 "tests": [ | |
291 { | |
292 "include": "common_tests.json", | |
293 "device type": "iPhone 5s", | |
294 "os": "10.0" | |
295 } | |
296 ] | |
297 } | |
298 ``` | |
299 Includes are not recursive, so `common_tests.json` may not itself include any | |
300 `include` dicts. | |
301 | |
302 ### Uploading compiled artifacts from a bot | |
303 | |
304 A bot may be configured to upload compiled artifacts. This is defined by the | |
305 `upload` key. For example: | |
306 ```json | |
307 { | |
308 "comments": [ | |
309 "Sample config for a bot which uploads artifacts." | |
310 ], | |
311 "gn_args": [ | |
312 "is_debug=true", | |
313 "target_cpu=\"x64\"" | |
314 ], | |
315 "upload": [ | |
316 { | |
317 "artifact": "Chromium.breakpad", | |
318 "bucket": "my-gcs-bucket", | |
319 }, | |
320 { | |
321 "artifact": "Chromium.app", | |
322 "bucket": "my-gcs-bucket", | |
323 "compress": true, | |
324 }, | |
325 { | |
326 "artifact": "Chromium.breakpad", | |
327 "symupload": true, | |
328 } | |
329 ] | |
330 } | |
331 ``` | |
332 After compilation, the bot will upload three artifacts. First the | |
333 `Chromium.breakpad` symbols will be uploaded to | |
334 `gs://my-gcs-bucket/<buildername>/<buildnumber>/Chromium.breakpad`. Next | |
335 `Chromium.app` will be tarred, gzipped, and uploaded to | |
336 `gs://my-gcs-bucket/<buildername>/<buildnumber>/Chromium.tar.gz`. Finally | |
337 the `Chromium.breakpad` symbols will be uploaded to the [breakpad] crash | |
338 reporting server where they can be used to symbolicate stack traces. | |
339 | |
340 If `artifact` is a directory, you must specify `"compress": true`. | |
341 | |
342 [analyzer]: ../tools/mb | |
343 [breakpad]: https://chromium.googlesource.com/breakpad/breakpad | |
344 [buildbucket]: https://cr-buildbucket.appspot.com | |
345 [chromium.fyi]: https://build.chromium.org/p/chromium.fyi/waterfall | |
346 [chromium.mac]: https://build.chromium.org/p/chromium.mac | |
347 [clang]: ../tools/clang | |
348 [commit queue]: https://dev.chromium.org/developers/testing/commit-queue | |
349 [gitiles]: https://gerrit.googlesource.com/gitiles | |
350 [GN]: ../tools/gn | |
351 [instructions]: ./ios_build_instructions.md | |
352 [iOS recipes]: https://chromium.googlesource.com/chromium/tools/build/+/master/s
cripts/slave/recipes/ios | |
353 [iOS simulator]: ../testing/iossim | |
354 [recipe module]: https://chromium.googlesource.com/chromium/tools/build/+/master
/scripts/slave/recipe_modules/ios | |
355 [recipes]: https://chromium.googlesource.com/infra/infra/+/HEAD/doc/users/recipe
s.md | |
356 [simulator]: https://developer.apple.com/library/content/documentation/IDEs/Conc
eptual/iOS_Simulator_Guide/Introduction/Introduction.html | |
357 [src/ios/build/bots]: ../ios/build/bots | |
358 [src/ios/build/bots/scripts]: ../ios/build/bots/scripts | |
359 [swarming]: https://github.com/luci/luci-py/tree/master/appengine/swarming | |
360 [swarming server]: https://chromium-swarm.appspot.com | |
361 [test runner]: ../ios/build/bots/scripts/test_runner.py | |
362 [tools/build]: https://chromium.googlesource.com/chromium/tools/build | |
363 [try job access]: https://www.chromium.org/getting-involved/become-a-committer#T
OC-Try-job-access | |
364 [try server]: https://build.chromium.org/p/tryserver.chromium.mac/waterfall | |
365 [tryserver.chromium.mac]: https://build.chromium.org/p/tryserver.chromium.mac/wa
terfall | |
366 [universal binary]: https://en.wikipedia.org/wiki/Universal_binary | |
367 [xctest]: https://developer.apple.com/reference/xctest | |
OLD | NEW |