| OLD | NEW |
| 1 # Continuous build and test infrastructure for Chromium for iOS | 1 # This document has moved |
| 2 | 2 |
| 3 See the [instructions] for how to check out and build Chromium for iOS. | 3 NOTE: Please update your link to this file! |
| 4 | 4 |
| 5 The Chromium projects use buildbot for continuous integration. This doc starts | 5 The new file location is [//docs/ios/infra.md](ios/infra.md) |
| 6 with an overview of the system, then gives detailed explanations about each | |
| 7 part. | |
| 8 | |
| 9 [TOC] | |
| 10 | |
| 11 ## Overview | |
| 12 | |
| 13 Commits are made using the [commit queue], which triggers a series of try jobs | |
| 14 to compile and test the proposed patch against Chromium tip of tree before | |
| 15 actually making the commit. If the try jobs succeed the patch is committed. A | |
| 16 newly committed change triggers the builders (or "bots") to compile and test | |
| 17 the change again. | |
| 18 | |
| 19 ## Bots | |
| 20 | |
| 21 Bots are slaves attached to a buildbot master (or "waterfall"). A buildbot | |
| 22 master is a server which polls for commits to a repository and triggers workers | |
| 23 to compile and test new commits whenever they are detected. [chromium.mac] is | |
| 24 the main waterfall for Mac desktop and iOS. [tryserver.chromium.mac] serves | |
| 25 as the try server for Mac desktop and iOS. | |
| 26 | |
| 27 The bots know how to check out a given revision of Chromium, compile, and test. | |
| 28 | |
| 29 ### Code location | |
| 30 | |
| 31 #### Master configs | |
| 32 | |
| 33 The masters are configured in [tools/build], a separate repository which | |
| 34 contains various infra-related scripts. | |
| 35 | |
| 36 #### Pollers | |
| 37 | |
| 38 [chromium.mac] uses a `GitilesPoller` which polls the Chromium repository for | |
| 39 new commits using the [gitiles] interface. When a new commit is detected, the | |
| 40 bots are triggered. | |
| 41 | |
| 42 #### Recipes | |
| 43 | |
| 44 The bots run [recipes], which are scripts that specify their sequence of steps | |
| 45 located in [tools/build]. An iOS-specific [recipe module] contains common | |
| 46 functionality that the various [iOS recipes] use. | |
| 47 | |
| 48 #### Configs | |
| 49 | |
| 50 Because the recipes live in another repository, changes to the recipes don't | |
| 51 go through the Chromium [commit queue] and aren't tested on the [try server]. | |
| 52 In order to allow bot changes to be tested by the commit queue, the recipes | |
| 53 for iOS are generic instead of bot-specific, and rely on configuration files | |
| 54 which live in master-specific JSON config files located in [src/ios/build/bots]. | |
| 55 These configs define the `gn_args` to use during compilation as well as the | |
| 56 tests to run. | |
| 57 | |
| 58 #### Scripts | |
| 59 | |
| 60 The [test runner] is the script which installs and runs the tests, interprets | |
| 61 the results, and collects any files emitted by the test ("test data"). It can | |
| 62 be found in [src/ios/build/bots/scripts], which means changes to the test runner | |
| 63 can be tested on the [try server]. | |
| 64 | |
| 65 ### Compiling with goma | |
| 66 | |
| 67 Goma is the distributed build system used by Chromium. It reduces compilation | |
| 68 time by avoiding recompilation of objects which have already been compiled | |
| 69 elsewhere. | |
| 70 | |
| 71 ### Testing with swarming | |
| 72 | |
| 73 Tests run on [swarming], a distributed test system used by Chromium. After | |
| 74 compilation, configured tests will be zipped up along with their necessary | |
| 75 dependencies ("isolated") and sent to the [swarming server] for execution. The | |
| 76 server issues tasks to its attached workers for execution. The bots themselves | |
| 77 don't run any tests, they trigger tests to be run remotely on the swarming | |
| 78 server, then wait and display the results. This allows multiple tests to be | |
| 79 executed in parallel. | |
| 80 | |
| 81 ## Try bots | |
| 82 | |
| 83 Try bots are bots which test proposed patches which are not yet committed. | |
| 84 | |
| 85 Request [try job access] in order to trigger try jobs against your patch. The | |
| 86 relevant try bots for an iOS patch are `ios-device`, `ios-device-xcode-clang`, | |
| 87 `ios-simulator`, and `ios-simulator-xcode-clang`. These bots can be found on | |
| 88 the Mac-specific [try server]. A try job is said to succeed when the build | |
| 89 passes (i.e. when the bot successfully compiles and tests the patch). | |
| 90 | |
| 91 `ios-device` and `ios-device-xcode-clang` both compile for the iOS device | |
| 92 architecture (ARM), and neither run any tests. A build is considered successful | |
| 93 so long as compilation is successful. | |
| 94 | |
| 95 `ios-simulator` and `ios-simulator-xcode-clang` both compile for the iOS | |
| 96 simulator architecture (x86), and run tests in the iOS [simulator]. A build is | |
| 97 considered successful when both compilation and all configured test succeed. | |
| 98 | |
| 99 `ios-device` and `ios-simulator` both compile using the version of [clang] | |
| 100 defined by the `CLANG_REVISION` in the Chromium tree. | |
| 101 | |
| 102 `ios-device-xcode-clang` and `ios-simulator-xcode-clang` both compile using the | |
| 103 version of clang that ships with Xcode. | |
| 104 | |
| 105 ### Scheduling try jobs using buildbucket | |
| 106 | |
| 107 Triggering a try job and collecting its results is accomplished using | |
| 108 [buildbucket]. The service allows for build requests to be put into buckets. A | |
| 109 request in this context is a set of properties indicating things such as where | |
| 110 to get the patch. The try bots are set up to poll a particular bucket for build | |
| 111 requests which they execute and post the results of. | |
| 112 | |
| 113 ### Compiling with the analyzer | |
| 114 | |
| 115 In addition to goma, the try bots use another time-saving mechanism called the | |
| 116 [analyzer] to determine the subset of compilation targets affected by the patch | |
| 117 that need to be compiled in order to run the affected tests. If a patch is | |
| 118 determined not to affect a certain test target, compilation and execution of the | |
| 119 test target will be skipped. | |
| 120 | |
| 121 ## Configuring the bots | |
| 122 | |
| 123 See the [configs code location](#Configs) for where to find the config files for | |
| 124 the bots. The config files are JSON which describe how the bot should compile | |
| 125 and which tests it should run. The config files are located in the configs | |
| 126 directory. The configs directory contains a named directory for each master. For | |
| 127 example: | |
| 128 ```shell | |
| 129 $ ls ios/build/bots | |
| 130 OWNERS scripts tests chromium.fyi chromium.mac | |
| 131 ``` | |
| 132 In this case, configs are defined for iOS bots on [chromium.fyi] and | |
| 133 [chromium.mac]. Inside each master-specific directory are JSON config files | |
| 134 named after each bot. For example: | |
| 135 ```shell | |
| 136 $ ls ios/build/bots/chromium.mac | |
| 137 ios-device.json ios-simulator.json | |
| 138 ``` | |
| 139 The `ios-device` bot on [chromium.mac] will read its configuration from | |
| 140 `chromium.mac/ios-device.json` in the configs directory. | |
| 141 | |
| 142 ### Example | |
| 143 | |
| 144 ```json | |
| 145 { | |
| 146 "comments": [ | |
| 147 "Sample config for a bot." | |
| 148 ], | |
| 149 "gn_args": [ | |
| 150 "is_debug=true", | |
| 151 "target_cpu=\"x64\"" | |
| 152 ], | |
| 153 "tests": [ | |
| 154 { | |
| 155 "app": "ios_chrome_unittests", | |
| 156 "device type": "iPhone 5s", | |
| 157 "os": "10.0", | |
| 158 "xcode version": "8.0" | |
| 159 } | |
| 160 ] | |
| 161 } | |
| 162 ``` | |
| 163 The `comments` key is optional and defines a list of strings which can be used | |
| 164 to annotate the config. You may want to explain why the bot exists and what it's | |
| 165 doing, particularly if there are extensive and atypical `gn_args`. | |
| 166 | |
| 167 The `gn_args` key is a required list of arguments to pass to [GN] to generate | |
| 168 the build files. Two GN args are required, `is_debug` and `target_cpu`. Use | |
| 169 `is_debug` to define whether to compile for Debug or Release, and `target_cpu` | |
| 170 to define whether to compile for x86, x64, arm, or arm64. The iOS bots typically | |
| 171 perform Debug builds for x86 and x64, and Release builds for arm and arm64. An | |
| 172 x86/x64 build can only be tested on the [iOS simulator], while an arm/arm64 | |
| 173 build can only be tested on a physical device. | |
| 174 | |
| 175 Since Chromium for iOS is shipped as a [universal binary], it's also fairly | |
| 176 common to set `additional_target_cpus`. For simulator builds, we typically set: | |
| 177 ```json | |
| 178 "gn_args": [ | |
| 179 "additional_target_cpus=[\"x86\"]", | |
| 180 "is_debug=true", | |
| 181 "target_cpu=\"x64\"" | |
| 182 ] | |
| 183 ``` | |
| 184 This builds universal binaries which run in 32-bit mode on 32-bit simulators and | |
| 185 64-bit mode on 64-bit simulators. For device builds we typically set: | |
| 186 ```json | |
| 187 "gn_args": [ | |
| 188 "additional_target_cpus=[\"arm\"]", | |
| 189 "is_debug=false", | |
| 190 "target_cpu=\"arm64\"" | |
| 191 ] | |
| 192 ``` | |
| 193 In order to build universal binaries which run in 32-bit mode on 32-bit devices | |
| 194 and 64-bit mode on 64-bit devices. | |
| 195 | |
| 196 The `tests` key is an optional list of dictionaries defining tests to run. There | |
| 197 are two types of test dictionary, `app` and `include`. An `app` dict defines a | |
| 198 specific compiled app to run, for example: | |
| 199 ```json | |
| 200 "tests": [ | |
| 201 { | |
| 202 "app": "ios_chrome_unittests", | |
| 203 "device type": "iPhone 5s", | |
| 204 "os": "10.0", | |
| 205 "xcode version": "8.0" | |
| 206 } | |
| 207 ] | |
| 208 ``` | |
| 209 This dict says to run `ios_chrome_unittests` on an `iPhone 5s` running iOS | |
| 210 `10.0` using Xcode `8.0`. A test dict may optionally define a list of `test | |
| 211 args`, which are arguments to pass directly to the test on the command line, | |
| 212 and it may define a boolean value `xctest` to indicate whether the test is an | |
| 213 [xctest] \(default if unspecified is `false`\). For example: | |
| 214 ```json | |
| 215 "tests": [ | |
| 216 { | |
| 217 "app": "ios_chrome_unittests", | |
| 218 "device type": "iPhone 5s", | |
| 219 "os": "10.0", | |
| 220 "test args": [ | |
| 221 "--foo", | |
| 222 "--bar" | |
| 223 ], | |
| 224 "xcode version": "8.0" | |
| 225 }, | |
| 226 { | |
| 227 "app": "ios_chrome_integration_egtests", | |
| 228 "device type": "iPhone 5s", | |
| 229 "os": "10.0", | |
| 230 "xcode version": "8.0", | |
| 231 "xctest": true | |
| 232 } | |
| 233 ] | |
| 234 ``` | |
| 235 This defines two tests to run, first `ios_chrome_unittests` will be run with | |
| 236 `--foo` and `--bar` passed directly to the test on the command line. Next, | |
| 237 `ios_chrome_integration_egtests` will be run as an xctest. `"xctest": true` | |
| 238 must be specified for all xctests, it is an error to try and launch an xctest as | |
| 239 a regular test. | |
| 240 | |
| 241 An `include` dict defines a list of tests to import from the `tests` | |
| 242 subdirectory in the configs directory. For example: | |
| 243 ```json | |
| 244 "tests": [ | |
| 245 { | |
| 246 "include": "common_tests.json", | |
| 247 "device type": "iPhone 5s", | |
| 248 "os": "10.0", | |
| 249 "xcode version": "8.0" | |
| 250 } | |
| 251 ] | |
| 252 ``` | |
| 253 This dict says to import the list of tests from the `tests` subdirectory and run | |
| 254 each one on an `iPhone 5s` running iOS `10.0` using Xcode `8.0`. Here's what | |
| 255 `common_tests.json` might look like: | |
| 256 ```json | |
| 257 "tests": [ | |
| 258 { | |
| 259 "app": "ios_chrome_unittests" | |
| 260 }, | |
| 261 { | |
| 262 "app": "ios_net_unittests" | |
| 263 }, | |
| 264 { | |
| 265 "app": "ios_web_unittests" | |
| 266 }, | |
| 267 ] | |
| 268 ``` | |
| 269 Includes may contain other keys besides `app` which can then be omitted in the | |
| 270 bot config. For example if `common_tests.json` specifies: | |
| 271 ```json | |
| 272 "tests": [ | |
| 273 { | |
| 274 "app": "ios_chrome_integration_egtests", | |
| 275 "xctest": true, | |
| 276 "xcode version": "8.0" | |
| 277 } | |
| 278 ] | |
| 279 ``` | |
| 280 Then the bot config may omit the `xctest` or `xcode version` keys, for example: | |
| 281 ```json | |
| 282 { | |
| 283 "comments": [ | |
| 284 "Sample config for a bot." | |
| 285 ], | |
| 286 "gn_args": [ | |
| 287 "is_debug=true", | |
| 288 "target_cpu=\"x64\"" | |
| 289 ], | |
| 290 "tests": [ | |
| 291 { | |
| 292 "include": "common_tests.json", | |
| 293 "device type": "iPhone 5s", | |
| 294 "os": "10.0" | |
| 295 } | |
| 296 ] | |
| 297 } | |
| 298 ``` | |
| 299 Includes are not recursive, so `common_tests.json` may not itself include any | |
| 300 `include` dicts. | |
| 301 | |
| 302 ### Uploading compiled artifacts from a bot | |
| 303 | |
| 304 A bot may be configured to upload compiled artifacts. This is defined by the | |
| 305 `upload` key. For example: | |
| 306 ```json | |
| 307 { | |
| 308 "comments": [ | |
| 309 "Sample config for a bot which uploads artifacts." | |
| 310 ], | |
| 311 "gn_args": [ | |
| 312 "is_debug=true", | |
| 313 "target_cpu=\"x64\"" | |
| 314 ], | |
| 315 "upload": [ | |
| 316 { | |
| 317 "artifact": "Chromium.breakpad", | |
| 318 "bucket": "my-gcs-bucket", | |
| 319 }, | |
| 320 { | |
| 321 "artifact": "Chromium.app", | |
| 322 "bucket": "my-gcs-bucket", | |
| 323 "compress": true, | |
| 324 }, | |
| 325 { | |
| 326 "artifact": "Chromium.breakpad", | |
| 327 "symupload": true, | |
| 328 } | |
| 329 ] | |
| 330 } | |
| 331 ``` | |
| 332 After compilation, the bot will upload three artifacts. First the | |
| 333 `Chromium.breakpad` symbols will be uploaded to | |
| 334 `gs://my-gcs-bucket/<buildername>/<buildnumber>/Chromium.breakpad`. Next | |
| 335 `Chromium.app` will be tarred, gzipped, and uploaded to | |
| 336 `gs://my-gcs-bucket/<buildername>/<buildnumber>/Chromium.tar.gz`. Finally | |
| 337 the `Chromium.breakpad` symbols will be uploaded to the [breakpad] crash | |
| 338 reporting server where they can be used to symbolicate stack traces. | |
| 339 | |
| 340 If `artifact` is a directory, you must specify `"compress": true`. | |
| 341 | |
| 342 [analyzer]: ../tools/mb | |
| 343 [breakpad]: https://chromium.googlesource.com/breakpad/breakpad | |
| 344 [buildbucket]: https://cr-buildbucket.appspot.com | |
| 345 [chromium.fyi]: https://build.chromium.org/p/chromium.fyi/waterfall | |
| 346 [chromium.mac]: https://build.chromium.org/p/chromium.mac | |
| 347 [clang]: ../tools/clang | |
| 348 [commit queue]: https://dev.chromium.org/developers/testing/commit-queue | |
| 349 [gitiles]: https://gerrit.googlesource.com/gitiles | |
| 350 [GN]: ../tools/gn | |
| 351 [instructions]: ./ios_build_instructions.md | |
| 352 [iOS recipes]: https://chromium.googlesource.com/chromium/tools/build/+/master/s
cripts/slave/recipes/ios | |
| 353 [iOS simulator]: ../testing/iossim | |
| 354 [recipe module]: https://chromium.googlesource.com/chromium/tools/build/+/master
/scripts/slave/recipe_modules/ios | |
| 355 [recipes]: https://chromium.googlesource.com/infra/infra/+/HEAD/doc/users/recipe
s.md | |
| 356 [simulator]: https://developer.apple.com/library/content/documentation/IDEs/Conc
eptual/iOS_Simulator_Guide/Introduction/Introduction.html | |
| 357 [src/ios/build/bots]: ../ios/build/bots | |
| 358 [src/ios/build/bots/scripts]: ../ios/build/bots/scripts | |
| 359 [swarming]: https://github.com/luci/luci-py/tree/master/appengine/swarming | |
| 360 [swarming server]: https://chromium-swarm.appspot.com | |
| 361 [test runner]: ../ios/build/bots/scripts/test_runner.py | |
| 362 [tools/build]: https://chromium.googlesource.com/chromium/tools/build | |
| 363 [try job access]: https://www.chromium.org/getting-involved/become-a-committer#T
OC-Try-job-access | |
| 364 [try server]: https://build.chromium.org/p/tryserver.chromium.mac/waterfall | |
| 365 [tryserver.chromium.mac]: https://build.chromium.org/p/tryserver.chromium.mac/wa
terfall | |
| 366 [universal binary]: https://en.wikipedia.org/wiki/Universal_binary | |
| 367 [xctest]: https://developer.apple.com/reference/xctest | |
| OLD | NEW |