Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(288)

Side by Side Diff: build/README.md

Issue 2095173002: Teach build.py to cross-compile go-based packages. (Closed) Base URL: https://chromium.googlesource.com/infra/infra.git@master
Patch Set: Teach build.py to cross-compile go-based packages. Created 4 years, 6 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | build/build.py » ('j') | build/build.py » ('J')
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 Overview 1 Overview
2 -------- 2 --------
3 3
4 Scripts and files in this directory describe how to build CIPD packages from 4 Scripts and files in this directory describe how to build CIPD packages from
5 the source code in infra.git repo. 5 the source code in infra.git repo.
6 6
7 There are two flavors of packages: 7 There are two flavors of packages:
8 8
9 * Packages with executables compiled from Go code. 9 * Packages with executables compiled from Go code.
10 * Single giant package with all python code and archived virtual environment 10 * Single giant package with all python code and archived virtual environment
11 needed to run it. 11 needed to run it.
12 12
13 13
14 Package definition 14 Package definition
15 ------------------ 15 ------------------
16 16
17 A package is defined in a *.yaml file with the following structure: 17 A package is defined in a *.yaml file with the following structure:
18 18
19 ```yaml 19 ```yaml
20 # Name of the package in CIPD repository. 20 # Name of the package in CIPD repository.
21 package: infra/example/package 21 package: infra/example/package
22 # Human readable description of the package. 22 # Human readable description of the package.
23 description: Example package 23 description: Example package
24 # Optional list of Buildbot CI builders to build this package on. If not 24 # Optional list of Buildbot CI builders to build this package on. If not
25 # specified the package will be build on all CI builders. When build.py script 25 # specified the package will be build on all CI builders. When build.py script
26 # is invoked manually (without --builder flag), this property is ignored. 26 # is invoked manually (without --builder flag), this property is ignored.
27 builders: 27 builders:
28 - infra-continuous-precise-64 28 - infra-continuous-precise-64
29 - ... 29 - ...
30 # If true, it means the package is friendly to different GOOS and GOARCH. If not
31 # set or false, this package will be skipped when doing cross-compilation.
32 supports_cross_compilation: true
33 # Optional list of go packages to 'go install' before zipping this package.
34 go_packages:
35 - github.com/luci/luci-go/client/cmd/cipd
36 - ...
30 # Path to the root of the package source files on the system we're building 37 # Path to the root of the package source files on the system we're building
31 # the package from. Can be absolute or relative to the path of the *.yaml 38 # the package from. Can be absolute or relative to the path of the *.yaml
32 # file itself. 39 # file itself.
33 root: ../.. 40 root: ../..
34 41
35 data: 42 data:
36 # 'dir' section adds a subdirectory of 'root' to the package. In this case 43 # 'dir' section adds a subdirectory of 'root' to the package. In this case
37 # it will scan directory <yaml_path>/../../a/b/c and put files into a/b/c 44 # it will scan directory <yaml_path>/../../a/b/c and put files into a/b/c
38 # directory of the package. 45 # directory of the package.
39 - dir: a/b/c 46 - dir: a/b/c
(...skipping 12 matching lines...) Expand all
52 59
53 Any string in package definition can reference a variable via ${var_name}, for 60 Any string in package definition can reference a variable via ${var_name}, for
54 example: 61 example:
55 62
56 ```yaml 63 ```yaml
57 package: infra/tools/cipd/${platform} 64 package: infra/tools/cipd/${platform}
58 ``` 65 ```
59 66
60 Available variables are defined in [build.py](build.py) in `get_package_vars`: 67 Available variables are defined in [build.py](build.py) in `get_package_vars`:
61 68
62 * `${exe_suffix}` is '.exe' on Windows and empty string on other platforms. 69 * `${exe_suffix}` is `.exe` on Windows and empty string on other platforms. If
63 * `${platform}` defines where build.py is running, as '(flavor)-(bitness)' 70 cross-compiling to Windows, it is also set to `.exe` regardless of the host
71 platform.
72 * `${platform}` defines where build.py is running (if not cross-compiling) or
73 what the target platform is (when cross-compiling), as `(flavor)-(bitness)`
64 string. It is suitable for packages that do not depend much on the exact 74 string. It is suitable for packages that do not depend much on the exact
65 version of the OS, for example packages with statically linked binaries. 75 version of the OS, for example packages with statically linked binaries.
66 Example values: 76 Example values:
67 * linux-amd64 77 * linux-amd64
68 * linux-386 78 * linux-386
69 * mac-amd64 79 * mac-amd64
70 * mac-386 80 * mac-386
71 * windows-amd64 81 * windows-amd64
72 * windows-386 82 * windows-386
73 * `${os_ver}` defines major and minor version of the OS/Linux distribution. 83 * `${os_ver}` defines major and minor version of the OS/Linux distribution.
74 It is useful if package depends on *.dll/*.so libraries provided by the OS. 84 It is useful if package depends on *.dll/*.so libraries provided by the OS.
75 Example values: 85 Example values:
76 * ubuntu14_04 86 * ubuntu14_04
77 * mac10_9 87 * mac10_9
78 * win6_1 88 * win6_1
89 Not set when cross-compiling.
79 * `${python_version}` defines python version as '(major)(minor)' string, 90 * `${python_version}` defines python version as '(major)(minor)' string,
80 e.g '27'. 91 e.g '27'. Not set when cross-compiling.
81 92
82 See [packages](packages/) for examples of package definitions. 93 See [packages](packages/) for examples of package definitions.
83 94
84 95
85 Build script 96 Build script
86 ------------ 97 ------------
87 98
88 [build.py](build.py) script does the following: 99 [build.py](build.py) script does the following:
89 100
90 * Ensures python virtual environment directory (ENV) is up to date. 101 * Ensures python virtual environment directory (ENV) is up to date.
91 * Rebuilds all infra Go code from scratch, with 'release' tag set. 102 * Rebuilds all necessary Go code from scratch and installs binaries into
92 * Enumerates packages/ directory for package definition files, builds and 103 `GOBIN`.
104 * Enumerates `packages/` directory for package definition files, builds and
93 (if `--upload` option is passed) uploads CIPD packages to 105 (if `--upload` option is passed) uploads CIPD packages to
94 [the repository](https://chrome-infra-packages.appspot.com). 106 [the repository](https://chrome-infra-packages.appspot.com).
95 * Stores built packages into out/ (as *.cipd files). 107 * Stores built packages into `out/` (as `*.cipd` files).
96 108
97 Package definition files can assume that Go infra code is built and all 109 Package definition files can assume that Go infra code is built and all
98 artifacts are installed in `GOBIN` (which is go/bin). 110 artifacts are installed in `GOBIN` (which is go/bin).
99 111
100 You can also pass one or more *.yaml file names to build only specific packages: 112 You can also pass one or more *.yaml file names to build only specific packages:
101 113
102 build.py infra_python cipd_client 114 build.py infra_python cipd_client
103 115
104 116
105 Verifying a package 117 Verifying a package
(...skipping 26 matching lines...) Expand all
132 * Runs `python test/<name>.py` with cwd == installation directory. 144 * Runs `python test/<name>.py` with cwd == installation directory.
133 * If test returns 0, considers it success, otherwise - failure. 145 * If test returns 0, considers it success, otherwise - failure.
134 146
135 Thus to test that infra_python.cipd package works, one can do the following: 147 Thus to test that infra_python.cipd package works, one can do the following:
136 148
137 ./build/build.py infra_python 149 ./build/build.py infra_python
138 ./build/test_packages.py infra_python 150 ./build/test_packages.py infra_python
139 151
140 test_packages.py is used on CI builders to verify packages look good before 152 test_packages.py is used on CI builders to verify packages look good before
141 uploading them. 153 uploading them.
154
155
156 Cross compilation of Go code
157 ----------------------------
158
159 `build.py` script recognizes `GOOS` and `GOARCH` environment variables used to
160 specify a target platform when cross-compiling Go code. When it detects them, it
161 builds only Go packages that have `supports_cross_compilation` property set to
162 true in the package definition YAML. It also changes the meaning of
163 `${platform}` and `${exe_suffix}` to match the values for the target platform.
164
165 Built packages have `+${platform}` suffix in file names and coexist with native
166 package in build output directory. When uploading packages (via `build.py
167 --no-rebuild --upload`), `GOOS` and `GOARCH` are used to figure out what flavor
168 of built packages to pick (what `+${platform}` to search for).
OLDNEW
« no previous file with comments | « no previous file | build/build.py » ('j') | build/build.py » ('J')

Powered by Google App Engine
This is Rietveld 408576698