| OLD | NEW |
|---|
| (Empty) |
| 1 Design | |
| 2 ====== | |
| 3 | |
| 4 | |
| 5 Overview | |
| 6 -------- | |
| 7 Allows trying out Skia code in the browser. | |
| 8 | |
| 9 | |
| 10 Security | |
| 11 -------- | |
| 12 | |
| 13 We're putting a C++ compiler on the web, and promising to run the results of | |
| 14 user submitted code, so security is a large concern. Security is handled in a | |
| 15 layered approach, using a combination of seccomp-bpf, chroot jail and rlimits. | |
| 16 | |
| 17 *seccomp-bpf* - Used to limit the types of system calls that the user code can | |
| 18 make. Any attempts to make a system call that isn't allowed causes the | |
| 19 application to terminate immediately. | |
| 20 | |
| 21 *chroot jail* - The code is run in a chroot jail, making the rest of the | |
| 22 operating system files unreachable from the running code. | |
| 23 | |
| 24 *rlimits* - Used to limit the resources the running code can get access to, | |
| 25 for example runtime is limited to 5s of CPU. | |
| 26 | |
| 27 User submitted code is also restricted in the following ways: | |
| 28 * Limited to 10K of code total. | |
| 29 * No preprocessor use is allowed (no lines can begin with #includes). | |
| 30 | |
| 31 | |
| 32 Architecture | |
| 33 ------------ | |
| 34 | |
| 35 The server runs on GCE, and consists of a Go Web Server that calls out to the | |
| 36 c++ compiler and executes code in a chroot jail. See the diagram below: | |
| 37 | |
| 38 +–––––––––––––+ | |
| 39 | | | |
| 40 | Browser | | |
| 41 | | | |
| 42 +––––––+––––––+ | |
| 43 | | |
| 44 +––––––+––––––+ | |
| 45 | | | |
| 46 | | | |
| 47 | Web Server | | |
| 48 | | | |
| 49 | (Go) | | |
| 50 | | | |
| 51 | | | |
| 52 +–––––––+–––––+ | |
| 53 | | |
| 54 +–––––––+––––––––––+ | |
| 55 | chroot jail | | |
| 56 | +––––––––––––––+| | |
| 57 | | seccomp || | |
| 58 | | +––––––––––+|| | |
| 59 | | |User code ||| | |
| 60 | | | ||| | |
| 61 | | +––––––––––+|| | |
| 62 | +––––––––––––––+| | |
| 63 | | | |
| 64 +––––––––––––––––––+ | |
| 65 | |
| 66 The user code is expanded into a simple template and linked against libskia | |
| 67 and a couple other .o files that contain main() and the code that sets up the | |
| 68 seccomp and rlimit restrictions. This code also sets up the SkCanvas that is | |
| 69 handed to the user code. Any code the user submits is restricted to running in | |
| 70 a single function that looks like this: | |
| 71 | |
| 72 | |
| 73 void draw(SkCanvas* canvas) { | |
| 74 // User code goes here. | |
| 75 } | |
| 76 | |
| 77 The user code is tracked by taking an MD5 hash of the code The template is | |
| 78 expanded out into <hash>.cpp, which is compiled into <hash>.o, which is then | |
| 79 linked together with all the other libs and object files to create an | |
| 80 executable named <hash>. That executable is copied into a directory | |
| 81 /home/webtry/inout, that is accessible to both the web server and the schroot | |
| 82 jail. The application is then run in the schroot jail, writing its response, | |
| 83 <hash>.png, out into the same directory, /home/webtry/inout/, where is it read | |
| 84 by the web server and returned to the user. | |
| 85 | |
| 86 Startup and config | |
| 87 ------------------ | |
| 88 The server is started and stopped via: | |
| 89 | |
| 90 sudo /etc/init.d/webtry [start|stop|restart] | |
| 91 | |
| 92 But sysv init only handles starting and stopping a program once, so we use | |
| 93 Monit to monitor the application and restart it if it crashes. The config | |
| 94 is in: | |
| 95 | |
| 96 /etc/monit/conf.d/webtry | |
| 97 | |
| 98 The chroot jail is implemented using schroot, its configuration | |
| 99 file is found in: | |
| 100 | |
| 101 /etc/schroot/chroot.d/webtry | |
| 102 | |
| 103 The seccomp configuration is in main.cpp and only allows the following system | |
| 104 calls: | |
| 105 | |
| 106 exit_group | |
| 107 exit | |
| 108 fstat | |
| 109 read | |
| 110 write | |
| 111 close | |
| 112 mmap | |
| 113 munmap | |
| 114 brk | |
| 115 | |
| 116 Database | |
| 117 -------- | |
| 118 | |
| 119 Code submitted is stored in an SQL database so that it can be referenced | |
| 120 later, i.e. we can let users bookmark their SkFiddles. | |
| 121 | |
| 122 The storage layer will be Cloud SQL (a cloud version of MySQL). Back of the | |
| 123 envelope estimates of traffic come out to a price of a about $1/month. | |
| 124 | |
| 125 All passwords for MySQL are stored in valentine. | |
| 126 | |
| 127 To connect to the database from the skia-webtry-b server: | |
| 128 | |
| 129 $ mysql --host=173.194.83.52 --user=root --password | |
| 130 | |
| 131 Initial setup of the database, the user, and the tables: | |
| 132 | |
| 133 CREATE DATABASE webtry; | |
| 134 USE webtry; | |
| 135 CREATE USER 'webtry'@'%' IDENTIFIED BY '<password is in valentine>'; | |
| 136 GRANT SELECT, INSERT, UPDATE ON webtry.webtry TO 'webtry'@'%'; | |
| 137 GRANT SELECT, INSERT, UPDATE ON webtry.workspace TO 'webtry'@'%'; | |
| 138 GRANT SELECT, INSERT, UPDATE ON webtry.workspacetry TO 'webtry'@'%'; | |
| 139 GRANT SELECT, INSERT, UPDATE ON webtry.source_images TO 'webtry'@'%'; | |
| 140 | |
| 141 // If this gets changed also update the sqlite create statement in webtry.go
. | |
| 142 | |
| 143 CREATE TABLE webtry ( | |
| 144 code TEXT DEFAULT '' NOT NULL, | |
| 145 create_ts TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL, | |
| 146 hash CHAR(64) DEFAULT '' NOT NULL, | |
| 147 width INTEGER DEFAULT 256 NOT NULL, | |
| 148 height INTEGER DEFAULT 256 NOT NULL, | |
| 149 source_image_id INTEGER DEFAULT 0 NOT NULL, | |
| 150 PRIMARY KEY(hash), | |
| 151 | |
| 152 FOREIGN KEY (source) REFERENCES sources(id) | |
| 153 ); | |
| 154 | |
| 155 CREATE TABLE workspace ( | |
| 156 name CHAR(64) DEFAULT '' NOT NULL, | |
| 157 create_ts TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL, | |
| 158 PRIMARY KEY(name), | |
| 159 ); | |
| 160 | |
| 161 CREATE TABLE workspacetry ( | |
| 162 name CHAR(64) DEFAULT '' NOT NULL, | |
| 163 create_ts TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL, | |
| 164 hash CHAR(64) DEFAULT '' NOT NULL, | |
| 165 width INTEGER DEFAULT 256 NOT NULL, | |
| 166 height INTEGER DEFAULT 256 NOT NULL, | |
| 167 source_image_id INTEGER DEFAULT 0 NOT NULL, | |
| 168 hidden INTEGER DEFAULT 0 NOT NULL, | |
| 169 | |
| 170 FOREIGN KEY (name) REFERENCES workspace(name), | |
| 171 ); | |
| 172 | |
| 173 CREATE TABLE source_images ( | |
| 174 id INTEGER PRIMARY KEY NOT NULL AUTO_INCREMENT, | |
| 175 image MEDIUMBLOB DEFAULT '' NOT NULL, -- Stored as PN
G. | |
| 176 width INTEGER DEFAULT 0 NOT NULL, | |
| 177 height INTEGER DEFAULT 0 NOT NULL, | |
| 178 create_ts TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL, | |
| 179 hidden INTEGER DEFAULT 0 NOT NULL | |
| 180 ); | |
| 181 | |
| 182 ALTER TABLE webtry ADD COLUMN source_image_id INTEGER DEFAULT 0 NOT NU
LL AFTER hash; | |
| 183 ALTER TABLE workspacetry ADD COLUMN source_image_id INTEGER DEFAULT 0 NOT NU
LL AFTER hash; | |
| 184 | |
| 185 Common queries webtry.go will use: | |
| 186 | |
| 187 INSERT INTO webtry (code, hash) VALUES('int i = 0;...', 'abcdef...'); | |
| 188 | |
| 189 SELECT code, create_ts, hash FROM webtry WHERE hash='abcdef...'; | |
| 190 | |
| 191 SELECT code, create_ts, hash FROM webtry ORDER BY create_ts DESC LIMIT 2; | |
| 192 | |
| 193 // To change the password for the webtry sql client: | |
| 194 SET PASSWORD for 'webtry'@'%' = PASSWORD('<password is in valentine>'); | |
| 195 | |
| 196 // Run before and after to confirm the password changed: | |
| 197 SELECT Host, User, Password FROM mysql.user; | |
| 198 | |
| 199 Common queries for workspaces: | |
| 200 | |
| 201 SELECT hash, create_ts FROM workspace ORDER BY create_ts DESC; | |
| 202 | |
| 203 INSERT INTO workspace (name, hash) VALUES('autumn-river-12354', 'abcdef...')
; | |
| 204 | |
| 205 SELECT name FROM workspace GROUP BY name; | |
| 206 | |
| 207 Common queries for sources: | |
| 208 | |
| 209 SELECT id, image, width, height, create_ts FROM source_images ORDER BY creat
e_ts DESC LIMIT 100; | |
| 210 | |
| 211 Password for the database will be stored in the metadata instance, if the | |
| 212 metadata server can't be found, i.e. running locally, then a local sqlite | |
| 213 database will be used. To see the current password stored in metadata and the | |
| 214 fingerprint: | |
| 215 | |
| 216 gcutil --project=google.com:skia-buildbots getinstance skia-webtry-b | |
| 217 | |
| 218 To set the mysql password that webtry is to use: | |
| 219 | |
| 220 gcutil --project=google.com:skia-buildbots setinstancemetadata skia-webtr
y-b --metadata=password:'[mysql client webtry password]' --fingerprint=[some fin
gerprint] | |
| 221 | |
| 222 To retrieve the password from the running instance just GET the right URL from | |
| 223 the metadata server: | |
| 224 | |
| 225 curl "http://metadata/computeMetadata/v1/instance/attributes/password" -H "X
-Google-Metadata-Request: True" | |
| 226 | |
| 227 N.B. If you need to change the MySQL password that webtry uses, you must change | |
| 228 it both in MySQL and the value stored in the metadata server. | |
| 229 | |
| 230 Source Images | |
| 231 ------------- | |
| 232 | |
| 233 For every try the user can select an optional source image to use as an input. | |
| 234 The id of the source image is just an integer and is stored in the database | |
| 235 along with the other try information, such as the code. | |
| 236 | |
| 237 The actual image itself is also stored in a separate table, 'sources', in the | |
| 238 database. On startup we check that all the images are available in 'inout', | |
| 239 and write out the images if not. Since they are all written to 'inout' we can | |
| 240 use the same /i/ image handler to serve them. | |
| 241 | |
| 242 When a user uploads an image it is decoded and converted to PNG and stored | |
| 243 as a binary blob in the database. | |
| 244 | |
| 245 The bitmap is available to user code as a module level variable: | |
| 246 | |
| 247 SkBitmap source; | |
| 248 | |
| 249 The bitmap is read, decoded and stored in source before the seccomp jail is | |
| 250 instantiated. | |
| 251 | |
| 252 | |
| 253 Squid | |
| 254 ----- | |
| 255 | |
| 256 Squid is configured to run on port 80 and run as an accelerator for the actual | |
| 257 Go program which is running on port 8000. The config for the squid proxy is | |
| 258 held in setup/sys/webtry_squid, which is copied into place during installation | |
| 259 and squid is kept running via monit. | |
| 260 | |
| 261 Workspaces | |
| 262 ---------- | |
| 263 | |
| 264 Workspaces are implemented by the workspace and workspacetry tables. The | |
| 265 workspace table keeps the unique list of all workspaces. The workspacetry table | |
| 266 keeps track of all the tries that have occured in a workspace. Right now the | |
| 267 hidden column of workspacetry is not used, it's for future functionality. | |
| 268 | |
| 269 Code Editor | |
| 270 ----------- | |
| 271 [CodeMirror](http://codemirror.net/) is used for rich code editing. The | |
| 272 following files are included from the official CodeMirror distribution and can | |
| 273 be updated in place (no local customizations): | |
| 274 | |
| 275 * codemirror.js - base CM implementation | |
| 276 * codemirror.css - base CM stylesheet | |
| 277 * clike.js - C-like syntax highlighting support | |
| 278 | |
| 279 Alternatively, we may consider pulling CM as an external dependency at some | |
| 280 point. | |
| 281 | |
| 282 Installation | |
| 283 ------------ | |
| 284 See the README file. | |
| 285 | |
| 286 | |
| OLD | NEW |
|---|
Chromium Code Reviews
Issue 688713002:
delete webtry from main skia repo; it's been moved to buildbots (Closed)
Base URL: https://skia.googlesource.com/skia.git@master