Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(183)

Side by Side Diff: perf/DESIGN.md

Issue 608273002: Add versioning to perf SQL database (Closed) Base URL: https://skia.googlesource.com/buildbot@master
Patch Set: Incorporating feedback 4 Created 6 years, 2 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | perf/Makefile » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 DESIGN 1 DESIGN
2 ====== 2 ======
3 3
4 4
5 Overview 5 Overview
6 -------- 6 --------
7 Provides interactive dashboard for Skia performance data. 7 Provides interactive dashboard for Skia performance data.
8 8
9 Code Locations 9 Code Locations
10 -------------- 10 --------------
(...skipping 178 matching lines...) Expand 10 before | Expand all | Expand 10 after
189 189
190 $ mysql -h 173.194.104.24 -u root -p 190 $ mysql -h 173.194.104.24 -u root -p
191 191
192 192
193 mysql> use skia 193 mysql> use skia
194 194
195 mysql> show tables; 195 mysql> show tables;
196 196
197 Initial setup of the database, the users, and the tables: 197 Initial setup of the database, the users, and the tables:
198 198
199 * Create the database and set up permissions. Execute the following after
200 you connect to a MySQL database (not necessary for SQLite).
201
199 CREATE DATABASE skia; 202 CREATE DATABASE skia;
200 USE skia; 203 USE skia;
201 CREATE USER 'readonly'@'%' IDENTIFIED BY <password in valentine>; 204 CREATE USER 'readonly'@'%' IDENTIFIED BY <password in valentine>;
202 GRANT SELECT ON *.* TO 'readonly'@'%'; 205 GRANT SELECT ON *.* TO 'readonly'@'%';
203 CREATE USER 'readwrite'@'%' IDENTIFIED BY <password in valentine>; 206 CREATE USER 'readwrite'@'%' IDENTIFIED BY <password in valentine>;
204 GRANT SELECT, DELETE, UPDATE, INSERT ON *.* TO 'readwrite'@'%'; 207 GRANT SELECT, DELETE, UPDATE, INSERT ON *.* TO 'readwrite'@'%';
205 208
206 CREATE TABLE shortcuts ( 209 * Create the versioned database tables.
207 id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
208 traces MEDIUMTEXT NOT NULL
209 );
210 210
211 CREATE TABLE clusters ( 211 We use the 'migrateDB' tool to keep the database in a well defined (versioned)
212 id INT NOT NULL AUTO_INCREMENT PRIMARY KEY, 212 state. The 'db_conn_string' flag allows to specify the target database.
213 ts BIGINT NOT NULL, 213 By default it will try to connect to the production environment.
214 hash TEXT NOT NULL, 214 But for testing a local MySQL database can be provided. If it cannot
215 regression FLOAT NOT NULL, 215 connect to MySQL it will fall back to SQLite.
216 cluster MEDIUMTEXT NOT NULL,
217 status TEXT NOT NULL,
218 message TEXT NOT NULL
219 );
220 216
221 CREATE TABLE tries ( 217 Bring the production database to the latest schema version:
222 issue VARCHAR(255) NOT NULL PRIMARY KEY, 218
223 lastUpdated BIGINT NOT NULL, 219 $ migrateDB -logtostderr=true
224 results LONGTEXT NOT NULL 220
225 ); 221 Bring a local database to the latest schema version:
222
223 $ migrateDB -logtostderr=true -db_conn_string="root:%s@tcp(localhost:3306)/ skia?parseTime=true"
224
225 Bring a local SQLite database to the latest schema version:
226
227 $ migrateDB -logtostderr=true -db_conn_string=""
228
226 229
227 Clustering 230 Clustering
228 ---------- 231 ----------
229 232
230 The clustering is done by using k-means clustering over normalized Traces. The 233 The clustering is done by using k-means clustering over normalized Traces. The
231 Traces are normalized by filling in missing data points so that there is a 234 Traces are normalized by filling in missing data points so that there is a
232 data point for every commit, and then scaling the data to have a mean of 0.0 235 data point for every commit, and then scaling the data to have a mean of 0.0
233 and a standard deviation of 1.0. See the docs for ctrace.NewFullTrace(). 236 and a standard deviation of 1.0. See the docs for ctrace.NewFullTrace().
234 237
235 The distance metric used is Euclidean distance between the traces. 238 The distance metric used is Euclidean distance between the traces.
(...skipping 169 matching lines...) Expand 10 before | Expand all | Expand 10 after
405 config, os, etc.) and select exactly two configs from it to compare against in 408 config, os, etc.) and select exactly two configs from it to compare against in
406 the query (say, "8888" and "gpu"). We then organize the data to calculate the 409 the query (say, "8888" and "gpu"). We then organize the data to calculate the
407 ratio of the benches from the two choices in the criteria (vertical) where all 410 ratio of the benches from the two choices in the criteria (vertical) where all
408 other parameters are the same. For instance, we calculate the ratio of benches 411 other parameters are the same. For instance, we calculate the ratio of benches
409 in the "config" vertical from the following two traces: 412 in the "config" vertical from the following two traces:
410 413
411 x86_64:HD7770:ShuttleA:Win8:gradient_create_opaque_640_480:gpu 414 x86_64:HD7770:ShuttleA:Win8:gradient_create_opaque_640_480:gpu
412 x86_64:HD7770:ShuttleA:Win8:gradient_create_opaque_640_480:8888 415 x86_64:HD7770:ShuttleA:Win8:gradient_create_opaque_640_480:8888
413 416
414 and put the value into the cell in a table that has 417 and put the value into the cell in a table that has
415 _row_gradient_create_opaque_640_480_ and 418 _row_gradient_create_opaque_640_480_ and
416 column _x86_64:HD7770:ShuttleA:Win8_. Basically, the table row will be the 419 column _x86_64:HD7770:ShuttleA:Win8_. Basically, the table row will be the
417 "test" name, and the column will be the rest of the keys. The number of columns 420 "test" name, and the column will be the rest of the keys. The number of columns
418 will be the number of perf bots we run (20+ for now). 421 will be the number of perf bots we run (20+ for now).
419 422
420 The value will then tell us if the performance is better (<1) or worse (>1) for 423 The value will then tell us if the performance is better (<1) or worse (>1) for
421 gpu against 8888. We can then heatmap-color the table cells by their value 424 gpu against 8888. We can then heatmap-color the table cells by their value
422 ranges, to provide a visual way for users to identify the problems in cell 425 ranges, to provide a visual way for users to identify the problems in cell
423 groups. By sorting the rows with aggregated performance, users will be able to 426 groups. By sorting the rows with aggregated performance, users will be able to
424 pinpoint the benches with worst/best relative performance to look into. 427 pinpoint the benches with worst/best relative performance to look into.
425 428
(...skipping 24 matching lines...) Expand all
450 453
451 But sysv init only handles starting and stopping a program once, so we use 454 But sysv init only handles starting and stopping a program once, so we use
452 Monit to monitor the application and restart it if it crashes. The config 455 Monit to monitor the application and restart it if it crashes. The config
453 is in: 456 is in:
454 457
455 /etc/monit/conf.d/perf 458 /etc/monit/conf.d/perf
456 459
457 Installation 460 Installation
458 ------------ 461 ------------
459 See the README file. 462 See the README file.
OLDNEW
« no previous file with comments | « no previous file | perf/Makefile » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698