Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(148)

Side by Side Diff: site/dev/testing/ct.md

Issue 1314413002: Update CT documentation on Skia site. (Closed) Base URL: https://skia.googlesource.com/skia.git@master
Patch Set: Updated 900k to 910k. Created 5 years, 3 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | site/dev/testing/ct-system.png » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 Cluster Telemetry 1 Cluster Telemetry
2 ================= 2 =================
3 3
4 ### Contents ### 4 ### Contents ###
5 5
6 * [Overview](#overview) 6 * [Overview](#overview)
7 * [Framework Usage](#framework_usage) 7 * [Framework Usage](#framework_usage)
8 * [System Architecture](#system_architecture) 8 * [System Architecture](#system_architecture)
9 + [System Diagram](#system_diagram) 9 + [System Diagram](#system_diagram)
10 + [Detailed explanation of steps](#detailed_explanation) 10 + [Detailed explanation of steps](#detailed_explanation)
11 * [Code](#code) 11 * [Code](#code)
12 * [Contact Us](#contact_us) 12 * [Contact Us](#contact_us)
13 13
14 <a name="overview"></a> 14 <a name="overview"></a>
15 Overview 15 Overview
16 -------- 16 --------
17 17
18 Cluster Telemetry allows you to run [telemetry](https://www.chromium.org/develop ers/telemetry)'s benchmarks, lua scripts and other tasks using multiple reposito ry patches through Alexa's [top 1 million](http://s3.amazonaws.com/alexa-static/ top-1m.csv.zip) web pages. 18 Cluster Telemetry allows you to run [telemetry](https://www.chromium.org/develop ers/telemetry)'s benchmarks, lua scripts and other tasks using multiple reposito ry patches through Alexa's [top 1 million](http://s3.amazonaws.com/alexa-static/ top-1m.csv.zip) web pages.
19 Developers can use the framework to measure the performance of their patch again st the top subset of the internet on both Desktop and Android. 19 Developers can use the framework to measure the performance of their patch again st the top subset of the internet on both Desktop and Android.
20 20
21 SKP files are a binary format for the draw commands Chromium sends to Skia for r asterization. The goal of the project started off with wanting to collect a larg e repository of 10k SKP files. This repository, after incremental changes in app roaches, has since grown to ~900k and now supports running all telemetry benchma rks. The top level feature request of this project was [skia:1268](https://code. google.com/p/skia/issues/detail?id=1268). 21 SKP files are a binary format for the draw commands Chromium sends to Skia for r asterization. The goal of the project started off with wanting to collect a larg e repository of 10k SKP files. This repository, after incremental changes in app roaches, has since grown to ~910k and now supports running all telemetry benchma rks. The top level feature request of this project was [skia:1268](https://code. google.com/p/skia/issues/detail?id=1268).
22 22
23 A web application has been created on App Engine that automates the process of c apturing new archives and running telemetry benchmarks at a click of a button; r esults are emailed to the requester and the web application contains complete hi story of runs with links to results. You can run telemetry benchmarks at http:// ct.skia.org. 23 A web application has been created on Google Compute Engine that automates the p rocess of capturing new archives and running telemetry benchmarks at a click of a button; results are emailed to the requester and the web application contains complete history of runs with links to results. You can run telemetry benchmarks at http://ct.skia.org.
24 24
25 The framework also contains the ability to run lua scripts on the SKP repository to scrape web pages. It only takes a few minutes to run a lua scrapping script on ~900k SKP files. 25 The framework also contains the ability to run lua scripts on the SKP repository to scrape web pages. It only takes a few minutes to run a lua scraping script o n ~900k SKP files.
26 26
27 These are the different parts of the framework: 27 Most users will use these two features:
28 28
29 * Chromium Perf Tryserver. Documentation [here](https://www.chromium.org/develop ers/cluster-telemetry). Webpage [here](http://skia-tree-status.appspot.com/skia- telemetry/chromium_try). 29 * Chromium Perf. Documentation [here](https://www.chromium.org/developers/cluste r-telemetry). Webpage [here](https://ct.skia.org/chromium_perf/).
30 * Skia Correctness Tryserver. Documentation [here](http://goto/skiatryclustertel emetry). Webpage [here](http://skia-tree-status.appspot.com/skia-telemetry/skia_ try). 30 * Run Lua Scripts. Documentation about lua bindings is [here](https://skia.org/u ser/special/lua). Webpage [here](https://ct.skia.org/lua_script/).
31 * Run Lua Scripts. Documentation about lua bindings is [here](https://skia.org/u ser/special/lua). Webpage [here](http://skia-tree-status.appspot.com/skia-teleme try/lua_script).
32 31
33 32
34 <a name="framework_usage"></a> 33 <a name="framework_usage"></a>
35 Framework Usage 34 Framework Usage
36 --------------- 35 ---------------
37 36
38 The Chromium Perf tryserver in CT has been used to gather perf data over the top 10k web pages for the following Chromium projects: 37 The Chromium Perf tryserver in CT has been used to gather perf data over the top 10k web pages for the following Chromium projects:
39 38
40 * Slimming paint 39 * Slimming paint
41 * Performance data for layer squashing and compositing overlap map 40 * Performance data for layer squashing and compositing overlap map
(...skipping 11 matching lines...) Expand all
53 52
54 * [Loading measurement: alexa top 1,000](https://docs.google.com/a/chromium.org/ document/d/1ca_Q7xePmCRqaYnHe7vkpCmKNFNLdDXvzgtUPt9iG8w/edit) 53 * [Loading measurement: alexa top 1,000](https://docs.google.com/a/chromium.org/ document/d/1ca_Q7xePmCRqaYnHe7vkpCmKNFNLdDXvzgtUPt9iG8w/edit)
55 * [Loading measurement: alexa top million](https://docs.google.com/a/google.com/ document/d/1hDDUUNE5OUV8eCjtOj7Ow6EZ2DSBCTjQirnA3Rp5pOg/edit) 54 * [Loading measurement: alexa top million](https://docs.google.com/a/google.com/ document/d/1hDDUUNE5OUV8eCjtOj7Ow6EZ2DSBCTjQirnA3Rp5pOg/edit)
56 * [Loading measurement: alexa top million netsim](https://docs.google.com/a/goog le.com/document/d/1cpLSSYpqi4SprkJcVxbS7af6avKM0qc-imxvkexmCZs/edit) 55 * [Loading measurement: alexa top million netsim](https://docs.google.com/a/goog le.com/document/d/1cpLSSYpqi4SprkJcVxbS7af6avKM0qc-imxvkexmCZs/edit)
57 * [Perf profile - alexa top million sites](https://docs.google.com/a/google.com/ document/d/1di__87watociuZj_dm22Cn72UM2xsZBXixbl8TCFQmw/edit) 56 * [Perf profile - alexa top million sites](https://docs.google.com/a/google.com/ document/d/1di__87watociuZj_dm22Cn72UM2xsZBXixbl8TCFQmw/edit)
58 57
59 The framework has also been used to run multiple lua scripts to scrape the SKP r epositories for the the following: 58 The framework has also been used to run multiple lua scripts to scrape the SKP r epositories for the the following:
60 chars-vs-glyphs, bitmap transform types, gradient color counter, 3 color gradien t checks, etc. 59 chars-vs-glyphs, bitmap transform types, gradient color counter, 3 color gradien t checks, etc.
61 This has been very useful for the Skia team to help determine which parts of the library to optimize and focus on. 60 This has been very useful for the Skia team to help determine which parts of the library to optimize and focus on.
62 61
63 All runs are recorded [here](http://skia-tree-status.appspot.com/skia-telemetry/ all_tasks). 62 All runs are recorded [here](https://ct.skia.org/history/).
64 63
65 64
66 <a name="system_architecture"></a> 65 <a name="system_architecture"></a>
67 System Architecture 66 System Architecture
68 ------------------ 67 -------------------
69 68
70 <a name="system_diagram"></a> 69 <a name="system_diagram"></a>
71 ### System Diagram 70 ### System Diagram
72 71
73 ![CT System Diagram](ct-system.png) 72 ![CT System Diagram](ct-system-diagram.svg)
74 73
75 74
76 <a name="detailed_explanation"></a> 75 <a name="detailed_explanation"></a>
77 ### Detailed explanation of steps 76 ### Detailed explanation of steps
78 77
79 1. User submits an Admin task (rebuild chrome, recreate pagesets, recreate webpa ge archives), Lua script task, or Telemetry benchmark task using the App Engine web application [here](http://ct.skia.org). 78 1. User submits a Lua script task, a Performance task, or an Admin task (rebuild chrome, recreate pagesets, recreate webpage archives, capture SKPs) using the G CE web application [here](http://ct.skia.org).
80 79
81 2. Each task is exposed by the web application in JSON. The CT master polls the web application and picks up new tasks. 80 2. Each task is exposed by the web application in JSON. The CT master polls the web application and picks up new tasks.
82 81
83 3. The master pushes new tasks to all the workers using the master scripts [here ](https://skia.googlesource.com/buildbot/+/master/ct/go/master_scripts/) (in a n ew process so that the poller is not blocked). The master scripts then check to see when the workers are done with the requested task. 82 3. The master pushes new tasks to all the workers using the master scripts [here ](https://skia.googlesource.com/buildbot/+/master/ct/go/master_scripts/). The ma ster scripts then check to see when the workers are done with the requested task .
84 83
85 4. The workers execute the task using the worker scripts [here](https://skia.goo glesource.com/buildbot/+/master/ct/go/worker_scripts/). All generated artifacts (CSV files, logs, SKP files, archives, etc) are then stored locally and copied t o Google Storage. 84 4. The workers execute the task using the worker scripts [here](https://skia.goo glesource.com/buildbot/+/master/ct/go/worker_scripts/). All generated artifacts (CSV files, logs, SKP files, archives, etc) are then stored locally and copied t o Google Storage.
86 85
87 5. The master scripts periodically check the workers to see when they are done w ith the requested task. Once the workers are done the generated artifacts are th en read from Google Storage and consolidated (if required). 86 5. The master scripts periodically check the workers to see when they are done w ith the requested task. Once the workers are done the generated artifacts are th en read from Google Storage and consolidated (if required).
88 87
89 6. The master scripts then email results of the task to the user who requested i t. The master scripts also update the status of the task to completed on App Eng ine. 88 6. The master scripts then email results of the task to the user who requested i t. The master scripts also update the status of the task to completed on App Eng ine.
90 89
91 90
92 <a name="code"></a> 91 <a name="code"></a>
93 Code 92 Code
94 ---- 93 ----
95 94
96 Cluster Telemetry is primarily written in Go with a few python scripts. The fram ework lives in [master/ct](https://skia.googlesource.com/buildbot/+/master/ct) a nd the appengine code lives in [master/appengine_scripts](https://skia.googlesou rce.com/buildbot/+/master/appengine_scripts/skia-tree-status). 95 Cluster Telemetry is primarily written in Go with a few python scripts. The fram ework lives in [master/ct](https://skia.googlesource.com/buildbot/+/master/ct).
97 96
98 <a name="contact_us"></a> 97 <a name="contact_us"></a>
99 Contact Us 98 Contact Us
100 ---------- 99 ----------
101 100
102 If you have questions, please email <cluster-telemetry@chromium.org> or contact rmistry@ directly. 101 If you have questions, please email <cluster-telemetry@chromium.org> or contact rmistry@ directly.
OLDNEW
« no previous file with comments | « no previous file | site/dev/testing/ct-system.png » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698