Chromium Code Reviews| OLD | NEW |
|---|---|
| 1 Cluster Telemetry | 1 Cluster Telemetry |
| 2 ================= | 2 ================= |
| 3 | 3 |
| 4 ### Contents ### | 4 ### Contents ### |
| 5 | 5 |
| 6 * [Overview](#overview) | 6 * [Overview](#overview) |
| 7 * [Framework Usage](#framework_usage) | 7 * [Framework Usage](#framework_usage) |
| 8 * [System Architecture](#system_architecture) | 8 * [System Architecture](#system_architecture) |
| 9 + [System Diagram](#system_diagram) | 9 + [System Diagram](#system_diagram) |
| 10 + [Detailed explanation of steps](#detailed_explanation) | 10 + [Detailed explanation of steps](#detailed_explanation) |
| 11 * [Code](#code) | 11 * [Code](#code) |
| 12 * [Contact Us](#contact_us) | 12 * [Contact Us](#contact_us) |
| 13 | 13 |
| 14 <a name="overview"></a> | 14 <a name="overview"></a> |
| 15 Overview | 15 Overview |
| 16 -------- | 16 -------- |
| 17 | 17 |
| 18 Cluster Telemetry allows you to run [telemetry](https://www.chromium.org/develop ers/telemetry)'s benchmarks, lua scripts and other tasks using multiple reposito ry patches through Alexa's [top 1 million](http://s3.amazonaws.com/alexa-static/ top-1m.csv.zip) web pages. | 18 Cluster Telemetry allows you to run [telemetry](https://www.chromium.org/develop ers/telemetry)'s benchmarks, lua scripts and other tasks using multiple reposito ry patches through Alexa's [top 1 million](http://s3.amazonaws.com/alexa-static/ top-1m.csv.zip) web pages. |
| 19 Developers can use the framework to measure the performance of their patch again st the top subset of the internet on both Desktop and Android. | 19 Developers can use the framework to measure the performance of their patch again st the top subset of the internet on both Desktop and Android. |
| 20 | 20 |
| 21 SKP files are a binary format for the draw commands Chromium sends to Skia for r asterization. The goal of the project started off with wanting to collect a larg e repository of 10k SKP files. This repository, after incremental changes in app roaches, has since grown to ~900k and now supports running all telemetry benchma rks. The top level feature request of this project was [skia:1268](https://code. google.com/p/skia/issues/detail?id=1268). | 21 SKP files are a binary format for the draw commands Chromium sends to Skia for r asterization. The goal of the project started off with wanting to collect a larg e repository of 10k SKP files. This repository, after incremental changes in app roaches, has since grown to ~900k and now supports running all telemetry benchma rks. The top level feature request of this project was [skia:1268](https://code. google.com/p/skia/issues/detail?id=1268). |
|
rmistry
2015/08/28 12:32:22
Could you please update this to ~910k.
dogben
2015/08/28 17:17:00
Done.
| |
| 22 | 22 |
| 23 A web application has been created on App Engine that automates the process of c apturing new archives and running telemetry benchmarks at a click of a button; r esults are emailed to the requester and the web application contains complete hi story of runs with links to results. You can run telemetry benchmarks at http:// ct.skia.org. | 23 A web application has been created on Google Compute Engine that automates the p rocess of capturing new archives and running telemetry benchmarks at a click of a button; results are emailed to the requester and the web application contains complete history of runs with links to results. You can run telemetry benchmarks at http://ct.skia.org. |
| 24 | 24 |
| 25 The framework also contains the ability to run lua scripts on the SKP repository to scrape web pages. It only takes a few minutes to run a lua scrapping script on ~900k SKP files. | 25 The framework also contains the ability to run lua scripts on the SKP repository to scrape web pages. It only takes a few minutes to run a lua scraping script o n ~900k SKP files. |
| 26 | 26 |
| 27 These are the different parts of the framework: | 27 Most users will use these two features: |
| 28 | 28 |
| 29 * Chromium Perf Tryserver. Documentation [here](https://www.chromium.org/develop ers/cluster-telemetry). Webpage [here](http://skia-tree-status.appspot.com/skia- telemetry/chromium_try). | 29 * Chromium Perf. Documentation [here](https://www.chromium.org/developers/cluste r-telemetry). Webpage [here](https://ct.skia.org/chromium_perf/). |
| 30 * Skia Correctness Tryserver. Documentation [here](http://goto/skiatryclustertel emetry). Webpage [here](http://skia-tree-status.appspot.com/skia-telemetry/skia_ try). | 30 * Run Lua Scripts. Documentation about lua bindings is [here](https://skia.org/u ser/special/lua). Webpage [here](https://ct.skia.org/lua_script/). |
| 31 * Run Lua Scripts. Documentation about lua bindings is [here](https://skia.org/u ser/special/lua). Webpage [here](http://skia-tree-status.appspot.com/skia-teleme try/lua_script). | |
| 32 | 31 |
| 33 | 32 |
| 34 <a name="framework_usage"></a> | 33 <a name="framework_usage"></a> |
| 35 Framework Usage | 34 Framework Usage |
| 36 --------------- | 35 --------------- |
| 37 | 36 |
| 38 The Chromium Perf tryserver in CT has been used to gather perf data over the top 10k web pages for the following Chromium projects: | 37 The Chromium Perf tryserver in CT has been used to gather perf data over the top 10k web pages for the following Chromium projects: |
| 39 | 38 |
| 40 * Slimming paint | 39 * Slimming paint |
| 41 * Performance data for layer squashing and compositing overlap map | 40 * Performance data for layer squashing and compositing overlap map |
| (...skipping 11 matching lines...) Expand all Loading... | |
| 53 | 52 |
| 54 * [Loading measurement: alexa top 1,000](https://docs.google.com/a/chromium.org/ document/d/1ca_Q7xePmCRqaYnHe7vkpCmKNFNLdDXvzgtUPt9iG8w/edit) | 53 * [Loading measurement: alexa top 1,000](https://docs.google.com/a/chromium.org/ document/d/1ca_Q7xePmCRqaYnHe7vkpCmKNFNLdDXvzgtUPt9iG8w/edit) |
| 55 * [Loading measurement: alexa top million](https://docs.google.com/a/google.com/ document/d/1hDDUUNE5OUV8eCjtOj7Ow6EZ2DSBCTjQirnA3Rp5pOg/edit) | 54 * [Loading measurement: alexa top million](https://docs.google.com/a/google.com/ document/d/1hDDUUNE5OUV8eCjtOj7Ow6EZ2DSBCTjQirnA3Rp5pOg/edit) |
| 56 * [Loading measurement: alexa top million netsim](https://docs.google.com/a/goog le.com/document/d/1cpLSSYpqi4SprkJcVxbS7af6avKM0qc-imxvkexmCZs/edit) | 55 * [Loading measurement: alexa top million netsim](https://docs.google.com/a/goog le.com/document/d/1cpLSSYpqi4SprkJcVxbS7af6avKM0qc-imxvkexmCZs/edit) |
| 57 * [Perf profile - alexa top million sites](https://docs.google.com/a/google.com/ document/d/1di__87watociuZj_dm22Cn72UM2xsZBXixbl8TCFQmw/edit) | 56 * [Perf profile - alexa top million sites](https://docs.google.com/a/google.com/ document/d/1di__87watociuZj_dm22Cn72UM2xsZBXixbl8TCFQmw/edit) |
| 58 | 57 |
| 59 The framework has also been used to run multiple lua scripts to scrape the SKP r epositories for the the following: | 58 The framework has also been used to run multiple lua scripts to scrape the SKP r epositories for the the following: |
| 60 chars-vs-glyphs, bitmap transform types, gradient color counter, 3 color gradien t checks, etc. | 59 chars-vs-glyphs, bitmap transform types, gradient color counter, 3 color gradien t checks, etc. |
| 61 This has been very useful for the Skia team to help determine which parts of the library to optimize and focus on. | 60 This has been very useful for the Skia team to help determine which parts of the library to optimize and focus on. |
| 62 | 61 |
| 63 All runs are recorded [here](http://skia-tree-status.appspot.com/skia-telemetry/ all_tasks). | 62 All runs are recorded [here](https://ct.skia.org/history/). |
| 64 | 63 |
| 65 | 64 |
| 66 <a name="system_architecture"></a> | 65 <a name="system_architecture"></a> |
| 67 System Architecture | 66 System Architecture |
| 68 ------------------ | 67 ------------------- |
| 69 | 68 |
| 70 <a name="system_diagram"></a> | 69 <a name="system_diagram"></a> |
| 71 ### System Diagram | 70 ### System Diagram |
| 72 | 71 |
| 73  | 72  |
| 74 | 73 |
| 75 | 74 |
| 76 <a name="detailed_explanation"></a> | 75 <a name="detailed_explanation"></a> |
| 77 ### Detailed explanation of steps | 76 ### Detailed explanation of steps |
| 78 | 77 |
| 79 1. User submits an Admin task (rebuild chrome, recreate pagesets, recreate webpa ge archives), Lua script task, or Telemetry benchmark task using the App Engine web application [here](http://ct.skia.org). | 78 1. User submits a Lua script task, a Performance task, or an Admin task (rebuild chrome, recreate pagesets, recreate webpage archives, capture SKPs) using the G CE web application [here](http://ct.skia.org). |
| 80 | 79 |
| 81 2. Each task is exposed by the web application in JSON. The CT master polls the web application and picks up new tasks. | 80 2. Each task is exposed by the web application in JSON. The CT master polls the web application and picks up new tasks. |
| 82 | 81 |
| 83 3. The master pushes new tasks to all the workers using the master scripts [here ](https://skia.googlesource.com/buildbot/+/master/ct/go/master_scripts/) (in a n ew process so that the poller is not blocked). The master scripts then check to see when the workers are done with the requested task. | 82 3. The master pushes new tasks to all the workers using the master scripts [here ](https://skia.googlesource.com/buildbot/+/master/ct/go/master_scripts/). The ma ster scripts then check to see when the workers are done with the requested task . |
| 84 | 83 |
| 85 4. The workers execute the task using the worker scripts [here](https://skia.goo glesource.com/buildbot/+/master/ct/go/worker_scripts/). All generated artifacts (CSV files, logs, SKP files, archives, etc) are then stored locally and copied t o Google Storage. | 84 4. The workers execute the task using the worker scripts [here](https://skia.goo glesource.com/buildbot/+/master/ct/go/worker_scripts/). All generated artifacts (CSV files, logs, SKP files, archives, etc) are then stored locally and copied t o Google Storage. |
| 86 | 85 |
| 87 5. The master scripts periodically check the workers to see when they are done w ith the requested task. Once the workers are done the generated artifacts are th en read from Google Storage and consolidated (if required). | 86 5. The master scripts periodically check the workers to see when they are done w ith the requested task. Once the workers are done the generated artifacts are th en read from Google Storage and consolidated (if required). |
| 88 | 87 |
| 89 6. The master scripts then email results of the task to the user who requested i t. The master scripts also update the status of the task to completed on App Eng ine. | 88 6. The master scripts then email results of the task to the user who requested i t. The master scripts also update the status of the task to completed on App Eng ine. |
| 90 | 89 |
| 91 | 90 |
| 92 <a name="code"></a> | 91 <a name="code"></a> |
| 93 Code | 92 Code |
| 94 ---- | 93 ---- |
| 95 | 94 |
| 96 Cluster Telemetry is primarily written in Go with a few python scripts. The fram ework lives in [master/ct](https://skia.googlesource.com/buildbot/+/master/ct) a nd the appengine code lives in [master/appengine_scripts](https://skia.googlesou rce.com/buildbot/+/master/appengine_scripts/skia-tree-status). | 95 Cluster Telemetry is primarily written in Go with a few python scripts. The fram ework lives in [master/ct](https://skia.googlesource.com/buildbot/+/master/ct). |
| 97 | 96 |
| 98 <a name="contact_us"></a> | 97 <a name="contact_us"></a> |
| 99 Contact Us | 98 Contact Us |
| 100 ---------- | 99 ---------- |
| 101 | 100 |
| 102 If you have questions, please email <cluster-telemetry@chromium.org> or contact rmistry@ directly. | 101 If you have questions, please email <cluster-telemetry@chromium.org> or contact rmistry@ directly. |
| OLD | NEW |