OLD | NEW |
1 # DOM Distiller | 1 # DOM Distiller |
2 | 2 |
3 DOM Distiller aims to provide a better reading experience by distilling the | 3 DOM Distiller aims to provide a better reading experience by distilling the |
4 content of the page. This distilled content can then be used in a variety of | 4 content of the page. This distilled content can then be used in a variety of |
5 ways. | 5 ways. |
6 | 6 |
7 The current efforts that will be powered by DOM Distiller: | 7 The current efforts that will be powered by DOM Distiller: |
8 - Reader mode: a mobile-friendly viewing mode for Chrome mobile | 8 |
| 9 - Reader mode: a mobile-friendly viewing mode for Chrome mobile |
9 | 10 |
10 ## How to use Reader mode on mobile Chrome | 11 ## How to use Reader mode on mobile Chrome |
11 | 12 |
12 - Open Chrome on your mobile device | 13 - Open Chrome on your mobile device |
13 - Navigate to [chrome://flags](chrome://flags) and search for "Reader mode" | 14 - Navigate to [chrome://flags](chrome://flags) and search for "Reader mode" |
14 (Menu -> Find in page -> Enable Reader Mode Toolbar Icon), or directly go to | 15 (Menu -> Find in page -> Enable Reader Mode Toolbar Icon), or directly go to |
15 [chrome://flags#enable-reader-mode-toolbar-icon](chrome://flags#enable-reader-
mode-toolbar-icon) | 16 [chrome://flags#enable-reader-mode-toolbar-icon] |
16 - Click Enable to turn on Reader mode | 17 (chrome://flags#enable-reader-mode-toolbar-icon) |
17 - Click "Relaunch Now" at the bottom of the page | 18 - Click Enable to turn on Reader mode |
18 - Next time you're trying to read a page, tap on the "Reader mode" icon in | 19 - Click "Relaunch Now" at the bottom of the page |
19 the toolbar to try it out! | 20 - Next time you're trying to read a page, tap on the "Reader mode" icon in the |
| 21 toolbar to try it out! |
20 | 22 |
21 # Continuous integration | 23 # Continuous integration |
22 | 24 |
23 - [](https://drone.io/github.com/chromium/dom-distiller/latest) | 25 - [![Build Status] |
24 - [drone.io waterfall](https://drone.io/github.com/chromium/dom-distiller) | 26 (https://drone.io/github.com/chromium/dom-distiller/status.png)] |
| 27 (https://drone.io/github.com/chromium/dom-distiller/latest) |
| 28 - [drone.io waterfall](https://drone.io/github.com/chromium/dom-distiller) |
25 | 29 |
26 # Get the code | 30 # Get the code |
27 | 31 |
28 In a folder where you want the code (outside of the chromium checkout): | 32 In a folder where you want the code (outside of the chromium checkout): |
29 | 33 |
30 ```bash | 34 ```bash |
31 git clone https://github.com/chromium/dom-distiller.git | 35 git clone https://github.com/chromium/dom-distiller.git |
32 ``` | 36 ``` |
| 37 |
33 A `dom-distiller` folder will be created in the folder you run that command. | 38 A `dom-distiller` folder will be created in the folder you run that command. |
34 | 39 |
35 # Environment setup | 40 # Environment setup |
36 | 41 |
37 Before you build for the first time, you need to install the build dependencies. | 42 Before you build for the first time, you need to install the build dependencies. |
38 | 43 |
39 For all platforms, it is require to download and install | 44 For all platforms, it is require to download and install [Google Chrome browser] |
40 [Google Chrome browser](https://www.google.com/chrome/browser/desktop/). | 45 (https://www.google.com/chrome/browser/desktop/). |
41 | 46 |
42 ChromeDriver requires Google Chrome to be installed at a specific location | 47 ChromeDriver requires Google Chrome to be installed at a specific location (the |
43 (the default location for the platform). See | 48 default location for the platform). See [ChromeDriver documentation] |
44 [ChromeDriver documentation](https://code.google.com/p/selenium/wiki/ChromeDrive
r) | 49 (https://code.google.com/p/selenium/wiki/ChromeDriver) for details. |
45 for details. | |
46 | 50 |
47 ## Developing on Ubuntu/Debian | 51 ## Developing on Ubuntu/Debian |
48 | 52 |
49 Install the dependencies by entering the `dom-distiller` folder and running: | 53 Install the dependencies by entering the `dom-distiller` folder and running: |
| 54 |
50 ```bash | 55 ```bash |
51 sudo ./install-build-deps.sh | 56 sudo ./install-build-deps.sh |
52 ``` | 57 ``` |
53 | 58 |
54 Ubuntu 14.04 64-bit is recommended. | 59 Ubuntu 14.04 64-bit is recommended. |
55 | 60 |
56 ## Developing on Mac OS X | 61 ## Developing on Mac OS X |
57 | 62 |
58 - Install JDK 7 using either your organizations software management tool, | 63 - Install JDK 7 using either your organizations software management tool, or |
59 or download it from | 64 download it from [Oracle] |
60 [Oracle](http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-
1880260.html). | 65 (http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880
260.html). |
61 - Install [Homebrew](http://brew.sh/). | 66 - Install [Homebrew](http://brew.sh/). |
62 - Install `ant` and `python` using Homebrew: | 67 - Install `ant` and `python` using Homebrew: |
63 ```bash | 68 |
64 brew install ant python | 69 ```bash |
65 ``` | 70 brew install ant python |
66 - Since both the protocol buffer compiler and Python bindings are needed, | 71 ``` |
67 install the `protobuf` package with the `--with-python` command line parameter: | 72 |
68 ```bash | 73 - Since both the protocol buffer compiler and Python bindings are needed, |
69 brew install protobuf --with-python | 74 install the `protobuf` package with the `--with-python` command line |
70 ``` | 75 parameter: |
71 - Create a folder named `buildtools` inside your DOM Distiller checkout | 76 |
72 - Download ChromeDriver (chromedriver_mac32.zip) from the | 77 ```bash |
73 [Download page](https://sites.google.com/a/chromium.org/chromedriver/downloads) | 78 brew install protobuf --with-python |
74 - Unzip the `chromedriver_mac32.zip` and ensure the binary ends up in your | 79 ``` |
75 `buildtools` folder. | 80 |
76 - Install the PyPI package management tool `pip` by running: | 81 - Create a folder named `buildtools` inside your DOM Distiller checkout |
77 ``` | 82 |
78 sudo easy_install pip | 83 - Download ChromeDriver (chromedriver_mac32.zip) from the [Download page] |
79 ``` | 84 (https://sites.google.com/a/chromium.org/chromedriver/downloads) |
80 - Install `selenium` using `pip`: | 85 |
81 ``` | 86 - Unzip the `chromedriver_mac32.zip` and ensure the binary ends up in your |
82 pip install --user selenium | 87 `buildtools` folder. |
83 ``` | 88 |
| 89 - Install the PyPI package management tool `pip` by running: |
| 90 |
| 91 ```bash |
| 92 sudo easy_install pip |
| 93 ``` |
| 94 |
| 95 - Install `selenium` using `pip`: |
| 96 |
| 97 ```bash |
| 98 pip install --user selenium |
| 99 ``` |
84 | 100 |
85 For the rest of this guide, there are sometimes references to a tool called | 101 For the rest of this guide, there are sometimes references to a tool called |
86 `xvfb` and specifically when running shell commands using `xvfb-run`. When you | 102 `xvfb` and specifically when running shell commands using `xvfb-run`. When you |
87 develop using a Mac OS X, you can remove that part of the command. For example | 103 develop using a Mac OS X, you can remove that part of the command. For example |
88 `xvfb-run echo` would just become `echo`. | 104 `xvfb-run echo` would just become `echo`. |
89 | 105 |
90 ## Developing with Vagrant | 106 ## Developing with Vagrant |
91 | 107 |
92 This option could be useful if you want to develop on an unsupported system | 108 This option could be useful if you want to develop on an unsupported system like |
93 like Windows or Red Hat Linux. Even if you are on a supported system but | 109 Windows or Red Hat Linux. Even if you are on a supported system but would rather |
94 would rather not touch the system too much, Vagrant is a viable alternative. | 110 not touch the system too much, Vagrant is a viable alternative. |
95 | 111 |
96 The Vagrant VM is based on Ubuntu 14.04. | 112 The Vagrant VM is based on Ubuntu 14.04. |
97 | 113 |
98 - [Install Vagrant](http://www.vagrantup.com/downloads.html) on your system. | 114 - [Install Vagrant](http://www.vagrantup.com/downloads.html) on your system. |
99 Version 1.7.2 or higher is recommended. | 115 Version 1.7.2 or higher is recommended. |
100 - Launch the Vagrant VM instance | 116 - Launch the Vagrant VM instance |
101 ```bash | 117 |
102 vagrant up | 118 ```bash |
103 ``` | 119 vagrant up |
104 - SSH to the VM | 120 ``` |
105 ```bash | 121 |
106 vagrant ssh | 122 - SSH to the VM |
107 ``` | 123 |
| 124 ```bash |
| 125 vagrant ssh |
| 126 ``` |
108 | 127 |
109 ## Tools for contributing | 128 ## Tools for contributing |
110 | 129 |
111 The DOM Distiller project uses the Chromium tools for collaboration. For code | 130 The DOM Distiller project uses the Chromium tools for collaboration. For code |
112 reviews, the | 131 reviews, the [Chromium Rietveld code review tool] |
113 [Chromium Rietveld code review tool](https://codereview.chromium.org/) is used | 132 (https://codereview.chromium.org/) is used and the set of tools found in |
114 and the set of tools found in `depot_tools` is also required. | 133 `depot_tools` is also required. |
115 | 134 |
116 To get `depot_tools`, follow the guide at | 135 To get `depot_tools`, follow the guide at [Chrome infrastructure documentation |
117 [Chrome infrastructure documentation for depot_tools](http://commondatastorage.g
oogleapis.com/chrome-infra-docs/flat/depot_tools/docs/html/depot_tools_tutorial.
html#_setting_up). | 136 for depot_tools] |
| 137 (http://commondatastorage.googleapis.com/chrome-infra-docs/flat/depot_tools/docs
/html/depot_tools_tutorial.html#_setting_up). |
118 | 138 |
119 The TL;DR of that is to run this from a folder where you install developer | 139 The TL;DR of that is to run this from a folder where you install developer |
120 tools, for example in your `$HOME` folder: | 140 tools, for example in your `$HOME` folder: |
| 141 |
121 ```bash | 142 ```bash |
122 git clone https://chromium.googlesource.com/chromium/tools/depot_tools | 143 git clone https://chromium.googlesource.com/chromium/tools/depot_tools |
123 export PATH="/path/to/depot_tools:$PATH" | 144 export PATH="/path/to/depot_tools:$PATH" |
124 ``` | 145 ``` |
125 | 146 |
126 You must also setup your local checkout needs to point to the Chromium Rietveld | 147 You must also setup your local checkout needs to point to the Chromium Rietveld |
127 server. This is a one-time setup for your checkout, so from your `dom-distiller` | 148 server. This is a one-time setup for your checkout, so from your `dom-distiller` |
128 checkout folder, run: | 149 checkout folder, run: |
| 150 |
129 ```bash | 151 ```bash |
130 git cl config | 152 git cl config |
131 ``` | 153 ``` |
132 - `Rietveld server`: `https://codereview.chromium.org` | 154 |
133 - You can leave the rest of the fields blank. | 155 - `Rietveld server`: `https://codereview.chromium.org` |
| 156 - You can leave the rest of the fields blank. |
134 | 157 |
135 # Building | 158 # Building |
136 | 159 |
137 ## Using ant | 160 ## Using ant |
138 | 161 |
139 `ant` is the tool we use to build, and the available targets can be listed using | 162 `ant` is the tool we use to build, and the available targets can be listed using |
140 `ant -p`, but the typical targets you might use when you work on this project | 163 `ant -p`, but the typical targets you might use when you work on this project |
141 is: | 164 is: |
142 | 165 |
143 - `ant test` Runs all tests. | 166 - `ant test` Runs all tests. |
144 - `ant test -Dtest.filter=$FILTER_PATTERN` where `$FILTER_PATTERN` is a [gtest\_
filter | 167 - `ant test -Dtest.filter=$FILTER_PATTERN` where `$FILTER_PATTERN` is a |
145 pattern] | 168 [gtest\_filter pattern] |
146 (https://code.google.com/p/googletest/wiki/AdvancedGuide#Running_a_Subset_of_the
_Tests). | 169 (https://code.google.com/p/googletest/wiki/AdvancedGuide#Running_a_Subset_of
_the_Tests). |
147 For example `*.FilterTest.*:*Foo*-*Bar*` would run all tests containing `.Filter
Test.` | 170 For example `*.FilterTest.*:*Foo*-*Bar*` would run all tests containing |
148 and `Foo`, but not those with `Bar`. | 171 `.FilterTest.` and `Foo`, but not those with `Bar`. |
149 - `ant gwtc` compiles .class+.java files to JavaScript. Standalone JavaScript | 172 - `ant gwtc` compiles .class+.java files to JavaScript. Standalone JavaScript |
150 is available at `war/domdistiller/domdistiller.nocache.js`. | 173 is available at `war/domdistiller/domdistiller.nocache.js`. |
151 - `ant gwtc.jstests` creates a standalone JavaScript for the tests. | 174 - `ant gwtc.jstests` creates a standalone JavaScript for the tests. |
152 - `ant package` Copies the main build artifacts into the `out/package` folder, | 175 - `ant package` Copies the main build artifacts into the `out/package` folder, |
153 typically the extracted JS and protocol buffer files. | 176 typically the extracted JS and protocol buffer files. |
154 | 177 |
155 # Contributing | 178 # Contributing |
156 | 179 |
157 You can use regular `git` command when developing in this project and use | 180 You can use regular `git` command when developing in this project and use `git |
158 `git cl` for collaboration. | 181 cl` for collaboration. |
159 | 182 |
160 ## Uploading a CL for review | 183 ## Uploading a CL for review |
161 | 184 |
162 On your branch, run: `git cl upload`. The first time you do this, you will have | 185 On your branch, run: `git cl upload`. The first time you do this, you will have |
163 to provide a username and password. | 186 to provide a username and password. |
164 - For username, use your @chromium.org. account. | 187 |
165 - For password, get it from | 188 - For username, use your @chromium.org. account. |
166 [GoogleCode.com settings page](https://code.google.com/hosting/settings) when | 189 - For password, get it from [GoogleCode.com settings page] |
167 logged into your @chromium.org account, and add the full | 190 (https://code.google.com/hosting/settings) when logged into your |
168 `machine code.google.com login` line to your `~/.netrc` file. | 191 @chromium.org account, and add the full `machine code.google.com login` line |
| 192 to your `~/.netrc` file. |
169 | 193 |
170 ## Committing a CL | 194 ## Committing a CL |
171 | 195 |
172 - Change upstream to remote master, push cl, then revert upstream to local: | 196 - Change upstream to remote master, push cl, then revert upstream to local: |
173 ```bash | 197 |
174 git branch -u origin/master | 198 ```bash |
175 git cl land | 199 git branch -u origin/master |
176 git branch -u master | 200 git cl land |
177 ``` | 201 git branch -u master |
178 - For username, use your GitHub account name (the username, not the full | 202 ``` |
179 e-mail). | 203 |
180 - For password, use your GitHub password. | 204 - For username, use your GitHub account name (the username, not the full |
181 - If you have two-factor authentication enabled, create a personal access | 205 e-mail). |
182 token at your | 206 |
183 [application settings page](https://github.com/settings/applications) and use | 207 - For password, use your GitHub password. |
184 that as your password. | 208 |
| 209 - If you have two-factor authentication enabled, create a personal access |
| 210 token at your [application settings page] |
| 211 (https://github.com/settings/applications) and use that as your |
| 212 password. |
185 | 213 |
186 ## Code formatting | 214 ## Code formatting |
187 | 215 |
188 Before uploading a CL it is recommended to run `git cl format`. However, this | 216 Before uploading a CL it is recommended to run `git cl format`. However, this |
189 requires adding symbolic links to your chromium checkout. | 217 requires adding symbolic links to your chromium checkout. |
190 | 218 |
191 Inside the `buildtools` folder of your checkout, add the following symbolic | 219 Inside the `buildtools` folder of your checkout, add the following symbolic |
192 links: | 220 links: |
193 - `clang_format`» → `/path/to/chromium/src/buildtools/clang_format/` | 221 |
194 - `linux64` ->» → `/path/to/chromium/src/buildtools/linux64/` (only for Lin
ux 64-bit platform) | 222 - `clang_format` → `/path/to/chromium/src/buildtools/clang_format/` |
195 - `mac` → `/path/to/chromium/mac/buildtools/linux64/` (only for Mac platfor
m) | 223 - `linux64` → `/path/to/chromium/src/buildtools/linux64/` (only for Linux |
| 224 64-bit platform) |
| 225 - `mac` → `/path/to/chromium/mac/buildtools/linux64/` (only for Mac |
| 226 platform) |
196 | 227 |
197 Doing this enables you to run the command `git cl format` to fix the formatting | 228 Doing this enables you to run the command `git cl format` to fix the formatting |
198 of your code. | 229 of your code. |
199 | 230 |
200 # Run in Chrome for desktop | 231 # Run in Chrome for desktop |
201 | 232 |
202 In this section, the following shell variables and are assumed correctly set: | 233 In this section, the following shell variables and are assumed correctly set: |
| 234 |
203 ```bash | 235 ```bash |
204 export CHROME_SRC=/path/to/chromium/src | 236 export CHROME_SRC=/path/to/chromium/src |
205 export DOM_DISTILLER_DIR=/path/to/dom-distiller | 237 export DOM_DISTILLER_DIR=/path/to/dom-distiller |
206 ``` | 238 ``` |
207 | 239 |
208 - Pull generated package (from ant package) into Chrome. You can use this handy | 240 - Pull generated package (from ant package) into Chrome. You can use this |
209 bash-function to help with that: | 241 handy bash-function to help with that: |
210 ```bash | 242 |
211 roll-distiller () { | 243 ```bash |
212 ( | 244 roll-distiller () { |
213 (cd $DOM_DISTILLER_DIR && ant package) && \ | 245 ( |
214 rm -rf $CHROME_SRC/third_party/dom_distiller_js/dist/* && \ | 246 (cd $DOM_DISTILLER_DIR && ant package) && \ |
215 cp -rf $DOM_DISTILLER_DIR/out/package/* $CHROME_SRC/third_party/dom_distille
r_js/dist/ && \ | 247 rm -rf $CHROME_SRC/third_party/dom_distiller_js/dist/* && \ |
216 touch $CHROME_SRC/components/resources/dom_distiller_resources.grdp | 248 cp -rf $DOM_DISTILLER_DIR/out/package/* $CHROME_SRC/third_party/dom_dist
iller_js/dist/ && \ |
217 ) | 249 touch $CHROME_SRC/components/resources/dom_distiller_resources.grdp |
218 } | 250 ) |
219 ``` | 251 } |
220 - From `$CHROME_SRC` run GYP to setup ninja build files using | 252 ``` |
221 ```bash | 253 |
222 build/gyp_chromium | 254 - From `$CHROME_SRC` run GYP to setup ninja build files using |
223 ``` | 255 |
| 256 ```bash |
| 257 build/gyp_chromium |
| 258 ``` |
224 | 259 |
225 ## Running the Chrome browser with distiller support | 260 ## Running the Chrome browser with distiller support |
226 | 261 |
227 - For running Chrome, you need to build the `chrome` target: | 262 - For running Chrome, you need to build the `chrome` target: |
228 ```bash | 263 |
229 ninja -C out/Debug chrome | 264 ```bash |
230 ``` | 265 ninja -C out/Debug chrome |
231 - Run chrome with DOM Distiller enabled: | 266 ``` |
232 ```bash | 267 |
233 out/Debug/chrome --enable-dom-distiller | 268 - Run chrome with DOM Distiller enabled: |
234 ``` | 269 |
235 - This adds a menu item `Distill page` that you can use to distill web pages. | 270 ```bash |
236 - You can also go to `chrome://dom-distiller` to access the debug page. | 271 out/Debug/chrome --enable-dom-distiller |
237 - To have a unique user profile every time you run Chrome, you can also add | 272 ``` |
238 `--user-data-dir=/tmp/$(mktemp -d)` as a command line parameter. | 273 |
239 On Mac OS X, you can instead write | 274 - This adds a menu item `Distill page` that you can use to distill web pages. |
240 `--user-data-dir=$(mktemp -d 2>/dev/null || mktemp -d -t 'chromeprofile')`. | 275 |
| 276 - You can also go to `chrome://dom-distiller` to access the debug page. |
| 277 |
| 278 - To have a unique user profile every time you run Chrome, you can also add |
| 279 `--user-data-dir=/tmp/$(mktemp -d)` as a command line parameter. On Mac OS |
| 280 X, you can instead write `--user-data-dir=$(mktemp -d 2>/dev/null || mktemp |
| 281 -d -t 'chromeprofile')`. |
241 | 282 |
242 ## Running the automated tests in Chromium | 283 ## Running the automated tests in Chromium |
243 | 284 |
244 - For running the tests, you need to build the `components_browsertests` target: | 285 - For running the tests, you need to build the `components_browsertests` |
245 ```bash | 286 target: |
246 ninja -C out/Debug components_browsertests | |
247 ``` | |
248 - Run the `components_browsertests` binary to execute the tests. You can prefix | |
249 the command with `xvfb-run` to avoid pop-up windows: | |
250 ```bash | |
251 xvfb-run out/Debug/components_browsertests | |
252 ``` | |
253 | 287 |
254 - To only run tests related to DOM Distiller, run: | 288 ```bash |
255 ```bash | 289 ninja -C out/Debug components_browsertests |
256 xvfb-run out/Debug/components_browsertests --gtest_filter=\*Distiller\* | 290 ``` |
257 ``` | |
258 | 291 |
259 - For running tests as isolates, you need to build `components_browsertests_run` | 292 - Run the `components_browsertests` binary to execute the tests. You can |
260 and execute them using the swarming tool: | 293 prefix the command with `xvfb-run` to avoid pop-up windows: |
261 ```bash | 294 |
262 ninja -C out/Debug components_browsertests_run | 295 ```bash |
263 python tools/swarming_client/isolate.py run -s out/Debug/components_browsertests
.isolated | 296 xvfb-run out/Debug/components_browsertests |
264 ``` | 297 ``` |
| 298 |
| 299 - To only run tests related to DOM Distiller, run: |
| 300 |
| 301 ```bash |
| 302 xvfb-run out/Debug/components_browsertests --gtest_filter=\*Distiller\* |
| 303 ``` |
| 304 |
| 305 - For running tests as isolates, you need to build |
| 306 `components_browsertests_run` and execute them using the swarming tool: |
| 307 |
| 308 ```bash |
| 309 ninja -C out/Debug components_browsertests_run |
| 310 python tools/swarming_client/isolate.py run -s out/Debug/components_browsert
ests.isolated |
| 311 ``` |
265 | 312 |
266 ## Running the content extractor | 313 ## Running the content extractor |
267 | 314 |
268 To extract the content from a web page directly, you can run: | 315 To extract the content from a web page directly, you can run: |
| 316 |
269 ```bash | 317 ```bash |
270 xvfb-run out/Debug/components_browsertests \ | 318 xvfb-run out/Debug/components_browsertests \ |
271 --gtest_filter='*MANUAL_ExtractUrl' \ | 319 --gtest_filter='*MANUAL_ExtractUrl' \ |
272 --run-manual \ | 320 --run-manual \ |
273 --test-tiny-timeout=600000 \ | 321 --test-tiny-timeout=600000 \ |
274 --output-file=./extract.out \ | 322 --output-file=./extract.out \ |
275 --url=http://www.example.com \ | 323 --url=http://www.example.com \ |
276 > ./extract.log 2>&1 | 324 > ./extract.log 2>&1 |
277 ``` | 325 ``` |
| 326 |
278 `extract.out` has the extracted HTML, `extract.log` has the console logging. | 327 `extract.out` has the extracted HTML, `extract.log` has the console logging. |
279 | 328 |
280 If you need more logging, you can add the following arguments to the command: | 329 If you need more logging, you can add the following arguments to the command: |
281 - Chrome browser: `--vmodule=*distiller*=2` | 330 |
282 - Content extractor: `--debug-level=99` | 331 - Chrome browser: `--vmodule=*distiller*=2` |
| 332 - Content extractor: `--debug-level=99` |
283 | 333 |
284 If this is something you often do, you can put the following function in a bash | 334 If this is something you often do, you can put the following function in a bash |
285 file you include (for example `~/.bashrc`) and use it for iterative development: | 335 file you include (for example `~/.bashrc`) and use it for iterative development: |
286 | 336 |
287 ```bash | 337 ```bash |
288 distill() { | 338 distill() { |
289 ( | 339 ( |
290 roll-distiller && \ | 340 roll-distiller && \ |
291 ninja -C out/Debug components_browsertests && | 341 ninja -C out/Debug components_browsertests && |
292 xvfb-run out/Debug/components_browsertests \ | 342 xvfb-run out/Debug/components_browsertests \ |
(...skipping 11 matching lines...) Expand all Loading... |
304 | 354 |
305 ```bash | 355 ```bash |
306 distill http://example.com/article.html | 356 distill http://example.com/article.html |
307 ``` | 357 ``` |
308 | 358 |
309 # Debug Code | 359 # Debug Code |
310 | 360 |
311 ## Interactive debugging | 361 ## Interactive debugging |
312 | 362 |
313 You can use the Chrome Developer Tools to debug DOM Distiller: | 363 You can use the Chrome Developer Tools to debug DOM Distiller: |
314 - Update the test JavaScript by running `ant gwtc.jstests` or `ant test`. | 364 |
315 - Open `war/test.html` in Chrome desktop | 365 - Update the test JavaScript by running `ant gwtc.jstests` or `ant test`. |
316 - Open the `Console` panel in Developer Tools (**Ctrl-Shift-J**). | 366 - Open `war/test.html` in Chrome desktop |
317 On Mac OS X you can use **⌥-⌘-I** (uppercase `I`) as the shortcut. | 367 - Open the `Console` panel in Developer Tools (**Ctrl-Shift-J**). On Mac OS X |
318 - Run all tests by calling: | 368 you can use **⌥-⌘-I** (uppercase `I`) as the shortcut. |
319 ```javascript | 369 - Run all tests by calling: |
320 org.chromium.distiller.JsTestEntry.run() | 370 |
321 ``` | 371 ```javascript |
322 - To run only a subset of tests, you can use a regular expression that matches a | 372 org.chromium.distiller.JsTestEntry.run() |
323 single test or multiple tests: | 373 ``` |
324 ```javascript | 374 |
325 org.chromium.distiller.JsTestEntry.runWithFilter('MyTestClass.testSomething') | 375 - To run only a subset of tests, you can use a regular expression that matches |
326 ``` | 376 a single test or multiple tests: |
| 377 |
| 378 ```javascript |
| 379 org.chromium.distiller.JsTestEntry.runWithFilter('MyTestClass.testSomething'
) |
| 380 ``` |
327 | 381 |
328 The `Sources` panel contains both the extracted JavaScript and all the Java | 382 The `Sources` panel contains both the extracted JavaScript and all the Java |
329 source files as long as you haven't disabled JavaScript source maps in | 383 source files as long as you haven't disabled JavaScript source maps in Developer |
330 Developer Tools. You can set breakpoints in the Java source files and then | 384 Tools. You can set breakpoints in the Java source files and then inspect all |
331 inspect all kinds of different interesting things when that breakpoint is hit. | 385 kinds of different interesting things when that breakpoint is hit. |
332 | 386 |
333 When a test fails, you will see several stack traces. One of these contains | 387 When a test fails, you will see several stack traces. One of these contains |
334 clickable links to the corresponding Java source files for the stack frames. | 388 clickable links to the corresponding Java source files for the stack frames. |
335 | 389 |
336 ## Developer extension | 390 ## Developer extension |
337 | 391 |
338 After running `ant package`, the `out/extension` folder contains an unpacked | 392 After running `ant package`, the `out/extension` folder contains an unpacked |
339 Chrome extension. This can be added to Chrome and used for development. | 393 Chrome extension. This can be added to Chrome and used for development. |
340 - Go to `chrome://extensions` | 394 |
341 - Enable developer mode | 395 - Go to `chrome://extensions` |
342 - Select to load an unpacked extension and point to the `out/extension` folder. | 396 - Enable developer mode |
| 397 - Select to load an unpacked extension and point to the `out/extension` |
| 398 folder. |
343 | 399 |
344 ### Features | 400 ### Features |
345 | 401 |
346 The extension currently supports profiling the extraction code. | 402 The extension currently supports profiling the extraction code. |
347 | 403 |
348 It also adds a panel to the Developer Tools which you can use to trigger | 404 It also adds a panel to the Developer Tools which you can use to trigger |
349 extraction on the inspected page. This can be used to trigger and profile | 405 extraction on the inspected page. This can be used to trigger and profile |
350 extraction on a mobile device which you are currently inspecting using | 406 extraction on a mobile device which you are currently inspecting using |
351 `chrome://inspect`. | 407 `chrome://inspect`. |
352 | 408 |
353 ## Logging | 409 ## Logging |
354 | 410 |
355 To add logging, you can use the LogUtil. You can use the Java function | 411 To add logging, you can use the LogUtil. You can use the Java function |
356 `LogUtil.logToConsole()`. Destination of logs: | 412 `LogUtil.logToConsole()`. Destination of logs: |
357 - `ant test`: Terminal. | 413 |
358 To get more verbose output, use `ant test -Dtest.debug_level=99`. | 414 - `ant test`: Terminal. To get more verbose output, use `ant test |
359 - Chrome browser: the Chrome log file, as set by shell variable | 415 -Dtest.debug_level=99`. |
360 `$CHROME_LOG_FILE`. A release mode build of Chrome will log all JavaScript | 416 - Chrome browser: the Chrome log file, as set by shell variable |
361 `INFO` there if you start Chrome with `--enable-logging`. You can add | 417 `$CHROME_LOG_FILE`. A release mode build of Chrome will log all JavaScript |
362 `--enable-logging=stderr` to have the log go to stderr instead of a file. | 418 `INFO` there if you start Chrome with `--enable-logging`. You can add |
363 - Content extractor: See | 419 `--enable-logging=stderr` to have the log go to stderr instead of a file. |
364 [documentation about `extract.log` above](#running-the-content-extractor). | 420 - Content extractor: See [documentation about `extract.log` above] |
| 421 (#running-the-content-extractor). |
365 | 422 |
366 For an example, see | 423 For an example, see |
367 `$DOM_DISTILLER_DIR/java/org/chromium/distiller/PagingLinksFinder.java`. | 424 `$DOM_DISTILLER_DIR/java/org/chromium/distiller/PagingLinksFinder.java`. |
368 | 425 |
369 Use `ant package '-Dgwt.custom.args=-style PRETTY'` for easier JavaScript | 426 Use `ant package '-Dgwt.custom.args=-style PRETTY'` for easier JavaScript |
370 debugging. | 427 debugging. |
371 | 428 |
372 # Mobile distillation from desktop | 429 # Mobile distillation from desktop |
373 | 430 |
374 1. In the tab with the interesting URL, bring up the Developer Tools emulation | 431 1. In the tab with the interesting URL, bring up the Developer Tools emulation |
375 panel (the mobile device icon). | 432 panel (the mobile device icon). |
376 2. Select the desired `Device` and reload the page. Verify that you get what you | 433 2. Select the desired `Device` and reload the page. Verify that you get what |
377 expect. For example a Nexus 4 might get a mobile site, whereas Nexus 7 might get | 434 you expect. For example a Nexus 4 might get a mobile site, whereas Nexus 7 |
378 the desktop site. | 435 might get the desktop site. |
379 3. The User-Agent can be copied directly out from the `UA` field. This field | 436 3. The User-Agent can be copied directly out from the `UA` field. This field |
380 does not even require reload after changing device, but it is good practice to | 437 does not even require reload after changing device, but it is good practice |
381 verify that you get what you expect. Copy this to the clipboard. | 438 to verify that you get what you expect. Copy this to the clipboard. |
382 4. (Re)start chrome with `--user-agent="$USER_AGENT_FROM_CLIPBOARD"`. Remember | 439 4. (Re)start chrome with `--user-agent="$USER_AGENT_FROM_CLIPBOARD"`. Remember |
383 to also add `--enable-dom-distiller`. | 440 to also add `--enable-dom-distiller`. |
384 5. Distill the same URL in viewer by either using the menu `Distill page` or by | 441 5. Distill the same URL in viewer by either using the menu `Distill page` or by |
385 going to `chrome://dom-distiller` and using the input field there. | 442 going to `chrome://dom-distiller` and using the input field there. |
386 6. Have fun scrutinizing the Chrome log file. | 443 6. Have fun scrutinizing the Chrome log file. |
387 | 444 |
388 If you want you can copy some of these User-Agent aliases into normal bash | 445 If you want you can copy some of these User-Agent aliases into normal bash |
389 aliases for easy access later. For example, Nexus 4 would be: | 446 aliases for easy access later. For example, Nexus 4 would be: |
390 ```bash | 447 |
| 448 ``` |
391 --user-agent="Mozilla/5.0 (Linux; Android 4.2.1; en-us; Nexus 4 Build/JOP40D) Ap
pleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.166 Mobile Safari/535.19" | 449 --user-agent="Mozilla/5.0 (Linux; Android 4.2.1; en-us; Nexus 4 Build/JOP40D) Ap
pleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.166 Mobile Safari/535.19" |
392 ``` | 450 ``` |
393 | 451 |
394 Steps 1-3 in the guide above can typically be done in a stable version of | 452 Steps 1-3 in the guide above can typically be done in a stable version of |
395 Chrome, whereas the rest of the steps is typically done in your own build of | 453 Chrome, whereas the rest of the steps is typically done in your own build of |
396 Chrome (hence the "(Re)" in step 4). Besides speed, this also facilitates | 454 Chrome (hence the "(Re)" in step 4). Besides speed, this also facilitates |
397 side-by-side comparison. | 455 side-by-side comparison. |
OLD | NEW |