Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(399)

Side by Side Diff: chrome/browser/predictors/resource_prefetch_predictor.h

Issue 2896713003: Create LoadingDataCollector class and have observers rely on it instead of ResourcePrefetchPredictor (Closed)
Patch Set: Rebase, merge loading_stats_collector. Created 3 years, 6 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
OLDNEW
1 // Copyright 2014 The Chromium Authors. All rights reserved. 1 // Copyright 2014 The Chromium Authors. All rights reserved.
2 // Use of this source code is governed by a BSD-style license that can be 2 // Use of this source code is governed by a BSD-style license that can be
3 // found in the LICENSE file. 3 // found in the LICENSE file.
4 4
5 #ifndef CHROME_BROWSER_PREDICTORS_RESOURCE_PREFETCH_PREDICTOR_H_ 5 #ifndef CHROME_BROWSER_PREDICTORS_RESOURCE_PREFETCH_PREDICTOR_H_
6 #define CHROME_BROWSER_PREDICTORS_RESOURCE_PREFETCH_PREDICTOR_H_ 6 #define CHROME_BROWSER_PREDICTORS_RESOURCE_PREFETCH_PREDICTOR_H_
7 7
8 #include <stddef.h> 8 #include <stddef.h>
9 9
10 #include <map> 10 #include <map>
(...skipping 58 matching lines...) Expand 10 before | Expand all | Expand 10 after
69 class LoadingStatsCollector; 69 class LoadingStatsCollector;
70 70
71 // Contains logic for learning what can be prefetched and for kicking off 71 // Contains logic for learning what can be prefetched and for kicking off
72 // speculative prefetching. 72 // speculative prefetching.
73 // - The class is a profile keyed service owned by the profile. 73 // - The class is a profile keyed service owned by the profile.
74 // - All the non-static methods of this class need to be called on the UI 74 // - All the non-static methods of this class need to be called on the UI
75 // thread. 75 // thread.
76 // 76 //
77 // The overall flow of the resource prefetching algorithm is as follows: 77 // The overall flow of the resource prefetching algorithm is as follows:
78 // 78 //
79 // * ResourcePrefetchPredictorObserver - Listens for URL requests, responses and 79 // * LoadingPredictorObserver - Listens for URL requests, responses and
80 // redirects (client-side redirects are not supported) on the IO thread (via 80 // redirects (client-side redirects are not supported) on the IO thread (via
81 // ResourceDispatcherHostDelegate) and posts tasks to the 81 // ResourceDispatcherHostDelegate) and posts tasks to the
82 // ResourcePrefetchPredictor on the UI thread. This is owned by the 82 // ResourcePrefetchPredictor on the UI thread. This is owned by the
83 // ProfileIOData for the profile. 83 // ProfileIOData for the profile.
84 // * ResourcePrefetchPredictorTables - Persists ResourcePrefetchPredictor data 84 // * ResourcePrefetchPredictorTables - Persists ResourcePrefetchPredictor data
85 // to a sql database. Runs entirely on the DB thread. Owned by the 85 // to a sql database. Runs entirely on the DB thread. Owned by the
86 // PredictorDatabase. 86 // PredictorDatabase.
87 // * ResourcePrefetchPredictor - Learns about resource requirements per URL in 87 // * ResourcePrefetchPredictor - Learns about resource requirements per URL in
88 // the UI thread through the ResourcePrefetchPredictorObserver and persists 88 // the UI thread through the LoadingPredictorObserver and persists
89 // it to disk in the DB thread through the ResourcePrefetchPredictorTables. It 89 // it to disk in the DB thread through the ResourcePrefetchPredictorTables. It
90 // initiates resource prefetching using the ResourcePrefetcherManager. Owned 90 // initiates resource prefetching using the ResourcePrefetcherManager. Owned
91 // by profile. 91 // by profile.
92 // * ResourcePrefetcherManager - Manages the ResourcePrefetchers that do the 92 // * ResourcePrefetcherManager - Manages the ResourcePrefetchers that do the
93 // prefetching on the IO thread. The manager is owned by the 93 // prefetching on the IO thread. The manager is owned by the
94 // ResourcePrefetchPredictor and interfaces between the predictor on the UI 94 // ResourcePrefetchPredictor and interfaces between the predictor on the UI
95 // thread and the prefetchers on the IO thread. 95 // thread and the prefetchers on the IO thread.
96 // * ResourcePrefetcher - Lives entirely on the IO thread, owned by the 96 // * ResourcePrefetcher - Lives entirely on the IO thread, owned by the
97 // ResourcePrefetcherManager, and issues net::URLRequest to fetch resources. 97 // ResourcePrefetcherManager, and issues net::URLRequest to fetch resources.
98 // 98 //
(...skipping 94 matching lines...) Expand 10 before | Expand all | Expand 10 after
193 193
194 ResourcePrefetchPredictor(const LoadingPredictorConfig& config, 194 ResourcePrefetchPredictor(const LoadingPredictorConfig& config,
195 Profile* profile); 195 Profile* profile);
196 ~ResourcePrefetchPredictor() override; 196 ~ResourcePrefetchPredictor() override;
197 197
198 // Starts initialization by posting a task to the DB thread to read the 198 // Starts initialization by posting a task to the DB thread to read the
199 // predictor database. Virtual for testing. 199 // predictor database. Virtual for testing.
200 virtual void StartInitialization(); 200 virtual void StartInitialization();
201 virtual void Shutdown(); 201 virtual void Shutdown();
202 202
203 // Thread safe.
204 static bool ShouldRecordRequest(net::URLRequest* request,
205 content::ResourceType resource_type);
206 static bool ShouldRecordResponse(net::URLRequest* response);
207 static bool ShouldRecordRedirect(net::URLRequest* response);
208
209 // Determines the resource type from the declared one, falling back to MIME 203 // Determines the resource type from the declared one, falling back to MIME
210 // type detection when it is not explicit. 204 // type detection when it is not explicit.
211 static content::ResourceType GetResourceType( 205 static content::ResourceType GetResourceType(
212 content::ResourceType resource_type, 206 content::ResourceType resource_type,
213 const std::string& mime_type); 207 const std::string& mime_type);
214 208
215 // Determines the ResourceType from the mime type, defaulting to the 209 // Determines the ResourceType from the mime type, defaulting to the
216 // |fallback| if the ResourceType could not be determined. 210 // |fallback| if the ResourceType could not be determined.
217 static content::ResourceType GetResourceTypeFromMimeType( 211 static content::ResourceType GetResourceTypeFromMimeType(
218 const std::string& mime_type, 212 const std::string& mime_type,
219 content::ResourceType fallback); 213 content::ResourceType fallback);
220 214
221 // 'ResourcePrefetchPredictorObserver' calls the below functions to inform the
222 // predictor of main frame and resource requests. Should only be called if the
223 // corresponding Should* functions return true.
224 void RecordURLRequest(const URLRequestSummary& request);
225 void RecordURLResponse(const URLRequestSummary& response);
226 void RecordURLRedirect(const URLRequestSummary& response);
227
228 // Called when the main frame of a page completes loading.
229 void RecordMainFrameLoadComplete(const NavigationID& navigation_id);
230
231 // Called after the main frame's first contentful paint.
232 void RecordFirstContentfulPaint(
233 const NavigationID& navigation_id,
234 const base::TimeTicks& first_contentful_paint);
235
236 // Called when ResourcePrefetcher is finished, i.e. there is nothing pending 215 // Called when ResourcePrefetcher is finished, i.e. there is nothing pending
237 // in flight. 216 // in flight.
238 void OnPrefetchingFinished( 217 void OnPrefetchingFinished(
239 const GURL& main_frame_url, 218 const GURL& main_frame_url,
240 std::unique_ptr<ResourcePrefetcher::PrefetcherStats> stats); 219 std::unique_ptr<ResourcePrefetcher::PrefetcherStats> stats);
241 220
242 // Returns true if prefetching data exists for the |main_frame_url|. 221 // Returns true if prefetching data exists for the |main_frame_url|.
243 virtual bool IsUrlPrefetchable(const GURL& main_frame_url) const; 222 virtual bool IsUrlPrefetchable(const GURL& main_frame_url) const;
244 223
245 // Returns true iff |resource| has sufficient confidence level and required 224 // Returns true iff |resource| has sufficient confidence level and required
(...skipping 12 matching lines...) Expand all
258 // Sets the |stats_collector| which is used to report histograms. 237 // Sets the |stats_collector| which is used to report histograms.
259 void SetStatsCollector(LoadingStatsCollector* stats_collector); 238 void SetStatsCollector(LoadingStatsCollector* stats_collector);
260 239
261 // Returns true iff there is PrefetchData that can be used for a 240 // Returns true iff there is PrefetchData that can be used for a
262 // |main_frame_url| and fills |prediction| with resources that need to be 241 // |main_frame_url| and fills |prediction| with resources that need to be
263 // prefetched. |prediction| pointer may be nullptr to get return value only. 242 // prefetched. |prediction| pointer may be nullptr to get return value only.
264 virtual bool GetPrefetchData(const GURL& main_frame_url, 243 virtual bool GetPrefetchData(const GURL& main_frame_url,
265 Prediction* prediction) const; 244 Prediction* prediction) const;
266 245
267 private: 246 private:
247 // 'LoadingPredictorObserver' calls the below functions to inform the
248 // predictor of main frame and resource requests. Should only be called if the
249 // corresponding Should* functions return true.
250 void RecordURLRequest(const URLRequestSummary& request);
251 void RecordURLResponse(const URLRequestSummary& response);
252 void RecordURLRedirect(const URLRequestSummary& response);
253
254 // Called when the main frame of a page completes loading.
255 void RecordMainFrameLoadComplete(const NavigationID& navigation_id);
256
257 // Called after the main frame's first contentful paint.
258 void RecordFirstContentfulPaint(
259 const NavigationID& navigation_id,
260 const base::TimeTicks& first_contentful_paint);
261
268 // Starts prefetching if it is enabled and prefetching data exists for the 262 // Starts prefetching if it is enabled and prefetching data exists for the
269 // |main_frame_url| either at the URL or at the host level. 263 // |main_frame_url| either at the URL or at the host level.
270 void StartPrefetching(const GURL& main_frame_url); 264 void StartPrefetching(const GURL& main_frame_url);
271 265
272 // Stops prefetching that may be in progress corresponding to 266 // Stops prefetching that may be in progress corresponding to
273 // |main_frame_url|. 267 // |main_frame_url|.
274 void StopPrefetching(const GURL& main_frame_url); 268 void StopPrefetching(const GURL& main_frame_url);
275 269
276 friend class LoadingPredictor; 270 friend class LoadingPredictor;
277 friend class ::PredictorsHandler; 271 friend class ::PredictorsHandler;
272 friend class LoadingDataCollector;
278 friend class ResourcePrefetchPredictorTest; 273 friend class ResourcePrefetchPredictorTest;
279 friend class ResourcePrefetchPredictorBrowserTest; 274 friend class ResourcePrefetchPredictorBrowserTest;
280 275
281 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, DeleteUrls); 276 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, DeleteUrls);
282 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, 277 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest,
283 LazilyInitializeEmpty); 278 LazilyInitializeEmpty);
284 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, 279 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest,
285 LazilyInitializeWithData); 280 LazilyInitializeWithData);
286 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, 281 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest,
287 NavigationLowHistoryCount); 282 NavigationLowHistoryCount);
288 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, NavigationUrlInDB); 283 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, NavigationUrlInDB);
289 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, NavigationUrlNotInDB); 284 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, NavigationUrlNotInDB);
290 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, 285 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest,
291 NavigationUrlNotInDBAndDBFull); 286 NavigationUrlNotInDBAndDBFull);
292 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, RedirectUrlNotInDB); 287 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, RedirectUrlNotInDB);
293 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, RedirectUrlInDB); 288 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, RedirectUrlInDB);
294 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, ManifestHostNotInDB); 289 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, ManifestHostNotInDB);
295 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, ManifestHostInDB); 290 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, ManifestHostInDB);
296 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, 291 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest,
297 ManifestHostNotInDBAndDBFull); 292 ManifestHostNotInDBAndDBFull);
298 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, 293 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest,
299 ManifestUnusedRemoved); 294 ManifestUnusedRemoved);
300 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, OnMainFrameRequest); 295 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, OnMainFrameRequest);
301 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, OnMainFrameRedirect); 296 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, OnMainFrameRedirect);
302 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, 297 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest,
303 OnSubresourceResponse); 298 OnSubresourceResponse);
304 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, GetCorrectPLT); 299 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, GetCorrectPLT);
305 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, HandledResourceTypes);
306 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, 300 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest,
307 PopulatePrefetcherRequest); 301 PopulatePrefetcherRequest);
308 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, PopulateFromManifest); 302 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, PopulateFromManifest);
309 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, GetRedirectEndpoint); 303 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, GetRedirectEndpoint);
310 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, GetPrefetchData); 304 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, GetPrefetchData);
311 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, 305 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest,
312 TestPrecisionRecallHistograms); 306 TestPrecisionRecallHistograms);
313 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, 307 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest,
314 TestPrefetchingDurationHistogram); 308 TestPrefetchingDurationHistogram);
315 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest, 309 FRIEND_TEST_ALL_PREFIXES(ResourcePrefetchPredictorTest,
316 TestRecordFirstContentfulPaint); 310 TestRecordFirstContentfulPaint);
317 311
318 enum InitializationState { 312 enum InitializationState {
319 NOT_INITIALIZED = 0, 313 NOT_INITIALIZED = 0,
320 INITIALIZING = 1, 314 INITIALIZING = 1,
321 INITIALIZED = 2 315 INITIALIZED = 2
322 }; 316 };
323 typedef GlowplugKeyValueData<PrefetchData, internal::LastVisitTimeCompare> 317 typedef GlowplugKeyValueData<PrefetchData, internal::LastVisitTimeCompare>
324 PrefetchDataMap; 318 PrefetchDataMap;
325 typedef GlowplugKeyValueData<RedirectData, internal::LastVisitTimeCompare> 319 typedef GlowplugKeyValueData<RedirectData, internal::LastVisitTimeCompare>
326 RedirectDataMap; 320 RedirectDataMap;
327 typedef GlowplugKeyValueData<precache::PrecacheManifest, 321 typedef GlowplugKeyValueData<precache::PrecacheManifest,
328 internal::ManifestCompare> 322 internal::ManifestCompare>
329 ManifestDataMap; 323 ManifestDataMap;
330 typedef GlowplugKeyValueData<OriginData, internal::LastVisitTimeCompare> 324 typedef GlowplugKeyValueData<OriginData, internal::LastVisitTimeCompare>
331 OriginDataMap; 325 OriginDataMap;
332 typedef std::map<NavigationID, std::unique_ptr<PageRequestSummary>> 326 typedef std::map<NavigationID, std::unique_ptr<PageRequestSummary>>
333 NavigationMap; 327 NavigationMap;
334 328
335 // Returns true if the main page request is supported for prediction.
336 static bool IsHandledMainPage(net::URLRequest* request);
337
338 // Returns true if the subresource request is supported for prediction.
339 static bool IsHandledSubresource(net::URLRequest* request,
340 content::ResourceType resource_type);
341
342 // Returns true if the subresource has a supported type.
343 static bool IsHandledResourceType(content::ResourceType resource_type,
344 const std::string& mime_type);
345
346 // Returns true if the request (should have a response in it) is "no-store". 329 // Returns true if the request (should have a response in it) is "no-store".
347 static bool IsNoStore(const net::URLRequest& request); 330 static bool IsNoStore(const net::URLRequest& request);
348 331
349 static void SetAllowPortInUrlsForTesting(bool state);
350
351 // Functions called on different network events pertaining to the loading of 332 // Functions called on different network events pertaining to the loading of
352 // main frame resource or sub resources. 333 // main frame resource or sub resources.
353 void OnMainFrameRequest(const URLRequestSummary& request); 334 void OnMainFrameRequest(const URLRequestSummary& request);
354 void OnMainFrameRedirect(const URLRequestSummary& response); 335 void OnMainFrameRedirect(const URLRequestSummary& response);
355 void OnSubresourceResponse(const URLRequestSummary& response); 336 void OnSubresourceResponse(const URLRequestSummary& response);
356 void OnSubresourceRedirect(const URLRequestSummary& response); 337 void OnSubresourceRedirect(const URLRequestSummary& response);
357 338
358 // Called when onload completes for a navigation. We treat this point as the 339 // Called when onload completes for a navigation. We treat this point as the
359 // "completion" of the navigation. The resources requested by the page up to 340 // "completion" of the navigation. The resources requested by the page up to
360 // this point are the only ones considered for prefetching. 341 // this point are the only ones considered for prefetching.
(...skipping 142 matching lines...) Expand 10 before | Expand all | Expand 10 after
503 484
504 private: 485 private:
505 ResourcePrefetchPredictor* predictor_; 486 ResourcePrefetchPredictor* predictor_;
506 487
507 DISALLOW_COPY_AND_ASSIGN(TestObserver); 488 DISALLOW_COPY_AND_ASSIGN(TestObserver);
508 }; 489 };
509 490
510 } // namespace predictors 491 } // namespace predictors
511 492
512 #endif // CHROME_BROWSER_PREDICTORS_RESOURCE_PREFETCH_PREDICTOR_H_ 493 #endif // CHROME_BROWSER_PREDICTORS_RESOURCE_PREFETCH_PREDICTOR_H_
OLDNEW

Powered by Google App Engine
This is Rietveld 408576698