Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(465)

Issue 2227973002: Add request throttler to thumbnail fetching for articles on mobile NTP (Closed)

Created:
4 years, 4 months ago by jkrcal
Modified:
4 years ago
CC:
chromium-reviews, ntp-dev+reviews_chromium.org
Base URL:
https://chromium.googlesource.com/chromium/src.git@master
Target Ref:
refs/pending/heads/master
Project:
chromium
Visibility:
Public.

Description

Add request throttler to thumbnail fetching for articles on mobile NTP As a safety mechanism, this CL introduces throttling mechanism for thumbnail fetching for articles for you on mobile NTP. Per default, the quota is unlimited but can be easily decreased by finch. Since all thumbnail requests are user initiated (by scrolling below the fold), the CL also extends the throttler to support separate quota for interactive requests (previously called "forced"). Lastly, it fixes a bug that the callback passed in in FetchSuggestionImage() got called with a wrong id (with the ID of the snippet "[url]" and not with the ID of the suggestion "[category_id]|[url]"). BUG=627073 Committed: https://crrev.com/ba735f09ac398664ea81fda2d948bef853469a4b Cr-Commit-Position: refs/heads/master@{#412555}

Patch Set 1 #

Total comments: 23

Patch Set 2 : Rebase #

Patch Set 3 : Marc's comments #

Total comments: 14

Patch Set 4 : Marc's comments #2 #

Total comments: 2

Patch Set 5 : Marc's comments #3 #

Total comments: 6

Patch Set 6 : Gayane's comments #

Patch Set 7 : Fixing the compile error #

Total comments: 4

Patch Set 8 : Rebase #

Patch Set 9 : Robert's and Tim's comments #

Total comments: 10

Patch Set 10 : Tim's comments #2 #

Total comments: 8

Patch Set 11 : Tim's comments #3 #

Patch Set 12 : One forgotten comment #

Total comments: 10

Patch Set 13 : Tims's comments #4 #

Patch Set 14 : Build issue #

Unified diffs Side-by-side diffs Delta from patch set Stats (+302 lines, -102 lines) Patch
M components/ntp_snippets/BUILD.gn View 1 2 3 4 5 6 7 8 9 10 11 12 13 1 chunk +1 line, -0 lines 0 comments Download
M components/ntp_snippets/ntp_snippets_fetcher.h View 1 2 3 4 5 6 7 8 9 10 1 chunk +2 lines, -2 lines 0 comments Download
M components/ntp_snippets/ntp_snippets_fetcher.cc View 1 2 3 4 5 6 7 8 9 10 1 chunk +2 lines, -2 lines 0 comments Download
M components/ntp_snippets/ntp_snippets_fetcher_unittest.cc View 1 2 3 4 5 6 7 8 9 10 12 chunks +13 lines, -13 lines 0 comments Download
M components/ntp_snippets/ntp_snippets_service.h View 1 2 3 4 5 6 7 8 9 10 11 12 5 chunks +24 lines, -10 lines 0 comments Download
M components/ntp_snippets/ntp_snippets_service.cc View 1 2 3 4 5 6 7 8 9 10 11 12 6 chunks +40 lines, -15 lines 0 comments Download
M components/ntp_snippets/ntp_snippets_service_unittest.cc View 1 2 3 4 5 6 7 8 9 10 11 12 10 chunks +78 lines, -3 lines 0 comments Download
M components/ntp_snippets/pref_names.h View 1 2 3 4 5 6 7 8 9 10 1 chunk +13 lines, -2 lines 0 comments Download
M components/ntp_snippets/pref_names.cc View 1 2 3 4 5 6 7 8 9 10 1 chunk +12 lines, -3 lines 0 comments Download
M components/ntp_snippets/request_throttler.h View 1 2 3 4 5 6 7 8 9 10 11 12 4 chunks +15 lines, -10 lines 0 comments Download
M components/ntp_snippets/request_throttler.cc View 1 2 3 4 5 6 7 8 9 10 11 12 5 chunks +73 lines, -31 lines 0 comments Download
M components/ntp_snippets/request_throttler_unittest.cc View 1 2 3 4 5 6 1 chunk +1 line, -1 line 0 comments Download
M tools/metrics/histograms/histograms.xml View 1 2 3 4 5 6 7 8 9 10 11 12 3 chunks +28 lines, -10 lines 0 comments Download

Messages

Total messages: 56 (31 generated)
jkrcal
Marc, PTAL! (Tim, FYI, as the CL is based on our discussions.)
4 years, 4 months ago (2016-08-09 13:18:44 UTC) #2
Marc Treib
https://codereview.chromium.org/2227973002/diff/1/components/ntp_snippets/ntp_snippets_service.cc File components/ntp_snippets/ntp_snippets_service.cc (right): https://codereview.chromium.org/2227973002/diff/1/components/ntp_snippets/ntp_snippets_service.cc#newcode645 components/ntp_snippets/ntp_snippets_service.cc:645: base::Bind(callback, it != snippets_.end() I don't understand the snippet_id-building ...
4 years, 4 months ago (2016-08-09 14:04:53 UTC) #5
jkrcal
Thanks, Marc. PTAL, again. Robert, PTAL at histograms.xml. https://codereview.chromium.org/2227973002/diff/1/components/ntp_snippets/ntp_snippets_service.cc File components/ntp_snippets/ntp_snippets_service.cc (right): https://codereview.chromium.org/2227973002/diff/1/components/ntp_snippets/ntp_snippets_service.cc#newcode645 components/ntp_snippets/ntp_snippets_service.cc:645: base::Bind(callback, ...
4 years, 4 months ago (2016-08-10 10:27:27 UTC) #9
Marc Treib
https://codereview.chromium.org/2227973002/diff/1/components/ntp_snippets/ntp_snippets_service.cc File components/ntp_snippets/ntp_snippets_service.cc (right): https://codereview.chromium.org/2227973002/diff/1/components/ntp_snippets/ntp_snippets_service.cc#newcode645 components/ntp_snippets/ntp_snippets_service.cc:645: base::Bind(callback, it != snippets_.end() On 2016/08/10 10:27:26, jkrcal wrote: ...
4 years, 4 months ago (2016-08-10 11:38:15 UTC) #10
jkrcal
Thanks! PTAL, again. https://codereview.chromium.org/2227973002/diff/1/components/ntp_snippets/ntp_snippets_service.cc File components/ntp_snippets/ntp_snippets_service.cc (right): https://codereview.chromium.org/2227973002/diff/1/components/ntp_snippets/ntp_snippets_service.cc#newcode645 components/ntp_snippets/ntp_snippets_service.cc:645: base::Bind(callback, it != snippets_.end() On 2016/08/10 ...
4 years, 4 months ago (2016-08-10 11:57:04 UTC) #12
Marc Treib
lgtm https://codereview.chromium.org/2227973002/diff/1/components/ntp_snippets/ntp_snippets_service.cc File components/ntp_snippets/ntp_snippets_service.cc (right): https://codereview.chromium.org/2227973002/diff/1/components/ntp_snippets/ntp_snippets_service.cc#newcode645 components/ntp_snippets/ntp_snippets_service.cc:645: base::Bind(callback, it != snippets_.end() On 2016/08/10 11:57:03, jkrcal ...
4 years, 4 months ago (2016-08-10 12:05:00 UTC) #13
jkrcal
Thanks, Marc! Robert, PTAL at histograms.xml (and happy birthday, by the way!) https://codereview.chromium.org/2227973002/diff/1/components/ntp_snippets/ntp_snippets_service.cc File components/ntp_snippets/ntp_snippets_service.cc ...
4 years, 4 months ago (2016-08-10 16:22:27 UTC) #14
gayane -on leave until 09-2017
https://codereview.chromium.org/2227973002/diff/80001/components/ntp_snippets/request_throttler.cc File components/ntp_snippets/request_throttler.cc (right): https://codereview.chromium.org/2227973002/diff/80001/components/ntp_snippets/request_throttler.cc#newcode50 components/ntp_snippets/request_throttler.cc:50: // When adding a new type here, extend also ...
4 years, 4 months ago (2016-08-10 18:47:55 UTC) #16
jkrcal
Thanks, Gayane! PTAL, again. https://codereview.chromium.org/2227973002/diff/80001/components/ntp_snippets/request_throttler.cc File components/ntp_snippets/request_throttler.cc (right): https://codereview.chromium.org/2227973002/diff/80001/components/ntp_snippets/request_throttler.cc#newcode50 components/ntp_snippets/request_throttler.cc:50: // When adding a new ...
4 years, 4 months ago (2016-08-11 08:06:28 UTC) #17
tschumann
https://codereview.chromium.org/2227973002/diff/1/components/ntp_snippets/ntp_snippets_service.cc File components/ntp_snippets/ntp_snippets_service.cc (right): https://codereview.chromium.org/2227973002/diff/1/components/ntp_snippets/ntp_snippets_service.cc#newcode645 components/ntp_snippets/ntp_snippets_service.cc:645: base::Bind(callback, it != snippets_.end() On 2016/08/10 16:22:27, jkrcal wrote: ...
4 years, 4 months ago (2016-08-11 11:09:40 UTC) #24
gayane -on leave until 09-2017
LGTM for histograms. Still need approval from rkaplow.
4 years, 4 months ago (2016-08-11 15:33:46 UTC) #27
rkaplow
https://codereview.chromium.org/2227973002/diff/120001/tools/metrics/histograms/histograms.xml File tools/metrics/histograms/histograms.xml (left): https://codereview.chromium.org/2227973002/diff/120001/tools/metrics/histograms/histograms.xml#oldcode86290 tools/metrics/histograms/histograms.xml:86290: <enum name="NtpRequestThrottlerStatus" type="int"> hm, if you're changing the meaning ...
4 years, 4 months ago (2016-08-11 17:13:15 UTC) #28
jkrcal
Thanks, Robert! PTAL, again. Tim, I added the unittest. https://codereview.chromium.org/2227973002/diff/120001/tools/metrics/histograms/histograms.xml File tools/metrics/histograms/histograms.xml (left): https://codereview.chromium.org/2227973002/diff/120001/tools/metrics/histograms/histograms.xml#oldcode86290 tools/metrics/histograms/histograms.xml:86290: ...
4 years, 4 months ago (2016-08-12 09:08:19 UTC) #29
tschumann
https://codereview.chromium.org/2227973002/diff/160001/components/ntp_snippets/ntp_snippets_service_unittest.cc File components/ntp_snippets/ntp_snippets_service_unittest.cc (right): https://codereview.chromium.org/2227973002/diff/160001/components/ntp_snippets/ntp_snippets_service_unittest.cc#newcode275 components/ntp_snippets/ntp_snippets_service_unittest.cc:275: void SetImageFetcherDelegate(ImageFetcherDelegate* delegate) override {} let's not mix up ...
4 years, 4 months ago (2016-08-12 09:55:36 UTC) #30
jkrcal
Thanks, Tim! PTAL, again. https://codereview.chromium.org/2227973002/diff/160001/components/ntp_snippets/ntp_snippets_service_unittest.cc File components/ntp_snippets/ntp_snippets_service_unittest.cc (right): https://codereview.chromium.org/2227973002/diff/160001/components/ntp_snippets/ntp_snippets_service_unittest.cc#newcode275 components/ntp_snippets/ntp_snippets_service_unittest.cc:275: void SetImageFetcherDelegate(ImageFetcherDelegate* delegate) override {} ...
4 years, 4 months ago (2016-08-12 12:06:10 UTC) #31
Marc Treib
https://codereview.chromium.org/2227973002/diff/160001/components/ntp_snippets/ntp_snippets_service_unittest.cc File components/ntp_snippets/ntp_snippets_service_unittest.cc (right): https://codereview.chromium.org/2227973002/diff/160001/components/ntp_snippets/ntp_snippets_service_unittest.cc#newcode275 components/ntp_snippets/ntp_snippets_service_unittest.cc:275: void SetImageFetcherDelegate(ImageFetcherDelegate* delegate) override {} On 2016/08/12 12:06:10, jkrcal ...
4 years, 4 months ago (2016-08-12 12:10:59 UTC) #32
tschumann
almost =) https://codereview.chromium.org/2227973002/diff/160001/components/ntp_snippets/ntp_snippets_service_unittest.cc File components/ntp_snippets/ntp_snippets_service_unittest.cc (right): https://codereview.chromium.org/2227973002/diff/160001/components/ntp_snippets/ntp_snippets_service_unittest.cc#newcode951 components/ntp_snippets/ntp_snippets_service_unittest.cc:951: EXPECT_TRUE(image.IsEmpty()); On 2016/08/12 12:06:09, jkrcal wrote: > ...
4 years, 4 months ago (2016-08-12 12:37:09 UTC) #35
jkrcal
Thanks, PTAL again. https://codereview.chromium.org/2227973002/diff/180001/components/ntp_snippets/ntp_snippets_service_unittest.cc File components/ntp_snippets/ntp_snippets_service_unittest.cc (right): https://codereview.chromium.org/2227973002/diff/180001/components/ntp_snippets/ntp_snippets_service_unittest.cc#newcode353 components/ntp_snippets/ntp_snippets_service_unittest.cc:353: auto image_fetcher = On 2016/08/12 12:37:09, ...
4 years, 4 months ago (2016-08-16 12:31:43 UTC) #38
Marc Treib
https://codereview.chromium.org/2227973002/diff/180001/components/ntp_snippets/ntp_snippets_service_unittest.cc File components/ntp_snippets/ntp_snippets_service_unittest.cc (right): https://codereview.chromium.org/2227973002/diff/180001/components/ntp_snippets/ntp_snippets_service_unittest.cc#newcode353 components/ntp_snippets/ntp_snippets_service_unittest.cc:353: auto image_fetcher = On 2016/08/16 12:31:43, jkrcal wrote: > ...
4 years, 4 months ago (2016-08-16 13:29:18 UTC) #39
tschumann
lgtm most of my comments can be addressed in a follow-up CL (if you're too ...
4 years, 4 months ago (2016-08-16 14:56:56 UTC) #40
rkaplow
lgtm
4 years, 4 months ago (2016-08-16 14:58:51 UTC) #41
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2227973002/260001
4 years, 4 months ago (2016-08-17 16:05:23 UTC) #51
commit-bot: I haz the power
Committed patchset #14 (id:260001)
4 years, 4 months ago (2016-08-17 16:30:46 UTC) #53
commit-bot: I haz the power
Patchset 14 (id:??) landed as https://crrev.com/ba735f09ac398664ea81fda2d948bef853469a4b Cr-Commit-Position: refs/heads/master@{#412555}
4 years, 4 months ago (2016-08-17 16:32:43 UTC) #55
jkrcal
4 years ago (2016-12-06 08:30:26 UTC) #56
Message was sent while issue was closed.
Tim, just for completeness: I've forgotten to send out drafted comments in a
long-closed CL.

https://codereview.chromium.org/2227973002/diff/220001/components/ntp_snippet...
File components/ntp_snippets/ntp_snippets_service.cc (right):

https://codereview.chromium.org/2227973002/diff/220001/components/ntp_snippet...
components/ntp_snippets/ntp_snippets_service.cc:679: // happened in the
meantime.
On 2016/08/16 14:56:55, tschumann wrote:
> i guess we want to record such events somewhere, hm?

This event is recorded in the request throttler. Do we need more?

https://codereview.chromium.org/2227973002/diff/220001/components/ntp_snippet...
components/ntp_snippets/ntp_snippets_service.cc:686: // The image fetcher calls
OnImageDataFetched() with the raw data (this object
On 2016/08/16 14:56:55, tschumann wrote:
> probably outside of this CL but I find comments like these a bit confusing.
> We probably should rename OnImageDataFetched() to CacheImageData().
> This would document that this is actually independent from the individual
> fetch-flow.
> For the code here, it really doesn't matter that image caching is taking place
> in-between; hence I'd be in favor of just dropping the comment.
> 
> That said, this class grows really, really large. The fact that
> NTPSnippetService also implements ImageFetcherDelegate adds unnecssary
> complexity (and after all the Service is conceptually not an
> ImagerFetcherDeletage ;-)).
> Instead, the cleaner solution would  be to define a CachedImageFetcher class
> that handles the caching aspects and looks like an image fetcher to the
> NTPSnippetService.
> 
> As said, not in this CL, but it's definitely a smell.

Ok, let's leave it up to a follow-up CL.

https://codereview.chromium.org/2227973002/diff/220001/components/ntp_snippet...
File components/ntp_snippets/ntp_snippets_service_unittest.cc (right):

https://codereview.chromium.org/2227973002/diff/220001/components/ntp_snippet...
components/ntp_snippets/ntp_snippets_service_unittest.cc:946: gfx::Image image =
gfx::test::CreateImage(1, 1);
On 2016/08/16 14:56:55, tschumann wrote:
> please add a comment explaining that we set up the picture like that  to
verify
> we get an empty image set in error cases.

Done.

https://codereview.chromium.org/2227973002/diff/220001/components/ntp_snippet...
File components/ntp_snippets/request_throttler.cc (right):

https://codereview.chromium.org/2227973002/diff/220001/components/ntp_snippet...
components/ntp_snippets/request_throttler.cc:154: const char*
RequestThrottler::GetRequestTypeAsString() const {
On 2016/08/16 14:56:55, tschumann wrote:
> actually this method name is misleading -- you don't return a string.
> GetRequestTypeName()?

Done.

https://codereview.chromium.org/2227973002/diff/220001/components/ntp_snippet...
components/ntp_snippets/request_throttler.cc:158: int
RequestThrottler::GetQuota(bool interactive_request) const {
On 2016/08/16 14:56:55, tschumann wrote:
> It seems like we should turn RequestTypeInfo into a proper class, move those
> methods onto the class and hide the members. Can happen in a follow-up CL but
a
> TODO() would be nice.

Addded a TODO.

Powered by Google App Engine
This is Rietveld 408576698