Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(207)

Issue 13762002: Switches to a Skia-specific appengine entry point that uses condensed data and taskqueue writes. (Closed)

Created:
7 years, 8 months ago by benchen
Modified:
7 years, 8 months ago
Reviewers:
borenet
CC:
skia-review_googlegroups.com
Visibility:
Public.

Description

Switches to a Skia-specific appengine entry point that uses condensed data and taskqueue writes (SkipBuildbotRuns). The default entry /add_point does not handle large data efficiently, so we've seen >30min timeouts for some bots to upload data. We now switch to using /skia_add_points that I'm writing for Skia, so we can condense the data to send to minimum (since all points in each batch have the same revision, platform and config), and dedicate the actual data processing to /skia_write_datastore which is a taskqueue task that has longer timeout and can run at app backend instead of blocking the bots. Initial testing from my MacBook on Mac 64 bench data via vpn gave only a little more than 1 second for uploading data for one config, about 15 seconds for all 12 configs. That's a big win against 20+ minutes. Committed: https://code.google.com/p/skia/source/detail?r=8560

Patch Set 1 #

Patch Set 2 : #

Unified diffs Side-by-side diffs Delta from patch set Stats (+23 lines, -29 lines) Patch
M bench/bench_graph_svg.py View 3 chunks +23 lines, -29 lines 0 comments Download

Messages

Total messages: 3 (0 generated)
benchen
Will submit the change to appengine entry url together with this change.
7 years, 8 months ago (2013-04-08 14:14:28 UTC) #1
borenet
LGTM, with the assumption that you've set up your appengine app to deal with the ...
7 years, 8 months ago (2013-04-08 14:52:32 UTC) #2
benchen
7 years, 8 months ago (2013-04-08 14:57:45 UTC) #3
Message was sent while issue was closed.
Committed patchset #2 manually as r8560 (presubmit successful).

Powered by Google App Engine
This is Rietveld 408576698