Codes for writing bench data to appengine datastore.

bench/bench_graph_svg.py

Index: bench/bench_graph_svg.py
===================================================================
--- bench/bench_graph_svg.py	(revision 7965)
+++ bench/bench_graph_svg.py	(working copy)
@@ -9,6 +9,9 @@
 import os
 import bench_util
 import json
+import httplib
+import urllib
+import urllib2
 import xml.sax.saxutils
 
 # We throw out any measurement outside this range, and log a warning.
@@ -19,9 +22,15 @@
 TITLE_PREAMBLE = 'Bench_Performance_for_Skia_'
 TITLE_PREAMBLE_LENGTH = len(TITLE_PREAMBLE)
 
+# Number of data points to send to appengine at once.
+DATA_SIZE = 100

[borenet, 2013/03/04 21:32:15]

Maybe this could be more descriptive?  Like DATA_POINTS_TO_SEND or something
similar?

[benchen, 2013/03/04 21:51:15]

Done.

+
 def usage():
     """Prints simple usage information."""
-    
+
+    print '-a <url> the url to use for adding bench values to app engine app.'
+    print '   Example: "https://skiadash.appspot.com/add_point".'

[rmistry, 2013/03/04 21:49:35]

This is going to upload to our instance of chromium's appengine? I thought we
were aiming to write to a SQL DB somewhere.

[benchen, 2013/03/04 21:59:18]

The SQL DB solution is through a separate script that won't run within buildbot
steps. Instead it'll be in batch mode done on Borg. This one is for loading data
for the other "saving face" solution. Sorry I thought this were pretty clear.
One advantage of this is that each bot uploads bench data even before uploading
raw bench logs to bigstore. But the datastore solution won't give us the SQL
functionality we desire.
The corresponding Chrome CL: https://codereview.chromium.org/12317053/

On 2013/03/04 21:49:35, rmistry wrote:

+    print '   If not set, will skip this step.'
     print '-b <bench> the bench to show.'
     print '-c <config> the config to show (GPU, 8888, 565, etc).'
     print '-d <dir> a directory containing bench_r<revision>_<scalar> files.'
@@ -286,7 +295,7 @@
     
     try:
         opts, _ = getopt.getopt(sys.argv[1:]
-                                 , "b:c:d:e:f:i:l:m:o:r:s:t:x:y:"
+                                 , "a:b:c:d:e:f:i:l:m:o:r:s:t:x:y:"
                                  , "default-setting=")
     except getopt.GetoptError, err:
         print str(err) 
@@ -299,6 +308,7 @@
     time_of_interest = None
     time_to_ignore = None
     bench_expectations = {}
+    appengine_url = None  # used for adding data to appengine datastore
     rep = None  # bench representation algorithm
     revision_range = '0:'
     regression_range = '0:'
@@ -369,10 +379,43 @@
         if exceptions:
             raise Exception('Bench values out of range:\n' +
                             '\n'.join(exceptions))
+    def write_to_appengine(lines, url, newest_revision, platform_and_alg):
+        """Writes latest bench values to appengine datastore."""
+        data = []
+        for line in lines:
+            bot = platform_and_alg[ : platform_and_alg.rfind('-')]
+            line_str = str(line)[ : str(line).find('_{')]
+            if line_str.find('.skp') < 0 or not line_str.endswith('_'):
+                # filter out non-picture and non-walltime benches
+                continue
+            bench, config = line_str.split('.skp', 1)
+            config = config [1 : -1]  # remove leading and trailing '_'

[borenet, 2013/03/04 21:32:15]

Can use config.strip('_')

[benchen, 2013/03/04 21:51:15]

Done.

+            rev, val = lines[line][-1]
+            if rev != newest_revision:
+                continue
+            data.append({'master': 'Skia', 'bot': bot,
+                         'test': config + '/' + bench,
+                         'revision': rev, 'value': val, 'error': 0})
+        while data:
+          curr_data = data[ : DATA_SIZE]
+          data = data[DATA_SIZE : ]
+          req = urllib2.Request(
+              appengine_url, urllib.urlencode({'data': json.dumps(curr_data)}))

[borenet, 2013/03/04 21:32:15]

Do we need a password?  I think we *should* need a password, since we don't want
anyone just uploading garbage data.

[benchen, 2013/03/04 21:51:15]

That's under development. We'll implement an IP filter and later account
authorization. But not sure how long it'll take... will leave as is for now
(before that, or before we integrate with chrome-perf).
On 2013/03/04 21:32:15, borenet wrote:

+          try:
+            urllib2.urlopen(req)
+          except urllib2.HTTPError, e:
+              sys.stderr.write("HTTPError: %d for JSON %s\n" % (e.code, data))
+          except urllib2.URLError, e:
+              sys.stderr.write("URLError: %s for JSON %s\n" % (
+                  str(e.reason), data))
+          except httplib.HTTPException, e:
+              sys.stderr.write("HTTPException for JSON %s\n" % lines)

[borenet, 2013/03/04 21:32:15]

Should we collect errors so that we can exit non-zero on failure?  I think we
want to know when we didn't upload as expected.

[benchen, 2013/03/04 21:51:15]

Don't want this to block any other steps on the bots yet - this hasn't gone
through load test etc. We're dogfooding it even ahead of Chrome. Will see how it
goes then decide what to do next.
On 2013/03/04 21:32:15, borenet wrote:

 
     try:
         for option, value in opts:
-            if option == "-b":
+            if option == "-a":
+                appengine_url = value
+            elif option == "-b":
                 bench_of_interest = value
             elif option == "-c":
                 config_of_interest = value
@@ -461,6 +504,10 @@
     output_xhtml(lines, oldest_revision, newest_revision, ignored_revision_data_points,
                  regressions, requested_width, requested_height, title)
 
+    if appengine_url:
+        write_to_appengine(lines, appengine_url, newest_revision,
+                           platform_and_alg)
+
     check_expectations(lines, bench_expectations, newest_revision,
                        platform_and_alg)