Use a datastore batcher for build queries.

milo/appengine/buildbot/datastore.go

Index: milo/appengine/buildbot/datastore.go
diff --git a/milo/appengine/buildbot/datastore.go b/milo/appengine/buildbot/datastore.go
new file mode 100644
index 0000000000000000000000000000000000000000..cef6ec9429be69fcb6b15535d22d227167e78565
--- /dev/null
+++ b/milo/appengine/buildbot/datastore.go
@@ -0,0 +1,31 @@
+// Copyright 2016 The LUCI Authors. All rights reserved.
+// Use of this source code is governed under the Apache License, Version 2.0
+// that can be found in the LICENSE file.
+
+package buildbot
+
+import (
+	ds "github.com/luci/gae/service/datastore"
+
+	"golang.org/x/net/context"
+)
+
+// buildQueryBatchSize is the batch size to use when querying build. It is
+// employed as an upper bound by getBuildQueryBatcher.
+//
+// This should be tuned to the observed query timeout for build loading. Since
+// loading is CPU-bound, this will probably be low. If build queries start
+// encountering datastore timeouts, reduce this value.
+const buildQueryBatchSize = 50
+
+// getBuildQueryBatcher returns a ds.Batcher tuned for executing queries on the
+// "buildbotBuild" entity.
+func getBuildQueryBatcher(c context.Context) *ds.Batcher {

[hinoka, 2017/01/31 19:05:32]

Would it make more sense to put this in the ds datastore module?

[dnj, 2017/01/31 19:19:39]

In the "ds" module, as in "luci/gae"? Batcher is a construct in luci/gae
already.

It's not the default behavior because it's potentially imperfect. It uses
cursors behind the scenes to resume datastore queries, so in some circumstances
the query result may not match the result of a full GetAll. It's not built-in
because we want the user to elect to have this imperfection, but it should be
fine for Milo's purposes, though. 

Also, the batch size is difficult to estimate. Here, it's MaxQueryDuration /
((QueryRPCOverhead * NumRPC) + (DeserializeOverhead * NumEntities)). Lots of
these are unknowns, and building a system to automatically adapt is non-trivial
and also doesn't exist ATM.

+	constraints := ds.Raw(c).Constraints()
+	if constraints.QueryBatchSize > buildQueryBatchSize {
+		constraints.QueryBatchSize = buildQueryBatchSize
+	}
+	return &ds.Batcher{
+		Size: constraints.QueryBatchSize,
+	}
+}