Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(407)

Side by Side Diff: scripts/master/pubsub_json_status_push.py

Issue 2422503002: Pubsub: Restrict full pending builds states to 25 per builder (from 75) (Closed)
Patch Set: Fixed bad rebase Created 4 years, 2 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # Copyright 2016 The Chromium Authors. All rights reserved. 1 # Copyright 2016 The Chromium Authors. All rights reserved.
2 # Use of this source code is governed by a BSD-style license that can be 2 # Use of this source code is governed by a BSD-style license that can be
3 # found in the LICENSE file. 3 # found in the LICENSE file.
4 4
5 import base64 5 import base64
6 import collections 6 import collections
7 import datetime 7 import datetime
8 import functools 8 import functools
9 import json 9 import json
10 import os 10 import os
(...skipping 354 matching lines...) Expand 10 before | Expand all | Expand 10 after
365 * builders: List of builders (builbot.status.builder.Builder). 365 * builders: List of builders (builbot.status.builder.Builder).
366 * slaves: List of slaves (buildbot.status.slave). 366 * slaves: List of slaves (buildbot.status.slave).
367 """ 367 """
368 builders = {builder_name: self._status.getBuilder(builder_name) 368 builders = {builder_name: self._status.getBuilder(builder_name)
369 for builder_name in self._status.getBuilderNames()} 369 for builder_name in self._status.getBuilderNames()}
370 builder_infos = {} 370 builder_infos = {}
371 for name, builder in builders.iteritems(): 371 for name, builder in builders.iteritems():
372 # This requires a deferred call since the data is queried from 372 # This requires a deferred call since the data is queried from
373 # the postgres DB (sigh). 373 # the postgres DB (sigh).
374 pending = yield builder.getPendingBuildRequestStatuses() 374 pending = yield builder.getPendingBuildRequestStatuses()
375 # Optimization cheat: only get the first 75 pending builds. 375 # Optimization cheat: only get the first 25 pending builds.
376 # This caps the amount of postgres db calls for really out of 376 # This caps the amount of postgres db calls and json size for really out
377 # control builders 377 # of control builders
378 pending = pending[:75] 378 pending = pending[:25]
379 pendingStates = yield defer.DeferredList( 379 pendingStates = yield defer.DeferredList(
380 [p.asDict_async() for p in pending]) 380 [p.asDict_async() for p in pending])
381 # Not included: basedir, cachedBuilds. 381 # Not included: basedir, cachedBuilds.
382 # cachedBuilds isn't useful and takes a ton of resources to compute. 382 # cachedBuilds isn't useful and takes a ton of resources to compute.
383 builder_info = { 383 builder_info = {
384 'slaves': builder.slavenames, 384 'slaves': builder.slavenames,
385 'currentBuilds': sorted(b.getNumber() for b in builder.currentBuilds), 385 'currentBuilds': sorted(b.getNumber() for b in builder.currentBuilds),
386 'pendingBuilds': len(pending), 386 'pendingBuilds': len(pending),
387 # p is a tuple of (success, payload) 387 # p is a tuple of (success, payload)
388 'pendingBuildStates': [p[1] for p in pendingStates if p[0]], 388 'pendingBuildStates': [p[1] for p in pendingStates if p[0]],
(...skipping 34 matching lines...) Expand 10 before | Expand all | Expand 10 after
423 return self 423 return self
424 424
425 @event_handler 425 @event_handler
426 def stepStarted(self, build, _step): 426 def stepStarted(self, build, _step):
427 # This info is included in the master json. 427 # This info is included in the master json.
428 return self 428 return self
429 429
430 @event_handler 430 @event_handler
431 def buildFinished(self, _builderName, build, _results): 431 def buildFinished(self, _builderName, build, _results):
432 self._recordBuild(build) 432 self._recordBuild(build)
OLDNEW
« no previous file with comments | « no previous file | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698