Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(1174)

Side by Side Diff: parallel_emerge

Issue 2946003: parallel_emerge: print total time and load in status output (Closed) Base URL: http://src.chromium.org/git/crosutils.git
Patch Set: Style fixes Created 10 years, 5 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 #!/usr/bin/python2.6 1 #!/usr/bin/python2.6
2 # Copyright (c) 2010 The Chromium OS Authors. All rights reserved. 2 # Copyright (c) 2010 The Chromium OS Authors. All rights reserved.
3 # Use of this source code is governed by a BSD-style license that can be 3 # Use of this source code is governed by a BSD-style license that can be
4 # found in the LICENSE file. 4 # found in the LICENSE file.
5 5
6 """Program to run emerge in parallel, for significant speedup. 6 """Program to run emerge in parallel, for significant speedup.
7 7
8 Usage: 8 Usage:
9 ./parallel_emerge --board=BOARD [emerge args] package 9 ./parallel_emerge --board=BOARD [emerge args] package
10 10
(...skipping 49 matching lines...) Expand 10 before | Expand all | Expand 10 after
60 # emerge args we are passing through. 60 # emerge args we are passing through.
61 PACKAGE = None 61 PACKAGE = None
62 EMERGE_ARGS = "" 62 EMERGE_ARGS = ""
63 BOARD = None 63 BOARD = None
64 64
65 # Runtime flags. TODO(): Maybe make these command-line options or 65 # Runtime flags. TODO(): Maybe make these command-line options or
66 # environment variables. 66 # environment variables.
67 VERBOSE = False 67 VERBOSE = False
68 AUTOCLEAN = False 68 AUTOCLEAN = False
69 69
70 # Global start time
71 GLOBAL_START = time.time()
72
70 73
71 def ParseArgs(argv): 74 def ParseArgs(argv):
72 """Set global vars based on command line. 75 """Set global vars based on command line.
73 76
74 We need to be compatible with emerge arg format. 77 We need to be compatible with emerge arg format.
75 We scrape --board=XXX and --jobs=XXX, and distinguish between args 78 We scrape --board=XXX and --jobs=XXX, and distinguish between args
76 and package names. 79 and package names.
77 TODO(): Robustify argument processing, as it's possible to 80 TODO(): Robustify argument processing, as it's possible to
78 pass in many two argument parameters that are difficult 81 pass in many two argument parameters that are difficult
79 to programmatically identify, although we don't currently 82 to programmatically identify, although we don't currently
(...skipping 386 matching lines...) Expand 10 before | Expand all | Expand 10 after
466 # List of total package installs represented in deps_map. 469 # List of total package installs represented in deps_map.
467 install_jobs = [x for x in deps_map if deps_map[x]["action"] == "merge"] 470 install_jobs = [x for x in deps_map if deps_map[x]["action"] == "merge"]
468 self._total_jobs = len(install_jobs) 471 self._total_jobs = len(install_jobs)
469 472
470 # Initialize the ready queue, these are jobs with no unmet dependencies. 473 # Initialize the ready queue, these are jobs with no unmet dependencies.
471 self._emerge_queue = [x for x in deps_map if not deps_map[x]["needs"]] 474 self._emerge_queue = [x for x in deps_map if not deps_map[x]["needs"]]
472 # Initialize the failed queue to empty. 475 # Initialize the failed queue to empty.
473 self._retry_queue = [] 476 self._retry_queue = []
474 self._failed = {} 477 self._failed = {}
475 478
479 def _LoadAvg(self):
480 loads = open('/proc/loadavg', 'r').readline().split()[:3]
481 return ' '.join(loads)
482
476 def _Status(self): 483 def _Status(self):
477 """Print status.""" 484 """Print status."""
478 print "Pending %s, Ready %s, Running %s, Retrying %s, Total %s" % ( 485 seconds = time.time() - GLOBAL_START
486 print "Pending %s, Ready %s, Running %s, Retrying %s, Total %s " \
487 "[Time %dm%ds Load %s]" % (
479 len(self._deps_map), len(self._emerge_queue), 488 len(self._deps_map), len(self._emerge_queue),
480 len(self._jobs), len(self._retry_queue), self._total_jobs) 489 len(self._jobs), len(self._retry_queue), self._total_jobs,
490 seconds / 60, seconds % 60, self._LoadAvg())
481 491
482 def _LaunchOneEmerge(self, target): 492 def _LaunchOneEmerge(self, target):
483 """Run emerge --nodeps to do a single package install. 493 """Run emerge --nodeps to do a single package install.
484 494
485 If this is a pseudopackage, that means we're done, and can select in in the 495 If this is a pseudopackage, that means we're done, and can select in in the
486 world file. 496 world file.
487 Args: 497 Args:
488 target: The full package name of the package to install. 498 target: The full package name of the package to install.
489 eg. "sys-apps/portage-2.17" 499 eg. "sys-apps/portage-2.17"
490 Returns: 500 Returns:
(...skipping 169 matching lines...) Expand 10 before | Expand all | Expand 10 after
660 670
661 if VERBOSE: 671 if VERBOSE:
662 PrintDepsMap(dependency_graph) 672 PrintDepsMap(dependency_graph)
663 673
664 # Run the queued emerges. 674 # Run the queued emerges.
665 scheduler = EmergeQueue(dependency_graph) 675 scheduler = EmergeQueue(dependency_graph)
666 scheduler.Run() 676 scheduler.Run()
667 677
668 print "Done" 678 print "Done"
669 679
OLDNEW
« no previous file with comments | « no previous file | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698