Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(417)

Side by Side Diff: parallel_emerge

Issue 2924008: Print status every 30 seconds so that buildbot doesn't kill us. (Closed) Base URL: ssh://git@chromiumos-git/crosutils.git
Patch Set: Created 10 years, 5 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 #!/usr/bin/python2.6 1 #!/usr/bin/python2.6
2 # Copyright (c) 2010 The Chromium OS Authors. All rights reserved. 2 # Copyright (c) 2010 The Chromium OS Authors. All rights reserved.
3 # Use of this source code is governed by a BSD-style license that can be 3 # Use of this source code is governed by a BSD-style license that can be
4 # found in the LICENSE file. 4 # found in the LICENSE file.
5 5
6 """Program to run emerge in parallel, for significant speedup. 6 """Program to run emerge in parallel, for significant speedup.
7 7
8 Usage: 8 Usage:
9 ./parallel_emerge --board=BOARD [emerge args] package 9 ./parallel_emerge --board=BOARD [emerge args] package
10 10
(...skipping 534 matching lines...) Expand 10 before | Expand all | Expand 10 after
545 target = self._retry_queue.pop(0) 545 target = self._retry_queue.pop(0)
546 self._emerge_queue.append(target) 546 self._emerge_queue.append(target)
547 print "Retrying emerge of %s." % target 547 print "Retrying emerge of %s." % target
548 548
549 def Run(self): 549 def Run(self):
550 """Run through the scheduled ebuilds. 550 """Run through the scheduled ebuilds.
551 551
552 Keep running so long as we have uninstalled packages in the 552 Keep running so long as we have uninstalled packages in the
553 dependency graph to merge. 553 dependency graph to merge.
554 """ 554 """
555 secs = 0
555 while self._deps_map: 556 while self._deps_map:
556 # If we have packages that are ready, kick them off. 557 # If we have packages that are ready, kick them off.
557 if self._emerge_queue and len(self._jobs) < JOBS: 558 if self._emerge_queue and len(self._jobs) < JOBS:
558 target = self._emerge_queue.pop(0) 559 target = self._emerge_queue.pop(0)
559 action = self._deps_map[target]["action"] 560 action = self._deps_map[target]["action"]
560 # We maintain a tree of all deps, if this doesn't need 561 # We maintain a tree of all deps, if this doesn't need
561 # to be installed just free up it's children and continue. 562 # to be installed just free up it's children and continue.
562 # It is possible to reinstall deps of deps, without reinstalling 563 # It is possible to reinstall deps of deps, without reinstalling
563 # first level deps, like so: 564 # first level deps, like so:
564 # chromeos (merge) -> eselect (nomerge) -> python (merge) 565 # chromeos (merge) -> eselect (nomerge) -> python (merge)
565 if action == "nomerge": 566 if action == "nomerge":
566 self._Finish(target) 567 self._Finish(target)
567 else: 568 else:
568 # Kick off the build if it's marked to be built. 569 # Kick off the build if it's marked to be built.
569 print "Emerging %s (%s)" % (target, action) 570 print "Emerging %s (%s)" % (target, action)
570 job = self._LaunchOneEmerge(target) 571 job = self._LaunchOneEmerge(target)
571 # Append it to the active jobs list. 572 # Append it to the active jobs list.
572 self._jobs.append(job) 573 self._jobs.append(job)
573 continue 574 continue
574 # Wait a bit to see if maybe some jobs finish. You can't 575 # Wait a bit to see if maybe some jobs finish. You can't
575 # wait on a set of jobs in python, so we'll just poll. 576 # wait on a set of jobs in python, so we'll just poll.
576 time.sleep(1) 577 time.sleep(1)
578 secs += 1
579 if secs % 30 == 0:
580 # Print an update.
581 self._Status()
577 582
578 # Check here that we are actually waiting for something. 583 # Check here that we are actually waiting for something.
579 if (not self._emerge_queue and 584 if (not self._emerge_queue and
580 not self._jobs and 585 not self._jobs and
581 self._deps_map): 586 self._deps_map):
582 # If we have failed on a package, retry it now. 587 # If we have failed on a package, retry it now.
583 if self._retry_queue: 588 if self._retry_queue:
584 self._Retry() 589 self._Retry()
585 # If we have failed a package twice, just give up. 590 # If we have failed a package twice, just give up.
586 elif self._failed: 591 elif self._failed:
(...skipping 73 matching lines...) Expand 10 before | Expand all | Expand 10 after
660 665
661 if VERBOSE: 666 if VERBOSE:
662 PrintDepsMap(dependency_graph) 667 PrintDepsMap(dependency_graph)
663 668
664 # Run the queued emerges. 669 # Run the queued emerges.
665 scheduler = EmergeQueue(dependency_graph) 670 scheduler = EmergeQueue(dependency_graph)
666 scheduler.Run() 671 scheduler.Run()
667 672
668 print "Done" 673 print "Done"
669 674
OLDNEW
« no previous file with comments | « no previous file | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698