Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(478)

Side by Side Diff: parallel_emerge

Issue 3168012: Don't pass on --jobs to individual emerge processes in parallel_emerge. (Closed) Base URL: ssh://git@chromiumos-git/crosutils.git
Patch Set: Add comments because Nick loves comments Created 10 years, 4 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 #!/usr/bin/python2.6 1 #!/usr/bin/python2.6
2 # Copyright (c) 2010 The Chromium OS Authors. All rights reserved. 2 # Copyright (c) 2010 The Chromium OS Authors. All rights reserved.
3 # Use of this source code is governed by a BSD-style license that can be 3 # Use of this source code is governed by a BSD-style license that can be
4 # found in the LICENSE file. 4 # found in the LICENSE file.
5 5
6 """Program to run emerge in parallel, for significant speedup. 6 """Program to run emerge in parallel, for significant speedup.
7 7
8 Usage: 8 Usage:
9 ./parallel_emerge [--board=BOARD] [--workon=PKGS] [--no-workon-deps] 9 ./parallel_emerge [--board=BOARD] [--workon=PKGS] [--no-workon-deps]
10 [emerge args] package" 10 [emerge args] package"
(...skipping 1276 matching lines...) Expand 10 before | Expand all | Expand 10 after
1287 self._show_output = show_output 1287 self._show_output = show_output
1288 1288
1289 if "--pretend" in emerge.opts: 1289 if "--pretend" in emerge.opts:
1290 print "Skipping merge because of --pretend mode." 1290 print "Skipping merge because of --pretend mode."
1291 sys.exit(0) 1291 sys.exit(0)
1292 1292
1293 # Setup scheduler graph object. This is used by the child processes 1293 # Setup scheduler graph object. This is used by the child processes
1294 # to help schedule jobs. 1294 # to help schedule jobs.
1295 emerge.scheduler_graph = emerge.depgraph.schedulerGraph() 1295 emerge.scheduler_graph = emerge.depgraph.schedulerGraph()
1296 1296
1297 # Calculate how many jobs we can run in parallel. We don't want to pass
1298 # the --jobs flag over to emerge itself, because that'll tell emerge to
1299 # hide its output, and said output is quite useful for debugging hung
1300 # jobs.
1297 procs = min(self._total_jobs, 1301 procs = min(self._total_jobs,
1298 emerge.opts.get("--jobs", multiprocessing.cpu_count())) 1302 emerge.opts.pop("--jobs", multiprocessing.cpu_count()))
1299 self._emerge_queue = multiprocessing.Queue() 1303 self._emerge_queue = multiprocessing.Queue()
1300 self._job_queue = multiprocessing.Queue() 1304 self._job_queue = multiprocessing.Queue()
1301 self._print_queue = multiprocessing.Queue() 1305 self._print_queue = multiprocessing.Queue()
1302 args = (self._emerge_queue, self._job_queue, emerge, package_db) 1306 args = (self._emerge_queue, self._job_queue, emerge, package_db)
1303 self._pool = multiprocessing.Pool(procs, EmergeWorker, args) 1307 self._pool = multiprocessing.Pool(procs, EmergeWorker, args)
1304 self._print_worker = multiprocessing.Process(target=PrintWorker, 1308 self._print_worker = multiprocessing.Process(target=PrintWorker,
1305 args=[self._print_queue]) 1309 args=[self._print_queue])
1306 self._print_worker.start() 1310 self._print_worker.start()
1307 1311
1308 # Initialize the failed queue to empty. 1312 # Initialize the failed queue to empty.
(...skipping 241 matching lines...) Expand 10 before | Expand all | Expand 10 after
1550 world_set.update(new_world_pkgs) 1554 world_set.update(new_world_pkgs)
1551 1555
1552 # Update environment (library cache, symlinks, etc.) 1556 # Update environment (library cache, symlinks, etc.)
1553 if deps.board and "--pretend" not in emerge.opts: 1557 if deps.board and "--pretend" not in emerge.opts:
1554 portage.env_update() 1558 portage.env_update()
1555 1559
1556 print "Done" 1560 print "Done"
1557 1561
1558 if __name__ == "__main__": 1562 if __name__ == "__main__":
1559 main() 1563 main()
OLDNEW
« no previous file with comments | « no previous file | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698