Robustify package upgrades and dependency checking.

Robustify package upgrades and dependency checking.

- Unmerge appropriate packages during upgrades and downgrades.
- Calculate time spent in dependency generation to the tenth of a second.
- Only track dependencies of packages that are actually being installed.
- Ignore PDEPEND, as it has no impact on dependency ordering.
- Only break dependency chains that go against Portage's install order.
- Rename Failed -> Retrying.
- Print emerge command lines as they are run.

TEST=Emerged hard-host-depends and ran build_packages with parallel_emerge
BUG=none

[Nick Sanders, 2010-07-02 00:07:16]

http://codereview.chromium.org/2886010/diff/1/2
File parallel_emerge (right):

http://codereview.chromium.org/2886010/diff/1/2#newcode347
parallel_emerge:347: if pkg in deps_ordering:
This will remove deps across non-installed packages which might (?) be bad..
Maybe do all the packages, then collapse the tree, that should allow the
sanitize step to work still..

http://codereview.chromium.org/2886010/diff/1/2#newcode348
parallel_emerge:348: action = packages[pkg]["action"]
can we add package[]["deps_info"] to contain deps_ordering info?

http://codereview.chromium.org/2886010/diff/1/2#newcode377
parallel_emerge:377: stack = oldstack + [dep]
checking if dep is in oldstack will allow us to find other cycles and not loop
forever.

http://codereview.chromium.org/2886010/diff/1/2#newcode490
parallel_emerge:490: cmdline += " && %s -C unmerge =%s" % (emerge_cmd, package)
unmerge?

http://codereview.chromium.org/2886010/diff/1/2#newcode640
parallel_emerge:640: scheduler = EmergeQueue(dependency_graph,
dependency_ordering)
then don't need deps_ordering anymore

[davidjames, 2010-07-02 02:46:44]

Thanks for the feedback. PTAL.

http://codereview.chromium.org/2886010/diff/1/2
File parallel_emerge (right):

http://codereview.chromium.org/2886010/diff/1/2#newcode347
parallel_emerge:347: if pkg in deps_ordering:
On 2010/07/02 00:07:16, Nick Sanders wrote:
> This will remove deps across non-installed packages which might (?) be bad..
> Maybe do all the packages, then collapse the tree, that should allow the
> sanitize step to work still..

Done.

http://codereview.chromium.org/2886010/diff/1/2#newcode348
parallel_emerge:348: action = packages[pkg]["action"]
On 2010/07/02 00:07:16, Nick Sanders wrote:
> can we add package[]["deps_info"] to contain deps_ordering info?

Done.

http://codereview.chromium.org/2886010/diff/1/2#newcode377
parallel_emerge:377: stack = oldstack + [dep]
On 2010/07/02 00:07:16, Nick Sanders wrote:
> checking if dep is in oldstack will allow us to find other cycles and not loop
> forever.

Done.

http://codereview.chromium.org/2886010/diff/1/2#newcode490
parallel_emerge:490: cmdline += " && %s -C unmerge =%s" % (emerge_cmd, package)
On 2010/07/02 00:07:16, Nick Sanders wrote:
> unmerge?

Done.

http://codereview.chromium.org/2886010/diff/1/2#newcode640
parallel_emerge:640: scheduler = EmergeQueue(dependency_graph,
dependency_ordering)
On 2010/07/02 00:07:16, Nick Sanders wrote:
> then don't need deps_ordering anymore

Done.

http://codereview.chromium.org/2886010/diff/1/2#newcode640
parallel_emerge:640: scheduler = EmergeQueue(dependency_graph,
dependency_ordering)
On 2010/07/02 00:07:16, Nick Sanders wrote:
> then don't need deps_ordering anymore

Done.

[Nick Sanders, 2010-07-02 03:01:43]

lgtm w/ following

http://codereview.chromium.org/2886010/diff/8001/9001
File parallel_emerge (right):

http://codereview.chromium.org/2886010/diff/8001/9001#newcode461
parallel_emerge:461: # Store the dependency ordering
stale

http://codereview.chromium.org/2886010/diff/8001/9001#newcode504
parallel_emerge:504: cmdline = cmdline = EmergeCommand() + " --nodeps --oneshot
=" + target
typo

http://codereview.chromium.org/2886010/diff/8001/9001#newcode507
parallel_emerge:507: emerge_cmd = "emerge"
Can we use included args to pickup any "-p" or "--root"

http://codereview.chromium.org/2886010/diff/8001/9001#newcode649
parallel_emerge:649: dependency_tree, dependency_ordering =
DepsToTree(deps_output)
also dependency_info

[davidjames, 2010-07-02 03:07:24]

Fixed. Will commit later after more testing.

http://codereview.chromium.org/2886010/diff/8001/9001
File parallel_emerge (right):

http://codereview.chromium.org/2886010/diff/8001/9001#newcode461
parallel_emerge:461: # Store the dependency ordering
On 2010/07/02 03:01:43, Nick Sanders wrote:
> stale

Done.

http://codereview.chromium.org/2886010/diff/8001/9001#newcode504
parallel_emerge:504: cmdline = cmdline = EmergeCommand() + " --nodeps --oneshot
=" + target
On 2010/07/02 03:01:43, Nick Sanders wrote:
> typo

Done.

http://codereview.chromium.org/2886010/diff/8001/9001#newcode507
parallel_emerge:507: emerge_cmd = "emerge"
On 2010/07/02 03:01:43, Nick Sanders wrote:
> Can we use included args to pickup any "-p" or "--root"

Done.

http://codereview.chromium.org/2886010/diff/8001/9001#newcode649
parallel_emerge:649: dependency_tree, dependency_ordering =
DepsToTree(deps_output)
On 2010/07/02 03:01:43, Nick Sanders wrote:
> also dependency_info 

Done.