[depot_tools] Find, upload and apply patchset dependencies

Find, upload and apply patchset dependencies.

Here is an explanation of the changes in each module:

* git_cl.py -
IF a local branch is being tracked AND a CL has been uploaded there THEN use the CL's issue number and latest patchset as a dependency.

* upload.py -
Uploads the patchset dependency, if it exists, to Rietveld (Rietveld will be able to parse this when https://codereview.chromium.org/1155513002/ lands).

* rietveld.py -
Adds utility methods to get patchset dependencies from the new Rietveld endpoint (the endpoint will exist when https://codereview.chromium.org/1155513002/ lands).

* apply_issue.py -
If CL3 depends on CL2 which in turn depends on CL1 then apply_issue will gather a list of all issues and patchsets to apply (Eg: [CL1:PS1, CL2:PS1, CL3:PS2]).
apply_issue will then loop over the list applying each dependency.
Note: The apply_issue.py diff looks much worse than it is. Please see my comment in
https://codereview.chromium.org/1149653002/diff/260001/apply_issue.py#oldcode169


Tested end-to-end using a test Git repository (https://skia.googlesource.com/skiabot-test/) and the following CLs created in my test Rietveld instance:
* https://skia-codereview-staging.appspot.com/931002  ('Branch1 CL')
* https://skia-codereview-staging.appspot.com/5001001 ('Branch2 CL')
* https://skia-codereview-staging.appspot.com/9881001 ('Branch3 CL')
* https://skia-codereview-staging.appspot.com/3951001 ('Branch3.1 CL')
Opt into the new UI and observe the new 'Depends on Patchset' and 'Dependent Patchsets' sections in the above CLs.


Design doc is here: https://docs.google.com/document/d/1KZGFKZpOPvco81sYVRCzwlnjGctup71RAzY0MSb0ntc/edit#heading=h.6r6lt4tsvssw

BUG=502255
Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=295778

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=295799

[rmistry, 2015-06-03 12:21:15]

Patchset #11 (id:200001) has been deleted

[rmistry, 2015-06-03 13:30:01]

https://codereview.chromium.org/1149653002/diff/260001/apply_issue.py
File apply_issue.py (left):

https://codereview.chromium.org/1149653002/diff/260001/apply_issue.py#oldcode169
apply_issue.py:169: print('Downloading the patch.')
The diff here looks worse than it is.

This whole section has been moved to line 203 on the right.
What has changed is:
* Print messages now tell you the issue url.
* We now use issue_to_apply and patchset_to_apply from the loop on line 200
instead of options.issue and options.patchset.

[rmistry, 2015-06-08 14:05:35]

rmistry@google.com changed reviewers:
+ agable@chromium.org

[agable, 2015-06-08 20:05:50]

agable@chromium.org changed reviewers:
+ jrobbins@chromium.org

[agable, 2015-06-08 20:05:53]

https://codereview.chromium.org/1149653002/diff/280001/apply_issue.py
File apply_issue.py (right):

https://codereview.chromium.org/1149653002/diff/280001/apply_issue.py#newcode172
apply_issue.py:172: depends_on_info_tokens = depends_on_info.split(':')
I realize this comment is more intended for the Rietveld CL, but... any reason
this has to parse it out, as opposed to having the get_depends_on_patchset
endpoint return a nice json blob or something?

https://codereview.chromium.org/1149653002/diff/280001/apply_issue.py#newcode181
apply_issue.py:181: print ('The patchset that was marked as a dependency no
longer '
Might be helpful to print something like 'And therefore it is likely that this
patch will not apply cleanly.'

https://codereview.chromium.org/1149653002/diff/280001/apply_issue.py#newcode214
apply_issue.py:214: patchset.patches = [patch for patch in patchset.patches
+jrobbins Take a look at the intra-line diffing done here. It's weirdly
inconsistent. This line and the similar one two lines down show two spaces being
added in the middle of the indent (why there? why not at the beginning or end of
the indent?). And most of the other lines in this area show the diff as being
the first two characters of real text on the line, which is clearly wrong (the
addition to each line is two spaces of indent). The lines with comments are even
weirder. Any idea what's going weird here, and how to fix it?

https://codereview.chromium.org/1149653002/diff/280001/git_cl.py
File git_cl.py (right):

https://codereview.chromium.org/1149653002/diff/280001/git_cl.py#newcode2074
git_cl.py:2074: # Look for dependent patchsets. See crbug/480453 for more
details.
nit: crbug.com/480453

https://codereview.chromium.org/1149653002/diff/280001/rietveld.py
File rietveld.py (right):

https://codereview.chromium.org/1149653002/diff/280001/rietveld.py#newcode94
rietveld.py:94: # The get_depends_on_patchset endpoint does not exist on this
Rietveld
Add a TODO to make this a real error eventually?

[rmistry, 2015-06-09 16:16:45]

https://codereview.chromium.org/1149653002/diff/280001/apply_issue.py
File apply_issue.py (right):

https://codereview.chromium.org/1149653002/diff/280001/apply_issue.py#newcode172
apply_issue.py:172: depends_on_info_tokens = depends_on_info.split(':')
On 2015/06/08 20:05:51, agable wrote:
> I realize this comment is more intended for the Rietveld CL, but... any reason
> this has to parse it out, as opposed to having the get_depends_on_patchset
> endpoint return a nice json blob or something?

Done here and in https://codereview.chromium.org/1155513002/

https://codereview.chromium.org/1149653002/diff/280001/apply_issue.py#newcode181
apply_issue.py:181: print ('The patchset that was marked as a dependency no
longer '
On 2015/06/08 20:05:53, agable wrote:
> Might be helpful to print something like 'And therefore it is likely that this
> patch will not apply cleanly.'

Done.

https://codereview.chromium.org/1149653002/diff/280001/apply_issue.py#newcode214
apply_issue.py:214: patchset.patches = [patch for patch in patchset.patches
On 2015/06/08 20:05:53, agable wrote:
> +jrobbins Take a look at the intra-line diffing done here. It's weirdly
> inconsistent. This line and the similar one two lines down show two spaces
being
> added in the middle of the indent (why there? why not at the beginning or end
of
> the indent?). And most of the other lines in this area show the diff as being
> the first two characters of real text on the line, which is clearly wrong (the
> addition to each line is two spaces of indent). The lines with comments are
even
> weirder. Any idea what's going weird here, and how to fix it?

Jason, this comment is for you.

https://codereview.chromium.org/1149653002/diff/280001/git_cl.py
File git_cl.py (right):

https://codereview.chromium.org/1149653002/diff/280001/git_cl.py#newcode2074
git_cl.py:2074: # Look for dependent patchsets. See crbug/480453 for more
details.
On 2015/06/08 20:05:53, agable wrote:
> nit: crbug.com/480453

Done.

https://codereview.chromium.org/1149653002/diff/280001/rietveld.py
File rietveld.py (right):

https://codereview.chromium.org/1149653002/diff/280001/rietveld.py#newcode94
rietveld.py:94: # The get_depends_on_patchset endpoint does not exist on this
Rietveld
On 2015/06/08 20:05:53, agable wrote:
> Add a TODO to make this a real error eventually?

Done.

[agable, 2015-06-09 17:20:45]

lgtm

[rmistry, 2015-06-19 16:34:57]

Patchset #16 (id:320001) has been deleted

[rmistry, 2015-06-19 16:41:34]

On 2015/06/09 17:20:45, agable wrote:
> lgtm

I plan to submit this on Monday morning after warning IRC.

[rmistry, 2015-06-19 16:41:48]

The CQ bit was checked by rmistry@google.com to run a CQ dry run

[rmistry, 2015-06-19 16:41:49]

The patchset sent to the CQ was uploaded after l-g-t-m from agable@chromium.org
Link to the patchset: https://codereview.chromium.org/1149653002/#ps340001
(title: "Rebase")

[commit-bot: I haz the power, 2015-06-19 16:42:10]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1149653002/340001

[rmistry, 2015-06-19 16:46:21]

The CQ bit was checked by rmistry@google.com to run a CQ dry run

[rmistry, 2015-06-19 16:46:21]

The patchset sent to the CQ was uploaded after l-g-t-m from agable@chromium.org
Link to the patchset: https://codereview.chromium.org/1149653002/#ps360001
(title: "Fix lint issue")

[commit-bot: I haz the power, 2015-06-19 16:46:38]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1149653002/360001

[commit-bot: I haz the power, 2015-06-19 16:49:30]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2015-06-19 16:49:31]

Dry run: This issue passed the CQ dry run.

[rmistry, 2015-06-22 12:16:54]

The CQ bit was checked by rmistry@google.com

[commit-bot: I haz the power, 2015-06-22 12:17:02]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1149653002/360001

[commit-bot: I haz the power, 2015-06-22 12:19:25]

Committed patchset #17 (id:360001) as
http://src.chromium.org/viewvc/chrome?view=rev&revision=295778

[rmistry, 2015-06-22 15:19:33]

A revert of this CL (patchset #17 id:360001) has been created in
https://codereview.chromium.org/1200773003/ by rmistry@google.com.

The reason for reverting is: Ran into a crash during the bot_update step here:
https://uberchromegw.corp.google.com/i/internal.infra.try/builders/infra-inte....

[rmistry, 2015-06-22 17:12:42]

Patchset #18 (id:380001) has been deleted

[rmistry, 2015-06-22 17:15:06]

I re-opened this CL and added a fix for the failure in
https://uberchromegw.corp.google.com/i/internal.infra.try/builders/infra-inte...
@patchset19.

PTAL.

[rmistry, 2015-06-23 11:10:52]

On 2015/06/22 17:15:06, rmistry wrote:
> I re-opened this CL and added a fix for the failure in
>
https://uberchromegw.corp.google.com/i/internal.infra.try/builders/infra-inte...
> @patchset19.
> 
> PTAL.

Aaron, I am going to TBR this fix because early morning is the best time to test
this and make sure everything is working, since not many are online right now. I
will address any comments in a follow up CL. Thanks!

[rmistry, 2015-06-23 11:22:00]

The CQ bit was checked by rmistry@google.com

[rmistry, 2015-06-23 11:22:00]

The patchset sent to the CQ was uploaded after l-g-t-m from agable@chromium.org
Link to the patchset: https://codereview.chromium.org/1149653002/#ps420001
(title: "Also catch ValueError")

[commit-bot: I haz the power, 2015-06-23 11:22:08]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1149653002/420001

[commit-bot: I haz the power, 2015-06-23 11:24:12]

Committed patchset #19 (id:420001) as
http://src.chromium.org/viewvc/chrome?view=rev&revision=295799