Gclient with git cache: delete conflicting mirror.

Gclient with git cache: delete conflicting mirror.

Consider an SVN repo which was mirrored to git by git_updater. When SVN repo is migrated to github, the git histories of chromium mirror and new github one are different. As a result, hashes of the objects do not match.

Before, gclient would just get stuck at trying to fetch the repo after changing its remote url with errors like this:

...
[0:01:21] error: refs/heads/master does not point to a valid object!
[0:01:21] error: refs/remotes/origin/HEAD does not point to a valid object!
[0:01:21] error: refs/remotes/origin/master does not point to a valid object!
[0:01:21] error: refs/heads/master does not point to a valid object!
[0:01:21] error: refs/remotes/origin/HEAD does not point to a valid object!
[0:01:21] error: refs/remotes/origin/master does not point to a valid object!
[0:01:21] fatal: bad object HEAD
[0:01:21] error: /b/git-cache/chromium.googlesource.com-external-github.com-google-open--vcdiff did not send all necessary objects


The solution is to notice such state, delete the .git folder, and clone again.


BUG=523239
R=akuegel@chromium.org

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=296417

[Adrian Kuegel, 2015-08-21 14:34:54]

In general I think this approach should be fine. We can probably just fix it by
deleting the checkout.

[Adrian Kuegel, 2015-08-21 15:35:58]

https://codereview.chromium.org/1303293002/diff/20001/gclient_scm.py
File gclient_scm.py (right):

https://codereview.chromium.org/1303293002/diff/20001/gclient_scm.py#newcode470
gclient_scm.py:470: self.Print("tAndrii31", "gotcha")
Do you want to keep this in?

https://codereview.chromium.org/1303293002/diff/20001/gclient_scm.py#newcode489
gclient_scm.py:489: self.Print("tAndrii41", "gotcha")
Same question as above.

[Adrian Kuegel, 2015-08-21 15:40:04]

Otherwise lgtm.

[tandrii(chromium), 2015-08-21 15:40:15]

fixed!

https://codereview.chromium.org/1303293002/diff/20001/gclient_scm.py
File gclient_scm.py (right):

https://codereview.chromium.org/1303293002/diff/20001/gclient_scm.py#newcode470
gclient_scm.py:470: self.Print("tAndrii31", "gotcha")
On 2015/08/21 15:35:58, Adrian Kuegel wrote:
> Do you want to keep this in?

nope, but it's helpful to distinguish the cases during debugging.

https://codereview.chromium.org/1303293002/diff/20001/gclient_scm.py#newcode489
gclient_scm.py:489: self.Print("tAndrii41", "gotcha")
On 2015/08/21 15:35:58, Adrian Kuegel wrote:
> Same question as above.

Done.

[tandrii(chromium), 2015-08-21 15:41:15]

tandrii@chromium.org changed reviewers:
+ hinoka@chromium.org

[tandrii(chromium), 2015-08-21 15:41:15]

+hinoka who is today's MTV trooper, so that he reverts it in case of problems.

[Adrian Kuegel, 2015-08-21 15:41:16]

akuegel@chromium.org changed reviewers:
- hinoka@chromium.org

[Adrian Kuegel, 2015-08-21 15:41:16]

https://codereview.chromium.org/1303293002/diff/40001/gclient_scm.py
File gclient_scm.py (right):

https://codereview.chromium.org/1303293002/diff/40001/gclient_scm.py#newcode465
gclient_scm.py:465: try:
Just a small nit: couldn't you put these lines into its own function and remove
the code duplication below?

[tandrii(chromium), 2015-08-21 15:41:23]

The CQ bit was checked by tandrii@chromium.org

[commit-bot: I haz the power, 2015-08-21 15:41:29]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1303293002/40001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1303293002/40001

[tandrii(chromium), 2015-08-21 15:41:39]

The CQ bit was unchecked by tandrii@chromium.org

[tandrii(chromium), 2015-08-21 15:52:57]

tandrii@chromium.org changed reviewers:
+ hinoka@chromium.org

[tandrii(chromium), 2015-08-21 15:53:01]

The CQ bit was checked by tandrii@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2015-08-21 15:53:08]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1303293002/80001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1303293002/80001

[commit-bot: I haz the power, 2015-08-21 15:55:09]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2015-08-21 15:55:10]

Dry run: This issue passed the CQ dry run.

[tandrii(chromium), 2015-08-24 16:28:32]

Ryan, friendly ping :)

[Ryan Tseng, 2015-08-24 22:40:45]

hinoka@google.com changed reviewers:
+ hinoka@google.com

[Ryan Tseng, 2015-08-24 22:40:46]

rs lgtm

https://codereview.chromium.org/1303293002/diff/80001/gclient_scm.py
File gclient_scm.py (right):

https://codereview.chromium.org/1303293002/diff/80001/gclient_scm.py#newcode1057
gclient_scm.py:1057: if ('fatal: bad object HEAD' in e.stderr
This sort of error detection scares me :/.  Theres no way to know if it'll carry
over from version to verison.

though failing to run rev-list on HEAD should be signal enough that HEAD is
broken somehow?

[Vadim Sh., 2015-08-24 22:43:12]

I'll commit this, since I have CL that triggers exact same behavior and I don't
want it to blow up everything: https://codereview.chromium.org/1311263003

Will verify it works as expected on try bots.

[Vadim Sh., 2015-08-24 22:43:17]

The CQ bit was checked by vadimsh@chromium.org

[Vadim Sh., 2015-08-24 22:43:17]

The patchset sent to the CQ was uploaded after l-g-t-m from akuegel@chromium.org
Link to the patchset: https://codereview.chromium.org/1303293002/#ps80001
(title: "refacotring")

[commit-bot: I haz the power, 2015-08-24 22:43:32]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1303293002/80001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1303293002/80001

[Vadim Sh., 2015-08-24 22:43:36]

The CQ bit was unchecked by vadimsh@chromium.org

[Vadim Sh., 2015-08-24 22:44:52]

On 2015/08/24 22:43:36, Vadim Sh. wrote:
> The CQ bit was unchecked by mailto:vadimsh@chromium.org

Unchecked because spotted "WIP: Gclient fix" title in email.

Though the CLs title is fine, it's gmail treading issue... Checking CQ box
again... Sorry.

[Vadim Sh., 2015-08-24 22:44:56]

The CQ bit was checked by vadimsh@chromium.org

[commit-bot: I haz the power, 2015-08-24 22:45:02]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1303293002/80001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1303293002/80001

[commit-bot: I haz the power, 2015-08-24 22:47:51]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2015-08-24 22:47:52]

Try jobs failed on following builders:
  depot_tools_presubmit on tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/depot_tools_pre...)

[hinoka, 2015-08-24 22:52:33]

lgtm

[Vadim Sh., 2015-08-24 22:53:48]

The CQ bit was checked by vadimsh@chromium.org

[commit-bot: I haz the power, 2015-08-24 22:54:02]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1303293002/80001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1303293002/80001

[commit-bot: I haz the power, 2015-08-24 22:55:56]

Committed patchset #5 (id:80001) as
http://src.chromium.org/viewvc/chrome?view=rev&revision=296417

[Vadim Sh., 2015-08-24 23:19:42]

On 2015/08/24 22:55:56, commit-bot: I haz the power wrote:
> Committed patchset #5 (id:80001) as
> http://src.chromium.org/viewvc/chrome?view=rev&revision=296417

It worked for my CL:
http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...

Thanks, Andrii! :)

[tandrii(chromium), 2015-08-25 08:00:52]

https://codereview.chromium.org/1303293002/diff/80001/gclient_scm.py
File gclient_scm.py (right):

https://codereview.chromium.org/1303293002/diff/80001/gclient_scm.py#newcode1057
gclient_scm.py:1057: if ('fatal: bad object HEAD' in e.stderr
On 2015/08/24 22:40:46, Ryan Tseng wrote:
> This sort of error detection scares me :/.  Theres no way to know if it'll
carry
> over from version to verison.
> 
> though failing to run rev-list on HEAD should be signal enough that HEAD is
> broken somehow?
So, I thought this way:

best case -> it works
ok case -> the error changes, so our waterfall gets red. Oh well, not any worse
than now.
worst case -> the error is same, but for some other weird reason. Then we'll mv
.git to _bad_scm, even though we didn't have to. Still tolerable.