Chromium-recipe: add rmtree before copy for llvm_tools_to_copy.

Chromium-recipe: add rmtree before copy for llvm_tools_to_copy.

BUG=636168

[mmoroz, 2016-10-07 12:20:02]

mmoroz@chromium.org changed reviewers:
+ inferno@chromium.org, ochang@chromium.org, tandrii@chromium.org,
thakis@chromium.org

[mmoroz, 2016-10-07 12:20:03]

https://codereview.chromium.org/2399353002/diff/80001/scripts/slave/recipe_mo...
File scripts/slave/recipe_modules/archive/api.py (right):

https://codereview.chromium.org/2399353002/diff/80001/scripts/slave/recipe_mo...
scripts/slave/recipe_modules/archive/api.py:280: self.m.file.rmtree('Purge ' +
tool,
I'm not sure that it will work fine if the file doesn't exist. I've tried to add
ignore_errors=True argument to the call, but got an error:

https://paste.googleplex.com/5613218135277568

[tandrii(chromium), 2016-10-07 12:23:31]

https://codereview.chromium.org/2399353002/diff/80001/scripts/slave/recipe_mo...
File scripts/slave/recipe_modules/archive/api.py (right):

https://codereview.chromium.org/2399353002/diff/80001/scripts/slave/recipe_mo...
scripts/slave/recipe_modules/archive/api.py:280: self.m.file.rmtree('Purge ' +
tool,
On 2016/10/07 12:20:02, mmoroz wrote:
> I'm not sure that it will work fine if the file doesn't exist. I've tried to
add
> ignore_errors=True argument to the call, but got an error:
> 
> https://paste.googleplex.com/5613218135277568

The error you got says "TypeError: Got extra keys while setting ConfigGroup:
{'ignore_errors': True}"

Are you sure your spelled kwarg correctly that it is accepted by rmtree?

[Oliver Chang, 2016-10-07 16:41:26]

https://codereview.chromium.org/2399353002/diff/80001/scripts/slave/recipe_mo...
File scripts/slave/recipe_modules/archive/api.py (right):

https://codereview.chromium.org/2399353002/diff/80001/scripts/slave/recipe_mo...
scripts/slave/recipe_modules/archive/api.py:280: self.m.file.rmtree('Purge ' +
tool,
On 2016/10/07 12:20:02, mmoroz wrote:
> I'm not sure that it will work fine if the file doesn't exist. I've tried to
add
> ignore_errors=True argument to the call, but got an error:
> 
> https://paste.googleplex.com/5613218135277568

Why rmtree, and not remove?

Also, did you figure out what was going wrong?

I see 2 errors (on lkgr only, and not on our libfuzzer bots):

shutil.Error:
`/b/c/b/ASAN_Release/src/third_party/llvm-build/Release+Asserts/bin/llvm-symbolizer`
and `/b/c/b/ASAN_Release/src/out/Release/llvm-symbolizer` are the same file

and

IOError: [Errno 2] No such file or directory:
'C:\\b\\c\\b\\Win_SyzyASAN_LKGR\\src\\third_party\\llvm-build\\Release+Asserts\\bin\\llvm-symbolizer.exe'

on syzyasan win bots.

The first error is weird, as I think that would imply the one in llvm-symbolizer
is a hard link of llvm-symbolizer. Not sure why it happens only on LKGR builds.

For the second one, I think we'll need to check if the binary exists before we
copy.

Maybe we should just add checks to make sure that the source exists, and the
destination doesn't, before doing the copy. The file recipe_module doesn't
actually pass your kwargs to the copy call:
https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/file/api.py

[mmoroz, 2016-10-07 16:50:22]

https://codereview.chromium.org/2399353002/diff/80001/scripts/slave/recipe_mo...
File scripts/slave/recipe_modules/archive/api.py (right):

https://codereview.chromium.org/2399353002/diff/80001/scripts/slave/recipe_mo...
scripts/slave/recipe_modules/archive/api.py:280: self.m.file.rmtree('Purge ' +
tool,
On 2016/10/07 16:41:26, Oliver Chang wrote:
> On 2016/10/07 12:20:02, mmoroz wrote:
> > I'm not sure that it will work fine if the file doesn't exist. I've tried to
> add
> > ignore_errors=True argument to the call, but got an error:
> > 
> > https://paste.googleplex.com/5613218135277568
> 
> Why rmtree, and not remove?
> 
> Also, did you figure out what was going wrong?
> 
> I see 2 errors (on lkgr only, and not on our libfuzzer bots):
> 
> shutil.Error:
>
`/b/c/b/ASAN_Release/src/third_party/llvm-build/Release+Asserts/bin/llvm-symbolizer`
> and `/b/c/b/ASAN_Release/src/out/Release/llvm-symbolizer` are the same file
> 
> and
> 
> IOError: [Errno 2] No such file or directory:
>
'C:\\b\\c\\b\\Win_SyzyASAN_LKGR\\src\\third_party\\llvm-build\\Release+Asserts\\bin\\llvm-symbolizer.exe'
> 
> on syzyasan win bots.
> 
> The first error is weird, as I think that would imply the one in
llvm-symbolizer
> is a hard link of llvm-symbolizer. Not sure why it happens only on LKGR
builds.
> 
> For the second one, I think we'll need to check if the binary exists before we
> copy.
> 
> Maybe we should just add checks to make sure that the source exists, and the
> destination doesn't, before doing the copy. The file recipe_module doesn't
> actually pass your kwargs to the copy call:
>
https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/file/api.py

Basically, I was working on this CL while waiting for someone to LGTM the
revert. I thought that I could implement some good fix faster then get a LGTM
for the revert. Since the revert has been landed, I would be happy to delete
this CL and leave this problem up to you :)


rmtree() because I haven't figured out that there is remove() function :)

Adding checks sounds good to me.

[Oliver Chang, 2016-10-07 17:38:44]

On 2016/10/07 16:50:22, mmoroz wrote:
>
https://codereview.chromium.org/2399353002/diff/80001/scripts/slave/recipe_mo...
> File scripts/slave/recipe_modules/archive/api.py (right):
> 
>
https://codereview.chromium.org/2399353002/diff/80001/scripts/slave/recipe_mo...
> scripts/slave/recipe_modules/archive/api.py:280: self.m.file.rmtree('Purge ' +
> tool,
> On 2016/10/07 16:41:26, Oliver Chang wrote:
> > On 2016/10/07 12:20:02, mmoroz wrote:
> > > I'm not sure that it will work fine if the file doesn't exist. I've tried
to
> > add
> > > ignore_errors=True argument to the call, but got an error:
> > > 
> > > https://paste.googleplex.com/5613218135277568
> > 
> > Why rmtree, and not remove?
> > 
> > Also, did you figure out what was going wrong?
> > 
> > I see 2 errors (on lkgr only, and not on our libfuzzer bots):
> > 
> > shutil.Error:
> >
>
`/b/c/b/ASAN_Release/src/third_party/llvm-build/Release+Asserts/bin/llvm-symbolizer`
> > and `/b/c/b/ASAN_Release/src/out/Release/llvm-symbolizer` are the same file
> > 
> > and
> > 
> > IOError: [Errno 2] No such file or directory:
> >
>
'C:\\b\\c\\b\\Win_SyzyASAN_LKGR\\src\\third_party\\llvm-build\\Release+Asserts\\bin\\llvm-symbolizer.exe'
> > 
> > on syzyasan win bots.
> > 
> > The first error is weird, as I think that would imply the one in
> llvm-symbolizer
> > is a hard link of llvm-symbolizer. Not sure why it happens only on LKGR
> builds.
> > 
> > For the second one, I think we'll need to check if the binary exists before
we
> > copy.
> > 
> > Maybe we should just add checks to make sure that the source exists, and the
> > destination doesn't, before doing the copy. The file recipe_module doesn't
> > actually pass your kwargs to the copy call:
> >
>
https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/file/api.py
> 
> Basically, I was working on this CL while waiting for someone to LGTM the
> revert. I thought that I could implement some good fix faster then get a LGTM
> for the revert. Since the revert has been landed, I would be happy to delete
> this CL and leave this problem up to you :)
> 
> 
> rmtree() because I haven't figured out that there is remove() function :)
> 
> Adding checks sounds good to me.

Created https://codereview.chromium.org/2394163006/. Closing this one.

[Oliver Chang, 2016-10-07 17:38:49]

Description was changed from

==========
Chromium-recipe: add rmtree before copy for llvm_tools_to_copy.

BUG=636168
==========

to

==========
Chromium-recipe: add rmtree before copy for llvm_tools_to_copy.

BUG=636168
==========