Prebuilts should not gate normal builds if the server responds with an error.

Prebuilts should not gate normal builds if the server responds with an error.

Recently the tegra2 prebuilts were uploaded with an internal only address and caused external builds to fail.

If a user cannot get the Packages file their build should continue as normal as if the server responded without a packages file in general.

BUG=chromium-os:9917
TEST=Set PORTAGE_BINHOST to invalid value and run build.

Change-Id: I95cab67da7a539afe8c48c831788eca29cc5dc69

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=3f89856

[Nick Sanders, 2010-12-21 01:32:31]

http://codereview.chromium.org/6085001/diff/1/parallel_emerge
File parallel_emerge (right):

http://codereview.chromium.org/6085001/diff/1/parallel_emerge#newcode1841
parallel_emerge:1841: print "Cannot resolve binhost. Building from source..."
Can we print like 10 lines of **********'s here and add an email address or bug
filing link? Not having prebuilts means the build is as good as broken...

[scottz, 2011-01-04 18:41:36]

As far as the building from source step is this something that only the builders
will run or is it something that runs for everyone using parallel_emerge? 

While I feel strongly that if prebuilts are not present the build should not
fail, if a developer requests a binary build and there are none it should fail
and tell them how they can complete their build with source. In the case of
buildbots we would probably want an option to modify the behavior so we can make
sure a critical binary build will still psas in this case.

Thoughts?

[sosa, 2011-01-04 18:43:43]

I believe this is why portage has usepkg vs. usepkgonly.

On Tue, Jan 4, 2011 at 10:41 AM,  <scottz@chromium.org> wrote:
> As far as the building from source step is this something that only the
> builders
> will run or is it something that runs for everyone using parallel_emerge?
>
> While I feel strongly that if prebuilts are not present the build should not
> fail, if a developer requests a binary build and there are none it should
> fail
> and tell them how they can complete their build with source. In the case of
> buildbots we would probably want an option to modify the behavior so we can
> make
> sure a critical binary build will still psas in this case.
>
> Thoughts?
>
> http://codereview.chromium.org/6085001/
>

[kliegs, 2011-01-04 19:15:17]

I think what Chris says is right - can't you run build_packages with
--usepkgonly which would do what you say?  Maybe we could document this more
or later create an interactive prompt of "binary packages not available,
build from source? Yes/No/Always/neVer"

On Tue, Jan 4, 2011 at 1:43 PM, Chris Sosa <sosa@chromium.org> wrote:

> I believe this is why portage has usepkg vs. usepkgonly.
>
> On Tue, Jan 4, 2011 at 10:41 AM,  <scottz@chromium.org> wrote:
> > As far as the building from source step is this something that only the
> > builders
> > will run or is it something that runs for everyone using parallel_emerge?
> >
> > While I feel strongly that if prebuilts are not present the build should
> not
> > fail, if a developer requests a binary build and there are none it should
> > fail
> > and tell them how they can complete their build with source. In the case
> of
> > buildbots we would probably want an option to modify the behavior so we
> can
> > make
> > sure a critical binary build will still psas in this case.
> >
> > Thoughts?
> >
> > http://codereview.chromium.org/6085001/
> >
>

[scottz, 2011-01-04 20:36:53]

So if we pass --usepkg to the bot and it fails to get a package it will continue
to build from source? If that is the case that works for me :)


On 2011/01/04 19:15:17, kliegs wrote:
> I think what Chris says is right - can't you run build_packages with
> --usepkgonly which would do what you say?  Maybe we could document this more
> or later create an interactive prompt of "binary packages not available,
> build from source? Yes/No/Always/neVer"
> 
> On Tue, Jan 4, 2011 at 1:43 PM, Chris Sosa <mailto:sosa@chromium.org> wrote:
> 
> > I believe this is why portage has usepkg vs. usepkgonly.
> >
> > On Tue, Jan 4, 2011 at 10:41 AM,  <mailto:scottz@chromium.org> wrote:
> > > As far as the building from source step is this something that only the
> > > builders
> > > will run or is it something that runs for everyone using parallel_emerge?
> > >
> > > While I feel strongly that if prebuilts are not present the build should
> > not
> > > fail, if a developer requests a binary build and there are none it should
> > > fail
> > > and tell them how they can complete their build with source. In the case
> > of
> > > buildbots we would probably want an option to modify the behavior so we
> > can
> > > make
> > > sure a critical binary build will still psas in this case.
> > >
> > > Thoughts?
> > >
> > > http://codereview.chromium.org/6085001/
> > >
> >

[davidjames, 2011-01-04 21:35:23]

On Tue, Jan 4, 2011 at 12:36 PM, <scottz@chromium.org> wrote:

> So if we pass --usepkg to the bot and it fails to get a package it will
> continue
> to build from source? If that is the case that works for me :)


Yup, that's always worked.

However, crossdev, which is used by setup_board to build cross-compilers,
 behaves incorrectly when the --usepkg flag is used. This is due to the fact
that the cross-compilers have interdependencies that are not reflected in
their version strings. E.g., if gcc is updated, you may need to also
recompile glibc, even if the version number of glibc has not changed. emerge
doesn't support that kind of magic out of the box, so it basically just does
the wrong thing with --usepkg. To work around this, the setup_board script
either uses the --usepkgonly flag or does not use packages at all when
emerging the toolchain.

Cheers,

David

[scottz, 2011-01-04 21:43:57]

This change LGTM to me.

I recall some talk about the setup_board issue, do we already have a bug running
on that? If I recall there was talk by Raymes about how we could address this.

-Scott


On 2011/01/04 21:35:23, davidjames wrote:
> On Tue, Jan 4, 2011 at 12:36 PM, <mailto:scottz@chromium.org> wrote:
> 
> > So if we pass --usepkg to the bot and it fails to get a package it will
> > continue
> > to build from source? If that is the case that works for me :)
> 
> 
> Yup, that's always worked.
> 
> However, crossdev, which is used by setup_board to build cross-compilers,
>  behaves incorrectly when the --usepkg flag is used. This is due to the fact
> that the cross-compilers have interdependencies that are not reflected in
> their version strings. E.g., if gcc is updated, you may need to also
> recompile glibc, even if the version number of glibc has not changed. emerge
> doesn't support that kind of magic out of the box, so it basically just does
> the wrong thing with --usepkg. To work around this, the setup_board script
> either uses the --usepkgonly flag or does not use packages at all when
> emerging the toolchain.
> 
> Cheers,
> 
> David

[Nick Sanders, 2011-01-04 22:29:07]

lgtm