Various image build improvements for ARM

Various image build improvements for ARM

Add an --arch option to build_image.sh, passed to install_packages.sh.
Use it to set APT::Architecture.

Add --mirror2/--suite2 options to build_image.sh, and --server2/--suite2
options to install_packages.sh (differing option names to match options
that were there already).  This allows some of the packages to be pulled
from ports.ubuntu.com rather than from local_packages, like this:

  ./build_image.sh --arch armel --mirror2 http://ports.ubuntu.com/ubuntu-ports --suite2 karmic

Copy only .debs from local_packages, not whatever other junk might be
lying around there.

Use copy: rather than file: for local_packages, so that the files will
be copied into the apt cache directory for the benefit of
dpkg_no_scripts.sh.

[tedbo, 2010-01-21 19:17:14]

Thanks for this change. Rather than adding a mirror2, etc, I was planning on
having mirror take a comma separated list to just blast into the sources.list
file like:

--mirror="http://foo.com/ubuntu karmic main universe,http://bar.com/blah
chromium main universe"

For now I think it is fine as-is; just a heads up that I may convert it to this.

http://codereview.chromium.org/554031/diff/2001/3002
File src/scripts/install_packages.sh (right):

http://codereview.chromium.org/554031/diff/2001/3002#newcode96
src/scripts/install_packages.sh:96: deb copy:"$SETUP_DIR" local_packages/
Why do you want to change this to copy into the apt cache? I had this as file
because I wanted to keep the build output packages separate from the ones pulled
down from other archives in terms of caching. In addition, I worried that we
might pull an older version of a built package from the apt cache if we didn't
up the version numbers across builds. Probably that worry is unfounded if the
apt-cache uses checksums properly?

[cjwatson, 2010-01-22 11:27:34]

Noted on mirror2.  A cleaner alternative might be to permit multiple --mirror
options; I haven't looked at shflags enough to know whether it can support that.

http://codereview.chromium.org/554031/diff/2001/3002
File src/scripts/install_packages.sh (right):

http://codereview.chromium.org/554031/diff/2001/3002#newcode96
src/scripts/install_packages.sh:96: deb copy:"$SETUP_DIR" local_packages/
On 2010/01/21 19:17:14, tedbo wrote:
> Why do you want to change this to copy into the apt cache? I had this as file
> because I wanted to keep the build output packages separate from the ones
pulled
> down from other archives in terms of caching.

As discussed yesterday, it's because otherwise dpkg_no_scripts.sh fails, I think
while unpacking the small initial set; I don't have the terminal scrollback any
more but I believe it was libacl1.

> In addition, I worried that we
> might pull an older version of a built package from the apt cache if we didn't
> up the version numbers across builds. Probably that worry is unfounded if the
> apt-cache uses checksums properly?

I'm certain that apt checks the checksum on each .deb at some point.  I'm almost
certain that this is done at the download cache level; even if it isn't, we will
get failures rather than corruption since it definitely checks that the checksum
of the thing it downloaded is what it was expecting.  But I'm pretty sure it
will just work.

Do you want me to drop this part of the change for now if you're still unsure
about it?  I'd like to get the rest in.

[tedbo, 2010-01-22 17:39:42]

LGTM

Thanks!

Unfortunately, I don't think that shflags supports "--mirror=aaa --mirror=bbb".

http://codereview.chromium.org/554031/diff/2001/3002
File src/scripts/install_packages.sh (right):

http://codereview.chromium.org/554031/diff/2001/3002#newcode96
src/scripts/install_packages.sh:96: deb copy:"$SETUP_DIR" local_packages/
No, please leave it in. I have it myself in my local tree.

On 2010/01/22 11:27:34, cjwatson wrote:
> On 2010/01/21 19:17:14, tedbo wrote:
> > Why do you want to change this to copy into the apt cache? I had this as
file
> > because I wanted to keep the build output packages separate from the ones
> pulled
> > down from other archives in terms of caching.
> 
> As discussed yesterday, it's because otherwise dpkg_no_scripts.sh fails, I
think
> while unpacking the small initial set; I don't have the terminal scrollback
any
> more but I believe it was libacl1.
> 
> > In addition, I worried that we
> > might pull an older version of a built package from the apt cache if we
didn't
> > up the version numbers across builds. Probably that worry is unfounded if
the
> > apt-cache uses checksums properly?
> 
> I'm certain that apt checks the checksum on each .deb at some point.  I'm
almost
> certain that this is done at the download cache level; even if it isn't, we
will
> get failures rather than corruption since it definitely checks that the
checksum
> of the thing it downloaded is what it was expecting.  But I'm pretty sure it
> will just work.
> 
> Do you want me to drop this part of the change for now if you're still unsure
> about it?  I'd like to get the rest in.