kernel, chromeos-initramfs: overhaul for recovery support

kernel, chromeos-initramfs: overhaul for recovery support

Add support recovery, developer recovery, and dev script recovery and
vfat and TPM in-kernel support. Adds tpmc and all the chromeos-recovery
copies.

TEST=With other changes:
     - Built factory installer and ensured it behaved normally
     - Built an image and mod'd for recovery and used in dev mode and normal mode
     - Installed normal mode on legacy firmware.
     - Built new recoery kernel and replaced recovery kernel on an older images
       - ensured it only works in normal mode
     - Tested dev mode to ensure keys exist on the drive and it doesn't delay
     - Tested dev mode to ensure keys exist on the drive and it does delay
     - Tested dev script in dev mode and not
     - Tested keyblocks that match but invalid signatures on the actual kernels (dev shim and dev install)
     - need to test with a /signed/ image on cros_fw
BUG=chromium-os:7451

[Nick Sanders, 2010-10-27 00:35:40]

It looks like this may have some problems for factory install.

http://codereview.chromium.org/4120005/diff/1/3
File chromeos-base/chromeos-initramfs/files/init (right):

http://codereview.chromium.org/4120005/diff/1/3#newcode118
chromeos-base/chromeos-initramfs/files/init:118: # Correctness wins over speed
for this.
We're going to be reading the whole thing anyway, while it is verified. Why do
want to do this? It's going to cost another 30 seconds or so in a manual section
of the factory line.

http://codereview.chromium.org/4120005/diff/1/3#newcode248
chromeos-base/chromeos-initramfs/files/init:248: busybox tr -s '[0-9]' '0' | \
no need for "busybox". busyshell auto finds it's supported programs

http://codereview.chromium.org/4120005/diff/1/3#newcode413
chromeos-base/chromeos-initramfs/files/init:413: CHSW=$(cat
/sys/devices/platform/chromeos_acpi/CHSW)
This needs to return "developer mode" on H2O boot as well

http://codereview.chromium.org/4120005/diff/1/3#newcode450
chromeos-base/chromeos-initramfs/files/init:450: log "Press Ctrl+Alt+F1 - F3 for
more detailed information."
Can you confirm that you've tested this and it works?

http://codereview.chromium.org/4120005/diff/1/3#newcode503
chromeos-base/chromeos-initramfs/files/init:503: exec &> /dev/tty3
Can you confirm that tty3 switching is possible here?

[Will Drewry, 2010-10-27 00:51:01]

BTW, is factory install working again? I'm seeing trouble at switch_root which
I'm beginning to diagnose, but I wanted to check in before I sink time into it.

Thanks!  I'll see if I can rework the dm check to avoid factory entirely.

http://codereview.chromium.org/4120005/diff/1/3
File chromeos-base/chromeos-initramfs/files/init (right):

http://codereview.chromium.org/4120005/diff/1/3#newcode118
chromeos-base/chromeos-initramfs/files/init:118: # Correctness wins over speed
for this.
On 2010/10/27 00:35:40, Nick Sanders wrote:
> We're going to be reading the whole thing anyway, while it is verified. Why do
> want to do this? It's going to cost another 30 seconds or so in a manual
section
> of the factory line.

I'll time the difference.  This will prepopulate the I/O cache and pre-validate
the image so it shouldn't be a huge additional hit.

If not, I can move full validation after the factory install check it just makes
the logic a bit ugly. :)

http://codereview.chromium.org/4120005/diff/1/3#newcode248
chromeos-base/chromeos-initramfs/files/init:248: busybox tr -s '[0-9]' '0' | \
On 2010/10/27 00:35:40, Nick Sanders wrote:
> no need for "busybox". busyshell auto finds it's supported programs

Duh - thanks. will fix

http://codereview.chromium.org/4120005/diff/1/3#newcode413
chromeos-base/chromeos-initramfs/files/init:413: CHSW=$(cat
/sys/devices/platform/chromeos_acpi/CHSW)
On 2010/10/27 00:35:40, Nick Sanders wrote:
> This needs to return "developer mode" on H2O boot as well

Not a problem. Done.

http://codereview.chromium.org/4120005/diff/1/3#newcode450
chromeos-base/chromeos-initramfs/files/init:450: log "Press Ctrl+Alt+F1 - F3 for
more detailed information."
On 2010/10/27 00:35:40, Nick Sanders wrote:
> Can you confirm that you've tested this and it works?

Yup - works like a charm!

http://codereview.chromium.org/4120005/diff/1/3#newcode503
chromeos-base/chromeos-initramfs/files/init:503: exec &> /dev/tty3
On 2010/10/27 00:35:40, Nick Sanders wrote:
> Can you confirm that tty3 switching is possible here?

Yeah - I had no problems.  However, once I get everything finalized, I'd like to
have some help making sure my tests match reality.

[Will Drewry, 2010-10-28 03:13:26]

Huge overhaul to address concerns and to save logs for the non-factory case.  I
need to rerun the factory installer one last time, but everything was working on
last test :)

PTAL! Thanks!

[Will Drewry, 2010-10-29 02:35:52]

All test cases below barring Patchset 15 & 16 which just updated a comment and
fixed the umount in touch_developer_mode:

- factory with matched root
  - works in dev (correct)
  - kern doesnt validate by default in normal but that may be andretti.
(correct)

- factory with mismatched root
  - official mode: doesn't boot (correct)
  - developer mode; doesn't boot (correct)

- full image --nosync_keys with matching kern
  - official: works (correct)
  - developer: works (correct)
  - legacy: works (correct)
    
- full image --nosync_keys with mismatched kern
  - official: fails (correct)
  - developer: works but key mismatch triggers 5 min delay (correct)
  - legacy: works 
  
- full image with --sync_keys
  - official: works normally (correct)
  - developer: works normally but didn't touch the .developer_mode file (ok)
  - legacy: works and re-mkfs's stateful (correctL side effect of dev detection)

- full image with --sync_keys with mismatched kerns
  - developer: works and keys match so no extra delay. (correct)
  - official: fails (correct)
  
- shim with mismatch keys:
  - in developer mode: delay (correct)
  - in official mode: fails (correct) 
  
- shim in with matching keys in an on-disk kernel:
  - in developer mode: UNTESTED
  - in official mode: UNTESTED 


(Note, dev_install builds are can be deprecated now)

[Will Drewry, 2010-10-29 02:44:45]

Patchset 16 fixed the ful+sync+dev touch file case it seems.  The failed umount
kept the dot file from being sync'd to disk.

Anyway, ptal. I'd love to land these before the week is over if you think it is
sane.  :)

Thanks!

[Will Drewry, 2010-10-29 15:44:34]

Additional test cases:

full image with a matching keyblock on the HDD but an invalid signature:
- official: fails because it depends on kern_b_hash (correct)
- developer: finds matching key then fails validation (correct)

- shim in with matching keys in an on-disk kernel:
  - in developer mode:  works with no delay (correct)
  - in official mode:  works with no delay [crosbug/8390] (correct)
(generated with last kernel and:
  ./make_developer_script_runner.sh  --kernel_image recovery_vmlinuz.image 
--developer_script hello_world.sh --developer_keyblock
/usr/share/vboot/devkeys/kernel.keyblock  --developer_private_key
/usr/share/vboot/devkeys/kernel_data_key.vbprivk  --image runner.bin
)

On 2010/10/29 02:44:45, Will Drewry wrote:
> Patchset 16 fixed the ful+sync+dev touch file case it seems.  The failed
umount
> kept the dot file from being sync'd to disk.
> 
> Anyway, ptal. I'd love to land these before the week is over if you think it
is
> sane.  :)
> 
> Thanks!

[Will Drewry, 2010-10-29 16:16:16]

Patchset 16 behavior tested.  This change and the script changes should be fully
functional for this mstone with clean up and enhancement bugs for the next.


On 2010/10/29 15:44:34, Will Drewry wrote:
> Additional test cases:
> 
> full image with a matching keyblock on the HDD but an invalid signature:
> - official: fails because it depends on kern_b_hash (correct)
> - developer: finds matching key then fails validation (correct)
> 
> - shim in with matching keys in an on-disk kernel:
>   - in developer mode:  works with no delay (correct)
>   - in official mode:  works with no delay [crosbug/8390] (correct)
> (generated with last kernel and:
>   ./make_developer_script_runner.sh  --kernel_image recovery_vmlinuz.image 
> --developer_script hello_world.sh --developer_keyblock
> /usr/share/vboot/devkeys/kernel.keyblock  --developer_private_key
> /usr/share/vboot/devkeys/kernel_data_key.vbprivk  --image runner.bin
> )
> 
> On 2010/10/29 02:44:45, Will Drewry wrote:
> > Patchset 16 fixed the ful+sync+dev touch file case it seems.  The failed
> umount
> > kept the dot file from being sync'd to disk.
> > 
> > Anyway, ptal. I'd love to land these before the week is over if you think it
> is
> > sane.  :)
> > 
> > Thanks!

[Nick Sanders, 2010-10-29 21:30:39]

lgtm