Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(10)

Issue 6058006: Preserves file system metadata between new build and latest shipping (Closed)

Created:
10 years ago by thieule
Modified:
9 years, 4 months ago
Reviewers:
petkov, gauravsh, adlr
CC:
chromium-os-reviews_chromium.org, Mandeep Singh Baines, anush, sosa
Visibility:
Public.

Description

Preserves file system metadata between new build and latest shipping image. This script preserves the root file system metadata as much as possible between the specified image and the latest shipping image. It preserves the metadata by ensuring that the files reuse the same inodes and that they are located at the same physical location on-disk. This leads to smaller auto-update delta payload and less disk reshuffling, extending the life of the SSD. It is called before the image is signed during the stamping process. Currently, this only supports x86-mario. BUG=chromium-os:10188 TEST=Build image, boot image, auto-update to new image, run suite_Smoke Change-Id: Ib05c59707174bf216beeccbd57f0a1f0c317f7c7

Patch Set 1 #

Total comments: 76
Unified diffs Side-by-side diffs Delta from patch set Stats (+264 lines, -0 lines) Patch
A align_rootfs View 1 chunk +264 lines, -0 lines 76 comments Download

Messages

Total messages: 9 (0 generated)
thieule
PTAL
10 years ago (2010-12-22 22:04:56 UTC) #1
petkov
A few quick comments. Also, I think this script should live under bin/cros_align_rootfs. http://codereview.chromium.org/6058006/diff/1/align_rootfs File ...
10 years ago (2010-12-22 22:45:03 UTC) #2
petkov
Btw, I'm still concerned about how this integrates into the release flow... Maybe it will ...
10 years ago (2010-12-22 22:46:51 UTC) #3
adlr
http://codereview.chromium.org/6058006/diff/1/align_rootfs File align_rootfs (right): http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode16 align_rootfs:16: # cros_make_image_bootable which requires this. i suspect this might ...
10 years ago (2010-12-22 22:49:50 UTC) #4
gauravsh
http://codereview.chromium.org/6058006/diff/1/align_rootfs File align_rootfs (right): http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode3 align_rootfs:3: # Copyright (c) 2009 The Chromium OS Authors. All ...
10 years ago (2010-12-22 22:55:26 UTC) #5
gauravsh
Forgot to mention in my comments, after discussing with Thieule, this script should live under ...
10 years ago (2010-12-22 22:57:20 UTC) #6
petkov
On 2010/12/22 22:57:20, gauravsh wrote: > Forgot to mention in my comments, after discussing with ...
10 years ago (2010-12-22 23:00:15 UTC) #7
thieule
I think we should considering the template image may or may not have its free ...
10 years ago (2010-12-22 23:04:29 UTC) #8
thieule
10 years ago (2010-12-23 19:26:27 UTC) #9
I'm about to send out a new CL with the code review feedbacks addressed.  The
script has been moved to
src/platform/vboot_reference/scripts/image_signing/align_rootfs.

http://codereview.chromium.org/6079004

PTAL

http://codereview.chromium.org/6058006/diff/1/align_rootfs
File align_rootfs (right):

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode3
align_rootfs:3: # Copyright (c) 2009 The Chromium OS Authors. All rights
reserved.
On 2010/12/22 22:45:03, petkov wrote:
> 2010

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode3
align_rootfs:3: # Copyright (c) 2009 The Chromium OS Authors. All rights
reserved.
On 2010/12/22 22:55:27, gauravsh wrote:
> 2010 (or rather 2011 if you end up submitting next year :) )

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode16
align_rootfs:16: # cros_make_image_bootable which requires this.
On 2010/12/22 22:49:50, adlr wrote:
> i suspect this might be a problem for the build bots. i would put in a TODO to
> fix this and maybe look into getting cros_make_image_bootable working outside
> the chroot

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode28
align_rootfs:28: # Parse command line.
On 2010/12/22 22:45:03, petkov wrote:
> you should do the flag processing inside main

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode38
align_rootfs:38: error "--board is required."
On 2010/12/22 22:45:03, petkov wrote:
> die instead of error and no exit

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode43
align_rootfs:43: error "--image is required."
On 2010/12/22 22:45:03, petkov wrote:
> die
> 
> also, did you mean to say that the image file does not exist?
> 

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode43
align_rootfs:43: error "--image is required."
On 2010/12/22 22:55:27, gauravsh wrote:
> the error displayed here is not accurate - since -f checks for file existence.
> suggest you move the -z check on flags to the check earlier, and check for
file
> existence here.

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode56
align_rootfs:56: add_cleanup_action() {
On 2010/12/22 22:55:27, gauravsh wrote:
> this trap setup and cleanup will conflict with
> vboot_reference/scripts/image_signing/common.sh, maybe it can be made a part
of
> common.sh and the cleanup function there?

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode79
align_rootfs:79: if [ ${FLAGS_board} == "x86-mario" ] ; then
On 2010/12/22 22:45:03, petkov wrote:
> this is a bit ugly. maybe it's better to remove the board flag and make the
> hardware class a required argument.

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode79
align_rootfs:79: if [ ${FLAGS_board} == "x86-mario" ] ; then
On 2010/12/22 22:55:27, gauravsh wrote:
> use '=' here

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode80
align_rootfs:80: HARDWARE_CLASS="IEC MARIO PONY 6101"
On 2010/12/22 22:55:27, gauravsh wrote:
> this could be better called HARDWARE_ID.

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode80
align_rootfs:80: HARDWARE_CLASS="IEC MARIO PONY 6101"
On 2010/12/22 22:55:27, gauravsh wrote:
> You might want to parameterize this now - as an optional command line
argument.

Making this a required command line arg per petkov suggestion.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode85
align_rootfs:85: error "Board ${FLAGS_board} not supported"
On 2010/12/22 22:45:03, petkov wrote:
> die

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode90
align_rootfs:90: get_latest_shipping_version() {
On 2010/12/22 22:55:27, gauravsh wrote:
> comment

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode91
align_rootfs:91: info "Pinging Omaha for the latest shipping version."
On 2010/12/22 22:45:03, petkov wrote:
> it might be easier/better to go through the recovery flow instead of omaha --
> the recovery URL should point to the latest recovery image which is just a
> gzipped image file. then you wouldn't need all the code that looks for the
> filename, download URL, etc.
> 

Is the recovery image always the latest shipping image?

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode101
align_rootfs:101: <o:app appid="{87efface-864d-49a5-9bb3-4b050a7c227a}"
On 2010/12/22 22:45:03, petkov wrote:
> you're hardcoding an AppId that corresponds to Mario hardware IDs. maybe make
it
> a flag with the right default value.

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode102
align_rootfs:102: version="ForcedUpdate" lang="en-US" track="dev-channel"
On 2010/12/22 22:45:03, petkov wrote:
> if you use version="0.0.0.0" you should get the latest version. I'm not sure
> what ForcedUpdate would give you from the live Omaha server.

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode104
align_rootfs:104: <o:ping a="-1" r="-1"></o:ping>
On 2010/12/22 22:45:03, petkov wrote:
> remove this ping otherwise you'd be counted as an active user every time you
run
> the script

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode110
align_rootfs:110: LATEST_SHIPPING_VERSION=$(wget -q --header="Content-Type:
text/xml" \
On 2010/12/22 22:45:03, petkov wrote:
> why is this a global? shouldn't you return this as a value from the function
> instead?

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode115
align_rootfs:115: download_image() {
On 2010/12/22 22:55:27, gauravsh wrote:
> add comment

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode157
align_rootfs:157: copy_root_fs() {
On 2010/12/22 22:55:27, gauravsh wrote:
> add comment about the arguments and what the function does

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode165
align_rootfs:165: ${SCRIPTS_DIR}/mount_gpt_image.sh -f "${image_dir}" -i
"${image_name}"
On 2010/12/22 22:55:27, gauravsh wrote:
> you can use mount_image_partition from
> vboot_reference/scripts/image_signing/common.sh here.
> 
> It does tracked mounts, so you won't have to worry about unmounting it
> explicitly (unless really needed).

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode173
align_rootfs:173: # Temporarily make immutable files on the dst rootfs mutable.
On 2010/12/22 22:45:03, petkov wrote:
> do we have such files? do we care about mutable/immutable given that rootfs is
> read-only?

There is one immutable file /boot/extlinux.sys that we have to temporarily make
mutable so we can overwrite it.  Even though the rootfs is mounted readonly, I
put the immutable flag back just to be clean.  I can remove the code that +i the
file if you like.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode176
align_rootfs:176: sed 's/\//\\\//g')
On 2010/12/22 22:45:03, petkov wrote:
> use sed s,,, rather than sed s/// for readability?

Ack, missed this on the original code review.  Sosa mentioned it and it
completely slipped my mind.

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode178
align_rootfs:178: eval $enum_files_cmd | sed "s/${dst_root_fs_dir_escaped}//" |
\
On 2010/12/22 22:49:50, adlr wrote:
> comment on what this command does?

I've removed this code.  It was a leftover from some older code where I was
implementing my own rsync instead of using rsync.  Should be easier to
understand now.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode179
align_rootfs:179: while read -r FILE ; do
On 2010/12/22 22:45:03, petkov wrote:
> while is FILE all caps?

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode195
align_rootfs:195: for FILE in ${immutable_files[*]} ; do
On 2010/12/22 22:45:03, petkov wrote:
> why is FILE all caps?

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode199
align_rootfs:199: ${SCRIPTS_DIR}/mount_gpt_image.sh -u
On 2010/12/22 22:45:03, petkov wrote:
> you'd try to unmount this twice, I think -- once here and once on EXIT? you
> don't need this unmount I guess.

I need these unmounts because I'll be replacing the rootfs in the image.  I've
added support for performing the cleanup action and removing from the list.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode203
align_rootfs:203: replace_root_fs() {
On 2010/12/22 22:55:27, gauravsh wrote:
> add comment about the arguments and what the function does

I've removed this function and using replace_image_partition() from common.sh
instead.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode208
align_rootfs:208: dd if="${root_fs}" of="${image}" bs=512 conv=notrunc \
On 2010/12/22 22:49:50, adlr wrote:
> maybe do bs=$((2 * 1024 * 1024)) ? take a look at fast_dd in
> platform/installer/chromeos-install . might be good to move that to
> chromeos-common

Function removed.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode213
align_rootfs:213: trap cleanup EXIT
On 2010/12/22 22:49:50, adlr wrote:
> many script trap on more than just exit, (i think) TERM and INT as well. Maybe
> add those?

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode213
align_rootfs:213: trap cleanup EXIT
On 2010/12/22 22:55:27, gauravsh wrote:
> if you use vboot_reference's common.sh, this will conflict with its own
cleanup
> handler.

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode221
align_rootfs:221: add_cleanup_action "sudo rm -f \"${latest_shipping_image}\""
On 2010/12/22 22:45:03, petkov wrote:
> why do this special add_cleanup_action rather than just:
> 
> trap "sudo rm -f \"${latest_shipping_image}\"" EXIT
> 
> then you don't need to run cleanup for EXIT and you don't need
> add_cleanup_action

I have other clean up actions that may need to be executed other than just this
one.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode231
align_rootfs:231: info "The latest shipping rootfs and the new rootfs are not
the same size."
On 2010/12/22 22:45:03, petkov wrote:
> warn instead?

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode240
align_rootfs:240: extract_root_fs "${latest_shipping_image}" "${temp_root_fs}"
On 2010/12/22 22:55:27, gauravsh wrote:
> extract_partition can be used here

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode243
align_rootfs:243: # Copy the new rootfs over template rootfs to preserve as much
of the
On 2010/12/22 22:49:50, adlr wrote:
> this is a super high level comment that could go at the top of the file. here
> you could just say something like "perform the actual copy"

Done.

http://codereview.chromium.org/6058006/diff/1/align_rootfs#newcode264
align_rootfs:264: main
On 2010/12/22 22:45:03, petkov wrote:
> main "$*"

Done.

Powered by Google App Engine
This is Rietveld 408576698