crosutils: Improve dd to usb in image_to_usb.sh

crosutils: Improve dd in image_to_usb.sh

1. add oflags=sync to allow interrupting dd without hanging system I/O for a long time
   (since bs=4M, the speed is almost the same)
2. use pv to provide progress report if available.
   (add pv into host-depends in http://codereview.chromium.org/3608004)

BUG=none
TEST=manually: 
1. install pv in chroot, then execute image_to_usb.sh --to /dev/sdX and verified there's progress bar
2. ctrl-c to break the dd process and verified it can stop immediately
3. remove pv from chroot, then execute image_to_usb.sh and verify that it still works


Change-Id: I62fc373a4feee6d7e61897325d9e1e6d84a74d63

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=4732d5d

[Kenneth Waters, 2010-10-01 17:16:24]

Add either "crosutils:" or "scripts:" to the front of the first line of the
change description.  For the TEST= line describe what you did to manually test
this change (i.e. "used image_to_usb.sh on a x86-generic image" or whatever you
actually did to test).

http://codereview.chromium.org/3581007/diff/1/2
File image_to_usb.sh (right):

http://codereview.chromium.org/3581007/diff/1/2#newcode239
image_to_usb.sh:239: if type pv >/dev/null 2>&1; then
Remove this conditional, keep only the pv branch, check this in after the other
change.

[sosa, 2010-10-01 17:22:02]

Also if you make this change you need to add 

assert_inside_chroot to the top since pv will not be available outside and
communicate this change to this script to the dev list.

[Kenneth Waters, 2010-10-01 17:25:26]

> Also if you make this change you need to add

I forgot that this runs outside the chroot for many developers.  It's probably
better to leave the conditional then, to not force it to run inside the chroot.

Is that OK with you Chris?

[Hung-Te, 2010-10-01 17:34:13]

The description title and TEST line is fixed. Thanks for the suggestion

[sosa, 2010-10-01 17:35:36]

Yup that sounds good.  Note if you want to redirect stdout and stderr
to one place you can use the shorthand &> /dev/null rather than
redirect stderr to stdout then to dev/null.

On Fri, Oct 1, 2010 at 10:25 AM,  <kwaters@chromium.org> wrote:
>> Also if you make this change you need to add
>
> I forgot that this runs outside the chroot for many developers.  It's
> probably
> better to leave the conditional then, to not force it to run inside the
> chroot.
>
> Is that OK with you Chris?
>
> http://codereview.chromium.org/3581007/show
>

[Kenneth Waters, 2010-10-01 17:36:44]

> Note if you want to redirect stdout and stderr
> to one place you can use the shorthand &> /dev/null rather than
> redirect stderr to stdout then to dev/null.

That's not POSIX and will break on dash, I'd like to discourage it.

[sosa, 2010-10-01 17:39:03]

I didn't know that.  Thanks for the info!

On Fri, Oct 1, 2010 at 10:36 AM,  <kwaters@chromium.org> wrote:
>> Note if you want to redirect stdout and stderr
>> to one place you can use the shorthand &> /dev/null rather than
>> redirect stderr to stdout then to dev/null.
>
> That's not POSIX and will break on dash, I'd like to discourage it.
>
> http://codereview.chromium.org/3581007/show
>

[Kenneth Waters, 2010-10-01 17:39:23]

lgtm.

[kliegs, 2010-10-01 17:43:38]

I've seen other CL's using this format (even after being mentioned that its
not POSIX).  Is this something we should be strictly enforcing in all CLs?
Or only for scripts on either target or host?

On Fri, Oct 1, 2010 at 1:36 PM, <kwaters@chromium.org> wrote:

>  Note if you want to redirect stdout and stderr
>> to one place you can use the shorthand &> /dev/null rather than
>> redirect stderr to stdout then to dev/null.
>>
>
> That's not POSIX and will break on dash, I'd like to discourage it.
>
>
> http://codereview.chromium.org/3581007/show
>

[kliegs, 2010-10-01 17:44:51]

Can image_to_usb run inside the chroot now? Last I heard it couldn't and we
were told to explicitly run from outside?

On Fri, Oct 1, 2010 at 1:25 PM, <kwaters@chromium.org> wrote:

> Also if you make this change you need to add
>>
>
> I forgot that this runs outside the chroot for many developers.  It's
> probably
> better to leave the conditional then, to not force it to run inside the
> chroot.
>
> Is that OK with you Chris?
>
>
> http://codereview.chromium.org/3581007/show
>

[Kenneth Waters, 2010-10-01 17:46:08]

> Can image_to_usb run inside the chroot now? Last I heard it couldn't and we
> were told to explicitly run from outside?

You have been able to for quite a while (I think robotboy@ fixed it).  Some of
the command line options require being inside the chroot.

[Kenneth Waters, 2010-10-01 17:52:51]

> I've seen other CL's using this format (even after being mentioned that its
> not POSIX).  Is this something we should be strictly enforcing in all CLs?
> Or only for scripts on either target or host?

I have mixed feelings about this.  Some bashisms are extremly helpful.

In ebuilds, I think we should use whatever bashisms we like.

However, for shell scripts I'd like to see POSIX shell compliance.  It's too
easy for code to migrate from the host and target side.  Those without lots of
Unix experience are unlikely to know the subtitles between the two and put
non-working code on the target.  This is exasperated by the fact that most/all
of our target shell is untested.

Observe, http://codereview.chromium.org/3432020 and
http://codereview.chromium.org/3466017 for an example of this biting us.

[kliegs, 2010-10-01 17:57:46]

Yah - I recall you catching me on that CL.  I ended up passing your comments
on to another CL but it got overlooked and I wasn't positive on the correct
procedure for host vs. target to push on it so was hoping to head towards
some consensus here.  I think it'd be good to have a single method for doing
this.  Scripts can easily migrate between host and target as you said
(explicitly by common include files or implicitly with cut and paste) so
making sure the code runs on both is a good thing.


On Fri, Oct 1, 2010 at 1:52 PM, <kwaters@chromium.org> wrote:

>  I've seen other CL's using this format (even after being mentioned that
>> its
>> not POSIX).  Is this something we should be strictly enforcing in all CLs?
>> Or only for scripts on either target or host?
>>
>
> I have mixed feelings about this.  Some bashisms are extremly helpful.
>
> In ebuilds, I think we should use whatever bashisms we like.
>
> However, for shell scripts I'd like to see POSIX shell compliance.  It's
> too
> easy for code to migrate from the host and target side.  Those without lots
> of
> Unix experience are unlikely to know the subtitles between the two and put
> non-working code on the target.  This is exasperated by the fact that
> most/all
> of our target shell is untested.
>
> Observe, http://codereview.chromium.org/3432020 and
> http://codereview.chromium.org/3466017 for an example of this biting us.
>
>
> http://codereview.chromium.org/3581007/show
>

[Kenneth Waters, 2010-10-01 18:02:33]

On 2010/10/01 17:57:46, kliegs wrote:
> Yah - I recall you catching me on that CL.  I ended up passing your comments
> on to another CL but it got overlooked and I wasn't positive on the correct
> procedure for host vs. target to push on it so was hoping to head towards
> some consensus here.  I think it'd be good to have a single method for doing
> this.  Scripts can easily migrate between host and target as you said
> (explicitly by common include files or implicitly with cut and paste) so
> making sure the code runs on both is a good thing.

There is no consistent style for our shell scripts.  Every few days I consider
publishing one (there is an internal one to start from), and then auditing all
of our shell.

I wish <<-EOF wasn't tied to hard tabs.

[sosa, 2010-10-01 18:05:55]

I would really like to see this too.  I have a P1 assigned to me for
doing this but haven't had any time to get to it.  Right now our style
in these scripts is all over the place.

On Fri, Oct 1, 2010 at 11:02 AM,  <kwaters@chromium.org> wrote:
> On 2010/10/01 17:57:46, kliegs wrote:
>>
>> Yah - I recall you catching me on that CL.  I ended up passing your
>> comments
>> on to another CL but it got overlooked and I wasn't positive on the
>> correct
>> procedure for host vs. target to push on it so was hoping to head towards
>> some consensus here.  I think it'd be good to have a single method for
>> doing
>> this.  Scripts can easily migrate between host and target as you said
>> (explicitly by common include files or implicitly with cut and paste) so
>> making sure the code runs on both is a good thing.
>
> There is no consistent style for our shell scripts.  Every few days I
> consider
> publishing one (there is an internal one to start from), and then auditing
> all
> of our shell.
>
> I wish <<-EOF wasn't tied to hard tabs.
>
> http://codereview.chromium.org/3581007/show
>

[kliegs, 2010-10-01 18:12:04]

I definitely support a style guide that at minimum is enforced on any future
CL's even if we don't have the time to back port it.  It will keep us
cleaner moving forward.

Had a tough time tracing a file which didn't indent functions the other day
so any steps towards cleaner shell scripts would be great.

Just because they're scripts doesn't mean they're not code!

-Jon

On Fri, Oct 1, 2010 at 2:05 PM, Chris Sosa <sosa@chromium.org> wrote:

> I would really like to see this too.  I have a P1 assigned to me for
> doing this but haven't had any time to get to it.  Right now our style
> in these scripts is all over the place.
>
> On Fri, Oct 1, 2010 at 11:02 AM,  <kwaters@chromium.org> wrote:
> > On 2010/10/01 17:57:46, kliegs wrote:
> >>
> >> Yah - I recall you catching me on that CL.  I ended up passing your
> >> comments
> >> on to another CL but it got overlooked and I wasn't positive on the
> >> correct
> >> procedure for host vs. target to push on it so was hoping to head
> towards
> >> some consensus here.  I think it'd be good to have a single method for
> >> doing
> >> this.  Scripts can easily migrate between host and target as you said
> >> (explicitly by common include files or implicitly with cut and paste) so
> >> making sure the code runs on both is a good thing.
> >
> > There is no consistent style for our shell scripts.  Every few days I
> > consider
> > publishing one (there is an internal one to start from), and then
> auditing
> > all
> > of our shell.
> >
> > I wish <<-EOF wasn't tied to hard tabs.
> >
> > http://codereview.chromium.org/3581007/show
> >
>