Enable LZMA compression in bmpbklk_utility.

Enable LZMA compression in bmpbklk_utility.

LZMA has better compression ratio and is also supported in u-boot already.
ARM BIOS will use LZMA to compress BMP files.

BUG=chromium-os:11017
TEST=manual
$ make
$ make runbmptests

Change-Id: I6b791e3284b65eb3085b0de548bd241eab2ee598

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=ee2bc91

[Hung-Te, 2011-02-15 07:55:10]

Did you tested that u-boot kernel really boots and displayed such LZMA'ed
pictures correctly? Please add that into your TEST=, thanks

http://codereview.chromium.org/6523019/diff/1/utility/bmpblk_util.c
File utility/bmpblk_util.c (right):

http://codereview.chromium.org/6523019/diff/1/utility/bmpblk_util.c#newcode152
utility/bmpblk_util.c:152: ibuf = ((void *)img) + sizeof(ImageInfo);
We should not perform arithmetic operation to void* pointers because it does not
have size. (If this is C++, you'll get error)

Please use uint8_t* instead, or ibuf = (void*)(img + 1);

http://codereview.chromium.org/6523019/diff/1/utility/bmpblk_util.c#newcode155
utility/bmpblk_util.c:155: obuf = malloc(osize);
Why not lzma_stream_buffer_bound?

http://codereview.chromium.org/6523019/diff/1/utility/bmpblk_util.c#newcode165
utility/bmpblk_util.c:165: fprintf(stderr, "Unalbe to initialize auto decoder
(error: %d)!\n", result);
Unable, typo

[Bill Richardson, 2011-02-15 23:07:58]

LGTM, after responding to Hung-Te's comments.

[Bill Richardson, 2011-02-15 23:23:22]

But watch out for http://code.google.com/p/chromium-os/issues/detail?id=12158

[Tom Wai-Hong Tam, 2011-02-16 10:58:03]

I spent some time to try if u-boot can decode the lzma format. It is
complicated. Here is my investigation.

Before this change, I use the command line tool "lzma" to encode a BMP. u-boot
can decode it successfully.

So I took a look at the file content. The file headers are different. Found that
u-boot only supports a legacy file header, LZMA_Alone format, not the new .xz
format. Hopefully, liblzma can support it after I changed to use
lzma_alone_encoder instead of lzma_easy_encoder.

After changing this, u-boot still can't decode it. Because there is a field
"uncompressed size" in the file header and liblzma fills it "unknown". Also
liblzma use an end-of-stream marker at the end of file. I guess that the reason.

So I found an old thread talking about it:
http://old.nabble.com/Squashfs-4.1-creates-invalid-.lzma-streams-td30217833.html

I am trying to found any other good solution. If no, I will use LZMA SDK instead
of liblzma.

http://codereview.chromium.org/6523019/diff/1/utility/bmpblk_util.c
File utility/bmpblk_util.c (right):

http://codereview.chromium.org/6523019/diff/1/utility/bmpblk_util.c#newcode152
utility/bmpblk_util.c:152: ibuf = ((void *)img) + sizeof(ImageInfo);
On 2011/02/15 07:55:10, Hung-Te wrote:
> We should not perform arithmetic operation to void* pointers because it does
not
> have size. (If this is C++, you'll get error)
> 
> Please use uint8_t* instead, or ibuf = (void*)(img + 1);

Done.

http://codereview.chromium.org/6523019/diff/1/utility/bmpblk_util.c#newcode155
utility/bmpblk_util.c:155: obuf = malloc(osize);
Since the exact uncompressed size is filled in when compressing. No need to
guess an upper bound.

http://codereview.chromium.org/6523019/diff/1/utility/bmpblk_util.c#newcode165
utility/bmpblk_util.c:165: fprintf(stderr, "Unalbe to initialize auto decoder
(error: %d)!\n", result);
On 2011/02/15 07:55:10, Hung-Te wrote:
> Unable, typo

Done.

[Hung-Te, 2011-02-16 11:46:16]

On 2011/02/16 10:58:03, Tom Wai-Hong Tam wrote:
> u-boot only supports a legacy file header, LZMA_Alone format, 
> not the new .xz
   xz (an archive format) is not LZMA (the codec), just like "gz is not deflate"
- so I think what you've found is true.
   We should also update the compression names in header from COMPRESS_LZMA to
something like COMPRESS_LZMA1, COMPRESS_LZMA2, or COMPRESS_LZMA_XZ (depends on
which we decide at last).
   (Maybe should also change the EFI one tot COMPRESS_LZMA_EFI1, since EFI also
supports other codecs)

> utility/bmpblk_util.c:155: obuf = malloc(osize);
> Since the exact uncompressed size is filled in when compressing. No need to
> guess an upper bound.
   So the compressor will never create a blob which is larger than the source
(ex, if the source is one byte -- although it's unlikely to happen in our case.
but what if one day we generated a mini blob that is smaller than LZMA headers?)

[Bill Richardson, 2011-02-16 16:55:29]

UEFI supports only one compression scheme, known as the "UEFI Compression
Algorithm". It is not LZMA, but is "a combination of the LZ77 algorithm and
Huffman Coding." There are actually two variants, because the algorithm in the
early EFI spec is very slightly different from the algorithm in the latest UEFI
spec. The latest UEFI spec doesn't acknowledge the existence of the earlier
version, but our BIOS is based on an early EFI code base and only understands
that earlier version, which is why I refer to it as COMPRESS_EFIv1 instead of
just COMPRESS_UEFI.

If we want to be forward-compatible, we can add COMPRESS_UEFI to the list, but
let's decide that with a bug and not here.

[Tom Wai-Hong Tam, 2011-02-17 04:49:34]

It was my mistake. I didn't assign enough memory in u-boot and resulted failed.
I tried it again. Simply using lzma_alone_encoder works in u-boot.

>> utility/bmpblk_util.c:155: obuf = malloc(osize);
>> Since the exact uncompressed size is filled in when compressing. No need to
>> guess an upper bound.
>   So the compressor will never create a blob which is larger than the source
>(ex, if the source is one byte -- although it's unlikely to happen in our case.
>but what if one day we generated a mini blob that is smaller than LZMA
headers?)
It is the case of compressing. The compressed size may be larger than the
uncompressed size so it needs lzma_stream_buffer_bound to obtain an upper bound.
But it is not the case of decompressing.

[Hung-Te, 2011-02-17 04:54:14]

On 2011/02/17 04:49:34, Tom Wai-Hong Tam wrote:
> It is the case of compressing.
  Oops, you're right - I got confused by the LZMA API names.

  LGTM and please update how you tested it in the commit description.

[Hung-Te, 2011-02-17 05:00:03]

On 2011/02/16 16:55:29, Bill Richardson wrote:
> UEFI supports only one compression scheme, 
> known as the "UEFI Compression Algorithm". 
> It is not LZMA, but is "a combination of the LZ77 algorithm and
> Huffman Coding."

   That's true for the original EDK. However for H2C, I rememeber
   they defined special GUIDs and named as "LZMA", and also in
   the Makefiles they called a filter program with "LZMA" in its name.
   The source also has "LZMADecode" in CustomizedDecompress.