Fix python to build with newlib.

Fix python to build with newlib.

Work around various issues with building python with
newlib.  Many of these fixes should be upstreamed
to newlib itself ASAP.

BUG=None
TEST=./build_all.py python
R=mseaborn@chromium.org

Committed: https://code.google.com/p/naclports/source/detail?r=823

[Mark Seaborn, 2013-07-12 19:47:29]

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl-python.sh
File libraries/python/nacl-python.sh (right):

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl-python....
libraries/python/nacl-python.sh:29: EXTRA_CONFIGURE_ARGS+=" --disable-ipv6"
Can you comment why this is needed?  What fails without this?  Does this disable
all networking usage, or just IPv6 and not IPv4?

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl-python....
libraries/python/nacl-python.sh:34: export LIBS="-lc -lnosys"
"-lc" should be implicit.  Can you omit that and just use "-lnosys"?

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl.patch
File libraries/python/nacl.patch (right):

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl.patch#n...
libraries/python/nacl.patch:51: +extern int lstat(const char *, struct stat *);
This is fixed in
https://src.chromium.org/viewvc/native_client?view=rev&revision=11693.  Can you
remove this?

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl.patch#n...
libraries/python/nacl.patch:55: +int fsync(int fd) {
This is fixed in
https://src.chromium.org/viewvc/native_client?view=rev&revision=11694

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl.patch#n...
libraries/python/nacl.patch:129: +#if defined __native_client__ && !defined
__GLIBC__
Can you link to https://code.google.com/p/nativeclient/issues/detail?id=3551
here?

https://codereview.chromium.org/18112024/diff/1/libraries/python/pkg_info
File libraries/python/pkg_info (left):

https://codereview.chromium.org/18112024/diff/1/libraries/python/pkg_info#old...
libraries/python/pkg_info:4: LIBC=glibc
BTW, does this change also enable building with PNaCl, and does that work OK at
the moment?

[Sam Clegg, 2013-07-12 21:54:29]

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl-python.sh
File libraries/python/nacl-python.sh (right):

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl-python....
libraries/python/nacl-python.sh:29: EXTRA_CONFIGURE_ARGS+=" --disable-ipv6"
On 2013/07/12 19:47:29, Mark Seaborn wrote:
> Can you comment why this is needed?  What fails without this?  Does this
disable
> all networking usage, or just IPv6 and not IPv4?

I'm just moving this from above, but you are right that this deserves a comment.

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl-python....
libraries/python/nacl-python.sh:34: export LIBS="-lc -lnosys"
On 2013/07/12 19:47:29, Mark Seaborn wrote:
> "-lc" should be implicit.  Can you omit that and just use "-lnosys"?

I found that I needed to put -lnosys after -lc (which means I need to be
explicit like this).  The problem occurs when a function libc is the first users
of a symbol in nosys.  If nosys has already been passed over by the linker the
symbol will not be found.  

In this case the error I get without putting -lc first is:

{{{
/home/sbc/nacl_sdk/pepper_canary/toolchain/linux_x86_newlib/bin/../lib/gcc/x86_64-nacl/4.4.3/../../../../x86_64-nacl/lib/../lib64/libcrt_common.a(lib_a-execv.o):
In function `execv':
execv.c:(.text+0xd): undefined reference to `_execve'
}}}

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl.patch
File libraries/python/nacl.patch (right):

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl.patch#n...
libraries/python/nacl.patch:51: +extern int lstat(const char *, struct stat *);
On 2013/07/12 19:47:29, Mark Seaborn wrote:
> This is fixed in
> https://src.chromium.org/viewvc/native_client?view=rev&revision=11693.  Can
you
> remove this?


Added TODO to remove when this lands.

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl.patch#n...
libraries/python/nacl.patch:55: +int fsync(int fd) {
On 2013/07/12 19:47:29, Mark Seaborn wrote:
> This is fixed in
> https://src.chromium.org/viewvc/native_client?view=rev&revision=11694

Added TODO to remove when this lands.

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl.patch#n...
libraries/python/nacl.patch:129: +#if defined __native_client__ && !defined
__GLIBC__
On 2013/07/12 19:47:29, Mark Seaborn wrote:
> Can you link to https://code.google.com/p/nativeclient/issues/detail?id=3551
> here?

Done.

https://codereview.chromium.org/18112024/diff/1/libraries/python/pkg_info
File libraries/python/pkg_info (left):

https://codereview.chromium.org/18112024/diff/1/libraries/python/pkg_info#old...
libraries/python/pkg_info:4: LIBC=glibc
On 2013/07/12 19:47:29, Mark Seaborn wrote:
> BTW, does this change also enable building with PNaCl, and does that work OK
at
> the moment?

yes, this change would enable pnacl, and it currently fails in translate:

{{{
/home/sbc/nacl_sdk/pepper_canary/toolchain/linux_x86_pnacl/newlib/bin/pnacl-translate
-arch x86-32 python.nexe.stripped -o python.nexe.x86-32.nexe
pnacl-llc:
/mnt/data/b/build/slave/linux-pnacl-x86_64/build/native_client/pnacl/git/llvm/lib/Target/X86/MCTargetDesc/X86AsmBackend.cpp:107:
virtual void<unnamed>::X86AsmBackend::applyFixup(const llvm::MCFixup&, char*,
unsigned int, uint64_t) const: Assertion `isIntN(Size * 8 + 1, Value) && "Value
does not fit in the Fixup field"' failed.
0  libLLVM-3.3svn.so 0xf6f90b8f llvm::sys::PrintStackTrace(_IO_FILE*) + 47
Stack dump:
0.	Program arguments:
/home/sbc/nacl_sdk/pepper_canary/toolchain/linux_x86_pnacl/host_x86_32/bin/pnacl-llc
-mtriple=i686-none-nacl-gnu -filetype=obj -reduce-memory-footprint
-mcpu=pentium4 python.nexe.stripped -o
/home/sbc/dev/naclports/out/repository/Python-2.7.5/build-nacl-pnacl-newlib/python.nexe.x86-32.nexe---python.nexe.stripped---.o


}}}

[Mark Seaborn, 2013-07-16 15:40:12]

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl.patch
File libraries/python/nacl.patch (right):

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl.patch#n...
libraries/python/nacl.patch:51: +extern int lstat(const char *, struct stat *);
On 2013/07/12 21:54:29, Sam Clegg wrote:
> On 2013/07/12 19:47:29, Mark Seaborn wrote:
> > This is fixed in
> > https://src.chromium.org/viewvc/native_client?view=rev&revision=11693.  Can
> you
> > remove this?
> 
> 
> Added TODO to remove when this lands.

The change to fix this should have rolled into Chrome several days ago.  Would
you mind trying this with the current SDK?  You should be able to remove this
now.

https://codereview.chromium.org/18112024/diff/1/libraries/python/pkg_info
File libraries/python/pkg_info (left):

https://codereview.chromium.org/18112024/diff/1/libraries/python/pkg_info#old...
libraries/python/pkg_info:4: LIBC=glibc
On 2013/07/12 21:54:29, Sam Clegg wrote:
> On 2013/07/12 19:47:29, Mark Seaborn wrote:
> > BTW, does this change also enable building with PNaCl, and does that work
> > OK at the moment?
> 
> yes, this change would enable pnacl, and it currently fails in translate:
> 
> {{{
>
/home/sbc/nacl_sdk/pepper_canary/toolchain/linux_x86_pnacl/newlib/bin/pnacl-translate
> -arch x86-32 python.nexe.stripped -o python.nexe.x86-32.nexe
> pnacl-llc:
>
/mnt/data/b/build/slave/linux-pnacl-x86_64/build/native_client/pnacl/git/llvm/lib/Target/X86/MCTargetDesc/X86AsmBackend.cpp:107:
> virtual void<unnamed>::X86AsmBackend::applyFixup(const llvm::MCFixup&, char*,
> unsigned int, uint64_t) const: Assertion `isIntN(Size * 8 + 1, Value) &&
"Value
> does not fit in the Fixup field"' failed.

I've fixed that, BTW:
https://code.google.com/p/nativeclient/issues/detail?id=3548  The fix should be
in the SDK by now.

https://codereview.chromium.org/18112024/diff/7001/libraries/python/nacl-pyth...
File libraries/python/nacl-python.sh (right):

https://codereview.chromium.org/18112024/diff/7001/libraries/python/nacl-pyth...
libraries/python/nacl-python.sh:19: CFLAGS="" LDFLAGS="" LogExecute ../configure
Can you comment setting CFLAGS="" LDFLAGS="", please?

https://codereview.chromium.org/18112024/diff/7001/libraries/python/nacl-pyth...
libraries/python/nacl-python.sh:29: EXTRA_CONFIGURE_ARGS+=" --disable-ipv6"
Should this be
EXTRA_CONFIGURE_ARGS="--disable-ipv6"
for the first arg?

[Sam Clegg, 2013-07-16 16:19:16]

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl.patch
File libraries/python/nacl.patch (right):

https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl.patch#n...
libraries/python/nacl.patch:51: +extern int lstat(const char *, struct stat *);
On 2013/07/16 15:40:12, Mark Seaborn wrote:
> On 2013/07/12 21:54:29, Sam Clegg wrote:
> > On 2013/07/12 19:47:29, Mark Seaborn wrote:
> > > This is fixed in
> > > https://src.chromium.org/viewvc/native_client?view=rev&revision=11693. 
Can
> > you
> > > remove this?
> > 
> > 
> > Added TODO to remove when this lands.
> 
> The change to fix this should have rolled into Chrome several days ago.  Would
> you mind trying this with the current SDK?  You should be able to remove this
> now.

This is still missing in the lastest SDK built against nacl r11718.

I guess this is because the TOOLCHAIN_REVISION needs to be rolled before the
header makes it into the toolchain?

https://codereview.chromium.org/18112024/diff/7001/libraries/python/nacl-pyth...
File libraries/python/nacl-python.sh (right):

https://codereview.chromium.org/18112024/diff/7001/libraries/python/nacl-pyth...
libraries/python/nacl-python.sh:19: CFLAGS="" LDFLAGS="" LogExecute ../configure
On 2013/07/16 15:40:12, Mark Seaborn wrote:
> Can you comment setting CFLAGS="" LDFLAGS="", please?

Done.

https://codereview.chromium.org/18112024/diff/7001/libraries/python/nacl-pyth...
libraries/python/nacl-python.sh:29: EXTRA_CONFIGURE_ARGS+=" --disable-ipv6"
On 2013/07/16 15:40:12, Mark Seaborn wrote:
> Should this be
> EXTRA_CONFIGURE_ARGS="--disable-ipv6"
> for the first arg?

Done.

[Mark Seaborn, 2013-07-16 16:33:22]

On 16 July 2013 09:19, <sbc@chromium.org> wrote:

>
> https://codereview.chromium.**org/18112024/diff/1/libraries/**
>
python/nacl.patch<https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl.patch>
> File libraries/python/nacl.patch (right):
>
> https://codereview.chromium.**org/18112024/diff/1/libraries/**
>
python/nacl.patch#newcode51<https://codereview.chromium.org/18112024/diff/1/libraries/python/nacl.patch#newcode51>
> libraries/python/nacl.patch:**51: +extern int lstat(const char *, struct
> stat *);
> On 2013/07/16 15:40:12, Mark Seaborn wrote:
>
>> On 2013/07/12 21:54:29, Sam Clegg wrote:
>> > On 2013/07/12 19:47:29, Mark Seaborn wrote:
>> > > This is fixed in
>> > > https://src.chromium.org/viewvc/native_client?view=rev&revision=11693.
>> Can you
>> > > remove this?
>> >
>> >
>> > Added TODO to remove when this lands.
>>
>
>  The change to fix this should have rolled into Chrome several days ago.
>>  Would
>>
> you mind trying this with the current SDK?  You should be able to remove
>> this
>>
> now.
>>
>
> This is still missing in the lastest SDK built against nacl r11718.
>
> I guess this is because the TOOLCHAIN_REVISION needs to be rolled before
> the header makes it into the toolchain?
>

Ah, OK, maybe the lstat() fix is not in yet.  But the fsync()/fdatasync()
additions to libnosys should definitely be in.  They don't require a
TOOL_REVISIONS update:

    r11694: (mseaborn) Add stubs for fsync() and fdatasync() to libnosys

Mark

-- 
You received this message because you are subscribed to the Google Groups
"Native-Client-Reviews" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to native-client-reviews+unsubscribe@googlegroups.com.
To post to this group, send email to native-client-reviews@googlegroups.com.
Visit this group at http://groups.google.com/group/native-client-reviews.
For more options, visit https://groups.google.com/groups/opt_out.

[Mark Seaborn, 2013-07-16 17:10:04]

LGTM, preferably with fsync()/fdatasync() defs removed if that works.

https://codereview.chromium.org/18112024/diff/15001/libraries/python/nacl-pyt...
File libraries/python/nacl-python.sh (right):

https://codereview.chromium.org/18112024/diff/15001/libraries/python/nacl-pyt...
libraries/python/nacl-python.sh:37: # which we do done provide.  TODO(sbc):
remove this once nacl_io supports
"do not provide"

[Sam Clegg, 2013-07-16 21:31:40]

Committed patchset #5 manually as r823 (presubmit successful).