Use seperate usr/lib and usr/include directories for i686-nacl

Use seperate usr/lib and usr/include directories for i686-nacl

This is what the naclports build system is expecting,
and I think its preferable anyway (for example, its
not a good idea to share include paths between
architectures in general).

In the long run I'd like to see lib32 move too, but
that is a bigger change.  This is just enough to make
naclports happy.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=4108
R=dschuff@chromium.org

Committed: 16ff1a6d2e820f1f686440c436d2bc610413879a

[Sam Clegg, 2015-02-21 01:24:24]

sbc@chromium.org changed reviewers:
+ dschuff@chromium.org

[Derek Schuff, 2015-02-26 01:10:51]

On 2015/02/21 01:32:31, Sam Clegg wrote:

I agree with you, e.g. on sharing include paths, but I had done it this way to
line up with gcc. Does this change make it match gcc?

[Sam Clegg, 2015-02-26 17:17:23]

On 2015/02/26 01:10:51, Derek Schuff wrote:
> On 2015/02/21 01:32:31, Sam Clegg wrote:
> 
> I agree with you, e.g. on sharing include paths, but I had done it this way to
> line up with gcc. Does this change make it match gcc?

gcc doesn't have usr/include or usr/lib at all.  We add them in via spec file
hacking.  And we use i686-nacl/usr/.  Having said that, for the non-usr include
and lib paths gcc does use x86_64-nacl/include and x86_64-nacl/lib32.   So this
change make clang match what we have for gcc today which is a bit of a mixed
bag.

[Derek Schuff, 2015-02-26 23:29:58]

On 2015/02/26 17:17:23, Sam Clegg wrote:
> On 2015/02/26 01:10:51, Derek Schuff wrote:
> > On 2015/02/21 01:32:31, Sam Clegg wrote:
> > 
> > I agree with you, e.g. on sharing include paths, but I had done it this way
to
> > line up with gcc. Does this change make it match gcc?
> 
> gcc doesn't have usr/include or usr/lib at all.  We add them in via spec file
> hacking.  And we use i686-nacl/usr/.  Having said that, for the non-usr
include
> and lib paths gcc does use x86_64-nacl/include and x86_64-nacl/lib32.   So
this
> change make clang match what we have for gcc today which is a bit of a mixed
> bag.

ok this change LGTM since the SDK/naclports already uses this multi-arch style.
we can discuss the more general multilib style of the toolchain/libc libs
separately (and maybe bring in Roland too) if you want.

[Sam Clegg, 2015-02-26 23:42:38]

Committed patchset #1 (id:1) manually as
16ff1a6d2e820f1f686440c436d2bc610413879a (presubmit successful).

[jvoung (off chromium), 2015-02-27 17:00:08]

jvoung@chromium.org changed reviewers:
+ jvoung@chromium.org

[jvoung (off chromium), 2015-02-27 17:00:09]

I'm trying to merge this change into: https://codereview.chromium.org/800553003/

My understanding is that this wasn't supposed to affect include only
usr/include.

However, errno.h, is no longer found, and errno.h is normally in:

ind toolchain/linux_x86/pnacl_newlib -name "errno.h"
toolchain/linux_x86/pnacl_newlib/i686_bc-nacl/include/errno.h
toolchain/linux_x86/pnacl_newlib/i686_bc-nacl/include/sys/errno.h
toolchain/linux_x86/pnacl_newlib/arm-nacl/include/errno.h
toolchain/linux_x86/pnacl_newlib/arm-nacl/include/sys/errno.h
toolchain/linux_x86/pnacl_newlib/arm_bc-nacl/include/errno.h
toolchain/linux_x86/pnacl_newlib/arm_bc-nacl/include/sys/errno.h
toolchain/linux_x86/pnacl_newlib/le32-nacl/include/errno.h
toolchain/linux_x86/pnacl_newlib/le32-nacl/include/sys/errno.h
toolchain/linux_x86/pnacl_newlib/x86_64-nacl/include/errno.h
toolchain/linux_x86/pnacl_newlib/x86_64-nacl/include/sys/errno.h

^^ HERE

toolchain/linux_x86/pnacl_newlib/x86_64_bc-nacl/include/errno.h
toolchain/linux_x86/pnacl_newlib/x86_64_bc-nacl/include/sys/errno.h


Error while building toolchain core_sdk_libs:


/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/x86_64-nacl-clang
-m32 -std=gnu99 -Wstrict-prototypes -O2 -g -fomit-frame\
> -pointer -fasynchronous-unwind-tables -Wall -Wundef -fdiagnostics-show-option
-Werror -Wno-variadic-macros -Wno-language-extension-token
-D__STDC_LIMIT_MACROS=1 -D__STDC_FORM\
> AT_MACROS=1 -D_GNU_SOURCE=1 -D_DEFAULT_SOURCE=1 -D_BSD_SOURCE=1
-D_POSIX_C_SOURCE=199506 -D_XOPEN_SOURCE=600 -DDYNAMIC_ANNOTATIONS_ENABLED=1
-DDYNAMIC_ANNOTATIONS_PREFIX=NACL\
> _ -DNACL_BUILD_ARCH=x86 -DNACL_BUILD_SUBARCH=32
-Itoolchain_build/out/core_sdk_libs_i686_work/nacl-x86-32-clang/include
-I/usr/local/google/home/nacl3 -c -o toolchain_build/o\
>
ut/core_sdk_libs_i686_work/nacl-x86-32-clang/obj/src/untrusted/pthread/nc_condvar.o
src/untrusted/pthread/nc_condvar.c -v
clang version 3.5.0
(https://chromium.googlesource.com/a/native_client/pnacl-clang.git
16ff1a6d2e820f1f686440c436d2bc610413879a)
(https://chromium.googlesource.com/a/native_client/pnacl-llvm.git
03e70587abbb52791c483e07a99917a67af35036)
Target: i386--nacl
Thread model: posix

"/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/x86_64-nacl-clang"
-cc1 -triple i386--nacl -S -disable-free -main-file-name nc_condvar.c
-mrelocation-model static -fmath-errno -no-integrated-as -mconstructor-aliases
-munwind-tables -fuse-init-array -target-cpu pentium4 -target-linker-version
2.24 -v -g -dwarf-column-info -coverage-file /tmp/nc_condvar-960267.s
-resource-dir
/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../lib/clang/3.5.0
-D __STDC_LIMIT_MACROS=1 -D __STDC_FORMAT_MACROS=1 -D _GNU_SOURCE=1 -D
_DEFAULT_SOURCE=1 -D _BSD_SOURCE=1 -D _POSIX_C_SOURCE=199506 -D
_XOPEN_SOURCE=600 -D DYNAMIC_ANNOTATIONS_ENABLED=1 -D
DYNAMIC_ANNOTATIONS_PREFIX=NACL_ -D NACL_BUILD_ARCH=x86 -D NACL_BUILD_SUBARCH=32
-I toolchain_build/out/core_sdk_libs_i686_work/nacl-x86-32-clang/include -I
/usr/local/google/home/nacl3 -internal-isystem
/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../lib/clang/3.5.0/include
-internal-isystem
/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../i686-nacl/usr/include
-internal-isystem
/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../i686-nacl/include
-O2 -Wstrict-prototypes -Wall -Wundef -Werror -Wno-variadic-macros
-Wno-language-extension-token -std=gnu99 -fno-dwarf-directory-asm
-fdebug-compilation-dir /usr/local/google/home/nacl3/native_client -ferror-limit
19 -fmessage-length 175 -mstackrealign -fobjc-runtime=gcc
-fdiagnostics-show-option -vectorize-loops -vectorize-slp -o
/tmp/nc_condvar-960267.s -x c src/untrusted/pthread/nc_condvar.c
clang -cc1 version 3.5.0 based upon LLVM 3.5.0 default target
x86_64-unknown-linux-gnu
ignoring nonexistent directory
"/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../i686-nacl/usr/include"
ignoring nonexistent directory
"/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../i686-nacl/include"
ignoring duplicate directory
"/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../lib/clang/3.5.0/include"
#include "..." search starts here:
#include <...> search starts here:
 toolchain_build/out/core_sdk_libs_i686_work/nacl-x86-32-clang/include
 /usr/local/google/home/nacl3

/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../lib/clang/3.5.0/include
End of search list.
src/untrusted/pthread/nc_condvar.c:11:10: fatal error: 'errno.h' file not found
#include <errno.h>
         ^
1 error generated.

[jvoung (off chromium), 2015-02-27 17:13:35]

https://codereview.chromium.org/946063002/diff/1/lib/Driver/ToolChains.cpp
File lib/Driver/ToolChains.cpp (left):

https://codereview.chromium.org/946063002/diff/1/lib/Driver/ToolChains.cpp#ol...
lib/Driver/ToolChains.cpp:2484: llvm::sys::path::append(P, "include");
This would have been x86_64-nacl/include, via the "x86_64-nacl/usr/include"
above.

Now it's changed to i686/include.

[Sam Clegg, 2015-02-27 17:13:56]

On 2015/02/27 17:00:09, jvoung wrote:
> I'm trying to merge this change into:
https://codereview.chromium.org/800553003/
> 
> My understanding is that this wasn't supposed to affect include only
> usr/include.
> 
> However, errno.h, is no longer found, and errno.h is normally in:
> 
> ind toolchain/linux_x86/pnacl_newlib -name "errno.h"
> toolchain/linux_x86/pnacl_newlib/i686_bc-nacl/include/errno.h
> toolchain/linux_x86/pnacl_newlib/i686_bc-nacl/include/sys/errno.h
> toolchain/linux_x86/pnacl_newlib/arm-nacl/include/errno.h
> toolchain/linux_x86/pnacl_newlib/arm-nacl/include/sys/errno.h
> toolchain/linux_x86/pnacl_newlib/arm_bc-nacl/include/errno.h
> toolchain/linux_x86/pnacl_newlib/arm_bc-nacl/include/sys/errno.h
> toolchain/linux_x86/pnacl_newlib/le32-nacl/include/errno.h
> toolchain/linux_x86/pnacl_newlib/le32-nacl/include/sys/errno.h
> toolchain/linux_x86/pnacl_newlib/x86_64-nacl/include/errno.h
> toolchain/linux_x86/pnacl_newlib/x86_64-nacl/include/sys/errno.h
> 
> ^^ HERE
> 
> toolchain/linux_x86/pnacl_newlib/x86_64_bc-nacl/include/errno.h
> toolchain/linux_x86/pnacl_newlib/x86_64_bc-nacl/include/sys/errno.h
> 
> 
> Error while building toolchain core_sdk_libs:
> 
> 
>
/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/x86_64-nacl-clang
> -m32 -std=gnu99 -Wstrict-prototypes -O2 -g -fomit-frame\
> > -pointer -fasynchronous-unwind-tables -Wall -Wundef
-fdiagnostics-show-option
> -Werror -Wno-variadic-macros -Wno-language-extension-token
> -D__STDC_LIMIT_MACROS=1 -D__STDC_FORM\
> > AT_MACROS=1 -D_GNU_SOURCE=1 -D_DEFAULT_SOURCE=1 -D_BSD_SOURCE=1
> -D_POSIX_C_SOURCE=199506 -D_XOPEN_SOURCE=600 -DDYNAMIC_ANNOTATIONS_ENABLED=1
> -DDYNAMIC_ANNOTATIONS_PREFIX=NACL\
> > _ -DNACL_BUILD_ARCH=x86 -DNACL_BUILD_SUBARCH=32
> -Itoolchain_build/out/core_sdk_libs_i686_work/nacl-x86-32-clang/include
> -I/usr/local/google/home/nacl3 -c -o toolchain_build/o\
> >
>
ut/core_sdk_libs_i686_work/nacl-x86-32-clang/obj/src/untrusted/pthread/nc_condvar.o
> src/untrusted/pthread/nc_condvar.c -v
> clang version 3.5.0
> (https://chromium.googlesource.com/a/native_client/pnacl-clang.git
> 16ff1a6d2e820f1f686440c436d2bc610413879a)
> (https://chromium.googlesource.com/a/native_client/pnacl-llvm.git
> 03e70587abbb52791c483e07a99917a67af35036)
> Target: i386--nacl
> Thread model: posix
> 
>
"/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/x86_64-nacl-clang"
> -cc1 -triple i386--nacl -S -disable-free -main-file-name nc_condvar.c
> -mrelocation-model static -fmath-errno -no-integrated-as -mconstructor-aliases
> -munwind-tables -fuse-init-array -target-cpu pentium4 -target-linker-version
> 2.24 -v -g -dwarf-column-info -coverage-file /tmp/nc_condvar-960267.s
> -resource-dir
>
/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../lib/clang/3.5.0
> -D __STDC_LIMIT_MACROS=1 -D __STDC_FORMAT_MACROS=1 -D _GNU_SOURCE=1 -D
> _DEFAULT_SOURCE=1 -D _BSD_SOURCE=1 -D _POSIX_C_SOURCE=199506 -D
> _XOPEN_SOURCE=600 -D DYNAMIC_ANNOTATIONS_ENABLED=1 -D
> DYNAMIC_ANNOTATIONS_PREFIX=NACL_ -D NACL_BUILD_ARCH=x86 -D
NACL_BUILD_SUBARCH=32
> -I toolchain_build/out/core_sdk_libs_i686_work/nacl-x86-32-clang/include -I
> /usr/local/google/home/nacl3 -internal-isystem
>
/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../lib/clang/3.5.0/include
> -internal-isystem
>
/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../i686-nacl/usr/include
> -internal-isystem
>
/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../i686-nacl/include
> -O2 -Wstrict-prototypes -Wall -Wundef -Werror -Wno-variadic-macros
> -Wno-language-extension-token -std=gnu99 -fno-dwarf-directory-asm
> -fdebug-compilation-dir /usr/local/google/home/nacl3/native_client
-ferror-limit
> 19 -fmessage-length 175 -mstackrealign -fobjc-runtime=gcc
> -fdiagnostics-show-option -vectorize-loops -vectorize-slp -o
> /tmp/nc_condvar-960267.s -x c src/untrusted/pthread/nc_condvar.c
> clang -cc1 version 3.5.0 based upon LLVM 3.5.0 default target
> x86_64-unknown-linux-gnu
> ignoring nonexistent directory
>
"/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../i686-nacl/usr/include"
> ignoring nonexistent directory
>
"/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../i686-nacl/include"
> ignoring duplicate directory
>
"/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../lib/clang/3.5.0/include"
> #include "..." search starts here:
> #include <...> search starts here:
>  toolchain_build/out/core_sdk_libs_i686_work/nacl-x86-32-clang/include
>  /usr/local/google/home/nacl3
> 
>
/usr/local/google/home/nacl3/native_client/toolchain_build/out/sdk_compiler_install/sdk_compiler/bin/../lib/clang/3.5.0/include
> End of search list.
> src/untrusted/pthread/nc_condvar.c:11:10: fatal error: 'errno.h' file not
found
> #include <errno.h>
>          ^
> 1 error generated.

Somehow its changed the /include as well as /usr/include.  Strange.

[Derek Schuff, 2015-02-27 17:51:57]

https://codereview.chromium.org/946063002/diff/1/lib/Driver/ToolChains.cpp
File lib/Driver/ToolChains.cpp (left):

https://codereview.chromium.org/946063002/diff/1/lib/Driver/ToolChains.cpp#ol...
lib/Driver/ToolChains.cpp:2484: llvm::sys::path::append(P, "include");
On 2015/02/27 17:13:35, jvoung wrote:
> This would have been x86_64-nacl/include, via the "x86_64-nacl/usr/include"
> above.
> 
> Now it's changed to i686/include.

you're right. this also breaks the src/clang/test/Driver/nacl-direct.c test
(because it was not updated).
So yeah we should just revert it for now.