Have CMake LLVM build share CFLAGS/CXXFLAGS with autoconf (e.g., libc++ flags).

Have CMake LLVM build share CFLAGS/CXXFLAGS with autoconf (e.g., libc++ flags).

That way, things are more uniform and we don't have
a divergence where an autoconf build of LLVM uses libc++
headers/library while the cmake build does not.
E.g., subzero is always built with libc++ and it would
fail to link against an old cmake build of llvm
(which uses libstdc++).

We need to disable go bindings test:
https://codereview.chromium.org/993513008/

Misc cleanup around the CompilersForHost code/comments
(update comments and remove unused glibc path).

BUG= https://code.google.com/p/nativeclient/issues/detail?id=4119
R=dschuff@chromium.org

Committed: https://chromium.googlesource.com/native_client/src/native_client/+/4dfff0ba4bd05d8e9ed3b7755a5cea1b29c9252b

[jvoung (off chromium), 2015-03-09 22:37:23]

jvoung@chromium.org changed reviewers:
+ dschuff@chromium.org, stichnot@chromium.org

[jvoung (off chromium), 2015-03-09 22:42:33]

https://codereview.chromium.org/978963002/diff/20001/toolchain_build/toolchai...
File toolchain_build/toolchain_build_pnacl.py (right):

https://codereview.chromium.org/978963002/diff/20001/toolchain_build/toolchai...
toolchain_build/toolchain_build_pnacl.py:649: # Older CMake ignore
CMAKE_*_LINKER_FLAGS during config step.
Well, I too ran into a trycompile link failure.

The link below suggests using LDFLAGS (or I guess the newest cmakes have a fix
finally).

[Derek Schuff, 2015-03-10 00:01:36]

dschuff@chromium.org changed reviewers:
+ bradnelson@google.com

[Derek Schuff, 2015-03-10 00:01:37]

the code LGTM.
If brad thinks we can make it work without putting his extra cflags in the
compiler command, maybe we can get rid of that code entirely and just put
everything in CFLAGS

https://codereview.chromium.org/978963002/diff/40001/toolchain_build/toolchai...
File toolchain_build/toolchain_build_pnacl.py (right):

https://codereview.chromium.org/978963002/diff/40001/toolchain_build/toolchai...
toolchain_build/toolchain_build_pnacl.py:225: extra_cc_args +=
[options.extra_cc_args]
+bradn:
This bit is what we added to allow specifying extra cflags on the
toolchain_build command line for pnacl-in-pnacl. This implementation puts the
flags directly as part of the cc command rather than e.g. passing them as cflags
via configure. Do you know if there was a reason they had to be that way, or if
it just happened by accident?

https://codereview.chromium.org/978963002/diff/40001/toolchain_build/toolchai...
toolchain_build/toolchain_build_pnacl.py:295: for linker_type in ['EXE',
'SHARED', 'MODULE']:
I seem to recall that there was some reason I made it just EXE and not all 3
before, but now I can't remember exactly what it was :(

https://codereview.chromium.org/978963002/diff/40001/toolchain_build/toolchai...
toolchain_build/toolchain_build_pnacl.py:653: env={'LDFLAGS' : ' '.join(
I've been trying pretty hard to avoid having env override in Command, but I'm
out of ideas as to how to avoid it this time :( Maybe add to the comment here
that once we no longer care about older cmakes, we can remove that feature.

[jvoung (off chromium), 2015-03-10 19:39:20]

https://codereview.chromium.org/978963002/diff/40001/toolchain_build/toolchai...
File toolchain_build/toolchain_build_pnacl.py (right):

https://codereview.chromium.org/978963002/diff/40001/toolchain_build/toolchai...
toolchain_build/toolchain_build_pnacl.py:295: for linker_type in ['EXE',
'SHARED', 'MODULE']:
On 2015/03/10 00:01:37, Derek Schuff wrote:
> I seem to recall that there was some reason I made it just EXE and not all 3
> before, but now I can't remember exactly what it was :(

Yeah, I'm not sure of the exact consequence. I started with EXE only, but then
added SHARED and MODULE and nothing was observably different. I looked at LLVM's
behavior w/ "-DLLVM_ENABLE_LIBCXX=OFF" switch to ON, and it would add
-stdlib=libc++ to EXE, SHARED, and MODULE, so I figured this might as well match
that.

https://codereview.chromium.org/978963002/diff/40001/toolchain_build/toolchai...
toolchain_build/toolchain_build_pnacl.py:653: env={'LDFLAGS' : ' '.join(
On 2015/03/10 00:01:37, Derek Schuff wrote:
> I've been trying pretty hard to avoid having env override in Command, but I'm
> out of ideas as to how to avoid it this time :( Maybe add to the comment here
> that once we no longer care about older cmakes, we can remove that feature.


Added a comment about deprecating this after the cmake fix propagates for a
while.

Any issue with caching?

[Derek Schuff, 2015-03-10 21:09:45]

https://codereview.chromium.org/978963002/diff/40001/toolchain_build/toolchai...
File toolchain_build/toolchain_build_pnacl.py (right):

https://codereview.chromium.org/978963002/diff/40001/toolchain_build/toolchai...
toolchain_build/toolchain_build_pnacl.py:295: for linker_type in ['EXE',
'SHARED', 'MODULE']:
On 2015/03/10 19:39:19, jvoung wrote:
> On 2015/03/10 00:01:37, Derek Schuff wrote:
> > I seem to recall that there was some reason I made it just EXE and not all 3
> > before, but now I can't remember exactly what it was :(
> 
> Yeah, I'm not sure of the exact consequence. I started with EXE only, but then
> added SHARED and MODULE and nothing was observably different. I looked at
LLVM's
> behavior w/ "-DLLVM_ENABLE_LIBCXX=OFF" switch to ON, and it would add
> -stdlib=libc++ to EXE, SHARED, and MODULE, so I figured this might as well
match
> that.

actually now that I think about it, i'm not sure -stdlib=libc++ is what you want
when linking a shared object? anyway if it all works then this is fine with me.

https://codereview.chromium.org/978963002/diff/60001/toolchain_build/command.py
File toolchain_build/command.py (right):

https://codereview.chromium.org/978963002/diff/60001/toolchain_build/command....
toolchain_build/command.py:203:
check_call_kwargs['env'].update(PlatformEnvironment(path_dirs))
can you do a run with --emit-signatures=foo and ensure that this argument shows
up in the hash? looking at Runnable's ReprForHash above, I'm not sure that it
will.

[jvoung (off chromium), 2015-03-10 22:30:18]

https://codereview.chromium.org/978963002/diff/60001/toolchain_build/command.py
File toolchain_build/command.py (right):

https://codereview.chromium.org/978963002/diff/60001/toolchain_build/command....
toolchain_build/command.py:203:
check_call_kwargs['env'].update(PlatformEnvironment(path_dirs))
On 2015/03/10 21:09:45, Derek Schuff wrote:
> can you do a run with --emit-signatures=foo and ensure that this argument
shows
> up in the hash? looking at Runnable's ReprForHash above, I'm not sure that it
> will.

The "foo" file has:

======================================================================
****************************** PACKAGE SIGNATURE ******************************
package:llvm_x86_64_linux
summary:[('platform', 'linux2'), ('machine', 'x86_64') ...

//...

command:
['cmake', '-G', 'Ninja',
'-DCMAKE_C_COMPILER=/usr/local/google/home/jvoung/nacl4/third_party/llvm-build/Release+Asserts/bin/clang',
'-DCMAKE_CXX_COMPILER=/usr/local/google/home/jvoung/na
cl4/third_party/llvm-build/Release+Asserts/bin/clang++',
'-DHAVE_SANITIZER_MSAN_INTERFACE_H=FALSE', '-DCMAKE_C_FLAGS=',
'-DCMAKE_CXX_FLAGS=-stdlib=libc++ -I%(abs_libcxx_x86_64_linux)s/in
clude/c++/v1', '-DCMAKE_EXE_LINKER_FLAGS=-L%(abs_libcxx_x86_64_linux)s/lib',
'-DCMAKE_SHARED_LINKER_FLAGS=-L%(abs_libcxx_x86_64_linux)s/lib',
'-DCMAKE_MODULE_LINKER_FLAGS=-L%(abs_libcxx_
x86_64_linux)s/lib', '-DBUILD_SHARED_LIBS=ON', '-DCMAKE_BUILD_TYPE=Release',
'-DCMAKE_INSTALL_PREFIX=%(output)s', '-DCMAKE_INSTALL_RPATH=$ORIGIN/../lib',
'-DLLVM_APPEND_VC_REV=ON', '-DLL
VM_BINUTILS_INCDIR=%(abs_binutils_pnacl_src)s/include', '-DLLVM_BUILD_TESTS=ON',
'-DLLVM_ENABLE_ASSERTIONS=ON', '-DLLVM_ENABLE_LIBCXX=OFF',
'-LLVM_ENABLE_WERROR=ON', '-DLLVM_ENABLE_ZLIB=
OFF', '-DLLVM_EXTERNAL_CLANG_SOURCE_DIR=%(clang_src)s',
'-DLLVM_EXTERNAL_SUBZERO_SOURCE_DIR=%(subzero_src)s',
'-DLLVM_TARGETS_TO_BUILD=X86;ARM;Mips', '%(llvm_src)s']
None
'env'
{'LDFLAGS': '-L%(abs_libcxx_x86_64_linux)s/lib'}
2865a36ac1d32273867d26a708f61378ce3e70f0
command:
'lib'
2865a36ac1d32273867d26a708f61378ce3e70f0
command:
'%(abs_libcxx_x86_64_linux)s/lib/libc++.so.1'
'lib/libc++.so.1'
2865a36ac1d32273867d26a708f61378ce3e70f0
command:
['ninja', '-v']
None
// ...

So 'env': {'LDFLAGS': ...} shows up.

====

In Runnable __str__, there is a loop over kwargs:

  values += [repr(k), ReprForHash(v)]

[Derek Schuff, 2015-03-10 22:52:19]

ok, LGTM

[Jim Stichnoth, 2015-04-09 17:14:16]

stichnot@chromium.org changed reviewers:
+ kschimpf@google.com

[jvoung (off chromium), 2015-04-09 22:51:10]

Rebased and re-running trybots.

After blowing away my pnacl_newlib the .so lookup problem for subzero seems to
have gone away.

Go bindings still fail:

./bin/llvm-go test llvm.org/llvm/bindings/go/llvm                               
                                                                               
(git)-[test_cmake] 
# llvm.org/llvm/bindings/go/llvm
/tmp/go-build503624422/llvm.org/llvm/bindings/go/llvm/_test/_obj_test/SupportBindings.cpp.o:SupportBindings.cpp:function
llvm::sys::DynamicLibrary::LoadLibraryPermanently(char const*, std::string*):
error: undefined reference to
'llvm::sys::DynamicLibrary::getPermanentLibrary(char const*, std::string*)'
collect2: error: ld returned 1 exit status
FAIL	llvm.org/llvm/bindings/go/llvm [build failed]

[jvoung (off chromium), 2015-04-09 23:02:26]

On 2015/04/09 22:51:10, jvoung wrote:
> Rebased and re-running trybots.
> 
> After blowing away my pnacl_newlib the .so lookup problem for subzero seems to
> have gone away.
> 
> Go bindings still fail:
> 
> ./bin/llvm-go test llvm.org/llvm/bindings/go/llvm                             
 
>                                                                               

> (git)-[test_cmake] 
> # llvm.org/llvm/bindings/go/llvm
>
/tmp/go-build503624422/llvm.org/llvm/bindings/go/llvm/_test/_obj_test/SupportBindings.cpp.o:SupportBindings.cpp:function
> llvm::sys::DynamicLibrary::LoadLibraryPermanently(char const*, std::string*):
> error: undefined reference to
> 'llvm::sys::DynamicLibrary::getPermanentLibrary(char const*, std::string*)'
> collect2: error: ld returned 1 exit status
> FAIL	llvm.org/llvm/bindings/go/llvm [build failed]

For the go bindings, I see this:

"Did you configure cmake to pass any additional flags to the compiler (maybe to
link against another standard library?)" -- "This should eventually be fixed by
passing CMAKE_C*_FLAGS through to where Go can see them."

https://llvm.org/bugs/show_bug.cgi?id=21552

Running the test with "-x" I see that the test is running a build script:

g++ -I . -fPIC -m64 -pthread -fmessage-length=0
-I/usr/local/google/home/jvoung/nacl4/native_client/toolchain_build/src/llvm/include
-I/usr/local/google/home/jvoung/nacl4/native_client/toolchain_build/out/llvm_x86_64_linux_work/include
-D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS
-D__STDC_LIMIT_MACROS -I $WORK/llvm.org/llvm/bindings/go/llvm/_test/_obj_test/
-std=c++11 -o
$WORK/llvm.org/llvm/bindings/go/llvm/_test/_obj_test/InstrumentationBindings.cpp.o
-c ./InstrumentationBindings.cpp
g++ -I . -fPIC -m64 -pthread -fmessage-length=0
-I/usr/local/google/home/jvoung/nacl4/native_client/toolchain_build/src/llvm/include
-I/usr/local/google/home/jvoung/nacl4/native_client/toolchain_build/out/llvm_x86_64_linux_work/include
-D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS
-D__STDC_LIMIT_MACROS -I $WORK/llvm.org/llvm/bindings/go/llvm/_test/_obj_test/
-std=c++11 -o
$WORK/llvm.org/llvm/bindings/go/llvm/_test/_obj_test/SupportBindings.cpp.o -c
./SupportBindings.cpp
g++ -I . -fPIC -m64 -pthread -fmessage-length=0 -o
$WORK/llvm.org/llvm/bindings/go/llvm/_test/_obj_test/_cgo_.o
$WORK/llvm.org/llvm/bindings/go/llvm/_test/_obj_test/_cgo_main.o
$WORK/llvm.org/llvm/bindings/go/llvm/_test/_obj_test/_cgo_export.o ...
-lLLVMOption -lLLVMObjCARCOpts -lLLVMMCJIT -lLLVMRuntimeDyld -lLLVMLinker
-lLLVMIRReader -lLLVMNaClBitReader -lLLVMNaClAnalysis -lLLVMInterpreter
-lLLVMInstrumentation -lLLVMExecutionEngine -lLLVMDebugInfo -lLLVMBitWriter
-lLLVMAsmParser -lLLVMMipsDisassembler -lLLVMMipsCodeGen -lLLVMMipsAsmParser
-lLLVMMipsDesc -lLLVMMipsInfo -lLLVMMipsAsmPrinter -lLLVMARMDisassembler
-lLLVMARMCodeGen -lLLVMNaClTransforms -lLLVMipo -lLLVMVectorize
-lLLVMARMAsmParser -lLLVMARMDesc -lLLVMARMInfo -lLLVMARMAsmPrinter
-lLLVMX86Disassembler -lLLVMX86AsmParser -lLLVMX86CodeGen -lLLVMSelectionDAG
-lLLVMAsmPrinter -lLLVMCodeGen -lLLVMScalarOpts -lLLVMProfileData
-lLLVMInstCombine -lLLVMTransformUtils -lLLVMipa -lLLVMAnalysis -lLLVMTarget
-lLLVMX86Desc -lLLVMObject -lLLVMMCParser -lLLVMBitReader -lLLVMMCDisassembler
-lLLVMX86Info -lLLVMX86AsmPrinter -lLLVMMC -lLLVMX86Utils -lLLVMCore
-lLLVMSupport -lrt -ldl -ltinfo -latomic -lpthread -lm


It's not picking up the fact that we changed CC or the flags, yet it wants to
link against the LLVM libraries that we've built.

[jvoung (off chromium), 2015-04-10 20:02:27]

Patchset #11 (id:200001) has been deleted

[jvoung (off chromium), 2015-04-10 20:02:33]

Patchset #10 (id:180001) has been deleted

[jvoung (off chromium), 2015-04-10 20:02:38]

Patchset #9 (id:160001) has been deleted

[jvoung (off chromium), 2015-04-10 20:43:40]

Committed patchset #9 (id:220001) manually as
4dfff0ba4bd05d8e9ed3b7755a5cea1b29c9252b (presubmit successful).