Subzero: Run sandboxed cross tests, and do some cleanup.

Subzero: Run sandboxed cross tests, and do some cleanup.

Tests all cross tests in both sandboxed and unsandboxed modes.  Unfortunately, crosstest run time is more than doubled because of LTO of the crosstest drivers.  (We may want to add "full" and "lite" versions of cross tests.)

LLVM triple strings are removed where possible (from .ll files), and when generated, we use just i686 or i686-nacl.

"Fix" the integrated assembler to emit the lock prefix after the 16-bit operand prefix, to make the validator happy.

Don't add external symbol references to the ELF file for llvm.* intrinsic functions.

Make the ELF object writer honor the -externalize argument.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=4092
R=jvoung@chromium.org

Committed: https://gerrit.chromium.org/gerrit/gitweb?p=native_client/pnacl-subzero.git;a=commit;h=a67fc448e9779ac8b7d9751bd7cf32f94047a345

[Jim Stichnoth, 2015-03-03 20:43:06]

stichnot@chromium.org changed reviewers:
+ jvoung@chromium.org, kschimpf@google.com, mtrofin@chromium.org

[Jim Stichnoth, 2015-03-03 22:44:03]

I got the spurious whitespace changes from clang-format rolled back, so now the
patch doesn't look twice as big as it ought to.  We should do a separate
clang-format CL based on the updated clang-format.

[jvoung (off chromium), 2015-03-03 22:58:02]

https://codereview.chromium.org/973823003/diff/1/src/IceELFObjectWriter.cpp
File src/IceELFObjectWriter.cpp (right):

https://codereview.chromium.org/973823003/diff/1/src/IceELFObjectWriter.cpp#n...
src/IceELFObjectWriter.cpp:551: if (Info)
oops didn't think an intrinsic name would make it this far down, but I guess
it's added to the extern syms list early?

https://codereview.chromium.org/973823003/diff/40001/pydir/crosstest.py
File pydir/crosstest.py (left):

https://codereview.chromium.org/973823003/diff/40001/pydir/crosstest.py#oldco...
pydir/crosstest.py:97: line = fix_target.sub('i686-pc-linux-gnu', line)
nice =)

https://codereview.chromium.org/973823003/diff/40001/pydir/run-pnacl-sz.py
File pydir/run-pnacl-sz.py (right):

https://codereview.chromium.org/973823003/diff/40001/pydir/run-pnacl-sz.py#ne...
pydir/run-pnacl-sz.py:115: # TODO(stichnot): use i686-nacl?
What are the issues with deciding between i686 or i686-nacl?

[Jim Stichnoth, 2015-03-03 23:18:51]

https://codereview.chromium.org/973823003/diff/1/src/IceELFObjectWriter.cpp
File src/IceELFObjectWriter.cpp (right):

https://codereview.chromium.org/973823003/diff/1/src/IceELFObjectWriter.cpp#n...
src/IceELFObjectWriter.cpp:551: if (Info)
On 2015/03/03 22:58:01, jvoung wrote:
> oops didn't think an intrinsic name would make it this far down, but I guess
> it's added to the extern syms list early?

Yeah, these llvm.* symbols were making it into the ELF file and causing
undefined symbol errors during the link.  I'm not sure why it wasn't happening
for the spec2k builds - maybe it was something about the gold arguments?  In any
case, it seemed like keeping them out of the ELF file was the right thing to do.

This definitely happens for the IceConverter, but I'm not sure whether the
PNaClTranslator does it as well.  Regardless, it seemed more appropriate to
continue dealing with intrinsic issues during lowering and afterwards.

https://codereview.chromium.org/973823003/diff/40001/pydir/run-pnacl-sz.py
File pydir/run-pnacl-sz.py (right):

https://codereview.chromium.org/973823003/diff/40001/pydir/run-pnacl-sz.py#ne...
pydir/run-pnacl-sz.py:115: # TODO(stichnot): use i686-nacl?
On 2015/03/03 22:58:02, jvoung wrote:
> What are the issues with deciding between i686 or i686-nacl?

If we are testing low-level aspects of sandboxing, and use --assemble, we would
probably want to use -triple=i686-nacl.  Maybe the TODO should say to add a
--sandbox argument and then select between i686 and i686-nacl?  Thoughts?

[jvoung (off chromium), 2015-03-03 23:52:53]

lgtm

https://codereview.chromium.org/973823003/diff/1/src/IceELFObjectWriter.cpp
File src/IceELFObjectWriter.cpp (right):

https://codereview.chromium.org/973823003/diff/1/src/IceELFObjectWriter.cpp#n...
src/IceELFObjectWriter.cpp:551: if (Info)
On 2015/03/03 23:18:51, stichnot wrote:
> On 2015/03/03 22:58:01, jvoung wrote:
> > oops didn't think an intrinsic name would make it this far down, but I guess
> > it's added to the extern syms list early?
> 
> Yeah, these llvm.* symbols were making it into the ELF file and causing
> undefined symbol errors during the link.  I'm not sure why it wasn't happening
> for the spec2k builds - maybe it was something about the gold arguments?  In
any
> case, it seemed like keeping them out of the ELF file was the right thing to
do.
> 

Okay, I agree it shouldn't be in the undefined symbols list, so it's not really
gold's problem.

> This definitely happens for the IceConverter, but I'm not sure whether the
> PNaClTranslator does it as well.  Regardless, it seemed more appropriate to
> continue dealing with intrinsic issues during lowering and afterwards.

I thought it would be better that earlier layers like
IceConverter/PNaClTranslator or the target lowering to "fully" deal with
intrinsics and not expose intrinsics to this layer. For example, at this layer
it should have "setjmp" but not "llvm.setjmp". Otherwise, this is fine for now
to get testing going.

https://codereview.chromium.org/973823003/diff/40001/pydir/run-pnacl-sz.py
File pydir/run-pnacl-sz.py (right):

https://codereview.chromium.org/973823003/diff/40001/pydir/run-pnacl-sz.py#ne...
pydir/run-pnacl-sz.py:115: # TODO(stichnot): use i686-nacl?
On 2015/03/03 23:18:51, stichnot wrote:
> On 2015/03/03 22:58:02, jvoung wrote:
> > What are the issues with deciding between i686 or i686-nacl?
> 
> If we are testing low-level aspects of sandboxing, and use --assemble, we
would
> probably want to use -triple=i686-nacl.  Maybe the TODO should say to add a
> --sandbox argument and then select between i686 and i686-nacl?  Thoughts?

Yeah I think clarify in the TODO to choose between i686 and i686-nacl if
sandboxing is being tested or not?

[Jim Stichnoth, 2015-03-04 00:12:58]

https://codereview.chromium.org/973823003/diff/40001/pydir/run-pnacl-sz.py
File pydir/run-pnacl-sz.py (right):

https://codereview.chromium.org/973823003/diff/40001/pydir/run-pnacl-sz.py#ne...
pydir/run-pnacl-sz.py:115: # TODO(stichnot): use i686-nacl?
On 2015/03/03 23:52:53, jvoung wrote:
> On 2015/03/03 23:18:51, stichnot wrote:
> > On 2015/03/03 22:58:02, jvoung wrote:
> > > What are the issues with deciding between i686 or i686-nacl?
> > 
> > If we are testing low-level aspects of sandboxing, and use --assemble, we
> would
> > probably want to use -triple=i686-nacl.  Maybe the TODO should say to add a
> > --sandbox argument and then select between i686 and i686-nacl?  Thoughts?
> 
> Yeah I think clarify in the TODO to choose between i686 and i686-nacl if
> sandboxing is being tested or not?

Done.

[Jim Stichnoth, 2015-03-04 00:13:23]

Committed patchset #4 (id:60001) manually as
a67fc448e9779ac8b7d9751bd7cf32f94047a345 (presubmit successful).