Subzero: Update tests and build scripts for sandboxing.

Subzero: Update tests and build scripts for sandboxing.

Spec2k now runs sandboxed, using all filetypes (asm, iasm, obj).

The new build-runtime.py script builds the native and sandboxed runtimes in build/runtime/ .  The various Subzero driver scripts are updated to use that.

Fixes a stack frame bug in sandboxed mode when the integrated assembler is used.  The stack adjustment for setting up a function call wasn't being rolled back for the second emitIAS() pass, so stack variables passed as arguments to the callee were being copied from the wrong stack slot.

Notes:

1. The hybrid Subzero/llc bisection debugging builds probably do not work as intended for -filetype=obj since the ELF emitter doesn't yet support -ffunction-sections.  (This was also true for non-sandboxed hybrid builds.)

2. The cross tests have not yet been adapted for testing sandboxing.  I'd prefer to first make progress on https://code.google.com/p/nativeclient/issues/detail?id=4085 in order to avoid blindly doubling the number of tests.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=4079
R=jvoung@chromium.org

Committed: https://gerrit.chromium.org/gerrit/gitweb?p=native_client/pnacl-subzero.git;a=commit;h=9738a9ea804be2aebc71bc0e5a58414e0774fe57

[Jim Stichnoth, 2015-02-22 18:34:44]

stichnot@chromium.org changed reviewers:
+ jvoung@chromium.org, kschimpf@google.com

[jvoung (off chromium), 2015-02-23 17:58:37]

https://codereview.chromium.org/944333002/diff/80001/pydir/build-runtime.py
File pydir/build-runtime.py (right):

https://codereview.chromium.org/944333002/diff/80001/pydir/build-runtime.py#n...
pydir/build-runtime.py:9: 
Usually nacl/chrome py files have 2 space indent, but I guess most of the files
are already 4 space indent.

https://codereview.chromium.org/944333002/diff/80001/pydir/build-runtime.py#n...
pydir/build-runtime.py:52: shellcmd(['mkdir',
Could have done os.makedirs() instead of shelling out / is more OS-portable, but
I guess this gets logging for free.

https://codereview.chromium.org/944333002/diff/80001/pydir/szbuild.py
File pydir/szbuild.py (right):

https://codereview.chromium.org/944333002/diff/80001/pydir/szbuild.py#newcode300
pydir/szbuild.py:300: opt_level=opt_level_map[opt_level], root=nacl_root),
Looks like opt_level isn't needed for this link invocation, and I don't think it
really matters for the unsandboxed one?

https://codereview.chromium.org/944333002/diff/80001/src/IceInstX8632.h
File src/IceInstX8632.h (right):

https://codereview.chromium.org/944333002/diff/80001/src/IceInstX8632.h#newco...
src/IceInstX8632.h:1039: //   20: cmpxchg ...
Weird, I thought it would have been fixed by:

https://codereview.chromium.org/569773002/diff/140001/test/MC/X86/AlignedBund...

This is with "nacl" as the OS in the target triple?

[Jim Stichnoth, 2015-02-23 21:38:19]

https://codereview.chromium.org/944333002/diff/80001/pydir/build-runtime.py
File pydir/build-runtime.py (right):

https://codereview.chromium.org/944333002/diff/80001/pydir/build-runtime.py#n...
pydir/build-runtime.py:9: 
On 2015/02/23 17:58:36, jvoung wrote:
> Usually nacl/chrome py files have 2 space indent, but I guess most of the
files
> are already 4 space indent.

Yeah, I tried to fit in with the existing files. :(

Probably there should be a separate CL that fixes all the .py files at once.

https://codereview.chromium.org/944333002/diff/80001/pydir/build-runtime.py#n...
pydir/build-runtime.py:52: shellcmd(['mkdir',
On 2015/02/23 17:58:36, jvoung wrote:
> Could have done os.makedirs() instead of shelling out / is more OS-portable,
but
> I guess this gets logging for free.

Done.  I don't care that much about logging this particular operation.

https://codereview.chromium.org/944333002/diff/80001/pydir/szbuild.py
File pydir/szbuild.py (right):

https://codereview.chromium.org/944333002/diff/80001/pydir/szbuild.py#newcode300
pydir/szbuild.py:300: opt_level=opt_level_map[opt_level], root=nacl_root),
On 2015/02/23 17:58:36, jvoung wrote:
> Looks like opt_level isn't needed for this link invocation, and I don't think
it
> really matters for the unsandboxed one?

Good point.  The "ld" man page (assuming that's what third-party clang actually
uses) says -O only affects generation of shared libraries, not executables, so
I'm removing this altogether.

https://codereview.chromium.org/944333002/diff/80001/src/IceInstX8632.h
File src/IceInstX8632.h (right):

https://codereview.chromium.org/944333002/diff/80001/src/IceInstX8632.h#newco...
src/IceInstX8632.h:1039: //   20: cmpxchg ...
On 2015/02/23 17:58:36, jvoung wrote:
> Weird, I thought it would have been fixed by:
> 
>
https://codereview.chromium.org/569773002/diff/140001/test/MC/X86/AlignedBund...
> 
> This is with "nacl" as the OS in the target triple?

Aha!  The szbuild.py was running llvm-mc with just -target=x86, but it should
also use -triple=i686 or -triple=i686-nacl.  So I can undo that workaround.

[jvoung (off chromium), 2015-02-23 23:58:38]

lgtm

https://codereview.chromium.org/944333002/diff/80001/pydir/build-runtime.py
File pydir/build-runtime.py (right):

https://codereview.chromium.org/944333002/diff/80001/pydir/build-runtime.py#n...
pydir/build-runtime.py:52: shellcmd(['mkdir',
On 2015/02/23 21:38:19, stichnot wrote:
> On 2015/02/23 17:58:36, jvoung wrote:
> > Could have done os.makedirs() instead of shelling out / is more OS-portable,
> but
> > I guess this gets logging for free.
> 
> Done.  I don't care that much about logging this particular operation.

Hmm actually, compared to "mkdir -p", you'll may need to be careful about the
directory already existing (so do a check?).

https://codereview.chromium.org/944333002/diff/100001/pydir/crosstest.py
File pydir/crosstest.py (left):

https://codereview.chromium.org/944333002/diff/100001/pydir/crosstest.py#oldc...
pydir/crosstest.py:116: '-arch=' + arch_map[args.target],
I guess this llvm-mc invocation is always native, but might be good to adjust
the commandlines to be consistent w/ szbuild

https://codereview.chromium.org/944333002/diff/100001/pydir/szbuild.py
File pydir/szbuild.py (right):

https://codereview.chromium.org/944333002/diff/100001/pydir/szbuild.py#newcod...
pydir/szbuild.py:225: 'llvm-mc -arch=x86 -triple={triple} ' +
You could just leave out arch and it will get the arch from the triple.

[Jim Stichnoth, 2015-02-24 00:37:57]

https://codereview.chromium.org/944333002/diff/80001/pydir/build-runtime.py
File pydir/build-runtime.py (right):

https://codereview.chromium.org/944333002/diff/80001/pydir/build-runtime.py#n...
pydir/build-runtime.py:52: shellcmd(['mkdir',
On 2015/02/23 23:58:38, jvoung wrote:
> On 2015/02/23 21:38:19, stichnot wrote:
> > On 2015/02/23 17:58:36, jvoung wrote:
> > > Could have done os.makedirs() instead of shelling out / is more
OS-portable,
> > but
> > > I guess this gets logging for free.
> > 
> > Done.  I don't care that much about logging this particular operation.
> 
> Hmm actually, compared to "mkdir -p", you'll may need to be careful about the
> directory already existing (so do a check?).

Oh, good point.  Fixed it to check for the directory existing.  Also, if
build/runtime exists and is for some reason a regular file instead of a
directory, it remove the file first.

https://codereview.chromium.org/944333002/diff/100001/pydir/crosstest.py
File pydir/crosstest.py (left):

https://codereview.chromium.org/944333002/diff/100001/pydir/crosstest.py#oldc...
pydir/crosstest.py:116: '-arch=' + arch_map[args.target],
On 2015/02/23 23:58:38, jvoung wrote:
> I guess this llvm-mc invocation is always native, but might be good to adjust
> the commandlines to be consistent w/ szbuild

Done.  (and tested by temporarily changing default filetype to asm)

https://codereview.chromium.org/944333002/diff/100001/pydir/szbuild.py
File pydir/szbuild.py (right):

https://codereview.chromium.org/944333002/diff/100001/pydir/szbuild.py#newcod...
pydir/szbuild.py:225: 'llvm-mc -arch=x86 -triple={triple} ' +
On 2015/02/23 23:58:38, jvoung wrote:
> You could just leave out arch and it will get the arch from the triple.

Ah, nice, done.

[Jim Stichnoth, 2015-02-24 00:39:10]

Committed patchset #7 (id:120001) manually as
9738a9ea804be2aebc71bc0e5a58414e0774fe57 (presubmit successful).