linux: don't instrument mmx functions

linux: don't instrument mmx functions

Shadow stacks use -finstrument-functions to hook all function entry/exit.
But mmx-related functions expect to be inlined so the compiler can
compile them out into mmx instructions.

This change excludes the mmx intrinsics header file from instrumentation.
Without it, builds wil fail with errors like the following:

../../third_party/gold/gold64: obj/media/libyuv_convert.a(obj/media/base/yuv_convert.yuv_convert.o): in function media::EmptyRegisterState():yuv_convert.cc(.text._ZN5media18EmptyRegisterStateEv+0x29): error: undefined reference to '_mm_empty()'

(_mm_empty() is an mmx intrinsic function.)


Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=123289

[Evan Martin, 2012-02-21 20:10:25]

glider: review
others: FYI

[Alexander Potapenko, 2012-02-22 11:50:33]

LGTM

[commit-bot: I haz the power, 2012-02-22 23:42:16]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/evan@chromium.org/9416068/1

[commit-bot: I haz the power, 2012-02-23 02:57:40]

Try job failure for 9416068-1 (retry) on win_rel for step "ui_tests".
It's a second try, previously, step "ui_tests" failed.
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=win_rel&nu...

[commit-bot: I haz the power, 2012-02-23 03:36:20]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/evan@chromium.org/9416068/1

[commit-bot: I haz the power, 2012-02-23 07:31:56]

Try job failure for 9416068-1 (retry) (retry) on win_rel for step "ui_tests".
It's a second try, previously, step "ui_tests" failed.
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=win_rel&nu...

[commit-bot: I haz the power, 2012-02-23 17:22:38]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/evan@chromium.org/9416068/1

[commit-bot: I haz the power, 2012-02-23 19:10:53]

Change committed as 123289

[Ami GONE FROM CHROMIUM, 2012-03-26 18:24:11]

Under clang this change emits this line for every .o built:
clang: warning: argument unused during compilation:
'-finstrument-functions-exclude-file-list=mmintrin.h'

Any reason not to make the -finstrument-functions-exclude-file-list=mmintrin.h
flag gcc-specific?

[Evan Martin, 2012-03-26 18:27:58]

On 2012/03/26 18:24:11, Ami Fischman wrote:
> Under clang this change emits this line for every .o built:
> clang: warning: argument unused during compilation:
> '-finstrument-functions-exclude-file-list=mmintrin.h'
> 
> Any reason not to make the -finstrument-functions-exclude-file-list=mmintrin.h
> flag gcc-specific?

SGTM.  But every time we diverge gcc and clang the build gets a little bit more
finicky, so it'd be nice to eventually make clang accept this flag.

(Are you building with -Dlinux_keep_shadow_stacks=1 on clang? I'm impressed that
works!)

[Ami GONE FROM CHROMIUM, 2012-03-26 18:38:19]

>
> SGTM.  But every time we diverge gcc and clang the build gets a little bit
> more
> finicky


Agreed.


> so it'd be nice to eventually make clang accept this flag.
>

But the flag is a workaround for gcc!  Wouldn't it be nicer to eventually
make gcc not need it?


> (Are you building with -Dlinux_keep_shadow_stacks=1 on clang? I'm
> impressed that works!)
>

You and me both.  I just added the -D to repro a heapcheck issue and didn't
think about the clanginess of my build until after it worked.

[Evan Martin, 2012-03-26 18:58:15]

On 2012/03/26 18:38:19, Ami Fischman wrote:
> You and me both.  I just added the -D to repro a heapcheck issue and didn't
> think about the clanginess of my build until after it worked.

FWIW, having spent some time looking into profiling, I conclude that it'd be
better to use a real stack unwinder like libunwind than relying on this shadow
stacks thing.

[Ami GONE FROM CHROMIUM, 2012-03-26 20:04:35]

>
> FWIW, having spent some time looking into profiling, I conclude that it'd
> be
> better to use a real stack unwinder like libunwind than relying on this
> shadow
> stacks thing.
>

I'm not horribly surprised, but I couldn't justify it without a bunch of
research + swapping in historical knowledge.
Feel like filing a crbug for this with the state you've recently
accumulated?

-a

[Evan Martin, 2012-03-26 20:30:51]

On Mon, Mar 26, 2012 at 1:04 PM, Ami Fischman <fischman@chromium.org> wrote:
>> FWIW, having spent some time looking into profiling, I conclude that it'd
>> be
>> better to use a real stack unwinder like libunwind than relying on this
>> shadow
>> stacks thing.
>
> I'm not horribly surprised, but I couldn't justify it without a bunch of
> research + swapping in historical knowledge.
> Feel like filing a crbug for this with the state you've recently
> accumulated?

I bet glider has more smart things to say than I do, but basically:
shadow stacks rely on the compiler to instrument every function, which
doesn't work through existing libraries (like any stack that goes
through gtk).

[Alexander Potapenko, 2012-03-27 11:32:01]

Most of the time we have the opposite problem: the system libraries are built
without frame pointers, so the stacks are often broken:
http://code.google.com/p/chromium/issues/detail?id=51988

IIRC libunwind does not work with tcmalloc out of the box because of a subtle
deadlock in dl_iterate_phdr which might be triggered while unwinding.

I think I've tried to combine the data from frame pointers and the shadow stack,
but considered it not worth the effort. The slowdown brought by the shadow
stacks is not that big, and we anyway use them only for leak checking, where the
guts of the third party libraries may not be very interesting.

OTOH the shadow stacks have some impact on the CPU profile, so it's better to
avoid using them with the tcmalloc profiler.