|
|
Created:
6 years, 7 months ago by Nico Modified:
6 years, 6 months ago CC:
chromium-reviews, awong Base URL:
svn://svn.chromium.org/chrome/trunk/src Visibility:
Public. |
DescriptionTurn on C++11 when building with gcc4.8+
Revert with prejudice (after filing bugs / pinging me ) if this regresses perf or similar.
BUG=377668
R=ajwong@chromium.org
TBR=brettw
Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=279220
Patch Set 1 #Patch Set 2 : ffffff #Patch Set 3 : gntoo #Patch Set 4 : literal #Patch Set 5 : cc #Patch Set 6 : gn #Patch Set 7 : aosp #Patch Set 8 : aosp target gcc is borked too #Patch Set 9 : tweak #Patch Set 10 : gnnnnnnnnn #Messages
Total messages: 17 (0 generated)
android_dbg looks happy, despite crbug.com/377715 … Do you think it's worth trying to land this and see how it fares? (I'm guessing it'll break all kinds of things.)
On 2014/05/27 13:46:20, Nico (traveling) wrote: > android_dbg looks happy, despite crbug.com/377715 … > > Do you think it's worth trying to land this and see how it fares? (I'm guessing > it'll break all kinds of things.) I think it makes sense to land if we want to collect many bugs quickly and be able to resolve them in parallel. Since benchmarks are our main motivation, let's have some scores first. Without speedups, not clear why we need to immediately increase the complexity.
On Tue, May 27, 2014 at 4:41 PM, <pasko@chromium.org> wrote: > On 2014/05/27 13:46:20, Nico (traveling) wrote: > >> android_dbg looks happy, despite crbug.com/377715 ... >> > > Do you think it's worth trying to land this and see how it fares? (I'm >> > guessing > >> it'll break all kinds of things.) >> > > I think it makes sense to land if we want to collect many bugs quickly and > be > able to resolve them in parallel. Since benchmarks are our main motivation, > let's have some scores first. Without speedups, not clear why we need to > immediately increase the complexity. > Actually, my main motivation is to get all platforms on C++11. Right now, it's a mix, and having C++11 on everywhere is supposed to _reduce_ complexity :-) Is there a good way to get perf numbers other than landing? I've been told that the easiest way to perf-test changes is to land them and then watch the perf waterfall. To unsubscribe from this group and stop receiving emails from it, send an email to chromium-reviews+unsubscribe@chromium.org.
On 2014/05/27 15:22:08, Nico (traveling) wrote: > On Tue, May 27, 2014 at 4:41 PM, <mailto:pasko@chromium.org> wrote: > > > On 2014/05/27 13:46:20, Nico (traveling) wrote: > > > >> android_dbg looks happy, despite crbug.com/377715 ... > >> > > > > Do you think it's worth trying to land this and see how it fares? (I'm > >> > > guessing > > > >> it'll break all kinds of things.) > >> > > > > I think it makes sense to land if we want to collect many bugs quickly and > > be > > able to resolve them in parallel. Since benchmarks are our main motivation, > > let's have some scores first. Without speedups, not clear why we need to > > immediately increase the complexity. > > > > Actually, my main motivation is to get all platforms on C++11. Right now, > it's a mix, and having C++11 on everywhere is supposed to _reduce_ > complexity :-) As far as I understand we still cannot move forward with _reducing_ complexity with switching Android build to C++11 until AOSP picks up GCC 4.8. The increased complexity I meant is rather small: more conditionals in common.gypi, but it sounds perfectly fine to make the change for performance reasons. > Is there a good way to get perf numbers other than landing? I've been told > that the easiest way to perf-test changes is to land them and then watch > the perf waterfall. I consider experimenting on perf waterfall to be a *bad* practice because: 1. we observe results on Android only a few days after landing the change 2. we have too many people experimenting with perf like that, so the change is likely to collide with someone else's => makes analysis hard 3. the change may mask a real unintentional regression 4. the change increases the amount of alerts for perf sheriffs, these folks are severely overloaded already, which leads them to making mistakes and missing real regressions I would say, landing a CL after making *some* experiments locally on 1-2 devices is good (saves a lot of effort not to run everywhere), otherwise mere experimentation on perf bots is icky. Many engineers do not share this opinion. Sadly. I realize running locally is a pain for you on a Mac. Options are: * I can run some tests for you * use perf trybot
On Tue, May 27, 2014 at 5:48 PM, <pasko@chromium.org> wrote: > On 2014/05/27 15:22:08, Nico (traveling) wrote: > > On Tue, May 27, 2014 at 4:41 PM, <mailto:pasko@chromium.org> wrote: >> > > > On 2014/05/27 13:46:20, Nico (traveling) wrote: >> > >> >> android_dbg looks happy, despite crbug.com/377715 ... >> >> >> > >> > Do you think it's worth trying to land this and see how it fares? (I'm >> >> >> > guessing >> > >> >> it'll break all kinds of things.) >> >> >> > >> > I think it makes sense to land if we want to collect many bugs quickly >> and >> > be >> > able to resolve them in parallel. Since benchmarks are our main >> motivation, >> > let's have some scores first. Without speedups, not clear why we need to >> > immediately increase the complexity. >> > >> > > Actually, my main motivation is to get all platforms on C++11. Right now, >> it's a mix, and having C++11 on everywhere is supposed to _reduce_ >> complexity :-) >> > > As far as I understand we still cannot move forward with _reducing_ > complexity > with switching Android build to C++11 until AOSP picks up GCC 4.8. The > increased > complexity I meant is rather small: more conditionals in common.gypi, but > it > sounds perfectly fine to make the change for performance reasons. > > > Is there a good way to get perf numbers other than landing? I've been told >> that the easiest way to perf-test changes is to land them and then watch >> the perf waterfall. >> > > I consider experimenting on perf waterfall to be a *bad* practice because: > 1. we observe results on Android only a few days after landing the change > 2. we have too many people experimenting with perf like that, so the > change is > likely to collide with someone else's => makes analysis hard > 3. the change may mask a real unintentional regression > 4. the change increases the amount of alerts for perf sheriffs, these > folks are > severely overloaded already, which leads them to making mistakes and > missing > real regressions > > I would say, landing a CL after making *some* experiments locally on 1-2 > devices > is good (saves a lot of effort not to run everywhere), otherwise mere > experimentation on perf bots is icky. Many engineers do not share this > opinion. > Sadly. > > I realize running locally is a pain for you on a Mac. Options are: > * I can run some tests for you > * use perf trybot > If you have time, it'd be nice if you could run tests. For perf trybots, is http://www.chromium.org/developers/performance-try-botsup-to-date? (I never used these bots before.) To unsubscribe from this group and stop receiving emails from it, send an email to chromium-reviews+unsubscribe@chromium.org.
On 2014/05/27 15:48:25, pasko wrote: > On 2014/05/27 15:22:08, Nico (traveling) wrote: > > On Tue, May 27, 2014 at 4:41 PM, <mailto:pasko@chromium.org> wrote: > > > > > On 2014/05/27 13:46:20, Nico (traveling) wrote: > > > > > >> android_dbg looks happy, despite crbug.com/377715 ... > > >> > > > > > > Do you think it's worth trying to land this and see how it fares? (I'm > > >> > > > guessing > > > > > >> it'll break all kinds of things.) > > >> > > > > > > I think it makes sense to land if we want to collect many bugs quickly and > > > be > > > able to resolve them in parallel. Since benchmarks are our main motivation, > > > let's have some scores first. Without speedups, not clear why we need to > > > immediately increase the complexity. > > > > > > > Actually, my main motivation is to get all platforms on C++11. Right now, > > it's a mix, and having C++11 on everywhere is supposed to _reduce_ > > complexity :-) > > As far as I understand we still cannot move forward with _reducing_ complexity > with switching Android build to C++11 until AOSP picks up GCC 4.8. The current AOSP master version already uses gcc 4.8 for target; the android_aosp bot is just pinned to a release version that's too old, for now. We're looking at updating the bot to use a newer manifest that is on gcc 4.8 and if nothing goes wrong we should be able to have that done in the next ~week. I'm following up with the build team about the status of updating the host toolchain on android, but we only build v8 and protoc for host I think.. > The increased > complexity I meant is rather small: more conditionals in common.gypi, but it > sounds perfectly fine to make the change for performance reasons. > > > Is there a good way to get perf numbers other than landing? I've been told > > that the easiest way to perf-test changes is to land them and then watch > > the perf waterfall. > > I consider experimenting on perf waterfall to be a *bad* practice because: > 1. we observe results on Android only a few days after landing the change > 2. we have too many people experimenting with perf like that, so the change is > likely to collide with someone else's => makes analysis hard > 3. the change may mask a real unintentional regression > 4. the change increases the amount of alerts for perf sheriffs, these folks are > severely overloaded already, which leads them to making mistakes and missing > real regressions > > I would say, landing a CL after making *some* experiments locally on 1-2 devices > is good (saves a lot of effort not to run everywhere), otherwise mere > experimentation on perf bots is icky. Many engineers do not share this opinion. > Sadly. > > I realize running locally is a pain for you on a Mac. Options are: > * I can run some tests for you > * use perf trybot
On 2014/05/27 15:51:21, Nico (traveling) wrote: > On Tue, May 27, 2014 at 5:48 PM, <mailto:pasko@chromium.org> wrote: > > > On 2014/05/27 15:22:08, Nico (traveling) wrote: > > > > On Tue, May 27, 2014 at 4:41 PM, <mailto:pasko@chromium.org> wrote: > >> > > > > > On 2014/05/27 13:46:20, Nico (traveling) wrote: > >> > > >> >> android_dbg looks happy, despite crbug.com/377715 ... > >> >> > >> > > >> > Do you think it's worth trying to land this and see how it fares? (I'm > >> >> > >> > guessing > >> > > >> >> it'll break all kinds of things.) > >> >> > >> > > >> > I think it makes sense to land if we want to collect many bugs quickly > >> and > >> > be > >> > able to resolve them in parallel. Since benchmarks are our main > >> motivation, > >> > let's have some scores first. Without speedups, not clear why we need to > >> > immediately increase the complexity. > >> > > >> > > > > Actually, my main motivation is to get all platforms on C++11. Right now, > >> it's a mix, and having C++11 on everywhere is supposed to _reduce_ > >> complexity :-) > >> > > > > As far as I understand we still cannot move forward with _reducing_ > > complexity > > with switching Android build to C++11 until AOSP picks up GCC 4.8. The > > increased > > complexity I meant is rather small: more conditionals in common.gypi, but > > it > > sounds perfectly fine to make the change for performance reasons. > > > > > > Is there a good way to get perf numbers other than landing? I've been told > >> that the easiest way to perf-test changes is to land them and then watch > >> the perf waterfall. > >> > > > > I consider experimenting on perf waterfall to be a *bad* practice because: > > 1. we observe results on Android only a few days after landing the change > > 2. we have too many people experimenting with perf like that, so the > > change is > > likely to collide with someone else's => makes analysis hard > > 3. the change may mask a real unintentional regression > > 4. the change increases the amount of alerts for perf sheriffs, these > > folks are > > severely overloaded already, which leads them to making mistakes and > > missing > > real regressions > > > > I would say, landing a CL after making *some* experiments locally on 1-2 > > devices > > is good (saves a lot of effort not to run everywhere), otherwise mere > > experimentation on perf bots is icky. Many engineers do not share this > > opinion. > > Sadly. > > > > I realize running locally is a pain for you on a Mac. Options are: > > * I can run some tests for you > > * use perf trybot > > > > If you have time, it'd be nice if you could run tests. I'm re-running thread_times.key_mobile_sites (appears that even with a crash we get some results ...) > For perf trybots, is > http://www.chromium.org/developers/performance-try-bots up-to-date (I > never used these bots before.) I haven't used it for some time. There is also: http://www.chromium.org/developers/tree-sheriffs/perf-sheriffs/bisecting-perf...
On 2014/05/27 15:52:38, Torne wrote: > On 2014/05/27 15:48:25, pasko wrote: > > On 2014/05/27 15:22:08, Nico (traveling) wrote: > > > On Tue, May 27, 2014 at 4:41 PM, <mailto:pasko@chromium.org> wrote: > > > > > > > On 2014/05/27 13:46:20, Nico (traveling) wrote: > > > > > > > >> android_dbg looks happy, despite crbug.com/377715 ... > > > >> > > > > > > > > Do you think it's worth trying to land this and see how it fares? (I'm > > > >> > > > > guessing > > > > > > > >> it'll break all kinds of things.) > > > >> > > > > > > > > I think it makes sense to land if we want to collect many bugs quickly and > > > > be > > > > able to resolve them in parallel. Since benchmarks are our main > motivation, > > > > let's have some scores first. Without speedups, not clear why we need to > > > > immediately increase the complexity. > > > > > > > > > > Actually, my main motivation is to get all platforms on C++11. Right now, > > > it's a mix, and having C++11 on everywhere is supposed to _reduce_ > > > complexity :-) > > > > As far as I understand we still cannot move forward with _reducing_ complexity > > with switching Android build to C++11 until AOSP picks up GCC 4.8. > > The current AOSP master version already uses gcc 4.8 for target; the > android_aosp bot is just pinned to a release version that's too old, for now. > We're looking at updating the bot to use a newer manifest that is on gcc 4.8 and > if nothing goes wrong we should be able to have that done in the next ~week. I'm > following up with the build team about the status of updating the host toolchain > on android, but we only build v8 and protoc for host I think.. Thanks! your words supercharge me with optimism!
On 2014/05/27 15:58:30, pasko wrote: > On 2014/05/27 15:51:21, Nico (traveling) wrote: > > On Tue, May 27, 2014 at 5:48 PM, <mailto:pasko@chromium.org> wrote: > > > > > On 2014/05/27 15:22:08, Nico (traveling) wrote: > > > > > > On Tue, May 27, 2014 at 4:41 PM, <mailto:pasko@chromium.org> wrote: > > >> > > > > > > > On 2014/05/27 13:46:20, Nico (traveling) wrote: > > >> > > > >> >> android_dbg looks happy, despite crbug.com/377715 ... > > >> >> > > >> > > > >> > Do you think it's worth trying to land this and see how it fares? (I'm > > >> >> > > >> > guessing > > >> > > > >> >> it'll break all kinds of things.) > > >> >> > > >> > > > >> > I think it makes sense to land if we want to collect many bugs quickly > > >> and > > >> > be > > >> > able to resolve them in parallel. Since benchmarks are our main > > >> motivation, > > >> > let's have some scores first. Without speedups, not clear why we need to > > >> > immediately increase the complexity. > > >> > > > >> > > > > > > Actually, my main motivation is to get all platforms on C++11. Right now, > > >> it's a mix, and having C++11 on everywhere is supposed to _reduce_ > > >> complexity :-) > > >> > > > > > > As far as I understand we still cannot move forward with _reducing_ > > > complexity > > > with switching Android build to C++11 until AOSP picks up GCC 4.8. The > > > increased > > > complexity I meant is rather small: more conditionals in common.gypi, but > > > it > > > sounds perfectly fine to make the change for performance reasons. > > > > > > > > > Is there a good way to get perf numbers other than landing? I've been told > > >> that the easiest way to perf-test changes is to land them and then watch > > >> the perf waterfall. > > >> > > > > > > I consider experimenting on perf waterfall to be a *bad* practice because: > > > 1. we observe results on Android only a few days after landing the change > > > 2. we have too many people experimenting with perf like that, so the > > > change is > > > likely to collide with someone else's => makes analysis hard > > > 3. the change may mask a real unintentional regression > > > 4. the change increases the amount of alerts for perf sheriffs, these > > > folks are > > > severely overloaded already, which leads them to making mistakes and > > > missing > > > real regressions > > > > > > I would say, landing a CL after making *some* experiments locally on 1-2 > > > devices > > > is good (saves a lot of effort not to run everywhere), otherwise mere > > > experimentation on perf bots is icky. Many engineers do not share this > > > opinion. > > > Sadly. > > > > > > I realize running locally is a pain for you on a Mac. Options are: > > > * I can run some tests for you > > > * use perf trybot > > > > > > > If you have time, it'd be nice if you could run tests. > > I'm re-running thread_times.key_mobile_sites (appears that even with a crash we > get some results ...) > > > For perf trybots, is > > http://www.chromium.org/developers/performance-try-bots up-to-date (I > > never used these bots before.) > > I haven't used it for some time. There is also: > http://www.chromium.org/developers/tree-sheriffs/perf-sheriffs/bisecting-perf... FYI I tried to run several changes through the perf try jobs and never managed to get anything useful from them (either couldn't run the test and it wasn't clear what was wrong or I inputed the wrong metric to gather and couldn't find the name of the right one). pdr@ had more success than I had so they would be a good contact if you want to go that route.
On 2014/05/27 15:59:57, pasko wrote: > On 2014/05/27 15:52:38, Torne wrote: > > On 2014/05/27 15:48:25, pasko wrote: > > > On 2014/05/27 15:22:08, Nico (traveling) wrote: > > > > On Tue, May 27, 2014 at 4:41 PM, <mailto:pasko@chromium.org> wrote: > > > > > > > > > On 2014/05/27 13:46:20, Nico (traveling) wrote: > > > > > > > > > >> android_dbg looks happy, despite crbug.com/377715 ... > > > > >> > > > > > > > > > > Do you think it's worth trying to land this and see how it fares? (I'm > > > > >> > > > > > guessing > > > > > > > > > >> it'll break all kinds of things.) > > > > >> > > > > > > > > > > I think it makes sense to land if we want to collect many bugs quickly > and > > > > > be > > > > > able to resolve them in parallel. Since benchmarks are our main > > motivation, > > > > > let's have some scores first. Without speedups, not clear why we need to > > > > > immediately increase the complexity. > > > > > > > > > > > > > Actually, my main motivation is to get all platforms on C++11. Right now, > > > > it's a mix, and having C++11 on everywhere is supposed to _reduce_ > > > > complexity :-) > > > > > > As far as I understand we still cannot move forward with _reducing_ > complexity > > > with switching Android build to C++11 until AOSP picks up GCC 4.8. > > > > The current AOSP master version already uses gcc 4.8 for target; the > > android_aosp bot is just pinned to a release version that's too old, for now. > > We're looking at updating the bot to use a newer manifest that is on gcc 4.8 > and > > if nothing goes wrong we should be able to have that done in the next ~week. > I'm > > following up with the build team about the status of updating the host > toolchain > > on android, but we only build v8 and protoc for host I think.. > > Thanks! your words supercharge me with optimism! bad news: -std=gnu++11 is about 30% *slower* with thread_times: tools/perf/run_benchmark -v --browser=android-chrome thread_times.key_mobile_sites --pageset-repeat=10 The effect is similar on all 5 sub-benchmarks. A change of this magnitude should be easy to profile/analyse. To be fair there is a slight chance that I messed up, since I tested with a quick hack: --- a/build/common.gypi +++ b/build/common.gypi @@@ -3557,6 -3557,6 +3557,10 @@@ '-fno-caller-saves', '-Wno-psabi', ], ++ 'cflags_cc': [ ++ '-std=gnu++11', ++ '-w', ++ ], # Android now supports .relro sections properly. # NOTE: While these flags enable the generation of .relro # sections, the generated libraries can still be loaded on
Thanks for testing! I'll investigate this once I'm near a linux box. On Tue, May 27, 2014 at 8:06 PM, <pasko@chromium.org> wrote: > On 2014/05/27 15:59:57, pasko wrote: > >> On 2014/05/27 15:52:38, Torne wrote: >> > On 2014/05/27 15:48:25, pasko wrote: >> > > On 2014/05/27 15:22:08, Nico (traveling) wrote: >> > > > On Tue, May 27, 2014 at 4:41 PM, <mailto:pasko@chromium.org> wrote: >> > > > >> > > > > On 2014/05/27 13:46:20, Nico (traveling) wrote: >> > > > > >> > > > >> android_dbg looks happy, despite crbug.com/377715 ... >> > > > >> >> > > > > >> > > > > Do you think it's worth trying to land this and see how it fares? >> > (I'm > >> > > > >> >> > > > > guessing >> > > > > >> > > > >> it'll break all kinds of things.) >> > > > >> >> > > > > >> > > > > I think it makes sense to land if we want to collect many bugs >> quickly >> and >> > > > > be >> > > > > able to resolve them in parallel. Since benchmarks are our main >> > motivation, >> > > > > let's have some scores first. Without speedups, not clear why we >> need >> > to > >> > > > > immediately increase the complexity. >> > > > > >> > > > >> > > > Actually, my main motivation is to get all platforms on C++11. Right >> > now, > >> > > > it's a mix, and having C++11 on everywhere is supposed to _reduce_ >> > > > complexity :-) >> > > >> > > As far as I understand we still cannot move forward with _reducing_ >> complexity >> > > with switching Android build to C++11 until AOSP picks up GCC 4.8. >> > >> > The current AOSP master version already uses gcc 4.8 for target; the >> > android_aosp bot is just pinned to a release version that's too old, for >> > now. > >> > We're looking at updating the bot to use a newer manifest that is on >> gcc 4.8 >> and >> > if nothing goes wrong we should be able to have that done in the next >> ~week. >> I'm >> > following up with the build team about the status of updating the host >> toolchain >> > on android, but we only build v8 and protoc for host I think.. >> > > Thanks! your words supercharge me with optimism! >> > > bad news: -std=gnu++11 is about 30% *slower* with thread_times: > tools/perf/run_benchmark -v --browser=android-chrome > thread_times.key_mobile_sites --pageset-repeat=10 > > The effect is similar on all 5 sub-benchmarks. A change of this magnitude > should > be easy to profile/analyse. > > To be fair there is a slight chance that I messed up, since I tested with a > quick hack: > --- a/build/common.gypi > +++ b/build/common.gypi > @@@ -3557,6 -3557,6 +3557,10 @@@ > '-fno-caller-saves', > '-Wno-psabi', > ], > ++ 'cflags_cc': [ > ++ '-std=gnu++11', > ++ '-w', > ++ ], > # Android now supports .relro sections properly. > # NOTE: While these flags enable the generation of > .relro > # sections, the generated libraries can still be > loaded on > > > > https://codereview.chromium.org/304453005/ > To unsubscribe from this group and stop receiving emails from it, send an email to chromium-reviews+unsubscribe@chromium.org.
I'm trying to reproduce your results. I built ChromeShell with and without this change, pushed it to the device, and ran the command you mentioned (except with `--browser=android-chrome-shell`). I made a copy of the results.html produced by the first run, and now have two html files with results. How do I get to the "30%" number from there? The html file contains a long table with all kinds of metrics.
Oh, if I scroll far enough to the right, I see "2% better" next to one number, and "5% better" next to another number. All other numbers look identical. So I guess I can't repro the 30% slowdown (?) On Mon, Jun 23, 2014 at 1:20 PM, <thakis@chromium.org> wrote: > I'm trying to reproduce your results. I built ChromeShell with and without > this > change, pushed it to the device, and ran the command you mentioned (except > with > `--browser=android-chrome-shell`). I made a copy of the results.html > produced by > the first run, and now have two html files with results. > > How do I get to the "30%" number from there? The html file contains a long > table > with all kinds of metrics. > > https://codereview.chromium.org/304453005/ > To unsubscribe from this group and stop receiving emails from it, send an email to chromium-reviews+unsubscribe@chromium.org.
Sure! What could p-o-s-s-i-b-l-y go wrong. LGTM
The CQ bit was checked by thakis@chromium.org
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/thakis@chromium.org/304453005/180001
Message was sent while issue was closed.
Committed patchset #10 manually as r279220 (presubmit successful). |