Chromium Code Reviews
|
|
Chromium Code Reviews
Issue
17029013:
Added /LARGEADDRESSAWARE linker flag on Windows. (Closed)
|
Created:
7 years, 6 months ago by Ken Russell (switch to Gerrit) Modified:
7 years, 6 months ago Reviewers:
cpu_(ooo_6.6-7.5), rvargas (doing something else), jschuh, laforge, kerz_chromium, laforge CC:
chromium-reviews, apatrick_chromium, jbauman, Sigurður Ásgeirsson, chrisha, Roger McFarlane (Chromium) Base URL:
http://git.chromium.org/chromium/src.git@master Visibility:
Public. |
DescriptionAdded /LARGEADDRESSAWARE linker flag on Windows.
Allows Unreal Engine Epic Citadel demo transpiled to Emscripten to run in Chrome on 32-bit Windows.
BUG=239803
TEST=http://www.unrealengine.com/html5 on Windows
Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=206560
Patch Set 1 #
Messages
Total messages: 24 (0 generated)
Please review. Tested with the Epic Citadel demo; works. I'm not sure the flag is in the best place. Please suggest an alternative if not.
I'm not that familiar with the current structure of common.gypi, but it looks like the right place for the flag. I heard Justin said he was confident there were no issues with address handling bugs, but I have no idea if that's the case (I would not be surprised if there are issues that don't manifest on the 64-bit build), so you better have his official blessing. I would actually suggest to delay this to M30 so that it runs for a full cycle on dev. That said, lgtm.
lgtm
kbr, it would be nice if in 6 weeks or so you post the UMA memory usage numbers of that chrome versus 28, in the bug. I would like to have numbers post-facto to decide if we should keep this.
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/kbr@chromium.org/17029013/1
On 2013/06/14 22:53:58, cpu wrote: > kbr, it would be nice if in 6 weeks or so you post the UMA memory usage numbers > of that chrome versus 28, in the bug. > > I would like to have numbers post-facto to decide if we should keep this. @cpu: I'll make these measurements then.
On 2013/06/14 23:32:09, kbr wrote: > On 2013/06/14 22:53:58, cpu wrote: > > kbr, it would be nice if in 6 weeks or so you post the UMA memory usage > numbers > > of that chrome versus 28, in the bug. > > > > I would like to have numbers post-facto to decide if we should keep this. > > @cpu: I'll make these measurements then. +guys from the Syzygy team.
On 2013/06/14 20:04:45, rvargas wrote: > I'm not that familiar with the current structure of common.gypi, but it looks > like the right place for the flag. > > I heard Justin said he was confident there were no issues with address handling > bugs, but I have no idea if that's the case (I would not be surprised if there > are issues that don't manifest on the 64-bit build), so you better have his > official blessing. I would actually suggest to delay this to M30 so that it runs > for a full cycle on dev. Who said what now? I said I'm okay with the risk assuming we don't see issues in canary/dev/beta. That's not the same thing as "confident there were no issues." Anyway, lgtm. Let it ride and lets see what we get. > > That said, lgtm.
Message was sent while issue was closed.
Change committed as 206560
Message was sent while issue was closed.
On 2013/06/15 04:21:00, Justin Schuh wrote: > On 2013/06/14 20:04:45, rvargas wrote: > > I'm not that familiar with the current structure of common.gypi, but it looks > > like the right place for the flag. > > > > I heard Justin said he was confident there were no issues with address > handling > > bugs, but I have no idea if that's the case (I would not be surprised if there > > are issues that don't manifest on the 64-bit build), so you better have his > > official blessing. I would actually suggest to delay this to M30 so that it > runs > > for a full cycle on dev. > > Who said what now? I said I'm okay with the risk assuming we don't see issues in > canary/dev/beta. That's not the same thing as "confident there were no issues." > > Anyway, lgtm. Let it ride and lets see what we get. > > > > > That said, lgtm. This change also seemed to fix the crashing of LLJS demo's here: http://mbebenita.github.io/LLJS/ :) nice work!
This'll kill SyzyASAN, unless we make this ASAN-conditional, or bump the shadow memory back up. On Jun 15, 2013 1:02 AM, <commit-bot@chromium.org> wrote: > Change committed as 206560 > > https://chromiumcodereview.**appspot.com/17029013/<https://chromiumcodereview... >
+cc laforge When we dropped the shadow memory to 256MB rather than 512MB the OOM error rate dropped by a factor of 3. I'd be curious to see the OOM rate with LAA enabled and the shadow memory bumped back up to 512MB. It shouldn't be as high as before because each process will have twice the addressable memory space. Depending on the results of this we can either keep the shadow at 512MB, or disable LAA for SyzyASAN builds and drop our shadow memory back down. Seem reasonable? Chris On Sat, Jun 15, 2013 at 9:51 AM, Sigurður Ásgeirsson <siggi@chromium.org> wrote: > This'll kill SyzyASAN, unless we make this ASAN-conditional, or bump the > shadow memory back up. > > On Jun 15, 2013 1:02 AM, <commit-bot@chromium.org> wrote: >> >> Change committed as 206560 >> >> https://chromiumcodereview.appspot.com/17029013/ > > -- > You received this message because you are subscribed to the Google Groups > "syzygy-team" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to syzygy-team+unsubscribe@chromium.org. > For more options, visit > https://groups.google.com/a/chromium.org/groups/opt_out. > >
Could someone elaborate on why LAA will "kill" SyzyASAN, and more specifically why we shouldn't first validate the impact of LAA, in a real Canary, before reverting back to a state that was generally worse for the vast majority of our users (hence the 3X change in OOM). Kind Regards, Anthony Laforge Technical Program Manager Mountain View, CA On Mon, Jun 17, 2013 at 7:36 AM, Chris Hamilton <chrisha@chromium.org>wrote: > +cc laforge > > When we dropped the shadow memory to 256MB rather than 512MB the OOM > error rate dropped by a factor of 3. I'd be curious to see the OOM > rate with LAA enabled and the shadow memory bumped back up to 512MB. > It shouldn't be as high as before because each process will have twice > the addressable memory space. Depending on the results of this we can > either keep the shadow at 512MB, or disable LAA for SyzyASAN builds > and drop our shadow memory back down. Seem reasonable? > > Chris > > On Sat, Jun 15, 2013 at 9:51 AM, Sigurður Ásgeirsson <siggi@chromium.org> > wrote: > > This'll kill SyzyASAN, unless we make this ASAN-conditional, or bump the > > shadow memory back up. > > > > On Jun 15, 2013 1:02 AM, <commit-bot@chromium.org> wrote: > >> > >> Change committed as 206560 > >> > >> https://chromiumcodereview.appspot.com/17029013/ > > > > -- > > You received this message because you are subscribed to the Google Groups > > "syzygy-team" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to syzygy-team+unsubscribe@chromium.org. > > For more options, visit > > https://groups.google.com/a/chromium.org/groups/opt_out. > > > > >
One of the first memory optimizations was to drop the shadow memory size by a factor of two, exploiting the fact that Chrome wasn't /LARGEADDRESSAWARE, and so there would be no valid memory addresses over the 2G threshold. In a process with a /LARGEADDRESSAWARE, ASAN will assert immediately on loading, hence "kill". On Mon, Jun 17, 2013 at 11:42 AM, Anthony LaForge <laforge@google.com>wrote: > Could someone elaborate on why LAA will "kill" SyzyASAN, and more > specifically why we shouldn't first validate the impact of LAA, in a real > Canary, before reverting back to a state that was generally worse for the > vast majority of our users (hence the 3X change in OOM). > > Kind Regards, > > Anthony Laforge > Technical Program Manager > Mountain View, CA > > > On Mon, Jun 17, 2013 at 7:36 AM, Chris Hamilton <chrisha@chromium.org>wrote: > >> +cc laforge >> >> When we dropped the shadow memory to 256MB rather than 512MB the OOM >> error rate dropped by a factor of 3. I'd be curious to see the OOM >> rate with LAA enabled and the shadow memory bumped back up to 512MB. >> It shouldn't be as high as before because each process will have twice >> the addressable memory space. Depending on the results of this we can >> either keep the shadow at 512MB, or disable LAA for SyzyASAN builds >> and drop our shadow memory back down. Seem reasonable? >> >> Chris >> >> On Sat, Jun 15, 2013 at 9:51 AM, Sigurður Ásgeirsson <siggi@chromium.org> >> wrote: >> > This'll kill SyzyASAN, unless we make this ASAN-conditional, or bump the >> > shadow memory back up. >> > >> > On Jun 15, 2013 1:02 AM, <commit-bot@chromium.org> wrote: >> >> >> >> Change committed as 206560 >> >> >> >> https://chromiumcodereview.appspot.com/17029013/ >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups >> > "syzygy-team" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an >> > email to syzygy-team+unsubscribe@chromium.org. >> > For more options, visit >> > https://groups.google.com/a/chromium.org/groups/opt_out. >> > >> > >> > >
On Mon, Jun 17, 2013 at 10:36 AM, Chris Hamilton <chrisha@chromium.org>wrote: > +cc laforge > > When we dropped the shadow memory to 256MB rather than 512MB the OOM > error rate dropped by a factor of 3. I'd be curious to see the OOM > rate with LAA enabled and the shadow memory bumped back up to 512MB. > It shouldn't be as high as before because each process will have twice > the addressable memory space. Depending on the results of this we can > either keep the shadow at 512MB, or disable LAA for SyzyASAN builds > and drop our shadow memory back down. Seem reasonable? > It depends. In practice, I think only users running 64 bit OSen will enjoy any benefit from /LARGEADDRESSAWARE, and the rest will be left with a 256Mb less available memory of their 32 bit address space. I don't know the fraction of users running 64 bit OSEn. > > Chris > > On Sat, Jun 15, 2013 at 9:51 AM, Sigurður Ásgeirsson <siggi@chromium.org> > wrote: > > This'll kill SyzyASAN, unless we make this ASAN-conditional, or bump the > > shadow memory back up. > > > > On Jun 15, 2013 1:02 AM, <commit-bot@chromium.org> wrote: > >> > >> Change committed as 206560 > >> > >> https://chromiumcodereview.appspot.com/17029013/ > > > > -- > > You received this message because you are subscribed to the Google Groups > > "syzygy-team" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to syzygy-team+unsubscribe@chromium.org. > > For more options, visit > > https://groups.google.com/a/chromium.org/groups/opt_out. > > > > >
Sorry for not being more clear to begin with. SyzyASAN currently refuses to run with binaries that are LAA because we hard code the shadow memory to 256MB (1/8th of the 2GB addressable memory of a non-LAA binary). This is a hard check in the code itself and the process will will simply die when loading the ASAN RTL into binaries that are LAA. We did this because Chrome was not previously LAA and this cut the size of the shadow memory in half, vastly reducing the OOM error rate on the canary channel. LAA was recently enabled for Chrome on trunk, thus we have two options: (1) Make SyzyASAN LAA as well, and double the shadow memory size. (2) Disable LAA for SyzyASAN builds. On 32-bit systems LAA apparently can give a process up to 3GB of addressable space, or 2.5GB after shadow memory. On 64-bit systems they can get the full 4GB, or 3.5GB after shadow. If the OOM was due to the process running out of addressable space then doubling the shadow memory and enabling LAA should likely be a wash. If the memory pressure is more due to actually running out of memory due to the large shadow memory, then the OOM will increase. I was thinking it might be worthwhile to do (1) in order to find out. If things are markedly worse we can go back to approach (2). Or, if opinions are strong we can simply do (2) to begin with. Chris On Mon, Jun 17, 2013 at 11:42 AM, Anthony LaForge <laforge@google.com> wrote: > Could someone elaborate on why LAA will "kill" SyzyASAN, and more > specifically why we shouldn't first validate the impact of LAA, in a real > Canary, before reverting back to a state that was generally worse for the > vast majority of our users (hence the 3X change in OOM). > > Kind Regards, > > Anthony Laforge > Technical Program Manager > Mountain View, CA > > > On Mon, Jun 17, 2013 at 7:36 AM, Chris Hamilton <chrisha@chromium.org> > wrote: >> >> +cc laforge >> >> When we dropped the shadow memory to 256MB rather than 512MB the OOM >> error rate dropped by a factor of 3. I'd be curious to see the OOM >> rate with LAA enabled and the shadow memory bumped back up to 512MB. >> It shouldn't be as high as before because each process will have twice >> the addressable memory space. Depending on the results of this we can >> either keep the shadow at 512MB, or disable LAA for SyzyASAN builds >> and drop our shadow memory back down. Seem reasonable? >> >> Chris >> >> On Sat, Jun 15, 2013 at 9:51 AM, Sigurður Ásgeirsson <siggi@chromium.org> >> wrote: >> > This'll kill SyzyASAN, unless we make this ASAN-conditional, or bump the >> > shadow memory back up. >> > >> > On Jun 15, 2013 1:02 AM, <commit-bot@chromium.org> wrote: >> >> >> >> Change committed as 206560 >> >> >> >> https://chromiumcodereview.appspot.com/17029013/ >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "syzygy-team" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an >> > email to syzygy-team+unsubscribe@chromium.org. >> > For more options, visit >> > https://groups.google.com/a/chromium.org/groups/opt_out. >> > >> > > > > -- > You received this message because you are subscribed to the Google Groups > "syzygy-team" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to syzygy-team+unsubscribe@chromium.org. > For more options, visit > https://groups.google.com/a/chromium.org/groups/opt_out. > >
Message was sent while issue was closed.
On 2013/06/15 04:21:00, Justin Schuh wrote: > On 2013/06/14 20:04:45, rvargas wrote: > > I'm not that familiar with the current structure of common.gypi, but it looks > > like the right place for the flag. > > > > I heard Justin said he was confident there were no issues with address > handling > > bugs, but I have no idea if that's the case (I would not be surprised if there > > are issues that don't manifest on the 64-bit build), so you better have his > > official blessing. I would actually suggest to delay this to M30 so that it > runs > > for a full cycle on dev. > > Who said what now? I said I'm okay with the risk assuming we don't see issues in > canary/dev/beta. Interesting wording ;-). Shouldn't then this bake longer on Dev before it goes to Beta?
The breakdown on Canary is 53% x64 and 47% x86. That being said, for SyzyASAN, my preference would be to start w/ LAA off for the immediate future until we have general consensus that LAA doesn't cause other stability problems. In that vein, adding Jason (who owns 29)...If we aren't highly confident in the stability implications of LAA, I'd rather not incur the costs of stabilization throughout the M29 life cycle (i.e. if there is any hint of badness, it comes off in Dev and doesn't go further). Kind Regards, Anthony Laforge Technical Program Manager Mountain View, CA On Mon, Jun 17, 2013 at 11:20 AM, <rvargas@chromium.org> wrote: > On 2013/06/15 04:21:00, Justin Schuh wrote: > >> On 2013/06/14 20:04:45, rvargas wrote: >> > I'm not that familiar with the current structure of common.gypi, but it >> > looks > >> > like the right place for the flag. >> > >> > I heard Justin said he was confident there were no issues with address >> handling >> > bugs, but I have no idea if that's the case (I would not be surprised if >> > there > >> > are issues that don't manifest on the 64-bit build), so you better have >> his >> > official blessing. I would actually suggest to delay this to M30 so >> that it >> runs >> > for a full cycle on dev. >> > > Who said what now? I said I'm okay with the risk assuming we don't see >> issues >> > in > >> canary/dev/beta. >> > > Interesting wording ;-). Shouldn't then this bake longer on Dev before it > goes > to Beta? > > https://codereview.chromium.**org/17029013/<https://codereview.chromium.org/1... >
If there's an ounce of risk, I think we should put to 30. We're already struggling with stabilizing without adding a new variable like this to the mix. What's the level of urgency for this? On Mon, Jun 17, 2013 at 11:30 AM, Anthony LaForge <laforge@google.com>wrote: > The breakdown on Canary is 53% x64 and 47% x86. That being said, for > SyzyASAN, my preference would be to start w/ LAA off for the immediate > future until we have general consensus that LAA doesn't cause other > stability problems. > > In that vein, adding Jason (who owns 29)...If we aren't highly confident > in the stability implications of LAA, I'd rather not incur the costs of > stabilization throughout the M29 life cycle (i.e. if there is any hint of > badness, it comes off in Dev and doesn't go further). > > Kind Regards, > > Anthony Laforge > Technical Program Manager > Mountain View, CA > > > On Mon, Jun 17, 2013 at 11:20 AM, <rvargas@chromium.org> wrote: > >> On 2013/06/15 04:21:00, Justin Schuh wrote: >> >>> On 2013/06/14 20:04:45, rvargas wrote: >>> > I'm not that familiar with the current structure of common.gypi, but it >>> >> looks >> >>> > like the right place for the flag. >>> > >>> > I heard Justin said he was confident there were no issues with address >>> handling >>> > bugs, but I have no idea if that's the case (I would not be surprised >>> if >>> >> there >> >>> > are issues that don't manifest on the 64-bit build), so you better >>> have his >>> > official blessing. I would actually suggest to delay this to M30 so >>> that it >>> runs >>> > for a full cycle on dev. >>> >> >> Who said what now? I said I'm okay with the risk assuming we don't see >>> issues >>> >> in >> >>> canary/dev/beta. >>> >> >> Interesting wording ;-). Shouldn't then this bake longer on Dev before it >> goes >> to Beta? >> >> https://codereview.chromium.**org/17029013/<https://codereview.chromium.org/1... >> > >
Message was sent while issue was closed.
On 2013/06/17 18:37:18, kerz_chromium wrote: > If there's an ounce of risk, I think we should put to 30. We're already > struggling with stabilizing without adding a new variable like this to the > mix. What's the level of urgency for this? This patch enables one of the key Emscripten-compiled demonstrations from Epic Games and Mozilla to run in Chrome on Windows. From the standpoint of demonstrating that Chrome supports leading-edge technologies and features I think it is important to turn on this flag. I personally doubt that there is any stability risk. It is a fact that allocations would be able to cross the 0x7FFFFFFF -> 0x80000000 boundary for the first time on Windows, but this may have been possible on other 32-bit platforms for some time now, so I don't think any new behaviors or bugs in pointer arithmetic will be exposed. Still, we should closely watch the canaries over the next week or so to confirm. We really should have this conversation on http://code.google.com/p/chromium/issues/detail?id=239803 , a more persistent location than this CL, if we intend to have a more in-depth conversation about this change.
On Mon, Jun 17, 2013 at 11:54 AM, Chris Hamilton <chrisha@chromium.org>wrote: > Sorry for not being more clear to begin with. > > SyzyASAN currently refuses to run with binaries that are LAA because > we hard code the shadow memory to 256MB (1/8th of the 2GB addressable > memory of a non-LAA binary). This is a hard check in the code itself > and the process will will simply die when loading the ASAN RTL into > binaries that are LAA. We did this because Chrome was not previously > LAA and this cut the size of the shadow memory in half, vastly > reducing the OOM error rate on the canary channel. > > LAA was recently enabled for Chrome on trunk, thus we have two options: > > (1) Make SyzyASAN LAA as well, and double the shadow memory size. > (2) Disable LAA for SyzyASAN builds. > > On 32-bit systems LAA apparently can give a process up to 3GB of > addressable space, or 2.5GB after shadow memory. 32 bit Windows will only do this if you boot with a flag that's non-default: http://msdn.microsoft.com/en-us/library/windows/hardware/ff556232(v=vs.85).aspx. If this non-default boot-time flag is not set, then LAA will have no effect on a 32 bit system. So in practice, users on 32 bit systems will have 2G-512Mb, save for the three to seven supernerds that have turned this on :). On 64-bit systems > they can get the full 4GB, or 3.5GB after shadow. If the OOM was due > to the process running out of addressable space then doubling the > shadow memory and enabling LAA should likely be a wash. If the memory > pressure is more due to actually running out of memory due to the > large shadow memory, then the OOM will increase. > > I was thinking it might be worthwhile to do (1) in order to find out. > If things are markedly worse we can go back to approach (2). Or, if > opinions are strong we can simply do (2) to begin with. > > Chris > > On Mon, Jun 17, 2013 at 11:42 AM, Anthony LaForge <laforge@google.com> > wrote: > > Could someone elaborate on why LAA will "kill" SyzyASAN, and more > > specifically why we shouldn't first validate the impact of LAA, in a real > > Canary, before reverting back to a state that was generally worse for the > > vast majority of our users (hence the 3X change in OOM). > > > > Kind Regards, > > > > Anthony Laforge > > Technical Program Manager > > Mountain View, CA > > > > > > On Mon, Jun 17, 2013 at 7:36 AM, Chris Hamilton <chrisha@chromium.org> > > wrote: > >> > >> +cc laforge > >> > >> When we dropped the shadow memory to 256MB rather than 512MB the OOM > >> error rate dropped by a factor of 3. I'd be curious to see the OOM > >> rate with LAA enabled and the shadow memory bumped back up to 512MB. > >> It shouldn't be as high as before because each process will have twice > >> the addressable memory space. Depending on the results of this we can > >> either keep the shadow at 512MB, or disable LAA for SyzyASAN builds > >> and drop our shadow memory back down. Seem reasonable? > >> > >> Chris > >> > >> On Sat, Jun 15, 2013 at 9:51 AM, Sigurður Ásgeirsson < > siggi@chromium.org> > >> wrote: > >> > This'll kill SyzyASAN, unless we make this ASAN-conditional, or bump > the > >> > shadow memory back up. > >> > > >> > On Jun 15, 2013 1:02 AM, <commit-bot@chromium.org> wrote: > >> >> > >> >> Change committed as 206560 > >> >> > >> >> https://chromiumcodereview.appspot.com/17029013/ > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "syzygy-team" group. > >> > To unsubscribe from this group and stop receiving emails from it, send > >> > an > >> > email to syzygy-team+unsubscribe@chromium.org. > >> > For more options, visit > >> > https://groups.google.com/a/chromium.org/groups/opt_out. > >> > > >> > > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "syzygy-team" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to syzygy-team+unsubscribe@chromium.org. > > For more options, visit > > https://groups.google.com/a/chromium.org/groups/opt_out. > > > > >
Okay... I'll make this conditional on the 'asan' GYP variable for now, reverting to non-LAA for ASAN builds. On Mon, Jun 17, 2013 at 2:59 PM, Sigurður Ásgeirsson <siggi@chromium.org> wrote: > On Mon, Jun 17, 2013 at 11:54 AM, Chris Hamilton <chrisha@chromium.org> > wrote: >> >> Sorry for not being more clear to begin with. >> >> SyzyASAN currently refuses to run with binaries that are LAA because >> we hard code the shadow memory to 256MB (1/8th of the 2GB addressable >> memory of a non-LAA binary). This is a hard check in the code itself >> and the process will will simply die when loading the ASAN RTL into >> binaries that are LAA. We did this because Chrome was not previously >> LAA and this cut the size of the shadow memory in half, vastly >> reducing the OOM error rate on the canary channel. >> >> LAA was recently enabled for Chrome on trunk, thus we have two options: >> >> (1) Make SyzyASAN LAA as well, and double the shadow memory size. >> (2) Disable LAA for SyzyASAN builds. >> >> On 32-bit systems LAA apparently can give a process up to 3GB of >> addressable space, or 2.5GB after shadow memory. > > > 32 bit Windows will only do this if you boot with a flag that's non-default: > http://msdn.microsoft.com/en-us/library/windows/hardware/ff556232(v=vs.85).aspx. > If this non-default boot-time flag is not set, then LAA will have no effect > on a 32 bit system. > So in practice, users on 32 bit systems will have 2G-512Mb, save for the > three to seven supernerds that have turned this on :). > >> On 64-bit systems >> they can get the full 4GB, or 3.5GB after shadow. If the OOM was due >> to the process running out of addressable space then doubling the >> shadow memory and enabling LAA should likely be a wash. If the memory >> pressure is more due to actually running out of memory due to the >> large shadow memory, then the OOM will increase. >> >> I was thinking it might be worthwhile to do (1) in order to find out. >> If things are markedly worse we can go back to approach (2). Or, if >> opinions are strong we can simply do (2) to begin with. >> >> Chris >> >> On Mon, Jun 17, 2013 at 11:42 AM, Anthony LaForge <laforge@google.com> >> wrote: >> > Could someone elaborate on why LAA will "kill" SyzyASAN, and more >> > specifically why we shouldn't first validate the impact of LAA, in a >> > real >> > Canary, before reverting back to a state that was generally worse for >> > the >> > vast majority of our users (hence the 3X change in OOM). >> > >> > Kind Regards, >> > >> > Anthony Laforge >> > Technical Program Manager >> > Mountain View, CA >> > >> > >> > On Mon, Jun 17, 2013 at 7:36 AM, Chris Hamilton <chrisha@chromium.org> >> > wrote: >> >> >> >> +cc laforge >> >> >> >> When we dropped the shadow memory to 256MB rather than 512MB the OOM >> >> error rate dropped by a factor of 3. I'd be curious to see the OOM >> >> rate with LAA enabled and the shadow memory bumped back up to 512MB. >> >> It shouldn't be as high as before because each process will have twice >> >> the addressable memory space. Depending on the results of this we can >> >> either keep the shadow at 512MB, or disable LAA for SyzyASAN builds >> >> and drop our shadow memory back down. Seem reasonable? >> >> >> >> Chris >> >> >> >> On Sat, Jun 15, 2013 at 9:51 AM, Sigurður Ásgeirsson >> >> <siggi@chromium.org> >> >> wrote: >> >> > This'll kill SyzyASAN, unless we make this ASAN-conditional, or bump >> >> > the >> >> > shadow memory back up. >> >> > >> >> > On Jun 15, 2013 1:02 AM, <commit-bot@chromium.org> wrote: >> >> >> >> >> >> Change committed as 206560 >> >> >> >> >> >> https://chromiumcodereview.appspot.com/17029013/ >> >> > >> >> > -- >> >> > You received this message because you are subscribed to the Google >> >> > Groups >> >> > "syzygy-team" group. >> >> > To unsubscribe from this group and stop receiving emails from it, >> >> > send >> >> > an >> >> > email to syzygy-team+unsubscribe@chromium.org. >> >> > For more options, visit >> >> > https://groups.google.com/a/chromium.org/groups/opt_out. >> >> > >> >> > >> > >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "syzygy-team" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an >> > email to syzygy-team+unsubscribe@chromium.org. >> > For more options, visit >> > https://groups.google.com/a/chromium.org/groups/opt_out. >> > >> > > > > -- > You received this message because you are subscribed to the Google Groups > "syzygy-team" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to syzygy-team+unsubscribe@chromium.org. > For more options, visit > https://groups.google.com/a/chromium.org/groups/opt_out. > >
This is being addressed here: https://codereview.chromium.org/17348007/ On Mon, Jun 17, 2013 at 4:29 PM, Chris Hamilton <chrisha@chromium.org> wrote: > Okay... I'll make this conditional on the 'asan' GYP variable for now, > reverting to non-LAA for ASAN builds. > > On Mon, Jun 17, 2013 at 2:59 PM, Sigurður Ásgeirsson <siggi@chromium.org> wrote: >> On Mon, Jun 17, 2013 at 11:54 AM, Chris Hamilton <chrisha@chromium.org> >> wrote: >>> >>> Sorry for not being more clear to begin with. >>> >>> SyzyASAN currently refuses to run with binaries that are LAA because >>> we hard code the shadow memory to 256MB (1/8th of the 2GB addressable >>> memory of a non-LAA binary). This is a hard check in the code itself >>> and the process will will simply die when loading the ASAN RTL into >>> binaries that are LAA. We did this because Chrome was not previously >>> LAA and this cut the size of the shadow memory in half, vastly >>> reducing the OOM error rate on the canary channel. >>> >>> LAA was recently enabled for Chrome on trunk, thus we have two options: >>> >>> (1) Make SyzyASAN LAA as well, and double the shadow memory size. >>> (2) Disable LAA for SyzyASAN builds. >>> >>> On 32-bit systems LAA apparently can give a process up to 3GB of >>> addressable space, or 2.5GB after shadow memory. >> >> >> 32 bit Windows will only do this if you boot with a flag that's non-default: >> http://msdn.microsoft.com/en-us/library/windows/hardware/ff556232(v=vs.85).aspx. >> If this non-default boot-time flag is not set, then LAA will have no effect >> on a 32 bit system. >> So in practice, users on 32 bit systems will have 2G-512Mb, save for the >> three to seven supernerds that have turned this on :). >> >>> On 64-bit systems >>> they can get the full 4GB, or 3.5GB after shadow. If the OOM was due >>> to the process running out of addressable space then doubling the >>> shadow memory and enabling LAA should likely be a wash. If the memory >>> pressure is more due to actually running out of memory due to the >>> large shadow memory, then the OOM will increase. >>> >>> I was thinking it might be worthwhile to do (1) in order to find out. >>> If things are markedly worse we can go back to approach (2). Or, if >>> opinions are strong we can simply do (2) to begin with. >>> >>> Chris >>> >>> On Mon, Jun 17, 2013 at 11:42 AM, Anthony LaForge <laforge@google.com> >>> wrote: >>> > Could someone elaborate on why LAA will "kill" SyzyASAN, and more >>> > specifically why we shouldn't first validate the impact of LAA, in a >>> > real >>> > Canary, before reverting back to a state that was generally worse for >>> > the >>> > vast majority of our users (hence the 3X change in OOM). >>> > >>> > Kind Regards, >>> > >>> > Anthony Laforge >>> > Technical Program Manager >>> > Mountain View, CA >>> > >>> > >>> > On Mon, Jun 17, 2013 at 7:36 AM, Chris Hamilton <chrisha@chromium.org> >>> > wrote: >>> >> >>> >> +cc laforge >>> >> >>> >> When we dropped the shadow memory to 256MB rather than 512MB the OOM >>> >> error rate dropped by a factor of 3. I'd be curious to see the OOM >>> >> rate with LAA enabled and the shadow memory bumped back up to 512MB. >>> >> It shouldn't be as high as before because each process will have twice >>> >> the addressable memory space. Depending on the results of this we can >>> >> either keep the shadow at 512MB, or disable LAA for SyzyASAN builds >>> >> and drop our shadow memory back down. Seem reasonable? >>> >> >>> >> Chris >>> >> >>> >> On Sat, Jun 15, 2013 at 9:51 AM, Sigurður Ásgeirsson >>> >> <siggi@chromium.org> >>> >> wrote: >>> >> > This'll kill SyzyASAN, unless we make this ASAN-conditional, or bump >>> >> > the >>> >> > shadow memory back up. >>> >> > >>> >> > On Jun 15, 2013 1:02 AM, <commit-bot@chromium.org> wrote: >>> >> >> >>> >> >> Change committed as 206560 >>> >> >> >>> >> >> https://chromiumcodereview.appspot.com/17029013/ >>> >> > >>> >> > -- >>> >> > You received this message because you are subscribed to the Google >>> >> > Groups >>> >> > "syzygy-team" group. >>> >> > To unsubscribe from this group and stop receiving emails from it, >>> >> > send >>> >> > an >>> >> > email to syzygy-team+unsubscribe@chromium.org. >>> >> > For more options, visit >>> >> > https://groups.google.com/a/chromium.org/groups/opt_out. >>> >> > >>> >> > >>> > >>> > >>> > -- >>> > You received this message because you are subscribed to the Google >>> > Groups >>> > "syzygy-team" group. >>> > To unsubscribe from this group and stop receiving emails from it, send >>> > an >>> > email to syzygy-team+unsubscribe@chromium.org. >>> > For more options, visit >>> > https://groups.google.com/a/chromium.org/groups/opt_out. >>> > >>> > >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "syzygy-team" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to syzygy-team+unsubscribe@chromium.org. >> For more options, visit >> https://groups.google.com/a/chromium.org/groups/opt_out. >> >>
Message was sent while issue was closed.
On 2013/06/18 21:45:02, chrisha wrote: > This is being addressed here: https://codereview.chromium.org/17348007/ > > On Mon, Jun 17, 2013 at 4:29 PM, Chris Hamilton <mailto:chrisha@chromium.org> wrote: > > Okay... I'll make this conditional on the 'asan' GYP variable for now, > > reverting to non-LAA for ASAN builds. > > > > On Mon, Jun 17, 2013 at 2:59 PM, Sigurður Ásgeirsson <mailto:siggi@chromium.org> > wrote: > >> On Mon, Jun 17, 2013 at 11:54 AM, Chris Hamilton <mailto:chrisha@chromium.org> > >> wrote: > >>> > >>> Sorry for not being more clear to begin with. > >>> > >>> SyzyASAN currently refuses to run with binaries that are LAA because > >>> we hard code the shadow memory to 256MB (1/8th of the 2GB addressable > >>> memory of a non-LAA binary). This is a hard check in the code itself > >>> and the process will will simply die when loading the ASAN RTL into > >>> binaries that are LAA. We did this because Chrome was not previously > >>> LAA and this cut the size of the shadow memory in half, vastly > >>> reducing the OOM error rate on the canary channel. > >>> > >>> LAA was recently enabled for Chrome on trunk, thus we have two options: > >>> > >>> (1) Make SyzyASAN LAA as well, and double the shadow memory size. > >>> (2) Disable LAA for SyzyASAN builds. > >>> > >>> On 32-bit systems LAA apparently can give a process up to 3GB of > >>> addressable space, or 2.5GB after shadow memory. > >> > >> > >> 32 bit Windows will only do this if you boot with a flag that's non-default: > >> > http://msdn.microsoft.com/en-us/library/windows/hardware/ff556232%28v=vs.85%2.... > >> If this non-default boot-time flag is not set, then LAA will have no effect > >> on a 32 bit system. > >> So in practice, users on 32 bit systems will have 2G-512Mb, save for the > >> three to seven supernerds that have turned this on :). > >> > >>> On 64-bit systems > >>> they can get the full 4GB, or 3.5GB after shadow. If the OOM was due > >>> to the process running out of addressable space then doubling the > >>> shadow memory and enabling LAA should likely be a wash. If the memory > >>> pressure is more due to actually running out of memory due to the > >>> large shadow memory, then the OOM will increase. > >>> > >>> I was thinking it might be worthwhile to do (1) in order to find out. > >>> If things are markedly worse we can go back to approach (2). Or, if > >>> opinions are strong we can simply do (2) to begin with. > >>> > >>> Chris > >>> > >>> On Mon, Jun 17, 2013 at 11:42 AM, Anthony LaForge <mailto:laforge@google.com> > >>> wrote: > >>> > Could someone elaborate on why LAA will "kill" SyzyASAN, and more > >>> > specifically why we shouldn't first validate the impact of LAA, in a > >>> > real > >>> > Canary, before reverting back to a state that was generally worse for > >>> > the > >>> > vast majority of our users (hence the 3X change in OOM). > >>> > > >>> > Kind Regards, > >>> > > >>> > Anthony Laforge > >>> > Technical Program Manager > >>> > Mountain View, CA > >>> > > >>> > > >>> > On Mon, Jun 17, 2013 at 7:36 AM, Chris Hamilton <mailto:chrisha@chromium.org> > >>> > wrote: > >>> >> > >>> >> +cc laforge > >>> >> > >>> >> When we dropped the shadow memory to 256MB rather than 512MB the OOM > >>> >> error rate dropped by a factor of 3. I'd be curious to see the OOM > >>> >> rate with LAA enabled and the shadow memory bumped back up to 512MB. > >>> >> It shouldn't be as high as before because each process will have twice > >>> >> the addressable memory space. Depending on the results of this we can > >>> >> either keep the shadow at 512MB, or disable LAA for SyzyASAN builds > >>> >> and drop our shadow memory back down. Seem reasonable? > >>> >> > >>> >> Chris > >>> >> > >>> >> On Sat, Jun 15, 2013 at 9:51 AM, Sigurður Ásgeirsson > >>> >> <mailto:siggi@chromium.org> > >>> >> wrote: > >>> >> > This'll kill SyzyASAN, unless we make this ASAN-conditional, or bump > >>> >> > the > >>> >> > shadow memory back up. > >>> >> > > >>> >> > On Jun 15, 2013 1:02 AM, <mailto:commit-bot@chromium.org> wrote: > >>> >> >> > >>> >> >> Change committed as 206560 > >>> >> >> > >>> >> >> https://chromiumcodereview.appspot.com/17029013/ > >>> >> > > >>> >> > -- > >>> >> > You received this message because you are subscribed to the Google > >>> >> > Groups > >>> >> > "syzygy-team" group. > >>> >> > To unsubscribe from this group and stop receiving emails from it, > >>> >> > send > >>> >> > an > >>> >> > email to mailto:syzygy-team+unsubscribe@chromium.org. > >>> >> > For more options, visit > >>> >> > https://groups.google.com/a/chromium.org/groups/opt_out. > >>> >> > > >>> >> > > >>> > > >>> > > >>> > -- > >>> > You received this message because you are subscribed to the Google > >>> > Groups > >>> > "syzygy-team" group. > >>> > To unsubscribe from this group and stop receiving emails from it, send > >>> > an > >>> > email to mailto:syzygy-team+unsubscribe@chromium.org. > >>> > For more options, visit > >>> > https://groups.google.com/a/chromium.org/groups/opt_out. > >>> > > >>> > > >> > >> > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "syzygy-team" group. > >> To unsubscribe from this group and stop receiving emails from it, send an > >> email to mailto:syzygy-team+unsubscribe@chromium.org. > >> For more options, visit > >> https://groups.google.com/a/chromium.org/groups/opt_out. > >> > >> LGTM |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
