Update the tcmalloc chromium branch to r144 (gperftools 2.0), and merge chromium-specific changes.

Update the tcmalloc chromium branch to r144 (gperftools 2.0), and merge chromium-specific changes.

This change is retry of r124832.  The original review discussion is at http://codereview.chromium.org/9311003/.

The major reason for us is to enable a fix for HEAP_PROFILE_MMAP.  The change is created by 3-way merge from 1) the original google-perftools r109 ( = vendor base), 2) the original gperftools 2.0  r144 ( = branch), and 3) the chromium branch ( = another branch) with the following additional changes :
* base/allocator/allocator.gyp is modified.
* Many [#include "third_party/tcmalloc/chromium/src/google/..."] are replaced with "gperftools/".  (Many files in Chromium)
* gperftools/tcmalloc.h (formerly google/tcmalloc.h) is replaced with the original (generated) one.
* windows/gperftools/tcmalloc.h (formerly windows/google/tcmalloc.h) is replaced with the original (generated) one.
* malloc_hook-like functions are moved to libc_override*.h in gperftools 2.0.  Some changes due to it.
* MALLOC_HOOK_MAYBE_VOLATILE is redefined using __MALLOC_HOOK_VOLATILE.  (config.h, tcmalloc.cc and libc_override_glibc.h)
* The macro "CRASH(...)" is replaced with "Log(kCrash, __FILE__, __LINE__, ...)".  (Many files)
* LARGE_PAGE-related parameters (which may affect performance?) are merged.  (common.h)
* RAW_VLOG() calls are removed.  (base/googleinit.h)
* sys_{mmap|munmap|mremap}(...) calls are tentatively replaced with syscall(SYS_{mmap|munmap|mremap}, ...).  (malloc_hook_mmap_linux.h)
* tc_mallinfo is declared only when HAVE_STRUCT_MALLINFO is defined.  (gperftools/tcmalloc.h)
* "libc_override_redefine.h" is not included in Windows.  (libc_override.h)
* Chromium-original "sys_alloc" is not declared.  (windows/port.cc)
* base/spinlock_win32-inl.h is reverted from r144 because 64-bit atomicops are not implemented on Windows.  (base/atomicops-internals-windows.h)

The vendor branch is updated in another change.


BUG=114302
TEST=run all existing tests.

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=126412

[Dai Mikurube (NOT FULLTIME), 2012-03-03 08:47:43]

John(jam), Tony(tony), Erik(arv), Steven(stevenjb)

I wonder if you could PTAL at changes in
- content/* (jam)
- webkit/extensions/v8/* (tony)
- chrome/browser/ui/webui/* (arv)
- chrome/browser/chromeos/status/* (stevenjb)
as you're OWNERs.  (Sorry for asking again, John and Tony.)

It changes only #include directories from google/ to gperftools/.  (Please see
http://code.google.com/p/gperftools/.)



Jim,

I tried performance experiments as follows.  Overall performance looks like
having no problems as links attached at bottom.

(I forgot to commit the update in another review page, so I created this new
one.)

In fact, dromaeo_jslib_test looks getting slow, but dromaeo_jslib_test tends to
get slower as r124830 (two revisions before committing the TCMalloc update) is
red, too.
http://build.chromium.org/p/chromium.perf/builders/Linux%20Perf%20%282%29/bui...

A problem might be failures in "sizes".  Static initializers look growing, but I
think we can't avoid it even if the TCMalloc causes them...
- http://build.chromium.org/p/chromium/buildstatus?builder=Linux&number=20880
-
http://build.chromium.org/p/chromium/buildstatus?builder=Linux%20x64&number=2...


Perf bots result of r124832 (committed the TCMalloc update) :

* Linux Builder
http://build.chromium.org/p/chromium.perf/builders/Linux%20Builder/builds/36235
* Linux Perf (1)
http://build.chromium.org/p/chromium.perf/builders/Linux%20Perf%20%281%29/bui...
* Linux Perf (2)
http://build.chromium.org/p/chromium.perf/builders/Linux%20Perf%20%282%29/bui...
* Linux Perf (3;lowmem)
http://build.chromium.org/p/chromium.perf/builders/Linux%20Perf%20%28lowmem%2...

* Win Builder
http://build.chromium.org/p/chromium.perf/builders/Win%20Builder%20%28dbg%29/...
* XP Perf (dbg)
http://build.chromium.org/p/chromium.perf/builders/XP%20Perf%20%28dbg%29/buil...
* Vista Perf (dog)
http://build.chromium.org/p/chromium.perf/builders/Vista%20Perf%20%28dbg%29/b...

fyi, ASAN bots are all green.


Perf bots result of r124833 (reverted the TCMalloc update) :

* Linux Builder
http://build.chromium.org/p/chromium.perf/builders/Linux%20Builder/builds/36236
* Linux Perf (1)
http://build.chromium.org/p/chromium.perf/builders/Linux%20Perf%20%281%29/bui...
* Linux Perf (2)
http://build.chromium.org/p/chromium.perf/builders/Linux%20Perf%20%282%29/bui...
* Linux Perf (3;lowmem)
http://build.chromium.org/p/chromium.perf/builders/Linux%20Perf%20%28lowmem%2...

* Win Builder
http://build.chromium.org/p/chromium.perf/builders/Win%20Builder%20%28dbg%29/...
* XP Perf (dbg)
http://build.chromium.org/p/chromium.perf/builders/XP%20Perf%20%28dbg%29/buil...
* Vista Perf (dog)
http://build.chromium.org/p/chromium.perf/builders/Vista%20Perf%20%28dbg%29/b...

[jam, 2012-03-05 06:56:26]

rubberstamp lgtm

[tony, 2012-03-05 19:27:30]

webkit LGTM

[arv (Not doing code reviews), 2012-03-05 21:21:16]

LGTm

[stevenjb, 2012-03-05 23:14:05]

rubberstamp lgtm for chrome/browser/chromeos/status/memory_menu_button.cc

[jar (doing other things), 2012-03-06 03:42:46]

I wanted to to have a look at the perfbots... but the overview pgae is broken,
an not displaying the range of changes from when you first landed <sigh>. 
Maruel thinks he fixed it, and it should be up in a bit.... I'll probably be
able to see the perf bots more easily tomorrow... and then I'll add my
blessings.




On 2012/03/03 08:47:43, Dai Mikurube wrote:
> John(jam), Tony(tony), Erik(arv), Steven(stevenjb)
> 
> I wonder if you could PTAL at changes in
> - content/* (jam)
> - webkit/extensions/v8/* (tony)
> - chrome/browser/ui/webui/* (arv)
> - chrome/browser/chromeos/status/* (stevenjb)
> as you're OWNERs.  (Sorry for asking again, John and Tony.)
> 
> It changes only #include directories from google/ to gperftools/.  (Please see
> http://code.google.com/p/gperftools/.)
> 
> 
> 
> Jim,
> 
> I tried performance experiments as follows.  Overall performance looks like
> having no problems as links attached at bottom.
> 
> (I forgot to commit the update in another review page, so I created this new
> one.)
> 
> In fact, dromaeo_jslib_test looks getting slow, but dromaeo_jslib_test tends
to
> get slower as r124830 (two revisions before committing the TCMalloc update) is
> red, too.
>
http://build.chromium.org/p/chromium.perf/builders/Linux%2520Perf%2520%25282%...
> 
> A problem might be failures in "sizes".  Static initializers look growing, but
I
> think we can't avoid it even if the TCMalloc causes them...
> - http://build.chromium.org/p/chromium/buildstatus?builder=Linux&number=20880
> -
>
http://build.chromium.org/p/chromium/buildstatus?builder=Linux%2520x64&number...
> 
> 
> Perf bots result of r124832 (committed the TCMalloc update) :
> 
> * Linux Builder
>
http://build.chromium.org/p/chromium.perf/builders/Linux%2520Builder/builds/3...
> * Linux Perf (1)
>
http://build.chromium.org/p/chromium.perf/builders/Linux%2520Perf%2520%25281%...
> * Linux Perf (2)
>
http://build.chromium.org/p/chromium.perf/builders/Linux%2520Perf%2520%25282%...
> * Linux Perf (3;lowmem)
>
http://build.chromium.org/p/chromium.perf/builders/Linux%2520Perf%2520%2528lo...
> 
> * Win Builder
>
http://build.chromium.org/p/chromium.perf/builders/Win%2520Builder%2520%2528d...
> * XP Perf (dbg)
>
http://build.chromium.org/p/chromium.perf/builders/XP%2520Perf%2520%2528dbg%2...
> * Vista Perf (dog)
>
http://build.chromium.org/p/chromium.perf/builders/Vista%2520Perf%2520%2528db...
> 
> fyi, ASAN bots are all green.
> 
> 
> Perf bots result of r124833 (reverted the TCMalloc update) :
> 
> * Linux Builder
>
http://build.chromium.org/p/chromium.perf/builders/Linux%2520Builder/builds/3...
> * Linux Perf (1)
>
http://build.chromium.org/p/chromium.perf/builders/Linux%2520Perf%2520%25281%...
> * Linux Perf (2)
>
http://build.chromium.org/p/chromium.perf/builders/Linux%2520Perf%2520%25282%...
> * Linux Perf (3;lowmem)
>
http://build.chromium.org/p/chromium.perf/builders/Linux%2520Perf%2520%2528lo...
> 
> * Win Builder
>
http://build.chromium.org/p/chromium.perf/builders/Win%2520Builder%2520%2528d...
> * XP Perf (dbg)
>
http://build.chromium.org/p/chromium.perf/builders/XP%2520Perf%2520%2528dbg%2...
> * Vista Perf (dog)
>
http://build.chromium.org/p/chromium.perf/builders/Vista%2520Perf%2520%2528db...

[Dai Mikurube (NOT FULLTIME), 2012-03-06 04:49:41]

Thank you for your lgtms, jam, tony, arv, stevenjb.

Jim,
I understood the situation.  Thanks for handling it!

[Dai Mikurube (NOT FULLTIME), 2012-03-07 21:34:14]

Hi Jim,

Perfbots may still be wrong?

On 2012/03/06 04:49:41, Dai Mikurube wrote:
> Thank you for your lgtms, jam, tony, arv, stevenjb.
> 
> Jim,
> I understood the situation.  Thanks for handling it!

[Dai Mikurube (NOT FULLTIME), 2012-03-10 10:13:03]

Did the performance experiment again, in r126020, and reverted in r126023.  

Look at the perf dashboard:
http://build.chromium.org/f/chromium/perf/dashboard/overview.html


Clobber builds: "sizes" are red in Linux / Linux x64.
Win: http://build.chromium.org/p/chromium/buildstatus?builder=Win&number=9857
Linux:
http://build.chromium.org/p/chromium/buildstatus?builder=Linux&number=21132
Linux x64:
http://build.chromium.org/p/chromium/buildstatus?builder=Linux%20x64&number=2...
or
http://build.chromium.org/p/chromium/buildstatus?builder=Linux+x64&number=24570
(replace %20 with a space)

[jar (doing other things), 2012-03-13 01:01:58]

Looking at revision 126020 on the various perf graphs, I've only noticed:

Bloat - HTTP: Dual Core XP:  all-time record: about 500MB over the current
peaks... but that is "only" about 2-3% higher than existing peaks.

I don't know what this means... but it was interesting.

Historically, I've seen reason to ignore smallish(?) changes in bloat... so I'm
going to cross my fingers and go with LGTM.  I'm hopeful that we won't see a
significant regression in the field.

Jim

p.s., I've also noticed that the there are no new-tab-warm perf results... but
that just means we're blind... not that this CL is problematic.


On 2012/03/10 10:13:03, Dai Mikurube wrote:
> Did the performance experiment again, in r126020, and reverted in r126023.  
> 
> Look at the perf dashboard:
> http://build.chromium.org/f/chromium/perf/dashboard/overview.html
> 
> 
> Clobber builds: "sizes" are red in Linux / Linux x64.
> Win: http://build.chromium.org/p/chromium/buildstatus?builder=Win&number=9857
> Linux:
> http://build.chromium.org/p/chromium/buildstatus?builder=Linux&number=21132
> Linux x64:
>
http://build.chromium.org/p/chromium/buildstatus?builder=Linux%2520x64&number...
> or
>
http://build.chromium.org/p/chromium/buildstatus?builder=Linux+x64&number=24570
> (replace %20 with a space)

[Dai Mikurube (NOT FULLTIME), 2012-03-13 01:13:07]

Thank you for looking at it, Jim!

I saw the peak in Bloat - HTTP, too.  I'll keep taking looks after committing
this update.

I'll commit it after dealing with the issue in static initializers.  (I'm
talking w/ thakis and pliard.)

[jar (doing other things), 2012-03-14 21:21:56]

The bloat change is very apparent, so the test landing was on target.

Have you looked at changing back to using "small" pages?  I'm wary that this is
at the heart of the issue.  A few percent in byte utilization adds up... and I
recall reading about this change a few months back with trepidations.

I *think* there is setting for small vs large pages, and I *think* the new
default is (questionably) large.

Jim

On 2012/03/13 01:13:07, Dai Mikurube wrote:
> Thank you for looking at it, Jim!
> 
> I saw the peak in Bloat - HTTP, too.  I'll keep taking looks after committing
> this update.
> 
> I'll commit it after dealing with the issue in static initializers.  (I'm
> talking w/ thakis and pliard.)

[Dai Mikurube (NOT FULLTIME), 2012-03-14 22:18:36]

Do you mean "Bloat - HTTP" shows worse values after r126412?  I don't see such
an obvious changes in looking at
http://build.chromium.org/f/chromium/perf/xp-release-dual-core/bloat-http/rep...
or other graphs...

Sorry, I think I don't understand your point...


On 2012/03/14 21:21:56, jar wrote:
> The bloat change is very apparent, so the test landing was on target.
> 
> Have you looked at changing back to using "small" pages?  I'm wary that this
is
> at the heart of the issue.  A few percent in byte utilization adds up... and I
> recall reading about this change a few months back with trepidations.
> 
> I *think* there is setting for small vs large pages, and I *think* the new
> default is (questionably) large.
> 
> Jim
> 
> On 2012/03/13 01:13:07, Dai Mikurube wrote:
> > Thank you for looking at it, Jim!
> > 
> > I saw the peak in Bloat - HTTP, too.  I'll keep taking looks after
committing
> > this update.
> > 
> > I'll commit it after dealing with the issue in static initializers.  (I'm
> > talking w/ thakis and pliard.)