Return configured processors in NumberOfProcessors() on posix platforms

Return configured processors in NumberOfProcessors() on posix platforms

On Linux/Mac/FreeBSD platform, NumberOfProcessors() was returning the number of
online processors. Change it to return the number of configured processors
(incl. offline processors).

1. Most callers of this API expect it to return totally available cpu cores,
   mainly for threading optimization.

2. The number of online processors is changed within seconds on some embedded
   platforms. It is wrong to pass the cached value of the number of currently
   online processors to Blink sandbox.

BUG=404364
TEST=Tested on nyan (it may have 1~4 cpu cores online)

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=290209

[jochen (gone - plz use gerrit), 2014-08-11 14:29:55]

if we have e.g. one core online, but 8 configured, we will completely overload
the system with threads (v8 spawns background threads based on the available
online cores).

can you explain why this change is needed?

[Justin Chuang, 2014-08-11 15:23:25]

On 2014/08/11 14:29:55, jochen (slow - soon OOO) wrote:
> if we have e.g. one core online, but 8 configured, we will completely overload
> the system with threads (v8 spawns background threads based on the available
> online cores).
> 
> can you explain why this change is needed?

The number of online cores change *within seconds* (I can show logs if needed). 

So the value of the current online core is only a transient value reflecting
*the current system load". It's not a value with system-wide meaning.
The totally available cores is what we care. If we cached online cpu count, V8
may be much slower than it can be. 

(For example, the system have 4 CPUs, but g_lazy_number_of_processors may be 1
because it has been lazy-initialized when loading is low)

To my understanding, on Linux, the number of online cpu cores is decided by
cpuquiet governor. There are several implementations of cpuquiet governors, and
each  can be customized with some parameters. For example, on Chrome OS kernel
3.10, we have 
   - 'runnable' governor - decide # of online CPUs depending on the number of
runnable threads
   - 'userspace' governor - decide # of online CPUs by userspace driver

Other POSIX systems (OS X, BSD) may also have its own implementation to decide
online cpu #.

From the perspective the Chrome, we can only decide the optimal number of
threads according to total configured cpu #. 
We cannot tweak it according to one hundred kernel configurations. 

Having said that, if the kernel implementation is too aggressive toward
power-saving and it does not increase online cpu cores fast enough, it can cause
throttling issues like you said. But I think it's rather an issue of bad kernel
tweaking on the embedded system than a Chrome issue.

[Justin Chuang, 2014-08-11 15:33:54]

Sorry the reply is too long. 

Let me recap: if the system has 4 configured CPUs, but only 1 CPU is online.

We should still create threads in V8 like there are 4 CPUs, and the kernel
should be smart enough to make 4 CPUs online.

(Since each V8 thread should be constantly in run queue, and kernel should
minimize system load, which relates to runqueue size)

[jochen (gone - plz use gerrit), 2014-08-13 13:05:35]

On 2014/08/11 at 15:33:54, jchuang wrote:
> Sorry the reply is too long. 
> 
> Let me recap: if the system has 4 configured CPUs, but only 1 CPU is online.
> 
> We should still create threads in V8 like there are 4 CPUs, and the kernel
should be smart enough to make 4 CPUs online.
> 
> (Since each V8 thread should be constantly in run queue, and kernel should
minimize system load, which relates to runqueue size)

ok, makes sense.

if you change this, can you also send a CL for v8's
src/base/platform/platform-posix.cc that has the same code?

[Justin Chuang, 2014-08-13 16:45:53]

On 2014/08/13 13:05:35, jochen (slow - soon OOO) wrote:

> ok, makes sense.
> 
> if you change this, can you also send a CL for v8's
> src/base/platform/platform-posix.cc that has the same code?

Thanks, will send CL for V8, too.

As for this CL, I'll commit after Friday to avoid impacting R38.

[jochen (gone - plz use gerrit), 2014-08-14 11:07:18]

this lgtm, but you will need a review from a //base owner.

[Justin Chuang, 2014-08-14 18:42:36]

+ 2 owners of base/

PTAL

[Mark Mentovai, 2014-08-14 19:46:57]

That seems fine, LGTM

[brettw, 2014-08-14 22:55:44]

Please just add one owner to avoid duplicating effort.

[Justin Chuang, 2014-08-15 02:59:04]

On 2014/08/14 22:55:44, brettw wrote:
> Please just add one owner to avoid duplicating effort.

sorry, noted

[Justin Chuang, 2014-08-15 11:38:56]

Pawel points out another potential bug:

  On ARM big.LITTLE platform with 4+4 cores, sysconf(_SC_NPROCESSORS_CONF)
returns 8. 

  All 8 cores can go online in HMP mode, but at most 4 cores can go online in
IKS mode.

  Maybe NumberOfProcessors() needs to handle big.LITTLE + Linux IKS as a special
case.


* Moreover, big.LITTLE + HMP also make it harder to decide the optimal number of
threads (using all cores is power-hungry).
But deeper discussion on managing power & performance is outside the scope of
this CL.

[Justin Chuang, 2014-08-18 03:13:02]

On 2014/08/15 11:38:56, Justin Chuang wrote:
> Pawel points out another potential bug:
> 
>   On ARM big.LITTLE platform with 4+4 cores, sysconf(_SC_NPROCESSORS_CONF)
> returns 8. 

Commit this first and leave big.LITTLE + IKS in a separate CL.

[Justin Chuang, 2014-08-18 03:13:06]

The CQ bit was checked by jchuang@chromium.org

[commit-bot: I haz the power, 2014-08-18 03:13:42]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/jchuang@chromium.org/454053002/40001

[commit-bot: I haz the power, 2014-08-18 04:52:56]

Committed patchset #3 (40001) as 290209