memory coordinator: Use free memory to guess available memory on Linux

memory coordinator: Use free memory to guess available memory on Linux

Though there will be eaisly discardable memory such as buffers and
caches, don't count them as "available" memory because discarding
them will affect the overall performance of the system.

BUG=617492
Committed: https://crrev.com/8e656e05f77fb80739dc80bdfa1abfce1f45173d
Cr-Commit-Position: refs/heads/master@{#427880}

[bashi, 2016-10-26 06:48:44]

The CQ bit was checked by bashi@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-10-26 06:49:01]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[bashi, 2016-10-26 06:49:53]

bashi@chromium.org changed reviewers:
+ chrisha@chromium.org, haraken@chromium.org

[bashi, 2016-10-26 06:49:53]

I'd like to change how to guess available memory on Linux which doesn't provide
"Available memory"[1] 

[1]
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=34...

I've been testing MemoryCoordinatorImpl and MemoryMonitor and found that it's
very difficult to enter throttled/suspended state on my Linux machine. I have to
consume almost all memory (95%+ of the total RAM) to make MCI change its state
to THROTTLED/SUSPENDED. Since the current implementation uses `free +_cached +
buffer` to guess available memory, disck caches and inactive anon pages were
discarded before entring THROTTLE/SUSPENDED state. Even though page swapping
didn't happen, I felt that my machine was very slow when the OS started
discarding page caches. I think it would be better not to count buffers and
caches as "available memory". WDYT?

[commit-bot: I haz the power, 2016-10-26 07:02:00]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-10-26 07:02:01]

Dry run: Try jobs failed on following builders:
  linux_chromium_asan_rel_ng on master.tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  linux_chromium_rel_ng on master.tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)

[bashi, 2016-10-26 07:05:43]

Patchset #1 (id:1) has been deleted

[bashi, 2016-10-26 07:05:50]

The CQ bit was checked by bashi@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-10-26 07:06:05]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[haraken, 2016-10-26 07:28:12]

LGTM

I appreciate you're experimenting with how MC works on real machines -- I think
that's really important.

[commit-bot: I haz the power, 2016-10-26 07:45:54]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-10-26 07:45:54]

Dry run: Try jobs failed on following builders:
  cast_shell_linux on master.tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/cast_shell_linu...)

[bashi, 2016-10-26 07:56:11]

The CQ bit was checked by bashi@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-10-26 07:56:25]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-10-26 08:52:32]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-10-26 08:52:33]

Dry run: This issue passed the CQ dry run.

[chrisha, 2016-10-26 21:29:37]

+1 to the motivation and the experimentation. Linux is a platform for which we
had no MPM implementation, so it was a best guess to begin with.

The ChromeOS logic is quite similar, so it might be worth testing there as well?

[chrisha, 2016-10-26 21:29:45]

Forgot the lgtm :)

[bashi, 2016-10-26 23:33:37]

On 2016/10/26 21:29:37, chrisha (slow) wrote:
> +1 to the motivation and the experimentation. Linux is a platform for which we
> had no MPM implementation, so it was a best guess to begin with.
> 
> The ChromeOS logic is quite similar, so it might be worth testing there as
well?

Thank you for review! I'm not very familiar with ChromeOS but I'll try to test
MCC on ChromeOS.

[bashi, 2016-10-26 23:33:45]

The CQ bit was checked by bashi@chromium.org

[bashi, 2016-10-26 23:33:46]

The patchset sent to the CQ was uploaded after l-g-t-m from haraken@chromium.org
Link to the patchset: https://codereview.chromium.org/2448923004/#ps40001
(title: "fix test expectation")

[commit-bot: I haz the power, 2016-10-26 23:34:20]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-10-26 23:39:36]

Committed patchset #2 (id:40001)

[commit-bot: I haz the power, 2016-10-26 23:42:20]

Description was changed from

==========
memory coordinator: Use free memory to guess available memory on Linux

Though there will be eaisly discardable memory such as buffers and
caches, don't count them as "available" memory because discarding
them will affect the overall performance of the system.

BUG=617492
==========

to

==========
memory coordinator: Use free memory to guess available memory on Linux

Though there will be eaisly discardable memory such as buffers and
caches, don't count them as "available" memory because discarding
them will affect the overall performance of the system.

BUG=617492

Committed: https://crrev.com/8e656e05f77fb80739dc80bdfa1abfce1f45173d
Cr-Commit-Position: refs/heads/master@{#427880}
==========

[commit-bot: I haz the power, 2016-10-26 23:42:21]

Patchset 2 (id:??) landed as
https://crrev.com/8e656e05f77fb80739dc80bdfa1abfce1f45173d
Cr-Commit-Position: refs/heads/master@{#427880}