Purge unpinned ashmem before parsing /proc/$pid/smaps.

build/android/pylib/android_commands.py

 # Copyright (c) 2012 The Chromium Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 
 """Provides an interface to communicate with the device via the adb command.
 
 Assumes adb binary is currently on system path.
 """
 
 import collections
@@ skipping 1391 matching lines @@
       if stats.device == 'mmcblk0':
         return {
             'num_reads': stats.num_reads_issued,
             'num_writes': stats.num_writes_completed,
             'read_ms': stats.ms_spent_reading,
             'write_ms': stats.ms_spent_writing,
         }
     logging.warning('Could not find disk IO stats.')
     return None
 
+  @staticmethod
+  def _GetExistingHostOutputPath(file_name):

[frankf, 2013/11/04 18:17:06]

If you decide to keep this, it should be moved to a utility module in
pylib/utils since it's not related to adb commands.

+    """Returns the most likely correct full path for the given file name.
+
+    Returns: The full path that was found or throws an exception if no path
+             could be found.
+    """
+    candidate_path = None
+    build_type = None
+    out_dir = os.path.join(
+        constants.DIR_SOURCE_ROOT, os.environ.get('CHROMIUM_OUT_DIR', 'out'))
+
+    if constants.BuildTypeIsSet():

[Philippe, 2013/10/30 17:18:21]

FYI, this condition is not always met which is why I wrote this function rather
than directly using constants.GetOutDirectory(). I'm making this function
private for now but I expect that other clients could make it public and use it
later.

[craigdh, 2013/10/31 00:10:13]

My understanding is that the goal is to support arbitrary out/* directories,
which is why all assumptions of Debug/Release were removed. To support an
arbitrary build type it must be passed in and set somewhere as there is no
reliable way to pick the right out directory. I'm opposed to magic searching for
common cases, explicit is better.

[Philippe, 2013/11/04 14:45:55]

I will let Marcus comment on this :)

+      candidate_path = os.path.join(
+          out_dir, constants.GetBuildType(), file_name)
+    else:
+      release_path = os.path.join(out_dir, 'Release', file_name)
+      debug_path = os.path.join(out_dir, 'Debug', file_name)
+
+      if os.path.exists(release_path):
+        if os.path.exists(debug_path):
+          # Retain the most recent file.
+          release_path_mtime = time.ctime(os.path.getmtime(release_path))
+          debug_path_mtime = time.ctime(os.path.getmtime(debug_path))
+          if release_path_mtime > debug_path_mtime:
+            constants.SetBuildType('Release')
+            return release_path
+          constants.SetBuildType('Debug')
+          return debug_path
+        candidate_path = release_path
+        build_type = 'Release'
+      else:
+        candidate_path = debug_path
+        build_type = 'Debug'

[digit1, 2013/11/04 14:15:55]

I'm ok with self-detection, but if you don't plan to remove this, please ensure
that there is a logging.warning() call here to notify the user. This will help
catch surprises later on.

[Philippe, 2013/11/04 14:45:55]

Yeah good idea. We got bitten more than once with this :)

+
+    if not os.path.exists(candidate_path):
+      raise Exception('File %s does not exist' % candidate_path)
+
+    if build_type:
+      constants.SetBuildType(build_type)
+    return candidate_path
+
+  def _PurgeUnpinnedAshmem(self):
+    """Purges the unpinned ashmem memory for the whole system.
+
+    This can be used to make memory measurements more stable in particular.
+    """
+    host_path = AndroidCommands._GetExistingHostOutputPath('purge_ashmem')
+    device_path = os.path.join(constants.TEST_EXECUTABLE_DIR, 'purge_ashmem')
+    self.PushIfNeeded(host_path, device_path)
+    if not self.RunShellCommand(device_path, log_result=True):
+      logging.warning('Could not purge ashmem. Measurement might be unstable.')
+
   def GetMemoryUsageForPid(self, pid):
     """Returns the memory usage for given pid.
 
     Args:
       pid: The pid number of the specific process running on device.
 
     Returns:
       A tuple containg:
       [0]: Dict of {metric:usage_kb}, for the process which has specified pid.
       The metric keys which may be included are: Size, Rss, Pss, Shared_Clean,
       Shared_Dirty, Private_Clean, Private_Dirty, Referenced, Swap,
       KernelPageSize, MMUPageSize, Nvidia (tablet only).
       [1]: Detailed /proc/[PID]/smaps information.
     """
     usage_dict = collections.defaultdict(int)
     smaps = collections.defaultdict(dict)
+    self._PurgeUnpinnedAshmem()

[tonyg, 2013/11/03 05:05:18]

Telemetry calls GetMemoryUsageForPid, and before we introduce this dependency on
a new binary, we need to check it into cloud storage like the other binaries
required to run on android. We don't want to require building anything in order
to run Telemetry on Android.

See: SetupPrebuiltTools in adb_commands.py

We also need to update instructions on the Telemetry wiki and error messages.

[Philippe, 2013/11/04 14:45:55]

Thanks for the instructions Tony. I have just pushed the 'purge_ashmem' ARM
release binary to cloud storage (since it looked good to David). I added the
corresponding SHA1 file and updated SetupPrebuildTools(). I haven't seen any
specific documentation that I should update?

     current_smap = ''
     for line in self.GetProtectedFileContents('/proc/%s/smaps' % pid,
                                               log_result=False):
       items = line.split()
       # See man 5 proc for more details. The format is:
       # address perms offset dev inode pathname
       if len(items) > 5:
         current_smap = ' '.join(items[5:])
       elif len(items) > 3:
         current_smap = ' '.join(items[3:])
@@ skipping 216 matching lines @@
   """
   def __init__(self, output):
     self._output = output
 
   def write(self, data):
     data = data.replace('\r\r\n', '\n')
     self._output.write(data)
 
   def flush(self):
     self._output.flush()