sandwich: Implements filter-cache sub-command.

tools/android/loading/chrome_cache.py

Index: tools/android/loading/chrome_cache.py
diff --git a/tools/android/loading/chrome_cache.py b/tools/android/loading/chrome_cache.py
index d6d65a265690fdc4a2fa9fea4a3238a4dd7511e6..16d83fbfcd280bc5591a605f908dbc5ed6f68f8a 100644
--- a/tools/android/loading/chrome_cache.py
+++ b/tools/android/loading/chrome_cache.py
@@ -8,6 +8,7 @@
 from datetime import datetime
 import json
 import os
+import shutil
 import subprocess
 import sys
 import tempfile
@@ -141,6 +142,8 @@ def ZipDirectoryContent(root_directory_path, archive_dest_path):
     root_directory_path: The directory's path to archive.
     archive_dest_path: Archive destination's path.
   """
+  if os.path.isfile(archive_dest_path):

[mattcary, 2016/02/26 09:33:08]

This doesn't seem to be necessary---zipfile will silently overwrite existing
files as far as I can tell. Seems harmless enough, though, even if my instinct
is always to remove code.

[pasko, 2016/02/26 17:10:06]

is it really necessary to remove before writing? According to the code below we
are not expecting a file to be there, so maybe this code is dead and not needed?

[gabadie, 2016/03/01 10:40:48]

Done.

+    os.remove(archive_dest_path)
   with zipfile.ZipFile(archive_dest_path, 'w') as zip_output:
     timestamps = {}
     root_directory_stats = os.stat(root_directory_path)
@@ -278,6 +281,30 @@ class CacheBackend(object):
     return stdout_data
 
 
+def ApplyUrlWhitelistToCacheArchive(cache_archive_path,
+                                    whitelisted_urls,
+                                    output_cache_archive_path):
+  """Generate a new cache archive containing only whitelisted urls.
+
+  Args:
+    cache_archive_path: Path of the cache archive to apply the white listing.
+    whitelisted_urls: Set of url to keep in cache.
+    output_cache_archive_path: Destination path of cache archive containing only
+      white-listed urls.
+  """
+  cache_temp_directory = tempfile.mkdtemp(suffix='.cache')
+  UnzipDirectoryContent(cache_archive_path, cache_temp_directory)
+  backend = CacheBackend(cache_temp_directory, 'simple')
+  cached_urls = backend.ListKeys()
+  for cached_url in cached_urls:
+    if cached_url not in whitelisted_urls:
+      backend.DeleteKey(cached_url)
+  for cached_url in backend.ListKeys():
+    assert cached_url in whitelisted_urls
+  ZipDirectoryContent(cache_temp_directory, output_cache_archive_path)
+  shutil.rmtree(cache_temp_directory)

[pasko, 2016/02/26 17:10:06]

the try..finally would avoid polluting the filesystem

[gabadie, 2016/03/01 10:40:48]

Good point. I think we should just do a util like
https://docs.python.org/3/library/tempfile.html#tempfile.TemporaryDirectory.

Done.

+
+
 if __name__ == '__main__':
   import argparse

[pasko, 2016/02/26 17:10:06]

why import here?

[gabadie, 2016/03/01 10:40:48]

Because this is a lib manual test (in the mean time of the integrated test for
sandwich). So we don't want to pollute the module import list when including
into another module.

[pasko, 2016/03/01 16:01:11]

isn't this against the style guide? To avoid the pollution please prefer to
create a separate file for manual testing.

[gabadie, 2016/03/01 16:49:09]

According to Benoit, that is a common practice.
https://code.google.com/p/chromium/codesearch#chromium/src/tools/android/load...

[pasko, 2016/03/01 17:55:53]

I disagree with Benoit here. In the Chromium codebase the only occurrences of
this are in tools/android/loading. And in this one weird place:
https://goo.gl/SozSvM

We should not do it.

   parser = argparse.ArgumentParser(description='Tests cache back-end.')

[pasko, 2016/02/26 17:10:06]

nit: backend in one word

[gabadie, 2016/03/01 10:40:48]

Looks like the two are correct. Keeping it to be consistent within this
directory.

[pasko, 2016/03/01 16:01:11]

They are both correct, but we should prefer using it as "backend" to be
consistent with the rest of Chrome.

Please also see Don Knuth's opinion on email vs e-mail:
http://www-cs-faculty.stanford.edu/~uno/email.html

But this is a nit for now, so you can leave it unchanged for now

[gabadie, 2016/03/01 16:49:09]

Acknowledged.