Extensions Docs Server: Efficient MemcacheFileSystem

chrome/common/extensions/docs/server2/memcache_file_system.py

Index: chrome/common/extensions/docs/server2/memcache_file_system.py
diff --git a/chrome/common/extensions/docs/server2/memcache_file_system.py b/chrome/common/extensions/docs/server2/memcache_file_system.py
index 5ac407237cabd5055aeee07a1def863a80b4d147..ff4a8668fa22ff93d15b25bc4c8e656395d74289 100644
--- a/chrome/common/extensions/docs/server2/memcache_file_system.py
+++ b/chrome/common/extensions/docs/server2/memcache_file_system.py
@@ -23,8 +23,12 @@ class MemcacheFileSystem(FileSystem):
       self._memcache.Set(path,
                          stat_info.version,
                          memcache.MEMCACHE_FILE_SYSTEM_STAT)
+      for child_path, child_version in stat_info.child_versions.iteritems():
+        self._memcache.Set(path.rsplit('/', 1)[0] + '/' + child_path,
+                           child_version,
+                           memcache.MEMCACHE_FILE_SYSTEM_STAT)
     else:
-      stat_info = self.StatInfo(version)
+      stat_info = self.StatInfo(version, {})

[not at google - send to devlin, 2012/08/13 05:34:15]

memcache can store arbitrary objects, I believe. If you store the StatInfo
object in memcache rather than just the version then you won't need to construct
this fake (and incorrect) {} object.

[cduvall, 2012/08/13 19:45:45]

Memcache can store dictionaries, but not objects. They need to have a pickle
method I believe. I changed this to use None instead of {}, which I think is a
little less confusing.

[not at google - send to devlin, 2012/08/13 23:02:14]

I just checked and it looks like pickle can handle at least simple objects (I
tested objects with simple and list fields). So... let's at least try it.

Here is my output from the python console, by the way:
Python 2.7.3 (default, Apr 13 2012, 00:05:08) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
...   def __init__(self, x, xs):
...     self.x = x
...     self.xs = xs
... 
False
10
[1, 2, 3]

[cduvall, 2012/08/14 18:15:00]

Aha! The problem with pickle was nested classes. It can't pickle a nested class.
Here's my output:
"PicklingError: Can't pickle <class 'file_system.StatInfo'>: attribute lookup
file_system.StatInfo failed"

I'm going to leave this just storing the version for now, and I can change it
later if we want.

[not at google - send to devlin, 2012/08/15 05:24:05]

Ok, let's un-nest StatInfo then (or figure out how to make pickle work).

     return stat_info
 
   def Read(self, paths, binary=False):