[BlobAsync] Disk support for blob storage

storage/browser/blob/blob_storage_registry.h

Index: storage/browser/blob/blob_storage_registry.h
diff --git a/storage/browser/blob/blob_storage_registry.h b/storage/browser/blob/blob_storage_registry.h
index 5211094b61a2bba35c6085525f865e4c2c27d96d..e030f07ff64cc1cb7fe68fcbacfda33df9964593 100644
--- a/storage/browser/blob/blob_storage_registry.h
+++ b/storage/browser/blob/blob_storage_registry.h
@@ -15,6 +15,7 @@
 #include "base/callback_forward.h"
 #include "base/containers/scoped_ptr_hash_map.h"
 #include "base/macros.h"
+#include "storage/browser/blob/blob_memory_controller.h"
 #include "storage/browser/blob/internal_blob_data.h"
 #include "storage/browser/storage_browser_export.h"
 #include "storage/common/blob_storage/blob_storage_constants.h"
@@ -22,6 +23,8 @@
 class GURL;
 
 namespace storage {
+class BlobDataHandle;
+class ShareableBlobDataItem;
 
 // This class stores the blob data in the various states of construction, as
 // well as URL mappings to blob uuids.
@@ -32,46 +35,49 @@ namespace storage {
 //   uuid. The user must keep track of these.
 class STORAGE_EXPORT BlobStorageRegistry {
  public:
-  // True means the blob was constructed successfully, and false means that
-  // there was an error, which is reported in the second argument.
-  using BlobConstructedCallback =
-      base::Callback<void(bool, IPCBlobCreationCancelCode)>;
-
-  enum class BlobState {

[michaeln, 2016/07/15 02:12:08]

i'd rather to retain the distinction between states the blob can be in and error
codes

[dmurph, 2016/07/15 20:18:16]

Hm... But we're pretty much always needing both. I kind of like the combination.

-    // The blob is pending transportation from the renderer. This is the default
-    // state on entry construction.
-    PENDING,
-    // The blob is complete and can be read from.
-    COMPLETE,
-    // The blob is broken and no longer holds data. This happens when there was
-    // a problem constructing the blob, or we've run out of memory.
-    BROKEN
+  struct STORAGE_EXPORT ItemCopyEntry {
+    ItemCopyEntry(scoped_refptr<ShareableBlobDataItem> source_item,
+                  size_t source_item_offset,
+                  scoped_refptr<ShareableBlobDataItem> dest_item);
+    ~ItemCopyEntry();
+    ItemCopyEntry(const ItemCopyEntry&);
+
+    scoped_refptr<ShareableBlobDataItem> source_item;
+    size_t source_item_offset = 0;
+    scoped_refptr<ShareableBlobDataItem> dest_item;
   };
 
   struct STORAGE_EXPORT Entry {

[michaeln, 2016/07/15 02:12:08]

I think it might help if there were a more clear distinction between an Entry
for a fully baked blob and an Entry for one that is in an intermediary state. I
think many of these data members only matter for prior to being fully baked. In
current source, the mutual exclusivity of data and data_builder is nice.

[dmurph, 2016/07/15 20:18:16]

Hm... what if we separated out these structs into a new file? We could put all
the little things, like PendingConstructionEntry, Entry, ItemCopyEntry, etc.

-    size_t refcount;
-    BlobState state;
-    std::vector<BlobConstructedCallback> build_completion_callbacks;
-
-    // Only applicable if the state == BROKEN.
-    IPCBlobCreationCancelCode broken_reason =
-        IPCBlobCreationCancelCode::UNKNOWN;
-
-    // data and data_builder are mutually exclusive.
-    std::unique_ptr<InternalBlobData> data;
-    std::unique_ptr<InternalBlobData::Builder> data_builder;
-
+    size_t refcount = 0;
+    BlobStatus status = BlobStatus::PENDING;
+    std::vector<BlobStatusCallback> build_completion_callbacks;
+
+    // Blob items. During construction these can be TYPE_BYTES_DESCRIPTION, and
+    // some files might be 'future' files (see BlobDataBuilder). This will never
+    // contain blob item references.
+    InternalBlobData data;
+    bool waiting_until_user_population = false;

[michaeln, 2016/07/15 02:12:08]

Can create a larger number of BlobStatus values to capture the different
intermediary PENDING states in place of these bools?

PENDING_MEMORY_ALLOCATION
PENDING_DATA_POPULATION

[dmurph, 2016/07/15 20:18:16]

Huh. This simplifies a little bit, let me know what you think.

+    BlobStatusCallback ready_for_user_population_callback;
+
+    // Metadata
     std::string content_type;
     std::string content_disposition;
 
-    Entry() = delete;
-    Entry(int refcount, BlobState state);
-    ~Entry();
+    // If the blob is destructed, we don't decrease memory in our manager unless
+    // this is true.
+    bool memory_accounted_for = false;
+    bool can_fit = true;
+    BlobMemoryController::PendingConstructionEntry pending_copies_memory_entry;
+    // These are copies from a referenced blob item to our blob items. Some of
+    // these entries may have changed from bytes to files if they were paged.
+    // We must handle this case.
+    std::vector<ItemCopyEntry> copies;
+
+    std::vector<std::unique_ptr<BlobDataHandle>> dependent_blobs;
+    size_t dependent_blobs_building = 0;
 
-    // Performs a test-and-set on the state of the given blob. If the state
-    // isn't as expected, we return false. Otherwise we set the new state and
-    // return true.
-    bool TestAndSetState(BlobState expected, BlobState set);
+    Entry();
+    ~Entry();
   };
 
   BlobStorageRegistry();