[BlobAsync] Disk support for blob storage

storage/browser/blob/blob_storage_registry.h

 // Copyright 2015 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 #ifndef STORAGE_BROWSER_BLOB_BLOB_STORAGE_REGISTRY_H_
 #define STORAGE_BROWSER_BLOB_BLOB_STORAGE_REGISTRY_H_
 
 #include <stddef.h>
 
 #include <map>
 #include <memory>
 #include <string>
 #include <vector>
 
 #include "base/callback_forward.h"
 #include "base/containers/scoped_ptr_hash_map.h"
 #include "base/macros.h"
+#include "storage/browser/blob/blob_memory_controller.h"
 #include "storage/browser/blob/internal_blob_data.h"
 #include "storage/browser/storage_browser_export.h"
 #include "storage/common/blob_storage/blob_storage_constants.h"
 
 class GURL;
 
 namespace storage {
+class BlobDataHandle;
+class ShareableBlobDataItem;
 
 // This class stores the blob data in the various states of construction, as
 // well as URL mappings to blob uuids.
 // Implementation notes:
 // * There is no implicit refcounting in this class, except for setting the
 //   refcount to 1 on registration.
 // * When removing a uuid registration, we do not check for URL mappings to that
 //   uuid. The user must keep track of these.
 class STORAGE_EXPORT BlobStorageRegistry {
  public:
-  // True means the blob was constructed successfully, and false means that
-  // there was an error, which is reported in the second argument.
-  using BlobConstructedCallback =
-      base::Callback<void(bool, IPCBlobCreationCancelCode)>;
+  struct STORAGE_EXPORT ItemCopyEntry {
+    ItemCopyEntry(scoped_refptr<ShareableBlobDataItem> source_item,
+                  size_t source_item_offset,
+                  scoped_refptr<ShareableBlobDataItem> dest_item);
+    ~ItemCopyEntry();
+    ItemCopyEntry(const ItemCopyEntry&);
 
-  enum class BlobState {

[michaeln, 2016/07/15 02:12:08]

i'd rather to retain the distinction between states the blob can be in and error
codes

[dmurph, 2016/07/15 20:18:16]

Hm... But we're pretty much always needing both. I kind of like the combination.

-    // The blob is pending transportation from the renderer. This is the default
-    // state on entry construction.
-    PENDING,
-    // The blob is complete and can be read from.
-    COMPLETE,
-    // The blob is broken and no longer holds data. This happens when there was
-    // a problem constructing the blob, or we've run out of memory.
-    BROKEN
+    scoped_refptr<ShareableBlobDataItem> source_item;
+    size_t source_item_offset = 0;
+    scoped_refptr<ShareableBlobDataItem> dest_item;
   };
 
   struct STORAGE_EXPORT Entry {

[michaeln, 2016/07/15 02:12:08]

I think it might help if there were a more clear distinction between an Entry
for a fully baked blob and an Entry for one that is in an intermediary state. I
think many of these data members only matter for prior to being fully baked. In
current source, the mutual exclusivity of data and data_builder is nice.

[dmurph, 2016/07/15 20:18:16]

Hm... what if we separated out these structs into a new file? We could put all
the little things, like PendingConstructionEntry, Entry, ItemCopyEntry, etc.

-    size_t refcount;
-    BlobState state;
-    std::vector<BlobConstructedCallback> build_completion_callbacks;
+    size_t refcount = 0;
+    BlobStatus status = BlobStatus::PENDING;
+    std::vector<BlobStatusCallback> build_completion_callbacks;
 
-    // Only applicable if the state == BROKEN.
-    IPCBlobCreationCancelCode broken_reason =
-        IPCBlobCreationCancelCode::UNKNOWN;
+    // Blob items. During construction these can be TYPE_BYTES_DESCRIPTION, and
+    // some files might be 'future' files (see BlobDataBuilder). This will never
+    // contain blob item references.
+    InternalBlobData data;
+    bool waiting_until_user_population = false;

[michaeln, 2016/07/15 02:12:08]

Can create a larger number of BlobStatus values to capture the different
intermediary PENDING states in place of these bools?

PENDING_MEMORY_ALLOCATION
PENDING_DATA_POPULATION

[dmurph, 2016/07/15 20:18:16]

Huh. This simplifies a little bit, let me know what you think.

+    BlobStatusCallback ready_for_user_population_callback;
 
-    // data and data_builder are mutually exclusive.
-    std::unique_ptr<InternalBlobData> data;
-    std::unique_ptr<InternalBlobData::Builder> data_builder;
-
+    // Metadata
     std::string content_type;
     std::string content_disposition;
 
-    Entry() = delete;
-    Entry(int refcount, BlobState state);
+    // If the blob is destructed, we don't decrease memory in our manager unless
+    // this is true.
+    bool memory_accounted_for = false;
+    bool can_fit = true;
+    BlobMemoryController::PendingConstructionEntry pending_copies_memory_entry;
+    // These are copies from a referenced blob item to our blob items. Some of
+    // these entries may have changed from bytes to files if they were paged.
+    // We must handle this case.
+    std::vector<ItemCopyEntry> copies;
+
+    std::vector<std::unique_ptr<BlobDataHandle>> dependent_blobs;
+    size_t dependent_blobs_building = 0;
+
+    Entry();
     ~Entry();
-
-    // Performs a test-and-set on the state of the given blob. If the state
-    // isn't as expected, we return false. Otherwise we set the new state and
-    // return true.
-    bool TestAndSetState(BlobState expected, BlobState set);
   };
 
   BlobStorageRegistry();
   ~BlobStorageRegistry();
 
   // Creates the blob entry with a refcount of 1 and a state of PENDING. If
   // the blob is already in use, we return null.
   Entry* CreateEntry(const std::string& uuid,
                      const std::string& content_type,
                      const std::string& content_disposition);
@@ skipping 33 matching lines @@
   using URLMap = std::map<GURL, std::string>;
 
   BlobMap blob_map_;
   URLMap url_to_uuid_;
 
   DISALLOW_COPY_AND_ASSIGN(BlobStorageRegistry);
 };
 
 }  // namespace storage
 #endif  // STORAGE_BROWSER_BLOB_BLOB_STORAGE_REGISTRY_H_