| OLD | NEW |
| 1 # Chrome's Blob Storage System Design | 1 # Chrome's Blob Storage System Design |
| 2 | 2 |
| 3 Elaboration of the blob storage system in Chrome. | 3 Elaboration of the blob storage system in Chrome. |
| 4 | 4 |
| 5 # What are blobs? | 5 # What are blobs? |
| 6 | 6 |
| 7 Please see the [FileAPI Spec](https://www.w3.org/TR/FileAPI/) for the full | 7 Please see the [FileAPI Spec](https://www.w3.org/TR/FileAPI/) for the full |
| 8 specification for Blobs, or [Mozilla's Blob documentation]( | 8 specification for Blobs, or [Mozilla's Blob documentation]( |
| 9 https://developer.mozilla.org/en-US/docs/Web/API/Blob) for a description of how | 9 https://developer.mozilla.org/en-US/docs/Web/API/Blob) for a description of how |
| 10 Blobs are used in the Web Platform in general. For the purposes of this | 10 Blobs are used in the Web Platform in general. For the purposes of this |
| (...skipping 64 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
| 75 size. | 75 size. |
| 76 | 76 |
| 77 # Blob Storage Limits | 77 # Blob Storage Limits |
| 78 | 78 |
| 79 We calculate the storage limits [here]( | 79 We calculate the storage limits [here]( |
| 80 https://cs.chromium.org/chromium/src/storage/browser/blob/blob_memory_controller
.cc?q=CalculateBlobStorageLimitsImpl&sq=package:chromium). | 80 https://cs.chromium.org/chromium/src/storage/browser/blob/blob_memory_controller
.cc?q=CalculateBlobStorageLimitsImpl&sq=package:chromium). |
| 81 | 81 |
| 82 **In-Memory Storage Limit** | 82 **In-Memory Storage Limit** |
| 83 | 83 |
| 84 * If the architecture is x64 and NOT Chrome OS or Android: `2GB` | 84 * If the architecture is x64 and NOT Chrome OS or Android: `2GB` |
| 85 * Otherwise: `total_physical_memory / 5` | 85 * If Chrome OS: `total_physical_memory / 5` |
| 86 * If Android: `total_physical_memory / 100` |
| 87 |
| 86 | 88 |
| 87 **Disk Storage Limit** | 89 **Disk Storage Limit** |
| 88 | 90 |
| 89 * If Chrome OS: `disk_size / 2` | 91 * If Chrome OS: `disk_size / 2` |
| 90 * If Android: `disk_size / 20` | 92 * If Android: `6 * disk_size / 100` |
| 91 * Else: `disk_size / 10` | 93 * Else: `disk_size / 10` |
| 92 | 94 |
| 93 Note: Chrome OS's disk is part of the user partition, which is separate from the | 95 Note: Chrome OS's disk is part of the user partition, which is separate from the |
| 94 system partition. | 96 system partition. |
| 95 | 97 |
| 96 **Minimum Disk Availability** | 98 **Minimum Disk Availability** |
| 97 | 99 |
| 98 We limit our disk limit to accomidate a minimum disk availability. The equation | 100 We limit our disk limit to accomidate a minimum disk availability. The equation |
| 99 we use is: | 101 we use is: |
| 100 | 102 |
| 101 `min_disk_availability = in_memory_limit * 2` | 103 `min_disk_availability = in_memory_limit * 2` |
| 102 | 104 |
| 103 ## Example Limits | 105 ## Example Limits |
| 104 | 106 |
| 105 (All sizes in GB) | |
| 106 | |
| 107 | Device | Ram | In-Memory Limit | Disk | Disk Limit | Min Disk Availability | | 107 | Device | Ram | In-Memory Limit | Disk | Disk Limit | Min Disk Availability | |
| 108 | --- | --- | --- | --- | --- | --- | | 108 | --- | --- | --- | --- | --- | --- | |
| 109 | Cast | 0.5 | 0.1 | 0 | 0 | 0 | | 109 | Cast | 512 MB | 102 MB | 0 | 0 | 0 | |
| 110 | Android Minimal | 0.5 | 0.1 | 8 | 0.4 | 0.2 | | 110 | Android Minimal | 512 MB | 5 MB | 8 GB | 491 MB | 10 MB | |
| 111 | Android Fat | 2 | 0.4 | 32 | 1.5 | 0.8 | | 111 | Android Fat | 2 GB | 20 MB | 32 GB | 1.9 GB | 40 MB | |
| 112 | CrOS | 2 | 0.4 | 8 | 4 | 0.8 | | 112 | CrOS | 2 GB | 409 MB | 8 GB | 4 GB | 0.8 GB | |
| 113 | Desktop 32 | 3 | 0.6 | 500 | 50 | 1.2 | | 113 | Desktop 32 | 3 GB | 614 MB | 500 GB | 50 GB | 1.2 GB | |
| 114 | Desktop 64 | 4 | 2 | 500 | 50 | 4 | | 114 | Desktop 64 | 4 GB | 2 GB | 500 GB | 50 GB | 4 GB | |
| 115 | 115 |
| 116 # Common Pitfalls | 116 # Common Pitfalls |
| 117 | 117 |
| 118 ## Creating Large Blobs Too Fast | 118 ## Creating Large Blobs Too Fast |
| 119 | 119 |
| 120 Creating a lot of blobs, especially if they are very large blobs, can cause | 120 Creating a lot of blobs, especially if they are very large blobs, can cause |
| 121 the renderer memory to grow too fast and result in an OOM on the renderer side. | 121 the renderer memory to grow too fast and result in an OOM on the renderer side. |
| 122 This is because the renderer temporarily stores the blob data while it waits | 122 This is because the renderer temporarily stores the blob data while it waits |
| 123 for the browser to request it. Meanwhile, Javascript can continue executing. | 123 for the browser to request it. Meanwhile, Javascript can continue executing. |
| 124 Transfering the data can take a lot of time if the blob is large enough to save | 124 Transfering the data can take a lot of time if the blob is large enough to save |
| 125 it directly to a file, as this means we need to wait for disk operations before | 125 it directly to a file, as this means we need to wait for disk operations before |
| 126 the renderer can get rid of the data. | 126 the renderer can get rid of the data. |
| 127 | 127 |
| 128 ## Leaking Blob References | 128 ## Leaking Blob References |
| 129 | 129 |
| 130 If the blob object in Javascript is kept around, then the data will never be | 130 If the blob object in Javascript is kept around, then the data will never be |
| 131 cleaned up in the backend. This will unnecessarily us memory, so make sure to | 131 cleaned up in the backend. This will unnecessarily use memory, so make sure to |
| 132 dereference blob objects if they are no longer needed. | 132 dereference blob objects if they are no longer needed. |
| 133 | 133 |
| 134 Similarily if a URL is created for a blob, this will keep the blob data around | 134 Similarily if a URL is created for a blob, this will keep the blob data around |
| 135 until the URL is revoked (and the blob object is dereferenced). However, the | 135 until the URL is revoked (and the blob object is dereferenced). However, the |
| 136 URL is automatically revoked when the browser context is destroyed. | 136 URL is automatically revoked when the browser context is destroyed. |
| 137 | 137 |
| 138 # How to use Blobs (Browser-side) | 138 # How to use Blobs (Browser-side) |
| 139 | 139 |
| 140 ## Building | 140 ## Building |
| 141 All blob interaction should go through the `BlobStorageContext`. Blobs are | 141 All blob interaction should go through the `BlobStorageContext`. Blobs are |
| (...skipping 197 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
| 339 | 339 |
| 340 The `BlobMemoryController` is responsable for: | 340 The `BlobMemoryController` is responsable for: |
| 341 | 341 |
| 342 1. Determining storage quota limits for files and memory, including restricting | 342 1. Determining storage quota limits for files and memory, including restricting |
| 343 file quota when disk space is low. | 343 file quota when disk space is low. |
| 344 2. Determining whether a blob can fit and the transportation strategy to use. | 344 2. Determining whether a blob can fit and the transportation strategy to use. |
| 345 3. Tracking memory quota. | 345 3. Tracking memory quota. |
| 346 4. Tracking file quota and creating files. | 346 4. Tracking file quota and creating files. |
| 347 5. Accumulating and evicting old blob data to files to disk. | 347 5. Accumulating and evicting old blob data to files to disk. |
| 348 | 348 |
| OLD | NEW |