SimpleCache: read small files all at once.

net/disk_cache/simple/simple_entry_impl.h

 // Copyright (c) 2013 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 #ifndef NET_DISK_CACHE_SIMPLE_SIMPLE_ENTRY_IMPL_H_
 #define NET_DISK_CACHE_SIMPLE_SIMPLE_ENTRY_IMPL_H_
 
 #include <stdint.h>
 
 #include <memory>
@@ skipping 64 matching lines @@
   SimpleEntryImpl(net::CacheType cache_type,
                   const base::FilePath& path,
                   uint64_t entry_hash,
                   OperationsMode operations_mode,
                   SimpleBackendImpl* backend,
                   net::NetLog* net_log);
 
   void SetActiveEntryProxy(
       std::unique_ptr<ActiveEntryProxy> active_entry_proxy);
 
   // Adds another reader/writer to this entry, if possible, returning |this| to

[pasko, 2017/08/04 01:28:34]

nit: perhaps mention somewhere in this file that it may read the file contents
in advance to return synchronously from ReadData?

[Maks Orlovich, 2017/08/04 18:35:44]

Acknowledged.

[pasko, 2017/08/09 12:28:21]

sry, it was a leftover comment when I thought that we return synchronously

   // |entry|.
   int OpenEntry(Entry** entry, const CompletionCallback& callback);
 
   // Creates this entry, if possible. Returns |this| to |entry|.

[pasko, 2017/08/04 01:28:34]

same nit about comment as OpenEntry

[Maks Orlovich, 2017/08/04 18:35:44]

Doesn't apply since Create fails if the entry already exists.

[pasko, 2017/08/09 12:28:21]

Acknowledged.

   int CreateEntry(Entry** entry, const CompletionCallback& callback);
 
   // Identical to Backend::Doom() except that it accepts a CompletionCallback.
   int DoomEntry(const CompletionCallback& callback);
 
   const std::string& key() const { return key_; }
   uint64_t entry_hash() const { return entry_hash_; }
 
   // The key is not a constructor parameter to the SimpleEntryImpl, because
   // during cache iteration, it's necessary to open entries by their hash
@@ skipping 194 matching lines @@
   void DoomOperationComplete(const CompletionCallback& callback,
                              State state_to_restore,
                              int result);
 
   // Reports reads result potentially refining status based on |crc_result|.
   // |crc_result| is permitted to be null.
   void RecordReadResultConsideringChecksum(
       int result,
       std::unique_ptr<SimpleSynchronousEntry::CRCRequest> crc_result) const;
 
   // Called after completion of asynchronous IO and receiving file metadata for

[pasko, 2017/08/04 01:28:34]

it can also be called after reading the stream from memory

[Maks Orlovich, 2017/08/04 18:35:43]

Rephrased.

   // the entry in |entry_stat|. Updates the metadata in the entry and in the
   // index to make them available on next IO operations.
   void UpdateDataFromEntryStat(const SimpleEntryStat& entry_stat);
 
   int64_t GetDiskUsage() const;
 
   // Used to report histograms.
   void RecordReadIsParallelizable(const SimpleEntryOperation& operation) const;
   void RecordWriteDependencyType(const SimpleEntryOperation& operation) const;
 
-  // Reads from the stream 0 data kept in memory.
-  int ReadStream0Data(net::IOBuffer* buf, int offset, int buf_len);
+  // Completes a read from the stream data kept in memory, logging metrics
+  // and updating metadata as appropriate. If |callback| is non-null, it will
+  // be posted to with the return code.
+  void ReadInMemoryStreamData(net::GrowableIOBuffer* in_buf,

[pasko, 2017/08/04 01:28:34]

naming bikeshed: 'Stream' is slightly confusing because we are not providing
stream index to the method. Also seems important to reflect the posting
activity. Suggestion: ReadFromBufferAndPostReply()?

[Maks Orlovich, 2017/08/04 18:35:44]

Done.

+                              int offset,
+                              int buf_len,
+                              net::IOBuffer* out_buf,
+                              const CompletionCallback& callback);
 
   // Copies data from |buf| to the internal in-memory buffer for stream 0. If
   // |truncate| is set to true, the target buffer will be truncated at |offset|
   // + |buf_len| before being written.
   int SetStream0Data(net::IOBuffer* buf,
                      int offset, int buf_len,
                      bool truncate);
 
   // Updates |crc32s_| and |crc32s_end_offset_| for a write of the data in
   // |buffer| on |stream_index|, starting at |offset| and of length |length|.
   void AdvanceCrc(net::IOBuffer* buffer,
                   int offset,
                   int length,
                   int stream_index);
 
   std::unique_ptr<ActiveEntryProxy> active_entry_proxy_;
 
   // All nonstatic SimpleEntryImpl methods should always be called on the IO
   // thread, in all cases. |io_thread_checker_| documents and enforces this.
   base::ThreadChecker io_thread_checker_;
 
   const base::WeakPtr<SimpleBackendImpl> backend_;
   const net::CacheType cache_type_;
   const scoped_refptr<base::TaskRunner> worker_pool_;
   const base::FilePath path_;
   const uint64_t entry_hash_;
   const bool use_optimistic_operations_;
+  bool is_initial_stream1_read_;  // used for metrics only.
   std::string key_;
 
   // |last_used_|, |last_modified_| and |data_size_| are copied from the
   // synchronous entry at the completion of each item of asynchronous IO.
   // TODO(clamy): Unify last_used_ with data in the index.
   base::Time last_used_;
   base::Time last_modified_;
   int32_t data_size_[kSimpleEntryStreamCount];
   int32_t sparse_data_size_;
 
@@ skipping 42 matching lines @@
   // Unlike other streams, stream 0 data is read from the disk when the entry is
   // opened, and then kept in memory. All read/write operations on stream 0
   // affect the |stream_0_data_| buffer. When the entry is closed,
   // |stream_0_data_| is written to the disk.
   // Stream 0 is kept in memory because it is stored in the same file as stream
   // 1 on disk, to reduce the number of file descriptors and save disk space.
   // This strategy allows stream 1 to change size easily. Since stream 0 is only
   // used to write HTTP headers, the memory consumption of keeping it in memory
   // is acceptable.
   scoped_refptr<net::GrowableIOBuffer> stream_0_data_;
+
+  // Sometimes stream 1 data is prefetched when stream 0 is first read, to make
+  // the first read call on stream 1 synchronous. If a write to the stream

[pasko, 2017/08/04 01:28:34]

oops, it's not synchronous because we PostTask every time. Apologies if I
suggested this formulation, I forgot that we do not return synchronously.

[Maks Orlovich, 2017/08/04 18:35:43]

So did I, actually. I think this was the original motivation Josh had, but it
kinda got overtaken by me liking single-read. Dropped this sentence.

+  // occurs on the entry the prefetch buffer is discarded. It may also be
+  // null if it wasn't prefetched in the first place.
+  scoped_refptr<net::GrowableIOBuffer> stream_1_prefetch_data_;
 };
 
 }  // namespace disk_cache
 
 #endif  // NET_DISK_CACHE_SIMPLE_SIMPLE_ENTRY_IMPL_H_