Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(106)

Issue 2742193002: Harden allocator for file-backed memory. (Closed)

Created:
3 years, 9 months ago by bcwhite
Modified:
3 years, 9 months ago
CC:
chromium-reviews, asvitkine+watch_chromium.org, vmpstr+watch_chromium.org, manzagop (departed)
Target Ref:
refs/pending/heads/master
Project:
chromium
Visibility:
Public.

Description

Harden allocator for file-backed memory. Data persisted to disk can end up with some oddities should a hard shutdown of the machine (such as from a power failure) cause an incomplete flush of the memory. It's possible that the actual data gets flushed but the "free_ptr" that indicates the end of used memory does not. The data is still iterable but might try to go beyond what is believed to be the end. The GetBlockData() checks against free_ptr have been removed, relying on the fixed mem_size_ boundary and the required existence of the header cookie to know if the block is valid. A "Flush()" call has been added to allow for signalling when essential changes have been made. It's an empty call except for file-backed memory. A call to it is done after initialization to ensure the file on disk appears initialized. A "memory state" field has been added to the shared header to indicate the overall state of the memory (initalized, deleted, etc.) and accessor methods added. This allows the caller to ensure that major changes to the memory segment get propogated. This is used to indicate if it has been "deleted" because actually deleting the file is prone to failure. Unfortunately, this requires a change to the SharedMetada structure that makes this code incompatible with previous versions. The first version with this change will be unable to read persistent data written by the previous version. It will find invalid data in the header and be unable to find the written data blocks. This is unfortunate but safe due to all the precautions against memory corruption already in the PMA. The PersistentHistogramAllocator has been updated to use this when deleting a file and the FileMetricsProvider now reads it and discards the data if it detects this. Still to do is make use of these features in the Breadcrumbs project. BUG=546019, 620813 Review-Url: https://codereview.chromium.org/2742193002 Cr-Commit-Position: refs/heads/master@{#457501} Committed: https://chromium.googlesource.com/chromium/src/+/42561dc4b39708c610a0149fb61b6f17c0e21f8e

Patch Set 1 #

Patch Set 2 : fix some build problems #

Total comments: 22

Patch Set 3 : addressed review comments by asvitkine #

Patch Set 4 : addressed review comments by asvitkine; added tests #

Total comments: 2

Patch Set 5 : addressed final review comments #

Unified diffs Side-by-side diffs Delta from patch set Stats (+156 lines, -23 lines) Patch
M base/metrics/persistent_histogram_allocator.cc View 1 chunk +2 lines, -0 lines 0 comments Download
M base/metrics/persistent_memory_allocator.h View 1 2 3 5 chunks +46 lines, -1 line 0 comments Download
M base/metrics/persistent_memory_allocator.cc View 1 2 3 4 14 chunks +79 lines, -17 lines 0 comments Download
M base/metrics/persistent_memory_allocator_unittest.cc View 1 2 3 4 chunks +14 lines, -2 lines 0 comments Download
M components/metrics/file_metrics_provider.h View 1 chunk +3 lines, -0 lines 0 comments Download
M components/metrics/file_metrics_provider.cc View 1 chunk +11 lines, -3 lines 0 comments Download
M tools/metrics/histograms/histograms.xml View 1 chunk +1 line, -0 lines 0 comments Download

Messages

Total messages: 35 (26 generated)
bcwhite
3 years, 9 months ago (2017-03-13 16:04:26 UTC) #10
Alexei Svitkine (slow)
https://codereview.chromium.org/2742193002/diff/20001/base/metrics/persistent_memory_allocator.cc File base/metrics/persistent_memory_allocator.cc (right): https://codereview.chromium.org/2742193002/diff/20001/base/metrics/persistent_memory_allocator.cc#newcode106 base/metrics/persistent_memory_allocator.cc:106: // architectures. Can you have a static_assert about that? ...
3 years, 9 months ago (2017-03-15 15:47:21 UTC) #12
bcwhite
https://codereview.chromium.org/2742193002/diff/20001/base/metrics/persistent_memory_allocator.cc File base/metrics/persistent_memory_allocator.cc (right): https://codereview.chromium.org/2742193002/diff/20001/base/metrics/persistent_memory_allocator.cc#newcode106 base/metrics/persistent_memory_allocator.cc:106: // architectures. On 2017/03/15 15:47:20, Alexei Svitkine (very slow) ...
3 years, 9 months ago (2017-03-15 19:21:49 UTC) #15
Alexei Svitkine (slow)
https://codereview.chromium.org/2742193002/diff/20001/base/metrics/persistent_memory_allocator.cc File base/metrics/persistent_memory_allocator.cc (right): https://codereview.chromium.org/2742193002/diff/20001/base/metrics/persistent_memory_allocator.cc#newcode119 base/metrics/persistent_memory_allocator.cc:119: // State of the memory, plus some padding to ...
3 years, 9 months ago (2017-03-15 20:05:29 UTC) #16
bcwhite
https://codereview.chromium.org/2742193002/diff/20001/base/metrics/persistent_memory_allocator.cc File base/metrics/persistent_memory_allocator.cc (right): https://codereview.chromium.org/2742193002/diff/20001/base/metrics/persistent_memory_allocator.cc#newcode119 base/metrics/persistent_memory_allocator.cc:119: // State of the memory, plus some padding to ...
3 years, 9 months ago (2017-03-16 15:53:03 UTC) #21
Alexei Svitkine (slow)
lgtm % comment https://codereview.chromium.org/2742193002/diff/60001/base/metrics/persistent_memory_allocator.cc File base/metrics/persistent_memory_allocator.cc (right): https://codereview.chromium.org/2742193002/diff/60001/base/metrics/persistent_memory_allocator.cc#newcode120 base/metrics/persistent_memory_allocator.cc:120: volatile std::atomic<char> memory_state; // MemoryState enum ...
3 years, 9 months ago (2017-03-16 16:12:53 UTC) #24
bcwhite
https://codereview.chromium.org/2742193002/diff/60001/base/metrics/persistent_memory_allocator.cc File base/metrics/persistent_memory_allocator.cc (right): https://codereview.chromium.org/2742193002/diff/60001/base/metrics/persistent_memory_allocator.cc#newcode120 base/metrics/persistent_memory_allocator.cc:120: volatile std::atomic<char> memory_state; // MemoryState enum values. On 2017/03/16 ...
3 years, 9 months ago (2017-03-16 17:20:44 UTC) #27
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2742193002/80001
3 years, 9 months ago (2017-03-16 17:21:40 UTC) #31
commit-bot: I haz the power
3 years, 9 months ago (2017-03-16 18:40:36 UTC) #35
Message was sent while issue was closed.
Committed patchset #5 (id:80001) as
https://chromium.googlesource.com/chromium/src/+/42561dc4b39708c610a0149fb61b...

Powered by Google App Engine
This is Rietveld 408576698