|
|
Created:
7 years ago by cmumford Modified:
6 years, 11 months ago CC:
chromium-reviews, jam, alecflett, joi+watch-content_chromium.org, darin-cc_chromium.org, dgrogan Base URL:
https://chromium.googlesource.com/chromium/src.git@master Visibility:
Public. |
DescriptionUnrevert 244240 "IndexedDBFactory now ForceCloses databases."
First implementation contained a memory leak in the test due to a
missing scoped_ptr.
BUG=259564
Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=244369
Patch Set 1 #
Total comments: 3
Patch Set 2 : Removed {} and ran git cl format. #
Total comments: 18
Patch Set 3 : Fixed iter's and -virtual #Patch Set 4 : Fully removed connection map from IndexedDBContextImpl. #Patch Set 5 : Added multimap for more efficient origin->db mapping. #
Total comments: 11
Patch Set 6 : Cleaned up iterator usage, removed local copy of conns. #
Total comments: 4
Patch Set 7 : ForceClosing connections using iterators to avoid possible infinite loop. #
Total comments: 2
Patch Set 8 : DCHECK'ing that connections_ is empty after force closing them all. #Patch Set 9 : Erasing with non-const iterator. #Patch Set 10 : Fixed memory leak #
Messages
Total messages: 29 (0 generated)
dgrogan@chromium.org: Please review changes in jsbell@chromium.org: Please review changes in Two things: 1. I had to modify the test to actually open/close databases. This was required by the fact that we now need to use a real IndexedDBFactory. 2. It looks like IndexedDBContextImpl::connections_ is only used for quota management now, and we could also rely on the factory's list of databases and delete connections_ entirely. My plan was to do that in a subsequent commit.
A few nits. Yeah, it'd be nice to get the context out of the business of tracking connections_ entirely. GetConnectionCount() - would need to delegate to factory loop over database databases; kinda lame. TransactionComplete() - only used in DCHECKs. ConnectionOpened() - this is doing (1) poke quota manager, and (2) update quota - doesn't need to track the connection. ConnectionClosed() - this is doing (1) poke quota manager, and (2) if this was the last connection in an origin, update quota. Those last two could initiated from the factory instead - on backing store open and close, for example. https://codereview.chromium.org/93873017/diff/1/content/browser/indexed_db/in... File content/browser/indexed_db/indexed_db_database.cc (right): https://codereview.chromium.org/93873017/diff/1/content/browser/indexed_db/in... content/browser/indexed_db/indexed_db_database.cc:1652: for (ConnectionSet::iterator i = conns.begin(); i != conns.end(); ++i) { Nit: no need for braces for single-line loop body. https://codereview.chromium.org/93873017/diff/1/content/browser/indexed_db/in... File content/browser/indexed_db/indexed_db_factory.cc (right): https://codereview.chromium.org/93873017/diff/1/content/browser/indexed_db/in... content/browser/indexed_db/indexed_db_factory.cc:95: for (i = dbs.begin(); i != dbs.end(); ++i) { Nit: no need for braces with single-line loop body. https://codereview.chromium.org/93873017/diff/1/content/browser/indexed_db/in... File content/browser/indexed_db/indexed_db_unittest.cc (right): https://codereview.chromium.org/93873017/diff/1/content/browser/indexed_db/in... content/browser/indexed_db/indexed_db_unittest.cc:122: const GURL& origin_url) Nit: indentation
On 2013/12/19 22:35:34, jsbell wrote: > A few nits. > > Yeah, it'd be nice to get the context out of the business of tracking > connections_ entirely. > > GetConnectionCount() - would need to delegate to factory loop over database > databases; kinda lame. > TransactionComplete() - only used in DCHECKs. > ConnectionOpened() - this is doing (1) poke quota manager, and (2) update quota > - doesn't need to track the connection. > ConnectionClosed() - this is doing (1) poke quota manager, and (2) if this was > the last connection in an origin, update quota. > > Those last two could initiated from the factory instead - on backing store open > and close, for example. > > https://codereview.chromium.org/93873017/diff/1/content/browser/indexed_db/in... > File content/browser/indexed_db/indexed_db_database.cc (right): > > https://codereview.chromium.org/93873017/diff/1/content/browser/indexed_db/in... > content/browser/indexed_db/indexed_db_database.cc:1652: for > (ConnectionSet::iterator i = conns.begin(); i != conns.end(); ++i) { > Nit: no need for braces for single-line loop body. > > https://codereview.chromium.org/93873017/diff/1/content/browser/indexed_db/in... > File content/browser/indexed_db/indexed_db_factory.cc (right): > > https://codereview.chromium.org/93873017/diff/1/content/browser/indexed_db/in... > content/browser/indexed_db/indexed_db_factory.cc:95: for (i = dbs.begin(); i != > dbs.end(); ++i) { > Nit: no need for braces with single-line loop body. > > https://codereview.chromium.org/93873017/diff/1/content/browser/indexed_db/in... > File content/browser/indexed_db/indexed_db_unittest.cc (right): > > https://codereview.chromium.org/93873017/diff/1/content/browser/indexed_db/in... > content/browser/indexed_db/indexed_db_unittest.cc:122: const GURL& origin_url) > Nit: indentation Josh: Let me know if you want me to drop the context in this CR and I'll do that.
https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... File content/browser/indexed_db/indexed_db_context_impl.cc (left): https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_context_impl.cc:348: DCHECK_EQ(connections_[origin_url].size(), 0UL); Maybe replace this with DCHECK_EQ(0UL, GetConnectionCount(origin_url)) for now? https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... File content/browser/indexed_db/indexed_db_database.cc (right): https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_database.cc:1650: ConnectionSet conns(connections_); Is this copy necessary? https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_database.cc:1651: for (ConnectionSet::iterator i = conns.begin(); i != conns.end(); ++i) Nit: We try not to use single-character variables. Prefer "it" or "iter" here. https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... File content/browser/indexed_db/indexed_db_database.h (right): https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_database.h:100: virtual void ForceClose(); Why is this virtual? https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_database.h:250: virtual ~IndexedDBDatabase(); Why is this virtual? https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... File content/browser/indexed_db/indexed_db_factory.cc (right): https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_factory.cc:93: std::vector<IndexedDBDatabase*> dbs = GetOpenDatabasesForOrigin(origin_url); GetOpenDatabasesForOrigin is documented in the header as "for diagnostics only" (since it's an O(N) scan). Either we're accepting that and the comment is now wrong, or we should consider if we want to maintain a multimap as well. https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_factory.cc:94: std::vector<IndexedDBDatabase*>::iterator i; Nit: change i to it or iter https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_factory.cc:95: for (i = dbs.begin(); i != dbs.end(); ++i) { Nit: No need for braces. The iterator initialization can also be done inside the for clause to scope the iterator.
> Let me know if you want me to drop the context in this CR and I'll do that. I think it's fine to keep this short and do further whittling in follow-up patches.
https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... File content/browser/indexed_db/indexed_db_context_impl.cc (left): https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_context_impl.cc:348: DCHECK_EQ(connections_[origin_url].size(), 0UL); On 2013/12/20 00:24:42, jsbell wrote: > Maybe replace this with DCHECK_EQ(0UL, GetConnectionCount(origin_url)) for now? You mean to take that DCHECK_EQ and move it to after the call to factory_->ForceClose()? I'll give that a try. https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... File content/browser/indexed_db/indexed_db_database.cc (right): https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_database.cc:1650: ConnectionSet conns(connections_); On 2013/12/20 00:24:42, jsbell wrote: > Is this copy necessary? I can't see an alternative. listset has no front method, so I can't do a while (!empty) type of loop. Nor does it have a reverse_iterator. And ForceClose modifies connections_ which invalidates the iterator and causes a crash. Let me know if I'm missing something. https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_database.cc:1651: for (ConnectionSet::iterator i = conns.begin(); i != conns.end(); ++i) On 2013/12/20 00:24:42, jsbell wrote: > Nit: We try not to use single-character variables. Prefer "it" or "iter" here. Done. https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... File content/browser/indexed_db/indexed_db_database.h (right): https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_database.h:100: virtual void ForceClose(); On 2013/12/20 00:24:42, jsbell wrote: > Why is this virtual? Woops - long story, but it (and the destructor) doesn't need to be virtual. https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... File content/browser/indexed_db/indexed_db_factory.cc (right): https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_factory.cc:93: std::vector<IndexedDBDatabase*> dbs = GetOpenDatabasesForOrigin(origin_url); On 2013/12/20 00:24:42, jsbell wrote: > GetOpenDatabasesForOrigin is documented in the header as "for diagnostics only" > (since it's an O(N) scan). Either we're accepting that and the comment is now > wrong, or we should consider if we want to maintain a multimap as well. I've got the multimap code written in another branch. Are we typically dealing with enough connections to justify the added complexity? Let me know and I'll bring the multimap in. At a minimum I need to fix the comment. https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_factory.cc:94: std::vector<IndexedDBDatabase*>::iterator i; On 2013/12/20 00:24:42, jsbell wrote: > Nit: change i to it or iter Done. https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_factory.cc:95: for (i = dbs.begin(); i != dbs.end(); ++i) { On 2013/12/20 00:24:42, jsbell wrote: > Nit: No need for braces. The iterator initialization can also be done inside the > for clause to scope the iterator. Done.
https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... File content/browser/indexed_db/indexed_db_context_impl.cc (left): https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_context_impl.cc:348: DCHECK_EQ(connections_[origin_url].size(), 0UL); On 2013/12/20 18:09:26, cmumford wrote: > On 2013/12/20 00:24:42, jsbell wrote: > > Maybe replace this with DCHECK_EQ(0UL, GetConnectionCount(origin_url)) for > now? > > You mean to take that DCHECK_EQ and move it to after the call to > factory_->ForceClose()? I'll give that a try. Yep, that's what I was thinking. https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... File content/browser/indexed_db/indexed_db_database.cc (right): https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_database.cc:1650: ConnectionSet conns(connections_); On 2013/12/20 18:09:26, cmumford wrote: > On 2013/12/20 00:24:42, jsbell wrote: > > Is this copy necessary? > > I can't see an alternative. listset has no front method, so I can't do a while > (!empty) type of loop. Nor does it have a reverse_iterator. And ForceClose > modifies connections_ which invalidates the iterator and causes a crash. Let me > know if I'm missing something. A few possibilities: * list_set does have begin() - should be equivalent to front() * unless there's a bug, the iterator shouldn't be invalidated during modification if it's incremented before the removal, similar to the STL erase(it++) idiom: it = set.begin(); while (it != set.end()) { con = *it; ++it; con->ForceClose(); } * a common pattern in these cases is to do a swap rather than a copy. However, that would empty out the "master" set, and there may be things that rely on things being in the set. I'd avoid this if possible. * Finally, if a copy is necessary, document it with a comment (since it's unusual) https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... File content/browser/indexed_db/indexed_db_factory.cc (right): https://codereview.chromium.org/93873017/diff/20001/content/browser/indexed_d... content/browser/indexed_db/indexed_db_factory.cc:93: std::vector<IndexedDBDatabase*> dbs = GetOpenDatabasesForOrigin(origin_url); On 2013/12/20 18:09:26, cmumford wrote: > On 2013/12/20 00:24:42, jsbell wrote: > > GetOpenDatabasesForOrigin is documented in the header as "for diagnostics > only" > > (since it's an O(N) scan). Either we're accepting that and the comment is now > > wrong, or we should consider if we want to maintain a multimap as well. > > I've got the multimap code written in another branch. Are we typically dealing > with enough connections to justify the added complexity? Let me know and I'll > bring the multimap in. At a minimum I need to fix the comment. You're probably right that our numbers are typically low. So adjusting the comments is probably fine, maybe add a // TODO(cmumford): Consider adding a multimap here.
I wound up completely removing IndexedDBContextImpl's connections_ map. I was originally going to clean that up later, but decided to switch over the other user of connections_ (quotas). Josh: I also added the multimap so that GetOpenDatabasesForOrigin is "properly fast". Let me know if you think this adds too much complexity - It seems reasonable to me.
Yay, nice to see the context getting simpler! https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... File content/browser/indexed_db/indexed_db_context_impl.cc (right): https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... content/browser/indexed_db/indexed_db_context_impl.cc:406: DCHECK(factory_->GetConnectionCount(origin_url) > 0); Suggest rewriting as: DCHECK(!factory_ || factory->...) https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... File content/browser/indexed_db/indexed_db_database.cc (right): https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... content/browser/indexed_db/indexed_db_database.cc:1595: // deleted before this function returns. A more common pattern is to take a self-reference within the method: scoped_refptr<IndexedDBTransaction> protect(this); That also protects against additional logic being added to the method. https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... File content/browser/indexed_db/indexed_db_factory.cc (right): https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... content/browser/indexed_db/indexed_db_factory.cc:116: // Iterate this way because ForceClose will call back into this instance Would the standard "advance iterator before erasing" pattern work here? while (range.first != range.second) { IndexedDBDatabase* db = range.first->second; ++range.first; db->ForceClose(); } https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... content/browser/indexed_db/indexed_db_factory.cc:371: std::pair<OriginDbMapIterator, OriginDbMapIterator> range = Can't this be: return origin_dbs_.count(origin_url) ? https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... File content/browser/indexed_db/indexed_db_factory.h (right): https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... content/browser/indexed_db/indexed_db_factory.h:53: typedef std::multimap<GURL, IndexedDBDatabase*> OriginDbMap; Nit: capitalize 'DB' or spell it out (i.e. 'Database') Nit: typedefs should be the first thing in the public: group - http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml?showone=Declar... (The indexeddb code violates this rule all over the place; we need to clean that up at some point.)
https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... File content/browser/indexed_db/indexed_db_context_impl.cc (right): https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... content/browser/indexed_db/indexed_db_context_impl.cc:406: DCHECK(factory_->GetConnectionCount(origin_url) > 0); On 2014/01/03 00:34:25, jsbell wrote: > Suggest rewriting as: > > DCHECK(!factory_ || factory->...) Done. https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... File content/browser/indexed_db/indexed_db_database.cc (right): https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... content/browser/indexed_db/indexed_db_database.cc:1595: // deleted before this function returns. On 2014/01/03 00:34:25, jsbell wrote: > A more common pattern is to take a self-reference within the method: > > scoped_refptr<IndexedDBTransaction> protect(this); > > That also protects against additional logic being added to the method. Yes, a much better approach. I'll do that. https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... File content/browser/indexed_db/indexed_db_factory.cc (right): https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... content/browser/indexed_db/indexed_db_factory.cc:116: // Iterate this way because ForceClose will call back into this instance On 2014/01/03 00:34:25, jsbell wrote: > Would the standard "advance iterator before erasing" pattern work here? > > while (range.first != range.second) { > IndexedDBDatabase* db = range.first->second; > ++range.first; > db->ForceClose(); > } I wasn't confident enough with the multimap iterator to know if that was a safe thing to do. Your suggestion does work however, and all tests pass. https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... content/browser/indexed_db/indexed_db_factory.cc:371: std::pair<OriginDbMapIterator, OriginDbMapIterator> range = On 2014/01/03 00:34:25, jsbell wrote: > Can't this be: > > return origin_dbs_.count(origin_url) ? That would return the number of open DB's for an origin, but not the number of connections right? https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... File content/browser/indexed_db/indexed_db_factory.h (right): https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... content/browser/indexed_db/indexed_db_factory.h:53: typedef std::multimap<GURL, IndexedDBDatabase*> OriginDbMap; On 2014/01/03 00:34:25, jsbell wrote: > Nit: capitalize 'DB' or spell it out (i.e. 'Database') > > Nit: typedefs should be the first thing in the public: group - > http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml?showone=Declar... > (The indexeddb code violates this rule all over the place; we need to clean that > up at some point.) Done.
https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... File content/browser/indexed_db/indexed_db_factory.cc (right): https://codereview.chromium.org/93873017/diff/150001/content/browser/indexed_... content/browser/indexed_db/indexed_db_factory.cc:371: std::pair<OriginDbMapIterator, OriginDbMapIterator> range = Oops, you're right, sorry.
Uploaded patchset 6 with requested changes.
lgtm with a couple nits https://codereview.chromium.org/93873017/diff/240001/content/browser/indexed_... File content/browser/indexed_db/indexed_db_context_impl.cc (right): https://codereview.chromium.org/93873017/diff/240001/content/browser/indexed_... content/browser/indexed_db/indexed_db_context_impl.cc:342: DCHECK_EQ(0UL, GetConnectionCount(origin_url)); Nit: This DCHECK could move outside the 'if' block. https://codereview.chromium.org/93873017/diff/240001/content/browser/indexed_... File content/browser/indexed_db/indexed_db_database.cc (right): https://codereview.chromium.org/93873017/diff/240001/content/browser/indexed_... content/browser/indexed_db/indexed_db_database.cc:1595: while (!connections_.empty()) If we ever break IndexedDBConnect::ForceClose() so it doesn't remove the connection, this would hang unhelpfully. Maybe change to iterating and then DCHECK that connections_ is empty?
https://codereview.chromium.org/93873017/diff/240001/content/browser/indexed_... File content/browser/indexed_db/indexed_db_context_impl.cc (right): https://codereview.chromium.org/93873017/diff/240001/content/browser/indexed_... content/browser/indexed_db/indexed_db_context_impl.cc:342: DCHECK_EQ(0UL, GetConnectionCount(origin_url)); On 2014/01/06 22:33:50, jsbell wrote: > Nit: This DCHECK could move outside the 'if' block. Done. https://codereview.chromium.org/93873017/diff/240001/content/browser/indexed_... File content/browser/indexed_db/indexed_db_database.cc (right): https://codereview.chromium.org/93873017/diff/240001/content/browser/indexed_... content/browser/indexed_db/indexed_db_database.cc:1595: while (!connections_.empty()) On 2014/01/06 22:33:50, jsbell wrote: > If we ever break IndexedDBConnect::ForceClose() so it doesn't remove the > connection, this would hang unhelpfully. Maybe change to iterating and then > DCHECK that connections_ is empty? Done.
still LGTM but one more nit https://codereview.chromium.org/93873017/diff/310001/content/browser/indexed_... File content/browser/indexed_db/indexed_db_database.cc (right): https://codereview.chromium.org/93873017/diff/310001/content/browser/indexed_... content/browser/indexed_db/indexed_db_database.cc:1599: } Can you add a DCHECK that connections_ is empty() once the loop completes?
All local unit tests passed with latest change. I'm going to commit. https://codereview.chromium.org/93873017/diff/310001/content/browser/indexed_... File content/browser/indexed_db/indexed_db_database.cc (right): https://codereview.chromium.org/93873017/diff/310001/content/browser/indexed_... content/browser/indexed_db/indexed_db_database.cc:1599: } On 2014/01/09 18:09:11, jsbell wrote: > Can you add a DCHECK that connections_ is empty() once the loop completes? Done.
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/cmumford@chromium.org/93873017/370002
Retried try job too often on mac_rel for step(s) base_unittests, browser_tests, interactive_ui_tests, net_unittests, unit_tests http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=mac_rel&nu...
erase in IndexedDBFactory::RemoveDatabaseFromMaps was failing because I was using a const_iterator which is OK in C++11 (clang), but not with gcc.
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/cmumford@chromium.org/93873017/350003
Retried try job too often on linux_aura for step(s) interactive_ui_tests http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=linux_aura...
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/cmumford@chromium.org/93873017/350003
Message was sent while issue was closed.
Change committed as 244240
Message was sent while issue was closed.
A revert of this CL has been created in https://codereview.chromium.org/134693005/ by cmumford@chromium.org. The reason for reverting is: Caused memory leaks (http://build.chromium.org/p/chromium.memory/builders/Linux%20ASAN%20Tests%20%...).
Message was sent while issue was closed.
Fixed memory leak
Memory leak is fixed (thx Josh)
leak fix lgtm
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/cmumford@chromium.org/93873017/920001
Message was sent while issue was closed.
Change committed as 244369 |