Refactor how ClientSocketPoolBaseHelper avoids re-entrancy.

Refactor how ClientSocketPoolBaseHelper avoids re-entrancy.
Specifically, we defer asynchronous user callbacks to tasks.
BUG=48861
Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=52509

[willchan no longer on Chromium, 2010-07-13 20:48:53]

This is just a refactor for simplicity.  The nice thing about doing things this
way is that the internal CSP state is always consistent.  We only slightly delay
calling the user callback for the async cases.
Steve: I'm not sure if this will collide with any more changes of yours.  It's a
heads up for you.

[Mike Belshe, 2010-07-13 21:28:11]

lgtm; just a few nits.  thanks for doing this before you go!

http://codereview.chromium.org/2994003/diff/1/5
File net/socket/client_socket_handle.cc (right):

http://codereview.chromium.org/2994003/diff/1/5#newcode43
net/socket/client_socket_handle.cc:43: // If we did not get initialized yet, so
we've got a socket request pending.
nit:  this comment reads funny - remove the "so".

http://codereview.chromium.org/2994003/diff/1/5#newcode76
net/socket/client_socket_handle.cc:76: CHECK_NE(ERR_IO_PENDING, result);
Should this function set is_initialized_ to true?

http://codereview.chromium.org/2994003/diff/1/6
File net/socket/client_socket_handle.h (right):

http://codereview.chromium.org/2994003/diff/1/6#newcode89
net/socket/client_socket_handle.h:89: // Returns true when Init() has completed
successfully.
Since set_is_initialized() is public, this comment is not quite true anymore?

http://codereview.chromium.org/2994003/diff/1/6#newcode99
net/socket/client_socket_handle.h:99: void set_is_initialized(bool value) {
is_initialized_ = value; }
I wonder if ClientSocketPool should be a friend to this class, and these methods
should be protected?

http://codereview.chromium.org/2994003/diff/1/8
File net/socket/client_socket_pool_base.cc (right):

http://codereview.chromium.org/2994003/diff/1/8#newcode547
net/socket/client_socket_pool_base.cc:547: //        waken.  This isn't optimal,
but there is no starvation, so to avoid
nit: /waken/woken/

http://codereview.chromium.org/2994003/diff/1/8#newcode677
net/socket/client_socket_pool_base.cc:677: if (group->pending_requests.empty())
nit:  the only caller of ProcessPendingRequest() already did this check. 
Probably do it in one or the other, not both.

http://codereview.chromium.org/2994003/diff/1/8#newcode812
net/socket/client_socket_pool_base.cc:812: handle->set_is_initialized(true);
I wonder if the initialized flag should be controlled only by the handle, rather
than by the CSP...

[willchan no longer on Chromium, 2010-07-13 22:44:09]

http://codereview.chromium.org/2994003/diff/1/5
File net/socket/client_socket_handle.cc (right):

http://codereview.chromium.org/2994003/diff/1/5#newcode43
net/socket/client_socket_handle.cc:43: // If we did not get initialized yet, so
we've got a socket request pending.
On 2010/07/13 21:28:12, Mike Belshe wrote:
> nit:  this comment reads funny - remove the "so".

Done.

http://codereview.chromium.org/2994003/diff/1/5#newcode76
net/socket/client_socket_handle.cc:76: CHECK_NE(ERR_IO_PENDING, result);
On 2010/07/13 21:28:12, Mike Belshe wrote:
> Should this function set is_initialized_ to true?

Yeah, that's much better.  Thanks for the suggestion.

http://codereview.chromium.org/2994003/diff/1/6
File net/socket/client_socket_handle.h (right):

http://codereview.chromium.org/2994003/diff/1/6#newcode89
net/socket/client_socket_handle.h:89: // Returns true when Init() has completed
successfully.
On 2010/07/13 21:28:12, Mike Belshe wrote:
> Since set_is_initialized() is public, this comment is not quite true anymore?

Done.

http://codereview.chromium.org/2994003/diff/1/6#newcode99
net/socket/client_socket_handle.h:99: void set_is_initialized(bool value) {
is_initialized_ = value; }
On 2010/07/13 21:28:12, Mike Belshe wrote:
> I wonder if ClientSocketPool should be a friend to this class, and these
methods
> should be protected?

Hm, I originally unfriended this class since I hate C++ friends, but
ClientSocketHandle and ClientSocketPool are getting pretty tightly bound, so
perhaps it makes sense.  I'm going to defer on this change for now.

http://codereview.chromium.org/2994003/diff/1/8
File net/socket/client_socket_pool_base.cc (right):

http://codereview.chromium.org/2994003/diff/1/8#newcode547
net/socket/client_socket_pool_base.cc:547: //        waken.  This isn't optimal,
but there is no starvation, so to avoid
On 2010/07/13 21:28:12, Mike Belshe wrote:
> nit: /waken/woken/

Done.

http://codereview.chromium.org/2994003/diff/1/8#newcode677
net/socket/client_socket_pool_base.cc:677: if (group->pending_requests.empty())
On 2010/07/13 21:28:12, Mike Belshe wrote:
> nit:  the only caller of ProcessPendingRequest() already did this check. 
> Probably do it in one or the other, not both.

Done.

http://codereview.chromium.org/2994003/diff/1/8#newcode812
net/socket/client_socket_pool_base.cc:812: handle->set_is_initialized(true);
On 2010/07/13 21:28:12, Mike Belshe wrote:
> I wonder if the initialized flag should be controlled only by the handle,
rather
> than by the CSP...

Done.

[eroman, 2010-07-13 23:41:10]

Have you considered making Request refcounted instead of having the pending
callbacks map?

(i.e. post a task to execute some InvokeCallback() method on the Request, which
is a no-op if it has been cancelled).

http://codereview.chromium.org/2994003/diff/8001/9001
File net/http/http_network_transaction_unittest.cc (right):

http://codereview.chromium.org/2994003/diff/8001/9001#newcode262
net/http/http_network_transaction_unittest.cc:262: ClientSocketHandle* handle) {
}
nit: as long as you are changing this line, can you make it {} ?

http://codereview.chromium.org/2994003/diff/8001/9004
File net/socket/client_socket_handle.cc (right):

http://codereview.chromium.org/2994003/diff/8001/9004#newcode18
net/socket/client_socket_handle.cc:18: socket_(NULL),
unrelated nit: socket_ is scoped_ptr<>, this initializer isn't strictly
necessary.

http://codereview.chromium.org/2994003/diff/8001/9004#newcode37
net/socket/client_socket_handle.cc:37:
socket_->NetLog().EndEvent(NetLog::TYPE_SOCKET_IN_USE, NULL);
See HandleInitCompletion() -- is_initialized_ may be set when socket_ is NULL,
causing a crash here.

http://codereview.chromium.org/2994003/diff/8001/9004#newcode81
net/socket/client_socket_handle.cc:81: is_initialized_ = true;
This seems confused.

The documentation for is_initialized() says: "returns true when Init() has
completed successfully.", however here you are setting true when result != OK,
i.e. on unsuccessful initialization.

http://codereview.chromium.org/2994003/diff/8001/9007
File net/socket/client_socket_pool_base.cc (right):

http://codereview.chromium.org/2994003/diff/8001/9007#newcode194
net/socket/client_socket_pool_base.cc:194: ClientSocketHandle* handle =
request->handle();
nit: can you condense this inside the check?

[willchan no longer on Chromium, 2010-07-13 23:48:27]

http://codereview.chromium.org/2994003/diff/8001/9001
File net/http/http_network_transaction_unittest.cc (right):

http://codereview.chromium.org/2994003/diff/8001/9001#newcode262
net/http/http_network_transaction_unittest.cc:262: ClientSocketHandle* handle) {
}
On 2010/07/13 23:41:10, eroman wrote:
> nit: as long as you are changing this line, can you make it {} ?

Done.

http://codereview.chromium.org/2994003/diff/8001/9004
File net/socket/client_socket_handle.cc (right):

http://codereview.chromium.org/2994003/diff/8001/9004#newcode18
net/socket/client_socket_handle.cc:18: socket_(NULL),
On 2010/07/13 23:41:10, eroman wrote:
> unrelated nit: socket_ is scoped_ptr<>, this initializer isn't strictly
> necessary.

Done.

http://codereview.chromium.org/2994003/diff/8001/9004#newcode37
net/socket/client_socket_handle.cc:37:
socket_->NetLog().EndEvent(NetLog::TYPE_SOCKET_IN_USE, NULL);
On 2010/07/13 23:41:10, eroman wrote:
> See HandleInitCompletion() -- is_initialized_ may be set when socket_ is NULL,
> causing a crash here.

I think you may have misread it.  is_initialized_ is set to true when
!!socket_.get(), so, only when socket_ is non-NULL.

http://codereview.chromium.org/2994003/diff/8001/9004#newcode81
net/socket/client_socket_handle.cc:81: is_initialized_ = true;
On 2010/07/13 23:41:10, eroman wrote:
> This seems confused.
> 
> The documentation for is_initialized() says: "returns true when Init() has
> completed successfully.", however here you are setting true when result != OK,
> i.e. on unsuccessful initialization.

Yeah, this is due to vandebo's change.  We now have "recoverable" socket errors,
for proxy sockets and http auth.  It's very non-obvious and I'm open to
suggestions on how to make it better, but we haven't been able to come up with
something better.

http://codereview.chromium.org/2994003/diff/8001/9007
File net/socket/client_socket_pool_base.cc (right):

http://codereview.chromium.org/2994003/diff/8001/9007#newcode194
net/socket/client_socket_pool_base.cc:194: ClientSocketHandle* handle =
request->handle();
On 2010/07/13 23:41:10, eroman wrote:
> nit: can you condense this inside the check?

Done.

[eroman, 2010-07-14 00:02:54]

http://codereview.chromium.org/2994003/diff/8001/9004
File net/socket/client_socket_handle.cc (right):

http://codereview.chromium.org/2994003/diff/8001/9004#newcode37
net/socket/client_socket_handle.cc:37:
socket_->NetLog().EndEvent(NetLog::TYPE_SOCKET_IN_USE, NULL);
On 2010/07/13 23:48:27, willchan wrote:
> On 2010/07/13 23:41:10, eroman wrote:
> > See HandleInitCompletion() -- is_initialized_ may be set when socket_ is
NULL,
> > causing a crash here.
> 
> I think you may have misread it.  is_initialized_ is set to true when
> !!socket_.get(), so, only when socket_ is non-NULL.

Oh yeah my bad, somehow I saw that as NULL.

[eroman, 2010-07-14 21:34:19]

http://codereview.chromium.org/2994003/diff/15001/16008
File net/socket/client_socket_pool_base.h (right):

http://codereview.chromium.org/2994003/diff/15001/16008#newcode421
net/socket/client_socket_pool_base.h:421: // exist in |pending_callback_map_|. 
We look up the callback and result code
This is very subtle.

The concern I have is in using a memory address as the key into the callback
map. Since across cancellations, it is possible for the key
(ClientSocketHandle*) to be freed. Hence we could allocate a new
ClientSocketHandle that just happens to have the same memory address as the old
cancelled one that was just freed, and end up with mismatched expectations
between the posted task and the pendingcallback map.

That said, the situations I considered happen to work, but I worry this won't be
future-proof.

For example, one ordering I considered was if you start off with a
CallbackResultPair keyed by memory address XXX in the pending callback map.

Now say that XXX is completed. An invoke-later task is posted for XXX.
Next, say that XXX is cancelled, so the pending callback result is removed from
the map. Next, a new a new ClientSocketHandle is allocated which happens to have
address XXX as well. We complete this new request right away, and post a second
invoke-callback-later task for XXX.

Now when processing the first invoke-callback-later task, we will end up
completing the second ClientSocketHandle. I guess this ends up being safe
though, since when we run the second invoke-callback task it will just be a
no-op.

Still if things change this could get broken. For example if the |int result|
were ever passed as a parameter of the posttask rather than part of the
pending-callback-map, then this would result in a mistake.

Maybe I am just crazy, but I wanted to bring this up.



Otherwise, LGTM.

[willchan no longer on Chromium, 2010-07-15 16:56:40]

eroman and I talked about this offline.  The plan is to add a
ScopedRunnableMethodFactory into the CallbackResultPair, so we can revoke the
task when it gets cancelled.  I don't think I'll have time to do that today
before I go on leave.  I'll put a TODO there and land as is.

On 2010/07/14 21:34:19, eroman wrote:
> http://codereview.chromium.org/2994003/diff/15001/16008
> File net/socket/client_socket_pool_base.h (right):
> 
> http://codereview.chromium.org/2994003/diff/15001/16008#newcode421
> net/socket/client_socket_pool_base.h:421: // exist in |pending_callback_map_|.

> We look up the callback and result code
> This is very subtle.
> 
> The concern I have is in using a memory address as the key into the callback
> map. Since across cancellations, it is possible for the key
> (ClientSocketHandle*) to be freed. Hence we could allocate a new
> ClientSocketHandle that just happens to have the same memory address as the
old
> cancelled one that was just freed, and end up with mismatched expectations
> between the posted task and the pendingcallback map.
> 
> That said, the situations I considered happen to work, but I worry this won't
be
> future-proof.
> 
> For example, one ordering I considered was if you start off with a
> CallbackResultPair keyed by memory address XXX in the pending callback map.
> 
> Now say that XXX is completed. An invoke-later task is posted for XXX.
> Next, say that XXX is cancelled, so the pending callback result is removed
from
> the map. Next, a new a new ClientSocketHandle is allocated which happens to
have
> address XXX as well. We complete this new request right away, and post a
second
> invoke-callback-later task for XXX.
> 
> Now when processing the first invoke-callback-later task, we will end up
> completing the second ClientSocketHandle. I guess this ends up being safe
> though, since when we run the second invoke-callback task it will just be a
> no-op.
> 
> Still if things change this could get broken. For example if the |int result|
> were ever passed as a parameter of the posttask rather than part of the
> pending-callback-map, then this would result in a mistake.
> 
> Maybe I am just crazy, but I wanted to bring this up.
> 
> 
> 
> Otherwise, LGTM.