net: Add Life of a URLRequest documentation.

net/docs/life-of-a-url-request.md

Index: net/docs/life-of-a-url-request.md
diff --git a/net/docs/life-of-a-url-request.md b/net/docs/life-of-a-url-request.md
new file mode 100644
index 0000000000000000000000000000000000000000..a475085b6bfd6d4028e3bf17a0c1d9c4df231906
--- /dev/null
+++ b/net/docs/life-of-a-url-request.md
@@ -0,0 +1,473 @@
+# Life of a URLRequest
+
+This document is intended as an overview of the core layers of the network
+stack, their basic responsibilities, how they fit together, and where some of
+the pain points are, without going into too much detail. Though it touches a
+bit on the renderer process and the content/loader stack, the focus is on net/
+itself.
+
+It's particularly targeted at people new to the Chrome network stack, but
+should also be useful for team members who may be experts at some parts of the
+stack, but are largely unfamiliar with other components. It starts by walking
+through how a basic request issued by Blink works its way through the network
+stack, and then moves on to discuss how various components plug in.
+
+
+# Anatomy of the Network Stack
+
+The main top-level network stack object is the URLRequextContext. The context
+has non-owning pointers to everything needed to create and issue a URLRequest.
+The context must outlive all requests that use it. Creating a context is a
+rather complicated process, and it's recommended that most embedders use
+URLRequestContextBuilder to do this. Chrome itself has several request
+contexts that the network stack team owns:
+
+* The proxy URLRequestContext, owned by the IOThread and used to get PAC
+scripts while avoiding re-entrancy.
+* The system URLRequestContext, also owned by the IOThread, used for requests
+that aren't associated with a profile.
+* Each profile, including incognito profiles, has a number of URLRequestContexts
+that are created as needed:
+    * The main URLRequestContext is mostly created in ProfileIOData, though it
+    has a couple components that are passed in from content's StoragePartition
+    code. Several other components are shared with the system URLRequestContext,
+    like the HostResolver.
+    * Each non-incognito profile also has a media request context, which uses a
+    different on-disk cache than the main request context. This prevents a
+    single huge media file from evicting everything else in the cache.
+    * On desktop platforms, each profile has a request context for extensions.
+    * Each profile has two contexts for each isolated app (One for media, one
+    for everything else).
+
+The "HttpNetworkSession" is another major network stack object. It has

[xunjieli, 2015/07/10 20:32:26]

nit: I think the quotes can be removed, since quotes aren't used in any other
places when referring to class names.

[mmenke, 2015/07/10 21:37:32]

Done.

+pointers to the network stack objects that more directly deal with sockets, and
+their dependendencies. Its main objects are the HttpStreamFactory, the socket
+pools, and the SPDY/QUIC session pools.
+
+This document does not mention either of these objects much, but at layers
+above the HttpStreamFactory, objects often grab their dependencies from the
+URLRequestContext, while the HttpStreamFactory and layers below it generally
+get their dependencies from the HttpNetworkSession.
+
+
+# How many "Delegates"?
+
+The network stack informs the embedder of important events for a request using
+two main interfaces: The URLRequest::Delegate interface and the NetworkDelegate

[xunjieli, 2015/07/10 20:32:26]

nit: lowercase "The" after the colon.

[mmenke, 2015/07/10 21:37:32]

Done.

+interface.
+
+The URLRequest::Delegate interface consists of small set callbacks needed to let

[xunjieli, 2015/07/10 20:32:26]

[a] small set [of] callbacks?

[mmenke, 2015/07/10 21:37:32]

Done.

+the embedder drive a request forward. URLRequest::Delegates generally own the
+URLRequest.
+
+The NetworkDelegate is a single object shared by all requests, and includes
+callbacks corresponding to most of the URLRequest::Delegate's callbacks, as well
+as an assortment of other methods. The NetworkDelegate is optional, the

[xunjieli, 2015/07/10 20:32:26]

nit: Add a conjunction in "The NetworkDelegate is optional, [and/but] the
URLRequest::Delegate is not." 

[mmenke, 2015/07/10 21:37:31]

Done.

+URLRequest::Delegate is not.
+
+
+# Overview
+
+A request for data is normally dispatched from the renderer to the browser
+process. There a URLRequest is created to drive the request. A
+protocol-specific job (e.g. HTTP, data, file) is attached to the request. That
+job first checks the cache, and then creates a network connection, if necessary,
+to actually fetch the data. That connection object interacts with network socket
+pools to potentially re-use sockets; the socket pools create and connect a
+socket if there is no appropriate existing socket. Once that socket exists, the
+HTTP request is dispatched, the response read and parsed, and the result
+returned back up the stack and sent over to the renderer.
+
+Of course, it's not quite that simple :-}.
+
+Consider a simple request issued by the renderer process. Suppose it's an HTTP
+request, the response is uncompressed, and no matching entry in the cache.
+
+### Request Starts in a Child Process

[xunjieli, 2015/07/10 20:32:26]

I find it a bit confusing that this doc mentioned "renderer process" and "child
process", as if they are interchangeable. Though later in the doc (Line 173), I
see that "child process" isn't necessarily a "renderer process".

Could you clarify a bit on what kinds of "child process" there are? I am
assuming they all use ResourceDispatcher & IPC to communicate with RDH. Is this
right?

Reading
http://dev.chromium.org/developers/design-documents/multi-process-architecture
does not enlighten me in this aspect, so I'd really appreciate a sentence or two
on whether the "child process" referred in this doc also includes other
processes like plugin processes.
 

[mmenke, 2015/07/10 21:37:32]

That's right.
I added a paragraph in the details section about the child processes, and switch
to "child" rather than "renderer" in most places, to be clearer.  Don't want to
go too much into processes, but since this is targeted at new Chromites, think
it makes sense to touch on them a little.

+
+* ResourceDispatcher to creates an IPCResourceLoaderBridge.

[xunjieli, 2015/07/10 20:32:26]

nit: remove "to".

[mmenke, 2015/07/10 21:37:32]

Done.

+* The IPCResourceLoaderBridge asks ResourceDispatcher to start the request.
+* ResourceDispatcher sends an IPC to the ResourceDispatcherHost in the
+browser process.
+
+### ResourceDispatcherHost set up the request in the Browser Process
+
+* ResourceDispatcherHost uses the URLRequestContext to create the URLRequest.
+* ResourceDispatcherHost creates a ResourceLoader and ResourceHandler chain to

[xunjieli, 2015/07/10 20:32:26]

nit: and [a] ResourceHandler chain.
(It reads as if it is a chain comprised of a ResourceLoader and a
ResourceHandler, but I think you meant a ResourceLoader and many handlers)

[mmenke, 2015/07/10 21:37:31]

Done, and rephrased to hopefully be clearer.

+manage the URLRequest.
+* ResourceLoader starts the URLRequest.
+
+### Check the Cache, Request an HttpStream
+
+* The URLRequest asks the URLRequestJobFactory to create a URLRequestJob,
+usually a URLRequestHttpJob for HTTP/HTTPS requests.
+* The URLRequestHttpJob asks the HttpCache to create an HttpTransaction
+(always an HttpCache::Transaction).
+* The HttpCache::Transaction sees there's no cache entry for the request,
+and creates an HttpNetworkTransaction.
+* The HttpNetworkTransaction calls into the HttpStreamFactory to request an
+HttpStream.
+
+### Create an HttpStream
+* HttpStreamFactory creates an HttpStreamFactoryImpl::Job.
+* HttpStreamFactoryImpl::Job calls into the TransportClientSocketPool to
+populate an ClientSocketHandle.
+* TransportClientSocketPool has no idle sockets, so it creates a
+TransportConnectJob and starts it.
+* TransportConnectJob creates a StreamSocket and establishes a connection.
+* TransportClientSocketPool puts the StreamSocket in the ClientSocketHandle,
+and calls into HttpStreamFactoryImpl::Job.
+* HttpStreamFactoryImpl::Job creates an HttpBasicStream, which takes
+ownership of the ClientSocketHandle.
+* It returns the HttpBasicStream to the HttpNetworkTransaction.
+
+### Send request and read the response headers
+
+* HttpNetworkTransaction gives the request headers to the HttpBasicStream,
+and tells it to start the request.
+* HttpBasicStream sends the request, and waits for the response.
+* The HttpBasicStream sends the response headers back to the
+HttpNetworkTransaction.
+* The response headers are sent up to the URLRequest, to the ResourceLoader,
+through the ResourceHandler stack.

[xunjieli, 2015/07/10 20:32:26]

nit: I would replace "stack" with "chain" so there is one fewer variation.

[mmenke, 2015/07/10 21:37:32]

Done.

+* They're then sent by the AsyncResourceHandler to the ResourceDispatcher.

[xunjieli, 2015/07/10 20:32:26]

I got confused by the sudden mention of AsyncResourceHandler here when I read
the doc first time, because I did not immediately associate AsyncResourceHandler
with the chain of ResourceHandlers. You mentioned later in the doc that
AsyncResourceHandler is the end of the chain. I would move that sentence here.

[mmenke, 2015/07/10 21:37:32]

Clarified that the AsyncResourceHandler is the last in the chain.

+
+### Response body is read
+
+* AsyncResourceHandler allocates a 512k ring buffer of shared memory to read
+the body of the request.
+* AsyncResourceHandler tells the ResourceLoader to read the response body to
+the buffer, 32kB at a time.
+* AsyncResourceHandler informs the ResourceDispatcher of each read.

[xunjieli, 2015/07/10 20:32:26]

How does AsyncResourceHandler inform the ResourceDispatcher, which is in the
child process? Should we mention this is through IPC?

[mmenke, 2015/07/10 21:37:32]

Done.

+* ResourceDispatcher tells the AsyncResourceHandler when it's done with the
+data with each read, so it knows when parts of the buffer can be reused.
+
+### URLRequest is Destroyed

[xunjieli, 2015/07/10 20:32:26]

nit: s/Destroyed/destroyed. 
I think we should stick with lowercase except first word or change all the words
for the section titles to have uppercase first letter. 

[mmenke, 2015/07/10 21:37:32]

Done to all the "###" titles (Hrm, I was really inconsistent there).

+
+* When complete, the RDH deletes the ResourceLoader, which deletes the

[xunjieli, 2015/07/10 20:32:26]

Does the chain of ResourceHandlers also get deleted?

[mmenke, 2015/07/10 21:37:32]

Done.

+URLRequest.
+* During destruction, the HttpNetworkTransaction determines if the socket is
+reusable, and if so, tells the HttpBasicStream to return it to the socket pool.
+
+# Details

[xunjieli, 2015/07/10 20:32:26]

I thought the life of a url is ended, because I didn't realize that the sections
above are **sub** sections of "Overview", and the real details begin here.

Maybe add some hints here by numbering the sections? For example, Overview can
be numbered 1, and subsections numbered from 1.1 to 1.7; Details can be numbered
2, and subsections numbered from 2.1 to 2.7. 

Or maybe combine the subsections of Overview with those of Details into one
subsection? since they exactly correspond?
Maybe something like:
# Request starts in a Child Process
[Indent] Bullet points from subsection in Overview
[no Indent] Paragraph from subsection in Details.
  
When I was reading Overview subsections, I had many questions which are answered
in Details subsections, so I think it will help to reduce cognitive load if the
contents are together.

+
+Continuing with a "simple" URLRequest, here's a bit more detail on how things
+work.
+
+### Request Starts in a Child Process
+
+Each child process has at most one ResourceDispatcher, which is responsible for
+all URL request-related communication with the browser process. When something
+in the renderer needs to issue a resource request, it calls into the
+ResourceDispatcher, which returns an IPCResourceLoaderBridge to the caller.
+The caller uses the bridge to start a request. When started, the
+ResourceDispatcher assigns the request a per-renderer ID, and then sends the
+ID, along with all information needed to issue the request, to the
+ResourceDispatcherHost in the browser process.
+
+### ResourceDispatcherHost set up the request in the Browser Process
+
+The ResourceDispatcherHost (RDH), along with most of the network stack, lives
+on the browser process's IO thread. The browser process only has one RDH,
+which is responsible for handling all network requests initiated by
+ResourceDispatchers in all child processes, not just renderer process.
+Browser-initiated don't go through the RDH, with some exceptions.

[xunjieli, 2015/07/10 20:32:26]

nit: s/"Browser-initiated"/"requests initiated by the browser process" or
something similar to make it clearer that we are talking about requests and
browser process here.

[mmenke, 2015/07/10 21:37:32]

Done.

+
+When the RDH sees the request, it calls into a URLRequestContext to create the
+URLRequest. The URLRequestContext has pointers to all the network stack
+objects needed to issue the request over the network, such as the cache, cookie
+store, and host resolver. The RDH then creates a chain of ResourceHandlers
+each of which can monitor/modify/delay/cancel the URLRequest and the
+information it returns. The only one of these I'll talk about here is the
+AsyncResourceHandler, which is the last ResourceHandler in the chain. The RDH
+then creates a ResourceLoader (Which is the URLRequest::Delegate), passes
+ownership of the URLRequest and the ResourceHandler chain to it, and then starts
+the ResourceLoader.
+
+The ResourceLoader checks that none of the ResourceHandlers want to cancel,
+modify, or delay the request, and then finally starts the URLRequest.
+
+### Check the Cache, Request an HttpStream
+
+The URLRequest then calls into the URLRequestJobFactory to create a
+URLRequestJob and then starts it. In the case of an HTTP or HTTPS request, this
+will be a URLRequestHttpJob. The URLRequestHttpJob attaches cookies to the
+request, if needed.
+
+The URLRequestHttpJob calls into the HttpCache to create an
+HttpCache::Transaction. If there's no matching entry in the cache, the
+HttpCache::Transaction will just call into the HttpNetworkLayer to create an
+HttpNetworkTransaction, and transparently wrap it.  The HttpNetworkTransaction
+then calls into the HttpStreamFactory to request an HttpStream to the server.
+
+### Create an HttpStream
+
+The HttpStreamFactoryImpl::Job creates a ClientSocketHandle to hold a socket,
+once connected, and passes it into the ClientSocketPoolManager. The
+ClientSocketPoolManager assembles the TransportSocketParams needed to
+establish the connection and creates a group name ("host:port") used to
+identify sockets that can be used interchangeably.
+
+The ClientSocketPoolManager directs the request to the
+TransportClientSocketPool, since there's no proxy and it will be using
+HTTP/1.x. The pool sends it on to the
+ClientSocketPoolBase<TransportSocketParams> it wraps, which sends it on to its

[xunjieli, 2015/07/10 20:32:26]

I am not 100% sure on what the three "it" refer to in this sentence.
The first "it" is probably the request. Not clear whether second 
"it" refers to... I think the second "it" is the pool, but I probably got it
wrong.

[mmenke, 2015/07/10 21:37:32]

Good point.  Removed almost all the its.

+ClientSocketPoolBaseHelper. If there isn't already an idle connection, and there
+are available socket slots, the ClientSocketPoolBaseHelper will create a new
+TransportConnectJob using the aforementioned params object. This Job will do the
+actual DNS lookup by calling into the HostResolverImpl, if needed, and then
+finally establish a connection.
+
+Once the socket is connected, ownership of the socket is passed to the
+ClientSocketHandle. The HttpStreamFactoryImpl::Job is then informed the
+connection attempt succeeded, and it then creates an HttpBasicStream, which
+takes ownership of the ClientSocketHandle. It then passes ownership of the
+HttpBasicStream back to the HttpNetworkTransaction.
+
+### Send request and read the response headers
+
+The Transaction passes the request headers to the HttpBasicStream, which uses an
+HttpStreamParser to (finally) format the request headers and body (if present)
+and send them to the server.
+
+The HttpStreamParser waits to receive the response and then parses the HTTP/1.x
+response headers, and then passes them up through both Transaction classes

[xunjieli, 2015/07/10 20:32:26]

nit: I'd spell out what these two Transaction classes are. I am assuming it is
the HttpNetworkTransaction and HttpCache::Transaction. But if I didn't pay much
attention, I probably would be lost here.

[mmenke, 2015/07/10 21:37:32]

Done.

+to the URLRequestHttpJob. The URLRequestHttpJob saves any cookies, if needed,
+and then passes the headers up to the URLRequest and on to the ResourceLoader.
+
+The ResourceLoader passes them through the chain of ResourceHandlers, and then
+they make their way to the AsyncResourceHandler. The AsyncResourceHandler uses
+the renderer process ID ("child ID") to figure out which process the request
+was associated with, and then sends the headers along with the request ID to
+that process's ResourceDispatcher. The ResourceDispatcher uses the ID to
+figure out which IPCResourceLoaderBridge the headers should be sent to, which
+sends them on to whatever created the IPCResourceLoaderBridge in the first
+place.
+
+### Response body is read
+
+Without waiting to hear back from the ResourceDispatcher, the ResourceLoader
+tells its ResourceHandler chain to allocate memory to receive the response
+body. The AsyncResourceHandler creates a 512KB ring buffer of shared memory,
+and then passes the first 32KB of it to the ResourceLoader for the first read.
+The ResourceLoader then passes a 32KB body read request down through the
+URLRequest all the way down to the HttpResponseParser. Once some data is read,
+possibly less than 32KB, the number of bytes read makes its way back to the
+AsyncResourceHandler, which passes the shared memory buffer and the offset and
+amount of data read to the renderer process.
+
+The AsyncResourceHandler relies on ACKs from the renderer to prevent it from
+overwriting data that the renderer has yet to consume. This process repeats
+until the response body is completely read.
+
+### URLRequest is destroyed
+
+When the URLRequest informs the ResourceLoader it's complete, the
+ResourceLoader tells the ResourceHandlers, and the AsyncResourceHandler tells
+the ResourceDispatcher the request is complete. The RDH then deletes
+ResourceLoader, which deletes the URLRequest.

[xunjieli, 2015/07/10 20:32:27]

nit: I'd mention what happens to the chain of ResourceHandlers. Do they also get
deleted by RDH?

[mmenke, 2015/07/10 21:37:32]

Done.

+
+When the HttpNetworkTransaction is being torn down, it figures out if the
+socket is reusable. If not, it tells the HttpBasicStream to close the socket.
+Either way, the ClientSocketHandle returns the socket is then returned to the
+socket pool, either for reuse or so the socket pool knows it has another free
+socket slot.
+
+
+# Additional Topics
+
+## HTTP Cache
+
+The HttpCache::Transaction sits between the URLRequestHttpJob and the
+HttpNetworkTransaction, and implements the HttpTransaction interface, just like
+the HttpNetworkTransaction. The HttpCache::Transaction checks if a request can
+be served out of the cache. If a request needs to be revalidated, it handles
+sending a 204 revalidation request over the network. It may also break a range
+request into multiple cached and non-cached contiguous chunks, and may issue
+multiple network requests for a single range URLRequest.
+
+One important detail is that it has a read/write lock for each URL. The lock
+technically allows multiple reads at once, but since an HttpCache::Transaction
+always grabs the lock for writing and reading before downgrading it to a read
+only lock, all requests for the same URL are effectively done serially. Blink
+merges requests for the same URL in many cases, which mitigates this problem to
+some extent.
+
+The HttpCache::Transaction uses one of three disk_cache::Backends to actually
+store the cache's index and files: The in memory backend, the blockfile cache
+backend, and the simple cache backend. The first is used in incognito. The
+latter two are both stored on disk, and are used on different platforms.
+
+## Cancellation
+
+A request can be cancelled by the renderer process Blink or by any of the

[xunjieli, 2015/07/10 20:32:26]

Why is the "renderer process" called Blink? I thought Blink is the rendering
engine. Can we use them interchangeably?

[mmenke, 2015/07/10 21:37:32]

Removed the Blink (Was supposed to be "Blink in the renderer process", but it
can be cancelled by a lot of other things.  Blink is a lot more than rendering
engine (Handles sending requests for subresources, for instance)...But not going
to get into that here.

+ResourceHandlers through the ResourceLoader. When the cancellation message

[xunjieli, 2015/07/10 20:32:26]

nit: maybe add the word "chain" here? 
e.g. any ResourceHandler on the chain of ResourceHandlers? A little bit verbose,
but it makes the concept a bit clearer.

[mmenke, 2015/07/10 21:37:32]

Done.

+reaches the URLRequest, it passes on the fact it's been cancelled back to the
+ResourceLoader, which then sends the message down the ResourceHandler chain.
+
+When an HttpNetworkTransaction for a cancelled request is being torn down, it
+figures out if the socket the HttpStream owns can potentially be reused, based
+on the protocol (HTTP / SPDY / QUIC) and any received headers. If the socket
+potentially can be reused, an HttpResponseBodyDrainer is created to try and
+read any remaining body bytes of the HttpStream, if any, before returning the
+socket to the SocketPool. If this takes too long, or there's an error, the
+socket is closed instead. Since this all happens at the layer below the cache,
+any drained bytes are not written to the cache, and as far as the cache layer is
+concerned, it only has a partial response.
+
+## Redirects
+
+The URLRequestHttpJob checks if headers indicate a redirect when it receives
+them from the next layer down (Typically the HttpCache::Transaction). If they
+indicate a redirect, it tells the cache the response is complete, ignoring the
+body, so the cache only has the headers. The cache then treats it as a complete
+entry, even if the headers indicated there will be a body.
+
+The URLRequestHttpJob then checks if the URLRequest if the request should be
+followed. First it checks the scheme. Then it informs the ResourceLoader
+about the redirect, to give it a chance to cancel the request. The information
+makes its way down through the AsyncResourceHandler to the ResourceDispatcher
+and on into Blink, which checks if the redirect should be followed.
+
+The ResourceDispatcher then asynchronously sends a message back to either
+follow the redirect or cancel the request. In either case, the old
+HttpTransaction is destroyed, and the HttpNetworkTransaction attempts to drain
+the socket for reuse, just as in the cancellation case. If the redirect is
+followed, the URLRequest calls into the URLRequestJobFactory to create a new
+URLRequestJob, and then starts it.
+
+## Filters (gzip, SDCH, etc)
+
+When the URLRequestHttpJob receives headers, it sends a list of all Content-
+Encoding values to Filter::Factory, which creates a (possibly empty) chain of

[xunjieli, 2015/07/10 20:32:26]

nit: I think there is a space between the dash and Encoding when the html is
generated. Maybe move "Content-" down?

[mmenke, 2015/07/10 21:37:32]

Done.

+filters. As body bytes are received, they're passed through the filters at the
+URLRequestJob layer and the decoded bytes are passed back to the embedder.
+
+Since this is done above the cache layer, the cache stores the responses prior
+to decompression. As a result, if files aren't compressed over the wire, they
+aren't compressed in the cache, either. This behavior can also create problems
+when responses are SDCH compressed, as a dictionary and a cached file encoded
+using it may have different lifetimes.
+
+## Socket Pools
+
+The ClientSocketPoolManager is responsible for assembling the parameters needed
+to connect a socket, and then sending the request to the right socket pool.
+Each socket request sent to a socket pool comes with a socket params object, a
+ClientSocketHandle, and a "group name". The params object contains all the
+information a ConnectJob needs to create a connection of a given type, and
+different types of socket pools take different params types. The
+ClientSocketHandle will take temporary ownership of the socket, once connected
+socket, and return it to the socket pool when done. All connections with the
+same group name in the same pool can be used to service the same connection
+request, so it consists of host, port, protocol, and whether "privacy mode" is
+used for requests using the socket or not.
+
+All socket pool classes derive from the ClientSocketPoolBase<SocketParamType>.
+The ClientSocketPoolBase handles managing sockets - which requests to create
+sockets for, which requests get connected sockets first, which sockets belong
+to which groups, connection limits per group, keeping track of and closing idle
+sockets, etc. Each ClientSocketPoolBase subclass has its own ConnectJob type,
+which establishes a connection using the socket params, before the pool hands
+out the connected socket.
+
+## Socket Pool Layering
+
+Some socket pools are layered on top other socket pools. This is done when a
+"socket" in a higher layer needs to establish a connection in a lower level
+pool and then take ownership of it as part of its connection process. See later
+sections for examples. There are a couple additional complexities here.
+
+From the perspective of the lower layer pool, all of its sockets that a higher
+layer pools owns are actively in use, even when the higher layer pool considers
+them idle. As a result, when a lower layer pool is at its connection limit and
+needs to make a new connection, it will ask any higher layer pools pools to
+close an idle connection if they have one, so it can make a new connection.
+
+Sockets in the higher layer pool must have their own distinct group name in the

[xunjieli, 2015/07/10 20:32:26]

This sentence is a bit hard to understand. Consider rephrasing a bit? 

Sockets in higher layer pools can be used in a lower layer pool, right? Maybe it
is clearer to just say that the sockets in different layer pools cannot share
the same group name.

[mmenke, 2015/07/10 21:37:32]

Tried to make it a bit clearer.

+lower layer pool as well. This is needed so the lower layer pool won't, for
+example, group SSL and HTTP connections to the same port together.
+
+## SSL
+
+When an SSL connection is needed, the ClientSocketPoolManager assembles the
+parameters needed both to connect the TCP socket and establish an SSL
+connection. It then passes them to the SSLClientSocketPool, which creates
+an SSLConnectJob using them. The SSLConnectJob's first step is to call into the
+TransportSocketPool to establish a TCP connection.
+
+Once a connection is established by the lower layered pool, the SSLConnectJob
+then starts SSL negotiation. Once that's done, the SSL socket is passed back to
+the HttpStreamFactoryImpl::Job that initiated the request, and things proceed
+just as with HTTP. When complete, the socket is returned to the
+SSLClientSocketPool.
+
+## Proxies
+
+The first step the HttpStreamFactoryImpl::Job performs, just before calling
+into the ClientSocketPoolManager to create a socket, is to check with the
+ProxyService to see if a proxy is needed for the URL it's been given. The
+ClientSocketPoolManager then uses this information to find the correct proxy
+socket pool to send the request to.
+
+TODO(mmenke): Discuss proxy configurations, WPAD, tracing proxy resolver.
+
+## Proxy Socket Pools
+
+Each SOCKS or HTTP proxy has its own completely independent set of socket
+pools. They have their own exclusive TransportSocketPool, their own protocol-
+specific pool above it, and their own SSLSocketPool above that. HTTPS proxies
+also have a second SSLSocketPool between the the HttpProxyClientSocketPool and
+the TransportSocketPool, since they can talk SSL to both the proxy and the
+destination server, layered on top of each other.
+
+## SPDY
+
+Once an SSL connection is established, the HttpStreamFactoryImpl::Job checks if
+SPDY was negotiated over the socket. If so, it creates a SpdySession using the
+socket, and a SpdyHttpStream. The SpdyHttpStream will be passed to the
+HttpNetworkTransaction, which drives the stream as usual.
+
+The SpdySession will be shared with other Jobs connecting to the same server,
+and future Jobs will find the SpdySession before they try to create a
+connection. HttpServerProperties also tracks which servers supported SPDY when
+we last talked to them. We only try to establish a single connection to servers
+we think speak SPDY when multiple HttpStreamFactoryImpl::Jobs are trying to
+connect to them, to avoid wasting resources.
+
+## QUIC
+
+HttpServerProperties also tracks which servers have advertised QUIC support in
+the past. If a server has advertised QUIC support, a second
+HttpStreamFactoryImpl::Job will be created for SPDY, and will be raced against
+the one for HTTP/HTTPS connection. Whichever connects first will be used.
+Existing QUIC sessions will be reused if available.
+
+TODO(mmenke): Discuss SPDY/QUIC proxies?
+
+## Prioritization
+
+URLRequests are assigned a priority on creation. It only comes into play in
+a couple places:
+
+* The ResourceScheduler lives outside net/, and in some cases, delays starting
+low priority requests on a per-tab basis.
+* DNS lookups are initiated based on the highest priority request for a lookup.
+* Socket pools hand out and create sockets on prioritization. However, idle
+sockets will be assigned to low priority requests in preference to creating new
+sockets for higher priority requests.

[xunjieli, 2015/07/10 20:32:26]

I don't understand this sentence. 
Do you instead mean that idle sockets will be assigned to **high** priority
requests in preference to creating new sockets?

[mmenke, 2015/07/10 21:37:31]

No...If we have no free socket slots, and a high priority request to
www.foo.com, and a low priority request to www.bar.com, and then a connection to
www.bar.com becomes idle, we'll service the low priority request, instead of
closing it to service the high priority request.

I've rephrased it a bit.  The detail may be too much information for the level
of detail this document should go into.

+* SPDY and QUIC both support sending priorities over-the-wire.
+
+At the socket pool layer, sockets are only assigned to socket requests once the
+socket is connected and SSL is negotiated, if needed. This is done so that if
+a higher priority request for a group reaches the socket pool before a
+connection is established, the first usable connection goes to the highest
+priority socket request.
+
+## Non-HTTP schemes
+
+The URLRequestJobFactory has a ProtocolHander for each supported scheme.
+Non-HTTP URLRequests have their own ProtocolHandlers. Some are implemented in
+net/, (like FTP, file, and data, though blink handles some data URLs
+internally), and others are implemented in content/ or chrome (like blob,
+chrome, and chrome-extension).