OLD | NEW |
(Empty) | |
| 1 # SDCH |
| 2 |
| 3 "SDCH" stands for "Shared Dictionary Compression over HTTP". It is a |
| 4 protocol for compressing URL responses used when the server and |
| 5 the client share a dictionary that can be referred to for |
| 6 compression/encoding and decompression/decoding. The details of the |
| 7 SDCH protocol are specified in |
| 8 [the spec](https://docs.google.com/a/chromium.org/document/d/1REMkwjXY5yFOkJwtJP
jCMwZ4Shx3D9vfdAytV_KQCUo/edit?pli=1) |
| 9 (soon to be moved to github) but in brief: |
| 10 |
| 11 1. If the client supports SDCH decoding, it advertises "sdch" in the |
| 12 "Accept-Encoding" header. |
| 13 2. If the server could have encoded a response with a dictionary (but |
| 14 didn't, because the client didn't have the dictionary), it includes |
| 15 an advisory "Get-Dictionary: <url>" header in its response. |
| 16 3. If the client has a dictionary that the server has previously |
| 17 advertised as being usable for encoding a particular requests, it |
| 18 advertises that dictionary as being available via an |
| 19 "Avail-Dictionary: <hash>" header in the request. |
| 20 4. If the server chooses to encode a response with a dictionary, it |
| 21 includes "sdch" in a "Content-Encoding" header, in which case the |
| 22 body will reference the dictionary to be used for decoding (which |
| 23 must be one the client advertised in the original request). |
| 24 Encodings may be chained; often responses are SDCH encoded, and then |
| 25 gzip encoded. |
| 26 |
| 27 ## SDCH in Chromium: Overview |
| 28 |
| 29 The SDCH implementation in Chromium is spread across several classes |
| 30 in several different directories: |
| 31 |
| 32 * SdchManager (in net/base): This class contains all |
| 33 dictionaries currently known to Chromium. Each URLRequestContext |
| 34 points to an SdchManager; at the chrome/ level, there is one |
| 35 SdchManager per profile. URLRequestHttpJob consults the SdchManager |
| 36 for what dictionaries should be advertised with a URLRequest, and |
| 37 notifies the SdchManager whenever it sees a "Get-Dictionary" |
| 38 header. The SdchManager does *not* mediate fetching of |
| 39 dictionaries; it is conceptually layered underneath URLRequest and |
| 40 has no knowledge of URLRequests. There are several nested classes of |
| 41 SdchManager (Dictionary, DictionarySet) used in the SDCH |
| 42 implementation; see sdch_manager.h for details. |
| 43 * SdchObserver (in net/base). This is an Abstract Base |
| 44 Class which other classes may implement if those classes wish to |
| 45 receive notifications about SDCH events. Such classes should also |
| 46 register as observers with the SdchManager. |
| 47 * SdchFilter (int net/filter). This class is derived from net::Filter |
| 48 that is used for decoding the SDCH response; it cooperates with |
| 49 SdchManager and the URLRequestJob to decode SDCH encoded responses. |
| 50 * SdchDictionaryFetcher (int net/url_request): |
| 51 This class implements the nuts&bolts of fetching an SDCH |
| 52 dictionary. |
| 53 * SdchOwner (in net/sdch): This class is an SdchObserver. |
| 54 It contains policy for the SDCH implementation, including mediation |
| 55 of fetching dictionaries, prioritization and eviction of |
| 56 dictionaries in response to new fetches, and constraints on the |
| 57 amount of memory that is usable by SDCH dictionaries. It initiates |
| 58 dictionary fetches as appropriate when it receives notification of |
| 59 a "Get-Dictionary" header from the SdchManager. |
| 60 |
| 61 A net/ embedder should instantiate an SdchManager and an SdchOwner, |
| 62 and guarantee that the SdchManager outlive the SdchOwner. |
| 63 |
| 64 Note the layering of the above classes: |
| 65 |
| 66 1. The SdchManager and SdchOwner classes have no knowledge of |
| 67 URLRequests. URLRequest is dependent on those classes, not the |
| 68 reverse. |
| 69 2. SdchDictionaryFetcher is dependent on URLRequest, but is still a |
| 70 utility class exported by the net/ library for use by higher levels. |
| 71 3. SdchOwner manages the entire system on behalf of the embedder. The |
| 72 intent is that the embedder can change policies through methods on |
| 73 SdchOwner, while letting the SdchOwner class take care of policy |
| 74 implementation. |
| 75 |
| 76 ## SDCH in Chromium: Debugging |
| 77 |
| 78 Data that is useful in debugging SDCH problems: |
| 79 |
| 80 * The SDCH UMA prefix is "Sdch3", and histograms that have been found |
| 81 useful for debugging include |
| 82 * ProblemCodes_* (though this requires trawling the source for each bucket). |
| 83 * ResponseCorruptionDetection.{Cached,Uncached}: An attempt to make |
| 84 sense of the twisted mess in SdchFilter::ReadFilteredData mentioned |
| 85 above. |
| 86 * BlacklistReason: Why requests avoid using SDCH when they could use |
| 87 it. |
| 88 * about:net-internals has an SDCH tab, showing loaded dictionaries and |
| 89 other information. Searching in net-internals for "Get-Dictionary", |
| 90 the URLRequest that actually fetches that dictionary, and then the |
| 91 hash of that dictionary (often used as the file name) can also be |
| 92 useful. |
| 93 |
| 94 ## SDCH in Chromium: Gotchas and corner cases |
| 95 |
| 96 There are a couple of known issues in SDCH in Chromium that developers |
| 97 in this space should be aware of: |
| 98 |
| 99 * As noted in the spec above, there have historically been problems |
| 100 with middleboxes stripping or corrupting SDCH encoded responses. |
| 101 For this reason, the protocol requires that if a server is not using |
| 102 SDCH encoding when it has previously advertised the availability of |
| 103 doing such, it includes an "X-SDCH-Encode: 0" header in the |
| 104 response. Servers don't always do this (especially multi-servers), |
| 105 and that can result in failed decodings and requests being dropped |
| 106 on the floor. The code to handle this is a twisted mess (see |
| 107 SdchFilter::ReadFilteredData()) and problems have often been seen |
| 108 from or associated with it. |
| 109 * If the decoding logic trips over a problem, it will often blacklist |
| 110 the server in question, temporarily (if it can recover that request) |
| 111 or permanently (if it can't). This can lead to a mysterious lack of |
| 112 SDCH encoding when it's expected to be present. |
| 113 * The network cache currently stores the response precisely as received from |
| 114 the network. This means that requests that don't advertise SDCH |
| 115 may get a cached value that is SDCH encoded, and requests that do |
| 116 advertise SDCH may get a cached value that is not SDCH encoded. |
| 117 The second case is handled transparently, but the first case may |
| 118 lead to request failure. |
| 119 |
OLD | NEW |