Implement exponential back-off mechanism and enforce it at the URLRequestHttpJob level.

net/url_request/request_throttler_manager.h

+// Copyright (c) 2010 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef NET_URL_REQUEST_REQUEST_THROTTLER_MANAGER_H_
+#define NET_URL_REQUEST_REQUEST_THROTTLER_MANAGER_H_
+
+#include <map>
+#include <string>
+
+#include "base/lock.h"
+#include "base/scoped_ptr.h"
+#include "base/singleton.h"
+#include "googleurl/src/gurl.h"
+#include "net/url_request/request_throttler_entry.h"
+
+// Class that registers a request throttler entry for each URL being accessed in
+// order to supervise traffic. URL requests for HTTP contents should register
+// their URLs in this request throttler manager on each request.
+class RequestThrottlerManager {
+ public:
+  // Must be called for every request, returns the request throttler entry
+  // associated with the URL. The caller must inform this entry of some events.
+  // Please refer to request_throttler_entry.h for further informations.

[Jói, 2010/11/12 00:12:08]

request_throttler_entry.h -> request_throttler_entry_interface.h

[yzshen, 2010/11/12 02:05:46]

Done.

+  scoped_refptr<RequestThrottlerEntryInterface> RegisterRequestUrl(
+      const GURL& url);
+
+  // This method is used by higher level modules which can notify this class if

[Jói, 2010/11/12 00:12:08]

This comment is not clear enough. Suggest the following edit:

Lets higher-level modules, that know how to parse particular response bodies,
notify of receiving malformed content for the given URL. This will be handled by
the throttler as if an HTTP 5xx response had been received to the request, i.e.
it will count as a failure.

[yzshen, 2010/11/12 02:05:46]

Thanks a lot!

+  // the response they received was malformed. It is useful because
+  // in the case where the response header returned 200 response code but had
+  // a malformed content body we will categorize it as a success and so we
+  // defer this decision to the module that had requested the resource.
+  void NotifyRequestBodyWasMalformed(const GURL& url);
+
+  // Registers a new entry in this service and overrides the existing entry (if
+  // any) for the URL.

[Jói, 2010/11/12 00:12:08]

if this is primarily for unit testing, we should say so

[yzshen, 2010/11/12 02:05:46]

It won't be for unit tests only, since I read the recent changes (those
submitted after my last sync) and found that cloud printing needs to use this
method once I sync to the tip of the tree and resolve the conflicts.

+  void OverrideEntry(const GURL& url,
+                     const scoped_refptr<RequestThrottlerEntry>& entry);
+
+  // Explicitly erases an entry.
+  // This is useful to remove those entries which have got infinite lifetime and
+  // thus won't be garbage collected.
+  void EraseEntry(const GURL& url);
+
+ protected:
+  RequestThrottlerManager();
+  virtual ~RequestThrottlerManager();
+
+  // Method that allows us to transform a URL into an ID that can be used in our
+  // map. Resulting IDs will be lowercase and be missing both the query string
+  // and fragment.
+  std::string GetIdFromUrl(const GURL& url);
+
+  // Method that ensures the map gets cleaned from time to time. The period at
+  // which garbage collecting happens is adjustable with the
+  // kRequestBetweenCollecting constant.
+  void GarbageCollectEntriesIfNecessary();
+  // Method that does the actual work of garbage collecting.
+  void GarbageCollectEntries();
+
+  // From each URL we generate an ID composed of the host and path
+  // that allows us to uniquely map an entry to it.
+  typedef std::map<std::string, scoped_refptr<RequestThrottlerEntry> >
+      UrlEntryMap;
+
+  friend struct DefaultSingletonTraits<RequestThrottlerManager>;
+
+  // Map that contains a list of URL ID and their matching
+  // RequestThrottlerEntry.
+  UrlEntryMap url_entries_;
+
+  // Lock to protect those non-static data members.
+  Lock lock_;
+
+ private:
+  // Maximum number of entries that we are willing to collect in our map.
+  static const unsigned int kMaximumNumberOfEntries;
+  // Number of requests that will be made between garbage collection.
+  static const unsigned int kRequestsBetweenCollecting;
+
+  // This keeps track of how many requests have been made. Used with
+  // GarbageCollectEntries.
+  unsigned int requests_since_last_gc_;
+
+  DISALLOW_COPY_AND_ASSIGN(RequestThrottlerManager);
+};
+
+#endif  // NET_URL_REQUEST_REQUEST_THROTTLER_MANAGER_H_