Implement exponential back-off mechanism and enforce it at the URLRequestHttpJob level.

net/base/run_all_unittests.cc

 // Copyright (c) 2010 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 #include "build/build_config.h"
 #include "base/metrics/histogram.h"
 #include "base/nss_util.h"
 #include "net/base/net_test_suite.h"
+#include "net/request_throttler/request_throttler_manager.h"
 #if defined(OS_WIN)
 #include "net/socket/ssl_client_socket_nss_factory.h"
 #endif
 
 int main(int argc, char** argv) {
   // Record histograms, so we can get histograms data in tests.
   base::StatisticsRecorder recorder;
   NetTestSuite test_suite(argc, argv);
 
 #if defined(OS_WIN)
   // Use NSS for SSL on Windows.  TODO(wtc): this should eventually be hidden
   // inside DefaultClientSocketFactory::CreateSSLClientSocket.
   net::ClientSocketFactory::SetSSLClientSocketFactory(
       net::SSLClientSocketNSSFactory);
   // We want to be sure to init NSPR on the main thread.
   base::EnsureNSPRInit();
 #endif
 
+  Singleton<RequestThrottlerManager>::get()->DisableThrottlingByDefault();

[joi, 2010/11/09 20:07:17]

I'm a bit wary of this. Since we're trying to make the throttler as unobtrusive
as possible, it would be nice to see that most of the tests pass without doing
anything special. The only tests that should really be affected are tests that
are making requests that fail with e.g. 4xx or 5xx HTTP error codes, and this
would hopefully be a small subset of the tests.

Can we keep the throttler enabled and turn it off (or add entries) for just the
tests needed? If it's causing lots of tests to fail right now I think that's
actually a bad sign.

[yzshen, 2010/11/09 22:32:22]

Yes, tests that deal with HTTP failures are a small subset and probably won't
cause trouble.

What bothers us is the sliding window mechanism. It is possible that automatic
tests run so fast that a lot of requests to fetch the same URL are issued in a
short period.

I met two test cases which tried to send 50 requests in a short period. But it
is possible that the sliding window results in flakiness in more obscure way, so
I explicitly disable it.
(Maybe this is a reason why sliding window should not belong to this level?)

+
   return test_suite.Run();
 }