Increase "log too large" threshold in MetricsService.

chrome/browser/metrics/metrics_service.cc

 // Copyright (c) 2012 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 //------------------------------------------------------------------------------
 // Description of the life cycle of a instance of MetricsService.
 //
 //  OVERVIEW
 //
 // A MetricsService instance is typically created at application startup.  It is
@@ skipping 229 matching lines @@
 const int kInitializationDelaySeconds = 30;
 #endif
 
 // The maximum number of events in a log uploaded to the UMA server.
 const int kEventLimit = 2400;
 
 // If an upload fails, and the transmission was over this byte count, then we
 // will discard the log, and not try to retransmit it.  We also don't persist
 // the log to the prefs for transmission during the next chrome session if this
 // limit is exceeded.
-const size_t kUploadLogAvoidRetransmitSize = 50000;
+const size_t kUploadLogAvoidRetransmitSize = 250000;

[Ilya Sherman, 2014/06/03 19:51:49]

This limit is now very close to kStorageByteLimitPerLogType from
metrics_log_manager.cc.  Should we bump that limit as well?  If not, we risk
dropping more small logs for the sake of storing a single large log.

[Ilya Sherman, 2014/06/03 19:51:49]

Optional nit: This might be a little easier to interpret as "250 * 1024" or "250
* 1000".

[Ilya Sherman, 2014/06/03 19:51:49]

What is actually accounting for the larger size of logs today?  I'm concerned
about just bumping this limit.  Should we be thinking about how to reduce the
size of the logs instead?  UMA ideally shouldn't have a significant impact on a
user's network or disk usage...

[Alexei Svitkine (slow), 2014/06/03 19:56:45]

I'm not sure when the old value was chosen or how it was picked. My guess is we
simply have more histograms now than before.

I agree that we should try to reduce the size of logs. We already compress them,
which provides on average a 50% saving for net transmissions. We probably should
persist compressed logs too, rather than uncompressed ones.

And we do want to start deprecating / removing histograms more aggressively.

Additional, there's probably more efficient ways to store them than in local
state - e.g. the base64 round-trip probably doesn't help performance.

Still, I don't think those efforts should block this change. Losing a large
number of logs like we do currently is quite bad.

[Ilya Sherman, 2014/06/03 20:25:19]

I agree that dropping lots of logs is quite bad, and that we should stop doing
that :)

However, while I don't think we should abandon this change in favor of something
more fundamental, I do think it's important to take the time to better
understand the problem before proceeding.  Specifically, I'd like to have a game
plan that goes beyond "let's bump this limit now, and then think about root
causes later" before we go ahead and bump the limit.  Otherwise, there's a good
chance that, despite good intentions, we'll just bump the limit and forget about
it.

I'd really like to better understand what's taking up space in the large logs. 
Is it really histograms?  Is it perhaps profiler data, or maybe even user
actions?  Also, are most logs around 50K now, or are we dealing with some sort
of multi-modal distribution?  Data would help.

In terms of compression, I agree that compression is a good idea.  We should
probably discard logs based on their compressed size, not their inflated size. 
We could then be more conservative about bumping this limit.

It might be the case that after deeper cogitation, we decide that bumping the
limit is really the right thing to do.  If so, that's fine.  I just want to make
sure that we take an appropriate amount of time to consider root causes and
other possible solutions before we just commit to using more resources.

 
 // Interval, in minutes, between state saves.
 const int kSaveStateIntervalMinutes = 5;
 
 // The metrics server's URL.
 const char kServerUrl[] = "https://clients4.google.com/uma/v2";
 
 // The MIME type for the uploaded metrics data.
 const char kMimeType[] = "application/vnd.chrome.uma";
 
@@ skipping 1070 matching lines @@
   RecordCurrentState(local_state_);
 }
 
 void MetricsService::RecordCurrentState(PrefService* pref) {
   pref->SetInt64(prefs::kStabilityLastTimestampSec, Time::Now().ToTimeT());
 
 #if defined(ENABLE_PLUGINS)
   plugin_metrics_provider_->RecordPluginChanges();
 #endif
 }