[NTP::SectionOrder] Track category index after it has been moved up.

components/ntp_snippets/content_suggestions_metrics.cc

 // Copyright 2016 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 #include "components/ntp_snippets/content_suggestions_metrics.h"
 
 #include <cmath>
 #include <string>
 #include <type_traits>
 
@@ skipping 37 matching lines @@
 const char kHistogramDismissedVisited[] =
     "NewTabPage.ContentSuggestions.DismissedVisited";
 const char kHistogramArticlesUsageTimeLocal[] =
     "NewTabPage.ContentSuggestions.UsageTimeLocal";
 const char kHistogramVisitDuration[] =
     "NewTabPage.ContentSuggestions.VisitDuration";
 const char kHistogramMoreButtonShown[] =
     "NewTabPage.ContentSuggestions.MoreButtonShown";
 const char kHistogramMoreButtonClicked[] =
     "NewTabPage.ContentSuggestions.MoreButtonClicked";
+const char kHistogramMovedUpCategoryNewIndex[] =
+    "NewTabPage.ContentSuggestions.MovedUpCategoryNewIndex";

[Marc Treib, 2017/02/01 13:04:23]

".CategoryMovedUpNewIndex"?

[vitaliii, 2017/02/01 13:30:18]

I like previous one more.

Which category? Moved up.

The new one feels like an action, "category moved up".

Also old one seems more consistent with "OpenedCategoryIndex".

Do you feel other way?

[Marc Treib, 2017/02/01 13:40:46]

Ah, I wasn't aware of OpenedCategoryIndex. I guess I'd have preferred
CategoryOpenedIndex there as well (or actually just CategoryOpened; the "Index"
is specified by the "units" field in the .xml), so that all the category-related
things start with "Category". But since it's too late for that, I guess it
doesn't matter anymore.

[vitaliii, 2017/02/01 14:19:37]

Acknowledged.

 const char kHistogramCategoryDismissed[] =
     "NewTabPage.ContentSuggestions.CategoryDismissed";
 const char kHistogramContentSuggestionsTimeSinceLastBackgroundFetch[] =
     "NewTabPage.ContentSuggestions.TimeSinceLastBackgroundFetch";
 
 const char kPerCategoryHistogramFormat[] = "%s.%s";
 
 // This mostly corresponds to the KnownCategories enum, but it is contiguous
 // and contains exactly the values to be recorded in UMA. Don't remove or
 // reorder elements, only add new ones at the end (before COUNT), and keep in
@@ skipping 255 matching lines @@
                                  kMaxSuggestionsPerCategory);
   }
 }
 
 void OnSuggestionTargetVisited(Category category, base::TimeDelta visit_time) {
   std::string name =
       GetCategoryHistogramName(kHistogramVisitDuration, category);
   base::UmaHistogramLongTimes(name, visit_time);
 }
 
+void OnCategoryMovedUp(int new_index) {
+  UMA_HISTOGRAM_EXACT_LINEAR(kHistogramMovedUpCategoryNewIndex, new_index,
+                             kMaxCategories);

[Marc Treib, 2017/02/01 13:04:23]

Hm. I guess you'd mostly look at the number of samples in the histogram over a
given period of time? Or do you see any use for the actual index values?

I'm also a bit worried that this will be hard to interpret: You'd have to
"normalize" over number of users of course, but also over total usage - if there
are more clicks overall, then there's probably also be more movement in the
categories. Or whenever we add a new category, there'll be more movement...

But I also can't immediately think of a better way to track this. Any ideas?

[vitaliii, 2017/02/01 13:30:18]

Partially true. We are worried that top most categories will wiggle around.
Thus, we could use data for the first 3 positions and ignore lower positions.
Number of users - probably, number of clicks - likely not. The order should
become more stable with time, so number of clicks should not matter. Ideally new
users will introduce some values here, but only before their order settles down.
If this highly depends on clicks, we set the thresholds too low. 
tschumann@ and I cannot find a better way as well, but this seemed good enough
to see how stable the order becomes with time.

[Marc Treib, 2017/02/01 13:40:46]

Alright, fair enough.

[Marc Treib, 2017/02/01 13:42:37]

(About tracking only the top 3 or something: No, if we do care about the index
(at least in some cases), then let's just track the index.)

[vitaliii, 2017/02/01 14:19:37]

Acknowledged.

The mention of top 3 was to explain why we need indices.

+}
+
 void OnMoreButtonShown(Category category, int position) {
   // The "more" card can appear in addition to the actual suggestions, so add
   // one extra bucket to this histogram.
   LogCategoryHistogramPosition(kHistogramMoreButtonShown, category, position,
                                kMaxSuggestionsPerCategory + 1);
 }
 
 void OnMoreButtonClicked(Category category, int position) {
   // The "more" card can appear in addition to the actual suggestions, so add
   // one extra bucket to this histogram.
   LogCategoryHistogramPosition(kHistogramMoreButtonClicked, category, position,
                                kMaxSuggestionsPerCategory + 1);
 }
 
 void OnCategoryDismissed(Category category) {
   UMA_HISTOGRAM_ENUMERATION(kHistogramCategoryDismissed,
                             GetHistogramCategory(category),
                             HistogramCategories::COUNT);
 }
 
 }  // namespace metrics
 }  // namespace ntp_snippets