RAPPOR implementation

Implementation of Randomized Aggregatable Privacy-Preserving Ordinal Responses (RAPPORs).

See the design doc at http://www.chromium.org/developers/design-documents/rappor

BUG=328168
TBR=darin@chromium.org

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=252492

[Steven Holte, 2013-10-28 21:02:44]

This is still a pretty early draft, but any pointers about where different bits
of code should live, suggestions for naming conventions, or similar code I
should look at for interface idioms would be appreciated.

https://codereview.chromium.org/49753002/diff/1/chrome/browser/metrics/metric...
File chrome/browser/metrics/metrics_log.cc (right):

https://codereview.chromium.org/49753002/diff/1/chrome/browser/metrics/metric...
chrome/browser/metrics/metrics_log.cc:808: void
MetricsLog::GetBloomFilterBits(std::string* bloom_filter_bits) {
This should probably be split into 4 parts:

A function for adding a collected value to the PPRR, a function for encoding the
PPRRs into the proto buf, a utility for breaking the URLs into bits for the
PPRR, and the actual collection of homepage.  Any suggestions about where each
of those should live?

https://codereview.chromium.org/49753002/diff/1/chrome/browser/metrics/metric...
File chrome/browser/metrics/metrics_service.cc (right):

https://codereview.chromium.org/49753002/diff/1/chrome/browser/metrics/metric...
chrome/browser/metrics/metrics_service.cc:401: // TODO make sure this doesn't
get uploaded anywhere.
Can anyone confirm that the prefs in this registry aren't picked up by other
services like Sync?

https://codereview.chromium.org/49753002/diff/1/chrome/common/metrics/proto/s...
File chrome/common/metrics/proto/system_profile.proto (right):

https://codereview.chromium.org/49753002/diff/1/chrome/common/metrics/proto/s...
chrome/common/metrics/proto/system_profile.proto:445: 
This probably should move somewhere else and be a repeated message containing a
name_hash.  Does moving it directly to ChromeUserMetricsExtension seem like the
right place?

[Ilya Sherman, 2013-10-28 23:38:27]

I'm not familiar with the context for this change.  Is there a design doc that I
can look at?

I'm hoping that by the time you're seeking a non-preliminary review for this CL,
you'll have a bug filed and detailed comments for the parts of the code that
merit them, but it's understandable that you don't have those yet while this is
still a draft CL.

[Steven Holte, 2013-10-28 23:50:00]

The only doc right now is this one:
https://docs.google.com/a/google.com/document/d/1Us5U2lA8g_2s5Bf5ag7LGa3SZ97s...

I'm working on one with more details about the planned use in UMA.

[Ilya Sherman, 2013-10-29 00:32:31]

On 2013/10/28 23:50:00, holte wrote:
> The only doc right now is this one:
>
https://docs.google.com/a/google.com/document/d/1Us5U2lA8g_2s5Bf5ag7LGa3SZ97s...
> 
> I'm working on one with more details about the planned use in UMA.

Thanks, that helped clarify things a fair bit.  I look forward to seeing the
UMA-specific design doc, and hope that the majority of that design doc can be
shared publicly and hosted on the chromium.org domain :)

https://codereview.chromium.org/49753002/diff/1/chrome/browser/metrics/metric...
File chrome/browser/metrics/metrics_log.cc (right):

https://codereview.chromium.org/49753002/diff/1/chrome/browser/metrics/metric...
chrome/browser/metrics/metrics_log.cc:808: void
MetricsLog::GetBloomFilterBits(std::string* bloom_filter_bits) {
On 2013/10/28 21:02:44, holte wrote:
> This should probably be split into 4 parts:
> 
> A function for adding a collected value to the PPRR, a function for encoding
the
> PPRRs into the proto buf, a utility for breaking the URLs into bits for the
> PPRR, and the actual collection of homepage.  Any suggestions about where each
> of those should live?

If you're intending to provide something of a parallel architecture to the
histograms architecture, then you'll probably want the code for "adding a
collected value to the PPRR" to live in //base/metrics.  Encoding the collected
values into the protocol buffer makes sense to keep in metrics_log.cc, most
likely in the anonymous namespace.  Breaking URLs into bits should probably be
an internal implementation detail for logging values to the code that lives in
//base/metrics.  The collection of the homepage should be done by whatever
feature code interacts with setting the homepage, though depending on when this
is meant to be recorded, there might need to be a call into that code from the
MetricsLog or from the MetricsService.

https://codereview.chromium.org/49753002/diff/1/chrome/browser/metrics/metric...
File chrome/browser/metrics/metrics_service.cc (right):

https://codereview.chromium.org/49753002/diff/1/chrome/browser/metrics/metric...
chrome/browser/metrics/metrics_service.cc:401: // TODO make sure this doesn't
get uploaded anywhere.
On 2013/10/28 21:02:44, holte wrote:
> Can anyone confirm that the prefs in this registry aren't picked up by other
> services like Sync?

I'm fairly confident that the difference between PrefRegistrySimple and
PrefRegistrySyncable is that PrefRegistrySimple is stored purely on the client.

https://codereview.chromium.org/49753002/diff/1/chrome/common/metrics/proto/s...
File chrome/common/metrics/proto/system_profile.proto (right):

https://codereview.chromium.org/49753002/diff/1/chrome/common/metrics/proto/s...
chrome/common/metrics/proto/system_profile.proto:445: 
On 2013/10/28 21:02:44, holte wrote:
> This probably should move somewhere else and be a repeated message containing
a
> name_hash.  Does moving it directly to ChromeUserMetricsExtension seem like
the
> right place?

If this is meant to be a general utility for particularly privacy-sensitive
histograms in a privacy-respectful way, then yes, moving it directly to
ChromeUserMetricsExtension seems like the right place.  OTOH, if this is
intended for reporting some specific system state outside of a histogram
context, then it is probably more appropriate to keep it in this file, but to
assign the field a more descriptive name.

[Alexei Svitkine (slow), 2013-12-11 15:27:04]

I'll review more later when I have a chance. Here's a few initial nits.

I'm also adding Jesse as a reviewer who can use some more experience reviewing
Chromium code. ;)

https://codereview.chromium.org/49753002/diff/220001/chrome/browser/chrome_br...
File chrome/browser/chrome_browser_main.cc (right):

https://codereview.chromium.org/49753002/diff/220001/chrome/browser/chrome_br...
chrome/browser/chrome_browser_main.cc:1130:
browser_process_->system_request_context());
Nit: Indent 2 more per C++ style.

https://codereview.chromium.org/49753002/diff/220001/components/rappor/bloom_...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/220001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:16: static const uint32_t hashSeedCount = 20;
Nit: No need for static keyword for things in the anon namespace. For these
constants and the function below.

https://codereview.chromium.org/49753002/diff/220001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:26: MurmurHash3_x86_32(str.data(), str.size(),
seed, &output);
Chrome isn't always x86_32. Is there a way to call a non-arch specific version
of this?

https://codereview.chromium.org/49753002/diff/220001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:49: for (size_t i = 0, len = strings.size(); i
< len; ++i) {
Nit: No need for {}s when body is a single line.

https://codereview.chromium.org/49753002/diff/220001/components/rappor/rappor...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/220001/components/rappor/rappor...
components/rappor/rappor_service.cc:20: using base::TimeDelta;
Nit: Remove this and just use the base:: prefix in the two places below.

https://codereview.chromium.org/49753002/diff/220001/components/rappor/rappor...
File components/rappor/rappor_service.h (right):

https://codereview.chromium.org/49753002/diff/220001/components/rappor/rappor...
components/rappor/rappor_service.h:64: friend class RapporServiceTest;
Nit: This should be just below the private: section.

https://codereview.chromium.org/49753002/diff/220001/components/rappor/rappor...
components/rappor/rappor_service.h:67: extern
base::LazyInstance<RapporService>::Leaky g_rappor_service;
I don't think we should have this as a global instance. Instead, hang it off the
browser process, like other services.

[Steven Holte, 2013-12-11 20:31:25]

https://codereview.chromium.org/49753002/diff/220001/chrome/browser/chrome_br...
File chrome/browser/chrome_browser_main.cc (right):

https://codereview.chromium.org/49753002/diff/220001/chrome/browser/chrome_br...
chrome/browser/chrome_browser_main.cc:1130:
browser_process_->system_request_context());
On 2013/12/11 15:27:05, Alexei Svitkine wrote:
> Nit: Indent 2 more per C++ style.

Done.

https://codereview.chromium.org/49753002/diff/220001/components/rappor/bloom_...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/220001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:16: static const uint32_t hashSeedCount = 20;
On 2013/12/11 15:27:05, Alexei Svitkine wrote:
> Nit: No need for static keyword for things in the anon namespace. For these
> constants and the function below.

Done.

https://codereview.chromium.org/49753002/diff/220001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:26: MurmurHash3_x86_32(str.data(), str.size(),
seed, &output);
On 2013/12/11 15:27:05, Alexei Svitkine wrote:
> Chrome isn't always x86_32. Is there a way to call a non-arch specific version
> of this?

I believe it is just optimized for x86_32.
According to the comment in third_party/smhasher/src/MurmurHash3.cpp:
// You can still
// compile and run any of them on any platform, but your performance with the
// non-native version will be less than optimal.

safe_browsing uses the same hash function:
chrome/renderer/safe_browsing/murmurhash3_util.cc

https://codereview.chromium.org/49753002/diff/220001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:49: for (size_t i = 0, len = strings.size(); i
< len; ++i) {
On 2013/12/11 15:27:05, Alexei Svitkine wrote:
> Nit: No need for {}s when body is a single line.

Done.

https://codereview.chromium.org/49753002/diff/220001/components/rappor/rappor...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/220001/components/rappor/rappor...
components/rappor/rappor_service.cc:20: using base::TimeDelta;
On 2013/12/11 15:27:05, Alexei Svitkine wrote:
> Nit: Remove this and just use the base:: prefix in the two places below.

Done.

https://codereview.chromium.org/49753002/diff/220001/components/rappor/rappor...
File components/rappor/rappor_service.h (right):

https://codereview.chromium.org/49753002/diff/220001/components/rappor/rappor...
components/rappor/rappor_service.h:64: friend class RapporServiceTest;
On 2013/12/11 15:27:05, Alexei Svitkine wrote:
> Nit: This should be just below the private: section.

Done.

https://codereview.chromium.org/49753002/diff/220001/components/rappor/rappor...
components/rappor/rappor_service.h:67: extern
base::LazyInstance<RapporService>::Leaky g_rappor_service;
On 2013/12/11 15:27:05, Alexei Svitkine wrote:
> I don't think we should have this as a global instance. Instead, hang it off
the
> browser process, like other services.

Done.

[Ilya Sherman, 2013-12-12 08:49:21]

Friendly reminder: If this is no longer just a draft CL, please update the CL
description and file a descriptive bug (at a minimum), and also write up a
design doc if possible (strongly encouraged, esp. since you can then link to
that design doc from the code).

[jwd, 2013-12-16 17:04:02]

Some initial comments.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:16: const uint32_t hashSeedCount = 20;
Use constant naming style kConstantName.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:17: const uint32_t
mockHashSeeds[hashSeedCount] = {
Dito.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:17: const uint32_t
mockHashSeeds[hashSeedCount] = {
This could use a better name I think. The current name makes it seem like it's
test related. 

You should probably also add some comments saying what they're for.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:23: inline uint32_t MurmurHash3String(const
std::string& str, uint32_t seed) {
Usually, only simple accessors are asked to be inlined in chromium.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
File components/rappor/bloom_filter.h (right):

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
components/rappor/bloom_filter.h:22: void Add(const std::vector<std::string>&
strings);
Overloaded functions are usually frowned upon in chromium code.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
components/rappor/bloom_filter.h:23: const ByteVector& bytes() const;
Can you give a comment for this function and for hash_count().

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
components/rappor/bloom_filter.h:28: uint32_t hash_count_;
Maybe a short comment here too.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/byteve...
File components/rappor/bytevector.cc (right):

https://codereview.chromium.org/49753002/diff/240001/components/rappor/byteve...
components/rappor/bytevector.cc:19: DCHECK(size() == rhs.size());
I would document the size equality restriction in the .h file.
Also, use DCHECK_EQ instead.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/byteve...
components/rappor/bytevector.cc:28: DCHECK(size() == rhs.size());
Dito

https://codereview.chromium.org/49753002/diff/240001/components/rappor/byteve...
File components/rappor/bytevector.h (right):

https://codereview.chromium.org/49753002/diff/240001/components/rappor/byteve...
components/rappor/bytevector.h:23: ByteVector operator&(ByteVector rhs) const;
Operator overloading is frowned upon in the c++ google style guide. I don't
think this case justifies an exception.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/log_up...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/240001/components/rappor/log_up...
components/rappor/log_uploader.h:20: // This class handles uploading logs to
Looks like you're missing an end of

[Steven Holte, 2013-12-16 23:02:16]

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:16: const uint32_t hashSeedCount = 20;
On 2013/12/16 17:04:03, Jesse Doherty wrote:
> Use constant naming style kConstantName.

Done.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:17: const uint32_t
mockHashSeeds[hashSeedCount] = {
On 2013/12/16 17:04:03, Jesse Doherty wrote:
> Dito.

Done.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:23: inline uint32_t MurmurHash3String(const
std::string& str, uint32_t seed) {
On 2013/12/16 17:04:03, Jesse Doherty wrote:
> Usually, only simple accessors are asked to be inlined in chromium.

Done.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
File components/rappor/bloom_filter.h (right):

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
components/rappor/bloom_filter.h:22: void Add(const std::vector<std::string>&
strings);
On 2013/12/16 17:04:03, Jesse Doherty wrote:
> Overloaded functions are usually frowned upon in chromium code.

Done.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
components/rappor/bloom_filter.h:23: const ByteVector& bytes() const;
On 2013/12/16 17:04:03, Jesse Doherty wrote:
> Can you give a comment for this function and for hash_count().

Done.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/bloom_...
components/rappor/bloom_filter.h:28: uint32_t hash_count_;
On 2013/12/16 17:04:03, Jesse Doherty wrote:
> Maybe a short comment here too.

Done.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/byteve...
File components/rappor/bytevector.cc (right):

https://codereview.chromium.org/49753002/diff/240001/components/rappor/byteve...
components/rappor/bytevector.cc:19: DCHECK(size() == rhs.size());
On 2013/12/16 17:04:03, Jesse Doherty wrote:
> I would document the size equality restriction in the .h file.
> Also, use DCHECK_EQ instead.

Done.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/byteve...
components/rappor/bytevector.cc:28: DCHECK(size() == rhs.size());
On 2013/12/16 17:04:03, Jesse Doherty wrote:
> Dito

Done.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/byteve...
File components/rappor/bytevector.h (right):

https://codereview.chromium.org/49753002/diff/240001/components/rappor/byteve...
components/rappor/bytevector.h:23: ByteVector operator&(ByteVector rhs) const;
On 2013/12/16 17:04:03, Jesse Doherty wrote:
> Operator overloading is frowned upon in the c++ google style guide. I don't
> think this case justifies an exception.

Done.

https://codereview.chromium.org/49753002/diff/240001/components/rappor/log_up...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/240001/components/rappor/log_up...
components/rappor/log_uploader.h:20: // This class handles uploading logs to
On 2013/12/16 17:04:03, Jesse Doherty wrote:
> Looks like you're missing an end of 

Done.

[jwd, 2013-12-17 23:00:25]

https://codereview.chromium.org/49753002/diff/250001/components/rappor/proto/...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/250001/components/rappor/proto/...
components/rappor/proto/rappor_metric.proto:14: // Next tag: 3
Update next tag, maybe drop it.

https://codereview.chromium.org/49753002/diff/250001/components/rappor/proto/...
components/rappor/proto/rappor_metric.proto:16: message RapporMetricProto {
indent inner message.

Do these have to be named *Proto? Is there a naming convention for this? At
least the inner one could drop it.

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor.h
File components/rappor/rappor.h (right):

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
components/rappor/rappor.h:56: const int zero_honesty_prob_index_;
This needs to be documented more.

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
File components/rappor/rappor_recorder.cc (right):

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
components/rappor/rappor_recorder.cc:45: Rappor* new_rappor = new Rappor(name,
16, 4, 4, 2, 4, 2);
This looks like voodoo. Can these constants be pulled out?

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
File components/rappor/rappor_recorder.h (right):

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
components/rappor/rappor_recorder.h:38: Rappor* GetRappor(const std::string&
name);
Can this return a weak_ptr? Similarly, can |Rappors| be a vector of weak_ptrs?

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
File components/rappor/rappor_reporter.cc (right):

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
components/rappor/rappor_reporter.cc:52: ByteVector
ByteVectorGenerator::GetRandomByteVector() {
Mention that it's expected 50% 1s or something like that.

Actually, isn't this dependent on the implementation of RandByte? Making the
comments in GetWeightedRandomByteVector not guaranteed?

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
components/rappor/rappor_reporter.cc:63: case -8:
What determines these constants? As in, why -8,...,8

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
File components/rappor/rappor_reporter_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
components/rappor/rappor_reporter_unittest.cc:17: ASSERT_EQ(16u, report.size());
Can the reporter be turned deterministic so that it can be more deeply tested?

[Steven Holte, 2013-12-18 02:06:11]

https://codereview.chromium.org/49753002/diff/250001/components/rappor/proto/...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/250001/components/rappor/proto/...
components/rappor/proto/rappor_metric.proto:16: message RapporMetricProto {
On 2013/12/17 23:00:25, Jesse Doherty wrote:
> indent inner message.
> 
> Do these have to be named *Proto? Is there a naming convention for this? At
> least the inner one could drop it.

I had included it in the name because UserActionEventProto did.  I don't know if
there are better conventions to follow.  I've dropped it from the inner message,
and I'm open to suggestions for message names.

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor.h
File components/rappor/rappor.h (right):

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
components/rappor/rappor.h:56: const int zero_honesty_prob_index_;
On 2013/12/17 23:00:25, Jesse Doherty wrote:
> This needs to be documented more.

Extracted to RapporParameters, and added comments.

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
File components/rappor/rappor_recorder.cc (right):

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
components/rappor/rappor_recorder.cc:45: Rappor* new_rappor = new Rappor(name,
16, 4, 4, 2, 4, 2);
On 2013/12/17 23:00:25, Jesse Doherty wrote:
> This looks like voodoo. Can these constants be pulled out?

Extracted the parameters into a RapporParameters object, which can be statically
initialized for each metric, and passed in here.

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
File components/rappor/rappor_reporter.cc (right):

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
components/rappor/rappor_reporter.cc:52: ByteVector
ByteVectorGenerator::GetRandomByteVector() {
On 2013/12/17 23:00:25, Jesse Doherty wrote:
> Mention that it's expected 50% 1s or something like that.
> 
> Actually, isn't this dependent on the implementation of RandByte? Making the
> comments in GetWeightedRandomByteVector not guaranteed?

Added comments about RandByte and RandBytes being expected to have uniform
distributions of bytes.  It should be approximately true for the pseudorandom
generator.

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
components/rappor/rappor_reporter.cc:63: case -8:
On 2013/12/17 23:00:25, Jesse Doherty wrote:
> What determines these constants? As in, why -8,...,8

Changed these to Probability enum values.

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
File components/rappor/rappor_reporter_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
components/rappor/rappor_reporter_unittest.cc:17: ASSERT_EQ(16u, report.size());
On 2013/12/17 23:00:25, Jesse Doherty wrote:
> Can the reporter be turned deterministic so that it can be more deeply tested?

I'm not sure that making it deterministic adds any meaningful additional
coverage.  We'd just be testing that our deterministic generator was
deterministic.

[Steven Holte, 2013-12-18 23:38:47]

https://codereview.chromium.org/49753002/diff/250001/components/rappor/proto/...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/250001/components/rappor/proto/...
components/rappor/proto/rappor_metric.proto:14: // Next tag: 3
On 2013/12/17 23:00:25, Jesse Doherty wrote:
> Update next tag, maybe drop it.

Done.

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
File components/rappor/rappor_recorder.h (right):

https://codereview.chromium.org/49753002/diff/250001/components/rappor/rappor...
components/rappor/rappor_recorder.h:38: Rappor* GetRappor(const std::string&
name);
On 2013/12/17 23:00:25, Jesse Doherty wrote:
> Can this return a weak_ptr? Similarly, can |Rappors| be a vector of weak_ptrs?

Hid this pointer inside a RecordSamples() method.

For |Rappors|, I changed GetRappors into CollectRappors, which clears the
Recorder, so the pointers returned are unique references to the objects and can
be deleted by the caller.  I'm not sure if using scoped_ptr for the vector
objects might be more appropriate?

[Alexei Svitkine (slow), 2013-12-19 19:47:01]

https://codereview.chromium.org/49753002/diff/290001/chrome/browser/browser_p...
File chrome/browser/browser_process_impl.cc (right):

https://codereview.chromium.org/49753002/diff/290001/chrome/browser/browser_p...
chrome/browser/browser_process_impl.cc:441: if (!rappor_service_.get()) {
Nit: No {}s

https://codereview.chromium.org/49753002/diff/290001/components/rappor.gypi
File components/rappor.gypi (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor.gypi#n...
components/rappor.gypi:43: # Protobuf compiler / generator for RAPPOR.
Nit: You can merge this with the main target to simplify things. See
variations.gypi.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/bloom_...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:31: DCHECK(hash_count <= kHashSeedCount);
DCHECK_LE

https://codereview.chromium.org/49753002/diff/290001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:50: uint32_t BloomFilter::hash_count() const {
return hash_count_; }
These can be inlined in the header.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/bloom_...
File components/rappor/bloom_filter.h (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/bloom_...
components/rappor/bloom_filter.h:20: BloomFilter(uint32_t bytes_size, uint32_t
hash_count);
Nit: Add explicit ~BloomFilter() declaration here and empty implementation in
the .cc file.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/bloom_...
components/rappor/bloom_filter.h:34: };
Nit: DISALLOW_COPY_AND_ASSIGN(BloomFilter);

https://codereview.chromium.org/49753002/diff/290001/components/rappor/byteve...
File components/rappor/bytevector.h (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/byteve...
components/rappor/bytevector.h:13: // methods for manipulating it's content as a
set of binary bits.
Nit: Move this comment above the class decl.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/byteve...
components/rappor/bytevector.h:21: ByteVector(const char* bytes, int
bytes_size);
Document these ctor's.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/byteve...
components/rappor/bytevector.h:25: ByteVector And(ByteVector rhs) const;
I'm not convinced we need a ByteVector class. Can these just be free-standing
functions that take std::vector<uint8_t> params? Also, do they really need to
return the byte vectors, or can they just modify things in place?

https://codereview.chromium.org/49753002/diff/290001/components/rappor/proto/...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/proto/...
components/rappor/proto/rappor_metric.proto:15: message RapporMetric {
Nit: Indent inner message.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor.h
File components/rappor/rappor.h (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor.h:24: // A class that maintains parameters and bloom
filter data for a Rappor Metric.
Maybe it should be called RapporMetric? Rappor by itself doesn't convey a lot of
meaning.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor.h:43: };
Nit: DISALLOW_COPY_AND_ASSIGN

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
File components/rappor/rappor_pref_names.h (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_pref_names.h:13: extern const char kRapporSecret[];
I'm not sure there's precedent for having prefs inside components.

If there isn't, these can still be in chrome/ and passed as parameters to the
ctor of the RapporService.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
File components/rappor/rappor_recorder.h (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_recorder.h:22:
rappor::g_rappor_recorder.Get().RecordSamples(parameters, samples);
I don't think we should have a global recorder or this macro. I think the API
should just be on the RapporService.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_recorder.h:28: typedef std::vector<Rappor*> Rappors;
This should use a ScopedVector<Rappor>.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_recorder.h:46: RapporRecorder();
These should be before any other methods.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
File components/rappor/rappor_reporter.cc (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_reporter.cc:22: #define HMAC_SEED 10
Use a const int in the anon namespace instead and follow kConstant naming
convention. Add a comment.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_reporter.cc:110: uint8_t randomBits;
Nit: random_bits

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_reporter.cc:124: const ByteVector
realbits(rappor.GetBytes());
Nit: real_bits, use this naming convention throughout.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_reporter.cc:126: HmacByteVectorGenerator hmacGenerator(
Nit: hmac_generator

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_reporter.cc:133:
(realbits.And(fakebits.Not())).Or(fakeones.And(fakebits));
Nit: No need for outer ()'s.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_service.cc:54:
rappor_metrics_proto_.SerializeToString(&log_text);
Check return value.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_service.cc:74: if (decoded) {
Nit: No {}s needed for single body if.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
File components/rappor/rappor_service.h (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_service.h:41: std::string rappor_secret_;
Nit: Variables should be below functions.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
File components/rappor/rappor_service_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_service_unittest.cc:14: const RapporParameters
kTestRapporParameters = {
Nit: Inline this into the test function, unless this is going to be used by
multiple tests..

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
File components/rappor/rappor_utils.h (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_utils.h:20: }
Nit: }  // namespace rappor

And add a newline before it.

[Steven Holte, 2013-12-20 03:03:55]

https://codereview.chromium.org/49753002/diff/290001/chrome/browser/browser_p...
File chrome/browser/browser_process_impl.cc (right):

https://codereview.chromium.org/49753002/diff/290001/chrome/browser/browser_p...
chrome/browser/browser_process_impl.cc:441: if (!rappor_service_.get()) {
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Nit: No {}s

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor.gypi
File components/rappor.gypi (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor.gypi#n...
components/rappor.gypi:43: # Protobuf compiler / generator for RAPPOR.
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Nit: You can merge this with the main target to simplify things. See
> variations.gypi.

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/bloom_...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:31: DCHECK(hash_count <= kHashSeedCount);
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> DCHECK_LE

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:50: uint32_t BloomFilter::hash_count() const {
return hash_count_; }
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> These can be inlined in the header.

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/bloom_...
File components/rappor/bloom_filter.h (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/bloom_...
components/rappor/bloom_filter.h:20: BloomFilter(uint32_t bytes_size, uint32_t
hash_count);
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Nit: Add explicit ~BloomFilter() declaration here and empty implementation in
> the .cc file.

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/bloom_...
components/rappor/bloom_filter.h:34: };
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Nit: DISALLOW_COPY_AND_ASSIGN(BloomFilter);

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/byteve...
File components/rappor/bytevector.h (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/byteve...
components/rappor/bytevector.h:13: // methods for manipulating it's content as a
set of binary bits.
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Nit: Move this comment above the class decl.

Changed class to typedef and moved+updated comment.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/byteve...
components/rappor/bytevector.h:21: ByteVector(const char* bytes, int
bytes_size);
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Document these ctor's.

N/A

https://codereview.chromium.org/49753002/diff/290001/components/rappor/byteve...
components/rappor/bytevector.h:25: ByteVector And(ByteVector rhs) const;
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> I'm not convinced we need a ByteVector class. Can these just be free-standing
> functions that take std::vector<uint8_t> params? Also, do they really need to
> return the byte vectors, or can they just modify things in place?

Changed ByteVector to a typedef and these to free-standing functions that modify
the vectors in place.  The interface still seems a bit rough, so I'll probably
take another pass at this.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/proto/...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/proto/...
components/rappor/proto/rappor_metric.proto:15: message RapporMetric {
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Nit: Indent inner message.

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor.h
File components/rappor/rappor.h (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor.h:24: // A class that maintains parameters and bloom
filter data for a Rappor Metric.
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Maybe it should be called RapporMetric? Rappor by itself doesn't convey a lot
of
> meaning.

I'll pick this up in the next pass.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor.h:43: };
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Nit: DISALLOW_COPY_AND_ASSIGN

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
File components/rappor/rappor_pref_names.h (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_pref_names.h:13: extern const char kRapporSecret[];
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> I'm not sure there's precedent for having prefs inside components.
> 
> If there isn't, these can still be in chrome/ and passed as parameters to the
> ctor of the RapporService.

See ./autofill/core/common/autofill_pref_names.h and
./policy/core/common/policy_pref_names.h.  It seems like a pretty reasonable
convention.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
File components/rappor/rappor_recorder.h (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_recorder.h:22:
rappor::g_rappor_recorder.Get().RecordSamples(parameters, samples);
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> I don't think we should have a global recorder or this macro. I think the API
> should just be on the RapporService.

Collapsed this class into RapporService

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_recorder.h:28: typedef std::vector<Rappor*> Rappors;
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> This should use a ScopedVector<Rappor>.

N/A, since RapporService just directly iterates over the Map.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_recorder.h:46: RapporRecorder();
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> These should be before any other methods.

N/A

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
File components/rappor/rappor_reporter.cc (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_reporter.cc:22: #define HMAC_SEED 10
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Use a const int in the anon namespace instead and follow kConstant naming
> convention. Add a comment.

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_reporter.cc:110: uint8_t randomBits;
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Nit: random_bits

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_reporter.cc:124: const ByteVector
realbits(rappor.GetBytes());
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Nit: real_bits, use this naming convention throughout.

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_reporter.cc:126: HmacByteVectorGenerator hmacGenerator(
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Nit: hmac_generator

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_reporter.cc:133:
(realbits.And(fakebits.Not())).Or(fakeones.And(fakebits));
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Nit: No need for outer ()'s.

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_service.cc:54:
rappor_metrics_proto_.SerializeToString(&log_text);
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Check return value.

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_service.cc:74: if (decoded) {
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Nit: No {}s needed for single body if.

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
File components/rappor/rappor_service.h (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_service.h:41: std::string rappor_secret_;
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Nit: Variables should be below functions.

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
File components/rappor/rappor_service_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_service_unittest.cc:14: const RapporParameters
kTestRapporParameters = {
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Nit: Inline this into the test function, unless this is going to be used by
> multiple tests..

Done.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
File components/rappor/rappor_utils.h (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor_utils.h:20: }
On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> Nit: }  // namespace rappor
> 
> And add a newline before it.

Done.

[Steven Holte, 2013-12-20 23:36:03]

Renamed Rappor to RapporMetric, and bytevector.cc to byte_vector_utils.cc. 
Split code from rappor_reporter.cc and merged it into the other two files.

Replace VectorAndNot with ByteVectorMerge.

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor.h
File components/rappor/rappor.h (right):

https://codereview.chromium.org/49753002/diff/290001/components/rappor/rappor...
components/rappor/rappor.h:24: // A class that maintains parameters and bloom
filter data for a Rappor Metric.
On 2013/12/20 03:03:55, holte wrote:
> On 2013/12/19 19:47:02, Alexei Svitkine wrote:
> > Maybe it should be called RapporMetric? Rappor by itself doesn't convey a
lot
> of
> > meaning.
> 
> I'll pick this up in the next pass.

Done.

[Alexei Svitkine (slow), 2013-12-24 16:59:29]

https://codereview.chromium.org/49753002/diff/330001/components/rappor/bloom_...
File components/rappor/bloom_filter.h (right):

https://codereview.chromium.org/49753002/diff/330001/components/rappor/bloom_...
components/rappor/bloom_filter.h:29: return bytes_;
Nit: This can go on the same line as above. Same for hash_count below.

https://codereview.chromium.org/49753002/diff/330001/components/rappor/log_up...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/330001/components/rappor/log_up...
components/rappor/log_uploader.cc:43: upload_interval_ =
TimeDelta::FromSeconds(kUnsentLogsIntervalSeconds);
Nit: Put this in the init list.

https://codereview.chromium.org/49753002/diff/330001/components/rappor/log_up...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/330001/components/rappor/log_up...
components/rappor/log_uploader.h:31: const char* server_url_;
Nit: Vars should come after methods.

https://codereview.chromium.org/49753002/diff/330001/components/rappor/log_up...
components/rappor/log_uploader.h:70: };
DISALLOW_COPY_AND_ASSIGN

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
File components/rappor/rappor_metric.h (right):

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
components/rappor/rappor_metric.h:28: RapporMetric(const RapporParameters*
parameters);
Nit: explicit.

What's the ownership model for |parameters|. Please document.

It seems your current code is keeping a weak ref to this. But the owner appears
to be whoever called the RapporService method. That doesn't seem like a good
ownership model. Maybe you could transfer ownership, by passing these as
scoped_ptr<RapporParameters> all the way down?

However, then the issue is logging new samples to an existing metric... Perhaps
you could create a helper RapporMetricRecorder class that handles this behind
the scenes (and observes RapporService destruction, so that when RapporService
goes away, the recorder acts sane).

Alternatively, just copy the params in this ctor and don't pass a weak pointer.

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
components/rappor/rappor_service.cc:97: delete rappor;
Instead of deleting things manually, use STLDeleteValues() from base/stl_util.h

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
File components/rappor/rappor_service.h (right):

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
components/rappor/rappor_service.h:36: // At startup, prefs needs to be called
with a list of all the pref names and
Nit: reword. Mention that this should be called before Start().

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
components/rappor/rappor_service.h:40: // Find a rappor by name, and create it
if it doesn't already exist, and adds
Nit: How about: "Records a set of samples on the rappor metric specified by
|parameters|. Creates and initializes the metric, if it doesn't yet exist."

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
components/rappor/rappor_service.h:42: void RecordSamples(const
RapporParameters* parameters,
Nit: pass by const ref (unless passing NULL is a supported use case, by I don't
think it is here).

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
components/rappor/rappor_service.h:61: // Find a rappor by name, and create it
if it doesn't already exist.
Nit: "rappor" -> "rappor metric".

[Steven Holte, 2014-01-04 00:12:54]

https://codereview.chromium.org/49753002/diff/330001/components/rappor/bloom_...
File components/rappor/bloom_filter.h (right):

https://codereview.chromium.org/49753002/diff/330001/components/rappor/bloom_...
components/rappor/bloom_filter.h:29: return bytes_;
On 2013/12/24 16:59:30, asvitkine (OOO until Jan 6th) wrote:
> Nit: This can go on the same line as above. Same for hash_count below.

Done.

https://codereview.chromium.org/49753002/diff/330001/components/rappor/log_up...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/330001/components/rappor/log_up...
components/rappor/log_uploader.cc:43: upload_interval_ =
TimeDelta::FromSeconds(kUnsentLogsIntervalSeconds);
On 2013/12/24 16:59:30, asvitkine (OOO until Jan 6th) wrote:
> Nit: Put this in the init list.

Done.

https://codereview.chromium.org/49753002/diff/330001/components/rappor/log_up...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/330001/components/rappor/log_up...
components/rappor/log_uploader.h:31: const char* server_url_;
On 2013/12/24 16:59:30, asvitkine (OOO until Jan 6th) wrote:
> Nit: Vars should come after methods.

Done.

https://codereview.chromium.org/49753002/diff/330001/components/rappor/log_up...
components/rappor/log_uploader.h:70: };
On 2013/12/24 16:59:30, asvitkine (OOO until Jan 6th) wrote:
> DISALLOW_COPY_AND_ASSIGN

Done.

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
File components/rappor/rappor_metric.h (right):

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
components/rappor/rappor_metric.h:28: RapporMetric(const RapporParameters*
parameters);
Changed to copy in ctor.

On 2013/12/24 16:59:30, asvitkine (OOO until Jan 6th) wrote:
> Nit: explicit.
> 
> What's the ownership model for |parameters|. Please document.
> 
> It seems your current code is keeping a weak ref to this. But the owner
appears
> to be whoever called the RapporService method. That doesn't seem like a good
> ownership model. Maybe you could transfer ownership, by passing these as
> scoped_ptr<RapporParameters> all the way down?
> 
> However, then the issue is logging new samples to an existing metric...
Perhaps
> you could create a helper RapporMetricRecorder class that handles this behind
> the scenes (and observes RapporService destruction, so that when RapporService
> goes away, the recorder acts sane).
> 
> Alternatively, just copy the params in this ctor and don't pass a weak
pointer.

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
components/rappor/rappor_service.cc:97: delete rappor;
On 2013/12/24 16:59:30, asvitkine (OOO until Jan 6th) wrote:
> Instead of deleting things manually, use STLDeleteValues() from
base/stl_util.h

Done.

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
File components/rappor/rappor_service.h (right):

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
components/rappor/rappor_service.h:36: // At startup, prefs needs to be called
with a list of all the pref names and
On 2013/12/24 16:59:30, asvitkine (OOO until Jan 6th) wrote:
> Nit: reword. Mention that this should be called before Start().

Done.

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
components/rappor/rappor_service.h:40: // Find a rappor by name, and create it
if it doesn't already exist, and adds
On 2013/12/24 16:59:30, asvitkine (OOO until Jan 6th) wrote:
> Nit: How about: "Records a set of samples on the rappor metric specified by
> |parameters|. Creates and initializes the metric, if it doesn't yet exist."

Done.

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
components/rappor/rappor_service.h:42: void RecordSamples(const
RapporParameters* parameters,
On 2013/12/24 16:59:30, asvitkine (OOO until Jan 6th) wrote:
> Nit: pass by const ref (unless passing NULL is a supported use case, by I
don't
> think it is here).

Done.

https://codereview.chromium.org/49753002/diff/330001/components/rappor/rappor...
components/rappor/rappor_service.h:61: // Find a rappor by name, and create it
if it doesn't already exist.
On 2013/12/24 16:59:30, asvitkine (OOO until Jan 6th) wrote:
> Nit: "rappor" -> "rappor metric".

Done.

[Alexei Svitkine (slow), 2014-01-09 19:23:08]

I did another round of review on this.

Ilya, would you mind also giving this a look before you disappear for vacation?

https://codereview.chromium.org/49753002/diff/610001/chrome/browser/prefs/pre...
File chrome/browser/prefs/pref_metrics_service.cc (right):

https://codereview.chromium.org/49753002/diff/610001/chrome/browser/prefs/pre...
chrome/browser/prefs/pref_metrics_service.cc:166:
rappor::GetUrlParts(homepage_url, &url_parts);
I think the service should just have a RecordURL() method that does this
internally, so the caller doesn't need this extra logic. (Then rappor_utils can
go away.)

https://codereview.chromium.org/49753002/diff/610001/components/rappor/bloom_...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/610001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:14: const uint32_t kHashSeedCount = 4;
You can probably get rid of kHashSeedCount and just use [] on the line below.
Then, you can use arraysize(kHashSeeds) in place of it.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:46: for (size_t i = 0, len = strings.size(); i
< len; ++i)
Nit: No need for |len|, just use |strings.size()| directly. (Compiler will
optimize it.)

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:89: abort();
Don't use abort() in Chrome code. You probably want NOTREACHED();

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:22: ByteVector* ByteVectorAnd(ByteVector*
lhs, const ByteVector& rhs);
Per style guide, non-const parameters should be last.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:24: ByteVector*
ByteVectorMerge(ByteVector* lhs,
This needs its own more specific comment to explain the functionality.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:32: explicit ByteVectorGenerator(size_t
byte_count);
Add an explicit destructor and define it in the C++ file. Same for the class
below.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:33: // Generates a random byte vector
where each bit is independently set with
Nit: Add an empty line above this.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:43: ByteVector GetRandomByteVector();
Add a line after this.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:45: };
DISALLOW_COPY_AND_ASSIGN()? Also below.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:51: HmacByteVectorGenerator(size_t
byte_count, const std::string& secret);
Document params.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:54: virtual uint8_t RandByte();
OVERRIDE

https://codereview.chromium.org/49753002/diff/610001/components/rappor/log_up...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/610001/components/rappor/log_up...
components/rappor/log_uploader.cc:52: if (!upload_timer_.IsRunning() &&
!callback_pending_) {
Nit: No need for {}s

https://codereview.chromium.org/49753002/diff/610001/components/rappor/log_up...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/610001/components/rappor/log_up...
components/rappor/log_uploader.h:23: LogUploader(const char* server_url, const
char* mime_type);
Document params. Add explicit destructor, define in .cc file.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/log_up...
components/rappor/log_uploader.h:25: void
SetRequestContext(net::URLRequestContextGetter* request_context);
Any reason this shouldn't be a ctor param?

https://codereview.chromium.org/49753002/diff/610001/components/rappor/log_up...
components/rappor/log_uploader.h:27: void QueueLog(const std::string& logData);
log_data. Also, document these.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/rappor...
File components/rappor/rappor_service_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/610001/components/rappor/rappor...
components/rappor/rappor_service_unittest.cc:17: return
&(rappor_service->rappor_metrics_proto_);
Nit: No need for ()'s.

[Steven Holte, 2014-01-09 22:02:59]

BTW, what should I do about this presubmit error?

These files spam the console log with printf/fprintf:
  chrome/browser/chrome_browser_main.cc

I'm not actually adding printfs to that file.

https://codereview.chromium.org/49753002/diff/610001/chrome/browser/prefs/pre...
File chrome/browser/prefs/pref_metrics_service.cc (right):

https://codereview.chromium.org/49753002/diff/610001/chrome/browser/prefs/pre...
chrome/browser/prefs/pref_metrics_service.cc:166:
rappor::GetUrlParts(homepage_url, &url_parts);
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> I think the service should just have a RecordURL() method that does this
> internally, so the caller doesn't need this extra logic. (Then rappor_utils
can
> go away.)

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/bloom_...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/610001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:14: const uint32_t kHashSeedCount = 4;
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> You can probably get rid of kHashSeedCount and just use [] on the line below.
> Then, you can use arraysize(kHashSeeds) in place of it.

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:46: for (size_t i = 0, len = strings.size(); i
< len; ++i)
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> Nit: No need for |len|, just use |strings.size()| directly. (Compiler will
> optimize it.)

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:89: abort();
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> Don't use abort() in Chrome code. You probably want NOTREACHED();

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:22: ByteVector* ByteVectorAnd(ByteVector*
lhs, const ByteVector& rhs);
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> Per style guide, non-const parameters should be last.

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:24: ByteVector*
ByteVectorMerge(ByteVector* lhs,
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> This needs its own more specific comment to explain the functionality.

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:32: explicit ByteVectorGenerator(size_t
byte_count);
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> Add an explicit destructor and define it in the C++ file. Same for the class
> below.

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:33: // Generates a random byte vector
where each bit is independently set with
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> Nit: Add an empty line above this.

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:43: ByteVector GetRandomByteVector();
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> Add a line after this.

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:45: };
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> DISALLOW_COPY_AND_ASSIGN()? Also below.

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:51: HmacByteVectorGenerator(size_t
byte_count, const std::string& secret);
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> Document params.

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:54: virtual uint8_t RandByte();
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> OVERRIDE

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/log_up...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/610001/components/rappor/log_up...
components/rappor/log_uploader.cc:52: if (!upload_timer_.IsRunning() &&
!callback_pending_) {
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> Nit: No need for {}s

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/log_up...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/610001/components/rappor/log_up...
components/rappor/log_uploader.h:23: LogUploader(const char* server_url, const
char* mime_type);
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> Document params. Add explicit destructor, define in .cc file.

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/log_up...
components/rappor/log_uploader.h:25: void
SetRequestContext(net::URLRequestContextGetter* request_context);
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> Any reason this shouldn't be a ctor param?

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/log_up...
components/rappor/log_uploader.h:27: void QueueLog(const std::string& logData);
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> log_data. Also, document these.

Done.

https://codereview.chromium.org/49753002/diff/610001/components/rappor/rappor...
File components/rappor/rappor_service_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/610001/components/rappor/rappor...
components/rappor/rappor_service_unittest.cc:17: return
&(rappor_service->rappor_metrics_proto_);
On 2014/01/09 19:23:09, Alexei Svitkine wrote:
> Nit: No need for ()'s.

Done.

[Alexei Svitkine (slow), 2014-01-09 22:19:30]

On Thu, Jan 9, 2014 at 5:03 PM, <holte@chromium.org> wrote:

> BTW, what should I do about this presubmit error?
>
> These files spam the console log with printf/fprintf:
>   chrome/browser/chrome_browser_main.cc
>
> I'm not actually adding printfs to that file.


You need to sync, the presubmit should be ignoring that file now:

https://code.google.com/p/chromium/codesearch#chromium/src/PRESUBMIT.py&l=894

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[Ilya Sherman, 2014-01-10 11:00:32]

Thanks.  This actually looks really good, I'm just a nitpicky reviewer.  Don't
be disheartened :)

https://codereview.chromium.org/49753002/diff/920001/chrome/browser/prefs/pre...
File chrome/browser/prefs/pref_metrics_service.cc (right):

https://codereview.chromium.org/49753002/diff/920001/chrome/browser/prefs/pre...
chrome/browser/prefs/pref_metrics_service.cc:34: "Settings.HomePage", 16 /*
Bloom filter size bytes */,
Optional nit: One arg per line might be a little clearer.

https://codereview.chromium.org/49753002/diff/920001/chrome/browser/prefs/pre...
chrome/browser/prefs/pref_metrics_service.cc:39: rappor::PROBABILITY_50 /* Zero
coin probability */
This is a lot of manual configuration.  Do all of these knobs really need to be
exposed, or can we just provide reasonable defaults for clients to use?

https://codereview.chromium.org/49753002/diff/920001/components/rappor.gypi
File components/rappor.gypi (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor.gypi#n...
components/rappor.gypi:1: # Copyright (c) 2013 The Chromium Authors. All rights
reserved.
nit: 2014, and no "(c)"

https://codereview.chromium.org/49753002/diff/920001/components/rappor/DEPS
File components/rappor/DEPS (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/DEPS#n...
components/rappor/DEPS:2: "+net",
nit: alphabetize?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:5: #include <string>
nit: Already included in the header file.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:8: #include "components/rappor/bloom_filter.h"
nit: This should be the *first* include in the implementation file, even before
system headers.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:13: // Distinct seeds are used to create
unique hash functions for the bloom filter.
Might be worth mentioning how these specific seeds were generated, and why there
are only four of them.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:29: DCHECK_LE(hash_count,
arraysize(kHashSeeds));
Please add an assertion that bytes_size > 0 (and probably > 1)

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:35: for (size_t j = 0; j < hash_count_; ++j) {
nit: Why use |j| rather than |i| here?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:44: for (size_t i = 0; i < strings.size();
++i)
nit: Please include curly braces to enclose all loop bodies, even when they are
just a single line.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
File components/rappor/bloom_filter.h (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.h:18: // BloomFilter is a simple bloom filter for
keeping track of a set of strings.
nit: "bloom filter" -> "Bloom filter", since "Bloom" is a last name (applies
throughout)

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.h:19: class BloomFilter {
Hmm, I'm going to be we already have a Bloom filter class in the Chromium
codebase, e.g. for safebrowsing.  Is this class really different enough to be
needed?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.h:21: BloomFilter(uint32_t bytes_size, uint32_t
hash_count);
nit: Please document the parameters, including the restriction that hash_count
must be <= 4.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.h:25: void AddStrings(const
std::vector<std::string>& strings);
nit: Please document these methods (fine to be very brief, which is a nice way
to quickly indicate that the methods don't do anything surprising).

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.h:33: // Stores the bit array of the bloom
filter.
Optional nit: "bit array" -> "byte array"?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.h:36: uint32_t hash_count_;
nit: This might be more clear if named something like "hash_function_count_".

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
File components/rappor/bloom_filter_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter_unittest.cc:16: ASSERT_EQ(0, filter.bytes()[1]);
Why is line 15 unsigned and line 16 signed?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter_unittest.cc:36: ASSERT_EQ(9u, filter.bytes()[1]);
nit: Please use EXPECT_EQ rather than ASSERT_EQ, throughout, except for the
size() assertions.  The Chromium style convention is to only use ASSERT_EQ in
order to prevent crashes.  (Honestly IMO it's a little silly that we bother to
distinguish, but it is the overwhelmingly prevailing style from what I've seen.)

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter_unittest.cc:37: }
Please add some more testing of potential edge-cases, e.g. tiny Bloom filters,
huge Bloom filters, repeated insertion of a value, hash collisions, etc.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:11: #include "crypto/hmac.h"
nit: This is already included in the header file.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:16: // Computes a bitwise and of byte
vectors and stores the result in rhs.
nit: "bitwise and" -> "bitwise AND", though there's really no need to duplicate
this comment from the header file.  Ditto below.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:19: for (size_t i = 0, len = lhs.size();
i < len; i++) {
nit: i++ -> ++i (applies throughout)

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:69: ByteVectorOr(GetRandomByteVector(),
&bytes));
Hmm, this seems a little bit inefficient.  If the inefficiency is (arguably)
necessary, please document why this approach is used rather than something more
direct like flipping a biased coin for each bit.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:81: NOTREACHED();
nit: Are there any other values in the enum?  Can the default statement be
removed so that the compiler can enforce that all cases are covered?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:88: : ByteVectorGenerator(byte_count),
hmac_(crypto::HMAC::SHA256) {
nit: The hmac_ initialization should be wrapped to the next line (one field per
line if any wrapping at all is necessary).

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:91: const uint64_t kHmacInitialState =
3141592653589793;
What's the advantage of having this be a constant rather than a random number? 
Also, if an arbitrary constant suffices, why not just pick 0?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:96: DCHECK(hmac_.DigestLength() >
sizeof(uint64_t));
Why do you use uint64_t here, but uint8_t below?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:97: hmac_state_ = kHmacInitialState;
nit: Why isn't this set as part of the initializer list?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:27: // Equivalent to (lhs & ~mask) | (rhs
& mask).  Stores the result in rhs.
nit: Might be valuable to document why this is a useful operation to perform on
byte vectors.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
File components/rappor/byte_vector_utils_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils_unittest.cc:45: ASSERT_EQ(random_75.size(),
2u);
It would be nice to have a statistical test that has some high number of 9's of
likelihood of success if things are working correctly, but that would identify
errors in the code if things were misconfigured.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:7: //#include
"chrome/browser/metrics/compression_utils.h"
nit: ?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:29: // the log to the prefs for transmission
during the next chrome session if this
nit: "chrome" -> "Chrome"

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:34: const size_t kMaxQueuedLogs = 10;
nit: It's generally better to limit based on size, rather than count.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:36: }  // anonymous namespace
nit: "anonymous namespace" -> "namespace" (for consistency with most of the
codebase)

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:41: net::URLRequestContextGetter*
request_context)
Who owns the request context?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:43: request_context_(request_context),
callback_pending_(false),
nit: One field per line, please.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:116: } else {
nit: No need for curlies.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:122: }
nit: No need for curlies.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:134: }
nit: No need for curlies.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.h:24: // mime_type of the uploaded data, and
request context to create uploads
nit: "request context" -> "request_context"

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.h:26: LogUploader(const char* server_url, const
char* mime_type,
nit: Please pass "const std::string&" rather than "const char*" unless you have
some really compelling reason to use C-strings.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.h:27: net::URLRequestContextGetter*
request_context);
nit: In header files, please wrap parameters so that there is one per line, and
all params are aligned with one another.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.h:54: const char* server_url_;
nit: This isn't const enough ;)  If you want to store const C-strings, they
should be stored as const char arrays, or as "const char* const"s.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.h:70: bool callback_pending_;
nit: I'd recommend adding a prefix like "has_" to emphasize that this is a
boolean value.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.h:72: // The interval between being told an
upload is done and starting the next
nit: "being told" -> "receiving confirmation" or something like that

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.h:77: };
How much of this code is fairly directly copied from the
//chrome/browser/metrics version of this class?  Could much of the code be
shared?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/proto/...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/proto/...
components/rappor/proto/rappor_metric.proto:19: // The reported bits for the
event
nit: This field is going to need considerably more documentation, especially
since it is an abritrarily-sized byte vector, which generally tends to make
people nervous unless they understand exactly what the restrictions on the
possible values for the byte vector are.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_metric.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric.cc:20: const ByteVector&
RapporMetric::GetBytes() const { return bloom_.bytes(); }
nit: It's fine to name this method "bytes()" and inline the implementation if
you want to, since the operation is cheap.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric.cc:22: ByteVector RapporMetric::GetReport(const
std::string& secret) const {
It would be very useful to include comments in this implementation function
explaining the algorithmic reasoning behind the operations performed.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric.cc:34: ByteVectorGenerator
coinGenerator(real_bits.size());
nit: coinGenerator -> coin_generator

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_metric.h (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric.h:21: // A RapporMetric is an object for
collects string samples into a bloom filter,
nit: "for collects" -> "that collects" or "for collecting"

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric.h:25: //
http://www.chromium.org/developers/design-documents/rappor
I'm going to insist that it's important to have thorough documentation available
at this link *prior* to landing this CL.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_metric_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric_unittest.cc:5: // Test of RapporMetric class
nit: No need for this comment IMO

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric_unittest.cc:14: 4 /* Bloom filter hash count */,
PROBABILITY_75 /* Fake data probability */,
Optional nit: Might be easier to read if written with one data member per line.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric_unittest.cc:33: }
Again, statistical tests with many 9's of likelihood of success would be very
useful for a class like this.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_parameters.h (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_parameters.h:37: // The probability that a one bit
reports as one.
nit: Is this a true one bit, or any one bit, after the fakes have been
substituted?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_parameters.h:39: // The probability that a zero bit
reports as one.
Ditto

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_parameters.h:41: };
nit: Should all of the fields in this struct be marked as const, or is that
overkill, do you think?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_pref_names.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_pref_names.cc:10: // A base-64 encoded randomly
generated byte string, which is used as a seed
nit: "encoded randomly" -> "encoded, randomly"

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_pref_names.cc:11: // for generating fake data.
nit: "for generating fake data" might be more easily understood by a casual
reader as something like "for randomizing reported data", or perhaps a different
word like "obfuscating", "obscuring", etc. in place of "randomizing".

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.cc:25: const char kServerUrl[] =
"http://localhost:1234/rappor/v2";
nit: Please update this, yadda yadda.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.cc:26: const char kMimeType[] =
"application/vnd.chrome.uma";
nit: This too, probably, though I bet it doesn't really matter.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.cc:36: delete uploader_;
No no no.  Please use a smart pointer instead.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.cc:75: std::string rappor_secret_64 =
pref_service->GetString(prefs::kRapporSecret);
nit: I think "_base64" would be clearer than just "_64", which makes me think
this is a 64-bit value or something.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.cc:79: return;
Is there a reason to expect that decoding might fail?  If so, please document
that.  (I'm assuming you're worried about corrupted prefs).

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.cc:121: // TODO(holte) break up host more
nit: "TODO(holte)" -> "TODO(holte):"

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.cc:127: RapporMetric*
RapporService::GetRapporMetric(
Please return a scoped_ptr to indicate transfer of memory ownership.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.cc:140: DCHECK_EQ(parameters.zero_coin_prob,
rappor->parameters()->zero_coin_prob);
nit: How about just DCHECK_EQ(parameters, rappor->parameters), or
DCHECK(parameters == rappor->parameters) if you don't want to write string
serialization code for the parameters struct?  That seems less likely to bitrot
in case any new fields are added to the struct.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_service.h (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:26: class RapporReporter;
Hmm, where is this class defined?  I don't recall seeing an implementation for
it.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:28: // This class provides base functionality
for logging rappor data.
nit: What does the "base" in "base functionality" refer to?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:34: // Starts the rappord system.  Should be
called when starting up.
nit: "rappord" -> "rappor"?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:37: // Registers the names of all of the
Preferences used by RapporService in the
nit: No need to capitalize "Preferences" here, I think.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:51: friend class RapporServiceTest;
Please don't add friend classes to new code, unless there's something really
unusual going on.  Friending a class gives it too much permission to muck about
with internal state.  Instead, expose the needed functionality via a set of
protected methods that a test-specific subclass can access or mock out.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:53: // Generates a rappor_secret and stores
it in preferences.  If already stored,
nit: "rappor_secret" isn't a variable name defined anywhere close to hear (that
I'm spotting, at least), so probably best to remove the underscore.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:54: // just retreives the stored value.
Is it important that a client's rappor secret remains constant for a long period
of time?  (I think it is, if I correctly understand the purpose of the secret.) 
If so, please document that either here or in the implementation file.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:58: void LogRappor(const RapporMetric&
rappor);
nit: "LogRappor" -> "LogMetric"; "rappor" -> "metric"

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:63: // Called whenever the logging interval
elapses.
What does the method do when it's called?  Please document.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:66: // Find a rappor metric by name, and
create it if it doesn't already exist.
nit: "and create it" -> "creating it"

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:69: // Client side secret used to generate
fake bits.
nit: "Client side" -> "Client-side"

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:70: std::string rappor_secret_;
nit: Probably no need for the "rappor_" prefix here or for any of the other
variable names, unless you're trying to distinguish multiple secrets within this
class.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:72: RapporMetricsProto rappor_metrics_proto_;
nit: Docs, please.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:74: // Timer which schedules calls
OnLogInterval()
nit: "calls" -> "calls to"

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:77: LogUploader* uploader_;
nit: Docs.  Also, who owns the memory for this object?

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:80: typedef std::map<std::string,
RapporMetric*> RapporMap;
Optional nit: This is definitely a personal style preference, but I'd prefer
that you not introduce this typedef, and just spell out the full type when it is
used.  IMO that is clearer, as it requires less cross-referencing.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:81: RapporMap rappors_;
nit: metrics_;

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:84: base::Lock lock_;
Why does the map need a lock?  Is this class thread-safe?  Why?  I'd strongly
encourage you to pick one thread and stick to it, unless there's a quite
compelling reason not to do so.

[Alexei Svitkine (slow), 2014-01-10 12:51:02]

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:44: for (size_t i = 0; i < strings.size();
++i)
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Please include curly braces to enclose all loop bodies, even when they
are
> just a single line.

Hmm, I advised the opposite. What's the reasoning behind this advise? I thought
Chromium preferred no {}'s for single line bodies, regardless of loop or if or
whatever.

[Ilya Sherman, 2014-01-10 22:34:47]

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:44: for (size_t i = 0; i < strings.size();
++i)
On 2014/01/10 12:51:02, Alexei Svitkine wrote:
> On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> > nit: Please include curly braces to enclose all loop bodies, even when they
> are
> > just a single line.
> 
> Hmm, I advised the opposite. What's the reasoning behind this advise? I
thought
> Chromium preferred no {}'s for single line bodies, regardless of loop or if or
> whatever.

Sorry, I should have marked this as an optional nit, since it's personal style
preference rather than something dictated by the style guide.  That said, in the
parts of the Chromium codebase that I've worked on, the prevailing style has
been to omit curly braces only for single-line if stmts.

[Steven Holte, 2014-01-14 00:47:54]

Partial response to Ilya's feedback.  Will address more in next patch.

https://codereview.chromium.org/49753002/diff/920001/chrome/browser/prefs/pre...
File chrome/browser/prefs/pref_metrics_service.cc (right):

https://codereview.chromium.org/49753002/diff/920001/chrome/browser/prefs/pre...
chrome/browser/prefs/pref_metrics_service.cc:34: "Settings.HomePage", 16 /*
Bloom filter size bytes */,
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Optional nit: One arg per line might be a little clearer.

Done.

https://codereview.chromium.org/49753002/diff/920001/chrome/browser/prefs/pre...
chrome/browser/prefs/pref_metrics_service.cc:39: rappor::PROBABILITY_50 /* Zero
coin probability */
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> This is a lot of manual configuration.  Do all of these knobs really need to
be
> exposed, or can we just provide reasonable defaults for clients to use?

We should have defaults for this eventually, but we don't have guidelines for
setting these yet.  I think that's something that will be easy to add as we add
additional metrics.

https://codereview.chromium.org/49753002/diff/920001/components/rappor.gypi
File components/rappor.gypi (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor.gypi#n...
components/rappor.gypi:1: # Copyright (c) 2013 The Chromium Authors. All rights
reserved.
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: 2014, and no "(c)"

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/DEPS
File components/rappor/DEPS (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/DEPS#n...
components/rappor/DEPS:2: "+net",
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: alphabetize?

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:5: #include <string>
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Already included in the header file.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:8: #include "components/rappor/bloom_filter.h"
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: This should be the *first* include in the implementation file, even
before
> system headers.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:8: #include "components/rappor/bloom_filter.h"
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: This should be the *first* include in the implementation file, even
before
> system headers.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:29: DCHECK_LE(hash_count,
arraysize(kHashSeeds));
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Please add an assertion that bytes_size > 0 (and probably > 1)

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:35: for (size_t j = 0; j < hash_count_; ++j) {
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Why use |j| rather than |i| here?

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
File components/rappor/bloom_filter.h (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.h:18: // BloomFilter is a simple bloom filter for
keeping track of a set of strings.
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "bloom filter" -> "Bloom filter", since "Bloom" is a last name (applies
> throughout)

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.h:19: class BloomFilter {
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Hmm, I'm going to be we already have a Bloom filter class in the Chromium
> codebase, e.g. for safebrowsing.  Is this class really different enough to be
> needed?

Yes.  Extracting safebrowsing's bloom filter looks very non-trivial.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.h:21: BloomFilter(uint32_t bytes_size, uint32_t
hash_count);
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Please document the parameters, including the restriction that hash_count
> must be <= 4.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.h:25: void AddStrings(const
std::vector<std::string>& strings);
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Please document these methods (fine to be very brief, which is a nice way
> to quickly indicate that the methods don't do anything surprising).

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.h:33: // Stores the bit array of the bloom
filter.
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Optional nit: "bit array" -> "byte array"?

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.h:36: uint32_t hash_count_;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: This might be more clear if named something like "hash_function_count_".

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
File components/rappor/bloom_filter_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter_unittest.cc:16: ASSERT_EQ(0, filter.bytes()[1]);
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Why is line 15 unsigned and line 16 signed?

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter_unittest.cc:36: ASSERT_EQ(9u, filter.bytes()[1]);
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Please use EXPECT_EQ rather than ASSERT_EQ, throughout, except for the
> size() assertions.  The Chromium style convention is to only use ASSERT_EQ in
> order to prevent crashes.  (Honestly IMO it's a little silly that we bother to
> distinguish, but it is the overwhelmingly prevailing style from what I've
seen.)

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:11: #include "crypto/hmac.h"
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: This is already included in the header file.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:16: // Computes a bitwise and of byte
vectors and stores the result in rhs.
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "bitwise and" -> "bitwise AND", though there's really no need to
duplicate
> this comment from the header file.  Ditto below.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:19: for (size_t i = 0, len = lhs.size();
i < len; i++) {
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: i++ -> ++i (applies throughout)

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:81: NOTREACHED();
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Are there any other values in the enum?  Can the default statement be
> removed so that the compiler can enforce that all cases are covered?

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:88: : ByteVectorGenerator(byte_count),
hmac_(crypto::HMAC::SHA256) {
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: The hmac_ initialization should be wrapped to the next line (one field
per
> line if any wrapping at all is necessary).

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:96: DCHECK(hmac_.DigestLength() >
sizeof(uint64_t));
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Why do you use uint64_t here, but uint8_t below?

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:7: //#include
"chrome/browser/metrics/compression_utils.h"
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: ?

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:29: // the log to the prefs for transmission
during the next chrome session if this
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "chrome" -> "Chrome"

Removed the sentence because it doesn't actually apply right now.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:34: const size_t kMaxQueuedLogs = 10;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: It's generally better to limit based on size, rather than count.

Added TODO

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:36: }  // anonymous namespace
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "anonymous namespace" -> "namespace" (for consistency with most of the
> codebase)

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:41: net::URLRequestContextGetter*
request_context)
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Who owns the request context?

Used scoped_refptr

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:43: request_context_(request_context),
callback_pending_(false),
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: One field per line, please.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:116: } else {
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: No need for curlies.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:122: }
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: No need for curlies.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.cc:134: }
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: No need for curlies.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.h:24: // mime_type of the uploaded data, and
request context to create uploads
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "request context" -> "request_context"

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.h:26: LogUploader(const char* server_url, const
char* mime_type,
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Please pass "const std::string&" rather than "const char*" unless you
have
> some really compelling reason to use C-strings.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.h:27: net::URLRequestContextGetter*
request_context);
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: In header files, please wrap parameters so that there is one per line,
and
> all params are aligned with one another.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.h:54: const char* server_url_;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: This isn't const enough ;)  If you want to store const C-strings, they
> should be stored as const char arrays, or as "const char* const"s.

changed to const std::string

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.h:70: bool callback_pending_;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: I'd recommend adding a prefix like "has_" to emphasize that this is a
> boolean value.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.h:72: // The interval between being told an
upload is done and starting the next
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "being told" -> "receiving confirmation" or something like that

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/log_up...
components/rappor/log_uploader.h:77: };
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> How much of this code is fairly directly copied from the
> //chrome/browser/metrics version of this class?  Could much of the code be
> shared?

Almost all of this code has some corresponding code in either MetricsService or
MetricsReportingScheduler.  I tried to write it in such a way that it could be
reused by the metrics code, but I didn't want to introduce significant UMA
changes in this CL, so this class would need to be extended a bit to work there,
and I'd prefer to defer that until a later change.  Some of the changes from
UMA: No serializing unsent logs to prefs, no prioritizing uploading initial
logs, disconnected log rotation time from upload success/failure.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/proto/...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/proto/...
components/rappor/proto/rappor_metric.proto:19: // The reported bits for the
event
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: This field is going to need considerably more documentation, especially
> since it is an abritrarily-sized byte vector, which generally tends to make
> people nervous unless they understand exactly what the restrictions on the
> possible values for the byte vector are.

Renamed messages, expanded the description, and added a link to the design doc.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_metric.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric.cc:20: const ByteVector&
RapporMetric::GetBytes() const { return bloom_.bytes(); }
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: It's fine to name this method "bytes()" and inline the implementation if
> you want to, since the operation is cheap.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric.cc:22: ByteVector RapporMetric::GetReport(const
std::string& secret) const {
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> It would be very useful to include comments in this implementation function
> explaining the algorithmic reasoning behind the operations performed.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric.cc:34: ByteVectorGenerator
coinGenerator(real_bits.size());
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: coinGenerator -> coin_generator

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_metric.h (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric.h:21: // A RapporMetric is an object for
collects string samples into a bloom filter,
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "for collects" -> "that collects" or "for collecting"

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric.h:25: //
http://www.chromium.org/developers/design-documents/rappor
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> I'm going to insist that it's important to have thorough documentation
available
> at this link *prior* to landing this CL.

That is the plan.  Ulfar has told me to expect an updated design doc from the
privacy team, so I'm waiting on that.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_metric_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric_unittest.cc:5: // Test of RapporMetric class
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: No need for this comment IMO

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric_unittest.cc:14: 4 /* Bloom filter hash count */,
PROBABILITY_75 /* Fake data probability */,
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Optional nit: Might be easier to read if written with one data member per
line.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric_unittest.cc:14: 4 /* Bloom filter hash count */,
PROBABILITY_75 /* Fake data probability */,
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Optional nit: Might be easier to read if written with one data member per
line.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_parameters.h (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_parameters.h:37: // The probability that a one bit
reports as one.
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Is this a true one bit, or any one bit, after the fakes have been
> substituted?

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_parameters.h:39: // The probability that a zero bit
reports as one.
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Ditto

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_parameters.h:41: };
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Should all of the fields in this struct be marked as const, or is that
> overkill, do you think?

I don't think that would be meaningful, since the same thing is accomplished by
making the instance of the object const.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_pref_names.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_pref_names.cc:10: // A base-64 encoded randomly
generated byte string, which is used as a seed
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "encoded randomly" -> "encoded, randomly"

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_pref_names.cc:11: // for generating fake data.
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "for generating fake data" might be more easily understood by a casual
> reader as something like "for randomizing reported data", or perhaps a
different
> word like "obfuscating", "obscuring", etc. in place of "randomizing".

Changed to "redacting collected data".

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_service.h (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:26: class RapporReporter;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Hmm, where is this class defined?  I don't recall seeing an implementation for
> it.

Removed.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:28: // This class provides base functionality
for logging rappor data.
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: What does the "base" in "base functionality" refer to?

Rewrote comment.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:34: // Starts the rappord system.  Should be
called when starting up.
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "rappord" -> "rappor"?

Rewrote comment.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:37: // Registers the names of all of the
Preferences used by RapporService in the
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: No need to capitalize "Preferences" here, I think.

Done.

[Alexei Svitkine (slow), 2014-01-14 17:55:48]

While you're still addressing Ilya's comments, the other thing I'd like to
mention is I think we should (initially) put this behind a field trial to roll
it out. That way, this can be landed even if this server-side part isn't ready
to receive the data yet and we can slowly roll it out.

[Steven Holte, 2014-01-15 04:53:43]

Incorporated/responded to the rest of Ilya's comments.  Will add finch
experiment in next revision.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter.cc:13: // Distinct seeds are used to create
unique hash functions for the bloom filter.
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Might be worth mentioning how these specific seeds were generated, and why
there
> are only four of them.

Expanded the description.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
File components/rappor/bloom_filter_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/bloom_...
components/rappor/bloom_filter_unittest.cc:37: }
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Please add some more testing of potential edge-cases, e.g. tiny Bloom filters,
> huge Bloom filters, repeated insertion of a value, hash collisions, etc.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:69: ByteVectorOr(GetRandomByteVector(),
&bytes));
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Hmm, this seems a little bit inefficient.  If the inefficiency is (arguably)
> necessary, please document why this approach is used rather than something
more
> direct like flipping a biased coin for each bit.

I'm not sure really what the more efficient way would be, or how I would flip
biased coins directly.  I've dropped the implementation of the probabilities
other than 75 and 50, because they aren't currently used anyway.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:91: const uint64_t kHmacInitialState =
3141592653589793;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> What's the advantage of having this be a constant rather than a random number?

> Also, if an arbitrary constant suffices, why not just pick 0?

Basically, we don't want to use 0 just in case it has some exploitable property.
 We can't use a runtime-random number, since it needs to be deterministic, so a
random-ish number is a compromise.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.cc:97: hmac_state_ = kHmacInitialState;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Why isn't this set as part of the initializer list?

It seems more clear to put the constant in a local scope.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils.h:27: // Equivalent to (lhs & ~mask) | (rhs
& mask).  Stores the result in rhs.
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Might be valuable to document why this is a useful operation to perform
on
> byte vectors.

Not really sure what to add here, since it's just a repeated idiom.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
File components/rappor/byte_vector_utils_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/byte_v...
components/rappor/byte_vector_utils_unittest.cc:45: ASSERT_EQ(random_75.size(),
2u);
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> It would be nice to have a statistical test that has some high number of 9's
of
> likelihood of success if things are working correctly, but that would identify
> errors in the code if things were misconfigured.

Added simple binomial tests.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.cc:25: const char kServerUrl[] =
"http://localhost:1234/rappor/v2";
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Please update this, yadda yadda.

Updated this, assuming the collector will extend the UMA one.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.cc:26: const char kMimeType[] =
"application/vnd.chrome.uma";
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: This too, probably, though I bet it doesn't really matter.

It doesn't seem to matter, but I changed it.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.cc:36: delete uploader_;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> No no no.  Please use a smart pointer instead.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.cc:75: std::string rappor_secret_64 =
pref_service->GetString(prefs::kRapporSecret);
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: I think "_base64" would be clearer than just "_64", which makes me think
> this is a 64-bit value or something.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.cc:79: return;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Is there a reason to expect that decoding might fail?  If so, please document
> that.  (I'm assuming you're worried about corrupted prefs).

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.cc:121: // TODO(holte) break up host more
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "TODO(holte)" -> "TODO(holte):"

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.cc:127: RapporMetric*
RapporService::GetRapporMetric(
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Please return a scoped_ptr to indicate transfer of memory ownership.

There is no transfer of ownership here.  This is a private method that gets a
value stored in the private map.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_service.h (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:51: friend class RapporServiceTest;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Please don't add friend classes to new code, unless there's something really
> unusual going on.  Friending a class gives it too much permission to muck
about
> with internal state.  Instead, expose the needed functionality via a set of
> protected methods that a test-specific subclass can access or mock out.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:53: // Generates a rappor_secret and stores
it in preferences.  If already stored,
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "rappor_secret" isn't a variable name defined anywhere close to hear
(that
> I'm spotting, at least), so probably best to remove the underscore.

This referenced rappor_secret_, now references secret_.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:54: // just retreives the stored value.
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Is it important that a client's rappor secret remains constant for a long
period
> of time?  (I think it is, if I correctly understand the purpose of the
secret.) 
> If so, please document that either here or in the implementation file.

Rewrote this comment a bit.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:58: void LogRappor(const RapporMetric&
rappor);
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "LogRappor" -> "LogMetric"; "rappor" -> "metric"

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:63: // Called whenever the logging interval
elapses.
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> What does the method do when it's called?  Please document.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:66: // Find a rappor metric by name, and
create it if it doesn't already exist.
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "and create it" -> "creating it"

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:69: // Client side secret used to generate
fake bits.
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "Client side" -> "Client-side"

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:70: std::string rappor_secret_;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Probably no need for the "rappor_" prefix here or for any of the other
> variable names, unless you're trying to distinguish multiple secrets within
this
> class.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:72: RapporMetricsProto rappor_metrics_proto_;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Docs, please.

Removed

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:74: // Timer which schedules calls
OnLogInterval()
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: "calls" -> "calls to"

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:77: LogUploader* uploader_;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: Docs.  Also, who owns the memory for this object?

Changed to scoped_ptr.  Added docstring.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:80: typedef std::map<std::string,
RapporMetric*> RapporMap;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Optional nit: This is definitely a personal style preference, but I'd prefer
> that you not introduce this typedef, and just spell out the full type when it
is
> used.  IMO that is clearer, as it requires less cross-referencing.

Done.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:81: RapporMap rappors_;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> nit: metrics_;

Changed to metrics_map_

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_service.h:84: base::Lock lock_;
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Why does the map need a lock?  Is this class thread-safe?  Why?  I'd strongly
> encourage you to pick one thread and stick to it, unless there's a quite
> compelling reason not to do so.

I would expect all of the upload/report generation functions to be called on the
same thread, but that the public RecordSample() method might be called from a
different thread.  I'm not familiar enough with how threads get used in Chrome
to know whether that's likely.

[Steven Holte, 2014-01-15 23:00:56]

Added a rollout field trial and statistical test of RapporMetric.

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
File components/rappor/rappor_metric_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/920001/components/rappor/rappor...
components/rappor/rappor_metric_unittest.cc:33: }
On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> Again, statistical tests with many 9's of likelihood of success would be very
> useful for a class like this.

Done.

[Alexei Svitkine (slow), 2014-01-21 18:14:03]

Some more comments your way. (Sorry for my latency on this, but many things are
on my plate...)

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/bloom...
File components/rappor/bloom_filter_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/bloom...
components/rappor/bloom_filter_unittest.cc:13: int CountBits(const ByteVector&
v) {
Is it me or is this duplicate in like 3 test files?

I suggest moving this to byte_vector_utils.h

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:76: DCHECK(hmac_.DigestLength() >
sizeof(uint8_t));
DCHECK_GT

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/byte_...
File components/rappor/byte_vector_utils_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/byte_...
components/rappor/byte_vector_utils_unittest.cc:10: 
Nit: Remove one blank line.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/byte_...
components/rappor/byte_vector_utils_unittest.cc:17: for(size_t i = 0; i <
v.size(); i++) {
Nit: space after for. also after "if" below.

Also, prefer ++i over i++ (here and below).

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/byte_...
components/rappor/byte_vector_utils_unittest.cc:20: if(byte & 1 << j) {
Nit: No {}'s.

Also, please but parens around one of the sub expression (so a person reading
the code doesn't need to remember order of operations between & and <<.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
File components/rappor/rappor_metric.h (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_metric.h:28: RapporMetric(const RapporParameters&
parameters);
Nit: add explicit keyword

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_metric.h:29: 
Nit: Add explicit dtor and implement in the .cc file.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_metric.h:38: ByteVector GetReport(const std::string&
secret) const;
Document this.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
File components/rappor/rappor_metric_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_metric_unittest.cc:25: TEST(RapporMetricTest,
BasicTest) {
Nit: Keep all the TEST methods together (at the end), don't interleave with
other code or const params.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_metric_unittest.cc:68: sample[0] = i;
It's kind of weird to generate these non-human readable strings as samples...

I'd instead generate actual human readable strings instead to be more similar to
actual test data. Maybe via StringPrintf("%d", i). Or would that make the
assertions below harder to make?

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
File components/rappor/rappor_parameters.h (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_parameters.h:42: std::ostream& operator<<(std::ostream&
out, const RapporParameters& rhs);
Generally, operator overloads are discouraged. I'd instead make a ToString() and
IsEqual() methods.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_service.cc:26: const char kServerUrl[] =
"https://clients4.google.com/rappor/v2";
Probably wouldn't be v2.

Maybe it's a good idea to use Finch params to provide the URL until we actually
get it finalized.

[Steven Holte, 2014-01-21 20:25:13]

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/bloom...
File components/rappor/bloom_filter_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/bloom...
components/rappor/bloom_filter_unittest.cc:13: int CountBits(const ByteVector&
v) {
On 2014/01/21 18:14:04, Alexei Svitkine wrote:
> Is it me or is this duplicate in like 3 test files?
> 
> I suggest moving this to byte_vector_utils.h

Done.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:76: DCHECK(hmac_.DigestLength() >
sizeof(uint8_t));
On 2014/01/21 18:14:04, Alexei Svitkine wrote:
> DCHECK_GT

Done.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/byte_...
File components/rappor/byte_vector_utils_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/byte_...
components/rappor/byte_vector_utils_unittest.cc:10: 
On 2014/01/21 18:14:04, Alexei Svitkine wrote:
> Nit: Remove one blank line.

Done.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/byte_...
components/rappor/byte_vector_utils_unittest.cc:17: for(size_t i = 0; i <
v.size(); i++) {
On 2014/01/21 18:14:04, Alexei Svitkine wrote:
> Nit: space after for. also after "if" below.
> 
> Also, prefer ++i over i++ (here and below).

Done.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/byte_...
components/rappor/byte_vector_utils_unittest.cc:20: if(byte & 1 << j) {
On 2014/01/21 18:14:04, Alexei Svitkine wrote:
> Nit: No {}'s.
> 
> Also, please but parens around one of the sub expression (so a person reading
> the code doesn't need to remember order of operations between & and <<.

Done.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
File components/rappor/rappor_metric.h (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_metric.h:28: RapporMetric(const RapporParameters&
parameters);
On 2014/01/21 18:14:04, Alexei Svitkine wrote:
> Nit: add explicit keyword

Done.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_metric.h:29: 
On 2014/01/21 18:14:04, Alexei Svitkine wrote:
> Nit: Add explicit dtor and implement in the .cc file.

Done.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_metric.h:38: ByteVector GetReport(const std::string&
secret) const;
On 2014/01/21 18:14:04, Alexei Svitkine wrote:
> Document this.

Done.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
File components/rappor/rappor_metric_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_metric_unittest.cc:25: TEST(RapporMetricTest,
BasicTest) {
On 2014/01/21 18:14:04, Alexei Svitkine wrote:
> Nit: Keep all the TEST methods together (at the end), don't interleave with
> other code or const params.

Done.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_metric_unittest.cc:68: sample[0] = i;
On 2014/01/21 18:14:04, Alexei Svitkine wrote:
> It's kind of weird to generate these non-human readable strings as samples...
> 
> I'd instead generate actual human readable strings instead to be more similar
to
> actual test data. Maybe via StringPrintf("%d", i). Or would that make the
> assertions below harder to make?

Done.

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
File components/rappor/rappor_parameters.h (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_parameters.h:42: std::ostream& operator<<(std::ostream&
out, const RapporParameters& rhs);
On 2014/01/21 18:14:04, Alexei Svitkine wrote:
> Generally, operator overloads are discouraged. I'd instead make a ToString()
and
> IsEqual() methods.

These only exist to make DCHECK_EQ work and I've added a comment to that effect.
 Is there a better way to enable that?

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_service.cc:26: const char kServerUrl[] =
"https://clients4.google.com/rappor/v2";
On 2014/01/21 18:14:04, Alexei Svitkine wrote:
> Probably wouldn't be v2.
> 
> Maybe it's a good idea to use Finch params to provide the URL until we
actually
> get it finalized.

Done.  Also changed service activation to depend on the server url being
non-empty, instead of a specific group name.

[Alexei Svitkine (slow), 2014-01-21 20:30:38]

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
File components/rappor/rappor_parameters.h (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_parameters.h:42: std::ostream& operator<<(std::ostream&
out, const RapporParameters& rhs);
On 2014/01/21 20:25:14, Steven Holte wrote:
> On 2014/01/21 18:14:04, Alexei Svitkine wrote:
> > Generally, operator overloads are discouraged. I'd instead make a ToString()
> and
> > IsEqual() methods.
> 
> These only exist to make DCHECK_EQ work and I've added a comment to that
effect.
>  Is there a better way to enable that?

How about just doing DCHECK_EQ() on ToString() result? Sure its costly, but its
only for debug builds anyway.

https://codereview.chromium.org/49753002/diff/1330001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/1330001/components/rappor/rappo...
components/rappor/rappor_service.cc:35: std::string GetServerUrl() {
Return a GURL. Also, make LogUploader() take a GURL.

https://codereview.chromium.org/49753002/diff/1330001/components/rappor/rappo...
components/rappor/rappor_service.cc:55: if (!FieldTrialIsActive())
Remove this and just do:

GURL server_url = GetServerUrl();
if (!server_url.is_valid())
  return;

(With a comment...)

[Steven Holte, 2014-01-21 21:05:31]

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
File components/rappor/rappor_parameters.h (right):

https://codereview.chromium.org/49753002/diff/1230001/components/rappor/rappo...
components/rappor/rappor_parameters.h:42: std::ostream& operator<<(std::ostream&
out, const RapporParameters& rhs);
On 2014/01/21 20:30:39, Alexei Svitkine wrote:
> On 2014/01/21 20:25:14, Steven Holte wrote:
> > On 2014/01/21 18:14:04, Alexei Svitkine wrote:
> > > Generally, operator overloads are discouraged. I'd instead make a
ToString()
> > and
> > > IsEqual() methods.
> > 
> > These only exist to make DCHECK_EQ work and I've added a comment to that
> effect.
> >  Is there a better way to enable that?
> 
> How about just doing DCHECK_EQ() on ToString() result? Sure its costly, but
its
> only for debug builds anyway.

Done.

https://codereview.chromium.org/49753002/diff/1330001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/1330001/components/rappor/rappo...
components/rappor/rappor_service.cc:35: std::string GetServerUrl() {
On 2014/01/21 20:30:39, Alexei Svitkine wrote:
> Return a GURL. Also, make LogUploader() take a GURL.

Done.

https://codereview.chromium.org/49753002/diff/1330001/components/rappor/rappo...
components/rappor/rappor_service.cc:55: if (!FieldTrialIsActive())
On 2014/01/21 20:30:39, Alexei Svitkine wrote:
> Remove this and just do:
> 
> GURL server_url = GetServerUrl();
> if (!server_url.is_valid())
>   return;
> 
> (With a comment...)

Done.

[Alexei Svitkine (slow), 2014-01-22 22:13:27]

So code is looking pretty good at this point, so I'm going to look more closely
at the API and some higher level questions. First, very important one I think,
is below.

https://codereview.chromium.org/49753002/diff/920001/chrome/browser/prefs/pre...
File chrome/browser/prefs/pref_metrics_service.cc (right):

https://codereview.chromium.org/49753002/diff/920001/chrome/browser/prefs/pre...
chrome/browser/prefs/pref_metrics_service.cc:39: rappor::PROBABILITY_50 /* Zero
coin probability */
On 2014/01/14 00:47:54, Steven Holte wrote:
> On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
> > This is a lot of manual configuration.  Do all of these knobs really need to
> be
> > exposed, or can we just provide reasonable defaults for clients to use?
> 
> We should have defaults for this eventually, but we don't have guidelines for
> setting these yet.  I think that's something that will be easy to add as we
add
> additional metrics.

This is one area that still concerns me. If the API is open to allow users to
specify their own values for these parameters, then I'm afraid that this may
compromise the guarnatees that the system is supposed to provide.

Specifically, could one come up with a set of values that would defeat the
design of the system that makes it privacy preserving? If so, then that's a bad
whole that I think we shouldn't expose by the API. Also, it's not just
mal-intent that I'm worried here - simply exposing this much complexity to
people adding metrics allows people to pick bad values by virtue of not
understanding the implications, etc.

I would strongly suggest defining "known-good" parameters in the component
itself and just having a set of enum values that's exposed through the API,
where each enum value maps to a set of parameters.

Separately, I'd also like to see some guidance provided from Ulfar for what
"acceptable" parameter ranges are and having strong validation to enforce this.

[ulfar, 2014-01-22 22:29:42]

Guidance coming.

U


On Wed, Jan 22, 2014 at 2:13 PM, <asvitkine@chromium.org> wrote:

> So code is looking pretty good at this point, so I'm going to look more
> closely
> at the API and some higher level questions. First, very important one I
> think,
> is below.
>
>
>
> https://codereview.chromium.org/49753002/diff/920001/
> chrome/browser/prefs/pref_metrics_service.cc
> File chrome/browser/prefs/pref_metrics_service.cc (right):
>
> https://codereview.chromium.org/49753002/diff/920001/
> chrome/browser/prefs/pref_metrics_service.cc#newcode39
> chrome/browser/prefs/pref_metrics_service.cc:39: rappor::PROBABILITY_50
> /* Zero coin probability */
> On 2014/01/14 00:47:54, Steven Holte wrote:
>
>> On 2014/01/10 11:00:32, Ilya Sherman (Away Jan11-Feb2) wrote:
>> > This is a lot of manual configuration.  Do all of these knobs really
>>
> need to
>
>> be
>> > exposed, or can we just provide reasonable defaults for clients to
>>
> use?
>
>  We should have defaults for this eventually, but we don't have
>>
> guidelines for
>
>> setting these yet.  I think that's something that will be easy to add
>>
> as we add
>
>> additional metrics.
>>
>
> This is one area that still concerns me. If the API is open to allow
> users to specify their own values for these parameters, then I'm afraid
> that this may compromise the guarnatees that the system is supposed to
> provide.
>
> Specifically, could one come up with a set of values that would defeat
> the design of the system that makes it privacy preserving? If so, then
> that's a bad whole that I think we shouldn't expose by the API. Also,
> it's not just mal-intent that I'm worried here - simply exposing this
> much complexity to people adding metrics allows people to pick bad
> values by virtue of not understanding the implications, etc.
>
> I would strongly suggest defining "known-good" parameters in the
> component itself and just having a set of enum values that's exposed
> through the API, where each enum value maps to a set of parameters.
>
> Separately, I'd also like to see some guidance provided from Ulfar for
> what "acceptable" parameter ranges are and having strong validation to
> enforce this.
>
> https://codereview.chromium.org/49753002/
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[Steven Holte, 2014-01-24 02:06:16]

Following up on an offline discussion with ulfar and vpihur yesterday, I've
added support for splitting the client population into cohorts, where each
cohort uses a different set of hash functions in the bloom filter, which will
help analysis in the case of hash collisions.

- The list of bloom filter seeds is removed, and the seeds are now just (cohort#
* numHashes + i) instead of pre-generated seeds.

- The cohort number is generated in RapporService and stored in preferences, and
is persistent per client like the rappor secret.

- All metrics generated by the client use the same cohort #, and it tags all of
it's upload with it's cohort #.

[ulfar, 2014-01-24 05:57:38]

To elaborate: Cohorts are useful for eliminating the uncertainty induced by
the Bloom filter hash functions.  By requiring the signal to be supported
by all cohorts, we can basically eliminate false positives.

U




On Thu, Jan 23, 2014 at 6:06 PM, <holte@chromium.org> wrote:

> Following up on an offline discussion with ulfar and vpihur yesterday, I've
> added support for splitting the client population into cohorts, where each
> cohort uses a different set of hash functions in the bloom filter, which
> will
> help analysis in the case of hash collisions.
>
> - The list of bloom filter seeds is removed, and the seeds are now just
> (cohort#
> * numHashes + i) instead of pre-generated seeds.
>
> - The cohort number is generated in RapporService and stored in
> preferences, and
> is persistent per client like the rappor secret.
>
> - All metrics generated by the client use the same cohort #, and it tags
> all of
> it's upload with it's cohort #.
>
> https://codereview.chromium.org/49753002/
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[Alexei Svitkine (slow), 2014-01-24 22:09:02]

More comments.

https://codereview.chromium.org/49753002/diff/1420001/components/rappor/rappo...
File components/rappor/rappor_metric.h (right):

https://codereview.chromium.org/49753002/diff/1420001/components/rappor/rappo...
components/rappor/rappor_metric.h:8: #include <assert.h>
Does the header file need all of these includes? If they're only used by the .cc
file, move them there.

https://codereview.chromium.org/49753002/diff/1420001/components/rappor/rappo...
File components/rappor/rappor_parameters.h (right):

https://codereview.chromium.org/49753002/diff/1420001/components/rappor/rappo...
components/rappor/rappor_parameters.h:8: #include <ostream>
Remove

https://codereview.chromium.org/49753002/diff/1420001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/1420001/components/rappor/rappo...
components/rappor/rappor_service.cc:7: #include <cstdlib>
Why is this needed?

https://codereview.chromium.org/49753002/diff/1420001/components/rappor/rappo...
components/rappor/rappor_service.cc:106:
report->set_name_hash(base::Hash(metric.parameters()->rappor_name));
I wouldn't use base::Hash() here - I think that one is intended to be used
internally in data structures, etc.

Plus, I don't think we should introduce yet another way to hash things. I
suggest using the same hash function we use for histogram names.

https://codereview.chromium.org/49753002/diff/1470001/components/rappor/bloom...
File components/rappor/bloom_filter.h (right):

https://codereview.chromium.org/49753002/diff/1470001/components/rappor/bloom...
components/rappor/bloom_filter.h:25: uint32_t hash_seed_offset);
Nit: 1 param per line and align.

Also, document hash_seed_offset.

https://codereview.chromium.org/49753002/diff/1470001/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/1470001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:30: int CountBits(const ByteVector& v);
Nit: Use more meaningful variable names than |v|. Here and in the impl.

https://codereview.chromium.org/49753002/diff/1470001/components/rappor/log_u...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/1470001/components/rappor/log_u...
components/rappor/log_uploader.cc:82: scoped_ptr<net::URLFetcher>
s(current_fetch_.Pass());
Use more meaningful variable names than |s|.

https://codereview.chromium.org/49753002/diff/1470001/components/rappor/log_u...
components/rappor/log_uploader.cc:92: discard_log = true;
Add a histogram to log these sort of events, so we can diagnose any issues
through UMA.

https://codereview.chromium.org/49753002/diff/1470001/components/rappor/rappo...
File components/rappor/rappor_metric.cc (right):

https://codereview.chromium.org/49753002/diff/1470001/components/rappor/rappo...
components/rappor/rappor_metric.cc:17: DCHECK_GE(cohort, 0);
Bad indentation.

https://codereview.chromium.org/49753002/diff/1470001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/1470001/components/rappor/rappo...
components/rappor/rappor_service.cc:91: void
RapporService::LoadCohort(PrefService* pref_service) {
Add a unit test for this and LoadSecret().

https://codereview.chromium.org/49753002/diff/1470001/components/rappor/rappo...
components/rappor/rappor_service.cc:151: if (!cohort_ < 0)
Make a helper function for this, rather than duplicating this logic in several
places.

https://codereview.chromium.org/49753002/diff/1470001/components/rappor/rappo...
File components/rappor/rappor_service.h (right):

https://codereview.chromium.org/49753002/diff/1470001/components/rappor/rappo...
components/rappor/rappor_service.h:78: // Finds a metric in the metrics_map_,
creating it if it doesn't already
Nit: When mentioning variable names in comments, enclose in |'s - i.e.
|metrics_map_|.

https://codereview.chromium.org/49753002/diff/1470001/components/rappor/rappo...
components/rappor/rappor_service.h:80: RapporMetric* LookupMetric(const
RapporParameters& parameters);
Nit: LookUpMetric

[Steven Holte, 2014-01-24 23:26:57]

https://codereview.chromium.org/49753002/diff/1420001/components/rappor/rappo...
File components/rappor/rappor_metric.h (right):

https://codereview.chromium.org/49753002/diff/1420001/components/rappor/rappo...
components/rappor/rappor_metric.h:8: #include <assert.h>
On 2014/01/24 22:09:03, Alexei Svitkine wrote:
> Does the header file need all of these includes? If they're only used by the
.cc
> file, move them there.

Removed

https://codereview.chromium.org/49753002/diff/1420001/components/rappor/rappo...
File components/rappor/rappor_parameters.h (right):

https://codereview.chromium.org/49753002/diff/1420001/components/rappor/rappo...
components/rappor/rappor_parameters.h:8: #include <ostream>
On 2014/01/24 22:09:03, Alexei Svitkine wrote:
> Remove

Done.

https://codereview.chromium.org/49753002/diff/1420001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/1420001/components/rappor/rappo...
components/rappor/rappor_service.cc:7: #include <cstdlib>
On 2014/01/24 22:09:03, Alexei Svitkine wrote:
> Why is this needed?
Removed

https://codereview.chromium.org/49753002/diff/1420001/components/rappor/rappo...
components/rappor/rappor_service.cc:106:
report->set_name_hash(base::Hash(metric.parameters()->rappor_name));
On 2014/01/24 22:09:03, Alexei Svitkine wrote:
> I wouldn't use base::Hash() here - I think that one is intended to be used
> internally in data structures, etc.
> 
> Plus, I don't think we should introduce yet another way to hash things. I
> suggest using the same hash function we use for histogram names.

Good catch! I thought I'd already had this set to use the same hash function as
histograms.  Since that function is in chrome/common/metrics, I moved it to
components/variations/metric_utils

[Steven Holte, 2014-02-03 20:27:37]

Some changes from discussion with vpihur/ulfar/tyler/dominic:

 - Modified the "Settings.HomePage" metric to record just eTLD+1 instead of
various components, use 2 hash functions in it's bloom filter, cleaned up code
that was used for recording multiple samples.

 - Include the real data in the seed used for generating the fake data, which
strengthens privacy when the data changes occasionally.

[Alexei Svitkine (slow), 2014-02-03 22:34:17]

https://codereview.chromium.org/49753002/diff/1820002/chrome/browser/prefs/pr...
File chrome/browser/prefs/pref_metrics_service.cc (right):

https://codereview.chromium.org/49753002/diff/1820002/chrome/browser/prefs/pr...
chrome/browser/prefs/pref_metrics_service.cc:33: const rappor::RapporParameters
kHomePageRapporParameters = {
As I said in a previous comment, I don't think we should expose the API such
that any possible parameters can be passed in.

Instead, I suggest making a set of "known-good" parameters inside rappor/ and an
enum that lists them and make the API take the enum to refer to a given
parameter config.

This way, only known good parameters (reviewed by OWNERS of rappor/) can be
used, rather than the current state where someone could pass in values that may
defeat the goals of RAPPOR.

[Steven Holte, 2014-02-03 23:58:04]

https://codereview.chromium.org/49753002/diff/1820002/chrome/browser/prefs/pr...
File chrome/browser/prefs/pref_metrics_service.cc (right):

https://codereview.chromium.org/49753002/diff/1820002/chrome/browser/prefs/pr...
chrome/browser/prefs/pref_metrics_service.cc:33: const rappor::RapporParameters
kHomePageRapporParameters = {
On 2014/02/03 22:34:18, Alexei Svitkine wrote:
> As I said in a previous comment, I don't think we should expose the API such
> that any possible parameters can be passed in.
> 
> Instead, I suggest making a set of "known-good" parameters inside rappor/ and
an
> enum that lists them and make the API take the enum to refer to a given
> parameter config.
> 
> This way, only known good parameters (reviewed by OWNERS of rappor/) can be
> used, rather than the current state where someone could pass in values that
may
> defeat the goals of RAPPOR.

Done.  These parameters now live in components/rappor/rappor_registry.cc

[Alexei Svitkine (slow), 2014-02-04 19:20:11]

https://codereview.chromium.org/49753002/diff/2020001/components/rappor/rappo...
File components/rappor/rappor_parameters.h (right):

https://codereview.chromium.org/49753002/diff/2020001/components/rappor/rappo...
components/rappor/rappor_parameters.h:44: extern const RapporParameters
kHomePageRapporParameters;
Remove?

https://codereview.chromium.org/49753002/diff/2020001/components/rappor/rappo...
File components/rappor/rappor_registry.cc (right):

https://codereview.chromium.org/49753002/diff/2020001/components/rappor/rappo...
components/rappor/rappor_registry.cc:13: "Settings.HomePage" /* Metric name */,
This wasn't quite what I meant.

I think the name should still be passed to the API, but the other params should
be in the registry referenced by the enum.

(And the enum name should be more generic - e.g. RAPPOR_URL rather than
RAPPOR_HOMEPAGE).

To avoid having to refactor things too much, maybe you can still keep the name
in RapporParameters, but have an empty name for all of these pre-defined ones
and have a call to RapporService copy the params over and fill the name.

https://codereview.chromium.org/49753002/diff/2020001/components/rappor/rappo...
File components/rappor/rappor_registry.h (right):

https://codereview.chromium.org/49753002/diff/2020001/components/rappor/rappo...
components/rappor/rappor_registry.h:17: extern const RapporParameters
g_rappor_registry[];
Can you expose a function rather than the array directly?

e.g.

const RapporParameters& GetRapporParameters(RapporRegistryIndex index);

The array can live entirely in the .cc file in the anon namespace.

[ulfar, 2014-02-04 19:36:34]

Probably RAPPOR_TLD_PLUS_ONE or something like that would be a good name.


On Tue, Feb 4, 2014 at 11:20 AM, <asvitkine@chromium.org> wrote:

>
> https://codereview.chromium.org/49753002/diff/2020001/
> components/rappor/rappor_parameters.h
> File components/rappor/rappor_parameters.h (right):
>
> https://codereview.chromium.org/49753002/diff/2020001/
> components/rappor/rappor_parameters.h#newcode44
> components/rappor/rappor_parameters.h:44: extern const RapporParameters
> kHomePageRapporParameters;
> Remove?
>
> https://codereview.chromium.org/49753002/diff/2020001/
> components/rappor/rappor_registry.cc
> File components/rappor/rappor_registry.cc (right):
>
> https://codereview.chromium.org/49753002/diff/2020001/
> components/rappor/rappor_registry.cc#newcode13
> components/rappor/rappor_registry.cc:13: "Settings.HomePage" /* Metric
> name */,
> This wasn't quite what I meant.
>
> I think the name should still be passed to the API, but the other params
> should be in the registry referenced by the enum.
>
> (And the enum name should be more generic - e.g. RAPPOR_URL rather than
> RAPPOR_HOMEPAGE).
>
> To avoid having to refactor things too much, maybe you can still keep
> the name in RapporParameters, but have an empty name for all of these
> pre-defined ones and have a call to RapporService copy the params over
> and fill the name.
>
> https://codereview.chromium.org/49753002/diff/2020001/
> components/rappor/rappor_registry.h
> File components/rappor/rappor_registry.h (right):
>
> https://codereview.chromium.org/49753002/diff/2020001/
> components/rappor/rappor_registry.h#newcode17
> components/rappor/rappor_registry.h:17: extern const RapporParameters
> g_rappor_registry[];
> Can you expose a function rather than the array directly?
>
> e.g.
>
> const RapporParameters& GetRapporParameters(RapporRegistryIndex index);
>
> The array can live entirely in the .cc file in the anon namespace.
>
> https://codereview.chromium.org/49753002/
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[Steven Holte, 2014-02-04 21:41:20]

https://codereview.chromium.org/49753002/diff/2020001/components/rappor/rappo...
File components/rappor/rappor_parameters.h (right):

https://codereview.chromium.org/49753002/diff/2020001/components/rappor/rappo...
components/rappor/rappor_parameters.h:44: extern const RapporParameters
kHomePageRapporParameters;
On 2014/02/04 19:20:12, Alexei Svitkine wrote:
> Remove?

Done.

https://codereview.chromium.org/49753002/diff/2020001/components/rappor/rappo...
File components/rappor/rappor_registry.cc (right):

https://codereview.chromium.org/49753002/diff/2020001/components/rappor/rappo...
components/rappor/rappor_registry.cc:13: "Settings.HomePage" /* Metric name */,
On 2014/02/04 19:20:12, Alexei Svitkine wrote:
> This wasn't quite what I meant.
> 
> I think the name should still be passed to the API, but the other params
should
> be in the registry referenced by the enum.
> 
> (And the enum name should be more generic - e.g. RAPPOR_URL rather than
> RAPPOR_HOMEPAGE).
> 
> To avoid having to refactor things too much, maybe you can still keep the name
> in RapporParameters, but have an empty name for all of these pre-defined ones
> and have a call to RapporService copy the params over and fill the name.

Split name and parameters.  PTAL.

https://codereview.chromium.org/49753002/diff/2020001/components/rappor/rappo...
File components/rappor/rappor_registry.h (right):

https://codereview.chromium.org/49753002/diff/2020001/components/rappor/rappo...
components/rappor/rappor_registry.h:17: extern const RapporParameters
g_rappor_registry[];
On 2014/02/04 19:20:12, Alexei Svitkine wrote:
> Can you expose a function rather than the array directly?
> 
> e.g.
> 
> const RapporParameters& GetRapporParameters(RapporRegistryIndex index);
> 
> The array can live entirely in the .cc file in the anon namespace.

Moved this to rappor_service.cc as kRapporParametersForType

[Alexei Svitkine (slow), 2014-02-05 18:07:00]

I've had a chance to do a more thorough review on this (actually looking closely
at the logic, etc). I have some more comments.

https://codereview.chromium.org/49753002/diff/2200001/chrome/common/metrics/m...
File chrome/common/metrics/metrics_log_base.cc (left):

https://codereview.chromium.org/49753002/diff/2200001/chrome/common/metrics/m...
chrome/common/metrics/metrics_log_base.cc:12: #include "base/sys_byteorder.h"
Is this header needed by this file anymore?

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/bloom...
File components/rappor/bloom_filter.h (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/bloom...
components/rappor/bloom_filter.h:24: BloomFilter(uint32_t bytes_size, uint32_t
hash_function_count,
Nit: 1 param per line; align.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/bloom...
components/rappor/bloom_filter.h:25: uint32_t hash_seed_offset);
Document hash_seed_offset.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:30: int CountBits(const ByteVector& v);
Use a more descriptive name than |v|.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:33: // likelihood of bits being true.
I'd expand the comment to mention that this class gets its random entropy from
crypto::RandBytes().

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:40: // Generates a random byte vector
where each bit is independently set with
I'd reword this a bit to make it clearer that the |probability| specifies the
likelyhood of each bit being set to 1.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:41: // the given probability.
Nit: enclose |probability| in |'s.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:62: // secret value to seed the
pseudo-random number generator.
I'd mention that secret is simply treated as an array of bytes (i.e. not human
readable).

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
components/rappor/log_uploader.cc:27: const size_t kUploadLogAvoidRetransmitSize
= 50000;
I would remove this logic to simplify things. Rappor logs shouldn't be that size
any time in the near future.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
components/rappor/log_uploader.cc:64: has_callback_pending_ = true;
DCHECK() that this is false before setting it?

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
components/rappor/log_uploader.cc:82: scoped_ptr<net::URLFetcher>
s(current_fetch_.Pass());
Use a more meaningful name than |s|.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
components/rappor/log_uploader.cc:84: int response_code =
source->GetResponseCode();
I would add some UMA histograms, at least the response code of the server. (So
we can verify that things are working correctly via UMA.)

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
components/rappor/log_uploader.cc:93: } else if (queued_logs_.size() >
kMaxQueuedLogs) {
Shouldn't this also be behind !upload_succeeded? (I know it doesn't make a
difference in the logic below right now, but it is confusing to set discard_log
in this case.)

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
components/rappor/log_uploader.cc:130: if (upload_interval_ > max_interval ||
upload_interval_.InSeconds() < 0)
Wait, why do you set it to the max interval if it's less than a second? Is this
trying to detect integer overflow or something? If so, it should have a comment.

Maybe this class could use some unit tests for these sorts of edge cases. (You
can probably simulate state without actually having it do any requests..)

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
components/rappor/log_uploader.h:33: // Adds an entry to the queue of logs to be
uploaded to the server.
Mention that |log| is the serialized proto bytes (rather than human readable
text).

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/proto...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:6: // Privacy-Preserving Ordinal
Responses) system.
Maybe link to the design doc on Chromium site here?

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
File components/rappor/rappor_metric.cc (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_metric.cc:28: const ByteVector real_bits(bytes());
I don't think this is necessary. You're making a copy of |bytes()| but keeping
it constant.

Just replace real_bits with bytes() throughout below.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_metric.cc:31: // client's secret key + real data as a
seed.
I'd explicitly mention that it's intended that the result is not human readable
but actually just raw bytes - even if some components of it (e.g. metric_name_)
are.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_metric.cc:33: std::string(real_bits.begin(),
real_bits.end());
Are we really sure that seeding the HMAC with the real data doesn't leak
information out? (Maybe a question for Ulfar).

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_metric.cc:55: ByteVectorMerge(*redacted_bits,
zero_coins, &one_coins);
Nit: Return directly.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.cc:80: bool metrics_logged =
LogMetrics(&reports);
Nit: Can just inline this into the if and remove the temp var.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.cc:121: secret_ = base::RandBytesAsString(128);
Can you make this a constant at the top of the file?

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.cc:128: ByteVector bytes =
metric.GetReport(secret_);
DCHECK(!secret_.empty()); before this

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.cc:129: std::string byte_string(bytes.begin(),
bytes.end());
Nit: Can inline this into the call below.

At which point, I suggest just removing this function and inlining the whole
thing into the function below, since it's just a couple of lines.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.cc:168: DCHECK_GE(cohort_, 0);
I would make a helper function IsInitialized() which wraps the cohort_ check and
have all these places do DCHECK(IsInitialized());

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
File components/rappor/rappor_service.h (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.h:42: static void
RegisterPrefs(PrefRegistrySimple* registry);
Move this below, so the static method is not between two non-static ones.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.h:53: bool LogMetrics(RapporReports* reports);
I would call this ExportMetrics() or something similar. "Log" is kind of
ambiguous and I'd first think of the RecordSample() call when I hear "log a
rappor metric".

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.h:62: int32_t cohort_;
Add a newline after this.

Though, I'd keep this private and add accessors for testing, if that's the
reason it's in this section now.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
File components/rappor/rappor_service_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service_unittest.cc:21: void SetCohort(uint32_t cohort)
{
I'd just name this SetCohortForTesting() and put it into the RapporService
class.

https://codereview.chromium.org/49753002/diff/2200001/components/variations/m...
File components/variations/metrics_util.cc (right):

https://codereview.chromium.org/49753002/diff/2200001/components/variations/m...
components/variations/metrics_util.cc:52: uint64 hash = HashToUInt64(hash_str);
Nit: Just return this directly.

[ulfar, 2014-02-05 18:32:46]

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
File components/rappor/rappor_metric.cc (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_metric.cc:33: std::string(real_bits.begin(),
real_bits.end());
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Are we really sure that seeding the HMAC with the real data doesn't leak
> information out? (Maybe a question for Ulfar).

Yup, HMAC is a PRF (pseudo-random function), according to the strong Crypto
definition thereof.  So, because we're adding the secret as well, we can be sure
that the output value here isn't correlated with the real data.

[Steven Holte, 2014-02-05 22:44:35]

Will look at adding test for LogUploader next.

https://codereview.chromium.org/49753002/diff/2200001/chrome/common/metrics/m...
File chrome/common/metrics/metrics_log_base.cc (left):

https://codereview.chromium.org/49753002/diff/2200001/chrome/common/metrics/m...
chrome/common/metrics/metrics_log_base.cc:12: #include "base/sys_byteorder.h"
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Is this header needed by this file anymore?

Removed, along with a couple other headers that don't seem needed.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/bloom...
File components/rappor/bloom_filter.h (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/bloom...
components/rappor/bloom_filter.h:24: BloomFilter(uint32_t bytes_size, uint32_t
hash_function_count,
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Nit: 1 param per line; align.

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/bloom...
components/rappor/bloom_filter.h:25: uint32_t hash_seed_offset);
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Document hash_seed_offset.

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:30: int CountBits(const ByteVector& v);
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Use a more descriptive name than |v|.

Renamed to |vector|.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:33: // likelihood of bits being true.
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> I'd expand the comment to mention that this class gets its random entropy from
> crypto::RandBytes().

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:40: // Generates a random byte vector
where each bit is independently set with
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> I'd reword this a bit to make it clearer that the |probability| specifies the
> likelyhood of each bit being set to 1.

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:41: // the given probability.
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Nit: enclose |probability| in |'s.

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:62: // secret value to seed the
pseudo-random number generator.
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> I'd mention that secret is simply treated as an array of bytes (i.e. not human
> readable).

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
components/rappor/log_uploader.cc:27: const size_t kUploadLogAvoidRetransmitSize
= 50000;
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> I would remove this logic to simplify things. Rappor logs shouldn't be that
size
> any time in the near future.

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
components/rappor/log_uploader.cc:64: has_callback_pending_ = true;
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> DCHECK() that this is false before setting it?

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
components/rappor/log_uploader.cc:82: scoped_ptr<net::URLFetcher>
s(current_fetch_.Pass());
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Use a more meaningful name than |s|.

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
components/rappor/log_uploader.cc:84: int response_code =
source->GetResponseCode();
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> I would add some UMA histograms, at least the response code of the server. (So
> we can verify that things are working correctly via UMA.)

Added one for response codes, and one for reason for discarding a log.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
components/rappor/log_uploader.cc:93: } else if (queued_logs_.size() >
kMaxQueuedLogs) {
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Shouldn't this also be behind !upload_succeeded? (I know it doesn't make a
> difference in the logic below right now, but it is confusing to set
discard_log
> in this case.)

Restructured this a bit to seperate cases, and added a histogram.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
components/rappor/log_uploader.cc:130: if (upload_interval_ > max_interval ||
upload_interval_.InSeconds() < 0)
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Wait, why do you set it to the max interval if it's less than a second? Is
this
> trying to detect integer overflow or something? If so, it should have a
comment.
> 
> Maybe this class could use some unit tests for these sorts of edge cases. (You
> can probably simulate state without actually having it do any requests..)

It's not less than a seconds, it's less than 0 seconds, but I added a comment.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/log_u...
components/rappor/log_uploader.h:33: // Adds an entry to the queue of logs to be
uploaded to the server.
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Mention that |log| is the serialized proto bytes (rather than human readable
> text).

Changed text to:
  // Adds an entry to the queue of logs to be uploaded to the server.  The
  // uploader makes no assumptions about the format of |log| and simply sends
  // it verbatim to the server.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/proto...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:6: // Privacy-Preserving Ordinal
Responses) system.
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Maybe link to the design doc on Chromium site here?

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
File components/rappor/rappor_metric.cc (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_metric.cc:28: const ByteVector real_bits(bytes());
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> I don't think this is necessary. You're making a copy of |bytes()| but keeping
> it constant.
> 
> Just replace real_bits with bytes() throughout below.

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_metric.cc:31: // client's secret key + real data as a
seed.
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> I'd explicitly mention that it's intended that the result is not human
readable
> but actually just raw bytes - even if some components of it (e.g.
metric_name_)
> are.

Expanded this comment a bit.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_metric.cc:55: ByteVectorMerge(*redacted_bits,
zero_coins, &one_coins);
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Nit: Return directly.

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.cc:80: bool metrics_logged =
LogMetrics(&reports);
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Nit: Can just inline this into the if and remove the temp var.

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.cc:121: secret_ = base::RandBytesAsString(128);
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Can you make this a constant at the top of the file?

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.cc:128: ByteVector bytes =
metric.GetReport(secret_);
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> DCHECK(!secret_.empty()); before this

N/A

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.cc:129: std::string byte_string(bytes.begin(),
bytes.end());
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Nit: Can inline this into the call below.
> 
> At which point, I suggest just removing this function and inlining the whole
> thing into the function below, since it's just a couple of lines.

Done

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.cc:168: DCHECK_GE(cohort_, 0);
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> I would make a helper function IsInitialized() which wraps the cohort_ check
and
> have all these places do DCHECK(IsInitialized());

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
File components/rappor/rappor_service.h (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.h:42: static void
RegisterPrefs(PrefRegistrySimple* registry);
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Move this below, so the static method is not between two non-static ones.

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.h:53: bool LogMetrics(RapporReports* reports);
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> I would call this ExportMetrics() or something similar. "Log" is kind of
> ambiguous and I'd first think of the RecordSample() call when I hear "log a
> rappor metric".

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service.h:62: int32_t cohort_;
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Add a newline after this.
> 
> Though, I'd keep this private and add accessors for testing, if that's the
> reason it's in this section now.

Moved to private.

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
File components/rappor/rappor_service_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/2200001/components/rappor/rappo...
components/rappor/rappor_service_unittest.cc:21: void SetCohort(uint32_t cohort)
{
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> I'd just name this SetCohortForTesting() and put it into the RapporService
> class.

Done.

https://codereview.chromium.org/49753002/diff/2200001/components/variations/m...
File components/variations/metrics_util.cc (right):

https://codereview.chromium.org/49753002/diff/2200001/components/variations/m...
components/variations/metrics_util.cc:52: uint64 hash = HashToUInt64(hash_str);
On 2014/02/05 18:07:01, Alexei Svitkine wrote:
> Nit: Just return this directly.

Done.

[Alexei Svitkine (slow), 2014-02-06 17:47:18]

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/log_u...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/log_u...
components/rappor/log_uploader.cc:114: ResponseCodeToStatus(response_code),
I know this how the MetricsService does it, but I'd actually just log the
response code directly rather than mapping it to this enum. See how
VariationsService does it.

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/log_u...
components/rappor/log_uploader.cc:122:
UMA_HISTOGRAM_ENUMERATION("Rappor.DiscardReason",
Make this a separate function in the anon namespace that takes the enum value.
(To avoid duplication the macro which expands to a lot of code.)

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/log_u...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/log_u...
components/rappor/log_uploader.h:56: void UploadFinished(bool server_is_healthy,
bool more_logs_remaining);
Nit: I'd name this OnUploadFinished().

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/rappo...
File components/rappor/rappor_metric.h (right):

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/rappo...
components/rappor/rappor_metric.h:35: const RapporParameters* parameters() const
{ return &parameters_; }
You should make this return a const ref, rather than a pointer (since it can't
be NULL).

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/rappo...
components/rappor/rappor_service.cc:151: bool RapporService::IsInitialized() {
Nit: Make this const

[Steven Holte, 2014-02-06 23:07:07]

Added LogUploader test.

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/log_u...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/log_u...
components/rappor/log_uploader.cc:114: ResponseCodeToStatus(response_code),
On 2014/02/06 17:47:20, Alexei Svitkine wrote:
> I know this how the MetricsService does it, but I'd actually just log the
> response code directly rather than mapping it to this enum. See how
> VariationsService does it.

Done.

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/log_u...
components/rappor/log_uploader.cc:122:
UMA_HISTOGRAM_ENUMERATION("Rappor.DiscardReason",
On 2014/02/06 17:47:20, Alexei Svitkine wrote:
> Make this a separate function in the anon namespace that takes the enum value.
> (To avoid duplication the macro which expands to a lot of code.)

Rewrote so the macros is only used once.

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/rappo...
File components/rappor/rappor_metric.h (right):

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/rappo...
components/rappor/rappor_metric.h:35: const RapporParameters* parameters() const
{ return &parameters_; }
On 2014/02/06 17:47:20, Alexei Svitkine wrote:
> You should make this return a const ref, rather than a pointer (since it can't
> be NULL).

Done.

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/2310001/components/rappor/rappo...
components/rappor/rappor_service.cc:151: bool RapporService::IsInitialized() {
On 2014/02/06 17:47:20, Alexei Svitkine wrote:
> Nit: Make this const

Done.

[Steven Holte, 2014-02-06 23:50:31]

Replaced MurmurHash3 with CityHash, which has better support in Dremel.

[Alexei Svitkine (slow), 2014-02-07 18:24:20]

LGTM % some final nits.

Also, please add an OWNERS file for this (see my reply on another thread).

https://codereview.chromium.org/49753002/diff/2310001/tools/metrics/histogram...
File tools/metrics/histograms/histograms.xml (right):

https://codereview.chromium.org/49753002/diff/2310001/tools/metrics/histogram...
tools/metrics/histograms/histograms.xml:15336: +    For each upload to the
Rappor server, log the response revieved from the
Nit: received

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/log_u...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/log_u...
components/rappor/log_uploader.cc:11: using base::TimeDelta;
Nit: Either remove this, or remove the namespace prefix from code below that
does base::TimeDelta.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/log_u...
File components/rappor/log_uploader_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:130: while(next > current) {
Nit: Space after while

[ulfar, 2014-02-07 19:00:37]

Comments on
https://sites.google.com/a/chromium.org/dev/developers/design-documents/rappor

On that feature description page, I think the mathematical notation hurts,
and doesn't help.  To be precise it would have to be significantly enhanced
(e.g., defining what h(q,k) means), and that would just make it longer and
more opaque. We have a precise definition in the other docs that will be
published.  So, please replace with textual descriptions.

Also, Bloom filters are usually described in terms of multiple hash
functions.  You can just say that the way those hash functions are
implemented is by concatenating the an integer and the value to be hashed
and passing to a standard hash function.

Finally, that description needs to discuss cohorts (i.e., that the hash
functions depend on a cohort number).  Those can be described by setting
the hash-function-instantiation integer to be CohortCount*HashFunctionIndex
+ CohortNumber, for a client's cohort number.


On Fri, Feb 7, 2014 at 10:24 AM, <asvitkine@chromium.org> wrote:

> LGTM % some final nits.
>
> Also, please add an OWNERS file for this (see my reply on another thread).
>
>
> https://codereview.chromium.org/49753002/diff/2310001/
> tools/metrics/histograms/histograms.xml
> File tools/metrics/histograms/histograms.xml (right):
>
> https://codereview.chromium.org/49753002/diff/2310001/
> tools/metrics/histograms/histograms.xml#newcode15336
> tools/metrics/histograms/histograms.xml:15336: +    For each upload to
> the Rappor server, log the response revieved from the
> Nit: received
>
> https://codereview.chromium.org/49753002/diff/2430001/
> components/rappor/log_uploader.cc
> File components/rappor/log_uploader.cc (right):
>
> https://codereview.chromium.org/49753002/diff/2430001/
> components/rappor/log_uploader.cc#newcode11
> components/rappor/log_uploader.cc:11: using base::TimeDelta;
> Nit: Either remove this, or remove the namespace prefix from code below
> that does base::TimeDelta.
>
> https://codereview.chromium.org/49753002/diff/2430001/
> components/rappor/log_uploader_unittest.cc
> File components/rappor/log_uploader_unittest.cc (right):
>
> https://codereview.chromium.org/49753002/diff/2430001/
> components/rappor/log_uploader_unittest.cc#newcode130
> components/rappor/log_uploader_unittest.cc:130: while(next > current) {
> Nit: Space after while
>
> https://codereview.chromium.org/49753002/
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[ulfar, 2014-02-07 19:05:57]

Also, the parameter configuration described in that doc is out of date.


On Fri, Feb 7, 2014 at 11:00 AM, Ulfar Erlingsson <ulfar@google.com> wrote:

> Comments on
>
https://sites.google.com/a/chromium.org/dev/developers/design-documents/rappor
>
> On that feature description page, I think the mathematical notation hurts,
> and doesn't help.  To be precise it would have to be significantly enhanced
> (e.g., defining what h(q,k) means), and that would just make it longer and
> more opaque. We have a precise definition in the other docs that will be
> published.  So, please replace with textual descriptions.
>
> Also, Bloom filters are usually described in terms of multiple hash
> functions.  You can just say that the way those hash functions are
> implemented is by concatenating the an integer and the value to be hashed
> and passing to a standard hash function.
>
> Finally, that description needs to discuss cohorts (i.e., that the hash
> functions depend on a cohort number).  Those can be described by setting
> the hash-function-instantiation integer to be CohortCount*HashFunctionIndex
> + CohortNumber, for a client's cohort number.
>
>
> On Fri, Feb 7, 2014 at 10:24 AM, <asvitkine@chromium.org> wrote:
>
>> LGTM % some final nits.
>>
>> Also, please add an OWNERS file for this (see my reply on another thread).
>>
>>
>> https://codereview.chromium.org/49753002/diff/2310001/
>> tools/metrics/histograms/histograms.xml
>> File tools/metrics/histograms/histograms.xml (right):
>>
>> https://codereview.chromium.org/49753002/diff/2310001/
>> tools/metrics/histograms/histograms.xml#newcode15336
>> tools/metrics/histograms/histograms.xml:15336: +    For each upload to
>> the Rappor server, log the response revieved from the
>> Nit: received
>>
>> https://codereview.chromium.org/49753002/diff/2430001/
>> components/rappor/log_uploader.cc
>> File components/rappor/log_uploader.cc (right):
>>
>> https://codereview.chromium.org/49753002/diff/2430001/
>> components/rappor/log_uploader.cc#newcode11
>> components/rappor/log_uploader.cc:11: using base::TimeDelta;
>> Nit: Either remove this, or remove the namespace prefix from code below
>> that does base::TimeDelta.
>>
>> https://codereview.chromium.org/49753002/diff/2430001/
>> components/rappor/log_uploader_unittest.cc
>> File components/rappor/log_uploader_unittest.cc (right):
>>
>> https://codereview.chromium.org/49753002/diff/2430001/
>> components/rappor/log_uploader_unittest.cc#newcode130
>> components/rappor/log_uploader_unittest.cc:130: while(next > current) {
>> Nit: Space after while
>>
>> https://codereview.chromium.org/49753002/
>>
>
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[ulfar, 2014-02-07 20:19:00]

https://codereview.chromium.org/49753002/diff/2430001/chrome/browser/prefs/pr...
File chrome/browser/prefs/pref_metrics_service.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/chrome/browser/prefs/pr...
chrome/browser/prefs/pref_metrics_service.cc:74: // pages, e.g.
plus.google.com).
Fix comment here.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/bloom...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/bloom...
components/rappor/bloom_filter.cc:27: uint32_t byte_index = (index / 8) %
bytes_.size();
I presume Chromium is full of assumptions about bytes being 8 bits, but it's
still funny to see this constant here.

Edit: Actually, since ByteVector is defined on uint8_t, this should always be
fine.  However, perhaps deserves a comment explaining why it is fine (or a
change in the type name of ByteVector).

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:100: ++hmac_state_;
Actually, we should do something a bit fancier, to be totally correct.
See NIST SP800-90A,
http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf

Or, more casually, see this:
http://crypto.stackexchange.com/questions/9076/using-a-hash-as-a-secure-prng

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/proto...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:17: message RapporReports {
What about version numbers?  We could piggy-back version numbers on cohort
numbers later, but building in support seems good as well.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:31: // used in the bloom filter.
Better comment:

// Which cohort this report belongs to.  The RAPPOR participants are partitioned
into cohorts
// in different ways, to allow better statistics and increased coverage.  In
particular, the cohort
// will serve to choose the hash functions used for Bloom-filter-based reports.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
File components/rappor/rappor_metric.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
components/rappor/rappor_metric.cc:29: // in the seed prevents fake data from
being correlated with real data.
// in the seed avoids correlations between the real and fake data.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
components/rappor/rappor_metric.cc:41: const ByteVector* redacted_bits =
better name would be fake_and_redacted_bits

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
components/rappor/rappor_metric.cc:53: return *ByteVectorMerge(*redacted_bits,
zero_coins, &one_coins);
A better comment would be 

// Create a randomized response report on the fake and redacted data, sending
// the outcome of flipping a zero coin for the zero bits in that data, and of 
// flipping a one coin for the one bits in that data, as the final report.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
File components/rappor/rappor_pref_names.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
components/rappor/rappor_pref_names.cc:11: // for redacting collected data.
// Important: This value should remain secret at the client, and never be
reported on the network, or to the server.

Add comment.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
components/rappor/rappor_service.cc:23: const int kNumCohorts = 8;
We'll may want a slightly higher number here, for the initial experiments. 
Perhaps 32.  Should check with Vasy.

[Steven Holte, 2014-02-07 21:08:15]

Adding OWNERS reviewers:
thakis@ for chrome/browsers
joi@ for components/
darin@ for third_party/smhasher
wtc@ for net/ and crypto/

https://codereview.chromium.org/49753002/diff/2310001/tools/metrics/histogram...
File tools/metrics/histograms/histograms.xml (right):

https://codereview.chromium.org/49753002/diff/2310001/tools/metrics/histogram...
tools/metrics/histograms/histograms.xml:15336: +    For each upload to the
Rappor server, log the response revieved from the
On 2014/02/07 18:24:21, Alexei Svitkine wrote:
> Nit: received

Done.

https://codereview.chromium.org/49753002/diff/2430001/chrome/browser/prefs/pr...
File chrome/browser/prefs/pref_metrics_service.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/chrome/browser/prefs/pr...
chrome/browser/prefs/pref_metrics_service.cc:74: // pages, e.g.
plus.google.com).
On 2014/02/07 20:19:02, ulfar wrote:
> Fix comment here.

Done.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/bloom...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/bloom...
components/rappor/bloom_filter.cc:27: uint32_t byte_index = (index / 8) %
bytes_.size();
On 2014/02/07 20:19:02, ulfar wrote:
> I presume Chromium is full of assumptions about bytes being 8 bits, but it's
> still funny to see this constant here.
> 
> Edit: Actually, since ByteVector is defined on uint8_t, this should always be
> fine.  However, perhaps deserves a comment explaining why it is fine (or a
> change in the type name of ByteVector).

Added Comment

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:100: ++hmac_state_;
On 2014/02/07 20:19:02, ulfar wrote:
> Actually, we should do something a bit fancier, to be totally correct.
> See NIST SP800-90A,
> http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf
> 
> Or, more casually, see this:
> http://crypto.stackexchange.com/questions/9076/using-a-hash-as-a-secure-prng

Will update this next pass.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/log_u...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/log_u...
components/rappor/log_uploader.cc:11: using base::TimeDelta;
On 2014/02/07 18:24:22, Alexei Svitkine wrote:
> Nit: Either remove this, or remove the namespace prefix from code below that
> does base::TimeDelta.

Done.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/log_u...
File components/rappor/log_uploader_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:130: while(next > current) {
On 2014/02/07 18:24:22, Alexei Svitkine wrote:
> Nit: Space after while

Done.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/proto...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:17: message RapporReports {
On 2014/02/07 20:19:02, ulfar wrote:
> What about version numbers?  We could piggy-back version numbers on cohort
> numbers later, but building in support seems good as well.

Done.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:31: // used in the bloom filter.
On 2014/02/07 20:19:02, ulfar wrote:
> Better comment:
> 
> // Which cohort this report belongs to.  The RAPPOR participants are
partitioned
> into cohorts
> // in different ways, to allow better statistics and increased coverage.  In
> particular, the cohort
> // will serve to choose the hash functions used for Bloom-filter-based
reports.

Done.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
File components/rappor/rappor_metric.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
components/rappor/rappor_metric.cc:29: // in the seed prevents fake data from
being correlated with real data.
On 2014/02/07 20:19:02, ulfar wrote:
> // in the seed avoids correlations between the real and fake data.

Done.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
components/rappor/rappor_metric.cc:41: const ByteVector* redacted_bits =
On 2014/02/07 20:19:02, ulfar wrote:
> better name would be fake_and_redacted_bits

Done.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
components/rappor/rappor_metric.cc:53: return *ByteVectorMerge(*redacted_bits,
zero_coins, &one_coins);
On 2014/02/07 20:19:02, ulfar wrote:
> A better comment would be 
> 
> // Create a randomized response report on the fake and redacted data, sending
> // the outcome of flipping a zero coin for the zero bits in that data, and of 
> // flipping a one coin for the one bits in that data, as the final report.

Done.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
File components/rappor/rappor_pref_names.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
components/rappor/rappor_pref_names.cc:11: // for redacting collected data.
On 2014/02/07 20:19:02, ulfar wrote:
> // Important: This value should remain secret at the client, and never be
> reported on the network, or to the server.
> 
> Add comment.

Done.

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/2430001/components/rappor/rappo...
components/rappor/rappor_service.cc:23: const int kNumCohorts = 8;
On 2014/02/07 20:19:02, ulfar wrote:
> We'll may want a slightly higher number here, for the initial experiments. 
> Perhaps 32.  Should check with Vasy.

Changed to 32.

[wtc, 2014-02-07 23:49:39]

On 2014/02/07 21:08:15, Steven Holte wrote:
> Adding OWNERS reviewers:
> ...
> wtc@ for net/ and crypto/

Patch set 34 doesn't contain any files in net/ and crypto/.

[Steven Holte, 2014-02-08 01:32:13]

Now using the HMAC_DRBG algorithm for pseudo-random bits.

@wtc (also @darin), I didn't modify net/crypto but I added them in
components/rappor/DEPS, so presubmit wants your LGTM.
Missing LGTM from OWNERS of dependencies added to DEPS:
    '+crypto',
    '+net',
    '+third_party/smhasher',

[ulfar, 2014-02-08 01:51:31]

https://codereview.chromium.org/49753002/diff/1790039/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/1790039/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:95: if (!keygen_hmac.Sign(initial_value +
secret, key.data(), key.size()))
Isn't the zero byte from V || 0x00 || provided_data missing here, between
initial_value and secret?

https://codereview.chromium.org/49753002/diff/1790039/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:107: if (requested_bytes_ %
hmac_.DigestLength()) {
make a variable for this.  For example,
size_t requested_byte_in_digest = requested_bytes_ % hmac_.DigestLength();
if (requested_byte_in_digest == 0) {

https://codereview.chromium.org/49753002/diff/1790039/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:117: DCHECK_LT(requested_bytes_, 1U <<
16);
Did you mean to conservatively say 2^16 here?  If so, need to explain that in
comment.

[Steven Holte, 2014-02-08 02:19:44]

https://codereview.chromium.org/49753002/diff/1790039/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/1790039/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:95: if (!keygen_hmac.Sign(initial_value +
secret, key.data(), key.size()))
On 2014/02/08 01:51:33, ulfar wrote:
> Isn't the zero byte from V || 0x00 || provided_data missing here, between
> initial_value and secret?

Done.

https://codereview.chromium.org/49753002/diff/1790039/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:107: if (requested_bytes_ %
hmac_.DigestLength()) {
On 2014/02/08 01:51:33, ulfar wrote:
> make a variable for this.  For example,
> size_t requested_byte_in_digest = requested_bytes_ % hmac_.DigestLength();
> if (requested_byte_in_digest == 0) {

Done.

https://codereview.chromium.org/49753002/diff/1790039/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:117: DCHECK_LT(requested_bytes_, 1U <<
16);
On 2014/02/08 01:51:33, ulfar wrote:
> Did you mean to conservatively say 2^16 here?  If so, need to explain that in
> comment.

No, it's bits vs. bytes.  Expanded comment.

[ulfar, 2014-02-08 03:20:22]

lgtm

LGTM on the code.

Comments on the text at
http://www.chromium.org/developers/design-documents/rappor still outstanding. 
Use descriptions and match with how the code works now.

[ulfar, 2014-02-08 03:20:34]

LGTM on the code.

Comments on the text at
http://www.chromium.org/developers/design-documents/rappor still outstanding. 
Use descriptions and match with how the code works now.

[Jói, 2014-02-10 12:57:43]

//components/* and //components/rappor/OWNERS LGTM. Please rely on an OWNER from
the latter-mentioned file for the main part of the review.

Cheers,
Jói

https://codereview.chromium.org/49753002/diff/2720001/components/OWNERS
File components/OWNERS (right):

https://codereview.chromium.org/49753002/diff/2720001/components/OWNERS#newco...
components/OWNERS:48: per-file rappor.gypi=asvitkine@chromium.org
Suggest rappor* rather than rappor.gypi in case you e.g. add other
rappor-related .gypi files, .grd files, etc. at this level.

[jwd, 2014-02-10 14:34:52]

lgtm

[Steven Holte, 2014-02-10 18:36:31]

Ulfar, I've updated the design doc at
http://www.chromium.org/developers/design-documents/rappor

https://codereview.chromium.org/49753002/diff/2720001/components/OWNERS
File components/OWNERS (right):

https://codereview.chromium.org/49753002/diff/2720001/components/OWNERS#newco...
components/OWNERS:48: per-file rappor.gypi=asvitkine@chromium.org
On 2014/02/10 12:57:44, Jói wrote:
> Suggest rappor* rather than rappor.gypi in case you e.g. add other
> rappor-related .gypi files, .grd files, etc. at this level.

Done.

[wtc, 2014-02-10 20:12:51]

Review comments on patch set 38:

I reviewed the use of crypto/ and net/ in components/rappor.

1. Only public classes in net/ are used (URLFetcher and related classes). This
looks good.

2. Only public classes/functions in crypto/ are used (crypto::HMAC and
crypto::RandBytes). However, I requested some comments to clarify the use of
HMAC_DRBG.

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:33: int CountBits(const ByteVector&
vector) {

This operation is called "population count" or "popcount" in the literature.
There are clever tricks in the books "Hacker's Delight" and vol. 4 of the Art of
Computer Programming if you are interested in this.

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:50: uint8_t
ByteVectorGenerator::RandByte() {

Requesting only one byte at a time from the PRNG can be very inefficient. I
suggest you make this function request a specified number of bytes from the
PRNG, for example:

  void ByteVectorGenerator::RandBytes(uint8_t*out, size_t len);

Then ByteVectorGenerator::GetRandomByteVector() can simply do:

ByteVector ByteVectorGenerator::GetRandomByteVector() {
  ByteVector bytes(byte_count_);
  RandBytes(&bytes[0], byte_count_);
  return bytes;
}

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:73: NOTREACHED();

This error probably needs to be detected in release builds also.

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:58: // deterministicly random bits.  The
implementation is equivalent to the

Nit: deterministicly => deterministically

Also, you can just say "pseudo-random" instead of "deterministically random".

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:62: // Ref:
http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf

IMPORTANT: sometimes we need to audit the Chromium source tree for correct use
of crypto. So the mention of "HMAC_DRBG" in this class is likely to cause this
class to be scrutinized in a crypto audit.

It would be nice if you could document why you need your own pseudo-random
number generator (PRNG). Here I asked two possible questions. But there are
other ways to document this.

1. Is HmacByteVectorGenerator used for cryptographic purposes? In other words,
do you just need the bits to be uniformly distributed, or do you also need them
to be unpredictable? The latter is a must for crypto applications.

2. If HmacByteVectorGenerator is used for non-crypto purposes, why do you use a
cryptographically secure pseudo-random number generator? The answer could be
"although a cryptographic PRNG exceeds my requirements, it saves me the time to
research a good non-cryptographic PRNG." But if that was the case, why couldn't
you just use crypto::RandomBytes?

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/log_u...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/log_u...
components/rappor/log_uploader.h:14: #include "net/url_request/url_fetcher.h"

Nit: a forward declaration of class URLFetcher may suffice, but it may not be
worth the trouble.

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/log_u...
File components/rappor/log_uploader_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:9: #include
"net/url_request/url_request_test_util.h"

I'm actually not sure if these two net/ headers are intended for external use.
I'll allow them because they are for testing only.

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/rappo...
File components/rappor/rappor_parameters.h (right):

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/rappo...
components/rappor/rappor_parameters.h:13: PROBABILITY_75,    // 75%

This should be documented. Is this the probability of getting a 1 bit?

[Steven Holte, 2014-02-10 22:50:53]

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:33: int CountBits(const ByteVector&
vector) {
On 2014/02/10 20:12:54, wtc wrote:
> 
> This operation is called "population count" or "popcount" in the literature.
> There are clever tricks in the books "Hacker's Delight" and vol. 4 of the Art
of
> Computer Programming if you are interested in this.

Since I'm only using this function for testing, I'm not really worried about
optimizing it.

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:50: uint8_t
ByteVectorGenerator::RandByte() {
On 2014/02/10 20:12:54, wtc wrote:
> 
> Requesting only one byte at a time from the PRNG can be very inefficient. I
> suggest you make this function request a specified number of bytes from the
> PRNG, for example:
> 
>   void ByteVectorGenerator::RandBytes(uint8_t*out, size_t len);
> 
> Then ByteVectorGenerator::GetRandomByteVector() can simply do:
> 
> ByteVector ByteVectorGenerator::GetRandomByteVector() {
>   ByteVector bytes(byte_count_);
>   RandBytes(&bytes[0], byte_count_);
>   return bytes;
> }

Done.

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:73: NOTREACHED();
On 2014/02/10 20:12:54, wtc wrote:
> 
> This error probably needs to be detected in release builds also.

This should be prevented by compiler errors due to missing enum values/invalid
conversions.

If we remove the PROBABILITY_50 case:
../../components/rappor/byte_vector_utils.cc:59:10: error: enumeration value
‘PROBABILITY_50’ not handled in switch [-Werror=switch]

If we tried to pass a number directly like GetWeightedRandomByteVector(3)
../../components/rappor/byte_vector_utils_unittest.cc:34:65: error: invalid
conversion from ‘int’ to ‘rappor::Probability’ [-fpermissive]

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:58: // deterministicly random bits.  The
implementation is equivalent to the
On 2014/02/10 20:12:54, wtc wrote:
> 
> Nit: deterministicly => deterministically
> 
> Also, you can just say "pseudo-random" instead of "deterministically random".

Done.

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:62: // Ref:
http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf
On 2014/02/10 20:12:54, wtc wrote:
> 
> IMPORTANT: sometimes we need to audit the Chromium source tree for correct use
> of crypto. So the mention of "HMAC_DRBG" in this class is likely to cause this
> class to be scrutinized in a crypto audit.
> 
> It would be nice if you could document why you need your own pseudo-random
> number generator (PRNG). Here I asked two possible questions. But there are
> other ways to document this.
> 
> 1. Is HmacByteVectorGenerator used for cryptographic purposes? In other words,
> do you just need the bits to be uniformly distributed, or do you also need
them
> to be unpredictable? The latter is a must for crypto applications.
> 
> 2. If HmacByteVectorGenerator is used for non-crypto purposes, why do you use
a
> cryptographically secure pseudo-random number generator? The answer could be
> "although a cryptographic PRNG exceeds my requirements, it saves me the time
to
> research a good non-cryptographic PRNG." But if that was the case, why
couldn't
> you just use crypto::RandomBytes?

Added a longer comment explaining the rationale of using HMAC_DRBG instead of
crypto::RandBytes

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/log_u...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/log_u...
components/rappor/log_uploader.h:14: #include "net/url_request/url_fetcher.h"
On 2014/02/10 20:12:54, wtc wrote:
> 
> Nit: a forward declaration of class URLFetcher may suffice, but it may not be
> worth the trouble.

Done.

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/log_u...
File components/rappor/log_uploader_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:9: #include
"net/url_request/url_request_test_util.h"
On 2014/02/10 20:12:54, wtc wrote:
> 
> I'm actually not sure if these two net/ headers are intended for external use.
> I'll allow them because they are for testing only.

They appear to be used in many uses of them outside of net/ (in _unittest.cc
files) including other modules in components/

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/rappo...
File components/rappor/rappor_parameters.h (right):

https://codereview.chromium.org/49753002/diff/2850001/components/rappor/rappo...
components/rappor/rappor_parameters.h:13: PROBABILITY_75,    // 75%
On 2014/02/10 20:12:54, wtc wrote:
> 
> This should be documented. Is this the probability of getting a 1 bit?

No, this is just a number (.75), so it's meaning is documented elsewhere i.e.

// The probability that a bit will be redacted with fake data.
Probability fake_prob;

[ulfar, 2014-02-10 22:59:06]

https://codereview.chromium.org/49753002/diff/2940001/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/2940001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:63: // have been produced by some value of
secret.
I would much rather say the following, which I think is clearer and more
correct:

// Conservatively, we're choosing to use HMAC_DRBG here, as it is (on of the)
best studied 
// and standardized way of generating deterministic, unpredictable sequences
based
// on a secret seed.

[Steven Holte, 2014-02-10 23:17:27]

https://codereview.chromium.org/49753002/diff/2940001/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/2940001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:63: // have been produced by some value of
secret.
On 2014/02/10 22:59:07, ulfar wrote:
> I would much rather say the following, which I think is clearer and more
> correct:
> 
> // Conservatively, we're choosing to use HMAC_DRBG here, as it is (on of the)
> best studied 
> // and standardized way of generating deterministic, unpredictable sequences
> based
> // on a secret seed.

Done.

[wtc, 2014-02-10 23:52:49]

Patch set 40 LGTM.

https://codereview.chromium.org/49753002/diff/3000001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3000001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:52: crypto::RandBytes(bytes.data(),
bytes.size());

IMPORTANT: the data() method of std::vector was added in C++11, so it may not be
supported by all compilers. If necessary, use &bytes[0] as a workaround.

https://codereview.chromium.org/49753002/diff/3000001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:104:
if(!hmac_.Sign(std::string(value_.begin(), value_.end()),

Nit: add a space between "if" and "(".

https://codereview.chromium.org/49753002/diff/3000001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:121: return bytes;

You may want to use something like this and get rid of RandByte:

ByteVector HmacByteVectorGenerator::GetRandomByteVector() {
  ByteVector bytes(byte_count_);
  uint8_t* data = bytes.data();
  size_t bytes_to_go = byte_count_;
  while (bytes_to_go) {
    size_t requested_byte_in_digest = requested_bytes_ % hmac_.DigestLength();
    if (requested_byte_in_digest == 0) {
      // Do step 4.1 of the HMAC_DRBG Generate Process for more bits.
      // V = HMAC(Key, V)
      if (!hmac_.Sign(std::string(value_.begin(), value_.end()),
          value_.data(), value_.size()))
        NOTREACHED();
    }
    size_t n = std::min(bytes_to_go,
                        hmac_.DigestLength() - requested_byte_in_digest);
    memcpy(data, &value_[requested_byte_in_digest], n);
    data += n;
    bytes_to_go -= n;
    requested_bytes_ += n;
    // Check max_number_of_bits_per_request from 10.1 Table 2
    // max_number_of_bits_per_request == 2^19 bits == 2^16 bytes
    DCHECK_LT(requested_bytes_, 1U << 16);
  }
  return bytes;
}

[wtc, 2014-02-10 23:56:15]

https://codereview.chromium.org/49753002/diff/3000001/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/3000001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:40: // Generates a random byte vector
where the bits are an independent random

Nit: remove "an".

[Steven Holte, 2014-02-11 00:36:11]

https://codereview.chromium.org/49753002/diff/3000001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3000001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:52: crypto::RandBytes(bytes.data(),
bytes.size());
On 2014/02/10 23:52:51, wtc wrote:
> 
> IMPORTANT: the data() method of std::vector was added in C++11, so it may not
be
> supported by all compilers. If necessary, use &bytes[0] as a workaround.

Done.

https://codereview.chromium.org/49753002/diff/3000001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:104:
if(!hmac_.Sign(std::string(value_.begin(), value_.end()),
On 2014/02/10 23:52:51, wtc wrote:
> 
> Nit: add a space between "if" and "(".

Done.

https://codereview.chromium.org/49753002/diff/3000001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:121: return bytes;
On 2014/02/10 23:52:51, wtc wrote:
> 
> You may want to use something like this and get rid of RandByte:
> 
> ByteVector HmacByteVectorGenerator::GetRandomByteVector() {
>   ByteVector bytes(byte_count_);
>   uint8_t* data = bytes.data();
>   size_t bytes_to_go = byte_count_;
>   while (bytes_to_go) {
>     size_t requested_byte_in_digest = requested_bytes_ % hmac_.DigestLength();
>     if (requested_byte_in_digest == 0) {
>       // Do step 4.1 of the HMAC_DRBG Generate Process for more bits.
>       // V = HMAC(Key, V)
>       if (!hmac_.Sign(std::string(value_.begin(), value_.end()),
>           value_.data(), value_.size()))
>         NOTREACHED();
>     }
>     size_t n = std::min(bytes_to_go,
>                         hmac_.DigestLength() - requested_byte_in_digest);
>     memcpy(data, &value_[requested_byte_in_digest], n);
>     data += n;
>     bytes_to_go -= n;
>     requested_bytes_ += n;
>     // Check max_number_of_bits_per_request from 10.1 Table 2
>     // max_number_of_bits_per_request == 2^19 bits == 2^16 bytes
>     DCHECK_LT(requested_bytes_, 1U << 16);
>   }
>   return bytes;
> }

Done.

https://codereview.chromium.org/49753002/diff/3000001/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/3000001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:40: // Generates a random byte vector
where the bits are an independent random
On 2014/02/10 23:56:16, wtc wrote:
> 
> Nit: remove "an".

Done.

[Alexei Svitkine (slow), 2014-02-11 16:59:05]

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:74: requested_bytes_(0) {
Why does this need to be a class member, rather than a local var in the function
where it's used?

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:88: &key[0], key.size()))
Nit: Add {}'s. Align params.

Alternatively, make a local var for "initial_value + char(0x00) + secret" so
that this if fits into one line.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:103: while (bytes_to_go) {
Nit: bytes_to_go > 0

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:104: size_t requested_byte_in_digest =
requested_bytes_ % hmac_.DigestLength();
The digest length is constant between iterations, right? If so, I'd cache it in
a variable to avoid calling this every iteration. (You also use it below).

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:106: // Do step 4.1 of the HMAC_DRBG
Generate Process for more bits.
This function is now pretty complex. Given it's implementing some spec (as far
as I can tell from this comment), are you sure we don't have a library
implementation somewhere that can be re-used?

Or perhaps this should be moved to the HMAC class?

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:108: if
(!hmac_.Sign(std::string(value_.begin(), value_.end()),
Can the param be a StringPiece rather than a std::string()? That way, there
won't be any copying.. Or is it because you're using |value_| as the output too,
so the copy needs to happen - if so document this with a comment.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:109: &value_[0], value_.size()))
Nit: Same comment as above about {}'s and aligning params.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/log_u...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/log_u...
components/rappor/log_uploader.h:19: class URLFetcher;
No need for indent here.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/proto...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:17: message RapporReports {
Add a comment, including a "Next tag: 4" line.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:27: message Report {
Probably a short comment explaining what each "report" is - i.e. that it's for
one metric?

[edknapp, 2014-02-11 17:27:58]

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:69:
HmacByteVectorGenerator::HmacByteVectorGenerator(size_t byte_count,
Please pass in entropy_input, nonce, and personalization_string separately.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:82: // HMAC_DRBG Update Process inlined
Please move this to a private function:
Update(provided_data)

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:56: // be generated from one request. 
Therefore, this one instance of this class
Something like:
This class only implements a single request from HMAC_DRBG and streams up to
2^19 bits from that request.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/rappo...
components/rappor/rappor_service.cc:29: const int kRapporSecretSize = 128;
kRapporSecretSize = HmacByteVectorGenerator::kKeySize
and create that constant in HmacByteVectorGenerator. 

Possible security levels for the SHA256 instantiation are 80, 112, 128, 192,
256. I'd recommend 128 bits.

[wtc, 2014-02-11 20:55:27]

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:74: requested_bytes_(0) {

On 2014/02/11 16:59:06, Alexei Svitkine wrote:
> Why does this need to be a class member, rather than a local var in the
function
> where it's used?

requested_bytes_ is the total number of bytes generated by this object. Although
it is only used in GetRandomByteVector, it needs to be a class member to count
the number of bytes across multiple calls.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:106: // Do step 4.1 of the HMAC_DRBG
Generate Process for more bits.

On 2014/02/11 16:59:06, Alexei Svitkine wrote:
> This function is now pretty complex. Given it's implementing some spec (as far
> as I can tell from this comment), are you sure we don't have a library
> implementation somewhere that can be re-used?

I don't know of such a class in the Chromium source tree. The random number
generators in Chromium are all singletons and don't allow the seed to be
specified.

> Or perhaps this should be moved to the HMAC class?

This thought also crossed my mind, but I'm afraid that people (including me)
will be tempted to ask for a more general interface to this class, and that will
make this code review drag on longer.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
File components/rappor/byte_vector_utils_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils_unittest.cc:47: EXPECT_EQ(random_75[0],
0x7f);

1. Nit: it seems that we should also test the generated byte at index 1.

2. Just FYI: you can download test vectors for HMAC_DRBG from the NIST website:
http://csrc.nist.gov/groups/STM/cavp/#05

[Steven Holte, 2014-02-11 22:08:12]

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:69:
HmacByteVectorGenerator::HmacByteVectorGenerator(size_t byte_count,
On 2014/02/11 17:28:00, edknapp wrote:
> Please pass in entropy_input, nonce, and personalization_string separately. 

Separated entropy_input, personalization_string, and referenced the part of
section 8.6.7 that allows acquiring entropy_input and nonce at the same time
from the same source.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:74: requested_bytes_(0) {
On 2014/02/11 16:59:06, Alexei Svitkine wrote:
> Why does this need to be a class member, rather than a local var in the
function
> where it's used?

This is the number of bytes streamed from the HMAC_DRBG Generate process, so
it's cumulative over multiple calls to GetRandomByteVector.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:82: // HMAC_DRBG Update Process inlined
On 2014/02/11 17:28:00, edknapp wrote:
> Please move this to a private function:
> Update(provided_data)

Done.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:88: &key[0], key.size()))
On 2014/02/11 16:59:06, Alexei Svitkine wrote:
> Nit: Add {}'s. Align params.
> 
> Alternatively, make a local var for "initial_value + char(0x00) + secret" so
> that this if fits into one line.

Done.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:103: while (bytes_to_go) {
On 2014/02/11 16:59:06, Alexei Svitkine wrote:
> Nit: bytes_to_go > 0

Done.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:104: size_t requested_byte_in_digest =
requested_bytes_ % hmac_.DigestLength();
On 2014/02/11 16:59:06, Alexei Svitkine wrote:
> The digest length is constant between iterations, right? If so, I'd cache it
in
> a variable to avoid calling this every iteration. (You also use it below).

Done.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:108: if
(!hmac_.Sign(std::string(value_.begin(), value_.end()),
On 2014/02/11 16:59:06, Alexei Svitkine wrote:
> Can the param be a StringPiece rather than a std::string()? That way, there
> won't be any copying.. Or is it because you're using |value_| as the output
too,
> so the copy needs to happen - if so document this with a comment.

value_ is a ByteVector, so StringPiece doesn't like it's iterators.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:109: &value_[0], value_.size()))
On 2014/02/11 16:59:06, Alexei Svitkine wrote:
> Nit: Same comment as above about {}'s and aligning params.

Done.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:56: // be generated from one request. 
Therefore, this one instance of this class
On 2014/02/11 17:28:00, edknapp wrote:
> Something like:
> This class only implements a single request from HMAC_DRBG and streams up to
> 2^19 bits from that request. 

Done.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
File components/rappor/byte_vector_utils_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/byte_...
components/rappor/byte_vector_utils_unittest.cc:47: EXPECT_EQ(random_75[0],
0x7f);
On 2014/02/11 20:55:29, wtc wrote:
> 
> 1. Nit: it seems that we should also test the generated byte at index 1.
> 
> 2. Just FYI: you can download test vectors for HMAC_DRBG from the NIST
website:
> http://csrc.nist.gov/groups/STM/cavp/#05

Changed test to generate 1 byte.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/log_u...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/log_u...
components/rappor/log_uploader.h:19: class URLFetcher;
On 2014/02/11 16:59:06, Alexei Svitkine wrote:
> No need for indent here.

Done.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/proto...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:17: message RapporReports {
On 2014/02/11 16:59:06, Alexei Svitkine wrote:
> Add a comment, including a "Next tag: 4" line.

Done.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:27: message Report {
On 2014/02/11 16:59:06, Alexei Svitkine wrote:
> Probably a short comment explaining what each "report" is - i.e. that it's for
> one metric?

Done.

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/rappo...
components/rappor/rappor_service.cc:29: const int kRapporSecretSize = 128;
On 2014/02/11 17:28:00, edknapp wrote:
> kRapporSecretSize = HmacByteVectorGenerator::kKeySize
> and create that constant in HmacByteVectorGenerator. 
> 
> Possible security levels for the SHA256 instantiation are 80, 112, 128, 192,
> 256. I'd recommend 128 bits. 

Done.

[Steven Holte, 2014-02-11 22:33:40]

Adding sky@ to replace thakis@ (Away) for chrome/browser/ review.

[edknapp, 2014-02-12 00:27:15]

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/rappo...
components/rappor/rappor_service.cc:29: const int kRapporSecretSize = 128;
On 2014/02/11 22:08:14, Steven Holte wrote:
> On 2014/02/11 17:28:00, edknapp wrote:
> > kRapporSecretSize = HmacByteVectorGenerator::kKeySize
> > and create that constant in HmacByteVectorGenerator. 
> > 
> > Possible security levels for the SHA256 instantiation are 80, 112, 128, 192,
> > 256. I'd recommend 128 bits. 
> 
> Done.

The new constant was created but I still see 128 here.

https://codereview.chromium.org/49753002/diff/3470001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3470001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:82: std::string
seed_material(entropy_input + personalization_string);
DCHECK(entropy_input.length() == kEntropyInputSize)

https://codereview.chromium.org/49753002/diff/3470001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:117: }
Steps 3-6:
if (provided_data.length() > 0) {
  K = HMAC(...);
  V = HMAC(...);
}

[sky, 2014-02-12 01:06:49]

Did this go through eng review?

https://codereview.chromium.org/49753002/diff/3470001/chrome/browser/chrome_b...
File chrome/browser/chrome_browser_main.cc (right):

https://codereview.chromium.org/49753002/diff/3470001/chrome/browser/chrome_b...
chrome/browser/chrome_browser_main.cc:1084:
browser_process_->rappor_service()->Start(
How will running this here effect startup?

https://codereview.chromium.org/49753002/diff/3470001/chrome/browser/prefs/pr...
File chrome/browser/prefs/pref_metrics_service.cc (right):

https://codereview.chromium.org/49753002/diff/3470001/chrome/browser/prefs/pr...
chrome/browser/prefs/pref_metrics_service.cc:1: // Copyright 2014 The Chromium
Authors. All rights reserved.
No need to change this.

https://codereview.chromium.org/49753002/diff/3470001/chrome/browser/prefs/pr...
chrome/browser/prefs/pref_metrics_service.cc:83:
g_browser_process->rappor_service()->RecordSample(
Don't you need to NULL check rappor_service()? Tests may have a different
BrowserProcess that returns NULL, right?

[Steven Holte, 2014-02-12 04:02:53]

sky@ - Yes, this went through eng review.  Here's the link to the eng review
proposal that we submitted:
https://docs.google.com/a/google.com/document/d/1mC88FoZGIIwNrCVqzZruTiGoYBDl...

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/3100001/components/rappor/rappo...
components/rappor/rappor_service.cc:29: const int kRapporSecretSize = 128;
On 2014/02/12 00:27:17, edknapp wrote:
> On 2014/02/11 22:08:14, Steven Holte wrote:
> > On 2014/02/11 17:28:00, edknapp wrote:
> > > kRapporSecretSize = HmacByteVectorGenerator::kKeySize
> > > and create that constant in HmacByteVectorGenerator. 
> > > 
> > > Possible security levels for the SHA256 instantiation are 80, 112, 128,
192,
> > > 256. I'd recommend 128 bits. 
> > 
> > Done.
> 
> The new constant was created but I still see 128 here. 

Oops.  Removed it.

https://codereview.chromium.org/49753002/diff/3470001/chrome/browser/chrome_b...
File chrome/browser/chrome_browser_main.cc (right):

https://codereview.chromium.org/49753002/diff/3470001/chrome/browser/chrome_b...
chrome/browser/chrome_browser_main.cc:1084:
browser_process_->rappor_service()->Start(
On 2014/02/12 01:06:50, sky wrote:
> How will running this here effect startup?

If you mean in terms of start up time, it should be negligible.  Start() will
just check a field trial, read two preferences, start a timer and return.  After
15 seconds pass it will create and upload a report.

https://codereview.chromium.org/49753002/diff/3470001/chrome/browser/prefs/pr...
File chrome/browser/prefs/pref_metrics_service.cc (right):

https://codereview.chromium.org/49753002/diff/3470001/chrome/browser/prefs/pr...
chrome/browser/prefs/pref_metrics_service.cc:1: // Copyright 2014 The Chromium
Authors. All rights reserved.
On 2014/02/12 01:06:50, sky wrote:
> No need to change this.

Done.

https://codereview.chromium.org/49753002/diff/3470001/chrome/browser/prefs/pr...
chrome/browser/prefs/pref_metrics_service.cc:83:
g_browser_process->rappor_service()->RecordSample(
On 2014/02/12 01:06:50, sky wrote:
> Don't you need to NULL check rappor_service()? Tests may have a different
> BrowserProcess that returns NULL, right?

I think there might not be tests that run this code, but I will add the NULL
check anyway.

https://codereview.chromium.org/49753002/diff/3470001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3470001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:82: std::string
seed_material(entropy_input + personalization_string);
On 2014/02/12 00:27:17, edknapp wrote:
> DCHECK(entropy_input.length() == kEntropyInputSize)

Done.

https://codereview.chromium.org/49753002/diff/3470001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:117: }
On 2014/02/12 00:27:17, edknapp wrote:
> Steps 3-6:
> if (provided_data.length() > 0) {
>   K = HMAC(...);
>   V = HMAC(...);
> }

Done.

[sky, 2014-02-12 14:42:07]

LGTM

[Alexei Svitkine (slow), 2014-02-12 18:13:03]

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:36: for(size_t i = 0; i < vector.size();
++i) {
Nit: space after for

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:82: DCHECK_EQ(entropy_input.size(),
kEntropyInputSize);
Nit: expected value should be on the left

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:109: if
(!hmac1.Init(std::string(key1.begin(), key1.end())))
I'm not super happy with the amount of copying of data going on in this code
(note just this line, in general).

For example, this line copies the |key1| byte vector into a string. I think you
can make a string piece out of |key1| by doing:

base::StringPiece(key1.data(), key1.size())

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:124: if (!hmac2.Sign(value2 + char(0x01)
+ provided_data, &key3[0], key3.size()))
This logic seems very similar to what you do for the about (1 vs. 2). Can you
make a helper function for this in the anon namespace, so the logic isn't
repeated (with the different char value being a param)?

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:136: size_t digest_length =
hmac_.DigestLength();
Nit: const

I'd also put this at the top of the function, so bytes, data and bytes_to_go are
together.

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.h:82: // HMAC_DRBG_Update
Expand comment to briefly explain what this does (e.g. what the params are used
for).

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.h:89: // Total number of bytes streamed from
the HMAC_DRBG Generate Process
Nit: add . at the end

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.h:90: size_t requested_bytes_;
Nit: I'd use a different name for this, "requested" makes me think of bytes
requested from this class (e.g. byte_count in the ctor). Maybe
"generated_bytes_"?

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
File components/rappor/byte_vector_utils_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils_unittest.cc:12: TEST(ByteVectorTest, TestOr)
{
Nit: No need for the actual test case names to start with "Test". I'd actually
name them after the methods they're testing when possible, e.g. in this case:

TEST(ByteVectorTest, ByteVectorOr) {

(Rename the ones below too.)

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/log_u...
File components/rappor/log_uploader_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:14: const GURL kTestServerURL =
GURL("http://a.com/");
Nit: Can just do:

const GURL kTestServerURL("http://a.com/");

[Steven Holte, 2014-02-12 22:28:48]

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:36: for(size_t i = 0; i < vector.size();
++i) {
On 2014/02/12 18:13:05, Alexei Svitkine wrote:
> Nit: space after for

Done.

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:82: DCHECK_EQ(entropy_input.size(),
kEntropyInputSize);
On 2014/02/12 18:13:05, Alexei Svitkine wrote:
> Nit: expected value should be on the left

Done.

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:109: if
(!hmac1.Init(std::string(key1.begin(), key1.end())))
On 2014/02/12 18:13:05, Alexei Svitkine wrote:
> I'm not super happy with the amount of copying of data going on in this code
> (note just this line, in general).
> 
> For example, this line copies the |key1| byte vector into a string. I think
you
> can make a string piece out of |key1| by doing:
> 
> base::StringPiece(key1.data(), key1.size())

Using StringPiece.

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:124: if (!hmac2.Sign(value2 + char(0x01)
+ provided_data, &key3[0], key3.size()))
On 2014/02/12 18:13:05, Alexei Svitkine wrote:
> This logic seems very similar to what you do for the about (1 vs. 2). Can you
> make a helper function for this in the anon namespace, so the logic isn't
> repeated (with the different char value being a param)?

Added helper function for K = HMAC(K, data)

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:136: size_t digest_length =
hmac_.DigestLength();
On 2014/02/12 18:13:05, Alexei Svitkine wrote:
> Nit: const
> 
> I'd also put this at the top of the function, so bytes, data and bytes_to_go
are
> together.

Done.

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.h:82: // HMAC_DRBG_Update
On 2014/02/12 18:13:05, Alexei Svitkine wrote:
> Expand comment to briefly explain what this does (e.g. what the params are
used
> for).

Done.

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.h:89: // Total number of bytes streamed from
the HMAC_DRBG Generate Process
On 2014/02/12 18:13:05, Alexei Svitkine wrote:
> Nit: add . at the end

Done.

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.h:90: size_t requested_bytes_;
On 2014/02/12 18:13:05, Alexei Svitkine wrote:
> Nit: I'd use a different name for this, "requested" makes me think of bytes
> requested from this class (e.g. byte_count in the ctor). Maybe
> "generated_bytes_"?

Done.

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
File components/rappor/byte_vector_utils_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils_unittest.cc:12: TEST(ByteVectorTest, TestOr)
{
On 2014/02/12 18:13:05, Alexei Svitkine wrote:
> Nit: No need for the actual test case names to start with "Test". I'd actually
> name them after the methods they're testing when possible, e.g. in this case:
> 
> TEST(ByteVectorTest, ByteVectorOr) {
> 
> (Rename the ones below too.)

Done.

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/log_u...
File components/rappor/log_uploader_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:14: const GURL kTestServerURL =
GURL("http://a.com/");
On 2014/02/12 18:13:05, Alexei Svitkine wrote:
> Nit: Can just do:
> 
> const GURL kTestServerURL("http://a.com/");

Done.

[Alexei Svitkine (slow), 2014-02-12 22:42:33]

re-LGTM with a nit

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3100002/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:124: if (!hmac2.Sign(value2 + char(0x01)
+ provided_data, &key3[0], key3.size()))
On 2014/02/12 22:28:50, Steven Holte wrote:
> On 2014/02/12 18:13:05, Alexei Svitkine wrote:
> > This logic seems very similar to what you do for the about (1 vs. 2). Can
you
> > make a helper function for this in the anon namespace, so the logic isn't
> > repeated (with the different char value being a param)?
> 
> Added helper function for K = HMAC(K, data)

Thanks, this is much cleaner!

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:18: base::StringPiece
ByteVectorToStringPiece(const ByteVector& lhs) {
Nit: I'd name this ByteVectorAsStringPiece(), since the string piece is just a
view over the existing byte vector rather than a conversion.

[edknapp, 2014-02-12 23:09:16]

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
File components/rappor/byte_vector_utils_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils_unittest.cc:88: 
Could you include the relevant sample test vectors from
http://csrc.nist.gov/groups/STM/cavp/

[Ilya Sherman, 2014-02-12 23:25:58]

Some comments on the code outside of //components/rappor, which I'll look at
next.  Note that Patch Set 47 has some upload issues, so I'm commenting on set
46.

Thanks, by the way, for filling in the design doc on chromium.org.  I think it's
really helpful to have that publicly available :)

Incidentally, have you given thought to giving the component a more transparent
name than "rappor"?  I'm a little worried that random Chromium developers will
stumble across this name and be puzzled by it.  This is probably a discussion
that's best to have off the CL; and if we arrive at a better name, it would
probably make sense to update the component name in a separate CL.  Just raising
it here while it's on my mind.

https://codereview.chromium.org/49753002/diff/3770001/chrome/browser/prefs/pr...
File chrome/browser/prefs/pref_metrics_service.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/chrome/browser/prefs/pr...
chrome/browser/prefs/pref_metrics_service.cc:83: if
(g_browser_process->rappor_service()) {
Can this actually ever be NULL?  If so, why?

https://codereview.chromium.org/49753002/diff/3770001/chrome/common/metrics/m...
File chrome/common/metrics/metrics_log_base.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/chrome/common/metrics/m...
chrome/common/metrics/metrics_log_base.cc:13: #include
"components/variations/metrics_util.h"
Why is the component name variations rather than metrics?  The desired
dependency is that variations is a subset of metrics, i.e. variations code can
depend on metrics code, but general metrics code shouldn't really depend on
variations code.

https://codereview.chromium.org/49753002/diff/3770001/components/OWNERS
File components/OWNERS (right):

https://codereview.chromium.org/49753002/diff/3770001/components/OWNERS#newco...
components/OWNERS:59: per-file rappor*=asvitkine@chromium.org
It's good to have more than a single owner for a component.  Any reason that
you're not including yourself as an OWNER?  If you're not yet a committer -- and
if so, I bet we can pretty safely nominate you based on the work in this CL -- I
would at least include a comment listing yourself as an OWNER-to-be.  That's
helpful because people will sometimes look at OWNERS files to understand who is
knowledgeable about a piece of code, in addition to simply seeking approval to
submit their own code.

https://codereview.chromium.org/49753002/diff/3770001/components/variations/m...
File components/variations/metrics_util.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/variations/m...
components/variations/metrics_util.h:16: uint32 HashName(const std::string&
name);
nit: This method should probably be renamed to something more specific, if it is
going to be a sibling of one called HashMetricName().

https://codereview.chromium.org/49753002/diff/3770001/tools/metrics/histogram...
File tools/metrics/histograms/histograms.xml (right):

https://codereview.chromium.org/49753002/diff/3770001/tools/metrics/histogram...
tools/metrics/histograms/histograms.xml:15993: +<histogram
name="Rappor.UploadResponseCode">
Hmm, do we not have an enum in this for response codes?  That's a little
surprising.

[Ilya Sherman, 2014-02-13 01:39:00]

Now with comments on the rappor code itself.  Since there are lots of nits, I've
emphasized some of my comments with "IMPORTANT".  Please pay extra attention to
those comments.  Thanks.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor.gypi
File components/rappor.gypi (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor.gypi#...
components/rappor.gypi:18: 'variations',
IMO it's not appropriate for the rappor component to depend on variations. 
Variations is about field trials; rappor has nothing to do with field trials
really (unless I'm missing something).  I know this doesn't feel like a big deal
right now, but getting this right at this time will make it easier to keep the
component layering healthy in the long term.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/OWNERS
File components/rappor/OWNERS (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/OWNER...
components/rappor/OWNERS:2: # Please include holte@ on reviews
Thanks for including this line, it's quite helpful :)

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
components/rappor/bloom_filter.cc:26: CityHash64WithSeed(str.data(), str.size(),
hash_seed_offset_ + i);
Please document why CityHash is used here, rather than a more commonly used hash
function.  What are the constraints that are important for this application, on
which CityHash shines?

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
File components/rappor/bloom_filter.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
components/rappor/bloom_filter.h:37: uint32_t hash_function_count() const {
return hash_function_count_; };
nit: Why does this need to be publicly exported?

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
components/rappor/bloom_filter.h:41: ByteVector bytes_;
nit: Please leave blank lines between documented member variables.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
components/rappor/bloom_filter.h:44: uint32_t hash_seed_offset_;
nit: Please add documentation for this member variable.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
File components/rappor/bloom_filter_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
components/rappor/bloom_filter_unittest.cc:48: EXPECT_EQ(1,
CountBits(filter.bytes()));
nit: Please directly #include the header where CountBits is defined.  (iwyu)

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:18: base::StringPiece
ByteVectorToStringPiece(const ByteVector& lhs) {
nit: Docs.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:19: return
base::StringPiece(reinterpret_cast<const char *>(&lhs[0]), lhs.size());
IMPORTANT: I don't think this is safe, as the byte vector might contain bytes
that evaluate to control characters or to the NUL byte.  I think you need to
actually make a copy, possibly base-64 encoded.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:22: std::string Concat(const ByteVector&
value, char c, const std::string& data) {
nit: Docs.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:23: return std::string(value.begin(),
value.end()) + c + data;
IMPORTANT: One of the chars you pass to Concat is the Nul character, below. 
Does that really work?  It seems unlikely to do what you want.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:26: // K = HMAC(K, data)
nit: Please expand the documentation to be written in the form of a sentence or
sentences.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:29: crypto::HMAC* out_hmac) {
Optional nit: "out_hmac" -> "result"

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:117: const size_t
HmacByteVectorGenerator::kEntropyInputSize = (128 / 8) * 3 / 2;
Please document what's happening in this computation.  Presumably the constants
+ computation are meant to be clearer than simply using 24, but it's not clear
to me what the semantics of this computation are.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:126: // HMAC_DRBG Update Process
It seems wrong for you to need to implement this within the rappor component.  I
would think this makes more sense in the crypto component.  Have you considered
moving the logic to there?

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:126: // HMAC_DRBG Update Process
The logic in this method is not obviously correct to me, because I'm not super
familiar with HMACs.  I think that means it's a good idea to provide additional
documentation here, most likely in the form of a link to the algorithm
description that this is implementing.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:137: // 3. If (provided_data = Null),
then return K and V.
This comment doesn't seem to match the code.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:148: const size_t digest_length =
hmac_.DigestLength();
Is this expected to be equal to the size of |value_|?  If so, it would be nice
to DCHECK that.  If not, I'm not sure how line 164 is guaranteed to be safe.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:168: // Check
max_number_of_bits_per_request from 10.1 Table 2
nit: Please provide a link to the reference that contains 10.1 Table 2.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:20: // Computes a bitwise OR of byte
vectors and stores the result in rhs.
nit: Please document the return value as well.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:23: // Merges the contents of lhs and rhs 
vectors according to a mask vector.
nit: I think it would be helpful to spell out, in English, that bit i in the
resulting vector is set to bit i from either the lhs or the rhs vector,
depending on whether bit i of the mask vector is 0 (lhs) or 1 (rhs).

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:24: // Equivalent to (lhs & ~mask) | (rhs
& mask).  Stores the result in rhs.
nit: Please document the return value as well.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:48: size_t byte_count_;
nit: This and the DISALLOW_COPY_AND_ASSIGN lines should have private visibility,
not protected.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:65: // secret value to seed the
pseudo-random number generator.  |secret| is
nit: There is no parameter named |secret| below.  Has this been replaced by one
or both of the two string variables?  Please update the documentation.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:73: static std::string
GenerateEntropyInput();
nit: Docs, please.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:79: virtual ByteVector
GetRandomByteVector() OVERRIDE;
nit: Please add the comment "// ByteVector implementation:" above this line.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:82: // Implements (Key, V) =
HMAC_DRBG_Update(provided_data, Key, V)
nit: IMO, you should probably use either (Key, Value) or (K, V); but not (Key,
V).

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:83: // V is read from and written to
value_
nit: Please end the sentence with a period.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:84: // Key is output by passing it to
hmac_.Init
nit: Ditto.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:92: size_t generated_bytes_;
nit: Please leave blank lines between documented member variables.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.cc:102: DiscardReason reason =
NUM_DISCARD_REASONS;
IMPORTANT: You should never try to log a value of NUM_DISCARD_REASONS.  Instead,
add an additional bucket for unknown failures.  I recommend making that the
first bucket, so that as the enum grows, that bucket doesn't randomly appear in
the middle.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.cc:119: // don't consider that a sign that the
server is in trouble.
It looks like this comment was copied from the UMA code.  Is this actually true
of the RAPPOR server?

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.cc:125: bool more_logs_remaining) {
nit: Indentation is slightly off.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.cc:147: }
nit: Please organize the .cc file so that the order of implementations matches
the order of declarations in the header file.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.h:31: const std::string& mime_type,
Why aren't the server_url and especially the mime_type simply constants defined
in the implementation file?

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.h:43: static base::TimeDelta
BackOffUploadInterval(base::TimeDelta);
nit: This doesn't seem like it needs to be public.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.h:70: scoped_refptr<net::URLRequestContextGetter>
request_context_;
nit: Docs, please.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.h:72: // The outstanding transmission appears as
a URL Fetch operation.
nit: Should "The outstanding transmission appears" instead be "The outstanding
transmission that appears"?

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.h:78: base::OneShotTimer<LogUploader>
upload_timer_;
nit: Docs, please.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
File components/rappor/log_uploader_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:15: const std::string kTestMimeType =
"text/plain";
These introduce static initializers, which are verboten in Chromium code. 
Instead, define const char[] arrays, and convert to the object types within test
function bodies.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:25: base::MessageLoopForUI loop_;
nit: Why is this needed?

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:36: base::TimeDelta
last_interval_set;
nit: All class member variables should be private.  You can simulate public
variables with accessors if need be.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:54: };
nit: DISALLOW_COPY_AND_ASSIGN.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:54: };
nit: This class should be declared in the anonymous namespace.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:60: "",
nit: Prefer std::string() to "".

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:66: // Log should be discarded
instead of retransmitted
nit: Please end the sentence with a period.  (Applies throughout.)

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/proto...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:20: optional int32 version = 1;
Hmm, is the Rappor scheme expected to change?  I'm wondering if it's really
worth including this field yet, or if we should wait to introduce it at the time
that the scheme does change (if ever).

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:26: optional int32 cohort = 2;
It would be nice to document the expected amount of entropy in this value.  Does
this act as something of a unique identifier, or is it intended to be chosen
from a relatively small sample space, so that a large number of users appear in
any given cohort?  Is it a purely random number for each user, or is there some
additional structure to how it is generated?

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:31: // The name of the metric,
hashed.
Please document which hash function is used.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_metric.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.cc:18: cohort *
parameters.bloom_filter_hash_function_count) {
Is it useful/desirable to DCHECK that this computed value is less than the
number of bytes in the Bloom filter?  Or, if it's ok for this to exceed the
Bloom filter's size, please add test code to exercise the relevant code paths. 
(You might already have such test code in place, in which case, good on ya', and
feel free to ignore this comment :)

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.cc:19: DCHECK_GE(cohort, 0);
nit: De-indent by two spaces.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.cc:32: std::string(bytes().begin(),
bytes().end());
IMPORTANT: Hmm, it doesn't seem safe to re-interpret the bytes as a string.  For
example, what if there are control characters, or NUL bytes?  I think you need
to base64-encode the bytes first, or something like that.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.cc:37: ByteVector fake_ones =
nit: I think "fake_bytes" might be a clearer name than "fake_ones".

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_metric.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.h:25: int32_t cohort);
nit: Please document the parameters, or at least document the class's member
variables.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.h:28: // Record additional samples in the Bloom
filter.
nit: "Record additional samples" -> "Records an additional sample"

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.h:31: // Retrieves the current Bloom filter
bits.
nit: It's a little confusing throughout this CL that you use "bits" in some
places and "bytes" in others.  It would be nice to be a little more consistent.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.h:45: BloomFilter bloom_;
nit: IMO "bloom_filter_" is a clearer, more descriptive (and hence better) name
than just "bloom_".

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_metric_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric_unittest.cc:81: //   [redacted_prob] * .75 +
(1-[redacted_prob] * .5) =
nit: It would help to explain where the .75 and the .5 in this calculation are
coming from, as you do in the other calculations above.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_parameters.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_parameters.h:23: std::string ToString() const;
nit: Why not export an operator== instead?

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_pref_names.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_pref_names.cc:11: // for redacting collected data.
IMPORTANT: Historically with UMA, we have been very cautious not to store
anything that looks like a unique identifier on the local client, even if it is
never transmitted via the network.  Is this client secret equivalent to a unique
identifier?  If so, we should have a deep privacy discussion about this.

Regardless of your answer above, please document the expected entropy of this
client secret -- i.e., how many random bytes are in the string?

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_pref_names.cc:16: // A randomly generated number, which
determines cohort data is reported for.
Essentially the same comment as above applies here.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_pref_names.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_pref_names.h:14: extern const char kRapporCohort[];
nit: This is currently not alphabetized ;)

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.cc:62: RapporService::~RapporService() {}
IMPORTANT: This should probably free any memory owned by the map, or else DCHECK
that the map is empty.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_service.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:11: #include "base/lazy_instance.h"
nit: This doesn't seem like it's used.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:12: #include "base/memory/weak_ptr.h"
nit: This doesn't seem like it's used.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:50: void SetSecretForTesting(std::string
secret) { secret_ = secret; }
nit: Pass by const-reference.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:50: void SetSecretForTesting(std::string
secret) { secret_ = secret; }
Optional nit: I prefer to restrict visibility of for-testing methods to
protected, rather than private.  However, the "ForTesting" suffix is sufficient,
so if you prefer the public visibility, I'm ok with it.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:59: bool ExportMetrics(RapporReports*
reports);
nit: Where is the "RapporReports" type declared?  I'm not sure what datatype
this is.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:62: // Creates and initializes the metric, if
it doesn't yet exist.
nit: Exposed for tests?

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:86: RapporMetric* LookupMetric(const
std::string& metric_name,
nit: Lookup -> LookUp (noun vs. verb)

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:99: scoped_ptr<LogUploader> uploader_;
nit: Can this be a direct class member, rather than a scoped_ptr?

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:101: // We keep all registered histograms in
a map, from name to histogram.
nit: "histogram" -> "metric"?

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:102: std::map<std::string, RapporMetric*>
metrics_map_;
Please document that the map owns the metrics it contains.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:105: base::Lock lock_;
IMPORTANT: Hmm, why does the map need a lock?  This class is unlikely to be
thread-safe.

[Alexei Svitkine (slow), 2014-02-13 04:22:31]

I'll let Steve reply to the other comments, but just wanted to reply to the
string/StringPiece comments.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:23: return std::string(value.begin(),
value.end()) + c + data;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> IMPORTANT: One of the chars you pass to Concat is the Nul character, below. 
> Does that really work?  It seems unlikely to do what you want.

This CL is using the std::string type (and StringPiece above) as just a
container for arbitrary bytes (non-text data). We do something similar in
MetricsService to store serialized protos (which are also not text). I believe
this is safe to do (see:
http://stackoverflow.com/questions/2845769/can-a-stdstring-contain-embedded-n...)
as long as all the things the things these strings are being passed to don't
expect them to be actual text.

[Steven Holte, 2014-02-13 05:11:10]

Ilya, could you expand more on what privacy concerns you have about storing a
random value in prefs?

https://codereview.chromium.org/49753002/diff/3770001/components/OWNERS
File components/OWNERS (right):

https://codereview.chromium.org/49753002/diff/3770001/components/OWNERS#newco...
components/OWNERS:59: per-file rappor*=asvitkine@chromium.org
On 2014/02/12 23:25:59, Ilya Sherman wrote:
> It's good to have more than a single owner for a component.  Any reason that
> you're not including yourself as an OWNER?  If you're not yet a committer --
and
> if so, I bet we can pretty safely nominate you based on the work in this CL --
I
> would at least include a comment listing yourself as an OWNER-to-be.  That's
> helpful because people will sometimes look at OWNERS files to understand who
is
> knowledgeable about a piece of code, in addition to simply seeking approval to
> submit their own code.

Added comment

https://codereview.chromium.org/49753002/diff/3770001/components/rappor.gypi
File components/rappor.gypi (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor.gypi#...
components/rappor.gypi:18: 'variations',
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> IMO it's not appropriate for the rappor component to depend on variations. 
> Variations is about field trials; rappor has nothing to do with field trials
> really (unless I'm missing something).  I know this doesn't feel like a big
deal
> right now, but getting this right at this time will make it easier to keep the
> component layering healthy in the long term.

No, it doesn't.  Unfortunately there a metrics component (yet) and it seems odd
to create one to put just MetricNameHash in it, and worse to leave it where it
was and depend on a chrome/ module.  The other alternative would be to move that
function to base/metrics/, which might be reasonable.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
components/rappor/bloom_filter.cc:26: CityHash64WithSeed(str.data(), str.size(),
hash_seed_offset_ + i);
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> Please document why CityHash is used here, rather than a more commonly used
hash
> function.  What are the constraints that are important for this application,
on
> which CityHash shines?

Added a comment which says "because it's supported in Dremel".

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
File components/rappor/bloom_filter.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
components/rappor/bloom_filter.h:37: uint32_t hash_function_count() const {
return hash_function_count_; };
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Why does this need to be publicly exported?

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
components/rappor/bloom_filter.h:41: ByteVector bytes_;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Please leave blank lines between documented member variables.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
components/rappor/bloom_filter.h:44: uint32_t hash_seed_offset_;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Please add documentation for this member variable.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
File components/rappor/bloom_filter_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/bloom...
components/rappor/bloom_filter_unittest.cc:48: EXPECT_EQ(1,
CountBits(filter.bytes()));
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Please directly #include the header where CountBits is defined.  (iwyu)

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:18: base::StringPiece
ByteVectorToStringPiece(const ByteVector& lhs) {
On 2014/02/12 22:42:36, Alexei Svitkine wrote:
> Nit: I'd name this ByteVectorAsStringPiece(), since the string piece is just a
> view over the existing byte vector rather than a conversion.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:18: base::StringPiece
ByteVectorToStringPiece(const ByteVector& lhs) {
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Docs.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:19: return
base::StringPiece(reinterpret_cast<const char *>(&lhs[0]), lhs.size());
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> IMPORTANT: I don't think this is safe, as the byte vector might contain bytes
> that evaluate to control characters or to the NUL byte.  I think you need to
> actually make a copy, possibly base-64 encoded.

It should be perfectly valid for the StringPiece to contain NUL bytes/control
characters.  Since the length in included in the constructor, they shouldn't
terminate the string.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:23: return std::string(value.begin(),
value.end()) + c + data;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> IMPORTANT: One of the chars you pass to Concat is the Nul character, below. 
> Does that really work?  It seems unlikely to do what you want.

Yes, it does.  std::string may contain null characters.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:26: // K = HMAC(K, data)
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Please expand the documentation to be written in the form of a sentence
or
> sentences.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:29: crypto::HMAC* out_hmac) {
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> Optional nit: "out_hmac" -> "result"

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:117: const size_t
HmacByteVectorGenerator::kEntropyInputSize = (128 / 8) * 3 / 2;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> Please document what's happening in this computation.  Presumably the
constants
> + computation are meant to be clearer than simply using 24, but it's not clear
> to me what the semantics of this computation are.

Expanded, and also updated security_strength to match the examples from NIST.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:126: // HMAC_DRBG Update Process
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> It seems wrong for you to need to implement this within the rappor component. 
I
> would think this makes more sense in the crypto component.  Have you
considered
> moving the logic to there?

This was discussed a bit earlier.  This isn't a general implementation of
HMAC_DRBG, so we don't want to encourage other components to start using it.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:126: // HMAC_DRBG Update Process
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> The logic in this method is not obviously correct to me, because I'm not super
> familiar with HMACs.  I think that means it's a good idea to provide
additional
> documentation here, most likely in the form of a link to the algorithm
> description that this is implementing.

There is a link in the header file, along with some additional explanation of
the assumptions this is implemented with.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:137: // 3. If (provided_data = Null),
then return K and V.
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> This comment doesn't seem to match the code.

Added a comment to the fact that we are ignoring the NULL case explicitly.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:148: const size_t digest_length =
hmac_.DigestLength();
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> Is this expected to be equal to the size of |value_|?  If so, it would be nice
> to DCHECK that.  If not, I'm not sure how line 164 is guaranteed to be safe.

Added DCHECK

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:168: // Check
max_number_of_bits_per_request from 10.1 Table 2
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Please provide a link to the reference that contains 10.1 Table 2.

See class declaration in header file.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:48: size_t byte_count_;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: This and the DISALLOW_COPY_AND_ASSIGN lines should have private
visibility,
> not protected.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:65: // secret value to seed the
pseudo-random number generator.  |secret| is
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: There is no parameter named |secret| below.  Has this been replaced by
one
> or both of the two string variables?  Please update the documentation.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:73: static std::string
GenerateEntropyInput();
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Docs, please.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:79: virtual ByteVector
GetRandomByteVector() OVERRIDE;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Please add the comment "// ByteVector implementation:" above this line.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:82: // Implements (Key, V) =
HMAC_DRBG_Update(provided_data, Key, V)
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: IMO, you should probably use either (Key, Value) or (K, V); but not (Key,
> V).

This is what the NIST spec uses.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:83: // V is read from and written to
value_
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Please end the sentence with a period.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:84: // Key is output by passing it to
hmac_.Init
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Ditto.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:92: size_t generated_bytes_;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Please leave blank lines between documented member variables.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.cc:102: DiscardReason reason =
NUM_DISCARD_REASONS;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> IMPORTANT: You should never try to log a value of NUM_DISCARD_REASONS. 
Instead,
> add an additional bucket for unknown failures.  I recommend making that the
> first bucket, so that as the enum grows, that bucket doesn't randomly appear
in
> the middle.

We never do log NUM_DISCARD_REASONS, as that's used here to indicate we
shouldn't discard it.  Also there isn't an unknown case, and I wouldn't expect
there to be one, since we aren't interpreting a code from an external source as
a reason.  The response_code histogram has that covered.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.cc:119: // don't consider that a sign that the
server is in trouble.
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> It looks like this comment was copied from the UMA code.  Is this actually
true
> of the RAPPOR server?

It will be, though the server doesn't exist yet.  The local servers I used for
testing kept that behavior. It seems like something it should do.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.cc:125: bool more_logs_remaining) {
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Indentation is slightly off.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.cc:147: }
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Please organize the .cc file so that the order of implementations matches
> the order of declarations in the header file.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.h:31: const std::string& mime_type,
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> Why aren't the server_url and especially the mime_type simply constants
defined
> in the implementation file?

Server URL at least, must be a parameter for now because we will be setting it
through a Finch trial.  Since the log uploader does not otherwise make any
assumptions about the format of it's payload, it would be nice to reuse it for
other things, so that's why mime_type is a parameter.  We aren't quite able to
reuse it for UMA, yet but that's something I would like to expand it to, later.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.h:43: static base::TimeDelta
BackOffUploadInterval(base::TimeDelta);
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: This doesn't seem like it needs to be public.

Changed to protected and marked exposed for tests.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.h:70: scoped_refptr<net::URLRequestContextGetter>
request_context_;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Docs, please.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.h:72: // The outstanding transmission appears as
a URL Fetch operation.
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Should "The outstanding transmission appears" instead be "The outstanding
> transmission that appears"?

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.h:78: base::OneShotTimer<LogUploader>
upload_timer_;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Docs, please.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
File components/rappor/log_uploader_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:15: const std::string kTestMimeType =
"text/plain";
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> These introduce static initializers, which are verboten in Chromium code. 
> Instead, define const char[] arrays, and convert to the object types within
test
> function bodies.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:25: base::MessageLoopForUI loop_;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Why is this needed?

Because otherwise
net::TestURLRequestContextGetter(base::MessageLoopProxy::current())
will crash.  Added a comment.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:36: base::TimeDelta
last_interval_set;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: All class member variables should be private.  You can simulate public
> variables with accessors if need be.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:54: };
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: DISALLOW_COPY_AND_ASSIGN.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:54: };
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: This class should be declared in the anonymous namespace.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:60: "",
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Prefer std::string() to "".

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:66: // Log should be discarded
instead of retransmitted
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Please end the sentence with a period.  (Applies throughout.)

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/proto...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:20: optional int32 version = 1;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> Hmm, is the Rappor scheme expected to change?  I'm wondering if it's really
> worth including this field yet, or if we should wait to introduce it at the
time
> that the scheme does change (if ever).

This was added in response to someone else's comment, but I don't feel strongly
either way.  We may add different types of Rappor metrics later (that may allow
correlations, numeric values etc.).

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:26: optional int32 cohort = 2;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> It would be nice to document the expected amount of entropy in this value. 
Does
> this act as something of a unique identifier, or is it intended to be chosen
> from a relatively small sample space, so that a large number of users appear
in
> any given cohort?  Is it a purely random number for each user, or is there
some
> additional structure to how it is generated?

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:31: // The name of the metric,
hashed.
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> Please document which hash function is used.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_metric.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.cc:18: cohort *
parameters.bloom_filter_hash_function_count) {
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> Is it useful/desirable to DCHECK that this computed value is less than the
> number of bytes in the Bloom filter?  Or, if it's ok for this to exceed the
> Bloom filter's size, please add test code to exercise the relevant code paths.

> (You might already have such test code in place, in which case, good on ya',
and
> feel free to ignore this comment :)

Changed "BloomFilterTest.HugeFilter" test to use a large seed offset.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.cc:19: DCHECK_GE(cohort, 0);
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: De-indent by two spaces.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.cc:32: std::string(bytes().begin(),
bytes().end());
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> IMPORTANT: Hmm, it doesn't seem safe to re-interpret the bytes as a string. 
For
> example, what if there are control characters, or NUL bytes?  I think you need
> to base64-encode the bytes first, or something like that.

std::string is not null terminated.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.cc:37: ByteVector fake_ones =
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: I think "fake_bytes" might be a clearer name than "fake_ones".

That seems like it might imply that the bits are masked in by bytes.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_metric.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.h:25: int32_t cohort);
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Please document the parameters, or at least document the class's member
> variables.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.h:28: // Record additional samples in the Bloom
filter.
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: "Record additional samples" -> "Records an additional sample"

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.h:31: // Retrieves the current Bloom filter
bits.
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: It's a little confusing throughout this CL that you use "bits" in some
> places and "bytes" in others.  It would be nice to be a little more
consistent.

Bits are the content, bytes are the format.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.h:45: BloomFilter bloom_;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: IMO "bloom_filter_" is a clearer, more descriptive (and hence better)
name
> than just "bloom_".

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_metric_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric_unittest.cc:81: //   [redacted_prob] * .75 +
(1-[redacted_prob] * .5) =
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: It would help to explain where the .75 and the .5 in this calculation are
> coming from, as you do in the other calculations above.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_parameters.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_parameters.h:23: std::string ToString() const;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Why not export an operator== instead?

Then the DCHECK won't show what is different.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_pref_names.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_pref_names.cc:11: // for redacting collected data.
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> IMPORTANT: Historically with UMA, we have been very cautious not to store
> anything that looks like a unique identifier on the local client, even if it
is
> never transmitted via the network.  Is this client secret equivalent to a
unique
> identifier?  If so, we should have a deep privacy discussion about this.
> 
> Regardless of your answer above, please document the expected entropy of this
> client secret -- i.e., how many random bytes are in the string?

Well this has at least 192 bits of entropy, should probably have double that,
and is randomly generated on the client, so it would definitely be unique.  Not
sure I understand the rational about not wanting to store such a value though. 
At that amount of entropy, documenting the amount of entropy seems a bit moot
for your purposes.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_pref_names.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_pref_names.h:14: extern const char kRapporCohort[];
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: This is currently not alphabetized ;)

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.cc:62: RapporService::~RapporService() {}
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> IMPORTANT: This should probably free any memory owned by the map, or else
DCHECK
> that the map is empty.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_service.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:11: #include "base/lazy_instance.h"
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: This doesn't seem like it's used.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:12: #include "base/memory/weak_ptr.h"
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: This doesn't seem like it's used.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:50: void SetSecretForTesting(std::string
secret) { secret_ = secret; }
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> Optional nit: I prefer to restrict visibility of for-testing methods to
> protected, rather than private.  However, the "ForTesting" suffix is
sufficient,
> so if you prefer the public visibility, I'm ok with it.

Leaving it public.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:50: void SetSecretForTesting(std::string
secret) { secret_ = secret; }
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Pass by const-reference.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:59: bool ExportMetrics(RapporReports*
reports);
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Where is the "RapporReports" type declared?  I'm not sure what datatype
> this is.

It's the proto/rappor_metric.pb.h type.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:62: // Creates and initializes the metric, if
it doesn't yet exist.
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Exposed for tests?

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:86: RapporMetric* LookupMetric(const
std::string& metric_name,
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Lookup -> LookUp (noun vs. verb)

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:99: scoped_ptr<LogUploader> uploader_;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Can this be a direct class member, rather than a scoped_ptr?

No, it's constructor requires the request_context passed to Start.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:101: // We keep all registered histograms in
a map, from name to histogram.
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: "histogram" -> "metric"?

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:102: std::map<std::string, RapporMetric*>
metrics_map_;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> Please document that the map owns the metrics it contains.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:105: base::Lock lock_;
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> IMPORTANT: Hmm, why does the map need a lock?  This class is unlikely to be
> thread-safe.

I'm not sure what thread the Timer executes it's callback on, but if it
shouldn't actually matter for measurements we are taking now, since
PrefMetricsService records it's sample shortly after Start().  I'll remove the
lock for now, since it's causing problems in the try jobs anyway.

https://codereview.chromium.org/49753002/diff/3770001/components/variations/m...
File components/variations/metrics_util.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/variations/m...
components/variations/metrics_util.h:16: uint32 HashName(const std::string&
name);
On 2014/02/12 23:25:59, Ilya Sherman wrote:
> nit: This method should probably be renamed to something more specific, if it
is
> going to be a sibling of one called HashMetricName().

I'd rather leave that for a more focused cleanup of the variations/metrics
dependencies.

https://codereview.chromium.org/49753002/diff/3770001/tools/metrics/histogram...
File tools/metrics/histograms/histograms.xml (right):

https://codereview.chromium.org/49753002/diff/3770001/tools/metrics/histogram...
tools/metrics/histograms/histograms.xml:15993: +<histogram
name="Rappor.UploadResponseCode">
On 2014/02/12 23:25:59, Ilya Sherman wrote:
> Hmm, do we not have an enum in this for response codes?  That's a little
> surprising.

It looks like we actually do, although of the only thing that uses it is
Wallet.ResponseCode.  There are a bunch of other histograms that report them,
but just don't use the enum (including Variations.SeedFetchResponseCode)

Modified to use the enum.

[Steven Holte, 2014-02-13 21:13:43]

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
File components/rappor/byte_vector_utils_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils_unittest.cc:88: 
On 2014/02/12 23:09:17, edknapp wrote:
> Could you include the relevant sample test vectors from
> http://csrc.nist.gov/groups/STM/cavp/

As Ed discovered in another thread, the test vectors NIST provides are generated
from the second request.  Since this HMAC_DRBG implementation only supports one
request, testing against those vectors is a bit problematic.  To mitigate this a
bit, I've modified the code to make the Update function more reusable and moved
it into the anonymous namespace as HMAC_DRBG_Update().

Then I did some manual testing by applying this patch, which makes the
constructor generate and discard a first request, and streams output from the
second, and then tests that against an NIST vector, and the test passed.  Not
sure what a better way to support this kind of testing would be.

diff --git a/components/rappor/byte_vector_utils.cc
b/components/rappor/byte_vector_utils.cc
index 7fb5554..e0571b9 100644
--- a/components/rappor/byte_vector_utils.cc
+++ b/components/rappor/byte_vector_utils.cc
@@ -153,7 +153,18 @@ HmacByteVectorGenerator::HmacByteVectorGenerator(
   // (value_ in initializer list)
 
   // 4. (Key, V) = HMAC_DRBG_Update(seed_material, Key, V)
-  if (!HMAC_DRBG_Update(seed_material, hmac1, &value_, &hmac_))
+  crypto::HMAC hmacZ(crypto::HMAC::SHA256);
+  if (!HMAC_DRBG_Update(seed_material, hmac1, &value_, &hmacZ))
+    NOTREACHED();
+  if (!HMAC_Rehash(hmacZ, &value_))
+    NOTREACHED();
+  if (!HMAC_Rehash(hmacZ, &value_))
+    NOTREACHED();
+  if (!HMAC_Rehash(hmacZ, &value_))
+    NOTREACHED();
+  if (!HMAC_Rehash(hmacZ, &value_))
+    NOTREACHED();
+  if (!HMAC_DRBG_Update("", hmacZ, &value_, &hmac_))
     NOTREACHED();
 }
 
diff --git a/components/rappor/byte_vector_utils_unittest.cc
b/components/rappor/byte_vector_utils_unittest.cc
index cb55317..4e3474b 100644
--- a/components/rappor/byte_vector_utils_unittest.cc
+++ b/components/rappor/byte_vector_utils_unittest.cc
@@ -5,6 +5,7 @@
 #include "components/rappor/byte_vector_utils.h"
 
 #include "base/rand_util.h"
+#include "base/strings/string_number_conversions.h"
 #include "testing/gtest/include/gtest/gtest.h"
 
 namespace rappor {
@@ -37,6 +38,16 @@ TEST(ByteVectorTest, ByteVectorGenerator) {
   EXPECT_EQ(random_75.size(), 2u);
 }
 
+const char entropy[] =
"ca851911349384bffe89de1cbdc46e6831e44d34a4fb935ee285dd14b71a7488";
+const char nonce[] = "659ba96c601dc69fc902940805ec0ca8";
+const char expected[] =
"e528e9abf2dece54d47c7e75e5fe302149f817ea9fb4bee6f4199697d04d5b89d54fbb978a15b5c443c9ec21036d2460b6f73ebad0dc2aba6e624abf07745bc107694bb7547bb0995f70de25d6b29e2d3011bb19d27676c07162c8b5ccde0668961df86803482cb37ed6d5c0bb8d50cf1f50d476aa0458bdaba806f48be9dcb8";
+
+std::string HexToString(const char* hex) {
+  ByteVector bv;
+  base::HexStringToBytes(hex, &bv);
+  return std::string(bv.begin(), bv.end());
+}
+
 TEST(ByteVectorTest, HmacByteVectorGenerator) {
   HmacByteVectorGenerator generator(1u,
       std::string(HmacByteVectorGenerator::kEntropyInputSize, 0x00), "");
@@ -48,6 +59,15 @@ TEST(ByteVectorTest, HmacByteVectorGenerator) {
   EXPECT_EQ(random_75[0], 0xdf);
 }
 
+TEST(ByteVectorTest, HmacNist) {
+  std::string entropy_input = HexToString(entropy) + HexToString(nonce);
+  HmacByteVectorGenerator generator(1024u / 8, entropy_input, "");
+  ByteVector random_50 = generator.GetWeightedRandomByteVector(PROBABILITY_50);
+
+  EXPECT_EQ(HexToString(expected),
+      std::string(random_50.begin(), random_50.end()));
+}
+
 TEST(ByteVectorTest, WeightedRandomStatistics50) {
   ByteVectorGenerator generator(50u);
   ByteVector random = generator.GetWeightedRandomByteVector(PROBABILITY_50);

[Alexei Svitkine (slow), 2014-02-13 21:25:36]

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
File components/rappor/byte_vector_utils_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils_unittest.cc:88: 
On 2014/02/13 21:13:45, Steven Holte wrote:
> On 2014/02/12 23:09:17, edknapp wrote:
> > Could you include the relevant sample test vectors from
> > http://csrc.nist.gov/groups/STM/cavp/
> 
> As Ed discovered in another thread, the test vectors NIST provides are
generated
> from the second request.  Since this HMAC_DRBG implementation only supports
one
> request, testing against those vectors is a bit problematic.  To mitigate this
a
> bit, I've modified the code to make the Update function more reusable and
moved
> it into the anonymous namespace as HMAC_DRBG_Update().
> 
> Then I did some manual testing by applying this patch, which makes the
> constructor generate and discard a first request, and streams output from the
> second, and then tests that against an NIST vector, and the test passed.  Not
> sure what a better way to support this kind of testing would be.

If you need to test a function that's in the anon namespace, one way you can do
this is to put it inside an "internal" namespace and declare it in the header.
So then the test code can still access it for testing.

For example:

https://code.google.com/p/chromium/codesearch#chromium/src/components/variati...

[edknapp, 2014-02-13 21:55:39]

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
File components/rappor/byte_vector_utils_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils_unittest.cc:88: 
What do you think of having each call to our GetRandomByteVector() correspond to
a call to the standard's HMAC_DRBG_Generate_algorithm? 

On 2014/02/13 21:13:45, Steven Holte wrote:
> On 2014/02/12 23:09:17, edknapp wrote:
> > Could you include the relevant sample test vectors from
> > http://csrc.nist.gov/groups/STM/cavp/
> 
> As Ed discovered in another thread, the test vectors NIST provides are
generated
> from the second request.  Since this HMAC_DRBG implementation only supports
one
> request, testing against those vectors is a bit problematic.  To mitigate this
a
> bit, I've modified the code to make the Update function more reusable and
moved
> it into the anonymous namespace as HMAC_DRBG_Update().
> 
> Then I did some manual testing by applying this patch, which makes the
> constructor generate and discard a first request, and streams output from the
> second, and then tests that against an NIST vector, and the test passed.  Not
> sure what a better way to support this kind of testing would be.
> 
> diff --git a/components/rappor/byte_vector_utils.cc
> b/components/rappor/byte_vector_utils.cc
> index 7fb5554..e0571b9 100644
> --- a/components/rappor/byte_vector_utils.cc
> +++ b/components/rappor/byte_vector_utils.cc
> @@ -153,7 +153,18 @@ HmacByteVectorGenerator::HmacByteVectorGenerator(
>    // (value_ in initializer list)
>  
>    // 4. (Key, V) = HMAC_DRBG_Update(seed_material, Key, V)
> -  if (!HMAC_DRBG_Update(seed_material, hmac1, &value_, &hmac_))
> +  crypto::HMAC hmacZ(crypto::HMAC::SHA256);
> +  if (!HMAC_DRBG_Update(seed_material, hmac1, &value_, &hmacZ))
> +    NOTREACHED();
> +  if (!HMAC_Rehash(hmacZ, &value_))
> +    NOTREACHED();
> +  if (!HMAC_Rehash(hmacZ, &value_))
> +    NOTREACHED();
> +  if (!HMAC_Rehash(hmacZ, &value_))
> +    NOTREACHED();
> +  if (!HMAC_Rehash(hmacZ, &value_))
> +    NOTREACHED();
> +  if (!HMAC_DRBG_Update("", hmacZ, &value_, &hmac_))
>      NOTREACHED();
>  }
>  
> diff --git a/components/rappor/byte_vector_utils_unittest.cc
> b/components/rappor/byte_vector_utils_unittest.cc
> index cb55317..4e3474b 100644
> --- a/components/rappor/byte_vector_utils_unittest.cc
> +++ b/components/rappor/byte_vector_utils_unittest.cc
> @@ -5,6 +5,7 @@
>  #include "components/rappor/byte_vector_utils.h"
>  
>  #include "base/rand_util.h"
> +#include "base/strings/string_number_conversions.h"
>  #include "testing/gtest/include/gtest/gtest.h"
>  
>  namespace rappor {
> @@ -37,6 +38,16 @@ TEST(ByteVectorTest, ByteVectorGenerator) {
>    EXPECT_EQ(random_75.size(), 2u);
>  }
>  
> +const char entropy[] =
> "ca851911349384bffe89de1cbdc46e6831e44d34a4fb935ee285dd14b71a7488";
> +const char nonce[] = "659ba96c601dc69fc902940805ec0ca8";
> +const char expected[] =
>
"e528e9abf2dece54d47c7e75e5fe302149f817ea9fb4bee6f4199697d04d5b89d54fbb978a15b5c443c9ec21036d2460b6f73ebad0dc2aba6e624abf07745bc107694bb7547bb0995f70de25d6b29e2d3011bb19d27676c07162c8b5ccde0668961df86803482cb37ed6d5c0bb8d50cf1f50d476aa0458bdaba806f48be9dcb8";
> +
> +std::string HexToString(const char* hex) {
> +  ByteVector bv;
> +  base::HexStringToBytes(hex, &bv);
> +  return std::string(bv.begin(), bv.end());
> +}
> +
>  TEST(ByteVectorTest, HmacByteVectorGenerator) {
>    HmacByteVectorGenerator generator(1u,
>        std::string(HmacByteVectorGenerator::kEntropyInputSize, 0x00), "");
> @@ -48,6 +59,15 @@ TEST(ByteVectorTest, HmacByteVectorGenerator) {
>    EXPECT_EQ(random_75[0], 0xdf);
>  }
>  
> +TEST(ByteVectorTest, HmacNist) {
> +  std::string entropy_input = HexToString(entropy) + HexToString(nonce);
> +  HmacByteVectorGenerator generator(1024u / 8, entropy_input, "");
> +  ByteVector random_50 =
generator.GetWeightedRandomByteVector(PROBABILITY_50);
> +
> +  EXPECT_EQ(HexToString(expected),
> +      std::string(random_50.begin(), random_50.end()));
> +}
> +
>  TEST(ByteVectorTest, WeightedRandomStatistics50) {
>    ByteVectorGenerator generator(50u);
>    ByteVector random = generator.GetWeightedRandomByteVector(PROBABILITY_50);

[Steven Holte, 2014-02-13 22:01:51]

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
File components/rappor/byte_vector_utils_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils_unittest.cc:88: 
On 2014/02/13 21:55:41, edknapp wrote:
> What do you think of having each call to our GetRandomByteVector() correspond
to
> a call to the standard's HMAC_DRBG_Generate_algorithm? 
> 
> On 2014/02/13 21:13:45, Steven Holte wrote:
> > On 2014/02/12 23:09:17, edknapp wrote:
> > > Could you include the relevant sample test vectors from
> > > http://csrc.nist.gov/groups/STM/cavp/
> > 
> > As Ed discovered in another thread, the test vectors NIST provides are
> generated
> > from the second request.  Since this HMAC_DRBG implementation only supports
> one
> > request, testing against those vectors is a bit problematic.  To mitigate
this
> a
> > bit, I've modified the code to make the Update function more reusable and
> moved
> > it into the anonymous namespace as HMAC_DRBG_Update().
> > 
> > Then I did some manual testing by applying this patch, which makes the
> > constructor generate and discard a first request, and streams output from
the
> > second, and then tests that against an NIST vector, and the test passed. 
Not
> > sure what a better way to support this kind of testing would be.
> > 
> > diff --git a/components/rappor/byte_vector_utils.cc
> > b/components/rappor/byte_vector_utils.cc
> > index 7fb5554..e0571b9 100644
> > --- a/components/rappor/byte_vector_utils.cc
> > +++ b/components/rappor/byte_vector_utils.cc
> > @@ -153,7 +153,18 @@ HmacByteVectorGenerator::HmacByteVectorGenerator(
> >    // (value_ in initializer list)
> >  
> >    // 4. (Key, V) = HMAC_DRBG_Update(seed_material, Key, V)
> > -  if (!HMAC_DRBG_Update(seed_material, hmac1, &value_, &hmac_))
> > +  crypto::HMAC hmacZ(crypto::HMAC::SHA256);
> > +  if (!HMAC_DRBG_Update(seed_material, hmac1, &value_, &hmacZ))
> > +    NOTREACHED();
> > +  if (!HMAC_Rehash(hmacZ, &value_))
> > +    NOTREACHED();
> > +  if (!HMAC_Rehash(hmacZ, &value_))
> > +    NOTREACHED();
> > +  if (!HMAC_Rehash(hmacZ, &value_))
> > +    NOTREACHED();
> > +  if (!HMAC_Rehash(hmacZ, &value_))
> > +    NOTREACHED();
> > +  if (!HMAC_DRBG_Update("", hmacZ, &value_, &hmac_))
> >      NOTREACHED();
> >  }
> >  
> > diff --git a/components/rappor/byte_vector_utils_unittest.cc
> > b/components/rappor/byte_vector_utils_unittest.cc
> > index cb55317..4e3474b 100644
> > --- a/components/rappor/byte_vector_utils_unittest.cc
> > +++ b/components/rappor/byte_vector_utils_unittest.cc
> > @@ -5,6 +5,7 @@
> >  #include "components/rappor/byte_vector_utils.h"
> >  
> >  #include "base/rand_util.h"
> > +#include "base/strings/string_number_conversions.h"
> >  #include "testing/gtest/include/gtest/gtest.h"
> >  
> >  namespace rappor {
> > @@ -37,6 +38,16 @@ TEST(ByteVectorTest, ByteVectorGenerator) {
> >    EXPECT_EQ(random_75.size(), 2u);
> >  }
> >  
> > +const char entropy[] =
> > "ca851911349384bffe89de1cbdc46e6831e44d34a4fb935ee285dd14b71a7488";
> > +const char nonce[] = "659ba96c601dc69fc902940805ec0ca8";
> > +const char expected[] =
> >
>
"e528e9abf2dece54d47c7e75e5fe302149f817ea9fb4bee6f4199697d04d5b89d54fbb978a15b5c443c9ec21036d2460b6f73ebad0dc2aba6e624abf07745bc107694bb7547bb0995f70de25d6b29e2d3011bb19d27676c07162c8b5ccde0668961df86803482cb37ed6d5c0bb8d50cf1f50d476aa0458bdaba806f48be9dcb8";
> > +
> > +std::string HexToString(const char* hex) {
> > +  ByteVector bv;
> > +  base::HexStringToBytes(hex, &bv);
> > +  return std::string(bv.begin(), bv.end());
> > +}
> > +
> >  TEST(ByteVectorTest, HmacByteVectorGenerator) {
> >    HmacByteVectorGenerator generator(1u,
> >        std::string(HmacByteVectorGenerator::kEntropyInputSize, 0x00), "");
> > @@ -48,6 +59,15 @@ TEST(ByteVectorTest, HmacByteVectorGenerator) {
> >    EXPECT_EQ(random_75[0], 0xdf);
> >  }
> >  
> > +TEST(ByteVectorTest, HmacNist) {
> > +  std::string entropy_input = HexToString(entropy) + HexToString(nonce);
> > +  HmacByteVectorGenerator generator(1024u / 8, entropy_input, "");
> > +  ByteVector random_50 =
> generator.GetWeightedRandomByteVector(PROBABILITY_50);
> > +
> > +  EXPECT_EQ(HexToString(expected),
> > +      std::string(random_50.begin(), random_50.end()));
> > +}
> > +
> >  TEST(ByteVectorTest, WeightedRandomStatistics50) {
> >    ByteVectorGenerator generator(50u);
> >    ByteVector random =
generator.GetWeightedRandomByteVector(PROBABILITY_50);
> 

Actually, after thinking about it a bit more, I figured out a reasonable way of
supporting the second request by adding a second constructor.  Since that's not
our normal behaviour, I've made that protected, but it does allow running
against the NIST vectors in tests.

[Ilya Sherman, 2014-02-13 23:23:06]

Thanks!

https://codereview.chromium.org/49753002/diff/3770001/components/rappor.gypi
File components/rappor.gypi (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor.gypi#...
components/rappor.gypi:18: 'variations',
On 2014/02/13 05:11:12, Steven Holte wrote:
> On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > IMO it's not appropriate for the rappor component to depend on variations. 
> > Variations is about field trials; rappor has nothing to do with field trials
> > really (unless I'm missing something).  I know this doesn't feel like a big
> deal
> > right now, but getting this right at this time will make it easier to keep
the
> > component layering healthy in the long term.
> 
> No, it doesn't.  Unfortunately there a metrics component (yet) and it seems
odd
> to create one to put just MetricNameHash in it, and worse to leave it where it
> was and depend on a chrome/ module.  The other alternative would be to move
that
> function to base/metrics/, which might be reasonable.

We should move code out of //chrome/browser/metrics into //components/metrics
anyway: [ http://crbug.com/332475 ].  While it's a little silly to have a
component with just a single method in it, we'll need the component in the
long-run anyway, so I don't think there's any harm to creating it now.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:19: return
base::StringPiece(reinterpret_cast<const char *>(&lhs[0]), lhs.size());
On 2014/02/13 05:11:12, Steven Holte wrote:
> On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > IMPORTANT: I don't think this is safe, as the byte vector might contain
bytes
> > that evaluate to control characters or to the NUL byte.  I think you need to
> > actually make a copy, possibly base-64 encoded.
> 
> It should be perfectly valid for the StringPiece to contain NUL bytes/control
> characters.  Since the length in included in the constructor, they shouldn't
> terminate the string.

Yes, looks like you're right -- I didn't realize that std::strings supported
internal NUL bytes.  That said, I still think the reinterpret_cast is unsafe
(see my comment on the latest patch set, which goes into more detail).

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:23: return std::string(value.begin(),
value.end()) + c + data;
On 2014/02/13 05:11:12, Steven Holte wrote:
> On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > IMPORTANT: One of the chars you pass to Concat is the Nul character, below. 
> > Does that really work?  It seems unlikely to do what you want.
> 
> Yes, it does.  std::string may contain null characters.

Yeah, you guys are right.  My bad.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:126: // HMAC_DRBG Update Process
On 2014/02/13 05:11:12, Steven Holte wrote:
> On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > The logic in this method is not obviously correct to me, because I'm not
super
> > familiar with HMACs.  I think that means it's a good idea to provide
> additional
> > documentation here, most likely in the form of a link to the algorithm
> > description that this is implementing.
> 
> There is a link in the header file, along with some additional explanation of
> the assumptions this is implemented with.

Please re-link to it in the implementation file, as that's where the intricate
logic is.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:168: // Check
max_number_of_bits_per_request from 10.1 Table 2
On 2014/02/13 05:11:12, Steven Holte wrote:
> On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > nit: Please provide a link to the reference that contains 10.1 Table 2.
> 
> See class declaration in header file.

Please include the link at each relevant location.  Someone reading through this
file a year from now won't necessarily know to look in the header file for the
reference.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:23: // Merges the contents of lhs and rhs 
vectors according to a mask vector.
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: I think it would be helpful to spell out, in English, that bit i in the
> resulting vector is set to bit i from either the lhs or the rhs vector,
> depending on whether bit i of the mask vector is 0 (lhs) or 1 (rhs).

^^^

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:24: // Equivalent to (lhs & ~mask) | (rhs
& mask).  Stores the result in rhs.
On 2014/02/13 01:39:03, Ilya Sherman wrote:
> nit: Please document the return value as well.

^^^

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
File components/rappor/log_uploader.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.cc:102: DiscardReason reason =
NUM_DISCARD_REASONS;
On 2014/02/13 05:11:12, Steven Holte wrote:
> On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > IMPORTANT: You should never try to log a value of NUM_DISCARD_REASONS. 
> Instead,
> > add an additional bucket for unknown failures.  I recommend making that the
> > first bucket, so that as the enum grows, that bucket doesn't randomly appear
> in
> > the middle.
> 
> We never do log NUM_DISCARD_REASONS, as that's used here to indicate we
> shouldn't discard it.  Also there isn't an unknown case, and I wouldn't expect
> there to be one, since we aren't interpreting a code from an external source
as
> a reason.  The response_code histogram has that covered. 

Ah, I see, I skimmed too quickly over the condition of the if-stmt that wraps
emitting to the histogram.  Nevermind :)

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.h:31: const std::string& mime_type,
On 2014/02/13 05:11:12, Steven Holte wrote:
> On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > Why aren't the server_url and especially the mime_type simply constants
> defined
> > in the implementation file?
> 
> Server URL at least, must be a parameter for now because we will be setting it
> through a Finch trial.  Since the log uploader does not otherwise make any
> assumptions about the format of it's payload, it would be nice to reuse it for
> other things, so that's why mime_type is a parameter.  We aren't quite able to
> reuse it for UMA, yet but that's something I would like to expand it to,
later.

Hmm, why are we setting the URL via a Finch trial?  Seems like there ought to
only be one RAPPOR server, and possibly a Finch trial to control whether RAPPOR
metrics are reported at all.  I think there's actually value to not controlling
the string via Finch, as it increases transparency to users about where exactly
their data is being reported to.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/proto...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:20: optional int32 version = 1;
On 2014/02/13 05:11:12, Steven Holte wrote:
> On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > Hmm, is the Rappor scheme expected to change?  I'm wondering if it's really
> > worth including this field yet, or if we should wait to introduce it at the
> time
> > that the scheme does change (if ever).
> 
> This was added in response to someone else's comment, but I don't feel
strongly
> either way.  We may add different types of Rappor metrics later (that may
allow
> correlations, numeric values etc.).

I don't feel strongly either, but this does seem to be purely storage overhead
right now.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_metric.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.cc:37: ByteVector fake_ones =
On 2014/02/13 05:11:12, Steven Holte wrote:
> On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > nit: I think "fake_bytes" might be a clearer name than "fake_ones".
> 
> That seems like it might imply that the bits are masked in by bytes.

fake_bits, then?

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_pref_names.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_pref_names.cc:11: // for redacting collected data.
On 2014/02/13 05:11:12, Steven Holte wrote:
> On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > IMPORTANT: Historically with UMA, we have been very cautious not to store
> > anything that looks like a unique identifier on the local client, even if it
> is
> > never transmitted via the network.  Is this client secret equivalent to a
> unique
> > identifier?  If so, we should have a deep privacy discussion about this.
> > 
> > Regardless of your answer above, please document the expected entropy of
this
> > client secret -- i.e., how many random bytes are in the string?
> 
> Well this has at least 192 bits of entropy, should probably have double that,
> and is randomly generated on the client, so it would definitely be unique. 
Not
> sure I understand the rational about not wanting to store such a value though.

> At that amount of entropy, documenting the amount of entropy seems a bit moot
> for your purposes.

Started an email thread to discuss this.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_service.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_service.h:105: base::Lock lock_;
On 2014/02/13 05:11:12, Steven Holte wrote:
> On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > IMPORTANT: Hmm, why does the map need a lock?  This class is unlikely to be
> > thread-safe.
> 
> I'm not sure what thread the Timer executes it's callback on, but if it
> shouldn't actually matter for measurements we are taking now, since
> PrefMetricsService records it's sample shortly after Start().  I'll remove the
> lock for now, since it's causing problems in the try jobs anyway.

The Timer runs on whatever thread it's instantiated on.  It's important to keep
in mind that most Chromium code is intentionally not thread-safe, to reduce
complexity.  Except in cases where performance really matters, we prefer message
passing, which copies data around between threads.  (No action to take on this
comment, just useful info for hacking on Chromium.)

https://codereview.chromium.org/49753002/diff/3770001/components/variations/m...
File components/variations/metrics_util.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/variations/m...
components/variations/metrics_util.h:16: uint32 HashName(const std::string&
name);
On 2014/02/13 05:11:12, Steven Holte wrote:
> On 2014/02/12 23:25:59, Ilya Sherman wrote:
> > nit: This method should probably be renamed to something more specific, if
it
> is
> > going to be a sibling of one called HashMetricName().
> 
> I'd rather leave that for a more focused cleanup of the variations/metrics
> dependencies.

You're welcome to split off the cleanup to a separate CL that you land first --
in fact, I'd encourage that -- but please don't land a CL that leaves this code
in a known bad state.  While I trust that your intentions are good, it's easy to
lose track of low-priority maintenance tasks, which builds up over time.  Thus,
I'm asking you to do this cleanup now, rather than punting it.

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/bloom...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/bloom...
components/rappor/bloom_filter.cc:9: // TODO(holte): we can't include "City.h"
due to type def conflicts.
nit: "type def" -> "typedef"

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/bloom...
components/rappor/bloom_filter.cc:27: // Using CityHash here because we have
support for it in Dremel.
Dremel supports lots of hash functions.  Presumably you care about speed or
something like that as well?

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:20: return
base::StringPiece(reinterpret_cast<const char *>(&lhs[0]), lhs.size());
So, reinterpret_cast is generally not safe.  The only guarantee is that if you
reinterpret_cast from type A to type B and then back to type A again, you get
the same result as you started with.  The state after casting to type B is
undefined.

In contrast, I think the explicit call to the std::string constructor on line 25
is safe.  Can you use a similar safe pattern here as well?  (FWIW, I think that
std::string safely implicitly converts to base::StringPiece -- in fact, I think
that's the whole point of base::StringPiece -- so you can safely return an
std::string from this method rather than a base::StringPiece.)

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/log_u...
File components/rappor/log_uploader_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:17: const char* kTestMimeType =
"text/plain";
nit: "const char* foo" -> "const char foo[]"

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:33: };
nit: The test harness itself should be declared outside of the anonymous
namespace.  Sorry, these rules are probably a bit esoteric-seeming, but they're
the best way to reduce symbol visibility at link time while still exporting the
symbols that need to be exported to avoid naming collisions and C++ name
mangling.

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/rappo...
components/rappor/rappor_service.cc:65: metrics_map_.clear();
nit: I think STLDeleteValues does both of these.

[Steven Holte, 2014-02-14 02:53:25]

https://codereview.chromium.org/49753002/diff/3770001/components/rappor.gypi
File components/rappor.gypi (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor.gypi#...
components/rappor.gypi:18: 'variations',
On 2014/02/13 23:23:08, Ilya Sherman wrote:
> On 2014/02/13 05:11:12, Steven Holte wrote:
> > On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > > IMO it's not appropriate for the rappor component to depend on variations.

> > > Variations is about field trials; rappor has nothing to do with field
trials
> > > really (unless I'm missing something).  I know this doesn't feel like a
big
> > deal
> > > right now, but getting this right at this time will make it easier to keep
> the
> > > component layering healthy in the long term.
> > 
> > No, it doesn't.  Unfortunately there a metrics component (yet) and it seems
> odd
> > to create one to put just MetricNameHash in it, and worse to leave it where
it
> > was and depend on a chrome/ module.  The other alternative would be to move
> that
> > function to base/metrics/, which might be reasonable.
> 
> We should move code out of //chrome/browser/metrics into //components/metrics
> anyway: [ http://crbug.com/332475 ].  While it's a little silly to have a
> component with just a single method in it, we'll need the component in the
> long-run anyway, so I don't think there's any harm to creating it now.

Done.  Though variations dependency is still required for field trial
parameters.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:126: // HMAC_DRBG Update Process
On 2014/02/13 23:23:08, Ilya Sherman wrote:
> On 2014/02/13 05:11:12, Steven Holte wrote:
> > On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > > The logic in this method is not obviously correct to me, because I'm not
> super
> > > familiar with HMACs.  I think that means it's a good idea to provide
> > additional
> > > documentation here, most likely in the form of a link to the algorithm
> > > description that this is implementing.
> > 
> > There is a link in the header file, along with some additional explanation
of
> > the assumptions this is implemented with.
> 
> Please re-link to it in the implementation file, as that's where the intricate
> logic is.

Added in several comments.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:168: // Check
max_number_of_bits_per_request from 10.1 Table 2
On 2014/02/13 23:23:08, Ilya Sherman wrote:
> On 2014/02/13 05:11:12, Steven Holte wrote:
> > On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > > nit: Please provide a link to the reference that contains 10.1 Table 2.
> > 
> > See class declaration in header file.
> 
> Please include the link at each relevant location.  Someone reading through
this
> file a year from now won't necessarily know to look in the header file for the
> reference.

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
File components/rappor/byte_vector_utils.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:23: // Merges the contents of lhs and rhs 
vectors according to a mask vector.
On 2014/02/13 23:23:08, Ilya Sherman wrote:
> On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > nit: I think it would be helpful to spell out, in English, that bit i in the
> > resulting vector is set to bit i from either the lhs or the rhs vector,
> > depending on whether bit i of the mask vector is 0 (lhs) or 1 (rhs).
> 
> ^^^

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/byte_...
components/rappor/byte_vector_utils.h:24: // Equivalent to (lhs & ~mask) | (rhs
& mask).  Stores the result in rhs.
On 2014/02/13 23:23:08, Ilya Sherman wrote:
> On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > nit: Please document the return value as well.
> 
> ^^^

Done.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
File components/rappor/log_uploader.h (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/log_u...
components/rappor/log_uploader.h:31: const std::string& mime_type,
On 2014/02/13 23:23:08, Ilya Sherman wrote:
> On 2014/02/13 05:11:12, Steven Holte wrote:
> > On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > > Why aren't the server_url and especially the mime_type simply constants
> > defined
> > > in the implementation file?
> > 
> > Server URL at least, must be a parameter for now because we will be setting
it
> > through a Finch trial.  Since the log uploader does not otherwise make any
> > assumptions about the format of it's payload, it would be nice to reuse it
for
> > other things, so that's why mime_type is a parameter.  We aren't quite able
to
> > reuse it for UMA, yet but that's something I would like to expand it to,
> later.
> 
> Hmm, why are we setting the URL via a Finch trial?  Seems like there ought to
> only be one RAPPOR server, and possibly a Finch trial to control whether
RAPPOR
> metrics are reported at all.  I think there's actually value to not
controlling
> the string via Finch, as it increases transparency to users about where
exactly
> their data is being reported to.

It gives us a little more flexibility while we are setting up the server.  Once
the server is running this can hard coded.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/proto...
File components/rappor/proto/rappor_metric.proto (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/proto...
components/rappor/proto/rappor_metric.proto:20: optional int32 version = 1;
On 2014/02/13 23:23:08, Ilya Sherman wrote:
> On 2014/02/13 05:11:12, Steven Holte wrote:
> > On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > > Hmm, is the Rappor scheme expected to change?  I'm wondering if it's
really
> > > worth including this field yet, or if we should wait to introduce it at
the
> > time
> > > that the scheme does change (if ever).
> > 
> > This was added in response to someone else's comment, but I don't feel
> strongly
> > either way.  We may add different types of Rappor metrics later (that may
> allow
> > correlations, numeric values etc.).
> 
> I don't feel strongly either, but this does seem to be purely storage overhead
> right now.

I suppose protobufs have optional fields and default values for a reason. 
Removing.

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
File components/rappor/rappor_metric.cc (right):

https://codereview.chromium.org/49753002/diff/3770001/components/rappor/rappo...
components/rappor/rappor_metric.cc:37: ByteVector fake_ones =
On 2014/02/13 23:23:08, Ilya Sherman wrote:
> On 2014/02/13 05:11:12, Steven Holte wrote:
> > On 2014/02/13 01:39:03, Ilya Sherman wrote:
> > > nit: I think "fake_bytes" might be a clearer name than "fake_ones".
> > 
> > That seems like it might imply that the bits are masked in by bytes.
> 
> fake_bits, then?

Done.

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/bloom...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/bloom...
components/rappor/bloom_filter.cc:9: // TODO(holte): we can't include "City.h"
due to type def conflicts.
On 2014/02/13 23:23:08, Ilya Sherman wrote:
> nit: "type def" -> "typedef"

Done.

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/bloom...
components/rappor/bloom_filter.cc:27: // Using CityHash here because we have
support for it in Dremel.
On 2014/02/13 23:23:08, Ilya Sherman wrote:
> Dremel supports lots of hash functions.  Presumably you care about speed or
> something like that as well?

It's pretty flexible.  Murmur, MD5, or SHA would probably also work.

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:20: return
base::StringPiece(reinterpret_cast<const char *>(&lhs[0]), lhs.size());
On 2014/02/13 23:23:08, Ilya Sherman wrote:
> So, reinterpret_cast is generally not safe.  The only guarantee is that if you
> reinterpret_cast from type A to type B and then back to type A again, you get
> the same result as you started with.  The state after casting to type B is
> undefined.
> 
> In contrast, I think the explicit call to the std::string constructor on line
25
> is safe.  Can you use a similar safe pattern here as well?  (FWIW, I think
that
> std::string safely implicitly converts to base::StringPiece -- in fact, I
think
> that's the whole point of base::StringPiece -- so you can safely return an
> std::string from this method rather than a base::StringPiece.)

The whole point of making this function was to avoid the extra copy involved
when creating a std::string, w.  From a practical standpoint, I think
reinterpeting an unsigned char* to a signed char* is pretty safe.  According to
http://stackoverflow.com/questions/15078638/can-i-turn-unsigned-char-into-cha...
the standard says that this doesn't violate strict-aliasing rules, but we aren't
gauranteed about the interpretation of the value being the "same".  If this
actually occurred, we should get test failures, at least from the
HmacByteVectorGenerator and HmacNist tests.

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/log_u...
File components/rappor/log_uploader_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:17: const char* kTestMimeType =
"text/plain";
On 2014/02/13 23:23:08, Ilya Sherman wrote:
> nit: "const char* foo" -> "const char foo[]"

Done.

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/log_u...
components/rappor/log_uploader_unittest.cc:33: };
On 2014/02/13 23:23:08, Ilya Sherman wrote:
> nit: The test harness itself should be declared outside of the anonymous
> namespace.  Sorry, these rules are probably a bit esoteric-seeming, but
they're
> the best way to reduce symbol visibility at link time while still exporting
the
> symbols that need to be exported to avoid naming collisions and C++ name
> mangling.

Done.

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/rappo...
File components/rappor/rappor_service.cc (right):

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/rappo...
components/rappor/rappor_service.cc:65: metrics_map_.clear();
On 2014/02/13 23:23:08, Ilya Sherman wrote:
> nit: I think STLDeleteValues does both of these.

So it does. Fixed.

[Ilya Sherman, 2014-02-14 05:23:08]

LGTM w/ the remaining nits addressed.  Thanks for your diligence on this code
review! :)

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/bloom...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/bloom...
components/rappor/bloom_filter.cc:27: // Using CityHash here because we have
support for it in Dremel.
On 2014/02/14 02:53:29, Steven Holte wrote:
> On 2014/02/13 23:23:08, Ilya Sherman wrote:
> > Dremel supports lots of hash functions.  Presumably you care about speed or
> > something like that as well?
> 
> It's pretty flexible.  Murmur, MD5, or SHA would probably also work.

Hmm, interesting.  I'd probably include that sentence in the documentation as
well, for extra clarity to anyone who might come along wondering if the newest,
hottest InstaHash function might be a good replacement here.

Just to appease my own curiosity, I've fired off an email to chromium-dev@
asking which hash function folks recommend.  No need to block on that thread to
resolve or anything, though :)

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/byte_...
File components/rappor/byte_vector_utils.cc (right):

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/byte_...
components/rappor/byte_vector_utils.cc:20: return
base::StringPiece(reinterpret_cast<const char *>(&lhs[0]), lhs.size());
On 2014/02/14 02:53:29, Steven Holte wrote:
> On 2014/02/13 23:23:08, Ilya Sherman wrote:
> > So, reinterpret_cast is generally not safe.  The only guarantee is that if
you
> > reinterpret_cast from type A to type B and then back to type A again, you
get
> > the same result as you started with.  The state after casting to type B is
> > undefined.
> > 
> > In contrast, I think the explicit call to the std::string constructor on
line
> 25
> > is safe.  Can you use a similar safe pattern here as well?  (FWIW, I think
> that
> > std::string safely implicitly converts to base::StringPiece -- in fact, I
> think
> > that's the whole point of base::StringPiece -- so you can safely return an
> > std::string from this method rather than a base::StringPiece.)
> 
> The whole point of making this function was to avoid the extra copy involved
> when creating a std::string, w.  From a practical standpoint, I think
> reinterpeting an unsigned char* to a signed char* is pretty safe.  According
to
>
http://stackoverflow.com/questions/15078638/can-i-turn-unsigned-char-into-cha...
> the standard says that this doesn't violate strict-aliasing rules, but we
aren't
> gauranteed about the interpretation of the value being the "same".  If this
> actually occurred, we should get test failures, at least from the
> HmacByteVectorGenerator and HmacNist tests.

Hmm, alright, I'm convinced.  Seems I need to brush up on my C++ miscellany :)

https://codereview.chromium.org/49753002/diff/4850001/components/OWNERS
File components/OWNERS (right):

https://codereview.chromium.org/49753002/diff/4850001/components/OWNERS#newco...
components/OWNERS:27: # OWNER-to-be: per-file metrics*=holte@chromium.org
Please add mpearson@, jar@, and isherman@ as owners for the metrics component as
well.

https://codereview.chromium.org/49753002/diff/4850001/components/metrics/OWNERS
File components/metrics/OWNERS (right):

https://codereview.chromium.org/49753002/diff/4850001/components/metrics/OWNE...
components/metrics/OWNERS:2: # Please include holte@ on reviews
Please add mpearson@, jar@, and isherman@ as owners for the metrics component as
well.

https://codereview.chromium.org/49753002/diff/4850001/components/metrics/metr...
File components/metrics/metrics_util.cc (right):

https://codereview.chromium.org/49753002/diff/4850001/components/metrics/metr...
components/metrics/metrics_util.cc:1: // Copyright (c) 2012 The Chromium
Authors. All rights reserved.
nit: Ditto.

https://codereview.chromium.org/49753002/diff/4850001/components/metrics/metr...
File components/metrics/metrics_util.h (right):

https://codereview.chromium.org/49753002/diff/4850001/components/metrics/metr...
components/metrics/metrics_util.h:1: // Copyright (c) 2012 The Chromium Authors.
All rights reserved.
nit: This is a new file, so "Copyright (c) 2012" -> "Copyright 2014"

https://codereview.chromium.org/49753002/diff/4850001/components/metrics/metr...
File components/metrics/metrics_util_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/4850001/components/metrics/metr...
components/metrics/metrics_util_unittest.cc:1: // Copyright (c) 2012 The
Chromium Authors. All rights reserved.
nit: Ditto.

[Steven Holte, 2014-02-14 23:01:05]

Also added components/metrics to chrome/ DEPS and chrome_common.gypi

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/bloom...
File components/rappor/bloom_filter.cc (right):

https://codereview.chromium.org/49753002/diff/4580001/components/rappor/bloom...
components/rappor/bloom_filter.cc:27: // Using CityHash here because we have
support for it in Dremel.
On 2014/02/14 05:23:09, Ilya Sherman wrote:
> On 2014/02/14 02:53:29, Steven Holte wrote:
> > On 2014/02/13 23:23:08, Ilya Sherman wrote:
> > > Dremel supports lots of hash functions.  Presumably you care about speed
or
> > > something like that as well?
> > 
> > It's pretty flexible.  Murmur, MD5, or SHA would probably also work.
> 
> Hmm, interesting.  I'd probably include that sentence in the documentation as
> well, for extra clarity to anyone who might come along wondering if the
newest,
> hottest InstaHash function might be a good replacement here.
> 
> Just to appease my own curiosity, I've fired off an email to chromium-dev@
> asking which hash function folks recommend.  No need to block on that thread
to
> resolve or anything, though :)

Done.

https://codereview.chromium.org/49753002/diff/4850001/components/OWNERS
File components/OWNERS (right):

https://codereview.chromium.org/49753002/diff/4850001/components/OWNERS#newco...
components/OWNERS:27: # OWNER-to-be: per-file metrics*=holte@chromium.org
On 2014/02/14 05:23:10, Ilya Sherman wrote:
> Please add mpearson@, jar@, and isherman@ as owners for the metrics component
as
> well.

Done.

https://codereview.chromium.org/49753002/diff/4850001/components/metrics/metr...
File components/metrics/metrics_util.cc (right):

https://codereview.chromium.org/49753002/diff/4850001/components/metrics/metr...
components/metrics/metrics_util.cc:1: // Copyright (c) 2012 The Chromium
Authors. All rights reserved.
On 2014/02/14 05:23:10, Ilya Sherman wrote:
> nit: Ditto.

Done.

https://codereview.chromium.org/49753002/diff/4850001/components/metrics/metr...
File components/metrics/metrics_util.h (right):

https://codereview.chromium.org/49753002/diff/4850001/components/metrics/metr...
components/metrics/metrics_util.h:1: // Copyright (c) 2012 The Chromium Authors.
All rights reserved.
On 2014/02/14 05:23:10, Ilya Sherman wrote:
> nit: This is a new file, so "Copyright (c) 2012" -> "Copyright 2014"

Done.

https://codereview.chromium.org/49753002/diff/4850001/components/metrics/metr...
File components/metrics/metrics_util_unittest.cc (right):

https://codereview.chromium.org/49753002/diff/4850001/components/metrics/metr...
components/metrics/metrics_util_unittest.cc:1: // Copyright (c) 2012 The
Chromium Authors. All rights reserved.
On 2014/02/14 05:23:10, Ilya Sherman wrote:
> nit: Ditto.

Done.

[edknapp, 2014-02-15 03:35:14]

LGTM for the HMAC_DRBG

[Steven Holte, 2014-02-18 20:30:53]

The CQ bit was checked by holte@chromium.org

[commit-bot: I haz the power, 2014-02-18 20:32:23]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/holte@chromium.org/49753002/5150001

[Steven Holte, 2014-02-18 20:41:16]

The CQ bit was unchecked by holte@chromium.org

[Steven Holte, 2014-02-20 20:05:48]

The CQ bit was checked by holte@chromium.org

[commit-bot: I haz the power, 2014-02-20 20:07:44]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/holte@chromium.org/49753002/5150001

[commit-bot: I haz the power, 2014-02-20 20:43:10]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-02-20 20:43:11]

Retried try job too often on android_dbg for step(s) slave_steps
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=android_db...

[Steven Holte, 2014-02-20 21:53:39]

The CQ bit was checked by holte@chromium.org

[commit-bot: I haz the power, 2014-02-20 21:58:09]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/holte@chromium.org/49753002/5640001

[commit-bot: I haz the power, 2014-02-20 22:10:12]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-02-20 22:10:14]

Retried try job too often on win_rel for step(s) base_unittests, browser_tests,
interactive_ui_tests, net_unittests, unit_tests
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=win_rel&nu...

[Steven Holte, 2014-02-20 22:24:48]

The CQ bit was checked by holte@chromium.org

[commit-bot: I haz the power, 2014-02-20 22:27:14]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/holte@chromium.org/49753002/5640001

[commit-bot: I haz the power, 2014-02-21 04:06:16]

Change committed as 252492