Protobuf for Traffic Annotation and first use by a URLFetcher.

tools/traffic_annotation/traffic_annotation.proto

+syntax = "proto3";
+package traffic_annotation;
+
+// TODO: include policies
+
+// Describes a specific kind of network traffic based on a fine-grained
+// semantic classification of all network traffic generated by Chrome.
+// Used for auditing purposes.
+message NetworkTrafficAnnotation {
+  // This is a globally unique identifier that must stay unchanged while the
+  // network request carries the same semantic meaning. If the network request
+  // gets a new meaning, this ID needs to be changed.
+  // The purpose of this ID is to give humans a chance to reference
+  // NetworkTrafficAnnotations externally even when those change a little bit
+  // (e.g. adding a new piece of data that is sent along with a network
+  // request).
+  // IDs of one component should have a shared prefix so that sorting all
+  // NetworkTrafficAnnotations by unique_id groups those that belong to the same
+  // component together.
+  // For example:
+  //   "spellchecker_lookup"
+  string unique_id = 1;
+
+  // Encapsulates information about the code location that generates this kind
+  // of network traffic.
+  message TrafficSource {
+    // File name where the network request is triggered.
+    // This is typically filled by the extractor and does not need to be
+    // specified in the source code. For manual whitelisting this needs to be
+    // specified.
+    string file = 1;
+
+    // Function name where the network request is instantiated.
+    // This is typically filled by the extractor and does not need to be
+    // specified in the source code. For manual whitelisting this needs to be
+    // specified.
+    string function = 2;
+
+    // __LINE__ in file, where the AuditPolicy object is instantiated.
+    // This is typically filled by the extractor and does not need to be
+    // specified in the source code.
+    // For whitelisted network requests in third_party/ that cannot be properly
+    // annotated in the source code, this attribute is empty.
+    int32 line = 3;
+
+    // For whitelisted network requests in third_party/ that cannot be properly
+    // annotated in the source code, this distinguishes between the first,
+    // second, ... annotated call.
+    // For annotations in the source code, this is not used because the line
+    // attribute uniquely identifies the network request.
+    int32 call_number = 4;
+  }
+
+  TrafficSource source = 2;
+
+  // Meta information about the network request.
+  message TrafficSemantics {
+    // Justification for an empty AuditPolicy policy.
+    // Typically this can be either a TODO or a hint that the annotation is
+    // made upstream in the code. For example, if net::URLFetcher::Create() has
+    // has annotation, the net::TCPClientSocket() that is used by the

[battre, 2017/01/19 08:10:15]

nit: has an annotation

[Ramin Halavati, 2017/01/24 08:49:38]

Done.

+    // URLFetcher does not need to be annotated as well.
+    string empty_policy_justification = 1;
+
+    // What component triggers the request. The components should be human
+    // readable and don’t need to reflect the components/ directory. Avoid
+    // abbreviations.
+    // Examples: spellchecker, component updater, website
+    string sender = 2;
+
+    // Plaintext description of the network request in language that is
+    // understandable by admins (ideally also users). Please avoid acronyms.
+    // Please describe the feature and the feature's value proposition as well.
+    // Examples:
+    // - Google Chrome can provide smarter spell-checking by sending text you
+    //   type into the browser to Google's servers, allowing you to use the same
+    //   spell-checking technology used by Google products, such as Docs.
+    //   If the feature is enabled, Chrome will send the entire contents of text
+    //   fields as you type in them to Google along with the browser’s default
+    //   language. Google returns a list of suggested spellings, which will be
+    //   displayed in the context menu.
+    // - A network request that comes from web content (a page the user visits)
+    string description = 3;
+
+    // What triggered the network request. Use a textual description. This
+    // should be a human readable string.
+    // For things that are clearly part of the website (resource load, form
+    // submission, fetch by a service worker,...), you *may* just put “website”
+    // here.
+    string trigger = 4;
+
+    // What nature of data is being sent. This should be a human readable
+    // string. Any user data and/or PII should be pointed out.
+    // Examples: “log files from /var/...”, “statistics about foobar”, “the
+    // signature of a form of a website”, “installed extensions and their
+    // version”, “a word on a website the user tapped on”
+    string data = 5;
+
+    enum Destination {
+      // A website the user visits (this may be google.com) or interacts with.
+      // For example search engines of the omnibar (also the default search
+      // engine) would be considered websites.

[battre, 2017/01/19 08:10:15]

I would change this:

A website the user visit or interacts with. The distinction from a google owned
service can be difficult when the user navigates to google.com or searches with
the omnibar. Therefore follow the following guideline: If the source code has
hardcoded that the request goes to Google (e.g. for ZeroSuggest), use
GOOGLE_OWNED_SERVICE. If the request can go to other domains and is perceived as
a part of a website rather than a native browser feature, use WEBSITE. In other
cases use OTHER.  

[Ramin Halavati, 2017/01/24 08:49:38]

Done.

+      WEBSITE = 0;
+      // A Google owned service, like SafeBrowsing, spellchecking, ...
+      GOOGLE_OWNED_SERVICE = 1;
+      // Other endpoints, e.g. a service hosting a PAC script

[battre, 2017/01/19 08:10:15]

Other endpoints, e.g. a service hosting a PAC script. In case of doubt, use this
category. We will audit it in the future to see whether we need more categories.

OTHER = 1000;

[Ramin Halavati, 2017/01/24 08:49:38]

Done.

+      OTHER = 2;
+    }
+    Destination destination = 6;
+
+    // Human readable description in case the destination points to OTHER.
+    string destination_other = 7;
+  }
+
+  TrafficSemantics semantics = 3;
+
+  message TrafficPolicy {
+    // Whether cookies/channel IDs/... can be sent or saved (use true if at
+    // least one is correct).
+    bool cookies_allowed = 1;
+
+    // If cookies_allowed is true and the request uses not the profile cookie
+    // store, please specify this here. You may use “system” to indicate that
+    // the System RequestContext and its cookie store are used or specify other
+    // exceptions (e.g. SafeBrowsing uses a separate cookie store).
+    string cookies_store_exceptions = 2;
+
+    // Human readable description of how to enable/disable a feature that
+    // triggers this network request by a user. Use “NA”, if no such setting
+    // exists (e.g. “Disable ‘Use a web service to help resolve spelling
+    // errors.’ in Chrome’s settings under Advanced”).
+    string setting = 3;
+
+    // Example policy configuration that disables this network request.
+    // This would be a text serialized protobuf of any enterprise policy.
+    // see out/Debug/gen/components/policy/cloud_policy.proto
+    repeated string policy = 4;
+    // TODO: repeated enterprise_management.CloudPolicySettings policy = 4;

[battre, 2017/01/19 08:10:15]

Please try to fix this.

[Ramin Halavati, 2017/01/24 08:49:38]

Done.

+
+    // Justification for not having a policy that disables this feature.
+    string policy_exception_justification = 5;
+  }
+
+  TrafficPolicy policy = 4;
+};
+
+// NetworkTrafficAnnotations that were extracted from the source code.
+message ExtractedNetworkTrafficAnnotation {
+  repeated NetworkTrafficAnnotation network_traffic_annotation = 1;
+};
+
+// NetworkTrafficAnnotations that had to go into a whitelist file because the
+// source code could not be annotated (e.g. because it is in a third-party
+// library).
+message WhitelistedNetworkTrafficAnnotations {
+  repeated NetworkTrafficAnnotation network_traffic_annotation = 1;
+};
+
+// All NetworkTrafficAnnotations from a Chromium configuration.
+message NetworkTrafficAnnotations {
+  ExtractedNetworkTrafficAnnotation extracted_network_traffic_annotations = 1;
+  WhitelistedNetworkTrafficAnnotations whitelisted_network_traffic_annotations =
+      2;
+};