Protobuf for Traffic Annotation and first use by a URLFetcher.

tools/traffic_annotation/traffic_annotation.proto

Index: tools/traffic_annotation/traffic_annotation.proto
diff --git a/tools/traffic_annotation/traffic_annotation.proto b/tools/traffic_annotation/traffic_annotation.proto
new file mode 100644
index 0000000000000000000000000000000000000000..9a190e92edb5132b102987dc4251574a6108942c
--- /dev/null
+++ b/tools/traffic_annotation/traffic_annotation.proto
@@ -0,0 +1,184 @@
+syntax = "proto3";
+package traffic_annotation;
+
+// TODO: include policies
+
+// Describes a specific kind of network traffic based on a fine-grained
+// semantic classification of all network traffic generated by Chrome.
+// Used for auditing purposes.
+message NetworkTrafficAnnotation {
+  //////////////////////////////////////////
+  // Section 1: Location of what is audited.

[battre, 2016/10/18 09:05:28]

please delete these lines.

[Ramin Halavati, 2016/10/18 10:03:22]

Done.

+
+  // Description of the call site that is audited.
+  // An annotation is required for calls from net/ (or go/cronet)
+  // 1) every call of net::URLFetcher::Create()
+  // 2) every call of net::URLRequestContext::CreateRequest()
+  // 3) every call of net::TCPClientSocket()
+  // 4) every call of net::UDPClientSocket()
+  // 5) every call of net::SSLClientSocket()
+  // 6) every call of implementations of
+  //       net::ClientSocketFactory::Create.*ClientSocket
+  // 7) every call of ResourceFetcher::Create() or
+  //    WebURLRequest::setRequestContext(
+  //        WebURLRequest::RequestContextInternal)
+  //    TODO: figure out how to deal with requests from the renderer
+  // TODO: investigate ClientSocketPoolManager.
+  //
+  // An annotation is required for calls via native sockets.
+  // A) every call of ::bind()
+  //
+  // For 1-6, the annotation needs to be assigned to the respective object
+  // inline in the same function that triggers the call. If a function
+  // contains K calls to functions 1-6, there need to be K annotations in
+  // that function accordingly.
+  // For A, it may be impossible to add the annotation inline, because
+  // the call happens in a third_party library. In this case, the annotation
+  // needs to be put into a special whitelist file.

[battre, 2016/10/18 09:05:28]

Please delete this block.

[Ramin Halavati, 2016/10/18 10:03:22]

Done.

+
+  // This is a globally unique identifier that must stay unchanged while the
+  // network request carries the same semantic meaning. If the network request
+  // gets a new meaning, this ID needs to be changed.
+  // The purpose of this ID is to give humans a chance to annotate specific
+  // audit policies and keep the annotation assigned to the policy even when

[battre, 2016/10/18 09:05:29]

The purpose of this ID is to give humans a chance to reference
NetworkTrafficAnnotations externally even when those change a little bit (...).

[Ramin Halavati, 2016/10/18 10:03:22]

Done.

+  // that changes a little bit (e.g. adding a new piece of data that is sent
+  // along with a network request).
+  // IDs of one component should have a shared prefix so that sorting all
+  // network requests by unique_id groups those policies that belong to the

[battre, 2016/10/18 09:05:28]

so that sorting all NetworkTrafficAnnotations by unique_id groups those that
belong to the same component together.

[Ramin Halavati, 2016/10/18 10:03:22]

Done.

+  // same component together.
+  // For example:
+  //   "spellchecker_lookup"
+  string unique_id = 1;
+
+  // Encapsulates information about the code location that generates this kind
+  // of
+  // network traffic.

[battre, 2016/10/18 09:05:28]

please fix line wrapping. Also below.

[Ramin Halavati, 2016/10/18 10:03:22]

Done.

+  message TrafficSource {
+    // File name where the network request is triggered.
+    // This is typically filled by the extractor and does not need to be
+    // specified
+    // in the source code. For manual whitelisting this needs to be specified.
+    string file = 1;
+
+    // Function name where the network request is instantiated.
+    // This is typically filled by the extractor and does not need to be
+    // specified
+    // in the source code. For manual whitelisting this needs to be specified.
+    string function = 2;
+
+    // __LINE__ in file, where the AuditPolicy object is instantiated.
+    // This is typically filled by the extractor and does not need to be
+    // specified
+    // in the source code.
+    // For whitelisted network requests in third_party that cannot be properly

[battre, 2016/10/18 09:05:28]

third_party/

[Ramin Halavati, 2016/10/18 10:03:22]

Done.

+    // annotated in the source code, this attribute is empty.
+    int32 line = 3;
+
+    // For whitelisted network requests in third_party that cannot be properly

[battre, 2016/10/18 09:05:28]

third_party/

[Ramin Halavati, 2016/10/18 10:03:22]

Done.

+    // annotated in the source code, this distinguishes between the first,
+    // second, … annotated call.

[battre, 2016/10/18 09:05:28]

please replace ... with three proper dots.

[Ramin Halavati, 2016/10/18 10:03:22]

Done.

+    // For annotations in the source code, this is not used because the line
+    // attribute uniquely identifies the network request.
+    int32 call_number = 4;
+  }
+
+  TrafficSource source = 2;
+
+  /////////////////////////////////////////////////////////
+  // Section 2: Meta information about the network request.

[battre, 2016/10/18 09:05:28]

Replace this just with // Meta information about the network request.

[Ramin Halavati, 2016/10/18 10:03:22]

Done.

+  message TrafficSemantics {
+    // Justification for an empty AuditPolicy policy.
+    // Typically this can be either a TODO or a hint that the annotation is
+    // made upstream in the code. For example, if net::URLFetcher::Create() has
+    // has annotation, the net::TCPClientSocket() that is used by the
+    // URLFetcher does not need to be annotated as well.
+    string empty_policy_justification = 1;
+
+    // What component triggers the request. The components should be human
+    // readable and don’t need to reflect the components/ directory. Avoid
+    // abbreviations.
+    // Examples: spellchecker, component updater, website
+    string sender = 2;
+
+    // Plaintext description of the network request in language that is
+    // understandable by admins (ideally also users). Please avoid acronyms.

[battre, 2016/10/18 09:05:28]

Please describe the feature and the feature's value proposition as well.

[Ramin Halavati, 2016/10/18 10:03:22]

Done.

+    // Examples:
+    // - “Sends a sequence of three consecutive words from a user input field
+    //   to Google’s online spell checker.”

[battre, 2016/10/18 09:05:28]

Please copy the text from the spell checker annotation that is also in this CL.

[battre, 2016/10/18 09:05:28]

please remove typographic quotes (here and elsewhere)

[Ramin Halavati, 2016/10/18 10:03:22]

Done.

[Ramin Halavati, 2016/10/18 10:03:22]

Done.

+    // - “A network request that comes from web content (a page the user
+    //   visits)”
+    string description = 3;
+
+    // What triggered the network request. Use a textual description. This
+    // should be a human readable string.
+    // For things that are clearly part of the website (resource load, form
+    // submission, fetch by a service worker,...), you *may* just put “website”
+    // here.
+    string trigger = 4;
+
+    // What nature of data is being sent. This should be a human readable
+    // string. Any user data and/or PII should be pointed out.
+    // Examples: “log files from /var/...”, “statistics about foobar”, “the
+    // signature of a form of a website”, “installed extensions and their
+    // version”, “a word on a website the user tapped on”
+    string data = 5;
+
+    enum Destination {
+      // A website the user visits (this may be google.com) or interacts with.
+      // For example search engines of the omnibar (also the default search
+      // engine) would be considered websites.
+      WEBSITE = 0;
+      // A Google owned service, like SafeBrowsing, spellchecking, ...
+      GOOGLE_OWNED_SERVICE = 1;
+      // Other endpoints, e.g. a service hosting a PAC script
+      OTHER = 2;
+    }
+    Destination destination = 6;
+    // Human readable description in case the destination points to OTHER.

[battre, 2016/10/18 09:05:28]

New line before this.

[Ramin Halavati, 2016/10/18 10:03:22]

Done.

+    string destination_other = 7;
+  }
+
+  TrafficSemantics semantics = 3;
+
+  message TrafficPolicy {
+    // Whether cookies/channel IDs/... can be sent or saved (use true if at
+    // least one is correct).
+    bool cookies_allowed = 1;
+
+    // If cookies_allowed is true and the request uses not the profile cookie
+    // store, please specify this here. You may use “system” to indicate that
+    // the System RequestContext and its cookie store are used or specify other
+    // exceptions (e.g. SafeBrowsing uses a separate cookie store).
+    string cookies_store_exceptions = 2;
+
+    // Human readable description of how to enable/disable a feature that
+    // triggers this network request by a user. Use “NA”, if no such setting
+    // exists (e.g. “Disable ‘Use a web service to help resolve spelling
+    // errors.’ in Chrome’s settings under Advanced”).
+    string setting = 3;
+
+    // Example policy configuration that disables this network request.
+    // This would be a text serialized protobuf of any enterprise policy.
+    // see out/Debug/gen/components/policy/cloud_policy.proto
+    repeated string policy = 4;
+    // TODO: repeated enterprise_management.CloudPolicySettings policy = 4;
+  }
+
+  TrafficPolicy policy = 4;
+
+  // Justification for not having a policy that disables this feature.
+  string policy_exception_justification = 5;

[battre, 2016/10/18 09:05:28]

Please move this into TrafficPolicy.

[Ramin Halavati, 2016/10/18 10:03:22]

Done.

+};
+
+message ExtractedAuditPolicies {

[battre, 2016/10/18 09:05:28]

ExtractedNetworkTrafficAnnotation (please fix the messagese / variables below as
well).

Please add some comments.

[Ramin Halavati, 2016/10/18 10:03:22]

Done.

+  repeated AuditPolicies audit_policies = 1;
+};
+
+message WhitelistedAuditPolicies {
+  repeated AuditPolicies audit_policies = 1;
+};
+
+message AuditPolicies {
+  ExtractedAuditPolicies extracted_audit_policies = 1;
+  WhitelistedAuditPolicies whitelisted_audit_policies = 2;
+};