DBus MediaAnalyticsClient and media_perception pb.

chromeos/dbus/proto/media_perception.proto

Index: chromeos/dbus/proto/media_perception.proto
diff --git a/chromeos/dbus/proto/media_perception.proto b/chromeos/dbus/proto/media_perception.proto
new file mode 100644
index 0000000000000000000000000000000000000000..05197ccad710b23e6b45fda6691023f4bbf05146
--- /dev/null
+++ b/chromeos/dbus/proto/media_perception.proto
@@ -0,0 +1,131 @@
+syntax = "proto2";
+
+option optimize_for = LITE_RUNTIME;
+
+package mri;
+
+// The output of a Drishti media perception graph. Implicitly tied to the
+// MediaPerception dictionary defined in Chromium source at
+// src/extensions/common/api/media_perception_private.idl for the
+// Chromium mediaPerceptionPrivate API. A duplicate copy of this proto also
+// exists in Chromium for the mediaPerceptionPrivate API implementation to

[tbarzic, 2017/05/05 21:10:42]

is this still valid statement?

I'd prefer to have a single copy of the proto :)

[Luke Sorenson, 2017/05/08 19:06:08]

Edited the text a little. To be clear there is just one copy of this proto in
the Chromium source. The reason I refer to this as a duplicate copy is because
of the one that exists with and is defined as part of the media analytics
process outside of Chrome.

[tbarzic, 2017/05/08 23:04:55]

Oh.
Can you please reword this a bit further. Someone reading this file only would
get very confused (like I got).
Maybe say that the proto has two copies - one being here - that have to be
updated together (rather than saying the proto has a copy in Chromium codebase,
even though this is that copy).

also, are we OK with refering to "dristhi" in open source code?

[Luke Sorenson, 2017/05/09 17:39:45]

Done.

Also removed references to "Drishti" since it's not necessary to have them.

+// deserialize this proto.
+//
+// The duplicate copy of this proto lives in Chromium source here:
+// src/chromeos/media_perception/media_perception.proto
+// If you change this file, you need to make sure to update this file in the
+// Chromium source tree as well.
+//
+// This message is packaged by the Drishti graph runner when a PerceptionSample
+// or array of PerceptionSamples comes out of the graph.
+message MediaPerception {
+  // The timestamp attached when this data originated from the analysis process.
+  optional uint64 timestamp = 1;
+
+  // A single FramePerception message or array of perceptions (if reporting the
+  // results from multiple frames).
+  repeated FramePerception frame_perception = 2;
+}
+
+// Used to transmit a history of image frames and their associated annotations.
+// This is accumulated over time by the Drishti graph runner.
+message Diagnostics {
+  repeated PerceptionSample perception_sample = 1;
+}
+
+message State {
+  enum Status {
+    TIMEOUT = 0;        // Unable to reach media analysis process.
+    UNINITIALIZED = 1;  // Media analytics working on loading configuration.
+    STARTED = 2;        // Analysis process running but not recieving frames.
+    RUNNING = 3;        // Analysis process running and injesting frames.
+    // Media analytics process waiting to be started.
+    SUSPENDED = 4;
+  }
+
+  // Note: RUNNING and SUSPENDED are the only two states which should be sent to
+  // SetState.
+  optional Status status = 1;
+
+  // Device context so that the media analytics process can better select the
+  // right video device to open.
+  optional string device_context = 2;
+}
+
+// This is the output of the MediaPerceptionSinkCalculator.
+message PerceptionSample {
+  optional FramePerception frame_perception = 1;
+  // The image frame data associated with the frame perception.
+  optional RawImageFrame raw_image_frame = 2;
+}
+
+// Note: this is a replica of image/content/flow/image/raw_image_data.proto
+// because this proto needs to be self-contained (to be checked in to Chromium
+// as well).
+message RawImageFrame {
+  optional int32 width = 1;
+  optional int32 height = 2;
+  // colorspace is defined in the same way as SimpleImage::ColorSpace.
+  optional int32 colorspace = 3;
+  // By default, 1 channel means Grayscale, 2 channels meangs Grayscale + Alpha,
+  // 3 channels means RGB, and 4 channels means RGBA.
+  optional int32 channels = 4;
+  // The raw pixel data as a string of uint8.
+  // The size of pixel_data is height*width*channels.
+  // Byte order is RGBARGBARGBA.
+  // TODO(lasoren): Replace with compressed image format.
+  optional bytes pixel_data = 5;
+}
+
+// The set of computer vision metadata for an image frame.
+message FramePerception {
+  optional uint64 frame_id = 1;
+
+  optional uint32 frame_width_in_px = 2;
+  optional uint32 frame_height_in_px = 3;
+
+  // The timestamp associated with the frame (when it enters the Drishti
+  // graph).
+  optional uint64 timestamp = 4;
+
+  // The list of entities detected for this frame.
+  repeated Entity entity = 5;
+}
+
+message Entity {
+  // A unique id associated with the detected entity, which can be used to track
+  // the entity over time.
+  optional uint32 id = 1;
+
+  enum EntityType {
+    UNSPECIFIED = 0;
+    FACE = 1;
+    PERSON = 2;
+  }
+
+  optional EntityType type = 2;
+
+  // Minimum box, which captures entire detected entity.
+  optional BoundingBox bounding_box = 3;
+
+  // A value for the quality of this detection.
+  optional float confidence = 4;
+}
+
+message BoundingBox {
+  // The points that define the corners of a bounding box.
+  optional Point top_left = 1;
+  optional Point bottom_right = 2;
+  // Indicates whether or not these coordinates are normalized to values between
+  // 0 and 1.
+  optional bool normalized = 3 [default = false];
+}
+
+message Point {
+  // x represents the horizontal distance from the top left corner of the image
+  // to the point.
+  optional float x = 1;
+  // y represents the vertical distance from the top left corner of the image to
+  // the point.
+  optional float y = 2;
+}