Add custom action support

components/arc/common/accessibility_helper.mojom

 // Copyright 2017 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
-// Next MinVersion: 3
+// Next MinVersion: 4
 
 module arc.mojom;
 
 import "screen_rect.mojom";
 
 // For future maintainers, each of the below enums were hand picked
 // from their equivalents in the Android source. Keep them in the
 // order given below and add as needed. The initial order matches the
 // order they appear in source files.
 
@@ skipping 46 matching lines @@
   PREVIOUS_HTML_ELEMENT,  // unused
   SCROLL_FORWARD,
   SCROLL_BACKWARD,
   COPY,
   PASTE,
   CUT,
   SET_SELECTION,
   EXPAND,
   COLLAPSE,
   DISMISS,
-  SET_TEXT
+  SET_TEXT,
+  CONTEXT_CLICK,
+  SCROLL_DOWN,
+  SCROLL_LEFT,
+  SCROLL_RIGHT,
+  SCROLL_TO_POSITION,
+  SCROLL_UP,
+  SET_PROGRESS,
+  SHOW_ON_SCREEN
 };
 
 // Possible boolean properties set on an AccessibilityNodeInfo.
 // https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo.html
 // The enum values appear in the same order as they do within
 // AccessibilityNodeInfo.java.
 [Extensible]
 enum AccessibilityBooleanProperty {
   CHECKABLE,
   CHECKED,
@@ skipping 40 matching lines @@
   TEXT_SELECTION_END,
   INPUT_TYPE,
   LIVE_REGION
 };
 
 // These fields are taken from List<int> instance members of
 // AccessibilityNodeInfo.
 [Extensible]
 enum AccessibilityIntListProperty {
   CHILD_NODE_IDS,
-  ACTIONS
+  ACTIONS,

[David Tseng, 2017/06/08 17:21:59]

This doesn't appear to be used. It might be safe to just rename it to
custom_action_ids.

[yawano, 2017/06/13 06:58:47]

Done.

+  CUSTOM_ACTION_IDS
+};
+
+[Extensible]
+enum AccessibilityStringListProperty {
+  CUSTOM_ACTION_DESCRIPTIONS
 };
 
 // AccessibilityNodeInfoData is a struct to contain info of
 // AccessibilityNodeInfo in Android.
 // https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo.html
 struct AccessibilityNodeInfoData {
   ScreenRect bounds_in_screen;
   [MinVersion=1]int32 id;
   [MinVersion=1]map<AccessibilityBooleanProperty, bool>? boolean_properties;
   [MinVersion=1]map<AccessibilityStringProperty, string>? string_properties;
   [MinVersion=1]map<AccessibilityIntProperty, int32>? int_properties;
   [MinVersion=1]
       map<AccessibilityIntListProperty, array<int32>>? int_list_properties;
+  [MinVersion=3]map<AccessibilityStringListProperty, array<string>>? string_list_properties;
 };
 
 // Filters the event type (and implicitly the data) sent by the ARC
 // accessibility service.
 [Extensible]
 enum AccessibilityFilterType {
   // No events will be sent.
   OFF,
 
   // Only send focus events along with the source focus node.
@@ skipping 18 matching lines @@
 // Next method ID: 2
 interface AccessibilityHelperHost {
   OnAccessibilityEventDeprecated@0(AccessibilityEventType event_type,
                                    AccessibilityNodeInfoData? event_source);
 
   // OnAccessibilityEvent is called when a converted Android accessibility event
   // is sent from Android.
   OnAccessibilityEvent@1(AccessibilityEventData event_data);
 };
 
-// Next method ID: 3
+// Next method ID: 4
 interface AccessibilityHelperInstance {
   Init@0(AccessibilityHelperHost host);
 
   // Perform the specified action on a node requested by a Chrome client.
   PerformAction@1(int32 id, AccessibilityActionType action);
 
   // Set a filter on the event types received.
   SetFilter@2(AccessibilityFilterType filter_type);
+
+  // Perform a custom action on a node requested by a Chrome client.
+  [MinVersion=3]PerformCustomAction@3(int32 id, int32 custom_action_id);

[David Tseng, 2017/06/08 17:21:59]

Considering we might want just one action list in the future, wouldn't it make
sense to try merging this with performAction (i.e. make performAction take an
ActionData)? On the Android end, therer isn't a distinction between a standard
action vs a custom action.

[yawano, 2017/06/09 08:03:01]

I would prefer to keep standard action and custom action separated in our mojo
interface. In Android, they are sharing ids with standard actions and custom
actions. But I'm afraid to share them in our mojo interface as well. It would be
difficult to keep maintaining our standard action ids in mojom file are synced
with those in Android framework.

If we need to manage those ids separately, we need to send them in the separated
int list from Android side to Chrome side. We will need to send descriptions of
standard actions as well in the future. Android supports to override standard
actions.

If we manage standard actions and custom actions separately for sending from
Android to Chrome, I think it makes sense to keep them separated in the other
way as well.

WDYT?

[David Tseng, 2017/06/09 18:01:32]

I should be more clear. I was thinking:
performAction(ActionData data);

// Passed to |performAction|. A caller must specify either standard_action_id or
custom_action_id.
struct ActionData {
  AccessibilityActionType standard_action_id;

  // Optional.
   int32 custom_id;

  ... (we can add additional args here).
};

[yawano, 2017/06/12 01:46:52]

My point was that it would make sense to keep standard actions and custom
actions separated if we treat them separated when sending them from Android to
Chrome side. It would be symmetric.

The pro of adding ActionData struct is that it makes easy to extend the API
later. Adding a field to the struct is much easier than changing the interface
of the method.

After a thought, I will change this to use ActionData. Some standard actions
take arguments, e.g. set text, text selection. We will need to make this
extensible in the future. This should be a good time to address the issue in
advance.

[David Tseng, 2017/06/12 16:55:01]

Totally understood. However, the automation api packs up actions into one data
structure and has one internal binding for it
(AutomationInternal::performAction). 
Yup; I was thinking of custom actions as a "standard" action with extra
arguments of action id. That could be one way to unify standard and custom --
add a custom action type to the AccessibilityActionType in the mojom as well as
in ax_enums.idl.

[yawano, 2017/06/13 06:58:47]

Changed PerformAction to take AccessibilityActionData.

 };