Implement SmartText selection.

content/public/android/java/src/org/chromium/content/browser/SelectionPopupController.java

 // Copyright 2015 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 package org.chromium.content.browser;
 
 import android.annotation.TargetApi;
 import android.app.Activity;
 import android.app.SearchManager;
 import android.content.ClipboardManager;
@@ skipping 36 matching lines @@
  * A class that handles input-related web content selection UI like action mode
  * and paste popup view. It wraps an {@link ActionMode} created by the associated view,
  * providing modified interaction with it.
  *
  * Embedders can use {@link ActionModeCallbackHelper} implemented by this class
  * to create {@link ActionMode.Callback} instance and configure the selection action
  * mode tasks to their requirements.
  */
 @TargetApi(Build.VERSION_CODES.M)
 public class SelectionPopupController extends ActionModeCallbackHelper {
-    private static final String TAG = "cr.SelectionPopCtlr";  // 20 char limit
+    private static final String TAG = "SelectionPopupCtlr"; // 20 char limit
 
     /**
      * Android Intent size limitations prevent sending over a megabyte of data. Limit
      * query lengths to 100kB because other things may be added to the Intent.
      */
     private static final int MAX_SHARE_QUERY_LENGTH = 100000;
 
     // Default delay for reshowing the {@link ActionMode} after it has been
     // hidden. This avoids flickering issues if there are trailing rect
     // invalidations after the ActionMode is shown. For example, after the user
     // stops dragging a selection handle, in turn showing the ActionMode, the
     // selection change response will be asynchronous. 300ms should accomodate
     // most such trailing, async delays.
     private static final int SHOW_DELAY_MS = 300;
 
+    // Display operation for ActionMode (show or invalidate) that is delayed because
+    // the selected text needs to be classified before. See comment to mPostponedDisplayOp.
+    private static final int POSTPONED_NONE = 0;
+    private static final int POSTPONED_SHOW = 1;
+    private static final int POSTPONED_INVALIDATE = 2;
+
     private final Context mContext;
     private final WindowAndroid mWindowAndroid;
     private final WebContents mWebContents;
     private final RenderCoordinates mRenderCoordinates;
     private final ImeAdapter mImeAdapter;
     private ActionMode.Callback mCallback;
 
     // Selection rectangle in DIP.
     private final Rect mSelectionRect = new Rect();
 
@@ skipping 25 matching lines @@
     private boolean mHasSelection;
 
     // Lazily created paste popup menu, triggered either via long press in an
     // editable region or from tapping the insertion handle.
     private PastePopupMenu mPastePopupMenu;
     private boolean mWasPastePopupShowingOnInsertionDragStart;
 
     // The client that processes textual selection, or null if none exists.
     private SelectionClient mSelectionClient;
 
+    // The classificaton result of the selected text or null if ContextSelectionProvider
+    // could not classify the selection.

[boliu, 2017/03/20 20:27:26]

what's the state of this if there's no selection?

[Tima Vaisburd, 2017/03/21 02:07:02]

Null. I rewrote as
"The classificaton result of the selected text if the selection exists and
ContextSelectionProvider was able to classify it, otherwise null."

+    private ContextSelectionProvider.Result mClassificationResult;
+
+    // Postponed display operation for ActionMode.
+    // TODO(timav): explain better.

[boliu, 2017/03/20 20:27:26]

later when?

[Tima Vaisburd, 2017/03/21 02:07:02]

Added an explanation.

+    private int mPostponedDisplayOp;
+
+    // The callback object that delivers result from SelectionClient.
+    private ContextSelectionProvider.ResultCallback mSelectonCallback =

[amaralp, 2017/03/21 02:34:14]

Why does this need to be a member of the class? Why not instantiate it directly
when calling ContextSelectionClient.create()?

[Tima Vaisburd, 2017/03/21 20:50:12]

No good reason.
Added a private class ContextSelectionCallback and did just that.

+            new ContextSelectionProvider.ResultCallback() {
+                @Override
+                public void onClassified(ContextSelectionProvider.Result result) {
+                    Log.v(TAG, "onClassified");

[boliu, 2017/03/20 20:27:26]

.

[Tima Vaisburd, 2017/03/21 02:07:02]

No more.

+
+                    // If the selection does not exist any more, discard |result|.
+                    if (!mHasSelection) {
+                        mPostponedDisplayOp = POSTPONED_NONE;
+                        return;
+                    }
+
+                    // Update the selection range if needed.
+                    if (!(result.startAdjust == 0 && result.endAdjust == 0)) {
+                        // This call causes SELECTION_HANDLES_MOVED event
+                        mWebContents.adjustSelectionByCharacterOffset(
+                                result.startAdjust, result.endAdjust);
+                    }
+
+                    final boolean hadPriorResult = mClassificationResult != null;
+                    mClassificationResult = result.hasNamedAction() ? result : null;
+
+                    // For simplicity always update the menu if we need to show assist item.
+                    // Also we need to update the menu if the assist item is going to disappear.
+                    if (mClassificationResult != null || hadPriorResult) mNeedsPrepare = true;
+
+                    switch (mPostponedDisplayOp) {
+                        case POSTPONED_NONE:
+                            break;
+
+                        case POSTPONED_SHOW:
+                            // TODO(timav): if the |result| contained the selection adjustment and

[amaralp, 2017/03/21 02:34:14]

One solution for the flickering could be to have the original (un-extended)
selection never appear. Instead of triggering the ContextSelectionClient on
|SELECTION_HANDLE_SHOWN| you could do it in Blink before the selection is made
and prevent the intermediate selection. What I'm thinking is that whenever a
touch selection occurs in Blink it could send an IPC to the browser with the
selected text and context. The browser figures out what more to select and sends
an IPC back to Blink.
This would also have the added benefit of not having to postpone the action
mode.

[Tima Vaisburd, 2017/03/21 20:50:12]

I like that idea although I receive a feedback that the next feature requirement
could be an animation of the selection expansion.

I thought, however, that for the IPC I either have to use the existing one or
change both this and contextual search to use the same mechanism. I wanted to
avoid the latter. Can we have this kind of improvement in later CLs?
 
I think I still need to postpone invalidate() after user stops the drag?

+                            // we called adjustSelectionByCharacterOffset() above, there is flicker
+                            // if SELECTION_HANDLES_MOVED comes later and invalidates selection
+                            // rectangle. Consider postponing till SELECTION_HANDLES_MOVED or
+                            // introduce delay.
+                            showActionMode();
+                            mPostponedDisplayOp = POSTPONED_NONE;
+                            break;
+
+                        case POSTPONED_INVALIDATE:
+                            invalidateActionMode();
+                            mPostponedDisplayOp = POSTPONED_NONE;
+                            break;
+
+                        default:
+                            assert false : "Invalid postponed display operation type.";
+                            break;
+                    }
+                }
+            };
+
     /**
      * Create {@link SelectionPopupController} instance.
      * @param context Context for action mode.
      * @param window WindowAndroid instance.
      * @param webContents WebContents instance.
      * @param view Container view.
      * @param renderCoordinates Coordinates info used to position elements.
      * @param imeAdapter ImeAdapter instance to handle cursor position.
      */
     public SelectionPopupController(Context context, WindowAndroid window, WebContents webContents,
@@ skipping 10 matching lines @@
         mRepeatingHideRunnable = new Runnable() {
             @Override
             public void run() {
                 assert mHidden;
                 final long hideDuration = getDefaultHideDuration();
                 // Ensure the next hide call occurs before the ActionMode reappears.
                 mView.postDelayed(mRepeatingHideRunnable, hideDuration - 1);
                 hideActionModeTemporarily(hideDuration);
             }
         };
+
+        mSelectionClient = ContextSelectionClient.create(mSelectonCallback, window, webContents);
     }
 
     /**
      * Update the container view.
      */
     void setContainerView(View view) {
         assert view != null;
 
         // Cleans up action mode before switching to a new container view.
         if (isActionModeValid()) finishActionMode();
@@ skipping 24 matching lines @@
     @Override
     public void setAllowedMenuItems(int allowedMenuItems) {
         mAllowedMenuItems = allowedMenuItems;
     }
 
     /**
      * Show (activate) android action mode by starting it.
      *
      * <p>Action mode in floating mode is tried first, and then falls back to
      * a normal one.
-     * @return {@code true} if the action mode started successfully or is already on.
+     * <p> If the action mode cannot be created the selection is cleared.
      */
-    public boolean showActionMode() {
-        if (isEmpty()) return false;
+    public void showActionMode() {

[boliu, 2017/03/20 20:27:26]

maybe call this showActionModeOrClearOnFailure

[Ted C, 2017/03/20 20:35:50]

Ah, that is why the other code changed.

[Tima Vaisburd, 2017/03/21 02:07:02]

Done.

+        if (isEmpty()) {
+            clearSelection();
+            return;
+        }
 
         // Just refreshes the view if it is already showing.
         if (isActionModeValid()) {
             invalidateActionMode();
-            return true;
+            return;
         }
 
         if (mView.getParent() != null) {
              // On ICS, startActionMode throws an NPE when getParent() is null.
             assert mWebContents != null;
             ActionMode actionMode = supportsFloatingActionMode()
                     ? startFloatingActionMode()
                     : mView.startActionMode(mCallback);
             if (actionMode != null) {
                 // This is to work around an LGE email issue. See crbug.com/651706 for more details.
                 LGEmailActionModeWorkaround.runIfNecessary(mContext, actionMode);
             }
             mActionMode = actionMode;
         }
         mUnselectAllOnDismiss = true;
-        return isActionModeValid();
+        if (!isActionModeValid()) clearSelection();
     }
 
     @TargetApi(Build.VERSION_CODES.M)
     private ActionMode startFloatingActionMode() {
         ActionMode actionMode = mView.startActionMode(
                 new FloatingActionModeCallback(this, mCallback), ActionMode.TYPE_FLOATING);
         return actionMode;
     }
 
     void showPastePopup(int x, int y) {
@@ skipping 61 matching lines @@
         return mPastePopupMenu != null && mPastePopupMenu.isShowing();
     }
 
     // Composition methods for android.view.ActionMode
 
     /**
      * @see ActionMode#finish()
      */
     @Override
     public void finishActionMode() {
+        mPostponedDisplayOp = POSTPONED_NONE;
         if (isActionModeValid()) {
             mActionMode.finish();
 
             // Should be nulled out in case #onDestroyActionMode() is not invoked in response.
             mActionMode = null;
         }
     }
 
     /**
      * @see ActionMode#invalidate()
      * Note that invalidation will also reset visibility state. The caller
      * should account for this when making subsequent visibility updates.
      */
     private void invalidateActionMode() {
         if (!isActionModeValid()) return;
         if (mHidden) {
             assert canHideActionMode();
             mHidden = false;
             mView.removeCallbacks(mRepeatingHideRunnable);
+            hideActionModeTemporarily(SHOW_DELAY_MS);
             mPendingInvalidateContentRect = false;
         }
 
         // Try/catch necessary for framework bug, crbug.com/446717.
         try {
             mActionMode.invalidate();
         } catch (NullPointerException e) {
             Log.w(TAG, "Ignoring NPE from ActionMode.invalidate() as workaround for L", e);
         }
     }
 
     /**
      * @see ActionMode#invalidateContentRect()
      */
-    public void invalidateContentRect() {
+    private void invalidateContentRect() {
         if (supportsFloatingActionMode()) {
             if (mHidden) {
                 mPendingInvalidateContentRect = true;
             } else {
                 mPendingInvalidateContentRect = false;
                 if (isActionModeValid()) mActionMode.invalidateContentRect();
             }
         }
     }
 
@@ skipping 82 matching lines @@
             // caused a resource loading failure to be logged. WebView
             // resource access needs to be improved so that this
             // logspam can be avoided.
             new MenuInflater(context).inflate(R.menu.select_action_menu, menu);
         }
     }
 
     private void createActionMenu(ActionMode mode, Menu menu) {
         mNeedsPrepare = false;
         initializeMenu(mContext, mode, menu);
+        updateContextDependantMenuItem(menu);
 
         if (!isSelectionEditable() || !canPaste()) {
             menu.removeItem(R.id.select_action_menu_paste);
         }
 
         if (isInsertion()) {
             menu.removeItem(R.id.select_action_menu_select_all);
             menu.removeItem(R.id.select_action_menu_cut);
             menu.removeItem(R.id.select_action_menu_copy);
             menu.removeItem(R.id.select_action_menu_share);
@@ skipping 22 matching lines @@
 
         initializeTextProcessingMenu(menu);
     }
 
     private boolean canPaste() {
         ClipboardManager clipMgr = (ClipboardManager)
                 mContext.getSystemService(Context.CLIPBOARD_SERVICE);
         return clipMgr.hasPrimaryClip();
     }
 
+    private void updateContextDependantMenuItem(Menu menu) {
+        Log.v(TAG, "updateContextDependantItem");

[boliu, 2017/03/20 20:27:26]

.

[Tima Vaisburd, 2017/03/21 02:07:02]

Removed. Also s/dependant/dependent/.

+        menu.removeItem(R.id.select_action_menu_context_dep);
+
+        if (mClassificationResult == null) return;
+
+        Log.v(TAG, "updateContextDependantMenuItem: adding item " + mClassificationResult.label);
+        menu.add(Menu.NONE, R.id.select_action_menu_context_dep, 1, mClassificationResult.label)
+                .setIcon(mClassificationResult.icon)
+                .setShowAsAction(MenuItem.SHOW_AS_ACTION_ALWAYS);
+    }
+
     /**
      * Intialize the menu items for processing text, if there is any.
      */
     private void initializeTextProcessingMenu(Menu menu) {
         if (Build.VERSION.SDK_INT < Build.VERSION_CODES.M
                 || !isSelectActionModeAllowed(MENU_ITEM_PROCESS_TEXT)) {
             return;
         }
 
         PackageManager packageManager = mContext.getPackageManager();
@@ skipping 21 matching lines @@
                 .setClassName(info.activityInfo.packageName, info.activityInfo.name);
     }
 
     @Override
     public boolean onActionItemClicked(ActionMode mode, MenuItem item) {
         if (!isActionModeValid()) return true;
 
         int id = item.getItemId();
         int groupId = item.getGroupId();
 
-        if (id == R.id.select_action_menu_select_all) {
+        if (id == R.id.select_action_menu_context_dep) {
+            doContextDependentAction();
+            mode.finish();
+        } else if (id == R.id.select_action_menu_select_all) {
             selectAll();
         } else if (id == R.id.select_action_menu_cut) {
             cut();
             mode.finish();
         } else if (id == R.id.select_action_menu_copy) {
             copy();
             mode.finish();
         } else if (id == R.id.select_action_menu_paste) {
             paste();
             mode.finish();
@@ skipping 38 matching lines @@
                 (int) (mSelectionRect.top * deviceScale),
                 (int) (mSelectionRect.right * deviceScale),
                 (int) (mSelectionRect.bottom * deviceScale));
 
         // The selection coordinates are relative to the content viewport, but we need
         // coordinates relative to the containing View.
         outRect.offset(0, (int) mRenderCoordinates.getContentOffsetYPix());
     }
 
     /**
+     * Perform an action that depends on the semantics of the selected text.
+     */
+    @VisibleForTesting
+    void doContextDependentAction() {
+        if (mClassificationResult == null) return;
+
+        assert mClassificationResult.onClickListener != null
+                || mClassificationResult.intent != null;
+
+        if (mClassificationResult.onClickListener != null) {
+            mClassificationResult.onClickListener.onClick(mView);
+            return;
+        }
+
+        if (mClassificationResult.intent != null) {
+            Context context = mWindowAndroid.getContext().get();
+            if (context == null) return;
+
+            context.startActivity(mClassificationResult.intent);
+            return;
+        }
+    }
+
+    /**
      * Perform a select all action.
      */
     @VisibleForTesting
     void selectAll() {
         mWebContents.selectAll();
         // Even though the above statement logged a SelectAll user action, we want to
         // track whether the focus was in an editable field, so log that too.
         if (isSelectionEditable()) {
             RecordUserAction.record("MobileActionMode.SelectAllWasEditable");
         } else {
@@ skipping 157 matching lines @@
         CharSequence result = data.getCharSequenceExtra(Intent.EXTRA_PROCESS_TEXT);
         if (result != null) {
             // TODO(hush): Use a variant of replace that re-selects the replaced text.
             // crbug.com/546710
             mWebContents.replace(result.toString());
         }
     }
 
     void restoreSelectionPopupsIfNecessary() {
         if (mHasSelection && !isActionModeValid()) {
-            if (!showActionMode()) clearSelection();
+            showActionMode();
         }
     }
 
     // All coordinates are in DIP.
     void onSelectionEvent(int eventType, int xAnchor, int yAnchor,
             int left, int top, int right, int bottom, boolean isScrollInProgress,
             boolean touchScrollInProgress) {
         // Ensure the provided selection coordinates form a non-empty rect, as required by
         // the selection action mode.
         if (left == right) ++right;
         if (top == bottom) ++bottom;
         switch (eventType) {
             case SelectionEventType.SELECTION_HANDLES_SHOWN:
                 mSelectionRect.set(left, top, right, bottom);
                 mHasSelection = true;
                 mUnselectAllOnDismiss = true;
-                if (!showActionMode()) clearSelection();
+                if (mSelectionClient != null && mSelectionClient.sendsMenuUpdates()) {
+                    mPostponedDisplayOp = POSTPONED_SHOW;
+                } else {
+                    showActionMode();
+                }
                 break;
 
             case SelectionEventType.SELECTION_HANDLES_MOVED:
                 mSelectionRect.set(left, top, right, bottom);
                 invalidateContentRect();
                 break;
 
             case SelectionEventType.SELECTION_HANDLES_CLEARED:
                 mHasSelection = false;
                 mUnselectAllOnDismiss = false;
                 mSelectionRect.setEmpty();
                 finishActionMode();
                 break;
 
             case SelectionEventType.SELECTION_HANDLE_DRAG_STARTED:
                 hideActionMode(true);
                 break;
 
             case SelectionEventType.SELECTION_HANDLE_DRAG_STOPPED:
-                hideActionMode(false);
+                if (mSelectionClient != null && mSelectionClient.sendsMenuUpdates()) {
+                    mPostponedDisplayOp = POSTPONED_INVALIDATE;
+                } else {
+                    hideActionMode(false);
+                }
                 break;
 
             case SelectionEventType.INSERTION_HANDLE_SHOWN:
                 mSelectionRect.set(left, top, right, bottom);
                 setIsInsertion(true);
                 break;
 
             case SelectionEventType.INSERTION_HANDLE_MOVED:
                 mSelectionRect.set(left, top, right, bottom);
                 if (!isScrollInProgress && isPastePopupShowing()) {
@@ skipping 40 matching lines @@
             int yAnchorPix = (int) (yAnchor * deviceScale);
             mSelectionClient.onSelectionEvent(eventType, xAnchorPix, yAnchorPix);
         }
     }
 
     /**
      * Clears the current text selection. Note that we will try to move cursor to selection
      * end if applicable.
      */
     void clearSelection() {
+        mClassificationResult = null;
         if (mWebContents == null || isEmpty()) return;
         mWebContents.collapseSelection();
     }
 
     void onSelectionChanged(String text) {
         mLastSelectedText = text;
         if (mSelectionClient != null) {
             mSelectionClient.onSelectionChanged(text);
         }
     }
@@ skipping 49 matching lines @@
         mIsInsertion = insertion;
     }
 
     private boolean isShareAvailable() {
         Intent intent = new Intent(Intent.ACTION_SEND);
         intent.setType("text/plain");
         return mContext.getPackageManager().queryIntentActivities(intent,
                 PackageManager.MATCH_DEFAULT_ONLY).size() > 0;
     }
 }