Allow Desktop Canvas to be scrolled out from under System UI.

remoting/android/java/src/org/chromium/chromoting/DesktopCanvas.java

Index: remoting/android/java/src/org/chromium/chromoting/DesktopCanvas.java
diff --git a/remoting/android/java/src/org/chromium/chromoting/DesktopCanvas.java b/remoting/android/java/src/org/chromium/chromoting/DesktopCanvas.java
index e6ae76baff7915e64059f84a2fa288be564d169e..b26434b4bb6e0b770aadaea0061e7ff4ccb62fa1 100644
--- a/remoting/android/java/src/org/chromium/chromoting/DesktopCanvas.java
+++ b/remoting/android/java/src/org/chromium/chromoting/DesktopCanvas.java
@@ -18,6 +18,11 @@ public class DesktopCanvas {
      */
     private static final float MAX_ZOOM_FACTOR = 100.0f;
 
+    /**
+     * Used to smoothly reduce the amount of padding whle the user is zooming.

[Lambros, 2016/09/30 01:59:37]

s/whle/while/

[joedow, 2016/09/30 19:42:57]

Done.

+     */
+    private static final float PADDING_REDUCTION_FACTOR = 0.85f;
+
     private final RenderStub mRenderStub;
     private final RenderData mRenderData;
 
@@ -41,6 +46,12 @@ public class DesktopCanvas {
      */
     private Rect mSystemUiOffsetPixels = new Rect();
 
+    /**
+     * Represents the additional space, in pixels, for each edge of the desktop image.  Used for
+     * positioning the desktop canvas to allow the user to scroll it below any visible System UI.

[Lambros, 2016/09/30 01:59:36]

Is this in screen or image coordinates?
Can you explain in more detail what this padding represents?
For example:
Is the padding conceptually added to, or subtracted from, the image?
How does it interact with the positioning algorithms? (How does the presence of
padding affect the result?)
How is it calculated?
When is it re-calculated and why?
What happens if these values are zero? Non-zero?
Maybe explain with an example - for example, if the toolbar appears/disappears
at the top of the screen, what effect does it have on mImagePadding.top ?

Feel free to write a very long comment if necessary :)

[joedow, 2016/09/30 19:42:56]

Added a block of text at the top of the class giving in depth details on
everything.

+     */
+    private RectF mImagePadding = new RectF();
+
     public DesktopCanvas(RenderStub renderStub, RenderData renderData) {
         mRenderStub = renderStub;
         mRenderData = renderData;
@@ -169,6 +180,16 @@ public class DesktopCanvas {
             mRenderData.transform.setScale(scale, scale);
         }
 
+        // Trim the image padding if the user is zooming out.  This prevents cases where the image
+        // pops to the center when it reaches its minimum size.  Note that we do not need to do
+        // anything when the user is zooming in as the canvas will expand and absorb the padding.
+        if (scaleFactor < 1.0f) {
+            mImagePadding.set(mImagePadding.left * PADDING_REDUCTION_FACTOR,
+                    mImagePadding.top * PADDING_REDUCTION_FACTOR,
+                    mImagePadding.right * PADDING_REDUCTION_FACTOR,
+                    mImagePadding.bottom * PADDING_REDUCTION_FACTOR);
+        }
+
         if (centerOnCursor) {
             setCursorPosition(mCursorPosition.x, mCursorPosition.y);
         } else {
@@ -198,6 +219,9 @@ public class DesktopCanvas {
         // Translate the image so the viewport center is displayed in the middle of the screen.
         mRenderData.transform.postTranslate(viewportTransX, viewportTransY);
 
+        // Remove or add image padding based on the new position.

[Lambros, 2016/09/30 01:59:37]

How is padding based on position?

[joedow, 2016/09/30 19:42:56]

Removed comment since it wan't useful.

+        adjustImagePadding();
+
         mRenderStub.setTransformation(mRenderData.transform);
     }
 
@@ -223,7 +247,9 @@ public class DesktopCanvas {
 
     /** Returns a region which defines the set of valid cursor values in image space. */
     private RectF getImageBounds() {
-        return new RectF(0, 0, mRenderData.imageWidth, mRenderData.imageHeight);
+        float[] maxPadding = getMaxPadding();
+        return new RectF(-maxPadding[0], -maxPadding[1], mRenderData.imageWidth + maxPadding[2],

[Lambros, 2016/09/30 01:59:36]

If my reading is correct, this is the full image rectangle, inflated outwards by
the maxPadding amount.
What has this got to do with allowed cursor values?
Maybe call this getMaxPaddedImageRect() or getMaxPaddedImageBounds() ?

I don't conceptually understand what maxPadding is, so I also don't get the
concept of this method.

[joedow, 2016/09/30 19:42:56]

I'll remove the maxPadding method since it is confusing and add comments here.

+                mRenderData.imageHeight + maxPadding[3]);
     }
 
     /** Returns a region which defines the set of valid viewport center values in image space. */
@@ -233,23 +259,105 @@ public class DesktopCanvas {
         mRenderData.transform.invert(screenToImage);
         screenToImage.mapVectors(screenVectors);
 
-        float xAdjust = 0.0f;
-        if (mRenderData.imageWidth < screenVectors[0]) {
-            // Image is narrower than the screen, so adjust the bounds to center it.
-            xAdjust = (screenVectors[0] - mRenderData.imageWidth) / 2.0f;
-        }
-
-        float yAdjust = 0.0f;
-        if (mRenderData.imageHeight < screenVectors[1]) {
-            // Image is shorter than the screen, so adjust the bounds to center it.
-            yAdjust = (screenVectors[1] - mRenderData.imageHeight) / 2.0f;
-        }
+        float[] letterBoxPadding = getLetterboxPadding();
+        screenToImage.mapVectors(letterBoxPadding);
 
         // screenCenter values are 1/2 of a particular screen dimension mapped to image space.
-        float screenCenterX = screenVectors[0] / 2.0f;
-        float screenCenterY = screenVectors[1] / 2.0f;
-        return new RectF(screenCenterX - xAdjust, screenCenterY - yAdjust,
-                mRenderData.imageWidth - screenCenterX + xAdjust,
-                mRenderData.imageHeight - screenCenterY + yAdjust);
+        float screenCenterX = (screenVectors[0] / 2.0f) - letterBoxPadding[0];
+        float screenCenterY = (screenVectors[1] / 2.0f) - letterBoxPadding[1];
+        RectF imageBounds = getImageBounds();
+        return new RectF(imageBounds.left + screenCenterX, imageBounds.top + screenCenterY,

[Lambros, 2016/09/30 01:59:36]

return imageBounds.inset(screenCenterX, screenCenterY);

[joedow, 2016/09/30 19:42:57]

Done.

+                imageBounds.right - screenCenterX, imageBounds.bottom - screenCenterY);
+    }
+
+    /** Returns the maximum amount of padding per edge (left, top, right, bottom) in image space. */
+    private float[] getMaxPadding() {
+        RectF systemUiOffsetPixels = getAdjustedSystemUiOffsets();
+        float[] maxPadding = {Math.max(mImagePadding.left, systemUiOffsetPixels.left),
+                Math.max(mImagePadding.top, systemUiOffsetPixels.top),
+                Math.max(mImagePadding.right, systemUiOffsetPixels.right),
+                Math.max(mImagePadding.bottom, systemUiOffsetPixels.bottom)};
+        Matrix screenToImage = new Matrix();
+        mRenderData.transform.invert(screenToImage);
+        screenToImage.mapVectors(maxPadding);
+
+        return maxPadding;
+    }
+
+    /** Returns the letter box padding needed in each dimension (x and y) in screen coordinates. */

[Lambros, 2016/09/30 01:59:37]

Maybe explain in a little more detail:
This is the amount of unused screen space if the desktop image were positioned
perfectly in the center of the screen.

[joedow, 2016/09/30 19:42:57]

Explained in detail at the top of the class.

+    private float[] getLetterboxPadding() {

[Lambros, 2016/09/30 01:59:36]

Maybe return PointF ?

[joedow, 2016/09/30 19:42:57]

I had originally left this as a float [] to make transforms simpler but I use it
in an unmodified form more frequently so this change seems fine.

+        float[] imageVectors = {mRenderData.imageWidth, mRenderData.imageHeight};
+        mRenderData.transform.mapVectors(imageVectors);
+
+        // We want to letterbox when the image is smaller than the screen in a specific dimension.
+        // Since we center the image, split the difference so it is equally distributed.
+        float widthAdjust =
+                Math.max(0.0f, ((float) mRenderData.screenWidth - imageVectors[0]) / 2.0f);
+        float heightAdjust =
+                Math.max(0.0f, ((float) mRenderData.screenHeight - imageVectors[1]) / 2.0f);
+
+        return new float[] {widthAdjust, heightAdjust};
+    }
+
+    /**
+     * Returns the amount of System UI, adjusted for letter boxing, which should be used for
+     * padding calculations.
+     */
+    private RectF getAdjustedSystemUiOffsets() {
+        float[] letterBoxPadding = getLetterboxPadding();
+        return new RectF(Math.max(mSystemUiOffsetPixels.left - letterBoxPadding[0], 0.0f),
+                Math.max(mSystemUiOffsetPixels.top - letterBoxPadding[1], 0.0f),
+                Math.max(mSystemUiOffsetPixels.right - letterBoxPadding[0], 0.0f),
+                Math.max(mSystemUiOffsetPixels.bottom - letterBoxPadding[1], 0.0f));
+    }
+
+    /** Updates the padding used to allow the user to scroll the image out from under System UI. */
+    private void adjustImagePadding() {
+        float[] letterBoxPadding = getLetterboxPadding();
+        RectF systemUiOffsetPixels = getAdjustedSystemUiOffsets();
+        float[] imagePoints = {0.0f, 0.0f, mRenderData.imageWidth, mRenderData.imageHeight};
+        mRenderData.transform.mapPoints(imagePoints);
+
+        mImagePadding.set(
+                adjustEdge(systemUiOffsetPixels.left, mImagePadding.left,
+                        imagePoints[0] - letterBoxPadding[0]),
+                adjustEdge(systemUiOffsetPixels.top, mImagePadding.top,
+                        imagePoints[1] - letterBoxPadding[1]),
+                adjustEdge(systemUiOffsetPixels.right, mImagePadding.right,
+                        mRenderData.screenWidth - imagePoints[2] - letterBoxPadding[0]),
+                adjustEdge(systemUiOffsetPixels.bottom, mImagePadding.bottom,
+                        mRenderData.screenHeight - imagePoints[3] - letterBoxPadding[1]));
+    }
+
+    /**
+     * Update the current edge padding based on the current System UI and image postion state.

[Lambros, 2016/09/30 01:59:36]

s/postion/position/

[joedow, 2016/09/30 19:42:56]

Done.

+     *
+     * @param systemUiPadding Amount of visible System UI in pixels.
+     * @param currentPadding Amount of padding used, in pixels, for position calculations.
+     * @param visiblePixels Amount of padding actually visible, in pixels.
+     * @return The new value, in pixels, to be used for positioning.
+     */
+    private float adjustEdge(float systemUiPadding, float currentPadding, float visiblePixels) {
+        float edgeValue;
+        visiblePixels = Math.max(visiblePixels, 0.0f);
+        if (systemUiPadding > 0.0f) {
+            if (systemUiPadding > currentPadding) {
+                // Add enough padding to allow scrolling the desktop out from under the System UI.
+                edgeValue = Math.min(systemUiPadding, visiblePixels);
+            } else if (systemUiPadding < currentPadding) {
+                // Remove unneeded padding from the canvas.
+                edgeValue = Math.max(systemUiPadding, Math.min(currentPadding, visiblePixels));
+            } else {
+                // If current and system padding sizes are the same, then keep the visible portion.
+                edgeValue = Math.min(currentPadding, visiblePixels);
+            }
+        } else if (currentPadding > 0.0f) {
+            // If we have existing padding, then trim if possible, otherwise keep the same value.
+            edgeValue = Math.min(currentPadding, visiblePixels);
+        } else {
+            // Use zero padding if there is no system or current padding specified.
+            edgeValue = 0.0f;
+        }
+        return edgeValue;
     }
 }