Allow Desktop Canvas to be scrolled out from under System UI.

remoting/android/java/src/org/chromium/chromoting/DesktopCanvas.java

 // Copyright 2015 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 package org.chromium.chromoting;
 
 import android.graphics.Matrix;
 import android.graphics.PointF;
 import android.graphics.Rect;
 import android.graphics.RectF;
 
 /**
  * This class is responsible for transforming the desktop image matrix.
+ *
+ * The transformation aspect is pretty simple, however the logic behind it is complex enough to
+ * warrant a longer chunk of comments to explain how that is accomplished.
+ *
+ * ===== Coordinate Systems =====
+ * First, there are two coordinate spaces at play here, screen space and image space, which map to
+ * coordinates on the device's screen and to the image used to display the remote desktop
+ * respectively.  We can convert from image coordinates to screen coordinates by using the transform
+ * matrix held by |mRenderData| and convert from screen coordinates to image coordinates using the
+ * inverse of that matrix.
+ *
+ * For consistency, position (viewport and cursor) member variables store padding in image space as
+ * do the internal methods which modify them (unless otherwise noted).
+ *
+ *
+ * ===== Viewport Concept =====
+ * Since the image used to display the remote desktop can be zoomed and panned such that only a
+ * portion of it is displayed on screen, this class defines that viewable area as the "viewport".
+ * A goal of this class is to ensure the viewport contains the maximum amount of image pixels as it
+ * can. Thus it defines a subset of valid positions which map to the image, these are called
+ * boundaries or bounds.  When a caller shifts the viewport, the new coordinate is checked to ensure
+ * it lies within the acceptable set of positions and is clamped to that viewable region if not.
+ * The viewport is tracked and bound by its center position.
+ *
+ * As an example, let's say the image is height 1000px and the screen height is 800 (both in screen
+ * coordinates).  The viewport center would be located at 400px in this case and the viewport bounds
+ * would range from 400px to 600px.  At 400px, the top of the image would align with the top of the
+ * screen and at 600px, the bottom of the image would align with the bottom of the screen.
+ *
+ *
+ * ===== Cursor Position and Desired Viewport Concept =====
+ * Moving the viewport position is simple and works well for direct input methods, such as Touch,
+ * however indirect methods, such as Trackpad mode, requires tracking of another position.  This
+ * second position can be thought of as the 'desired' viewport position and coincides with the
+ * cursor position.  It is called the desired position because it is the original position of the
+ * viewport before the bounds are applied.  This class attempts to keep the desired and actual
+ * viewport centers at the same coordinate, however when the user can move the cursor to a position
+ * which is outside of the allowed coordinates for the viewport (i.e. the edge of the screen).
+ * We can't move the viewport center to that location without breaking our goal of maximizing the
+ * viewable area of the screen, so instead we track the cursor position and begin shifting the
+ * viewport again once the cursor moves back into the region of allowable viewport positions.
+ *
+ *
+ * ===== Letterbox Padding =====
+ * One last note about the image and screen.  When the image has been zoomed such that it is larger
+ * than the screen, we are free to move the viewport around as described above.  However once the
+ * image is zoomed out such that one of the dimensions is smaller than the corresponding screen
+ * dimension, we want to center the image within the viewport (and the screen).  This is done by
+ * adding conceptual padding to the edges of the screen.  As an example, when the user zooms the
+ * image out and the height of the image (say 799px) drops below the screen height (say 800px), we
+ * will add 0.5px to the top and bottom edges of the image in our position calculations.  As the
+ * image height (in screen space) decreases (e.g. to 600px), the amount of padding will increase.
+ * This ensures that the content is centered as the user zooms in and out.  The same would occur
+ * if the image width (in screen space) dropped below the screen width with padding being added to
+ * the right and left edges of the image.  This padding is referred to as "letterbox" padding.
+ * Note that this padding is never applied to all four edges.  The calculations used below allow one
+ * image dimension to decrease below the screen dimension but it prevents additional attempts to
+ * zoom out once the second image dimension matches the screen dimension.

[Lambros, 2016/10/03 23:04:10]

I'm still struggling to understand this exact concept.
It sounds like "letterbox padding" is an artificial extension of the bounds of
the image, to prevent excessive movement of the image when it is (fully or
nearly) zoomed out. It completely "locks" movement in, say, the vertical
direction if the vertical image size is smaller than the screen.
When System UI appears, the "padding" is maybe adjusted. How? I can see 3
distinct schemes:
* Reduced equally at top and bottom, because the viewable area is reduced by the
system UI. Padding is always symmetrically applied.
* Reduced at the side where System UI appears, down to a minimum of zero
(padding is never negative). Maybe if you hit zero, you start taking away
padding from the opposite side? Padding is always applied in such a way that
Padding+SystemUI+Image == Screen (except when this would make padding negative).
* Letterbox padding is not affected by system UI at all. There is a separate
concept of "adjusted padding" that takes System UI into account.

[joedow, 2016/10/04 20:47:47]

Discussed offline.  I've simplifed the code a bit and removed this text (since I
don't think it provides value anymore).

+ *
+ *
+ * ===== Viewport and System UI Concepts =====
+ * System UI is defined as pixels displayed by the operating system.  For us this boils down to four
+ * UI elements: status bar (located on the top edge of the screen), toolbar (located below the
+ * status bar), navigation bar (located either at the bottom or right/left edge of the screen), and
+ * the soft input method (docked to bottom of the screen, possibly above the navigation bar).
+ *
+ * Android has a mechanism to automatically resize the size of a UI View when System UI is displayed
+ * or changes size however this causes a jarring repositioning effect.  We use this mechanism in
+ * Pre-KitKat versions of Android as there isn't much of a choice.  On KitKat and later, we actually
+ * position the desktop image in a layer below the System UI so we can display the image within the
+ * viewport in a consistent fashion and avoid those jarring reposition effects that occur on older
+ * OS versions.  Though the experience is smoother, it also creates a new problem which is how do
+ * users view and interact with remote UI elements on the image which are obscured by System UI.
+ *
+ * To address this problem, we allow the user to pan the remote image out from under the System UI.
+ * Our goal is to keep the viewport location consistent regardless of the System UI state whenever
+ * possible which leads to the following sections on padding and viewport position with System UI.
+ *
+ *
+ * ===== System UI Padding =====
+ * In order to allow the user to pan the image out from under System UI, we need to know how much
+ * space that UI occupies on the screen.  This information is provided by the View whenever the size
+ * changes.  This information is applied to the viewport boundaries to allow the user to move the
+ * viewport further in that direction than they could have previously.
+ *
+ * If we use the same sizes as before (image is 1000px and screen is 800px in screen coordinates),
+ * the allowable viewport position values in the y axis are 400px through 600px.  If System UI is
+ * shown at the top and bottom of the screen (say 50px along both edges) then we need to adjust the
+ * viewport to allow panning past that UI, the updated range of allowed values is 350px to 650px.
+ * This adjustment allows the user to pan the desktop image out from under any System UI and fixes
+ * the original problem.
+ *
+ * There are two caveats though.  The first is how to handle the case when the System UI changes
+ * size or disappears.  The second is a complication due to letterbox padding.  We'll address each
+ * in their own sections.
+ *
+ *
+ * ===== System UI and Letterbox Padding Overlap =====
+ * If we did not account for letterbox padding when adjusting the viewport bounds, the behavior
+ * described in the System UI Padding section would fall apart when the image size (in screen space)
+ * decreased below the screen size.  The effect would be that the user could pan the desktop image
+ * out from under the System UI (good) and then continue panning a distance defined by the amount of
+ * letterbox padding added to that edge.  Functionally this is OK, but visually it looks broken.  We
+ * address this problem by accounting for the overlap between the System UI and Letterbox padding.
+ *
+ * For this example, we will consider a single dimension (e.g. height) and assume a screen size of
+ * 800px, an image size (once again in screen space) of 600px, letterbox padding on top and bottom
+ * edges of 100px, and System UI at the top and bottom of the screen at 50px each.
+ *
+ * Without the System UI, the viewport center would be locked to 400px (the actual center of the
+ * image + letterbox padding).  When the System UI was displayed, the range would be changed by 50px
+ * to 350px to 450px.  This would allow the user to pan the image canvas by 50 px in either
+ * direction.  The problem here is that this panning is not useful as none of the image is obscured
+ * by the System UI.  Therefore we account for the letterbox padding and see that no scrolling is
+ * required and keep the viewport locked at 400px.
+ *
+ * The previous example showed an example interaction between a small System UI and the viewport
+ * bounds with letterbox padding, now let's consider a large System UI.  In this example let's use
+ * a Soft Input Method UI which is 400 px tall and anchored to the bottom of the screen.  Using the
+ * previous values (without letterboxing), the viewport range would be adjusted to 400px - 800px.
+ * This allows the image to be fully seen by the user, but includes a 100px black edge between the
+ * content and the System UI.  What is needed is to account for the letterbox padding here as well.
+ * We subtract the padding (100px) from the allowable range and end up with 400px - 700px.  The

[Lambros, 2016/10/03 23:04:10]

Shouldn't this be 500px - 700px ?

[joedow, 2016/10/04 20:47:47]

Comment block removed.

+ * image is fully visible and no letterbox padding is displayed.

[Lambros, 2016/10/03 23:04:10]

But the image isn't fully visible? It's 600px tall but the viewable area is only
400px.

[joedow, 2016/10/04 20:47:47]

Comment block removed.

+ *
+ *
+ * ===== Image Padding and Ratcheting =====
+ * On to the problem of consistency.  The sections above have solved the visibility problem, with
+ * the methods described above, the image can always be viewed and all coordinates on the image can
+ * be interacted with using the cursor or touch.  So now we move on to the problem of handling the
+ * boundaries when the System UI changes.
+ *
+ * First a description of the behavior we desire in this scenario.  Let's say the user has panned
+ * their image out from under the status and toolbar (~150px).  When those UI elements are dismissed
+ * we have two options, immediately trim the padding used for it and snap the viewport back to its
+ * normal position or allow the viewport to remain until the extra padding is no longer required.

[Lambros, 2016/10/03 23:04:10]

"extra padding"? The amount of padding is *less* when there is System UI. So
when System UI is dismissed, there is no "extra padding" to remove. Instead,
there is a possibility of adding some new padding, now there is more screen
space to put the image in.

[joedow, 2016/10/04 20:47:47]

N/A Comment block removed.

+ * We have chosen the second approach.
+ *
+ * The idea is that after the System UI has been dismissed, we allow the viewport to remain as-is,
+ * but we don't want to keep this padding forever.  The compromise is to 'eat' the unneeded padding
+ * as the user pans the canvas, effectively creating a ratcheting effect.
+ *
+ * We accomplish this goal using a concept of image padding which tracks the actual amount of
+ * padding in use for viewport calculations.  In order to determine how much padding to use, we look
+ * at the adjusted System UI padding, current image padding, and the number of visible pixels.

[Lambros, 2016/10/03 23:04:10]

I don't understand "visible pixels" or the different types of padding.
You have lots of different kinds of padding:
"letterbox padding"
"adjusted System UI padding"
"current image padding"

I understand the first of these. I sort-of get what "adjusted system UI padding"
is trying to do, but not quite fully. I don't know what its definition is, or
what properties it has, or what invariants it satisfies, so it's basically
impossible for me to conceptualize or reason about it.
I have no idea what "current image padding" is.

[joedow, 2016/10/04 20:47:47]

Comment block removed.

+ *
+ * More examples would be useful here.  Instead of talking about a dimension, let's focus on a
+ * single edge for this set of example scenarios.  Everything starts with System UI being displayed,
+ * otherwise there is no need for padding.  The System UI for this example has already been adjusted

[Lambros, 2016/10/03 23:04:10]

"otherwise there is no need for padding"
But letterbox padding is still applied when there is no System UI at all.
You probably don't mean "letterbox padding", but there are so many different
kinds of padding now :)

[joedow, 2016/10/04 20:47:47]

Comment block removed.

+ * to account for letterbox padding and totals 50px.
+ * Note: The Visible Pixels are also adjusted to account for letterbox padding.
+ *
+ * Example: No padding is ever used in viewport calculations.
+ *   The user pans the image but never moves the image out from under the System UI.
+ *      Edge Calculation:  System UI: 50px,  Image Padding: 0px,  Visible Pixels 0px
+ *      Edge Padding result: 0px
+ *   The System UI then disappears:
+ *      Edge Calculation:  System UI: 0px,  Image Padding: 0px,  Visible Pixels 0px
+ *      Edge Padding result: 0px
+ *
+ * Example: Some padding is used in viewport calculations.
+ *   The user pans the image halfway out from under the System UI.
+ *      Edge Calculation:  System UI: 50px,  Image Padding: 0px,  Visible Pixels 25px
+ *      Edge Padding result: 25px
+ *   The System UI then disappears (the viewport remains stationary):
+ *      Edge Calculation:  System UI: 0px,  Image Padding: 25px,  Visible Pixels 25px
+ *      Edge Padding result: 25px
+ *   The user pans away from the edge a bit (say 10px):
+ *      Edge Calculation:  System UI: 0px,  Image Padding: 25px,  Visible Pixels 15px
+ *      Edge Padding result: 15px
+ *   The user attempts to pan back toward the edge but cannot go past the 15px padding.
+ *   The user then pans 75px away from the edge:
+ *      Edge Calculation:  System UI: 0px,  Image Padding: 15px,  Visible Pixels -75px
+ *      Edge Padding result: 0px
+ *
+ *
+ * ===== Image Padding and Zooming =====
+ * In the previous section, the ratcheting mechanism was discussed which reduces the padding when
+ * the user pans away from the edge of the screen.  However what happens when a user zooms when
+ * padding is applied?  If the user is zooming in, then nothing is required as the image will be
+ * expanded and the padding will be 'absorbed' by the ratcheting effect described previously.  The
+ * problem occurs when the user is zooming out.  In that case, the edge of the image is not changed
+ * and the effect looks odd (but is functionally OK).
+ *
+ * The solution to this problem is to reduce the amount of padding by a constant factor.  This
+ * reduction creates an easing effect as the position of the image slides toward the edge as the
+ * padding is reduced.
+ *
+ *
+ * ===== Image Padding and Cursors =====
+ * All of the problems and solutions above apply equally to Touch and Trackpad modes.  However
+ * Trackpad mode has two additional problems.
+ *
+ * The first is the problem of newly displayed System UI obscurring the cursor.  Imagine you move
+ * the cursor to the upper corner of the screen and then display the toolbar.  The toolbar will be
+ * displayed over the top of the cursor which is not a good idea.  Our goal is to minimize screen
+ * translations, but in this case, it is more important that the user does not lose sight of the
+ * cursor.  Thus we automatically move the viewport so the cursor is guaranteed to be visible.  If
+ * the cursor is centered on the screen, then no translation is needed.
+ *
+ * The second is also related to visibility but is not solved by the solution above.  For this case,
+ * let's assume the cursor is centered in the viewport.  Next the user brings up the Soft Keyboard.
+ * The Keyboard's height is near that of the screen center so the cursor is obscured.  The user is
+ * not able to see what they are interacting with and selecting a text box is impossible.  For this
+ * scenario, we artificially shift the 'screen center' position by viewport_size / 2.  This ensures
+ * the cursor is visible and the user can view and interact with any element on the remote desktop.
+ *
  */
 public class DesktopCanvas {
     /**
-     * Maximum allowed zoom level - see {@link #repositionImageWithZoom()}.
+     * Maximum allowed zoom level - see {@link #scaleAndRepositionImage()}.
      */
     private static final float MAX_ZOOM_FACTOR = 100.0f;
 
+    /**
+     * Used to smoothly reduce the amount of padding while the user is zooming.
+     */
+    private static final float PADDING_REDUCTION_FACTOR = 0.85f;
+
     private final RenderStub mRenderStub;
     private final RenderData mRenderData;
 
     /**
-     * Represents the actual center of the viewport.  This value needs to be a pair of floats so the
-     * desktop image can be positioned with sub-pixel accuracy for smoother panning animations at
-     * high zoom levels.
+     * Represents the actual center of the viewport in image space.  This value needs to be a pair
+     * of floats so the desktop image can be positioned with sub-pixel accuracy for smoother panning
+     * animations at high zoom levels.
      */
     private PointF mViewportPosition = new PointF();
 
     /**
-     * Represents the desired center of the viewport.  This value may not represent the actual
-     * center of the viewport as adjustments are made to ensure as much of the desktop is visible as
-     * possible.  This value needs to be a pair of floats so the desktop image can be positioned
-     * with sub-pixel accuracy for smoother panning animations at high zoom levels.
+     * Represents the desired center of the viewport in image space.  This value may not represent
+     * the actual center of the viewport as adjustments are made to ensure as much of the desktop is
+     * visible as possible.  This value needs to be a pair of floats so the desktop image can be
+     * positioned with sub-pixel accuracy for smoother panning animations at high zoom levels.
      */
     private PointF mCursorPosition = new PointF();
 
     /**
-     * Represents the amount of space, in pixels, used by System UI.
+     * Represents the amount of space, in screen pixels, used by System UI.
      */
     private Rect mSystemUiOffsetPixels = new Rect();
 
+    /**
+     * Represents the additional space, in screen pixels, for each edge of the desktop image.
+     * Used to allow the user to position the image canvas out from under any visible System UI.
+     */
+    private RectF mImagePadding = new RectF();
+
     public DesktopCanvas(RenderStub renderStub, RenderData renderData) {
         mRenderStub = renderStub;
         mRenderData = renderData;
     }
 
     /**
      * Sets the desired center position of the viewport (a.k.a. the cursor position) and ensures
      * the viewport is updated to include the cursor within it.
      *
      * @param newX The new x coordinate value for the desired center position.
@@ skipping 108 matching lines @@
         mRenderData.transform.mapVectors(imageSize);
 
         if (imageSize[0] < mRenderData.screenWidth && imageSize[1] < mRenderData.screenHeight) {
             // Displayed image is too small in both directions, so apply the minimum zoom
             // level needed to fit either the width or height.
             float scale = Math.min((float) mRenderData.screenWidth / mRenderData.imageWidth,
                                    (float) mRenderData.screenHeight / mRenderData.imageHeight);
             mRenderData.transform.setScale(scale, scale);
         }
 
+        // Trim the image padding if the user is zooming out.  This prevents cases where the image
+        // pops to the center when it reaches its minimum size.  Note that we do not need to do
+        // anything when the user is zooming in as the canvas will expand and absorb the padding.
+        if (scaleFactor < 1.0f) {
+            mImagePadding.set(mImagePadding.left * PADDING_REDUCTION_FACTOR,
+                    mImagePadding.top * PADDING_REDUCTION_FACTOR,
+                    mImagePadding.right * PADDING_REDUCTION_FACTOR,
+                    mImagePadding.bottom * PADDING_REDUCTION_FACTOR);
+        }
+
         if (centerOnCursor) {
             setCursorPosition(mCursorPosition.x, mCursorPosition.y);
         } else {
             // Find the new screen center (it was probably changed during the zoom operation) and
             // update the viewport and cursor.
             float[] mappedPoints = {
                     ((float) mRenderData.screenWidth / 2), ((float) mRenderData.screenHeight / 2)};
             Matrix screenToImage = new Matrix();
             mRenderData.transform.invert(screenToImage);
             screenToImage.mapPoints(mappedPoints);
             // The cursor is mapped to the center of the viewport in this case.
             setCursorPosition(mappedPoints[0], mappedPoints[1]);
         }
     }
 
     /**
      * Repositions the image by translating it (without affecting the zoom level).
      */
     private void repositionImage() {
         // Map the current viewport position to screen coordinates and adjust the image position.
         float[] viewportPosition = {mViewportPosition.x, mViewportPosition.y};
         mRenderData.transform.mapPoints(viewportPosition);
 
         float viewportTransX = ((float) mRenderData.screenWidth / 2) - viewportPosition[0];
         float viewportTransY = ((float) mRenderData.screenHeight / 2) - viewportPosition[1];
 
         // Translate the image so the viewport center is displayed in the middle of the screen.
         mRenderData.transform.postTranslate(viewportTransX, viewportTransY);
 
+        adjustImagePadding();
+
         mRenderStub.setTransformation(mRenderData.transform);
     }
 
     /**
      * Updates the given point such that it refers to a coordinate within the bounds provided.
      *
      * @param point The point to adjust, the original object is modified.
      * @param bounds The bounds used to constrain the point.
      */
     private void constrainPointToBounds(PointF point, RectF bounds) {
         if (point.x < bounds.left) {
             point.x = bounds.left;
         } else if (point.x > bounds.right) {
             point.x = bounds.right;
         }
 
         if (point.y < bounds.top) {
             point.y = bounds.top;
         } else if (point.y > bounds.bottom) {
             point.y = bounds.bottom;
         }
     }
 
-    /** Returns a region which defines the set of valid cursor values in image space. */
+    /** Returns a region used to constrain the cursor position in image space. */
     private RectF getImageBounds() {
-        return new RectF(0, 0, mRenderData.imageWidth, mRenderData.imageHeight);
+        // Expand the allowable cursor positions to include any applicable padding (includes both
+        // unused System UI padding and currently used image padding).  Doing this allows the user
+        // to move their cursor into this space to indicate they want to scroll it into the viewable
+        // area.
+        RectF systemUiOffsetPixels = getAdjustedSystemUiOffsets();
+        float[] paddingBuffer = {Math.max(mImagePadding.left, systemUiOffsetPixels.left),
+                Math.max(mImagePadding.top, systemUiOffsetPixels.top),
+                Math.max(mImagePadding.right, systemUiOffsetPixels.right),
+                Math.max(mImagePadding.bottom, systemUiOffsetPixels.bottom)};
+        Matrix screenToImage = new Matrix();
+        mRenderData.transform.invert(screenToImage);
+        screenToImage.mapVectors(paddingBuffer);
+
+        return new RectF(-paddingBuffer[0], -paddingBuffer[1],
+                mRenderData.imageWidth + paddingBuffer[2],
+                mRenderData.imageHeight + paddingBuffer[3]);
     }
 
     /** Returns a region which defines the set of valid viewport center values in image space. */
     private RectF getViewportBounds() {
         float[] screenVectors = {(float) mRenderData.screenWidth, (float) mRenderData.screenHeight};
         Matrix screenToImage = new Matrix();
         mRenderData.transform.invert(screenToImage);
         screenToImage.mapVectors(screenVectors);
 
-        float xAdjust = 0.0f;
-        if (mRenderData.imageWidth < screenVectors[0]) {
-            // Image is narrower than the screen, so adjust the bounds to center it.
-            xAdjust = (screenVectors[0] - mRenderData.imageWidth) / 2.0f;
-        }
-
-        float yAdjust = 0.0f;
-        if (mRenderData.imageHeight < screenVectors[1]) {
-            // Image is shorter than the screen, so adjust the bounds to center it.
-            yAdjust = (screenVectors[1] - mRenderData.imageHeight) / 2.0f;
-        }
+        PointF letterBoxPadding = getLetterboxPaddingOnScreen();
+        float[] letterBoxPaddingVectors = {letterBoxPadding.x, letterBoxPadding.y};
+        screenToImage.mapVectors(letterBoxPaddingVectors);
 
         // screenCenter values are 1/2 of a particular screen dimension mapped to image space.
-        float screenCenterX = screenVectors[0] / 2.0f;
-        float screenCenterY = screenVectors[1] / 2.0f;
-        return new RectF(screenCenterX - xAdjust, screenCenterY - yAdjust,
-                mRenderData.imageWidth - screenCenterX + xAdjust,
-                mRenderData.imageHeight - screenCenterY + yAdjust);
+        float screenCenterX = (screenVectors[0] / 2.0f) - letterBoxPaddingVectors[0];
+        float screenCenterY = (screenVectors[1] / 2.0f) - letterBoxPaddingVectors[1];
+        RectF imageBounds = getImageBounds();
+        imageBounds.inset(screenCenterX, screenCenterY);
+        return imageBounds;
+    }
+
+    /** Returns the letterbox padding needed in each dimension (x and y) in screen coordinates. */
+    private PointF getLetterboxPaddingOnScreen() {
+        float[] imageVectors = {mRenderData.imageWidth, mRenderData.imageHeight};
+        mRenderData.transform.mapVectors(imageVectors);
+
+        // We want to letterbox when the image is smaller than the screen in a specific dimension.
+        // Since we center the image, split the difference so it is equally distributed.
+        float widthAdjust =
+                Math.max(0.0f, ((float) mRenderData.screenWidth - imageVectors[0]) / 2.0f);
+        float heightAdjust =
+                Math.max(0.0f, ((float) mRenderData.screenHeight - imageVectors[1]) / 2.0f);
+
+        return new PointF(widthAdjust, heightAdjust);
+    }
+
+    /**
+     * Returns the amount of System UI, adjusted for letterbox padding overlap, to use for padding.
+     */
+    private RectF getAdjustedSystemUiOffsets() {
+        // Ignore negative padding since that means no adjustment is necessary.  We use a floor of
+        // 0.0f to simplify calculations using the returned values.
+        PointF letterBoxPadding = getLetterboxPaddingOnScreen();
+        return new RectF(Math.max(mSystemUiOffsetPixels.left - letterBoxPadding.x, 0.0f),
+                Math.max(mSystemUiOffsetPixels.top - letterBoxPadding.y, 0.0f),
+                Math.max(mSystemUiOffsetPixels.right - letterBoxPadding.x, 0.0f),
+                Math.max(mSystemUiOffsetPixels.bottom - letterBoxPadding.y, 0.0f));
+    }
+
+    /**
+     * Updates the padding used to allow the user to scroll the image out from under System UI.
+     * This is done by taking the adjusted System UI size, current padding, and visible pixels for
+     * each edge of the screen.
+     */
+    private void adjustImagePadding() {
+        PointF letterBoxPadding = getLetterboxPaddingOnScreen();
+        RectF systemUiOffsetPixels = getAdjustedSystemUiOffsets();
+        float[] imagePoints = {0.0f, 0.0f, mRenderData.imageWidth, mRenderData.imageHeight};
+        mRenderData.transform.mapPoints(imagePoints);
+
+        mImagePadding.set(
+                adjustEdgePadding(systemUiOffsetPixels.left, mImagePadding.left,
+                        imagePoints[0] - letterBoxPadding.x),
+                adjustEdgePadding(systemUiOffsetPixels.top, mImagePadding.top,
+                        imagePoints[1] - letterBoxPadding.y),
+                adjustEdgePadding(systemUiOffsetPixels.right, mImagePadding.right,
+                        mRenderData.screenWidth - imagePoints[2] - letterBoxPadding.x),
+                adjustEdgePadding(systemUiOffsetPixels.bottom, mImagePadding.bottom,
+                        mRenderData.screenHeight - imagePoints[3] - letterBoxPadding.y));
+    }
+
+    /**
+     * Update the current edge padding based on the current System UI and image position state.
+     *
+     * @param systemUiPadding Amount of visible System UI in pixels.
+     * @param currentPadding Amount of padding used, in pixels, for position calculations.
+     * @param visiblePixels Amount of padding actually visible, in pixels.
+     * @return The new value, in pixels, to be used for positioning.
+     */
+    private float adjustEdgePadding(
+            float systemUiPadding, float currentPadding, float visiblePixels) {
+        float edgeValue;
+        visiblePixels = Math.max(visiblePixels, 0.0f);
+        if (systemUiPadding > 0.0f) {
+            if (systemUiPadding > currentPadding) {
+                // Add enough padding to allow scrolling the desktop out from under the System UI.
+                edgeValue = Math.min(systemUiPadding, visiblePixels);
+            } else if (systemUiPadding < currentPadding) {
+                // Remove unneeded padding from the canvas.
+                edgeValue = Math.max(systemUiPadding, Math.min(currentPadding, visiblePixels));
+            } else {
+                // If current and system padding sizes are the same, then keep the visible portion.
+                edgeValue = Math.min(currentPadding, visiblePixels);
+            }
+        } else if (currentPadding > 0.0f) {
+            // If we have existing padding, then trim if possible, otherwise keep the same value.
+            edgeValue = Math.min(currentPadding, visiblePixels);
+        } else {
+            // Use zero padding if there is no system or current padding specified.
+            edgeValue = 0.0f;
+        }
+        return edgeValue;
     }
 }