OLD | NEW |
(Empty) | |
| 1 # QR Scanner |
| 2 |
| 3 The QR Scanner provides a way of scanning QR codes and bar codes directly from |
| 4 Chrome. It is developed behind the `EnableQRCodeReader` experimental flag. |
| 5 |
| 6 [TOC] |
| 7 |
| 8 ## Usage |
| 9 |
| 10 1. Create a delegate implementing the `QRScannerViewControllerDelegate` |
| 11 protocol. |
| 12 2. Initialize `QRScannerViewController` with this delegate. |
| 13 3. Present the view controller returned by `getViewControllerToPresent`. |
| 14 |
| 15 ## Behavior |
| 16 |
| 17 The QR Scanner is presented as a full-screen view controller displaying a video |
| 18 preview, a control for the camera's torch, and a control for closing the QR |
| 19 scanner. |
| 20 |
| 21 ### Presentation |
| 22 |
| 23 The QR Scanner is presented using a custom transition animation which makes it |
| 24 appear to be originally positioned under the presenting view controller. |
| 25 |
| 26 ### Scanning |
| 27 |
| 28 * Codes are only recognized inside the viewport. |
| 29 * A flash animation is played when a code is recognized. |
| 30 * If VoiceOver is enabled, an announcement is played instead of the animation. |
| 31 * Scanning a QR code or any other code type which can contain alphanumeric |
| 32 strings places the scanned result in the Omnibox, and the user has to press |
| 33 the "Go" button on the keyboard to load the result. Non-URL strings will be |
| 34 loaded in search. |
| 35 * Scanning a bar code type which can only contain numbers will load search |
| 36 results containing the bar code immediately without waiting for user |
| 37 confirmation. |
| 38 |
| 39 ### Torch |
| 40 |
| 41 * Torch is switched off every time the QR scanner is opened, closed, or the |
| 42 camera session is interrupted. |
| 43 * The torch button always reflects the current torch state. |
| 44 * The torch button is in a disabled state if the camera does not have torch, |
| 45 the torch is unavailable, or the camera is not yet loaded. |
| 46 |
| 47 ### Errors |
| 48 |
| 49 * A dialog is displayed if the camera is unavailable, camera permissions are |
| 50 not granted, the camera is in use by another application, or the application |
| 51 is in Split View on iPad. |
| 52 * Pressing the "Cancel" button of any error dialog dismisses the QR Scanner |
| 53 view controller. |
| 54 * If the camera becomes available when a dialog is presented, the dialog is |
| 55 automatically dismissed. |
| 56 |
| 57 ## Entry points |
| 58 |
| 59 The QR scanner can be accessed from the 3D Touch application shortcuts on |
| 60 supported devices. The `SpotlightActions` experiment allows the QR scanner to be |
| 61 accessed from Spotlight search. More info about Spotlight actions can be found |
| 62 at go/chrome-ios-spotlight and crbug.com/608733. Planned and rejected entry |
| 63 points are described in the design doc at go/chrome-ios-qr-code. |
| 64 |
| 65 Tests for QR Scanner are a part of `ios_chrome_ui_egtests`. |
| 66 |
| 67 ## Controller architecture |
| 68 |
| 69 * **QRScannerViewController** is the entry point for the feature. It connects |
| 70 the `CameraController` and the `QRScannerView` and is responsible for |
| 71 displaying alerts from `QRScannerAlerts`. |
| 72 * **CameraController** manages the `AVCaptureSession` for the camera. It is |
| 73 responsible for loading the camera, listening for camera notifications, |
| 74 receiving the scanned result and informing the `QRScannerViewController` |
| 75 about changes to the state of the camera or the torch. |
| 76 |
| 77 Operations performed by `CameraController` are done on a separate dispatch |
| 78 queue, as recommended by the [documentation][avcapturesession] for |
| 79 `AVCaptureSession`. |
| 80 |
| 81 [avcapturesession]: https://developer.apple.com/reference/avfoundation/avcapture
session |
| 82 |
| 83 ### Initialization |
| 84 |
| 85 `QRScannerViewController` owns an instance of `CameraController` and |
| 86 `QRScannerView` and is their delegate. |
| 87 |
| 88 1. The `getViewControllerToPresent` method checks if camera permission is |
| 89 granted by calling `checkPermissionsAndLoadCamera` of the |
| 90 `CameraController`. |
| 91 2. If the camera permission is denied, an error dialog will be returned from |
| 92 `getViewControllerToPresent`. If the user has not previously granted camera |
| 93 permission to the application, the `QRScannerViewController` instance will |
| 94 be returned, and an error will be displayed by the QR scanner if the user |
| 95 denies the permission in the system dialog. The error dialog prompts the |
| 96 user to change this setting and includes a link to the Settings app, if |
| 97 available. |
| 98 3. If the camera permission is granted, the `QRScannerViewController` will be |
| 99 returned as the view controller to present, and the `CameraController` will |
| 100 start loading the camera on a separate dispatch queue. |
| 101 |
| 102 #### Camera |
| 103 |
| 104 Camera initialization is handled by the `loadCaptureSession` method of the |
| 105 `CameraController`. |
| 106 |
| 107 1. Camera state is set to `CAMERA_UNAVAILABLE` and an error dialog is |
| 108 displayed if: |
| 109 * The back camera of the device is not found, |
| 110 * There was an error initializing the camera input, |
| 111 * The video input cannot be attached to the `AVCaptureSession`, |
| 112 * The metadata output cannot be attached to the `AVCaptureSession`, |
| 113 * The metadata output does not support QR code recognition. |
| 114 2. After a successful initialization, camera state is set to |
| 115 `CAMERA_AVAILABLE`, which is reported asynchronously to |
| 116 `QRScannerViewController`, and `CameraController` starts listening for |
| 117 camera notifications. |
| 118 3. Camera starts recording on `viewWillAppear`. |
| 119 |
| 120 #### Torch |
| 121 |
| 122 * Torch availability is checked when the camera initialization completes. |
| 123 * Torch is considered available, if the properties `hasTorch` and |
| 124 `isTorchAvailable` of the `AVCaptureDevice` are both `YES`. |
| 125 * During initialization, it is also checked if the torch supports the torch |
| 126 modes `AVCaptureTorchModeOn` and `AVCaptureTorchModeOff`. |
| 127 * Torch mode is set to off on initialization. |
| 128 |
| 129 #### Camera preview |
| 130 |
| 131 The `AVCaptureVideoPreviewLayer` is created by the `QRScannerView`: |
| 132 |
| 133 1. The `QRScannerView` is initialized by the `QRScannerViewController`. |
| 134 2. On `viewDidLoad`, `QRScannerViewController` calls `loadVideoPreviewLayer:` |
| 135 with the loaded preview. If the camera is already loaded, |
| 136 `CameraController` attaches the preview to the `AVCaptureSession`. Otherwise |
| 137 the preview is attached immediately after the `AVCaptureSession` is |
| 138 initialized. |
| 139 |
| 140 #### Viewport |
| 141 |
| 142 The rectangle of interest for the metadata output of the capture session is |
| 143 calculated to lie exactly inside the viewport drawn by the `QRScannerView`. |
| 144 Resetting the viewport causes the video preview to freeze for a short while, |
| 145 that is why the viewport is only set when the preview is hidden. |
| 146 |
| 147 1. `QRScannerViewController` sets the viewport on `viewDidAppear`, to make sure |
| 148 that the preview layer is of the correct size and position when the viewport |
| 149 is calculated. |
| 150 2. If the capture session is loaded, the viewport is set immediately. Otherwise |
| 151 the viewport is set after the capture session is loaded. |
| 152 3. `CameraController` calls the `cameraIsReady` method of its delegate to |
| 153 notify the `QRScannerViewController` that the viewport was successfully set |
| 154 and the camera preview can be displayed. |
| 155 |
| 156 ### Camera state and notifications |
| 157 |
| 158 `CameraController` is listening for the following notifications: |
| 159 |
| 160 1. `AVCaptureSessionRuntimeErrorNotification`, handled by setting the camera |
| 161 state to `CAMERA_UNAVAILABLE`, |
| 162 2. `AVCaptureSessionWasInterruptedNotification`, handled by setting the camera |
| 163 state to one of: |
| 164 * `APPLICATION_IN_BACKGROUND`, |
| 165 * `CAMERA_IN_USE_BY_ANOTHER_APPLICATION`, |
| 166 * `MULTIPLE_FOREGROUND_APPS`, |
| 167 based on the value of `AVCaptureSessionInterruptionReasonKey` in the |
| 168 notification's user info. |
| 169 3. `AVCaptureSessionInterruptionEndedNotification`, handled by setting the |
| 170 camera state to `CAMERA_AVAILABLE`. |
| 171 4. `AVCaptureDeviceWasDisconnected`, handled by setting the camera state to |
| 172 `CAMERA_UNAVAILABLE`. |
| 173 |
| 174 ### Torch state |
| 175 |
| 176 The current state of the torch is obtained using key-value observing of the |
| 177 `AVCaptureDevice` object. |
| 178 |
| 179 * Torch state is set based on the value of the `torchActive` property, and |
| 180 the delegate is informed using `torchStateChanged:`. |
| 181 * Torch availability is set based on the values of `hasTorch` and |
| 182 `torchAvailable`. Torch is only considered available if both properties are |
| 183 `YES` and the delegate is informed using `torchAvailabilityChanged:`. |
| 184 |
| 185 The delegate sets the value of the torch using `setTorchMode:` and is informed |
| 186 of the outcome asynchronously. |
| 187 |
| 188 ### Scanning |
| 189 |
| 190 `CameraController` implements the `AVCaptureMetadataOutputObjectsDelegate` and |
| 191 receives the scanned result on the main queue. |
| 192 |
| 193 * The scanned result must be an `AVMetadataMachineReadableCodeObject`. |
| 194 * Only results which are non-empty strings are passed on to the |
| 195 `QRScannerViewController`. |
| 196 * `CameraController` checks the type of the scanned code, and if the code can |
| 197 only contain numbers, sets the `loadImmediately` argument to `YES`. |
| 198 * If a valid code was scanned, the `CameraController` stops the capture |
| 199 session. |
| 200 |
| 201 Supported codes (from [Machine Readable Object Types][machinereadableobjects]): |
| 202 |
| 203 * Numeric-only bar codes: UPC-E, EAN-8, EAN-13, Interleaved 2 of 5, ITF-14 |
| 204 * Alphanumeric bar codes: Code 39, Code 39 Mod 43, Code 93, Code 128 |
| 205 * 2D alphanumeric codes: PDF417, AztecCode, DataMatrix, QR Code |
| 206 |
| 207 [machinereadableobjects]: https://developer.apple.com/reference/avfoundation/avm
etadatamachinereadablecodeobject/1668878-machine_readable_object_types?language=
objc |
| 208 |
| 209 ## Views |
| 210 |
| 211 The QR scanner consists of three views: |
| 212 |
| 213 * **VideoPreviewView** holds the camera preview. |
| 214 * **PreviewOverlayView** holds layers drawing the darker preview overlay and |
| 215 the viewport border. |
| 216 * **QRScannerView** contains the `VideoPreviewView`, `PreviewOverlayView` and |
| 217 controls as subviews. |
| 218 |
| 219 ### Transition animation |
| 220 |
| 221 `QRScannerTransitioningDelegate` implements a custom transition animation: |
| 222 the `QRScannerViewController` appears to be positioned below its presenting view |
| 223 controller. The presenting view controller slides up for presentation and down |
| 224 for dismissal. |
| 225 |
| 226 ### Screen rotation |
| 227 |
| 228 * `QRScannerView` and `PreviewOverlayView` rotate normally. |
| 229 `VideoPreviewView` does not rotate: on `viewWillTransitionToSize:` the view |
| 230 is animated to rotate in the opposite direction. This avoids resetting the |
| 231 viewport rectangle of interest on every screen rotation, because resetting |
| 232 it causes the video to pause for a while. The view is not positioned using |
| 233 AutoLayout. |
| 234 * `PreviewOverlayView` is a square that is `sqrt(2)`-times bigger than |
| 235 max(width, height) of the `QRScannerView`, to avoid redrawing the |
| 236 viewport. This also makes the viewport rotate in place. The view is |
| 237 positioned using AutoLayout. |
| 238 |
| 239 ### Split View |
| 240 |
| 241 * Camera is unavailable in Split View. |
| 242 * When the camera becomes available, the viewport rectangle is reset, |
| 243 otherwise the viewport would be in the wrong place when Split View is |
| 244 cancelled. |
| 245 |
| 246 ## Accessibility |
| 247 |
| 248 * Standard UI elements have accessibility labels and identifiers. |
| 249 * The value of the torch button is communicated using its accessibility value. |
| 250 * If VoiceOver is on, an accessibility announcement is played when a code is |
| 251 scanned, and the result is loaded after the announcement finishes. |
| 252 * If the result is loaded immediately, no additional accessibility notificatio
ns are |
| 253 posted. |
| 254 * If the result is placed in Omnibox for the user to review, the Omnibox |
| 255 should be focused afterwards. |
| 256 |
| 257 ## Metrics |
| 258 |
| 259 The following metrics are collected: |
| 260 |
| 261 * `ApplicationShortcut.ScanQRCodePressed` when the user opens the QR scanner |
| 262 from 3D Touch application shortcuts. |
| 263 * `MobileQRScannerClose` when the user closes the QR scanner without scanning |
| 264 a code. |
| 265 * `MobileQRScannerError` when the user closes the QR scanner from an error |
| 266 dialog. |
| 267 * `MobileQRScannerScannedCode` when the user scans a code. |
| 268 * `MobileQRScannerTorchOn` when the user switches on the torch. |
| 269 |
| 270 ## Known issues |
| 271 |
| 272 Screen rotation on iPad is not handled the same way as on iPhone, as of iOS 9. |
| 273 The counter-rotation animation is not played at the same time as the screen |
| 274 rotation animation, and the camera preview appears to be rotating. This effect |
| 275 is most visible when rotating the screen by 180 degrees, which results in an |
| 276 apparent double-rotation by 360 degrees. |
| 277 |
| 278 ## See also |
| 279 |
| 280 * go/chrome-ios-qr-code for the original design doc. |
OLD | NEW |