| OLD | NEW |
| (Empty) |
| 1 UI Events | |
| 2 ========= | |
| 3 | |
| 4 Scope | |
| 5 ----- | |
| 6 | |
| 7 The following input devices are supported by sky: | |
| 8 - fingers on multitouch screens | |
| 9 - mice, including mouse wheels | |
| 10 - styluses on screens | |
| 11 - other devices that emulate mice (track pads, track balls) | |
| 12 - keyboards | |
| 13 | |
| 14 The following input devices are not supported natively by sky, but can | |
| 15 be used by connecting directly to the mojo application servicing the | |
| 16 relevant device: | |
| 17 - joysticks | |
| 18 - track balls that move focus (or raw data from track balls) | |
| 19 - raw data from track pads (e.g. multitouch gestures) | |
| 20 - raw data from styluses that have their own absolute pads | |
| 21 - raw data from mice (e.g. to handle mouse capture in 3D games) | |
| 22 | |
| 23 The following interactions are intended to be easy to handle: | |
| 24 - one finger starts panning, another finger is placed on the surface | |
| 25 (and ignored), the first finger is lifted, and the second finger | |
| 26 continues panning (without the scroll position jumping when the | |
| 27 first finger is lifted) | |
| 28 - right-clicking doesn't trigger buttons by default | |
| 29 - fingers after the first within a surface don't trigger buttons by | |
| 30 default | |
| 31 - if there are two independent surfaces, they capture fingers | |
| 32 unrelated to each other | |
| 33 | |
| 34 Frameworks are responsible for converting pointer events described | |
| 35 below into widget-specific events such as the following: | |
| 36 - a click/tap/activation, as distinct from a short drag | |
| 37 - a context menu request (e.g. right-click, long-press) | |
| 38 - a drag (moving an item) | |
| 39 - a pan (scroll) | |
| 40 - a zoom/rotation (whether using two finger gestures, or one finger | |
| 41 with the double-tap-and-hold gesture) | |
| 42 - a double-tap autozoom | |
| 43 | |
| 44 In particular, this means distinguishing whether a finger tap consists | |
| 45 of a tap, a drag, or a long-press; it also means distinguishing | |
| 46 whether a drag, once established as such, should be treated as a pan | |
| 47 or a drag, and deciding whether a secondary touch should begin a | |
| 48 zoom/rotation or not. | |
| 49 | |
| 50 | |
| 51 Pointer events | |
| 52 -------------- | |
| 53 | |
| 54 Each touch or pointer is tracked individually. | |
| 55 | |
| 56 New touches and pointers can appear and disappear over time. | |
| 57 | |
| 58 Each pointer has a list of current targets. | |
| 59 | |
| 60 When a new one enters the system, a non-bubbling 'pointer-added' event | |
| 61 is fired at the application's document, and the pointer's current | |
| 62 targets list is initialised to just that Document object. | |
| 63 | |
| 64 When it is removed, a non-bubbling 'pointer-removed' event is fired at | |
| 65 the application's document and at any other objects in the pointer's | |
| 66 current targets list. Currently, at the time of a pointer-removed, the | |
| 67 list will always consist of only the document. | |
| 68 | |
| 69 A pointer can be "up" or "down". Initially all pointers are "up". | |
| 70 | |
| 71 A pointer switches from "up" to "down" when it is a touch or stylus | |
| 72 that is in contact with the display surface, or when it is a mouse | |
| 73 that is being clicked, and from "down" back to "up" when this ends. | |
| 74 (Note that clicking a button on a stylus doesn't change it from up to | |
| 75 down. A stylus can have a button pressed while "up".) In the case of a | |
| 76 mouse with multiple buttons, the pointer switches back to "up" only | |
| 77 when all the buttons have been released. | |
| 78 | |
| 79 When a pointer switches from "up" to "down", the following algorithm | |
| 80 is run: | |
| 81 | |
| 82 1. Hit test the position of the pointer, let 'node' be the result. | |
| 83 2. Fire a bubbling pointer-down event at the layoutManager for | |
| 84 'node', with an empty array as the default return value. Let | |
| 85 'result1' be the returned value. | |
| 86 3. If result1 is not an array of EventTarget objects, set it to the | |
| 87 empty array and (if this is debug mode) report the issue. | |
| 88 4. Fire a bubbling pointer-down event at the Element for 'node', with | |
| 89 an empty array as the default return value. Let 'result2' be the | |
| 90 returned value. | |
| 91 5. If result2 is not an array of EventTarget objects, set it to the | |
| 92 empty array and (if this is debug mode) report the issue. | |
| 93 6. Let result be the concatenation of result1's contents, result2's | |
| 94 contents, and the application document. | |
| 95 7. Let 'result' be this pointer's current targets. | |
| 96 | |
| 97 When an object is one of the current targets of a pointer and no other | |
| 98 pointers have that object as a current target so far, and either there | |
| 99 are no buttons (touch, stylus) or only the primary button is active | |
| 100 (mouse) and this is not an inverted stylus, then that pointer is | |
| 101 considered the "primary" pointer for that object. The pointer remains | |
| 102 the primary pointer for that object until the corresponding pointer-up | |
| 103 event (even if the buttons change). | |
| 104 | |
| 105 When a pointer moves, a non-bubbling 'pointer-move' event is fired at | |
| 106 each of the pointer's current targets in turn (maintaining the order | |
| 107 they had in the 'pointer-down' event, if there's more than one). If | |
| 108 the return value of a 'pointer-moved' event is 'cancel', and the | |
| 109 pointer is currently down, then the pointer is canceled (see below). | |
| 110 | |
| 111 When a pointer's button state changes but this doesn't impact whether | |
| 112 it is "up" or "down", e.g. when a mouse with a button down gets a | |
| 113 second button down, or when a stylus' buttons change state, but the | |
| 114 pointer doesn't simultaneously move, then a 'pointer-moved' event is | |
| 115 fired anyway, as described above, but with dx=dy=0. | |
| 116 | |
| 117 When a pointer switches from "down" to "up", a non-bubbling | |
| 118 'pointer-up' event is fired at each of the pointer's current targets | |
| 119 in turn (maintaining the order they had in the 'pointer-down' event, | |
| 120 if there's more than one), and then the pointer's current target list | |
| 121 is emptied except for the application's document. The buttons exposed | |
| 122 on the 'pointer-up' event are those that were down immediately prior | |
| 123 to the buttons being released. | |
| 124 | |
| 125 At the time of a 'pointer-up' event, for each object that is a current | |
| 126 target of the pointer, and for which the pointer is considered the | |
| 127 "primary" pointer for that object, if there is another pointer that is | |
| 128 already down, which is of the same kind, which also has that object as | |
| 129 a current target, and that has either no buttons or only its primary | |
| 130 button active, then that pointer becomes the new "primary" pointer for | |
| 131 that object before the 'pointer-up' event is sent. Otherwise, the | |
| 132 "primary" pointer stops being "primary" just _after_ the 'pointer-up' | |
| 133 event. (This matters for whether the 'primary' field is set.) | |
| 134 | |
| 135 When a pointer is canceled, if it is "down", pretend that the pointer | |
| 136 moved to "up", sending 'pointer-up' as described above, and entirely | |
| 137 empty its current targets list. AFter the pointer actually switches | |
| 138 from "down" to "up", replace the current targets list with an object | |
| 139 that only contains the application's document. | |
| 140 | |
| 141 Nothing special happens when a pointer's current target moves in the | |
| 142 DOM. | |
| 143 | |
| 144 The x and y position of an -up or -down event always match those of | |
| 145 the previous -moved or -added event, so their dx and dy are always 0. | |
| 146 | |
| 147 Positions are floating point numbers; they can have subpixel values. | |
| 148 | |
| 149 | |
| 150 These data of all these events is an object with the following fields: | |
| 151 | |
| 152 pointer: an integer assigned to this touch or pointer when it | |
| 153 enters the system, never reused, increasing | |
| 154 monotonically every time a new value is assigned, | |
| 155 starting from 1 (if the system gets a new tap every | |
| 156 microsecond, this will cause a problem after 285 | |
| 157 years) | |
| 158 | |
| 159 kind: one of 'touch', 'mouse', 'stylus', 'inverted-stylus' | |
| 160 | |
| 161 x: x-position relative to the top-left corner of the | |
| 162 surface of the node on which the event was fired | |
| 163 | |
| 164 y: y-position relative to the top-left corner of the | |
| 165 surface of the node on which the event was fired | |
| 166 | |
| 167 dx: difference in x-position since last pointer-moved | |
| 168 event | |
| 169 | |
| 170 dy: difference in y-position since last pointer-moved | |
| 171 event | |
| 172 | |
| 173 buttons: a bitfield of the buttons pressed, from the following | |
| 174 list: | |
| 175 | |
| 176 1: primary mouse button (not available on stylus) | |
| 177 | |
| 178 2: secondary mouse button, primary stylus button | |
| 179 | |
| 180 3: middle mouse button, secondary stylus button | |
| 181 | |
| 182 4: back button | |
| 183 | |
| 184 5: forward button | |
| 185 | |
| 186 additional buttons can be represented by numbers | |
| 187 greater than six: | |
| 188 | |
| 189 n: (n-2)th mouse button, ignoring any buttons that | |
| 190 are explicitly back or forward buttons | |
| 191 | |
| 192 (n-4)th stylus button, again ignoring any | |
| 193 explictly back or forward buttons | |
| 194 | |
| 195 note that stylus buttons can be pressed even when the | |
| 196 pointer is not "down" | |
| 197 | |
| 198 e.g. if the left mouse button and the right mouse | |
| 199 button are pressed at the same time, the value will | |
| 200 be 3 (bits 1 and 2); if the right mouse button and | |
| 201 the back button are pressed at the same time, the | |
| 202 value will be 10 (bits 2 and 4) | |
| 203 | |
| 204 down: true if the pointer is down (in pointer-down event or | |
| 205 subsequent pointer-move events); false otherwise (in | |
| 206 pointer-added, pointer-up, and pointer-removed | |
| 207 events, and in pointer-move events that aren't | |
| 208 between pointer-down and pointer-up events) | |
| 209 | |
| 210 primary: true if this is a primary pointer/touch (see above) | |
| 211 can only be set for pointer-moved and pointer-up | |
| 212 | |
| 213 obscured: true if the system was rendering another view on top | |
| 214 of the sky application at the time of the event (this | |
| 215 is intended to enable click-jacking protections) | |
| 216 | |
| 217 | |
| 218 When down is true: | |
| 219 | |
| 220 pressure: the pressure of the touch as a number ranging from | |
| 221 0.0, indicating a touch with no discernible pressure, | |
| 222 to 1.0, indicating a touch with "normal" pressure, | |
| 223 and possibly beyond, indicating a stronger touch; for | |
| 224 devices that do not detect pressure (e.g. mice), | |
| 225 returns 1.0 | |
| 226 | |
| 227 pressure-min: the minimum value that pressure can return for this | |
| 228 pointer | |
| 229 | |
| 230 pressure-max: the maximum value that pressure can return for this | |
| 231 pointer | |
| 232 | |
| 233 | |
| 234 When kind is 'touch', 'stylus', or 'stylus-inverted': | |
| 235 | |
| 236 distance: distance of detected object from surface (e.g. | |
| 237 distance of stylus or finger from screen), if | |
| 238 supported and down is not true, otherwise 0.0. | |
| 239 | |
| 240 distance-min: the minimum value that distance can return for this | |
| 241 pointer (always 0.0) | |
| 242 | |
| 243 distance-max: the maximum value that distance can return for this | |
| 244 pointer (0.0 if not supported) | |
| 245 | |
| 246 | |
| 247 When kind is 'touch', 'stylus', or 'stylus-inverted' and down is true: | |
| 248 | |
| 249 radius-major: the radius of the contact ellipse along the major | |
| 250 axis, in pixels | |
| 251 | |
| 252 radius-minor: the radius of the contact ellipse along the major | |
| 253 axis, in pixels | |
| 254 | |
| 255 radius-min: the minimum value that could be reported for | |
| 256 radius-major or radius-minor for this pointer | |
| 257 | |
| 258 radius-max: the maximum value that could be reported for | |
| 259 radius-major or radius-minor for this pointer | |
| 260 | |
| 261 | |
| 262 When kind is 'touch' and down is true: | |
| 263 | |
| 264 orientation: the angle of the contact ellipse, in radians in the | |
| 265 range | |
| 266 | |
| 267 -pi/2 < orientation <= pi/2 | |
| 268 | |
| 269 ...giving the angle of the major axis of the ellipse | |
| 270 with the y-axis (negative angles indicating an | |
| 271 orientation along the top-left / bottom-right | |
| 272 diagonal, positive angles indicating an orientation | |
| 273 along the top-right / bottom-left diagonal, and zero | |
| 274 indicating an orientation parallel with the y-axis) | |
| 275 | |
| 276 | |
| 277 When kind is 'stylus' or 'stylus-inverted': | |
| 278 | |
| 279 orientation: the angle of the stylus, in radians in the range | |
| 280 | |
| 281 -pi < orientation <= pi | |
| 282 | |
| 283 ...giving the angle of the axis of the stylus | |
| 284 projected onto the screen, relative to the positive | |
| 285 y-axis of the screen (thus 0 indicates the stylus, if | |
| 286 projected onto the screen, would go from the contact | |
| 287 point vertically up in the positive y-axis direction, | |
| 288 pi would indicate that the stylus would go down in | |
| 289 the negative y-axis direction; pi/4 would indicate | |
| 290 that the stylus goes up and to the right, -pi/2 would | |
| 291 indicate that the stylus goes to the left, etc) | |
| 292 | |
| 293 tilt: the angle of the stylus, in radians in the range | |
| 294 | |
| 295 0 <= tilt <= pi/2 | |
| 296 | |
| 297 ...giving the angle of the axis of the stylus, | |
| 298 relative to the axis perpendicular to the screen | |
| 299 (thus 0 indicates the stylus is orthogonal to the | |
| 300 plane of the screen, while pi/2 indicates that the | |
| 301 stylus is flat on the screen) | |
| 302 | |
| 303 | |
| 304 TODO(ianh): add an API that exposes the currently existing pointers, | |
| 305 so that you can determine e.g. if you have a mouse. | |
| 306 | |
| 307 TODO(ianh): determine what the update frequency of these events should | |
| 308 be. One set of events per frame? Multiple updates per frame? | |
| 309 | |
| 310 | |
| 311 Wheel events | |
| 312 ------------ | |
| 313 | |
| 314 When a wheel input device is turned, a 'wheel' event that bubbles is | |
| 315 fired at the application's document, with the following fields: | |
| 316 | |
| 317 wheel: an integer assigned to this wheel by the system. The | |
| 318 same wheel on the same system must always be given | |
| 319 the same ID. The primary wheel (e.g. the vertical | |
| 320 wheel on a mouse) must be given ID 1. | |
| 321 | |
| 322 delta: an floating point number representing the fraction of | |
| 323 the wheel that was turned, with positive numbers | |
| 324 representing a downward movement on vertical wheels, | |
| 325 rightward movement on horizontal wheels, and a | |
| 326 clockwise movement on wheels with a user-facing side. | |
| 327 | |
| 328 Additionally, if the wheel is associated with a pointer (e.g. a mouse | |
| 329 wheel), the following fields must be present also: | |
| 330 | |
| 331 pointer: the integer assigned to the pointer in its | |
| 332 'pointer-add' event (see above). | |
| 333 | |
| 334 x: x-position relative to the top-left corner of the | |
| 335 display, in global layout coordinates | |
| 336 | |
| 337 y: x-position relative to the top-left corner of the | |
| 338 display, in global layout coordinates | |
| 339 | |
| 340 Note: The only wheels that are supported are mouse wheels and physical | |
| 341 dials. Track balls are not reported as mouse wheels. | |
| 342 | |
| 343 | |
| 344 Text input events | |
| 345 ----------------- | |
| 346 | |
| 347 TODO(ianh): keyboard events | |
| OLD | NEW |