Visual motion feedback for user interface   

This application claims the benefit of U.S. Provisional Patent Application No. 61/304,004, filed on Feb. 12, 2010, entitled “MULTI-LAYER USER INTERFACE WITH FLEXIBLE MOVEMENT,” which is incorporated herein by reference.

The design of an effective user interface poses many challenges. One challenge is how to provide a user with an optimal amount of visual information or functionality, given the space limitations of a display and the needs of a particular user. This challenge can be especially acute for devices with small displays, such as smartphones or other mobile computing devices. This is because there is often more information available to a user performing a particular activity (e.g., browsing for audio or video files in a library of files) than can fit on the display. A user can easily become lost unless careful attention is paid to how information is presented on the limited amount available display space. Visual cues are useful for indicating, for example, a user\'s location when browsing a list or other collection of data, since it is often not possible to show an entire collection (e.g., a list of contacts stored in a smartphone) on a small display.

Another challenge is how to provide a high level of functionality while maintaining a satisfying and consistent user experience. As devices have become more complex, and as consumers have become more demanding, it has become increasingly difficult to design user interfaces that are convenient and pleasing for users, without sacrificing reliability, flexibility, functionality or performance.

Whatever the benefits of previous techniques, they do not have the advantages of the techniques and tools presented below.

SUMMARY

Techniques and tools are described that relate to different aspects of a user interface that provides visual feedback in response to user input. For example, boundary effects are presented to provide visual cues to a user to indicate that a boundary in a movable user interface element (e.g., the end of a scrollable list) has been reached. As another example, parallax effects are presented in which multiple parallel or substantially parallel layers in a multi-layer user interface move at different rates, in response to user input. As another example, simulated inertia motion of UI elements is used to provide a more natural feel for touch input. Various combinations of features are described. For example, simulated inertia motion can be used in combination with parallax effects, boundary effects, or other types of visual feedback.

In one aspect, a user interface (UI) system receives gesture information corresponding to a gesture on a touch input device. The UI system calculates simulated inertia motion for a movable user interface element based at least in part on the gesture information, and potentially on other inertia information such as a friction coefficient or a parking speed coefficient. Based at least in part on the gesture information and on the simulated inertia motion, the UI system calculates a post-gesture position of the movable user interface element. The UI system determines that the post-gesture position exceeds a gesture boundary of the movable user interface element, and calculates a distortion effect (e.g., a squeeze, compression or squish effect) in the movable user interface element to indicate that the gesture boundary has been exceeded. Calculating the distortion effect can include, for example, determining an extent by which the gesture boundary has been exceeded, determining a compressible area of the movable user interface element, determining a scale factor for the distortion effect based at least in part on the compressible area and the extent by which the gesture boundary has been exceeded, and scaling the compressible area according to the scale factor. The distortion effect can be calculated based on a distortion point (which, for compression, can be referred to as a compression point or squeeze point), which can indicate the part of the UI element to be distorted.

In another aspect, user input (e.g., a gesture on a touch screen) indicates movement in a graphical user interface element having plural movable layers. Based at least in part on inertia information and the user input, a UI system calculates a first motion having a first movement rate in a first layer of the plural movable layers, and calculates a parallax motion in a second layer of the plural movable layers. The parallax motion is based at least in part on the first motion (and potentially simulated inertia motion), and the parallax motion comprises a movement of the second layer at a second movement rate that differs from the first movement rate. The parallax motion can be calculated based on, for example, a parallax constant for the second layer, or an amount of displayable data in the second layer.

In another aspect, a UI system receives gesture information corresponding to a gesture on a touch input device, the gesture information indicating a movement of a user interface element having a movement boundary. Based at least in part on the gesture information, the UI system computes a new position of the user interface element. Based at least in part on the new position, the UI system determines that the movement boundary has been exceeded. The UI system determines an extent by which the movement boundary has been exceeded, determines a compressible area of the user interface element, determines a scale factor for a distortion effect based at least in part on the compressible area and the extent by which the movement boundary has been exceeded, and presents a distortion effect in the user interface element. The distortion effect comprises a visual compression of content in the compressible area (e.g., text, images, graphics, video or other displayable content) according to the scale factor. Depending, for example, on the size of the compressible area and the size of the display area, some parts of the compressible area may not be visible on a display, so the distortion can be virtual (e.g., in areas that are not visible on a display) or the distortion can be actually displayed, or some parts of the distorted content can be displayed while other parts of the distorted content are not displayed. The visual compression is in a dimension that corresponds to the movement of the user interface element. For example, a vertical movement in a UI element that exceeds a movement boundary can cause content in the UI element to be vertically compressed or squeezed.

The foregoing and other objects, features, and advantages of the invention will become more apparent from the following detailed description, which proceeds with reference to the accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-1C and 2 are flow charts showing example techniques for presenting motion feedback in user interface elements, according to one or more described embodiments.

FIG. 3 is a diagram showing a boundary effect, according to one or snore described embodiments.

FIGS. 4A-4C are diagrams showing parallax effects, according to one or more described embodiments.

FIGS. 5 and 6A-6E are diagrams showing parallax effects and boundary effects in a user interface having parallel layers, according to one or more described embodiments.

FIGS. 7A, 7B, 8A and 8B are diagrams showing gesture boundary areas which can be used to determine whether to present boundary effects, according to one or more described embodiments.

FIG. 9 is a diagram showing example pinch and stretch gestures, according to one or more described embodiments.

FIG. 10 is a graph showing changes in position over time of a UI element that exhibits a boundary feedback effect, according to one or more described embodiments.

FIG. 11 is a system diagram showing a UI system in which described embodiments can be implemented.

FIG. 12 illustrates a generalized example of a suitable computing environment in which several of the described embodiments may be implemented.

FIG. 13 illustrates a generalized example of a suitable implementation environment in which one or more described embodiments may be implemented.

FIG. 14 illustrates a generalized example of a mobile computing device in which one or more described embodiments may be implemented.

DETAILED DESCRIPTION

Techniques and tools are described that relate to different aspects of a user interface that provides visual feedback in response to user input. For example, boundary effects are presented to provide visual cues to a user to indicate that a boundary in a movable user interface element (e.g., the end of a scrollable list) has been reached. As another example, parallax effects are presented in which multiple parallel or substantially parallel layers in a multi-layer user interface move at different rates, in response to user input. As another example, simulated inertia motion of UI elements is used to provide a more natural feel for touch input. Various combinations of features are described. In one implementation, a UI system that accepts touch input includes detailed motion rules (e.g., rules for interpreting different kinds of touch input, rules for presenting inertia motion in UI elements in response to touch input, rules for determining boundaries in UI elements, etc.). The motion rules can be combined with various combinations of optional motion features such as parallax effects, boundary effects, and other visual feedback. The visual feedback that is presented according to motion rules and optional motion features in a UI element can depend on many factors, such as the type of the UI element and the content of the UI element.

Various alternatives to the implementations described herein are possible. For example, techniques described with reference to flowchart diagrams can be altered by changing the ordering of stages shown in the flowcharts, by repeating or omitting certain stages, etc. As another example, systems described with reference to system diagrams can be altered by changing the ordering of processing stages shown in the diagrams, by repeating or omitting certain stages, etc. As another example, user interfaces described with reference to diagrams can be altered by changing the content or arrangement of user interface features shown in the diagrams, by omitting certain features, etc. As another example, although some implementations are described with reference to specific devices and user input mechanisms (e.g., mobile devices with a touchscreen interface), described techniques and tools can be used with other devices and/or user input mechanisms.

The various techniques and tools can be used in combination or independently. Different embodiments implement one or more of the described techniques and tools.

As devices have become more complex, and as consumers have become snore demanding, it has become increasingly difficult to design user interfaces that are convenient and pleasing for users, without sacrificing reliability, flexibility, functionality or performance. The feel of a user interface (UI) is becoming increasingly important to distinguish the underlying product from its competitors. An important contributor to the feel of a UI is how it reacts when a user interacts with it. This is especially true for touch-based interfaces.

Accordingly, techniques and tools are described for providing feedback (e.g., visual cues such as parallax effects, boundary effects, etc.) to users in response to user input (e.g., touch input). In some embodiments, movements in elements (also referred to as “controls”) are based at least in part on user input (e.g., gestures on a touchscreen) and an inertia model. For example, a movement in a UI element can be extended beyond the actual size of a gesture on a touchscreen by applying inertia to the movement. Applying inertia to a movement in a UI element typically involves performing one more calculations using gesture information (e.g., a gesture start position, a gesture end position, gesture velocity and/or other information) and one or more inertia motion values (e.g., friction coefficients) to determine a post-gesture state (e.g., a new position) for the UI element. Simulated inertia motion can be used in combination with other effects (e.g., parallax effects, boundary effects, etc.) to provide feedback to a user. In any of the examples herein, movements in UI elements can be rendered for display (e.g., depicting calculated distortion, parallax, or other effects, if any).

Movement in UI elements typically depends to some extent on user interaction. For example, a user that wishes to navigate from one part of a UI element to another (e.g., from the beginning of a scrollable list to the end of the list) provides user input to indicate a desired movement. The user input can then cause movement in the UI element and potentially other elements in the user interface. In some embodiments, a user causes movement in a display area of a device by interacting with a touchscreen. The interaction can include, for example, contacting the touchscreen with a fingertip, stylus or other object and moving it (e.g., with a flicking or sweeping motion) across the surface of the touchscreen to cause movement in a desired direction. Alternatively, a user can interact with a user interface in some other way, such as by pressing buttons (e.g., directional buttons) on a keypad or keyboard, moving a trackball, pointing and clicking with a mouse, making a voice command, etc.

The actual amount and direction of the user\'s motion that is necessary to produce particular movements can vary depending on implementation or user preferences. For example, a user interface system can include a default setting that is used to calculate the amount of motion (e.g., in terms of pixels) as a function of the size or rate of a user movement. As another example, a user can adjust a touchscreen sensitivity control, such that the same motion of a fingertip or stylus on a touchscreen will produce smaller or larger movements, depending on the setting of the control. Gestures can be made in various directions to cause movement in UI elements. For example, upward and downward gestures can cause upward or downward movements, respectively, while rightward and leftward movements can cause rightward and leftward movements, respectively. Upward/downward motion can even be combined with left/right motion for diagonal movements. Other kinds of motion, such as non-linear motion (e.g., curves) or bi-directional motion (e.g., pinch or stretch motions made with multiple contact points on a touchscreen) also can be used to cause movement in UI elements.

FIG. 1A is a flow chart showing a general technique 100 for providing motion feedback in a UI. At 101, a device receives user input indicating motion in a UI element. For example, a UI system on a mobile device receives gesture information corresponding to a gesture on a touchscreen on the mobile device. At 102, the device determines whether inertia will be applied to the motion indicated by the user input. For example, a UI system determines based on gesture information (e.g., gesture start position, gesture end position, gesture direction, gesture velocity) whether to apply inertia to the motion in the UI element. At 103, the device determines whether visual effects (e.g., boundary effects, parallax effects, etc.) will be applied to the motion indicated by the user input. For example, the device determines whether to apply a distortion effect (e.g., a compression or squeeze effect) to indicate that a boundary in the UI element (e.g., a boundary at the end of a scrollable list) has been reached. As another example, the device determines whether to apply a parallax effect (e.g., by moving parallel layers in a multi-layer UI element at different rates). The applied effects also can be based on inertia, where inertia is applied to the motion indicated by the user input. For example, if a UI system applies inertia to a movement and calculates, based on the inertia, a new position for a UI element that is outside a boundary for the UI element, the UI system can apply a boundary effect to provide a visual indicator that the boundary has been reached. At 104, the motion in the UI element is rendered for display.

FIG. 19 is a flow chart showing a technique 110 for providing boundary effects in combination with inertia motion. At 111, a UI system receives gesture information corresponding to a gesture. For example, the UI system receives gesture coordinates and velocity information for the gesture. At 112, the UI system calculates inertia motion based on the gesture information. For example, the UI system determines that inertia motion is applied based on the velocity information, and calculates a duration of inertia motion for the gesture. At 113, the UI system calculates a post-gesture position based on the gesture information and the inertia motion. For example, the UI system calculates the post-gesture position based on the gesture coordinates and the duration of the inertia motion. At 114, the UI system determines that a boundary for the UI element has been exceeded. For example, the UI system compares one or more coordinates (e.g., vertical or horizontal coordinates) of the post-gesture position and determines an extent by which the post-gesture position exceeds the boundary. At 115, the UI system calculates a distortion effect to indicate that the boundary has been exceeded. For example, the UI system calculates a squeeze or compression effect in the content of the UI element based on the extent to which the post-gesture position exceeds the boundary.

FIG. 1C is a flow chart showing a technique 120 for providing parallax effects in combination with inertia motion. At 121, a UI system receives user input indicating motion in a UI element having plural layers. For example, the UI system receives gesture coordinates and velocity information for a gesture on a touch screen, where the gesture is directed to a content layer in multi-layer UI. At 122, the UI system calculates motion in a first layer based on inertia information and the user input. For example, the UI system determines that inertia motion should be applied to movement in the content layer based on the velocity information, and calculates a duration of inertia motion for the movement. At 123, the UI system calculates a parallax motion in a second layer based on the first motion in the first layer. For example, the UI system calculates the parallax motion in a layer above the content layer based on the motion in the content layer, with the parallax motion having a different movement rate than the motion in the content layer. The parallax motion also can include inertia motion, or inertia motion can be omitted in the parallax motion.

In any of the above techniques, any combination of the inertia, boundary, parallax, distortion, and other effects described herein can be applied. Depending on implementation and the type of processing desired, processing stages shown in example techniques 100, 110, 120 can be rearranged, added, omitted, split into multiple stages, combined with other stages, and/or replaced with like stages.

FIG. 2 is a flow chart showing a detailed example technique 200 for providing visual feedback in a UI in response to a user gesture.

At 210, a UI system on a device receives touch input information in a touch input stream. For example, the touch input stream comprises data corresponding to a gesture on a touchscreen of a mobile device. Data received from the touch input stream can include, for example, gesture information such as a gesture start position, a gesture end position, and timestamps for the gesture. The touch input stream is typically received from a device operating system, which converts raw data received from a touch input device (e.g., a touchscreen) into gesture information. Alternatively, data received from the touch input stream can include other information, or gesture information can be received from some other source.

At 212, filtering is applied to the touch input stream. In the filtering stage, one or more algorithms are applied to the touch input stream coming from the OS to filter out or correct anomalous data. For example, the filtering stage can correct misaligned touch data caused by jittering (e.g., values that are not aligned with previous inputs) or filter out spurious touch contact points (e.g., incorrect interpretation of a single touch point as multiple touch points that are close together), etc. As another example, if only single-touch-point gestures are allowed, the filtering stage can convert any multi-touch input into a single-touch input. Alternatively, touch input filtering can be performed during generating of the touch input stream (e.g., at the device OS). As another alternative, touch input filtering can be performed during a coordinate space transform stage (e.g., coordinate space transform 220). As another alternative, touch input filtering can be omitted.

At 220, the UI system applies a coordinate space transform to data in the touch input stream corresponding to the gesture. For example, a coordinate space transform is applied to the data from the touch input stream in order to account for possible rotations of the device, scale changes, influence from other animations, etc., in order to properly interpret the original input stream. For example, if a UI element is rotated 90 degrees such that vertical movement in the UI element becomes horizontal movement (or vice versa), a vertical gesture can be transformed to a horizontal gesture (or vice versa) to account for the rotation of the device. If no adjustments are necessary, the coordinate space transform can leave gesture information unchanged. Alternatively the coordinate space transform state can be omitted.

At 230, the UI system calculates the velocity at the end of the gesture. For example, the velocity is calculated by determining a first position near the end of the gesture and an end position of the gesture, and dividing by the time elapsed during the movement from the first position near the end of the gesture to the end position. In one implementation, the first position is determined by finding the gesture position at approximately 100 ms prior to the end of the gesture. Measuring velocity near the end of the gesture can help to provide a more accurate motion resulting from the gesture than measuring velocity over the entire course of the gesture. For example, if a gesture starts slowly and ends with a higher velocity, measuring the velocity at the end of the gesture can help to more accurately reflect the user\'s intended gesture (e.g., a strong flick). Alternatively, the velocity is calculated by determining the distance (e.g., in pixel units) between the start position for the gesture and the end position of the gesture, and dividing by the time elapsed during the movement from the start position to the end position. The time elapsed can be calculated, for example, by computing the difference between a timestamp associated with the start position and a timestamp associated with the end position.

At 240, the UI system determines whether the gesture is an inertia gesture. As used herein, an inertia gesture refers to a gesture, such as a flick gesture, capable of causing movement in one or more user interface elements to which inertia can be applied. The UI system can distinguish between a non-inertia gesture and an inertia gesture by determining how quickly the user\'s finger, stylus, etc., was moving when it broke contact with the touchscreen, and whether the velocity exceeds a threshold. If the gesture ends with a velocity above the threshold, the gesture can be interpreted as an inertia gesture. For example, a gesture that starts with panning motion at a velocity below the threshold and ends with a velocity above the threshold can be interpreted as ending with a flick that causes movement to which inertia can be applied. If the gesture ends with a velocity below the threshold, the gesture can be interpreted as a non-inertia gesture. Exemplary techniques and tools used in some implementations for gesture interpretation are described in detail below.

If the gesture is an inertia gesture (e.g., a flick gesture), at 250 the UI system determines whether inertia will be applied to the motion indicated by the gesture. For example, the UI system determines based on gesture information (e.g., end-of-gesture velocity) and/or other information (e.g., user preferences) whether to apply inertia to the motion in the UI element. Despite being considered an inertia gesture, a gesture such as a flick may still not have inertia applied to its resulting movements, such as when a flick gesture is received for a UI element that does not support inertia movements, or for a UI element for which inertia movement has been deactivated (e.g., according to user preference).

If inertia is not to be applied (e.g., when the gesture is not an inertia gesture), at 254 the UI system computes a new position for the UI element based on gesture information (e.g., end-of-gesture position coordinates). If inertia is to be applied, at 252 the UI system computes a new position based on the gesture information (e.g., end-of-gesture position coordinates) and simulated inertia. For example, the simulated inertia involves treating a UI element, or part of a UI element, as a physical object of non-zero mass that moves according to an approximation of Newtonian physics. The approximation can include, for example, a friction coefficient and/or other parameters that control how the movement is calculated and/or rendered.

When the new position of the UI element has been computed (with or without simulated inertia), the UI system determines at 260 whether boundary feedback will be presented. Determining whether boundary feedback will be presented involves determining whether the new position is within boundaries (if any) of the UI element. For example, in a scrollable list, the UI system can determine whether the new position is calculated to be outside the boundaries of the scrollable list (e.g., below the end of a vertically scrollable list). Some UI elements may not have boundaries that can be exceeded by any permitted motion. For example, a UI element may take the form of a wrappable list, which may have a default entry position but no beginning or end. If the wrappable list is axis-locked (e.g., if movement is only permitted along a vertical axis for a vertically scrolling list), the list may have no boundaries that can be exceeded by any permitted motion. For UI elements without any boundaries, or without boundaries that can be exceeded by permitted motion, the determination of whether the new position is within boundaries can be skipped. Axis locking is described in more detail below.

If boundary feedback is to be presented, at 262 the UI system applies a boundary effect to the UI element. For example, the UI system can apply a visual distortion effect such as a “squish” or compression of text, images or other visual information in the UI element, to provide a visual cue that a boundary of the UI element has been reached. Boundary effects are described in more detail below.

The UI system determines at 270 whether parallax feedback will be presented. Determining whether parallax feedback will be presented involves determining whether the UI element has multiple parallel layers or substantially parallel layers that can be moved at different rates based on the same gesture. If parallax feedback is to be presented, at 272 the UI system applies a parallax effect to the UI element. In general, a parallax effect involves movement of multiple parallel layers, or substantially parallel layers, at different rates. Example parallax effects are described in more detail below.

The processing stages in example technique 200 indicate example flows of information in a UI system. Depending on implementation and the type of processing desired, processing stages can be rearranged, added, omitted, split into multiple stages, combined with other stages, and/or replaced with like stages.

For example, although example technique 200 shows stages of receiving data from a touch input stream, applying touch input filtering, applying a coordinate space transform, calculating a velocity at the end of a gesture, and determining whether the gesture is an inertia gesture, such processing stages are only exemplary. Gesture information (e.g., gesture velocity, position, whether the gesture is a candidate for simulated inertia, etc.) can be obtained in other ways. As an example, a module that determines whether to apply inertia motion and determines whether to apply boundary feedback or parallax effects can obtain gesture data from another source, such as another module that accepts touch input and makes calculations to obtain gesture information (e.g., gesture velocity, end-of-gesture position).

As another example, although example technique 200 shows a determination of whether to present boundary feedback occurring before a determination of whether to present parallax feedback, such an arrangement is only exemplary. A determination of whether to present boundary feedback and/or parallax feedback can be performed in other ways. As examples, once a new position has been calculated, determinations of whether to present boundary feedback and/or parallax feedback can occur in parallel, or the determination of whether to present a parallax effect can occur before the determination of whether to present a boundary effect. Such arrangements can be useful, for example, where a gesture may cause movements in multiple parallel layers of a UI element prior to reaching a boundary of the element. A UI system also can determine (e.g., based on characteristics of a current UI element) whether boundary effects and/or parallax effects are not available (e.g., for UI elements that do not have multiple layers or boundaries), and skip processing stages that are not relevant.

Boundary feedback can be used to provide visual cues to a user to indicate that a boundary (e.g., a boundary at the end, beginning, or other location) in a UI element (e.g., a data collection such as a list) has been reached. In described implementations, a UI system presents a boundary effect in a UI element (or a portion of a UI element) by causing the UI element to be displayed in a visually distorted state, such as a squeezed or compressed state (i.e., a state in which text, images or other content is shown to be smaller than normal in one or more dimensions), to indicate that a boundary of the UI element has been reached.

Described techniques and tools for presenting boundary feedback can be applied to any UI element with one or more boundaries that can be manipulated by moving the element. For example, described techniques and tools can be used in an email viewer, such that text in a scrollable email message is distorted (e.g., squeezed or compressed) to indicate that the end of the email message has been reached.

Boundary effects (e.g., distortion effects) can be presented in different ways. For example, a boundary effect can be held in place for different lengths of time depending on user input and/or design choice. A boundary effect can end, for example, by returning the UI element to a normal (e.g., undistorted) state when a user lifts a finger, stylus or other object to end an interaction with a touchscreen after reaching a boundary, or when an inertia motion has completed. As another example, distortion effects other than a squish, squeeze or compression can be used. One alternative distortion effect is a visual stretch. A stretch effect can be used, for example, in combination with a snap-back animation to indicate that boundary has been reached.

Boundary effects can be presented even when it is possible to continue a movement beyond a boundary. For example, if a user scrolls to the end of a vertically-oriented list, causing a distortion of text or images at the end of the list, further motion can cause the list to wrap past the boundary and back to the beginning of the list. The UI also can show an element (or part of an element) at the top of the list to indicate that further movement can allow the user to wrap back to the beginning of the list.

FIG. 3 is a diagram showing a graphical user interface (GUI) presented by a UI system that uses a distortion effect to indicate that a boundary of UI element has been reached. According to the example shown in FIG. 3, a user 302 (represented by the hand icon) interacts with a list comprising list elements (“Contact1,” “Contact2,” etc.). In this example, distortion effects depend at least in part on the location of a squeeze point 396. Some list elements with distortion effects are shown as being outside display area 300.

FIG. 3 shows example states 390-394. In state 390, user 302 interacts with a touchscreen by making an upward motion, indicated by an initial gesture position 350 and an end-of-gesture touch position 352. The interaction can include, for example, contacting the touchscreen with a fingertip, stylus or other object and moving it (e.g., with a flicking or sweeping motion) along the surface of the touchscreen. Although FIG. 3 shows user 302 interacting with the touchscreen at particular locations in the display area 300, the UI system allows interaction with other parts of the touchscreen to cause movement in the list. Furthermore, although the example shown in FIG. 3 shows user 302 making an upward motion to scroll towards the end of the list, user 302 also can make other motions (e.g., downward motions to scroll towards the beginning of the list). The UI system can interpret different kinds of upward or downward user movements, even diagonal movements extending to the right or left of the vertical plane, as a valid upward or downward motion.

From state 390, the upward motion causes a distortion effect shown in state 392. In this example, the upward motion is finger-tracking motion caused by a drag gesture, but distortion effects also can be caused by other motion resulting from other kinds of gestures, such as inertia motion caused by a flick gesture. The distortion effect indicates that a boundary in the list has been reached. In the example shown in state FIG. 3, the entire list is treated as a single surface, as indicated by the single dimension line to the right of the list in states 390, 392 and 394, respectively. In state 392, the list has been squeezed or compressed in a vertical dimension, as shown by the reduced length of the dimension to the right of the list. The text of each list element has been squeezed or compressed in a vertical dimension. The elements are distorted proportionally. The effect in state 392 is as if all the list elements are being compressed against a barrier at the squeeze point 396.

In the example shown in state 392, the squeeze point 396 is indicated at the top of a list, outside the display area 300. Other squeeze points are also possible. For example, the squeeze point could be at the center of a list (e.g., at item 50 in a 100 item list) or at the top of a visible portion of a list. In this example, the list can be considered as having two parts—one part above the squeeze point, and one part below the squeeze point—where only one part of the list is squeezed. The squeeze point can change dynamically, depending on the state of the list and/or display. For example, a squeeze point can move up or down (e.g., in response to where the center of the list is) as elements are added to or removed from the list, or a squeeze point can update automatically (e.g., when the end of the list has been reached) to be at the top of a visible portion of the list. As another example, a squeeze point can be placed outside of a list. This can be useful to provide more consistent visual feedback, such as when a UI element does not fill the visible area.

In state 394, the list has returned to the undistorted state shown in state 390. For example, the list can return to the undistorted state after the gesture shown in state 390 is ended (e.g., when the user breaks contact with the touchscreen).

The upward motion shown in FIG. 3 is only an example of a possible user interaction. The same motion and/or other user interactions (e.g., motions having different sizes, directions, or velocities) can cause different effects, different display states, different transitions between display states, etc. For example, a motion that causes a distortion effect in a UI element (e.g., at the end of a vertically scrollable list) also can cause another portion of the UI element (e.g., a list item at the beginning of a vertically scrollable list) to be displayed to indicate availability of a wrapping feature in the list. Further movement can then cause wrapping in the UI element (e.g., from the end back to the beginning of a vertically scrollable list).

States 390-394 are only examples of possible states. In practice, a UI element can exist in any number of states (e.g., in intermediate states between example states 390-394, etc.) in addition to, or as alternatives to, the example states 390-394. For example, it is preferable to show a gradual transition between an undistorted state (e.g., state 390) and a distorted state (e.g., state 392), or from a distorted state to an undistorted state, to provide a more natural feel and avoid the appearance of abrupt changes in the display. Intermediate states, such as states that may occur between state 390 and state 392, or between state 392 and state 394 can show gradually increasing or decreasing degrees of distortion, as appropriate.

In described embodiments, a UI system can present parallel, or substantially parallel, movable layers. The UI system can present a parallax effect, in which layers move at different speeds relative to one another. The effect is referred to as a parallax effect because, in a typical example, a layer that is of interest to a user moves at a faster rate than other layers, as though the layer of interest were closer to the user than the other, slower-moving layers. However, the term “parallax effect” as used herein refers more generally to effects in which layers move at different rates relative to one another.

The rate of movement in each layer can depend on several factors, including the amount of data to be presented visually (e.g., text or graphics) in the layers, or the arrangement of the layers relative to one another. The amount of data to be presented visually in a layer can measured by, for example, determining the length as measured in a horizontal direction of the data as rendered on a display or as laid out for possible rendering on the display. Length can be measured in pixels or by some other suitable measure (e.g., the number of characters in a string of text). A layer with a larger amount of data and moving at a faster rate can advance by a number of pixels that is greater than a layer with a smaller amount of data moving at a slower rate. Layer movement rates can be determined in different ways. For example, movement rates in slower layers can be derived from movement rates in faster layers, or vice versa. Or, layer movement rates can be determined independently of one another. Layers that exhibit parallax effects can be overlapping layers or non-overlapping layers.

When user interaction causes movement in layers, the movement of the layers is a typically a function of the length of the layers and the size and direction of the motion made by the user. For example, a leftward flicking motion on a touchscreen produces a leftward movement of the layers relative to the display area. Depending on implementation and/or user preferences, user input can be interpreted in different ways to produce different kinds of movement in the layers. For example, a UI system can interpret any movement to the left or right, even diagonal movements extending well above or below the horizontal plane, as a valid leftward or rightward motion of a layer, or the system can require more precise movements. As another example, a UI system can require that a user interact with a part of a touchscreen corresponding to the display area occupied by a layer before moving that layer, or the system can allow interaction with other parts of the touchscreen to cause movement in a layer. As another example, a user can use an upward or downward motion to scroll up or down in a part of the content layer that does not appear on the display all at once.

In some embodiments, lock points indicate corresponding positions in layers with which a display area of a device will be aligned. For example, when a user navigates to a position on a content layer such that the left edge of the display area is at a left-edge lock point “A,” the left edge of display area will also be aligned at a corresponding left-edge lock point “A” in each of the other layers. Lock points also can indicate alignment of a right edge of a display area (right-edge lock points), or other types of alignment (e.g., center lock points). Typically, corresponding lock points in each layer are positioned to account for the fact that layers will move at different speeds. For example, if the distance between a first lock point and a second lock point in a content layer is twice as great as the distance between corresponding first and second lock points in a background layer, the background layer moves at half the rate of the content layer when transitioning between the two lock points.

In addition to indicating corresponding positions in layers, lock points can exhibit other behavior. For example, lock points can indicate positions in a content layer to which the layer will move when the part of the layer corresponding to the lock point comes into view on the display. This can be useful, for example, when an image, list or other content element comes partially into view near an edge of the display area—the content layer can automatically bring the content element completely into view by moving the layer such that an edge of the display area aligns with an appropriate lock point. A lock animation can be performed at the end of a gesture, such as a flick or pan gesture, to align the layers with a particular lock point. As an example, a lock animation can be performed at the end of a gesture that causes movement of a content layer to a position between two elements in a content layer (e.g., where portions of two images in a content layer are visible in a display area). A UI system can select an element (e.g., by checking which element occupies more space in the display area) and transition to focus on that element using the lock animations. This can improve the overall look of the layers and can be effective in bringing information or functional elements into view in a display area. A lock animation also can be used together with simulated inertia motion. For example, a lock animation can be presented after inertia motion stops, or a lock animation can be blended with inertia motion (such as by extending inertia motion to a lock point, or ending inertia motion early by gradually coming to a stop at a lock point) to present a smooth transition to a lock point.

The amounts and rates of movements presented in parallax effects can be calculated and presented in different ways. In a detailed example described below, equations are described for calculating parallax effect movements in which a parallax constant is used to determine anew position for a layer after a gesture. As another example, motion in layers and/or other elements, such as lists, can be calculated based on motion ratios. For example, a UI system can calculate motion ratios for a background layer and a title layer by dividing the width of the background layer and the width of the title layer, respectively, by a maximum width of the content layer. Taking into account the widths of the background layer and the title layer, a system can map locations of lock points in the background layer and the title layer, respectively, based on the locations of corresponding lock points in the content layer.

Movement of various layers can differ depending on context. For example, a user can navigate left from the beginning of a content layer to reach the end of a content layer, and can navigate right from the end of the content layer to reach the beginning of a content layer. This wrapping feature provides more flexibility when navigating through the content layer. Wrapping can be handled by the UI system in different ways. For example, wrapping can be handled by producing an animation that shows a rapid transition from the end of layers such as title layers or background layers back to the beginning of such layers, or vice-versa. Such animations can be combined with ordinary panning movements in the content layer, or with other animations in the content layer, such as locking animations. However, wrapping functionality is not required.

FIGS. 4A-4C are diagrams showing a GUI presented by a UI system with three layers 410, 412, 414 and a background layer 450. In this example, a user 302 (represented by the hand icon) interacts with content layer 414 by interacting with a touchscreen having a display area 300.

Background layer 450 floats behind the other layers. Data to be presented visually in background layer 450 can include, for example, an image that extends beyond the boundaries of display area 300. The content layer 414 includes content elements (e.g., images) 430A-H. Layers 410, 412 include text information (“Category” and “Selected Subcategory,” respectively). The length of content layer 414 is indicated to be approximately twice the length of layer 412, which is in turn indicated to be approximately twice the length of layer 410. The length of background layer 450 is indicated to be slightly less than the length of layer 412.

In FIGS. 4A-4C, the direction of motion that can be caused in the layers 410, 412, 414, 450 by user 302 is indicated by a left-pointing arrow and a right-pointing arrow. These arrows indicate possible movements (left or right horizontal movements) of layers 410, 412, 414, 450 in response to user movements. In this example, the system interprets user movements to the left or right, even diagonal movements extending above or below the horizontal plane, as a valid leftward or rightward motion of a layer. Although FIGS. 4A-4C show user 302 interacting with a portion of display area 300 that corresponds to content layer 414, the system also allows interaction with other parts of the touchscreen (e.g., parts that correspond to portions of display area 300 occupied by other layers) to cause movement in layers 410, 412, 414, 450.

When user input indicates a motion to the right or left, the system produces a rightward or leftward movement of the layers 410, 412, 414, 450 relative to display area 300. The amount of movement of layers 410, 412, 414, 450 is a function of the data in the layers and the size or rate of the motion made by the user.

In FIGS. 4A-4C, example left-edge lock points “A,” “B” and “C” are indicated for layers 410, 412. 414. 450. The left-edge lock points indicate the corresponding position of the left edge of the display area 300 on each layer. For example, when a user navigates to a position on content layer 414 such that the left edge of display area 300 is at lock point “A,” the left edge of display area 300 will also be aligned at lock point “A” of the other layers 410, 412, 450, as shown in FIG. 4A. In FIG. 4B, the left edge of display area 300 is at lock point “B” in each of the layers 410, 412, 414, 450. In FIG. 4C, the left edge of the display area 300 is at lock point “C” in each of the layers 410, 412, 414, 450.

The lock points shown in FIGS. 4A-4C are not generally representative of a complete set of lock points, and are limited to lock points “A,” “B” and “C” only for brevity. For example, left-edge lock points can be set for each of the content elements 430A-430H. Alternatively, fewer lock points can be used, or lock points can be omitted. As another alternative, lock points can indicate other kinds of alignment. For example, right-edge lock points can indicate alignment with the right edge of display area 300, or center lock points can indicate alignment with the center of display area 300.

In this example, layers 410, 412, 414, 450 move according to the following rules, except during wrapping animations: 1. Content layer 414 will move at approximately twice the rate of layer 412, which is approximately half the length of layer 414. 2. Layer 412 will move at approximately twice the rate of layer 410, which is approximately half the length of layer 412. 3. Content layer 414 will move at approximately four times the rate of layer 410, which is approximately ¼ the length of layer 414. 4. Background layer 450 will move slower than layer 410. Although background layer 450 is longer than layer 410, the distance to be moved between neighboring lock points (e.g., lock points “A” and “B”) in layer 410 is greater than the distance between the corresponding lock points in background layer 450.

Movement of layers 410, 412, 414, 450 may differ from the rules described above in some circumstances. In this example, wrapping is permitted. User 302 can navigate left from the beginning of content layer 414 (the position shown in FIG. 4A), and can navigate right from the end of content layer 414 (the position shown in FIG. 4C). During a wrapping animation, some layers may move faster or slower than during other kinds of movements. In this example, the image in background layer 450 and the text in layers 410 and 412 moves faster when user input causes wrapping back to the beginning of content layer 414. In FIG. 4C, display area 300 shows portions of one and two letters, respectively, in layers 410 and 412, at the end of the respective text strings. Display area 300 also shows the rightmost portion of the image in background layer 450. A wrapping animation to return to the state shown in FIG. 4A can include bringing the leftmost portion of the image in background layer 450 and the beginning of the text in layers 410, 412 into view from the right. This results in a more rapid movement in layers 410, 412 and 450 than in other contexts, such as the transition from the state shown FIG. 4A to the state shown in FIG. 4B.

Parallax effects can be used in combination with boundary effects and inertia motion. For example, boundary effects can be used to indicate when a user has reached a boundary of a layer, or a boundary of an element within a layer. As another example, inertia motion can be used to extend motion of UI elements caused by some gestures (e.g., flick gestures). If inertia motion causes movement of a UI element (e.g., a layer) to extend beyond a boundary, a UI system can present a boundary effect.

FIG. 5 is a diagram showing two layers 530, 532. Display area 300 is indicated by a dashed line and has dimensions typical of displays on smartphones or similar mobile computing devices. The content layer 532 includes content elements 540-544. In this example, each content element 540-544 comprises an image representing a music album, and text indicating the title of the respective album. The list header layer 530 includes a text string (“Albums”). According to the example shown in FIG. 5, a user 302 (represented by the hand icon) interacts with content layer 532 by interacting with a touchscreen having the display area 300. The interaction can include, for example, contacting the touchscreen with a fingertip, stylus or other object and moving it (e.g., with a flicking or sweeping motion) across the surface of the touchscreen.

FIG. 5 shows example display states 590-594. In display state 590, user 302 interacts with a touchscreen by making a flick gesture 510, which is indicated by a leftward-pointing arrow. The flick gesture 510 causes an inertia motion in content layer 532, which continues to move after the gesture 510 has ended. Although FIG. 5 shows user 302 interacting with the touchscreen at a particular location in the display area 300, the UI system allows interaction with other parts of the touchscreen to cause movement. Furthermore, although the example shown in FIG. 5 shows user 302 making a leftward flick gesture, user 302 also can make other motions (e.g., rightward motions to scroll towards the beginning of the list). The UI system can interpret different kinds of leftward or rightward user movements, even diagonal movements extending below or above the horizontal plane, as a valid leftward or rightward motion.

In response to the flick gesture 510, the UI system produces leftward movement of the layers 530, 532 relative to the display area 300. For example, from display state 590, the flick gesture 510 causes a leftward movement in the layers and leads to display state 592, in which element 540 is no longer visible, and elements 542 and 544 have moved to the left. The text string (“Albums”) in the list header layer 530 also has moved to the left, but at a slower rate (in terms of pixels) than the content layer 532. The movement of the layers 530, 532 is a function of the data in the layers and the velocity of the flick gesture 510.

From display state 592, the inertia motion causes continued leftward movement of the layers 530, 532 without further input from the user 302, and leads to display state 594 in which element 542 is no longer visible. The inertia motion causes the content layer to extend beyond a boundary (not shown) to the right of the element 544 in the content layer 532, which results in a distortion effect in which an image and text in element 544 is squeezed or compressed in a horizontal dimension. The compression is indicated by the reduced length of the dimension lines above the image and text (“Rock & Roll Part in”) of element 544, respectively. The text string (“Albums”) in the list header layer 530 also has moved to the left, but at a slower rate (in terms of pixels) than the content layer 532. The text in list header layer 530 is uncompressed. The distortion effect gives user 302 an indication that the end of the content layer 532 has been reached.

Although a motion that is calculated to extend beyond a boundary may result in a distortion effect, the boundary need not prevent further movement in the direction of the motion. For example, if wrapping functionality is available, further movement beyond the boundary can cause the content layer 530 to wrap back to the beginning (e.g., back to display state 590). In state 594, element 540 at the beginning of the collection is partially visible, indicating that wrapping is available.

The display can return from display state 594 to display state 592, transitioning from a display state with a distortion effect to an undistorted display state. This can occur, for example, without any additional input by the user. The length of time that it takes to transition between states can vary depending on implementation.

Flick gesture 510 is only an example of a possible user interaction. The same gesture 510 and/or other user interactions (e.g., motions having different sizes, directions, or velocities) can cause different effects, different display states, different transitions between display states, etc. Some display states (e.g., display state 594) may occur only if a gesture results in a post-gesture position that is calculated to go beyond a boundary for the layer.

Display states 590-594 are only examples of possible display states. In practice, a display can exist in any number of states (e.g., in intermediate states between example states 590-594, in states with different visible UI elements, etc.) in addition to, or as alternatives to, the example display states 590-594. For example, it is preferable to show a gradual transition between an undistorted state (e.g., state 592) and a distorted state (e.g., state 494), or from a distorted state to an undistorted state, to provide a more natural feel and avoid the appearance of abrupt changes in the display. Intermediate states, such as states that may occur between state 592 and state 594, can show gradually increasing or decreasing degrees of distortion, as appropriate. As another example, a UI system can provide a boundary effect by compressing the elements 542 and 544 shown in display state 592 without moving the elements 542 and 544 to the left in the display area 300.

Described techniques and tools can be used on display screens in different orientations, such as landscape orientation. Changes in display orientation can occur, for example, where a UI has been configured (e.g., by user preference) to be oriented in landscape fashion, or where a user has physically rotated a device. One or more sensors (e.g., an accelerometer) in the device can be used to detect when a device has been rotated, and adjust the display orientation accordingly.

In the example shown in FIG. 5, the display area 300 is oriented in landscape fashion. Content (e.g., data collection elements 540-544 in content layer 532) and/or other user interface features in the display area 300 can be dynamically adjusted to take into account effects of a reorientation (e.g., a new effective width of the display area 300, interpreting directions of user interactions differently, etc.). For example, distortion effects can be adjusted, such as by compressing data collection elements in a horizontal dimension instead of a vertical dimension, to account for display reorientation.

However, such adjustments are not required. For example, if a display area has equal height and width, reorientation of the display area to a landscape orientation will not change the effective width of the display area.

FIGS. 6A-6E are diagrams showing a content layer 614 that moves in tandem with layer 612 above it. In this example, a user 302 (represented by the hand icon) navigates through content layer 614 by interacting with a touchscreen having the display area 300. The interaction can include, for example, contacting the touchscreen with a fingertip, stylus or other object and moving it (e.g., with a flicking or sweeping motion) across or along the surface of the touchscreen. The content layer 614 includes game icons 640, 642, 644, lists 650, 652, 654, and avatar 630 (which is described in more detail below in Example 8). The other layers 610, 612 include text information (“Games” in layer 610; “Spotlight,” “Xbox Live, “Requests” and “Collection” in layer 612).

The direction of motion that can be caused by user 302 is indicated by a left-pointing arrow and a right-pointing arrow in FIGS. 6A-6E, along with additional up- and down-pointing arrows in FIGS. 6A and 6E. The right-pointing and left-pointing arrows indicate possible movements (left or right horizontal movements) of the layers 610, 612, 614 in response to user movements. In addition to movements of entire layers, a user also can cause movements in elements or parts of layers, depending on the data in the layer and how the layer is arranged. For example, a user can cause movements (e.g., vertical movements) in layer elements (e.g., lists in a content layer) that are orthogonal to movements (e.g., horizontal movements) that can be caused in a layer as a whole. Such can include scrolling vertically in a list embedded in a content layer that moves horizontally. Alternatively, a system that presents layers that move vertically can allow horizontal movements in layer elements.

The up-pointing and down-pointing arrows indicate possible movements of the list 650 in response to user movements. The amount of movement of list 650 can be a function of the size or rate of the motion made by user 302, and the data in list 650. Thus, scrolling of the list 650 can be element-by-element, page-by-page of elements, or something in between that depends on size or rate of the motion. In this example, list 650 includes only one element that is not visible in the display area 300, as shown in FIG. 6A, so a range of small or large downward movements may be enough to scroll to the end of list 650. In the example shown in FIG. 6E, an upward user movement has caused a boundary effect in list 650, in which the text of elements in the list are squeezed or compressed in a vertical dimension. This effect gives user 302 an indication that the end of the list has been reached.

In this example, the amount of movement in layers 610, 612, 614 is a function of the data in the layers and the size or rate of the motion made by the user. Horizontal movement in layers 610, 612, 614 proceeds according to the following rules, except during wrapping animations: 1. The horizontal movement of content layer 614 is locked to layer 612. 2. Layers 612 and 614 will each move at approximately three times the rate of layer 610, which is approximately ⅓ the length of layers 612 and 614.

Movement in the layers 610, 612, 614 may differ from the rules described above in some circumstances. In the example shown in FIGS. 6A-6E, wrapping is permitted. The arrows indicate that a user can navigate left from the beginning of the content layer 614 (the position shown in FIG. 6A and FIG. 6E), and can navigate right from the end of the content layer 614 (the position shown in FIG. 6D). During a wrapping animation, some layers may move faster or slower than during other kinds of movements. For example, the text in layer 610 can move faster when wrapping back to the beginning of content layer 614. In FIG. 6D, display area 300 shows portions of two letters in layer 610, at the end of the “Games” text string. A wrapping animation to return to the state shown in FIG. 6A can include bringing the data in layers 610, 612, 614 (including the text of layer 610) into view from the right, resulting in a more rapid movement in layer 610 than in other contexts, such as a transition from the state shown FIG. 6A to the state shown in FIG. 6B.

In FIGS. 6A-6E, example lock points “A,” “B,” “C” and “D” are indicated for layers 610 and 612. In terms of horizontal motion, content layer 614 is locked to layer 612; the lock points indicated for layer 612 also apply to layer 614. The lock points for each layer indicate the corresponding position of the left edge of the display area 300 on each layer. For example, when a user navigates to a position on content layer 614 such that the left edge of the display area 300 is at lock point “A,” the left edge of display area 300 also is aligned at lock point “A” of the other layers 610, 612, as shown in FIGS. 6A and 6E. In FIG. 6B, the left edge of the display area 300 is at lock point “B” in each of the layers 610, 612, 614. In FIG. 6C, the left edge of the display area 300 is at lock point “C” in each of the layers 610, 612, 614. In FIG. 6D, the left edge of the display area 300 is at lock point “D” in each of the layers 610, 612, 614.

The lock points shown in FIGS. 6A-6E are not generally representative of a complete set of lock points, and are limited to lock points “A,” “B,” “C” and “D” only for brevity. For example, right-edge lock points can be added to obtain alignment with the right edge of display area 300, or center lock points can be added to obtain alignment with the center of display area 300. Alternatively, fewer lock points can be used, more lock points can be used, or lock points can be omitted.

User 302 can move left or right in content layer 614 after making an up or down movement in list 650. The current position of list 650 can be saved, or the system can revert to a default position (e.g., the top-of-list position indicated in FIG. 6A) when navigating left or right in content layer 614 from list 650. Although the arrows in FIGS. 6A-6E (and other figures) that indicate possible movements are shown for purposes of explanation, the display area 300 can itself display graphical indicators (such as arrows or chevrons) of possible movements for the layers and/or list.

The system can interpret user movements to the left or right, even diagonal movements extending above or below the horizontal plane, as a valid leftward or rightward motion. Similarly, the system can interpret upward or downward movements, even diagonal movement extending to the left or right of the vertical plane, as a valid upward or downward motion. Although FIGS. 6A-6E show the user 302 interacting with a portion of the display area 300 that corresponds to the content layer 614, the system also allows interaction with other parts of the touchscreen (e.g., those that correspond to display area occupied by other layers) to cause movement in the layers 610, 612, 614, list 650, or other UI elements.

In FIGS. 6A-6E, avatar 630 can provide a visual cue to indicate a relationship between or draw attention to parts of the content layer 614.

In FIG. 6B, avatar 630 is positioned between list 652 and list 654. In FIG. 6C, avatar 630 floats behind the text of list 654, but remains completely within display area 300. In FIG. 6D, avatar 630 is only partially within display area 300; the part that is within the display area floats behind game icons 640, 642, 644. The positioning of avatar 630 at the left edge of display area 300 can indicate to the user 302 that information associated with avatar 630 is available if the user 302 navigates in the direction of avatar 630. Avatar 630 can move at varying speeds. For example, avatar 630 moves faster in the transition between FIGS. 6B and 6C than it does in the transition between FIGS. 6C and 6D.

Alternatively, avatar 630 can move in different ways, or exhibit other functionality. For example, a UI system can present a distortion effect in avatar 630 to indicate a user\'s location in a data collection with which the avatar is associated. Avatar 630 also can be locked to particular position (e.g., a lock point) in content layer 614 or in some other layer, such that avatar 630 moves at the same horizontal rate as the layer to which it is locked. As another alternative, avatar 630 can be associated with a list that can be scrolled up or down, such as list 650, and move up or down as the associated list is scrolled up or down.

In this section, a detailed implementation is described comprising aspects of motion feedback including boundary effects and parallax effects, with reference to the following detailed example.

In this detailed example, a set of equations, coefficients and rules are described that can allow a UI system (e.g., a UI system provided as part of a mobile device operating system) to interpret user input such as touch gestures (including multi-touch gestures with more than one touch contact point) and generate motion feedback in response to user input. Features described in this detailed example include inertia movement, panning and zooming operations, boundary effects, parallax effects, and combinations thereof. Described features can help to provide natural-looking, smooth motion in response to user input (e.g., touch gestures).

In this detailed example, processing tasks can be handled by different software modules. For example, a module called “ITouchSession” provides coefficients, gesture positions, and gesture velocity information, and a dynamic motion module in a mobile device operating system uses information provided by ITouchSession to generate motion feedback (e.g., parallax effects, boundary effects, etc.). Preferably, gesture information provided to the dynamic motion module is accurate (e.g., with minimal jitter in position information), detailed (e.g., with time stamps on touch input), and low-latency (e.g., under 30 ms). The information (e.g., motion feedback information) generated by the dynamic motion module can be used by other modules, as well. For example, web browsers or other applications that run on the mobile device operating system can use information generated by the dynamic motion module.

In this detailed example, the dynamic motion resulting from user interaction is defined by a set of motion rules. The motion rules define how different visual elements react on screen in response to different gestures. For example, some rules apply to finger-tracking gestures such as panning or dragging gestures, some rules apply to flick or toss gestures, and some rules apply to pinch or stretch gestures. Additionally, some rules, such as inertia rules, may apply to more than one type of gesture. The specific motion rules that apply to different UI elements (or “controls”) are determined by factors such as the control type and control content; not all motion rules will apply to all UI elements. For example, rules for pinch and stretch gestures do not apply to UI elements where pinch and stretch gestures are not recognized. The motion resulting from the application of motion rules to the input stream generated by the user can be further refined by an optional set of modifiers, which are collectively called “optional motion features.”

In this detailed example, touch input interactions that result in dynamic motion comply with the motion rules. Additionally, different UI elements (or “controls”) can apply zero or more optional motion features, which can be determined by factors such as the desired motion, control type and control content. For example, a list control may opt to enhance motion feedback with boundary effects, while a panorama control may apply a parallax feature to some of its layers.

In addition, when a user interacts with a UI element, it can be helpful to provide some immediate (or substantially immediate) visual feedback to the user (e.g., a change in movement in the UI element, or some other effect such as a tilt or highlight). Immediate or substantially immediate feedback helps the user to know that the user interface is responsive to the user\'s actions.

In this detailed example, the following motion rules apply in UI elements where the rules (e.g., rules relating to finger-tracking gestures, inertia, boundaries, pinch/stretch gestures) are relevant to the types of motion that are permitted in the respective UI elements. The motion rules can be modified for some UI elements, such as where optional motion features apply to a UI element.

For finger tracking movements (e.g., movements caused by dragging or panning gestures), the content at the initial gesture point moves in direct correspondence to the gesture. For example, content under the user\'s finger at an initial touch point moves with the user\'s finger during the gesture. The current position of a visual element is given by the following equation:

p=p0+(q−q0)  (Eq. 1)

where p is the (x, y) vector that represents the current position of the visual element, p0 is the (x0, y0) vector that represents the visual element position at the beginning of the gesture, q is the (x, y) vector that represents the current touch contact position, and q0 is the (x0, y0) vector that represents the touch contact position at the beginning of the gesture.

In a UI element that allows inertia movement (e.g., a scrolling list), when the user finishes a gesture (e.g., by lifting a finger or other object to end the interaction with the touchscreen), a velocity and direction for that movement is identified, and the motion initially continues in the same direction and speed as the gesture, as if the visual element was a real, physical object with a non-zero mass. If the motion is not stopped for some other, permissible reason (e.g., where the UI element reaches a boundary or is stopped by another user gesture), the motion gradually decelerates over time, eventually coming to a stop. The deceleration proceeds according to a combination of equations and coefficients, which can vary depending on implementation. Default system-wide coefficient values can be made available. Default system-wide coefficients can help to maintain a consistent feeling across all controls. Alternatively, different equations or coefficients can be used, such as where a particular control has its own friction coefficient for modeling different kinds of motion.

The velocity (e.g., in pixels/second) at the end of the gesture is computed by the following equation:

v=(q−q0)/(t−t0),  (Eq. 2)

where v is the (vx, vy) velocity vector that represents the inertia velocity at the end of the gesture, q is the (x, y) vector that represents the touch contact position at the end of the gesture, q0 is the (x0, y0) vector that represents the touch contact position at the time t0, t is the timestamp of the last touch input of the gesture, and t0 is the timestamp of the least recent touch input that happened within some fixed period of time from the last touch input. Alternatively, the velocity can be calculated in another way. For example, a weighted sum of velocities at different time instances can be calculated, with greater weighting for velocities at the end of the gesture. In this detailed example, calculating the velocity is the responsibility of ITouchSession. However, velocity calculations can be handled by other modules.

The duration of the inertia motion can be computed according to the following equation:

t max = ln   γ  v 0  ln   μ ( Eq .  3 )

where tmax is the duration of the inertia motion, |v0| is the magnitude of the initial velocity vector (|v0|>γ), μ is a friction coefficient (e.g., MotionParameter_Friction, 0<μ<1), and y is a parking speed coefficient (e.g., MotionParameter_ParkingSpeed, 0<γ<|v0|) that is used to indicate a threshold velocity, below which inertia motion will stop. In one implementation, the friction coefficient is 0.4, and the parking speed coefficient is 60.0. The duration is computed at the start of the inertia motion, and need not be computed again.

The following equation will compute the current velocity vector v at any given time t:

v=V0·μt  (Eq. 4).

The new position p′ for the visual element can be computed based on its last known position p and the time elapsed since the last position update (Δt), as shown in the following equation:

p′=p+v·Δt  (Eq. 5).