From 30b6637fe84813ad7119e92dd49df7e12692c694 Mon Sep 17 00:00:00 2001 From: Henrik Rydberg Date: Thu, 28 Apr 2011 16:01:13 +0200 Subject: Document the gesture frame logic Add some notes on the math of the gesture frame computations. Signed-off-by: Henrik Rydberg --- docs/gestures.txt | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 docs/gestures.txt (limited to 'docs/gestures.txt') diff --git a/docs/gestures.txt b/docs/gestures.txt new file mode 100644 index 0000000..5a1eb45 --- /dev/null +++ b/docs/gestures.txt @@ -0,0 +1,58 @@ +Introduction +------------ + +This document describes how the gestures are extracted from multi-finger +actions. The process is divided into gestural transformations, gesture +recognition, and gesture instantiation. + +Gestural Transformations +------------------------ + +All two-finger transformations are extracted. These are all exact, in the +sense that continuously transforming the original finger positions, frame +per frame, will exactly follow the actual finger positions. + +In addition to two-finger transformations, a global gesture is also +extracted. The rotation and scaling is taken from the contact pair with the +longest distance between contacts. This approximates the behavior of a +region, such that complex transformations could, in principle, be happening +inside the region, but at a distance, the transformation will look like it +was performed with two fingers. + +At each time step, a gestural transformation is goverened by rotation, +scaling and translation. The point around which rotation and scaling is +performed is called the pivot. To form as natural gestures as possible, +this point always lies within the area formed by the contacts themselves. +It is placed at the point which, after rotation and scaling, leaves the +transformed contacts as close to the actual positions as possible. The +translation, or drag, is always a fraction of the movement of the center +point. The fraction is called moveness, and is related to the distance +between the pivot and the center point. When the pivot is at the center +point, the moveness is one and the drag is the same as the center movement. +When the pivot is at one of the contacts, as in rotation around a finger, +the moveness is zero, and consequently the drag is zero. The relations +between the pivot, the moveness and the drag are detailed in the document +pivot.txt. + +Gesture Recognition +------------------- + +Each gestural transformation can give rise to one or several gesture +primitives. Based on the rotation, scaling, moveness and drag values, the +gesture primitives drag, pinch and rotate are recognized. The onset of a +gesture primitive is governed by a threshold and a timeout. Basically, if +performed distinctly enough, the gesture will be triggered, and will remain +active until a finger is lifted or added. + +Gesture Instantiation +--------------------- + +Given a set of detected gesture primitives, only some will trigger actual +gesture events. First, the set of available primitives are matched against +available listeners. Primitives not listened for are dropped. The remaining +set is arranged according to priority. A pointer gesture has lower priority +than a two-finger gesture, which has lower priority than a tap, for +instance. As long as the gesture primitives of higher priority are expected +but not activated, all gestures are held back. Once one of the highest +priority primitives are activated, all primitives of lower priority are +cancelled, and events are emitted. -- cgit v1.2.3