Use Your Hand as a 3D Mouse, or, Relative Orientation from Extended Sequences of Sparse Point and Line Correspondences Using the Affine Trifocal TensorLars Bretzner and Tony LindebergIn: Proc. 5th European Conference on Computer Vision (H. Burkhardt and B. Neumann, eds.), vol. 1406 of Lecture Notes in Computer Science, (Freiburg, Germany), pp. 141157, Springer Verlag, Berlin, June 1998.AbstractThis paper addresses the problem of computing threedimensional structure and motion from an unknown rigid configuration of point and lines viewed by an affine projection model. An algebraic structure, analogous to the trilinear tensor for three perspective cameras, is defined for configurations of three centered affine cameras. This centered affine trifocal tensor contains 12 coefficients and involves linear relations between point correspondences and trilinear relations between line correspondences It is shown how the affine trifocal tensor relates to the perspective trilinear tensor, and how threedimensional motion can be computed from this tensor in a straightforward manner. A factorization approach is also developed to handle point features and line features simultaneously in image sequences.This theory is applied to a specific problem of humancomputer interaction of capturing threedimensional rotations from gestures of a human hand. A qualitative model is presented, in which three fingers are represented by their position and orientation, and it is shown how three point correspondences (blobs at the finger tips) and three line correspondences (ridge features at the fingers) allow the affine trifocal tensor to be determined, from which the rotation is computed. Besides the obvious application, this test problem illustrates the usefulness of the affine trifocal tensor in a situation where sufficient information is not available to compute the perspective trilinear tensor, while the geometry requires point correspondences as well as line correspondences over at least three views. Keywords: affine trifocal tensor, trilinear tensor, affine, perspective, projection, structure, motion, factorization, tracking, point, blob, line, ridge, scale selection, humancomputer interaction, computer vision. PDF: (271 kb) Video illustrating how hand motions computed in this way can be used for controlling the motion of other objects:
Applications of visual control using the 3D hand mouse are described in the following patent application (now released): Background and related material: (Later version of this work with a more extensive experimental evaluation) (The feature tracker with automatic scale selection that this work builds upon) Related project: Computer vision based humancomputer interaction
Responsible for this page:
Tony Lindeberg
