3D Hand Motion EstimationWe have studied how three-dimensional motion information can be transferred from the users hand via a monocular camera, without need for external equipment. One long term goal is to be able to replace devices such as computer gloves and trackballs in HMI tasks like CAD and navigation in 3D virtual worlds.
After transforming the features of the hand into points and lines, we use the point and line correspondences over multiple frames to estimate the rigid 3D motion of the hand. The motion parameters can then be used to control other objects, real or virtual (Patent pending). (You can see a movie illustrating these ideas, by clicking on the image below.)