3D Hand Motion Estimation

We have studied how three-dimensional motion information can be transferred from the users hand via a monocular camera, without need for external equipment. One long term goal is to be able to replace devices such as computer gloves and trackballs in HMI tasks like CAD and navigation in 3D virtual worlds.

After transforming the features of the hand into points and lines, we use the point and line correspondences over multiple frames to estimate the rigid 3D motion of the hand. The motion parameters can then be used to control other objects, real or virtual (Patent pending). (You can see a movie illustrating these ideas, by clicking on the image below.)

The 3-D hand mouse -- A demonstration of how it is possible to measure the three-dimensional motion (translations and rotations) of a human hand, and to use such motion estimates to control the three-dimensional motion of other computerized devices. (In the video clip below, the cube moves according to the estimated motion of the hand.)

References

• Geometric framework underlying the 3-D hand mouse (Original publication on the 3-D hand mouse)
• Qualitative hand model used for tracking features on the hand
• The feature tracker with automatic scale selection that this work builds upon

Responsible for this page: Tony Lindeberg Lars Bretzner

---