Structure and Motion Estimation using Sparse Point and Line Correspondences in Multiple Affine Views

Lars Bretzner and Tony Lindeberg

Technical report ISRN KTH/NA/P--99/13--SE

Earlier version presented in Proc. 5th European Conference on Computer Vision (H. Burkhardt and B. Neumann, eds.), vol. 1406 of Lecture Notes in Computer Science, (Freiburg, Germany), pp. 141--157, Springer Verlag, Berlin, June 1998.

Abstract

This paper addresses the problem of computing three-dimen\-sional structure and motion from an unknown rigid configuration of points and lines viewed by an affine projection model. An algebraic structure, analogous to the trilinear tensor for three perspective cameras, is defined for configurations of three centered affine cameras. This centered affine trifocal tensor contains 12 non-zero coefficients and involves linear relations between point correspondences and trilinear relations between line correspondences. It is shown how the affine trifocal tensor relates to the perspective trilinear tensor, and how three-dimensional motion can be computed from this tensor in a straightforward manner. A factorization approach is developed to handle point features and line features simultaneously in image sequences, and degenerate feature configurations are analysed. This theory is applied to a specific problem in human-computer interaction of capturing three-dimensional rotations from gestures of a human hand. This application to quantitative gesture analyses illustrates the usefulness of the affine trifocal tensor in a situation where sufficient information is not available to compute the perspective trilinear tensor, while the geometry requires point correspondences as well as line correspondences over at least three views.

Keywords: affine trifocal tensor, trilinear tensor, affine, perspective, projection, structure, motion, factorization, tracking, point, blob, line, ridge, scale selection, human-computer interaction, computer vision.

PDF: (320 kb)

Background and related material: (Earlier version of this work at ECCV98) (The qualitiative feature hierarchy used for supporting the tracker as well as a more extensive description of the application to human computer interaction) (The feature tracker with automatic scale selection that this work builds upon)