Feature Tracking with Automatic Selection of Spatial Scales

Lars Bretzner and Tony Lindeberg

Technical report ISRN KTH NA/P--96/21--SE. Department of Numerical Analysis and Computing Science, Royal Institute of Technology, S-100 44 Stockholm, Sweden, May 1996.

Shortened version in Computer Vision and Image Understanding, vol. 71, pp. 385--392, Sept. 1998.

Shortened version in Linde, Sparr (Eds.): Proc. Swedish Symposium on Image Analysis, SSAB'96, Lund, Sweden, pages 24--28, march 1996.

Abstract

When observing a dynamic world, the size of image structures may vary over nada. This article emphasizes the need for including explicit mechanisms for automatic scale selection in feature tracking algorithms in order to: (i) adapt the local scale of processing to the local image structure, and (ii) adapt to the size variations that may occur over time.

The problems of corner detection and blob detection are treated in detail, and a combined framework for feature tracking is presented in which the image features at every time moment are detected at locally determined and automatically selected nadaes. A useful property of the scale selection method is that the scale levels selected in the feature detection step reflect the spatial extent of the image structures. Thereby, the integrated tracking algorithm has the ability to adapt to spatial as well as temporal size variations, and can in this way overcome some of the inherent limitations of exposing fixed-scale tracking methods to image sequences in which the size variations are large.

In the composed tracking procedure, the scale information is used for two additional major purposes: (i) for defining local regions of interest for searching for matching candidates as well as setting the window size for correlation when evaluating matching candidates, and (ii) stability over time of the scale and significance descriptors produced by the scale selection procedure are used for formulating a multi-cue similarity measure for matching.

Experiments on real-world sequences are presented showing the performance of the algorithm when applied to (individual) tracking of corners and blobs. Specifically, comparisons with fixed-scale tracking methods are included as well as illustrations of the increase in performance obtained by using multiple cues in the feature matching step.

Keywords: feature, tracking, motion, blob, corner, scale, scale-space, scale selection, similarity, multi-scale representation, computer vision

PDF: (Long version 588 kb) (Short version 319 kb)

Background and related material: (General reference of feature detection with automatic scale selection) (Specific application of scale selection principle to junction detection) (Monograph on scale-space theory) (Other publications on scale-space theory) (Encyclopedia entry on scale-space theory)

Further work: (Integration with qualitative multi-scale feature hierarchy and application to human-computer interaction)