Temporal scale selection in timecausal scale spaceTony LindebergJournal of Mathematical Imaging and Vision, 58(1): 57101, 2017. Digitally published with DOI:10.1007/s1085101606913 in January 2017. AbstractWhen designing and developing scale selection mechanisms for generating hypotheses about characteristic scales in signals, it is essential that the selected scale levels reflect the extent of the underlying structures in the signal.This paper presents a theory and indepth theoretical analysis about the scale selection properties of methods for automatically selecting local temporal scales in timedependent signals based on local extrema over temporal scales of scalenormalized temporal derivative responses. Specifically, this paper develops a novel theoretical framework for performing such temporal scale selection over a timecausal and timerecursive temporal domain as is necessary when processing continuous video or audio streams in real time or when modelling biological perception. For a recently developed timecausal and timerecursive scalespace concept defined by convolution with a scaleinvariant limit kernel, we show that it is possible to transfer a large number of the desirable scale selection properties that hold for the Gaussian scalespace concept over a noncausal temporal domain to this temporal scalespace concept over a truly timecausal domain. Specifically, we show that for this temporal scalespace concept, it is possible to achieve true temporal scale invariance although the temporal scale levels have to be discrete, which is a novel theoretical construction. The analysis starts from a detailed comparison of different temporal scalespace concepts and their relative advantages and disadvantages, leading the focus to a class of recently extended timecausal and timerecursive temporal scalespace concepts based on firstorder integrators or equivalently truncated exponential kernels coupled in cascade. Specifically, by the discrete nature of the temporal scale levels in this class of timecausal scalespace concepts, we study two special cases of distributing the intermediate temporal scale levels, by using either a uniform distribution in terms of the variance of the composed temporal scalespace kernel or a logarithmic distribution. In the case of a uniform distribution of the temporal scale levels, we show that scale selection based on local extrema of scalenormalized derivatives over temporal scales makes it possible to estimate the temporal duration of sparse local features defined in terms of temporal extrema of first or secondorder temporal derivative responses. For dense features modelled as a sine wave, the lack of temporal scale invariance does, however, constitute a major limitation for handling dense temporal structures of different temporal duration in a uniform manner. In the case of a logarithmic distribution of the temporal scale levels, specifically taken to the limit of a timecausal limit kernel with an infinitely dense distribution of the temporal scale levels towards zero temporal scale, we show that it is possible to achieve true temporal scale invariance to handle dense features modelled as a sine wave in a uniform manner over different temporal durations of the temporal structures as well to achieve more general temporal scale invariance for any signal over any temporal scaling transformation with a temporal scaling factor that is an integer power of the distribution parameter of the timecausal limit kernel. It is shown how these temporal scale selection properties developed for a pure temporal domain carry over to feature detectors defined over timecausal spatiotemporal and spectrotemporal domains. Background and related material: (Theory for timecausal and timerecursive receptive fields) (Underlying computational theory for visual receptive fields) (General framework for invariant visual receptive fields under natural image transformations) (Underlying mathematical necessity results regarding scale covariant, affine covariant and Galilean covariant receptive fields)
Responsible for this page:
Tony Lindeberg
