Interest point detection and scale selection in space-time
Ivan Laptev and Tony Lindeberg
in Proc. Scale Space Methods in Computer Vision (2003),
Isle of Skye, UK, Volume 2695 of Springer Verlag Lecture Notes
in Computer Science, pp.372-387.
Abstract
Several types of interest point detectors have been proposed
for spatial images.
This paper investigates how this notion can be
generalised to the detection of interesting events in space-time data.
Moreover, we develop a mechanism for spatio-temporal scale selection
and detect events at scales corresponding to their extent
in both space and time.
To detect spatio-temporal events, we build on the idea of the Harris
and Förstner interest point operators and detect regions in space-time
where the image structures have significant local variations in both
space and time.
In this way, events that correspond to curved space-time structures
are emphasised, while structures with locally constant motion are
disregarded.
To construct this operator, we start from a multi-scale windowed
second moment matrix in space-time, and combine the determinant
and the trace in a similar way as for the spatial Harris operator.
All space-time maxima of this operator are then adapted to
characteristic scales by maximising a scale-normalised space-time
Laplacian operator over both spatial scales and temporal scales.
The motivation for performing temporal scale selection as a complement
to previous approaches of spatial scale selection is to be able to
robustly capture spatio-temporal events of different temporal extent.
It is shown that the resulting approach is truly scale invariant with
respect to both spatial scales and temporal scales.
The proposed concept is tested on synthetic and real image sequences.
It is shown that the operator responds to distinct and stable points
in space-time that often correspond to interesting events.
The potential applications of the method are discussed.
PDF:
(1.1Mb)
PowerPoint presentation:
(2.7Mb)
Demonstractions:
handwavedemo.avi (4.6Mb)
footballdemo.avi (2.8Mb)
walk07a_classharris.avi (2.6Mb)
Related publications:
(Velocity-adaptation of spatio-temporal receptive fields for
direct recognition of activities: An experimental study)
(Time-recursive velocity-adapted spatio-temporal scale-space filters)
(Linear spatio-temporal scale-space)
(Separable scale-space with causal time direction)
(Automatic selection of temporal scales in time-causal scale-space)
(Monograph on scale-space theory)
Ivan Laptev