PYRA real-time vision library:
I'm currently working on opening up parts of all the code I've developed
during the years. The first package to become available is pyraasm-0.9.tar.gz,
a library that includes x86 and amd64 assembly optimized functions for
filtering,
colour to grey-level conversion, Harris' corner detection, as well as
disparities using local area correlation. The research mentioned below
relies on this package and other packages that are in preparation.
With the package
a disparity map like disp.jpg can, for example,
be computed from trees11.jpg
and trees14.jpg in 7.9 ms on a 2.6 GHz Athlon
with floating points. Harris' corners can similarly be extracted in 1.0 ms
or an image be low-pass filtered in 0.3 ms. Note: the package requires the
assembler NASM for compilation.
A CUDA implementation of SIFT:
If you are interested in a CUDA implementation of Lowe's Scale Invariant
Feature Transform (SIFT) you can find one here;
cudaSIFT.tar.gz. On a GeForce 8800 GTS graphics board SIFT features are
extracted
from a VGA image at a computational cost of about 15-20 ms, depending on
the number of features. That is about 10x faster than my own SSE assembly
implementation of the same method. How it compares to other implementations,
I don't know. CUDA is an C-like API from
NVIDIA that facilitates
implementation of General Purpose algorithms for GPUs. Hence the term and
community GPGPU.
Some recent experiments:
Here are some movies from a recent experiment performed using a
binocular head with two peripheral and two foveal cameras. The first
experiment shows the interface of a visual front-end delivering cues,
such as binocular disparities and epipolar geometry. In the
second experiment object hypotheses are derived from the disparities,
a saccade
is triggered towards an object of interest, the object is recognised
and finally pose estimation initiated. The last movie illustrates
the attention-recognition loop. All movies are played at the correct
real-time speed.
The current left wide field and foveal views of the binocular head can
be found here.
Publications:
K. Kuebner, M. Björkman, B. Rasolzadeh, M. Schmidt and D. Kragic,
``Integration of Visual and Shape Attributes for Object Action Complexes'',
in International Conference on Computer Vision Systems, ICVS'08, pp. 13-22, Santorini, Greece, May 2008.
M. Björkman and J-O. Eklundh,
``Vision in the Real World: Finding, Attending and Recognizing Objects'',
International Journal of Imaging Systems and Technology, 16(5), pp. 189-209, 2006.
(pdf)
P. Jensfelt, D. Kragic, J. Folkesson and M. Björkman,
``A Framework for Vision Based Bearing Only 3D SLAM'',
in IEEE International Conference on Robotics and Automation, ICRA'06, Orlando, USA, pp. 1944-1950, May 2006.
(pdf)
B. Rasolzadeh, M. Björkman and J-O. Eklundh,
``An attentional system combining top-down and bottom-up influences'',
International Cognitive Vision Workshop (ICVW), ECCV'06, 2006.
D. Kragic and M. Björkman,
``Strategies for object manipulation using foveal and peripheral vision'',
in International Conference on Computer Vision Systems, ICVS'06,
New York, USA, 2006.
(pdf)
M. Björkman and J-O. Eklundh,
``Foveated Figure-Ground Segmentation and Its Role in Recognition'',
British Machine Vision Conference, Sep 2005.
(pdf)
D. Kragic, M. Björkman, H.I. Christensen and J-O. Eklundh,
``Vision for Robotic Object Manipulation in Domestic Settings'',
Robotics and Autonomous Systems, Volume 52, Issue 1, pp. 85-100, July 2005.
(pdf)
J-O. Eklundh and M. Björkman,
``Recognition of Objects in the Real World from a Systems Perspective'',
Kuenstliche Intelligenz, 19(2), pp. 12-17, 2005.
(pdf)
M. Björkman and J-O. Eklundh,
``Attending, Foveating and Recognizing Objects in Real World Scenes'',
British Machine Vision Conference, Sep 2004.
(pdf)
D. Kragic, M. Björkman H.I. Christensen and J-O. Eklundh,
``Issues and Strategies for Robotic Object Manipulation in Domestic Settings'',
Workshop on Advances in Robot Vision: From Domestic Environments to Medical
Applications, IROS'04, 2004.
(pdf)
M. Björkman and D. Kragic,
``Combination of Foveal and Peripheral Vision for Object Recognition
and Pose Estimation'',
International Conference on Robotics and Automation, Apr 2004.
(pdf)
M. Björkman,
``Real-Time Motion and Stereo Cues for Active Visual Observers'',
doctoral dissertation, ISRN KTH/NA/P--02/13--SE, NADA,
Royal Institute of Technology, Jun 2002.
(pdf)
M. Björkman and J-O. Eklundh,
``Real-Time Epipolar Geometry Estimation of Binocular Stereo Heads'',
IEEE Trans. Pattern Analysis and Machine Intelligence 24(3), pp. 425-432,
Mar 2002. (postscript)
M. Björkman and J-O. Eklundh,
``Visual Cues for a Fixating Active Agent'',
in Proc. Robot Vision 2001, (Auckland, New Zealand), Feb 2001.
(postscript)
M. Björkman and J-O. Eklundh,
``A Real-Time System for Epipolar Geometry and Ego-Motion
Estimation'', in Proc. IEEE Computer Vision
and Pattern Recognition, vol. 2, (Hilton Head, SC),
pp. 506--513, Jun 2000.
(postscript)
M. Björkman and J-O. Eklundh,
``Dynamic Fixation in an Active Visual Agent'',
Tech. Report ISRN KTH/NA/P-00/09-SE, NADA,
Royal Institute of Technology, Mar 2000.
(postscript)
M. Björkman and J-O. Eklundh,
``Real-Time Epipolar Geometry Estimation and Disparity'',
in Proc. International Conference on Computer Vision, (Kerkyra,
Greece), pp. 234--141, Sep 1999.
(postscript)
F. Dahlgren, P. Stenström and M. Björkman,
``Reducing the Read-Miss Penalty for Flat COMA Protocols'',
The Computer Journal 40(4), pp. 208-219, 1997.
(postscript)
M. Björkman, F. Dahlgren and P. Stenström,
``Using Hints to Reduce the Read Miss Penalty for Flat COMA Protocols'',
in Proc. of the 28th Hawaii International Conference on System Sciences,
pp. 242-251, 1995.
(postscript)
/ Celebrandil of Phenomena (ex-Fairlight, ex-North Star)