We have presented a model based tracking system that estimates the full pose of an object in a realistic environment. The system also has the ability of automatic initial pose estimation. For this purpose a PCA based method is used where the training images are obtained with no particular lighting or background conditions. The system has successfully been used in a number of robotic applications such as pickup and delivery of objects in a living room scenario. Through separation of the various step of visual servoing: recognition, pose estimation, pose refinement and model based tracking it is possible to provide efficient techniques that allow real-time implementation for handling of ``complex'' objects in realistic settings. The methods have been extensively tested for mobile manipulation.
Future research will consider the tessellation of the view space for
training. For relatively simply objects it is well known that the
theoretical complexity of the aspect space is at worst 
while
in practical cases it is much less. For complex (non-polyhedral)
objects the complexity of the aspect space is considerably higher and
there is thus a need to investigate the generalization properties of
the presented strategy for such types of objects. In addition for such
objects there is also a need for introduction of more complex
strategies for visual servoing as the physical layout of the objects
much be taken into account as part of the approach and grasp
evaluation.