# Disparity Selection for Vergence in Binocular Pursuit

## Atsuto Maki, Tomas Uhlin and Jan-Olof Eklundh

To appear in; Proc. 4th IAPR International Workshop on Machine Vision Applications, Kawasaki, Japan, Dec. 1994.

## Extended Abstract

### Overview

Pursuit of a target using a stereo camera system requires mechanisms for version, to track the target on the horopter, and vergence, to fixate it in depth. Since those two mechanisms are orthogonal to each other, pursuit can be performed by controlling them independently. The research described here is concerned with the mechanism for vergence. To keep the cameras nadaed on aCVAP/ target, the vergence system must measure the current vergence error. We utilize binocular disparity as the most useful visual cue to vergence, because of the straightforward mapping into vergence error. As the disparity detector, a new method which is promising in this respect based on the output phase of bandpass filters \cite{sang88} \cite{knut89} is employed. Since this produces disparity estimates at all points in the image, a crucial problem in applying this framework to vergence in binocular pursuit is to estimate the disparity corresponding to the target. I.e., a selection process has to take place among the multiple disparities observed in the image. The objective of the work presented here is to solve the problem of disparity selection for vergence control of our vision system, the KTH-head \cite{pahl92}, on which we implement the presented techniques to perform real-time smooth pursuit. Typical for this problem is that not only do we want to estimate a disparity at each instant of time, but this disparity should also be consistent with the target and stable over time. Two different approaches to solve this problem are proposed, one is based on histogramming, and the other by weighting disparities based on the assumption that the target is close to the predicted position of the target in the image. Their individual performance is investigated and compared with each other both experimentally and theoretically.

### Disparity Estimation

As the method to acquire the disparity map of the scene from a pair of stereo images, we employ the phase-based approach considering the advantage in terms of the computational cost and the stability against varying brightness. The basic concept of the phase-based approach is to convolve the left and right stereo images with a complex filter, and then estimate the local disparity by computing the complex phase difference of the filter output. For each coordinate $(x,y)$ in the image, the algorithm provides the disparity estimation $D(x,y)$ and a confidence value $C(x,y)$, representing the the feasibility of the disparity estimation (Maki et al, 1993).

### Histogram-based Selection

In a scene where several objects exist at different depths, multiple disparities are observed. In order to find the disparities that are present and select the one that corresponds to the target, we compute a histogram $H(D_d)$ with respect to the discrete disparities $D_d$: \begin{eqnarray} H(D_d) = \sum_{x,y} C(x,y) ~~\mbox{for}~~ \{(x,y) ~|~ D_d<=D(x,y)Center-oriented Selection

### Weighting Selection

In the situation when the version component of the pursuit system keeps track of the target in the image, the disparity information relevant to the vergence error will be observed around the estimated position of the target. Using the confidence values as a weighting factor for the individual disparity estimates and combining this with a Gaussian envelope, $G(x,y;\sigma)$ in order to suppress the peripheral disparities, center-oriented disparity is estimated using the following: \begin{eqnarray} D_c=\frac{\displaystyle{\sum_{x,y} C(x,y) G(x-x_0,y-y_0;\sigma) D(x,y)}}{\displaystyle{\sum_{x,y} C(x,y) G(x-x_0,y-y_0;\sigma)}}. \end{eqnarray} The Gaussian envelope is placed at the predicted position of the target $(x_0,y_0)$. The standard deviation $\sigma$ can be adjusted according to the size of the target and the priority given to the vicinity of the predicted target position.

### Summary

In this article, we have considered the problem of disparity selection, a vital component of successful camera vergence in binocular pursuit, in which also the incorporation of time in disparity estimation plays a central role in the success. The figure shows the scheme of the algorithm. Based on the disparity and confidence maps produced by the phase-based method, a histogram-based approach and a weighting approach, are introduced to solve the problem of disparity selection. Experimental results including the comparison of the approaches will be presented in the paper along with a description of the algorithms. \caption{The scheme of the algorithm. The paper describes the part marked by the dashed line.} \label{fig:frame} \bibliography{/home/maki/bib/mva94}

Full paper: PostScript 164k

Atsuto Maki <maki@bion.kth.se>
Tomas Uhlin <tomas@bion.kth.se>
Jan-Olof Eklundh <joe@bion.kth.se>