Active binocular vision: phase-based registration and optimal foveation

Active binocular vision systems are powerful tools in machine vision. With a virtually unlimited field-of-view they have access to huge amounts of information, yet are able to confine their resources to specific regions of interest. Since they can dynamically interact with the environment, they are able to successfully address problems that are ill-posed to passive systems. A primary goal of an active binocular vision systems is to ascertain depth information. Since they employ two cameras and are able to sample a scene from two distinct vantage points, they are well suited for such a task. The depth recovery process is composed of two interrelated components: image registration and sampling. Image registration is the process of determining corresponding points between the stereo images. Once points in the images have been matched, 3D information can be recovered via triangulation. Image sampling determines how the image is discretized and represented. Image registration and sampling are highly interdependent. The choice of sampling scheme can profoundly impact the accuracy and complexity of the registrations process. In many situations, particular registration algorithms are simply incompatible with some sampling schemes. In this dissertation we meticulously address both registration and sampling in the context of stereopis for active binocular vision systems. Throughout the development of this work, contributions in each area are addressed with an eye toward their eventual integration into a cohesive registration procedure appropriate for active binocular vision systems. The actual synthesis is a daunting task that is beyond the scope of this single dissertation. The focus of this work is to assiduously analyze both registration and sampling, establishing a solid foundation for their future aggregation. One of the most successful approaches to image registration is phase-differencing. Phase-differencing algorithms provide a fast, powerful means for depth recovery. Unfortunately, phase-differencing techniques suffer from two significant impediments: phase nonlinearities and neglect of multispectral information. This dissertation uses the amenable properties of white noise images to analytically quantify the behavior of phase in these regions of phase nonlinearity. The improved understanding gained from this analysis enables us to create a new, more effective method for identifying these regions based on the second derivative of phase. We also suggest a novel approach that combines our method of nonlinear phase detection with strategies of both phase-differencing and local correlation. This hybrid approach retains the advantageous properties of phase-differencing while incorporating the multispectral aspects of local correlation. This task of registration is greatly simplified if the camera geometry is known and the search for corresponding points can be restricted to epipolar lines. Unfortunately, computation of epipolar lines for an active system requires calibration which can be both highly complex and inaccurate. While it is possible to register images without calibration information, such unconstrained algorithms are usually time consuming and prone to error. In this dissertation we propose compromise. Even without the instantaneous knowledge of the system geometry, we can restrict the region of correspondence by imposing limits on the possible range of configurations, and as a result, confine our search for matching points to what we refer to as epipolar spaces. For each point in one image, we define the corresponding epipolar space in the other image as the union of all associated epipolar lines over all possible system geometries. Epipolar spaces eliminate the need for calibration at the cost of an increased search region. Since the average size of a search space is directly related to the accuracy and efficiency of any registration algorithm, it is essential to mitigate the increase. The major contribution of this dissertation is the derivation of an optimal nonuniform sampling that minimizes the average area per epipolar space.