Mathematical Theories of Human Vision

Computational vision is at the heart of robotics and biomedicine, but is still quite primitive when compared with our own visual sense. We effortlessly demonstrate enormous flexibility and generality, which hides its staggering complexity: More than one-third of the primate brain is used to process visual information. How should the function of billions of neurons be characterized in algorithmic terms? How should we associate information-processing tasks–such as edge detection or stereo in vision–to collections of neurons? How can such algorithms be designed, tested, and applied to real-world tasks?

Steven Zucker’s group is pioneering an approach to developing an abstract theory of vision and neural computation. They are studying how one class of computations (linear complementarity problems) generalizes the computational competence of visual cortex from filtering, local selection, and constraint satisfaction to solving polymatrix games. The natural "unit" of computation is a particular cell assembly, with subtle advantages in reliability and accuracy.

Central to the abstraction underlying early vision is the observation that visual cortex is organized largely around measurements of the orientation of entities in the visual field. This suggests a differential-geometric approach, letting visual-orientation selectivity be a substrate for representing those tangents that approximate the curves that bound objects, that define highlights and other surface markings, and that group into sets of curves to provide texture flows (e.g., hair, fur) and other visual patterns. Zucker’s group is therefore developing a unified framework for early vision, in which all tasks are formulated in geometric terms and computations are distributed over spaces of position cross orientation. Models for edge and curve detection, for stereo, and for texture and shading are being developed. A unique advantage of such mathematical models is the analysis of interactions; for example, how shading flows into edges. The stereo algorithms are being applied to the interpretation of biomedical imagery in collaboration with the Medical Image Processing and Analysis Group in the Department of Diagnostic Radiology.