Geometric Fourier Analysis in Computational Vision

Transcription

1 Geometric Fourier Analysis in Computational Vision Biological and Mathematical Background Jacek Turski University of Houston-DTN 9 th International Conference on Cognitive and Neural Systems Boston University, May 18-21, 2005

2 Outline I. The Brain Visual pathway 1. Biological Facts 2. Local V1 Topography 3. On Global Topography 4. Cyclopean Symmetry II. GFA for Computational Vision 5. Mathematical Background 6. The Conformal Camera 7. Projective Fourier Analysis III. On Binocular Image Processing 8. Cortical Image Processing 9. Binocular Vision - Cortical Images - Horopter

3 References [1] M. Balasubraminian, J. Polimeni and E.L. Schwartz, The V1-V2-V3 complex: Quasiconformal dipole maps in primate striate and extra-striate cortex, Neural Networks, 15, , [2] J. Turski, Geometric Fourier Analysis for Computational Vision, JFAA, 11, 1-23, Acknowledgement: Some pictures have been borrowed from Eric Schwartz web page

4 1. Biological Facts Image is transmitted from the retina to the visual cortex along the visual pathway in a precise, although changing, retinotopic arrangement, resulting in many maps of features (topography, orientation, eye dominance, motion direction, etc) superimposed in visual cortex. 50% of the primary visual cortex (V1) is used to process input from the foveal region of 4% of the retinal area -80 º -60 º -40 º -20 º 0 º 80 º 60 º 40 º 20 º

5 2. Local V1 Topography With the ganglion density (away from the central part of the fovea) and a uniform V1 packing call density, gives the coordinate mapping r r 2 rrdrd cdud k lnr, k lnz which defines the local V1 topography c z k lnz

6 3. On Global Topography After the work of Eric Schwartz and his group: (1) The mapping w k lnz a is an accepted approximation of the V1 topographic structure. (2) A better model represents a full topography in terms of the mapping w k ln za zb

7 4. Cyclopean Symmetry Author: Herman Gomes

8 5. Mathematical Background Geometric Fourier Analysis (GFA) Harmonic Analysis associated with the Unitary Group Representations Note: The Classical Fourier Transform is associated with the unitary representations of the group of translations. We constructed Computational Harmonic Analysis of the group SL(2,C) -- the group that provides image projective transformations in the Conformal Camera. Remarkably, the resulting image representation is also well adapted to retinotopic mapping of the brain s visual pathway.

9 The Conformal Camera The Camera 6. The Conformal Camera Geometry of the Image Plane Image plane: x 1,1,x 3 x 3 ix 1 x p Complex Projective transformations z g z dzc bza ; g a b c d SL2, z h z p are finite iterations of hz 1 z and k z z z z x 2 x 1 z k z Complex Projective Geometry p 2 S 0,1,0 x 2 1 The embedding: x 2 iy x 3ix 1 z 1 z 2 2 g z 1 z 2 induces g z on slopes of lines z 2 z z 1 Image conformal projective transformations: fz fg 1 z

10 6. The Conformal Camera Image plane 2 S 0,1,0 The conformal camera reduces the projective degrees of freedom to a minimal set of image projective transformations. We will show later that this set of projective transformations is relevant to the way the human vision system acquires the understanding of 3D scenes.

11 7. Projective Fourier Analysis Geometric Fourier Analysis of the Conformal Camera G ive n the action of SL2,: z g z, de compos e a patte rn f L 2 in te rms of the irre ducible unitary re pre s e ntations of SL2, pre s e nt in L 2. The Projective Fourier Transform (PFT) fs, k i 2 fz z is1 z z k dzd z The Inverse PFT fz k fs, k z is1 k z z k ds

12 7. Projective Fourier Analysis The Borel Characters k,s (B) = z -is ( z/ z ) -k Gauss Decomposition SL2,. NB, where N and B z 0 z 1, implies that the Borel subgroup B exhausts the projective part of SL2,.

13 8. Cortical Image Processing PFT is the standard FT in log-polaru, where u ln r We discretize PFT and compute it by FFT in log-polar Nonuniform Log-polar Sampling Retinal Image Inverse DPFT in Log-polar Coordinates Cortical Image

14 8. Cortical Image Processing The Discrete PFT and Its Inverse (Log-polar) M1 f m,n k0 N1 l0 f k,l e u k e i2u m k/t e i2 nll f k,l 1 M1 MN N1 m0 n0 f m,n e u k e i2u m k/t e i2 nll Projective covariance of PFT f m,n 1 MN M1 k0 N1 f k,l e u m,n l0 e i2u m,n k/t e i m,n ll u m,n i m,n lnz m,n lng 1 z m,n

15 9. Binocular Vision Right Hemisphere Right Visual Field Right Eye Fovea V2 & V3 Cortical Projection of the Left Visual Field LGN V1 Fixation point Left Visual Field Left Eye Fovea LGN V2 & V3 V1 Cortical Projection of the Right Visual Field Left Hemisphere Binocular Disparity

16 - Cortical Images The Projection onto the Right Eye Retina Cortical Image from The Right Eye The Projection onto the Left Eye Retina Cortical Image from The Left Eye

17 - Horopter Horopter = Zero-Disparity Curve Fixation point M V2 &V3 V2 & V3 Cortical Image of the Left Visual Field V1 V1 Cortical Image of the Right Visual Field Theorem. For binocular system with the conformal cameras, the horopters in the visual plane are conics, closely matching the empirical horopters. With retinas acting as if they were spheres, the geometric horopters are circles, known as the Vieth-Muller circles.

18 Conclusions Because: (1) the camera with silicon retina sensors produces the image similar to the topographic image in V1, (2) the line singularity most likely exists in the fovea, the head eye visual cortex integrated system makes possible an efficient, biologically realistic computational approach to binocular vision for robotic systems

19 The End

20 APPENDIX [SM] Sharon, E. and Mumford, D. 2D-Shape Analysis using Conformal mapping, IEEE on Conference CVPR Vol. 2, 2004.