Stereo matching approach based on wavelet analysis for 3D reconstruction in neurovision system 1

Transcription

1 Stereo matchig approach based o wavelet aalysis or 3D recostructio i eurovisio system Yige Xiog Visio Iteraces & Sys. Lab. (VISLab) CSE Dept.Wright State U. OH ABSTRACT I this paper a stereo matchig approach or 3D recostructio based o wavelet aalysis is preseted. It ca be used i euro-visio system. The approach ca be divided ito two parts. First the stereo matchig problem is solved with wavelet aalysis. Dyadic discrete wavelet aalysis is adopted i this process ad stereo matchig process is realized with global optimizatio. A coheret hierarchical matchig strategy is costructed so that the stereo matchig process ca be accomplished with coarse to ie techiques. Secod a 3D obect recostructio eural etwork is costructed by usig BP eural etwork. By eedig the image correspodig poits betwee the let image ad right image i a stereo image pair the 3D coordiates o poits o obect surace ca be obtaied usig this eural etwork ad the coiguratio ad shape o the obect ca be recostructed. With multiple 3D recostructio eural etworks the 3D recostructio processes ca be perormed i parallel. The examples or both sythetic ad real images are show i the experimet ad good results are obtaied. Keywords: 3D recostructio Stereo matchig Neural etwork Wavelet aalysis Computer visio. INTRODUCTION Computer visio system takes a importat part i itelliget mauacturig eviromet. Whether the system is perormig ispectios locatig parts or decidig what parts are i the ield o view it is recogizig a particular patter i a itesity image. While may applicatios could be solved by perormig oly 2D obect recogitio. More complex applicatios ad the eed or added lexibility are requirig the acquisitio o three dimesioal (3D) iormatio alog with 3D obect recogitio. Stereo visio is eeded i this system. Stereo visio is a techique or buildig a 3D descriptio o a scee observed rom several viewpoits. I the passive stereo visio usually two cameras are used to observe the same scee rom two slightly dieret viewpoits ad the stereo image pairs are obtaied. Ater the let image ad right image i the stereo image pair are matched. The 3D descriptio o the scee ca be built. Stereo matchig process takes a importat part i the 3D descriptio. The obective o image matchig is to establish correspodece betwee the selected primitives so that the matched primitive arises rom the same elemet i the scee. For the stereo visio the stereo matchig problem is to establish the correspodece betwee let image ad right image i a stereo image pair. There have bee several attempts to ormulate solutios to the problem which may be categorized ito eature ad area based methods ~5. May o problems ecoutered i stereo matchig i the spatial domai ca be avoided by treatig the stereo matchig i requecy domai by usig local phase. The local phase has bee used or stereo matchig process 6-8 ad this is the third category amed phase based approach. Recetly wavelet aalysis has bee used or stereo matchig process 9~. Wavelet decompositio provides a mathematically precise deiitio to the cocept o multiresolutio. The multiresolutio represetatios derived rom wavelet rames provide a mathematically coheret hierarchical ramework to accomplish a coarse to ie icremetal matchig strategy. I this paper a stereo matchig approach based o wavelet aalysis is preseted. First a coheret hierarchical matchig strategy is costructed so that the stereo matchig process ca be accomplished with coarse to ie techiques. The dyadic discrete wavelet aalysis is adopted i this process ad the stereo matchig process is realized with global optimizatio. Secod a 3D obect recostructio eural etwork is costructed by usig BP eural etwork. By eedig the image correspodig poits betwee the let image ad right image i a stereo image pair the 3D coordiates o This work is supported by Guagdog Provice Natural Sciece Foudatio o Chia uder Grat No. 72

2 poits o obect surace ca be obtaied usig this eural etwork ad the coiguratio ad shape o the obect ca be recostructed. With multiple 3D recostructio eural etworks the 3D recostructio processes ca be perormed i parallel D wavelet aalysis 2. STEREO MATCHING WITH WAVELET ANALYSIS The 2D wavelet trasorm o the uctio ( x L 2 ( R) T Ψ with a give mother wavelet Ψ is deied by + + ( s u v) ( x Ψs u v ( x = dxdy Ater discretizig both the scale actor s ad the traslatio ( u v) () We ca obtai the dyadic scale space. s = ad ( u k v = l) 2 = with ( ) Z 3 k l (2) where Z deotes the set o itegers. Let D deote the dierece betwee two approximatios A ad A D where A is a idetity operator. = A A = 2 (3) A is the approximatio o a give uctio ( x at a scale s = 2 2 =. There are three compoets or the dierece betwee two approximatios A ad A D = A A = D D (4) so the multiresolutio aalysis o a uctio ( x ca be writte as ( x = A = A 2 = A + 2 = [ D ] 2 o (5) 3 Give a discrete image ( x with a limited support wavelet pyramid ivolves computig the coeiciets A D p = 23 p o each level. x y m 2 d = 2 the actual procedure or costructig the a k l p k l which ca be grouped ito our matrices

3 A = ) D = d ) or ( a k l p ( p k l k l = 2 2 (6) Let h ad g be the impulse respose o the ilter φ ad ψ the coeiciets a k l ad d p k l p = 2 3 ca be computed via a iterative procedure. The wavelet pyramid o image ( x ad its costructig process are illustrated i igure ad igure 2. Fig.. Wavelet pyramid o a image ( x Fig. 2. The lowchart or the aalysis rom level level to 2.2. Image matchig with global optimizatio Stereo matchig process is to id correspodece betwee let image ad right image i stereo image pair. For ay image poit ( k l ) o the reerece image its approximate correspodece ( k l ) o the matched image may be obtaied through some geeral strategies such as the spiral ad hierarchical parallax propagatio. The simplest way to catch the precise correspodece ( k l ) is the discrete search i a small eighborhood o ( k l ) which is deied by a distace threshold T r mi ( k l ) S (( k l)( k l )) ( k l ) : ( k l ) ( k l ) T r (7) The distace threshold Tr should be deied i such a way that the allowed errors i the parallax propagated rom the last higher level ca be corrected. I geeral T 2. Note that the search takes place ot oly o ormal iteger r

4 positios but also o diagoal positios. The computatioal structure ad the data low o wavelet rame processig or stereo matchig is show i igure 3. Fig. 3. The computatioal structure ad the data low 3 THREE DIMENSIONAL RECONSTRUCTION I the covetioal 2D to 3D poits coversio process a elaborate system calibratio to determie the relatioships betwee the two cameras ad the global co-ordiate system is required. Usig 3D poits kow i 3D space ad correspodig 2D poits o images to create the relatioship betwee the cameras ad global co-ordiate system is usually the approach. The each three dimesioal poit is oud by eedig the two matchig poits ito equatios goverig the trasormatios. Usually the Colliear is required. Istead o covetioal methods here we costruct a our layers back propagatio eural etwork to perorm the operatio. The iput layer has our euros or X ad y

5 values o the matched poits o the let ad right images o a stereo image pair. The etwork has two hidde layers. Each cotais 5 euros or storig the complex relatioship betwee 2D image ad 3D space. The output layer has 3 euros or the 3D poits. The architecture o the eural etwork is give as Figure 4. By eedig the matched poits ito the iput layer the eural etwork ca output the 3D poits o the obect. I multiple etworks are available poits ca be coverted i parallel. 4. EXPERIMENTAL RESULTS AND DISCUSSION We have tested the approach preseted i this paper with a series o sythesized images ad ature images. All images show here are composed o about 52x52 pixels. Durig the stereo matchig processes the epiplor lies are used to provide useul costrai or eural etworks. The example is a stereo image pair o a turbie blade (show i igure 5). The let ad right images are take at parallel positio. Figure 4 The architecture o 2D to 3D poit coversio eural etwork Figure 6 shows the 3D plot o the turbie blade recostructed rom the results o stereo matchig. It ca be see that the 3D shape ad surace are well recovered. This example is desiged to test the ability o the approach to be used i practical situatio. From this test we ca see that the results ca be used i the 3D recostructio o obect. (a) let image (b) right image Figure 5 The stereo image pair o turbie blade Figure 6 The result o 3D recostructio 5. CONCLUSION I this paper a stereo matchig approach or 3D recostructio i euro-visio system is preseted. The stereo matchig problems are solved based o the wavelet aalysis. I the dyadic scale space The image ca be decomposed ito a set o requecy chaels havig the same badwidth o a logarithmic scale. By adaptatio to scale ad traslatio a course to ie image matchig process ca be costructed. With the image matchig strategy the correspodig poits betwee let image ad right image i stereo image pair ca be obtaied. A 3D obect recostructio eural etwork is costructed by usig BP eural etwork so the 3D coiguratio ad shape ca be recostructed by this eural etwork. With multiple eural etworks the 3D recostructio processes ca be perormed i parallel. Some examples ad applicatios or this approach are also preseted. We have tested this approach with

6 some sythesize ad ature images. From the results o the examples ad applicatios we ca see that the shapes ad surace o obects are well recovered rom the results o stereo matchig. The approach preseted i this paper ca be used i computer visio ad itelliget assembly system. REFERENCES. Takeo Kaade ad Masatoshi Okutomi A Stereo Matchig Algorithm with a Adaptive Widow: Theory ad Experimet Proceedigs o the 99 IEEE Iteratioal Coerece o Robotics ad Automatio Sacrameto Calioria ppl88-95 April M. J. Haah A System or Digital Stereo Image Matchig Photogrammetric Egieerig ad Remote Sesig 55(2) Dec G.Medioi ad R.Nevatia Segmet-based Stereo Matchig Comput. Visio Graphics Image Processig Vol M.lto ad A.lshii Three-View Stereo Aalysis IEEE Trasactios o Patter Aalysis ad Machie Itelligece Vol.PAMI-8No.4July D.Steve Cochra ad Gerard Medioi 3-D Surace Descriptio rom Biocular Stereo IEEE Trasactios o Patter Aalysis ad Machie Itelligece Vol. 4 No. pp ctober M.Wolgag Theimer ad A.Haspeter Mallot Phase-Based Biocular Vergece Cotrol ad Depth Recostructio Usig Active Visio CVGIP: Image Uderstadig Vol.6 No.3. pp November Juyag Weg Widowed Fourier Phase: Completeess ad Sigal Recostructio IEEE Trasactios o Sigal Processig. Vol.4 No.2 February pp Yali Xiog ad A.Steve Shaer "Variable Widow Gabor Filters ad Their Use i Focus ad Correspodece" Proc. Computer Visio Patt.Recog.pp Jaworska T A ew approach to stereo image matchig based o multiresolutio wavelet aalysis. MGV vol. 9 o. /2 2 pp N. G. Kigsbury Complex wavelets or shit ivariat aalysis ad ilterig o sigals Joural o Applied Computatio ad Harmoic Aalysis Jue 2.. N. G. Kigsbury A Dual-Tree Complex Wavelet Trasorm with improved orthogoality ad symmetry properties Proc. IEEE Co. o Image Processig Vacouver September -3 2.