Semarang, Indonesia. Sepuluh Nopember Institute of Technology, Surabaya, Indonesia

Transcription

1 3D Surfaces Reconstructon of Seafloor Images Usng Multvew Camera Based on Image Regstraton Pulung Nurtanto Andono 1,a, I Ketut Eddy Purnama 2,b, Mochamad Harad 2,c, Tach Watanabe 3,d, and Kuno Kondo 3,e 1 Informatcs Engneerng Dept., Faculty of Computer Scence, Dan Nuswantoro Unversty, Semarang, Indonesa 2 Electrcal Engneerng Dept., Faculty of Industral Technology, Sepuluh Nopember Insttute of Technology, Surabaya, Indonesa 3 School of Meda Scence, Tokyo Unversty of Technology, Tokyo, Japan a pulung@research.dnus.ac.d, b ketut@ee.ts.ac.d, c mochar@ee.ts.ac.d, d earth@aqua.meda.teu.ac.jp, e kondo@meda.teu.ac.jp Keywords: 3D Reconstructon, GPA, ICP, Regstraton, SIFT, Stereo Vson. Abstract. In ths paper, we evaluate the effectveness of GPA-ICP for 3D surface reconstructon of underwater coral reef mages. Instead of usng orgnal ICP we have used GPA-ICP for fnal algnment to buld 3D model. In land 3D reconstructon, GPA-ICP s preferable for effcency and robustness. Ths s challengng task to mplement GPA-ICP for 3D reconstructon of underwater mages. Ths paper generates 3D pont cloud of underwater coral reef mages from dataset acqured by stereo camera system. The performance of GPA-ICP s measured by vsual appearance and Mean Square Error between the pont and ther closest neghbor compared to the orgnal ICP. The result shows that GPA-ICP s more accurate and faster than the orgnal ICP. Introducton 3D reconstructon s an mportant computer vson technque and wdely used for several real-world applcaton, such as bometrc recognton, underwater exploraton and vrtual realty. Stereo magery can be used to develop 3D model. Stereo mages acqured from stereo magery are calbrated to determne nternal and external parameter of the camera. Usng both ntrnsc and extrnsc parameter, feature pont extracted by matchng algorthm can be used to generate 3D pont cloud. Fnally, wth famous Iteratve Closest Pont (ICP) algorthm, we can regster several 3D pont clouds nto 3D model. Orgnal ICP was frst ntroduced by Besl and McKay [1]. ICP s wdely adopted for regstraton of 3D pont cloud data set. Unfortunately, orgnal ICP process can be tme-consumng [2]. Several technques were proposed to enhance the performance of orgnal ICP. Toldo et. al. [3] proposed to embed Generalzed Procrustes Analyss nto an ICP. GPA-ICP s global regstraton strategy that attempt to algn multple pont cloud at the same tme. GPA-ICP s proposed for the reason that GPA-ICP s more robust than orgnal ICP [4]. Moreover, GPA-ICP s computatonally efcent compared to orgnal ICP [3]. In ths paper, our goal s to mplement GPA-ICP for fnal algnment n 3D surfaces reconstructon of seafloor mages. It s challengng task, snce 3D reconstructon of underwater mage has many obstacle, such as absorpton and scatterng of lght are occurrng that would hamper or degrade the mage regstraton. Indeed, ths problem can affect the qualty of 3D reconstructon. The remander of ths paper s organzed as follows. Secton II presents relevance lterature used for 3D Surface Reconstructon of seafloor mages. Secton III demonstrates and analyss the result of 3D surface reconstructon. The concluson and road ahead are n secton IV.

2 The Proposed Approach of 3D Surface Reconstructon Our approach s composed of the followng man steps: 1. Calbrate the camera to know the ntrnsc and extrnsc parameter of the camera. 2. Enhance the qualty of underwater mage by Contrast Lmted Adaptve Hstogram Equalzaton (CLAHE). 3. Stereo mage matchng by Scale Invarant Feature Transform (SIFT). 4. Image regstraton by GPA-ICP. Each step s crucal to create a hgh qualty 3D surface reconstructon. However, ths study only evaluates the orgnal ICP and GPA-ICP for underwater 3D surface reconstructon. Ths study s based on prevous research n [5] and [6]. The bref theores of each step are descrbed n the followng subsecton. Camera Calbraton. Sample mage of the dataset needs to be selected for calbratng the camera. Camera calbraton has been performed usng MATLAB Calbraton Toolbox. Camera calbraton was used to obtan ntrnsc and extrnsc parameter of the camera. Intrnsc parameter conssts of focal length, prncple pont, pxel sze, and dstorton coeffcent. Meanwhle, extrnsc parameter descrbes the poston of the camera n the world coordnate. Intrnsc and extrnsc matrx s shown n (1) and (2). s. p A. R t. P (1) X u fx 0 Cx r11 r12 r13 t x Y s. v 0 f y Cy r21 r22 r23 t y Z r31 r32 r33 tz 1 (2) Where A s ntrnsc parameter, R t s extrnsc parameter, [ X, Y, Z ] s the 3D pont coordnates n the world coordnate space. Fg. 1 shows camera coordnate system concdes wth the world coordnate system. Ths paper uses MATLAB Calbraton Toolbox to obtan ntrnsc and extrnsc parameters. Chessboard cm s used to perform camera calbraton. Sze of square n chessboard s cm (Fg. 2). Center of Projecton y x Image Plane Image Pont ( x, y, f ) 3D Pont x, y, z ) ( S S S C Z f Fg. 1. The camera coordnate system concdes wth the world coordnate system Y X Y 300 O 300 O X Fg. 2. Camera Calbraton usng Checkerboard

3 Contrast Lmted Adaptve Hstogram Equalzaton. CLAHE s the extenson of Adaptve Hstogram Equalzaton method (AHE). The am of CLAHE s to enhance the qualty of mage by utlzng a parameter lmt value of hstogram n order to handle over brghtness and contrast on an mage [7]. CLAHE works by dvde an mage nto several non-overlappng regons. Then for each regon, hstogram s calculated. Next, hstogram s clpped by a desred lmt for contrast expanson. The dstrbuton of the pxel for the hstogram can be transform nto unform, exponental, and Raylegh dstrbuton [7]. CLAHE lmts the maxmum value of hstogram. The clp lmt can be obtaned by: M 1 ( Smax 1 N 100 Where s clp lmt factor, M regon sze, N s grayscale value. The maxmum clp lmt s obtaned for =100. Scale Invarant Feature Transform. Our study utlze SIFT based mage matchng to obtan the matchng pont n mage par. These matchng ponts are useful to reconstruct 3D model. SIFT method was developed by Lowe [8]. The method ncludes four phase: 1. Scale-space extreme detecton The frst stage n the process of computng s to dentfy all potental key pont on all scales. Scale an mage space s defned as a functon of whch s the product of convoluton between the Gaussan kernel wth the mage. To fnd features on the magery used operators Dfference of Gaussan (DoG) by complng octave mage pyramd wth dfferent scales. 2. Keypont localzaton From the keypont canddates obtaned from a scale-space extrema detecton, hgh stablty keypont wll be selected. The emergence of feature-level stablty s based on the features of each octave. 3. Orentaton assgnment Orentaton of the keypont based on local gradent drecton of each mage. Any operaton performed on the mage based on the drecton, orentaton and locaton of the keypont. 4. Keypont descrptor Local gradent mage s computed at each scale regon around the keypont. In that stuaton, then transformed to local dstortons and llumnaton changes n the area around the keypont. SIFT used Gaussan to develop scale space therefore t can be called Gaussan scale space. Next stage s compute the Dfference of Gaussan Scale (DoG). (3) = (4) Where s the convoluton of the orgnal mage wth Gaussan flter on the scale and s the convoluton of orgnal mage wth Gaussan flter on scale, k 2 Iteratve Closest Pont. ICP s wdely used for mage regstraton algorthm [1]. Let set of 3D pont n base cloud and q Q dstance, orgnal ICP fnd the closest pont between step s calculates translaton t and rotaton R between p Pbe the be the set of another pont n target cloud. Usng Eucldean p and q as a correspondence pont. The next p and q usng the least square method.

4 Transform the pont q to the pont p wth R and t that mnmzes the sum of square dstance E usng the followng equaton. If the dstance E s less than the gven threshold, then the teraton process s stopped. Fg. 3 llustrates the correspondence estmaton n ICP. P Q Fg. 3. Correspondence estmaton between base cloud P (red) and target cloud Q (blue) E R, t mn p 2 Rq t (5) Rt, Generalzed Procrustes Analyss. Many extenson of ICP have been proposed to solve the shortcomng of ICP, such as speed, range, rate of convergence, and robustness. GPA attempts to mnmze the followng objectve functon: mn m m T T T T T tr c X T jt c X T jt c X T jt c XT jt (6) 1 j1 where tr stands for trace of the matrx, X1, X 2,..., X m are m model pont matrces, whch contan the same set of p ponts n k dmensonal m dfferent coordnate systems, c s the scale factor of model X, T s the orthogonal rotaton matrx of X, t s the translaton vector of X and p T j s a unt vector. Sad X c XT jt, and K s geometrcal centrod. The detal and the mathematcal explanaton of GPA nto an ICP have been descrbed by Toldo et.al. [3]. Experment Result and Analyss Camera Olympus μ Tough-8010 and resoluton of pxels used to collect our underwater mages dataset at Karmunjawa Island Central Java Indonesa. Karmunjawa s a Natonal Marne Park declared as a Natural Conservaton Area by Decree of the Mnster of Forestry, located at South Lattude and ' ' East Longtude n the Java Sea, north of Java, Indonesa. We explore the underwater coral reef mages n 4-5 meter deep. The performance of orgnal ICP and GPA-ICP s measured by vsual appearance and Mean Square Error (MSE) between the pont and ther closest neghbor compared to the orgnal ICP. Respectvely, Fg. 4 and 5 show the regstraton error and computatonal tme of orgnal ICP and GPA-ICP. It shows that GPA-ICP s more accurate and faster than orgnal ICP. Fg. 6 and 7 show the result of 3D pont cloud and 3D surface reconstructon of seafloor mages wth dfferent frame number. Based on these fgures, GPA-ICP produces more 3D pont cloud and more detal 3D surface reconstructon compared to ICP.

5 Tme (n second) MSE Frame Number ICP GPA-ICP Fg. 4. The MSE dstance error of dfferent frame number ICP GPA-ICP Frame Number Fg. 5. The regstraton tme of dfferent frame number Concluson and Future Work In ths paper, we have evaluated GPA-ICP based matchng technque for 3D surface reconstructon of underwater coral reef mages. In our evaluaton, we have been able to show that the performance of GPA-ICP yelds best results, more accurate and lower computatonal tme whch comparable to the orgnal ICP. In the future, a partcular dea we are plannng s to mplement or evaluate our 3D surface reconstructon approach to marne envronment technology, such as rugosty measurement. We can construct vrtual chan from 3D surface reconstructon of underwater coral reef mages to measure the rugosty.

6 Orgnal ICP GPA-ICP (a) (b) (c) (d) (e) (f) Fg. 6. 3D pont cloud. Frst Row. 2 nd frame. Second Row. 20 th frame. Thrd Row. 30 th frame.

7 Orgnal ICP GPA-ICP (a) (b) (c) (d) (e) (f) Fg. 7. 3D surface reconstructon. Frst Row. 2 nd frame. Second Row. 20 th frame. Thrd Row. 30 th frame.

8 References [1] P.J Besl and Nel D. McKay, "A method for regstraton of 3-D shapes," n IEEE Transactons on Pattern Analyss and Machne Intellgence, 1992, pp [2] Anuj Sehgal, Danel Cernea, and and Mlena Makaveeva, "Real-Tme Scale Invarant 3D Range Pont Cloud Regstraton," Image Analyss and Recognton Lecture Notes n Computer Scence, vol. 6111, pp , [3] Roberto Toldo, Alberto Benat, and Fabo Croslla, "Global regstraton of multple pont clouds embeddng the Generalzed Procrustes Analyss nto an ICP framework," n 3DPVT 2010 Conference, [4] Surya Prakash, "Human Recognton usng 2D and 3D Ear Images," Indan Insttute of Technology Kanpur, Inda, Doctoral Thess [5] Pulung Nurtanto Andono, Eko Mulyanto Yunarno, Mochamad Harad, and Valentjn Venus, "3D reconstructon of under water coral reef mages usng low cost mult-vew cameras," n Internatonal Conference on Multmeda Computng and Systems (ICMCS), Tanger, Maroko, 2012, pp [6] Pulung Nurtanto Andono, I Ketut Eddy Purnama, and Mochamad Harad, "Underwater Image Enhancement Usng Adaptve Flterng For Enhanced Sft-Based Image Matchng," Journal of Theoretcal and Appled Informaton Technology, vol. 51, no. 3, pp , May [7] Suprjanto, Ganto, E. Julastut, Azhar, and Lus Epslawat, "Image Contrast Enhancement for Flm-Based Dental Panoramc Radography," n Internatonal Conference on System Engneerng and Technology, [8] Davd G Lowe, Dstnctve Image Featuresfrom Scale-Invarant Keyponts.: Internatonal Journal of Computer Vson, 2004.