Depth error correction for projector-camera based consumer depth cameras

Transcription

1 Computatonal Vsual Meda Vol. 4, No. 2, June 2018, Research Artcle Depth error correcton for projector-camera based consumer depth cameras Hrotake Yamazoe 1 ( ), Hrosh Habe 2, Ikuhsa Mtsugam 3, and Yasush Yag 4 c The Author(s) Ths artcle s publshed wth open access at Sprngerlnk.com Abstract Ths paper proposes a depth measurement error model for consumer depth cameras such as the Mcrosoft Knect, and a correspondng calbraton method. These devces were orgnally desgned as vdeo game nterfaces, and ther output depth maps usually lack suffent accuracy for 3D measurement. Models have been proposed to reduce these depth errors, but they only consder camera-related causes. Snce the depth sensors are based on projectorcamera systems, we should also consder projectorrelated causes. Also, prevous models requre dsparty observatons, whch are usually not output by such sensors, so cannot be employed n practce. We gve an alternatve error model for projector-camera based consumer depth cameras, based on ther depth measurement algorthm, and ntrnsc parameters of the camera and the projector; t does not need dsparty values. We also gve a correspondng new parameter estmaton method whch smply needs observaton of a planar board. Our calbrated error model allows use of a consumer depth sensor as a 3D measurng devce. Expermental results show the valdty and effectveness of the error model and calbraton procedure. Keywords consumer depth camera; ntrnsc calbraton; projector; dstorton 1 College of Informaton Sence and Engneerng, Rtsumekan Unversty, Shga, , Japan. E-mal: yamazoe@fc.rtsume.ac.jp ( ). 2 Faculty of Sence and Engneerng, Knda Unversty, Osaka, , Japan. E-mal: habe@knda.ac.jp. 3 Graduate School of Informaton Sences, Hroshma Cty Unversty, Hroshma, , Japan. E-mal: mtsugam@hroshma-cu.ac.jp. 4 The Insttute of Sentfc and Industral Research, Osaka Unversty, Osaka, , Japan. E-mal: yag@am.sanken.osaka-u.ac.jp. Manuscrpt receved: ; accepted: Introducton Recently, varous consumer depth cameras such as the Mcrosoft Knect V1/V2, Asus Xton, etc. have been released. Snce such consumer depth sensors are nexpensve and easy to use, these devces are wdely deployed n varous felds for a wde varety of applcatons [1, 2]. These consumer depth cameras can be dvded nto two categores: () projector-camera based systems n whch a projector casts a structured pattern onto the surface of a target object, and () tme-of-flght (ToF) sensors that measure the tme taken for lght to travel from a source to an object and back to a sensor. ToF sensors generally gve more accurate depths than projector-camera based ones, whch are, however, stll useful because of ther smplty and low cost. Snce projector-camera based devces nclude cameras and a projector, output errors may be caused n errors n determnng the ntrnsc parameters. As long as such devces are used as human nterfaces for vdeo games, such errors are unmportant. For example, even when a Knect V1 captures a planar object, the resultant depth maps have errors (see Fg. 1, left) as also reported elsewhere [3, 4]. Thus, n ths paper, we focus on projector-camera based consumer depth cameras and propose a depth error correcton method based on ther depth measurement algorthm. Varous ntrnsc calbraton methods have already been proposed for Knect and other projector-camera based depth cameras [3 11]. Smsek et al. [3] and Herrera et al. [4] proposed calbraton and depth correcton methods for Knect that reduce the depth observaton errors. Raposo et al. [6] extended Herrera et al. s method to mprove stablty and 103

2 104 H. Yamazoe, H. Habe, I. Mtsugam, et al. Fg. 1 Left: observaton errors n Knect output. Rght: compensated values usng our method. speed. However, ther methods only consdered dstorton due to the nfrared (IR) cameras. Snce projector-camera based depth sensors nclude cameras and a projector, we should also consder projector-related sources of error. We prevously proposed a depth error model for Knect ncludng projector-related dstorton [5]. Darwsh et al. [8] also proposed a calbraton algorthm that consders both camera and projectorrelated parameters for Knect. However, these methods as well as other prevous methods requre dsparty observatons, and these are not generally provded by such sensors. Thus, methods that requre dsparty observatons cannot be employed n practce for error compensaton for data from exstng commeral sensors. Some researchers employ non-parametrc models for depth correcton but a calbraton board needs to be shown perpendcular to the sensor [9, 11], or ground truth data obtaned by smultaneous localzaton and mappng (SLAM) are requred [12, 13]. Jn et al. [10] proposed a calbraton method usng cubods, but, ther method s also based on dsparty observatons. Other researchers proposed error dstrbuton models for Knect [14, 15], but ths research dd not focus on error compensaton. To provde straghtforward procedures for calbraton and error compensaton for depth data, ncludng prevously captured data, our method ntroduces a parametrc error model that consders () both camera and projector dstorton, and () errors n the parameters used to convert dsparty observatons to actual dsparty. To estmate the parameters n the error model, we propose a smple method that resembles the common color camera calbraton method [16]. Havng placed a planar calbraton board n front of the depth camera and captured a set of mages, our method effently optmzes the parameters allowng us to reduce the depth measurement errors (see Fg. 1, rght). Our compensaton model only requres depth data, wthout the need for dsparty observatons. Thus we can apply our error compensaton to any depth data captured by projector-camera based depth cameras. We note that the calbraton method ntroduced n ths paper s desgned for Knect because t s the most common projector-camera based depth sensor. However, t potentally generally more useful because t s based on a prnple common to other projectorcamera based depth sensors. Secton 2 descrbes the measurement algorthm used by Knect, and Secton 3 descrbes our parametrc error model and parameter estmaton. Secton 4 shows expermental results demonstratng the effectveness of our proposed method, whle Secton 5 summarzes our paper. 2 Depth error model 2.1 Depth measurement by Knect Snce our method s based on the measurement algorthm used by the Knect, we frst outlne ths algorthm and ths depth sensor, whch conssts of an IR camera and an IR projector. The IR projector projects speal fxed patterns (speckle patterns) on the target observed by the IR camera. By comparng the observed and reference patterns captured n advance, Knect estmates depth nformaton for the target. The reference patterns are observatons made by the IR camera when the IR projector casts the speckle pattern on the reference plane Π 0 [17] (see Fg. 2). Here, we assume pattern P (x p ) s projected n the drecton of pont x p =[x p,y p ] T onto the reference plane Π 0, and pattern P (x p )onπ 0 s projected onto the 2D poston x (Π 0) = [x (Π 0),y (Π 0) ] T for the IR camera. We obtan the followng relatonshp: x (Π 0) = x p + fw/z 0 (1) where w s the baselne dstance between the camera and the projector, f s the focal length of the IR camera (and the IR projector), and Z 0 s the dstance between the reference plane Π 0 and the Knect. Next, we consder the target observaton measured at pont Q and assume that pattern P (x p ) s observed at x usng the IR camera. By consderng

3 Depth error correcton for projector-camera based consumer depth cameras 105 Fg. 2 Depth measurement by Knect. the reference patterns, the pattern s observed poston when t s projected onto reference plane Π 0, x (Π 0), can be obtaned. We calculate dsparty δ from the reference plane observaton at x as follows: δ = x x (Π 0) = x x p fw/z 0 (2) Then X, whch s the 3D poston of pont Q, can be calculated as X = [ (x x cc )Z, (y ] T y cc )Z fw, (3) f f fw/z 0 + δ where x cc and y cc are the IR camera s prnpal ponts and Z s the depth of pont Q. Knect does not output dsparty values, but only normalzed observatons δ from 0 to 2047 (n Knect dsparty unts: kdu) [17], where δ = mδ + n. The drver software for Knect (Knect for Wndows SDK and OpenNI) uses these to calculate and output depth values Z based on the followng equaton: fw Z = fw/z 0 + mδ + n = fw (4) d The dsparty between the camera and projector d can be expressed as follows: d = fw/z 0 + mδ + n (5) Note that recent versons of the drver software do not support output of dspartes δ, so these are generally unobtanable. Instead, we propose a method to calbrate and compensate the depth data obtaned by Knect that does not requre ether the dsparty or normalzed dsparty observatons. 2.2 Depth error model The depth measurement model descrbed above holds only n an deal case. In practce, when Knect observes a planar target, the output depth maps have errors, as prevously noted [3, 4] (see Fg. 1). To be able to compensate for them, we consder not only camera dstorton but also the projector dstorton n our model Dstorton parameters A well-known lens dstorton model s x = x +(x x cc ) [ k c1 u T u + k c2 (u T u ) 2] y +(y y cc ) [ k c1 u T u + k c2 (u T u ) 2] (6) where x = [x,y ] T and x are the deal and observed, dstorted 2D postons, and u gves the normalzed coordnates of x. k c1 and k c2 are the dstorton parameters of the IR camera. We assume the same dstorton model can be used for the projector: x p = x p +(x p x pc ) [ k p1 u T p u p + k p2 (u T p u p ) 2] y p +(y p y pc ) [ k p1 u T p u p + k p2 (u T p u p ) 2] (7) where x p and x p are the deal and dstorted 2D postons, and u p gves the normalzed coordnates of x p. [x pc,y pc ] T s the prnpal pont of the projector, and k p1 and k p2 are the dstorton parameters of the IR projector. We now consder pattern P (x p ) projected n the drecton of pont x p (see Fg. 3). However, because of projector dstorton, pattern P (x p ) s actually projected n the drecton of pont x p, and s projected onto pont Q. In the camera, pattern P (x p ) s actually projected onto poston x because of the camera dstorton. Let d be the observed (dstorted) dsparty at x and d = x x p (8) On the other hand, consderng Q n Fg. 3, the deal dsparty d correspondng to pont Q should be d = x x p = d ɛ c ɛ p (9) where ɛ c = x x and ɛ p = x p x p Proposed error model Equaton (9) expresses the relaton between the deal dsparty d and the observed dsparty d. In practce, snce the parameters n Eq. (5),.e., f, w, Z 0, m, and n nclude errors, we need to

4 106 H. Yamazoe, H. Habe, I. Mtsugam, et al. Fg. 3 Proposed error model. compensate for them. Here, let d and d be the deal values and the values calculated by Eq. (5) based on the error parameters and the observed dsparty. Consderng the errors n the parameters n Eq. (5) and collectng the coeffents, the followng relatons can be obtaned: d = α d + β (10) where α and β are parameters for compensatng errors n f, w, Z 0, m, andn. A detaled dervaton of Eq. (10) s shown n Appendx A. Thus, the deal dsparty d can be expressed as follows: d = d ɛ c ɛ p =(α d + β) ɛ c ɛ p (11) By ntrodung α and β, we can compensate for errors n the parameters n Eq. (5) wthout observng the normalzed dsparty tself. Therefore, we calbrate not only the dstorton parameters of the camera and the projector but also α and β allowng us to compensate for errors n these values. In the next secton, we descrbe parameter estmaton for ths error model. 3 Algorthm 3.1 Overvew In consumer depth sensors, snce the projecton patterns cannot be controlled, we cannot drectly estmate the projector s dstorton parameters. Instead, we estmate the error model parameters usng the process flow shown n Fg. 4. Frst, we obtan N IR mages and correspondng depth data for a calbraton board (of known sze and pattern) n arbtrary poses and postons. Ths lets us perform ntrnsc calbraton of the IR camera by Fg. 4 Process flow. Zhang s method [16]. As descrbed n the prevous secton, we model the depth errors based on Eq. (11), and deal dsparty d and observed dsparty d are requred. Here, we assume that the poses and postons of the board estmated by ntrnsc camera calbraton are deal depth values, and calculate deal dsparty d from these poses and postons. The observed dsparty values can be calculated from the observed depth values. Next we estmate the error model parameters by mnmzng Eq. (11) based on d and d. Table 1 summarzes the notaton used n the followng. 3.2 Camera calbraton Frst, ntrnsc calbraton of the IR camera s performed usng the N mages captured by the IR cameras, usng Zhang s method [16]. For camera calbraton, X k, x bk, the sze of the chessboard, and the number of checker patterns on the chessboard should be gven. Zhang s method can estmate the focal length (f), the prnpal pont (u cc, v cc ), and the camera dstorton parameters (k c = {k c1,k c2 }). The dsparty dfferences caused by the camera lens Table 1 Notaton ndex of ponts j ndex of observatons, j =1,...,N k ndex of chessboard corner ponts x 2D postons n mage j X 3D postons at x obtaned by sensor X k 3D postons of chessboard corners (Z bk =0) 2D postons of corners n mage j x bk X bk 3D postons at x bk obtaned by sensor

5 Depth error correcton for projector-camera based consumer depth cameras 107 dstorton may be consdered. Let k c be camera the dstorton parameter, and ɛ c be dsparty error caused by k c. Then k c and ɛ c can be expressed as follows: { kc = k c +Δk c ɛ c = ɛ c +Δɛ c (12) and Eq. (11) can be rewrtten as follows: d =(α d + β) Δɛ c ɛ p (13) In addton, we can obtan the board s poses and postons n each mage j: (R, t ). Ths nformaton s used n the followng processes. 3.3 Projector calbraton and dsparty compensaton parameter estmaton Next, we estmate the dstorton parameters for the projector and the dsparty converson parameters. To do so, we use the relatons n Eq. (13) to gve the followng equaton: (α j {d d + β Δɛ ɛ p )} = 0 (14) The deal dspartes d can be calculated from the estmated poses and postons R, t as follows: d = wf Z b (x ) (15) where X b (x ) gves the 3D poston on the board correspondng to drecton x (see Fg. 5), and Z b (x )sthez component of X b (x ). The observed dsparty values d can be estmated usng: Fg. 5 d = wf o Z Relatonshp between 2D and 3D observatons. (16) Z where s the observed depth value at poston x. f o s the focal length for the depth sensor used for calculatng observed dsparty values d, and can be estmated from X. ɛ can be expressed as follows: ɛ = x x [ = ( x x cc ) k c1 (ŭ )T ŭ + k c2 ((ŭ )T ŭ )2] where x = ( [ x x x cc y cc ]) [kc1(ŭ (17) )T ŭ + kc2((ŭ )T ŭ )2] (18) and we employ the approxmate undstorted model [18]. Based on the above equatons, we can estmate Δk c, k p, α, andβ by mnmzaton as below: [ˆkp ˆβ], ˆα, = arg mn Δk c,k p,α,β j {d (α d +β Δɛ ɛ p )} (19) In our experments, we employed the mult-start algorthm n the MATLAB and Global Optmzaton Toolbox (verson 2017b) for optmzaton. Snce t requres ntal values, they were determned as follows. Note that w s not ncluded n the parameters to be calbrated, because w and dspartes d and d are proportonal. Instead, we employ w = 75mm based on Ref. [19]. We frst consder the ntal values of converson parameters ˆα and ˆβ. From Eqs. (15) and (16), we obtan the followng relatons by substtutng 0 for ɛ and ɛ p nto Eq. (11): d =ˆα d + ˆβ (20) allowng ˆα and ˆβ to be determned by least-squares fttng. Next, we consder the ntal values for dstorton errors ˆɛ p. Consderng the relatons between the camera and the projector and Eq. (8), we can estmate x p and x p as follows: [ ] w x p m = A p X b 1 0 (21) 0 [ ] α d + β x p = x (c) 0 (22)

6 108 H. Yamazoe, H. Habe, I. Mtsugam, et al. where A p are ntrnsc parameters of the projector. Usng Eq. (7), we can obtan the followng equaton: ˆɛ p = = x p x p ( [ ]) x p [ˆkp1(ŭ x pc y pc p )T ŭ p + ˆk p2((ŭ p )T ŭ p )2 ] (23) We then estmate the optmal values of ɛ p (and k p1,k p2 ), α, andβ by mnmzng Eq. (19) from these ntal values. 3.4 Depth compensaton usng calbraton results Fnally, we descrbe the compensaton process for the depth data obtaned from the depth sensors. Here, we consder the 3D data X at pxel x. Frst, we obtan d and ɛ c from Eqs. (16) and (17). x p can be calculated from Eq. (22). Next, we calculate ɛ p from Eq. (23). Then, the compensated dsparty d can be calculated from Eq. (11), so the compensated depth Z can be obtaned as Z = fw (24) d and the compensated 3D data X s gven by [ (x x cc )Z X = f 4 Experments, (y ] T y cc )Z,Z (25) f We performed the followng experments to confrm the valdty of our proposed error model and error compensaton. In the experment, we used a Knect for Xbox (Devce 1, abbrevated Dev.1, etc.), a Knect for Wndows (Devce 2), and an ASUS Xton Pro (Devce 3); all of these devces are based on the same Prmesense measurement algorthm [20]. We compared the compensated results usng the followng three models and the observed raw data: (a) our proposed method; (b) model consderng camera dstorton and converson parameters (wthout ɛ p ); (c) model consderng only camera dstorton errors (wth Δɛ c ); (d) no compensaton,.e., observed raw data. We captured 12 observatons of the chessboard n dfferent arbtrary poses and postons n the experments. The dstances between the board and the devce were about mm. A leave-oneout method was used for evaluatng the valdty of the proposed error model: one observaton was used for evaluaton and the remanng observatons were used for estmatng error model parameters. From the observatons, we manually obtaned the 2D postons of the chessboard corners (54 ponts per mage). Table 2 shows the resdual errors after the calbraton phase, and Table 3 shows the errors n evaluatons. Here, the errors were calculated as the averaged dstances between the compensated (or observed) postons and ground truth postons of the chessboard corners. We used the 3D postons obtaned from the color camera observatons as the ground truth postons. These comparatve results show that all three models can reduce errors compared to the uncompensated results, n both the calbraton and evaluaton phases. The errors compensated by (a) our proposed model were the lowest, followed by (b) the model that consdered camera dstorton and lnear relatons, and then (c) the model that consdered only camera dstorton. The number of parameters used n these models also has the same orderng: (a) has the most, followed by (b) and then (c). These results suggest that usng all parameters consdered n our proposed error model are helpful n mprovng the qualty of the 3D depth data. After calbraton, we evaluated the flatness of the compensated observatons for the chessboard, measurng plane fttng errors wthn the chessboard regons. Table 4 shows comparatve results for these plane fttng errors. These results show that the plane fttng errors n compensated observatons from our proposed model Table 2 Comparson of averaged errors durng calbraton (a) (b) (c) (d) proposed w/o ɛ p w ɛ c w/o comp. (mm) (mm) (mm) (mm) Dev Dev Dev Table 3 Comparson of averaged errors n evaluaton (a) (b) (c) (d) proposed w/o ɛ p w ɛ c w/o comp. (mm) (mm) (mm) (mm) Dev Dev Dev

7 Depth error correcton for projector-camera based consumer depth cameras 109 Table 4 Comparson of plane fttng errors n evaluaton (a) (b) (c) (d) proposed w/o ɛ p w ɛ c w/o comp. (mm) (mm) (mm) (mm) Dev Dev Dev (a) decreased, but on the other hand, typcally for other methods (b) and (c), the plane fttng errors ncreased. These results suggest that all parameters consdered n our proposed error model are requred to mprove the qualty of the 3D depth data. Next, we evaluated the method s robustness to errors n the gven baselne length w. Our method assumes the target devce s baselne length s that gven n such artcles as Ref. [19]. However, f t s not gven, we need to measure t ourselves. In such cases, the measured length may nclude errors. Thus, we evaluated the robustness to errors n the baselne length w of up to ±2 mm. Table 5 shows the errors when the baselne ncludes errors. As can be seen, our proposed model can reduce errors between the compensated postons and the ground truth postons even when the gven baselne length ncludes errors. Ths s because our model consders errors n the baselne length w as one of the parameters n Eq. (5). These expermental results, confrm that our proposed model can mprove the qualty of 3D depth data obtaned by consumer depth cameras such as Knect and Xton. 5 Summary In ths paper, we have proposed and evaluated a depth error model for projector-camera based consumer depth cameras such as the Knect, and Table 5 Resdual errors wth varyng baselne length errors Error Dev.1 Dev.2 Dev.3 (mm) (mm) (mm) (mm) w/o comp an error compensaton method based on calbraton of the parameters nvolved. Snce our method only requres depth data wthout dsparty observatons, we can apply t to any depth data captured by projector-camera based depth cameras such as the Knect and Xton. Our error model consders () both camera and projector dstorton, and () errors n the parameters used to convert from normalzed dsparty to depth data. The optmal model parameters can be estmated by showng a chessboard to the depth sensor usng multple arbtrary dstances and poses. Expermental results show that the proposed error model can reduce depth measurement errors for both Knect and Xton by about 70%. Our proposed model has sgnfcant advantages when usng a consumer depth camera as a 3D measurng devce. Future work ncludes further nvestgaton of the error model, mprovement of the optmzaton approach for parameter estmaton, and mplementaton of a calbraton tool based on the proposed error model for varous projectorcamera based depth cameras, such as the Intel RealSense and Ocptal Structure Sensor, as well as the Mcrosoft Knect. Appendx A Dervaton of Eq. (10) Consderng errors n the parameters n Eq. (5), d can be expressed as follows: d = f w/ Z 0 + mδ + n (26) where f, w, Z0, m, and n denote the parameters whch contan errors. Let Δ( f w/ Z 0 ), Δ m, andδ n be the errors n these parameters. Then d, the deal value of the observed dsparty, can be obtaned as follows: ( d = f w/ Z0 +Δ( f w/ Z 0 )) +( mδ m) δ +( n +Δ n) (27) All parameters except δ are fxed values, so we can obtan Eq. (10) by collectng coeffents: d = α d + β (28) Acknowledgements Ths work was supported by the JST CREST Behavor Understandng based on Intenton-Gat Model project.

8 110 H. Yamazoe, H. Habe, I. Mtsugam, et al. References [1] Zhang, Z. Mcrosoft Knect sensor and ts effect. IEEE Multmeda Vol. 19, No. 2, 4 10, [2] Han, J.; Shao, L.; Xu, D.; Shotton, J. Enhanced computer vson wth Mcrosoft Knect sensor: A revew. IEEE Transactons on Cybernetcs Vol. 43, No. 5, , [3] Smsek, J.; Jancosek, M.; Pajdla, T. 3D wth Knect. In: Proceedngs of the IEEE Internatonal Conference on Computer Vson Workshops, , [4] Herrera, D.; Kannala, J.; Hekklä, J. Jont depth and color camera calbraton wth dstorton correcton. IEEE Transactons on Pattern Analyss and Machne Intellgence Vol. 34, No. 10, , [5] Yamazoe, H.; Habe, H.; Mtsugam, I.; Yag, Y. Easy depth sensor calbraton. In: Proceedngs of the 21st Internatonal Conference on Pattern Recognton, , [6] Raposo, C.; Barreto, J. P.; Nunes, U. Fast and accurate calbraton of a Knect sensor. In: Proceedngs of the Internatonal Conference on 3D Vson, , [7] Xang, W.; Conly, C.; McMurrough, C. D.; Athtsos, V. A revew and quanttatve comparson of methods for Knect calbraton. In: Proceedngs of the 2nd Internatonal Workshop on Sensor-based Actvty Recognton and Interacton, Artcle No. 3, [8] Darwsh, W.; Tang, S.; L, W.; Chen, W. A new calbraton method for commeral RGB-d sensors. Sensors Vol. 17, No. 6, 1204, [9] Wess, A.; Hrshberg, D.; Black, M. J. Home 3D body scans from nosy mage and range data. In: Proceedngs of the Internatonal Conference on Computer Vson, , [10] Jn, B.; Le, H.; Geng, W. Accurate ntrnsc calbraton of depth camera wth cubods. In: Computer Vson ECCV ECCV Lecture Notes n Computer Sence, Vol Fleet, D.; Pajdla, T.; Schele, B.; Tuytelaars, T. Eds. Sprnger, Cham, , [11] D Ccco, M.; Iocch, L.; Grsett, G. Nonparametrc calbraton for depth sensors. Robotcs and Autonomous Systems Vol. 74, , [12] Techman, A.; Mller, S.; Thrun, S. Unsupervsed ntrnsc calbraton of depth sensors va SLAM. Robotcs: Sence and Systems Vol. 248, 3, [13] Wang, H.; Wang, J.; Lang, W. Onlne reconstructon of ndoor scenes from RGB-D streams. In: Proceedngs of the IEEE Conference on Computer Vson and Pattern Recognton, , [14] Park, J.-H.; Shn, Y.-D.; Bae, J.-H.; Baeg, M.-H. Spatal uncertanty model for vsual features usng a Knect TM sensor. Sensors Vol. 12, No. 7, , [15] Nguyen, C. V.; Izad, S.; Lovell, D. Modelng Knect sensor nose for mproved 3D reconstructon and trackng. In: Proceedngs of the 2nd Internatonal Conference on 3D Imagng, Modelng, Processng, Vsualzaton & Transmsson, , [16] Zhang, Z. A flexble new technque for camera calbraton. IEEE Transactons on Pattern Analyss and Machne Intellgence Vol. 22, No. 11, , [17] Khoshelham, K.; Elbernk, S. O. Accuracy and resoluton of Knect depth data for ndoor mappng applcatons. Sensors Vol. 12, No. 2, , [18] Hekkla, J. Geometrc camera calbraton usng rcular control ponts. IEEE Transactons on Pattern Analyss and Machne Intellgence Vol. 22, No. 10, , [19] Dal Mutto, C.; Zanuttgh, P.; Cortelazzo, G. M. Tmeof-Flght Cameras and Mcrosoft Knect TM. Sprnger Sence & Busness Meda, [20] Freedman, B.; Shpunt, A.; Arel, Y. Dstancevaryng llumnaton and magng technques for depth mappng. U.S. Patent 8,761, Hrotake Yamazoe receved hs B.E., M.E., and Ph.D. degrees n engneerng from Osaka Unversty n 2000, 2002, and 2005, respectvely. He was wth the Advanced Telecommuncatons Research Insttute Internatonal (ATR) durng , the Insttute of Sentfc and Industral Research, Osaka Unversty durng , and Osaka School of Internatonal Publc Polcy, Osaka Unversty durng He s now a lecturer n the College of Informaton Sence and Engneerng, Rtsumekan Unversty. Hs research nterests nclude computer vson and wearable computng. He s a member of IEICE, IPSJ, ITE, HIS, IEEE, and ACM. Htosh Habe receved hs B.E. and M.E. degrees n electrcal engneerng and D.Info. degree n ntellgence sence and technology from Kyoto Unversty, Japan, n 1997, 1999, and 2006, respectvely. After workng for Mtsubsh Electrc Corporaton from 1999 to 2002, Kyoto Unversty from 2002 to 2006, Nara Insttute of Sence and Technology

9 Depth error correcton for projector-camera based consumer depth cameras 111 from 2006 to 2011, and Osaka Unversty from 2011 to 2012, he s now a lecturer n the Department of Informatcs, Knda Unversty, Japan. From 2010 to 2011, he was a vstng researcher at the Department of Engneerng, Unversty of Cambrdge, UK. Hs research nterests nclude computer vson, pattern recognton, and mage processng. He s a member of IEEE, ACM, IEICE, IPSJ, and JSAI. Ikuhsa Mtsugam receved hs B.S. degree n engneerng from Kyoto Unversty n 2001, and M.S. and Ph.D. degrees n engneerng from Nara Insttute of Sence and Technology n 2003 and 2007, respectvely. He then started workng for the Academc Center for Computng and Meda Studes, Kyoto Unversty, and n 2010 became an assstant professor of the Insttute of Sentfc and Industral Research, Osaka Unversty. He s currently an assoate professor of the Graduate School of Informaton Sences, Hroshma Cty Unversty. Hs research nterests are computer vson, human nterfaces, and human behavor analyss. He s a member of IEEE, IEICE, IPSJ, and VRSJ. Yasush Yag s the Executve Vce Presdent of Osaka Unversty. He receved hs Ph.D. degree from Osaka Unversty n In 1985, he joned the Product Development Laboratory, Mtsubsh Electrc Corporaton, where he worked on robotcs and nspecton. He became a research assoate n 1990, a lecturer n 1993, an assoate professor n 1996, and a professor n 2003 at Osaka Unversty. He was also drector of the Insttute of Sentfc and Industral Research, Osaka Unversty from 2012 to Internatonal conferences for whch he has served as Char nclude: FG1998 (Fnanal Char), OMINVIS2003 (Organzng Char), ROBIO2006 (Program Co-char), ACCV2007 (Program Char), PSVIT2009 (Fnanal Char), ICRA2009 (Techncal Vst Char), ACCV2009 (General Char), ACPR2011 (Program Co-char), and ACPR2013 (General Char). He has also served on the IEEE ICRA Conference Edtoral Board ( ). He s an edtoral board member of IJCV and Edtor-n-Chef of IPSJ Transactons on Computer Vson & Applcatons. He was awarded an ACM VRST2003 Honorable Menton Award, IEEE ROBIO2006 Fnalst for T.J. Tan Best Paper n Robotcs, IEEE ICRA2008 Fnalst for Best Vson Paper, MIRU2008 Nagao Award, and PSIVT2010 Best Paper Award. Hs research nterests are computer vson, medcal engneerng, and robotcs. He s a fellow of IPSJ and a member of IEICE, RSJ, and IEEE. Open Access The artcles publshed n ths journal are dstrbuted under the terms of the Creatve Commons Attrbuton 4.0 Internatonal Lcense ( creatvecommons.org/lcenses/by/4.0/), whch permts unrestrcted use, dstrbuton, and reproducton n any medum, provded you gve approprate credt to the orgnal author(s) and the source, provde a lnk to the Creatve Commons lcense, and ndcate f changes were made. Other papers from ths open access journal are avalable free of charge from To submt a manuscrpt, please go to edtoralmanager.com/cvmj.