Decoding positional and color information from a coded pattern Nelson L. Chang, Suk Hwan Lim, and Feng Tang

Transcription

1 Deoding positional and olor information from a oded pattern Nelson L. Chang, Suk Hwan Li and Feng Tang HP Laboratories HPL Keyword(s): single shot oded patter amera alibratio geometri alibratio olor onsisteny, augmented reality Abstrat: The paper presents deoding positional and olor information using visual oded patterns for effiient geometri alibration and olor onsisteny aross multiple ameras. The patterns are generated from an alphabet of basis olors plaed in unique spatial onfigurations alled ChromaCodes. The imaged patterns are automatially deoded to derive the information needed for both geometri and olor alibration. Previous 2-D strutured patterns are typially designed for obtaining only geometry whereas the proposed patterns apture olor information as well. Moreover, the unique deodability of the odes allows them to overome the problem of pattern visibility in all views and enables effetive geometri alibration and olor onsisteny with even partial and/or oluded view of the pattern. Experimental results demonstrate that a single shot of the pattern is suffiient to enapsulate enough information to ompute both geometry and olor aross one or more ameras, espeially important for realtime or interative appliations. External Posting Date: September 27, 2010 [Fulltext] Approved for External Publiation Internal Posting Date: September 27, 2010 [Fulltext] Published and presented at the IEEE Internatioanl Conferene on Image Proessing, September , Hong Kong. Copyright 2010 IEEE International Conferene on Image Proessing

2 DECODING POSITIONAL AND COLOR INFORMATION FROM A CODED PATTERN Nelson L. Chang, Suk Hwan Li and Feng Tang Hewlett-Pakard Laboratories, Palo Alt CA 94304, USA { nelson.hang suk-hwan.lim feng.tang hp.om ABSTRACT The paper presents deoding positional and olor information using visual oded patterns for effiient geometri alibration and olor onsisteny aross multiple ameras. The patterns are generated from an alphabet of basis olors plaed in unique spatial onfigurations alled ChromaCodes. The imaged patterns are automatially deoded to derive the information needed for both geometri and olor alibration. Previous 2-D strutured patterns are typially designed for obtaining only geometry whereas the proposed patterns apture olor information as well. Moreover, the unique deodability of the odes allows them to overome the problem of pattern visibility in all views and enables effetive geometri alibration and olor onsisteny with even partial and/or oluded view of the pattern. Experimental results demonstrate that a single shot of the pattern is suffiient to enapsulate enough information to ompute both geometry and olor aross one or more ameras, espeially important for realtime or interative appliations. Index Terms single shot oded patter amera alibratio geometri alibratio olor onsisteny, augmented reality 1. INTRODUCTION It is ommon to use known referene patterns for many amerabased appliations. For geometri alibration of ameras, suh a pattern enables solving for and properly modeling various amera non-idealities suh as lens distortion as well as the relative position and orientation of the amera with respet to some world origin. Typially, a pattern is omprised of robust, easy to detet features (e.g. the intersetions of a blak/white hekerboard pattern or orners of a multi-box patter that refer to speifi world oordinates and are used to solve for the intrinsi and extrinsi amera parameters [2, 21]. Beause the patterns do not expliitly enode positional information with respet to the patter they tend to be partiularly sensitive to olusions, inomplete visibility (i.e. not ompletely in view), or other violations of spatial oherene. Moreover, with ameras in general position with very wide baselines, it is likely that the referene pattern will not be ompletely visible in all ameras simultaneously. For olor alibratio a referene hart (e.g., the Mabeth hart) helps in the estimation of the olor transformation between the olor spae of the referene to that of the amera(s), often to ensure olor onsisteny aross devies. Currently, it is neessary for someone to manually identify the squares of the hart. While hypothesis testing may be used to loate eah of the squares, issues arise when the hart is partially obstruted or out of view. Als while white balaning ould be performed by measuring pixel values in a blak/white hekerboard patter only a simple olor model (i.e. gain and offset for eah olor) an be estimated. More omplex olor models require more olors in the patterns. Figure 1. Example 12x12 pattern onsisting of unique overlapping 2x2 ChromaCodes made up of four basis olors (red (R), green (G), blue (B), blak (K)). 1-D sequene {R,G,R,B,R,K,G,B,G,K,B,K} (the leftmost olum was used to generate the pattern through suessive olumn shifts. In this paper, we present oded patterns that simultaneously enode the positional and olor information with respet to some referene oordinate system. The patterns are onstruted from uniquely deodable olor odes referred to as ChromaCodes. When imaged by one or more ameras, a single shot of suh pattern an be automatially analyzed to establish the geometri mapping and olor transformation between the amera and referene oordinate systems simultaneously. Moreover, it is robust to inomplete visibility and olusions, thereby reduing reliane on spatial oherene. Calibration using ChromaCoded patterns is very useful when the amera pose and/or illumination hange ontinuously (e.g. in appliations suh as augmented reality), thereby requiring repeated automati alibration. It is espeially important when one wants to measure the olor and/or illumination at preisely the same loation in the sene. 2. CHROMACODED PATTERNS The proposed patterns are formed from a set of C basis olors, onfigured in R x S overlapping odes referred olletively as ChromaCodes. The odes diretly enode both positional and loal olor information with respet to a referene oordinate system. By desig every one is unique and quikly deodable, with eah ode appearing at most one in a single pattern. One of the primary design parameters is the number of basis olors. As the number of C basis olors inreases, so do the number of uniquely deodable odes and hene the effetive resolution of the overall pattern. However, this inrease omes at the ost of dereased disriminability and deoding robustness, espeially for unalibrated ameras. Thus, the number of basis olors governs the ability of the amera to reliably distinguish among the primaries, leading to a tradeoff between auray and maximum pattern resolution. In a similar fashio varying the size of the RxS spatial onfiguration leads to a tradeoff between effetive spatial resolution and robust deoding in the presene of olusions and surfae deformations. At one extreme, the ode of size R=S=1 is

3 Basis Color Classifiation Corner Validation Deoding Optimization amera image Corner Detetion point orrespondenes the least affeted by pattern deformations, but the pattern resolution would be onstrained to be equal to the number of basis olors and lead to many false positives. At the other extreme, large ode dimensions leads to a bigger ditionary of uniquely deodable odes, eah less likely to our naturally in the sene. However, this advantage omes with inreased suseptibility to olusions, surfae deformations, and spatial oherene. As desribed, eah ode in a RxS onfiguration an take on any of C olors, leading to a maximum of C RS possible ChromaCodes. One an view the ChromaCoded patterns more generally as 1- orientable aperiodi C-ary arrays [3], where eah valid ode appears at most one in a given pattern. Other patterns also inlude the general non-binary perfet and semi-perfet maps [11]. Here, a set of C basis symbols is onsidered to reate a wraparound pattern with unique RxS onfigurations, enumerating every, or almost every, possible onfiguration. For odd number of basis olors C, there exists a simple pattern onstrution by taking a 1-D C-ary de Bruijn sequene and repliating shifted versions of the sequene to form a (C R,C S ;R,S) non-binary perfet map [7]. For many pratial appliations, the exat boundaries of the ChromaCodes will not be known a priori, and the imaged pattern annot be assumed to be fronto-parallel and retangular in shape. With patterns suh as perfet maps, every possible ode appears exatly one (inluding wraparound), resulting in onfigurations with adjaent ells of the same basis olor. These onfigurations are more diffiult to loalize and should be avoided. Thus, we prefer to have odes whose adjaent ells differ in olor and thus have as many disernable olor edges as possible. Sine we require only a unique set of odes within the pattern and do not need properties like periodiity or perfetness, we adopt a similar strategy as above to maximize the adjaent ell edges for arbitrary C (not just odd). Instead of using a de Bruijn sequene, we start with a 1-D sequene formed by all C(C-1) pairs of basis olors; this starting sequene is equivalent to a universal yle of 2-permutations from the set of C olors [5]. The initial sequene fills the leftmost olumn (0 th olum of the 2-D patter and eah subsequent olumn is repliated from the previous one, shifted up (and wrapped around if neessary) by the olumn number. The underlying referene oordinate is speified at the enter of eah ode, suh that any adjaent RxS grouping results in a provably unique onfiguration and with at most two adjaent ells having the same basis olor in a given ode. We make the following observations about this onstrution: Observation 1: The onstrution guarantees unique overlapping RxS odes for 2 R C(C-1) and 2 S C(C-1). Observation 2: The onstrution guarantees that at least three of the four interior olor edges in any 2x2 subarray will be disernable. Observation 3: Wraparound odes along the boundaries are not unique. Figure 2. Analysis and deoding workflow Observation 1 omes from notiing that the left two ells of any 2x2 subarray in the leftmost olumn will be mathed with eah pair from the universal yle in suessio thus leading to uniqueness for any 2x2. With eah 2x2 subarray unique, it follows that a larger subarray will also be unique. If we label the inter-ell boundaries as N,E,S,W, Observation 2 omes from the fat that a) there are no repeats in a given olumn (guaranteeing W and E olor edges), and b) no two adjaent olumns are idential (ensuring the N and/or S olor edge). Observation 3 follows from the fat that every neighboring pair of the universal yle is paired with every other one within the interior of the pattern. We thus have a simple pattern onstrution to form a C(C- 1)xC(C-1) pattern (J=I=C(C-1)) out of the total C RS possible odes that maximizes the number of disernable inter-ell olor edges. Smaller patterns are obtained by ropping a subwindow of the full pattern. Figure 1 shows an example 12x12 pattern of overlapping 2x2 odes using four basis olors (red (R), green (G), blue (B), and blak (K)) generated by the 1-D sequene {R,G,R,B,R,K,G,B,G,K,B,K} (i.e. C=4 and R=S=2). The result is a 11x11 retangular array suh that any adjaent 2x2 grouping results in a unique onfiguration and with at most two adjaent ells having the same basis olor in a given ode. The ChromaCoded patterns bears similarity to other patterns derived from de Bruijn sequenes. Muh of the previous work has foused on using these as strutured light patterns to solve the orrespondene problem and reover 3-D shape [1]. One-shot tehniques fall predominantly into two amps: The first amp uses 1-D olored stripes generated by de Bruijn sequenes for dynami 3-D shape reovery; for example, Zhang et al. [20] use a 5-ary de Bruijn sequene of order 3 to enode the 1-D planes from a alibrated set up. 3-D shape is then obtained by solving for rayplane intersetions. The seond amp onstruts 2-D patterns from distint olored dots based on perfet maps or M-arrays [14]. For example, Morano et al. [12] reated pseudo-random olor patterns that maximizes hamming distane using a brute fore tehnique. Claes and Bruyninkx [6] onstrut a 6-ary 3x3 de Bruijn pattern for visual servoing a roboti arm. With a slightly different take, Salvi et al. [13] used equally spaed horizontal and vertial slits using 3- ary de Bruijn sequenes of order 3 for eah orientation to enode geometri information at their intersetions. In ontrast, the proposed ChromaCodes are formed by a grid of adjaent ells, without requiring equal spaing and enabling reasonable ode density in a single pattern for a given spatial resolution. ChromaCodes and their ode design are more resilient to spatial deformations and olusions. In addition to enoding positional informatio the olors of the ells may also be used for estimating loal olor information of the sene. The odes are designed to easily deteted, and the pattern leads to a simple onstrution method for any C, R, S. 3. PATTERN ANALYSIS AND DECODING

4 1) P r (j,i) P r (j+2,i) P r (j+1,i) ˆ ( j 2, 1) P 2) ˆ ( j 1, 1) P ( j 2, 2) ( j 2, 3) referene spae amera spae Figure 3. Candidate edges in referene and amera spaes for the dynami programming framework. By desig the ChromaCoded patterns enapsulate the orrespondene mapping, as well as the appropriate olor transformatio between a amera spae and the referene spae. When aptured by one or more ameras, these patterns may be quikly deoded to robustly and aurately identify the amera spae points and hene loate the ChromaCoded pattern in a single pass. As shown in Figure 2, the entire workflow for the deoding proess for eah amera image onsists of orner detetio basis olor lassifiatio orner validatio and deoding optimization. In order to ensure that the pattern is always deteted, the deoding proess starts off with a simple detetor/lassifier suh that there are many andidate points for the pattern. Eah subsequent step imposes a different prior knowledge about the pattern and uses it to filter out the andidate points Corner detetion The first step is to detet salient orners in the image. A olorbased extension of the Harris orner detetor [19] is used to more robustly loate all relevant orners in the image. Non-maximal suppression and loal lustered filtering are performed to further improve robustness. Only the top 10% of andidate orners are retained. At this stage, it is more important to apture all relevant orners even if there are a large number of false positives Basis olor lassifiation Every amera image pixel is lassified to one or more of the basis olors. To support unalibrated ameras with possibly arbitrary olor shifts or spetral response, we opt for three parallel olor lassifiers where the false negatives for eah lassifier may be minimized. Eah amera pixel is lassified to be at most three olors, one from eah omplementary set {(red (R), green (G), blue (B)), (blak (K), white (W)), (yan (C), magenta (M), yellow(y))}. Dividing the symbols into omplementary sets allows us to have many basis olors while minimizing the false negatives during this lassifiation stage. Determining whih omplementary set the pixel belongs to would be done in a later stage (deoding optimizatio with the use of spatial struture of the ChromaCoded pattern. Additional lassifiers may be added when using more basis olors Corner validation The list of points returned by the orner detetor may onsist of many false positives and thus needs to be pruned. We ombine the results of the previous two stages to validate the orners with respet to the pattern. Sine a valid orner point should lie at the intersetion of four basis olors, we analyze a 3x3 path with respet to the orner point and determine whether the majority of eah quadrant onsists of a single basis olor. Corner points that have four valid quadrants are ompared to the list of approved ChromaCodes for further pruning. This step leads to a list of ˆ 1) P ˆ ( j 1, 1) P Figure 4. Computing likelihood of being the orret olor edge. andidate orner points that orrespond to a possibly valid ode in the referene pattern. These amera spae orner points are then added as the possible andidates for their orresponding ChromaCode oordinate Deoding optimization The final step is to prune the existing orners and identify atual ChromaCodes in the sene. Eah ChromaCode oordinate may be assigned possibly many andidate amera spae orner points. At most only one suh orner point an atually orrespond to the ChromaCode oordinate. As suh, this step needs to be robust to possibly spurious points that happen to deode to a valid ChromaCode as well as handle possible olusions and non-planar deformations of the pattern. We employ a dynami program along eah row to leverage the spatial relationship of the entire pattern and optimize the pruning and identifying proess. The andidate points between adjaent ChromaCode oordinates are examined. An additional andidate plaeholder is added to signify an oluded point. Instead of optimizing point positions diretly, we enfore onstraints with the edge segments to better exploit edge onnetivity along a given row in the pattern. Edge nodes are formed between every ombination of andidate points in adjaent oordinates, inluding the oluded plaeholder. We notate the underlying JxI referene grid oordinates as P r (j,i)=(x ji,y ji ), indexed by j=1 J, i=1 I. Similarly, we define the orresponding points in the amera spae as P (j,i)=(u ji,v ji ) (Figure 3). Als denote E r (j,i) as the edge segment onneting referene points P r (j,i) and P r (j+1,i), i.e. E r (j,i)={p r (j,i),p r (j+1,i)}. Likewise, denote E (j,i)={p (j,i),p (j+1,i)} be the orresponding edge segment in the amera spae. We define E (j, as the edge segment onneting andidate amera points m) and ( j 1,, i.e. E ˆ ˆ P m), P ( j 1,. The segment length and orientation d m) ( j 1, (1) ˆ ˆ 1 P m) P ( j 1, os (2) m) ( j 1, are reorded. Edges with nearly zero lengths (i.e. d(j,<ε) are disarded. Eah edge node is assigned an initial ost C o (j, based on the likelihood of forming an edge that borders the two basis olors; as seen in Figure 4, points are sampled just above and below the edge and ompared to the expeted basis olors. The dynami program finds the minimum ost solution through the andidate edges. The ost C(j, of onneting edge ˆ 2) P ˆ ( j 1, 1) P

5 Figure 5. Camera alibration performane (in terms of RMS reprojetion error) of different ChromaCoded patterns as a funtion of pattern visibility (perentage). segment E (j, and E (j+1, is given by C( j, Co( j, Cdistd( j, (3) Cangle Col where d d( j, d( j 1, is the edge length differene, C dist d penalizes for large differenes in edge lengths (the pattern edges are assumed to be roughly similar), ( j 1, is the edge orientation differene, C angle θ penalizes for large differenes in orientation (the edges along a given row are assumed to be loally linear), and C ol penalizes if seleting an oluded plaeholder (is zero otherwise). To ensure edge onnetivity, only n=o ases are onsidered. Suppose g ji represents the set of andidate edge segments for E (j,. A given solution path G through the data inurs a ost T ( G) C( x, ). Thus, the dynami program x g xi finds the path G that minimizes T(G). Additional proessing may be used to prune invalid andidate edges and disard outliers. In the end, we obtain the point orrespondenes between oordinates P r (j,i) in the ChromaCoded referene grid and their ounterparts P (j,i) in the amera image(s). 4. EXPERIMENTAL RESULTS In this setio we desribe various experiments using up to four heterogeneous ameras (labeled A to D) to demonstrate the effetiveness of the proposed ChromaCoded patterns. Without loss of generality, oded patterns with R=S=2 and varying number of basis olors are onsidered. In a first series of experiments, ChromaCodes may be used to automatially identify robust features for amera alibration. Sine the fous of the approah is on the automati detetion and deoding of the pattern features and not on the atual alibration algorith the experiments assume a Pattern\Camera A B D RGBK RGBKW RGBKCMY RGBKWCMY Table 1. RMS error (pixel) of the feature point olors before and after applying the estimated olor transformation to amera C. fixed alibration methodology (in this ase, Bouguet s MATLAB Camera Calibration toolbox [2]). Reprojetion error is used as a relative measure of the auray for the set of feature points. We ompare the results for different 9x9 (ropped) patterns, wherein eah ell of the pattern measures 22mm x 22mm and the printed pattern is matted onto a flat surfae. In eah ase, the four heterogeneous ameras apture a total of six different views of eah pattern. The feature points are automatially omputed and serve as diret input for amera alibration. The features are then refined to subpixel auray based on initial results and the reprojetion error is finally omputed. Regardless of the number of frames used, the error is roughly around 0.25 pixels, suggesting good alibration auray using any of the patterns. We find these results to be omparable in auray to manually seleting features on a similarly sized hekerboard pattern [17]. The error does not derease with additional frames beause we believe it falls within the auray of the subpixel interpolation. A seond series of experiments validate that ChromaCodes perform well in the presene of olusions. To examine the performane as a funtion of visibility, an LCD monitor is used to display eah of the four patterns, with a portion of the pattern obsured and the remaining points used for amera alibration. As seen in Figure 5, the overall reprojetion error is somewhat higher but suggests good alibration performane even with lose to 50% of the points obstruted. In addition to geometri alibratio ChromaCodes an simultaneously improve the olor onsisteny aross multiple ameras. The proposed odes diretly enapsulate the olor information of the sene. Feature points are automatially deoded and used to solve for the intrinsis of eah amera. We model the inter-amera olor transformation as a 3x4 affine matrix, and solve using linear least squares with the enter olors of eah ell after gamma orretion. Camera C is seleted as the target olor spae and the other ameras are transformed to it. Table 1 illustrates the RMS omponent olor pixel error before and after applying the estimated olor transformation. Note the signifiant improvement in using the patterns for olor onsisteny aross all the frames. Figure 6 illustrates some of the alibration and olor results. Two representative frames from ameras C and D are shown in Figures 6(a) and (b); note the wide baseline and severe olor (a) (b) () (d) Figure 6. Representative results for amera alibration and olor onsisteny: Original images from (a) amera C and (b) amera D; () undistorted image after alibration; (d) resulting image after olor orretion.

6 amera image referene pattern H -1 Warp Compute Color C warped entroid sampling Diff & Filter adjusted olusion mask Figure 7. Framework for omputing geometry and olor transformations. differenes. Figure 6() shows the result of using the omputed geometri parameters to undistort the image. Here, the barrel distortion from amera D has been automatially orreted. Figure 6(d) shows the olor orreted result. Note how onsistent the result is ompared with the referene amera shown in Figure 6(a). The proposed ChromaCoded patterns an also be used for automati ontent insertio espeially important for real-time augmented reality appliations. A oded pattern is used to define the region to be replaed, instead of segmenting objets through Chroma Keying [15] or omputational matting [10, 16]. For eah amera, the desired ontent undergoes a geometri and olor transformation to bring the ontent to the amera s spae (see Figure 7). By desig the pattern and its deoding are not restrited to non-planar surfaes. However, we assume a planar pattern to simplify real-time warping and solve for the homography using linear least squares with outlier rejetion using the pairwise orrespondenes. Eah measured amera olor is related to its expeted referene olor by a 3x4 affine olor transformation matrix, omputed using linear least squares with the olor of loal neighborhoods around the entroids of eah warped ell. Points that deviate signifiantly from the predited olor are masked off and are onsidered to be oluders, thereby reating an olusion mask as a byprodut. The proposed approah urrently runs unoptimized at about 5-10 fps, thus approahing real-time video rates. Figure 8 shows an example for two ameras apturing a moving pattern. The top row shows the aptured images. The next row demonstrates the results of texture mapping an input image using just the omputed geometri mapping. The following row demonstrates inorporating the omputed olor transformation. Notie how well the two Figure 8. Example of automati ontent insertion: (a) original amera images; (b) applying only geometri warps; () orreting for olor; (d) fatoring in the olusion mask. inserted images blend with their respetive amera s olor spae. The final row inorporates the omputed olusion mask to give a reasonable approximation of preserving depth ordering. Figures 9 and 10 highlight the lear benefits of using the proposed ChromaCoded patterns. As shown in the top row of Figure 9, muh of the pattern is oblique to the amera and offsreen in both ases. The unique pattern enoding allows robust reovery of the neessary geometry and olor transformations to properly insert the ontent (bottom row of Figure 9). Similarly, Figure 10 demonstrates the approah s automati responsiveness to global brightness hanges. Unlike traking frame orners [9] or onventional patterns for augmented reality [8], ChromaCodes does not require full visibility or planarity. In ontrast to sene-dependent natural images [18], it is more resilient to olusions and better suited for simultaneous olor reovery with its uniformly distributed odes. Figure 11 shows additional frames from a video sequene using ChromaCoded patterns for seamless ontent insertion. 5. CONCLUSIONS We have presented oded patterns designed to effiiently enode both geometri and olor information. The result is the ability to use a single pattern to quikly solve geometri alibration as well as olor onsisteny aross multiple ameras. The advantage of

7 Figure 9. Automatially inserted ontent in the presene of oblique angles and partial pattern visibility. using ChromaCodes omes from its simple onstrutio the addition of olor informatio as well as improved resiliene to partial visibility and olusions. Co-loated positional and olor information helps for interative appliations like augmented reality when the amera pose and/or illumination frequently hange. We are urrently investigating the use of ChromaCoded patterns for these and other appliations. REFERENCES [1] J. Batlle, E. Mouaddib, J. Salvi, Reent Progress in Coded Strutured Light as a Tehnique to Solve the Correspondene Problem: A Survey, Pattern Reognitio vol. 31, no. 7, pp , [2] J.-Y. Bouguet, Camera Calibration Toolbox for MATLAB, June [3] J. Burns and C. J. Mithell, Coding Shemes for Two-Dimensional Position Sensing, HPL-92-19, January [4] Y. H. Chen et al., A Vision-Based Augmented-Reality System for Multiuser Collaborative Environments, TMM, vol. 10, no. 4, pp , [5] F. Chung, P. Diaonis, R. Graha Universal Cyles for Combinatorial Strutures, Disrete Math., 110: 43 59, [6] K. Claes and H. Bruyninkx, Robot Positioning using Strutured Light Patterns suitable for Self Calibration and 3D Traking, International Conferene on Advaned Robotis, Jeju, Korea, [7] T. Etzio Construtions for Perfet Maps and Pseudo-random Arrays, IEEE Transations on Information Theory, 34(5): , [8] H. Kato and M. Billinghurst, Marker Traking and HMD Calibration for a Video-based Augmented Reality Conferening Syste 2nd Int'l Workshop on Augmented Reality, , San Franis CA, Ot [9] M. C. Leung et al., A Projetor-based Hand-held Display Syste Pro. IEEE CVPR, June [10] A. Levi A. Rav-Aha, D. Lishinski. Spetral Matting. IEEE Transations on Pattern Analysis and Mahine Intelligene, 30(10): , [11] F. J. MaWilliams and N. J. A. Sloane, Pseudo-random Sequenes and Arrays, Proeedings of the IEEE, 64: , [12] R. A. Morano et al., Strutured Light using Pseudorandom Codes, PAMI, vol. 20, no. 3, pp , [13] J. Salvi et al., A Robust-Coded Pattern Projetion for Dynami 3D Sene Measurement, Pattern Reognition Letters, vol. 19, pp , [14] J. Salvi et al., Pattern Codifiation Strategies in Strutured Light Systems, Pattern Reognitio vol, 37, no. 4, pp , [15] A. R. Smith and J. F. Blin Blue Sreen Matting, ACM SIGGRAPH, pp , Figure 10. Automatially inserted ontent responding to global brightness hanges. [16] J. Su J. Jia, C.-K. Tang, H.-Y. Shu Poisson Matting, ACM SIGGRAPH, pp , [17] R. Y. Tsai, A Versatile Camera Calibration Tehnique for High Auray 3D Mahine Vision Metrology using Off-the-shelf TV Cameras and Lenses, IEEE Journal of Robotis and Automatio vol. RA-3, no. 4, pp , [18] D. Wagner et al., Pose Traking from Natural Features on Mobile Phones, Pro. ISMAR, [19] J. van de Weijer et al., Edge and Color Detetion by Photometri Quasi-Invariants, IEEE PAMI, vol. 27, no. 4, pp , [20] L. Zhang et al., Rapid Shape Aquisition using Color Strutured Light and Multi-pass Dynami Programming, International Symposium on 3D Data Proessing Visualization and Transmissio pp , [21] Z. Zhang, A Flexible New Tehnique for Camera Calibratio IEEE PAMI, vol. 22, no. 11, pp , 2000.

8 Figure 11. Representative frames of a video sequene with automatially inserted ontent using ChromaCoded patterns.