Combining complementary edge, point and color cues in model-based tracking for highly dynamic scenes

Transcription

1 Combnng complementary edge, pont and color cues n model-based trackng for hghly dynamc scenes Antone Pett, Erc Marchand, Keyvan Kanan Abstract Ths paper focuses on the ssue of estmatng the complete 3D pose of the camera wth respect to a complex object, n a potentally hghly dynamc scene, through modelbased trackng. We propose to robustly combne complementary geometrcal edge and pont features wth color based features n the mnmzaton process. A Kalman flterng and pose predcton process s also suggested to handle potental large nterframe motons. In order to deal wth complex 3D models, our method takes advantage of hardware acceleraton. Promsng results, outperformng classcal state-of-art approaches, have been obtaned on varous real and synthetc mage sequences, wth a focus on space robotcs applcatons. I. INTRODUCTION Determnng the complete 3D pose of the camera wth respect to the object s a key requrement n many robotc applcatons nvolvng 3D objects, for nstance n the case of autonomous, vson-based and uncooperatve space rendezvous wth space targets or debrs [13]. Based on the knowledge of the 3D model of the target, common approaches address ths problem by usng ether texture [1], edge features [6, 5, 13] or color or ntensty features [11, 17]. In order to cope wth advantages and drawbacks of these dfferent types of cues, some researches have focused on combnng them. Dfferent ways of handlng the ntegraton have been consdered. Some studes propose a sequental ntegraton of edge and texture cues [1, 3], where the computed domnant moton [1] provdes a predcton of the projected edges n the mage, mprovng the ntalzaton of an edge-based mnmzaton process. [3] uses optcal flow of pxels lyng on the projected object to ntalze a maxmzaton process of the separaton between object and background along the object contours usng color or lumnance probabltes. Other methods smultaneously merge features of dfferent types wthn probablstc or flterng technques such as Kalman flters w.r.t. the pose parameters [8, 7], by ntegratng nterest ponts matchng n an Extended Kalman Flter for [8], and ntegratng optcal flow estmaton for [7]. [18] elaborated a smlar hybrd soluton wthn a probablstc Expectaton Maxmzaton (EM) framework whch ams at optmzng the posteror probabltes of both edge and FAST pont features. Works of [21, 15, 16, 12] rely on a determnstc teratve mnmzaton of a global error functon combnng the dfferent features. [21] proposes for nstance to ntegrate n a classcal edge-based error functon [6, 5] geometrcal A. Pett s wth INRIA Rennes - Bretagne Atlantque, Lagadc Team, France, Antone.Gullaume.Pett@nra.fr. E. Marchand s wth Unversté de Rennes 1, IRISA, Lagadc Team, France, marchand@rsa.fr. K. Kanan s wth Astrum, Toulouse, France dstances between keypont features detected and matched n two consecutve frames and the projecton of ther 3D postons. [16] nstead reles on optcal flow estmaton of some regularly spread pxels lyng on planar patches on the object, provdng pont correspondences between successve mages and thus pont to pont dstances to be mnmzed w.r.t. the camera dsplacement through a homography. In contrast to [21, 16], [15] proposes a texture-based error functon whch s drectly based on the dfference between ntenstes of some pxels selected on reference planar patches of the object. [12, 14] combne n the objectve functon to be optmzed, w.r.t. the camera pose, a classcal geometrcal nformaton provded by the dstances between model and mage edges wth color nformaton through object/background color separaton along the model edges. Followng deas suggested by [12, 14], and by [21, 2], whch propose hybrd methods ncorporatng edge and keypont features, we propose to ntegrate wthn a determnstc pose estmaton process the dfferent consdered vsual features, n order to take advantage of ther complementarty and to overcome the lmtatons of classcal edge-based approaches. A low-level multple hypotheses edge matchng process s also embedded n our framework. Lke n our prevous works [14], the model projecton and edge generaton phase reles on the graphcs process unts (GPU) n order to handle complex 3D models, and to be reasonably tme-consumng. Snce the applcaton context of ths work manly deals wth space robotcs applcatons (space rendezvous and proxmty operatons) for whch the targeted object generally nvolves constant velocty motons, a Kalman flterng framework on the pose parameters s desgned, enablng a predcton step for the pose, provdng a fner ntalzaton, ntended to cope wth potental large nter-frame motons and to avod local mnma. II. COMBINING KEYPOINT, EDGE AND COLOR FEATURES IN A MODEL-BASED TRACKING FRAMEWORK Our problem s restrcted to model-based trackng, usng a 3D model of the target. The goal s to estmate the camera pose r by mnmzng, wth respect to r, the functon accountng for errors e (r) between a set of vsual features extracted from the mage and the forward projecton of ther 3D homologues n the mage plane: (r) = ρ(e (r)) (1) where ρ s a robust estmator, whch reduces the senstvty to outlers. Ths s a non-lnear mnmzaton problem wth

2 respect to the pose parameters r, and we follow a Gauss- Newton mnmzaton framework to handle t, by teratvely updatng the pose r: r k+1 = r k δr (2) δr = λ(dj) + De (3) where (DJ) + s the pseudo nverse of DJ, wth J the Jacoban matrx of the error vector e(r). λ s a proportonal gan and D s a weghtng matrx assocated to the Tukey robust estmator. Wth the homogeneous matrx M k+1, the new pose r k+1 can be updated usng the exponental map [9]: M k+1 = exp([δr]) M k (4) Our challenge s to ntegrate, wthn ths framework, geometrcal edge-based features wth color features along slhouette edges and geometrcal features based on Harrs corner ponts extracted n the mage, tracked through the KLT algorthm [19]. The objectve s to fuse n the crteron to be optmzed a geometrcal nformaton provded by dstances between edges, wth dstances between keyponts and wth a denser color nformaton through object/background color separaton along the slhouette model edges. The goal s to beneft from the complementarty of these features and to overcome the lmtatons of classcal sngle cue approaches. Geometrcal features based on edges or keyponts such as Harrs corners rely on lne-to-pont or pont-to-pont correspondences and on geometrcal dstances accountng for these correspondences. On one hand, edges have the advantage of beng robust to llumnaton condtons but suffer from havng smlar appearances, resultng n ambgutes between dfferent edges and potental local mnma. On the other hand, pont features can be descrbed more specfcally, mposng a better spato-temporal constrant. Though begn locally performed, the KLT algorthm allows a larger convergence radus. However, these keyponts are more senstve to llumnaton condtons, for both extracton and trackng steps. Besdes, pose errors resultng from the trackng phase at the prevous frame are ntegrated n the back-projecton process of the keyponts, leadng to potental drft problems across the sequence. Whereas for edges, the edge matchng process n the mage nstead rely on the absolute reference of the projecton of the 3D model. Wth color or ntensty edge-based feature, the dea s to avod any mage extracton or segmentaton that could lead to outlers and msmatches. By processng a dense nformaton along the slhouette of the projected 3D model by modelng the color (or lumnance) appearance on both sdes of the edges, usng smple statstcs, and optmzng ther separaton, a better accuracy can be acheved. A man advantage s a better robustness to mage or moton blur, background clutter or nose. However, among drawbacks these features need color contrast to perform effcently and are lmted by ther computatonal costs. By combnng these features, can be rewrtten as: = w g g + w c c + w p p (5) refers to the geometrcal edge-based error functon, c stands for the color-based one and p corresponds to the keypont features. w g, w c and w p are the respectve weghtng parameters. Let us note that the nvolved vsual features rely on the projecton of the 3D model. g A. Geometrcal edge features III. VISUAL FEATURES 1) Model projecton and generaton of model edge ponts: As n our prevous works [13, 14], we propose to automatcally manage the projecton of the model and to determne the vsble and promnent edges from the rendered scene, by consderng the drect use of a complete model, whch can be textured or not. By usng the graphcs process unts (GPU) and a 3D renderng engne (OpenSceneGraph here), we avod any manual pre-processng. For each acqured mage I k+1, the model s rendered wth respect to the prevous pose r k. Processng the depth map, we can obtan a set of N g 2D control ponts {x } Ng =1 whch belong to target rms, edges and vsble textures from the rendered scene. The correspondng 3D ponts X can then easly be reconstructed. 2) Error computaton and Jacoban matrx: the edgebased functon g s computed n a smlar way to [14]. From the model edge ponts we perform a 1D search along the normal of the underlyng edge of each x (r k ). A common approach s to choose on the scan lne the pxel wth the maxmum gradent as the matchng edge pont x n the new mage. The consdered approach s based on the ECM algorthm [2]. Once correspondences between the set of control ponts {x } Ng =1 and the set of mage edge ponts {x }Ng =1 are establshed, our approach consders the dstance between the projected 3D lne l (r) underlyng the projected control pont x (r) (projected from the 3D pont X ) and the selected matchng pont x n the mage. The error functon g (r) to be mnmzed wth respect to the pose r can be wrtten as: 1 N g g (r) = ρ g (e g (r)) (6) where e g (r) = σ 1 g d (l (r), x )) and d (l (r), x )) s the dstance between the pont x and the correspondng lne l (r). ρ g s a Tukey robust estmator and σ g s a normalzaton factor, the estmator of the standard devaton of the error e g. σ g 2 = 1 N g ρg (d (l (r f ), x )) wth r f the pose computed at the end of the prevous mnmzaton process. The crteron g mproves other approaches [22, 12, 4] whch consder the dstance between x (r) and x along the 2D normal vector n to the edge underlyng x (r), but neglectng the dependence of n w.r.t. to the pose r, what s takng nto n our computaton of the Jacoban matrx of e g. A key requrement s to compute the 3D equaton n the scene or object frame of the lne L such that l (r) = pr(l, r). Ths s necessary n order to compute the error e g and the correspondng Jacoban matrx J e g and the complete computaton of J e g. For more detals, see [14, 5].

3 As n [13], multple hypotheses can be consdered n the process. These hypotheses correspond to potental edges, whch are dfferent local extrema x,j of the gradent along the scan lne. And g becomes: g = 1 ρ g (σg 1 mn d (l (r), x N g j,j)) (7) where ponts x,j are the selected canddates for each 2 control pont x. For σ g, we use the estmate: σ g = 1 N g ρg (mn j d (l (r), x,j )). B. Color features The color-based functon c s elaborated to characterze the separaton between both sdes of projected model edges belongng to the slhouette, by relyng on color nformaton. c s computed the same way as n [14], and the prncple steps are brefly recalled hereafter. The goal s to compute local color statstcs (means and covarances) along the normal to the projected slhouette model edges, on both sdes. Then for each pxel along the normal, we determne a resdual representng the consstency of the pxel wth these statstcs, accordng to a fuzzy membershp rule to each sde. More formally, gven the set of N s projected slhouette model edge ponts {x (r)} Ns =1, we compute color statstcs, so to say RGB means I O and I B and covarances R O and R B, on both sde of the edge (object O and background B) usng 2D + 1 pxels along the edge normal n, up to a dstance L. These statstcs are then mxed accordng to a fuzzy membershp rule, gvng means Î,j(r) and covarances ˆR,j (r) for the pxels y,j on the normal n, wether they are on the object or background sde. Î,j(r) can be seen as a desred color value for y,j. An error e c,j (r) can then be defned as: e c,j(r) = (Î,j(r) I(y,j )) T 1 ˆR,j (Î,j(r) I(y,j ))) (8) Usng a M-estmator (Tukey) to cope wth outlers, c can be wrtten as: c = 1 ρ c ( e c N,j(r)) (9) c j wth N c = 2DN s accountng for the number of color features. For more accuracy and temporal smoothness, we propose to ntroduce temporal consstency, by ntegratng the color statstcs computed on the prevous frame P I for the slhouette edge ponts x (r k ) at the frst teraton of the mnmzaton process. Wth α a weghtng factor ( < α < 1), t gves e c,j (r) = αî,j(r) + (1 α)( P Î,j (r)) I(y,j ). For the computaton of the correspondng Jacoban matrx J e c,j, see [14]. C. Geometrcal keypont based features Another class of vsual features whch can be used are keyponts tracked across the mage sequence. As prevously suggested by [3] or by [21, 16] wthn there hybrd approaches, the dea s to desgn a texture-based objectve functon p accountng for geometrcal dstances between keyponts extracted and tracked over successve mages. But n contrast to [16], whch process 2D-2D pont correspondences to estmate the 2D transformaton from I k to I k+1 of planar local regons underlyng the ponts, we use 2D-3D correspondences to drectly mnmze p w.r.t. the pose r. More specfcally, let us denote {x } Np =1 a set of detected nterests ponts n frame I k. Assumng the pose r k has been properly estmated, we can restrct these ponts to be lyng on the projected 3D model wth respect to r k. Snce we rely on a complete 3D model, the depth of the ponts n the scene can be accurately retreved, and usng r k, we can backproject these ponts on the 3D model, gvng a set {X } Np =1 of 3D ponts of the 3D model. Ths s a major dfference wth respect to [21] whch ams at smultaneously optmzng the camera poses and projectons of the matched ponts n two successve frames, relaxng the assumpton of havng accurate prevous pose estmate and 3D model, but ncreasng computatons. Our knowledge of a complete 3D model, along wth the use of convenent renderng technques allows us the keep ths assumpton vald. As keyponts, we employ the Harrs corners detector nsde the slhouette of the projected model n the mage to extract the set {x } Np =1 n I k. Then, the KLT trackng algorthm enables to track ths set of ponts n frame I k+1, resultng n a correspondng set {x }Np =1. From the correspondences between {X } Np =1 and {x }Np =1, p can be computed as follows: p (r) = 1 N p ρ p (e p N ) (1) p wth e p = σ 1 p (x (r) x ) and x (r) = pr(x, r)). ρ p s the Tukey robust estmator assocated to these errors. σ p accounts for the standard devaton of errors e p. Smlarly to σ g, we use the estmate σ 2 p = 1 N p ρp (x (r f ) x ). The Jacoban matrx J e p s computed as follows: J e p = σ p 1 x (11) r = K σ p [ 1/Z x/z xy (1 + x 2 ) y 1/Z y/z (1 + y 2 ) xy x wth (x, y) the meter coordnates of the mage pont x (r) = pr(x, r)), and Z the depth of the correspondng 3D pont. K s the calbraton matrx of the camera. D. Combnaton of the vsual features The combnaton of the three types of features and ther respectve errors e g (r), ec,j (r) and ep (r) n the mnmzaton framework s acheved by stackng the error vectors e g, ec,j and e p nto a global error vector e and ther correspondng Jacoban matrces J e g, J e c and J e p nto a global Jacoban matrx J: e = [ λ g e gt λc e ct λp e ] pt T (12) J = [ λ g J e g T λc J e c T λp J e p T ] T ] (13) where λ g = w g /N g, λ c = w c /N c and λ p = w p /N p, e s a N g + N c + N p vector and J s a (N g + N c + N p ) 6

4 matrx. Regardng the weghtng matrx D, t s wrtten as D = blockdag(d g, D c, D p ), where D g, D c and D p are the respectve weghtng matrces assocated to the robust estmators ρ g, ρ c and ρ p. A. Pose uncertanty IV. FILTERING AND POSE PREDICTION An mportant tool to set up s the measurement of the qualty and relablty of the trackng process, based on the errors provded by the dfferent cues ntegrated n the objectve functon. For ths purpose, we can compute the covarance matrx Σ δr on the parameters of the pose error δr, whch results from the errors e, based on equaton (4). We can ndeed assume that the pose error δr follows a Gaussan dstrbuton δr N (, Σ δr ). We also assume that error e follows a Gaussan dstrbuton e N (, Σ e ), wth Σ e = blockdag(λ g I Ng N g, λ c I Nc N c, λ p I Np N p ), snce the errors are normalzed. From equaton (3) gvng the value of δr, Σ δr can be wrtten as: [ ] Σ δr = E δrδr T = E [(DJ) ] + De e T D T ((DJ) + ) T Snce the uncertanty les on e we have: Σ δr = (DJ) + DΣ e D T ((DJ) + ) T (14) We can actually determne a covarance matrx for the whole set of the vsual features or one for each type, gvng three covarance matrces, Σ g δr, Σc δr, and Σp δr. B. Kalman flterng and pose predcton Kalman flterng: n order to smooth pose estmates, we propose to ncorporate a flterng process, relyng on the Kalman flterng theory. To acheve ths, we employ an lnear Kalman flter on the parameters of the camera velocty v, whch s ntegrated to determne the pose so that: M k+1 = exp([v k+1 δt]) M k, at each tme step k. We assume a constant velocty dynamc model. Ths model s partcularly sutable n our context of space rendezvous snce constant velocty motons are generally nvolved. As the state of the system ( x k, Σ k ) we smply choose the velocty parameters, so that x k = v k s the actual system state at tme step k, wth x k = v k the a posteror estmate of x k, and Σ k the correspondng covarance matrx: x k = x k + η k and Mk = exp([η k δt])m k (15) wth η k N (, Σ k ), and M k the posteror estmate of the actual pose M k. Wth a constant velocty model, the true state at tme step k+1 s evolved from the state at k accordng to: x k+1 = x k + n k+1 (16) wth n k N (, Q k ) the state nose, and Q k = dagblock(σv, Σω) the state nose covarance matrx, wth Σv = σvi and Σω = σωi 2 3 3, σv and σω beng state nose standard devatons respectvely on the translaton and rotaton parameters of v. As an observaton z k+1, the mnmzaton process descrbed n secton II provdes us wth a pose measure M m k+1, and wth a measure of the velocty vk+1 m : vk+1 m = exp 1 ( M k M m 1 k+1 ) (17) z k+1 = vk+1 m = x k+1 + w k+1 (18) wth w k N (, R k ), wth R k the observaton nose covarance matrx. The nose w k can actually be nterpreted as equal to δr k, whch s the pose error at the end of the pose estmaton process, so that R k = Σ δr (equaton 14). The predcton step can then be acheved, gvng pror estmate of the future state and the pose: x k+1 k = x k, Σ k+1 k = Σ k + Q k+1 (19) and M k+1 k = exp( [ x k+1 k ] ) Mk (2) The update step s classcally performed, resultng n the Kalman gan K k+1 and n posteror state and pose estmates x k+1, Σ k+1 and M k+1 : K k+1 = Σ k+1 k (Σ k+1 k + R k ) 1 (21) x k+1 = x k+1 k + K k+1 (z k+1 x k+1 k ) (22) Σ k+1 = (I K k+1 )Σ k+1 k (23) M k+1 = exp( [ K k+1 (z k+1 x k+1 k ) ] ) M k+1 k (24) Pose predcton: from the estmate pose M k+1, we suggest to use t to provde a predcted pose M p k+1 whch s ntended to ntalze the pose estmaton phase for the next tme step. A natural dea s to choose the pror estmate at pred k + 1 so that M k+1 = exp([ x k+1]) M k+1. Ths predcton step s partcularly useful when large nterframe motons are observed n the mage, snce the model pred projecton wth respect to M k+1 can brng the error functon closer to ts actual mnmum, avodng local ones. V. EXPERIMENTAL RESULTS We valdate the proposed trackng method, both qualtatvely on real mages and quanttatvely on synthetc mages, and ts benefts are verfed. Let us frst classfy the dfferent contrbutons proposed n ths paper: Effcent model projecton and model edge control pont generaton, and pose estmaton based on geometrcal edge features: C. Integraton of a multple-hypothess framework n the edge regstraton process: C1. Integraton of the color-based vsual features: C2. Temporal consstency for the color-based objectve functon: C3. Integraton of the geometrcal keypont features: C4. Kalman flterng and pose predcton step: C5. The soluton combnng (C, C1, C2, C3) s actually very close to the one proposed n [14] except the ncorporaton of a multple-hypotheses scheme takng advantage of lne clusterng. We refer t as N M (for Nomnal Mode). We propose to evaluate the man contrbutons of ths paper ( C4 and C5) and to compare them wth () whch has proven n [14] to have superor performances wth respect to [5, 12, 13].

5 In terms of mplementaton, a standard laptop (2.8GHz Intel Core 7 CPU) has used and for all the followng tests. A. Results on synthetc mages We consder the synthetc sequence, proposed n [14], featurng a Spot satellte and whch s provded wth groundtruth on the pose. The results of the trackng of the target can be seen on Fgures 1 and 2 where the errors on the pose parameters are plotted (translaton and rotaton, represented by Euler angles). For our new solutons (N M, C4) and (, C5), and for our prevous one (), the trackng s properly performed, as depcted on the mage sequence on Fgure 1 for (N M, C4, C5). In terms of pose errors, though the mprovement s qute slght on ths sequence, the ncorporaton of keyponts enables to avod some peaks on the pose parameters errors whch are encountered by (), especally when the panels of satellte tends to flp n the mage (around frames 75, see second mage n Fgure 1) or when the target tends to get far, wth low lumnosty (frames 9-135). Due to the temporal constrant mposed by the trackng of the keyponts, these ponts can prevent from some jtter effect on the pose. By employng the Kalman flter and the pose predcton step, the pose get smoothed. The advantage of the predcton step can be enhanced by temporally down-samplng ths sequence, snce trackng can be correctly handled wth a down-samplng factor f up to f = 7 (see provded vdeo), and enables to deal wth very large nter-frame motons (up to 2 pxels). Error (m) Error (m) Error (m) ,C ,C5 Translaton / z Translaton / y Translaton / x -.5,C Error (rad.) Error (rad.) Error (rad.) ,C ,C5 Rotaton / z (Roll) Rotaton / y (Yaw) Rotaton / x (Ptch) ,C Fg. 2: Errors on the estmated camera pose over the whole sequence, (), (, C4) and (, C4, C5) wth f = 1. (a) frame 4 (b) frame 1 (c) frame 38 Fg. 1: Trackng for the Spot sequence wth the proposed method (, C4, C5) for frames 2, 75 and 144. Tracked Harrs corners wth the KLT algorthm are represented n wth blue dots. Red dots are ther reprojecton at the end of the pose estmaton process. B. Results on real mages A frst sequence shows the Soyuz TMA-3M undockng from the Internatonal Space Staton (ISS) We have also run the Nomnal Mode N M, and the proposed soluton (N M, C4). Dfferent temporal down-samplng factors have been tred out. Wth f = 1, we observe that (, C4) s able to track the target on a longer sequence than () (Fgure 3(e)). Wth f = 5, the advantage of ntroducng keyponts s obvous snce trackng for (, C4) s not affected by the down-samplng (Fgures 3(a)- 3(d)), n contrast to (N M) whch rapdly fals (Fgures 3(f)). The ablty of keyponts of beng properly tracked wth large nter-frame motons (up to 25 pxels) s here stressed out. The uncertanty of the pose for both methods s represented on Fgure 4 by the global covarance matrx Σ δr, showng, for f = 5, larger uncertanty when both solutons tends to fal (around frame 2 for and around frame 42 for (, C4)). (d) frame 42 (e) frame 167 (f) frame 4 Fg. 3: Trackng for the Soyuz sequence usng (, C4) wth f = 5 (a-d), usng wth f = 1 (e) and f = 5 (f). The second sequence features a fgurne of Maro, whch s 35cm hgh, and made of curved and complex shapes. The 3D model was obtaned usng a Knect sensor and the ReconstructMe software. Despte rough modelzaton of some parts, ths model, whch s made of 15 vertces, for a 5.5MB sze, has been drectly used n our trackng algorthm, showng the convenence of the proposed method and the effcency of the model projecton system. Wth (), trackng gets quckly lost (Fgure 5(f)) due to the large nter-frame moton, where (N M, C4) acheves t successfully throughout the sequence, even n the case of very large motons n the mage (up to 4 pxels) and mportant moton blur (see Fgure 5(a) and provded vdeo).

6 Standard devaton (m) Translaton v Standard devaton (rad.) Rotaton ω Standard devaton (m) Translaton v Edge covarance Color covarance Pont covarance Global covarance Standard devaton (rad.) Rotaton ω Edge covarance Color covarance Pont covarance Global covarance Fg. 4: Covarances on the pose dsplacement for () and (, C4). Square root of traces on translaton and rotaton parameters of Σ δr are represented. The uncertanty of the pose for both methods s also represented (Fgure 6), for the dfferent vsual features, wth Σ g δr, Σc δr, and Σp δr, and for the whole set wth Σ δr. We can observe that the effectve fal of the trackng for around frame 5 does not have much effect on the global covarance, however, the covarances generated by the keyponts and the geometrcal edge features take much larger values. Let us fnally note that the proposed algorthm (, C4, C5) runs at around 8 fps n the presented cases. (a) frame 14 (b) frame 25 (c) frame 25 (d) frame 26 (e) frame 495 (f) frame 15 () Fg. 5: Trackng for the Maro sequence wth (, C4) (a-e) and (f). VI. CONCLUSION In ths paper we have presented a robust and hybrd approach of 3D vsual model-based object trackng.our approach has been tested va varous experments, on both synthetc and real mages, n whch the ntegraton of keyponts has proven to be partcularly effcent n the case of hghly dynamc scenes wth large nter-frame motons. Our method has also shown to be promsng n terms of robustness wth respect to llumnaton condtons, moton blur and nose, outperformng state-of-the-art approaches, for nstance n the case of space rendezvous. REFERENCES [1] G. Bleser, Y. Pastarmov, and D. Strcker. Real-tme 3d camera trackng for ndustral augmented realty applcatons. Journal of WSCG, pp , 25. [2] P. Bouthemy. A maxmum lkelhood framework for determnng movng edges. IEEE T. on PAMI, 11(5): , May [3] T. Brox et al. Hgh accuracy optcal flow serves 3-D pose trackng: explotng contour and flow based constrants. In ECCV 6, pp , Graz, May 26. [4] C. Cho and H. I. Chrstensen. Robust 3d vsual trackng usng partcle flterng on the specal eucldean group: A combned approach of keypont and edge features. IJRR, 31(4): , Aprl 212. Standard devaton (m) Translaton v Edge covarance Color covarance Pont covarance Global covarance Standard devaton (rad.) Rotaton ω Edge covarance Color covarance Pont covarance Global covarance Fg. 6: Covarances on the pose dsplacement for ()(top) and (, C4)(bottom). Sqare root of traces on translaton and rotaton parameters of Σ g δr (Edge), Σc δr (Color), Σ p δr (Pont) and Σ δr (Global) are represented. [5] A.I. Comport, E. Marchand, M. Pressgout, and F. Chaumette. Realtme markerless trackng for augmented realty: the vrtual vsual servong framework. IEEE T-VCG, 12(4): , July 26. [6] T. Drummond and R. Cpolla. Real-tme vsual trackng of complex structures. IEEE PAMI, 24(7): , July 22. [7] M. Haag and H.H. Nagel. Combnaton of edge element and optcal flow estmates for 3D-model-based vehcle trackng n traffc mage sequences. IJCV, 35(3): , December [8] V. Kyrk and D. Kragc. Integraton of model-based and model-free cues for vsual object trackng n 3D. In IEEE ICRA 5, pp , Barcelona, 25. [9] Y. Ma, S. Soatto, J. Košecká, and S. Sastry. An nvtaton to 3-D vson. Sprnger, 24. [1] E. Marchand, P. Bouthemy, F. Chaumette, and V. Moreau. Robust real-tme vsual trackng usng a 2D-3D model-based approach. In IEEE ICCV 99, pp , Kerkra, [11] G. Pann, Al. Ladkos, and A. Knoll. An effcent and robust real-tme contour trackng system. In ICVS, pp , 26. [12] G. Pann, E. Roth, and A. Knoll. Robust contour-based object trackng ntegratng color and edge lkelhoods. In Proc. of VMV 28, pp , Konstanz, 28. [13] A. Pett, E. Marchand, and K. Kanan. Trackng complex targets for space rendezvous and debrs removal applcatons. In IEEE/RSJ IROS 12, pp , Vlamoura, 212. [14] A. Pett, E. Marchand, and K. Kanan. A robust model-based tracker for space applcatons: combnng edge and color nformaton. In IEEE/RSJ IROS 213, Tokyo, 213. [15] M. Pressgout and E. Marchand. Real-Tme Hybrd Trackng usng Edge and Texture Informaton IJRR, 26(7): , 27. [16] M. Pressgout, E. Marchand, and E. Mémn. Hybrd trackng approach usng optcal flow and pose estmaton. In IEEE ICIP 8, pp , San Dego 28. [17] V. Prsacaru and I. Red. Pwp3d: Real-tme segmentaton and trackng of 3d objects. In Brtsh Machne Vson Conf., September 29. [18] E. Rosten and T. Drummond. Fusng ponts and lnes for hgh performance trackng. In IEEE ICCV, pp , Bejng, 25. [19] J. Sh and C. Tomas. Good features to track. In IEEE CVPR, pp , Seattle, [2] M. Tamaazoust et al. Nonlnear refnement of structure from moton reconstructon by takng advantage of a partal knowledge of the envronment. In IEEE CVPR, pp , 211. [21] L. Vacchett, V. Lepett, and P. Fua. Combnng edge and texture nformaton for real-tme accurate 3d camera trackng. In ACM/IEEE ISMAR 24, pp , Arlngton, 24. [22] H. Wuest and D. Strcker. Trackng of ndustral objects by usng cad models. Journal of Vrtual Realty and Broadcastng, 4(1), 27.