Alignment of Non-Overlapping Sequences
|
|
- Gavin Terry
- 6 years ago
- Views:
Transcription
1 Algnment of Non-Overlappng Sequences Yaron Casp Mchal ran Dept. of Computer Scence and Appled Math The Wezmann nsttute of Scence Rehovot, srael Ths paper shows how two mage sequences that have no spatal overlap between ther felds of vew can be algned both n tme and n space. Such algnment s possble when the two cameras are attached closely together and are moved jontly n space. The common moton nduces smlar changes over tme wthn the two sequences. Ths correlated temporal behavor, s used to recover the spatal and temporal transformatons between the two sequences. The requrement of coherent appearance n standard mage algnment technques s therefore replaced by coherent temporal behavor, whch s often easer to satsfy. Ths approach to algnment can be used not only for algnng non-overlappng sequences, but also for handlng other cases that are nherently dffcult for standard mage algnment technques. We demonstrate applcatons of ths approach to three real-world problems: () algnment of non-overlappng sequences for generatng wde-screen moves, () algnment of mages (sequences) obtaned at sgnfcantly dfferent zooms, for survellance applcatons, and, () mult-sensor mage algnment for mult-sensor fuson. 1 ntroducton The problem of mage algnment (or regstraton) has been extensvely researched, and successful approaches have been developed for solvng ths problem. Some of these approaches are based on matchng extracted local mage features. Other approaches are based on drectly matchng mage ntenstes. A revew of some of these methods can be found n [19] and [13]. However, all these approaches share one basc assumpton: that there s suffcent overlap between the two mages to allow extracton of common mage propertes, namely, that there s suffcent smlarty between the two mages ( Smlarty of mages s used here n the broadest sense. t could range from graylevel smlarty, to feature smlarty, to smlarty of frequences, and all the way to statstcal smlarty such as mutual nformaton [21]). n ths paper the followng queston s addressed: Can two mages be algned when there s very lttle smlarty between them, or even more extremely, when there s no spatal overlap at all between the two mages? When dealng wth ndvdual mages, the answer tends to be No. However, ths s not the case when dealng wth mage sequences. An mage sequence contans much more nformaton than any ndvdual frame does. n partcular, temporal changes (such as dynamc changes n the scene, or the nduced mage moton) are encoded between vdeo frames, but do not appear n any ndvdual frame. Such nformaton can form a powerful cue for algnment of two (or more) sequences. Casp and ran [6] and Sten [18] have llustrated an applcablty of such an approach for algnng two sequences based on common dynamc scene nformaton. However, they assumed that the same temporal changes n the scene (e.g., movng objects) are vsble to both vdeo cameras, leadng to the requrement that there must be sgnfcant overlap n the FOV s (felds-of-vew) of the two cameras. n ths paper we show that when two cameras are attached closely to each other (so that ther centers of projectons are very close), and move jontly n space, then the nduced frame-to-frame transformatons wthn each sequence have correlated behavor across the two sequences. Ths s true even when the sequences have no spatal overlap. Ths correlated temporal behavor s used to recover both the spatal and temporal transformatons between the two sequences. Unlke carefully calbrated stereo-rgs [17], our approach does not requre any pror nternal or external camera calbraton, nor any sophstcated hardware. Our approach bears resemblance to the approaches suggested by [7, 11, 22] for auto-calbraton of stereo-rgs. But unlke these methods, we do not requre that the two cameras observe and match the same scene features, nor that ther FOV s wll overlap. The need for coherent appearance, whch s a fundamental assumpton n mage algnment methods, s replaced here wth the requrement of coherent temporal behavor. A smlar approach was used for hand eye calbraton n robotcs research e.g., [20, 12]. Coherent temporal behavor s often easer to satsfy (e.g., by movng the two cameras jontly n space). Our approach s therefore useful not only n the case of non-overlappng sequences, but also n other cases whch are nherently dffcult for standard mage algnment technques. Ths gves rse to a varety of real-world applcatons, n-
2 n+1 2 Ι 2 Ι n+1 1 Fgure 1: Two vdeo cameras are attached to each other, so that they have the same center of projecton, but non-overlappng feldsof-vew. The two cameras are moved jontly n space, producng two separate vdeo sequences 1 ::: n+1 and 0 1 ::: 0 n+1. cludng: () Mult-sensor algnment for mage fuson. Ths requres accurate algnment of mages (sequences) obtaned by sensors of dfferent sensng modaltes (such as nfra- Red and vsble lght). Such mages dffer sgnfcantly n ther appearance due to dfferent sensor propertes [21]. () Algnment of mages (sequences) obtaned at dfferent zooms. The problem here s that dfferent mage features are promnent at dfferent mage resolutons [8]. Algnment of a wde-fov sequence wth a narrow-fov sequence s useful for detectng small zoomed-n objects n (or outsde) a zoomed-out vew of the scene. Ths can be useful n survellance applcatons. () Generaton of wde-screen moves from multple non-overlappng narrow FOV moves (such as n MAX moves). Our approach can handle such cases. Results are demonstrated n the paper on complex real-world sequences, as well as on manpulated sequences wth ground truth. 2 Problem Formulaton We examne the case when two vdeo cameras havng (approxmately) the same center of projecton but dfferent 3D orentaton, move jontly n space (see Fg. 1). The felds of vew of the two cameras do not necessarly overlap. The nternal parameters of the two cameras are dfferent and unknown, but fxed along the sequences. The external parameters relatng the two cameras (.e., the relatve 3D orentaton) are also unknown but fxed. Let S = 1 :::n+1 and S 0 = 1 0 ::: 0 m+1 be the two sequences of mages recorded by the two cameras 1. When temporal synchronzaton (e.g., tme stamps) s not avalable, then and 0 may not be correspondng frames n tme. Our goal s to recover the transformaton that algns the two sequences both n tme and n space. Note the term algnment here has a broader meanng than the usual one, as the sequences may not overlap n space, and may not be synchronzed n tme. Here we refer 1 The subscrpt s used represents the frame tme ndex, and the superscrpt prme s used to dstngush between the two sequences S and S 0. Ι 1 Sequence 1 Sequence 2 T1 T2 T n n H H t 1 m T 1 T T m-1 Fgure 2: Problem formulaton. The two sequences are spatally related by a fxed but unknown nter-camera homography H, and temporally related by a fxed and unknown tme shft t. Gven the frame-to-frame transformatons T 1 ::: Tn and T1 0 ::: T m, 0 we want to recover H and t. to algnment as dsplayng one sequence n the spatal coordnate system of the other sequence, and at the correct tme shft, as f obtaned by the other camera. When the two cameras have the same center of projecton (and dffer only n ther 3D orentaton and ther nternal calbraton parameters), then a smple fxed homography H (a 2D projectve transformaton) descrbes the spatal transformaton between temporally correspondng pars of frames across the two sequences [10]. f there were enough common features (e.g., p and p 0 ) between temporally correspondng frames (e.g., and 0), then t would be easy to recover the nter-camera homography H, as each such par of correspondng mage ponts provdes two lnear constrans on H: p 0 = Hp. Ths, n fact, s how most mage algnment technques work [10]. However, ths s not the case here. The two sequence do not share common features, because there s no spatal overlap between the two sequences. nstead, the homography H s recovered from the nduced frame-to-frame transformatons wthn each sequence. Let T 1 :::Tn and T1 0 :::T0 m be the sequences of frameto-frame transformatons wthn the vdeo sequences S and S 0, respectvely. T s the transformaton relatng frame to +1. These transformatons can be ether 2D parametrc transformatons (e.g., homographes or affne transformatons) or 3D transformatons/relatons (e.g., fundamental matrces). We next show how we can recover the spatal transformaton H and the temporal shft t between the two vdeo sequences drectly from the two sequences of transformatons T 1 :::Tn and T1 0 :::T0 m. The problem formulated above s llustrated n Fg Recoverng Spatal Algnment Between Sequences Let us frst assume that the temporal synchronzaton s known. Such nformaton s often avalable (e.g., from tme stamps encoded n each of the two sequences). Secton 4 shows how we can recover the temporal shft between the two sequences when that nformaton s not avalable. Therefore, wthout loss of generalty, t s assumed that
3 and 0 are correspondng frames n tme n sequences S and S 0, respectvely. Two cases are examned: () The case when the scene s planar or dstant from the cameras. We refer to these scenes as 2D scenes. n ths case the frame-to-frame transformatons T can be modeled by homographes (Sec. 3.1). () The case of a non-planar scene. We refer to these scenes as 3D scenes. n ths case the frame-to-frame relaton can be modeled by a fundamental matrx (Sec. 3.2). 3.1 Planar or Dstant (2D) Scenes When the scene s planar or dstant from the cameras, or when the jont 3D translaton of the two cameras s neglgble relatve to the dstance of the scene, then the nduced mage motons wthn each sequence (.e., T 1 :::Tn and T1 0 :::T0 n ) can be descrbed by 2D parametrc transformatons [10]. T thus denotes the homography between frame and +1, represented by 3 3 non-sngular matrces. We next show that temporally correspondng transformatons T and T 0 are also related by the fxed nter-camera homography H (whch relates frames and 0). Let P be a 3D pont n the planar (or the remote) scene. Denote by p and p 0 ts mage coordnates n frames and 0, respectvely (the pont P need not to be vsble n the frames,.e., P need not be wthn the FOV of the cameras). Let p+1 and p 0 +1 be ts mage coordnates n frames +1 and +1 0, respectvely. Then, p +1 = Tp and p 0 +1 = T 0 p0. Because the coordnates of the vdeo sequences S and S 0 are related by a fxed homography H, then: p 0 = Hp and p 0 +1 = Hp+1. Therefore: HTp = Hp+1 = p 0 +1 = T 0 p0 = T 0 Hp (1) Each p could theoretcally have a dfferent scalar assocated wth the equalty n Eq. (1). However, t s easy to show that because the relaton n Eq. (1) holds for all ponts p, therefore all these scalars are equal, and hence: HT = T 0 H: (2) Because H s nvertble, we may wrte T 0 = HTH ;1,or T 0 = s HTH ;1 (3) where s s a (frame-dependent) scale factor. Eq. (3) s true for all frames (.e., for any par of correspondng transformatons T and T 0, = 1::n). Eq. (3) shows that there s a smlarty relaton 2 (or conjugacy relaton) between the two matrces T and T 0 (up to a scale factor). A smlar observaton was made for case of hand-eye calbraton (e.g., [20, 12]), and for auto-calbraton of a stereo-rg (e.g. [22]). 2 A matrx A s sad to be smlar to a matrx B f there exsts an nvertble matrx M such that A = MBM ;1. See [9]. The term conjugate matrces can be used as well. Denote by eg(a) = [ ] t a 3 1 vector contanng the egenvalues of a 3 3 matrx A (n decreasng order). Then t s known ([9] pp. 898.) that: () f A and B are smlar matrces, then they have the same egenvalues: eg(a) = eg(b), and, () The egenvalues of a scaled matrx are scaled: eg(sa) = s(eg(a)). Usng these two facts and Eq. (3) we obtan: eg(t 0 )=s eg(t) (4) where s s the scale factor defned by Eq. (3). Eq. (4) mples that eg(t) and eg(t 0 ) are parallel. Ths gves rse to a measure of smlarty between two matrces T and T 0 sm(t T 0 )= eg(t ) t eg(t 0 ) jjeg(t)jj jjeg(t 0 : )jj (5) where jj jj s the vector norm. For real valued egenvalues, Eq. (5) provdes the cosne of the angle between the two vectors eg(t) and eg(t 0 ). Ths property wll be used later for obtanng the temporal synchronzaton (Secton 4). Ths measure s also used for outler rejecton of bad frame-toframe transformaton pars, T and T 0. The remander of ths secton explans how the fxed nter-camera homography H s recovered from the lst of frame-to frame transformatons T 1 ::Tn and T1 0 :: T n, 0 and dscusses unqueness of the soluton. For each par of temporally correspondng transformatons T and T 0 n sequences S and S 0, we frst compute ther egenvalues eg(t) and eg(t 0 ). The scale factor s whch relates them s then estmated from Eq. (4) usng least squares mnmzaton. (three equatons one unknown). Once s s estmated, Eq. (3) (or Eq. (2)) can be rewrtten as: sht ; T 0 H =0 (6) Eq. (6) s lnear n the unknown components of H. Rearrangng the components of H n a 9 1 column vector ~ h = [H11 H 12 H 13 H 21 H 22 H 23 H 31 H 32 H 33 ] t, Eq. (6) can be rewrtten as a set of lnear equatons n ~ h: 2 M ~ h = ~ 0 (7) where M s a 9 9 matrx defned by T, T 0 and s : M = 4 s t T ; 0 T11 ;T 0 12 ;T 0 13 ;T 0 21 st t ; T 0 22 ;T 0 23 ;T 0 31 ;T 0 32 st t ; T 0 33 and s the 3 3 dentty matrx. Eq. (7) mples that each par of correspondng transformatons T and T 0 contrbutes 9 lnear constrans n the unknown homography H (.e., ~ h). t can be shown [5] that f T (and hence also T 0 ) have 3 dfferent egenvalues, then H can be determned by a sngle such par of transformatons
4 (a) (b) (a) (b) (c) Fgure 3: Algnment of non-overlappng sequences. (a) and (b) are temporally correspondng frames from sequences S and S 0. The correct tme shft was automatcally detected. (c) shows one frame n the combned sequence after spatotemporal algnment. Note the accuracy of the spatal and temporal algnment of the runnng person. For full sequences see up to three degrees of freedom. Therefore, at least two such pars of ndependent transformatons are needed to unquely determne the homography H (up to a scale factor). The constrants from all the transformatons T 1 :: Tn can be combned nto a sngle set of lnear and T 0 1 :: T0 n equatons n ~ h: where A s a 9n9 matrx: A = 2 64 M 1. A ~ h = ~ 0 (8) Mn Eq. (8) s a homogeneous set of lnear equatons n ~ h, that can be solved n a varety of ways [3]. n partcular, ~ h may be recovered by computng the egenvector whch corresponds to the smallest egenvalue of the matrx A t A D Scenes When the scene s nether planar nor dstant, the relaton between two consecutve frames of an uncalbrated camera s descrbed by the fundamental matrx [10]. n ths case the nput to our algorthm s two sequences of fundamental matrces between successve frames, denoted by F 1 :::Fn and F1 0 :::F n 0. Namely, f p 2 and p are correspondng mage ponts, then: p t +1 F p =0. Although the relatons wthn each sequence are characterzed by fundamental matrces, the nter-camera transformaton remans a (c) Fgure 4: Algnment of non-overlappng sequences. (a) and (b) are temporally correspondng frames from sequences S and S 0. The correct tme shft was automatcally detected. (c) shows one frame n the combned sequence. Correspondng vdeo frames were averaged after spato-temporal algnment. The small overlappng area was not used n the estmaton process, but only for verfcaton (see text). Note the accuracy of the spatal and temporal algnment of the soccer player n the overlappng regon. For full sequences see homography H. Ths s because the two cameras stll share the same center of projecton (Sec. 2). Each fundamental matrx F can be decomposed nto a homography + eppole as follows [10]: F =[e] x T where e s the eppole relatng frames and +1, the matrx T s the nduced homography from to +1 va any plane (real or vrtual). [] x s the cross product matrx ([v] x ~w = ~v ~w). The homographes, T 1 ::: Tn and T1 0 ::: T n 0, and the eppoles e 1 ::: en and e 0 1 ::: e0 n, mpose separate constrants on the nter-camera homography H. These constrants can be used separately or jontly to recover H. () Homography-based constrants: The homographes T 1 :: Tn and T1 0 :: T0 n (extracted from the fundamental matrces F 1 :: Fn and F1 0 :: F n 0, respectvely), may correspond to dfferent 3D planes. n order to apply the algorthm of Sec. 3.1 usng these homographes, we need mpose plane-consstency across the two sequences (to guarantee that temporally correspondng homographes correspond to the same plane n the 3D world). One possble way for mposng plane-consstency across (and wthn) the two sequences s by usng the Plane+Parallax approach [16, 14]. However, ths approach requres that a real phys-
5 cal planar surface be vsble n all vdeo frames. Alternatvely, the threadng method of [1] or other methods for computng consstent set of camera matrces (e.g., [2]), can mpose plane-consstency wthn each sequence, even f no real physcal plane s vsble n any of the frames. Plane consstency across the two sequences can be guaranteed e.g., f [1] s ntated at frames whch are known to smultaneously vew the same real plane n both sequences. However, the two cameras can see dfferent portons of the plane (allowng for non-overlappng FOVs), and need not see the plane at any of the other frames. Ths approach s therefore less restrctve than the Plane+Parallax approach. () Eppole-based constrants: The fundamental matrces F 1 ::Fn and F 0 1 ::F 0 n also provde a lst of eppoles e 1 ::: en and e 0 1 ::: e0 n. These eppoles are unquely defned (there s no ssue of plane consstency here). Snce the two cameras have the same center of projecton, then for any frame : e 0 = He, or more specfcally: (e 0 ) x = [h 1h 2 h 3 ] e [h 7 h 8 h 9 ] e (e 0 ) y = [h 4h 5 h 6 ] e [h 7 h 8 h 9 ] e Multplyng by the domnator and rearrangng terms yelds two new lnear constrans on H for every par of correspondng eppoles e and e 0 : e t ~ 0 t (e 0 ) x e t ~ 0 t e t (e 0 ) y e 29 t ~ h =0 (10) where ~ 0 t =[0 0 0]. Every par of temporally correspondng eppoles, e and e 0, thus mposes two lnear constrants on H. These 2n constrants ( = 1 :: n) can be added to the set of lnear equatons n Eq. (8) whch are mposed by the homographes. Alternatvely, the eppole-related constrants can be used alone to solve for H, thus avodng the need to enforce plane-consstency on the homographes. Theoretcally, four pars of correspondng eppoles e and e 0 are suffcent. 4 Recoverng Temporal Synchronzaton Between Sequences So far we have assumed that the temporal synchronzaton between the two sequences s known and gven. Namely, that frame n sequence S corresponds to frame 0 n sequence S 0, and therefore the transformaton T corresponds to transformaton T 0. Such nformaton s often avalable from tme stamps. However, when such synchronzaton s not avalable, we can recover t. Gven two unsynchronzed sequences of transformatons T 1 :::Tn and T1 0 :::T m, 0 we wsh to recover the unknown temporal shft t between them. Let T and T+t 0 be temporally correspondng transformatons (namely, they occurred at the (9) same tme nstance). Then from Eq. (4) we know that they should satsfy eg(t) k eg(t+t 0 ) (.e., the 3 1 vectors of egenvalues should be parallel). n other words, the smlarty measure sm(tt T t 0 0 +t) of Eq. (5) should equal 1 (correspondng to cos(0),.e., an angle of 0 between the two vectors). All pars of correspondng transformatons T and T+t 0 must smultaneously satsfy ths constrant for the correct tme shft t. Therefore, we recover the unknown temporal tme shft t by maxmzng the followng objectve functon: SM(t) =X sm(t T+t) 2 (11) The maxmzaton s currently performed by an exhaustve search over a fnte range of vald tme shfts t. To address larger temporal shfts, we apply a herarchcal search. Coarser temporal levels are constructed by composng transformatons to obtan fewer transformaton between more dstant frames. The objectve functon of Eq. (11) can be generalzed to handle sequences of dfferent frame rates, such as sequences obtaned by NTSC cameras (30 frame/sec) vs. PAL cameras (25 frames/sec). The rato between frames correspondng to equal tme steps n the two sequences s 25 : 30 = 5 : 6. Therefore, the objectve functon that should be maxmzed for an NTSC-PAL par of sequences s: 5(+1) SM(t) =Xsm(T 5 T 0 6(+1)+t ) 2 (12) 6+t Where T j s the transformaton from frame to frame j. n our experments, all sequences were obtaned by PAL vdeo cameras. Therefore only the case of equal framerate (Eq. (11)) was expermentally verfed. We found ths method to be very robust. t successfully recovered the temporal shft up to feld (sub-frame) accuracy. Sub-feld accuracy may be further recovered by nterpolatng the values of SM(t) obtaned at dscrete tme shfts. 5 Applcatons Ths secton llustrates the applcablty of our method to solvng some real-world problems, whch are partcularly dffcult for standard mage algnment technques. These nclude: () Algnment of non-overlappng sequences for generaton of wde-screen moves from multple narrowscreen moves (such as n MAX flms), () Algnment of sequences obtaned at sgnfcantly dfferent zooms (e.g., for survellance applcatons), and () Algnment of multsensor sequences for mult-sensor fuson. We show results of applyng the method to complex real-world sequences. n addton, n order to emprcally quantfy the accuracy of
6 Wde vew Zoomed Vew Algned Vews (1.a) (1.b) (1.c) (2.a) (2.b) (2.c) (3.a) (3.b) (3.c) Fgure 5: Fndng zoomed regon. Ths fgure dsplays three dfferent examples (one at each row), each one wth dfferent zoom factor. The left column (1.a, 2.a, 3.a) dsplay one frame from each of the three wde-fov sequences. The temporally correspondng frames from the correspondng narrow-fov sequences are dsplayed n the center column. The correct tme shft was automatcally detected for each par of narrow/wde FOV sequences. Each mage on the rght column shows super-poston of correspondng frames of the two sequences after spato-temporal algnment, dsplayed by color averagng. For full sequences see our method, we also appled t to pars of sequences generated from a real sequence by warpng t wth known (ground truth) homographes. All sequences whch we expermented wth were captured by of the shelf consumer CCD cameras. The cameras were attached to each other, to mnmze the dstance between ther centers of projectons. The jont camera moton was performed manually (.e., a person would manually hold and rotate the two attached cameras). No temporal synchronzaton tool was used. The frame-to-frame nput transformatons wthn each sequence (homographes T 1 :::Tn and T1 0 :::T n 0 ) were extracted usng the method descrbed n [15]. The nput sequences were usually several seconds long to guaranty sgnfcant enough moton. The temporal tme shft was recovered usng the algorthm descrbed n Sec. 4 up to feld accuracy. naccurate frame-to-frame transformatons T were pruned out by usng two outler detecton mechansms. These are dscussed n detal n [5]. Fnally, the best thrty or so transformatons were used n the estmaton of the nter-camera homography H (usng the algorthm descrbed n Sec. 3.1). 5.1 Algnment of Non-Overlappng Sequences Fg. 3 shows an example of algnment of nonoverlappng sequences. The left camera s zoomed-n and rotated relatve to the rght camera. The correct spato-temporal algnment can be seen n Fg. 3.c. Note the accurate algnment of the runnng person both n tme and n space. Our approach to sequence algnment can be used to generate wde-screen moves from two (or more) narrow feldof-vew moves (such as n MAX moves). Such an example s shown n Fg. 4. To verfy the accuracy of algnment (both n tme and n space), we allowed for a very small overlap between the two sequences. However, ths mage regon was not used n the estmaton process, to mtate the case of truly non-overlappng sequences. The overlappng regon was used only for dsplay and verfcaton purposes. Fg. 4.c shows the result of combnng the two sequences (by averag-
7 Vsble R Output (a) (b) (c) Fgure 6: Mult-sensor Algnment. (a) and (b) are temporally correspondng frames from the vsble-lght and R sequences, respectvely (the temporal algnment was automatcally detected). The nsde of the buldng s vsble only n the vsble-lght sequence, whle the R sequence captures the detals outdoors (e.g., the dark trees, the sgn, the bush). (c) shows the results of fusng the two sequences after spato-temporal algnment. The fused sequence preserves the detals from both sequences. Note the hgh accuracy of algnment (both n tme and n space) of the walkng lady. For more detals see text. For full sequences see ng correspondng frames) after spato-temporal algnment. Note the accurate spatal as well as temporal algnment of the soccer players n the averaged overlappng regon. n order to emprcally verfy the accuracy of our method, the real vdeo sequence of Fg. 7 was splt n the mddle, producng two non-overlappng sequences of half-a-frame wdth each. The true (ground truth) homography n ths case corresponds to a horzontal shft by the wdth of a frame (352 pxels). The frame-to-frame transformaton (T 1 :::Tn and T1 0 :::T n) 0 were estmated separately wthn each sequence usng [15]. The temporal shft (t = 0) was recovered correctly from these transformatons, and the nter-camera homography H was recovered up to a msalgnment error of less than 0.7 pxel over the entre mage. See Table 1 for summary of the quanttatve expermental results. 5.2 Algnment of Sequences Obtaned at Dfferent Zooms Often n survellance applcatons two cameras are used, one wth a wde FOV (feld-of-vew) for observng large scene regons, and the other camera wth a narrow FOV (zoomed-n) for detectng small objects. Matchng two such mages obtaned at sgnfcantly dfferent zooms s a dffcult problem for standard mage algnment methods, snce the two mages dsplay dfferent features whch are promnent at the dfferent resolutons. Our sequence algnment approach may be used for such scenaros. Fg. 5 shows three such examples. The results of the spato-temporal algnment (rght colunm of Fg. 5) are dsplayed n the form of averagng temporally correspondng frames after algnment accordng to the computed homography and the computed tme shft. n the frst example (top row of Fg. 5) the zoom dfference between the two cameras was approxmately 1:3. n the second example (second row) t was 1:4, and n the thrd example (bottom row) t was 1:8. Note the small red flowers n the zoomed vew (Fg. 5.2.b). These can barely be seen n the correspondng low resoluton wde-vew frame (Fg. 5.2.a). The same holds for the Pagoda n Fg. 5.3.b To emprcally verfy the accuracy of our method n the presence of large zooms and large rotatons, we ran the algorthm on followng three manpulated sequences wth known (ground truth) manpulatons: We warped the sequence of Fg. 7 once by a zoom factor of 2, once by a zoom factor of 4, and once rotated t by 180 o. The results are summarzed n Table 1. n each of these cases, the recovered homography was composed wth the nverse of the ground-truth homography: H true ;1 H recovored. deally, the composed homography should be the dentty matrx. The errors reported n Table 1 are the maxmal resdual msalgnment nduced by the composed homography over the entre mage. 5.3 Mult-Sensor Algnment mages obtaned by sensors of dfferent modaltes, e.g., R (nfra-red) and vsble lght, can vary sgnfcantly n ther appearance. Features appearng n one mage may not appear n the other, and vsa versa. Ths poses a problem for mage algnment methods. Our sequence algnment approach, however, does not requre coherent appearance between the two sequences, and can therefore be appled to solve the problem. Fg. 6 shows an example of two such sequences, one captured by a near R camera, whle the other by a regular vdeo (vsble-lght) camera. The scene was shot n twlght. n the sequence obtaned by the regular vdeo camera (Fg.6.(a)), the outdoor scene s barely vsble, whle the nsde of the buldng s clearly vsble. The
8 (a) (c) (d) Fgure 7: The sequence used for emprcal evaluaton. (a,b,c) are three frames (0,150,300) out of the orgnal 300 frames. Ths sequence was used as the base sequence for the quanttatve experments summarzed n Table 1. R camera, on the other hand, captures the outdoor scene n great detal, whle the ndoor part (llumnated by cold neon lght) was nvsble to the R camera (Fg. 6.(b)). The result of the spato-temporal algnment s llustrated by fusng temporally correspondng frames. The R camera provdes only ntensty nformaton, and was therefore fused only wth the ntensty (Y) component of the vsble-lght camera (usng the mage-fuson method of [4]). The chrome components ( and Q) of the vsble-lght camera supply the color nformaton. The reader s encouraged to vew color sequences at 6 Concluson Ths paper presents an approach for algnng two sequences (both n tme and n space), even when there s no common spatal nformaton between the sequences. Ths was made possble by replacng the need for coherent appearance (whch s a fundamental requrement n standard mages algnment technques), wth the requrement of coherent temporal behavor, whch s often easer to satsfy. We demonstrated applcatons of ths approach to real-world problems, whch are nherently dffcult for regular mage algnment technques. Acknowledgment The authors would lke to thank R. Basr and L. Zelnk- Manor for ther helpful comments. References [1] S. Avdan and A. Shashua. Thereadng fundamaental matrces. n European Conference on Computer Vson, pages , Freburg, June [2] P. A. Beardsley, P. H. S. Torr, and A. Zsserman. 3D model aquston from extended mage sequences. n Proc. 4th European Conference on Computer Vson, LNCS 1065, Cambrdge, pages , [3] A. Bjorck. Numercal Methodes for Least Squares Problems. SAM, Phladelpha, [4] P.R. Burt and R.J. Kolczynsk. Enhanced mage capture through fuson. n nternatonal Conference on Computer Vson, [5] Y. Casp and M. ran. Algnment of non overlappng sequences. jornal verson. Appled Recovered Max Resdual Transformaton Transformaton Msalgnment Horzontal shft Horzontal shft of 352 pxels of pxels 0.7 pxels Zoom factor = 2 Zoom factor = pxels Zoom factor = 4 Zoom factor = pxels Rotaton by 180 o Rotaton by 180:00 o 0.01 pxels Table 1: Quanttatve results. Ths table summarzes the quanttatve results wth respect to ground truth. Each row corresponds to one experment. n each experment a real vdeo sequence (Fg. 7) was warped ( manpulated ) by a known homography, to generate a second sequence. The left column descrbes the type of spatal transformaton appled to the sequence, the center column descrbes the recovered transformaton, and the rght column descrbes the resdual error between the ground-truth homography and the recovered homography (measured n maxmal resdual msalgnment n the mage space). n all 4 cases the correct temporal shft was recovered accurately. See text for further detals. [6] Y. Casp and M. ran. A step towards sequence-to-sequence algnment. n EEE Conference on Computer Vson and Pattern Recognton, pages , [7] D. Demrdjan, A. Zsserman, and R. Horaud. Stereo autocalbraton from one plane. n European Conference on Computer Vson, [8] Y. Dufournaud, C. Schmd, and R. Horaud. Matchng mages wth dfferent resolutons. n EEE Conference on Computer Vson and Pattern Recognton, pages , [9] C. E. Pearson (ed.). Handbook of appled mathematcs - Second Edton. Van Nostrand Renhold Company, New York, [10] R. Hartley and A. Zsserman. Multple Vew Geometry n Computer Vson. Cambrdge unversty press, Cambrdge, [11] R. Horaud and G. Csurka. reconstructon usng motons of a stereo rg. n nternatonal Conference on Computer Vson, pages , [12] R. Horaud and F. Dornaka. Hand-eye calbraton. nternatonal Journal of Robotcs Research, 14(3): , June [13] M. ran and P. Anandan. About drect methods. n Vson Algorthms Workshop, pages , Corfu, [14] M. ran, P. Anandan, and D. Wenshall. From reference frames to reference planes: Mult-vew parallax geometry and applcatons. n European Conference on Computer Vson, Freburg, June [15] M. ran, B. Rousso, and S. Peleg. Computng occludng and transparent motons. nternatonal Journal of Computer Vson, 12(1):5 16, January [16] R. Kumar, P. Anandan, and K. Hanna. Drect recovery of shape from multple vews: A parallax based approach. n nternatonal Conference on Pattern Recognton, [17] C.C. Slama. Manual of Photogrammetry. Amercan Socety of Photogrammetry and Remote Sensng, [18] G. P. Sten. Trackng from multple vew ponts: Self-calbraton of space and tme. n DARPA U Workshop, pages , [19] P.H.S. Torr and A. Zsserman. Feature based methods for structure and moton estmaton. n Vson Algorthms Workshop, [20] R. Y. Tsa and R. K. Lenz. A new technque for full autonomous and effcent 3d robotcs hand/eye calbraton. EEE Journal of Robotcs and Automaton, 5(3): , June [21] P. Vola and W. Wells. Algnment by maxmzaton of mutual nformaton. n nternatonal Conference on Computer Vson, pages 16 23, [22] A. Zsserman, P.A. Beardsley, and.d. Red. Metrc calbraton of a stereo rg. n Workshop on Representatons of Vsual Scenes, 1995.
Aligning Non-Overlapping Sequences,
Internatonal Journal of Computer Vson 48(1), 39 51, 2002 c 2002 Kluwer Academc Publshers. Manufactured n The Netherlands. Algnng Non-Overlappng Sequences, YARON CASPI AND MICHAL IRANI Department of Computer
More informationSLAM Summer School 2006 Practical 2: SLAM using Monocular Vision
SLAM Summer School 2006 Practcal 2: SLAM usng Monocular Vson Javer Cvera, Unversty of Zaragoza Andrew J. Davson, Imperal College London J.M.M Montel, Unversty of Zaragoza. josemar@unzar.es, jcvera@unzar.es,
More informationImage Alignment CSC 767
Image Algnment CSC 767 Image algnment Image from http://graphcs.cs.cmu.edu/courses/15-463/2010_fall/ Image algnment: Applcatons Panorama sttchng Image algnment: Applcatons Recognton of object nstances
More informationStructure from Motion
Structure from Moton Structure from Moton For now, statc scene and movng camera Equvalentl, rgdl movng scene and statc camera Lmtng case of stereo wth man cameras Lmtng case of multvew camera calbraton
More informationWhat are the camera parameters? Where are the light sources? What is the mapping from radiance to pixel color? Want to solve for 3D geometry
Today: Calbraton What are the camera parameters? Where are the lght sources? What s the mappng from radance to pel color? Why Calbrate? Want to solve for D geometry Alternatve approach Solve for D shape
More informationMulti-stable Perception. Necker Cube
Mult-stable Percepton Necker Cube Spnnng dancer lluson, Nobuuk Kaahara Fttng and Algnment Computer Vson Szelsk 6.1 James Has Acknowledgment: Man sldes from Derek Hoem, Lana Lazebnk, and Grauman&Lebe 2008
More informationInverse-Polar Ray Projection for Recovering Projective Transformations
nverse-polar Ray Projecton for Recoverng Projectve Transformatons Yun Zhang The Center for Advanced Computer Studes Unversty of Lousana at Lafayette yxz646@lousana.edu Henry Chu The Center for Advanced
More informationEcient Computation of the Most Probable Motion from Fuzzy. Moshe Ben-Ezra Shmuel Peleg Michael Werman. The Hebrew University of Jerusalem
Ecent Computaton of the Most Probable Moton from Fuzzy Correspondences Moshe Ben-Ezra Shmuel Peleg Mchael Werman Insttute of Computer Scence The Hebrew Unversty of Jerusalem 91904 Jerusalem, Israel Emal:
More informationA Binarization Algorithm specialized on Document Images and Photos
A Bnarzaton Algorthm specalzed on Document mages and Photos Ergna Kavalleratou Dept. of nformaton and Communcaton Systems Engneerng Unversty of the Aegean kavalleratou@aegean.gr Abstract n ths paper, a
More informationRange images. Range image registration. Examples of sampling patterns. Range images and range surfaces
Range mages For many structured lght scanners, the range data forms a hghly regular pattern known as a range mage. he samplng pattern s determned by the specfc scanner. Range mage regstraton 1 Examples
More informationRobust Recovery of Camera Rotation from Three Frames. B. Rousso S. Avidan A. Shashua y S. Peleg z. The Hebrew University of Jerusalem
Robust Recovery of Camera Rotaton from Three Frames B. Rousso S. Avdan A. Shashua y S. Peleg z Insttute of Computer Scence The Hebrew Unversty of Jerusalem 994 Jerusalem, Israel e-mal : roussocs.huj.ac.l
More informationParallelism for Nested Loops with Non-uniform and Flow Dependences
Parallelsm for Nested Loops wth Non-unform and Flow Dependences Sam-Jn Jeong Dept. of Informaton & Communcaton Engneerng, Cheonan Unversty, 5, Anseo-dong, Cheonan, Chungnam, 330-80, Korea. seong@cheonan.ac.kr
More informationReducing Frame Rate for Object Tracking
Reducng Frame Rate for Object Trackng Pavel Korshunov 1 and We Tsang Oo 2 1 Natonal Unversty of Sngapore, Sngapore 11977, pavelkor@comp.nus.edu.sg 2 Natonal Unversty of Sngapore, Sngapore 11977, oowt@comp.nus.edu.sg
More informationPROJECTIVE RECONSTRUCTION OF BUILDING SHAPE FROM SILHOUETTE IMAGES ACQUIRED FROM UNCALIBRATED CAMERAS
PROJECTIVE RECONSTRUCTION OF BUILDING SHAPE FROM SILHOUETTE IMAGES ACQUIRED FROM UNCALIBRATED CAMERAS Po-Lun La and Alper Ylmaz Photogrammetrc Computer Vson Lab Oho State Unversty, Columbus, Oho, USA -la.138@osu.edu,
More informationComputer Animation and Visualisation. Lecture 4. Rigging / Skinning
Computer Anmaton and Vsualsaton Lecture 4. Rggng / Sknnng Taku Komura Overvew Sknnng / Rggng Background knowledge Lnear Blendng How to decde weghts? Example-based Method Anatomcal models Sknnng Assume
More informationCorrespondence-free Synchronization and Reconstruction in a Non-rigid Scene
Correspondence-free Synchronzaton and Reconstructon n a Non-rgd Scene Lor Wolf and Assaf Zomet School of Computer Scence and Engneerng, The Hebrew Unversty, Jerusalem 91904, Israel e-mal: {lwolf,zomet}@cs.huj.ac.l
More informationn others; multple brghtness values n one mage may map to a sngle brghtness value n the other mage, and vce versa. In other words, the two mages are us
Robust Mult-Sensor Image Algnment Mchal Iran Dept. of Appled Math and CS The Wezmann Insttute of Scence 76100 Rehovot, Israel P. Anandan Mcrosoft Corporaton One Mcrosoft Way Redmond, WA 98052, USA Abstract
More informationAn Image Fusion Approach Based on Segmentation Region
Rong Wang, L-Qun Gao, Shu Yang, Yu-Hua Cha, and Yan-Chun Lu An Image Fuson Approach Based On Segmentaton Regon An Image Fuson Approach Based on Segmentaton Regon Rong Wang, L-Qun Gao, Shu Yang 3, Yu-Hua
More informationContent Based Image Retrieval Using 2-D Discrete Wavelet with Texture Feature with Different Classifiers
IOSR Journal of Electroncs and Communcaton Engneerng (IOSR-JECE) e-issn: 78-834,p- ISSN: 78-8735.Volume 9, Issue, Ver. IV (Mar - Apr. 04), PP 0-07 Content Based Image Retreval Usng -D Dscrete Wavelet wth
More informationCalibration of an Articulated Camera System
Calbraton of an Artculated Camera System CHEN Junzhou and Kn Hong WONG Department of Computer Scence and Engneerng The Chnese Unversty of Hong Kong {jzchen, khwong}@cse.cuhk.edu.hk Abstract Multple Camera
More information3D Metric Reconstruction with Auto Calibration Method CS 283 Final Project Tarik Adnan Moon
3D Metrc Reconstructon wth Auto Calbraton Method CS 283 Fnal Project Tark Adnan Moon tmoon@collge.harvard.edu Abstract In ths paper, dfferent methods for auto camera calbraton have been studed for metrc
More informationTN348: Openlab Module - Colocalization
TN348: Openlab Module - Colocalzaton Topc The Colocalzaton module provdes the faclty to vsualze and quantfy colocalzaton between pars of mages. The Colocalzaton wndow contans a prevew of the two mages
More informationFEATURE EXTRACTION. Dr. K.Vijayarekha. Associate Dean School of Electrical and Electronics Engineering SASTRA University, Thanjavur
FEATURE EXTRACTION Dr. K.Vjayarekha Assocate Dean School of Electrcal and Electroncs Engneerng SASTRA Unversty, Thanjavur613 41 Jont Intatve of IITs and IISc Funded by MHRD Page 1 of 8 Table of Contents
More informationAmnon Shashua Shai Avidan Michael Werman. The Hebrew University, objects.
Trajectory Trangulaton over Conc Sectons Amnon Shashua Sha Avdan Mchael Werman Insttute of Computer Scence, The Hebrew Unversty, Jerusalem 91904, Israel e-mal: fshashua,avdan,wermang@cs.huj.ac.l Abstract
More informationImprovement of Spatial Resolution Using BlockMatching Based Motion Estimation and Frame. Integration
Improvement of Spatal Resoluton Usng BlockMatchng Based Moton Estmaton and Frame Integraton Danya Suga and Takayuk Hamamoto Graduate School of Engneerng, Tokyo Unversty of Scence, 6-3-1, Nuku, Katsuska-ku,
More informationProper Choice of Data Used for the Estimation of Datum Transformation Parameters
Proper Choce of Data Used for the Estmaton of Datum Transformaton Parameters Hakan S. KUTOGLU, Turkey Key words: Coordnate systems; transformaton; estmaton, relablty. SUMMARY Advances n technologes and
More informationFitting & Matching. Lecture 4 Prof. Bregler. Slides from: S. Lazebnik, S. Seitz, M. Pollefeys, A. Effros.
Fttng & Matchng Lecture 4 Prof. Bregler Sldes from: S. Lazebnk, S. Setz, M. Pollefeys, A. Effros. How do we buld panorama? We need to match (algn) mages Matchng wth Features Detect feature ponts n both
More informationImage warping and stitching May 5 th, 2015
Image warpng and sttchng Ma 5 th, 2015 Yong Jae Lee UC Davs PS2 due net Frda Announcements 2 Last tme Interactve segmentaton Feature-based algnment 2D transformatons Affne ft RANSAC 3 1 Algnment problem
More informationCalibration of an Articulated Camera System with Scale Factor Estimation
Calbraton of an Artculated Camera System wth Scale Factor Estmaton CHEN Junzhou, Kn Hong WONG arxv:.47v [cs.cv] 7 Oct Abstract Multple Camera Systems (MCS) have been wdely used n many vson applcatons and
More informationA Comparison and Evaluation of Three Different Pose Estimation Algorithms In Detecting Low Texture Manufactured Objects
Clemson Unversty TgerPrnts All Theses Theses 12-2011 A Comparson and Evaluaton of Three Dfferent Pose Estmaton Algorthms In Detectng Low Texture Manufactured Objects Robert Krener Clemson Unversty, rkrene@clemson.edu
More informationSubspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points;
Subspace clusterng Clusterng Fundamental to all clusterng technques s the choce of dstance measure between data ponts; D q ( ) ( ) 2 x x = x x, j k = 1 k jk Squared Eucldean dstance Assumpton: All features
More informationA Fast Visual Tracking Algorithm Based on Circle Pixels Matching
A Fast Vsual Trackng Algorthm Based on Crcle Pxels Matchng Zhqang Hou hou_zhq@sohu.com Chongzhao Han czhan@mal.xjtu.edu.cn Ln Zheng Abstract: A fast vsual trackng algorthm based on crcle pxels matchng
More informationCS 534: Computer Vision Model Fitting
CS 534: Computer Vson Model Fttng Sprng 004 Ahmed Elgammal Dept of Computer Scence CS 534 Model Fttng - 1 Outlnes Model fttng s mportant Least-squares fttng Maxmum lkelhood estmaton MAP estmaton Robust
More informationResolving Ambiguity in Depth Extraction for Motion Capture using Genetic Algorithm
Resolvng Ambguty n Depth Extracton for Moton Capture usng Genetc Algorthm Yn Yee Wa, Ch Kn Chow, Tong Lee Computer Vson and Image Processng Laboratory Dept. of Electronc Engneerng The Chnese Unversty of
More informationCalibration of an Articulated Camera System
Calbraton of an Artculated Camera System CHEN Junzhou and Kn Hong WONG Department of Computer Scence and Engneerng The Chnese Unversty of Hong Kong {jzchen, khwong}@cse.cuhk.edu.hk Abstract Multple Camera
More informationHermite Splines in Lie Groups as Products of Geodesics
Hermte Splnes n Le Groups as Products of Geodescs Ethan Eade Updated May 28, 2017 1 Introducton 1.1 Goal Ths document defnes a curve n the Le group G parametrzed by tme and by structural parameters n the
More informationA Robust Method for Estimating the Fundamental Matrix
Proc. VIIth Dgtal Image Computng: Technques and Applcatons, Sun C., Talbot H., Ourseln S. and Adraansen T. (Eds.), 0- Dec. 003, Sydney A Robust Method for Estmatng the Fundamental Matrx C.L. Feng and Y.S.
More informationMulti-view 3D Position Estimation of Sports Players
Mult-vew 3D Poston Estmaton of Sports Players Robbe Vos and Wlle Brnk Appled Mathematcs Department of Mathematcal Scences Unversty of Stellenbosch, South Afrca Emal: vosrobbe@gmal.com Abstract The problem
More informationAn Optimal Algorithm for Prufer Codes *
J. Software Engneerng & Applcatons, 2009, 2: 111-115 do:10.4236/jsea.2009.22016 Publshed Onlne July 2009 (www.scrp.org/journal/jsea) An Optmal Algorthm for Prufer Codes * Xaodong Wang 1, 2, Le Wang 3,
More informationMOTION PANORAMA CONSTRUCTION FROM STREAMING VIDEO FOR POWER- CONSTRAINED MOBILE MULTIMEDIA ENVIRONMENTS XUNYU PAN
MOTION PANORAMA CONSTRUCTION FROM STREAMING VIDEO FOR POWER- CONSTRAINED MOBILE MULTIMEDIA ENVIRONMENTS by XUNYU PAN (Under the Drecton of Suchendra M. Bhandarkar) ABSTRACT In modern tmes, more and more
More informationThe Codesign Challenge
ECE 4530 Codesgn Challenge Fall 2007 Hardware/Software Codesgn The Codesgn Challenge Objectves In the codesgn challenge, your task s to accelerate a gven software reference mplementaton as fast as possble.
More informationAn Application of the Dulmage-Mendelsohn Decomposition to Sparse Null Space Bases of Full Row Rank Matrices
Internatonal Mathematcal Forum, Vol 7, 2012, no 52, 2549-2554 An Applcaton of the Dulmage-Mendelsohn Decomposton to Sparse Null Space Bases of Full Row Rank Matrces Mostafa Khorramzadeh Department of Mathematcal
More informationAnalysis of Continuous Beams in General
Analyss of Contnuous Beams n General Contnuous beams consdered here are prsmatc, rgdly connected to each beam segment and supported at varous ponts along the beam. onts are selected at ponts of support,
More information3D Rigid Facial Motion Estimation from Disparity Maps
3D Rgd Facal Moton Estmaton from Dsparty Maps N. Pérez de la Blanca 1, J.M. Fuertes 2, and M. Lucena 2 1 Department of Computer Scence and Artfcal Intellgence ETSII. Unversty of Granada, 1871 Granada,
More informationMulti-body Segmentation: Revisiting Motion Consistency Λ
Mult-body Segmentaton: Revstng Moton Consstency Λ Lh Zelnk-Manor Moshe Machlne Mchal Iran Dept. of Computer Scence and Appled Math The Wezmann Insttute of Scence 76 Rehovot, Israel Abstract Dynamc analyss
More informationy and the total sum of
Lnear regresson Testng for non-lnearty In analytcal chemstry, lnear regresson s commonly used n the constructon of calbraton functons requred for analytcal technques such as gas chromatography, atomc absorpton
More informationReal-time Motion Capture System Using One Video Camera Based on Color and Edge Distribution
Real-tme Moton Capture System Usng One Vdeo Camera Based on Color and Edge Dstrbuton YOSHIAKI AKAZAWA, YOSHIHIRO OKADA, AND KOICHI NIIJIMA Graduate School of Informaton Scence and Electrcal Engneerng,
More informationNew dynamic zoom calibration technique for a stereo-vision based multi-view 3D modeling system
New dynamc oom calbraton technque for a stereo-vson based mult-vew 3D modelng system Tao Xan, Soon-Yong Park, Mural Subbarao Dept. of Electrcal & Computer Engneerng * State Unv. of New York at Stony Brook,
More informationFitting and Alignment
Fttng and Algnment Computer Vson Ja-Bn Huang, Vrgna Tech Many sldes from S. Lazebnk and D. Hoem Admnstratve Stuffs HW 1 Competton: Edge Detecton Submsson lnk HW 2 wll be posted tonght Due Oct 09 (Mon)
More informationDirect Methods for Visual Scene Reconstruction
To appear at the IEEE Workshop on Representatons of Vsual Scenes, June 24, 1995, Cambrdge, MA 1 Drect Methods for Vsual Scene Reconstructon Rchard Szelsk and Sng Bng Kang Dgtal Equpment Corporaton Cambrdge
More informationAn efficient method to build panoramic image mosaics
An effcent method to buld panoramc mage mosacs Pattern Recognton Letters vol. 4 003 Dae-Hyun Km Yong-In Yoon Jong-Soo Cho School of Electrcal Engneerng and Computer Scence Kyungpook Natonal Unv. Abstract
More informationMULTISPECTRAL IMAGES CLASSIFICATION BASED ON KLT AND ATR AUTOMATIC TARGET RECOGNITION
MULTISPECTRAL IMAGES CLASSIFICATION BASED ON KLT AND ATR AUTOMATIC TARGET RECOGNITION Paulo Quntlano 1 & Antono Santa-Rosa 1 Federal Polce Department, Brasla, Brazl. E-mals: quntlano.pqs@dpf.gov.br and
More informationS1 Note. Basis functions.
S1 Note. Bass functons. Contents Types of bass functons...1 The Fourer bass...2 B-splne bass...3 Power and type I error rates wth dfferent numbers of bass functons...4 Table S1. Smulaton results of type
More informationR s s f. m y s. SPH3UW Unit 7.3 Spherical Concave Mirrors Page 1 of 12. Notes
SPH3UW Unt 7.3 Sphercal Concave Mrrors Page 1 of 1 Notes Physcs Tool box Concave Mrror If the reflectng surface takes place on the nner surface of the sphercal shape so that the centre of the mrror bulges
More informationDetecting Irregularities in Images and in Video
Detectng Irregulartes n Images and n Vdeo Oren Boman Mchal Iran Dept. of Computer Scence and Appled Math The Wezmann Insttute of Scence 76100 Rehovot, Israel Abstract We address the problem of detectng
More informationA Fast Content-Based Multimedia Retrieval Technique Using Compressed Data
A Fast Content-Based Multmeda Retreval Technque Usng Compressed Data Borko Furht and Pornvt Saksobhavvat NSF Multmeda Laboratory Florda Atlantc Unversty, Boca Raton, Florda 3343 ABSTRACT In ths paper,
More informationComputer Vision I. Xbox Kinnect: Rectification. The Fundamental matrix. Stereo III. CSE252A Lecture 16. Example: forward motion
Xbox Knnect: Stereo III Depth map http://www.youtube.com/watch?v=7qrnwoo-8a CSE5A Lecture 6 Projected pattern http://www.youtube.com/watch?v=ceep7x-z4wy The Fundamental matrx Rectfcaton The eppolar constrant
More informationAlignment of Non-Overlapping Sequences
Alignment of Non-Overlapping Sequences Yaron Caspi Michal Irani Dept. of Computer Science and Applied Math The Weizmann Institute of Science 76100 Rehovot, Israel PLEASE PRINT IN COLOR Abstract This paper
More informationMETRIC ALIGNMENT OF LASER RANGE SCANS AND CALIBRATED IMAGES USING LINEAR STRUCTURES
METRIC ALIGNMENT OF LASER RANGE SCANS AND CALIBRATED IMAGES USING LINEAR STRUCTURES Lorenzo Sorg CIRA the Italan Aerospace Research Centre Computer Vson and Vrtual Realty Lab. Outlne Work goal Work motvaton
More informationDynamic Camera Assignment and Handoff
12 Dynamc Camera Assgnment and Handoff Br Bhanu and Ymng L 12.1 Introducton...338 12.2 Techncal Approach...339 12.2.1 Motvaton and Problem Formulaton...339 12.2.2 Game Theoretc Framework...339 12.2.2.1
More informationTsinghua University at TAC 2009: Summarizing Multi-documents by Information Distance
Tsnghua Unversty at TAC 2009: Summarzng Mult-documents by Informaton Dstance Chong Long, Mnle Huang, Xaoyan Zhu State Key Laboratory of Intellgent Technology and Systems, Tsnghua Natonal Laboratory for
More information3D Modeling Using Multi-View Images. Jinjin Li. A Thesis Presented in Partial Fulfillment of the Requirements for the Degree Master of Science
3D Modelng Usng Mult-Vew Images by Jnjn L A Thess Presented n Partal Fulfllment of the Requrements for the Degree Master of Scence Approved August by the Graduate Supervsory Commttee: Lna J. Karam, Char
More informationFor instance, ; the five basic number-sets are increasingly more n A B & B A A = B (1)
Secton 1.2 Subsets and the Boolean operatons on sets If every element of the set A s an element of the set B, we say that A s a subset of B, or that A s contaned n B, or that B contans A, and we wrte A
More informationDiscriminative Dictionary Learning with Pairwise Constraints
Dscrmnatve Dctonary Learnng wth Parwse Constrants Humn Guo Zhuoln Jang LARRY S. DAVIS UNIVERSITY OF MARYLAND Nov. 6 th, Outlne Introducton/motvaton Dctonary Learnng Dscrmnatve Dctonary Learnng wth Parwse
More informationROBOT KINEMATICS. ME Robotics ME Robotics
ROBOT KINEMATICS Purpose: The purpose of ths chapter s to ntroduce you to robot knematcs, and the concepts related to both open and closed knematcs chans. Forward knematcs s dstngushed from nverse knematcs.
More informationLecture 4: Principal components
/3/6 Lecture 4: Prncpal components 3..6 Multvarate lnear regresson MLR s optmal for the estmaton data...but poor for handlng collnear data Covarance matrx s not nvertble (large condton number) Robustness
More informationAn Entropy-Based Approach to Integrated Information Needs Assessment
Dstrbuton Statement A: Approved for publc release; dstrbuton s unlmted. An Entropy-Based Approach to ntegrated nformaton Needs Assessment June 8, 2004 Wllam J. Farrell Lockheed Martn Advanced Technology
More informationEVALUATION OF RELATIVE POSE ESTIMATION METHODS FOR MULTI-CAMERA SETUPS
EVALUAION OF RELAIVE POSE ESIMAION MEHODS FOR MULI-CAMERA SEUPS Volker Rodehorst *, Matthas Henrchs and Olaf Hellwch Computer Vson & Remote Sensng, Berln Unversty of echnology, Franklnstr. 8/9, FR 3-,
More informationOptimal Combination of Stereo Camera Calibration from Arbitrary Stereo Images.
Tna Memo No. 1991-002 Image and Vson Computng, 9(1), 27-32, 1990. Optmal Combnaton of Stereo Camera Calbraton from Arbtrary Stereo Images. N.A.Thacker and J.E.W.Mayhew. Last updated 6 / 9 / 2005 Imagng
More information6.854 Advanced Algorithms Petar Maymounkov Problem Set 11 (November 23, 2005) With: Benjamin Rossman, Oren Weimann, and Pouya Kheradpour
6.854 Advanced Algorthms Petar Maymounkov Problem Set 11 (November 23, 2005) Wth: Benjamn Rossman, Oren Wemann, and Pouya Kheradpour Problem 1. We reduce vertex cover to MAX-SAT wth weghts, such that the
More informationLine-based Camera Movement Estimation by Using Parallel Lines in Omnidirectional Video
01 IEEE Internatonal Conference on Robotcs and Automaton RverCentre, Sant Paul, Mnnesota, USA May 14-18, 01 Lne-based Camera Movement Estmaton by Usng Parallel Lnes n Omndrectonal Vdeo Ryosuke kawansh,
More informationShape Representation Robust to the Sketching Order Using Distance Map and Direction Histogram
Shape Representaton Robust to the Sketchng Order Usng Dstance Map and Drecton Hstogram Department of Computer Scence Yonse Unversty Kwon Yun CONTENTS Revew Topc Proposed Method System Overvew Sketch Normalzaton
More informationIntroduction to Multiview Rank Conditions and their Applications: A Review.
Introducton to Multvew Rank Condtons and ther Applcatons: A Revew Jana Košecká Y Ma Department of Computer Scence, George Mason Unversty Electrcal & Computer Engneerng Department, Unversty of Illnos at
More informationSupport Vector Machines
/9/207 MIST.6060 Busness Intellgence and Data Mnng What are Support Vector Machnes? Support Vector Machnes Support Vector Machnes (SVMs) are supervsed learnng technques that analyze data and recognze patterns.
More informationLecture #15 Lecture Notes
Lecture #15 Lecture Notes The ocean water column s very much a 3-D spatal entt and we need to represent that structure n an economcal way to deal wth t n calculatons. We wll dscuss one way to do so, emprcal
More informationLECTURE : MANIFOLD LEARNING
LECTURE : MANIFOLD LEARNING Rta Osadchy Some sldes are due to L.Saul, V. C. Raykar, N. Verma Topcs PCA MDS IsoMap LLE EgenMaps Done! Dmensonalty Reducton Data representaton Inputs are real-valued vectors
More informationPlanar Catadioptric Stereo: Multiple-View Geometry and Image-Based Camera Localization
Manuscrpt Marottn et al., Spec. Issue Vs. Serv. Clck here to vew lnked References Planar Catadoptrc Stereo: Multple-Vew Geometry and Image-Based Camera Localzaton Gan Luca Marottn a,stefano Schegg b,fabo
More informationAIMS Computer vision. AIMS Computer Vision. Outline. Outline.
AIMS Computer Vson 1 Matchng, ndexng, and search 2 Object category detecton 3 Vsual geometry 1/2: Camera models and trangulaton 4 Vsual geometry 2/2: Reconstructon from multple vews AIMS Computer vson
More informationMOTION BLUR ESTIMATION AT CORNERS
Gacomo Boracch and Vncenzo Caglot Dpartmento d Elettronca e Informazone, Poltecnco d Mlano, Va Ponzo, 34/5-20133 MILANO boracch@elet.polm.t, caglot@elet.polm.t Keywords: Abstract: Pont Spread Functon Parameter
More informationReal-time Joint Tracking of a Hand Manipulating an Object from RGB-D Input
Real-tme Jont Tracng of a Hand Manpulatng an Object from RGB-D Input Srnath Srdhar 1 Franzsa Mueller 1 Mchael Zollhöfer 1 Dan Casas 1 Antt Oulasvrta 2 Chrstan Theobalt 1 1 Max Planc Insttute for Informatcs
More informationMultiple Frame Motion Inference Using Belief Propagation
Multple Frame Moton Inference Usng Belef Propagaton Jang Gao Janbo Sh The Robotcs Insttute Department of Computer and Informaton Scence Carnege Mellon Unversty Unversty of Pennsylvana Pttsburgh, PA 53
More informationProf. Feng Liu. Spring /24/2017
Prof. Feng Lu Sprng 2017 ttp://www.cs.pd.edu/~flu/courses/cs510/ 05/24/2017 Last me Compostng and Mattng 2 oday Vdeo Stablzaton Vdeo stablzaton ppelne 3 Orson Welles, ouc of Evl, 1958 4 Images courtesy
More information3D vector computer graphics
3D vector computer graphcs Paolo Varagnolo: freelance engneer Padova Aprl 2016 Prvate Practce ----------------------------------- 1. Introducton Vector 3D model representaton n computer graphcs requres
More informationSum of Linear and Fractional Multiobjective Programming Problem under Fuzzy Rules Constraints
Australan Journal of Basc and Appled Scences, 2(4): 1204-1208, 2008 ISSN 1991-8178 Sum of Lnear and Fractonal Multobjectve Programmng Problem under Fuzzy Rules Constrants 1 2 Sanjay Jan and Kalash Lachhwan
More informationGeometric Primitive Refinement for Structured Light Cameras
Self Archve Verson Cte ths artcle as: Fuersattel, P., Placht, S., Maer, A. Ress, C - Geometrc Prmtve Refnement for Structured Lght Cameras. Machne Vson and Applcatons 2018) 29: 313. Geometrc Prmtve Refnement
More informationLecture 13: High-dimensional Images
Lec : Hgh-dmensonal Images Grayscale Images Lecture : Hgh-dmensonal Images Math 90 Prof. Todd Wttman The Ctadel A grayscale mage s an nteger-valued D matrx. An 8-bt mage takes on values between 0 and 55.
More informationModel-Based Bundle Adjustment to Face Modeling
Model-Based Bundle Adjustment to Face Modelng Oscar K. Au Ivor W. sang Shrley Y. Wong oscarau@cs.ust.hk vor@cs.ust.hk shrleyw@cs.ust.hk he Hong Kong Unversty of Scence and echnology Realstc facal synthess
More informationVideo Object Tracking Based On Extended Active Shape Models With Color Information
CGIV'2002: he Frst Frst European Conference Colour on Colour n Graphcs, Imagng, and Vson Vdeo Object rackng Based On Extended Actve Shape Models Wth Color Informaton A. Koschan, S.K. Kang, J.K. Pak, B.
More informationSelf-Calibration from Image Triplets. 1 Robotics Research Group, Department of Engineering Science, Oxford University, England
Self-Calbraton from Image Trplets Martn Armstrong 1, Andrew Zsserman 1 and Rchard Hartley 2 1 Robotcs Research Group, Department of Engneerng Scence, Oxford Unversty, England 2 The General Electrc Corporate
More informationSimultaneous Object Pose and Velocity Computation Using a Single View from a Rolling Shutter Camera
Smultaneous Object Pose and Velocty Computaton Usng a Sngle Vew from a Rollng Shutter Camera Omar At-Ader, Ncolas Andreff, Jean Marc Lavest, and Phlppe Martnet Unversté Blase Pascal Clermont Ferrand, LASMEA
More informationSteps for Computing the Dissimilarity, Entropy, Herfindahl-Hirschman and. Accessibility (Gravity with Competition) Indices
Steps for Computng the Dssmlarty, Entropy, Herfndahl-Hrschman and Accessblty (Gravty wth Competton) Indces I. Dssmlarty Index Measurement: The followng formula can be used to measure the evenness between
More informationCorner-Based Image Alignment using Pyramid Structure with Gradient Vector Similarity
Journal of Sgnal and Informaton Processng, 013, 4, 114-119 do:10.436/jsp.013.43b00 Publshed Onlne August 013 (http://www.scrp.org/journal/jsp) Corner-Based Image Algnment usng Pyramd Structure wth Gradent
More informationLong-Term Moving Object Segmentation and Tracking Using Spatio-Temporal Consistency
Long-Term Movng Obect Segmentaton Trackng Usng Spato-Temporal Consstency D Zhong Shh-Fu Chang {dzhong, sfchang}@ee.columba.edu Department of Electrcal Engneerng, Columba Unversty, NY, USA Abstract The
More informationSolving two-person zero-sum game by Matlab
Appled Mechancs and Materals Onlne: 2011-02-02 ISSN: 1662-7482, Vols. 50-51, pp 262-265 do:10.4028/www.scentfc.net/amm.50-51.262 2011 Trans Tech Publcatons, Swtzerland Solvng two-person zero-sum game by
More informationUAV global pose estimation by matching forward-looking aerial images with satellite images
The 2009 IEEE/RSJ Internatonal Conference on Intellgent Robots and Systems October -5, 2009 St. Lous, USA UAV global pose estmaton by matchng forward-lookng aeral mages wth satellte mages Kl-Ho Son, Youngbae
More informationActive Contours/Snakes
Actve Contours/Snakes Erkut Erdem Acknowledgement: The sldes are adapted from the sldes prepared by K. Grauman of Unversty of Texas at Austn Fttng: Edges vs. boundares Edges useful sgnal to ndcate occludng
More informationFinding Intrinsic and Extrinsic Viewing Parameters from a Single Realist Painting
Fndng Intrnsc and Extrnsc Vewng Parameters from a Sngle Realst Pantng Tadeusz Jordan 1, Davd G. Stork,3, Wa L. Khoo 1, and Zhgang Zhu 1 1 CUNY Cty College, Department of Computer Scence, Convent Avenue
More informationA 3D Human Skeletonization Algorithm for a Single Monocular Camera Based on Spatial Temporal Discrete Shadow Integration
appled scences Artcle A 3D Human Skeletonzaton Algorthm for a Sngle Monocular Camera Based on Spatal Temporal Dscrete Shadow Integraton Je Hou *, Baolong Guo, Wangpeng He and Jnfu Wu School of Aerospace
More informationFitting: Deformable contours April 26 th, 2018
4/6/08 Fttng: Deformable contours Aprl 6 th, 08 Yong Jae Lee UC Davs Recap so far: Groupng and Fttng Goal: move from array of pxel values (or flter outputs) to a collecton of regons, objects, and shapes.
More informationSuppression for Luminance Difference of Stereo Image-Pair Based on Improved Histogram Equalization
Suppresson for Lumnance Dfference of Stereo Image-Par Based on Improved Hstogram Equalzaton Zhao Llng,, Zheng Yuhu 3, Sun Quansen, Xa Deshen School of Computer Scence and Technology, NJUST, Nanjng, Chna.School
More information