Real-time Feature-based Video Mosaicing at 500 fps

Transcription

1 213 IEEE/RSJ Internatonal Conference on Intellgent Robots and Systems (IROS) November 3-7, 213. Tokyo, Japan Real-tme Feature-based Vdeo Mosacng at 5 fps Ken-ch Okumura, Sushl Raut, Qngy Gu, Tadayosh Aoyama, Takesh Takak, and Idaku Ish Abstract We conducted hgh-frame-rate (HFR) vdeo mosacng for real-tme synthess of a panoramc mage by mplementng an mproved feature-based vdeo mosacng algorthm on a feld-programmable gate array (FPGA)-based hgh-speed vson platform. In the mplementaton of the mosacng algorthm, feature pont extracton was accelerated by mplementng a parallel processng crcut module for Harrs corner detecton n the FPGA on the hgh-speed vson platform. Feature pont correspondence matchng can be executed for hundreds of selected feature ponts n the current frame by searchng those n the prevous frame n ther neghbor ranges, assumng that frame-to-frame mage dsplacement becomes consderably smaller n HFR vson. The system we developed can mosac mages at 5 fps as a sngle syntheszed mage n real tme by sttchng the mages based on ther estmated frame-to-frame changes n dsplacement and orentaton. The results of an experment conducted, n whch an outdoor scene was captured usng a hand-held camera-head that was quckly moved by hand, verfy the performance of our system. I. INTRODUCTION Vdeo mosacng s a vdeo processng technque n whch multple mages are merged nto a composte mage that covers a larger, more seamless vew than the feld of vew of the camera. Vdeo mosacng has been used to generate panoramc pctures n many applcatons such as trackng, survellance, and augmented realty. Most vdeo mosacng methods [1 determne the transform parameters between mages by calculatng feature ponts usng feature extractors such as the KLT tracker [2 and SIFT features [3 to match feature pont correspondences between mages. Many realtme vdeo mosacng approaches for fast mage regstraton have been proposed. Kourog et al. proposed a fast mage regstraton method that uses pseudo moton vectors estmated as gradent-based optcal flow [4. Cvera et al. developed a drft-free vdeo mosacng system that obtans a consstent mage from mages at 3 fps usng an EKF-SLAM approach when prevously vewed scenes are revsted [5. de Souza et al. performed real-tme vdeo mosacng wthout over-deformaton of mosacs usng a non-rgd deformaton model [6. Most of these proposals only apply to standard vdeos at relatvely low frame rates (e.g., NTSC 3 fps). Further, the camera has to be operated slowly, wthout any large dsplacement between frames, n order to prevent dffcultes n matchng feature pont correspondences. However, n scenaros nvolvng rapd camera moton, vdeo mages at such low frame rates are nsuffcent for trackng feature ponts n the mages. Thus, there s a demand for real-tme K. Okumura, S. Raut, Q. Gu, T. Aoyama, T. Takak, and I. Ish. are wth Hroshma Unversty, Hroshma , Japan (correspondng author (I. Ish) Tel: ; e-mal: okumura@robotcs.hroshma-u.ac.jp). hgh-frame-rate (HFR) vdeo mosacng at rates of several hundred frames per second and greater. Many feld-programmable gate array (FPGA)-based HFR vson systems have been developed for real-tme vdeo processng at hundreds of frames per second and greater [7 [9. Although HFR vson systems can smultaneously calculate scalar mage features, they cannot transfer the entre mage at hgh speed to a personal computer (PC) owng to the lmtatons n the transfer speed of ts nner bus. We recently developed IDP Express [1, an HFR vson platform that can smultaneously process an HFR vdeo and drectly map t onto memory allocated n a PC. Implementng a realtme functon to extract feature ponts n mages and match ther correspondences between frames on such an FPGAbased hgh-speed vson platform would make t possble to accelerate vdeo mosacng at a hgher frame rate than 3 fps, even when the camera moves quckly. In ths study, we developed a real-tme vdeo mosacng system that operates mages at 5 fps. The system can smultaneously extract and enable correspondence between feature ponts n mages for estmaton of frameto-frame dsplacement, and sttch the captured mages nto a panoramc mage to gve a seamless wder feld of vew. II. ALGORITHM Most vdeo mosacng algorthms are realzed by executng the followng processes [1: (1) feature pont detecton, (2) feature pont matchng, (3) transform estmaton, and (4) mage composton. In many applcatons, mage composton at dozens of frames per second s suffcent for the human eye to montor a panoramc mage. However, feature trackng at dozens of frames per second s not always accurate when the camera s movng rapdly, because most feature ponts are mstracked when mage dsplacements between frames become large. In HFR vson, frame-to-frame dsplacement n feature ponts becomes consderably smaller. Narrowng the search range by assumng HFRs reduces the naccuracy and computatonal load needed to match feature ponts between frames even when hundreds of feature ponts are observed from a rapdly movng camera, whereas mage composton at an HFR requres a heavy computatonal load. We ntroduce a real-tme vdeo mosacng algorthm that can resolve ths trade-off between trackng accuracy and computatonal load n vdeo mosacng for rapd camera moton. In our algorthm, steps (1) (3) are executed at an HFR on an HFR vson platform, and step (4) s executed at dozens of frames per second to enable the human eye to montor the panoramc mage. In ths study, we focus on acceleraton of steps (1) (3) for HFR feature pont trackng /13/$ IEEE 2665

2 at hundreds of frames per second or more on the bass of the followng concepts: (a) Feature pont detecton accelerated by hardware logc Generally, feature pont detecton requres pxel-level computaton of the order O(N 2 ), where N 2 s the mage sze. Gradent-based feature pont detecton s sutable for acceleraton by hardware logc because the local calculaton of brghtness gradents can be easly parallelzed as pxellevel computaton. In ths study, we accelerate gradent-based feature pont detecton by mplementng ts parallel crcut logc on an FPGA-based HFR vson platform. (b) Reducton n the number of feature ponts Thousands of feature ponts are not always requred for estmatng transform parameters between mages n vdeo mosacng. The computatonal load can be reduced by selectng a smaller number of feature ponts because featurelevel computaton of the order O(M 2 ) s generally requred n feature pont matchng; M s the number of selected feature ponts. In ths study, the number of feature ponts used n feature pont matchng s reduced by excludng closely crowded feature ponts as they often generate correspondng errors between mages n feature pont matchng. (c) Narrowed search range by assumng HFR vdeo Feature pont matchng stll requres heavy computaton of the order O(M 2 ) f t s requred that all the feature ponts correspond to each other between frames, even when a reduced number of feature ponts are used. In an HFR vdeo, t can be assumed that frame-to-frame mage dsplacement grows consderably smaller, whch allows a smaller search range to be used for ponts correspondng to ponts n the precedng frame. Ths narrowed search range reduces the computatonal load of feature pont matchng to correspondng feature ponts between frames on the order of O(M). Our algorthm for real-tme HFR vdeo mosacng conssts of the followng processes: (1) Feature pont detecton (1-a) Feature extracton To extract feature ponts such as upper-left vertexes, we use the followng brghtness gradent matrx C(x,: C(x, = [ I x 2 (x, I x(x,i y(x, I x(x,i y(x, I y 2, (1) (x, x N a (x) where N a (x) s the a a adjacent area of pxel x = (x,y), and I x(x, and I y(x, ndcate the followng postve values of I x (x, and I y (x,, respectvely: { I ξ(x, Iξ (x, (I = ξ (x, > ) (ξ = x,y), (2) (otherwse) where I x (x, and I y (x, are x and y dfferentals of the nput mage I(x, at pxel x at tme t. λ(x, s defned as a feature for feature trackng usng the Harrs corner detecton [11 as follows: λ(x, = det C(x, κ(tr C(x,) 2. (3) Here, κ s a tunable senstve parameter, and values n the range.4.15 have been reported as feasble. (1-b) Feature pont detecton Thresholdng s conducted for λ(x, wth a threshold λ T to obtan a map of feature ponts, R(x,, as follows: { 1 (λ(x, > λt ) R(x, =. (4) (otherwse) The number of feature ponts when R(x, = 1 s counted n the p p adjacent area of x as follows: P(x, = R(x,, (5) x N p (x) where P(x, ndcates the densty of the feature ponts. (1-c) Selecton of feature ponts To reduce the number of feature ponts, closely crowded feature ponts are excluded by countng the number of feature ponts n the neghborhood. The reduced set of feature ponts R ( s calculated at tme t as follows: R ( = {x P(x, P }, (6) where P s a threshold used to sparsely select feature ponts. We assume that the number of feature ponts s less than M. (2) Feature pont matchng (2-a) Template matchng To enable correspondence between feature ponts at the current tme t and those at the prevous tme t p = t n t, template matchng s conducted for all the selected feature ponts n an mage. t s the shortest frame nterval of a vson system, and n s an nteger. For template matchng to enable the correspondence of the -th feature pont at tme t p belongng to R (t p ), x (t p ) (1 M), to the -th feature pont at tme t belongng to R (, x ( (1 M), the sum of absolute dfference (SAD) s calculated as follows: E(,;t,t p ) = I(x (+ξ, I(x (t p )+ξ,t p ), (7) ξ=(ξ,η) W m where W m s an m m wndow of template matchng. To reduce the number of msmatched ponts, ˆx(x (t p );, whch ndcates the feature pont at tme t correspondng to the -th feature pont x (t p ) at tme t p, and ˆx(x (;t p ), whch ndcates the feature pont at tme t p correspondng to the -th feature pont x ( at tme t, are bdrectonally searched by selectng the feature ponts when E(,;t,t p ) s at a mnmum n ther adjacent areas as follows: ˆx(x (t p ); = x ()( = arg mn E(,;t,t p ), (8) x ( N b (x (t p)) ˆx(x (;t p ) = x ( )(t p ) = arg mn E(,;t,t p ), (9) x (t p ) N b (x () where () and ( ) are determned as the ndex numbers of the feature pont at tme t, correspondng to x (t p ), and the feature pont at tme t p, correspondng to x (, respectvely. The par of feature ponts between tme t and t p are selected when the correspondng feature ponts are mutually selected as the same ponts as follows: {ˆx(x (t x(x (t p ); = p ); ( = ( ())) (otherwse), (1) 2666

3 where ndcates that no feature pont at tme t corresponds to the -th feature pont x (t p ) at tme t p, and the feature ponts at tme t that have no correspondng pont at tme t p, are adjudged to be newly appearng feature ponts. We assume that the mage dsplacement between tmes t and t p s small, and the feature pont x ( at tme t can be matched wth a feature pont at tme t p n the b b adjacent area of x (. The processes descrbed n Eqs. (8) (1) are conducted for all the feature ponts belongng to R (t p ) and R (, and we can reduce the computatonal load of feature pont matchng n the order of O(M) by settng a narrowed search range wth a small value for b. (2-b) Frame nterval selecton To avod accumulated regstraton errors caused by small frame-to-frame dsplacements, the frame nterval of feature pont matchng s adjusted for feature pont matchng at tme t + t usng the followng averaged dstance among the correspondng feature ponts at tmes t and t p : x(x (t p ); x (t p ) d(t;t p ) = x (t p ) Q(t;t p ) S(Q(t;t p )), (11) where Q(t;t p ) s a set of the feature ponts x (t p ) at tme t p that satsfy x(x (t p ); ( = 1,,M). S(Q(t;t p )) s the number of elements belongng to Q(t;t p ). Dependng on whether d(t;t p ) s larger than d, n = n( s determned for feature pont matchng at tme t+ t; the frame nterval of feature pont matchng s ntally set to t. (2-b-) d(t;t p ) d, The mage dsplacement between tmes t and t p = t p ( s too small for feature pont matchng. To ncrease the frameto-frame dsplacement at the next tme t+ t, the parameter n(t+ s ncremented by one as follows: n(t+ = n(+1. (12) Ths ndcates that the feature ponts at tme t + t are matched wth those at tme t p (t+ = t n( t. Wthout conductng any process n steps (3) and (4) below, return to step (1) for the next tme nterval t+ t. (2-b-) d(t;t p ) > d To reset the frame-to-frame dsplacement at tme t+ t, n(t+ s set to one as follows: n(t+ = 1. (13) Ths ndcates that the feature ponts at tme t + t are matched wth those at tme t p (t+ = t. Thereafter, go to steps (3) and (4); the selected pars of feature ponts, x ( ( Q(t;t p )) and x(x (;t p ), are used n steps (3) and (4). (3) Transform estmaton Usng the selected pars of feature ponts, affne parameters between the two mages at tmes t and t p are estmated as follows: x T ( = [ a1 a 2 a 3 a 4 [ x T a5 (x (;t p )+ a 6, (14) = A(t;t p ) x T (x (;t p )+b T (t;t p ), (15) where a j (j = 1,,6) are components of the transform matrxa(t;t p ) and translaton vectorb(t;t p ) that express the affne transform relatonshp between the two mages at tmes t and t p. In ths study, they are estmated by mnmzng the followng Tukey s bweght evaluaton functons, E 1 and E 2 : E l = w l d 2 l (l = 1,2). (16) x (t p ) Q(t;t p ) Devatons d 1 and d 2 are gven as follows: d 1 = x (a 1 x +a 2 y +a 5 ), (17) d 2 = y (a 3 x +a 4 y +a 6 ), (18) where x(x (;t p ) = (x,y ) and x ( = (x,y ) ndcate the temporally estmated locaton of the -th feature pont at tme t n Tukey s bweght method. The followng processes are teratvely executed to reduce estmaton errors caused by dstantly msmatched pars of feature ponts. Here, the weghts w 1 and w 2 are set to one and x ( set to x ( as ther ntal values. (3-a) Affne parameter estmaton On the bass of the least squares method to mnmze E 1 and E 2, affne parameters a j (j = 1,,6) are estmated by calculatng the weghted product sums of the xy coordnates of the selected feature ponts as follows: 1 w 1 x 2 w 1 x a y w 1 x w 1 x x 1 a 2 = w 1 x y w 1 y 2 w 1 y w 1 x y a 5 w 1 x w 1 y, (19) w 1 w 1 x a 3 a 4 a 6 1 w 2 x 2 w 2 x y w 2 x w 2 y x = w 2 x y w 2 y 2 w 2 y w 2 y y w 2 x w 2 y. (2) w 2 w 2 y Usng the affne parameters estmated n Eqs. (19) and (2), the temporally estmated locatons of feature ponts can be updated as follows: x T ( = [ a1 a 2 a 3 a 4 [ x T a5 (x (;t p )+ a 6. (21) (3-b) Updatng weghts To reduce the errors caused by dstantly msmatched pars of feature ponts, w 1 and w 2 are updated for the -th par of feature ponts usng the devatons d 1 and d 2 as follows: ( [ w1 ( = w 2 ) 2 1 d2 1 Wu 2 ) 2 1 d2 2 Wu 2 [ (d 1 W u,d 2 W u ). (22) (otherwse) W u s a parameter used to determne the cut pont for evaluaton functons at u tme teraton. As the number of 2667

4 teratons u s larger, W u s set to a smaller value. Increment u by one and return to step (3-a) durng u U. After U tme teraton, the affne parameters between two nput mages at tmes t and t p are determned, and the affne transform matrx and vector are obtaned. A(t;t p ) and b(t;t p ) are accumulated wth the affne parameters estmated at tme t p as follows: A( = A(t;t p )A(t p ), (23) b T ( = A(t p )b T (t;t p )+b T (t p ), (24) where A( and b( ndcate the affne parameters of the nput mage at tme t, compared wth the nput mage at tme ; A() and b() are ntally gven as the unt matrx and zero vector, respectvely. (4) Image composton To montor the panoramc mage at ntervals of t, the panoramc mage G(x, s updated by attachng an affnetransformed nput mage at tme t to the panoramc mage G(x,t t ) at tme t t as follows: { G(x, = I(x, (f I(x, ) G(x,t t ) (otherwse), (25) where the panoramc mage G(x, s ntally gven as an empty mage at tme ; x ndcates the affne-transformed coordnate vector at tme t as follows: x T = A(x T +b T (. (26) Our vdeo mosacng algorthm can be effcently executed as a mult-rate vdeo processng method. The nterval n steps (1) and (2) can be set to the shortest frame nterval t for accurate feature pont trackng n HFR vson, whle that n step (3) depends on the frame-to-frame mage dsplacement; t s the shortest frame nterval when there s sgnfcant camera moton, and t becomes larger when the camera moton becomes smaller. On the other hand, the nterval n step (4), t, can be ndependently set to dozens of mllsecond to enable the human eye to montor the panoramc mage. III. SYSTEM IMPLEMENTATION A. Hgh-speed vson platform: IDP Express To realze real-tme vdeo mosacng at an HFR, our vdeo-mosacng method was mplemented on the hgh-speed vson platform IDP Express [1. The mplemented system conssted of a camera head, a dedcated FPGA board (IDP Express board), and a PC. The camera head was hghly compact and could easly be held by a human hand. Its dmensons and weght were mm and 145 g, respectvely. On the camera head, eght-bt RGB mages of pxels could be captured at 2 fps wth a Bayer flter on ts CMOS mage sensor. The IDP Express board s desgned for hgh-speed processng and recordng of mages transferred from the camera head at 2 fps. On the IDP Express board, hardware logc can be mplemented on a user-specfed FPGA (Xlnx XC3S5-4FG9) for acceleraton of mage processng algorthms. The nput mages, and the processed results on the color nput mage F( x, 8btx4 Fg. 1. color-to-gray convertor 4 parallel gray mage I( x, 8btx4 Feature Pont Extracton Crcut Module Harrs Corner Detector Submodule multplcaton & summaton gradent matrx C I' 2 x 2 I' y I ' I' x y 2bt 3 4 trace & determnant Tr C 36bt 4 det C 35btx4 multplcaton & subtracton feature λ( x, 35btx4 thresholdng submodule 4 parallel 4 parallel 16 parallel 4 parallel feature pont map B( x, P( x, 1btx4 feature pont counter 4 parallel number of feature ponts 8btx4 color nput mage F( x, 8btx4 Schematc data flow of a feature pont extracton crcut module. IDP Express board, can be memory-mapped va 16-lane PCIe buses n real tme at 2 fps onto the allocated memores n the PC, and arbtrary processng for the executon of software on the PC can be programmed wth an ASUSTek P5E manboard, Core 2 Quad93 bulk 2.5 GHz CPU, 4 GB memory, and a Wndows XP Professonal 32-bt OS. B. Implemented hardware logc To accelerate steps (1-a) and (1-b) n our vdeo mosacng method, we desgned a feature pont extracton crcut module n the user-specfc FPGA on the IDP Express; steps (1-a) and (1-b) have a computatonal complexty of O(N 2 ). The other processes (steps (1-c), (2), (3), and (4)) were mplemented n software on the PC because, wth ther computatonal complexty of O(M), they can be accelerated by reducng the number of feature ponts. Fgure 1 s a schematc data flow of the feature pont extracton crcut module mplemented n the FPGA on the IDP Express board. It conssts of a color-to-gray converter submodule, a Harrs feature extractor submodule, and a feature pont counter submodule. Raw eght-bt nput color mages wth a Bayer flter, F(x,, are scanned n unts of four pxels from the upper left to the lower rght usng X and Y sgnals at MHz. The color-togray converter submodule converts RGB mages nto eghtbt gray-level mages I(x, n parallel wth the four pxels after RGB converson for nput mages wth a Bayer flter. In the Harrs feature detector submodule, I x and I y are calculated usng 3 3 Prewtt operators, and I x 2, I xi y, and I y 2 n the adjacent 3 3 pxels (a = 3) are calculated as 2-bt data usng 16 adders and 12 multplers n parallel n unts of four pxels at MHz. The feature λ(x, s calculated as 35-bt data by subtractng det C(x, wth a three-bt shft value of (Tr C(x,) 2 n unts of four pxels when κ =.625; 16 multplers, 24 adders, and four threebt shfters are mplemented for calculatng λ(x,. The feature pont counter submodule can obtan the postons of feature ponts as a bnary map B(x, by thresholdng λ(x, wth a threshold λ T n parallel n unts of four pxels at MHz; B(x, ndcates whether a feature pont s located at pxel x at tme t. The number of feature ponts n the 5 5 adjacent area (p = 5) s counted for all the feature ponts as a map P(x, usng 96 adders n parallel n unts of four pxels; P(x, s used for checkng closely crowded feature ponts n step (1-c). 2668

5 The color nput mage F(x, and the number of feature ponts P(x, are outputted to FIFO memory for an external PC wth X and Y sgnals. The delay tme n outputtng F(x, and P(x, usng the feature pont extracton crcut module s 45 clocks (1 clock = 13.2 ns) after the raster scannng has been performed, whle the delay tme n outputtng them to the PC s one frame (.5 ms). C. Specfcaton We mplemented steps (1-a) and (1-b) of the above crcut module n the FPGA (Xlnx XC3S5-4FGG9) on the IDP Express board. The resources consumed by the FPGA are lsted n Table I. The extracted feature ponts can be outputted to the PC for mages at 2 fps. Steps (1-c), (2), (3), and (4) were mplemented n software on the PC, and the followng parameters were used n ths study. The threshold P n step (1-c) was determned n order to reduce the number of feature ponts (M 3), dependng on the expermental scene. Step (2-a) executed 5 5 (m = 5) template matchng wth bdrectonal search n the adjacent area (b = 31). In step (2-b), the frame nterval of feature pont matchng was determned wth d = 7. Step (3) executed Tukey s bweght method wth L = 1 teraton; the parameters for the bweght evaluaton functons were set at W l = 11 l (l = 1,,1). Table II summarzes the executon tme taken for vdeo mosacng. The executon tmes of steps (1-a) and (1-b) nclude the mage acquston tme for a mage on the FPGA board. The total executon tme of steps (1), (2), and (3) s less than t = 2 ms; t was worst when step (3) was executed at all the frames. The executon tme of step (4) was much larger than that of the other steps. Step (4) was mplemented as a mult-threaded process that was not able to dsturb the real-tme processes of the other steps when ts nterval was set to t = 6 ms for montorng by the human eye. We confrmed that vdeo mosacng can be executed for mages n real tme at 5 fps, ncludng feature pont extracton and trackng for M 3. TABLE I FPGA RESOURCE CONSUMPTION. Devce Type Xlnx XC3S5 Slce 5,864/33,28 (17%) Slce Flp Flop 9,15/66,56 (13%) 4 nput LUT 6,38/66,56 (9%) Bounded IOB 195/633 (3%) Block RAM 15/14 (14%) MULT18X18s 28/14 (26%) GCLK 2/8 (25%) TABLE II EXECUTION TIMES. tme [ms (1-a,1-b) Feature extracton / Thresholdng@.86 (1-c) Selecton of feature ponts.5 (2-a) Template matchng.97 (2-b) Frame nterval selecton.1 (3) Affne transform estmaton.9 (4) Image composton Total ((1) (3)) 1.98 nput mage extracted feature ponts Fg. 2. Expermental scene. t =.2 s t =.4 s t =.6 s t =.8 s t = 1. s Fg [pxel x pos on 24 [pxel 12 y pos on 9 Input mages and extracted feature ponts. 6 [ms nterval of feature pont matchng me [s Fg. 4. Frame nterval of feature pont matchng, and xy coordnates of translaton vector. IV. EXPERIMENT In ths secton, we present the expermental results of realtme HFR vdeo mosacng for an outdoor scene captured usng a camera head moved quckly and perodcally by a human hand. Fgure 2 shows the expermental outdoor scene. It was captured from the top of an eght-story buldng n Hroshma Unversty by an operator who manually moved the hand-held camera head wth hs hand from left to rght wth perodc up-and-down motons approxmately fve tmes per three second, and then decelerated the camera s moton. The frame rate and exposure tme of the IDP Express system were set to 5 fps and.25 ms, respectvely. To select 3 feature ponts or less from a mage, the threshold parameters were set to λ T = and P = 25. Fgure 3 shows the fve-nput mage sequence and the extracted feature ponts, taken at ntervals of.2 s. The affne 2669

6 t=.s t=.5s t=1.s t=1.5s t=2.s t=2.5s t=3.s t=3.5s t=4.s t=4.5s Fg. 5. Syntheszed panoramc mages matrx and translaton vector were set as unt matrx and zero vector, respectvely, at t =. It can be seen that most of the feature ponts n the nput mages were correctly extracted. Fgure 4 shows the frame nterval of feature pont matchng, and the x- and y-coordnates for translaton vector b( for t =. 5. s. Correspondng to the camera s moton, t can be seen that a short frame nterval was set for the hgh-speed scene, and a long frame nterval for the low-speed scene. Fgure 5 shows the syntheszed panoramc mages of pxels, taken at ntervals of.5 s. Wth tme, the panoramc mage was extended by accurately sttchng affne transformed nput mages over the panoramc mage at the prevous frame, and large buldngs and background forests on the far sde were observed n a sngle panoramc mage at t = 4.5 s. These fgures ndcate that our system can accurately generate a sngle syntheszed mage n real tme by sttchng nput mages at 5 fps when the camera head s quckly moved by a human hand. V. CONCLUSION We developed a hgh-speed vdeo mosacng system that can sttch mages at 5 fps to gve a sngle syntheszed panoramc mage n real tme. On the bass of the expermental results, we plan to mprove our vdeo mosacng system for more robust and long-tme vdeo mosacng, and to extend the applcaton of ths system to vdeo-based SLAM and other survellance technologes n the real world. REFERENCES [1 B. Ztova and J. Flusser, Image regstraton methods: A survey, Image Vs. Comput., 21, 977 1, 23. [2 J. Sh and C. Tomas, Good features to track, Proc. IEEE Conf. Comput. Vs. Patt. Recog., 593-6, [3 D.G. Lowe, Dstnctve mage features from scale-nvarant keyponts, Int. J. Comput. Vs., 6, 91 11, 24. [4 M. Kourog, T. Kurata, J. Hoshno, and Y. Muraoka, Real-tme mage mosacng from a vdeo sequence, Proc. Int. Conf. Image Proc., , [5 J. Cvera, A.J. Davson, J.A. Magallon, and J.M.M. Montel, Drftfree real-tme sequental mosacng, Int. J. Comput. Vs., 81, , 29. [6 R.H.C. de Souza, M. Okutom, and A. Tor, Real-tme mage mosacng usng non-rgd regstraton, Proc. 5th Pacfc Rm Conf. on Advances n Image and Vdeo Technology, , 211. [7 Y. Watanabe, T. Komuro, and M. Ishkawa, 955-fps real-tme shape measurement of a movng/deformng object usng hgh-speed vson for numerous-pont analyss, Proc. IEEE Int. Conf. Robot. Autom., , 27. [8 S. Hra, M. Zakoj, A. Masubuch, and T. Tsubo, Realtme FPGAbased vson system, J. Robot. Mechat., 17, 41 49, 25. [9 I. Ish, R. Sukenobe, T. Tanguch, and K. Yamamoto, Development of hgh-speed and real-tme vson platform, H 3 Vson, Proc. IEEE/RSJ Int. Conf. Intell. Rob. Sys., , 29. [1 I. Ish, T. Tatebe, Q. Gu, Y. Morue, T. Takak, and K. Tajma, 2 fps real-tme vson system wth hgh-frame-rate vdeo recordng, Proc. IEEE Int. Conf. Robot. Autom., , 21. [11 C. Harrs and M. Stephens, A combned corner and edge detector, Proc. the 4th Alvey Vs. Conf., ,