A new algorithm for small object tracking based on super-resolution technique

Transcription

1 A new algorihm for small objec racking based on super-resoluion echnique Yabunayya Habibi, Dwi Rana Sulisyaningrum, and Budi Seiyono Ciaion: AIP Conference Proceedings 1867, (2017); doi: / View online: hps://doi.org/ / View Table of Conens: hp://aip.sciaion.org/oc/apc/1867/1 Published by he American Insiue of Physics Aricles you may be ineresed in Be-safe ravel, a web-based geographic applicaion o explore safe-roue in an area AIP Conference Proceedings 1867, (2017); / Consrained H conrol for low bandwidh acive suspensions AIP Conference Proceedings 1867, (2017); / A characerizaion of S-prime submodules of a free module over a principal ideal domain AIP Conference Proceedings 1867, (2017); / Caegorizing documen by fuzzy C-Means and K-neares neighbors approach AIP Conference Proceedings 1867, (2017); / D muliplayer virual pes game using Google Card Board AIP Conference Proceedings 1867, (2017); / Implemenaion of he common phrase index mehod on he phrase query for informaion rerieval AIP Conference Proceedings 1867, (2017); /

2 A New Algorihm for Small Objec Tracking Based on Super-Resoluion Technique Yabunayya Habibi 1, a), Dwi Rana Sulisyaningrum 1, b) 1, c) and Budi Seiyono 1 Mahemaics Deparemen, Insiue of Technology Sepuluh Nopember, Jalan Raya ITS, kepuih, Sukolilo, Surabaya, Jawa imur 60111, Indonesia. a) Corresponding auhor: habibiexebpp@gmail.com b) dwrana@gmail.com c) masbudiseiyono@gmail.com Absrac. Objec racking in a video is a problem of esimaing he rajecory of an objec in he image plane as i moves around a scene. In general, objec racking is a quie complicaed problem. Difficulies in objec racking occur due o some consrains or condiions such as objec moion, changing appearance paerns, non-rigid objec srucures, occlusions, and camera moion. Level of problems would be higher if he objec racking has relaively small. If i happens, an objec will be difficul o idenify and racking becomes less precision because small objec has lile informaion. In order o overcome hese problem, he racking will be inegraed wih super-resoluion where a high-resoluion image will be buil from several low-resoluion image. In his research, racking of moving objec using adapive paricle filer which adapive moion model is applied o ge beer proposal disribuion approach. The simulaion shows ha racking inegraion wih superresoluion significanly increase he accuracy of small objec racking. INTRODUCTION Video surveillance is a opic of research in compuer vision ha ries o deec, recognize and rack objecs from a sequence of images. Objec racking is an imporan ask in many compuer vision applicaions such as surveillance, vehicle navigaion, and auonomous robo navigaion. Objec racking in a video is a problem of esimaing he rajecory of an objec in he image plane as i moves around a scene. One of he advanages in he miliary field is he uilizing on racking as video surveillance for naional parol such as guard in he border area or around he base hen deecion and racking of he moving enemy on he balefield or auomaic conrol sysem o rack he arge or he arge posiion wih a higher precision in firing missiles. In oher filed such as spors, racking of moving objecs performed o deermine he movemen and posiion players on he field so ha a spors analys can deermine he paern of he game from players and uilizaion of Hawk-eye is used o rack he posiion of he ball so i will grealy assis he ask of linesmen. There are many hings o do in racking process bu i depends on he level whom user need for rerieving informaion [1]. In general, racking is a problem ha is quie complicaed. This problem can arise due o he movemen of objecs quickly, occlusion or he objec invisible, noise in image sequences, he srucure of non-rigid objecs such as objec roaing and changing he scale, and he camera moves o follow he movemen of objecs and more [2]. One of he more difficul problems will occur if objec racking is performed a considerable disance, so a objecs look relaively small such as racking of paragliding or plane movemen, balls in spors like soccer video. If his case happen, he racking process become less precision because small objec isn enough informaion o be idenified and regarded as noise [3]. One way o overcome hese problems and improve informaion of he small objec is inegraion a super- Inernaional Conference on Mahemaics: Pure, Applied and Compuaion AIP Conf. Proc. 1867, ; doi: / Published by AIP Publishing /$

3 resoluion echnique in racking process. Super-resoluion is a process of high-resoluion images ha are buil from several low-resoluion image [4][5]. Super-resoluion will improve he visual qualiy of small objecs so ha objecs has more informaion o be racked. Some earlier relaed research abou racking and super-resoluion is a sudy from Sun [5]. Sun used emplae maching for racking and super resoluion o enhance image qualiy by reconsrucing Projecion Ono Convex Ses (POCs). However, final in his research uilized racking resul as enhance image on super-resoluion o improvemen visual qualiy. Inegraion beween super-resoluion and racking has been esablished by Mise and Breckon [6] wih a super-resoluion imaging approach based on combinaion of he Sum of Absolu Diff erences (SAD) and gradien descen. The research used for improve arge appearance ha assiss he overall racking on high dynamic scene. Then, earlier research abou small objec racking has been proposed by Davieshy [7]. This paper will presen our approach ha inegraes a super-resoluion echnique ino racking in order o improve visual qualiy o ge more informaion of he small objec. The racking mehod using adapive paricle filer proposed by Huang [8] whils also can handles problem in racking of small objec. Improvemen visual qualiy wih super-resoluion from small objec will help he racking process become more precision. TRACKING SYSTEM Adapive Paricle Filer Paricle Filer is a mehod wih applying adapive moion models o ge a beer disribuion approach. To furher refine he exising disorders, moion coninuiy and smoohness of he rack combined wih correlaion emplae in he observaion likelihood [9]. This secion will be described lieraure review and he basic heory abou adapive paricle filer ha i proposed by [8]. Adapive Paricle Filer An image can be represened in a wo-dimensional marix which each value in he marix represens a value of illuminaion inensiy. Sae vecor is a variable sae ha describes he behavior of a sysem. If an objec s image is defined as X wih ( x, as cenroid of objec, hen objec racking can be modeled as X X v 1 (1) where v is moion vecor for objec obained from moion esimaion and a noise. The moion vecors can be represened by a ranslaional or oher models ha can esimae he objec moion from a sequenial image of video. In he observaion model, he calculaion of he weighs paricle measure based on inensiy, moion and rajecory. I can use he likelihood funcion defined as in mo rj in mo O O 1 1 P ( Z X ) P( Z X ) P( Z X ) P( Z X ) (2) where Z { Z, Z, Z } is inensiy measuremen, moion measuremen and rajecory measuremen, respecively. Z is he observaion vecor ha is aff eced by sae vecor X. From equaion above, inensiy measuremen is assumed independen from eiher bu no for moion measuremen and rajecory measuremen because boh of hem is foreign muual (he equaion is no processed in he same sae). If an objec isn deeced by moion esimaion hen O 0 and 1 oherwise. The inensiy measuremen is calculaed wih similariy beween image blocks (emplae) and candidae paricle. A uniformly disribued of Cluer wih J candidaes from correlaion surface and hen inensiy likelihood is formulaed as where q j is prior probabiliy and a normalizaion facor J 0 U(.) CN q j N( r, ) j 1 P( Z X ) q (3) C N. rj

4 The moion objec likelihood is calculaed based on d mo or he difference beween he paricle posiion change ( x, and average objec posiion change in previous ime as xs xs 1 x, k y y s k s k k 10). Then, he moion objec likelihood calculaion obained as follows P 1 2 d mo 1 2 mo 2 mo ( Z X ) e 1 s y k s 1 ([8] use (4) 2 where ( x, is paricle posiion change, ( x, is average objec posiion change in previous ime and is variance for moion objec likelihood. While, he rajecory likelihood is calculaed from paricle closeness o a rajecory ha is obained from previous posiion objec. Trajecory Likelihood can be wrien as follows m P mo 2 rj 2 rj ( Z X ) e 2 d rj F 1 (5) 2 where he closeness meric d y a x wih polynomial order m as rajecory funcion. [8] defined a forgoen f rj i 0 facor F 0, (0 ) 1. In cerain condiion. f i i If an objec in previous frame ( cur 1) isn deeced by moion esimaion O cur 1 0, a projeciion one predicion resuls on he esimaed rajecory will be done if objec is no deeced by moion esimaion. Given wo posiions when he objec is deeced in previous frame, namely X on frame j h and X on frame i h wih ( i j), he esimaed rajecory calculaed as wih a predicion X cur rj j ~ 0 ˆ 0 (1 ) X X (6) f cur cur f cur i X ˆ cur Xi ( Xi X j), i j i j (7) where a projecion of Xˆ cur namely X ~ cur is defined as he poin on he closes and 0 is he number of previous frame ha isn deeced by moion esimaion. i SUPER-RESOLUTION Super-resoluion is a process of high-resoluion images ha are buil from several low-resoluion image [5][4]. Low resoluion images can be used form a single image or images series from he same scene. Because of he same scene, a process will obain a informaion ha can be used o reconsruc he high-resoluion image [6] [10]. This paper focuses on a muli-frame super-resoluion reconsrucion echnique and hen a simplified ilusraed from superresoluion process from a images series such low qualiy video frame

5 FIGURE 1. The process of Super-resoluion image from a sequenial low resoluion image In general, super-resoluion consiss of wo process, namely he regisraion and reconsrucion image. Regisraion process including an imporan process in super-resoluion moreover images are regisraion wih sub-pixel accuracy because i very imporan aspec in here consrucion process [4]. For implemenaion, Phased Based Image Maching (PBIM) mehod will be used for he regisraion process super-resoluion. Phased Based Image Maching This mehod is acively developed in recen imes because PBIM basically use a discree Fourier ransform and is ofen used in image regisraion process because of is reabiliy and he compuong ime required is quie simple [4]. In PBIM, The calculaion sample cross correlaion beween he reference and arge image using he Fas Fourier Transform (FFT) and find he locaion of he peak which is a grea ranslaion beween he wo images. Suppose wo images f ( n 1, n2 ) and g ( n 1, n2 ) wih N1 N2 dimension and Cross phase specrum (normalized cross specrum) defined as follows ˆ F( ) G( ) J ( ) R( ) e (8) F( ) G( ) And is invers rˆ( n N1 N2 1 1, n2 ) Rˆ( ) N1N 2 n1 0 n2 e k1n1 n2 j2 ( ) N1 N2 where F ( k 1, ) and G ( k 1, ) a discree Fourier ransform of he spaial domain image. The resuls of his mehod is pixels ranslaion ha will be used in he reconsrucion process wih POCs. Super-resoluion image reconsrucion on saed redevelopmen or rearrangemens wih projecions o a high-resoluion grid afer value from he movemen of he regisraion process have been obained [4]. The reconsrucion process in he super-resoluion image based on image regisraion is illusraed as follows. (9)

6 FIGURE 2. Super-resoluion echnique using sub-pixel image regisraion From he illusraion above, he super-resoluion will be reconsruced or a redevelopmen from a low-resoluion image sequence afer hrough he image regisraion process. Projecion Ono Convex Ses Projecion Ono Convex Ses (POCS) algorihm in image reconsrucion is an algorihm ha quie simple bu i also can provide informaion more deail in he images reconsruced super-resoluion from several low-resoluion image wih moion blur and noise [4]. The firs, POCS as images reconsruced super-resoluion was suggesed by Sark and Oskoui [11]. The main concep is o esimae he super-resoluion image is consrained in a closed convex se and ge resuls by ieraing [12]. Le a Low-resoluion image g ( x, which a high resoluion image f ( x, who experienced a shif s, s ), a process of degradaion or blurring by a poin spread funcion h ( x, and he addiion ( x y of noise N( x, can be modeled as g( x, h( x, f ( x x, y s ) N( x, (10) So from he equaion obained convex se C i s y { f : g( x, h( x, f ( x, N( x, } (11) Suppose given projecion operaor who projec an image o he se of convex closed Pc, f k he magnificaion of he image by inerpolaing algorihm afer k ieraion, and f 0 firs ieraion from high resoluion so a projecion reconrucion equaion o convex se as follow f k 1 TcmTcm 1... Tc2Tc1 f k (12) where Tci I i ( Pci I) wih 0 i 2 [4][12]. The equaion above can be solved wih obain compleion ieraively on orhogonal projecion o convex se by he consrains from noise level of low-resoluion image. If projecion operaor is subsiued ino equaion above o obain following equaion ' gi hi f k 2 f k 1 f k i h 2 i (13) h ' i where g i is i h elemen of vecor g( x, and h is a row of i from marix h ( x,. The ieraion process will coninue unil he erminaion crieria is obained and he ieraion will sop[4]. i

7 TRACKING INTEGRATION WITH SUPER-RESOLUTION This secion will discuss our proposed racking inegraion wih super-resoluion echnique. Super-resoluion imaging applied ino frame of video in order o improve informaion of he racked arge (small objec). Figure 3 shows block diagram of he racking inegraion wih super-resoluion Inpu Video Conver video o frame Pre-processing Super-resoluion Image Regisraion wih PBIM Inisialisasi Objek dengan ROI Image Reconrucion wih POCS iniializaion =1 Objec Tracking =+1 End Frame? No Adapive Paricle Filer Moion Esimaion Nex Frame ( Frame +1) Yes FIGURE 3. Block diagram of racking based on super-resoluion Firsly, he process begins wih inpu video and convering video o he frame o ge all of he video frame. From of he frames, he resul frames will be processed furher called pre-processing. Pre-processing process is done o ge he hisogram ha is evenly disribued wih hisogram equalizaion o all video frames and he resul will be used as new frame. Then, he resul of pre-processing will be used as inpu o he super-resoluion process. Super-resoluion process using super-resoluion echnique of muli-frame ha consiss of wo sages, regisraion and reconrucion. In a frame sequence, he objec racking process is processed on each frame in accordance wih he specified objec. ROI (Region of Ineres) is a par of he seleced image as an area for separaing beween he foreground and background (segmenaion) as a reference for objec racking. Iniialize of arge is acceped as reference image iniialized by user wih ROI (Region Of Ineres). In he nex sage, super-resoluion resuls or new frame are used for racking of moving objec wih Adapive Paricle Filer mehod. A moion esimaion is used o deermine he moion vecor based on he model in equaion (1). Objec racking process sar from frame 2 h unil he number of video frames and according o he seleced objec wih ROI hen mark i wih a bounding box. RESULT AND DISCUSSION We are using MATLAB sofware o implemens his concep wih four video daase in.avi and min 25 fps for simulaion in order o deermine is performance over a series simulaion. Every video have various characerisic especially for objec on every video ha is presened in Table 1. In Tabel 1, Helicoper video has 60 frames wih a

8 large resoluion ( ) pixels for avoid a misake of objec deecion as noise if he video is displayed in a lower resoluion. Bicycle video and Paragliding video respecively has 50 frames and 60 frames wih ( ) pixels which conain a very low resoluion objec. Then, Moorcycle video has 63 frame wih ( ) pixes which has a objec wih a endency of he similariy color o background and small par wih inensiy of diff eren color inside i. Video TABLE 1, Video Daase Screenshoo Video Number of Frame Moorcycle 63 Bicycle 50 Helicoper 59 Paragliding 60 From every video above, each frame of video will be processed become high resoluion image, super-resoluion, wih objec size according Lefevre and Vincen [3] ypically beween 10 and 100 pixels. The super-resoluion resul is shown in Table 2 ha illusraes seleced objec by ROI

9 Video Objec Size (pixel) TABLE 2. Super-resoluion Image resuls Grayscale Low Objec Image Resoluion Image wih Hisogram Equalizaion Super-resoluion Image Moorcycle (44 47) Bicycle (23 15) Helicoper (12 37) Paragliding (17 16) This paper gives hree level of accuracy in objec racking where he assessmen is done visually wih he deails shown in Figure 4. (a) (b) (c) FIGURE 4. Three level of accuracy. (a) Precision, (b) Less-precision, (c) No precision From he picure above, he objec racking is caegorized ino precision if mos of he objecs are in he area bounding box. Objec racking is caegorized ino less precision racking if here are only a small par of he racked objec is in he bounding box area and no precision if no par of he objec is in he bounding box area. The same racking mehod wihou super-resoluion process will be used o rack video above o make a comparison and find ou significan resuls from our proposed

10 (a) (b) (c) (d) (e) (f) FIGURE 5. Illusraion racking resuls In Figure 5, secion (a), (b) and (c) show he racking resul wih caegory precision based on he level of accuracy as in Figure 5. Objec racking wih less precision is shown in Figure 5 secion (d), (e) and an imprecision racking shown in Figure 5 secion (f). For furher calculaion of he overall video, objec racking resuls wihou super-resoluion process are presened in Table below. The comparison among boh of hem based on he number of frame from racked objec corresponding o he level of accuracy above hen he resul is shown in Table 2 Number TABLE 2. The Comparison of he accuracy resul (frames) Tracking based on superresoluion Tracking wihou super-resoluion Video No Lessprecision precision No Lessprecision Precision Precision precision 1 Moorcycle.avi Bicycle.avi Helikoper.avi Paragliding.avi Moorcycle video is failed o rack moving moorcycle furher because he acquired video is done in he area of he shadow of a building so objec has a endency of he same inensiy wih background. I is caused by moion esimaion failure o deec objec alhough a small par of he objec in moorcycle video has diff erence in color inensiy wih background. A Tracking wihou super-resoluion process on moorcycle video shows ha objec can only be racked unil frame 21 h and he nex frame racking becomes less-precision or no precision. Bicycle video is also failed for furher racking process because objec is smaller and has similariies wih he background (in his case he background is he sree where people cycling). Moreover, Helikoper and Paragliding video wih racking wihou super-resoluion process show a racking wih accuracy quie good bu somehing o know furher abou effec of he super-resoluion process in racking. Table 2 also describes he racking resuls wih our proposed mehod such as a racking on he helicoper video where he objec can be racked correcly up o 36 frames. However, objec from frame 38 h unil he las has missed in racking process because of he vibraion eff ec during video acquisiion so ha objec becomes blurred for a momen. I is likewise on he video moorcycle, a moorcycle movemen can be raced correcly unil frame 54 h. Paragliding video and Bicycle video successfully racked unil las frame for rack a man. The racking resul in Table 2 show an increase of racking accuracy wih our proposed mehod ha uilize super-resoluion resul ino racking process. An increase of racking accuracy significanly occurred on he moorcycle video. The difference of visualizaion in racking resul from hese video is shown in Figure

11 FIGURE 6. A scene racking resuls from Bicycle video. Top image is racking resuls wihou super-resoluion. The boom image is racking resul based on super-resoluion An increase in accuracy of racking occurs on he Moorcycle video and Paragliding video. In addiion, a small increase of accuracy on Helicoper video due o obsacles in he process of racking such as some difficuly of racking described in he Yilmaz e al [2]. CONCLUSIONS This paper presens an approach ha inegraes a super-resoluion echnique ino racking process. The racking mehod using adapive paricle filer proposed by Huang [8] which an adapive moion model is applied o ge good proposal disribuion wih varied diversiy of paricles. Super-resoluion succesfully improves visual qualiy o ge more informaion of he racked arge (small objec). In order o make beer resul, muli-frame super-resoluion is processing ino racking where phased based image maching (PBIM) is used in he regisraion process o ge pixels ranslaion and Projecion Ono Convex Ses (POCs) as redevelopmen of super-resoluion image in reconrucion process. The simulaion resuls on he four video daase wih differen characerisic and condiion show ha our approach significanly increase of racking accuracy. REFERENCES 1. K. R. Rupesh, A Survey on Objec Deecion and Tracking Algorihms, Tesis, Naional Insiue of Technology Rourkela, Rourkela, A. Yilmaz, O. Javed, and M. Shah, Objec racking: A survey, ACM Compuing Survey, Vol 38, No 4, Aricle 13, December, S. Lefevre, T. Foissoe, and N. Vincen, Deecion and Tracking of Small Objecs by Deails Removal, Proceedings of 4h European Workshop on Image Analysis for Mulimedia Ineracive Services, Universiy of London, Queen Mary, pp , April B. Seiyono, M. Hariadi, H. M. Purnomo, Survei of Superresoluion using Phased Based Image Maching, Jurnal Of Theoriical And Applied Informaion echnology, Vol.43, No.2, Sepember S. Hao, L. Lin, Z. Weiping, and L. Limin, Locaion and Super-resoluion Enhancemen of License Plaes Based on Video Sequences, Firs Inernaional Conference on Informaion Science and Engineering, pp , December O. Mise and T.P. Breckon, Super-Resoluion Imaging Applied o Moving Targes in High Dynamic Scenes, Proceedings of SPIE - The Inernaional Sociey for Opical Engineering, Vol.889, pp , Ocober D. Davies, P. Palmer, and M. Mirmehdi, Deecion and racking of very small low conras objec, Proceedings of he Briish Machine Conference, BMVA press, pp , Y. Huang and J. Llach, Tracking he small objec hrough Cluer wih Adapive Paricle Filer, Audio, Language and Image Processing, pp , July C. Sauff er and W. E. L. Grimson, Adapive background mixure models for real-ime racking, Compuer Vision and Paern Recogniion, IEEE Compuer Sociey Conference, Vol.2, pp.252, N. Panchal, B. Limbasiya, and A. Prajapai, Survey On Muli-Frame Image Super-Resoluion, Inernaional Journal Of Scienific and Technology Research, Vol 2, pp.11, H. Sark and P. Oskoui, High-resoluion image recovery from image-plane arrays, using convex projecions, JOSA A, Opical Sociey of America, Vol.6, No.11, pp , C. Fan, J. Zhu, J. Gong, and C. Kuang, POCS Super-resoluion Sequence Image Reconsrucion Based on Improvemen Approach of Keren Regisraion Mehod,Sixh Inernasional Conference on Inelligen Sysems Design and Applicaions, Vol 2, pp , Ocober