Performance Analysis of Various Feature Detector and Descriptor for Real-Time Video based Face Tracking

Transcription

1 Iteratioal Joural Computer Applicatios ( ) Performace Aalysis Various Feature Detector ad for Real-Time Video based Face Trackig Akash Patel P.G. studet Computer Egieerig SCET, Surat, Idia. D. R. Kasat Associate Prsor SCET college Surat, Idia Sajeev Jai, Ph. D Director MITS Gwalior, Idia V. M. Thakare, Ph. D Head CSE Dept. Amravati Uiversity Amravati, Idia ABSTRACT This paper prets the performace aalysis various cotemporary feature detector ad dcriptor pair for real time face trackig. The feature detectors/dcriptors are mostly used i image ig applicatios. Some feature detectors/dcriptors like STAR, FAST, BRIEF, FREAK, ad ORB ca also be used for SLAM applicatios due to their high performace. However usig oly oe the feature detectors for object trackig may ot provide good accuracy due to various challeg i trackig like abrupt chage i object motio, o-rigid object structure, chage i appearace object, occlusios i the scee ad camera motio. But it ca be combied other object trackig algorithm to improve the overall trackig accuracy. I this paper we have measured the trackig speed ad accuracy the feature detectors i real time video for face trackig usig parameters like average umb key poits, average detectio time key-poit, frame per secod ad umb usig OpeCV. Geeral Terms Object trackig, Image ig. Keywords Face trackig, Feature detectors ad Feature dcriptors. 1. ITRODUCTIO Visual object trackig ca be defied as the procs trackig a movig object(s) cotiuously usig a camera. The goal object trackig is to determie the positio the object i fram cotiuously ad reliably i video [1]. It is very importat task i may computer visio applicatios. This procs should keep track its motio, orietatio, occlusio i scee etc. Trackig ca be simplified by imposig costraits o the motio ad/or appearace objects [2]. Feature detectors are used to fid itert poits i give image. It aims at computig abstractios image iformatio whereas feature extractio aims at how to reprets the key poits image. Feature extractio is basically a special form dimioality reductio. The detectors/dcriptors are used as first step i may applicatios like object trackig, localizatio, image ig ad recogitio. The detectio, dcriptio ad ig feature poits plays a vital role i most the cotemporary algorithms for SLAM (Simultaeous Localizatio ad Mappig) [3, 4]. I past years several ew detectors (FAST [6], SURF [7], ad CeSurE-based STAR [8]) ad dcriptors (SIFT [5], SURF [7], BRIEF [9], ORB [10], BRISK [11], ad FREAK [13]) have bee proposed. They have bee succsfully applied to the object detectio ad trackig task. Curretly, to the extet our kowledge there is o comparative study the ewt poit detectors ad dcriptors with regard to their applicability i face trackig. I [14] author has compared various feature dcriptors for Pedtria detectio. I [15] ad [16] the authors has dcribed the dired characteristics the feature detectors ad dcriptors for visual SLAM, but they have ot preted ay experimetal rults. This paper pret the performace aalysis the detector dcriptor pairs i the cotext face trackig. The measure the pair s efficiecy was based o the various parameters like average umb key poits, average detectio time key-poit, detectio frame per secod ad umb. The videos were take from several real-time situatios usig Webcam supportig rolutio up to 720p ad speed up to 30 fps. The followig paper is orgaized as follows. The Sectio 2 prets the short summary feature detector ad dcriptor evaluated i the study. Sectio 3 prets the evaluatio methodology ad rult aalysis ad the sectio 4 cotai the cocludig remarks. 2. VARIOUS FEATURE DETECTORS AD DESCRIPTORS 2.1 FAST feature detector The FAST [6] (Featur from Accelerated Segmet Tt) feature detector was the first algorithm based o AST (Accelerated Segmet Tt). It first exami the valu the itity fuctio pixels i a circle radius r aroud the cadidate poit p. They have cosidered pixel o a circle bright if its itity value is brighter by at least t(thrhold), ad dark if its itity value is darker by at least t tha the itity value p. They have classified a cadidate pixel as a feature o a basis a segmet tt if a cotiguous, at least pixels log arc bright or dark pixels is foud i the circle tha it is cosidered as feature. They have used ID3 [17] algorithm to optimize the order i which pixels are tted, rultig i high computatioal efficiecy. The segmet tt aloe produc small sets adjacet positive rpos. To further refie the rults, they have used a additioal corer-s measure for o-maximum supprsio (MS). To improve the speed the MS is applied oly to a small fractio pixels that positively passed the segmet tt. 37

2 Iteratioal Joural Computer Applicatios ( ) 2.2 SURF feature detector/dcriptor The SURF [7] (Speeded Up Robust Featur) is a robust local feature detector ad dcriptor. It is ispired by the SIFT [5] detector/dcriptor. Its mai objective was to overcome SIFT s mai weaks its computatioal complexity ad hece a low executio speed. SURF is several tim faster tha SIFT ad it is more robust agaist differet image trasformatios tha SIFT as claimed by authors. The detectio step i SURF tak advatage the use Haar wavelet approximatio the blob detector based o the Hsia determiat. The approximatios Haar wavelets ca be efficietly computed usig itegral imag, regardls the scale. For accurate localizatio multi-scale SURF featur iterpolatio is required. For the feature dcriptor they have used Haar wavelets i cojuctio with itegral imag to ecode the distributio pixel itity valu i the eighborhood the feature while accoutig the feature s scale. They have computed the dcriptor for a give feature at scale s which begis with the assigmet the domiat orietatio to make the dcriptor rotatio ivariat. 2.3 CeSurE based STAR feature detector The STAR keypoit detector was implemeted as a part the OpeCV computer visio library. It is derived from CeSurE (Ceter Surroud Extrema) feature detector [8]. The authors aimed at the formatio a multi-scale detector with full spatial rolutio. As defied i [8], the subsamplig performed by SIFT [5] ad SURF [7] affects the accuracy feature localizatio. The detector us a bi-level approximatio the Laplacia Gaussias (LoG) filter. The circular shape the mask is replaced by a approximatio that prerv rotatioal ivariace ad eabl the use itegral imag for efficiet computatio. They have created scale-space without iterpolatio, by applyig masks differet size. 2.4 BRIEF corer dcriptor The BRIEF [9] (Biary Robust Idepedet Elemetary Featur) dcriptor proposed i [8] us biary strigs for feature dcriptio ad subsequet ig. This eabl the use Hammig distace to compute the dcriptor similarity. Such similarity measure ca be computed very efficietly much faster tha the commoly used L2 orm. Due to BRIEF s sitivity to oise, the image is smoothed with a simple averagig filter before applyig the actual dcriptor. The value each bit cotributig to the dcriptor depeds o the rult a compariso betwee the itity valu two poits iside a image segmet cetered o the curretly dcribed feature. The bit corrpodig to a give poit pair is set to if the itity value the first poit this pair is higher tha the itity value the secod poit, ad ret otherwise. The samplig strategy for the selectio poit for the pairs to be compared was selected based o experimets with uiform ad Gaussia radom samplig usig differet distributio parameters. The proposed dcriptor is 512-bit log ad computed over a pixel image patch. The iitial smoothig is performed with a 9 9 pixel rectagular averagig filter. The basic form BRIEF is ot ivariat w.r.t. rotatio. 2.5 ORB feature detector/dcriptor ORB [10] is basically a fusio FAST (Featur from Accelerated Segmet Tt) [6] keypoit detector ad BRIEF (Biary Robust Idepedet Elemetary Featur) [9] dcriptor with may modificatios to ehace the performace. It us FAST to fid, ad the apply Harris corer measure to fid top poits amog them. It also use pyramid to produce multiscale-featur. But oe problem is that, FAST do t compute the orietatio. So, Authors came up with followig modificatio. It comput the itity weighted cetroid the patch with located corer at ceter. The directio the vector from this corer poit to cetroid giv the orietatio. To improve the rotatio ivariace, momets are computed with x ad y which should be i a circular regio radius r, where r is the size the patch. For dcriptor, ORB us modified versio BRIEF dcriptor. Stadard BRIEF dcriptor performs poorly with rotatio. So ORB steer BRIEF accordig to the orietatio. For ay feature set biary tts at locatio (x i, y i ), defie a 2 matrix, S which cotais the coordiat the pixels. The usig the orietatio patch, θ, its rotatio matrix is foud ad rotat the S to get steered(rotated) versio S θ. ORB discretize the agle to icremets 2π/30 (12 degre), ad costruct a lookup table pre-computed BRIEF patters. As log as the keypoit orietatio θ is cosistet across views, the correct set poits S θ will be used to compute its dcriptor. 2.6 BRISK feature detector/dcriptor The BRISK [11] is a keypoit detector ad dcriptor ispired by AGAST [12] ad BRIEF [9]. For detectig the featur it us AGAST [12] which is improvemet FAST i speed while maitaiig the same detectio performace. To achieve scale ivariace, it detects the i a scalpace pyramid, performig o-maxima supprsio ad iterpolatio across all scal. Istead usig leared or radom patter like i BRIEF ad ORB they have used symmetric patter to dcribe the featur. They have used several log-distace sample poit comparisos to determie orietatio ad for log-distace compariso the vector displacemet betwee the sample poits is stored ad weighted by the relative differece i itity. The, to determie the domiat gradiet directio patch the weighted vectors are averaged. 2.7 FREAK feature dcriptor The FREAK [13] (Fast Retia Keypoit) is a ovel dcriptor biologically ispired by huma visual system. It provid the dcriptor with feature orietatio by summig the timated local gradiets over selected poit pairs. It us a specific poit samplig patter that allows applyig coarser discretizatio rotatio, which allows savigs i memory space. They have used a special, biologically ispired samplig patter. While the rultig dcriptor is still a biary strig like BRIEF [9], the samplig patter allows for the use a coarse-to-fie approach to feature dcriptio. It first compar the poit pairs carryig the iformatio o most distictive characteristics the feature eighborhood. This allows for faster rejectio false ad shorteig the computatio time. 3. EXPERIMETS 3.1 Dataset We have used our ow dataset for ttig various detectordcriptor pairs. We have tted each pair i several realworld situatios. The face was moved left/right to tt the effect rotatio for each detector/dcriptor pair. The Logitech C270 Webcam supportig rolutio up to 720p ad speed up to 30 fps is used for takig the videos. 38

3 Iteratioal Joural Computer Applicatios ( ) 3.2 Evaluatio The OpeCV C/C++ library for Widows is used to perform all the tts. All the tts were executed o a laptop with a Itel 2 d ge core-i5 2430M 2.4GHz procsor ad 4GB RAM. The video was captured at rolutio. The followig procedure is adopted for ttig each detectordcriptor pair: 1. The user selects the itert object from the live video. 2. The selected area is cropped from frame ad it is cosidered as object image. 3. The selected poit feature detector is applied o object image ad curret video frame (i.e. scee). 4. The poit featur dcriptors are calculated for both imag usig the selected dcriptor algorithm. 5. The featur from both imag are ed usig hammig distace based brute-force er fuctio by miimizig the distace betwee their dcriptors. 6. The distace d betwee dcriptor object image ad scee image is calculated. 7. The mea this distace array is calculated usig this formula: mea(μ) = Where is total umb dcriptors (1) 8. The deviatio distace array is calculated usig this formula: deviatio σ = (2) 9. The the parameters like average umb key poits, ad Time take per frame are also calculated for each pair. Here the mea ad deviatio distace array are used as efficiecy measure. The larger mea shows that the average distace betwee dcriptor referece image ad scee image is large. So it poits towards lser efficiecy ig. While the deviatio reprets the average amout differece betwee other dcriptor value ad the mea value. The deviatio teds to icrease whe object is moved i the scee. Firstly the tt was performed o the facial imag havig plai backgroud i good lightig. At first we have measured all the parameters as dcribed above for each detectordcriptor pair o straight face. The the face is moved left ad right as show i fig. 3.1 ad agai all the parameters are measured. The same tt is repeated for 3 tim for each pair. The rults this tt are show i table The we have performed similar tts i low light coditio. The rults low-light tt are show i table I followig tabl the umb ad umber shows the average value. For mea ad deviatio the rage is show. The fps field shows the average frame per secod video. The distace mea ad deviatio value 1 i=1 d i i=1 (d i μ) 2 icreas as the face mov left/right. The FREAK dcriptor has hight deviatio while the SURF detector/dcriptor has lowt average mea distace ad deviatio but its performace is slower tha all ad it detects ls umb tha others. Biary vector dcriptor BRIEF ad ORB are showig good performace. I low light coditio the umb decreas drastically for FAST ad ORB detectors. BRISK detector was tted o low thrhold (T=10) for low light coditio because at default thrhold (T=30) it was ot detectig ay keypoit i the face. Table 3.1 Tt rults for FAST detector er er Mea BRIEF BRISK ORB FREAK Table 3.2 Tt rults for SURF detector er Mea BRIEF BRISK ORB FREAK Table 3.3 Tt rults for STAR detector er Mea BRIEF BRISK ORB FREAK Table 3.4 Tt rults for ORB detector er Mea ORB BRISK FREAK

4 Iteratioal Joural Computer Applicatios ( ) Table 3.5 Tt rults for BRISK detector Mea BRISK BRIEF FREAK The Fig. 3-1 shows the trackig rult for FAST/BRIEF pair i good light coditio. While the Fig. 3-2 shows the trackig rult i low light coditio. From both the fig. we ca say that the umber are very ls i low light coditio. Table 3.6 Tt rults for FAST detector (low-light) e s Mea BRIEF BRISK ORB FREAK Table 3.7 Tt rults for SURF detector (low-light) Mea BRIEF BRISK ORB FREAK Table 3.8 Tt rults for STAR detector (low-light) e s Mea BRIEF BRISK ORB FREAK Figure 3-1 FAST/BRIEF (I good lightig) Table 3.9 Tt rults for ORB detector (low-light) Mea ORB BRISK FREAK Table 3.10 Tt rults for BRISK detector (low-light) Mea BRISK BRIEF FREAK Figure 3-2 FAST/BRIEF (I low light) 40

5 Iteratioal Joural Computer Applicatios ( ) 4. COCLUSIOS We have compared various cotemporary feature detector ad dcriptor pair to fid the bt combiatio for real time visual face trackig. The experimets show that i low light coditio umb ad are decreasig. The biary dcriptors BRIEF ad ORB are showig good performace with detectors like FAST ad STAR. While the recetly proposed FREAK remov so may keypoit whe combied with SURF, ORB ad BRISK ad it has more deviatio compared to other whe object mov so it is ot showig cosistet performace as claimed i [13]. The SURF detector has the lowt distace deviatio ad mea so it is accurate. But it tak almost double time tha other detectors. So it is ls suitable for SLAM applicatios. So i short FAST/BRIEF or ORB is more suitable for real time visual face trackig. 5. REFERECES [1] W. Hu, T. Ta, L. Wag, ad S. Maybak, A survey o visual surveillace object motio ad behaviors, IEEE Tras. Syst. Ma Cyber.-C vol. 34 (3), 2004, pp [2] A. Yilmaz, O. Javed, ad M. Shah. Object trackig: A survey. ACM Computig Survey, vol. 38(4), [3] J. Daviso, I. Reid,. Molto, ad O. Stasse, MooSLAM: Real-Time Sigle Camera SLAM, IEEE Tras. PAMI, vol. 29(6), 2007, pp [4] A. Schmidt, A. Kasiński, The Visual SLAM System for a Hexapod Robot, Lecture ot i Computer Sciece, vol. 6375, 2010, pp [5] D. Lowe, Object recogitio from local scale-ivariat featur, i: Proceedigs the Iteratioal Cerece o Computer Visio ICCV, Corfu, 1999, pp [6] E. Roste, ad T. Drummod, Machie learig for highspeed corer detectio, i Proc. Europea C. o Computer Visio, 2006, pp [7] H. Bay, A. Ess, T. Tuytelaars, L. Va Gool, SURF: Speeded Up Robust Featur, Computer Visio ad Image Uderstadig, vol. 110(3), 2008, pp [8] M. Agrawal, K. Koolige, ad M.R. Blas, CeSurE: Ceter surroud extremas for real time feature detectio ad ig, Lecture ot i Computer Sciece, vol. 5305, 2008, pp [9] M. Caloder, V. Lepetit, C. Strecha, ad P. Fua, BRIEF: Biary Robust Idepedet Elemetary Featur, i Proceedigs ECCV 2010, pp [10] E. Rublee, V. Rabaud, K. Koolige, ad G. R. Bradski, ORB: A efficiet alterative to SIFT or SURF, i Proc. ICCV, 2011, pp [11] S. Leuteegger, M. Chli, ad R. Siegwart, Brisk: Biary robust ivariat scalable, i Proc. It. C. Computer Visio, 2011, pp [12] E. Mair, G. D. Hager, D. Burschka, M. Suppa, ad G. Hirziger, Adaptive ad geeric corer detectio based o the accelerated segmet tt, I Proceedigs the Europea Cerece o Computer Visio (ECCV), [13] A. Alahi, R. Ortiz, ad P. Vadergheyst, FREAK: Fast Retia Keypoit, I Proc. IEEE Cerece o Computer Visio ad Patter Recogitio, 2012, pp [14] Schaeffer, Camero. "A Compariso Keypoit s i the Cotext Pedtria Detectio: FREAK vs. SURF vs. BRISK", [15] O. Martíez, A. Gil, M. Ballta, ad O. Reioso, Itert Poit Detectors for Visual SLAM, I Curret Topics i Artificial Itelligece, Spriger Berli Heidelberg, 2007, pp [16] M. Ballta, A. Gil, O. Martíez, ad O. Reioso, Local s for Visual SLAM, i Proc. Workshop o Robotics ad Mathematics, [17] Quila, J.R., Iductio decisio tree, Machie learig, vol. 1(1) 1986, pp IJCA TM : 41