Superpixel Tracking. School of Information and Communication Engineering, Dalian University of Technology, China 2

Transcription

1 Superpixel Traking Shu Wang1, Huhuan Lu1, Fan Yang1, and Ming-Hsuan Yang2 1 Shool of Information and Communiation Engineering, Dalian University of Tehnology, China 2 Eletrial Engineering and Computer Siene, University of California at Mered, United States Abstrat While numerous algorithms have been proposed for objet traking with demonstrated suess, it remains a hallenging problem for a traker to handle large hange in sale, motion, shape deformation with olusion. One of the main reasons is the lak of effetive image representation to aount for appearane variation. Most trakers use high-level appearane struture or low-level ues for representing and mathing target objets. In this paper, we propose a traking method from the perspetive of mid-level vision with strutural information aptured in superpixels. We present a disriminative appearane model based on superpixels, thereby failitating a traker to distinguish the target and the bakground with mid-level ues. The traking task is then formulated by omputing a target-bakground onfidene map, and obtaining the best andidate by maximum a posterior estimate. Experimental results demonstrate that our traker is able to handle heavy olusion and reover from drifts. In onjuntion with online update, the proposed algorithm is shown to perform favorably against existing methods for objet traking. 1. Introdution The reent years have witnessed signifiant advanes in visual traking with the development of effiient algorithms and fruitful appliations. Examples abound, ranging from algorithms that resort to low-level visual ues to high-level strutural information with adaptive models to aount for appearane variation as a result of objet motion [1, 3, 1, 8, 21, 11]. While low-level ues are effetive for feature traking and sene analysis, they are less effetive in the ontext of objet traking [23]. On the other hand, numerous works have demonstrated that adaptive appearane models play a key role in ahieving robust objet traking [9, 4, 13, 1, 11, 2]. In [13], an inremental visual traker () with adaptive appearane model that aims to aount for appearane variation of rigid or limited deformable motion is presented. Although it has been shown to perform well when target objets undergo lighting and pose variation, this method is less bird2 #7 bird2 #11 bird2 #19 lemming #867 lemming #155 lemming #112 transformer #17 transformer #52 transformer #124 woman #32 woman #65 woman #236 Figure 1. Four ommon hallenges enountered in traking. The results by our traker, [13], [11], PROST [2], Trak [1] and PDAT [1] methods are represented by yellow, red, white, green, yan, and magenta retangles. Existing trakers are not able to effetively handle heavy olusion, large variation of pose and sale, and non-rigid deformation, while our traker gives more robust results. effetive in handling heavy olusion or non-rigid distortion as a result of the adopted holisti appearane model. The ensemble traker [2] formulates the task as a pixelbased binary lassifiation problem. Although this method is able to differentiate between target and bakground, the pixel-based representation is rather limited and thereby onstrains its ability to handle heavy olusion and lutter. The ment-based traker [1] aims to solve partial olusion with a representation based on histograms of loal pathes. The traking task is arried out by ombing votes of mathing loal pathes using a template. Nevertheless, the template is not updated and thereby it is not expeted to handle appearane hange due to large variation in sale and shape deformation.

2 In addition to aount for appearane variation, reent works have foused on reduing visual drifts. In [3], an algorithm extends multiple instane learning to an online setting for objet traking. Whereas it is able to redue visual drifts, this method is not able to handle large nonrigid shape deformation. The PROST method [2] extends the traking-by-detetion framework with multiple modules for reduing drifts. Although this traker is able to handle ertain drifts and shape deformation, it is not lear how this method an be extended to handle targets undergoing non-rigid motion or large pose variation. The visual traking deomposition () approah effetively extends the onventional partile filter framework with multiple motion and observation models to aount for appearane variation aused by hange of pose, lighting and sale as well as partial olusion [11]. Nevertheless, as a result of the adopted generative representation sheme, this traker is not equipped to distinguish target and bakground pathes. Consequently, bakground pixels within a retangular template are inevitably onsidered as parts of foreground objet, thereby introduing signifiant amount of noise in updating the appearane model. Mid-level visual ues have been effetive representations with suffiient information of image struture and great flexibility when ompared with high-level appearane models and low-level features. In partiular, superpixels have been one of the most promising representations with demonstrated suess in image segmentation and objet reognition [18, 15, 22, 12, 17]. These methods are able to segment images into numerous superpixels with evident boundary information of objet parts from whih effetive representations an be onstruted. In [19], a traking method based on superpixel is proposed, whih regards traking task as a figure/ground segmentation aross frames. However, as it proesses every entire frame individually with Delaunay triangularization and CRF for region mathing, the omputational omplexity is rather high. Further, it is not designed to handle omplex senes inluding heavy olusion and luttered bakground as well as large lighting hange. Similarly, a non-parametri method [14] also aims to segment one single salient foreground objet from bakground.. In this paper, we exploit effetive and effiient mid-level visual ues for objet traking with superpixels. During the training stage, the segmented superpixels are grouped for onstruting a disriminative appearane model to distinguish foreground objets from luttered bakgrounds. In the test phase, a onfidene map at superpixel level is omputed using the appearane model to obtain the most likely target loation with maximum a posteriori (MAP) estimates. The appearane model is onstantly updated to aount for variation aused by hange in both the target and the bakground. Experimental results on various sequenes show that the proposed algorithm performs favorably against existing state-of-the-art methods. In partiular, our algorithm is able to trak objets undergoing large nonrigid motion, rapid movement, large variation of pose and sale, heavy olusion and drifts. 2. Proposed Algorithm We present details of the proposed image representation sheme and traking algorithm in this setion Bayesian Traking Formulation Our algorithm is formulated within the Bayesian framework in whih the maximum a posterior estimate of the state given the observations up to time t is omputed by p(x t Y 1:t ) = αp(y t X t ) p(xt X t 1 )p(x t 1 Y 1:t 1 )dx t 1 (1) where X t is the state at time t, Y 1:t is all the observations up to time t, and α is a normalization term. In this work, the target state is defined as X t = (Xt, Xt s ), where Xt represents the enter loation of the target and Xt s denotes its sale. As demonstrated by numerous works in the objet traking literature, it is ritial to onstrut an effetive observation model p(y t X t ) and an effiient motion model p(x t X t 1 ). In our formulation, a robust disriminative appearane model is onstruted whih, given an observation, omputes the likelihood of it belonging to the target or the bakground. Thus the observation estimate of a ertain target andidate X t is proportional to its onfidene: p(y t X t ) Ĉ(X t) (2) where Ĉ(X t) represents the onfidene of an observation at state X t being the target. The state estimate of the target ˆX t at time t an be obtained by the MAP estimate over the N samples at eah time t. Let X (l) t denote the l-th sample of the state X t, ˆX t = arg max p(x (l) t Y 1:t ) l = 1,..., N (3) X (l) t In the following, the superpixel-based disriminative appearane model for traking is introdued in Setion 2.2, followed by onstrution of the onfidene map based on this model in Setion 2.3. The observation and motion models are presented in Setion 2.4, and then the update sheme Superpixel-based Disriminative Appearane Model To onstrut an appearane model for both the target and the bakground, prior knowledge regarding the label of eah

3 (a) (b) () (d) (e) (f) (g) (h) (i) Figure 2. Illustration of onfidene map for state predition. (a) a new frame at time t. (b) surrounding region of the target in the last frame, (1) i.e., at state Xt. () segmentation result of (b). (d) the omputed onfidene map of superpixels using Eq. 7 and Eq. 8. The superpixels olored with red indiate strong likelihood of belonging to the target, and those olored with dark blue indiate strong likelihood of belonging to bakground. (e) the onfidene map of the entire frame. (f), (g) and (h), (i) show two target andidates with high and low onfidene, respetively. pixel an be learned from a set of m training frames. That is, for a ertain pixel at loation (i, j) in the t-th frame pixel(t, i, j), we have: 1 if pixel(t, i, j) target yt (i, j) = (4) 1 if pixel(t, i, j) bakground where yt (i, j) denotes the label of pixel(t, i, j). Assume that the target objet an be represented by a set of superpixels without signifiantly destroying the boundaries between target and bakground (i.e., only few superpixels ontain almost equal amount of target pixels and bakground pixels), prior knowledge regarding the target and the bakground appearane an be modeled by 1 if sp(t, r) target yt (r) = (5) 1 if sp(t, r) bakground where sp(t, r) denotes the r-th superpixel in the t-th frame, and yt (r) denotes its orresponding label. However, suh prior knowledge is not at our disposal in most traking senarios, and one feasible way to ahieve this is to infer prior knowledge from a set of samples, {Xt }m t=1 prior to the traking proess starts. We present a method to extrat similar information as Eq. 5 from a small set of samples. First, we segment the surrounding region1 of the target in the t-th training frame into Nt superpixels. Eah superpixel sp(t, r) (t = 1,..., m, r = 1,..., Nt ) is represented by a feature vetor ftr. Next, we apply the mean shift lustering algorithm [6] on the total feature pool F = {ftr t = 1,..., m; r = 1,..., Nt }, and obtain n different lusters. In the feature spae, eah luster lst(i) (i = 1,..., n) is represented by its luster enter f (i), its luster radius r (i) and its own luster members {ftr ftr lst(i)}. Now that every lst(i) orresponds to its own image region S(i) in the training frames (image regions that superpixel members of lst(i) over), we ount two sores for 1 The surrounding region is a square area entered at the loation of 1 target Xt, and its side length is equal to λs [S(Xt )] 2, where S(Xt ) represents the area size of target area Xt. The parameter λs is a stable parameter, whih ontrols the size of this surrounding region, and is set to 1.5 in all experiments. eah lst(i): S + (i) and S (i). The former denotes size of luster area S(i) overlapping the target area at state Xt in the orresponding training frames, and the latter denotes the size of S(i) outside the target area. Intuitively, the greater the ratio S + (i)/s (i) is, the more likely superpixel members of lst(i) appear in target area in training frames. Consequentially, we give eah luster a target-bakground onfidene measure between 1 and -1 to indiate how probable its superpixel members belong to the target or bakground: Ci = S + (i) S (i), i = 1,..., n. S + (i) + S (i) (6) Our superpixel-based disriminative appearane model is onstruted based on four fators: luster onfidenes Ci, luster enters f (i), luster radius r (i) and luster members {ftr ftr lst(i)}, whih are used for determining the luster for a ertain superpixel. By applying the onfidene measures of eah luster to superpixels in the training frames, we are able to learn similar prior knowledge as Eq. 5 from a set of training images. The merits of the proposed superpixel-based disriminative appearane model are shown by Figure 4 and Setion 3: Few bakground superpixels appearing in the target area (as a result of drifts or olusions), are likely to be lustered into the same group with other bakground superpixels, and thus have negligible effet to our algorithm during training and update Confidene Map When a new frame arrives, we first extrat a surrounding region2 of the target and segment it into Nt superpixels (See Figure 2 (b) and ()). To ompute a onfidene map for urrent frame, we evaluate every superpixel and ompute its onfidene measure. The onfidene measure of a superpixel depends on two fators: the luster it belongs to, and the distane between this superpixel and the orresponding luster enter in the feature spae. The rationale for the first riterion is that if a ertain superpixel belongs to lst(i) in the feature spae, then the target-bakground onfidene of 2A 1 square area entered at Xt 1 with side length λs [S(Xt 1 )] 2.

4 warped image onfidene map Figure 3. Confidene map. Four target andidate regions orresponding to states X (i) t, i = 1,..., 4 are shown both in warped image and the onfidene map. These andidates onfidene regions M i, i = 1,..., 4 have the same anonial size (upper right) after normalization. Based on Eq. 1, andidate X (1) t, X (2) t have similar positive onfidene C 1, C 2, and X (3) t, X (4) t have similar negative onfidene C 3, C 4. However, andidate X (2) t overs less target area than X (1) t, and X (4) t overs more bakground area than X (3) t. Intuitively, targetbakground onfidene of X (1) t should be higher than X (2) t, while onfidene of X (4) t should be lower than X (3) t. These two fators are onsidered in omputing onfidene map as desribed in Setion 2.4. lst(i) indiates how likely it belongs to the target or bakground. The seond term is a weighting term that takes the distane metri into onsideration. The farther the feature of a superpixel ft r lies from the orresponding luster enter f (i) in feature spae, the less likely this superpixel belongs to lst(i). The onfidene measure of eah superpixel is omputed as follows: w(r, i) = exp( λ d f r t f(i) 2 r (i) ) r = 1,..., N t, i = 1,..., n (7) C s r = w(r, i) C i, r = 1,..., N t (8) where w(r, i) denotes the weighting term based on the distane between ft r (the feature of sp(t, r), the r-th superpixel in the t-th frame) and f (i) (the feature enter of the luster that sp(t, r) belongs to). The parameter r (i) denotes the luster radius of lst(i) in the feature spae, and λ d is a normalization term (set to 2 in all experiments). By taking these two terms into aount, Cr s is the onfidene measure for superpixel r at the t-th frame, sp(t, r). We obtain a onfidene map for eah pixel on the entire urrent frame as follows. We assign every pixel in the superpixel sp(t, r) with superpixel onfidene Cr s, and every pixel outside this surrounding region with -1. Figure 2 (a)- (e) shows the steps how the onfidene map is omputed with a new frame arriving at time t. This onfidene map is omputed based on our appearane model desribed in Setion 2.2. In turn, the following steps for identifying the likely loations of the target in objet traking are based on this onfidene map Observation and Motion Models The motion (or dynamial) model is assumed to be Gaussian distributed: p(x t X t 1 ) = N (X t ; X t 1, Ψ) (9) where Ψ is a diagonal ovariane matrix whose elements are the standard deviations for loation and sale, i.e., σ and σ s. The values of σ and σ s ditate how the proposed algorithm aounts for motion and sale hange (See details in the supplemental material). We then normalize all these andidate image regions into anonial size maps {M l } N l=1 (the size of the target orresponding to X t 1 is used as the anonial size). We denote v l (i, j) the value at loation (i, j) of the normalized onfidene map M l of X (l) t, and then we aumulate the onfidene for the state X (l) t : (i,j) M l v l (i, j) (1) However, this target-bakground onfidene C l does not deal with saling well. In order to make the traker robust to the saling of the target, we weigh C l with respet to the size of eah andidate as follows: Ĉ l = C l [S(X (l) t )/S(X t 1 )], l = 1,..., N (11) where S(X t ) represents the area size of target state X t. For the target andidates with positive onfidene values (i.e., indiating they are likely to be targets), the ones with larger area size should be weighted more. For the target andidates with negative onfidene values, the ones with larger area size should be weighted less. This weighting sheme ensures our observation model p(y t Xt s ) adaptive to sale. Figure 3 illustrates this weighting sheme. We then normalize the final onfidene of all targets {Ĉl} N l=1 within the range of [,1] for omputing likelihood of X (l) t for our observation model: p(y t X (l) t ) = Ĉl, l = 1,..., N (12) where Ĉl denotes the normalized onfidene value for eah sample. With the observation model p(y t X (l) t ) and the motion model p(x (l) t X t 1 ), the MAP state estimate ˆX t an be omputed with Eq. 3. Figure 2 (f)-(i) show two drawn samples and their orresponding onfidene maps. As shown in

5 C3.7 C4.3 C6 1 Feature Spae C5.8 C 2 1 C1.8 (a) (b) () (d) (e) Figure 4. Reovering from drifts. (a) a target objet with visual drifts. (b) the surrounding region of the target is segmented into superpixels. () lustering results of (b) in feature spae and the target-bakground onfidene of eah luster. (d) the onfidene map in a new frame omputed with lustering results. (e) the MAP estimate of the target area (the traker reovers from drifts). This illustration shows even if our traker experienes drifts during traking (See (a)), our appearane model obtains suffiient information from surrounding bakground area by update, and provides our traker with a more disriminative power against drifts than holisti appearane models. these examples, the onfidene maps failitate the proess of determining the most likely target loation Online Update with Olusion and Drifts We apply superpixel segmentation to the surrounding region of the target (rather than the entire image) for effiient and effetive objet traking. A sliding window update sheme is adopted, in whih a sequene of H frames is stored during traking proess. For every U frames, we put a new frame into this sequene, and delete the oldest one. That is, this proess retains a reord from the past H U frames. For eah frame in this sequene, the estimated state ˆX t and the result of superpixel segmentation are saved. We update the appearane model with the retained sequene every W frames 3, and this proess is the same as the training proess desribed in Setion 2.2. With the proposed disriminative appearane model using mid-level ues, we present a simple but effiient method to handle olusion in objet traking. For a state X (l) t at time t, its onfidene C l (from Eq. 1) is bounded within a range: [ S(X (l) t ), S(X (l) t )]. The upper bound indiates that all pixels in the image region orresponding to X (l) t are assigned with highest onfidene of belonging to the target, and onversely the lower bound indiates all pixels belonging to the bakground. We set a threshold θ o to detet heavy or full olusions: µ C max({c l } N l=1 ) > θ S(X (l) o (13) t ) 2 where µ C is the average of onfidene (from Eq. 1) of the target estimates in the retained sequene of H frames. The numerator of the left hand side of this formula reflets the differene between the onfidene C l of the MAP estimate of urrent frame, and the average onfidene of target in the retained sequene. The denominator is a normalizing term to onfine the left hand side to the range of [-1,1]. If the onfidene C l of the MAP estimate of urrent frame is muh 3 The length of information sequene H and the spaing interval U is set 1 and 3 in our experiments. The update frequeny W is set between 5 and 1. Table 1. Proposed algorithm. Initialization: for t = 1 to m (e.g., m is set to 4 in all experiments) 1. Initialize parameters of our algorithm in the first frame. 2. Segment the surrounding region of X t for training into N t superpixels, and extrat their features {ft r}n t r=1. end Obtain a feature pool F = {ft r t = 1,..., m; r = 1,..., Nt}. Apply mean shift lustering and obtain the superpixel-based disriminative appearane model by Eq. 6. Traking: for t = m + 1 to the end of the sequene end 1. Segment a surrounding region of X t 1 into N t superpixels and extrat their features. Compute the target-bakground onfidene map using Eq. 7 and Eq Sample N andidate states {X (l) t } N l=1 with the onfidene map. 3. Compute motion parameters p(x (l) t X t 1 ) by Eq. 9 and their likelihoods p(y t X (l) t ) by Eq Estimate MAP state ˆX t using Eq Detet full olusion with Eq Add one frame into the update sequene every U frames 7. Update the appearane model every W frames. less than the average of onfidene of the retained sequene, that means that the MAP target estimate is still of high probability to be bakground area, then Eq. 13 is satisfied and a heavy olusion is deemed to our. In suh situations, the target estimate X t 1 of the last frame is onsidered the target estimate ˆX t for the urrent frame. Furthermore, instead of deleting the oldest (first) frame when we add one new frame to the end of the retained sequene, we delete the k- th (e.g., k = 8, k < H) frame of the sequene. In this manner, our traker does not delete all information of target when long-duration olusion ours, and meanwhile does not ontinue to learn from oluded examples. Without this mehanism, our traker may update with wrong examples when the target objet is oluded or un-oluded. All superpixels in the urrent frame are regarded as lying in the bakground area and µ C is saved as the onfidene at the urrent frame. As will be shown in the experiments, robust results an be obtained with this sheme.

6 The onfidene map with update is also used to reover our traker from drifts. Figure 4 illustrates how the proposed method reovers from drifts with the information from superpixels and onfidene map. The main steps of the proposed algorithm are summarized in Table Experimental Results We present the experimental setups and empirial results as well as observations in this setion Experimental Setups We utilize normalized histogram in the HSI olor spae as the feature for eah superpixel. The SLIC algorithm [17] is applied to segment frames into superpixels where the spatial proximity weight and number of superpixels are set to 1 and 3, respetively. The bandwidth of the mean shift lustering [6] is set to the range of.15 and.2. We note that the bandwidth needs to be wide enough to separate superpixels from the target and bakground into different lusters. To ollet a training dataset in the initialization step, the target regions in the first 4 frames are either loated by an objet detetor or manually ropped. The σ and σs in Eq. 9 are set between 3 and 8 in antiipation of the fastest motion speed or hanging sale of the target objets. The threshold to detet olusion θo is between the range of.1 and.3. We evaluate our algorithm on 1 hallenging sequenes (6 from prior work [1, 1, 2, 11] and 4 from our own). These sequenes inlude most hallenging fators in visual traking: omplex bakground, moving amera, fast movement, large variation in pose and sale, half or full olusion, shape deformation and distortion (See Figure 1, Figure 5 and Figure 6). The quantitative evaluations of the Mean Shift (MS) [5], adaptive olor-based partile filter (PF) [16], [13], Trak [1], Trak [3], PROST [2], [11] methods and our traker are presented in Table 2, Table 3 and Figure 7. More results and videos an be found in the supplementary material and at our web site ( In addition, our work an easily be extended to segment salient foreground target from bakground, and results are presented in supplemental material. All the MATLAB ode and datasets are available on our web site Empirial Results We first evaluate our algorithm with the sequenes used in prior works: singer1 and basketball from [11], transformer from PDAT [1], lemming and liquor from PROST [2], and woman from Trak [1]. We then test 4 sequenes from our own dataset: bolt, bird1, bird2, and girl. For fair omparison, we arefully adjust the parameters of every traker with the ode provided by the authors Sequene lemming liquor singer1 basketball woman transformer bolt bird1 bird2 girl MS PF PROST SPT Table 2. Traking results. The numbers denote average errors of enter loation in pixels. girl #745 bolt #26 liquor #1236 basketball #169 singer1 #77 woman #273 Figure 5. Traking results with omparisons to olor-based trakers. The results by the MS traker, PF method and our algorithm are represented by red ellipse, green ellipse and yellow retangles. It is evident that our traker is able to handle luttered bakground (girl and basketball sequenes), drasti movement (bolt sequene), heavy olusion (liquor and woman sequenes) and lighting ondition hange (singer1 sequene). and use the best result from 5 runs, or taken diretly from the presented results in the prior works. Comparison with olor-based trakers: As shown in Figure 5, the adaptive olor-based partile filter [16] an neither deal with luttered bakground, drasti movement nor heavy olusion. The mean shift traker with adaptive sale [5] does not perform well when there is a large appearane hange due to non-rigid motion, lighting hange and heavy olusion (Figure 5). We note that this traker is designed to handle sale hange. However, it is less effetive in dealing with lighting and olusion. On the other hand, the disriminative appearane model based on mid-level representation alleviates negative influenes from noise and bakground lutter. Consequently, our traker is able to trak objets undergoing heavy olusion, non-rigid deformation and lighting hange in lutter bakgrounds (Figure 5). Comparison with other state-of-the-art trakers: Visual drifts: While trakers based on holisti appearane models are able to trak objets in many senarios, they are less effetive in handling drifts. The main reason is

7 basketball #35 basketball #485 basketball #725 bolt #2 bolt #184 bolt #35 girl #117 girl #1395 girl #15 liquor #778 liquor #1187 liquor #1722 bird1 #33 bird1 #1 bird1 #185 bird1 #268 bird1 #314 bird1 #371 Figure 6. Traking results. The results by our traker,,, PROST, Trak and Trak methods are represented by yellow, red, white, green, blue and yan retangles. Sequene MS lemming 171 liquor 413 singer1 64 basketball 78 woman 35 transformer 28 bolt 15 bird1 1 bird2 36 girl 79 PF PROST SPT Table 3. Traking results. The numbers denote the ount of suessful frame based on evaluation metri of the PASCAL VOC objet detetion [7] whih is also used in other traking algorithm [2]. Note that we use elliptial target area for the mean shift traker (MS) and the adaptive olor-based partile filter (PF) to alulate the metri used in PASCAL VOC tests for fair omparison. that these trakers typially fous on learning target appearane rather than the bakground (i.e., with a generative approah). As shown in the first row (bird2 sequene) of Figure 1, the and methods drift away from the target into bakground regions when heavy olusions our in frame 11 and 19. In the basketball and bolt sequenes (shown in Figure 6), the, Trak and Trak methods drift to bakground area in early frames for that they are not designed for non-rigid deformation. Although the traker ahieves the seond best results in these two sequenes, its traking results are not as aurate as ours. The reason is that it does not distinguish the target from the bakground, and onsiders some bakground pixels as parts of the target, thereby rendering impreise traking results. In ontrast, the disriminative appearane model of our traker utilizes bakground information effetively and avoids suh drifting problems throughout these two sequenes. Large variation of pose and sale: The seond row (lemming sequene) in Figure 1 shows that, the, Trak and PROST methods perform well as the methods with holisti appearane models are effetive for traking rigid targets (one traker in PROST is an off-line template). They are able to trak the target well when there is no large hange in sale and pose (e.g., out-of-plane rotation). However, it is not surprising that their holisti appearane models (where target objets are enlosed with retangles for representation) are not effetive in aounting for appearane hange due to large pose hange. On the other hand, our traker is more robust to pose variation due to the use of mid-level appearane model, and outperforms other trakers as the proposed superpixel-based disriminative appearane model learns the differene between the target and bakground with updates, whih makes our traker able to handle saling and olusion throughout this sequene. Large shape deformation: The third row (transformer sequene) of Figure 1 shows one example when drasti shape deformation ours, traking algorithms using holisti appearane models or blobs are unlikely to perform well (, and ). The path-based dynami appearane traker (PDAT) [1] is able to trak the target objet in this sequene as its representation sheme is based on loal pathes and not sensitive to non-rigid shape deformation. Nevertheless, without suffiient usage of the appearane information of both target and bakground, the traking results are less aurate. Our appearane model utilizes information of both target and bakground on loal mid-level ues, and distinguishes target parts from bakground bloks preisely. Thus our traker gives the most aurate results. Heavy olusion: The target in the liquor sequene undergoes heavy olusion for many times (the seond row of Figure 6). Sine our superpixel-based disriminative ap-

8 lemming bolt PROST liquor bird1 PROST basketball bird woman girl Figure 7. Traking results omparison of, Visual Traking Deomposition (), Trak, Trak, PROST and our traker. pearane model is able to alleviate influene from bakground pixels and learns the appearane of both target and bakground with superpixels, our traker is able to detet and handle all heavy olusions aordingly. Although the PROST method may reover from drifts after olusion, it does not sueed all the time. Furthermore, the other trakers fail for that they are not able to handle large appearane hange due to heavy olusion or reover from drifts. In the bird1 sequene (third row in Figure 6), the target objet undergoes signifiant non-rigid deformation, rapid motion, pose hange, and olusion for a long duration. Unless a taker is able to distinguish foreground from bakground based on low-level or mid-level ues, it is unlikely to handle heavy olusion and non-rigid deformation simultaneously. Our disriminative appearane model with superpixels enables our traker to detet full olusion and aount for shape deformation at the same time. The other trakers fail mainly due to large appearane hange aused by heavy olusion and shape deformation. In addition to the above-mentioned results, our traker outperforms other state-of-the-art methods in dealing with heavy olusion in the woman and girl sequenes (shown in Figure 1 and Figure 6). 4. Conlusion In this paper, we propose a robust traker based on a disriminative appearane model and superpixels. We show that the use of superpixels provide flexible and effetive mid-level ues, whih are inorporated in an appearane model to distinguish the foreground target and the bakground. The proposed appearane model is used for objet traking to aount for large appearane hange due to shape deformation, olusion and drifts. Numerous experimental results and evaluations demonstrate the proposed traker performs favorably against existing state-of-the-art algorithms in the literature. Referenes [1] A. Adam, E. Rivlin, and I. Shimshoni. Robust fragments-based traking using the integral histogram. In CVPR, pages 79885, 26. [2] S. Avidan. Ensemble traking. In CVPR, pages 49451, 25. [3] B. Babenko, M.-H. Yang, and S. Belongie. Visual traking with online multiple instane learning. In CVPR, pages 98399, 29. [4] R. Collins and Y. Liu. On-line seletion of disriminative traking features. In ICCV, pages , 23. [5] R. T. Collins. Mean-shift blob traking through sale spae. In CVPR (2), pages 23424, 23. [6] D. Comaniiu and P. Meer. Mean shift: A robust approah toward feature spae analysis. PAMI, 24(5):63619, 22. [7] M. Everingham, L. J. V. Gool, C. K. I. Williams, J. M. Winn, and A. Zisserman. The pasal visual objet lasses (vo) hallenge. International Journal of Computer Vision, 88(2):33338, 21. [8] B. Han, Y. Zhu, D. Comaniiu, and L. S. Davis. Visual traking by ontinuous density propagation in sequential Bayesian filtering framework. PAMI, 31(5):91993, 29. [9] A. D. Jepson, D. J. Fleet, and T. F. El-Maraghi. Robust online appearane models for visual traking. In CVPR, pages , 21. [1] J. Kwon and K. M. Lee. Traking of a non-rigid objet via pathbased dynami appearane modeling and adaptive Basin Hopping Monte Carlo sampling. In CVPR, pages , 29. [11] J. Kwon and K. M. Lee. Visual traking deomposition. In CVPR, pages , 21. [12] A. Levinshtein, A. Stere, K. Kutulakos, D. Fleet, S. Dikinson, and K. Siddiqi. Turbopixels: Fast superpixels using geometri flows. PAMI, 31(12): , 29. [13] J. Lim, D. Ross, R.-S. Lin, and M.-H. Yang. Inremental learning for visual traking. In NIPS, pages MIT Press, 25. [14] L. Lu and G. D. Hager. A nonparametri treatment for loation/segmentation based visual traking. In CVPR, 27. [15] G. Mori, X. Ren, A. A. Efros, and J. Malik. Reovering human body onfigurations: Combining segmentation and reognition. In CVPR, pages , 24. [16] K. Nummiaro, E. Koller-Meier, and L. J. V. Gool. An adaptive olorbased partile filter. Image Vision Comput., 21(1):9911, 23. [17] A. Radhakrishna, A. Shaji, K. Smith, A. Luhi, P. Fua, and S. Susstrunk. Sli superpixels. Tehnial Report 1493, EPFL, 21. [18] X. Ren and J. Malik. Learning a lassifiation model for segmentation. In ICCV, pages 117, 23. [19] X. Ren and J. Malik. Traking as repeated figure/ground segmentation. In CVPR, 27. [2] J. Santner, C. Leistner, A. Saffari, T. Pok, and H. Bishof. PROST: Parallel robust online simple traking. In CVPR, pages 72373, 21. [21] D.-N. Ta, W.-C. Chen, N. Gelfand, and K. Pulli. SURFTra: Effiient traking and ontinuous objet reognition using loal feature desriptors. In CVPR, pages , 29. [22] A. Vedaldi and S. Soatto. Quik shift and kernel methods for mode seeking. In ECCV, pages 75718, 28. [23] A. Yilmaz, O. Javed, and M. Shah. Objet traking: A survey. ACM Computing Surveys, 38(4):145, 26.