Background Subtraction via Fast Robust Matrix Completion

Transcription

1 Background Subtraction via Fast Robust Matrix Completion Behnaz Rezaei and Sarah Ostadabbas Augmented Cognition Lab (ACLab) Electrical and Computer Engineering Department Northeastern University, Boston, MA 025 Abstract Background subtraction is the primary task of the majority of video inspection systems. The most important part of the background subtraction which is common among different algorithms is background modeling. In this regard, our paper addresses the problem of background modeling in a computationally efficient way, which is important for current eruption of big data processing coming from high resolution multi-channel videos. Our model is based on the assumption that background in natural images lies on a low-dimensional subspace. We formulated and solved this problem in a low-rank matrix completion framework. In modeling the background, we benefited from the in-face extended Frank-Wolfe algorithm for solving a defined convex optimization problem. We evaluated our fast robust matrix completion (frmc) method on both background models challenge (BMC) and Stuttgart artificial background subtraction (SABS) datasets. The results were compared with the robust principle component analysis (RPCA) and lowrank robust matrix completion (RMC) methods, both solved by inexact augmented Lagrangian multiplier (IALM). The results showed faster computation, at least twice as when IALM solver is used, while having a comparable accuracy even better in some challenges, in subtracting the backgrounds in order to detect moving objects in the scene.. Introduction Background subtraction or foreground detection is the principle task of almost every video inspection algorithm before operating further processing designed for a particular computer vision application. In essence, a robust background subtraction is achieved by creating a model of the background, as depicted in Fig.. The performance of the background subtraction algorithm depends on how well the background model can adapt to the slight and sudden changes of the background scenes. Recently, a huge body of work in this area has focused on modeling the back- Figure : Standard structure common in the majority of the robust background subtraction methods. ground as a low-dimensional subspace in the high dimensional space of video frames. Following the modeling of the background as a lowdimensional subspace, there are various ways to formulate the task of background subtraction as an optimization problem, which has been employed in different works [7, 2, 5, 36]. Although these methods perform well in background/foreground separation, they are designed based on a quadratic optimization problem with heavy structural properties in their iterations. Therefore, the issue of low speed and high memory usage of these methods makes them impractical for online video inspection and prevents them from scaling to big data domain... Prior Works in Background Modeling The most common modeling of the background is based on the probabilistic modeling, which is first proposed in Stauffer and Grimson [29] and then modified by Hayman and Eklundh [4]. In this method, which is called Gaussian mixture model (GMM), distribution of the pixel colors is estimated by sum of different Gaussian distributions and the parameters of each distribution are learned through an online expectation maximization (EM) algorithm. Although GMM can manage the slight changes in the background illumination, immediate variations of the background often appear in the foreground. Moreover, if the initial video frames used for the parameter learning are noisy, the trained model will substantially suffer from the noise. In order to 87

2 improve the performance of the GMM approach, there are various works trying to make this algorithm more robust to noise by proposing different learning methods or modifying the adaptation of the algorithm [9, 30, 37]. Beside GMM, fuzzy and neuro-fuzzy background subtraction methods are also classified as probabilistic background modeling. Authors in [28] provided a comprehensive review of these methods and their performance evaluation. Another class of algorithms for modeling the background is based on the implicit or explicit decomposition of an observation matrix into a low-rank matrix and an additive part. Robust principle component analysis (RPCA) was the first problem formulation for this matrix decomposition, which decomposes a matrix into a superposition of a low-rank and a sparse matrix [7]. Inspired by RPCA, various problem formulations for decomposing a matrix into a low-rank matrix and an additive matrix were proposed such as non-negative matrix factorization (NMF), robust subspace tracking (RST), robust matrix completion (RMC), and robust low-rank minimization (RLRM) [2, 5, 2, 36]. The main difference between these methods is the optimization problem defined by each method and the solver that they used to tackle the problem. Practically, in all of the above methods, background sequence is modeled as the low-rank/low-dimensional subspace and the moving objects form the outliers, which are represented by an additive matrix to the low-rank matrix. Although these algorithms work visually well in modeling the background and its gradual changes over time, they are based on an optimization problem that requires to be solved by computationally expensive iterations. Despite various efforts on reducing their computational complexity, it seems that none of them are able to address the challenges of large scale datasets in videos from real-world applications. A survey on background subtraction algorithms with low-dimensional subspace learning framework can be found in [2, 3, 27]. below, a short review on the state-of-the-art works regarding the above mentioned methods is provided. Initial work on RPCA as one of the methods based on decomposing a matrix into a low rank and a sparse matrix was developed in parallel by three research groups [7, 8, 33]. They all addressed this problem under minimal assumption by solving a convex optimization problem called principle component pursuit (PCP). After the initial effort, many algorithms have been proposed to solve the PCP making an attempt to reduce the computation and memory cost of the problem [9, 20, 26, 35]. Inexact augmented Lagrangian multiplier (IALM) is a successful effort in this regard for solving the problem of RPCA based on the augmented Lagrangian multiplier (ALM) algorithm without taking the unnecessary singular value decomposition (SVD) steps [8]. Works categorized in NMF framework are constructed on decomposing a matrix into a product of two non-negative matrices. Conventional methods of NMF are designed to model the Gaussian or Poisson distributions, so they try to fit one of two mentioned distribution to the background and foreground. Consequently, they do not perform well for background/foreground separation, where their distribution has thick tail. Nonetheless, there have been works improving the performance of NMF algorithms for background modeling using different distance/loss functions [2, 6, 7, 32]. RST algorithms try to address tracking of non-stationary subspaces or in other words, online separation of the foreground and background. Associated subspaces can have low-rank or sparse structures similar to previously discussed methods. Following the idea of RST, there are different algorithms, which try to separate background and foreground iteratively using incremental gradient descent constrained on Grassmannian manifolds [5, 34]. Likewise, both RMC and RLRM methods work based on representing the background as a low-rank matrix..2. Our Contribution Natural videos with static background as the main region of the frames due to the high correlation among frames can be modeled as a low-rank matrix with gross perturbations. Based on this assumption, we propose a method, named fast robust matrix completion (frmc) to model the background in the framework of matrix completion in order to detect the foreground without any prior knowledge about the moving objects. To recover the low-dimensional subspace spanning the background, we formed a quadratic optimization problem, as stated in Equation (3). For solving the corresponding convex optimization problem, which results in a lowdimensional representation of the background, we benefited from the computationally efficient algorithm of in-face extended Frank-Wolfe method proposed in []. Changing the problem formulation and using this solver resulted in more than two times faster computation, while preserving the performance compared to the RPCA with IALM solver. In our proposed frmc, designed based on the in-face extended Frank-Wolfe solver, iterations start from a rank- matrix and SVD gets updated in each iteration based on the previous SVD. In this procedure rank of the updated matrix increases in each iteration by at most one. In contrast, IALM which is currently known as the efficient solver for the RPCA and matrix completion (MC) problems computes a partial SVD in each iteration. Partial SVD returns the singular values of a high rank matrix greater than a specific value. The rank of the updated matrix in IALM monotonically increases with higher slope in each iteration. Since we do not need to calculate SVD in each iteration, the frmc is considerably faster in high-dimensional data processing. Furthermore, there is no need to store the whole matrix representing the background. Instead, we only have to store the left and right hand side singular vectors with singular values 872

3 of a low-rank matrix. Underlying idea inherited from the original Frank-Wolfe method for solving a convex quadratic optimization is solving a linear optimization sub-problem in each iteration to update the next iteration. In-face extended Frank-Wolf makes the iterations as low as possible, while keeping the error in objective function in desired level by proposing the in-face direction in updating phase. Updating the next iteration toward in-face direction without increasing the rank makes this solver efficient in scaling to huge-size convex optimization, which has shown increasing interest in the computer vision field. We evaluated the computational complexity and functionality of our proposed frmc compared to RPCA and RMC both solved with IALM. Evaluation were according to two publicly available datasets including the background models challenge (BMC 202) and the Stuttgart artificial background subtraction (SABS) datasets. 2. Methodology Problem Formulation: Given an observation matrix V consisting of each video frame as its columns, the objective is to recover the underlying low-rank matrix, B, from the foreground perturbations F as: min rank(b) s.t. V=B+F () where the recovered low-rank matrix B works as the background model for the observed video. In the following, we first describe the way that we implicitly separate the moving objects from the background using the framework of the matrix completion as presented in Section 2.. This new formulation allows us to employ a more efficient solver with lower iterations as well as lower computations in each iterations, which results in our computationally efficient frmc method. The details of this solver called in-face extended Frank-Wolf is given in Section 2.2. In Section 2.3, to provide a comparison platform, we then give an overview on the RPCA and how it is used for modeling the background using explicit decomposition of a matrix into a low-rank and sparse matrix. RPCA is the most common formulation for the low-rank representation of the background when it is solved with PCP [7]. As RPCA has shown promising results for the background subtraction, we implemented this method using the IALM solver introduced in [8] and compared results of our proposed problem formulation with this algorithm. In addition, in order to compare the performance and speed of our suggested solver with IALM for the same optimization problem, we used IALM for our problem formulation in Equation () in a form of a conventional robust matrix completion (RMC) algorithm. The performance evaluation and comparison results are given in Section Matrix Completion Framework One way to think about the background subtraction problem is to consider the background throughout the video as a collection of high-dimensional data lying in a lowdimensional subspace due to the high correlation of the backgrounds among video frames. So far, most of the optimization problems formulated for modeling the background as a low-rank matrix have imposed the sparse structure to the representing matrix of the foreground by l -norm minimization. However, it is not the only way to impose sparsity. As it is theoretically and experimentally shown, under mild conditions, Frobenius norm can be effective in imposing sparsity [23 25]. Considering this theory, we modeled the background as a low-dimensional subspace and implicitly impose sparsity to the foreground by minimizing the Frobenius norm, which is strictly convex compared to using l -norm minimization. In order to form our optimization problem we used the low-rank matrix completion (LRMC) concept. In LRMC, we have a partially observed data matrix D R n,n 2 containing n 2 observations, each of dimension n, which only a fraction of its entries are available and the task is to predict unobserved entries of the D matrix. Matrix Z R n,n 2 is the estimated matrix of D containing the prediction of unobserved entries. In general, without any assumption on the underlying structure of the estimated matrix Z, this problem is ill-posed and the data matrix can be filled up with any real values. The most common assumption without having any prior knowledge about the data distribution is to restrict matrix Z to a low-rank matrix []. This assumption is accurate in many real applications with high dimensionality, including background subtraction or moving object detection. The relaxed optimization problem of the LRMC is stated as: min f(z) := /2 (i, j) Ω (z i j d i j ) 2 s.t. Z < δ (2) where f( ) is the least squares error between the available observed data entries d i j and estimated data entries z i j, and Ω is the subset containing all indices of observed entries.. indicates the nuclear norm of a matrix, which is defined as sum of its singular values, and δ is the constraining upper band for the nuclear norm ball of estimated low-rank matrix Z. It is assumed that the observed entries are contaminated by noise with bounded energy and minimizing the Z imposes the low-rank constraint on the estimated matrix. Although in the background subtraction context we do not have any missing data, instead we want to recover the background from observations corrupted by foreground (as noise). Considering the fact that background is lying on a low-dimensional subspace compared to the space domain of the video frames, task of the background subtraction can 873

4 be restated in the LRMC framework. If data matrix V is formed by arranging the video frames in each column of it, in order to recover the background from video frames, we estimate the low-rank matrix B out of the uncontaminated entries of the observed video V. On the other hand, foreground is the difference of the estimated background video and the original video, which in the LRMC is the sparse bounded energy perturbation of the background. Low-rank background model in the context of the LRMC is restated in solving the following optimization problem: min B V 2 F s.t. B < δ (3) where. F indicates the Frobenius norm of a matrix defined as root sum squared of matrix entries, and (B V) is related to the foreground perturbations in videos Extended Frank Wolfe Method for LRMC Basically, LRMC optimization formulation is a specific case of RPCA objective function, therefore this problem can be solved using the IALM method. However, IALM has scaling problem to massive datasets, which arise in image/video processing. Multi-channel videos usually have millions of dimensions, hence implementation of IALM, considering that it requires taking an SVD in each iteration, is not a feasible solution. One of the algorithms designed for convex problems with better scalability is Frank- Wolfe method and its extensions featuring linear convergence [, 3, 22]. In order to recover the substantial lowrank matrix B, which represents the background throughout video frames, we have used in-face extended Frank-Wolfe algorithm proposed in []. In fact, Equation (2) is an example of a more general problem as below, which is addressed by Frank-Wolf method: f := min f(x) x S where S is a closed and bounded convex set, and f( ) is a differentiable convex function defined on subset S. Original Frank-Wolfe method finds the sub-optimal solution for this problem based on the procedure described in Algorithm.. C i is the optional lower bound for the optimal objective function f and is updated in steps 3 and 4 of each iteration. This lower bound is effective in conditioning the number of iterations. ᾱ i is the updating step size, which can be selected by line-search or simple rule of ᾱ i := i+ 2. There are strong mathematical verification for the computational guarantee of Frank-Wolf method [0]. The main computational load of the Frank-Wolf algorithm is solving the linear optimization problem in step 2 of Algorithm.. In this work, we used in-face extended Frank-Wolfe method to solve the problem of foreground detection introduced in Equation (3). This method computes and works (4) Algorithm : Original Frank-Wolfe method for the optimization problem in Equation (4). Result: x initialization: x 0, lower bound: C, i 0 ; while not converged do. compute f(x); 2. x i argmin{ f(x i )+ f(x i ) T (x x i )} ; 3. C ω i f(x i )+ f(x i ) T ( x x i ); // updating the best bound 4. C i max{c i,c ω i }; // updating x 5. x i+ x i + ᾱ i ( x i x i ), ᾱ i [0,]; end with points that have specific structure of low-rank in the case when x is a data matrix. The main advantage of this algorithm toward optimizing the objective function f(x) is iterating in low-dimensional faces of S. Such low-rank iteration not only make the output solution low-rank, but also result in an essential reduction in computation cost. This method adopted for the specific problem of foreground detection in Equation (3) is presented in Algorithm. 2. In this algorithm, B i = UDV T denotes the SVD of the current iteration, and u i and v i are the left and right singular vectors corresponding to the largest singular values of the matrix f(b i ). At step 2 of each iteration, algorithm works for the in-face direction q i which preserves the next estimated point in the minimal face F C (B i ). In step 4, algorithm chooses between three possible next steps: B i b that lies in the relative boundary of the current minimal face, B i a that may not lie in the relative boundary of the current minimal face, and a regular Frank-Wolf update in step 4(c). Each of these steps are chosen based on the decrease in the optimality bound gap criterion: f(b i b/a ) C i f(b i ) C i + γ /2 2 L D 2 (5) where B i a and B i b are the two in-face candidates for the next update of the B i. L is the Lipschitz constant of the gradient of f(.) on nuclear norm ball, and D is the diameter of the convex bounded set in the optimization problem, which in our case is the diameter of the nuclear norm ball equal to 2δ. γ and γ 2 are two constants that control the convergence properties of the algorithm. When we want the criterion in Equation (5) be easily satisfied particularly in low-rank matrix completion, γ and γ 2 are preferred to be lower than higher. Defining the problem of background subtraction as the convex optimization problem in Equation (3) and benefiting from high performance in-face extended Frank-Wolf with verified convergence properties to solve the proposed prob- 874

5 Algorithm 2: In-face extended Frank-Wolfe method for the optimization problem in Equation (3). Result: B Input: V; Constants: 0 γ γ 2, L L=, D D=2δ ; Definition: f(b i )=/2 B i V 2 F, f(b i )=(B i V), B X := tr{b T X} ; Initialization: B 0 δu 0 v T 0,i 0 lower bound : C max{ f(0)+ f(0)}; while not converged do. f(b i )=(B i V), C i C i ; 2. compute direction q i ˆB i argmax f(b i ) B; B F C (B i ) q i B i ˆB i case : in the case B i int(c) and when B i (C) F C (B i )= C ˆB i = δu i v T i ; case 2: in the case B i (C), rank(b i )=r B i = UDV T ˆB i U ˆM i V T = δuu i u T i VT 3. compute step size α stop i {α : B i + αq i F C (B i )}; B i b := Bi + α stop i q i ; B i a := B i + β i q i where β i [0,α stop i ]; end 4. choose next iterate: (a) if f(b i b ) C i f(b i ) C i + γ 2 L D 2 : B i+ B i a; (b) if f(b i a) C i f(b i ) C i + γ 2 2 L D 2 : B i+ B i b ; (c) Else: do the original Frank-Wolfe step with the input x i = B i and update lower bound described in Algorithm. ; lem configure our frmc method An Overview of Robust Principle Component Analysis Under two assumptions that the low-rank matrix is not sparse and sparsity pattern of the sparse component is selected randomly, RPCA can decompose a matrix into a superposition of a low-rank matrix and a sparse matrix. Despite the differences between RPCA and matrix completion, the recovery of the low-rank matrix in the RPCA can be treated as a low-rank matrix completion. In this case, the matrix completion deviates from recovering a low-rank matrix from a fraction of its components to recovering a lowrank matrix by having an unknown fraction of them available, while the rest of them are grossly corrupted [7]. J. Candes in [7] approaches the problem of recovering the incomplete and corrupted entries as a convex optimization: min B + λ F s.t. V=B+F (6) where. is l -norm of a matrix defined as the sum of absolute values of matrix entries, and λ is a positive regularizing coefficient. Efficient algorithm for this optimization is augmented Lagrangian multiplier (ALM) introduced in [20, 35]. ALM algorithm inspired from the theory that first-order iterative thresholding algorithms can be used efficiently for both l - norm and nuclear-norm minimization [, 5, 6]. The original ALM is designed for solving the constrained optimization problem of: min f(x) s.t. g(x) = 0 (7) where f : R n R and g : R n R n 2. ALM algorithm converts constrained problem in Equation (7) to an unconstrained problem by adding a penalty term that punishes violations from the equality constraint: L ρ (X,Y)= f(x)+ Y,g(X) + ρ 2 g(x) 2 F (8) where Y is a regularizer matrix, Y,g(X) is defined as trace(y T g(x)) and ρ is a positive scalar. ALM algorithm formulation for the RPCA problem is identified as: then: L ρ (B,F,Y)= X =(B,F), f(x)= B + λ F, g(x)=v B F. B + λ F + Y,V B F + ρ 2 V B F 2 F (9) (0) The state-of-the art solver for the unconstrained optimization problem in Equation (0) in terms of speed and computational guarantee is the IALM, which its major computational cost is a partial SVD in each iteration. Considering n as the max(n,n 2 ), it is shown that for incoherent B, correct recovery occurs with high probability when the rank of B is in the order of n/µ(logn) 2, where µ > and the number of nonzero elements in F is on the order of n 2. A choice of λ = / n works with high probability for recovering the low-rank matrix [9, 20]. As we pointed out earlier, IALM solver can be used for solving the unconstrained optimization problem related to the low-rank matrix completion as a special case of the objective function of RPCA demonstrated as: L ρ (B,Y)= B + Y,V B + ρ 2 V B 2 F () 875

6 3. Experimental Analysis 3.. Performance Evaluation on BMC Dataset In order to validate and compare our proposed frmc formulation in terms of computational time and detection performance, we tested frmc, RPCA, and RMC on the BMC 202 dataset [3]. This dataset contains 9 real videos included with encrypted ground-truth images of the foreground to test the background models. The software given by the website (called BMCWizard ) measures two types of metrics for performance evaluation: static quality metrics and application quality metrics. Regarding the static quality two scores are measured in this application: F-measure and peak signal-noise ratio (PSNR). These two benchmarks are introduced to evaluate the raw behavior of each algorithm for moving object segmentation. Considering F as the set of n foreground frames processed by the background subtraction algorithm, and G as the ground-truth images, these two metrics are defined as follows: F = n n i= 2 Prec i Rec i Prec i + Rec i where : T P i T P i Rec i (P)= ; Prec i (P)= ; T P i + FN i T P i + FP i T N i T N i Rec i (N)= ; Prec i (N)= T N i + FP i T N i + FN i Rec i = /2(Rec i (P)+Rec i (P)); Prec i = /2(Prec i (P)+Prec i (P)) (2) where for given frame i, T P i and FP i are the true and false positive detection, and T N i and FN i are the true and false negative detection. And, PSNR= n n i= 0log 0 n n j= F i ( j) G i ( j) 2 (3) where F i ( j) is the jth pixel of the image i (of size n ) in the frame sequence F. For the application quality metrics that consider the problem of background subtraction in a visual and perceptual way, BMCWizard calculates two other benchmark: Structural similarity (SSIM) and D-score. SSIM is calculated as: SSIM(F,G)= n n i= (2µ Fi µ Gi + c )(2cov Fi G i + c 2 ) (µ 2 F i + µ 2 G i + c )(σ 2 F i + σ 2 G i + c 2 ) (4) where µ Fi, µ Gi are the means and σ Fi, σ Gi are the standard deviations and cov Fi G i is the covariance of the F i and G i. c and c 2 are two constants chosen as and , respectively. Associated to the application quality, D-score considers localization of errors based on the real object position. To compute this benchmark, only mistakes in the background subtraction algorithm is taken into account: D score(f i ( j))=exp(( log 2 (2DT(F i ( j)) 5/2) 2 ) (5) where DT(F i ( j)) is given by minimal distance between the pixel F i ( j) and the nearest reference point (here by Baddeley distance). With this metric, local/far errors will produce a near zero D-score. In contrast, medium range errors produce high D-score; a good D-score tends to zero. We tested our algorithm on six videos of the BMC dataset in comparison with both RPCA and RMC. The performance metrics calculated by BMCWizard are reported in Table. As demonstrated by the results, with almost the same performance (even better in some metrics) our frmc algorithm processes the same video in less than half of the time required by RPCA. Experiments were executed in MATLAB installed on a machine with Intel(R) Xeon(R)@ 3.60GHz, 6 Cores CPU and 32GB RAM. Some examples of background and foreground masks resulted from RPCA, RMC and our frmc background subtraction methods are shown in Fig. 2. As you can see, all outcomes are very comparable in mask detection Performance Evaluation on SABS Dataset As the second part of our performance evaluation we tested the functionality of frmc, RPCA, and RMC algorithms in background/ foreground separation on the SABS dataset [4]. This dataset contains 9 artificial videos covering the typical challenges in the background subtraction. The considered challenges are: Gradual illumination changes: to measure the robustness of the background models in gradual changes of the environment (e.g. variation of light intensity in outdoor settings). Sudden illumination changes: considering the strong changes in the background appearance, which can cause false positive detection (e.g. sudden switch off). Dynamic background: taking into account situations with some moving components that are relevant to the background (e.g. traffic lights or trees). Camouflage: one of the important challenges in background modeling especially in surveillance applications is similar appearance of some objects to the background, making the precise classification difficult. Shadows: shadows are irrelevant areas that prevent the classifier to separate the nearby foreground objects as they overlap. Video noise: inevitable issue in recording the video is the sensor noise or compression artifact that degrades the signal quality. 876

7 Table : Computational time and benchmark metrics for the background subtraction task of our frmc algorithm compared to RPCA and RMC evaluated on the BMC dataset. Algorithm Running time (min) F-measure D-score Precision PSNR Ssim Big trucks RPCA RMC frmc Wandering students RPCA RMC frmc Rabbit in the night RPCA RMC frmc Beware of the trains RPCA RMC frmc Train in the tunnel RPCA RMC frmc (a) (b) (c) Figure 2: Result of applying the three methods (a) RPCA, (b) RMC, and (c) frmc on BMC wandering students video demonstrated for two different frames; first row is the original frame, second row is the binary mask of the foreground by thresholding the detected foreground, and last row is the background image. Each video reflects one or more of the above challenges. We tuned the parameters of the algorithms based on the Basic video in the dataset and use the same parameters for all the videos. The benchmark metrics and execution time of each method are reported in Table 2. The metrics consist of F-measure and precision defined in Equation (2) and were measured using the evaluation framework provided by the institute for visualization and interactive systems (VIS) 2. We demonstrated the result of applying the RPCA, RMC, and frmc on the SABS light switch video in Fig. 3. This video covers the challenge of sudden illumination changes and as you can see from the images, RMC cannot model the background satisfactorily and the window is shown as part of the foreground which confirms the low score of RMC in Table 2 for this video. 2 information-visualisation-and-visual-analytics/ visual-analytics-of-video-data/sabs.html 877

8 (a) (b) (c) Figure 3: Result of applying the three methods (a) RPCA, (b) RMC, and (c) frmc on SABS light switch video demonstrated for two different frames; first row is the original frame, second row is the binary mask of the foreground by thresholding the detected foreground, and last row is the background image. As you can see RMC fails in modeling the background in abrupt change of light. Table 2: Computational time and benchmark metrics for the background subtraction task of our frmc algorithm compared to RPCA and RMC evaluated on the SABS datasets. shows the best result for each experiment reported by VIS for SABS dataset. Algorithm Running time (min) Precision F-measure Basic RPCA RMC frmc Best result Light switch RPCA RMC frmc Best result Camouflage RPCA RMC frmc Best result No camouflage RPCA RMC frmc Best result Noisy night RPCA RMC frmc Best result MPEG4 (40Kbps) RPCA RMC frmc Best result Conclusion In this paper, we proposed our fast robust matrix completion (frmc) method to address the problem of time efficient background subtraction by finding the low-dimensional subspace which background lies on that. For recovering the associated subspace, we represented our problem in a matrix completion framework, and benefited from the in-face extended Frank-Wolfe algorithm to solve corresponding optimization problem. We validated our algorithm on the BMC 202 and SABS datasets in comparison with RPCA and RMC algorithms both solved by IALM solver. With almost the same performance or better in some videos, we accomplished the task with less than half of the execution time required for the RPCA and RMC. References [] A. Beck and M. Teboulle. A fast iterative shrinkagethresholding algorithm for linear inverse problems. SIAM journal on imaging sciences, 2():83 202, [2] T. Bouwmans, N. S. Aybat, and E.-h. Zahzah. Handbook of Robust Low-Rank and Sparse Matrix Decomposition: Applications in Image and Video Processing. CRC Press, [3] T. Bouwmans, A. Sobral, S. Javed, S. K. Jung, and E.-H. Zahzah. Decomposition into low-rank plus additive matrices for background/foreground separation: A review for a comparative evaluation with a large-scale dataset. Computer Science Review, 23: 7, [4] Brutzer, S. and Höferlin, Benjamin and Heidemann, Gunther. Evaluation of Background Subtraction Techniques for Video Surveillance. In Computer Vision and Pattern Recognition (CVPR), pages IEEE, [5] J.-F. Cai, E. J. Candès, and Z. Shen. A singular value thresholding algorithm for matrix completion. SIAM Journal on 878

9 Optimization, 20(4): , [6] J.-F. Cai, S. Osher, and Z. Shen. Linearized bregman iterations for compressed sensing. Mathematics of Computation, 78(267):55 536, [7] E. J. Candès, X. Li, Y. Ma, and J. Wright. Robust principal component analysis? Journal of the ACM (JACM), 58(3):, 20., 2, 3, 5 [8] V. Chandrasekaran, S. Sanghavi, P. A. Parrilo, and A. S. Willsky. Rank-sparsity incoherence for matrix decomposition. SIAM Journal on Optimization, 2(2): , [9] Y.-T. Chen, C.-S. Chen, C.-R. Huang, and Y.-P. Hung. Efficient hierarchical method for background subtraction. Pattern Recognition, 40(0): , [0] R. M. Freund and P. Grigas. New analysis and results for the frank wolfe method. Mathematical Programming, 55(- 2):99 230, [] R. M. Freund, P. Grigas, and R. Mazumder. An extended frank wolfe method with in-face directions, and its application to low-rank matrix completion. SIAM Journal on Optimization, 27():39 346, , 3, 4 [2] N. Guan, D. Tao, Z. Luo, and J. Shawe-Taylor. Mahnmf: Manhattan non-negative matrix factorization. arxiv preprint arxiv: , 202., 2 [3] J. Guélat and P. Marcotte. Some comments on wolfe s away step. Mathematical Programming, 35():0 9, [4] E. Hayman and J.-O. Eklundh. Statistical background subtraction for a mobile observer. In ICCV, volume, [5] J. He, L. Balzano, and A. Szlam. Incremental gradient on the grassmannian for online foreground and background separation in subsampled video. In Computer Vision and Pattern Recognition (CVPR), 202 IEEE Conference on, pages IEEE, 202., 2 [6] A. Kumar and V. Sindhwani. Near-separable non-negative matrix factorization with and bregman loss functions. In Proceedings of the 205 SIAM International Conference on Data Mining, pages SIAM, [7] A. Kumar, V. Sindhwani, and P. Kambadur. Fast conical hull algorithms for near-separable non-negative matrix factorization. In ICML (), pages , [8] Z. Lin, M. Chen, and Y. Ma. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arxiv preprint arxiv: , , 3 [9] Z. Lin, A. Ganesh, Wright, and et. al. Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix. Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP), 6(6), , 5 [20] Z. Lin, R. Liu, and Z. Su. Linearized alternating direction method with adaptive penalty for low-rank representation. In Advances in neural information processing systems, pages , 20. 2, 5 [2] H. Mansour and A. Vetro. Video background subtraction using semi-supervised robust matrix completion. In 204 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages , May [22] C. Mu, Y. Zhang, J. Wright, and D. Goldfarb. Scalable robust matrix recovery: Frank wolfe meets proximal methods. SIAM Journal on Scientific Computing, 38(5):A329 A337, [23] X. Peng, C. Lu, Z. Yi, and H. Tang. Connections between nuclear-norm and frobenius-norm-based representations. IEEE Transactions on Neural Networks and Learning Systems, [24] X. Peng, J. Lu, Z. Yi, and R. Yan. Automatic subspace learning via principal coefficients embedding. IEEE Transactions on Cybernetics, 206. [25] X. Peng, Z. Yu, Z. Yi, and H. Tang. Constructing the l2- graph for robust subspace learning and subspace clustering. IEEE transactions on cybernetics, [26] Y. Shen, Z. Wen, and Y. Zhang. Augmented lagrangian alternating direction method for matrix separation based on low-rank factorization. Optimization Methods and Software, 29(2): , [27] A. Sobral, T. Bouwmans, and E.-h. Zahzah. Comparison of Matrix Completion Algorithms for Background Initialization in Videos, pages Springer International Publishing, Cham, [28] A. Sobral and A. Vacavant. A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos. Computer Vision and Image Understanding, 22:4 2, [29] C. Stauffer and W. E. L. Grimson. Adaptive background mixture models for real-time tracking. In Computer Vision and Pattern Recognition, 999. IEEE Computer Society Conference on., volume 2, pages IEEE, 999. [30] O. Tuzel, F. Porikli, and P. Meer. A bayesian approach to background modeling. In Computer Vision and Pattern Recognition-Workshops, CVPR Workshops. IEEE Computer Society Conference on, pages IEEE, [3] A. Vacavant, T. Chateau, A. Wilhelm, and L. Lequièvre. A benchmark dataset for outdoor foreground/background extraction. In Asian Conference on Computer Vision, pages Springer, [32] H. Woo and H. Park. Robust asymmetric nonnegative matrix factorization. Computational and Applied Mathematics Reports, University of California, USA, [33] J. Wright, A. Ganesh, S. Rao, Y. Peng, and Y. Ma. Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. In Advances in neural information processing systems, pages , [34] J. Xu, V. K. Ithapu, L. Mukherjee, J. M. Rehg, and V. Singh. Gosus: Grassmannian online subspace updates with structured-sparsity. In Proceedings of the IEEE International Conference on Computer Vision, pages , [35] X. Yuan and J. Yang. Sparse and low-rank matrix decomposition via alternating direction methods. preprint, 2, , 5 [36] X. Zhou, C. Yang, and W. Yu. Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(3):597 60, 203., 2 [37] Z. Zivkovic and F. Van Der Heijden. Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern recognition letters, 27(7): ,