Orthogonal Matching Pursuit: Recursive Function Approximation with Applications to Wavelet. Y. C. Pati R. Rezaiifar and P. S.
|
|
- Buck Carson
- 6 years ago
- Views:
Transcription
1 / To appear in Proc. of the 27 th Annual Asilomar Conference on Signals Systems and Computers, Nov. {3, 993 / Orthogonal Matching Pursuit: Recursive Function Approximation with Applications to Wavelet Decomposition Y. C. Pati R. Rezaiifar and P. S. Krishnaprasad Information Systems Laboratory Dept. of Electrical Engineering Institute for Systems Research Dept. of Electrical Engineering Stanford University, Stanford, CA 935 University of Maryland, College Par, MD 272 Abstract In this paper we describe a recursive algorithm to compute representations of functions with respect to nonorthogonal and possibly overcomplete dictionaries of elementary building blocs e.g. ane (wavelet) frames. We propose a modication to the Matching Pursuit algorithm of Mallat and Zhang (992) that maintains full bacward orthogonality of the residual (error) at every step and thereby leads to improved convergence. We refer to this modied algorithm as Orthogonal Matching Pursuit (OMP). It is shown that all additional computation required for the OMP algorithm may be performed recursively. Introduction and Bacground Given a collection of vectors D = fx i g in a Hilbert space H, let us dene V = Spanfx n g; and W = V? (in H): We shall refer to D as a dictionary, and will assume the vectors x n, are normalized (x n = ). In [3] Mallat and Zhang proposed an iterative algorithm that they termed Matching Pursuit (MP) to construct representations of the form P V n a n x n ; () where P V is the orthogonal projection operator onto V. Each iteration of the MP algorithm results in an intermediate representation of the form i= a i x ni + R f + R f; where f is the current approximation, and R f the current residual (error). Using initial values of R f, f =, and =, the MP algorithm is comprised of the following steps, (I) Compute the inner-products fhr f; x n ig n. (II) Find n + such that (III) Set, R f; x n+ sup j where <. jhr f; x j ij ; f + = f + R f; x n+ xn+ R + R f? R f; x n+ xn+ (IV) Increment, ( +), and repeat steps (I){ (IV) until some convergence criterion has been satised. The proof of convergence [3] of MP relies essentially on the fact that R + f; x n+ =. This orthogonality of the residual to the last vector selected leads to the following \energy conservation" equation. R f 2 = R + f 2 + R f; x n+ 2 : (2) It has been noted that the MP algorithm may be derived as a special case of a technique nown as Projection Pursuit (c.f. [2]) in the statistics literature. A shortcoming of the Matching Pursuit algorithm in its originally proposed form is that although asymptotic convergence is guaranteed, the resulting approximation after any nite number of iterations will in general be suboptimal in the following sense. Let N <,
2 be the number of MP iterations performed. Thus we have f N = N? = R f; x n+ xn+: Dene V N = Spanfx n ; : : :; x nn g. We shall refer to f N as an optimal N-term approximation if f N = P VN f, i.e. f N is the best approximation we can construct using the selected subset fx n ; : : :; x nn g of the dictionary D. (Note that this notion of optimality does not involve the problem of selecting an \optimal" N-element subset of the dictionary.) In this sense, f N is an optimal N-term approximation, if and only if R N f 2 V? N. As MP only guarantees that R N f? x nn, f N as generated by MP will in general be suboptimal. The diculty with such suboptimality is easily illustrated by a simple example in IR 2. Let x, and x 2 be two vectors in IR 2, and tae f 2 IR 2, as shown in Figure. Figure (b) is a plot of R f 2 (b) Normalized Error π/3 x 2 π/8 x Figure : Matching pursuit example in IR 2 : Dictionary D = fx ; x 2 g and a vector f 2 IR 2 versus. Hence although asymptotic convergence is guaranteed, after any nite number of steps, the error may still be quite large. In this paper we propose a renement of the Matching Pursuit (MP) algorithm that we refer to as Orthogonal Matching Pursuit (OMP). For nonorthogonal dictionaries, OMP will in general converge faster than MP. For any nite size dictionary of N elements, OMP converges to the projection onto the span of the dictionary elements in no more than N steps. Furthermore after any nite number of iterations, OMP A simlar diculty with the Projection Pursuit algorithm was noted by Donoho et.al. [] who suggested that bactting may be used to improve the convergence of PPR. Although the technique is not fully described in [] it appears that it is in the same spirit as the technique we present here. f gives the optimal approximation with respect to the selected subset of the dictionary. This is achieved by ensuring full bacward orthogonality of the error i.e. at each iteration R f 2 V?. For the example in Figure, OMP ensures convergence in exactly two iterations. It is also shown that the additional computation required for OMP, taes a simple recursive form. We demonstrate the utility of OMP by example of applications to representing functions with respect to time-frequency localized ane wavelet dictionaries. We also compare the performance of OMP with that of MP on two numerical examples. 2 Orthogonal Matching Pursuit Assume we have the following th -order model for f 2 H, a n x n+r f; with hr f; x n i = ; n = ; : : :: (3) The superscript, in the coecients a n, show the dependence of these coecients on the model-order. We would lie to update this th -order model to a model of order +, + a + n x n + R + f; with hr + f; x n i =, n = ; : : : +. () Since elements of the dictionary D are not required to be orthogonal, to perform such an update, we also require an auxiliary model for the dependence of x + on the previous x n 's (n = ; : : :). Let, x + = b nx n + ; with h ; x n i = ; n = ; : : :: P (5) Thus b nx n = P V x +, and = P V? x +, is the component of x + which is unexplained by fx ; : : :; x g. Using the auxiliary model (5), it may be shown that the correct update from the th -order model to the model of order +, is given by a + n = a n? b n; n = ; : : :; (6) and a + + = ; where = hr f; x + i h ; x + i = = hr f; x + i 2 hr f; x + i x + 2? P b n hx n ; x + i : 2
3 It also follows that the residual R + f satises, R R + f + ; and R f 2 = R + f 2 + jhr f; x + ij 2 2. The OMP Algorithm 2 : (7) The results of the previous section may be used to construct the following algorithm that we will refer to as Orthogonal Matching Pursuit (OMP). Initialization: f = ; R f; D = f g x = ; a = ; = (I) Compute fhr f; x n i ; x n 2 D n D g. (II) Find x n+ 2 D n D such that R f; x n+ sup j jhr f; x j ij ; < : (III) If R f; x n+ <, ( > ) then stop. (IV) Reorder the dictionary D, by applying the permutation + $ n +. (V) Compute b n, such P that, x + = b nx n + and h ; x n i = ; n = ; : : :; : (VI) Set, a + + = =?2 hr f; x + i, (VII) Set a + n = a n? b n; n = ; : : :; ; and update the model, f + = + a + n x n R + f? f + D + = D [ fx+ g: +, and repeat (I){(VII). Theorem 2. For f 2 H, let R f be the residuals generated by OMP. Then (i) lim! R f? P V? : (ii) f N = P VN f; N = ; ; 2; : : :. Proof: The proof of convergence parallels the proof of Theorem in [3]. The proof of the the second property follows immediately from the orthogonality conditions of Equation (3). Remars: The ey dierence between MP and OMP lies in Property (iii) of Theorem 2.. Property (iii) implies that at the N th step we have the best approximation we can get using the N vectors we have selected from the dictionary. Therefore in the case of nite dictionaries of size M, OMP converges in no more than M iterations to the projection of f onto the span of the dictionary elements. As mentioned earlier, Matching Pursuit does not possess this property. 2.3 Some Computational Details As in the case of MP, the inner products fhr f; x j ig may be computed recursively. For OMP we may express these recursions implicitly in the formula hr f; x j i = hf? f ; x j i = hf; x j i? a n hx n ; x j i : (8) The only additional computation required for OMP, arises in determining the b n's of the auxiliary model (5). To compute the b n's we rewrite the normal equations associated with (5) as a system of linear equations, v = A b ; (9) where v = [hx + ; x i ; hx + ; x 2 i : : : ; hx + ; x i] T b = b ; b 2 ; : : :; b T 2.2 Some Properties of OMP As in the case of MP, convergence of OMP relies on an energy conservation equation that now taes the form (7). The following theorem summarizes the convergence properties of OMP. and A = 2 6 hx ; x i hx 2 ; x i hx ; x i hx ; x 2 i hx 2 ; x 2 i hx ; x 2 i hx ; x i hx 2 ; x i hx ; x i : 3
4 Note that the positive constant used in Step (III) of OMP guarantees nonsingularity of the matrix A, hence we may write b = A? v : () However, since A + may be written as A + = A v v ; () (where denotes conjugate transpose) it may be shown using the bloc matrix inversion formula that A A? =? + b b?b +?b ; (2) where = =(? v b ). Hence A? +, and therefore b +, may be computed recursively using A?, and b from the previous step. 3 Examples In the following examples we consider representations with repect to an ane wavelet frame constructed from dilates and translates of the second derivate of a Gaussian, i.e. D = f m;n ; m; n 2 Zg where, m;n(x) = 2 m=2 (2 m x? n); and the analyzing wavelet is given by, (x) = 3 p =2? x 2? e?x2 =2 : Note that for wavelet dictionaries, the initial set of inner products fhf; m;n ig, are readily computed by one convolution followed by sampling at each dilation level m. The dictionary used in these examples consists of a total of 35 vectors. In our rst example, both OMP and MP were applied to the signal shown in Figure 2. We see from Figure 2(b) that OMP clearly converges in far fewer iterations than MP. The squared magnitude of the coecients a, of the resulting representation is shown in Figure 3. We could also compare the two algorithms on the basis of required computational eort to compute representations of signals to within a prespecied error. However such a comparison can only be made for a given signal and dictionary, as the number of iterations required for each algorithm depends on both the signal and the dictionary. For example, for the signal of Example I, we see from Figure that it is Original Signal and OMP Approximation Normalized L2 Error MP OMP Figure 2: Example I : Original signal f, with OMP approximation superimposed, (b) Squared L 2 norm of residual R f versus iteration number, for both OMP (solid line) and MP (dashed line). Dilation Index Translation Index Figure 3: Distribution of coecients obtained by applying OMP in Example I. Shading is proportional to squared magnitude of the coecients a, with dar colors indicating large magnitudes. to 8 times more expensive to achieve a prespecied error using OMP even though OMP converges in fewer iterations. On the other hand for the signal shown in Figure 5, which lies in the span of three dictionary vectors, it is approximately 2 times more expensive to apply MP. In this case OMP converges in exactly three iterations. Summary and Conclusions In this paper we have described a recursive algorithm, which we refer to as Orthogonal Matching Pursuit (OMP), to compute representations of signals with respect to arbitrary dictionaries of elementary functions. The algorithm we have described is a modication of the Matching Pursuit (MP) algorithm of Mallat and Zhang [3] that improves convergence us-
5 Cost (FLOPS) MP OMP Log of Normalized L2 Error Figure : Computational cost (FLOPS) versus approximation error for both OMP (solid line) and MP (dashed line) applied to the signal in Example I...2 at each step. Acnowledgements This research of Y.C.P. was supported in part by NASA Headquarters, Center for Aeronautics and Space Information Sciences (CASIS) under Grant NAGW9,S6, and in part by the Advanced Research Projects Agency of the Department of Defense monitored by the Air Force Oce of Scientic Research under Contract F This research of R.R. and P.S.K was supported in part by the Air Force Oce of Scientic Research under contract F J-5, the AFOSR University Research Initiative Program under Grant AFOSR-9-5, by the Army Research Oce under Smart Structures URI Contract no. DAAL3-92-G-2, and by the National Science Foundation's Engineering Research Centers Program, NSFD CDR (b) Normalized L2 Error MP OMP Figure 5: Example II: Original signal f, (b) Squared L 2 norm of residual R f versus iteration number, for both OMP (solid line) and MP (dashed line). References [] D. Donoho, I. Johnstone, P. Rousseeuw, and W. Stahel. The Annals of Statistics, 3(2):96{ 5, 985. Discussion following article by P. Huber. [2] P. J. Huber. Projection pursuit. The Annals of Statistics, 3(2):35{75, 985. [3] S. Mallat and Z. Zhang. Matching pursuits with time-frequency dictionaries. Preprint. Submitted to IEEE Transactions on Signal Processing, 992. ing an additional orthogonalization step. The main benet of OMP over MP is the fact that it is guaranteed to converge in a nite number of steps for a nite dictionary. We also demonstrated that all additional computation that is required for OMP may be performed recursively. The two algorithms, MP and OMP, were compared on two simple examples of decomposition with respect to a wavelet dictionary. It was noted that although OMP converges in fewer iterations than MP, the computational eort required for each algorithm depends on both the class of signals and choice of dictionary. Although we do not provide a rigorous argument here, it seems reasonable to conjecture that OMP will be computationally cheaper than MP for very redundant dictionaries, as nowledge of the redundancy is exploited in OMP to reduce the error as much as possible 5
Orthogonal Matching Pursuit: Recursive Function Approximat ion with Applications to Wavelet Decomposition
Orthogonal Matching Pursuit: Recursive Function Approximat ion with Applications to Wavelet Decomposition Y. C. PATI Information Systems Laboratory Dept. of Electrical Engineering Stanford University,
More informationY. C. PATI Information Systems Laboratory Dept. of Electrical Engineering Stanford University, Stanford, CA 94305
i 4ô- / To appear in Proc. of the Annual Asilomar Conference on Signals Systems and Computers, Nov. 1-3, 1993/ Orthogonal Matching Pursuit: Recursive Function Approximation with Applications to Wavelet
More informationThe Viterbi Algorithm for Subset Selection
524 IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 5, MAY 2015 The Viterbi Algorithm for Subset Selection Shay Maymon and Yonina C. Eldar, Fellow, IEEE Abstract We study the problem of sparse recovery in
More informationEfficient Implementation of the K-SVD Algorithm and the Batch-OMP Method
Efficient Implementation of the K-SVD Algorithm and the Batch-OMP Method Ron Rubinstein, Michael Zibulevsky and Michael Elad Abstract The K-SVD algorithm is a highly effective method of training overcomplete
More informationEfficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit
Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit Ron Rubinstein, Michael Zibulevsky and Michael Elad Abstract The K-SVD algorithm is a highly effective method of
More informationHowever, m pq is just an approximation of M pq. As it was pointed out by Lin [2], more precise approximation can be obtained by exact integration of t
FAST CALCULATION OF GEOMETRIC MOMENTS OF BINARY IMAGES Jan Flusser Institute of Information Theory and Automation Academy of Sciences of the Czech Republic Pod vodarenskou vez 4, 82 08 Prague 8, Czech
More informationX.-P. HANG ETAL, FROM THE WAVELET SERIES TO THE DISCRETE WAVELET TRANSFORM Abstract Discrete wavelet transform (DWT) is computed by subband lters bank
X.-P. HANG ETAL, FROM THE WAVELET SERIES TO THE DISCRETE WAVELET TRANSFORM 1 From the Wavelet Series to the Discrete Wavelet Transform the Initialization Xiao-Ping hang, Li-Sheng Tian and Ying-Ning Peng
More information3.1. Solution for white Gaussian noise
Low complexity M-hypotheses detection: M vectors case Mohammed Nae and Ahmed H. Tewk Dept. of Electrical Engineering University of Minnesota, Minneapolis, MN 55455 mnae,tewk@ece.umn.edu Abstract Low complexity
More informationAdaptive Estimation of Distributions using Exponential Sub-Families Alan Gous Stanford University December 1996 Abstract: An algorithm is presented wh
Adaptive Estimation of Distributions using Exponential Sub-Families Alan Gous Stanford University December 1996 Abstract: An algorithm is presented which, for a large-dimensional exponential family G,
More informationEdge intersection graphs. of systems of grid paths. with bounded number of bends
Edge intersection graphs of systems of grid paths with bounded number of bends Andrei Asinowski a, Andrew Suk b a Caesarea Rothschild Institute, University of Haifa, Haifa 31905, Israel. b Courant Institute,
More informationDepartment of Electronics and Communication KMP College of Engineering, Perumbavoor, Kerala, India 1 2
Vol.3, Issue 3, 2015, Page.1115-1021 Effect of Anti-Forensics and Dic.TV Method for Reducing Artifact in JPEG Decompression 1 Deepthy Mohan, 2 Sreejith.H 1 PG Scholar, 2 Assistant Professor Department
More informationInternational Journal of Foundations of Computer Science c World Scientic Publishing Company DFT TECHNIQUES FOR SIZE ESTIMATION OF DATABASE JOIN OPERA
International Journal of Foundations of Computer Science c World Scientic Publishing Company DFT TECHNIQUES FOR SIZE ESTIMATION OF DATABASE JOIN OPERATIONS KAM_IL SARAC, OMER E GEC_IO GLU, AMR EL ABBADI
More informationExtra-High Speed Matrix Multiplication on the Cray-2. David H. Bailey. September 2, 1987
Extra-High Speed Matrix Multiplication on the Cray-2 David H. Bailey September 2, 1987 Ref: SIAM J. on Scientic and Statistical Computing, vol. 9, no. 3, (May 1988), pg. 603{607 Abstract The Cray-2 is
More informationProblem Set 3. MATH 778C, Spring 2009, Austin Mohr (with John Boozer) April 15, 2009
Problem Set 3 MATH 778C, Spring 2009, Austin Mohr (with John Boozer) April 15, 2009 1. Show directly that P 1 (s) P 1 (t) for all t s. Proof. Given G, let H s be a subgraph of G on s vertices such that
More informationLecture 27: Fast Laplacian Solvers
Lecture 27: Fast Laplacian Solvers Scribed by Eric Lee, Eston Schweickart, Chengrun Yang November 21, 2017 1 How Fast Laplacian Solvers Work We want to solve Lx = b with L being a Laplacian matrix. Recall
More informationComments on the randomized Kaczmarz method
Comments on the randomized Kaczmarz method Thomas Strohmer and Roman Vershynin Department of Mathematics, University of California Davis, CA 95616-8633, USA. strohmer@math.ucdavis.edu, vershynin@math.ucdavis.edu
More informationIterative Closest Point Algorithm in the Presence of Anisotropic Noise
Iterative Closest Point Algorithm in the Presence of Anisotropic Noise L. Maier-Hein, T. R. dos Santos, A. M. Franz, H.-P. Meinzer German Cancer Research Center, Div. of Medical and Biological Informatics
More informationOn the Relationships between Zero Forcing Numbers and Certain Graph Coverings
On the Relationships between Zero Forcing Numbers and Certain Graph Coverings Fatemeh Alinaghipour Taklimi, Shaun Fallat 1,, Karen Meagher 2 Department of Mathematics and Statistics, University of Regina,
More informationThe Simplex Algorithm with a New. Primal and Dual Pivot Rule. Hsin-Der CHEN 3, Panos M. PARDALOS 3 and Michael A. SAUNDERS y. June 14, 1993.
The Simplex Algorithm with a New rimal and Dual ivot Rule Hsin-Der CHEN 3, anos M. ARDALOS 3 and Michael A. SAUNDERS y June 14, 1993 Abstract We present a simplex-type algorithm for linear programming
More informationarxiv: v1 [physics.comp-ph] 25 Dec 2010
APS/123-QED Iteration Procedure for the N-Dimensional System of Linear Equations Avas V. Khugaev a arxiv:1012.5444v1 [physics.comp-ph] 25 Dec 2010 Bogoliubov Laboratory of Theoretical Physics, Joint Institute
More informationCardinal B-spline dictionaries on a compact interval
Cardinal B-spline dictionaries on a compact interval arxiv:math/543v [math.fa] 24 Jan 25 Miroslav Andrle and Laura Rebollo-Neira NCRG, Aston University, Birmingham B4 7ET, UK Abstract A prescription for
More informationTilings of the Euclidean plane
Tilings of the Euclidean plane Yan Der, Robin, Cécile January 9, 2017 Abstract This document gives a quick overview of a eld of mathematics which lies in the intersection of geometry and algebra : tilings.
More informationSorting. There exist sorting algorithms which have shown to be more efficient in practice.
Sorting Next to storing and retrieving data, sorting of data is one of the more common algorithmic tasks, with many different ways to perform it. Whenever we perform a web search and/or view statistics
More informationChapter 3. Quadric hypersurfaces. 3.1 Quadric hypersurfaces Denition.
Chapter 3 Quadric hypersurfaces 3.1 Quadric hypersurfaces. 3.1.1 Denition. Denition 1. In an n-dimensional ane space A; given an ane frame fo;! e i g: A quadric hypersurface in A is a set S consisting
More informationreasonable to store in a software implementation, it is likely to be a signicant burden in a low-cost hardware implementation. We describe in this pap
Storage-Ecient Finite Field Basis Conversion Burton S. Kaliski Jr. 1 and Yiqun Lisa Yin 2 RSA Laboratories 1 20 Crosby Drive, Bedford, MA 01730. burt@rsa.com 2 2955 Campus Drive, San Mateo, CA 94402. yiqun@rsa.com
More informationIterative Algorithms I: Elementary Iterative Methods and the Conjugate Gradient Algorithms
Iterative Algorithms I: Elementary Iterative Methods and the Conjugate Gradient Algorithms By:- Nitin Kamra Indian Institute of Technology, Delhi Advisor:- Prof. Ulrich Reude 1. Introduction to Linear
More information1 INTRODUCTION The LMS adaptive algorithm is the most popular algorithm for adaptive ltering because of its simplicity and robustness. However, its ma
MULTIPLE SUBSPACE ULV ALGORITHM AND LMS TRACKING S. HOSUR, A. H. TEWFIK, D. BOLEY University of Minnesota 200 Union St. S.E. Minneapolis, MN 55455 U.S.A fhosur@ee,tewk@ee,boley@csg.umn.edu ABSTRACT. The
More informationarxiv: v1 [math.co] 25 Sep 2015
A BASIS FOR SLICING BIRKHOFF POLYTOPES TREVOR GLYNN arxiv:1509.07597v1 [math.co] 25 Sep 2015 Abstract. We present a change of basis that may allow more efficient calculation of the volumes of Birkhoff
More informationSparse Reconstruction / Compressive Sensing
Sparse Reconstruction / Compressive Sensing Namrata Vaswani Department of Electrical and Computer Engineering Iowa State University Namrata Vaswani Sparse Reconstruction / Compressive Sensing 1/ 20 The
More informationSPARSE COMPONENT ANALYSIS FOR BLIND SOURCE SEPARATION WITH LESS SENSORS THAN SOURCES. Yuanqing Li, Andrzej Cichocki and Shun-ichi Amari
SPARSE COMPONENT ANALYSIS FOR BLIND SOURCE SEPARATION WITH LESS SENSORS THAN SOURCES Yuanqing Li, Andrzej Cichocki and Shun-ichi Amari Laboratory for Advanced Brain Signal Processing Laboratory for Mathematical
More informationRegularity Analysis of Non Uniform Data
Regularity Analysis of Non Uniform Data Christine Potier and Christine Vercken Abstract. A particular class of wavelet, derivatives of B-splines, leads to fast and ecient algorithms for contours detection
More informationWE consider the gate-sizing problem, that is, the problem
2760 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I: REGULAR PAPERS, VOL 55, NO 9, OCTOBER 2008 An Efficient Method for Large-Scale Gate Sizing Siddharth Joshi and Stephen Boyd, Fellow, IEEE Abstract We consider
More informationIntroduction to Topics in Machine Learning
Introduction to Topics in Machine Learning Namrata Vaswani Department of Electrical and Computer Engineering Iowa State University Namrata Vaswani 1/ 27 Compressed Sensing / Sparse Recovery: Given y :=
More informationSparse Solutions to Linear Inverse Problems. Yuzhe Jin
Sparse Solutions to Linear Inverse Problems Yuzhe Jin Outline Intro/Background Two types of algorithms Forward Sequential Selection Methods Diversity Minimization Methods Experimental results Potential
More informationBipartite Graph based Construction of Compressed Sensing Matrices
Bipartite Graph based Construction of Compressed Sensing Matrices Weizhi Lu, Kidiyo Kpalma and Joseph Ronsin arxiv:44.4939v [cs.it] 9 Apr 24 Abstract This paper proposes an efficient method to construct
More informationA second order algorithm for orthogonal projection onto curves and surfaces
A second order algorithm for orthogonal projection onto curves and surfaces Shi-min Hu and Johannes Wallner Dept. of Computer Science and Technology, Tsinghua University, Beijing, China shimin@tsinghua.edu.cn;
More informationThe Encoding Complexity of Network Coding
The Encoding Complexity of Network Coding Michael Langberg Alexander Sprintson Jehoshua Bruck California Institute of Technology Email: mikel,spalex,bruck @caltech.edu Abstract In the multicast network
More informationCounting the number of spanning tree. Pied Piper Department of Computer Science and Engineering Shanghai Jiao Tong University
Counting the number of spanning tree Pied Piper Department of Computer Science and Engineering Shanghai Jiao Tong University 目录 Contents 1 Complete Graph 2 Proof of the Lemma 3 Arbitrary Graph 4 Proof
More informationInstitute for Advanced Computer Studies. Department of Computer Science. Direction of Arrival and The Rank-Revealing. E. C. Boman y. M. F.
University of Maryland Institute for Advanced Computer Studies Department of Computer Science College Park TR{91{166 TR{2813 Direction of Arrival and The Rank-Revealing URV Decomposition E. C. Boman y
More informationA Novel Approach for Image Compression using Matching Pursuit Signal Approximation and Simulated Annealing
A Novel Approach for Image Compression using Matching Pursuit Signal Approximation and Simulated Annealing Ahmed M. Eid Amin ahmedamin@ieee.org Supervised by: Prof. Dr. Samir Shaheen Prof. Dr. Amir Atiya
More informationCenter for Automation Research. University of Maryland. the characterization and analysis of spectral signatures
Higher Order Statistical Learning for Vehicle Detection in Images A.N. Rajagopalan Philippe Burlina Rama Chellappa Center for Automation Research University of Maryland College Park, MD - 20742, USA. Abstract
More informationNew Worst-Case Upper Bound for #2-SAT and #3-SAT with the Number of Clauses as the Parameter
Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI-10) New Worst-Case Upper Bound for #2-SAT and #3-SAT with the Number of Clauses as the Parameter Junping Zhou 1,2, Minghao
More informationSHIP WAKE DETECTION FOR SAR IMAGES WITH COMPLEX BACKGROUNDS BASED ON MORPHOLOGICAL DICTIONARY LEARNING
SHIP WAKE DETECTION FOR SAR IMAGES WITH COMPLEX BACKGROUNDS BASED ON MORPHOLOGICAL DICTIONARY LEARNING Guozheng Yang 1, 2, Jing Yu 3, Chuangbai Xiao 3, Weidong Sun 1 1 State Key Laboratory of Intelligent
More informationCOMPUTABILITY THEORY AND RECURSIVELY ENUMERABLE SETS
COMPUTABILITY THEORY AND RECURSIVELY ENUMERABLE SETS JOSHUA LENERS Abstract. An algorithm is function from ω to ω defined by a finite set of instructions to transform a given input x to the desired output
More informationStatement-Level Communication-Free. Partitioning Techniques for. National Central University. Chung-Li 32054, Taiwan
Appeared in the Ninth Worshop on Languages and Compilers for Parallel Comping, San Jose, CA, Aug. 8-0, 996. Statement-Level Communication-Free Partitioning Techniques for Parallelizing Compilers Kuei-Ping
More informationImplementation of QR Up- and Downdating on a. Massively Parallel Computer. Hans Bruun Nielsen z Mustafa Pnar z. July 8, 1996.
Implementation of QR Up- and Downdating on a Massively Parallel Computer Claus Btsen y Per Christian Hansen y Kaj Madsen z Hans Bruun Nielsen z Mustafa Pnar z July 8, 1996 Abstract We describe an implementation
More informationTelecommunication and Informatics University of North Carolina, Technical University of Gdansk Charlotte, NC 28223, USA
A Decoder-based Evolutionary Algorithm for Constrained Parameter Optimization Problems S lawomir Kozie l 1 and Zbigniew Michalewicz 2 1 Department of Electronics, 2 Department of Computer Science, Telecommunication
More informationChapter 18 out of 37 from Discrete Mathematics for Neophytes: Number Theory, Probability, Algorithms, and Other Stuff by J. M. Cargal.
Chapter 8 out of 7 from Discrete Mathematics for Neophytes: Number Theory, Probability, Algorithms, and Other Stuff by J. M. Cargal 8 Matrices Definitions and Basic Operations Matrix algebra is also known
More informationEdge-disjoint Spanning Trees in Triangulated Graphs on Surfaces and application to node labeling 1
Edge-disjoint Spanning Trees in Triangulated Graphs on Surfaces and application to node labeling 1 Arnaud Labourel a a LaBRI - Universite Bordeaux 1, France Abstract In 1974, Kundu [4] has shown that triangulated
More informationSecret Instantiation in Ad-Hoc Networks
Secret Instantiation in Ad-Hoc Networs Sandeep S. Kularni Mohamed G. Gouda Anish Arora Department of Computer Department of Computer Sciences Department of Computer Science and Engineering The University
More informationRules for Identifying the Initial Design Points for Use in the Quick Convergent Inflow Algorithm
International Journal of Statistics and Probability; Vol. 5, No. 1; 2016 ISSN 1927-7032 E-ISSN 1927-7040 Published by Canadian Center of Science and Education Rules for Identifying the Initial Design for
More informationCOUNTING PERFECT MATCHINGS
COUNTING PERFECT MATCHINGS JOHN WILTSHIRE-GORDON Abstract. Let G be a graph on n vertices. A perfect matching of the vertices of G is a collection of n/ edges whose union is the entire graph. This definition
More information2386 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 6, JUNE 2006
2386 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 6, JUNE 2006 The Encoding Complexity of Network Coding Michael Langberg, Member, IEEE, Alexander Sprintson, Member, IEEE, and Jehoshua Bruck,
More informationNoncrossing Trees and Noncrossing Graphs
Noncrossing Trees and Noncrossing Graphs William Y. C. Chen and Sherry H. F. Yan 2 Center for Combinatorics, LPMC, Nanai University, 30007 Tianjin, P.R. China chen@nanai.edu.cn, 2 huifangyan@eyou.com Mathematics
More informationFACE RECOGNITION USING INDEPENDENT COMPONENT
Chapter 5 FACE RECOGNITION USING INDEPENDENT COMPONENT ANALYSIS OF GABORJET (GABORJET-ICA) 5.1 INTRODUCTION PCA is probably the most widely used subspace projection technique for face recognition. A major
More informationconsisting of compact sets. A spline subdivision scheme generates from such
Spline Subdivision Schemes for Compact Sets with Metric Averages Nira Dyn and Elza Farkhi Abstract. To dene spline subdivision schemes for general compact sets, we use the representation of spline subdivision
More informationIntersection of sets *
OpenStax-CNX module: m15196 1 Intersection of sets * Sunil Kumar Singh This work is produced by OpenStax-CNX and licensed under the Creative Commons Attribution License 2.0 We have pointed out that a set
More informationDiffusion Wavelets for Natural Image Analysis
Diffusion Wavelets for Natural Image Analysis Tyrus Berry December 16, 2011 Contents 1 Project Description 2 2 Introduction to Diffusion Wavelets 2 2.1 Diffusion Multiresolution............................
More informationStructured System Theory
Appendix C Structured System Theory Linear systems are often studied from an algebraic perspective, based on the rank of certain matrices. While such tests are easy to derive from the mathematical model,
More informationHydraulic pump fault diagnosis with compressed signals based on stagewise orthogonal matching pursuit
Hydraulic pump fault diagnosis with compressed signals based on stagewise orthogonal matching pursuit Zihan Chen 1, Chen Lu 2, Hang Yuan 3 School of Reliability and Systems Engineering, Beihang University,
More informationAlternating Projections
Alternating Projections Stephen Boyd and Jon Dattorro EE392o, Stanford University Autumn, 2003 1 Alternating projection algorithm Alternating projections is a very simple algorithm for computing a point
More informationVidhya.N.S. Murthy Student I.D Project report for Multimedia Processing course (EE5359) under Dr. K.R. Rao
STUDY AND IMPLEMENTATION OF THE MATCHING PURSUIT ALGORITHM AND QUALITY COMPARISON WITH DISCRETE COSINE TRANSFORM IN AN MPEG2 ENCODER OPERATING AT LOW BITRATES Vidhya.N.S. Murthy Student I.D. 1000602564
More informationIMAGE DENOISING USING NL-MEANS VIA SMOOTH PATCH ORDERING
IMAGE DENOISING USING NL-MEANS VIA SMOOTH PATCH ORDERING Idan Ram, Michael Elad and Israel Cohen Department of Electrical Engineering Department of Computer Science Technion - Israel Institute of Technology
More information. Introduction Image moments and various types of moment-based invariants play very important role in object recognition and shape analysis [], [2], [
On the Calculation of Image Moments Jan Flusser and Tomas Suk Institute of Information Theory and Automation Academy of Sciences of the Czech Republic Pod vodarenskou vez 4, 82 08 Prague 8, Czech Republic
More informationMinimization of the Truncation Error by Grid Adaptation
NASA/CR-999-0979 ICASE Report No. 99-46 Minimization of the Truncation Error by Grid Adaptation Nail K. Yamaleev NASA Langley Research Center, Hampton, Virginia Institute for Computer Applications in Science
More informationARITHMETIC operations based on residue number systems
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II: EXPRESS BRIEFS, VOL. 53, NO. 2, FEBRUARY 2006 133 Improved Memoryless RNS Forward Converter Based on the Periodicity of Residues A. B. Premkumar, Senior Member,
More informationOpen and Closed Sets
Open and Closed Sets Definition: A subset S of a metric space (X, d) is open if it contains an open ball about each of its points i.e., if x S : ɛ > 0 : B(x, ɛ) S. (1) Theorem: (O1) and X are open sets.
More informationFast Algorithms for Regularized Minimum Norm Solutions to Inverse Problems
Fast Algorithms for Regularized Minimum Norm Solutions to Inverse Problems Irina F. Gorodnitsky Cognitive Sciences Dept. University of California, San Diego La Jolla, CA 9293-55 igorodni@ece.ucsd.edu Dmitry
More informationhas phone Phone Person Person degree Degree isa isa has addr has addr has phone has phone major Degree Phone Schema S1 Phone Schema S2
Schema Equivalence in Heterogeneous Systems: Bridging Theory and Practice R. J. Miller y Y. E. Ioannidis z R. Ramakrishnan x Department of Computer Sciences University of Wisconsin-Madison frmiller, yannis,
More informationImage reconstruction based on back propagation learning in Compressed Sensing theory
Image reconstruction based on back propagation learning in Compressed Sensing theory Gaoang Wang Project for ECE 539 Fall 2013 Abstract Over the past few years, a new framework known as compressive sampling
More informationELEG Compressive Sensing and Sparse Signal Representations
ELEG 867 - Compressive Sensing and Sparse Signal Representations Gonzalo R. Arce Depart. of Electrical and Computer Engineering University of Delaware Fall 211 Compressive Sensing G. Arce Fall, 211 1 /
More informationMATH 423 Linear Algebra II Lecture 17: Reduced row echelon form (continued). Determinant of a matrix.
MATH 423 Linear Algebra II Lecture 17: Reduced row echelon form (continued). Determinant of a matrix. Row echelon form A matrix is said to be in the row echelon form if the leading entries shift to the
More informationA simple algorithm for the inverse field of values problem
A simple algorithm for the inverse field of values problem Russell Carden Department of Computational and Applied Mathematics, Rice University, Houston, TX 77005-183, USA E-mail: Russell.L.Carden@rice.edu
More informationAD0A STANFORD UNIV CA DEPT OP COMPUTER SCIENCE F/6 12/1 VERIFICATION OF LINK-LEVEL PROTOCOLS,(U) JAN 81 D E KNUTH. N0R11476-C0330 UNCLASSIFIED
AD0A099 140 STANFORD UNIV CA DEPT OP COMPUTER SCIENCE F/6 12/1 VERIFICATION OF LINK-LEVEL PROTOCOLS,(U) JAN 81 D E KNUTH. N0R11476-C0330 UNCLASSIFIED STANCS-81-84 NL Report. N STAN I [EVEV iv Verification
More informationBasis Selection For Wavelet Regression
Basis Selection For Wavelet Regression Kevin R. Wheeler Caelum Research Corporation NASA Ames Research Center Mail Stop 269-1 Moffett Field, CA 94035 wheeler@mail.arc.nasa.gov Atam P. Dhawan College of
More informationMatching Pursuit Filter Design
Matching Pursuit Filter Design P Jonathon Phillips * US Army Research Laboratory Ft. Belvoir, VA 22060 Abstract A method has been devised of using localized information to detect objects with varying signatures
More informationDiscriminative sparse model and dictionary learning for object category recognition
Discriative sparse model and dictionary learning for object category recognition Xiao Deng and Donghui Wang Institute of Artificial Intelligence, Zhejiang University Hangzhou, China, 31007 {yellowxiao,dhwang}@zju.edu.cn
More informationAdvanced Operations Research Techniques IE316. Quiz 1 Review. Dr. Ted Ralphs
Advanced Operations Research Techniques IE316 Quiz 1 Review Dr. Ted Ralphs IE316 Quiz 1 Review 1 Reading for The Quiz Material covered in detail in lecture. 1.1, 1.4, 2.1-2.6, 3.1-3.3, 3.5 Background material
More informationLOW-DENSITY PARITY-CHECK (LDPC) codes [1] can
208 IEEE TRANSACTIONS ON MAGNETICS, VOL 42, NO 2, FEBRUARY 2006 Structured LDPC Codes for High-Density Recording: Large Girth and Low Error Floor J Lu and J M F Moura Department of Electrical and Computer
More informationHyperplane Ranking in. Simple Genetic Algorithms. D. Whitley, K. Mathias, and L. Pyeatt. Department of Computer Science. Colorado State University
Hyperplane Ranking in Simple Genetic Algorithms D. Whitley, K. Mathias, and L. yeatt Department of Computer Science Colorado State University Fort Collins, Colorado 8523 USA whitley,mathiask,pyeatt@cs.colostate.edu
More informationComputation of the Constrained Infinite Time Linear Quadratic Optimal Control Problem
Computation of the Constrained Infinite Time Linear Quadratic Optimal Control Problem July 5, Introduction Abstract Problem Statement and Properties In this paper we will consider discrete-time linear
More informationTHE EFFECT OF JOIN SELECTIVITIES ON OPTIMAL NESTING ORDER
THE EFFECT OF JOIN SELECTIVITIES ON OPTIMAL NESTING ORDER Akhil Kumar and Michael Stonebraker EECS Department University of California Berkeley, Ca., 94720 Abstract A heuristic query optimizer must choose
More informationNewton and Quasi-Newton Methods
Lab 17 Newton and Quasi-Newton Methods Lab Objective: Newton s method is generally useful because of its fast convergence properties. However, Newton s method requires the explicit calculation of the second
More informationRecent Developments in Model-based Derivative-free Optimization
Recent Developments in Model-based Derivative-free Optimization Seppo Pulkkinen April 23, 2010 Introduction Problem definition The problem we are considering is a nonlinear optimization problem with constraints:
More informationI R TECHNICAL RESEARCH REPORT. Evolutionary Policy Iteration for Solving Markov Decision Processes
TECHNICAL RESEARCH REPORT Evolutionary Policy Iteration for Solving Markov Decision Processes by Hyeong Soo Chang, Hong-Gi Lee, Michael Fu, and Steven Marcus TR 00-31 I R INSTITUTE FOR SYSTEMS RESEARCH
More informationUsing Local Trajectory Optimizers To Speed Up Global. Christopher G. Atkeson. Department of Brain and Cognitive Sciences and
Using Local Trajectory Optimizers To Speed Up Global Optimization In Dynamic Programming Christopher G. Atkeson Department of Brain and Cognitive Sciences and the Articial Intelligence Laboratory Massachusetts
More informationMinimum-Cost Spanning Tree. as a. Path-Finding Problem. Laboratory for Computer Science MIT. Cambridge MA July 8, 1994.
Minimum-Cost Spanning Tree as a Path-Finding Problem Bruce M. Maggs Serge A. Plotkin Laboratory for Computer Science MIT Cambridge MA 02139 July 8, 1994 Abstract In this paper we show that minimum-cost
More informationParallel Evaluation of Hopfield Neural Networks
Parallel Evaluation of Hopfield Neural Networks Antoine Eiche, Daniel Chillet, Sebastien Pillement and Olivier Sentieys University of Rennes I / IRISA / INRIA 6 rue de Kerampont, BP 818 2232 LANNION,FRANCE
More informationMath 5BI: Problem Set 2 The Chain Rule
Math 5BI: Problem Set 2 The Chain Rule April 5, 2010 A Functions of two variables Suppose that γ(t) = (x(t), y(t), z(t)) is a differentiable parametrized curve in R 3 which lies on the surface S defined
More informationDistributed Compressed Estimation Based on Compressive Sensing for Wireless Sensor Networks
Distributed Compressed Estimation Based on Compressive Sensing for Wireless Sensor Networks Joint work with Songcen Xu and Vincent Poor Rodrigo C. de Lamare CETUC, PUC-Rio, Brazil Communications Research
More informationUMIACS-TR March Direction-of-Arrival Estimation Using the. G. Adams. M. F. Griffin. G. W. Stewart y. abstract
UMIACS-TR 91-46 March 1991 CS-TR-2640 Direction-of-Arrival Estimation Using the Rank-Revealing URV Decomposition G. Adams M. F. Griffin G. W. Stewart y abstract An algorithm for updating the null space
More informationAdaptive Image Compression Using Sparse Dictionaries
Adaptive Image Compression Using Sparse Dictionaries Inbal Horev, Ori Bryt and Ron Rubinstein Abstract Transform-based coding is a widely used image compression technique, where entropy reduction is achieved
More informationTHE TRANSITIVE REDUCTION OF A DIRECTED GRAPH*
SIAM J. COMPUT. Vol. 1, No. 2, June 1972 THE TRANSITIVE REDUCTION OF A DIRECTED GRAPH* A. V. AHO, M. R. GAREY" AND J. D. ULLMAN Abstract. We consider economical representations for the path information
More informationtime using O( n log n ) processors on the EREW PRAM. Thus, our algorithm improves on the previous results, either in time complexity or in the model o
Reconstructing a Binary Tree from its Traversals in Doubly-Logarithmic CREW Time Stephan Olariu Michael Overstreet Department of Computer Science, Old Dominion University, Norfolk, VA 23529 Zhaofang Wen
More informationDirectional Derivatives. Directional Derivatives. Directional Derivatives. Directional Derivatives. Directional Derivatives. Directional Derivatives
Recall that if z = f(x, y), then the partial derivatives f x and f y are defined as and represent the rates of change of z in the x- and y-directions, that is, in the directions of the unit vectors i and
More informationChapter 7. Nearest Point Problems on Simplicial Cones 315 where M is a positive denite symmetric matrix of order n. Let F be a nonsingular matrix such
Chapter 7 NEAREST POINT PROBLEMS ON SIMPLICIAL CONES Let ; = fb. 1 ::: B. n g be a given linearly independent set of column vectors in R n, and let b 2 R n be another given column vector. Let B = (B. 1
More informationModified Iterative Method for Recovery of Sparse Multiple Measurement Problems
Journal of Electrical Engineering 6 (2018) 124-128 doi: 10.17265/2328-2223/2018.02.009 D DAVID PUBLISHING Modified Iterative Method for Recovery of Sparse Multiple Measurement Problems Sina Mortazavi and
More informationA Course in Machine Learning
A Course in Machine Learning Hal Daumé III 13 UNSUPERVISED LEARNING If you have access to labeled training data, you know what to do. This is the supervised setting, in which you have a teacher telling
More informationWeighted Geodetic Convex Sets in A Graph
IOSR Journal of Mathematics (IOSR-JM) e-issn: 2278-5728, p-issn: 2319-765X. PP 12-17 www.iosrjournals.org Weighted Geodetic Convex Sets in A Graph Jill K. Mathew 1 Department of Mathematics Mar Ivanios
More information11 The Regular Pentagon
11 The Regular Pentagon 11.1 The Euclidean construction with the Golden Ratio The figure on page 561 shows an easy Euclidean construction of a regular pentagon. The justification of the construction begins
More information