Matrix algorithms: fast, stable, communication-optimizing random?!
|
|
|
- Lorraine Horton
- 5 years ago
- Views:
Transcription
1 Matrix algorithms: fast, stable, communication-optimizing random?! Ioana Dumitriu Department of Mathematics University of Washington (Seattle) Joint work with Grey Ballard, James Demmel, Olga Holtz, Robert Kleinberg IPAM: Convex Optimization and Algebraic Geometry September 28, 2010 Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
2 1 Motivation and Background Main Concerns in Numerical Linear Algebra Size issues 2 Reducing the flops Stability of FMM Stability of FLA Why Randomize? 3 Reducing communication Why is communication bad? Randomized Spectral Divide and Conquer Communication Costs 4 Conclusions Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
3 Motivation and Background Main Concerns in Numerical Linear Algebra The big four issues in matrix algorithms. Accuracy of computation (how far the computed values are from the actual ones, in the presence of rounding error). Stability (how much the computed result will change if we perturb the problem a little bit). Speed/complexity (how many operations they require, e.g., multiplications). Parallelism (how to distribute the computation to multiple processors in order to optimize speed if possible). Communication complexity (how to minimize the amount of back-and-forth between levels of memory or processors.) Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
4 Motivation and Background Main Concerns in Numerical Linear Algebra The big four issues in matrix algorithms. Accuracy of computation (how far the computed values are from the actual ones, in the presence of rounding error). Stability (how much the computed result will change if we perturb the problem a little bit). Speed/flops complexity (how many operations they require, e.g., multiplications). Parallelism (how to distribute the computation to multiple processors in order to optimize speed if possible). Communication complexity (how to minimize the amount of back-and-forth between levels of memory or processors.) Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
5 Motivation and Background Main Concerns in Numerical Linear Algebra The big four issues in matrix algorithms. Accuracy of computation (how far the computed values are from the actual ones, in the presence of rounding error). Stability (how much the computed result will change if we perturb the problem a little bit). Speed/flops complexity (how many operations they require, e.g., multiplications). Parallelism (how to distribute the computation to multiple processors in order to optimize speed if possible). Communication complexity (how to minimize the amount of back-and-forth between levels of memory or processors.) Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
6 Motivation and Background Main Concerns in Numerical Linear Algebra The big four issues in matrix algorithms. Accuracy of computation (how far the computed values are from the actual ones, in the presence of rounding error). Stability (how much the computed result will change if we perturb the problem a little bit). Speed/flops complexity (how many operations they require, e.g., multiplications). Parallelism (how to distribute the computation to multiple processors in order to optimize speed if possible). Communication complexity (how to minimize the amount of back-and-forth between levels of memory or processors.) Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
7 Motivation and Background Main Concerns in Numerical Linear Algebra The big five issues in matrix algorithms. Accuracy of computation (how far the computed values are from the actual ones, in the presence of rounding error). Stability (how much the computed result will change if we perturb the problem a little bit). Speed/flops complexity (how many operations they require, e.g., multiplications). Parallelism (how to distribute the computation to multiple processors in order to optimize speed if possible). Communication complexity (how to minimize the amount of back-and-forth between levels of memory or processors). Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
8 Motivation and Background Size issues Size matters. We re concerned with industrial-sized problems and asymptotic complexities. Constants in O( ) terms may be large enough to make everyday use impractical. Some problems will reduce down to manageable size... then apply classical algorithms. Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
9 Reducing the flops Stability of FMM Fast matrix multiplication What s the minimum number of operations needed to multiply two matrices? LinAlg 101 student: O(n 3 ) ThCS/LinAlg 501 student: far fewer... Late 60s (Strassen): at most c n log 2 7 are needed Early 90 s (Coppersmith-Winograd) at most c n are needed... but is it true that only c n 2+ɛ are needed?... Matrix multiplication exponent: ω is the smallest number such that there exist fast matrix multiplication algorithms running in O(n ω+ɛ ) time for any ɛ > 0; ω 2. Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
10 Reducing the flops Stability of FMM Fast matrix multiplication What s the minimum number of scalar multiplications needed to multiply two matrices? LinAlg 101 student: O(n 3 ) ThCS/LinAlg 501 student: far fewer... Late 60s (Strassen): at most c n log 2 7 are needed Early 90 s (Coppersmith-Winograd) at most c n are needed... but is it true that only c n 2+ɛ are needed?... Matrix multiplication exponent: ω is the smallest number such that there exist fast matrix multiplication algorithms running in O(n ω+ɛ ) time for any ɛ > 0; ω 2. Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
11 Reducing the flops Stability of FMM Fast matrix multiplication What s the minimum number of scalar multiplications needed to multiply two matrices? LinAlg 101 student: O(n 3 ) ThCS/LinAlg 501 student: far fewer... Late 60s (Strassen): at most c n log 2 7 are needed Early 90 s (Coppersmith-Winograd) at most c n are needed... but is it true that only c n 2+ɛ are needed?... Matrix multiplication exponent: ω is the smallest number such that there exist fast matrix multiplication algorithms running in O(n ω+ɛ ) time for any ɛ > 0; ω 2. Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
12 Reducing the flops Stability of FMM Fast matrix multiplication What s the minimum number of scalar multiplications needed to multiply two matrices? LinAlg 101 student: O(n 3 ) ThCS/LinAlg 501 student: far fewer... Late 60s (Strassen): at most c n log 2 7 are needed Early 90 s (Coppersmith-Winograd) at most c n are needed... but is it true that only c n 2+ɛ are needed?... Matrix multiplication exponent: ω is the smallest number such that there exist fast matrix multiplication algorithms running in O(n ω+ɛ ) time for any ɛ > 0; ω 2. Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
13 Reducing the flops Stability of FMM Fast matrix multiplication What s the minimum number of scalar multiplications needed to multiply two matrices? LinAlg 101 student: O(n 3 ) ThCS/LinAlg 501 student: far fewer... Late 60s (Strassen): at most c n log 2 7 are needed Early 90 s (Coppersmith-Winograd) at most c n are needed... but is it true that only c n 2+ɛ are needed?... Matrix multiplication exponent: ω is the smallest number such that there exist fast matrix multiplication algorithms running in O(n ω+ɛ ) time for any ɛ > 0; ω 2. Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
14 Reducing the flops Stability of FMM Fast matrix multiplication What s the minimum number of scalar multiplications needed to multiply two matrices? LinAlg 101 student: O(n 3 ) ThCS/LinAlg 501 student: far fewer... Late 60s (Strassen): at most c n log 2 7 are needed Early 90 s (Coppersmith-Winograd) at most c n are needed... but is it true that only c n 2+ɛ are needed?... Matrix multiplication exponent: ω is the smallest number such that there exist fast matrix multiplication algorithms running in O(n ω+ɛ ) time for any ɛ > 0; ω 2. Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
15 Reducing the flops Stability of FMM Fast matrix multiplication What s the minimum number of scalar multiplications needed to multiply two matrices? LinAlg 101 student: O(n 3 ) ThCS/LinAlg 501 student: far fewer... Late 60s (Strassen): at most c n log 2 7 are needed Early 90 s (Coppersmith-Winograd) at most c n are needed... but is it true that only c n 2+ɛ are needed?... Matrix multiplication exponent: ω is the smallest number such that there exist fast matrix multiplication algorithms running in O(n ω+ɛ ) time for any ɛ > 0; ω 2. Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
16 Reducing the flops Stability of FMM Fast matrix multiplication is stable! No concrete progress in the 00s, but new promising paths via group-theoretic algorithms (Cohn, Kleinberg, Szegedy, & Umans) We may not know what ω is, but we do now know that the algorithm that achieves it does so stably. (Demmel, D., Holtz, & Kleinberg 07) In other words, errors in computation (due to roundoff) will not accumulate to the point where the calculation is completely unreliable, and small changes in the problem will not incur big changes in the result. No algorithm running in less than O(n 3 ) can be entry-wise stable (Miller 75), but we achieve norm-stability. Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
17 Reducing the flops Stability of FMM Fast matrix multiplication is stable! Proved (Demmel, D., Holtz, Kleinberg 07) that Cohn, Umans et al algorithms are norm-stable. Bilinear non-commutative algorithms are norm-stable. Combined this with result that the exponent of matrix multiplication is achieved by bilinear non-commutative algorithms (folklore; Raz 03). Therefore the exponent of matrix multiplication can be achieved by stable algorithms. What about other linear algebra operations? Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
18 Reducing the flops Stability of FLA Fast linear algebra is stable? Assume now a black box algorithm for fast and stable matrix multiplication (any exponent 3 > γ ω). Can we use it to do other linear algebra operations fast and stably? Known: complexity of (virtually all) Linear Algebra = complexity of MM. Not known (previously): how to achieve it stably. Must use blocked algorithms... new block algorithms (the old can do the trick). Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
19 Reducing the flops Stability of FLA Fast linear algebra is stable? Assume now a black box algorithm for fast and stable matrix multiplication (any exponent 3 > γ ω). Can we use it to do other linear algebra operations fast and stably? Known: complexity of (virtually all) Linear Algebra = complexity of MM. Not known (previously): how to achieve it stably. Must use blocked algorithms... new block algorithms (the old can do the trick). Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
20 Reducing the flops Stability of FLA Fast linear algebra is stable! Proved (Demmel, D., Holtz, 07) that FLA is stable: e.g., LU, QR, EIG/SVD, inversion, Sylvester eqs, A\b, etc. Essentially, MM, LU, and QR will get you everything else. For some ops (e.g. LU, matrix inversion) we need to raise bit complexity to achieve backward stability (need to count bit operations); others ops (QR, EIG/SVD) work in arithmetic operations. In particular, we need a randomized rank-revealing factorization to do Divide and Conquer (both for symmetric and non-symmetric case). Most of these are deterministic, but we need to randomize to be able to do EIG/SVD. All are parallelizable! Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
21 Reducing the flops Stability of FLA Fast (randomized) linear algebra is stable! Proved (Demmel, D., Holtz, 07) that FLA is stable: e.g., LU, QR, EIG/SVD, inversion, Sylvester eqs, A\b, etc. Most of these are deterministic, but we need to randomize to be able to do EIG/SVD. In particular, we need a randomized rank-revealing factorization to do Divide and Conquer (both for symmetric and non-symmetric case). All are parallelizable! Essentially, MM, LU, and QR will get you everything else. For some ops (e.g. LU, matrix inversion) we need to raise bit complexity to achieve backward stability (need to count bit operations); others ops (QR, EIG/SVD) work in arithmetic operations. Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
22 Reducing the flops Why Randomize? How do we find the rank of a matrix fast? SVD. Can t use, circular. QR. If matrix singular, wil break. QR with pivoting. O(n 4 )! not fast at all. Use a random orthogonal (Haar) matrix to muddle columns! Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
23 Reducing the flops Why Randomize? How do we find the rank of a matrix fast? SVD. Can t use, circular. QR. If matrix singular, wil break. QR with pivoting. O(n 4 )! not fast at all. Use a random orthogonal (Haar) matrix to muddle columns! Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
24 Reducing the flops Why Randomize? How do we find the rank of a matrix fast? SVD. Can t use, circular. QR, no pivoting. If matrix singular, wil break. QR with pivoting. O(n 4 )! not fast at all. Use a random orthogonal (Haar) matrix to muddle columns! Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
25 Reducing the flops Why Randomize? How do we find the rank of a matrix fast? SVD. Can t use, circular. QR, no pivoting. If matrix singular, will break. QR with pivoting. O(n 4 )! not fast at all. Use a random orthogonal (Haar) matrix to muddle columns! Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
26 Reducing the flops Why Randomize? How do we find the rank of a matrix fast? SVD. Can t use, circular. QR, no pivoting. If matrix singular, will break. QR with pivoting. O(n 4 )! not fast at all. Use a random orthogonal (Haar) matrix to muddle columns! Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
27 Reducing the flops Why Randomize? How do we find the rank of a matrix fast? SVD. Can t use, circular. QR, no pivoting. If matrix singular, will break. QR with pivoting. Worst-case O(n 4 )! Use a random orthogonal (Haar) matrix to muddle columns! Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
28 Reducing the flops Why Randomize? How do we find the rank of a matrix fast? SVD. Can t use, circular. QR, no pivoting. If matrix singular, will break. QR with pivoting. Worst-case O(n 4 )! Use a random orthogonal (Haar) matrix to muddle columns! Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
29 Reducing the flops Randomized URV (RURV) Why Randomize? Suppose want to find rank(a) in exact arithmetic. Construct Haar (random orthogonal/unitary) matrix V; [U, R] = qr(a V T ); (do Gram-Schmidt on product) A = URV! R s diagonal entries are all 0 beyond the rank, all non-zero before, with probability 1. What about in floating-point, with numerical rank? Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
30 Randomized URV Reducing the flops Why Randomize? Numerical rank r: σ 1... σ r σ r+1... σ n ; σ 1 /σ r = O(1), σ r σ r+1. The same algorithm works for numerical rank with high probability, assumming an appropriately large gap σ r σ r+1. Idea existed before, e.g., Rokhlin et al., Mahoney et al., Tropp et al., for r = o(n). Here r = Θ(n) is necessary, for Divide and Conquer. Proof uses the order of smallest eigenvalue for a β = 1, 2 Jacobi matrix (D., 2010). The resulting algorithm is easily parallelizable and also minimizes communication. Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
31 Reducing communication Communication Cost Model Why is communication bad? Algorithms have two costs: 1 arithmetic (flops) 2 communication: moving data between levels of a memory hierarchy (sequential case) processors over a network (parallel case) Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
32 Reducing communication Communication Cost Model Why is communication bad? Running time of an algorithm is sum of 3 terms: # flops * time per flop # words moved / bandwidth # messages * latency Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
33 Reducing communication Why is communication bad? Communication Cost Model Exponentially growing gaps between Sequentially: time per flop 1 / network BW network latency improving 59% per year vs. 26% per year vs. 15% per year In parallel: time per flop 1 / memory BW memory latency improving 59% per year vs. 23% per year vs. 5.5% per year Need to reorganize linear algebra to avoid communication (# words and # messages moved) Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
34 Reducing communication Why is communication bad? Eigenvalue Algorithms on Sequential Machines Communication lower bounds ( ) n 3 # words = Ω M ( ) n 3 # messages = Ω M 3/2 Factors by which algorithms exceed lower bounds: New Algorithms LAPACK Algorithm # words # messages # words # messages Symm Eig/SVD M M O (1) O (1) O O Nonsymm Eig Optimal! M is fast memory size, n is problem size New algorithms have same communication complexity as QR decomposition and achieve lower bound Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
35 Reducing communication Why is communication bad? Eigenvalue Algorithms on Parallel Machines Communication lower bounds ( ) n 2 # words = Ω P ( ) # messages = Ω P Factors by which algorithms exceed lower bounds: New Algorithms ScaLAPACK Algorithm # words # messages # words # messages Symm Eig/SVD O (log P) O P n O (log P) O (log P) P Nonsymm Eig Nearly optimal! O log P O(n log P) P is number of processors, n is problem size New algorithms have same communication complexity as QR decomposition and achieve lower bound (up to polylog factors) Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
36 Reducing communication Randomized Spectral Divide and Conquer History of Spectral Divide and Conquer Ideas go back to Bulgakov, Godunov, Malyshev ( 88-89) Bai, Demmel, Gu 97 reduced to matmul, QR, generalized QR with pivoting Demmel, D., Holtz, 07 instead of QR with pivoting, use RURV (randomized URV) requires matmul and QR, no column pivoting Demmel, Grigori, Hoemmen, Langou 08 communication-optimal QR decomposition ("CAQR") (tangential) Ballard, Demmel, Holtz, Schwartz communication lower bounds on LA communication-optimal Cholesky Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
37 Reducing communication Randomized Spectral Divide and Conquer Overview of (Ballard, D., Demmel) Algorithm 10 One step of divide and conquer: ( 1 Compute I + (A 1 ) 2k) 1 implicitly maps eigenvalues of A to 0 and 1 (roughly) 2 Compute rank-revealing decomposition to find invariant subspace 3 Output block-triangular matrix [ ] A new = U A11 A AU = 12 ε A 22 block sizes chosen to minimize norm of ε eigenvalues of A 11 all lie outside unit circle, eigenvalues of A 22 lie inside unit circle, subproblems solved recursively Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
38 Reducing communication Randomized Spectral Divide and Conquer Randomized Bisection, symmetric case Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
39 Reducing communication Randomized Spectral Divide and Conquer Randomized Bisection, symmetric case Pick interval around midpoint Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
40 Reducing communication Randomized Spectral Divide and Conquer Randomized Bisection, symmetric case Split randomly inside interval Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
41 Reducing communication Randomized Spectral Divide and Conquer Randomized Bisection, symmetric case Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
42 Reducing communication Randomized Spectral Divide and Conquer Randomized Bisection, non-symmetric case Pick inner circle around center Gershgorin bounding disc Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
43 Reducing communication Randomized Spectral Divide and Conquer Randomized Bisection, non-symmetric case θ Choose random angle Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
44 Reducing communication Randomized Spectral Divide and Conquer Randomized Bisection, non-symmetric case Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
45 Reducing communication Randomized Spectral Divide and Conquer Randomized Bisection, non-symmetric case Choose random perpendicular in range Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
46 Reducing communication Randomized Spectral Divide and Conquer Randomized Bisection, non-symmetric case Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
47 Reducing communication Communication Costs Communication Upper Bound (sequential case) M = memory size, γ = cost of flop, β = inverse bandwidth, α = latency Cost of one step of divide-and-conquer is ( ) ( ) n 3 n 3 C D+C (n) = α O M 3/2 + β O + γ O(n 3 ) M Assuming we split the spectrum by some fraction 1 > f > 0 each time (likely b/c of randomization!): total cost of the entire algorithm is asymptotically the same! Algorithm attains lower bound. Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
48 Reducing communication Communication Costs Communication Upper Bound (parallel case) P = # processors, γ = cost of flop, β = inverse bandwidth, α = latency Cost of one step of divide-and-conquer is ( ) ( ) ( ) n 2 n 3 C D+C (n, P) = α O P log P + β O log P + γ O P P log P Subproblems can be solved in parallel. Assuming we split the spectrum by some fraction 1 > f > 0 each time (likely b/c of randomization!): total cost of the entire algorithm is asymptotically the same! Algorithm attains lower bound (to within logarithmic factors). Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
49 Conclusions What to take home If one can reach the complexity exponent of matrix multiplication, one can do all linear algebra fast and stably. The above works for large matrices (constants may be large). Randomizing can mean stabilizing. Divide and Conquer is a very good strategy (for EIG/SVD). Communication is bad and must be avoided, even if it means doing more work. Ioana Dumitriu (UW) Matrix algorithms September 28, / 49
LAPACK. Linear Algebra PACKage. Janice Giudice David Knezevic 1
LAPACK Linear Algebra PACKage 1 Janice Giudice David Knezevic 1 Motivating Question Recalling from last week... Level 1 BLAS: vectors ops Level 2 BLAS: matrix-vectors ops 2 2 O( n ) flops on O( n ) data
COMPUTATIONAL LINEAR ALGEBRA
COMPUTATIONAL LINEAR ALGEBRA Matrix Vector Multiplication Matrix matrix Multiplication Slides from UCSD and USB Directed Acyclic Graph Approach Jack Dongarra A new approach using Strassen`s algorithm Jim
CS 6210 Fall 2016 Bei Wang. Review Lecture What have we learnt in Scientific Computing?
CS 6210 Fall 2016 Bei Wang Review Lecture What have we learnt in Scientific Computing? Let s recall the scientific computing pipeline observed phenomenon mathematical model discretization solution algorithm
CS Software Engineering for Scientific Computing Lecture 10:Dense Linear Algebra
CS 294-73 Software Engineering for Scientific Computing Lecture 10:Dense Linear Algebra Slides from James Demmel and Kathy Yelick 1 Outline What is Dense Linear Algebra? Where does the time go in an algorithm?
Lecture 11: Randomized Least-squares Approximation in Practice. 11 Randomized Least-squares Approximation in Practice
Stat60/CS94: Randomized Algorithms for Matrices and Data Lecture 11-10/09/013 Lecture 11: Randomized Least-squares Approximation in Practice Lecturer: Michael Mahoney Scribe: Michael Mahoney Warning: these
AMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 20: Sparse Linear Systems; Direct Methods vs. Iterative Methods Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 26
Communication-optimal Parallel and Sequential Cholesky decomposition
Communication-optimal arallel and Sequential Cholesky decomposition Grey Ballard James Demmel Olga Holtz Oded Schwartz Electrical Engineering and Computer Sciences University of California at Berkeley
AMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 1: Course Overview; Matrix Multiplication Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 21 Outline 1 Course
A GPU-based Approximate SVD Algorithm Blake Foster, Sridhar Mahadevan, Rui Wang
A GPU-based Approximate SVD Algorithm Blake Foster, Sridhar Mahadevan, Rui Wang University of Massachusetts Amherst Introduction Singular Value Decomposition (SVD) A: m n matrix (m n) U, V: orthogonal
Outline. Parallel Algorithms for Linear Algebra. Number of Processors and Problem Size. Speedup and Efficiency
1 2 Parallel Algorithms for Linear Algebra Richard P. Brent Computer Sciences Laboratory Australian National University Outline Basic concepts Parallel architectures Practical design issues Programming
Modern GPUs (Graphics Processing Units)
Modern GPUs (Graphics Processing Units) Powerful data parallel computation platform. High computation density, high memory bandwidth. Relatively low cost. NVIDIA GTX 580 512 cores 1.6 Tera FLOPs 1.5 GB
How to Write Fast Numerical Code Spring 2012 Lecture 9. Instructor: Markus Püschel TAs: Georg Ofenbeck & Daniele Spampinato
How to Write Fast Numerical Code Spring 2012 Lecture 9 Instructor: Markus Püschel TAs: Georg Ofenbeck & Daniele Spampinato Today Linear algebra software: history, LAPACK and BLAS Blocking (BLAS 3): key
Communication Efficient Gaussian Elimination with Partial Pivoting using a Shape Morphing Data Layout
Communication Efficient Gaussian Elimination with Partial Pivoting using a Shape Morphing Data Layout Grey Ballard James Demmel Benjamin Lipshitz Oded Schwartz Sivan Toledo Electrical Engineering and Computer
Lesson 2 7 Graph Partitioning
Lesson 2 7 Graph Partitioning The Graph Partitioning Problem Look at the problem from a different angle: Let s multiply a sparse matrix A by a vector X. Recall the duality between matrices and graphs:
Name: Math 310 Fall 2012 Toews EXAM 1. The material we have covered so far has been designed to support the following learning goals:
Name: Math 310 Fall 2012 Toews EXAM 1 The material we have covered so far has been designed to support the following learning goals: understand sources of error in scientific computing (modeling, measurement,
An I/O-Complexity Lower Bound for All Recursive Matrix Multiplication Algorithms by Path-Routing. Jacob N. Scott
An I/O-Complexity Lower Bound for All Recursive Matrix Multiplication Algorithms by Path-Routing by Jacob N. Scott A dissertation submitted in partial satisfaction of the requirements for the degree of
Computational Methods. Sources of Errors
Computational Methods Sources of Errors Manfred Huber 2011 1 Numerical Analysis / Scientific Computing Many problems in Science and Engineering can not be solved analytically on a computer Numeric solutions
Fast Approximation of Matrix Coherence and Statistical Leverage
Fast Approximation of Matrix Coherence and Statistical Leverage Michael W. Mahoney Stanford University ( For more info, see: http:// cs.stanford.edu/people/mmahoney/ or Google on Michael Mahoney ) Statistical
Solving Sparse Linear Systems. Forward and backward substitution for solving lower or upper triangular systems
AMSC 6 /CMSC 76 Advanced Linear Numerical Analysis Fall 7 Direct Solution of Sparse Linear Systems and Eigenproblems Dianne P. O Leary c 7 Solving Sparse Linear Systems Assumed background: Gauss elimination
Numerical Analysis and Statistics on Tensor Parameter Spaces
Numerical Analysis and Statistics on Tensor Parameter Spaces SIAM - AG11 - Tensors Oct. 7, 2011 Overview Normal Mean / Karcher Mean Karcher mean / Normal mean - finding representatives for a set of points
Using recursion to improve performance of dense linear algebra software. Erik Elmroth Dept of Computing Science & HPC2N Umeå University, Sweden
Using recursion to improve performance of dense linear algebra software Erik Elmroth Dept of Computing Science & HPCN Umeå University, Sweden Joint work with Fred Gustavson, Isak Jonsson & Bo Kågström
Chapter 4. Divide-and-Conquer. Copyright 2007 Pearson Addison-Wesley. All rights reserved.
Chapter 4 Divide-and-Conquer Copyright 2007 Pearson Addison-Wesley. All rights reserved. Divide-and-Conquer The most-well known algorithm design strategy: 2. Divide instance of problem into two or more
How to Write Fast Numerical Code
How to Write Fast Numerical Code Lecture: Dense linear algebra, LAPACK, MMM optimizations in ATLAS Instructor: Markus Püschel TA: Daniele Spampinato & Alen Stojanov Today Linear algebra software: history,
Communication-avoidance!
Communication-avoidance! and! Automatic performance tuning! Nick Knight! UC-Berkeley Parlab (Bebop group)! 1 Students and Collaborators (Bebop) PIs! James Demmel! Kathy Yelick! Students! Marghoob Mohiyuddin!
Contents. Implementing the QR factorization The algebraic eigenvalue problem. Applied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in
CS321 Introduction To Numerical Methods
CS3 Introduction To Numerical Methods Fuhua (Frank) Cheng Department of Computer Science University of Kentucky Lexington KY 456-46 - - Table of Contents Errors and Number Representations 3 Error Types
Graph Expansion and Communication Costs of Algorithms
Graph Expansion and Communication Costs of Algorithms Grey Ballard James Demmel Olga Holtz Oded Schwartz Abstract The communication complexity of algorithms is shown to be closely related to the expansion
Chapter 1 A New Parallel Algorithm for Computing the Singular Value Decomposition
Chapter 1 A New Parallel Algorithm for Computing the Singular Value Decomposition Nicholas J. Higham Pythagoras Papadimitriou Abstract A new method is described for computing the singular value decomposition
Computational Methods in Statistics with Applications A Numerical Point of View. Large Data Sets. L. Eldén. March 2016
Computational Methods in Statistics with Applications A Numerical Point of View L. Eldén SeSe March 2016 Large Data Sets IDA Machine Learning Seminars, September 17, 2014. Sequential Decision Making: Experiment
CSCE 411 Design and Analysis of Algorithms
CSCE 411 Design and Analysis of Algorithms Set 4: Transform and Conquer Slides by Prof. Jennifer Welch Spring 2014 CSCE 411, Spring 2014: Set 4 1 General Idea of Transform & Conquer 1. Transform the original
Sankalchand Patel College of Engineering - Visnagar Department of Computer Engineering and Information Technology. Assignment
Class: V - CE Sankalchand Patel College of Engineering - Visnagar Department of Computer Engineering and Information Technology Sub: Design and Analysis of Algorithms Analysis of Algorithm: Assignment
Chapter 1 A New Parallel Algorithm for Computing the Singular Value Decomposition
Chapter 1 A New Parallel Algorithm for Computing the Singular Value Decomposition Nicholas J. Higham Pythagoras Papadimitriou Abstract A new method is described for computing the singular value decomposition
Divide-and-Conquer. Combine solutions to sub-problems into overall solution. Break up problem of size n into two equal parts of size!n.
Chapter 5 Divide and Conquer Slides by Kevin Wayne. Copyright 25 Pearson-Addon Wesley. All rights reserved. Divide-and-Conquer Divide-and-conquer. Break up problem into several parts. Solve each part recursively.
Advanced Algorithms. Problem solving Techniques. Divide and Conquer הפרד ומשול
Advanced Algorithms Problem solving Techniques. Divide and Conquer הפרד ומשול 1 Divide and Conquer A method of designing algorithms that (informally) proceeds as follows: Given an instance of the problem
Iterative Algorithms I: Elementary Iterative Methods and the Conjugate Gradient Algorithms
Iterative Algorithms I: Elementary Iterative Methods and the Conjugate Gradient Algorithms By:- Nitin Kamra Indian Institute of Technology, Delhi Advisor:- Prof. Ulrich Reude 1. Introduction to Linear
17/05/2018. Outline. Outline. Divide and Conquer. Control Abstraction for Divide &Conquer. Outline. Module 2: Divide and Conquer
Module 2: Divide and Conquer Divide and Conquer Control Abstraction for Divide &Conquer 1 Recurrence equation for Divide and Conquer: If the size of problem p is n and the sizes of the k sub problems are
Curriculum Map: Mathematics
Curriculum Map: Mathematics Course: Honors Advanced Precalculus and Trigonometry Grade(s): 11-12 Unit 1: Functions and Their Graphs This chapter will develop a more complete, thorough understanding of
MATH 3511 Basics of MATLAB
MATH 3511 Basics of MATLAB Dmitriy Leykekhman Spring 2012 Topics Sources. Entering Matrices. Basic Operations with Matrices. Build in Matrices. Build in Scalar and Matrix Functions. if, while, for m-files
Parallel Linear Algebra in Julia
Parallel Linear Algebra in Julia Britni Crocker and Donglai Wei 18.337 Parallel Computing 12.17.2012 1 Table of Contents 1. Abstract... 2 2. Introduction... 3 3. Julia Implementation...7 4. Performance...
Blocked Schur Algorithms for Computing the Matrix Square Root. Deadman, Edvin and Higham, Nicholas J. and Ralha, Rui. MIMS EPrint: 2012.
Blocked Schur Algorithms for Computing the Matrix Square Root Deadman, Edvin and Higham, Nicholas J. and Ralha, Rui 2013 MIMS EPrint: 2012.26 Manchester Institute for Mathematical Sciences School of Mathematics
The divide and conquer strategy has three basic parts. For a given problem of size n,
1 Divide & Conquer One strategy for designing efficient algorithms is the divide and conquer approach, which is also called, more simply, a recursive approach. The analysis of recursive algorithms often
A few multilinear algebraic definitions I Inner product A, B = i,j,k a ijk b ijk Frobenius norm Contracted tensor products A F = A, A C = A, B 2,3 = A
Krylov-type methods and perturbation analysis Berkant Savas Department of Mathematics Linköping University Workshop on Tensor Approximation in High Dimension Hausdorff Institute for Mathematics, Universität
Lecture 15: More Iterative Ideas
Lecture 15: More Iterative Ideas David Bindel 15 Mar 2010 Logistics HW 2 due! Some notes on HW 2. Where we are / where we re going More iterative ideas. Intro to HW 3. More HW 2 notes See solution code!
COMP 558 lecture 19 Nov. 17, 2010
COMP 558 lecture 9 Nov. 7, 2 Camera calibration To estimate the geometry of 3D scenes, it helps to know the camera parameters, both external and internal. The problem of finding all these parameters is
Multiple-choice (35 pt.)
CS 161 Practice Midterm I Summer 2018 Released: 7/21/18 Multiple-choice (35 pt.) 1. (2 pt.) Which of the following asymptotic bounds describe the function f(n) = n 3? The bounds do not necessarily need
Lecture 9 - Matrix Multiplication Equivalences and Spectral Graph Theory 1
CME 305: Discrete Mathematics and Algorithms Instructor: Professor Aaron Sidford (sidford@stanfordedu) February 6, 2018 Lecture 9 - Matrix Multiplication Equivalences and Spectral Graph Theory 1 In the
Numerical Linear Algebra
Numerical Linear Algebra Probably the simplest kind of problem. Occurs in many contexts, often as part of larger problem. Symbolic manipulation packages can do linear algebra "analytically" (e.g. Mathematica,
CSC Design and Analysis of Algorithms
CSC 8301- Design and Analysis of Algorithms Lecture 6 Divide and Conquer Algorithm Design Technique Divide-and-Conquer The most-well known algorithm design strategy: 1. Divide a problem instance into two
The Limits of Sorting Divide-and-Conquer Comparison Sorts II
The Limits of Sorting Divide-and-Conquer Comparison Sorts II CS 311 Data Structures and Algorithms Lecture Slides Monday, October 12, 2009 Glenn G. Chappell Department of Computer Science University of
CSC Design and Analysis of Algorithms. Lecture 6. Divide and Conquer Algorithm Design Technique. Divide-and-Conquer
CSC 8301- Design and Analysis of Algorithms Lecture 6 Divide and Conquer Algorithm Design Technique Divide-and-Conquer The most-well known algorithm design strategy: 1. Divide a problem instance into two
Reducing the Influence of Tiny Normwise Relative Errors on Performance Profiles. Dingle, Nicholas J. and Higham, Nicholas J. MIMS EPrint: 2011.
Reducing the Influence of Tiny Normwise Relative Errors on Performance Profiles Dingle, Nicholas J. and Higham, Nicholas J. 20 MIMS EPrint: 20.90 Manchester Institute for Mathematical Sciences School of
Roundoff Noise in Scientific Computations
Appendix K Roundoff Noise in Scientific Computations Roundoff errors in calculations are often neglected by scientists. The success of the IEEE double precision standard makes most of us think that the
Divide-and-Conquer. Dr. Yingwu Zhu
Divide-and-Conquer Dr. Yingwu Zhu Divide-and-Conquer The most-well known algorithm design technique: 1. Divide instance of problem into two or more smaller instances 2. Solve smaller instances independently
Contents. I Basics 1. Copyright by SIAM. Unauthorized reproduction of this article is prohibited.
page v Preface xiii I Basics 1 1 Optimization Models 3 1.1 Introduction... 3 1.2 Optimization: An Informal Introduction... 4 1.3 Linear Equations... 7 1.4 Linear Optimization... 10 Exercises... 12 1.5
MA/CSSE 473 Day 17. Divide-and-conquer Convex Hull. Strassen's Algorithm: Matrix Multiplication. (if time, Shell's Sort)
MA/CSSE 473 Day 17 Divide-and-conquer Convex Hull Strassen's Algorithm: Matrix Multiplication (if time, Shell's Sort) MA/CSSE 473 Day 17 Student Questions Exam 2 specification Levitin 3 rd Edition Closest
MATH 5520 Basics of MATLAB
MATH 5520 Basics of MATLAB Dmitriy Leykekhman Spring 2011 Topics Sources. Entering Matrices. Basic Operations with Matrices. Build in Matrices. Build in Scalar and Matrix Functions. if, while, for m-files
Lecture 27: Fast Laplacian Solvers
Lecture 27: Fast Laplacian Solvers Scribed by Eric Lee, Eston Schweickart, Chengrun Yang November 21, 2017 1 How Fast Laplacian Solvers Work We want to solve Lx = b with L being a Laplacian matrix. Recall
Geometric Mean Algorithms Based on Harmonic and Arithmetic Iterations
Geometric Mean Algorithms Based on Harmonic and Arithmetic Iterations Ben Jeuris and Raf Vandebril KU Leuven, Dept. of Computer Science, 3001 Leuven(Heverlee), Belgium {ben.jeuris,raf.vandebril}@cs.kuleuven.be
vector space retrieval many slides courtesy James Amherst
vector space retrieval many slides courtesy James Allan@umass Amherst 1 what is a retrieval model? Model is an idealization or abstraction of an actual process Mathematical models are used to study the
Lecture 4 CS781 February 3, 2011
Lecture 4 CS78 February 3, 2 Topics: Data Compression-Huffman Trees Divide-and-Conquer Solving Recurrence Relations Counting Inversions Closest Pair Integer Multiplication Matrix Multiplication Data Compression
A new implementation of the dqds algorithm
Kinji Kimura Kyoto Univ. Kinji Kimura (Kyoto Univ.) 1 / 26 We propose a new implementation of the dqds algorithm for the bidiagonal SVD. Kinji Kimura (Kyoto Univ.) 2 / 26 Our aim(1) srand(1); n=70000;
CS/ENGRD 2110 Object-Oriented Programming and Data Structures Spring 2012 Thorsten Joachims. Lecture 10: Asymptotic Complexity and
CS/ENGRD 2110 Object-Oriented Programming and Data Structures Spring 2012 Thorsten Joachims Lecture 10: Asymptotic Complexity and What Makes a Good Algorithm? Suppose you have two possible algorithms or
Mathematics 6 12 Section 26
Mathematics 6 12 Section 26 1 Knowledge of algebra 1. Apply the properties of real numbers: closure, commutative, associative, distributive, transitive, identities, and inverses. 2. Solve linear equations
What is MATLAB? What is MATLAB? Programming Environment MATLAB PROGRAMMING. Stands for MATrix LABoratory. A programming environment
What is MATLAB? MATLAB PROGRAMMING Stands for MATrix LABoratory A software built around vectors and matrices A great tool for numerical computation of mathematical problems, such as Calculus Has powerful
AMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 5: Sparse Linear Systems and Factorization Methods Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 18 Sparse
Chapter 5. Divide and Conquer. Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved.
Chapter 5 Divide and Conquer Slides by Kevin Wayne. Copyright 25 Pearson-Addison Wesley. All rights reserved. Divide-and-Conquer Divide-and-conquer. Break up problem into several parts. Solve each part
Introduction to Algorithms
Lecture 1 Introduction to Algorithms 1.1 Overview The purpose of this lecture is to give a brief overview of the topic of Algorithms and the kind of thinking it involves: why we focus on the subjects that
Blocked Schur Algorithms for Computing the Matrix Square Root
Blocked Schur Algorithms for Computing the Matrix Square Root Edvin Deadman 1, Nicholas J. Higham 2,andRuiRalha 3 1 Numerical Algorithms Group edvin.deadman@nag.co.uk 2 University of Manchester higham@maths.manchester.ac.uk
Degeneracy of the Linear Seventeen-Point Algorithm for Generalized Essential Matrix
J Math Imaging Vis 00 37: 40-48 DOI 0007/s085-00-09-9 Authors s version The final publication is available at wwwspringerlinkcom Degeneracy of the Linear Seventeen-Point Algorithm for Generalized Essential
Review Questions 26 CHAPTER 1. SCIENTIFIC COMPUTING
26 CHAPTER 1. SCIENTIFIC COMPUTING amples. The IEEE floating-point standard can be found in [131]. A useful tutorial on floating-point arithmetic and the IEEE standard is [97]. Although it is no substitute
Lecture 8: Mergesort / Quicksort Steven Skiena
Lecture 8: Mergesort / Quicksort Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.stonybrook.edu/ skiena Problem of the Day Give an efficient
Divide-and-Conquer. The most-well known algorithm design strategy: smaller instances. combining these solutions
Divide-and-Conquer The most-well known algorithm design strategy: 1. Divide instance of problem into two or more smaller instances 2. Solve smaller instances recursively 3. Obtain solution to original
Sorting. Quicksort analysis Bubble sort. November 20, 2017 Hassan Khosravi / Geoffrey Tien 1
Sorting Quicksort analysis Bubble sort November 20, 2017 Hassan Khosravi / Geoffrey Tien 1 Quicksort analysis How long does Quicksort take to run? Let's consider the best and the worst case These differ
Outline and Reading. Analysis of Algorithms 1
Outline and Reading Algorithms Running time ( 3.1) Pseudo-code ( 3.2) Counting primitive operations ( 3.4) Asymptotic notation ( 3.4.1) Asymptotic analysis ( 3.4.2) Case study ( 3.4.3) Analysis of Algorithms
Biostatistics 615/815 Lecture 13: Programming with Matrix
Computation Biostatistics 615/815 Lecture 13: Programming with Hyun Min Kang February 17th, 2011 Hyun Min Kang Biostatistics 615/815 - Lecture 13 February 17th, 2011 1 / 28 Annoucements Computation Homework
Algorithm Analysis. (Algorithm Analysis ) Data Structures and Programming Spring / 48
Algorithm Analysis (Algorithm Analysis ) Data Structures and Programming Spring 2018 1 / 48 What is an Algorithm? An algorithm is a clearly specified set of instructions to be followed to solve a problem
Module 5.5: nag sym bnd lin sys Symmetric Banded Systems of Linear Equations. Contents
Module Contents Module 5.5: nag sym bnd lin sys Symmetric Banded Systems of nag sym bnd lin sys provides a procedure for solving real symmetric or complex Hermitian banded systems of linear equations with
Lecture 3: Sorting 1
Lecture 3: Sorting 1 Sorting Arranging an unordered collection of elements into monotonically increasing (or decreasing) order. S = a sequence of n elements in arbitrary order After sorting:
Advanced Algorithms Class Notes for Monday, November 10, 2014
Advanced Algorithms Class Notes for Monday, November 10, 2014 Bernard Moret Divide-and-Conquer: Matrix Multiplication Divide-and-conquer is especially useful in computational geometry, but also in numerical
GEOMETRIC TOOLS FOR COMPUTER GRAPHICS
GEOMETRIC TOOLS FOR COMPUTER GRAPHICS PHILIP J. SCHNEIDER DAVID H. EBERLY MORGAN KAUFMANN PUBLISHERS A N I M P R I N T O F E L S E V I E R S C I E N C E A M S T E R D A M B O S T O N L O N D O N N E W
Divide and Conquer. Algorithm Fall Semester
Divide and Conquer Algorithm 2014 Fall Semester Divide-and-Conquer The most-well known algorithm design strategy: 1. Divide instance of problem into two or more smaller instances 2. Solve smaller instances
CS 97SI: INTRODUCTION TO PROGRAMMING CONTESTS. Jaehyun Park
CS 97SI: INTRODUCTION TO PROGRAMMING CONTESTS Jaehyun Park Today s Lecture Algebra Number Theory Combinatorics (non-computational) Geometry Emphasis on how to compute Sum of Powers n k=1 k 2 = 1 6 n(n
Floating-Point Numbers in Digital Computers
POLYTECHNIC UNIVERSITY Department of Computer and Information Science Floating-Point Numbers in Digital Computers K. Ming Leung Abstract: We explain how floating-point numbers are represented and stored
(Creating Arrays & Matrices) Applied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB (Creating Arrays & Matrices) Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional
Floating-Point Numbers in Digital Computers
POLYTECHNIC UNIVERSITY Department of Computer and Information Science Floating-Point Numbers in Digital Computers K. Ming Leung Abstract: We explain how floating-point numbers are represented and stored
Alternating Projections
Alternating Projections Stephen Boyd and Jon Dattorro EE392o, Stanford University Autumn, 2003 1 Alternating projection algorithm Alternating projections is a very simple algorithm for computing a point
Divide and Conquer Kernel Ridge Regression
Divide and Conquer Kernel Ridge Regression Yuchen Zhang John Duchi Martin Wainwright University of California, Berkeley COLT 2013 Yuchen Zhang (UC Berkeley) Divide and Conquer KRR COLT 2013 1 / 15 Problem
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 3, MARCH
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 3, MARCH 2005 343 Rate-Distortion Optimized Tree-Structured Compression Algorithms for Piecewise Polynomial Images Rahul Shukla, Member, IEEE, Pier Luigi
CALCULATING RANKS, NULL SPACES AND PSEUDOINVERSE SOLUTIONS FOR SPARSE MATRICES USING SPQR
CALCULATING RANKS, NULL SPACES AND PSEUDOINVERSE SOLUTIONS FOR SPARSE MATRICES USING SPQR Leslie Foster Department of Mathematics, San Jose State University October 28, 2009, SIAM LA 09 DEPARTMENT OF MATHEMATICS,
QR Decomposition on GPUs
QR Decomposition QR Algorithms Block Householder QR Andrew Kerr* 1 Dan Campbell 1 Mark Richards 2 1 Georgia Tech Research Institute 2 School of Electrical and Computer Engineering Georgia Institute of
Jane Li. Assistant Professor Mechanical Engineering Department, Robotic Engineering Program Worcester Polytechnic Institute
Jane Li Assistant Professor Mechanical Engineering Department, Robotic Engineering Program Worcester Polytechnic Institute (3 pts) Compare the testing methods for testing path segment and finding first
Parallel Implementations of Gaussian Elimination
s of Western Michigan University vasilije.perovic@wmich.edu January 27, 2012 CS 6260: in Parallel Linear systems of equations General form of a linear system of equations is given by a 11 x 1 + + a 1n
1. For the sieve technique we solve the problem, recursively mathematically precisely accurately 2. We do sorting to, keep elements in random positions keep the algorithm run in linear order keep the algorithm
Mathematical preliminaries and error analysis
Mathematical preliminaries and error analysis Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan August 28, 2011 Outline 1 Round-off errors and computer arithmetic IEEE
2.7 Numerical Linear Algebra Software
2.7 Numerical Linear Algebra Software In this section we will discuss three software packages for linear algebra operations: (i) (ii) (iii) Matlab, Basic Linear Algebra Subroutines (BLAS) and LAPACK. There
CSE373: Data Structure & Algorithms Lecture 21: More Comparison Sorting. Aaron Bauer Winter 2014
CSE373: Data Structure & Algorithms Lecture 21: More Comparison Sorting Aaron Bauer Winter 2014 The main problem, stated carefully For now, assume we have n comparable elements in an array and we want
Parameterization of triangular meshes
Parameterization of triangular meshes Michael S. Floater November 10, 2009 Triangular meshes are often used to represent surfaces, at least initially, one reason being that meshes are relatively easy to
Topics in geometry Exam 1 Solutions 7/8/4
Topics in geometry Exam 1 Solutions 7/8/4 Question 1 Consider the following axioms for a geometry: There are exactly five points. There are exactly five lines. Each point lies on exactly three lines. Each
A Geometric Analysis of Subspace Clustering with Outliers
A Geometric Analysis of Subspace Clustering with Outliers Mahdi Soltanolkotabi and Emmanuel Candés Stanford University Fundamental Tool in Data Mining : PCA Fundamental Tool in Data Mining : PCA Subspace
arxiv: v1 [cs.ds] 25 Sep 2018
![arxiv: v1 [cs.ds] 25 Sep 2018](/thumbs/92/108440307.jpg "arxiv: v1 [cs.ds] 25 Sep 2018")
Improved Parallel Cache-Oblivious Algorithms for Dynamic Programming and Linear Algebra Yan Gu Carnegie Mellon University arxiv:1809.09330v1 [cs.ds] 25 Sep 2018 Abstract For many cache-oblivious algorithms