AMS526: Numerical Analysis I (Numerical Linear Algebra)

Size: px
Start display at page:

Download "AMS526: Numerical Analysis I (Numerical Linear Algebra)"

Transcription

1 AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 1: Course Overview; Matrix Multiplication Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 21

2 Outline 1 Course Overview 2 Matrix-Vector Multiplication 3 Matrix-Matrix Multiplication Xiangmin Jiao Numerical Analysis I 2 / 21

3 Course Description What is numerical linear algebra? Solving linear algebra problems using efficient algorithms on computers Topics: Direct and iterative methods for solving simultaneous linear equations. Matrix factorization, conditioning, stability, sparsity, and efficiency. Computation of eigenvalues and eigenvectors. Singular value decomposition Required Textbook (adopted in Fall 2012) David S. Watkins, Fundamentals of Matrix Computations, 3rd edition, Wiley 2010, ISBN Xiangmin Jiao Numerical Analysis I 3 / 21

4 Prerequisite Prerequisite/Co-requisite: AMS 510 (linear algebra portion) or equivalent undergraduate-level linear algebra course. Familiarity with following concepts is assumed: Vector spaces, Gaussian elimination, Gram-Schmidt orthogonalization, and eigenvalues/eigenvectors AMS 595 (co-requisite for students without programming experience in C) This MUST NOT be your first course in linear algebra, or you will get lost To review fundamental concepts of linear algebra, see textbook such as Gilbert Strang, Linear Algebra and Its Applications, 4th Edition, Brooks Cole, Xiangmin Jiao Numerical Analysis I 4 / 21

5 Why Learn Numerical Linear Algebra? Numerical linear algebra is foundation of scientific computations Many problems ultimately reduce to linear algebra concepts or algorithms, either analytical or computational Examples: Finite-element analysis, data fitting, PageRank (Google) Focus of this course: Fundamental concepts, algorithms, and programming Xiangmin Jiao Numerical Analysis I 5 / 21

6 Course Outline Gaussian elimination and its variants (2.5 weeks) Sensitivity of linear systems (2.5 weeks) Linear least squares problem (1.5 weeks) Singular value decomposition (1.5 weeks) Eigenvalues and eigenvectors (3 weeks) Iterative methods for linear systems (2 weeks) Course webpage: Note: Course schedule online is tentative and is subject to change. Xiangmin Jiao Numerical Analysis I 6 / 21

7 Some Notes of Textbook and Notation We will cover all chapters of third-edition of Watkins, Fundamentals of Matrix Computations Previously this course used Trefethen and Bau, Numerical Linear Algebra, SIAM, 1997, which is a very insightful book Why change to Fundamentals of Matrix Computations (FMC)? FMC takes more progressive approach, so easier for students Starts with linear systems instead of SVD; section on iterative methods is easier to follow Starts with real numbers and switch to complex numbers later Has more examples FMC has more coverage on sparse matrices, which are important FMC uses non-standard names sometimes, so we sometimes will use different terminology from textbook Some explanations in lectures will still come from Trefethen and Bau Xiangmin Jiao Numerical Analysis I 7 / 21

8 Course Policy Assignments (written or programming) Assignments are due in class one to two weeks after assigned You can discuss course materials and homework problems with others, but you must write your answers completely independently Do NOT copy solutions from any source. Do NOT share your solutions to others Exams and tests Grading All exams are closed-book However, one-page cheat sheet is allowed Assignments: 30% Two tests: 40% Final exam: 30% Xiangmin Jiao Numerical Analysis I 8 / 21

9 Outline 1 Course Overview 2 Matrix-Vector Multiplication 3 Matrix-Matrix Multiplication Xiangmin Jiao Numerical Analysis I 9 / 21

10 Definition Suppose A is n m, x is size m, and b is size n a 11 a a 1m x 1 a 21 a a 2m A =......, x = x 2., b = a n1 a n2... a nm x m All entries belong to R, the field of real numbers. Space of m-vectors is R m, and the space of n m matrices is R n m Matrix-vector product b = Ax is defined as m b i = a ij x j j=1 The map x Ax is linear, which means that for any x, y R m and any α R A(x + y) = Ax + Ay, A(αx) = αax. Xiangmin Jiao Numerical Analysis I 10 / 21 b 1 b 2. b n

11 Linear Combination Let a j denote jth column of matrix A Alternatively, matrix-vector product can be viewed as a 11 a 12 a 21 a 22 b = Ax = m x j a j = x 1 j=1. a n1 + x 2. a n2 i.e., b is a linear combination of column vectors of A Two different views of matrix-vector products: + + x m 1 b i = m j=1 a ijx j : A acts on x to produce b; scalar operations; 2 b = m j=1 x ja j : x acts on A to produce b; vector operations. If A is n m, Ax can be viewed as a mapping from R m to R n a 1m a 2m. a nm Xiangmin Jiao Numerical Analysis I 11 / 21

12 Pseudo-Code for Matrix-Vector Multiplication Pseudo-code for b = Ax for i = 1 to n b i 0 for j = 1 to m b i b i + a ij x j Pseudo-code are not real code, in that they do not run on any computer. They are human readable, but they should be straightforward to convert into real computer codes in any programming language (e.g., C, FORTRAN, MATLAB, etc.) There is not a standard convention for pseudo-code Xiangmin Jiao Numerical Analysis I 12 / 21

13 Flop Count It is important to assess the efficiency of algorithms. But how? We could implement different algorithms and do head-to-head comparison, but implementation details can affect true performance We could estimate cost of all operations, but it is very tedious Relatively simple and effective approach is to estimate amount of floating-point operations, or flops, and focus on asymptotic analysis as sizes of matrices approach infinity Idealization Count each operation +,,, /, and as one flop, and make no distinction of real and complex numbers. This estimation is very crude, since it omits data movement in memory, which is non-negligible on modern computer architectures Matrix-vector product requires about 2mn flops, because b i b i + a ij x j requires two flops Suppose m = n, it takes quadratic time in n (or O(n 2 )) Xiangmin Jiao Numerical Analysis I 13 / 21

14 Outline 1 Course Overview 2 Matrix-Vector Multiplication 3 Matrix-Matrix Multiplication Xiangmin Jiao Numerical Analysis I 14 / 21

15 Matrix-Matrix Multiplication If A is n m and X is m p, then B = AX is n p, with entries defined by m b ij = a ik x kj. k=1 Written in columns, we have m b j = Ax j = x kj a k. k=1 In other word, jth column of B is A times jth column of X Xiangmin Jiao Numerical Analysis I 15 / 21

16 Pseudo-Code for Matrix-Matrix Multiplication Pseudo-code for B = AX for i = 1 to n for j = 1 to p b ij 0 for k = 1 to m b ij b ij + a ik x kj Matrix-matrix multiplication requires about 2mnp flops, because b ij b ij + a ik x kj requires two flops Suppose m = n = p, it takes cubic time in n (or O(n 3 )) Xiangmin Jiao Numerical Analysis I 16 / 21

17 Special Cases Matrix-Matrix Multiplication Vector is a special case of a matrix Matrix-vector multiplication is a special case of matrix-matrix Multiplication Other special cases of vector-vector multiplication Inner product: u T v (where u R n and v R n ) Outer product: uv T (where u R n and v R m ) Xiangmin Jiao Numerical Analysis I 17 / 21

18 Block Matrices Matrix can be partitioned into blocks. For example, A R n m and X R m p can be partitioned into A = m 1 m 2 [ A11 A 12 A 21 A 22 Then B = AX can be written as p ] 1 p [ 2 n1 X11 X, X = 12 n 2 X 21 X 22 ] m1 m 2, p 1 p 2 [ B11 B 12 B 21 B 22 ] [ n1 A11 A = 12 n 2 A 21 A 22 ] [ X11 X 12 X 21 X 22 ] where B ij = A i1 X 1j + A i2 X 2j, i, j = 1, 2. More generally, the matrix can be decomposed into more than two block rows/columns Xiangmin Jiao Numerical Analysis I 18 / 21

19 Block Matrix Operators Suppose A has s 1 block rows and s 2 block columns, and X has s 2 block rows and s 3 block columns. Matrix-matrix multiplication can be rewritten using block matrix operations: Pseudo-code for B = AX using block-matrix operators for i = 1 to s 1 for j = 1 to s 3 B ij 0 for k = 1 to s 2 B ij B ij + A ik X kj Block matrix operations do not reduce flop counts, but they can improve performance because of improved cache performance and reduced data movement on modern computer architecture, by fitting matrix blocks into cache at once Xiangmin Jiao Numerical Analysis I 19 / 21

20 Fast Matrix-Matrix Multiplication Multiplying two n n matrices requires 2n 3 flops using inner-product-based algorithm Is this optimal? Xiangmin Jiao Numerical Analysis I 20 / 21

21 Fast Matrix-Matrix Multiplication Multiplying two n n matrices requires 2n 3 flops using inner-product-based algorithm Is this optimal? Strassen s method requires O(n s ) flops, where s = log It is recursive algorithm applied to matrices of size 2 k 2 k For matrices of sizes not of 2 k 2 k, fill missing rows and columns with zeros Asymptotically faster algorithm, due to D. Coppersmith and S. Winograd (1990), requires O(n ) flops Asymptotically fastest algorithm currently known, due to V. Williams (2011), requires O(n ) flops Xiangmin Jiao Numerical Analysis I 20 / 21

22 Matrix-Matrix Multiplication in Practice In practice, inner-product-based algorithm is almost always used Earlier authors estimated that Strassen s algorithm is faster for matrices with widths of n 100 for optimized implementations Compared to a highly optimized traditional multiplication on current architectures, Strassen s algorithm is not faster unless n 1000, and the benefit is marginal for matrix sizes of several thousand Coppersmith and Winograd s algorithm has a huge constant C in front of it, so it is never fast enough in practice for realistic problem sizes The asymptotically faster methods are less accurate than standard multiplication when using floating-point numbers (more prone to rounding errors and cancellation errors) Lower-complexity algorithms are sometimes used to prove theoretical time bounds of other algorithms For sparse matrices, efficient matrix-matrix multiplication should take advantage of sparsity (see SMMP for further reading) Xiangmin Jiao Numerical Analysis I 21 / 21

David C Lay Linear Algebra 4th Edition

David C Lay Linear Algebra 4th Edition DAVID C LAY LINEAR ALGEBRA 4TH EDITION PDF - Are you looking for david c lay linear algebra 4th edition Books? Now, you will be happy that at this time david c lay linear algebra 4th edition PDF is available

More information

AM205: lecture 2. 1 These have been shifted to MD 323 for the rest of the semester.

AM205: lecture 2. 1 These have been shifted to MD 323 for the rest of the semester. AM205: lecture 2 Luna and Gary will hold a Python tutorial on Wednesday in 60 Oxford Street, Room 330 Assignment 1 will be posted this week Chris will hold office hours on Thursday (1:30pm 3:30pm, Pierce

More information

Modern GPUs (Graphics Processing Units)

Modern GPUs (Graphics Processing Units) Modern GPUs (Graphics Processing Units) Powerful data parallel computation platform. High computation density, high memory bandwidth. Relatively low cost. NVIDIA GTX 580 512 cores 1.6 Tera FLOPs 1.5 GB

More information

(Creating Arrays & Matrices) Applied Linear Algebra in Geoscience Using MATLAB

(Creating Arrays & Matrices) Applied Linear Algebra in Geoscience Using MATLAB Applied Linear Algebra in Geoscience Using MATLAB (Creating Arrays & Matrices) Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra)

AMS526: Numerical Analysis I (Numerical Linear Algebra) AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 20: Sparse Linear Systems; Direct Methods vs. Iterative Methods Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 26

More information

Numerical Linear Algebra

Numerical Linear Algebra Numerical Linear Algebra Probably the simplest kind of problem. Occurs in many contexts, often as part of larger problem. Symbolic manipulation packages can do linear algebra "analytically" (e.g. Mathematica,

More information

Numerical Analysis I - Final Exam Matrikelnummer:

Numerical Analysis I - Final Exam Matrikelnummer: Dr. Behrens Center for Mathematical Sciences Technische Universität München Winter Term 2005/2006 Name: Numerical Analysis I - Final Exam Matrikelnummer: I agree to the publication of the results of this

More information

CSCE 689 : Special Topics in Sparse Matrix Algorithms Department of Computer Science and Engineering Spring 2015 syllabus

CSCE 689 : Special Topics in Sparse Matrix Algorithms Department of Computer Science and Engineering Spring 2015 syllabus CSCE 689 : Special Topics in Sparse Matrix Algorithms Department of Computer Science and Engineering Spring 2015 syllabus Tim Davis last modified September 23, 2014 1 Catalog Description CSCE 689. Special

More information

REGULAR GRAPHS OF GIVEN GIRTH. Contents

REGULAR GRAPHS OF GIVEN GIRTH. Contents REGULAR GRAPHS OF GIVEN GIRTH BROOKE ULLERY Contents 1. Introduction This paper gives an introduction to the area of graph theory dealing with properties of regular graphs of given girth. A large portion

More information

Bindel, Fall 2016 Matrix Computations (CS 6210) Notes for

Bindel, Fall 2016 Matrix Computations (CS 6210) Notes for 1 Logistics Notes for 2016-08-24 CS 6210 is a graduate-level introduction to numerical linear algebra. We will study direct and iterative methods for linear systems, least squares problems, eigenvalue

More information

Realization of Hardware Architectures for Householder Transformation based QR Decomposition using Xilinx System Generator Block Sets

Realization of Hardware Architectures for Householder Transformation based QR Decomposition using Xilinx System Generator Block Sets IJSTE - International Journal of Science Technology & Engineering Volume 2 Issue 08 February 2016 ISSN (online): 2349-784X Realization of Hardware Architectures for Householder Transformation based QR

More information

AMS 148: Chapter 1: What is Parallelization

AMS 148: Chapter 1: What is Parallelization AMS 148: Chapter 1: What is Parallelization 1 Introduction of Parallelism Traditional codes and algorithms, much of what you, the student, have experienced previously has been computation in a sequential

More information

Lecture #3: PageRank Algorithm The Mathematics of Google Search

Lecture #3: PageRank Algorithm The Mathematics of Google Search Lecture #3: PageRank Algorithm The Mathematics of Google Search We live in a computer era. Internet is part of our everyday lives and information is only a click away. Just open your favorite search engine,

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra)

AMS526: Numerical Analysis I (Numerical Linear Algebra) AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 5: Sparse Linear Systems and Factorization Methods Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 18 Sparse

More information

AMS527: Numerical Analysis II

AMS527: Numerical Analysis II AMS527: Numerical Analysis II A Brief Overview of Finite Element Methods Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao SUNY Stony Brook AMS527: Numerical Analysis II 1 / 25 Overview Basic concepts Mathematical

More information

Lecture 9 - Matrix Multiplication Equivalences and Spectral Graph Theory 1

Lecture 9 - Matrix Multiplication Equivalences and Spectral Graph Theory 1 CME 305: Discrete Mathematics and Algorithms Instructor: Professor Aaron Sidford (sidford@stanfordedu) February 6, 2018 Lecture 9 - Matrix Multiplication Equivalences and Spectral Graph Theory 1 In the

More information

LAPACK. Linear Algebra PACKage. Janice Giudice David Knezevic 1

LAPACK. Linear Algebra PACKage. Janice Giudice David Knezevic 1 LAPACK Linear Algebra PACKage 1 Janice Giudice David Knezevic 1 Motivating Question Recalling from last week... Level 1 BLAS: vectors ops Level 2 BLAS: matrix-vectors ops 2 2 O( n ) flops on O( n ) data

More information

EE/AA 578: Convex Optimization

EE/AA 578: Convex Optimization EE/AA 578: Convex Optimization Instructor: Maryam Fazel University of Washington Fall 2016 1. Introduction EE/AA 578, Univ of Washington, Fall 2016 course logistics mathematical optimization least-squares;

More information

CS 6210 Fall 2016 Bei Wang. Review Lecture What have we learnt in Scientific Computing?

CS 6210 Fall 2016 Bei Wang. Review Lecture What have we learnt in Scientific Computing? CS 6210 Fall 2016 Bei Wang Review Lecture What have we learnt in Scientific Computing? Let s recall the scientific computing pipeline observed phenomenon mathematical model discretization solution algorithm

More information

Matrix algorithms: fast, stable, communication-optimizing random?!

Matrix algorithms: fast, stable, communication-optimizing random?! Matrix algorithms: fast, stable, communication-optimizing random?! Ioana Dumitriu Department of Mathematics University of Washington (Seattle) Joint work with Grey Ballard, James Demmel, Olga Holtz, Robert

More information

Introduction to Algorithms

Introduction to Algorithms Lecture 1 Introduction to Algorithms 1.1 Overview The purpose of this lecture is to give a brief overview of the topic of Algorithms and the kind of thinking it involves: why we focus on the subjects that

More information

2.3 Algorithms Using Map-Reduce

2.3 Algorithms Using Map-Reduce 28 CHAPTER 2. MAP-REDUCE AND THE NEW SOFTWARE STACK one becomes available. The Master must also inform each Reduce task that the location of its input from that Map task has changed. Dealing with a failure

More information

Solving Sparse Linear Systems. Forward and backward substitution for solving lower or upper triangular systems

Solving Sparse Linear Systems. Forward and backward substitution for solving lower or upper triangular systems AMSC 6 /CMSC 76 Advanced Linear Numerical Analysis Fall 7 Direct Solution of Sparse Linear Systems and Eigenproblems Dianne P. O Leary c 7 Solving Sparse Linear Systems Assumed background: Gauss elimination

More information

Contents. Implementing the QR factorization The algebraic eigenvalue problem. Applied Linear Algebra in Geoscience Using MATLAB

Contents. Implementing the QR factorization The algebraic eigenvalue problem. Applied Linear Algebra in Geoscience Using MATLAB Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in

More information

Parallel Implementations of Gaussian Elimination

Parallel Implementations of Gaussian Elimination s of Western Michigan University vasilije.perovic@wmich.edu January 27, 2012 CS 6260: in Parallel Linear systems of equations General form of a linear system of equations is given by a 11 x 1 + + a 1n

More information

2.7 Numerical Linear Algebra Software

2.7 Numerical Linear Algebra Software 2.7 Numerical Linear Algebra Software In this section we will discuss three software packages for linear algebra operations: (i) (ii) (iii) Matlab, Basic Linear Algebra Subroutines (BLAS) and LAPACK. There

More information

UML CS Algorithms Qualifying Exam Fall, 2003 ALGORITHMS QUALIFYING EXAM

UML CS Algorithms Qualifying Exam Fall, 2003 ALGORITHMS QUALIFYING EXAM NAME: This exam is open: - books - notes and closed: - neighbors - calculators ALGORITHMS QUALIFYING EXAM The upper bound on exam time is 3 hours. Please put all your work on the exam paper. (Partial credit

More information

Linear Algebra Lay 4th Edition Solution

Linear Algebra Lay 4th Edition Solution We have made it easy for you to find a PDF Ebooks without any digging. And by having access to our ebooks online or by storing it on your computer, you have convenient answers with linear algebra lay 4th

More information

Lecture 27: Fast Laplacian Solvers

Lecture 27: Fast Laplacian Solvers Lecture 27: Fast Laplacian Solvers Scribed by Eric Lee, Eston Schweickart, Chengrun Yang November 21, 2017 1 How Fast Laplacian Solvers Work We want to solve Lx = b with L being a Laplacian matrix. Recall

More information

Outline. Parallel Algorithms for Linear Algebra. Number of Processors and Problem Size. Speedup and Efficiency

Outline. Parallel Algorithms for Linear Algebra. Number of Processors and Problem Size. Speedup and Efficiency 1 2 Parallel Algorithms for Linear Algebra Richard P. Brent Computer Sciences Laboratory Australian National University Outline Basic concepts Parallel architectures Practical design issues Programming

More information

CS 6210 Fall 2016 Bei Wang. Lecture 1 A (Hopefully) Fun Introduction to Scientific Computing

CS 6210 Fall 2016 Bei Wang. Lecture 1 A (Hopefully) Fun Introduction to Scientific Computing CS 6210 Fall 2016 Bei Wang Lecture 1 A (Hopefully) Fun Introduction to Scientific Computing About this class Technical content followed by fun investigations Stay engaged in the classroom Share your SC-related

More information

Matrix Multiplication

Matrix Multiplication Matrix Multiplication CPS343 Parallel and High Performance Computing Spring 2013 CPS343 (Parallel and HPC) Matrix Multiplication Spring 2013 1 / 32 Outline 1 Matrix operations Importance Dense and sparse

More information

Dense Matrix Algorithms

Dense Matrix Algorithms Dense Matrix Algorithms Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar To accompany the text Introduction to Parallel Computing, Addison Wesley, 2003. Topic Overview Matrix-Vector Multiplication

More information

Maths for Signals and Systems Linear Algebra in Engineering. Some problems by Gilbert Strang

Maths for Signals and Systems Linear Algebra in Engineering. Some problems by Gilbert Strang Maths for Signals and Systems Linear Algebra in Engineering Some problems by Gilbert Strang Problems. Consider u, v, w to be non-zero vectors in R 7. These vectors span a vector space. What are the possible

More information

How to Write Fast Numerical Code Spring 2012 Lecture 9. Instructor: Markus Püschel TAs: Georg Ofenbeck & Daniele Spampinato

How to Write Fast Numerical Code Spring 2012 Lecture 9. Instructor: Markus Püschel TAs: Georg Ofenbeck & Daniele Spampinato How to Write Fast Numerical Code Spring 2012 Lecture 9 Instructor: Markus Püschel TAs: Georg Ofenbeck & Daniele Spampinato Today Linear algebra software: history, LAPACK and BLAS Blocking (BLAS 3): key

More information

COMPUTATIONAL DYNAMICS

COMPUTATIONAL DYNAMICS COMPUTATIONAL DYNAMICS THIRD EDITION AHMED A. SHABANA Richard and Loan Hill Professor of Engineering University of Illinois at Chicago A John Wiley and Sons, Ltd., Publication COMPUTATIONAL DYNAMICS COMPUTATIONAL

More information

n = 1 What problems are interesting when n is just 1?

n = 1 What problems are interesting when n is just 1? What if n=1??? n = 1 What problems are interesting when n is just 1? Sorting? No Median finding? No Addition? How long does it take to add one pair of numbers? Multiplication? How long does it take to

More information

Matrix Multiplication

Matrix Multiplication Matrix Multiplication CPS343 Parallel and High Performance Computing Spring 2018 CPS343 (Parallel and HPC) Matrix Multiplication Spring 2018 1 / 32 Outline 1 Matrix operations Importance Dense and sparse

More information

Two-view geometry Computer Vision Spring 2018, Lecture 10

Two-view geometry Computer Vision Spring 2018, Lecture 10 Two-view geometry http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 10 Course announcements Homework 2 is due on February 23 rd. - Any questions about the homework? - How many of

More information

EE/CSCI 451 Midterm 1

EE/CSCI 451 Midterm 1 EE/CSCI 451 Midterm 1 Spring 2018 Instructor: Xuehai Qian Friday: 02/26/2018 Problem # Topic Points Score 1 Definitions 20 2 Memory System Performance 10 3 Cache Performance 10 4 Shared Memory Programming

More information

D-BAUG Informatik I. Exercise session: week 5 HS 2018

D-BAUG Informatik I. Exercise session: week 5 HS 2018 1 D-BAUG Informatik I Exercise session: week 5 HS 2018 Homework 2 Questions? Matrix and Vector in Java 3 Vector v of length n: Matrix and Vector in Java 3 Vector v of length n: double[] v = new double[n];

More information

AMath 483/583 Lecture 11

AMath 483/583 Lecture 11 AMath 483/583 Lecture 11 Outline: Computer architecture Cache considerations Fortran optimization Reading: S. Goedecker and A. Hoisie, Performance Optimization of Numerically Intensive Codes, SIAM, 2001.

More information

Numerical Algorithms

Numerical Algorithms Chapter 10 Slide 464 Numerical Algorithms Slide 465 Numerical Algorithms In textbook do: Matrix multiplication Solving a system of linear equations Slide 466 Matrices A Review An n m matrix Column a 0,0

More information

School of Computing and Information Sciences. Course Title: Data Structures Date: 3/30/2010 Course Number: COP 3530 Number of Credits: 3

School of Computing and Information Sciences. Course Title: Data Structures Date: 3/30/2010 Course Number: COP 3530 Number of Credits: 3 Course Title: Date: 3/30/2010 Course Number: Number of Credits: 3 Subject Area: Programming Subject Area Coordinator: Tim Downey email: downeyt@cis.fiu.edu Catalog Description: Basic concepts of data organization,

More information

Mathematics 4330/5344 #1 Matlab and Numerical Approximation

Mathematics 4330/5344 #1 Matlab and Numerical Approximation David S. Gilliam Department of Mathematics Texas Tech University Lubbock, TX 79409 806 742-2566 gilliam@texas.math.ttu.edu http://texas.math.ttu.edu/~gilliam Mathematics 4330/5344 #1 Matlab and Numerical

More information

Conic Duality. yyye

Conic Duality.  yyye Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 1 Conic Duality Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/

More information

Curriculum Map: Mathematics

Curriculum Map: Mathematics Curriculum Map: Mathematics Course: Honors Advanced Precalculus and Trigonometry Grade(s): 11-12 Unit 1: Functions and Their Graphs This chapter will develop a more complete, thorough understanding of

More information

AMath 483/583 Lecture 11. Notes: Notes: Comments on Homework. Notes: AMath 483/583 Lecture 11

AMath 483/583 Lecture 11. Notes: Notes: Comments on Homework. Notes: AMath 483/583 Lecture 11 AMath 483/583 Lecture 11 Outline: Computer architecture Cache considerations Fortran optimization Reading: S. Goedecker and A. Hoisie, Performance Optimization of Numerically Intensive Codes, SIAM, 2001.

More information

Dense matrix algebra and libraries (and dealing with Fortran)

Dense matrix algebra and libraries (and dealing with Fortran) Dense matrix algebra and libraries (and dealing with Fortran) CPS343 Parallel and High Performance Computing Spring 2018 CPS343 (Parallel and HPC) Dense matrix algebra and libraries (and dealing with Fortran)

More information

A Few Numerical Libraries for HPC

A Few Numerical Libraries for HPC A Few Numerical Libraries for HPC CPS343 Parallel and High Performance Computing Spring 2016 CPS343 (Parallel and HPC) A Few Numerical Libraries for HPC Spring 2016 1 / 37 Outline 1 HPC == numerical linear

More information

Project Report. 1 Abstract. 2 Algorithms. 2.1 Gaussian elimination without partial pivoting. 2.2 Gaussian elimination with partial pivoting

Project Report. 1 Abstract. 2 Algorithms. 2.1 Gaussian elimination without partial pivoting. 2.2 Gaussian elimination with partial pivoting Project Report Bernardo A. Gonzalez Torres beaugonz@ucsc.edu Abstract The final term project consist of two parts: a Fortran implementation of a linear algebra solver and a Python implementation of a run

More information

55:148 Digital Image Processing Chapter 11 3D Vision, Geometry

55:148 Digital Image Processing Chapter 11 3D Vision, Geometry 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence

More information

Multiple View Geometry in Computer Vision

Multiple View Geometry in Computer Vision Multiple View Geometry in Computer Vision Prasanna Sahoo Department of Mathematics University of Louisville 1 Structure Computation Lecture 18 March 22, 2005 2 3D Reconstruction The goal of 3D reconstruction

More information

Multithreaded Algorithms Part 2. Dept. of Computer Science & Eng University of Moratuwa

Multithreaded Algorithms Part 2. Dept. of Computer Science & Eng University of Moratuwa CS4460 Advanced d Algorithms Batch 08, L4S2 Lecture 12 Multithreaded Algorithms Part 2 N. H. N. D. de Silva Dept. of Computer Science & Eng University of Moratuwa Outline: Multithreaded Algorithms Part

More information

Rectangular Matrix Multiplication Revisited

Rectangular Matrix Multiplication Revisited JOURNAL OF COMPLEXITY, 13, 42 49 (1997) ARTICLE NO. CM970438 Rectangular Matrix Multiplication Revisited Don Coppersmith IBM Research, T. J. Watson Research Center, Yorktown Heights, New York 10598 Received

More information

Iterative Algorithms I: Elementary Iterative Methods and the Conjugate Gradient Algorithms

Iterative Algorithms I: Elementary Iterative Methods and the Conjugate Gradient Algorithms Iterative Algorithms I: Elementary Iterative Methods and the Conjugate Gradient Algorithms By:- Nitin Kamra Indian Institute of Technology, Delhi Advisor:- Prof. Ulrich Reude 1. Introduction to Linear

More information

How to Write Fast Numerical Code

How to Write Fast Numerical Code How to Write Fast Numerical Code Lecture: Dense linear algebra, LAPACK, MMM optimizations in ATLAS Instructor: Markus Püschel TA: Daniele Spampinato & Alen Stojanov Today Linear algebra software: history,

More information

Parallel Algorithms CSE /22/2015. Outline of this lecture: 1 Implementation of cilk for. 2. Parallel Matrix Multiplication

Parallel Algorithms CSE /22/2015. Outline of this lecture: 1 Implementation of cilk for. 2. Parallel Matrix Multiplication CSE 539 01/22/2015 Parallel Algorithms Lecture 3 Scribe: Angelina Lee Outline of this lecture: 1. Implementation of cilk for 2. Parallel Matrix Multiplication 1 Implementation of cilk for We mentioned

More information

Sparse matrices: Basics

Sparse matrices: Basics Outline : Basics Bora Uçar RO:MA, LIP, ENS Lyon, France CR-08: Combinatorial scientific computing, September 201 http://perso.ens-lyon.fr/bora.ucar/cr08/ 1/28 CR09 Outline Outline 1 Course presentation

More information

Parallelizing The Matrix Multiplication. 6/10/2013 LONI Parallel Programming Workshop

Parallelizing The Matrix Multiplication. 6/10/2013 LONI Parallel Programming Workshop Parallelizing The Matrix Multiplication 6/10/2013 LONI Parallel Programming Workshop 2013 1 Serial version 6/10/2013 LONI Parallel Programming Workshop 2013 2 X = A md x B dn = C mn d c i,j = a i,k b k,j

More information

Geometric camera models and calibration

Geometric camera models and calibration Geometric camera models and calibration http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 13 Course announcements Homework 3 is out. - Due October

More information

Chapter 5. Divide and Conquer. Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved.

Chapter 5. Divide and Conquer. Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved. Chapter 5 Divide and Conquer Slides by Kevin Wayne. Copyright 25 Pearson-Addison Wesley. All rights reserved. Divide-and-Conquer Divide-and-conquer. Break up problem into several parts. Solve each part

More information

Database Systems (INFR10070) Dr Paolo Guagliardo. University of Edinburgh. Fall 2016

Database Systems (INFR10070) Dr Paolo Guagliardo. University of Edinburgh. Fall 2016 Database Systems (INFR10070) Dr Paolo Guagliardo University of Edinburgh Fall 2016 Databases are everywhere Electronic commerce, websites (e.g., Wordpress blogs) Banking applications, booking systems,

More information

INTRODUCTION TO CAD/CAM SYSTEMS IM LECTURE HOURS PER WEEK PRESENTIAL

INTRODUCTION TO CAD/CAM SYSTEMS IM LECTURE HOURS PER WEEK PRESENTIAL COURSE CODE INTENSITY MODALITY CHARACTERISTIC PRE-REQUISITE CO-REQUISITE CREDITS ACTUALIZATION DATE INTRODUCTION TO CAD/CAM SYSTEMS IM0242 3 LECTURE HOURS PER WEEK 48 HOURS CLASSROOM ON 16 WEEKS, 96 HOURS

More information

Sparse matrices, graphs, and tree elimination

Sparse matrices, graphs, and tree elimination Logistics Week 6: Friday, Oct 2 1. I will be out of town next Tuesday, October 6, and so will not have office hours on that day. I will be around on Monday, except during the SCAN seminar (1:25-2:15);

More information

Introduction to Programming in C Department of Computer Science and Engineering. Lecture No. #16 Loops: Matrix Using Nested for Loop

Introduction to Programming in C Department of Computer Science and Engineering. Lecture No. #16 Loops: Matrix Using Nested for Loop Introduction to Programming in C Department of Computer Science and Engineering Lecture No. #16 Loops: Matrix Using Nested for Loop In this section, we will use the, for loop to code of the matrix problem.

More information

Mathematics and Computer Science

Mathematics and Computer Science Technical Report TR-2006-010 Revisiting hypergraph models for sparse matrix decomposition by Cevdet Aykanat, Bora Ucar Mathematics and Computer Science EMORY UNIVERSITY REVISITING HYPERGRAPH MODELS FOR

More information

CS/ENGRD 2110 Object-Oriented Programming and Data Structures Spring 2012 Thorsten Joachims. Lecture 10: Asymptotic Complexity and

CS/ENGRD 2110 Object-Oriented Programming and Data Structures Spring 2012 Thorsten Joachims. Lecture 10: Asymptotic Complexity and CS/ENGRD 2110 Object-Oriented Programming and Data Structures Spring 2012 Thorsten Joachims Lecture 10: Asymptotic Complexity and What Makes a Good Algorithm? Suppose you have two possible algorithms or

More information

CS 450 Numerical Analysis. Chapter 7: Interpolation

CS 450 Numerical Analysis. Chapter 7: Interpolation Lecture slides based on the textbook Scientific Computing: An Introductory Survey by Michael T. Heath, copyright c 2018 by the Society for Industrial and Applied Mathematics. http://www.siam.org/books/cl80

More information

Autar Kaw Benjamin Rigsby. Transforming Numerical Methods Education for STEM Undergraduates

Autar Kaw Benjamin Rigsby.   Transforming Numerical Methods Education for STEM Undergraduates Autar Kaw Benjamin Rigsy http://nmmathforcollegecom Transforming Numerical Methods Education for STEM Undergraduates http://nmmathforcollegecom LU Decomposition is another method to solve a set of simultaneous

More information

ECEN 615 Methods of Electric Power Systems Analysis Lecture 11: Sparse Systems

ECEN 615 Methods of Electric Power Systems Analysis Lecture 11: Sparse Systems ECEN 615 Methods of Electric Power Systems Analysis Lecture 11: Sparse Systems Prof. Tom Overbye Dept. of Electrical and Computer Engineering Texas A&M University overbye@tamu.edu Announcements Homework

More information

Image Compression using Singular Value Decomposition

Image Compression using Singular Value Decomposition Applications of Linear Algebra 1/41 Image Compression using Singular Value Decomposition David Richards and Adam Abrahamsen Introduction The Singular Value Decomposition is a very important process. In

More information

DM545 Linear and Integer Programming. Lecture 2. The Simplex Method. Marco Chiarandini

DM545 Linear and Integer Programming. Lecture 2. The Simplex Method. Marco Chiarandini DM545 Linear and Integer Programming Lecture 2 The Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1. 2. 3. 4. Standard Form Basic Feasible Solutions

More information

AMSC/CMSC 460 Final Exam, Fall 2007

AMSC/CMSC 460 Final Exam, Fall 2007 AMSC/CMSC 460 Final Exam, Fall 2007 Show all work. You may leave arithmetic expressions in any form that a calculator could evaluate. By putting your name on this paper, you agree to abide by the university

More information

Teaching Manual Math 2131

Teaching Manual Math 2131 Math 2131 Linear Algebra Labs with MATLAB Math 2131 Linear algebra with Matlab Teaching Manual Math 2131 Contents Week 1 3 1 MATLAB Course Introduction 5 1.1 The MATLAB user interface...........................

More information

MA/CSSE 473 Day 17. Divide-and-conquer Convex Hull. Strassen's Algorithm: Matrix Multiplication. (if time, Shell's Sort)

MA/CSSE 473 Day 17. Divide-and-conquer Convex Hull. Strassen's Algorithm: Matrix Multiplication. (if time, Shell's Sort) MA/CSSE 473 Day 17 Divide-and-conquer Convex Hull Strassen's Algorithm: Matrix Multiplication (if time, Shell's Sort) MA/CSSE 473 Day 17 Student Questions Exam 2 specification Levitin 3 rd Edition Closest

More information

CMPSCI 250: Introduction to Computation. Lecture #1: Things, Sets and Strings David Mix Barrington 22 January 2014

CMPSCI 250: Introduction to Computation. Lecture #1: Things, Sets and Strings David Mix Barrington 22 January 2014 CMPSCI 250: Introduction to Computation Lecture #1: Things, Sets and Strings David Mix Barrington 22 January 2014 Things, Sets, and Strings The Mathematical Method Administrative Stuff The Objects of Mathematics

More information

calibrated coordinates Linear transformation pixel coordinates

calibrated coordinates Linear transformation pixel coordinates 1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial

More information

Exam Design and Analysis of Algorithms for Parallel Computer Systems 9 15 at ÖP3

Exam Design and Analysis of Algorithms for Parallel Computer Systems 9 15 at ÖP3 UMEÅ UNIVERSITET Institutionen för datavetenskap Lars Karlsson, Bo Kågström och Mikael Rännar Design and Analysis of Algorithms for Parallel Computer Systems VT2009 June 2, 2009 Exam Design and Analysis

More information

CPSC 340: Machine Learning and Data Mining. Ranking Fall 2016

CPSC 340: Machine Learning and Data Mining. Ranking Fall 2016 CPSC 340: Machine Learning and Data Mining Ranking Fall 2016 Assignment 5: Admin 2 late days to hand in Wednesday, 3 for Friday. Assignment 6: Due Friday, 1 late day to hand in next Monday, etc. Final:

More information

Course Number 432/433 Title Algebra II (A & B) H Grade # of Days 120

Course Number 432/433 Title Algebra II (A & B) H Grade # of Days 120 Whitman-Hanson Regional High School provides all students with a high- quality education in order to develop reflective, concerned citizens and contributing members of the global community. Course Number

More information

Introduction to Computer Systems

Introduction to Computer Systems Introduction to Computer Systems Syllabus Web Page http://www.cs.northwestern.edu/~pdinda/icsclass Instructor Peter A. Dinda 1890 Maple Avenue, Room 338 847-467-7859 pdinda@cs.northwestern.edu Office hours:

More information

Algorithms and Data Structures

Algorithms and Data Structures Algorithms and Data Structures or, Classical Algorithms of the 50s, 60s, 70s Richard Mayr Slides adapted from Mary Cryan (2015/16) with small changes. School of Informatics University of Edinburgh ADS

More information

CVEN 302. Computer Applications in Engineering and Construction. Dr. Tony Cahill Environmental and Water Resources Division

CVEN 302. Computer Applications in Engineering and Construction. Dr. Tony Cahill Environmental and Water Resources Division CVEN 302 Computer Applications in Engineering and Construction Dr. Tony Cahill Environmental and Water Resources Division Instructors Instructor: Tony Cahill Office: WERC 205J Office Hours: T/R 3:00 4:00PM.

More information

Chapter 18. Geometric Operations

Chapter 18. Geometric Operations Chapter 18 Geometric Operations To this point, the image processing operations have computed the gray value (digital count) of the output image pixel based on the gray values of one or more input pixels;

More information

Parallel Numerical Algorithms

Parallel Numerical Algorithms Parallel Numerical Algorithms Chapter 5 Vector and Matrix Products Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign CS 554 / CSE 512 Michael T. Heath Parallel

More information

Extra-High Speed Matrix Multiplication on the Cray-2. David H. Bailey. September 2, 1987

Extra-High Speed Matrix Multiplication on the Cray-2. David H. Bailey. September 2, 1987 Extra-High Speed Matrix Multiplication on the Cray-2 David H. Bailey September 2, 1987 Ref: SIAM J. on Scientic and Statistical Computing, vol. 9, no. 3, (May 1988), pg. 603{607 Abstract The Cray-2 is

More information

STA141C: Big Data & High Performance Statistical Computing

STA141C: Big Data & High Performance Statistical Computing STA141C: Big Data & High Performance Statistical Computing Lecture 0: Course information Cho-Jui Hsieh UC Davis April 3, 2018 Course Information Website: http://www.stat.ucdavis.edu/~chohsieh/teaching/

More information

GEOMETRIC TOOLS FOR COMPUTER GRAPHICS

GEOMETRIC TOOLS FOR COMPUTER GRAPHICS GEOMETRIC TOOLS FOR COMPUTER GRAPHICS PHILIP J. SCHNEIDER DAVID H. EBERLY MORGAN KAUFMANN PUBLISHERS A N I M P R I N T O F E L S E V I E R S C I E N C E A M S T E R D A M B O S T O N L O N D O N N E W

More information

Multiple View Geometry in Computer Vision

Multiple View Geometry in Computer Vision Multiple View Geometry in Computer Vision Prasanna Sahoo Department of Mathematics University of Louisville 1 Projective 3D Geometry (Back to Chapter 2) Lecture 6 2 Singular Value Decomposition Given a

More information

Lecture Notes for Chapter 2: Getting Started

Lecture Notes for Chapter 2: Getting Started Instant download and all chapters Instructor's Manual Introduction To Algorithms 2nd Edition Thomas H. Cormen, Clara Lee, Erica Lin https://testbankdata.com/download/instructors-manual-introduction-algorithms-2ndedition-thomas-h-cormen-clara-lee-erica-lin/

More information

Chapter 18 out of 37 from Discrete Mathematics for Neophytes: Number Theory, Probability, Algorithms, and Other Stuff by J. M. Cargal.

Chapter 18 out of 37 from Discrete Mathematics for Neophytes: Number Theory, Probability, Algorithms, and Other Stuff by J. M. Cargal. Chapter 8 out of 7 from Discrete Mathematics for Neophytes: Number Theory, Probability, Algorithms, and Other Stuff by J. M. Cargal 8 Matrices Definitions and Basic Operations Matrix algebra is also known

More information

Matrix Multiplication and All Pairs Shortest Paths (2002; Zwick)

Matrix Multiplication and All Pairs Shortest Paths (2002; Zwick) Matrix Multiplication and All Pairs Shortest Paths (2002; Zwick) Tadao Takaoka, University of Canterbury www.cosc.canterbury.ac.nz/tad.takaoka INDEX TERMS: all pairs shortest path problem, matrix multiplication,

More information

Finite Element Analysis Prof. Dr. B. N. Rao Department of Civil Engineering Indian Institute of Technology, Madras. Lecture - 24

Finite Element Analysis Prof. Dr. B. N. Rao Department of Civil Engineering Indian Institute of Technology, Madras. Lecture - 24 Finite Element Analysis Prof. Dr. B. N. Rao Department of Civil Engineering Indian Institute of Technology, Madras Lecture - 24 So in today s class, we will look at quadrilateral elements; and we will

More information

CITS2401 Computer Analysis & Visualisation

CITS2401 Computer Analysis & Visualisation FACULTY OF ENGINEERING, COMPUTING AND MATHEMATICS CITS2401 Computer Analysis & Visualisation SCHOOL OF COMPUTER SCIENCE AND SOFTWARE ENGINEERING Topic 3 Introduction to Matlab Material from MATLAB for

More information

Generalized trace ratio optimization and applications

Generalized trace ratio optimization and applications Generalized trace ratio optimization and applications Mohammed Bellalij, Saïd Hanafi, Rita Macedo and Raca Todosijevic University of Valenciennes, France PGMO Days, 2-4 October 2013 ENSTA ParisTech PGMO

More information

COMP 558 lecture 19 Nov. 17, 2010

COMP 558 lecture 19 Nov. 17, 2010 COMP 558 lecture 9 Nov. 7, 2 Camera calibration To estimate the geometry of 3D scenes, it helps to know the camera parameters, both external and internal. The problem of finding all these parameters is

More information

Vector: A series of scalars contained in a column or row. Dimensions: How many rows and columns a vector or matrix has.

Vector: A series of scalars contained in a column or row. Dimensions: How many rows and columns a vector or matrix has. ASSIGNMENT 0 Introduction to Linear Algebra (Basics of vectors and matrices) Due 3:30 PM, Tuesday, October 10 th. Assignments should be submitted via e-mail to: matlabfun.ucsd@gmail.com You can also submit

More information

Matlab- Command Window Operations, Scalars and Arrays

Matlab- Command Window Operations, Scalars and Arrays 1 ME313 Homework #1 Matlab- Command Window Operations, Scalars and Arrays Last Updated August 17 2012. Assignment: Read and complete the suggested commands. After completing the exercise, copy the contents

More information

CSE 202 Divide-and-conquer algorithms. Fan Chung Graham UC San Diego

CSE 202 Divide-and-conquer algorithms. Fan Chung Graham UC San Diego CSE 22 Divide-and-conquer algorithms Fan Chung Graham UC San Diego Announcements Homework due today before the class. About homework, write your own homework, allowing oral discussion with one fixed partner.

More information