Jord Cortadella Department of Computer Scence Introducton to Programmng Dept. CS, UPC 2 An essental algorthm n Lnear Algebra wth multple applcatons: Solvng lnear systems of equatons Fndng the nverse of a matrx Computng determnants Computng ranks and bases of vector spaces Named after Carl Fredrch Gauss (1777-1855), but known much before by the Chnese. Alan Turng contrbuted to provde modern numercal algorthms for computers (characterzaton of ll-condtoned systems). Introducton to Programmng Dept. CS, UPC 3 Introducton to Programmng Dept. CS, UPC 4
// Type defntons for real vectors and matrces usng rvector vector<double>; usng rmatrx vector<rvector>; // Pre: A s an n n matrx, b s an n-element vector. // Returns x such that Axb. In case A s sngular, // t returns a zero-szed vector. // Post: A and b are modfed. rvector SystemEquatons(rmatrx& A, rvector& b) { bool nvertble GaussElmnaton(A, b); f (not nvertble) return rvector(); // A s n row echelon form return BackSubsttuton(A, b); Axb A x b Assumpton: trangular system of equatons wthout zeroes n the dagonal Introducton to Programmng Dept. CS, UPC 5 Introducton to Programmng Dept. CS, UPC 6 Introducton to Programmng Dept. CS, UPC 7 Introducton to Programmng Dept. CS, UPC 8
// Pre: A s an nvertble n n matrx n row echelon form, // b s a n-element vector // Returns x such that Axb rvector BackSubsttuton(const rmatrx& A, const rvector& b) { nt n A.sze(); rvector x(n); // Creates the vector for the soluton // Calculates x from x[] to x[] for (nt n - 1; > ; --) { // The values x[+1..] have already been calculated double s ; for (nt j + 1; j < n; ++j) s s + A[][j] x[j]; x[] (b[] s)/a[][]; return x; k A b Invarant: The rows..k have been reduced (zeroes at the left of the dagonal) Introducton to Programmng Dept. CS, UPC 9 Introducton to Programmng Dept. CS, UPC 1 j k What s the best pvot? (max absolute value) swap rows For each row, add a multple of row k such that A[][k] becomes zero. The coeffcent s A[][k]/A[k][k]. Introducton to Programmng Dept. CS, UPC 11 Introducton to Programmng Dept. CS, UPC 12
// Pre: A s an n n matrx, b s a n-element vector. // Returns true f A s nvertble, and false f A s sngular. // Post: If A s nvertble, A and b are the result of // the (A s n row echelon form). bool GaussanElmnaton(rmatrx& A, rvector& b) { nt n A.sze(); Dscusson: how large should a pvot be? // Reduce rows 1.. (use the pvot n prevous row k) for (nt k ; k < n - 1; ++k) { // Rows..k have already been reduced nt max fnd_max_pvot(a, k); // fnds the max pvot f (abs(a[max][k]) < 1e-1) return false; // Sngular matrx // Pre: A s an n n matrx, k s the ndex of a row. // Returns the ndex of the row wth max absolute value // for the subcolumn A[k..][k]. nt fnd_max_pvot(const rmatrx& A, nt k) { nt n A.sze(); double max k; // ndex of the row wth max pvot double max_pvot abs(a[k][k]); swap(a[k], A[max]); swap(b[k], b[max]); // Swap rows k and max // Force s n column A[k+1..][k] for (nt k + 1; < n; ++) { double c A[][k]/A[k][k]; // coeffcent to scale row A[][k] ; for (nt j k + 1; j < n; ++j) A[][j] A[][j] c A[k][j]; b[] b[] c b[k]; return true; // We have an nvertble matrx for (nt k + 1; < n; ++) { double a abs(a[][k]); f (a > max_pvot) { max_pvot a; max ; return max; Introducton to Programmng Dept. CS, UPC 13 Solvng multple systems of equatons Introducton to Programmng Dept. CS, UPC 14 // Pre: A s an nvertble n n matrx n row echelon form, // B s a n m matrx // Returns X such that AXB rmatrx BackSubsttuton(const rmatrx& A, const rmatrx& B) { nt n A.sze(); nt m B[].sze(); rmatrx X(n, rvector(m)); // Creates the soluton matrx // Calculates X from X[] to X[] for (nt n - 1; > ; --) { // The values X[+1..] have already been calculated for (nt k ; k < m; ++k) { double s ; for (nt j + 1; j < n; ++j) s s + A[][j] X[j][k]; X[][k] (B[][k] s)/a[][]; return X; Introducton to Programmng Dept. CS, UPC 15 Introducton to Programmng Dept. CS, UPC 16
Inverse of a Matrx // Pre: A s an n n matrx, B s an n m matrx. // Returns true f A s nvertble, and false f A s sngular. // Post: If A s nvertble, A and B are the result of // the (A s n row echelon form). bool GaussanElmnaton(rmatrx& A, rmatrx& B) { nt n A.sze(); nt m B[].sze(); Reduce the problem to a set of systems of lnear equatons: // Reduce rows 1.. (use the pvot n prevous row k) for (nt k ; k < n - 1; ++k) { // Rows..k have already been reduced nt max fnd_max_pvot(a, k); // fnds the max pvot f (abs(a[max][k]) < 1e-1) return false; // Sngular matrx swap(a[k], A[max]); swap(b[k], B[max]); // Swap rows k and max // Force s n column A[k+1..][k] for (nt k + 1; < n; ++) { double c A[][k]/A[k][k]; // coeffcent to scale row A[][k] ; for (nt j k + 1; j < n; ++j) A[][j] A[][j] c A[k][j]; for (nt l ; l < m; ++l) B[][l] B[][l] c B[k][l]; return true; // We have an nvertble matrx Introducton to Programmng Dept. CS, UPC 17 Computng determnants Rules of determnants: 1. Swappng two rows: determnant multpled by -1 2. Multplyng a row by a scalar: the determnant s multpled by the same scalar 3. Addng to one row the scalar multple of another row: determnant does not change Algorthm: Do and remember the number of row swaps (odd or even). The determnant s the product of the elements n the dagonal (negated n case of an odd number of swaps). Rule 2 s not used, unless some row s scaled. Introducton to Programmng Dept. CS, UPC 18 Summary s the most used algorthm n Lnear Algebra. Complexty: O(n 3 ). There are many good packages to solve Lnear Algebra operatons (LAPACK, LINPACK, Matlab, Mathematca, NumPy, R, ). Introducton to Programmng Dept. CS, UPC 19 Introducton to Programmng Dept. CS, UPC 2