Lecture 8 Mathematics

Transcription

1 CS 491 CAP Intro to Competitive Algorithmic Programming Lecture 8 Mathematics Uttam Thakore University of Illinois at Urbana-Champaign October 14, 2015

2 Outline Number theory Combinatorics & probability Exponentiation & recurring series Linear algebra & root-finding Language considerations CS 491 CAP Intro to Competitive Algorithmic Programming 2

3 Number Theory CS 491 CAP Intro to Competitive Algorithmic Programming 3

4 Outline Number theory Primality and prime factorization GCD/LCM and Euclid s algorithm Combinatorics & probability Exponentiation & recurring series Linear algebra & root-finding Language considerations CS 491 CAP Intro to Competitive Algorithmic Programming 4

5 Prime number Integer > 1 that has only two divisors, 1 and itself Rest of the positive integers are called composite First few prime numbers: 2, 3, 5, 7, 11, 13, 17, 19, Is there an efficient algorithm to check if a given integer is prime? CS 491 CAP Intro to Competitive Algorithmic Programming 5

6 Primality testing algorithm Naively, can loop from 2 to N 1 and check if divisible VERY SLOW Better solution: for i = 2.. sqrt(n) if N % i == 0 return Composite return Prime CS 491 CAP Intro to Competitive Algorithmic Programming 6

7 Why it works Any composite number N can be expressed as N = a b WLOG, a b, then clearly a Thus, no need to loop through all N Complexity: O( N) There is a much faster algorithm based on probabilistic guarantees, but out of scope for this lecture Miller-Rabin algorithm N CS 491 CAP Intro to Competitive Algorithmic Programming 7

8 Sieve of Eratosthenes What if we want to generate a list of prime numbers less than N? Observation: if p is a prime number, then 2p, 3p, 4p, are all composite Create a boolean table of size N P[i] will be true if i is prime Can create a HashMap instead if N is large CS 491 CAP Intro to Competitive Algorithmic Programming 8

9 Sieve of Eratosthenes Set P[2.. N] to initially true for i = 2.. sqrt(n): if P[i] == true: for j = 2.. N / i: P[i * j] = false Complexity: O(N log log N) CS 491 CAP Intro to Competitive Algorithmic Programming 9

10 Greatest Common Divisor (GCD) Given two integers a, b, GCD(a, b) is defined as the largest integer that divides both a and b Examples: GCD(16, 24) = 8 GCD( 5, 15) = 5 CS 491 CAP Intro to Competitive Algorithmic Programming 10

11 Euclidean Algorithm Not going to prove it, but GCD a, b = GCD(b, a mod b) Since GCD(a, 0) = a, can recursively compute GCD Very fast CS 491 CAP Intro to Competitive Algorithmic Programming 11

12 Least Common Multiple LCM(a, b) is defined as the smallest positive integer n that both a and b divides Examples: LCM(3, 5) = 15 LCM(6, 24) = 24 LCM(6, 8) = 24 LCM a, b = a b GCD(a,b) To avoid possible intermediate overflow, compute as a GCD a, b b (in that order) in programming languages CS 491 CAP Intro to Competitive Algorithmic Programming 12

13 Questions so far? CS 491 CAP Intro to Competitive Algorithmic Programming 13

14 Combinatorics and Probability CS 491 CAP Intro to Competitive Algorithmic Programming 14

15 Outline Number theory Combinatorics & probability Basic combinatorics concepts Pascal s triangle Principle of Inclusion Exclusion Basic probability theory Exponentiation & recurring series Linear algebra & root-finding Language considerations CS 491 CAP Intro to Competitive Algorithmic Programming 15

16 Permutations P n, r is defined as the number of ways to arrange r of n objects, where order matters P n, r = n! (n r)! There are n ways to pick an object for the first slot, n - 1 ways to pick an object for the second slot, etc. CS 491 CAP Intro to Competitive Algorithmic Programming 16

17 Combinations C n, r is defined as the number of ways to pick r objects from n objects Also notated as n choose k or n C k or C n, r = P n,r r! = n! r! n r! Permutation cares about the order of the picked objects, while combination doesn t So we divide by r! n k CS 491 CAP Intro to Competitive Algorithmic Programming 17

18 Pascal s Rule Pascal s rule states: C n, r = C n 1, r + C n 1, r 1 C n, 0 = 1 for any n Using this identity, we can quickly construct a table that stores C(n,r), i.e. T[n][r] = C(n,r) Useful because the query time is constant CS 491 CAP Intro to Competitive Algorithmic Programming 18

19 Principle of Inclusion Exclusion Given two sets A, B: A B = A + B A B More generally, given n sets A 1, A 2, A 3,, A n : CS 491 CAP Intro to Competitive Algorithmic Programming 19

20 Applications of PIE Derangement problem SPOJ NOVICE62 Description: Given an integer n, determine the number of permutations of length n where no element appears in its original position Examples: 1, 3, 4, 2 (invalid, because 1 is in its original position) 4, 3, 2, 1 (valid) CS 491 CAP Intro to Competitive Algorithmic Programming 20

21 Probability theory Topics: Probability distribution/density functions (PDFs) and cumulative distribution/density functions (CDFs) Independence of probability Expectation Common distributions CS 491 CAP Intro to Competitive Algorithmic Programming 21

22 Definition of probability Consider a fair die (can roll between a 1 and a 6) with equal probability Each possible result is an outcome The set of outcomes, 1, 2, 3, 4, 5, 6, is called the sample space Each subset of possible outcomes is an event E.g., rolling an even number The probability mass function (pmf), p x, relates each outcome to its probability p(1) = 1 6, p(2) = 1 6 CS 491 CAP Intro to Competitive Algorithmic Programming 22

23 Types of distributions Discrete distributions: Sample space is a countable set of outcomes Examples: rolling dice, flipping a coin Defined by a probability mass function (pmf) Continuous distributions: Sample space is an uncountable set of outcomes Think ranges of real numbers Examples: picking a real number between 0 and 1 Defined by a probability density function (pdf) CS 491 CAP Intro to Competitive Algorithmic Programming 23

24 Expected value Suppose a random variable X can take value x 1 with probability p 1, x 2 with probability p 2, up to x k with probability p k. Then, k E[X] = σ i=1 x i p i Given a (continuous) probability density function p x, E X = x p x dx CS 491 CAP Intro to Competitive Algorithmic Programming 24

25 Linearity of expectation Given two random variables X and Y E X + Y = E X + E Y E X 1 + X X n = E X 1 + E X E X n Seemingly difficult problems can sometimes be solved easily using this property Note: E XY E X E Y except in particular cases! CS 491 CAP Intro to Competitive Algorithmic Programming 25

26 Independence & expectation Independence: Two events A and B are independent if: P A B = P A P B Two random variables X and Y are independent if: p X,Y x, y = p X x p Y y Given two independent random variables X and Y: E XY = E X E Y CS 491 CAP Intro to Competitive Algorithmic Programming 26

27 Common types of distributions Learn some properties (expectation, distribution) of the following: Uniform distribution Bernoulli distribution Binomial distribution Geometric distribution Exponential distribution CS 491 CAP Intro to Competitive Algorithmic Programming 27

28 Exponentiation & Recurring Series CS 491 CAP Intro to Competitive Algorithmic Programming 28

29 Outline Number theory Combinatorics & probability Exponentiation & recurring series Fast exponentiation Recursively-defined series Linear algebra & root-finding Language considerations CS 491 CAP Intro to Competitive Algorithmic Programming 29

30 Fast exponentiation Recursive computation of a n : a n = 1 n = 0 a n = 1 a nτ2 2 a a nτ2 2 n is even n is odd Can be used for computation of a large power of anything (integers, doubles, matrices, etc.) in logarithmic time Particularly useful with matrices and modular arithmetic CS 491 CAP Intro to Competitive Algorithmic Programming 30

31 Generating linear recurring series Consider the Fibonacci series F 0 = 0, F 1 = 1 F n = F n-1 + F n-2 Generating normally would require generation of n-1 previous values CS 491 CAP Intro to Competitive Algorithmic Programming 31

32 Generating recursively-defined series Consider another representation F n+1 F n = F n F n 1 = n F 1 F 0 Enter fast exponentiation Now, computation takes logarithmic time Can be used to generate any linear recursively defined series CS 491 CAP Intro to Competitive Algorithmic Programming 32

33 Questions so far? CS 491 CAP Intro to Competitive Algorithmic Programming 33

34 Linear Algebra & Root-Finding CS 491 CAP Intro to Competitive Algorithmic Programming 34

35 Outline Number theory Combinatorics & probability Exponentiation & recurring series Linear algebra & root-finding Solving linear equations Finding polynomial (and non-polynomial) roots Language considerations CS 491 CAP Intro to Competitive Algorithmic Programming 35

36 Linear algebra Types of problems: Solve a system of linear equations Invert a matrix Find the rank of a matrix Compute the determinant of a matrix All of the above can be computed using Gaussian elimination Check out Wikipedia for the algorithm Good to include in your reference materials CS 491 CAP Intro to Competitive Algorithmic Programming 36

37 Finding polynomial roots 1 st -order polynomial: ax + b = 0 Can solve directly 2 nd -order polynomial: ax 2 + bx + c = 0 Can solve using the quadratic formula What about 3 rd -order and up? Yes, some formulas exist, but they can get very complicated CS 491 CAP Intro to Competitive Algorithmic Programming 37

38 Bisection method Algorithm: Start with an interval [a, b] that contains only a single root Calculate c, the midpoint Examine the sign of f(c) and replace either a or b with c so the new interval contains the root Repeat until convergence Uses binary search Can be used to find roots of nonpolynomial functions as well! Image: Wikipedia CS 491 CAP Intro to Competitive Algorithmic Programming 38

39 More advanced techniques Remember your calculus? Newton s method Secant method Etc. CS 491 CAP Intro to Competitive Algorithmic Programming 39

40 Questions so far? CS 491 CAP Intro to Competitive Algorithmic Programming 40

41 Language Considerations CS 491 CAP Intro to Competitive Algorithmic Programming 41

42 Outline Number theory Combinatorics & probability Exponentiation & recurring series Linear algebra & root-finding Language considerations Double formatting BigInteger & BigDecimal considerations CS 491 CAP Intro to Competitive Algorithmic Programming 42

43 Basic functions Available in <cmath> (C++), java.lang.math (Java), or math module (Python) Functions: sin, cos, tan, asin, acos, atan, atan2 And their hyperbolic equivalents abs (and fabs in C++/Python), copysign (C++11/Python) and signum (Java) ceil, floor, round (C++/Java), trunc (Python) max, min sqrt, pow, hypot, log, log10, log1p, exp, expm1 Constants: e (call exp(1) in C++), π (call acos(-1) in C++) Complex numbers: <complex> (C++) and cmath module (Python) CS 491 CAP Intro to Competitive Algorithmic Programming 43

44 When to use float vs. double floats are 32-bit data types, with 23 bits dedicated to the significand Maximum precision: ~7 significant digits doubles are 64-bit data types, with 52 bits dedicated to the significand Maximum precision: ~15 significant digits ALWAYS USE DOUBLES! CS 491 CAP Intro to Competitive Algorithmic Programming 44

45 Output formatting Some problems may call for formatting a number to k decimal places Use printf (C++), System.out.printf (Java), or print (Python) to do formatting Example: print/printf( %.3f, d) will print d to 3 decimal places For string formatting, can also use stringstream and sprintf (C++), String.format (Java), or string.format (Python) CS 491 CAP Intro to Competitive Algorithmic Programming 45

46 Output formatting Beware the rounding! GNU C++ uses the round-half-to-even rounding mode, which rounds to the nearest even number if the last digit is halfway between numbers Java uses the round-half-up rounding mode, which rounds up if the last digit is halfway between numbers For example, in C++, printf( %.1f, ) will print 123.4, whereas in Java, it will print You can change the rounding mode in C/C++ using fesetround in <cfenv> and in Java using java.math.roundingmode CS 491 CAP Intro to Competitive Algorithmic Programming 46

47 Arbitrary precision numbers Normal numeric types have fixed limits (e.g., 32-bit or 64-bit integer/float) We want types that can support arbitrary-length numbers Python: int and decimal Note: Python decimal and Java BigDecimal differ in functionality Java: BigInteger and BigDecimal Support addition, subtraction, multiplication, division *, exponentiation, negation BigInteger also supports bitwise operations Read API for more details lots of good stuff there! Not available in standard C++ (there are external libraries, but can t use them in ICPC) CS 491 CAP Intro to Competitive Algorithmic Programming 47

48 Notes on BigInteger and BigDecimal BigInteger stores arbitrary precision integers, but it is slow Only use when you can t do the work with longs BigDecimal stores arbitrary precision decimals numbers, but it can t deal with repeating decimals! BigDecimal.ONE.divide(BigDecimal.valueOf(3)) will throw an exception! You can t divide by anything that can t be prime factorized by 2 and 5 Python s decimal can deal with repeating decimals, but it truncates CS 491 CAP Intro to Competitive Algorithmic Programming 48

49 What s left? Trigonometry and geometry were covered in the Computational Geometry lecture More advanced number theory (totient function, modulo arithmetic, etc.) will not be covered Refer to TopCoder tutorials and CP book You rarely need more complicated math for easier competitions CS 491 CAP Intro to Competitive Algorithmic Programming 49

50 Questions? CS 491 CAP Intro to Competitive Algorithmic Programming 50

51 Problem set Problem set will be released tomorrow CS 491 CAP Intro to Competitive Algorithmic Programming 51

52 Resources for this lecture Chapter 5 of Competitive Programming by Steven Halim TopCoder Tutorial Understanding Probabilities Lecture 2 from Stanford s CS 97 SI course CS 491 CAP Intro to Competitive Algorithmic Programming 52