UCT Algorithm Circle: Probabilistic Algorithms

Transcription

1 UCT Algorithm Circle: 24 February 2011

2 Outline 1 2 3

3 Probabilistic algorithm A probabilistic algorithm is one which uses a source of random information The randomness used may result in randomness in the output, or the output may not be affected by it The running time may be good in the expected case, and bad in the worst case We can model such algorithms as probabilistic turing machines

4 Probabilistic algorithm A probabilistic algorithm is one which uses a source of random information The randomness used may result in randomness in the output, or the output may not be affected by it The running time may be good in the expected case, and bad in the worst case We can model such algorithms as probabilistic turing machines

5 Probabilistic algorithm A probabilistic algorithm is one which uses a source of random information The randomness used may result in randomness in the output, or the output may not be affected by it The running time may be good in the expected case, and bad in the worst case We can model such algorithms as probabilistic turing machines

6 Las Vegas algorithm A Las Vegas algorithm is guaranteed to always return the correct result The output may be different if the algorithm is run again (e.g. a non-stable sort may return different, but correct, results) The worst-case running time may be significantly different from the expected time Example: quicksort

7 Las Vegas algorithm A Las Vegas algorithm is guaranteed to always return the correct result The output may be different if the algorithm is run again (e.g. a non-stable sort may return different, but correct, results) The worst-case running time may be significantly different from the expected time Example: quicksort

8 Las Vegas algorithm A Las Vegas algorithm is guaranteed to always return the correct result The output may be different if the algorithm is run again (e.g. a non-stable sort may return different, but correct, results) The worst-case running time may be significantly different from the expected time Example: quicksort

9 Las Vegas algorithm A Las Vegas algorithm is guaranteed to always return the correct result The output may be different if the algorithm is run again (e.g. a non-stable sort may return different, but correct, results) The worst-case running time may be significantly different from the expected time Example: quicksort

10 Monte Carlo method A Monte Carlo method is one in which the results are random; and not necessarily correct The results may be correct some fraction of the time, or they may be a numerical approximation to some quantity Generally a better result can be obtained by running the algorithm for longer, as Monte Carlo methods generally work by random sampling The running time is usually not random Example: Metropolis-Hastings algorithm

11 Monte Carlo method A Monte Carlo method is one in which the results are random; and not necessarily correct The results may be correct some fraction of the time, or they may be a numerical approximation to some quantity Generally a better result can be obtained by running the algorithm for longer, as Monte Carlo methods generally work by random sampling The running time is usually not random Example: Metropolis-Hastings algorithm

12 Monte Carlo method A Monte Carlo method is one in which the results are random; and not necessarily correct The results may be correct some fraction of the time, or they may be a numerical approximation to some quantity Generally a better result can be obtained by running the algorithm for longer, as Monte Carlo methods generally work by random sampling The running time is usually not random Example: Metropolis-Hastings algorithm

13 Monte Carlo method A Monte Carlo method is one in which the results are random; and not necessarily correct The results may be correct some fraction of the time, or they may be a numerical approximation to some quantity Generally a better result can be obtained by running the algorithm for longer, as Monte Carlo methods generally work by random sampling The running time is usually not random Example: Metropolis-Hastings algorithm

14 Monte Carlo method A Monte Carlo method is one in which the results are random; and not necessarily correct The results may be correct some fraction of the time, or they may be a numerical approximation to some quantity Generally a better result can be obtained by running the algorithm for longer, as Monte Carlo methods generally work by random sampling The running time is usually not random Example: Metropolis-Hastings algorithm

15 Outline 1 2 3

16 Building a binary search tree We can quickly build a binary search tree from a list by adding elements one by one, starting at the root In the case of strange input data, this may give an unbalanced tree If items are added in a random order, the expected height will be small regardless of the input ordering

17 Building a binary search tree We can quickly build a binary search tree from a list by adding elements one by one, starting at the root In the case of strange input data, this may give an unbalanced tree If items are added in a random order, the expected height will be small regardless of the input ordering

18 Building a binary search tree We can quickly build a binary search tree from a list by adding elements one by one, starting at the root In the case of strange input data, this may give an unbalanced tree If items are added in a random order, the expected height will be small regardless of the input ordering

19 Polygon triangulation Seidel s algorithm for polygon triangulation requires processing the edges of a polygon sequentially and hence updating a tree The tree will potentially become unbalanced Randomization makes this less likely This results in an algorithm of O(nlog (n)) In this case, similar to with constructing a binary search tree, randomisation is useful because it makes the worst case performance unlikely to occur

20 Polygon triangulation Seidel s algorithm for polygon triangulation requires processing the edges of a polygon sequentially and hence updating a tree The tree will potentially become unbalanced Randomization makes this less likely This results in an algorithm of O(nlog (n)) In this case, similar to with constructing a binary search tree, randomisation is useful because it makes the worst case performance unlikely to occur

21 Polygon triangulation Seidel s algorithm for polygon triangulation requires processing the edges of a polygon sequentially and hence updating a tree The tree will potentially become unbalanced Randomization makes this less likely This results in an algorithm of O(nlog (n)) In this case, similar to with constructing a binary search tree, randomisation is useful because it makes the worst case performance unlikely to occur

22 Probabilistic primality testing A number of efficient algorithms for checking primality of numbers are in co RP. For Miller-Rabin, for example, when the algorithm indicates compositeness, it is guaranteed, and the probability of the test incorrectly identifying a composite as prime can be made exponentially smaller by linearly increasing the running time. This can be used to make the algorithm effectively never fail, by using a few tens of iterations.

23 Probabilistic primality testing A number of efficient algorithms for checking primality of numbers are in co RP. For Miller-Rabin, for example, when the algorithm indicates compositeness, it is guaranteed, and the probability of the test incorrectly identifying a composite as prime can be made exponentially smaller by linearly increasing the running time. This can be used to make the algorithm effectively never fail, by using a few tens of iterations.

24 Probabilistic primality testing A number of efficient algorithms for checking primality of numbers are in co RP. For Miller-Rabin, for example, when the algorithm indicates compositeness, it is guaranteed, and the probability of the test incorrectly identifying a composite as prime can be made exponentially smaller by linearly increasing the running time. This can be used to make the algorithm effectively never fail, by using a few tens of iterations.

25 Genetic algorithms Suppose that we want to maximise some function over some parameter space, but we don t know anything about the function other than that it is (somewhat) continuous. We can choose random points in the space, and select the best one. We can also make use of the fact that the function is continuous to search for better solutions near good solutions. This leads to algorithms such as genetic algorithms and simulated annealing.

26 Genetic algorithms Suppose that we want to maximise some function over some parameter space, but we don t know anything about the function other than that it is (somewhat) continuous. We can choose random points in the space, and select the best one. We can also make use of the fact that the function is continuous to search for better solutions near good solutions. This leads to algorithms such as genetic algorithms and simulated annealing.

27 Genetic algorithms Suppose that we want to maximise some function over some parameter space, but we don t know anything about the function other than that it is (somewhat) continuous. We can choose random points in the space, and select the best one. We can also make use of the fact that the function is continuous to search for better solutions near good solutions. This leads to algorithms such as genetic algorithms and simulated annealing.

28 Rabin s nearest neighbor algorithm Suppose we have a set of points on the plane, and we want to find the two points which have the least distance from each other There exist deterministic algorithms which are O(nlog(n)) to solve this However, there is a probabilistic algorithm which is O(n) expected time, which works by selecting a random subset of points of size n, and hence finding an upper bound d on the minimum distance

29 Rabin s nearest neighbor algorithm Suppose we have a set of points on the plane, and we want to find the two points which have the least distance from each other There exist deterministic algorithms which are O(nlog(n)) to solve this However, there is a probabilistic algorithm which is O(n) expected time, which works by selecting a random subset of points of size n, and hence finding an upper bound d on the minimum distance

30 Outline 1 2 3

31 Probabilistic Turing machine Similar to a non-deterministic turing machine, but the state transition function assigns probabilities to each future state From these probabilities, we can find the probability that the accepting and rejecting states are reached

32 Complexity classes The following are defined with respect to probabilistic turing machines, and refer to decision problems ZPP: Zero-error Probabilistic Polynomial time. The class of problems which can be solved in expected polynomial time. BPP: Bounded-error Probabilistic Polynomial time. The class of problems which can be solved in expected polynomial time with a probability of error of at most 1/3. RP: Randomized Polynomial time The class of decision problems for which False is always returned if the answer is False, and True is returned in at least 1/2 of cases if the answer is True. The polynomial time is expected value. co RP: Similar to RP, with True and False switched. QBP: Uses quantum stuff, otherwise similar to BPP.

33 Complexity classes The following are defined with respect to probabilistic turing machines, and refer to decision problems ZPP: Zero-error Probabilistic Polynomial time. The class of problems which can be solved in expected polynomial time. BPP: Bounded-error Probabilistic Polynomial time. The class of problems which can be solved in expected polynomial time with a probability of error of at most 1/3. RP: Randomized Polynomial time The class of decision problems for which False is always returned if the answer is False, and True is returned in at least 1/2 of cases if the answer is True. The polynomial time is expected value. co RP: Similar to RP, with True and False switched. QBP: Uses quantum stuff, otherwise similar to BPP.

34 Complexity classes The following are defined with respect to probabilistic turing machines, and refer to decision problems ZPP: Zero-error Probabilistic Polynomial time. The class of problems which can be solved in expected polynomial time. BPP: Bounded-error Probabilistic Polynomial time. The class of problems which can be solved in expected polynomial time with a probability of error of at most 1/3. RP: Randomized Polynomial time The class of decision problems for which False is always returned if the answer is False, and True is returned in at least 1/2 of cases if the answer is True. The polynomial time is expected value. co RP: Similar to RP, with True and False switched. QBP: Uses quantum stuff, otherwise similar to BPP.

35 Complexity classes The following are defined with respect to probabilistic turing machines, and refer to decision problems ZPP: Zero-error Probabilistic Polynomial time. The class of problems which can be solved in expected polynomial time. BPP: Bounded-error Probabilistic Polynomial time. The class of problems which can be solved in expected polynomial time with a probability of error of at most 1/3. RP: Randomized Polynomial time The class of decision problems for which False is always returned if the answer is False, and True is returned in at least 1/2 of cases if the answer is True. The polynomial time is expected value. co RP: Similar to RP, with True and False switched. QBP: Uses quantum stuff, otherwise similar to BPP.

36 Complexity classes The following are defined with respect to probabilistic turing machines, and refer to decision problems ZPP: Zero-error Probabilistic Polynomial time. The class of problems which can be solved in expected polynomial time. BPP: Bounded-error Probabilistic Polynomial time. The class of problems which can be solved in expected polynomial time with a probability of error of at most 1/3. RP: Randomized Polynomial time The class of decision problems for which False is always returned if the answer is False, and True is returned in at least 1/2 of cases if the answer is True. The polynomial time is expected value. co RP: Similar to RP, with True and False switched. QBP: Uses quantum stuff, otherwise similar to BPP.

37 Complexity classes The following are defined with respect to probabilistic turing machines, and refer to decision problems ZPP: Zero-error Probabilistic Polynomial time. The class of problems which can be solved in expected polynomial time. BPP: Bounded-error Probabilistic Polynomial time. The class of problems which can be solved in expected polynomial time with a probability of error of at most 1/3. RP: Randomized Polynomial time The class of decision problems for which False is always returned if the answer is False, and True is returned in at least 1/2 of cases if the answer is True. The polynomial time is expected value. co RP: Similar to RP, with True and False switched. QBP: Uses quantum stuff, otherwise similar to BPP.