Algoritmos e Incerteza (PUC-Rio INF979, 017.1) Lecture 1: An Introduction to Online Algorithms Mar 1, 017 Lecturer: Marco Molinaro Scribe: Joao Pedro T. Brandao Online algorithms differ from traditional algorithms in how the instances are processed. In traditional or deterministic algorithms, all of an instance I, is known before the process begins, where as in an online algorithms a decision must be made as each element of the instance is revealed. For example, in finance, when trading stocks we want to be able to predict whether a stock s price will go up or down based on its history. At each moment we must decide whether to buy, sell, or hold. We can thus characterize an online algorithm as the following: 1. The instance I has values in discrete time t, where t = 1,,..., n.. We gain new information at every moment t, e.g. Every minute there is a new price to a particular stock.. At every moment t, we make a decision, e.g. which stock to buy, sell, or hold.. The objective is to either maximize or minimize a certain objective function, e.g. to maximize profits. 1 The Ski-Rental Problem Problem: A ski resort is open during ski season. The season is determined by weather conditions, specifically if there is enough snow on the tracks. At some unknown day in the future, the resort will close. Each day you check if the resort is open or closed. If it s open, you choose whether you want to rent or buy a pair of skis. The rent is valid for the one day, while buying allows you to ski for the remainder of the season. The objective is to minimize your cost. To better understand the problem, we ll work with an example. Assume that ski rental is $1 per day and it costs $b to buy a pair of skis. Each day we re told whether the ski tracks are open or closed, we ll denote l as the last day the resort is open. The instance of this problem is then characterized by b and l, where b is known at time t = 0 and l is unknown. Lets assume that the cost for buying skis is $10, and the resort is opened for four days. Below we exemplify with an instance and a solution. Example: b = 10, and l = 1
Time t Track Status Decision Cost t = 1 Open Rent +1 t = Open Rent +1 t = Open Buy +10 t = Open N.A. +0 t = 5 Closed N.A. +0 Total Cost: 1 If we know before hand that the last open day is the fourth day, the optimal solution has total cost $. It is achieved by renting every day which is cheaper than purchasing the skis. Question: How do we know how good the above algorithm is, given the uncertainty of the instance? Moreover, how can we objectively measure and compare different algorithms to determine which is the best? Answer: Compare the online algorithm to the offline optimal solution, i.e. the optimal solution knowing the last open day of the ski resort. Definition: An algorithm, ALGO is α-competitive if, for every instance I ALGO(I) α Where ALGO(I) returns the cost of the online algorithm and the cost of the optimal offline algorithm. In the Ski-Rental problem, the OP T (b, l) is equal to the minimum value of b and l. OP T (b, l) = minb, l} 1.1 An Algorithm for Ski-Rental Problem Lets assume the cost to buy a pair of skis is $10, and, as usual, we don t know how many days the tracks will remain open, i.e. b = 10 and, l unknown. Any algorithm will need to make a decision when to buy the skis. Lets determine that our algorithm, ALGO decides to buy on the first day (if opened), t = 1. How competitive is ALGO? In evaluating competitiveness of an online algorithm, it is useful to think of an analogy of a twoplayer game, where the first player chooses an algorithm that minimizes the cost, and the second player chooses an instance that maximizes the first player s algorithm. In this case, the worst possible instance would be if the ski tracks close on the following day, l = 1. The algorithm s competitiveness is then 10, α = 10. There is no smaller α for this algorithm. ALGO(I) 10
A more general algorithm, would be to buy on day b. The worst possible instance in this case is when the last day is also day b, i.e. l = b. Intuitively, the algorithm is at least twice as costly as the offline optimal solution. Lemma: Proof. The algorithm ALGO, that chooses to buy at day b is -competitive. ALGO(I) = l if l < b, b 1 + b if l b. If l < b, then ALGO(I) = l l =. If l b then ALGO(I) = b l b =. In either case, ALGO never exceeds twice the value returned by the optimal offline solution. ALGO(I) 1. Lower Bound for Deterministic Algorithms The following lemma gives a slightly tighter bound to the previous result. Lemma: For any algorithm ALGO, there exists an instance I, such that ALGO(I) 1 Proof. First, note that ALGO is equivalent to deciding to buy on day x. Up to day x 1 our cost is $x 1, we buy on day x, raising our cost to $x 1 + b. In which ever day x is, the worst instance, I x, is the one where the ski tracks close on day x + 1, i.e. l = x. Finally, we have that ALGO(I x ) = (x 1) + b while OP T (I x ) = minx, b}. Therefore, ALGO(I x ) minx, b} 1 = OP T (I x ) 1 As desired. 1. Random Algorithms The main idea is to include a random variable into our decision making process. For example, consider the algorithm, A, it flips a coin; If it s heads, A buys on day one, If it s tails, A buys on day 0 (if tracks are open).
Question: What is A s cost? How can we quantify the cost if we can get different costs for the same instance? Answer: We take the expected value of the cost. E A = 1 [cost of buying on day 1] + 1 [cost of buying on day 0] Definition: A random algorithm, A, is α-competitive if for any instance I, E A α The advantage of random algorithms lies in their ability to behave as multiple deterministic algorithms. In the previous example, A buys on the first day half of the time, and the other half buys on day 0. It is then hard to find a single instance that is simultaneously costly to more than one behavior. The following theorem determines a better competitiveness than the deterministic algorithm. Theorem 1.1. ([1]) There exists a random algorithm that is problem. e -competitive for the ski-rental e 1 Proof. (Sketch) We assume there is a probability distribution p t associated with each day t, such that there is always a probability p t that you buy on day t. We must then calculate the theoretical value of E A, for every instance I. Then find the worst instance I and optimize the probabilities p t for that instance. We ll consider a simpler version of the problem where b = and l. The table below shows the competitiveness of each algorithm A i in each instance I j, where i determines which day to buy, and j determines the last day of the ski tracks. Each entry of the table is the ratio between the A algorithm and the optimal solution for that instance, i.e. i (I j ) α. OP T j (I j ) A 1 A A A φ I 1 1 1 1 I 1 1 1 I 1 The best deterministic algorithms are A and A φ that are both -competitive. If we mix algorithms we could potentially improve the competitiveness. We can mix the two best algorithms. Consider that the algorithm A runs algorithm A and A φ, each with 1 probability. Algorithm A has the following results for the different instances:
A I 1 1 I 5 I It is still -competitive. We were unable to improve on the previous result. Note how algorithms A and A φ are simultaneously costly (in relation to the optimal) on instance I. Ideally, we d choose two or more algorithms that are complementary to each other. For this next try, we ll be choosing algorithms A 1 and A, note how they are complementary to each other. This time, we will not be attributing a priori probabilities to them. Let algorithm A 1 occur with probability p and algorithm A occur with probability 1 p. We have the following results for the algorithm A. A I 1 p + 1 I I p p We assume that each instance I j can occur with equal probability, so P r(i 1 ) = P r(i ). We conclude that p = 1. By taking the expected value of A, we obtain that the algorithm A is -competitive, a strict improvement on our previous findings. 1. Lower Bound for Randomized Algorithms Theorem 1.. No algorithm (even randomized) is better than e -competitive. e 1 Example: Let b = 10, and let algorithm A choose to buy at time t with probability p t. How can we determine an instance l that make A s cost high? Difficulty: We cannot find a single instance that stops exactly after the randomized algorithm buys. We could, instead, look at EA((b = 10, l)) and for each l we get to choose a bad instance. Unfortunately, E is not always easy to evaluate for more complex problems. 5
Main idea: Yao s MINIMAX principle gives reduction to deterministic case: Randomized algorithm on deterministic instance Deterministic algorithm on random instance A randomized algorithm can be thought of as A 1 A,... A, p 1 p,... where A i is a deterministic algorithm and A has probability p i of running algorithm A i. We want to show that there is no randomized algorithm that is better than e -competitive, e.g. there are no e 1 randomized algorithm that is 1.1-competitive, equivalently: For every randomized algorithm A, there exists I such that E > 1.1 j p ja j (I) > 1.1 A random instance I can be thought of as I 1 I,... I q 1 q,..., where I i is a deterministic instance and I has probability q i of instantiating to I i. Theorem 1.. (Yao s MINIMAX principle) For all randomized algorithm A and instance I as detailed above we have [ E ] max min I E [ ] I det A Where the on the left side we re running the randomized algorithm A over the worst fixed instance and right side we re running the best deterministic algorithm over a random instance. Observation 1. Under technical conditions the inequality becomes equality (non-trivial, uses, e.g., von Neumann s Minimax). Example: Back to the matrix example. Consider the random instance I 1 with probability 1 I = I with probability 6
The expected competitive ratio is a weighted average over ratio on the possible instances: E I = 1 A(I 1 ) OP T (I 1 ) + A(I ) OP T (I ) The table below shows the expected ratios of the deterministic algorithms for this random instance I. I A 1 A A A φ For this random instance, all deterministic algorithms have ratio at least. It is one bad random instance for all algorithms. Thus, the right-hand side of Yao s Principle is equal to, which then shows that our algorithm A = 1A 1 + A is optimal. Proof of Yao s Minimax Principle. max I E A E IE A (maximum average) E I E A = E AE I (exchanging sums) E A E I min j E I 5 min j E I A j (I) min A j (I) (average minimum) det A E I References [1] A. R. Karlin, M. S. Manasse, L. A. McGeoch, and S. Owicki. Competitive randomized algorithms for nonuniform problems. Algorithmica, 11(6):5 571, 199. 7