Shows nodes and links, Node pair A-B, and a route between A and B.

Transcription

1 Solving Stochastic Optimization Problems on Computational Grids Steve Wright Argonne ational Laboratory University of Chicago University of Wisconsin-Madison Dundee, June 26, Outline Stochastic Programming (SP) Formulation and Basic Algorithms for SP Condor and metaeos Asynchronous Trust-Region Algorithm Computational Results: Algorithm Performance Sampling Methodology Computational Results: Quality of Solutions Joint work with Je Linderoth (Axioma Inc) and Alex Shapiro (Georgia Tech). Stochastic Programming Optimization of a model with uncertainty. Example: etwork Planning Adding capacity on a telecommunications network for private-line services. (Sen et al., 1994.) Often formulated mathematically as min x f(x) def = E g(x; ) = Z g(x; )p()d; (p is probability density function) subject to constraints on x 2 R n. A Shows nodes and links, ode pair A-B, and a route between A and B. B Arises in planning-under-uncertainty applications, where each represents a possible scenario (a possible way in which the model could evolve). Space can contain nite or innite scenarios. g(x; ) could be the value function of some second level optimization problem parametrized by x. (Recourse.) Add capacity to some links, to attempt to meet (uncertain) demand for trac between nodes. Sample demand prole: Third node pair: 0 (prob :855) 5:39 (prob :095) 75:1 (prob :05)

2 Data: - network topology: n = 89 links. - point-to-point pairs: i = 1; 2; : : : ; demands d i for each pair i are random and independent, with 3 to 7 possible scenarios. Total about scenarios! Decision variables: x j, j = 1; 2; : : : ; n: amount of capacity to add on link j. Total new capacity bounded by B. Objective: minimize the expected amount of unmet demand, summed over the m point-to-point pairs. We can't hope to solve the problem by accounting for all the possible scenarios exhaustively it's much too large. Practical approach is to use sampling to select a subset of scenarios, randomly. The sample average approximation (SAA) is still large, but manageable. 2-stage stochastic LP with recourse min x Q(x) = c T x + E P Q(x;!) subj. to Ax = b; x 0; where P is a probability measure on the space (; F), and Q(x;!) = min y q(!) T y subject to W y = h(!) T(!)x; y 0: x = rst-stage vars, y = second-stage vars. Sampled approximation: Sample points! j, j = 1; 2; : : : ; from P, and solve min x Q(x) = c T x + 1 P j=1 Q(x;! j ) subj. to Ax = b; x 0: Each Q(x;! j ) is convex, piecewise-linear in x. \bundle" methods Q(x) subgradients Build up a lower bounding, piecewise linear approximation to Q(x), based on function values Q(x`) and subgradients g` at iterates x`. Compute subgradients of Q by nding dual solutions j of the secondstage LP's for j = 1; 2; : : : ; (concurrently!) summing: Q(x;! j ) : min yj q(! j ) T y j ; subj. to W y = h(! j ) T(! j )x; y j 0; c 1 X T(! j ) T j : j=1 x Model function M k (x) after k iterates is h M k (x) = Q(x`) + (g`) T (x x`) i : sup `=0;1;:::;k Choose next iterate as x k+1 = arg min M k (x); subj. to Ax = b; x 0: which can be formulated as: min x; ; subject to Ax = b; x 0; Q(x`) + (g`) T (x x`); ` = 0; 1; : : : ; k: (Each constraint is called a cut.)

3 Example: After rst two iterations 0; 1: Q(x) enhancements x 0 x1 M (x) 1 x 2 is the minimizer of M 1 ; add new subgradient to obtain M 2 ; take minimizer to obtain x 3 : Q(x) x 2 x3 x M (x) 2 x trust-region (allows more steady progress, exploits good starting point); algorithm that allows deletion of old cuts; group the second-stage problems Q(x;! j ) into T \chunks" t, t = 1; 2; : : : ; T, with f1; 2; : : : ; g = [ t=1;2;:::;t t : and assign each t to a worker processor; multiple cuts at each x (each chunk can return its own subgradients); asynchronous variant is preferred for our target parallel platform. trust-region (TR) Choose next iterate as x k+ = arg min M k (x); subj. to Ax = b; x 0; kx x k k 1 k ; where k is the trust-region radius. Trivial to modify the LP subproblem: just add the bounds k e x x k k e: If candidate point x k+ is \signicantly better" (achieves some fraction of the decrease predicted by the model) then set x k+1 x k+. Possibly delete cuts, increase the trust region. Otherwise, set x k+1 x k and add subgradient information from x k+ to improve the model. Possibly delete uninteresting cuts, decrease trust region. TR properties Denoting the solution set by S, Can delete cuts liberally, between major iterations; dist(x k ; S)! 0; The algorithm may still be too synchronous: requires complete evaluation of Q(x) at a candidate iterate x before proceeding.

4 related work Condor! Marsten et al \box step" (1975) builds up an exact model of the problem over the TR. Lemarechal \bundle" (1975 et seq.) Kiwiel (1983 et seq.) Ruszczynski \regularized decomposition" (1986) specically for stochastic programming. The last three give quadratic programming subproblems, not LP. More like a Levenberg approach than a trust-region approach. High-throughput computing on pools of distributively owned computers Developed at Wisconsin: Livny. Condor pools consist of user workstations, nodes from multiprocessor systems and clusters. Handles scheduling, matching of user requirements to machine characteristics. Checkpointing and migration. Flocking and Glide-in mechanisms allow jobs to execute across multiple pools. a challenging environment... The Condor environment is powerful and inexpensive, but challenging to algorithm designers and implementers. dynamic/opportunistic: size and composition of worker pool changes unpredictably during computation heterogeneous: old workstations and fast new linux machines, dierent versions of Solaris, software licenses valid on only some machines. latency unpredictable, generally slow: workers can be next to each other in a rack, or separated by 6000 miles and the Internet. Problems that are large and compute-intensive and algorithms that are asynchronous work best on this platform. MW (Master-Worker) Many interesting optimization algorithms can be shoehorned into the master-worker paradigm. MW is a runtime support library for implementing master-worker computations on the Condor system. (Yoder, Kulkarni, Linderoth, Goux) MW abstracts the issues of resource management and communication. Condor handles resource management; Communication is either via shared les or Condor-PVM.

5 TR may not be asynchronous enough! The TR approach still synchronizes on the function evaluation at each candidate point x k+. If there are T chunks of second-stage scenarios, can't use more than T processors. For many problems of interest, we cannot make T very large (10 { 100) without making the work-perchunk too small and creating too much contention at the master. May wait for a long time for the last chunk to be evaluated, if its host is suspended or disappears. An asynchronous trust-region () algorithm increases parallelism and throughput. Maintain an incumbent x I : the best point found so far (smallest value of Q). Maintain a basket B of 3 20 other x points possible new incumbents for which the second-stage LPs are currently being solved. When space becomes available in B, generate a new candidate point by solving a TR subproblem around the current incumbent: kx x I k 1. (x I becomes the parent incumbent of the new point.) (continued) convergence of When evaluation of a point x 2 B is completed, accept it as the new incumbent if Q(x) < Q(x I ); and Q(x) gives a signicant decrease over its parent incumbent. Populate B initially by solving TR subproblems around early incumbents, using partial subgradient information. (Synchronicity parameter.) Strategies for cut deletion and adjustment of trust region are adapted from the strategies for the synchronous TR algorithm. Because of the strategy for selecting incumbents, and for cut management and adjustment of, can re-use much of the theory for the serial case. From a given incumbent, can trace back a chain through successive parents, to the initial point. A sucient decrease condition is satised with respect to each link in this chain. By applying synchronous TR theory to this chain, and assuming that all tasks terminate nitely, we have dist(x I ; S)! 0.

6 SS, = 10; M 7.06M = 100; M 70.6M. SS, ALS TR TR TR TR TR TR = M 7.06M. SS, SS: computational results rst stage: 89 variables, 1 constraint; second stage: 706 variables, 175 constraints, 2284 nonzeros. Study the eect of asynchronicity, parallelism on large sampled instances. We report results for = 10 4 and = 10 5 scenarios, with synch parameter = :7. par. eciency cuts/iter procs av. iter chunks run wall clock (min) run iter jbj chunks cuts/iter procs av. eciency par. clock (min) wall run iter jbj chunks wall clock (min) cuts/iter procs av. par. eciency

7 TR = Columns: 1: storm, Solved second-stage linear programs during the run (3472 per second). Average task 774 seconds. Total computation time 9014 hours (more than one year). storm, = 250; M 315M. storm: computational results wall clock (min) Cargo ight scheduling problem (Mulvey and Ruszczynski). rst stage: 121 variables; second stage: 1259 variables. For a scenario sampled approx, LP has size 132; 000; ; 750; 121 Started from a solution for a 3000-scenario approximation, whose quality is very good. TR takes a single step and terminates, doesn't take any steps, just veries quality of starting point. chunks cuts/iter procs av. eciency par. iter jbj run (For a chunk of 2000 scenarios, task size is about 150 seconds.) storm with 10 7 scenarios LP has size approximately 5: rows; 1: columns: Used machines at Wisconsin, CSA (Illinois), ew Mexico, Argonne, Italy, Columbia. 800 machines requested, 556 actually used during the run (average of 433 at any one time). #workers Sec. wall clock (hrs) procs av. eciency par. cuts/iter chunks iter jbj run

8 notation reminder solution quality Can we get useful estimates for the optimal objective values of the true problem from the sampled problem? Can we get condence intervals? How do the solutions of the sampled approximation relate to those of the real problem? Using, along with relevant theory (some recent), we have performed computational and statistical studies of these issues for some dif- cult problems from the literature. min Q(x) = X Ax=b;x0 ct x + K p i Q(x;! i ); i=1 (where K 10 70, say) while sample average approximation (SAA) can be formed by sampling points f! 1 ;! 2 ; : : : ;! g from the distribution P and solving X min Q (x) = c T x + 1 Q(x;! j ): Ax=b;x0 Denote j=1 Z = min Q(x); Z = min Q (x): Ax=b;x0 Ax=b;x0 Can use Monte Carlo sampling (sampling with replacement). Can also use variance reduction techniques to reduce Var(Z ). We implemented Latin hypercube sampling (sampling without replacement). lower bound for Z condence interval for lower bound It's well known that EZ Z : This is true for any unbiased estimator. In particular can use certain variance reduction techniques (e.g. Latin hypercube) to select the sample f! 1 ;! 2 ; : : : ;! g. Generate M batches each a sampled approximation of size of the form f! (i) 1 ;!(i) i = 1; 2; : : : ; M and solve the M SAAs to obtain optimal values Z (1) Then estimate EZ by ; Z(2) ; : : : ; Z(M) : L M = M 1 M X i=1 Z (i) : g, 2 ; : : : ;!(i) Have in the limit that p M [LM EZ ] (0; 2 L) where 2 L = VarZ : Can approximate L 2 estimator s L (M) 2 = 1 MX M 1 i=1 Then dening z such that by the sample variance 2 Z (i) L M P f(0; 1) z g = 1 ; our estimate of the width of the (1 2) con- dence interval is z s L (M)= p M: (For = :025 have z 1:96.)

9 application scenarios stage 1 stage 2 name hydro power LandS 6: gbd cargo ights storm vehicle assignment 1: term network design ssn upper bound for Z Given any feasible point ^x, we have Q(^x) Q(x ): Choose an ^x that appears to be nearly optimal, e.g. minimizer of some Q. Choose T i.i.d. samples, each of size (using Monte Carlo or Latin Hypercube): Dening f! (i) 1 ;!(i) 2 ; : : : ;!(i) g; i = 1; 2; : : : ; T; bq (i) (^x) = ct ^x + 1 X j=1 we get an unbiased estimator: U ;T = T 1 T X i=1 Q(^x;! (i) j ): bq (i) (^x): condence interval for upper bound Since the batches are i.i.d. we have p T h U ;T Q(^x) i (0; 2 U) where U 2 = Var Q b (^x). Approximate U 2 by sample variance estimator s U (T) 2 = 1 TX bq (i) T 1 (^x) 2 ;T U : i=1 Get condence interval width z s U (T)= p T : test problems Performed experiments with ve problems from the literature. Solved SAA's for sample sizes ranging from 50 to Solved between 9 and 12 SAAs (M) for each value of. In upper-bound evaluation, for each optimizing ^x from the SAAs, T = 50 and = 20; 000. Report the value of ^x for which the estimate U ;T is lowest, together with its condence interval. In selecting samples of size (for SAA) and (for evaluation), used both Monte Carlo and Latin Hypercube distribution, as reported in the tables.

10 results on bounds: SS Results of Mak et al (1999). lower upper batch/sample estimate % condence 0:21 0:11 Using dierent techniques, Mak et al, generate an approximate solution ^x using = 2000, and obtain upper bound 10:06 0:12 (95% condence interval); with 95% likelihood, the optimal Z is within 0:77 of this value. solution estimates for SS 95% condence intervals. Monte Carlo: Lower Upper 50 4:11 1:23 12:88 0: :66 1:31 11:31 0: :54 0:34 10:42 0: :31 0:23 10:20 0: :98 0:21 10:01 0:09 Latin Hypercube: Lower Upper 50 10:10 0:81 11:39 0: :90 0:36 10:52 0: :87 0:22 10:05 0: :83 0:29 9:97 0: :84 0:10 9:90 0: SS Monte Carlo SS Latin Hypercube

11 solution estimates for gbd 95% condence intervals. Monte Carlo: Lower Upper :62 66: :33 1: :24 42: :13 2: :66 13: :42 2: :50 12: :35 4: :13 4: :81 3:56 Latin Hypercube: Lower Upper :21 10: :62 0: :62 0: :62 0: :62 0: :62 0: :62 0: :62 0: :62 0: :62 0: gbd Monte Carlo gbd Latin Hypercube solution estimates for storm 95% condence intervals. Monte Carlo: Lower Upper : : :00 964: : : : : :58 415: : : :42 484: : : :67 188: : :66 Latin Hypercube: Lower Upper :90 107: :00 17: :90 101: :00 20: :60 28: :00 22: :20 14: :00 17: :10 7: :00 15:04

12 1.552e e+06 Storm Monte Carlo 1.55e e e e e e e e+06 Storm Latin Hypercube 1.551e e e e e e e+06 solution estimates for 20term 95% condence intervals. Monte Carlo: Lower Upper :33 944: :00 41: :89 800: :00 40: :33 194: :00 36: :22 234: :00 46: :78 85: :00 51:05 Latin Hypercube: Lower Upper :57 371: :00 4: :00 252: :00 5: :43 117: :00 5: :00 95: :00 5: :57 38: :00 5:80 20term Monte Carlo

13 term Latin Hypercube LandS Monte Carlo LandS Latin Hypercube conditioning and exact solutions Recent results (Shapiro and Homem-de-Mello, 2000) indicate that a discrete SAA can identify the exact solution of a stochastic linear program over a discrete probablity space. If solution is unique, chance of identifying it exactly approaches 1 exponentially rapidly in : P(^x = x ) 1 e ; some > 0: We investigated the solutions obtained for the nest SAAs (=5000) for each problem instance, using Latin Hypercube sampling. We plotted distance matrices for six SAA solutions for each problem.

14 2.49e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e distance of gbd solutions Sharp, well dened minimizer. Solutions are far apart, though their objective values appear to be similar. Indicates a shallow minimum distance of SS solutions distance of 20term solutions distance of storm solutions Also a well dened minimizer.

15 Latin hypercube sampling Aim to reduce variance in Z. Example: Scenario space! 1! 2! 3, where each! is distributed according to: P(! = A) = :5; P(! = B) = :25; P(! = C) = :25: A B C The following slides will not be used in this talk but are retained in the le for reference. ω 1 ω 2 ω 3 Sample size = 4. Divide [0; 1] into 4 intervals, allow exactly one sample in each interval for each random variable ! 1 B C A A! 2 A A B C! 3 A C A B We use Latin hypercube sampling, larger sample sizes. Lower bound: 39 batches of size = Obtained L M = 9:9163; standard error = :0273: 95% condence interval is [9:8610; 9:9716]. Upper bound: For each of the SA solutions ^x obtained from the lower bound calculation, took 21 batches of size P = 20; 000. Used these to estimate U P (^x) for each ^x, together with its std error. For the \best" ^x, obtained 9:9001; standard error :0190: 95% condence interval is [9:8614; 9:9397]. Suggests strongly that the optimal Z is close to 9:91.