Randomized rounding of semidefinite programs and primal-dual method for integer linear programming. Reza Moosavi Dr. Saeedeh Parsaeefard Dec.

Transcription

1 Randomized rounding of semidefinite programs and primal-dual method for integer linear programming Dr. Saeedeh Parsaeefard

2 Semidefinite Programming ()

3 1

4 Integer Programming integer programming problem Integer variables NP-hard Linear Non linear 4

5 Integer Programming Heuristic Approach Metaheuristic Approach Simulated Annealing Genetic algorithms Tabu search No guarantee for optimality of solution Low complexity Approximation algorithms Guarantee a maximum distance from optimal solution Low complexity Strong math base 5

6 Approximation Algorithms α approximation algorithm Minimization problem Solution α. OPT for α > 1 Maximization problem Solution α. OPT for α < 1 6

7 Approximation Algorithms Rounding data and dynamic programming Deterministic rounding of linear programming Random sampling and randomized rounding of linear programs Randomized rounding of semidefinite programming Simple, Good bound The primal-dual method Combinatorial problem, Online algorithms, prove of bound 7

8 2 Semidefinite Programming

9 Semidefinite Programming Semidefinite programming uses symmetric, positive semidefinite matrices Definition A matrix X ज n n is positive semidefinite if and only if for all y R n, y T Xy 0. X ज n n is symmetric, then the following statements are equivalent: 1. X is positive semidefinite; 2. X has non-negative eigenvalues; 3. X = V T V for some V ज m n where m n; 4. X = i=1 λ i ω i ω T i for some λ i 1&ω i ω T j 0 for i j. 0 and vectors w i R n such that ω i ω T i = 9

10 Semidefinite Programming Maximize (Minimize) i,j c ij x ij Subject to: i,j a ijk x ij = b k, k, x ij = x ji, i, j, X = x ij 0. 10

11 Semidefinite Programming Vector programming If x ij = v i T v j = v i v j, then X = V T V. Maximize (Minimize) i,j c ij (v i. v j ) Subject to: i,j a ijk (v i. v j ) = b k, k, v i R n, i = 1,, n. 11

12 MAX CUT G = (V, E) w ij 0 for each edge (i, j) Partition vertex set into two parts, U and W = V U Maximize the weight of edges whose two endpoints are in different parts U = {i: y i = 1}, W = {i: y i = 1} Maximize 1/2 (i,j) E w ij (1 y i. y j ) Subject to: y i { 1, 1} i = 1,, n, 12

13 MAX CUT OPT is the value of the optimal solution of maximum cut problem Vector programming Optimal value Z VP If v i = y i, 0,0,, 0, then v i. v i = 1 & v i. v j = y i y j. Z VP OPT Maximize 1/2 (i,j) E w ij (1 v i. v j ) Subject to: v i. v i = 1, i = 1,, n, v i R n, i = 1,, n. 13

14 MAX CUT The Algorithm 1. Pick a random vector r = (r 1,, r n ), each component with distribution N(0,1) 2. Set unit vector v i =(0,,0,1,0,,0), 3. Put i U if v i. r 0 and i W otherwise. 14

15 MAX CUT Another way of looking at the algorithm 1. All vectors v i lie on the unit sphere, since v i. v i = 1 and they are unit vectors. 2. The hyperplane with normal r containing the origin splits the sphere in half 3. vertices in one half (the half such that v i. r 0 ) are put into U, and all vertices in the other half are put into W 15

16 MAX CUT Theorem Rounding the vector programming by choosing a random hyperplane is a α =0.878-approximation algorithm for the maximum cut problem. M. X. Goemans and D. P. Williamson. Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM, 42: ,

17 Quadratic Programming We restrict the objective matrix A 0, because objective is not negative Optimal value OPT Maximize 1 i,j n a ij x i x j Subject to: x i 1, +1, i = 1,, n, A = a ij 0 17

18 Quadratic Programming Vector programming Optimal value Z VP Maximize 1 i,j n a ij ( v i. v j ) Subject to: v i. v i = 1, i = 1,, n, v i R n, i = 1,, n. 18

19 Quadratic Programming The Algorithm 1. choose a random hyperplane with normal r. 2. Set x i = 1 if v i. r 0 and x i = 1 otherwise. Theorem This approach gives an α = 2 -approximation algorithm for the quadratic π programming problem Nesterov, Y., Semidefinite relaxation and nonconvex quadratic optimization. Optimization methods and software, (1-3): p

20 Correlation Clustering Undirected graph G = (V, E) Each edge i, j E has two nonnegative weight w ij + 0 and w ij 0. Goal is to cluster the vertices into several sets to maximize total w ij + of the nodes in similar cluster plus w ij of nodes in different clusters 20

21 Correlation Clustering Partition S is clustering of vertices δ(s) is set of edges with endpoints in different sets E(S) is set of edges with both endpoints in same set Find S to maximize i,j E(S) w ij + + i,j δ(s) w ij Let e k be kth unit vector, where its kth element is equal to one and zero elsewhere. x i = e i if i is in kth cluster Maximize (i,j) E (w ij + x i. x j + w ij (1 x i. x j )) Subject to: x i e 1, e 2,, e n i. 21

22 Correlation Clustering Vector programming Optimal value Z CC OPT Maximize (i,j) E (w ij + v i. v j + w ij (1 v.i. v j )) Subject to: v i. v i = 1, i. v i. v j 0, i, j v i R n, i. 22

23 Correlation Clustering The Algorithm 1. we choose two random hyperplane, with two independent vectors r 1 and r This approach partitions the vertices into 4 sets. R 1 = {i V: r 1. v i 0, r 2. v i 0} R 2 = {i V: r 1. v i 0, r 2. v i < 0} R 3 = {i V: r 1. v i < 0, r 2. v i 0} R 4 = {i V: r 1. v i < 0, r 2. v i < 0} 23

24 Correlation Clustering Theorem The approach gets a α = 3 -approximation algorithm 4 Swamy, C. Correlation clustering: maximizing agreements via semidefinite programming. in Proceedings of the fifteenth annual ACM- SIAM symposium on Discrete algorithms Society for Industrial and Applied Mathematics. 24

25 Paper Example 1 multiple multicast sessions Several overloaded link N alternative path 25

26 Paper Example 1 Semidefinite programming Prove 2-approximation min x M m 1 n 1 N s. t. x 1 m M n 1 M m 1 N mn mn mn m mn pn mn r x B x C n N x 0,1 S. Q. Zhang, Q. Zhang, H. Bannazadeh, and A. Leon-Garcia, Routing algorithms for network function virtualization enabled multicast topology on SDN, IEEE Transactions on Network and Service Management, vol. 12, pp , Dec

27 Paper Example 1 27

28 Paper Example 1 28

29 Paper Example 2 For correlative clustering If small cells i, j in a cluster x i,j = 1, x i,j = 0 otherwise No frequency reuse (decrease sum rate) Channel gain (Interference) F: set of femtocells M: maximum cluster size 29

30 Paper Example 2 max w x w (1 x ) x i, j i F j F s. t. x 1, i F i, i i, j j, k i, k i, j i, j i, j i, j i, j n i, j x x x 1, i, j, k F, k>i, j i,k j F x M, i F x 0, i, j F X=(x ) 0. Obtain a solution with performance Branch and Bound A. Abdelnasser, E. Hossain and D. I. Kim, Clustering and resource allocation for dense femtocells in a two-tier cellular OFDMA network, IEEE Trans. Wireless Commun., vol. 13, no. 3, pp , Mar

31 Paper Example 2 31

32 Paper Example 2 32

33 3

34 dual method Integer Programming Relax variable Find dual problem Find feasible solution of integer programming based dual problem 34

35 dual method Instead of actually solving the dual LP, we can construct a feasible dual solution with the same properties. According to dual problem, obtain primal solution Obtain a bound for solution dual method is also used in combinatorial optimization and online algorithms 35

36 dual method linear programming 36

37 Set Cover Problem Input a ground set of elements E = e 1,..., e n. Some subsets of those elements S 1, S 2,..., S m E. Nonnegative weight w j for each subset S j. The goal is to find a minimum-weight collection of subsets that covers all of E j=1 m Minimize w j. x j Subject to: j:ei S j x j 1, i = 1,, n, x j {0,1}, j = 1,, m. 37

38 Set Cover Problem Relax integer programming to linear programming Obtain dual problem i=1 n Maximize y i Subject to: i:ei S j y i w j, j = 1,, m, y j 0, i = 1,, n. 38

39 Set Cover Problem The Algorithm R. Bar-Yehuda and S. Even. A linear time approximation algorithm for the weighted vertex cover problem. Journal of Algorithms, 2: ,

40 Set Cover Problem Theorem The above algorithm is an f-approximation algorithm for the set cover problem. f = max {j: e i S j } i R. Bar-Yehuda and S. Even. A linear time approximation algorithm for the weighted vertex cover problem. Journal of Algorithms, 2: ,

41 The feedback vertex set Undirected graph G = (V, E) Weights w i for vertices i V The goal is to choose a minimum-cost subset of vertices S V such that every cycle C in the graph contains some vertex of S. Find a minimum-cost set of vertices S such that the induced graph G[V S] is acyclic. Minimize i V w i. x i Subject to: i C x i 1, C C, x i {0,1}, i V. 41

42 The feedback vertex set Relax integer programming to linear programming Obtain dual problem Maximize C C y C Subject to: C C:i C y C w i, i V, y C 0, C C. 42

43 The feedback vertex set The Algorithm R. Bar-Yehuda, D. Geiger, J. Naor, and R. M. Roth. Approximation algorithms for the feedback vertex set problem with applications to constraint satisfaction and Bayesian inference. SIAM Journal on Computing, 27: ,

44 The feedback vertex set Theorem the Algorithm is an (4 log 2 n )-approximation algorithm for the feedback vertex set problem. R. Bar-Yehuda, D. Geiger, J. Naor, and R. M. Roth. Approximation algorithms for the feedback vertex set problem with applications to constraint satisfaction and Bayesian inference. SIAM Journal on Computing, 27: ,

45 Shortest Path Undirected graph G = (V, E) Costs c e for edges e E Find minimum-cost path from s to d S set of all s t cuts δ S set of all edges that one endpoint in S and the other endpoint not in S. Minimize e E c e. x e Subject to: e δ(s) x e 1, S S, x e {0,1}, e E. 45

46 Shortest Path Relax integer programming to linear programming Obtain dual problem Maximize S S y S Subject to: S S:e δ(s) y S c e, e E, y S 0, S S. 46

47 Shortest Path The Algorithm *F δ C = A. Hoffman. On simple combinatorial optimization problems. Discrete Mathematics, 106/107: ,

48 Shortest Path Theorem The Algorithm finds a shortest path from s to t. A. Hoffman. On simple combinatorial optimization problems. Discrete Mathematics, 106/107: ,

49 4

50 Randomized rounding of semidefinite programs for integer programming primal-dual method for integer programming Guarantee a maximum distance from optimal solution Low complexity Strong math base Apply to some problem 50

51 References [1] Williamson, D.P. and D.B. Shmoys, The design of approximation algorithms. 2011: Cambridge university press. [2] Strang, G., et al., to linear algebra. Vol : Wellesley-Cambridge Press Wellesley, MA. [3] Wolkowicz, H., R. Saigal, and L.Vandenberghe, Handbook of semidefinite programming: theory, algorithms, and applications. Vol : Springer Science & Business Media. [4] Goemans, M.X. and D.P. Williamson, Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM (JACM), (6): p [5] Fernandez De La Vega, W., MAX CUT has a randomized approximation scheme in dense graphs. Random Structures & Algorithms, (3): p [6] Fotakis, D., A primal-dual algorithm for online non-uniform facility location. Journal of Discrete Algorithms, (1): p [7] Nesterov, Y., Semidefinite relaxation and nonconvex quadratic optimization. Optimization methods and software, (1-3): p [8] Swamy, C. Correlation clustering: maximizing agreements via semidefinite programming. in Proceedings of the fifteenth annual ACM-SIAM symposium on Discrete algorithms Society for Industrial and Applied Mathematics. [9] Williamson, D.P., On the design of approximation algorithms for a class of graph problems. 1993, Massachusetts Institute of Technology. [10] Karger, D., R. Motwani, and M. Sudan. Approximate graph coloring by semidefinite programming. in Foundations of Computer Science, 1994 Proceedings., 35th Annual Symposium on IEEE. [11] Zhang, S.Q., et al., Routing algorithms for network function virtualization enabled multicast topology on SDN. IEEE transactions on Network and Service Management, (4): p

52 References [12] Abdelnasser, A., E. Hossain, and D.I. Kim, Clustering and resource allocation for dense femtocells in a two-tier cellular OFDMA network. IEEE Transactions on Wireless Communications, (3): p [13] R. Bar-Yehuda and S. Even. A linear time approximation algorithm for the weighted vertex cover problem. Journal of Algorithms, 2: , [14] R. Bar-Yehuda, D. Geiger, J. Naor, and R. M. Roth. Approximation algorithms for the feedback vertex set problem with applications to constraint satisfaction and Bayesian inference. SIAM Journal on Computing, 27: , [15] A. Hoffman. On simple combinatorial optimization problems. Discrete Mathematics, 106/107: ,

53 Thank You For Your Attention