IEOR E4008: Computational Discrete Optimization

Transcription

1 Yuri Faenza IEOR Department Jan 23th, 2018

2 Logistics Instructor: Yuri Faenza Assistant IEOR from 2016 Research area: Discrete Optimization Schedule: MW, 10:10-11:25 Room: 303 Mudd Office Hours: M, 5:30-7pm or by appointment Mudd 334

3 Website Course website on Courseworks: Slides, lecture notes, links to further material, I ll (try to) upload the slides of each lecture before class starts, so you can print them if you want to Piazza: Online discussions on topics of the class

4 Grading 30%: Assignments, roughly one every two weeks 10%: Class participation 60%: Project - I will list some projects ~ - you can come up with idea for c project, and talk to me

5 Examples of final projects How much would we save if all taxis were replaced by car sharing? How to solve discrete optimization problems with algorithms inspired by statistical physics and genetics How to summarize the content of documents using machine learning and submodular functions How can we visit all streets in a neighborhood as quickly as possible? How can we build index funds using algorithms on graphs?

6 Examples of assignments In an image, each pixel is represented by a 3-dimensional vector in the RGB space, ie each component represents the amount of one of the three primary colors (red, green blue) that appears in the pixel Each component has value between 0 and 256 Adapt one of the algorithms seen in class to the following problem You are given an image as a set of pixels in the RGB space, and an integer k Reducethesizeofthepicturebyrepresentingitusingatmostk colors

7 Image compression via clustering: an example Original image, 302KB 2colors,13KB 16 colors, 44KB 32 colors, 73KB IEORE4004: Optimization Models and Methods

8 What is this course about? Discrete optimization: choosethesolutionofmaximumprofit(orminimum cost) from a discrete family Some discrete optimization problems you have probably seen in basic classes: Many problems on graphs (eg shortest path) : " in

9 What is this course about? Discrete optimization: choosethesolutionofmaximumprofit(orminimum cost) from a discrete family Some discrete optimization problems you have probably seen in basic classes: Many problems on graphs (eg shortest path) Integer Programming f Linear min cx Ax b x Z n programming integer I program

10 Why a class focused on Discrete Optimization? Many applications: production planning network design biomedicine DNA assembling machine learning In general, very hard to solve not a unique approach

11 - - tire Example: online ads Bis 25$ Where does the 97% of Google revenue come from 1? Advertising! Google Adwords: Company i chooses: we car di cci = race cci amonthlybudgetb i ; for each word w acostc(i, w) that the company is willing to pay to have its ad shown when word w is searched on Google,,w ) -_ w ) = 70$ s $,w ) = 3$ When w is searched, Google chooses a set of ads to display If the user clicks on the ad of company i, Google gets c(i, w) from the company (unless the company has finished its budget B i ) 1 Source:

12 Which word produces the biggest revenue 2? Which is the most expensive 3? 2 Source: 3 Source:

13 The Adwords problem Who should we assign each word to? WORDS TO Y OTA CAR BUDGET RACE G- H

14 agg The Adwords problem CAR 3 RACE WORDS 4 [ I f 4O 13 ) Bugg,, 3f) 1[ I I GM Who should we assign each word to? 7) CAR a- ti RACE 8 TOYOTA 2X CAR 2 x RACE STRATEGY :/GRCED WHO STILL HAS G SOME BUDGET LEFT 8 -, q, µ, won, I TO THE HIGHEST BIDDER can TOYOTA G TOYOTA 9 RACE c- n - - I a I 10

15 The Adwords problem WORDS CAR RACE I L TOYOTA 8 2x CAR 2x RACE TSU yy3 ) BUDGET CAR TOYOTA can Gn 3 [ I [ gu RACE TOYOTA 3 Who should we assign each word to? RACE g- n y - II

16 The Adwords problem Who should we assign each word to? What if arrivals are random? Dr Balasubramanian Sivan, Google Research

17 The Stable Marriage problem A marriage is stable if m, w : m prefers w to his partner w a m " hav x, m,w as above is called a " blocking pain, -

18 The Stable Marriage problem my W1 is a blocking pain I not a stable matching

19 The Stable Marriage problem NO BLOCKING PAIR STABLE MATCHING

20 The Stable Marriage problem Applications include: Allocations of students to high schools in NYC, of doctors to hospitals, of kidney donors to patients,

21 A mathematical challenge From a Procter & Gamble ad from 1962: Imagine that Toody and Muldoon want to drive around the country and visit each of the 33 locations represented by dots on the contest map, and that in doing so, they want to travel the shortest possible route You should plan a route for them from location to location which will result in the shortest total mileage from Chicago, Illinois back to Chicago, Illinois TO $ in 1962 correspond to $ in today s value

22 TSP and applications Travelling Salesman Problem (TSP): find a tour of minimum length Today s algorithms can solve problems with tens of thousands of cities, and there are still open challenges Applications: Pick-up & delivery, Guiding industrial machines, Mapping genomes, FREE App SOLVING TSP INSTANCES : CONCORDE

23 Discrete Optimization for Machine Learning i - "shhgreturj Ip,? ' " ',p4 Let f ( S ) = AREA COVERED BY ACTIVATING seasons IN POSITION S f ( sit ) + fcsntsfcs ) + FCT)

24 Discrete Optimization for Machine Learning Those functions have diminishing returns and are called Submodular Functions

25 What is this course about? 1: Problems and Applications Minimum spanning tree, steiner minimum tree, bin packing Network design and beyond TSP: approximation algorithms, heuristics, Integer Programming and branch and bound Extensions: vehicle routing problems Matching problems, stable marriage problem Applications to the New York school system and to the hospital-intern problem Beyond stability Online matching and online advertisement: adwords and secretary problems Submodular functions and their applications in machine learning: MAP inference, document summarization, influence in social networks

26 What is this course about? 2: Algorithms For most of those problems, we will see one or more algorithms for their solutions Why? Often, those algorithms contain ideas that are problem-independent and can be reused for your favourite application How well we want to solve the problem depends on the application

27 What is this course about? 3: Implementations We will test our algorithm in practice using: Python, Gurobi, and their interaction

28 Do we really need to learn all this? Objection 1: There are only a finite number of solutions to these problems Idon tneedasmartalgorithm,ihave asupercomputer! Objection 2: I ll buy an expensive solver and let it do the job!

29 Do we really need to learn all this? Objection 1: There are only a finite number of solutions to these problems Idon tneedasmartalgorithm,ihave asupercomputer! Objection 2: I ll buy an expensive solver and let it do the job! Let s prove those objections wrong!

30 Number of solutions to TSP vs very large sets cities tours Number of cells in a human body: Number of particles in the visible universe: (, n cities # feasible solutions roughly n!

31 in Graphs, trees Graph G(V, E) In is, } n=s edges lk#hkvslkind oh 4floss of J 3

32 Graphs, trees Graph G(V, E) Degree OY degrees

33 Graphs, trees Graph G(V, E) Degree Path o - o - o - O - - o repetition allowed ( of either nodes or edges )

34 Graphs, trees Graph G(V, E) Degree Path / as a Walk (open and closed) ' path ", but repetitions allowed first node #, last node " " first nodes last node -1 open walk closed walk

35 Graphs, trees Graph G(V, E) Degree Path Walk (open and closed) Cycle / % every path is an open walk every cycle is a closed walk

36 Graphs, trees TREE Graph G(V, E) Degree Path Walk (open and closed) Cycle O Tree I NOT A TREE ( 7 cycle ) CONNECTED GRAPH O WITH NO CYCLE A O TREE o ( NOT CONNECTED ) O o

37 Graphs, trees Graph G(V, E) 2 Degree 3 Path Walk (open and closed) Cycle I 4 25 I 3 Tree Weighted graph! weight can be on the edges or on the nodes C or both)

38 Graphs, trees O Graph G(V, E) Degree O O O Path Walk (open and closed) Cycle Tree Weighted graph Subgraph s a subgraph O O of O O [ We Cdn choose any and edges ] selection of nodes

39 Graphs, trees O Vl O 3 Graph G(V, E) Degree Path Walk (open and closed) Cycle Tree Weighted graph Subgraph Induced Subgraph Go ok ou s the subgraph of VI O o 's " f induced by VI Vz, Vs Va choose we a subset of vertices and, take edges between them ]

40 Graphs, trees Graph G(V, E) Degree Path Walk (open and closed) Cycle Tree Weighted graph Subgraph Induced Subgraph Multigraph c) the same edge can :O: Xan than once

41 TSP: a formulation The Travelling Salesman 1 Problem y (TSP): cities 00 Given: A graph G(V, E) withweightsw 0ontheedges Find: a tour ieacycle passing through all vertices of the graph of minimum total weight O between roads cities 1 5 t I tht 6 = t h t 3 t s = 14 the 16 we prefer 2

42 Greedy algorithm for TSP Input: AgraphG(V, E); w : E R 0 L =[0] For i =0,,n 2: Pick j V \ L minimizing w(l[i], j) Set L[i +1]=j ± Output L 1 IT MAY NOT GIVE You THG OPTIMUM 2 IT HAV Bf FAL AWAY But From Ttb OPTIMUM IT IS FAST AND EASY To IMPLEMENT

43 Pros and cons of Greedy Pros: Fast Intuitively, seems to do the right thing Cons: We have no idea how far it is from the optimum