Introduction. Introduction. Heuristic Algorithms. Giovanni Righini. Università degli Studi di Milano Department of Computer Science (Crema)

Transcription

1 Introduction Heuristic Algorithms Giovanni Righini Università degli Studi di Milano Department of Computer Science (Crema)

2 Objectives The course aims at illustrating the main algorithmic techniques for the analysis and the design of effective and efficient heuristic algorithms for complex decision problems, especially combinatorial optimization problems. The course contributes to the Analytics and Optimization curriculum within the M.Sc. in Informatics and in Mathematics. The aim of this (new and original) curriculum is to provide knowledge on decision-making and optimization based on the use of data, models and algorithms. This currently goes under the name (Business) Analytics and it is one of the most required professional/educational profiles worldwide.

3 Course programme Introduction Basic concepts Classification Constructive algorithms Local search algorithms Approximation algorithms with guaranteed error bounds Math-heuristics

4 Prerequisites The course has some prerequisites: Computer programming Algorithms and data structures (Operations research) Students will be required to implement some of the algorithms and to do computational tests with them. For this purpose some lectures will be organized as lab sessions.

5 References There are many excellent books on heuristic and meta-heuristic algorithms. A classical reference is the book: E. Aarts, J.K. Lenstra eds., Local search in combinatorial optimization, Wiley 1997 (which is available in our dept. library), but there are also more recent references. Papers on specific topics will be made available through the course web site.

6 Heuristic algorithms The word heuristic comes from the greek word eurisko, which means I find. It has been used in the last century or so, to indicate a practical decision rule or a practical way to find a solution to a problem, relying upon experience and common sense. These rules do not aim at satisfying any formal or theoretical property; sometimes heuristics have been defined just as practical alternatives in contrast to formal mathematical techniques (=algorithms!).

7 Why using heuristics? Heuristic algorithms are used to solve large instances of computationally difficult problems, because the computation of an exact solution would require an excessive amount of computing time. It is usually purposeless to use heuristics for polynomially solvable problems, for small scale instances.

8 Heuristic algorithms for combinatorial optimization Heuristics can be applied to many different types of problems: optimization problems classification/recognition problems (machine learning) control problems etc... This course deals with heuristics for combinatorial optimization problems only, because: a huge amount of problems, arising in any type of industry, can be represented as c.o. problems; c.o. problems are in general very difficult; their combinatorial structure is a useful source of inspiration for the design of heuristics. It is useful to complement this course with the Combinatorial Optimization course.

9 Heuristics and meta-heuristics These algorithms can be roughly classified into two types: specific algorithms for specific problems Heuristics general ideas for almost any problem Meta-heuristics In general, special-purpose heuristic algorithms are more effective than general-purpose meta-heuristics. On the other hand, meta-heuristics are more easily applicable to a wide variety of different problems.

10 Not only heuristics Sometimes it may happen that the best way to compute good solutions to a combinatorial optimization problem is an exact optimization algorithm after suitable modifications that make it faster at the expense of the optimality guarantee. Any optimization algorithm can be turned into a heuristic algorithm. Therefore this course can be complemented with the Complements of O.R. course.

11 Remarks Remark 1: Heuristic algorithms is a dangerous field of study, because: any idea (good or bad) can be turned into a (new!) heuristic algorithm and publicized as such; it is not needed to work with mathematical models of combinatorial optimization problems; nobody expects theorems or theoretical results on heuristic algorithms. These are ideal conditions for junk science. One of the main educational goals of this course will be to develop a scientific approach to the design and the analysis of heuristic algorithms.

12 Remarks Remark 2: I like this kind of heuristics, This heuristics is new/fashionable are not scientifically sound reasons for using it. The (comparative) analysis of heuristics is an experimental discipline, similar to natural science disciplines. Remark 3: Heuristic algorithms design and analysis is a borderline field between O.R. and Informatics. In can be correctly labeled in both ways (although this is not always recognized). A correct approach to heuristic algorithms requires knowledge from both domains (which is not so common). Hence is it advisable to complement this course with the Operations Research course.

13 Remarks Remark 4: heuristic is not magic and less predictable is not more powerful. The temptation is to think that if the problem is so complex that we do not know how to solve it, than the algorithm must be so complex that we do not know what it does. Remark 5: understanding why a heuristic algorithm works is more important than showing that it is the best known algorithm for a specific set of instances.