Artificial Intelligence

Artificial Intelligence Informed Search and Exploration Chapter 4 (4.3 4.6) Searching: So Far We ve discussed how to build goal-based and utility-based agents that search to solve problems We ve also presented both uninformed (or blind) and informed (or heuristic) approaches for search What we ve covered so far explore search space systematically, which could enumerate the entire state space before finding a solution Last Time: Search Strategies This Time Uninformed: Use only information available in the problem formulation Breadth-first Uniform-cost Depth-first Depth-limited Iterative deepening Local search algorithms Hill climbing Simulated annealing Genetic algorithm Informed: Use heuristics to guide the search Best first: Greedy search A* search Local Search Algorithms Optimization Other search strategies, can use local search algorithms: keep a single current state, and try to improve it, in order to solve the problem at hand The path is irrelevant; the goal state itself is the solution. We do not necessarily have a designated start state The objective is to search through the problem space to find other solutions that are better, the best, or that meet certain criteria (goal) Problems where we search through complete solutions to find the best solution are often referred to as optimization problems Most optimization tasks belong to a class of computational problems called NP Non-deterministic Polynomial time solvable For NP problems, state spaces are usually exponential, so systematic search methods are not time or space efficient 1

Optimization Problems Optimization Problems As it turns out, many real-world problems that we might want an agent to solve are similarly hard optimization problems: Bin-packing - The goal is to pack a collection of objects into the minimum number of fixed-size "bins" Logistics planning VLSI layout/circuit design Theorem-proving Navigation/routing Production scheduling Learning the parameters for a neural network For optimization problems, there is a well-defined objective function that we are trying to optimize In addition to finding goals, local search algorithms are useful for solving pure optimization problems, in which the aim is to find the best state according to an objective function Local Search Local search Local search is a type of greedy, complete search that focuses on a specific (or local) part of the search space, rather than trying to branch out into all of it We only consider the neighborhood of the current state rather than the entire state space (multiple paths) Consider the state space landscape A landscape has both "location" (defined by the state) and "elevation" (defined by the value of the heuristic cost function or objective function) If elevation corresponds to cost, then the aim is to find the lowest valley a global minimum If elevation corresponds to an objective function, then the aim is to find the highest peak a global maximum (You can convert from one to the other just by inserting a minus sign.) Local search Hill-Climbing (HC) Local search algorithms explore this landscape A complete local search algorithm always finds a goal if one exists An optimal algorithm always finds a global minimum/maximum The most common local search strategy is called hillclimbing, if the task is to maximize the objective function Called gradient ascent if we are maximizing else Called gradient descent if we are minimizing We consider all the successors of the current node, expand the best one, and throw the rest away 2

Hill-Climbing Hill-Climbing The hill-climbing search algorithm is simply a loop that continually moves in the direction of increasing value that is, uphill It terminates when it reaches a "peak" where no neighbor has a higher value The algorithm does not maintain a search tree, so the current node data structure need only record the state and its objective function value Hill-climbing does not look ahead beyond the immediate neighbors of the current state Hill climbing is sometimes called greedy local search Perform quite well Hill climbing often makes very rapid progress towards a solution, because it is usually quite easy to improve a bad state Hill-Climbing Hill-Climbing Issues Example: Hill-climbing search 1. Pick a solution from the search space and evaluate it. Define this as the current solution. 2. Generate a new solution by applying a transformation to the current solution and evaluate it. 3. If the new solution is better than the current then make it the new current solution; otherwise discard it. 4. Repeat steps 2 and 3 until there are no more possible transformations. Often gets stuck for the following reasons: Local maxima Ridges Plateau (...sideways moves... Objective Surfaces Objective Surface The objective surface is a plot of the objective function s landscape The various levels of optimality can be seen on the objective surface 3

Escaping Local Optima Random Restarting Local optima are OK, but sometimes we want to find the absolute best solution There are several ways we can try to avoid local optima and find more globally optimal solutions: Random Restarting Simulated Annealing If at first you don t succeed, try, try again! The idea here is to run the standard HC search algorithm several times, each with different, randomized initial states Since HC is a local search strategy, trying multiple initial states allows us to locally explore a wider range of the search space Random Restarting Random Restarting If we pick lucky initial states, we can find the global optimum! It turns out that, if each HC run has a probability p of success, the number of restarts needed is approximately 1/p For example, with 8-Queens, there is a probability of success p» 0.14»1/0.14 So, on average, we would need only 7 randomly initialized trails of the basic HC search to find a solution Simulated Annealing (SA) Simulated Annealing In metallurgy, annealing is the process used to temper or harden metals and glass by heating them to a high temperature and then gradually cooling them, thus allowing the material to coalesce into a low-energy crystalline state Builds on an analogy with thermodynamics. The Boltzmann probability distribution describes the relative probabilities of finding a system in different states as a function of temperature Let us consider to switch from HC to gradient descent (minimizing the cost) 4

Simulated Annealing Simulated Annealing According to thermodynamics, to grow a crystal: Start by heating a row of materials in a molten state The crystal melt is cooled If the temperature is reduced too quickly, irregularities occur and it does not reach its ground state By analogy, SA relies on a good cooling schedule, which maps the current time to a temperature T, to find the optimal solution Usually exponential Can be very difficult to devise Imagine the task of getting a ping pong ball into the deepest crevice in a bumpy surface If we let the ball roll come to rest in a local minimum The simulated annealing solution is to start shaking hard (i.e. at a high temperature) and then gradually reduce the intensity of the shaking (i.e. lower the temperature) Simulated Annealing Simulated Annealing 1) In some way, the simulated-annealing algorithm is similar to hill climbing. In stead of picking the best move, it picks a random move If improve the situation always accepted Else it accepts the situation with some probability less then 1 The probability decreases exponentially with the badness of the move the amount the evaluation is worsened Requirements for simulated annealing: A description of possible system states (representation). A generator of random changes in the configuration (search operator). An evaluation function E (analog of energy) for minimization. A parameter T (analog of temperature) and an annealing schedule. The probability also decreases as the temperature T goes down Genetic Algorithms Evolutionary Analogy So far the optimization strategies we ve discussed search for a single solution, one state at a time within a neighborhood Genetic algorithms (GAs) are a unique search approach that maintains a population of states, or individuals, which evolves Also called evolutionary search Consider a population of rabbits: Some individuals are faster and smarter than others Slower, dumber rabbits are likely to be caught and eaten by foxes Fast, smart rabbits survive to do what rabbits to best: make more rabbits!! 5

Evolutionary Analogy Evolutionary Analogy The rabbits that survive breed with each other to generate offspring, which starts to mix up their genetic material Fast rabbits might breed with fast rabbits Fast rabbits with slow rabbits Smart with not-so-smart, etc Furthermore, nature occasionally throws in a wild hare because genes can mutate In this analogy, an individual rabbit represents a solution to the problem (i.e. a single point in the state space) The state description is its DNA, if you will The foxes represent the problem constraints Solutions that do well are likely to survive What we need to create are notions of natural selection, reproduction, and mutation Core Elements of GAs Genetic Algorithm Example For selection, we use a fitness function to rank the individuals of the population For reproduction, we define a crossover operator which takes state descriptions of individuals and combines them to create new ones For mutation, we can merely choose individuals in the population and alter part of its state POP = initialpopulation // build a new population repeat { // with every generation NEW_POP = empty for I = 1 to POP_SIZE { X = fit individual // natural selection Y = fit individual CHILD = crossover(x,y) // reproduction if small random probability then mutate(child) // mutation add CHILD to NEW_POP } POP = NEW_POP } until solution found or enough time elapsed return most fit individual in POP Genetic Algorithm Example Selection The previous algorithm completely replaces the population for each new generation but we can allow individuals from older generations to live on Reproduction here is only between two parents (as in nature), but we can allow for more!! The population size also is fixed but we can have this vary from one generation to the next Selection (either to reproduce or live on) from one generation to the next relies on the fitness function We can usually think of the fitness function as being a heuristic, or the objective function We want to apply pressure that good solutions survive and bad solutions die Too much and we converge to sub-optimal solutions Too little and we don t make much progress 6

Selection Reproduction Deterministic selection 1. Rank all the individuals using the fitness function and choose the best k to survive 2. Replace the rest with offspring Can lead fast convergence (and local optima) Stochastic selection Randomly choice of the k to survive Slower to converge and could lose good solutions The unique thing about GAs is the ability of solutions to inherit properties from other solutions in the population The basic way to perform a crossover operation is to splice together parts of the state description from each parent Reproduction Mutation There are many different ways to choose crossover point(s) for reproduction: Single-point: choose the center, or some optimal point in the state description take the first half of one parent, the second half of the other Random: choose the split point randomly (or proportional to the parents fitness scores) Uniform: choose each element of the state description independently, at random (or proportional to fitness) Ect... There are also a variety of ways to mutate individuals in the population The first question to consider it who to mutate Alter the most fit? Least fit? Random? Mutate children only, or surviving parents as well? How many to mutate? The second question is how to mutate Totally arbitrarily? Mutate to a better neighbor? GAs and Creativity Genetic Algorithm Example The objectives of the GA search: The population as a whole is trying to converge to an optimal solution Because solutions can evolve from a variety of factors, without prodding from us as to which direction to go (as in local search), very novel problem solutions can be found/discovered Constructing a jumbo jet (Cannot guarantee optimal solution): The shape of the wings Be able to fly Fuel-efficient Limited size Weight Stable Intact Etc... 7

References Summary 1) Roger Eriksson, Department of Computing Science, College Skövde, Sweden Iterative improvement algorithms keep only a single state in memory. Can get stuck in local extrema/optima Local search methods are more appropriate for solving complete search and optimization problems State spaces can be prohibitively large The goal is different than with systematic search strategies Summary Next Time! There are several effective ways of escaping local optima for local searching, which exploit different properties: Random restarting tries several times from different parts of the search space Simulated annealing allows for a variety of moves by searching stochastically (it is complete and optimal given a slow enough cooling schedule) Game Playing! Section 6.1-6.4 Local beam search keeps track of k states rather than just one, Genetic algorithm is a variant of stochastic local beam search 8