EXPERIMENTAL INVESTIGATION OF THE APPLICABILITY OF ANT COLONY OPTIMIZATION ALGORITHMS FOR PROJECT SCHEDULING JADE HERBOTS WILLY HERROELEN ROEL LEUS

Transcription

1 EXPERIMENTAL INVESTIGATION OF THE APPLICABILITY OF ANT COLONY OPTIMIZATION ALGORITHMS FOR PROJECT SCHEDULING JADE HERBOTS WILLY HERROELEN ROEL LEUS

2 Experimental Investigation of the Applicability of Ant Colony Optimization Algorithms for Project Scheduling Jade Herbots Willy Herroelen Roel Leus Department of Applied Economics, KU.Leuven, November 26, 2004 Abstract Ant Colony Optimization (ACO) algorithms have recently been developed. These are algorithms that have taken inspiration from the observation of ant colonies' foraging behaviour. Although a lot of effort has already been put into the development of ACO algorithms for multiple fields of application, the research for the resource-constrained project scheduling problem (RCPSP) has remained scarce. This problem is an N P-hard optimization problem that aims at minimizing the makespan of a project consisting of activities which have to be processed under two types of constraints, namely precedence and resource constraints. In this paper, we investigate the applicability of ACO for the RCPSP. First, we study the basic features of ant algorithms and look at some implementations for the RCPSP. Then, we break down the algorithm into its constituent procedures and examine various mechanisms to corne to a better understanding of the algorithmic performance. Computational experiments are performed on standard benchmark datasets. Our experiments result in two full-fledged ant algorithms for the RCPSP. Finally, we compare the performance of these two algorithms with existing ACO algorithms and with other non-hybrid heuristics taken from literature. This comparison allows us to predict very good results for hybrid versions of ACO algorithms for the RCPSP. 1 Introduction Project scheduling, as a part of project management, covers a broad variety of optimization problems. In this article we study the resource-constrained project 1

3 scheduling problem (RCPSP), also denoted as m, 11cpmlCmax in the classification scheme of Herroelen et al. (Herroelen et al. 1998). The RCPSP aims at minimizing the makespan of a project consisting of activities which have to be processed under two types of constraints, namely precedence and renewable resource constraints. The former prevent activities from being processed before their immediate predecessor activities have been finished. The latter limit the availability of renewable resources during each time unit. Since the RCPSP belongs to the class of N P-hard optimization problems (Blazewicz et al. 1986), optimal solutions can only be found in reasonable times for relatively small projects. Even so, numerous exact solution methods have been developed, we mention Brucker et al. (1998), Demeulemeester & Herroelen (1992) and Sprecher (2000). Since practical cases typically relate to larger projects, the development of good heuristics was very much required. This encouraged a lot of researchers to compose powerful algorithms. A classification and evaluation of different heuristics for the RCPSP can be found in Kolisch & Hartmann (1999) and Hartmann & Kolisch (2000). Recently Dorigo & Di Caro (1999) have developed a new class of metaheuristics inspired by the behaviour of ants, called Ant Colony Optimization (ACO) algorithms. In ACO algorithms, artificial ants stepwize build solutions by making probabilistic decisions based on local information. At each step, an ant chooses one possibility from a set of feasible options, thus progressing towards a complete solution. Although promising results have already been achieved for many types of problems, the applicability of ACO for the RCPSP has not yet been thoroughly investigated. In this paper, we examine whether ACO algorithms may be of use to the RCPSP. In section 2 we describe the RCPSP and explain how to build a solution. Section 3 gives an outline of a typical ACO algorithm for the RCPSP, whereas the existing ACO algorithms for the RCPSP are discussed in section 4. In section 5 we perform some algorithmic enhancements and in section 6, we compare our results with figures from the literature. Finally, we conclude in section 7. 2 Resource-constrained project scheduling problem The RCPSP seeks to minimize the duration of a project subject to finish-start zero-lag precedence constraints and renewable resource constraints. A project 2

4 consists of a set J = {O, 1,..., n, n + I} of activities. The precedence relations prevent an activity j from being started before all its immediate predecessor activities, gathered in the set Pj, have been processed. The renewable resources are comprised in the set K = {I,..., IKI}, each resource type k has a limited capacity of Rk units during each time period. The processing of activity j requires rjk units of resource k E K during each period of the non-preemptable activity duration Pj. The values of Pj, rjk and Rk are considered to be deterministic. We assume activities 0 and n + 1 to be dummies, having zero duration and zero resource usage. For each activity j a latest start time LST j and a latest finish time LFT j can be calculated respectively through forward and backward pass recursion (critical path analysis, see e.g. Demeulemeester & Herroelen 2002). Solving an RCPSP boils down to building a feasible schedule of minimal duration. A project schedule is represented by S = (Fl, F2,..., Fn) with Fj the finishing time of activity j. To create such a schedule, meta-heuristics usually operate first on a schedule representation. The schedule representation can be transformed into a schedule through a generation scheme. Different types of both the schedule representation and the generation scheme exist. Hartmann (1999) distinguishes five different schedule representations, of which the activity list (AL) representation is one of the most widely-spread. An activity list is a precedence feasible list, meaning that each activity j is positioned after all activities in Pj. In this paper we will only make use of activity lists as a schedule representation. Next, a schedule generation scheme (SGS) is used to build a schedule from the activity lists. Two types of SGS exist, namely the serial (SSGS) and the parallel (PSGS) (Kolisch & Hartmann 1999). The former makes use of activityincrementation and the latter of time-incrementation. Activity-incrementation selects one activity in each of the n stages and schedules it at the earliest precedence- and resource-feasible finishing time. When the SSGS is supplied with an activity list, SSGS always chooses the first feasible activity in the list. In the case of PSGS, we perform time-incrementation. PSGS starts with a partial schedule that contains only the start. activity 0 at time O. Then, we determine the next time instance tg together with the set of eligible activities Dg at tg and select the activities to schedule at tg in the order as they appear in the activity list and calculate the following t g. This procedure is repeated until all the activities have been scheduled. In contrast to the serial SGS, the parallel SGS does not always have an activity list that can be turned into an optimal 3

5 schedule. However, since we are primarily interested in finding good solutions and less in finding optimal solutions per se, there is no reason to eliminate the PSGS from our experiments. Both the serial and the parallel generation scheme come in three versions described by Demeulemeester & Herroelen (2002), namely the forward, the backward and the bidirectional scheme. The forward scheme is the default procedure and was described above. For the backward scheme, we go about in a similar fashion, but the activities are planned in reverse order, scheduling the dummy end activity first and ending with the dummy start activity. When all the activities have been planned, the schedule is moved forward until the dummy start activity starts at time O. The bidirectional scheme is a combination of the forward and the backward scheme: in every iteration activities are planned in a forward or backward fashion according to the specifications of the serial or parallel scheme. In a final step, the backward planned activities are moved forward. In this section we have described the generation scheme and the schedule represention that will be used to solve RCPSP instances in this article. This is not only of importance to fully understand the presented algorithms, but it is also crucial when comparing the performance of different algorithms for the RCPSP. Since heuristics developed by different authors are usually tested on computers with different computer architectures and operating systems, we can not compare results that are obtained using the same amount of CPU-time. Therefore, in line with Kolisch & Hartmann (1999), a limit is imposed on the number of generated and evaluated schedules in order to provide a fair basis for comparison. 3 Ant Colony Optimization algorithms Ant Colony Optimization (ACO) is a metaheuristic proposed by Dorigo & Sttitzle (2004). It was succesfully implemented for solving a variety of complex problems, such as the traveling salesperson problem (TSP) (Dorigo & Gambardella 1997, Sttitzle & Dorigo 1999), the quadratic assignment problem (QAP) (Gambardella et al. 1999), vehicle routing problems (Bullnheimer et al. 2001) and production scheduling problems (Colomi et al. 1994, Shyu et al. 2004). ACO was inspired by the foraging behaviour of ant colonies. While walking, each ant drops a chemical substance called pheromone on the ground. When 4

6 an ant arrives at a decision point, e.g. an intersection, it makes a probabilistic decision biased by the amount of pheromone present on each of the possible roads. The information from the amount of pheromone helps ants to find the shortest path between their nest and a food source. The natural behaviour of ants was copied in several ACO algorithms for various types of problems. In this paper, we only discuss ACO algorithms for the RCPSP. The presented algorithms all go about in a similar fashion: several generations of ants build an activity list starting from the dummy start activity. Then, position by position is filled until the dummy end activity is reached. While walking through the network the ant drops pheromone and simultaneously old pheromone evaporates. When the activity list is constructed, we build a schedule by using a generation scheme. After evaluating the constructed schedule, additional pheromone trails are dropped on succesful paths. This form of indirect communication, also known as stigmergy, permits a colony of ants to converge to very good solutions. In this section we present the basic features of the existing ACO algorithms for the RCPSP. All algorithms follow the same pseudo-code: procedure Ant Colony Algorithm initialization while (number_of_generations < GEN) do {for z = 1 to number_of_ants {construct activity list generate schedule if (makespan < best_solution) then {best_solution := makespan} } if (number_of_generations_same_solution >= g_max) then {best_solution := iteration_solution} pheromone updates alter parameters } while (number_of_evaluations <= EVALUATIONS) do {local search} Some basic elements of the code will now be discussed in greater detail. The remaining functions of the procedure will be explained later on in the text. 5

7 3.1 An ant colony An ant colony consists of a generation of ants. The number of ants in a generation typically varies between one and ten ants. During the processing of the algorithm, each ant of the generation constructs an activity list. When the ant has terminated its path and has performed the necessary updates, it dies. If the ant is of the 'synchronized' type, the algorithm waits until all ants of the current generation have died to release another generation of ants. In case of 'non-synchronized' ants, a new ant is launched the moment an old ant dies. ACO algorithms for the RCPSP mostly make use of synchronized ants because of the programming difficulties related to the non-synchronized species. 3.2 Constructing an activity list Step by step, each ant builds an activity list, starting from dummy activity 0 to dummy activity n + 1. At each step, the ant chooses the next activity from the set Dg containing all the activities of which the predecessors have ended. The probability Pij of selecting activity j from the set Dg in step i is determined by heuristic ('TJij) and pheromone information (T ij). These two values embody the usefulness of placing activity j in the ith place of the activity list. The pheromone information refers to the chemical substance dropped by real ants on paths that have been visited, this factor is inherent to any ACO algorithm. The heuristic information comprises specifications of the problem instance. The values for 'TJij and Tij are stored respectively in matrices Hand T. According to Merkle et al. (2002) the calculation of Pij can be done through direct evaluation by using Eq. (1): (1) or through summation evaluation as in Eq. (2): ( t, bi- k. Tkj])O'. ['TJij],'l k=l Pij = -L-=--'('-t,- -b-i--k-.-t-'-kh-l)-o',.,,--.[-'tji-hl-,'l (2) hedg k=l I > 0 determines the relative influence of the pheromone values from previous decisions. When calculating the probability of placing activity j in position i, we 6

8 take the pheromone values for placing activity j in previous positions (k) in the activity list into acount. This associates a higher probability of being chosen with activities for which placements in previous positions in the activity list delivered good results. The parameters ex > 0 and f3 > 0 represent the relative influence of the pheromone trails compared to the heuristic information. Besides the exclusive use of direct or summation evaluation, we can apply a combination of both evaluation methods (Merkle et al. 2002). In that case, parameter c, 0 ::::; c ::::; 1, denotes the relative influence of the direct versus the summation evaluation. c is set to 1 when we exclusively use direct evaluation. In the other extreme case of pure summation evaluation, c is set to zero. In order to calculate the probabilities we replace every Tij in Eq. (1) with the T;rvalues calculated in Eq. (3): T;j := c Xi Tij + (1 - c). Yi. L,i-k. Tkj k=l (3) with Xi:= L (t,i-k. Tkh) and Yi:= L Tih. hedg k=l hedg 3.3 Pheromone updates Pheromone updates imply a change in the pheromone values in matrix T. In combination with the RCPSP, only the online delayed pheromone update is commonly used. This method implies that the pheromone values are only updated when each ant of a generation has constructed its activity list. In general, two types of updates are being performed. First, an amount of pheromone is evaporated on each trail to prevent old trails from having too much impact on present decisions. This mechanism is called pheromone trail evaporation and is characterized by an evaporation rate p, 0 ::::; p ::::; 1. During the process each element of the matrix T is altered according to the following expression: T:=(l-p) T (4) During the second pheromone update one of the ants adds extra pheromone to paths that delivered good solutions. The amount of pheromone to deposit and the choice of paths to update depend on the specifications of the algorithm. 7

9 3.4 Local search Dorigo & Gambardella (1997) have shown that local search can improve ACO's performance. Local search methods can be implemented during or after the ACO phase. 4 Existing ACO algorithms for the RCPSP In this section we discuss two existing ACO algorithms for the RCPSP, both developed by Merkle et al. (2002), namely Simple Ant System and Ant System. The authors were inspired by Ant System Traveling Salesperson Problem (AS TSP) for the traveling salesperson problem (TSP) (Dorigo et al. 1996). The algorithms are constructed according to the description of the procedure Ant Colony Algorithm in section 3. Simple Ant System does not possess all the functionalities of Ant System for the RCPSP. 4.1 Simple Ant System Simple Ant System for the RCPSP or s-as-rcpsp was developed by Merkle et al. (2002). Our implementation is based on the description in the forementioned article and is represented schematically by the pseudo-code of section 3. The algorithm consists of three major parts: the initialization, the construction of activity lists and the updates. Initialization. At the start of the algorithm all parameters are set to their initial values, the matrices T and H are constructed and the first generation of ants is created in order to build a set of activity lists. In s-as-rcpsp, five synchronized ants are created per generation and they reproduce for 1000 generations, thus building 5000 schedules. The parameter values were set as follows: a = 1, f3 = 1, c = 0.6 and p = The matrix T is initialized with small positive values. We chose 0.5 from the set {O; 0.5; 1; 1.5} since this value delivered the best results. The elements of the H-matrix were calculated through the normalized Latest Starting Time-heuristic (nlst), described in Eq. (5). 7)ij = maxlstk - LSTj + 1 kedg (5) Merkle et al. (2002) use an adaptation of the LST-heuristic because the 8

10 relative differences between the LST-values of the activities that become plannable in the final phases can be very small and so it is better to use the absolute differences to the maximum LST-value over all eligible activities. Activity list construction. This topis was described in section 3.2. s-as RCPSP uses the combination evaluation method from Eq. (3) with c = 0.6. In other words, a combination of direct and summation evaluation is adhered to, with a slightly higher influence of the direct evaluation. 'Y = 1, so that the pheromone values used for determining the previous places in the activity list have the same influence as the current pheromone trails. When an activity list is built, s-as-rcpsp uses SSGS to generate a schedule. The pheromone updates. The pheromone evaporation takes place according to Eq. (4) and the description in section 3.3. When all the ants of a generation have built a solution, we update the best solution found so far and the best solution found by the current generation of ants. The best solution has to be interpreted as the activity list that delivered the schedule with the smallest makespan. For every activity j in position i in one of the fore-mentioned activity lists, we replace the value Tij in the 1 matrix T by Tij+P' 2T*' with T* the makespan of the best-found schedule so far. The s-as-rcpsp makes no use of local search techniques. Table 1 represents the average percentage deviation from a lower bound for our own implementation of the s-as-rcpsp. We consider four testsets from the PSPLIB 1 (Kolisch & Sprecher 1997), referred to as J30, J60, J90 and J120, depending on the number of activities each problem instance consists of. The evaluation of the calculated makespans is based on the average percentage deviations from the currently best known lower bounds for the testsets from the PSPLIB (based on the results in the PSPLIB on April 9, 2004). For J30 the best known lower bounds are equal to the optimal solutions of the problem instances. All programs are implemented using Visual C and run on a computer equipped with a 1 GHz Pentium III processor. 1 http) / / datasm.html 9

11 Table 1: Average percentage deviations from the lower bounds for s-as-rcpsp 4.2 Ant System Merkle et al. (2002) also developed a second and better performing ACO algorithm for the RCPSP, namely Ant System or AS-RCPSP. The difference with s-as-rcpsp is that not all parameters are kept constant and that additional mechanisms such as ignoring an elitist solution, local search and bidirectional planning are incorporated. Variable parameter values. In the AS-RCPSP, only a = 1, j3 = 1, c = 0.5 and 'Y = 1 remain constant during the execution of the algorithm. The evaporation rate p initially has a value of 0.025, but is increased up to during the last 200 generations. This causes the pheromone trails to evaporate more quickly during the final stage of the algorithm, thus allowing old solutions to have less influence on current decisions and promoting a more extensive search of the area around recently discovered good solutions. We start with a lower evaporation rate however, in order to prevent the algorithm from converging too quickly. Ignoring an elitist solution. Normally, ACO algorithms add pheromone to paths that led to good solutions, thus permitting an intensive search of the area around these good solutions. Unfortunately, this strategy may cause a premature convergence of the search process, so that the solution space is not thoroughly explored. To prevent this phenomenon from occurring, Merkle et al. (2002) have introduced 9max, the maximum number of iterations the best solution can be left unchanged. When 9max is reached, the best solution will be replaced by the best result from the operating generation of ants. We have not included this in our implementation since no significant improvements were accomplished by using this technique. Local search. At the start of the algorithm five synchronized ants are created, which reproduce themselves for 850 generations as to generate 4250 activity lists. The remaining 750 lists are constructed through local search. We have used 2-opt, a fastest descent strategy to perform the local search. 2-opt swaps pairs of activities in an activity list. The swap is fixed for testing the remaining swaps in the activity list, on two conditions. First, we 10

12 check if the precedence constraints have not been violated due to the swap and secondly, the makespan of the schedule generated from the activity list has to be smaller than the makespan of the original schedule. Bidirectional planning. Merkle et al. (2002) make use of two types of ants, namely forward and backward running ants. The former operate on the original RCPSP whereas the latter build an activity list starting with the final position and then working backwards. AS-RCPSP maintains a colony of forward and a colony of backward ants during the first 100 generations. Afterwards, the results of both colonies are compared for the last 25 generations and the algorithm proceeds with the colony that realized the best average results. Table 2 represents the average deviations from the lower bounds of the AS RCPSP for the different test sets of the PSPLIB. These figures are the result of our own implementations. The algorithm was allowed to construct 5000 schedules. Table 2: Average percentage deviations from the lower bounds for AS-RCPSP 5 Algorithmic enhancements In this section, a number of elements are selected from the Ant Colony algorithm procedure as described in section 3 in an attempt to identify opportunities for algorithmic enhancements. For each element, possible variations will be scrutinized and the most promising options will be combined into a full-fledged ACO algorithm. 5.1 Initialization The values of the entries of matrix T were initially set to 0.5. To provide us with the heuristic information T/ij' we have calculated the priority values with successful heuristics for the RCPSP retrieved from Klein (2000) in combination with different generation schemes. Although the probability calculations in ACO are only partially influenced by the heuristic information, this strategy 11

13 did turn out to be succesful. Three combinations were initially retained from Klein (2000), the LST-heuristic in combination with SSGS, the Latest Finish Time or LFT-rule together with the PSGS and finally, the Weighted Resource Utilization and Precedence or WRUP-rule in combination with PSGS. The last heuristic was chosen because it considers the resource utilization and the number of direct successors of an activity, which allows us to include important features of the RCPSP into our algorithm. For the same reasons as in section 4.1, we have opted for normalized versions of the heuristics. We have implemented the nlst described in Eq. (5), the nlft calculated as in Eq. (6) and the nwrup derived from Eq. (7). TJij = maxlftk - LFTj + 1 kedg (6) I I ",Tjl. ( ",Tkl) TJij = W Sj + (1 - w) L..J - - mm W ISkl + (1 - w) L..J IEK Rl kedg IEK Rl (7) In Eq. (7), w E [0,1]. The n WRUP consists of four terms with the first two adding up to the WRUP-value. The first term considers the immediate successors of activity j comprised in set Sj. The second term represents the influence of the resources. Tjl is the number of resources of type I required by activity j, Rl is the number of available resources of type I and K denotes the set of resource types. The final two terms are included for normalization purposes. Intuitively, it seems suitable to increase the influence of the heuristic information at the beginning of the algorithm and decrease it towards the end. This was implemented through manipulation of the parameter fj while using combination evaluation as described in Eq. (3). However, testing of three different mechanisms for changing fj showed that this was not a good option. The worst performance (average deviation from the lower bounds of 4.422%) was registered when fj declined linearly to O. Since fj represents the influence of the heuristic values in the calculation of the probabilities, this means that the influence of the heuristic gradually drops until the search process is only guided by the pheromone trails. From the poor results, we can conclude that the heuristic information remains necessary for an effective exploration of the solution space. The foregoing reasoning leads us to set the value of fj to a constant value of 1. The parameter 'Y was set equal to 1 and c is determined by experimenting with smaller testsets. In Table 3 we have represented the average deviation from the lower bounds for test set J90 and 5000 schedules. The three algorithms 12

14 were tested with a forward, backward and bidirectional scheme as referred to in section 2. The best result was obtained for the PSGS in combination with the nlft-heuristic. The second best performance was achieved by the SSGS. and the nlst-heuristic. In the remainder of this text these two versions will be the basis for further implementations. The two algorithms will be denoted as ALG1 and ALG2. SSGS/nLST PSGS/nLFT PSGS/nWRUP forward 4.0l7(c = 0,6) 3.896( c = 0,6) 4.491(c = 0,8) bidirectional 4.392( c = 0,5) 6.710(c = 0,6) 6.966(c = 0,8) backward 4.266(c = 0,4) 6.283(c = 0,8) 4.461(c = 0,8) Table 3: Comparison of the average percentage deviations from lower bounds for different generation schemes for J Construct activity list For constructing the activity list we have implemented two strategies new to the RCPSP, inspired by ACS (Dorigo & Gambardella 1997). The first one starts by picking a random number q between 0 and 1. When q is smaller than qo, a parameter in the range [0,1], the next activity in the activity list is determined as the activity with the largest value for P'ij' In the other case, when qo is strictly larger than q, the following activity is selected by means of a probabilistic choice. Higher values for qo result in an extensive search of paths with high pheromone values. By decreasing qo, options with lower pheromone values will be chosen more easily, thus avoiding convergence of the search process. Tests on smaller testsets have revealed that qo = 0.5 was the best option when using a constant value. For the case where the value of qo is varied throughout the search procedure, we have tried out two options for increasing qo. An increase is carried through in the first place to allow the algorithm to explore a larger area during the early stages of the algorithm ('exploration') and secondly, to conduct a more local search in the final steps ('intensification'). The first version, referred to as 'increasing 1', starts off with qo = 0.5. After 500 generations qo is set to 0.9. In the second implementation, 'increasing 2', qo has an initial value of 0.5 but, after 500 generations, qo is calculated as qo = number _of_generations/looo, which induces a more gradual augmentation of parameter qo. This last strategy proved to be very succesful (Table 4). The second strategy for constructing the activity list consists of performing a 13

15 qo = 0 qo = 0.5 increasing 1 increasing2 ALGI ALG Table 4: Average percentage deviations from the lower bounds for J90 pheromone update (evaporation) each time an ant has constructed its path. This urges ants of the same generation to visit paths that strongly differ from each other, which in time enlarges the search space. We used different evaporation rates for the implementation. p = turned out to be the most succesful, both for ALGI and ALG2. However, the implementation led to a worsening of the overall results compared to the results in Table 3. For J90, we recorded average deviations from the lower bounds of 4.335% for ALGI and 3.953% for ALG2. We additionally tried to perform a local update by adding an amount of pheromone to each activity i of the visited path during each iteration. The amount of pheromone was set equal to the evaporation rate divided by the product of the number of activities and the LFT i of the visited activity. This formula was based on Ant Colony System for the traveling salesman problem (TSP) (Dorigo & Gambardella 1997);. P was set to 0.5. This did not lead to an improvement of the results: the average deviations from the lower bounds for test set J90 were 4.110% for ALGI and 3.977% for ALG Ignoring an elitist solution Merkle et al. (2002) have implemented a mechanism that replaces the overall best solution with the generation best solution when the overall best solution remains unaltered during 9max iterations. However, we found no evidence of the positive effects of this mechanism. Merkle et al. (2002) assumed that the neighbourhood of the replaced solution contained a lot of equal or better solutions that would easily be found by the algorithm when continuing its search. Our results for the test set J90 (Table 5) can not confirm this. Therefore we conclude that the replaced solutions are not retrieved and that the results are worsened through the solution exchange. 14

16 Table 5: Average percentage deviations from the lower bounds for ALGI for J Pheromone updates We have implemented two alternative methods for updating matrix T. These are based on the Max - Min Ant System for the TSP (Sttitzle & Hoos 2000). The first method consists of placing an upper bound on the pheromone values. The upper bound, T max, is calculated as: T max = _1_. ~, with T* the 1- p T* makespan of the overall best solution. The upper bound limits the amount of pheromone on a trail. In this way, it restricts the differences between the pheromone values. Throughout the search procedure, the probabilities of the eligible acivities remain closer to each other, keeping all the theoretically eligible activities also eligible in practice. In a second version, we have also imposed a minimum value T min for the h 1 T max(l - Pdec). h T max avg - 1 Pdec Tmax + avg - 1. Tmin p eromone tral s: T min = ( )' Wit Pdec = ( ) and avg = number _of _activities/2. If Tmin > Tmax, we set Tmin equal to T max. Before running our program, the elements of matrix T were set to 10, so that they would be equal to T max after the first generation. The average deviation from the lower bounds for J90 amounted to 4.885% for both versions, which is significantly worse than when using our original program. By only using T min and T max during the first 500 generations, we have tried to promote the exploration during the first part of the search process. The average deviation from the lower bounds for J90 amounted to 4.340%. Without using Tmin, we realized an average deviation from the lower bounds of 4.058%, which is still not an improvement. Therefore, we conclude that the implementation of T min and T max is not useful to the RCPSP. Next, we have tried out an additional strategy for performing the pheromone updates, namely the exclusive updating of the iteration best solution (Merkle et al. 2002). This contrasts with the s-as-rcpsp and the AS-RCPSP, where both the overall best solution and the iteration best solution are updated. We have applied a mixed strategy in which we exclusively update the best solution from the iteration during the first 25 iterations. The next 50 iterations, we additionally update the overall best solution every five iterations. During the next 50 iterations, every three iterations, the overall best solution is used for 15

17 the updates and during the following 50 iterations, only two iterations separate the overall best solution update. Above 250 iterations we only perform updates of the overall best solution. The increase of the frequency of using the global best solution to perform the updates, can be justified by our former conclusions that it seems sensible to direct the search process towards areas around good solutions towards the end of the algorithm. The implementation of this mechanism for ALG 1 led to an average deviation from the lower bounds of 4.270% for the testset J90. This means that it remains best to perform updates of the overall best solution as well as of the best solution of the iteration during the whole search process. 5.5 Local search Finally, we have tested the performance of the local search mechanism. We use 2-opt, the same local search method opted for by Merkle et al. (2002). Table 6 represents the average deviations from the lower bounds for testset J90 using local search after a variable number of generations. These results demonstrate that better results are obtained thanks to local search. The best results were obtained by running ACO for 900 generations and then using local search until 5000 schedules have been built. This statement holds for both ALGI and ALG2. Table 6 additionally demonstrates that it is useful to implement an ACO algorithm, since our results show that when a local search is called after a structured search has been performed for a certain number of generations, the algorithm delivers better results. local local local local no local search search search search search after 800 after 850 after 900 after 950 gen. gen. gen. gen. ALGI ALG Table 6: Average percentage deviations from the lower bounds when using local search for J90 16

18 5.6 Results of the computational experiments The test results of this section will be used to build two final versions of the algorithm. Our first algorithm is based upon ALGI and the second upon ALG2. We do not include local search, in order to obtain two non-hybrid heuristics. ALG 1 makes use of SSGS in combination with the nlst heuristic. The second version builds its schedules according to the PSGS in combination with the nlft-heuristic. We use qo for the probability calculations. Initially qo is set to 0, but after a number of generations qo is calculated as number _ of _generations Itotal_number _of_generations. We have pinned (3 and I down to the value 1. The other parameters were derived through tests on smaller datasets. c was hereby set to In combination with the setting for c, parameter p was also tested for different values. We select as the starting value, however, after 700 generations p climbs to The final parameter is the number of generations from which qo linearly increases which is equal to 500. The results of this implementation are reported in Table 7 for ALGI for the datasets J30, J60, J90 and J120 The results of Table 7: Average percentage deviations from the lower bounds for ALGI ALG2 for the different test sets of the PSPLIB are gathered in Table 8. Each of the algorithms was allowed to construct 5000 schedules. Table 8: Average percentage deviations from the lower bounds for ALG2 6 Positioning the results In this section, our results will be placed in a broader context. First, we compare the results of our two implementations of existing ACO algorithms. Afterwards, we look at the performance of other heuristics for the RCPSP retrieved from the literature. Unless stated otherwise, each algorithm was allowed to evaluate 5000 activity lists. The tables in which the results are gathered, are built up as follows: the second row gives the sums of the makespans of the schedules of 17

19 all the problem instances from the specified testset. The row 'Av. Dev. LB' indicates the average deviations from the lower bounds. These lower bounds are retrieved from the PSPLIB. For the set J30, this value is equal to the optimal makespans of the problem instances. The row 'Av. Dev. Best' represents the average deviations from the best reported solutions in the PSPLIB. The last update of these solutions dated from April 9, The row 'Number Best' indicates the number of times our algorithm found the best known solution. In brackets, we mention the total number of problem instances in the testset. The row 'Av. CPU-time' shows the average processing time the computer needed to calculate the solution. The final row 'Max CPU-time' indicates the maximum processing time needed to process an instance from the dataset. 6.1 Evaluation and comparison of the results of existing ACO algorithms To start with, we compare our implementations of the s-as-rcpsp and the AS RCPSP. From Table 9 and Table 10 we conclude that the AS-RCPSP delivers better results than the the s-as-rcpsp for all datasets and this without using a significantly higher CPU-time. J30 J60 J90 J120 Total Av. Dev. LB 0.496% 3.437% 4.017% 1l.383% Av. Dev. Best 0.496% l.296% 1.710% 6.038% Number Best 392(480) 349(480) 346(480) 160(600) Av. CPU-time l Max. CPU-time Table 9: Results for s-as-rcpsp for 5000 activity lists J30 J60 J90 J120 Total Av. Dev. LB 0.407% 3.412% 3.834% % Av. Dev. Best 0.407% l.274% l.544% 4.906% Number Best 403(480) 349(480) 348(480) 159(600) Av. CPU-time Max. CPU-time Table 10: Results for AS-RCPSP for 5000 activity lists The results in Table 9 and Table 10 differ from the results reported in Merkle 18

20 et al. (2002). These authors only use the testset J120 from the PSPLIB and compute the average deviation from the critical path based lower bounds (Stinson et al. 1978). As Table 11 points out, our versions of the s-as-rcpsp and the AS-RCPSP perform slightly worse than the implementation of Merkle et al. (2002). Although we have tried to stay as close as possible to the original designs, the differences in outcome can not be denied. The most logical explanation for this divergence lies in differences in programming techniques, in implementation and in parameter settings. s-as-rcpsp AS-RCPSP our implementation Merkle et al. (2002) Table 11: Average percentage deviations from the critical path lower bounds for J Comparison with other RCPSP-heuristics The results of our own algorithms, ALGI and ALG2, are gathered in Table 12 and Table 13. A few remarks are appropriate: although our algorithms perform better than our implementation of the AS-RCPSP for the test sets of J90 and J120, they perform worse for the datasets J30 and J60. This difference is probably due to the use of qo for the probability calculation. Additional testing showed that succesful strategies using qo can also be developed for J30 and J60. However, this requires tuning of the parameters according to the size of the RCPSP. An important downside to tuning is that the algorithm loses much of its generality, which is why we do not pursue this issue further. J30 J60 J90 J120 Total Av. Dev. LB 0.723% 3.577% 3.859% 9.916% Av. Dev. Best 0.723% 1.425% 1.566% 4.664% Number Best 381 (480) 347(480) 350(480) 176(600) Av. CPU-time Max. CPU-time Table 12: Results for ALGI for 5000 activity lists In order to provide a better idea of the quality of our algorithms, we compare them with some figures from literature. The comparison is based on the average 19

21 J30 J60 J90 J120 Total Av. Dev. LB 1.368% 3.901% 3.859% 8.820% Av. Dev. Best 1.368% % 1.605% 3.681% Number Best 293(480) 269( 480) 283(480) 114(600) Av. CPU-time Max. CPU-time Table 13: Results for ALG2 for 5000 activity lists deviation from the critical path lower bounds and this for the test set J120 from the PSPLIB. Most heuristics were allowed to calculate 5000 schedules. The results of the heuristics that did not construct exactly 5000 schedules, but delivered a comparable computing effort, are indicated between brackets in Table 14. Our algorithms are compared with seven other types of non-hybrid heuristics: 1) three deterministic single/pass heuristics with regret-based random sampling (single pass/sampling) (Kolisch 1996a, Kolisch 1996b); 2) two genetic algorithms (GA) (Hartmann 1998, Alcaraz & Maroto 2001); 3) one simulated annealing algorithm (SA) (Bouleimen & Lecocq 2003); 4) two random sampling methods (random sampling) (Kolisch 1995); 5) one time-oriented branch-and-bound approach (B&B) (Dorndorf et al. 2000); 6) one list scheduling heuristic that uses solutions obtained by Lagrangian relaxation of a time-indexed integer programming formulation (LS with LR-IP) (Mohring et al. 2000); 7) one ant colony algorithm (s-as-rcpsp) (Merkle et al. 2002). The benchmarks we consider in Table 14 are all non-hybrid heuristics. Therefore we have chosen to include s-as-rcpsp rather than the better performing AS-RCPSP which makes use of local search methods. Since the publication of these non-hybrid methods, a lot of new, powerful heuristics have been developed that obtain even better results, among which we can cite Hartmann & Kolisch (2002), Alcaraz & Maroto (2004), Tormos & Lova (2001), Tormos & Lova (2003), Valls et al. (2002) and Debels et al. (2004). However, all of the cited heuristics are hybrid versions of metaheuristics. Because Ant Colony Optimization algorithms outperform the other non-hybrid heuristics for the RCPSP, we believe that great possibilities for ACO lie in the creation of hybrid versions. As a final remark we point out that Merkle et al. (2002) have only compared 20

22 Algorithm Author Av. Dev. CP LB ALG % LS with LR-IP Mohring et al., 2000 (36.2%) GA 1 Alcaraz and Maroto, % s-as-rcpsp Merkle et al., % GA 2 Hartmann % B&B Dorndorf et al., 2000 (37.1%) SA Bouleimen and Lecocq, % ALGI 37.85% single pass/sampling 1 Kolisch % single pass/sampling 2 Kolisch % single pass/sampling 3 Kolisch % random sampling 1 Kolisch % random sampling 2 Kolisch % Table 14: Average deviations from critical path lower bounds for J120 the results of the heuristics for test set J120, but our results clearly indicate that good results for certain testsets are no guarantee for competitive results on problem instances of a different size. Therefore it is not sure that the favorable reported performance of s-as-rcpsp and AS-RCPSP can be repeated for problems with a different number of instances, unless the parameter settings are allowed to change. 7 Conclusions Ant Colony Optimization algorithms have been recently developed. They achieved promising results for all kinds of optimization problems, varying from routing problems to assignment problems. For the RCPSP, however, the research efforts remained limited. In this article, we have looked at several ACO heuristics developed for the RCPSP and explored the possibilities for improvement. To this aim, we have dissected the ACO algorithm and tested some new mechanisms. Based on our results, we have retained a mechanism that causes the search process to converge to paths with high probability values during the second phase of the algorithm. A downside to this mechanism is that its performance highly depends on the parameter values. From our tests with varying parameters we have learned that the parameter values have a strong impact on the search process and a proportional amount of attention should be paid to their determination. 21

23 Additional experiments have allowed us to conclude that it is useful to let the pheromone trails evaporate more quickly towards the end of the algorithm, in order to give less influence to older trails. Although ACO algorithms are characterized by their use of pheromone for finding good solutions, we have demonstrated in this article that the heuristic information is equally important for an efficient search. The comparison of our test results with other non-hybrid heuristics from the existing literature showed a very positive picture of the Ant Colony Optimization algorithms. This leads us to believe that great improvements can be realized through the development of hybrid Ant Colony Optimization algorithms. References Alcaraz, J. & Maroto, C. (2001). A robust genetic algorithm for resource allocation in project scheduling. Annals of Operations Research, 102, pp Alcaraz, J. & Maroto, C. (2004). Improving the performance of genetic algorithms for the rcps problem. Ninth International Workshop on Project Management and Scheduling. Euro Working Group on Project Management and Scheduling. pp Blazewicz, J., Cellary, W., Slowinski, R. & Weglarz, J. (1986). Scheduling under resource constraints - deterministic models. Annals of Operations Research, 7, pp Bouleimen, K. & Lecocq, H. (2003). A new efficient simulated annealing algorithm for the resource-constrained project scheduling problem and its multiple mode version. European Journal of Operational Research, 149, pp Brucker, P., Knust, S., Schoo, A. & Thiele, O. (1998). A branch and bound algorithm for the resource-constrained project scheduling problem. European Journal of Operational Research, 107, pp Bullnheimer, B., Hartle, R. & Strauss, C. (2001). An improved ant system algorithm for the vehicle routing problem. Annals of Operations Research, 89, pp

24 Colorni, A., Dorigo, M., Maniezzo, M. & Trubian, M. (1994). Ant system for job-shop scheduling. JORBEL - Belgium Journal of Operational Research, Statistics and Computer Science, 34, pp Debels, D., De Reyck, B., Leus, R. & Vanhoucke, M. (2004). A scatter-search meta-heuristic for the resource-constrained project scheduling problem. European Journal of Operational Research, to appear. Demeulemeester, E. & Herroelen, W. (1992). A branch-and-bound procedure for the multiple resource-constrained project scheduling problem. Management Science, 38, pp Demeulemeester, E. & Herroelen, W. (2002). Project scheduling - A research handbook. Vol. 49 of International Series in Operations Research fj Management Science. Kluwer Academic Publishers, Boston. Dorigo, M. & Di Caro, G. (1999). The Ant Colony Optimization meta-heuristic. McGraw-Hill. pp In: D. Corne, M. Dorigo and F. Glover: New Ideas in Optimization. Dorigo, M. & Gambardella, L. (1997). Ant colony system: A cooperative learning approach to the traveling salesman problem. IEEE Transactions on Evolutionary Computation, 1(1), pp Dorigo, M., Maniezzo, V. & Colorni, A. (1996). Ant system: Optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man and Cybernetics, Part B, 26, pp 29-4l. Dorigo, M. & Stiitzle, T. (2004). Ant Colony Optimization. The MIT Press. Dorndorf, U., Pesch, E. & Phan Huy, T. (2000). A branch and bound algorithm for the resource-constrained project scheduling problem. Mathematical Methods of Operations Research, 51(3), pp Gambardella, L., Taillard, E. & Dorigo, M. (1999). Ant colonies for the quadratic assignment problem. Journal of the Operational Research Society, 50, pp Hartmann, S. (1998). A competitive genetic algorithm for resource-constrained project scheduling. Naval Research Logistics, 45(7), pp Hartmann, S. (1999). Self-adapting genetic algorithms with an application to project scheduling. Research Report 506. Universitat Kiel. 23

25 Hartmann, S. & Kolisch, R. (2000). Experimental evaluation of state-of-the-art heuristics for the resource-constrained project scheduling problem. European Journal of Operational Research, 127, pp Hartmann, S. & Kolisch, R. (2002). A self-adaptive genetic algorithm for project scheduling under resource constraints. Naval Research Logistics, 49, pp Herroelen, W., Demeulemeester, E. & De Reyck, B. (1998). A classification scheme for project scheduling problems. Kluwer. pp In: Weglarz J.: Handbook on recent advances in project scheduling. Klein, R. (2000). Bidirectional planning: Improving priority rule-based heuristics for scheduling resource-constrained projects. European Journal of Operational Research, 127(3), pp Kolisch, R. (1995). Production and Logistics. Physica-Verlag, Heidelberg, Germany. chapter Project scheduling under resource constraints. Kolisch, R. (1996a). Efficient priority rules for the resource-constrained project scheduling problem. Journal of Operations Management, 14, pp Kolisch, R. (1996b). Serial and parallel resource-constrained project scheduling methods revisited: Theory and computation. European Journal of Operational Research, 90, pp Kolisch, R. & Hartmann, S. (1999). Heuristic algorithms for solving the resource-constrained project scheduling problem: classification and computational analysis. Kluwer Academic Publishers. In: Weglarz J.: Project scheduling: recent models, algorithms and applications. Kolisch, R. & Sprecher, A. (1997). PSPLIB - a project scheduling library. European Journal of Operational Research, 96, pp Merkle, D., Middendorf, M. & Schmeck, H. (2002). Ant colony optimization for resource-constrained project scheduling. IEEE Transactions on evolutionary computation, 6(4), pp Mohring, R., Schulz, A., Stork, F. & Uetz, M. (2000). Solving project scheduling problems by minimum cut computations. Zeitschrijt fur Operations Research, 29, pp