2012 4th Internatonal Conference on Computer Research and Development IPCSIT vol.39 (2012) (2012) IACSIT Press, Sngapore Meta-heurstcs for Multdmensonal Knapsack Problems Zhbao Man + Computer Scence Department, The Unversty of Hull, Hull, UK Abstract. In ths research, we present the use of genetc algorthms and constrant handlng technques used n genetc algorthms to solve Multdmensonal Knapsack Problems (MKP). We also nvestgate and compare how the use of dfferent genetc algorthm operators and parameters could obtan dfferent solutons for the MKP. Three proposed genetc algorthms (basc GA, GA wth penalty functon1, and GA wth penalty functon2) are mplemented. The performance of these genetc algorthms s compared based on some small MKP. Fnally, a well-known MKP s modfed as a fcttous captal budgetng problem, and the results are reported. The experment results show that genetc algorthms are capable to solve MKP and captal budgetng problems. Keywords: Genetc Algorthm; Multdmensonal Knapsack Problem; Constrant Handlng Technques; Captal Budgetng Problem 1. Introducton The Multdmensonal Knapsack Problem (MKP) s a well-known NP-hard problem [1], whch arses n several practcal problems such as captal budgetng problem. The MKP can be defnes as follows: Maxmze p j x j (1) n j= 1 n Subject to rx b j j j= 1, = 1,, m, (2) x j {0,1}, j = 1,, n. (3) Where n s the number of tems, m s the number of constrants, p j 0 s the proft of the jth tem, r j 0, for = 1,, m, s the weght of the jth tem, and b 0, for = 1,, m, s the capacty of the knapsack. There are m constrants descrbed n equaton (2) and each of them s called a knapsack constrant, hence, the MKP s also called the m-dmensonal knapsack problem. Many practcal problems can be defned as a MKP. For example, n captal budgetng problem, project j has proft p j and consumes r j unts of resource. The goal s to fnd a subset of n projects such that the total proft s maxmzed wthout volatng all resource constrants. In last two decades, a number of papers nvolvng the use of Genetc Algorthms (GAs) to solve MKP. Genetc algorthms, sometmes called natural selecton, are search methods based on the prncples of survval of the fttest. It was ntroduced and orgnally developed extensvely by John Holland [6] n the + Correspondng author. Tel.: + (44)1482465045; fax: +(44)1482466666. E-mal address: Z.Man@2009.hull.ac.uk
mddle of 1970s. The readers are referred to [4, 5] for a detaled steps of constructon of a basc genetc algorthm. 2. Introducton of constrant handlng technques Constrants must not be volated. However, ether some encodng methods or after the use of recombnaton and mutaton operators, may cause llegal ndvduals. In many of the applcatons where GAs are ntended to be the problem of fndng a feasble soluton s tself NP-hard [2]. Mchalewcz [7] also ndcated that each evolutonary computaton technque used to a specfed problem should address the ssue of handlng unfeasble ndvduals. Two constrant handlng technques,.e., the use of penalty functons and repar algorthms wll be mplemented n our proposed GAs. (1) Penalty functons. One most advantage of penalty functon s that nfeasble solutons are allowed n a populaton so that one feasble and hghly ft soluton mght be generated by the combnaton of two nfeasble ndvduals. (2) Repar algorthm. In MKP, t s relatvely easy to repar an nfeasble soluton and transform t nto a feasble one. Repar functons are not only used to make an nfeasble soluton feasble, but also to mprove the ftness of solutons. 3. GAs for multdmensonal knapsack problem In ths secton, a complete mplementaton of genetc algorthms wll be ntroduced. The experments are based on those standard MKP test problems from OR-Lbrary ssued by [2]. 3.1. Populaton sze and ntal populaton For populaton sze, t s expected that the bgger the populaton s the better the results wll be found. However, a bgger populaton also leads to an addtonal computaton tme per generaton. We wll dscuss the dfferences between populaton sze equals to 100 and 200 later on. We adopt a random ntalsaton scheme n order to acheve suffcent dversfcaton. In the frst nstance, each of the ntal feasble soluton was randomly constructed. However, t s very dffcult to fnd a feasble soluton for those large test problems. We consder t s due to the fact that fndng a feasble soluton s tself NP-hard. Hence, a prmtve heurstc was used where a soluton wth all zeros s ntally generated and a varable s randomly selected repeatedly and set to one f the soluton s feasble. The heurstc termnates whle the selected varable cannot be added to the soluton. The pseudo-code for the ntalsaton as follows (see Algorthm 1). Algorthm 1. Intalse P(0) for the MKP Let: I = {1,, m} and J = {1,, n} ; Let: R = the accumulated resources of constrant n S ; /* S s soluton set*/ for k = 1 to N do set S k [ j] 0, j J ; set R = 0 ; set T J ; /*T s a dummy set*/ randomly select a j T and set T T j ; whle R + rj b, I do set S k [ j] 1; set R R + rj, I ; randomly select a j T and set T T j ; end whle end for
3.2. Selecton operators Parent selecton s the task of assgnng reproductve opportuntes to each ndvdual n the populaton. Two selecton strateges,.e., roulette-wheel selecton and tournament selecton, are mplemented n our experments and the expermental results wll be compared n secton 3. 3.3. Crossover operators The dea behnd the crossover s let offsprngs to nhert some both parents propertes. Two crossover operators are tested n our experment,.e., one-pont crossover and unform crossover. 3.4. Mutaton operator Once a chld soluton has been generated from crossover, a mutaton procedure s performed to change some selected bts n the chld soluton at random. 3.5. Replacement After applyng the above three operators, the new offsprngs have been generated. However, how to replace the old populaton (generaton) wth the new offsprngs s a queston. We adopt the steady-state-noduplcates strategy, whch replaces the worst one of the current populaton by the one new generated ndvdual, and checks that no duplcate offsprngs are added to the populaton. Ths adds the computatonal overhead but more search space s explored. 3.6. Penalty functons It s very lkely that one good soluton mght be generated by the combnaton of two nfeasble solutons. We tested two penalty functons (penalty functon 1 and 3) proposed n [9], and the experment results wll be analysed and dscussed n secton 3. 3.7. Construct the parameters and operators n GAs Table I shows the parameters and strateges used n our experments. Snce there s not one set of parameters that s sutable for all types of problems, we have to test durng the experment n order to fnd out the best set. Some of the parameters and strateges wll be set the same as those showed n the lterature [2, 3, and 8] snce they have been mproved to generate better solutons. 4. Experments and analysss Table. I: Parameters used n GAs parameters / strategy settng populaton sze 100 ntalsaton ntalse algorthm selecton (two optons) roulette wheel tournament (k = 2) crossover (two optons) one pont unform crossover probablty 0.9 mutaton bt flp mutaton probablty 2/n replacement strategy steady-state-no-duplcates stoppng crtera 6 run over 10 generatons mprovements penalty functon, repar operator, local mprovement operator 4.1. Selecton and crossover operators selecton We have conducted two selecton methods and two crossover methods. The am of ths experment s to fnd out one combnaton of these two methods n order to generate the best soluton. Fg. 1 shows the results based on one largest problem. In Fg. 1, the tournament selecton wth unform crossover ncreases fast and produces the best solutons. Hence, ths combnaton wll be chosen n our experment.
Fg. 1: Test case m=30, n=500, problem 29, proft wth dfferent selecton and crossover operators. Fg. 2: Test case m=30, n=500, problem 29, proft wth dfferent populaton szes. 4.2. Populaton sze selecton In ths secton, dfferent populaton szes (100 and 200) are tested based on some of largest problems. The fragment data for proft wth dfferent populaton szes from 10,000 to 1,000,000 generatons are dsplayed n Fg. 2. In Fg. 2, t can be seen that the genetc algorthm wth populaton sze equals to 200 ncreases faster and generates better solutons than those of populaton sze equals to 100. However, there s no sgnfcant gap between these two szes. We decde to choose 100 as the populaton sze to save experment tme. 4.3. Algorthms selecton Three GAs are mplemented n ths experment,.e., the basc GA, GA wth penalty functon 1, and GA wth penalty functon 2. The am of ths secton s to choose one of the best algorthms to solve the MKP. Fg. 3 shows the experment results n soluton qualty base on one small problem. Fg. 3: Dfferent GAs comparson regardng to proft based on problem WEISH13. For small problems, n most cases, classcal GA produces better performance snce t has the fastest ncrease at begnnng and fnds the optmal soluton very early. GA wth penalty functon 1 generally has the worst performance. GA wth penalty functon 2 also performed well n some cases. For example, n Fg. 3, the optmal soluton (proft: 6159) s found by both GA and GA wth penalty functon 2. 5. Case study In ths secton, we use the classcal GA to solve a smple fcttous captal budgetng problem. In order to smulate the real world captal budgetng problems, we add two more constrants to those orgnal constrants n MKP. One s called dependency constrant (e.g., f project 1 s chosen then project 5 should be chosen as well, and vce versa), the other s called exclusve constrant (e.g., f project 1 s selected then project 5 should not be selected, and vce versa). We smply change the problem SENTO1 (n OR-Lbrary, see the WWW address http://people.brunel.ac.uk/~mastjjb/jeb/orlb/fles/mknap2.txt) by addng the above constrants to a real captal budgetng problems. The experment result s shown as follows:
Frstly, before the extra constrants are added to ths problem, we found the well-known soluton values (7772) and the optmal soluton s dsplayed as follows: 01001 00110 10110 10110 10001 01001 00000 01000 00000 11000 00100 10010 Next, the dependency constrant (project 1 and 5 are dependent n solutons) was added. The best soluton values (7703) were found after 10 runs. The best soluton s shown below, where both project 1 and 5 are selected: 11001 00100 00110 10110 10001 01001 00000 11000 00000 11000 00100 10010 Fnally, the exclusve constrant (project 1 and 5 are exclusve n solutons) was added. Below s the optmal soluton (7772) we obtaned. 01001 00110 10110 10110 10001 01001 00000 01000 00000 11000 00100 10010 It s the same as the orgnal optmal soluton, because ths soluton s also feasble n exclusve case. Note that, ths s a smple example showng how to use GAs to solve real world project selecton and captal budgetng problems based on MKP. It s sure that there are more other complcated constrants n real world problems. 6. Summary and future work All three proposed GAs perform well for small problems. Although the best performance s obtaned from the classcal GA, t s hard to say whch s better compared wth others. Hence, further comparson s needed n large problems. Note that, although classcal GA performs well for small problems, some of the optmal solutons are not found. Ths means the performance of classcal GA need to mprove as well. Future work wll be based on testng the performance (both soluton qualty and CPU consumed tme) of GAs wth local mprovement. 7. References [1] Frevlle A., The multdmensonal 0 1 knapsack problem: An overvew, European Journal of Operatonal Research 155 (2004), pp 1 21. [2] Chu P. and Beasley J., A genetc algorthm for the multdmensonal knapsack problem, Journal of Heurstcs 4 (1998), pp 63 86. [3] Radl G.R., An mproved genetc algorthm for the multconstrant knapsack problem, n: Proceedngs of the 5th IEEE Internatonal Conference on Evolutonary Computaton, 1998, pp 207 211. [4] Burke E. and Kendall G., Search Methodologes: Introductory tutorals n optmzaton and decson support technque, chapter 4. 2005, New York: Sprnger. [5] Negnevtsky M., Artfcal ntellgence: A gude to ntellgent systems. 2005, Harlow, England; New York: Addson-Wesley. [6] Holland J.H., Adaptaton n natural and artfcal systems: an ntroductory analyss wth applcatons to bology, control, and artfcal ntellgence. 1975, Ann Arbor: Unversty of Mchgan Press. [7] Mchalewcz Z., A survey of constrant handlng technques n evolutonary computaton Method. In 4th Annual Conference on Evolutonary Programmng. 1995, pp 135-155. [8] Ackeln U., Genetc Algorthms for multple-choce Optmsaton Probems, n European Busness Management School. 1999, Unversty of Wales. [9] Djannaty F. and Doostdar S., A Hybrd Genetc Algorthm for the Multdmensonal Knapsack Problem, Int. J. Contemp. Math. Scences, Vol. 3, 2008, no. 9, pp 443 456.