Sub-Exponential Algorithms for 0/1 Knapsack and Bin Packing

Transcription

1 Sub-Expoetial Algorithms for 0/1 Kapsack ad Bi Packig Thomas E. O Neil Computer Sciece Departmet Uiversity of North Dakota Grad Forks, ND, USA Abstract - This paper presets simple algorithms for 0/1 Kapsack ad Bi Packig with a fixed umber of bis that achieve time complexity p 2 O x where x is the total bit legth of a list of sizes for objects. The algorithms are adaptatios of a method that achieves a similar complexity for the Partitio ad Subset Sum problems. The method is show to be geeral eough to be applied to other optimizatio or decisio problem based o a list of umeric sizes or weights. This establishes that 0/1 Kapsack ad Bi Packig have sub-expoetial time complexity usig iput legth as the complexity parameter. It also supports the expectatio that all NP-complete problems with pseudo-polyomial time algorithms ca be solved determiistically i sub-expoetial time. Keywords: 0/1 Kapsack, dyamic programmig, Bi Packig, sub-expoetial time, NP-complete problems. 1 Itroductio The comparative complexity of problems withi the class NP-Complete has bee a recurrig theme i computer sciece research sice the problems were defied ad cataloged i the early years of the disciplie [2]. I 1990, Stears ad Hut [7] classified a problem to have power idex i if the fastest algorithm that solves it requires 2 O i steps. Assumig that Satisfiability has power idex 1, they argued that the Clique ad Partitio problems have power idex oe-half. Their aalysis is based o two algorithms with time complexity p 2 O x, where x is the legth i bits of the iput represetatios ad p() is a polyomial fuctio of the umber of graph edges (for Clique) or the umber of itegers i the iput set (for Partitio). These results were iterpreted to provide strog evidece that Clique ad Partitio were easier problems tha Satisfiability ad most other NP-Complete problems. I a subsequet study, Impagliazzo, Paturi, ad Zae [3] preseted aother framework for compariso of NPcomplete problems. Istead of adoptig the power idex termiology of Stears ad Hut, they categorized problems based o weakly expoetial ( 2 1 ) or strogly expoetial ( 2 ) lower bouds (assumig that Satisfiability will oe day be prove to be strogly expoetial) ad sub-expoetial ( 2 o ) upper bouds. To avoid icosistecies related to the characterizatio of iput legth, they defied a family of reductios (the Sub- Expoetial Reductio Family) that would allow the complexity measure to be parameterized. This framework tolerated polyomial differeces i the legths of problem istaces, ad there was o complexity distictio amog Clique, Idepedet Set, Vertex Cover or k-sat. These coclusios are ot cosistet with those of Stears ad Hut, where both Clique ad Partitio were easier tha Satisfiability. It is clear that represetatios ad complexity measures for problem istaces play a critical role i complexity aalysis. I classical complexity theory, the complexity measure is the legth of the iput strig. This parameter is formally determied, simply by coutig the bits i the strig. The advatage of usig the formal measure is that it requires o sematic iterpretatio of the iput strig, ad problems with vastly differet sematics ca by grouped together i formal complexity classes. Withi the class NP-complete, we fid that for may problems, the use of simple sematic complexity measures will ot clash with detailed aalysis based o the formal measure. This is geerally true of strog NP-complete problems, where the objects i the iput represetig variables or odes or set elemets ca be umbered (i biary). The umbers are just labels used for idetificatio of the objects. There are other problems i the class, however, where the iput cotais a list of weights or values, ad aalysis based o sematic measures such as the umber of objects versus the sum (or maximum) of the values ca give radically differet results: expoetial time with oe measure, polyomial time with the other. This collectio of problems icludes Partitio, Subset Sum, 0/1 Kapsack, ad Bi Packig, which we will refer to as the Subset Sum family. The safest approach to aalysis of these problems is to use the formal complexity measure, which icorporates both relevat sematic parameters, ad i this paper we show that the Subset Sum family of pseudopolyomial-time problems is 2 O x (which is subexpoetial).

2 Stears ad Hut [7] were apparetly the first to demostrate that a algorithm for the Partitio problem exhibits sub-expoetial time. The sigificace of this result was probably obscured by the claim i the same paper that the Clique problem is also sub-expoetial, while its dual problem Idepedet Set remais strogly expoetial. This apparet aomaly is a represetatio-depedet distictio, ad it disappears whe a symmetric represetatio for the problem istace is used [5]. The complexity distictio betwee Partitio ad Satisfiability, however, appears to have stroger credibility. I [6] it is show that the sub-expoetial upper boud for Partitio is also valid for Subset Sum. The algorithm for Subset Sum is a variat of dyamic programmig that is much simpler ad more geeral tha the backtrackig/dyamic programmig hybrid that Stears ad Hut desiged for Partitio. I this paper, the sub-expoetial Subset Sum algorithm is adapted to 0/1 Kapsack ad Bi Packig with a fixed umber of bis, establishig that these problems are also subexpoetial with respect to the formal complexity measure (total bit-legth of iput, deoted x). We also abstract from the previous methods a lemma that idetifies the property of ordered sets of itegers that is exploited to achieve subexpoetial time. More recet complexity studies i the research literature for problems i the Subset Sum family do ot typically use the iput legth as the complexity parameter. The curret upper boud for both Subset Sum ad 0/1 Kapsack is apparetly 2 O /2 whe the umber of objects i the list is used as the complexity measure [8]. A lower boud of 2 /2 / for Kapsack has also bee demostrated i [1]. The lower boud applies oly to algorithms withi a model defied geerally eough to iclude most backtrackig ad dyamic programmig approaches. The sub-expoetial bouds derived here usig the formal complexity measure complemet rather tha supersede the strogly expoetial bouds derived usig the umber of objects i the iput list (deoted ) as the complexity parameter. 2 Geeralized Dyamic Programmig The Stears ad Hut algorithm for Partitio [7] combies backtrackig with dyamic programmig. Such hybrid approaches had bee previously described i operatios research literature (e.g. [4]). The iput set is ordered ad divided ito a deser ad a sparser subset. Backtrackig is employed o the sparse subset, while dyamic programmig is used for the dese subset. The results are combied to achieve time complexity 2 O x, where x is the total legth i bits of the iput. I this paper we employ a simpler algorithm that achieves the same goal. The approach was first developed for Subset Sum ad Partitio [6]. Similar to covetioal dyamic programmig, it represets a breadth-first eumeratio of partial solutios. The problem istace is a list of objects, each of which has a size. The algorithm maitais a pool of partial solutios as it processes each object. The list of objects is ordered by size, ad the largest objects are processed first. I cotrast with covetioal dyamic programmig, the pool of solutios is dyamically allocated (hece the acroym DDP, for dyamic dyamic programmig). It first grows ad the shriks as more objects are processed. The etire pool of solutios is traversed for each object, updatig each solutio by possibly subtractig the curret object's size from the solutio's remaiig capacity. Each solutio is also evaluated relative to the sum of sizes of the objects yet to be processed. The sum of remaiig sizes ca be used to prue the pool of solutios depedig o problem sematics. This pruig relative to the sum of sizes of the uprocessed objects places a sub-expoetial upper boud o the umber of partial solutios i the pool. The time aalysis of the DDP method relies o a simple lemma (abstracted from the aalysis i [6]) that allows us to boud the k th value i a ordered list as a fuctio of its positio i the list ad the total bit-legth of the etire list (see Lemma 1 below). Boudig the k th value allows us to boud the sum of the first k values as well. This, i tur, leads to a boud o the legth of the pool of partial solutios i DDP algorithms. Lemma 1: Let L represet a list of positive atural umbers i o-decreasig order, let L[k] represet the k th umber i the list, let b k be the bit legth of the k th umber, ad let b be total umber of bits i the etire list: b = b i = 1 lg L[i]. The L[k] < 2 (b k+1)/( k+1)+1. Proof: A upper boud o the value of L[k] for ay list with total bit legth b is obtaied by reservig as few bits as possible for the smaller umbers i the list ad as may bits as possible for L[k] ad the umbers that follow it. This is accomplished by settig L[1] through L[k-1] to 1 ad distributig the remaiig bits equally amog the higher k + 1 umbers. I that case, L[k] has o more tha (b k+1)/ ( k+1) bits, establishig L[k] < 2 (b k+1)/( k+1)+1. 3 The Kapsack Problem The 0/1 Kapsack problem is defied as follows: give a set of objects S with sizes s[1..] ad values v[1..], fid a subset of objects with the highest value whose size is less tha or equal to C, the capacity of the kapsack [2]. The problem ca also be expressed as a decisio problem, where we determie the existece of a subset whose value is greater tha or equal to a target value V.

3 //* Give a set of objects whose sizes are specified i a array s[1..] i o-decreasig order ad whose values are stored i a array v[1..], fid the highest valued subset whose total size is less tha or equal to capacity C. */ public it Kapsack() 1) bestval 0; 2) sizeofrest 3.1 The Kapsack algorithm s[ i] ; valueofrest v[i] ; 3) Pool {(C, 0)}; 4) for i 1 to 5) size s[ i +1]; value v[ i +1]; 6) NewList { }; 7) for each sack i Pool 8) if (sack.capacity < size ) 9) cotiue; 10) else if (sack.capacity > sizeofrest) 11) bestval max (bestval, sack.value + valueofrest) 12) remove sack from Pool; 13) else 14) bestval max (bestval, sack.value + value); 15) NewList.apped ((sack.capacity size, sack.value + value)); ed for 16) sizeofrest sizeofrest size; valueofrest valueofrest value; 17) Pool merge(pool, NewList); ed for 18) retur bestval; Figure 1. The Kapsack algorithm. I adaptig the DDP method to the Kapsack problem, we ca iterate either the size or the value array as the cotrol for the outer loop. Here we use the size array. The algorithm keeps a pool of (capacity, value) pairs represetig partially filled kapsacks, iitially cotaiig a empty sack represeted as (C, 0), where C is the capacity of the empty sack. For each object i S ad for each sack curretly i the pool, we add a ew sack represetig the curret sack plus the curret object. This is accomplished by subtractig the object size from the sack's remaiig capacity ad addig the object value to the sack's value. Pseudo-code for the Kapsack algorithm is show i Figure 1. Lies 1-3 iitialize the global Pool, the bestval variable, ad variables represetig the cumulative size ad value of the remaiig objects. There is oe iteratio of the outer for loop (lies 4-17) for each object i the set S = {y 1, y 2,..., y }. The size array s, i which s[i] is the size of object y i, is assumed to be i o-decreasig order, ad the largest umbers are processed first, so object y i+1 is processed durig the i th iteratio. The pool of partially filled sacks is updated by the ier for loop (lies 7-15). For each sack i the pool, s[ i+1] is subtracted from its capacity ad v[ i+1] is added to its value, placig the ew (capacity, value) o a secod ordered sack list. The pool ad the ew sack list are merged i the last step of the outer loop (lie 17). The best value for a filled sack is updated whe appropriate i lies 11 ad 14, wheever a updated sack is created. At completio of the outer loop, the best value is retured. The algorithm does ot retur the cotets of the sack with the best value, but this could be accomplished by addig a referece to a subset object to the (capacity, value) pairs i the pool, icreasig the time complexity by o more tha a factor of. The ier loop has two coditios that moderate the legth of the pool. Lies 8 ad 9 skip sacks that ca't hold the curret object. Also, i lies 10-12, sacks with eough capacity to hold all remaiig objects are removed from the pool after updatig the bestval variable. If all remaiig objects will fit i a sack, there is o process them oe-byoe. The outer loop also has logic to cotrol the size of the pool. The last step i the outer loop is a sequetial merge operatio that adds the ew partially filled sacks to the pool. If two sacks with the same capacity are ecoutered durig the merge, oly the sack with the higher value is added to the pool. Thus the capacities of all sacks i the pool are uique. 3.2 Time Aalysis of Kapsack The time aalysis closely follows the method used for the Subset Sum algorithm i [6]. Let S = {y 1, y 2,..., y } ad assume the sizes are stored i o-decreasig order (s[i] s[i+1]). The total umber of steps is determied by the size of Pool. With each iteratio of the outer for loop, Pool is traversed ad possibly exteded (requirig 2 passes oe by the ier for loop ad the other by the sequetial merge step). The total amout of work is closely estimated (withi a factor of 2) by Pool i (1) where Pool i is the legth of Pool at the begiig of outer loop iteratio i. Sice the merge operatio elimiates duplicatio of capacities, we ca describe legth of Pool(i) as at most MaxC(i), the largest capacity of ay sack o the list at the begiig of iteratio i. The list is actually smaller tha this, sice all the capacities betwee zero ad the maximum are ot preset. We also kow that the legth of the list ca, at most, double with each loop iteratio, so regardless

4 of the maximum value i the list, its legth caot exceed 2 i. This gives us Pool i mi 2 i, MaxC i. (2) The legth of Pool will grow rapidly ad later possibly shrik as i approaches. Our goal is to fid a upper boud for MaxC(i). Iitially MaxC(1) = C, which is the capacity of the empty sack. Oly smaller-capacity sacks are added to the list, ad evetually the larger-capacity sacks are removed whe the coditio i lie 10 becomes true, so i 1 MaxC i s [ j] i 1 s[ i 1]. (3) j=1 Boudig MaxC(i) thus reduces to fidig a upper boud for s[ i + 1], ad Lemma 1 is ivoked for this purpose. To complete the aalysis, we boud the step couts as a fuctio of b, the total bit legth of the size array s. We cosider two cases. Case 1. b. Here we have Pool i mi 2 i, MaxC i 2 b. (4) Case 2. > b. I this case we split the summatio at i = b. b 1 Pool i mi 2 i, MaxC i (5) mi 2 i, MaxC i mi 2 i,maxc i (6) i= b b 1 2 b 1 mi 2 i, MaxC i (7) i= b b 1 2 b 1 b 1 MaxC b (8) b 1 b 1 2 b 1 b 1 j=1 s[ j]. (9) b 1 2 b 1 b 1 2 s[ b 1] (10) At this poit, we employ Lemma 1 to compute the boud for s[k] where k= b 1, ad we cotiue by replacig s [ b 1] with 2 b 1 : b 1 2 b 1 b b 1 (11) b b 1 (12) b. (13) This establishes that the time complexity of Kapsack is O p 2 b for a polyomial fuctio p(). The argumet b is the total bit legth of the list of sizes. The etire iput for the problem also icludes the capacity C ad a list of values. We ca't make ay specific assumptios about the relative magitudes of the sizes ad values, but we are certai that if x is the total iput legth, the b will be smaller tha x, ad the O p 2 b step cout will also be O p 2 x. 4 The Bi Packig Problem The Bi Packig problem is defied as follows: give a set of objects S with sizes s[1..], determie whether the objects will fit ito a fixed umber of k bis, each with a capacity of B. The problem ca also be expressed as a optimizatio problem i which the smallest B is determied [2]. Whe B is equal to the sum of all sizes divided by k, the problem represets a geeralizatio of the Partitio problem. /* Give a set of objects whose sizes are specified i a array s[1..] i o-decreasig order, determie whether all objects ca be stored i k bis, each with capacity B. */ public boolea BiPack() 1) sizeofrest = s[i] ; 2) Pool = {(B, B,, B)}; 3) for i 1 to 4) extsize s[ i +1]; 5) NewList { }; 6) for each bituple i Pool 7) if (bituple.capacity[1] < extsize ) 8) cotiue; 9) else if (bituple.capacity[1] > sizeofrest) 10) retur true; 11) else 12) for j 1 to k 13) ewtuple update(bituple, j, extsize); 14) if (ewtuple!= ull) 15) NewList.isert (ewtuple); ed for ed for 16) Pool NewList; 17) sizeofrest sizeofrest extsize; ed for 18) retur false; Figure 2. The BiPack algorithm. 4.1 The BiPack Algorithm Whe we adapt the DDP strategy to Bi Packig, we fid a few sigificat differeces from the Kapsack versio. The BiPack algorithm is show i Figure 2. The pool of partial solutios must be a list of k-tuples, where each compoet of a tuple is the remaiig capacity of oe of the bis (see lie 2). Also, we are ot searchig for a subset. All the objects i the origial set S must be icluded i the solutio. This has implicatios for the logic i the ested loops of the algorithm. Ay partial solutio i the ier loop that caot accommodate the ext object ca be

5 discarded (lies 7-8), ad the pool of updated partial solutios created by the ier loop replaces the pool from the previous iteratio of the outer loop (rather tha mergig with the previous pool; see lie 16). We also fid that the test eforcig the upper limit o the size of the pool (relative to the sum of the remaiig object sizes) triggers early termiatio (lies 9-10). This versio of the algorithm does ot specify what objects are placed i what bis, but this iformatio could be icluded by associatig a referece to a size object to each partial solutio. This would icrease the time complexity by o more tha a factor of. 4.2 Time Aalysis of BiPack The time aalysis of BiPack follows the same geeral logic as the aalysis for Kapsack. The major differece is the growth rate of the pool of partial solutios. While the pool ca double i legth with each iteratio of the ier loop i Kapsack, it ca icrease i legth by a factor of k i BiPack. Aother sigificat differece is the cost of suppressig duplicates i the pool of partial solutios. We make the coservative assumptio that the isertio of a updated partial solutio i the pool takes liear time i the curret legth of the pool. We demostrate below that i spite of these sigificat differeces, the time complexity of the algorithm remais sub-expoetial. To proceed with the aalysis, let S = {y 1, y 2,..., y }, ad assume the sizes are stored i o-decreasig order (s[i] s[i+1]). As with Kapsack, The total umber of steps is closely related to the size of Pool. With each iteratio of the outer for loop, Pool is traversed ad replaced with a updated versio (called NewList). Each isertio ito NewList requires liear time. The total amout of work is therefore estimated as Pool i 2 (14) where Pool i is the legth of Pool at the begiig of outer loop iteratio i. Sice the isert operatio of lie 15 elimiates duplicatio of capacities, we ca describe legth of Pool(i) as at most MaxC(i) k. If MaxC(i) is the largest capacity of ay bi i ay tuple o the list at the begiig of iteratio i, the umber of distict tuples caot exceed this quatity raised to the power k. This grossly overestimates the umber of tuples, sice the capacities withi each tuple are i o-icreasig order ad sice all the tuples have the same sum. It is a iterestig coutig problem to determie a tight upper boud for the umber of tuples, but the loose boud is sufficiet to establish the desired complexity result. We also kow that the legth of the list ca, at most, grow by a factor of k with each loop iteratio, so regardless of the maximum value i the list, its legth caot exceed k i. This gives us Pool i mi k i, MaxC i k. (15) Lies 9 ad 10 assure us that the algorithm termiates if MaxC(i) exceeds the sum of the remaiig object sizes, so we have i 1 MaxC i s [ j] i 1 s[ i 1]. (16) j=1 To complete the aalysis, we boud the step couts as a fuctio of x, the total bit legth of the size array s. As before, we cosider two cases. Case 1. x. Here we have Pool i 2 mi k i,maxc i k 2 (17) k 2 k 2 x 2 2lg k x. (18) Case 2. > x. I this case we split the summatio at i = x. Pool i 2 mi k i,maxc i k 2 (19) x 1 mi k i, MaxC i k 2 i= x mi k i, MaxC i k 2 (20) x 1 k x 1 x 1 MaxC x k 2 (21) The by Lemma 1: x 1 k x 1 x 1 x 1 2 x 1 2k (22) ad by algebraic simplificatio: 2k 1 2 2k x 1 (23) Sice k is a costat, this establishes that the time complexity of BiPack is p 2 O x for a polyomial fuctio p(). 5 Coclusio The algorithms i the previous sectios demostrate that dyamic programmig with dyamic allocatio (DDP) ca be used to prove that 0/1 Kapsack ad Bi Packig with a fixed umber of bis have time complexity p 2 O x where x is the total bit legth of iput umbers. This places these problems with Partitio ad Subset Sum i the subclass of NP-complete problems that have sub-expoetial upper bouds o ruig time, whe iput legth is used as the complexity parameter. The Kapsack problem was formulated as a optimizatio problem above, while Bi Packig was preseted as a decisio problem. It is apparet that the Kapsack algorithm ca be modified to solve the decisio versio of the problem without chagig its time complexity. It is also possible to modify BiPack to fid the smallest bi capacity eeded to store all objects i k bis, as log as k is costat, without chagig its time complexity. Give the simplicity ad geerality of Lemma

6 1, which provides the foudatio for the time aalyses, we expect that the DDP method ca be applied to ay NPcomplete problem ivolvig a list of weighted objects that has pseudo-polyomial time complexity. Refereces [1] M. Alekhovich, A. Borodi, J. Buresh-Oppeheim, R. Impagliazzo, A. Mage, ad T. Pitassi, Toward a Model for Backtrackig ad Dyamic Programmig, Proceedigs of the 20 th Aual IEEE Coferece o Computatioal Complexity, pp (2005). [2] M. Garey ad D. Johso, Computers ad Itractability: A Guide to the Theory of NP- Completeess, Freema Press, Sa Fracisco, CA (1979). [3] R. Impagliazzo, R. Paturi, ad F. Zae, Which Problems Have Strogly Expoetial Complexity?, Joural of Computer ad System Scieces 63, pp , Elsevier Sciece (2001). [4] S. Martello ad P. Toth, A mixture of dyamic programmig ad brach-ad-boud for the subset sum problem, Maagemet Sciece 30(6), pp (1984). [5] T. E. O'Neil, The Importace of Symmetric Represetatio, Proceedigs of the 2009 Iteratioal Coferece o Foudatios of Computer Sciece (FCS 2009), pp , CSREA Press (2009). O [6] T. E. O'Neil ad S. Kerli, A Simple x 2 Algorithm for Partitio ad Subset Sum, Proceedigs of the 2010 Iteratioal Coferece o Foudatios of Computer Sciece (FCS 2010), pp , CSREA Press (2010). [7] R. Stears ad H. Hut, Power Idices ad Easier Hard Problems, Mathematical Systems Theory 23 (1990), pp [8] G. J. Woegiger, Exact Algorithms for NP-Hard Problems: A Survey, Lecture Notes i Computer Sciece 2570, pp , Spriger-Verlaug, Berli (2003).