Implementing Dynamic Programming Recurrences in Constraint Handling Rules with Rule Priorities

Transcription

1 Implemetig Dyamic Programmig Recurreces i Costrait Hadlig Rules with Rule Priorities Ahmed Magdy 1, Frak Raiser, ad Thom Frühwirth 1 Computer Sciece ad Egieerig, Germa Uiversity i Cairo ahmed.mabrouk@studet.guc.edu.eg Istitute of Software Egieerig ad Compiler Costructio, Uiversity of Ulm {frak.raiser,thom.fruehwirth}@ui-ulm.de Abstract. Dyamic Programmig (DP) is a importat techique used i solvig optimizatio problems. A close correspodece betwee DP recurreces ad Costrait Hadlig Rules with rule priorities (CHR rp ) yields atural implemetatios of DP problems i CHR rp. I this work, we evaluate differet implemetatio techiques with respect to their rutime. From our results we derive a set of guidelies for implemetig arbitrary DP problems i CHR rp. 1 Itroductio Costrait Hadlig Rules (CHR) ([1]) is a simple high-level programmig laguage that is embedded i a host laguage. It combies the elemets of Costrait (Logic) Programmig ad rule-based laguages ([]). It was foud for the purpose of embeddig user-defied costrait solvers i the host laguage. It evolved through the years to become a more powerful geeral-purpose programmig laguage ([1]). A extesio of CHR was itroduced called Costrait Hadlig Rules with Rule Priorities (CHR rp ) ([3]). It gives CHR rules priorities to cotrol the executio order. It simplifies implemetig rule-based algorithms because these algorithms ofte require some rules to be tried before others for efficiecy or correctess reasos ([4]). A importat coclusio, reached by [5], implies that every algorithm ca be implemeted i CHR with the best-kow time ad space complexity. Aother coclusio, reached by [6], states that the efficiecy of the CHR rp implemetatio comes close to the state-of-the-art K.U.Leuve CHR system (ad sometimes eve surpasses it). Dyamic Programmig (DP) is a importat techique used i solvig optimizatio problems. DP follows the optimal substructure property where a solutio of a large problem cotais withi it optimal solutios to smaller subproblems. There is correspodece betwee Dyamic Programmig recurreces ad CHR rp rules that we will exploit i this paper.

2 I this paper, a implemetatio to Matrix Chai Multiplicatio DP problem is itroduced ad evaluated. Differet modificatios to the previous problem are proposed ad evaluated i order to improve the rutime. Other modificatios are tested o four DP problems which are Kapsack ([7]), Edit Distace ([8]), Viterbi ([9]), CYK ([1]). The guidelies are proposed that help i implemetig DP problems i CHR rp. These guidelies are formulated based o the evaluatios doe o the five DP problems. Prelimiaries This sectio reviews CHR rp sytax ad sematics. For a more thorough itroductio, see [3] or [6]. This sectio also reviews dyamic programmig. For a more thorough itroductio, see [11]..1 Costrait Hadlig Rules with Rule Priorities Sytax. A costrait is of the form c(t 1,..., t ) with arity. Each t k is a value defied by the host laguage. There are two types of costraits: builti costraits ad CHR costraits. The host laguage provides data types ad pre-defied costraits which are called the built-i costraits. CHR costraits are also called user-defied costraits ad are solved by the CHR rp program. CHR rp programs cosist of three types of rules show below. Simplificatio H r <=> g B pragma priority(p). Propagatio H k ==> g B pragma priority(p). Simpagatio H k \ H r <=> g B pragma priority(p). where r represets a optioal rule ame. H r ad H k are called heads of the rule ad represet oe or more CHR costraits. g is called the guard of the rule ad it is zero or more built-i costraits that must be satisfied to apply the rule. B is the body of the rule. It s a sequece of CHR costraits ad built-i costraits. p ca be a static umber or a arithmetic expressio. The variables i the arithmetic expressio ca oly be from the variables i the head costraits of the correspodig rule. Sematics. A costrait store is a multiset of costraits. Iitially, the store starts with the costraits of the iitial query. The, the rules of the CHR rp program are applied, which maipulates the cotets of the costrait store. A rule is applied if its head costraits are i the costrait store ad the rule s guard holds. The body of the applied rule is added to the costrait store. The differece betwee the three rules is i the way the program deals with the head costraits if the rule fires. I simplificatio rules, head costraits are removed from the costrait store. I propagatio rules, head costraits are kept i the costrait store. Simpagatio rules combie the actios of propagatio ad simplificatio rules: head costraits before the backslash are kept i the costrait store, ad head costraits after the backslash are removed from the costrait store ([1]).

3 Joi Orderig ad Idexig. Fidig parter costraits to match a rule is crucial i CHR. Joi orderig is the order i which these parter costraits are looked up. The time complexity for that operatio should be miimized to obtai the optimal complexity. The time complexity of a program depeds o the joi orderig ([1]). Therefore, the lookup of parter costraits should be efficiet. Idexig is a importat approach that decrease the lookup time. There are several approaches for idexig. The traditioal oe uses attributed variables for costat lookup time ([1]).. Dyamic Programmig Dyamic programmig (DP) is a importat techique used i solvig may optimizatio problems. Optimizatio problems are the kid of problems that have several solutios but oe of them is better tha the others. The idea of dyamic programmig is that some itermediate computatio may be repeated due to overlappig sub-problems. Therefore, itermediate results are stored ad reused whe ecessary. DP exhibits the optimal substructure property. A problem exhibits this property whe the optimal solutio of the large problem cotais withi it optimal solutios of its smaller sub-problems ([11]). Matrix Chai Multiplicatio. It is the problem of multiplyig matrices. The order by which matrices are multiplied affects the umber of scalar multiplicatios. To multiply three matrices, there are two ways: (A 1 A )A 3 ad A 1 (A A 3 ). Assume A 1 has dimesios 1 1, A has dimesios 1 5, ad A 3 has dimesios 5 5. The first optio will result i = 75 }{{}}{{} A 1 A (A 1A ) A 3 multiplicatios. The secod optio will result i = }{{}}{{} A A 3 A 1 (A A 3) 75. DP ca be used to fid the optimal umber of scalar multiplicatios to multiply a chai of matrices. The the optimal order ca be calculated. The umber of scalar multiplicatios ca be computed usig (1). m(i, j) = { i = j mi i<k j {m(i, k) + m(k + 1, j) + p i 1 p k p j } i < j (1) where m(i, j) represets the cost of multiplyig matrices from matrix i to matrix j, p i the secod dimesio of matrix i, ad p the first dimesio of the first matrix 3. The optimal umber of scalar multiplicatios ca be computed by calculatig m(1, ), where is the umber of matrices to multiply. The DP algorithm rus i O( 3 ) time complexity ad O( ) space complexity ([11]). 3 The first dimesio of matrix i must be equal to the secod dimesio of matrix i 1

4 3 Implemetatio I this sectio, a implemetatio of the Matrix Chai Multiplicatio is itroduced. Costrait p/ represets the matrices dimesios. The first attribute is the idex, ad the secod oe is the dimesio. It resembles the p i i (1). The umber of matrices is stored i umofmatrices/1 costrait as its oly attribute. The costrait that holds itermediate calculated values is called cell/3. The first ad secod attributes are the same as the first ad secod argumets of (1). The third oe represets the value calculated. The recursive part i the recurrece is represeted as a propagatio rule at priority two. cell(i,k,c1), cell(k1,j,c), p(i,p1), p(k,p), p(j,p3), curretsize(d) ==> K1=:=K+1, D=:=J-I, I=:=I-1 R is C1+C+P1*P*P3, cell(i,j,r) pragma priority(). The head costraits of the rule (without curretsize(d) costrait) resembles the right had side of (1). The body of the rule resembles the left had side of (1). Costraits matched with the propagatio rules that represet the recurrece should cotai optimal values. Usig a o-optimal value will add to the rutime. Therefore, a ew costrait (curretsize/1) is defied to esure that the costraits matched are optimal. The attribute of the costrait represets the size of sub-problems that ca be solved by the propagatio rule. The attribute starts with the smallest possible sub-problem ad is icremeted util the whole problem is solved. m(i, j) calculates the umber of scalar multiplicatios to multiply matrix i to matrix j which are j i matrices. This differece is a idicatio to the size of the problem. The problem m(i, j) depeds o sub-problems m(i, k), m(k + 1, j), where k varies from i + 1 to j. This implies that both sub-problems are smaller i size tha the m(i, j) problem sice k i j i ad j k + 1 j i. The attribute of the curretsize costrait is icremeted at priority three after all cell costraits from the propagatio rules are geerated. Iitially, curretsize s attribute is set to zero. This satisfies the base case of the recurrece whe j i =. At the base case, m(i, j) is set to zero. Therefore, cell(i,i,) costraits are added to the store. Selectig the optimal choice (miimum umber of scalar multiplicatios) is doe by a simpagatio rule at the highest priority. After the program fiishes computatios, the result will be C i the costrait cell(1,x,c), where X is the umber of matrices. 4 Evaluatio I this sectio, the iitial implemetatio of the Matrix Chai Multiplicatio is evaluated. Furthermore, two modificatios to the iitial implemetatio are

5 proposed to improve the rutime. These modificatios are the evaluated to determie their effect. Two other modificatios are evaluated o Kapsack s problem implemetatio. The evaluatios are performed o a Dual-Core processor. GHz machie usig SWI-Prolog Iitial Implemetatio Hypothesis. The Matrix Chai Multiplicatio implemetatio rus i O( 3 ) Test. The implemetatio is tested o sizes of the problem. For each size, te trials are radomly geerated. Average time betwee the te trials is calculated ad plotted. Smallest test case is two matrices ad the icremet betwee each set ad the ext is oe matrix. The geerated graph is show i Fig CHR^rp f(x) = x^5 / Fig. 1. Matrix Chai Multiplicatio i CHR rp. Result. The geerated graph ca be approximated by a polyomial of 5 th order. Therefore, a factor of is added to the rutime. 4. Replacig Guards with Costraits Guards are ot idexed. Replacig the guards of the form A=:="arithmetic expressio" with costraits ca lead to idexig of these costraits ad improvig the rutime i fidig the matchig parter costraits. Hypothesis. Replacig guards from the propagatio rules that correspods to the recurrece with costraits improves the rutime.

6 Test. Test cases from the previous bechmark are used i this test case to compare the results. The geerated graph is plotted agaist the previous graph i Fig. 14 Iitial implemetatio Guarded costraits Fig.. Guards vs. Guard costraits. Result. icreased by a factor ragig from 1.1 to 1.. Therefore, the hypothesis is ot true. 4.3 Editig Order of Costraits The order of occurrece of costraits i a rule ca ifluece the order of matchig parter costraits. If the compiler matches parter costraits i the order as they appear i the rule, the there will be more tha a costat factor i the complexity to fid the matchig costraits. Cosider the followig example: x(i), y(j), z(i,j) <=>... where the umber of y costraits is big relative to x ad z costraits. If x(i) is the costrait that tries to fid parter costraits, the it will try to match y(j). Therefore, it will try several y(j) till it reaches the oe that matches with z(i,j). Therefore, i the worst case, the umber of y costraits will be added to rutime. The factor that icreased i the rutime i the previous implemetatio ca be because of the joi orderig where the umber of head costraits icreased ad they are ot ordered. Therefore, the order of costraits i the recurrece rules is edited so that the rule starts with the curretsize costrait, followed by a costrait that has the same attribute of the curretsize costrait, followed by a costrait that have commo attributes with the previous costraits ad so o.

7 Hypothesis. Orderig costraits i the recurrece rules improves the rutime. Test. This modificatio is tested o the implemetatio itroduced i subsectio 4. because this implemetatio cotais costraits that share may attributes where idexig will be applied. Therefore, by applyig this modificatio to that implemetatio, the differece i rutime will be observable. The geerated graph is plotted i Fig.3.85 Ordered costraits f(x) = x^3 / Fig. 3. Ordered costraits i Matrix Chai Multiplicatio. Results. The rutime is improved by a factor of. Therefore, the hypothesis is true. 4.4 Other Evaluatios Two modificatios are evaluated o the Kapsack problem. The first oe is typig costraits attributes to groud. This modificatio improved the rutime by a factor early equal to 1.. The secod modificatio is typig costraits attribute to iteger. This modificatio resulted i the same rutime as typig costraits to groud. 4 5 Implemetig DP problems i CHR rp I this sectio, guidelies are itroduced that help i implemetig DP recurreces i CHR rp. 4 Geerated graphs are show i Appedix A.

8 Guidelie 1. DP depeds o reusig previously calculated values. Therefore, costrait cell(+,+,+) is defied to store these values. The first two attributes represet the argumets of the DP recurrece. The last attribute represets the value calculated. Typig costrait s attribute to groud improved rutime as observed i subsectio 4.4 Guidelie. Optimizatio Rule is defied to make the optimal choice ad remove the o-optimal costrait from the costrait store. It is defied at priority oe with a simpagatio rule to drop the o-optimal costrait. Some DP problems do ot have a optimizatio rule. The CYK ([1]) problem is such a example. The optimizatio rule looks like : cell(x,y,z1) \ cell(x,y,z) <=> "optimizatio choice" true pragma priority(1). Guidelie 3. Recurrece Rules are defied to perform the actual computatio of the DP recurrece ad geerate cell costraits. The recurrece rules are propagatio rules. This esures that previously calculated results remai i the store to be further used. They are defied at priority two. Their head costraits represet the right had side of the recurrece, while their body represet the left had side of the recurrece. Due to the ucertaity of the order of applicatio of the recurrece rules (sice all recurrece rules have the same priority), a o-optimal costrait of a smaller sub-problem could be used to solve a bigger oe before optimizig the smaller sub-problem. That would cause to use o-optimal values which would lead to uecessary computatios ad adds to the rutime. Hece, explicit cotrol of the order i which rules are fired is used to esure that the values used are optimal. Guidelie 4. A curretsize(+) costrait is defied to eforce explicit cotrol of the order i which rules are fired. Its attribute represets the size of the sub-problems that ca be solved by the recurrece rules ad hece, is problem specific. Not all problems require that kid of cotrol. For a problem like the CYK ([1]) problem, there is o optimal choice made. Therefore, curretsize costrait is ot used for the CYK implemetatio. The attribute of the curretsize costrait is icremeted whe all the subproblems of that size are solved. It is icremeted at priority 3 with a simpagatio rule of the form : maximum(x) \ curretsize(y) <=> Y<=X NextY is Y+1, curretsize(nexty) pragma priority(3). where maximum s attribute represets the maximum size of the problem.

9 Guidelie 5. If the recurrece rule requires a guard of the form A=:="arithmetic expressio", the replace it with a costrait that satisfies it. cell(a,b,x), cell(c,d,y) ==> A=:=B+C... is trasformed to : cell(a,b,x), cell(c,d,y), guardsum(a,b,c) ==>... This is doe, because whe the compiler idexes the costraits it does ot iclude the guards i the idexig. By replacig it with a costrait, it is idexed. The guardsum costraits are added to the store whe costraits they will deped o are added to the store. The overhead of geeratig the guardsum costraits is liear i the umber of costraits they deped o because for each costrait geerated there is at least oe costrait that matches it. Hece, geeratig the costraits itroduces a costat factor oly to the overall complexity. It was observed i subsectio 4. that this modificatio icreased the rutime. However, orderig the costraits after applyig this guidelie improved the rutime by more tha a costat factor. Guidelie 6. The order of head costraits withi a rule is sigificat. The compiler tries to match costraits i the order they appear i the rule as observed i subsectio 4.3. Hece, to make use of idexig of costraits, the recurrece rule starts with the curretsize costrait, followed by a costrait that has the same attribute of the curretsize costrait, followed by a costrait that has commo attributes with the previous costrait, ad so o. Guidelie 7. Result Rules are defied to get the result ad prit it. They are defied at priorities four ad five. Two priorities are used because the result of some DP problems is the maximum or miimum of several values. Therefore, the value is calculated at priority four ad prited at priority five. These guidelies are applied o the five DP problems ad the evaluated. All geerated graphs could be approximated to the complexities of the correspodig problems 5. 6 Coclusio Five DP problems were implemeted the evaluated. Modificatios were proposed ad evaluated i order to improve the rutime. The, guidelies followig the five implemetatios ad modificatios are itroduced that helps i implemetig DP recurreces i CHR rp. 5 Geerated graphs are show Appedix B

10 Future Work. Guidelies itroduced are tested o DP problems that have a recurrece cosistig of two attributes oly. Tests should be doe to see if these guidelies ca be exteded o DP problems that have recurreces cosistig of three or more attributes. curretsize costrait is problem specific. However, it is related to the recurrece formulatio. The size of the implemeted problem was defied by reasoig but o direct mappig from the recurrece was itroduced. A possible research is to fid the attributes that cotrol the size of the problem directly from the recurrece. Space complexity was ot ivestigated i this paper. Modificatios to the implemetatios ca be researched to reach the optimal theoretical space complexity. Theoretical complexity aalysis ca be doe to prove that this formulatio is i the optimal time complexity. Related Work. A Viterbi implemetatio was proposed by [1] ad bechmarked. This implemetatio was the exteded ad optimized to achieve the theoretical best time complexity. This paper was published so recetly that comparisos betwee our results ad their results could ot be doe. Refereces 1. Seyers, J., Va Weert, P., Schrijvers, T., De Koick, L.: As time goes by: Costrait Hadlig Rules A survey of CHR research betwee 1998 ad 7. TPLP 1(1) (1) Schrijvers, T.: Aalyses, optimizatios ad extesios of Costrait Hadlig Rules. PhD thesis, K.U.Leuve, Belgium (5) 3. De Koick, L., Schrijvers, T., Demoe, B.: User-defiable rule priorities for CHR. I Leuschel, M., Podelski, A., eds.: PPDP 7, ACM Press (7) Gabbrielli, M., Meo, M.C., Mauro, J.: O the expressive power of priorities i CHR. I López-Fraguas, F., ed.: PPDP 9, ACM Press (9) Seyers, J., Schrijvers, T., Demoe, B.: The computatioal power ad complexity of Costrait Hadlig Rules. ACM TOPLAS 31() (9) 6. De Koick, L., Stuckey, P.J., Duck, G.J.: Optimized compilatio of CHR rp. Techical Report CW 499, K.U.Leuve, Dept. Comp. Sc., Leuve, Belgium (7) 7. Martello, S., Toth, P.: Kapsack Problems: Algorithms ad Computer Implemetatio. Joh Wiley ad Sos (199) 8. Wager, R.A., Fischer, M.J.: The strig-to-strig correctio problem. J. ACM 1(1) (1974) Rabier, L.R., Juag, B.H.: A itroductio to hidde markov models. 3(1) (1986) Sipser, M.: Itroductio to Theory of Computatio. d ed. Thomso Course Techology (6) 11. Corme, T.H., Leiserso, C.E., Rivest, R.L., Stei, C.: Itroductio to Algorithms. d ed. McGraw-Hill Sciece / Egieerig / Math (3) 1. Christiase, H., Have, C.T., Lasse, O.T., Petit, M.: The viterbi algorithm expressed i Costrait Hadlig Rules. I Va Weert, P., De Koick, L., eds.: CHR 1, K.U.Leuve, Dept. Comp. Sc., Techical report (1) To appear.

11 A Kapsack Modificatios A.1 Groud Typig versus Not Typig Hypothesis Typig variables to groud is more efficiet. Test This optimizatio is tested o the Kapsack problem. I defiitio of costraits, costrait/3 is trasformed to costrait(+,+,+). For example, cell/3 is trasformed to cell(+,+,+). 14 Iitial implemetatio Groud typig Fig. 4. Typed vs. o-typed. Results Typig is ideed more efficiet by a costat factor approximately equal 1.. A. Typig Groud versus Typig Itegers It s possible to specify the type of variable to be atural iteger. This ca give more iformatio to the compiler to do more optimizatio. Hypothesis Typig itegers is more efficiet. Test This optimizatio is tested o the Kapsack problem. I the defiitio of costraits, costrait(+,+,+) is trasformed to costrait(+it,+it,+it). For example, cell(+,+,+) is trasformed to cell(+it,+it,+it). This trasformatio is doe for all costraits that have iteger attributes.

12 11 1 Groud typig Iteger typig Fig. 5. Groud-typed vs. iteger-typed. Result They produced the same performace. B Fial Implemetatios I this sectio, fial implemetatios of the five DP problems are evaluated after applyig the guidelies Fial Implemetatio f(x) = 3.8^(x/145) Fig. 6. Kapsack.

13 Fial Implemetatio f(x) = x / Fig. 7. Edit Distace with varyig first strig. 18 Fial Implemetatio f(x) = x / m Fig. 8. Edit Distace with varyig secod strig.

14 .85 Fial Implemetatio f(x) = x^3 / Fig. 9. Matrix Chai Multiplicatio. 1 Fial Implemetatio f(x) = x^ / Fig. 1. Viterbi with varyig umber of states.

15 Fial Implemetatio f(x) = x/ m Fig. 11. Viterbi with varyig sequece legth i sparse trasitio matrix.. Fial Implemetatio f(x) = x / m Fig. 1. Viterbi with varyig sequece legth i complete trasitio matrix.

16 .6.4 Fial Implemetatio f(x) = x/ Fig. 13. Viterbi with varyig the degree of sparsity of the trasitio matrix. 18 Fial Implemetatio f(x) = x^3 / L Fig. 14. CYK.