Multiple-Criteria Decision Analysis: A Novel Rank Aggregation Method

Transcription

1 3537 Multiple-Criteria Deision Analysis: A Novel Rank Aggregation Method Derya Yiltas-Kaplan Department of Computer Engineering, Istanbul University, 34320, Avilar, Istanbul, Turkey dyiltas@ istanbul.edu.tr ABSTRACT Ranking among several objets is a very ruial operation for different appliations to find a vote value for eah objet against the others. Multiple metris an be ombined to get a single vote value of an objet. There are many studies in the literature that onvert the ranking problem into a graph struture to solve it with a disrete mathematial proess. Generally, these studies define multiple metris as matrix forms and then relate them with the omputations of eigenvetors to find the best ranked objet. However, due to the dynami nature of the metri values, ranking approahes should be fast and less omplex. In this study a different approah for the ranking proess with multiple metris is proposed. This approah is fast and easy to implement. In order to test the approah, a network senario is designed with omputer programs. The experimental results show that this method outperforms a ommon onventional method in terms of various metri values, namely transmission time, paket loss rate, jitter, availability, and throughput. As a onsequene, the proposed method gives the average value of eah individual metri as more advantageous and without resaling the numerial values. Keywords - Deision theory, multi-riteria deision analysis, multi-objetive deision, rank aggregation, rank entrality Date of Submission: Marh 02, 2018 Date of Aeptane: Marh 23, INTRODUCTION graph theory properties suh as isomorphism, predeessor and suessor verties. Ranking appliations over user/ustomer preferenes on In addition to eigenvetors, in some studies Markov some measurements used in e-ommere, multi-agent systems, web servies related to searh engines, or some soial ativities like sports tournaments and vote inquiries. In reent studies some ranking algorithms have been hains were applied to represent objets as basi state strutures. The state sequenes were related to some state distributions, namely probability distributions [4]. When the system reahed a onstant point with a stati state proposed to satisfy the rank of objets in various distribution, namely stationary distribution, the information-based problems suh as shema mathing in approximations on eigenvetors easily gave this database integration, searh engine responses, deisionmaking proesses, and multi-objetive seletions. There are also some studies that exhibit deision making methods whih onlude joint deisions aording to the preferenes of several parties in a problem. One of these studies represents various voting rules inluding ranking steps to get the last deision among many possible alternatives [1]. In this ase the voting shemes are benefiial for several kinds of daily problems suh as task or resoure alloations. Most of the ranking algorithms in the literature onvert the ranking problem into a graph struture in order to solve it with the disrete mathematis tools. The graph struture is desribed as G=(V, E), in whih the variables symbolize a set of ranked objets and pairs of ompared objets [2]. The studies using the graphs represent multiple metris in matrix forms and then give the eigenvetors to find the best ranked objet. In a sample study the relationship between the ranking algorithms and Internet tehnologies was provided with the help of mathematial axioms and their properties in soial hoies whih were obtained from the websites [3]. The ranking was performed as graph-theoreti onsidering the stationary probability distribution besides a node-related matrix. In this study, the authors proposed several axioms based on distribution. The ranking senarios were evaluated to define spam pages espeially for web searh results from keywords and the usage of searh engine results. For details about some appliations and methods see Dwork et al. [4]. In the literature the ranking algorithms whih give some solutions to the multiple metri or multi-objetive deision problems are also known as rank aggregation methods or rank aggregators. (Note that in this paper, both terms are used alternately due to their usages in the ited referenes.) Some methods were ited by Gormez et al. [5]. These methods generally provide the deisions among the objets with pairwise omparisons. Mostly this proess is applied via the ounts of outperformane between any two objets. A study on web servies has provided different ranking methods for several quality attributes of the seleted web pages [6]. Some metris suh as file size, reliability, response time, and data freshness were used as the quality attributes for omputations. The ranking proess was onstruted in three different ways whih over the rank aggregation or linear sore ombination steps. The authors proved that the linear ombination of all values obtained as individually normalized quality metris was the best way among them. However, their normalization formula may not be suitable for the metris with very small values suh as deimal numbers.

2 3538 Starting from the rank aggregation analysis of the web and some other web searh and multimedia database appliations, Domshlak et al. [7] proposed some rank aggregation methods for a database integration operation, namely shema mathing to find the best ranked mappings. They presented two general algorithms to whih they adapted their problem and ompared the results with their novel algorithms. Their algorithms over matrix-based operators and some properties of funtions suh as ommutativity that are beyond the sope of this paper. In another study the rank aggregation was ombined with the similarity analysis of the objets [8]. The similarity property was measured through a predetermined funtion. Some measures suh as Kendall Tau distane and Spearman footrule distane were used to provide the similarity measurements of the ranking values of the objet pairs [4, 8, 9]. Eah objet had a ranking value and the sum of the distanes between its value and those of all other objets. The basi purpose here was to find the ranking with a minimum distane [9]. The aggregate similarity position of an objet was evaluated under the similarity funtion and the aggregate similarity ranked list [8]. The main goal was to derease the similarity distane between an aggregate list and the input lists. In the last part the similarity information was used with several aggregation methods to solve a searh engine proess [8]. The keywords for web searh were expanded to propose relevant keywords automatially. The Kendall Tau test and Eulidean distane were also applied to some problem solutions by Yazdani et al. [10]. Their study gives a method to obtain the topology robustness through multiple metris for the networks with various dimensions and onfigurations. The authors used several metris suh as node-onnetivity, edgeonnetivity, ritial breakdown ratio, and spetral gap. They provided some network examples from hierarhial and distributed onfigurations. Making a diret omparison between the multiple metris did not give a good measurement of network robustness as in a sample study [10]. For this reason, a sore initialization was proposed in the beginning of the solution and through these values a rank aggregation method was performed during the proess. Eah metri firstly overed the sore initialization as an input independently and then generated an aggregate ranking with other metris. Here, eah metri was standardized based on a perentage of an optimal theoretial value, but the authors mentioned that these theoretial optimal values might not be possible and there should be some trade-off between some metris. A Kendall Tau test from the literature was applied to make a rank orrelation statistially based on some oeffiients between metri pairs [10]. The Eulidean distane was evaluated over the metris of a given network and the omplete network to find their loseness. The perentage operation, whih gives the base of ranking in this study, overed metri orrelations due to some frations. However, in some problems the parameters may not be orrelated aording to some formulas beause they are independent. Also, the adaptation of this solution to another problem requires different formulas, so it is not an adaptive and flexible solution. Aording to a statistial framework, the rank aggregators that are optimized with Kendall Tau alulations are diffiult to ompute; yet have a high performane [11]. The optimization is performed with respet to the Kendall Tau error measures. This framework monitors the performane of an aggregate ranker based on individual input rankers. It is obvious that the initialization values of the starting ranks during the optimization affet the last performane of the rank aggregators. Shahnai et al. [12] generalized the rank aggregation problem whih handles the input lists inluding the permutations of predetermined objets. In their study an input list ontained a set of multiple orderings, namely multiple permutations of the objets. An overall permutation of the objets was ompared with the individual permutations inside the set. At this stage Kemeny distane was used additionally to find the objet number staying in different orders of two orderings. So the authors tried to obtain a single objet order whih minimizes the total distane. This also means the minimization of the total disagreements upon the objet orders inside the lists. As the authors said, the problem ours as NP-hard for four different input lists. This theorem is also mentioned in a different study [13]. In this paper s method five different metris whih an be adapted as five different input lists are used as in below. Additionally, some leture notes gave detailed theoretial information about rank aggregation methods, espeially on the basis of the metasearh problem that is the aggregation of searh results from several searh engines [13]. The above-mentioned Kendall Tau, Spearman footrule, and Kemeny distanes an be observed along their formulas from these leture notes. The rest of this paper is organized as follows. In Setion 2, the theoretial information about the onventional ompared method and the proposed one is defined. In Setion 3, the numerial omputations based on omputer programs are given. Lastly, several important onlusion points are mentioned in Setion THEORETICAL BASELINE There are several algorithms for rank aggregation as disussed above. In this setion it is foused on a onventional method and then proposed the new method and ompared both approahes. 2.1 Conventional rank aggregation method In the literature Rank Centrality (RC) is one of the main rank aggregation algorithms that find a single sore for eah objet [14]. This method beomes onventional as mentioned before, using a graph in whih the nodes symbolize the ranked objets. Random walks along the edges of the graph represent pairwise omparisons of objets staying at the end nodes. The walking frequeny on a node or the stationary distribution gives a sore of the relevant objet. This distribution measures the importane of a node between the other nodes giving the term network entrality.

3 3539 Now some important points and formulas of the RC method are briefly given. a is a fration that represents the number of times objet j outperforms objet i [14]: a k l 1/ k) l Y 1 ( (1) l Here, Y represents the result in omparison l. If j outperforms i, the result is 1; otherwise it is 0. The omparison of the objets is made aording to the formulas: A and A ji a (2) a a ji a ji (3) a a ji In (2)-(3), the denominators are always equal to 1. Additionally, if the objets are not ompared to eah other, A or A ji beomes 0. The value A onverges to the weight w fration i j w w j for large k as k. P transition matrix and stationary distribution are omputed based on the A values. Here, is the top left eigenvetor of P. At the end eah node gets a rank value or a numerial sore [14]. 2.2 Proposed method In this subsetion a new rank aggregation method is proposed. This method performs an easy omputation throughout many objets and multiple metris with their raw values and also without onsidering any normalization step. This method shall be alled as Win Rank and the objets are labelled as winner andidates with an index ( ). The number of all objets is C and M={M 1,M 2,...,M d } is a set of metris with d. It is assumed that RV represents the ranking value of objet with respet to the ith metri. RV is an integer value that gives the order of objet among overall C objets based on their asending/desending sorted values. For a numerial example, it is assumed that five objets (C=5) and an individual metri M 1 are stayed. The third olumn of Table 1 is obtained aording to the given first two olumns. Table 1. Ranking Values of Five Objets Based on Their M 1 Values. M 1 RVM Table 1 diretly represents ranking values of the objets on their asending sorted M 1 values. In Win Rank eah metri has its own arrangement type as asending or desending, whih may be different from the others; the arrangement type is related to the metri harateristis. That is for a metri, if small numerial values are advantageous, the sorting should be asending; if large values are advantageous, the desending arrangement is hosen. If a desending sort is used in Table 1, the RVM 1 olumn ours as 2, 3, 1, 5, and 4 respetively from top to bottom. After ompleting the omputation of the ranking values whih depend on all metris individually, the total ranking value (TRV) of objet is omputed as an aggregation along all metris as: d TRV ( ) RV (4) i 1 At the end, a omparative degree of an objet is found as Win Order WO() based on the TRV() results with a new proess similar to the asending sort for RV. An example for this proess is given in Table 2. Here it is assumed that there are five objets (C=5) and five metris (d=5). TRV() is the sum of five RV values for eah objet as in (4). In this way, the last olumn is obtained aording to the first two olumns in Table 2. The winner of this example is Objet 2, as shown in Table 2. The steps of Win Rank an be seen from the pseudoode in Fig. 1. Table 2. WO() of Five Objets Based on Their TRV() Values. TRV() WO()

4 3540 Selet all objets and randomly hoose their metri values eah from a predetermined set. For i=1 to d Sort all C objets asending/desending with respet to their values and find the order number of eah one. For =1 to C Assign the objet s order number to TRV()+= RV ; RV ; If i==d Sort all TRV() values asending and find the new order number for eah objet. Assign the objet s new order number to WO(); The objet with WO()=1 beomes the winner. Figure 1. Pseudoode of Win Rank. 3. EMPIRICAL EVALUATION In this setion several numerial omparisons between the onventional RC method and the Win Rank method are presented. It should be noted that Win Rank an be used with any kind of ranking-based seletions suh as deision-making proesses. Also, the sample struture given in this setion an be easily adapted to any other areas by reorganizing the design and the metris. In the Win Rank, an ad ho network is set with 100 nodes and 116 edges totally. An edge is loated in between two nodes, so all edges are onnetion ways for the relevant node pairs. Eah edge is assoiated with several metris suh as transmission time, paket loss rate, jitter, availability, and link throughput. These metris an be hanged aording to the problem area. The values of these metris are randomly hosen from the ranges as shown in Table 3. multipliative property over multiple edges. It is assumed that only the availability is multipliative and the others are additive. This means that, for example, the overall jitter is omputed along a path, summing up the jitter values of the edges whih belong to this path. This operation is the same for transmission time, paket loss rate, and link throughput. On the other hand, the availability values of the relevant edges are multiplied to find an overall availability for a speifi path. Additionally, as mentioned in Setion 2.2, the arrangement type of availability and link throughput are desending and all others are asending. Now note that the objets are the paths. In the experiments 25 different soure-destination pairs are hosen for the omputation. Several paths (objets) starting at the soure and ending at the destination nodes are determined. Two different data sets whih have different metri and path numbers are onstruted. Suh distintion is preferred to show speifially the effet of the path numbers on the results. In the first experiment set 500 paths are used to ompare four metris, i.e. transmission time, paket loss rate, jitter, and availability. Figs. 2-5 show results of the omparison between RC and Win Rank methods on all metris that get their average values from the winner paths relevant to any hop ount. Table 3. Metris and Their Ranges. Metri name Metri range Transmission [1, 200] time Paket loss rate {10-7, 10-6, 10-5,10-4, 10-3, 10-2 } Jitter [1, 20] Availability [0.999, ] Link throughput [1, 5000] An edge array inluding at least one edge between any two nodes gives a path struture. There may be several paths in between two speifi nodes, namely soure and destination. The main target is to find the best path between any seleted soure-destination node pair with respet to the metri values. This is similar to the detetion of the shortest path between any two ities as in logistis. But here there are multiple metris, not only one metri suh as the distane. For eah path it should be had one aggregated value based on eah metri value of the relevant edges. The used metris have an additive or Figure 2. Transmission time (experiment set 1).

5 3541 Figure 3. Paket loss rate (experiment set 1). Figure 5. Availability (experiment set 1). Eah path has a hop ount whih defines the number of edges it inludes. As seen in Figs. 2-5, a metri value on the y-axis is demonstrated as the average value of all winner paths having the relevant hop ount on the x-axis. Figs. 2-5 demonstrate that Win Rank outperforms RC at almost all values. Note that large values of availability are preferable beause they have a desending arrangement as mentioned before. On the ontrary, small values of transmission time, paket loss rate, and jitter are preferable. In the seond experiment set, 5000 paths are used with onsidering the link throughput as well as the other metris. Figs show omparison results for the 5 metris. Figure 4. Jitter (experiment set 1).

6 3542 Figure 6. Transmission time (experiment set 2). Figure 8. Jitter (experiment set 2). Figure 7. Paket loss rate (experiment set 2). Figure 9. Availability (experiment set 2).

7 3543 winner between the objets in a deision making proess an be easily deteted aording to the omparisons in this method. 5. ACKNOWLEDGEMENTS This work was supported by Sientifi Researh Projets Coordination Unit of Istanbul University. Projet Number: The author sinerely thanks Dr. Ergün Gümüş for sharing his omments during the preparation of this study. REFERENCES [1] V. Conitzer, Making deisions based on the preferenes of multiple agents, Communiations of the ACM, 53, 2010, [2] L.H. Lim, X. Jiang, Y. Yao, and Y. Ye, Graph helmholtzian and rank learning. f, 2008, Last aessed: Marh Figure 10. Aggregated throughput (experiment set 2). Aording to the results in Figs. 6-10, Win Rank also outperforms RC in all metris for the 5000-path set. Note that the throughput is handled as the aggregated value of all links along an entire path, not as the minimum throughput value in between all links values towards a path as in the usual network appliations. Thus large values of the link throughput are preferable here. As seen in Figs. 6-10, a large number of paths gives a more distint differene between RC and Win Rank than that from a small number as in Figs CONCLUSIONS A new rank aggregation method is onstruted to solve the ranking problem easily. It was not needed to use any onstant matrix struture whih should have been set ontrolling the metri values. The raw data was used diretly in the omputations. In other words, the metri values were not saled into any weight fration. The proposed method, namely Win Rank was ompared with a onventional one using matrix forms and eigenvetors. Win Rank outperforms this ommon method for both small and large numbers of objets. Win Rank gives better results in almost all metris and hop ounts. An empirial testbed was used for two network senarios, but the proposed method an be speifially adapted to any other senario in various orrelative problem solutions that over different testbeds and metris. Additionally, the omputer based implementation of Win Rank is very easy and extendible for new areas overing deision making proesses espeially with many omparison metris. For eah metri, the ranking value is omputed independently. Then the total value of all ranking results of the metri values is obtained. Consequently, in any researh area, the [3] A. Altman and M. Tennenholtz, Ranking systems: The PageRank axioms, Pro. 6th ACM Conferene on Eletroni Commere, Vanouver, British Columbia, Canada, 2005, 1-8. [4] C. Dwork, R. Kumar, M. Naor, and D. Sivakumar, Rank aggregation methods for the web, Pro. Tenth Int. World Wide Web Conferene (WWW10), Hong Kong, 2001, [5] Z. Gormez, E. Gumus, A. Sertbas, and O. Kursun, Comparison of aggregators for multi-objetive SNP seletion, Pro. 35th Annual International Conferene of the IEEE EMBS, Osaka, Japan, 2013, [6] F.E.M. Arasi, A. Anand, and S. Kumar, QoS based ranking for omposite web servies, International Journal of Siene, Engineering and Tehnology Researh (IJSETR), 3, 2014, [7] C. Domshlak, A. Gal, and H. Roitman, Rank aggregation for automati shema mathing, IEEE Transations on Knowledge and Data Engineering, 19, 2007, [8] D. Sulley, Rank aggregation for similar items. ilarrank.pdf, 2006, Last aessed: Marh [9] L.P. Dinu and F. Manea, An effiient approah for the rank aggregation problem, Theoretial Computer Siene, 359, 2006, [10] A. Yazdani, L. Dueñas-Osorio, and Q. Li, A soring mehanism for the rank aggregation of network robustness, Commun Nonlinear Si Numer Simulat, 18, 2013,

8 3544 [11] S. Adalı, B. Hill, and M. Magdon-Ismail, Information vs. robustness in rank aggregation: Models, algorithms and a statistial framework for evaluation, Last aessed: Marh [12] H. Shahnai, L. Zhang, and T. Matsui, On rank aggregation of multiple orderings in network design, Last aessed: Marh [13] R. Kumar, Rank aggregation. Leture Notes, University of Rome, Italy, [14] S. Negahban, S. Oh, and D. Shah, Rank entrality: Ranking from pair-wise omparisons, Cornell University Library. Presented in part at NIPS, 2012 in Lake Tahoe. Last aessed: Marh Biography Derya Yiltas-Kaplan was born in Midyat/Mardin in Turkey. She graduated from primary and middle shool in Mardin, the high shool in Aksaray. She reeived the BS, MS, and PhD degrees in omputer engineering from Istanbul University, Istanbul, Turkey, in 2001, 2003 and 2007, respetively. She was a post-dotorate researher at the North Carolina State University and she reeived postdotorate researh sholarship from The Sientifi and Tehnologial Researh Counil of Turkey during the period of April 2008-April She is urrently working as a faulty member in the Department of Computer Engineering at Istanbul University.