Dynamic Grouping Strategy in Cloud Computing

Transcription

1 IEEE CGC 2012, November 1-3, Xiangtan, Hunan, China Dynamic Grouping Strategy in Cloud Computing Presenter : Qin Liu a Joint work with Yuhong Guo b, Jie Wu b, and Guojun Wang a a Central South University, China b Temple University, USA 1

2 Outline Introduction Preliminaries K-Mean-based Dynamic Grouping Extensions Evaluation Conclusion & Future work 2

3 Introduction 3

4 Cloud computing model o Cloud computing has emerged as a new type of commercial paradigm due to its overwhelming advantages, such as flexibility, scalability, and cost efficiency F 1 : 1 :{A, B} B} A, B Cloud F 2 : 2 :{B, D} D} Bob F 1 F F 3 : 3 :{C, D} D} 4

5 Application scenario University A Cloud Online library Access right Online library 5 Staffs &students

6 Problem: Cost grows linearly University A Alice {A,B} {F1,F2,F4} Redundant files: F1,F3: 2 F2,F4: 3 Bob Clark {A} {F1,F2} {C,D} {F2,F3,F4} {C} {F3,F4} 6 Donland

7 New chanllenge Alice University A {A,B} Deploy proxies inside Aggregate user queries Distribute search results Bob {F1, F2, F4} {A} {F1, F2} Proxy 1 {A,B} {F1, F2, F4} 7 Clark {C} {C, D} {F2, F3, F4} Proxy 2 Donland Return 6 files How to group queries to minimize the returned files? {F3, F4} {C, D} {F2, F3, F4}

8 Naïve solution: Random grouping Alice Clark Bob Donland University A {A,B} {F1, F2, F4} {C, D} {F2, F3, F4} {A} {F1, F2} {C} {F3, F4} Proxy 1 Proxy 2 Problems: Random grouping cause a waste of bandwidth {A,B} {F1, F2, F3 F4} {C, D} {F1,F2, F3, F4} Return 8 files Waste 20% bandwidth 8

9 Naïve solution: One proxy Alice University A {A,B} {F1, F2, F4} Problem: One proxy causes performance bottleneck and single point of failure Clark Bob Donland {C, D} {F2, F3, F4} {A} {F1, F2} {C} {F3, F4} Proxy {A,B,C, D} {F1,F2,F3,F4} Return 4 files Best performance 9

10 K-Mean-based Dynamic Grouping (KMDG) Classify n queries into k groups in the case of k proxy servers, so that each group size equals to n/k and the number of returned files is minimized. NP-Hard problem Heuristic grouping strategy Basic strategy: KMDG (based on K-Means) Extensions: KMDG1-Robust version KMDG2-Relax the constraint of equal group size 10

11 Design goals Effectiveness Cost efficiency Obtain optimal results within a polynomial time Load balancing Balance the bandwidth among proxies Minimize the bandwidth at at the cloud Robustness Obtain search results even if if some machines fail 11

12 Preliminaries 12

13 System model The cloud, many users, and many proxy servers Query router (QR) Aggregation and distribution machines (ADMs) 13

14 System model Given a public dictionary consisting of all keywords University A A user query/a combined query is a 0-1 bit string Alice Clark Bob {A,B} <1100> {C, D} <0011> {A} <1000> {C} <0010> Proxy {A,B,C, D} <1111> Dictionary=<A,B,C D> 14 Donland

15 Problem formulation Assumption Keywords are uniformly distributed in the file set The probability of each keyword in a file is the same 15 Minimize the total returned files = Minimize the number of keywords in combined queries Group queries with the most common keywords together Minimize the total number of 1s in the combined queries

16 Parameter analysis The expected value of the number of returned files can be calculated with Eq.1 d: the number of keywords in the dictionary γ: the average number of keywords in a file k: the number of groups/proxies t: the number of files in the cloud (grouping cost) : the average number of keywords in the j-th combined query (1) 16

17 K-Mean-based Dynamic Grouping (KMDG) 17

18 Definitions 18

19 High level ideas Suppose each query denotes a node 19

20 High level ideas An improvement: First choose a random node as the seed. For the i-th seed, choose the one with the total maximal distance with all i-1 seeds 20

21 High level ideas Closet: the minimal number of increased 1s after being combined with the seed s1=1100, s2=0011, s3=1001, s4=0101; Q=

22 High level ideas After choosing k seeds, grouping process is performed in the same way in next round 22

23 Algorithm 23

24 Example Sample Queries 24

25 Extensions 25

26 KMDG1-Algorithm Each user generates 2 α k query copies with the constraint that α copies are in different groups 26

27 KMDG1-Example 27

28 KMDG2-Algorithm Relax the constraint of equal group size to further reduce bandwidth 28

29 KMDG2-Example 29

30 Evaluation 30

31 Parameters Simulations are conducted with MATLAB R2010a, running on a local machine with an Intel Core 2 Duo E GHz CPU and 8 GB RAM Summary of parameters 31

32 Performance Comparison of total number of 1s. X-axis denotes the number of users and Y-axis denotes the total number of 1s. 32

33 Performance Comparison of bandwidth at the cloud. X-axis denotes the number of users and Y-axis denotes bandwidth at the cloud (MB) 33

34 Load balancing Comparison of imbalanced transfer-in bandwidth. X-axis denotes the number of users and Y-axis denotes the imbalanced transfer-in bandwidth (MB) 34

35 Load balancing Comparison of imbalanced transfer-out bandwidth. X-axis denotes the number of users and Y-axis denotes the imbalanced transfer-out bandwidth (MB) 35

36 Conclusion & Future work 1 A dynamic grouping strategy is proposed to achieve cost efficiency, load balancing, and robustness in cloud computing 2 Experiment results show that KMDG can largely reduce the bandwidth incurred at the cloud 3 Conduct experiments on other keyword distributions to verify the effectiveness of KMDG 36

37 37 Thank you!