[Type text] [Type text] [Type text] Xinquan Yang 1,2 1 School of Computer and Information Engineering, Heze University, Heze, ,

Transcription

1 [Type text] [Type text] [Type text] ISSN : Volume 10 Issue 22 BoTechnology 2014 An Indan Journal FULL PAPER BTAIJ, 10(22), 2014 [ ] Research on multdmensonal ndexng based on cloud computng system Xnquan Yang 1,2 1 School of Computer and Informaton Engneerng, Heze Unersty, Heze, , (CHINA) 2 Insttute of Embedded System and Internet of Thngs, Heze Unersty. Heze, , (CHINA) ABSTRACT Rapd deelopment of cloud computng technology realzes the storage and management of mass data, but changes of data model make the ndexng technology of tradtonal database management system cannot process data drectly, whch means research of ndexng technology n cloud enronment s needed to prode effcent support for retreal tasks of mass data n cloud enronment. Wth the poor research of multdmensonal ndexng based on cloud computng system, ths paper studes the multdmensonal ndexng n cloud computng system, ntroduces t, descrbes the concept of multdmensonal ndexng and ntroduces the generally used multdmensonal data processng method. Then t researches multdmensonal ndexng based on cloud computng system. Ths paper ncludes bref ntroducton of dstrbuted storage system and somethng related to consstency Hash and the process of multdmensonal data ndexng drops to one-dmensonal ndexng, proposes multdmensonal system of M- Index for the problem of current dstrbuted storage system data ndexng doesn t support more than one program, phases n pyramd technque, proceeds dmensonalty reducton on two dmensonal space, so as to offer reference for mass data storage ndexng n cloud enronment. KEYWORDS Mass data; Multdmensonal ndexng; Dstrbuted storage; Pyramd technque. Trade Scence Inc.

2 BTAIJ, 10(22) 2014 Xnquan Yang INTRODUCTION Wth the deelopment of network and nternet, tradtonal data management technology cannot satsfes the users needs of mass data management any more, whle cloud computng can sole ths problem well whch prodes a bg mpetus to the deelopment of mass data management [1]. As a new network technology, cloud computng attracts many users because of ts huge storage and low cost [2]. Users can share and store resources anytme and anywhere through cloud serce on nternet. As the mportant representaton of cloud computng, sertzaton of computng and resources can prode mass data management, computng and applcaton deployment to users [3]. There are two ndexng technques n cloud enronment: undmensonal data ndexng and multdmensonal data ndexng. Ths paper focuses on multdmensonal ndexng. MULTIDIMENSIONAL INDEXING IN CLOUD ENVIRONMENT In cloud enronment, mass data processng program wll be dded nto n small subprograms automatcally through net, mass data of users wll be stored n serer by net dstrbuton and be processed by systematc search and computng analyss consstng of many serers, and the results wll be returned to users. In cloud computng enronment, subset of dstrbuted data s stored n network nodes, ts search strategy s deployed n need-satsfyng characterstc for users to rapdly search data by correspondng column. Many data queres can be realzed n dstrbuted network, ncludng settng up relatonshp, one-dmensonal query, neghbor query, range query and pont locaton query. Multdmensonal ndexng means regardng all the data attrbutes nformaton of products n cloud computng as multdmensonal data, such as the sze of e-book, publshng tme and prce, transformng these multdmensonal attrbutes nto multdmensonal data, buldng a multdmensonal ndexng based on these multdmensonal data n order to proceed multdmensonal ndexng and range queres at the same tme rather than undmensonal query n dfferent dmensons. Fgure 1 shows the most used multdmensonal data processng method. Fgure 1 : Multdmensonal data processng method The maor ssue of the establshment of multdmensonal data ndexng structure n cloud enronment s to ensure load balance between space and host process. When proceedng centralzed routng nqury, carryng out nserts, updates and deletes, t can ensure low consumpton of mutual nformaton. Usually, cloud computng system expects data keeps smooth dstrbuton n network so as to ensure mostly the same spatal load of each host. The man form of multdmensonal data ndexng structure s to map multdmensonal space to one-dmensonal space, sort the multdmensonal spatal obects by one-dmensonal alue, dde ndexng space nto lots of lttle peces and code eery model by z-transform. Releant work ntroducton MULTIDIMENSIONAL INDEXING IN CLOUD COMPUTING ENVIRONMENT Dstrbuted storage system GFS s a dstrbuted fle system deeloped by Google, ts operaton mode s to dde data fles nto peces and store them dstrbutely. It supports parallel fles readng [4]. On the bass of GFS, Google deeloped Bgtable, a dstrbuted storage system wth smlar functons of dstrbuted database. Bgtable s manly used to store structural data or sem-structural data and prode data storage and query serce for Google search engne and Google map [5].

3 13800 Research on multdmensonal ndexng based on cloud computng system BTAIJ, 10(22) 2014 Dynamo deeloped by Amazon s also a representate dstrbuted storage system wth the feature that users can adust the scope of data center dynamcally accordng to work loan [6] and t uses P2P structure n the system to reduce query tme. Cassandra s a structural data storage system ntegratng Dynamo and Bgtable, t s used broadly. Data storage platforms of some applcatons lke Facebook and Dgg are all Cassandra [7]. PLUTS, a dstrbuted database system deeloped by YaHoo, can process masse dstrbuted data. Consstency hash MIT proposes consstency Hash algorthm for dstrbuted hash table, realzng the real applcaton of DHT n P2P enronment [8]. Many algorthms can realze Hash, ncludng Chord, CAN and Pastry, ths paper focuses on Hash algorthm wth data center storage node bult wth Chord. Identfers of each node and key word n Chord system are M bt bnary strng acqured through hash operaton of node IP and keyword whch sorts all nodes clockwse by the sze of dentfers to form an annular topologcal structure. Fgure 2 shows the Chord network when m=4. Fgure 2 : Chord network when m=4 Multdmensonal data ndexng Bref ntroducton of multdmensonal data ndexng mechansm whch can execute complex query, ncludng ntroducton of topologcal and ndexng mechansm of oerlay network n data desgn center and defnton of multdmensonal data ndexng bult wth pyramd technology. Oerlay network topology In data center based on shared nothng structure, storage dece of eery node s ndependent. Fgure 3 shows oerlay network topology, Fgure 4 ddes N (0 < n) nto two knds of nodes: storage nodes SN and ndexng nodes IN. SN s used to store data allocated to N, and IN s used to manage metadata of data n SN. Data center wll buld an annular oerlay network of IN accordng to Chord protocol. At the same tme, there are two data ndexes for mantenance on each IN, namely multdmensonal ndex M-Index and local data ndex L-Index. In oerlay network topology shown n Fgure 3, man task of M-Index s to mantan route nformaton transmtted n oerlay network, whle L-Index s n charge of managng metadata nformaton so as to aod falure of sngle pont and sae storage space. Fgure 3 : Oerlay network topology

4 BTAIJ, 10(22) 2014 Xnquan Yang Fgure 4 : Applcaton of B-tree Indexng mechansm Data stored n cloud computng enronment are mostly multdmensonal data, for example, the metadata of an audo fle has such nformaton as memory, duraton and compresson rato. Ths paper plans data n a unfed way by abstractng t to a bnary key-alue par, namely metadata attrbute and attrbute alue represented as: D a, (0 d) (1) Prmary key of data n consstency Hash only contans a certan metadata, makng dstrbuted storage system cannot carry out multdmensonal query. Establshment of multdmensonal data ndexng by pyramd technology n ths research s targeted to M-Index by reducng dmensonalty of d-dmensonal ector M 0,, d 1 nto one-dmensonal ndex d wth pyramd technology, whle M-Index ensures no loss of data to keep data ntegraton and aod omsson n data query. M-Index sets pont 0.5,, 0.5 P as center pont, then allocates d-dmensonal space 0,1 d nto 2d subspaces named pyramd wth P on the top and d-1-dmensonal space at the bottom. Codng rule of pyramd s: any one-dmensonal ector Dm 0 m d n d-dmenson s perpendcular to the bottom of two pyramds, when m-dmensonal coordnate m 0.5 n one pyramd, ths pyramd s coded as m pyramd, when m 0.5, ths pyramd s coded as m d pyramd. M-Index defnes d-dmensonal ector as pont V n d-dmenson, and one-dmensonal ndex after dmenson reducton as the dstance between pont and pont P. Dmenson reducton may produce the same dstance between pont and pont P, generatng redundance n query. To reduce redundance, dfferent dstance functons are requred n dfferent pyramds. The process of M-Index reduces multdmensonal metadata to one-dmensonal ndex s as shown n below: (1)Normalzaton. Gng that pyramd technology s targeted to 0,1 d dmenson, M-Index should transform metadata frst by transformng the alues of ( 0 d) < nto the range of 0,1 as shown n formula (2): (1) mn (0 d) mn (2) mn represents the mnmum alue of and represents the mum alue, (1) represents the normalzed alue of for expresson. (2) Code the pyramd the decson pont V belongs to. Gng that dfferent dstance functons set by M-Index n dfferent pyramds hae dfferent expressons, computng code of pyramd the decson pont V belongs to before computng dstance s requred, as shown n formula (3):, f ( 0.5), d, f ( 0.5), (3) ( ( k,0 (, k) d, k : k )) (4)

5 13802 Research on multdmensonal ndexng based on cloud computng system BTAIJ, 10(22) 2014 represents the one-dmensonal ector wth the largest dfference of coordnate between pont V and pont P. d d d dmenson s (3) Calculaton of the dfference of coordnate between pont V and pont P n or as shown n formula (5): h 0.5 ( d) mod d or (5) (4) Calculaton of one-dmensonal ndex of pont d y s as shown n formula (6): d h (6) Takng two dmenson as example, after aboe calculaton, two dmenson s dded nto 4 pyramds wth pont P 0.5, 0.5 as ertex as shown n Fgure 5. Hypothetcally, the sze of audo data s 2MB, duraton s 60s, wrtten as: D s 2 MB, t 60s ; alue nteral s [0,4MB], [0,240s]. Accordng to the aboe formulas, D-dmensonal ndex s reduced to one-dmensonal ndex d, d y y Fgure 5 : Pyramd of two-dmensonal space Introducton of consstency Hash may break the order of alues wthn one-dmensonal ndexng range d, ndexng system wth prmary key of d y cannot realze support to nteral query. Therefore, dde ths nteral d, d nto n+1 subnterals, transmt nteral query nto a sngle alue query accordng to the ntersecton of query mn nteral and subnteral n data query whle keepng load balance of data dstrbuton, select common prefx as prmary key to adust the sze of nteral dynamcally, use tree structure to manage common prefx of subnteral. For example, prefx bnary-tree n Fgure 6 s a prefx bnary-tree wth depth of 5 whch needs further research. mn, d Fgure 6 : Prefx bnary-tree wth depth of 5

6 BTAIJ, 10(22) 2014 Xnquan Yang CONCLUSION Storage of mass data n cloud enronment brngs dffcult to data ndexng. Wth the deelopment of cloud computng technology, multdmensonal ndexng technology n cloud enronment s also mproed. To tackle the problem that data ndexng of dstrbuted storage system n cloud enronment doesn t support complex ndexng wth seeral programs, ths paper proposes multdmensonal ndexng mechansm M-Index and ntroduces pyramd technology to reduce multdmensonal data to one-dmensonal ndexng. REFERENCE [1] Chen Quan, Deng Qann; Cloud computng and ts key technques [J], Journal of Computer Applcatons, 29(9), (2009). [2] Zhang Jan; Analyss of concept and nfluence of cloud computng [J], Telecommuncatons Technology, 01, (2009). [3] P.Mell, T.Grance; The NIST defnton of cloud computng, SP [R], Gathersburg: Natonal Insttute of Strabdards and Technology, (2011). [4] S.Ghemawat, H.Goboff, S.Leung; The google fle system[c]//proc of the 19 th ACM symp on operatng systems prncples, New YORK:ACM, (2003). [5] F.Chang, J.Dean, S.Ghemawat et al.; Bgtable:A dstrbuted storage ststem for structured data[j], ACM Trans on Computer syetems, 26(2), 1-26 (2008). [6] G.DeCanda, D.Hastorun, M.Jampan et a.; Dynamo: Amazons hghly aalable key-alue store[j], //Proc of the 21 st Acm Symp on Operatng Systems Orncples, New York:ACM, (2007). [7] A.Lakshman, P.Malk; Cassandra: A decentralzed structured storage system[j], ACM SIGOPS Pperatng Systems Reew, 44(2), (2010). [8] D.Karger, E.Lehman, T.Leghton et al.; Consstent hashng and random trees: Dstrbuted cachng protocols for releng hot spots on the World Wde Web[C]//Proc of 29 th Annual ACM Symp on Theory of Computng, New York:ACM, (1997).