Remote Sensing Image Retrieval Algorithm based on MapReduce and Characteristic Information

Transcription

1 Remote Sensng Image Retreval Algorthm based on MapReduce and Characterstc Informaton Zhang Meng 1, 1 Computer School, Wuhan Unversty Hube, Wuhan Informaton Center, Wuhan Unversty Hube, Wuhan Abstract In order to mprove the retreval effcency and accuracy of remote sensng mage, and ths paper proposed a remote sensng mage retreval algorthm based on MapReduce. Frstly, the mage color and texture features of emote sensng are extracted, and then the Map functon s used to compute smlarty among the retreval remote sensng mages and the feature lbrary he accordng to color, color features, fnally, the ntermedate results of nodes are collected the node s obtaned by usng the Reduce functon, and the remote sensng mages are sorted to accordng to the smlarty to obtan the remote sensng mage retreval results. Test results show that the proposed algorthm can fast and accurate retreval n remote sensng mage, not only mprove the remote sensng mage retreval effcency, and also mprove the remote sensng mage retreval accuracy. Keywords - remote sensng mage;feature extracton;cloud computng;retreval algorthm I. INTRODUCTION Wth the development of satellte remote sensng technology, remote sensng mages data ncrease daly, there are some dsadvantages exsted n the tradtonal manual retreval method such as large workload and low effcency, whch could not meet requrements of remote sensng mage applcaton, whle the automatc retreval of remote sensng mages based on computers could enhance retreval effcency and effectveness, therefore, desgnng effcent and hgh accuracy remote sensng mage retreval algorthm has become a sgnfcant subject n the research at present. Amng at the automatc retreval of remote sensng mages, scholars home and abroad have conducted a large amount of researches, among whch CBIR based on content has advantages of quck speed and hgh precson and t has become the man retreval algorthm, frstly through drawng some characterstcs of the remote sensng mages such as color, type as well as texture to descrbe the content of the remote sensng mages, then match wth feature database n the remote sensng mages to obtan the retreval results [- 4]. Tradtonal sgnal node module s dffcult to meet realtme requrement[5,6]. Dstrbuted processng technology could dstrbute tasks to varous workng nodes and then treat, jontly accomplsh the tasks through collaboraton among nodes, therefore, dstrbuted processng technology has provded a new knd of soluton for remote sensng mages retreval[7]. Dstrbute processng technology at present manly has grd computng and cloud computng, n whch Hadoop s a basc archtecture for dstrbute processng system, the user could develop MapReduce program wthout understandng underlyng detals, conductng large scale of data analyss wth Hadoop has become the man parallel processng module n cloud computng and has been wdely used n vrtual database, large scale data processng, bo-medcne as well as classfcaton of patent mages[8]. To ncrease retreval effcency and accuracy rate of remote sensng mages, ths thess has put forward a knd of retreval algorthm of remote sensng mages based on MapReduce. Frst of all, drawng the remote sensng mages and texture features, then matchng wth remote sensng mages accordng to color features wth Map functon, and conductng collecton on ntermedate results of varous computng nodes wth Reduce functon and sortng of remote sensng mages accordng to the smlarty at last so that obtanng retreval result of remote sensng mages. The test result shows that the algorthm n ths thess could retreve the remote sensng mages fast and accurately, whch not only enhances retreval effcency of remote sensng mages but ncreases accuracy of retreval of remote sensng mages. II. CHARACTERISTICS OF REMOTE SENSING IMAGE AND SIMILARITY MATCHING Remote sensng mage retreval system based on CBIR draws remote sensng mage features to be retreved frst and then compute feature smlarty n remote sensng mage database, realze mage retreval accordng to the smlarty at last. A. Drawng remote sensng mage Color s an mportant characterstc n dstngushng classfcaton of remote sensng mages, drawng color features of remote sensng mages n RGB color space and obtanng 4 color features ncludng RGB mean value, R mean value, G mean value as well as B mean value. Texture descrbes space changes n remote sensng mages and draws texture features of remote sensng mages DOI /IJSSST.a ISSN: x onlne, prnt

2 wth Gabor flter. Gabor flter h(x,y) and Fourer H(u,v) transformaton forms are: hxy (, ) gx ( y)exp fx ( u f) v (1) Huv (, ) exp a In whch 1 x y gxy (, ) exp ( xy, ) ( xcos ysn, xsn ycos ) ( uv, ) ( ucos vsn, usn vcos ) B ( 1) f B ( 1) ln (3) 1 a In the equaton, f represents center frequency n bandpass zone of the flter, B represents tape wdth of the flter, θ represents drecton angle of chef axs of the flter and σ represents the varance. Determnng Gabor flter parameters accordng to equatons (1)-(3), then computng foldng energy values of respectve flters and mages, settng the mean value and varance of mage flter energy values as texture features of the remote sensng mage, that s texture texture texture texture Ftexture { 0,0, 0,0,, k 1, l 1, k 1, l 1} (4) In the equaton, K represents number of center frequency, L represents number of drecton angle. Computatonal formula for energy mean value of sub mage and mean square devaton E (x,y) texture x y n n (5) texture E (x,y) texture x y n n Therefore, 4 texture features of remote sensng mages have been obtaned and then there are 8 remote sensng mage features totally composed of color and texture features. B. Smlarty matchng Suppose that the remote sensng mage to be retreved s p0, there are n mages p(=1,,,n) n the remote sensng mage database, ts color feature s shown as c Rm and texture feature t Rk, M and K are dmensons for color and texture respectvely, computng smlarty between p0 and p(=1,,,n) accordng to the formula (6) R0 wd 1 t wd c (6) In the equaton, w 1 and w are weghts and w1 w 1, Dt and Dc show the smlarty values between the color and () the texture respectvely, ther computatonal formula s as follows: 1/ M m m ( t0 t ) m 1 D 1 t 1/ M m m max ( t0 t ) m1 1/ K k k ( t0 t ) k 1 Dc 1 1/ K k k max ( c0 c ) k 1 Conductng sortng on mages n the remote sensng mage database on Ro(=1,,,n) n descendng order and selectng prevous m mage as the retreval result. III MAPREDUCE REMOTE SENSING IMAGE RETRIEVAL A. MapReduce mage storage Image storage s the foundaton for the automatc retreval of remote sensng mage, t s a computng process n data ntensve type, ths thess adopts MapReduce dstrbuted processng to upload mages to HDFS. The specfc content s as follows: (1) Map stage. Adoptng Map functon and readng one remote sensng mage each tme and then drawng color and texture features of mage. () Reduce stage. Storng feature data of remote sensng mage drawn nto HDFS. HBase s a contrbuted database facng rows, therefore, HDFS remote sensng mage storage adopts HBase table format, specfc desgn of HBase table s shown n table 1. TABLE 1.HBASE TABLE DESIGNING OF REMOTE SENSING IMAGE Remote sensng mage d Orgnal document of mage Color feature Texture feature 001 fle001 c1 t1 00 fle001 c t 00n fle00n cn tn Procedure for mage storage based on MapReduce s shown n fgure. (7) DOI /IJSSST.a ISSN: x onlne, prnt

3 Fgure.Storage procedure for remote sensng mage B. MapReduce remote sensng mage retreval Because the remote sensng mage and ts features are stored n HBase, when HBase data collecton s so large, long tme shall be spent on scannng the table as a whole. To reduce tme for mage retreval and enhance retreval effcency, conductng parallel computng on remote sensng mage retreval wth MapReduce computng module, the specfc frame s shown n fgure 3 and specfc mplementaton process s shown n fgure 4. map(key,value) Begn Csearch=ReadSearchCharact( ); //read features of remote sensng mage to be retreved Cdatabase=value; //read data n remote sensng features database Path = GetPcturePath( value) ; / /read mage route n remote sensng mage database SmByColor=CompareByColor(Csearch, Cdatabase) ; / /computng smlarty of remote sensng mage color SmByTexture = CompareByTexture(Csearch, Cdatabase); //computng smlarty of remote sensng mage texture Sm=SmByColor*w1 + SmByTexture*w; //computng matchng smlarty Commt(Sm,Path); End Reduce functon s defned as reduce(key,value): Begn Sort(key,value); //conductng sort on remote sensng mage accordng to sze of smlarty Commt(key,value); //key refers to the value of smlarty,value refers to route of smlar remote sensng mages End Fgure 3.Workng procedure for remote sensng mage retreval Steps for remote sensng mage retreval based on MapReduce are as follows: Step 1: Map stage. Read remote sensng mage to be retreved from HDFS cache and draw ts color and texture features, then match wth features n mage n HBase, map output s the value of <smlarty, mage ID >. Step : Conductng sort and redraw of all values of map outputs <smlarty, mage ID > and then nput to reducer agan. Step 3: Reduce stage. Collectng all of values of <smlarty, mage ID > and then conductng sort of smlarty on these values and wrtng N values nto HDFS. Step 5: Outputtng those mage IDs that are the most smlar to the remote sensng mages to be retreved. Map functon s defned as: Fgure 4.Process of remote sensng mage retreval based on MapReduce DOI /IJSSST.a ISSN: x onlne, prnt

4 IV. SYSTEM TEST AND ANALYSIS A. Test envronment Adoptng one man engne and 3 ordnary machnes to consst of one Hadoop dstrbuted system through Lnux envronment and ther confguraton s shown n table. There are 000 remote sensng mages collected totally. To make the result of remote sensng mage retreval put forward n ths thess more convncng, we conducted contrast experment adoptng B/S sngle node system. TABLE. CONFIGURATION OF VARIOUS NODES Nodes Operaton system IP CPU RAM Man engne Lnux Core X 3.3GHz 4G Ordnary1 Lnux Core GHz G Ordnary Lnux Core GHz G Ordnary3 Lnux Core GHz G B. Test analyss on storage performance Adoptng dfferent amount of remote sensng mages and the storage tme of mages under dfferent nodes s shown n fgure 5. It can be seen from fgure 5 that when the amount of remote sensng mages s less than 500, there s no bg dfference n storage tme between B/S sngle node system and Hadoop dstrbuted system and the advantage s not obvous. When the amount of remote sensng mages s more than 500, storage tme n B/S sngle node system has ncreased greatly whle slow n Hadoop dstrbuted system, ths shows that uploadng remote sensng mages nto HDFS wth MapReduce method wll enhance storage effcency. When the amount of mages s more than 000, storage tme n nodes and 3 nodes dstrbuted system show ncrease n ndex form, ths has shown that Map tasks are more than 3 at ths tme meanwhle t wll dstrbute many tasks on some nodes, however, one node can only execute one Map task n one tme, so t ncreases number of nodes n Hadoop dstrbuted system whch enhance executon effcency of remote sensng mage retreval system. mage database s small, mult-node retreval tme n Hadoop dstrbuted system s longer than that n B/S sngle node system and one node system, t s manly because conductng parallel computng adoptng mult-node and ncrease n the amount of calculaton and tme, when the number of mages s more than 1000, retreval tme of mages n mult-node dstrbuted system s obvously less than the sngle node, t s manly because advantage n conductng parallel computng wth MapReduce to dstrbute the task of remote sensng mage retreval to varous nodes whch ncreases effcency of remote sensng mage retreval. D. System load test Under 3 nodes, forwardng remote sensng mage retreval task to Hadoop dstrbuted system, testng load condtons of varous nodes under dfferent tme ponts and dfferent amounts, recordng CPU utlzaton ratos of varous nodes are shown n fgures 7 and 8 respectvely. Fgure 7. CPU utlzaton rato n processng 00 remote sensng mages It can be seen from fgure 7 that when the amount of mages processng(00)due to small amount of mages and only one Map task, t dstrbutes to node1 to process and fnshes at t5, node1 begns to execute Reduce task. Fgure 5. Change curve of storage tme for remote sensng mage. C. Test analyss on remote sensng mage retreval Remote sensng mage retreval tme consumpton n dfferent scale of remote sensng mage database under dfferent nodes s shown n fgure 6. It can be seen from fgure 6 that when the amount of mages n remote sensng Fgure 8.CPU utlzaton rato n processng 000 remote sensng mages It can be seen from fgure 8 that when the amount of remote sensng mages processng s large(000), because there are many Map tasks to be executed at the same tme, 3 Map tasks on 3 nodes at T1 and T3 have been fnshed, because Map task on node3 at T4 has been fnshed and s free, therefore executng Reduce task on node3 and ths has realzed executon of automatc transfer of node task n DOI /IJSSST.a ISSN: x onlne, prnt

5 heavy load to free node, whch has kept balance of system loads. Meanwhle due to collaboraton between Map and Reduce tasks, t has full taken advantage of data processng capacty on varous nodes and enhanced data effcency of varous nodes. E. Comparson of results of remote sensng mage retreval Conductng retreval on many categores of remote sensng mages wth Hadoop dstrbuted system and B/S sngle node system, the average retreval results are shown n table 3. It can be seen from the table that precson rato and recall rato of Hadoop dstrbuted system are superor to B/S sngle node system, whch shows that Hadoop dstrbuted system has enhanced retreval qualty of remote sensng mages. TABLE 3.COMPARISON OF RESULTS OF MANY CATEGORIES OF REMOTE SENSING IMAGES Hadoop dstrbuted Dfferent system B/S sngle node system categores precson rato(%) recall rato(%) precson rato(%) recall rato(%) Plantaton Wasteland Houses Lakes Rvers Roads and squares VI. CONCLUSION Amng at enormous amount of dffcultes of remote sensng mage retreval effcency n tradtonal methods, ths thess has put forward a remote sensng mage retreval algorthm based on MapReduce wth the advantage of Hadoop dstrbuted technology. The test result shows that the algorthm n ths thess could retreve remote sensng mages fast and accurately, whch not only enhances retreval effcency of remote sensng mages but ncreases retreval accuracy of remote sensng mage and has wde applcaton prospect n automatc retreval of remote sensng mages. REFERENCES [1] L Chao Feng, Zeng Sheng Gen, Xu Le, ntellgent processng of remote sensng mage, Bejng: Electroncs Industry Press, 007,pp [] Smpson, J. J., J. T. Mcntr. A Recurrent Neural Network Classfer for Improved Retrevals of Area Extent of Snow Cover. IEEE Transactons on Geoscences and Remote Sensng, 001, 39,pp [3] Smeulders A W.M., Worrng M, Santn S, et al. Content -based mage retreval at the end of the early years. IEEE Trans. On Pattern Analyss and Machne Intellgence. 000, (1),pp.1-3. [4] Guo Zh Qang Ca Song Classfcaton algorthm of colorful remote sensng mage and Matlab realzaton Wuhan Scence and Engneerng Unversty learned journal, 006,8(1), pp [5] Wang Xan We, Da Qng Yun, Jang Wen Chao, Cao Jang Zhong. Retreval method for appearance desgn patent mage.mn-sze computer system, 01, 33(3), pp [6] Sanjay Ghemawat,Howard Goboff,Shun-Tak Leung.The Googl Fle System. Pro-C eedngs of the 19th ACM Symposum on Operatng Systems Prncples.Bolton Landng:ACM,003,1,pp [7] Jeffrey Dean, Sanjay Ghemawat. MapReduce: a flexble data processng tool. Communcatons of The Acm, 010, 53( 1), pp [8] Tan Xa. Large-scale SMS messages mnng based on MapReduce. Proceedngs of the Internatonal Symposum on Computatonal Intellgence and Desgn,London, 008,13,pp [9] Konstantn Shvaclko, Harong Kuang, Sanjay Rada, et al. Hadoop dstrbuted fle system for the Grd. Proceedngs of the Nuclear scence Symposum Conference Record, IEEE, 009,pp DOI /IJSSST.a ISSN: x onlne, prnt