Two-Stage Data Dstrbuton for Dstrbuted Survellance Vdeo Processng wth Hybrd Storage Archtecture Yangyang Gao, Hatao Zhang, Bngchang Tang, Yanpe Zhu, Huadong Ma Bejng Key Lab of Intellgent Telecomm. Software and Multmeda, Bejng Unversty of Posts and Telecomm., Bejng, Chna Emal: gyyzyp@163.com; zht@bupt.edu.cn; zyjhtangtang@163.com; ypzhu@bupt.edu.cn; mhd@bupt.edu.cn Abstract Dstrbuted computng framework such as Hadoop s now a promnent soluton for the effcent bg data processng. However, the current data placement strateges n these frameworks lack n consderaton of the types of storage devces and the features of survellance vdeo tasks. So usng the current dstrbuted computng framework for large-scale vdeo processng n the cloud computng cluster wth hybrd storage archtecture wll lead to the low utlzaton of the hgh-performance devces and the skew of the vdeo task completon tme. In ths paper, we propose a novel Two- Stage Data Dstrbuton Approach (TSDDA) for optmzng the data placement n the cloud computng platform. Our approach adopts an accurate Processng Tme Predcton Model (PTPM) to estmate the processng tme of the vdeo task by ncorporatng the several mportant vdeo task features, the heterogeneous I/O capabltes and processng abltes n the hybrd cluster. In our approach, an ntal data placement algorthm s used to place the data needed by the vdeo processng s on the approprate processng nodes n the process of the ntal data loadng, and then a dynamc data mgraton algorthm s further used to mgrate the vdeo data from HDD to SSD for further mprovng the SSD s utlzaton and the processng effcency durng the process of the executon. Fnally, we conduct the extensve experments, and the expermental results verfy the accuracy of PTPM and show that the proposed TSDDA outperforms the current methods. Keywords-Vdeo survellance; data dstrbuton; data placement; data mgraton; hybrd storage archtecture I. INTRODUCTION Durng the last few years, the combnaton of ntellgent vdeo analyss technology and cloud computng s wdely accepted as a promsng soluton to deal wth the challenges of large-scale survellance vdeo analyss [1] [2]. Cloud computng platform can provde compute resources and storage resources dynamcally and manage all knds of workload for cloud applcatons. Nowadays, Sold-State Drve (SSD) s wdely used as an elemental storage devce n the cloud computng platform due to ther low access latency and hgh data bandwdth [3]. However, as the rato of cost versus performance of SSD s stll low compared wth Hard Dsk Drve (HDD) devces, SSD alone as a prmary storage s not the current manstream soluton. Hybrd storage strategy whch takes advantage of both SSD and HDD s used as a promsng practce to mprove the total storage performance n an economcal fashon [4], [5]. Apache Hadoop platform has become one of the most promnent frameworks to handle bg data analytcs n the cloud cluster [6], n whch, the data processng frameworks such as MapReduce and Spark are the key components for the large-scale data analytcs, and the underlyng storage s typcally provded through Hadoop Dstrbuted Fle System (HDFS). The current Apache Hadoop platform uses the round-robn scheme for data placement when the nodes are equpped wth multple storage devces, and makes no dstncton between SSD and HDD [3]. Even the recently released Hadoop can support user to defne where and how to store data n dfferent types of storage devces n the process of ntal data placement. However, when the system reads data, t does not consder the heterogenety of the storage devces. Usng the default data dstrbuton approach of Hadoop n the cloud cluster wth the hybrd storage archtecture for large-scale vdeo processng wll cause the low SSD utlzaton and the long task processng tme. Several recent lteratures [6], [9], [16] focus on supportng for the hybrd storage archtecture n Apache Hadoop platform for mprovng the effcency of the dstrbuted data processng. But they usually do not consder the features of survellance vdeo processng tasks and the cluster heterogenety smultaneously, and ths wll cause the unbalanced workload and the low task executon effcency. As the vdeo fles are dfferent on the duraton, the resoluton and the frame rate and the dfferent vdeo processng tasks have dfferent computatonal requrements, the processng tme of these vdeo fles may be dfferent sgnfcantly. Further, the heterogeneous node processng capablty results n a sgnfcant dfference of the executon tme of the dstrbuted processng tasks. Consequently, the straggler task wll serously defer the overall fnsh tme. In ths paper, nstead of usng the Apache Hadoop platform for large-scale vdeo processng, we frstly present a contaner-based survellance vdeo processng cloud platform. Secondly, we propose a two-stage data dstrbuton approach (TSDDA), whch ncludes the ntal data placement and the dynamc data mgraton, for mprovng the effcency of vdeo task processng. Specfcally, we defne a processng tme predcton model for the dstrbuted survellance vdeo processng tasks, whch ncorporates the mportant feature
parameters ncludng the duraton, the resoluton, the frame rate of vdeo, the node processng and storage capabltes. At the process of ntal data loadng, our platform can place the vdeo data to the approprate processng nodes accordng to the proposed tme predcton model. In the process of task executon, each node can adaptvely mgrate vdeo data from HDD to SSD for further mprovng SSD utlzaton and decreasng the tme of vdeo processng usng our dynamc data mgraton algorthm. Fnally, we mplement the proposed survellance vdeo processng platform and run the representatve vdeo synopss functon. The expermental results show that our platform can mprove the effcency of vdeo processng and SSD utlzaton than the current wdely used methods. The rest of the paper s organzed as follows. Secton II descrbes the archtecture and workflow of the system. We propose a two-stage data dstrbuton approach n Secton III. Secton IV presents the platform mplementaton and the expermental results. Secton V outlnes the related work. We conclude the paper n Secton VI. II. CONTAINER BASED DISTRIBUTED VIDEO PROCESSING SYSTEM Our dstrbuted vdeo processng platform s bult upon Docker contaner vrtualzaton technology. It s fully ntegrated wth the vdeo survellance system that comples wth ITU-T standard [12]. Fgure 1 llustrates the archtecture of the proposed dstrbuted vdeo processng platform. Fgure 1. System Archtecture. 1) VS: Vdeo Survellance (VS) system s a telecommuncaton subsystem focusng on vdeo applcaton technology, whch s used to remotely capture vdeo, and present them to end users [10]. The VS can send the requests of the vdeo processng tasks to the controller component for effcently processng the survellance vdeo n parallel. 2) Controller: Controller component s responsble for recevng the servce requests from the VS, assgnng the vdeo process tasks to the contaner cluster and forwardng the processng results generated by contaner cluster to the VS. 3) Contaner Cluster: Contaner cluster conssts of many low-cost heterogeneous SH-Servers. Each SH-Server s a physcal server whch s equpped wth both SSD and HDD, and s also confgured wth a Docker contaner engne for the creaton and destructon of contaner nstance whch ncorporates a specfc vdeo processng program to complete the assgned tasks, such as vdeo transcodng and object trackng. In addton, the node manager s nstalled n each SH-Server, and ts responsblty s to control the Docker engne to generate the contaner nstance for the vdeo tasks assgned to the node and perform the data mgraton n the process of task executon. 4) Image Regstry: Image regstry s an onlne mage repostory whch stores all mages, and each mage corresponds to a specal vdeo processng task. In ths paper, we focus on the workflow of the survellance vdeo processng n our dstrbuted vdeo processng platform. In the VS, Network Vdeo Recorder (NVR) s used to receve and store the dgtal vdeo streams from IP cameras. Frstly, the VS ntates a survellance vdeo servce processng request whch nvolves several metadata such as the type of vdeo processng task and the storage locaton of the data, and forwards ths request to the controller component. When the controller component receves the servce request from the VS, t analyzes the address of the vdeo data to be processed n the NVR and the vdeo task type, and then controls the data loadng from the NVR to the dstrbuted vdeo processng platform. The vdeo data s then placed n the approprate SH-Servers accordng to our proposed two-stage data dstrbuton approach whch can contrbute to the hgh SSD utlzaton and the local data processng. After all vdeo processng tasks are completed, the controller component forwards the vdeo analyss results to the VS. III. TWO-STAGE DATA DISTRIBUTION APPROACH A. Processng Tme Predcton Model For realzng effcent processng of the vdeo tasks and preventng the performance degradaton due to the unreasonable data dstrbuton strategy durng the executon, we need an accurate tme predcton model for each subtask of vdeo processng, and then use ths model to assgn the vdeo subtask on the nodes n our dstrbuted computng platform. The processng tme of a vdeo task manly ncludes the computng tme and the data I/O tme. The computng tme s manly related to three aspects ncludng the computng capablty of the node, the vdeo task type, and the vdeo characterstcs whch are respectvely the vdeo duraton, the vdeo resoluton, and the frame rate [10] n our paper. The I/O tme s manly related to the volume of the survellance vdeo data and the performance characterstcs of the storage devce. We defne the dfferent types of the survellance vdeo resoluton as R = {r 1, r 2,..., r m }. The dfferent frame
rates of the survellance vdeo are dvded nto the dfferent ranks, and are defned as F = {f 1, f 2,..., f n }. The dfferent vdeo task types are defned as Z = {z 1, z 2,..., z q }. The concept of the survellance vdeo qualty s defned as Q = R F = {(r 1, f 1 ), (r 1, f 2 ),..., (r m, f n )}, whch s the Cartesan product of R and F, and each element of Q stands for a basc vdeo qualty rank, called QR. The QR of a Survellance Vdeo Data Block (SVDB) belongs to the set Q. Further, we defne another set S = Q Z (1) whch s the Cartesan product of Q and Z, and each element of S stands for a unque Survellance Vdeo Processng Mold (SVPM) whch corresponds to a specfc vdeo task type z and a specfc basc vdeo qualty q, where z Z and q Q. For example, Q = {(r 1, f 1 ), (r 2, f 2 )}, Z = {z 1, z 2 }, so S = {(z 1, r 1, f 1 ), (z 1, r 2, f 2 ), (z 2, r 1, f 1 ), (z 2, r 2, f 2 )}. Based on the defnton of the set S, we defne another set D = {d s1, d s2,..., d sq m n }. (2) There s a one-to-one mappng relatonshp between the elements n the set D and the set S. And for each SVPM s j n set S, the element d sj stands for the average volume sze of the temporary data generated by processng the unt-tme vdeo data, and t can be obtaned by runnng a large number of SVDBs whose SVPM s s j and calculatng the average of the volume sze of the temporary data generated by each SVDB. It s assumed that there are N nodes n the cluster. Then we defne E c = {e c,,s 1,..., e c,s q m n } 1 N. (3) and for any node c, the value of e c,s j stands for the average computng tme for processng unt-tme vdeo data whose SVPM s s j, and t can be obtaned by runnng a large number of SVDBs whose SVPM s s j n the node c and calculatng the average of the computng tme of each SVDB. For an SVDB, based on the above defntons, we can obtan the processng tme of an SVDB when processng t n the node c j usng the followng equaton P,cj = e cj,s k ts + W + d sk ts r cj + d s k ts w cj (4) where s k (s k S) s the SVPM of the th SVDB and e cj,s k s the average processng tme of the unt-tme vdeo data that corresponds to the th SVDB. ts = t 2 t1, and t1 and t 2 are respectvely the start tme and the stop tme of the th SVDB. d sk (d k D) s the average volume sze of the temporary data generated by processng unt-tme vdeo data that corresponds to the th SVDB. W s the volume sze of the th SVDB. r cj and w cj are respectvely the average data readng/wrtng rate of the node c j. For the node that has both SSD and HDD, the (r cj, w cj ) should be n two sets. One stands for the average data readng/wrtng rate of SSD n the node c j, and another one stands for the average data readng/wrtng rate of HDD n the node c j. The value of r cj and w cj for the node c j can be obtaned by usng the popular benchmark hdparm [13]. We defne Φ,cj = ts r cj + ts w cj and ξ,cj = W r cj respectvely, then the equaton (4) can be expressed as follows. P,cj = ts e cj,s k + Φ,cj d sk + ξ,cj (5) where ts, Φ,cj and ξ,cj are all constants assocated wth the average I/O rate of the node c j and the duraton of the th SVDB. A vdeo processng task usually relates to multple SVDBs. The processng tme of a vdeo task runnng n the node c j can be obtaned usng the followng equaton T cj = M P,cj (6) =1 where M s the number of the SVDBs to be processed. At ths pont, we buld the PTPM to predct the processng tme of a vdeo processng n the cluster wth the hybrd storage archtecture. B. Intal Data Placement In ths subsecton, we descrbe the ntal data placement algorthm based on the PTPM proposed above n the process of the vdeo data loadng from NVRs to our contaner cluster. Assume there are k vdeo processng s each of whch relates to a group of SVDBs. For example, the functon of one s to run the vdeo synopss for the vdeo blocks collected from the same survellance camera. The objectve of the ntal data placement algorthm s to place each group of SVDBs related to one vdeo on the approprate storage medum so that processng effcency of the s can be mproved. We defne the computng nodes n the dstrbuted computng platform as C = {c 1, c 2,..., c N }, each node s expressed as c = (ssd, hdd, ssd used, hdd used ) ( = 1,..., N), n whch ssd and hdd stand for the, ssd tmp, hdd tmp sze of the storage spaces of SSD and HDD respectvely, ssd used and hdd used stand for the used storage spaces of SSD and HDD respectvely, ssd tmp and hdd tmp stand for the allocated storage spaces of SSD and HDD respectvely for placng the temporary data generated by the vdeo processng s. J = { 1, 2,..., k } s defned as the set. Further, DBS = {db 1, db 2,..., db k } s defned as the ordered data block group set n whch db s related wth the n the set J ( = 1,..., k).
The ntal data placement adopts the mnmum load prorty matchng strategy. The basc prncple of ths algorthm s to select a from set J n turn. And for each, we obtan the canddate node set C n whch each node meets the storage need of the, select the node whch has the current lowest load from C, and then place the data block group db on t. Step 1: We defne the character constant S whch stands for SSD meda, the character constant H whch stands for HDD meda, and ntalze ssd used., hdd used., ssd tmp., hdd tmp., c load. whch stands for the load of the node to zero for all nodes, selectmeda. for all node s set to S, selectmeda. stands for the type of storage medum n the node for placng data. Step 2: Select from the set J and remove t from J. Step 3: Compute the total volume sze of the temporary data generated by, whch s defned as db tmp followng equaton db tmp = usng K d sk ts j (7) j=1 where ts j = t 2 j t1 j, and t1 j and t2 j are respectvely the start tme and the stop tme of the jth SVDB n db and d sk (d sk D) s the average volume sze of temporary data generated by processng the jth SVDB whose SVPM s s k (s k S). Step 4: For each node c j, calculate the remanng storage spaces respectvely f placng the data db on SSD or HDD usng the followng equaton (8) ssd j {ssd used j or the equaton (9) hdd j {hdd used j + db + max{0, db tmp + db + max{0, db tmp ssd tmp j }} (8) hdd tmp j }}. (9) If the value of the equaton (8) 0, then add the node c j to the set C and update the varable selectmeda j = S. If the value of the equaton (8) < 0 and the value of the equaton (9) 0 then add the node c j to the set C and update the varable selectmeda j = H. Step 5: Intalze the varable = N one, the varable Meda = S, and load mn = Infnte, where Infnte s a great value. s the node for processng, Meda s the storage meda (SSD or HDD) for placng db and db tmp n the node. Step 6: Select the node c j from C and remove t from C. Step 7: Calculate the processng tme T cj for accordng to PTPM and then obtan the upcomng load c upload j = c load j + T cj. (10) Step 8: If c upload j < load mn, then update load mn = c upload j, = c j, and Meda = selectemeda j. Step 9: Repeat Step 5 Step 8 untl C s empty. Step 10: If M eda equals to S, then place the data group db nto the SSD medum of the node, and update the value of ssd used usng the followng equaton (11) ssd used =ssd used ssd tmp db + max(db tmp +, ssd tmp ), (11) and the value of ssd tmp usng the followng equaton (12) ssd tmp = max{dbtmp, ssd tmp }, (12) else place the data group db nto the HDD medum of the node and update the value of hdd used usng the followng equaton (13) hdd used =hdd used hdd tmp db + max(db tmp +, hdd tmp ), (13) and the value of hdd tmp usng the followng equaton (14) hdd tmp = max{dbtmp, hdd tmp }. (14) Then, update c load = load mn. Step 11: Repeat Step 2 Step 10 untl J s empty, namely all the data block groups n the set DBS have been dealt wth. The ntal data placement algorthm s gven n Algorthm 1. C. Dynamc Data Mgraton In ths subsecton, we detal the dynamc data mgraton algorthm. The basc prncple s to mgrate the vdeo data from the HDD to the SSD once the SSD has enough free space n the process of the task executon and ths can further mprove the SSD s utlzaton and the processng effcency of s. After the data loadng, the node manager n each node starts to control the Docker engne to create contaner nstance to process vdeo data. For each node c j n the cluster, the set s dvded nto three subset: set ssd, sethdd and set pror. The data group related to the each n set ssd s placed nto the SSD storage medum. The data group related to each n set hdd s placed nto the HDD storage medum. And set pror s empty n the ntal stage. Assume that the currently completed vdeo task n the node c j s defned as cur and ts processng tme s defned as Tcur whch can be obtaned accordng to PTPM. stands for the vdeo processng that wll be from the set by the node manager for further processng, and T s ts processng tme. dom grate ndcates whether a data mgraton task s currently n
Algorthm 1 Intal Data Placement Algorthm 1: for any c j C do 2: ssd used j = 0, ssd tmp j = 0, hdd used j = 0, hdd tmp j = 0, c load j = 0, selectmeda j = S; 3: end for 4: whle J s not empty do 5: J.get(); 6: Calculate db tmp of accordng to the equaton (7); 7: for any c j C do 8: Calculate the remanng storage space f place the data of on the SSD n the node c j usng the equaton (8); 9: f SSD n the node c j meets the storage need for then 10: Add c j to C ; selectmeda j = S; contnue; 11: end f 12: Calculate the remanng storage space f place the data of on the HDD n the node c j usng the equaton (9); 13: f HDD n the node c j meets the storage need for then 14: Add c j to C ; selectmeda j = H; 15: end f 16: end for 17: = None, Meda = S, load mn = Infnte; 18: whle C s not empty do 19: c j C.get() 20: Calculate the processng tme T cj for accordng to PTPM and then calculate the upcomng load c upload j usng the equaton (10); 21: f c upload j < load mn then 22: Update load mn = c upload j, = c j and Meda = selectemeda j; 23: end f 24: end whle 25: f Meda equals to S then 26: Place the data group db nto the SSD medum of the node and update ssd used and ssd tmp accordng to the equatons (11) and (12); 27: else 28: Place the data group db nto the HDD medum of the node and update hdd used accordng to equatons (13) and (14); 29: end f 30: Update c load = load mn; 31: end whle and hdd tmp progress and db mgrate cur s the correspondng data group that wll be mgrated from HDD to SSD for further processng, Tcur mgrate s the remanng tme for completng the current data mgraton task. db mgrate s the data group from HDD for the data mgraton task and T mgrate s the tme for completng the data mgraton for db mgrate. The node manager selects and db mgrate accordng to our dynamc data mgraton algorthm once the current cur s completed. The detals of the dynamc data mgraton algorthm are as follows. Step 1: If domgrate s T rue, namely, a data mgraton task s currently n progress, then update the value of the remanng tme for the current data mgraton task accordng to T mgrate cur = T mgrate cur T cur. (15) If Tcur mgrate < 0, then update domgrate = F alse. Step 2: If set pror s not empty, then select from set pror, and remove t from set pror. Calculate the processng tme T of accordng to PTPM, and goto Step 12. Step 3: If set ssd s not empty, then select from the set set sdd, remove the from setsdd, and calculate the processng tme T of accordng to PTPM. Step 4: If domgrate s T rue, then goto Step 12, else goto Step 5. Step 5: Select j from set hdd whch meets and ssd used c db cur + db j <= ssd c (16) ssd tmp c db tmp j >= 0 (17) and has the maxmum processng tme. Step 6: If j s not None, calculate the data mgraton tme accordng to T mgrate = max{ db j w sdd c, db j r hdd c }, (18) else goto Step 11. Step 7: For all s n set ssd, calculate the total processng tme accordng to T ssd total = T, set ssd. (19) If Ttotal ssd T mgrate, then update dom grate = T rue, db mgrate = db j, and goto Step 12, else goto Step 8. Step 8: For all s n set hdd except j, calculate the total process tme accordng to T hdd total = T, set hdd. (20) If Ttotal hdd T mgrate, then goto Step 9 else goto Step 11. Step 9: Sort the s n set hdd except j accordng to the descendng order of the processng tme, and then select the frst n s whch meet and n =1 T n+1 T =1 T mgrate (21) T mgrate, (22) remove them from set hdd to setpror, update domgrate = T rue.
Step 10: If set pror then select from set pror from set pror. Step 11: If set hdd then select from set hdd s not empty and s None,, and remove the s not empty and s None,, and remove the from set hdd. Step 12: Update cur =, = None. If domgrate s T rue and db mgrate s not None, then update db mgrate cur = db mgrate, db mgrate = None. Start the data mgraton task whch mgrates db mgrate cur from HDD to SSD and controls the Docker engne to create the contaner nstance to start the cur smultaneously. The dynamc data mgraton algorthm s gven n Algorthm 2. A. System Implementaton IV. EXPERIMENTAL RESULTS Before performng the performance valdaton, we ntroduce the system mplementaton detals and the expermental envronment. Our system s mplemented accordng to the archtecture presented n Secton II. In our prevous work [11], we have mplemented a standard vdeo survellance system. The dstrbuted vdeo processng platform s bult on a cluster wth 10 nodes and the confguratons of the nodes are shown n Table I. By usng hdparm [13], we obtan the average data readng and wrtng rate for SSD n each node are 1149.66(M/Sec) and 865.54(M/Sec) respectvely, and the average data readng and wrtng rate for HDD n each node are 199.82(M/Sec) and 197.78(M/Sec) respectvely. The operatng system used n each server s Ubuntu 14.04.1 LTS. The expermental vdeo data s obtaned from the real vdeo survellance system deployed n Fuzhou, Chna. In our experments, we have mplemented the representatve vdeo synopss applcaton for evaluatng our two-stage data dstrbuton approach. B. Expermental Analyss For verfyng the accuracy of PTPM, we choose a survellance vdeo data set whch has 3200 SVDBs. The frame rate of the vdeo s 25 fps. There are three knds of the vdeo resoluton, whch are respectvely L(640*360), M(1280*720), and H(1920*1080). The vdeo block duraton s respectvely 1320s and 2640s. Then, the set of SVPM s S = {(V S, L), (V S, M), (V S, H)}. Frstly, we calculate the value of d sj for each SVPM s j n the set S and the value of each e c,sj by processng 2000 SVDBs n each node. Then, we use the PTPM to predct the processng tme of other SVDBs for vdeo synopss applcaton and compare the predcted tme wth the practcal values measured by the offlne processng and the values of CTPM [10] whch predct the vdeo processng tme only accordng to the CPU computng tme. In order to mnmze the nfluence of Algorthm 2 Dynamc Data Mgraton Algorthm 1: f domgrate s T rue then 2: Update Tcur mgrate accordng to the equaton (15) 3: f Tcur mgrate 0 then 4: Update domgrate = F alse; 5: end f 6: end f 7: f set pror s not empty then 8: set pror.get(); Start processng ; break; 9: end f 10: f set ssd s not empty then 11: set ssd.get(); 12: f domgrate s T rue then 13: Start processng ; break; 14: end f 15: end f 16: Select j from set hdd whch meets the constrants (16) and (17) and has maxmum processng tme; 17: f j s None then 18: f s None and set hdd s not empty then 19: set hdd.get(); 20: end f 21: Start processng ; break; 22: end f 23: Calculate the data mgraton tme accordng to the equaton (18); 24: Calculate the total processng tme Ttotal ssd for all s n set ssd accordng to the equaton (19); then 26: Update domgrate = T rue, db mgrate 25: f T ssd total T mgrate data group db mgrate smultaneously; break; 27: end f = db j; Mgrate the from HDD to SSD and start processng 28: Calculate the total processng tme Ttotal hdd for all s n set hdd accordng to the equaton (20); then 30: Sort s n set hdd accordng to the processng tme n descendng order, and then select the frst n s whch meet the constrants (21) and (22). Remove them from set hdd to set pror ; 31: f s None then 29: f T hdd total T mgrate 32: set pror.get(); 33: end f 34: Update domgrate = T rue, db mgrate data group db mgrate smultaneously; break; 35: end f 36: f s not None then 37: Start processng ; 38: end f = db j; Mgrate the from HDD to SSD and start processng the unstable system factors, we use the average processng tme of each qualty of vdeo for accuracy comparson. As shown n Fgure 2, we can get the concluson that the PTPM can accurately estmate the vdeo processng tme under the varous expermental confguratons. In order to verfy the hgh SSD utlzaton and the effectveness of our two-stage data dstrbuton approach for the survellance vdeo processng. We frst construct the four representatve vdeo data set wth the dfferent number of
Table I PHYSICAL SERVERS. Physcal Servers Instance Type CPU SSD HDD number 1 24 core 128GB 1T 2 2 24 core 128GB 2T 3 3 32 core 128GB 2T 2 4 44 core 128GB 2T 2 5 44 core 128GB 1T 1 Fgure 3. Average task processng tme. Fgure 2. Accuracy of PTPM. SVDBs and the dfferent vdeo qualty confguratons. The data set 1 has 830 SVDBs whch nclude 280 L25 SVDBs, 300 M25 SVDBs, and 250 H25 SVDBs. The data set 2 has 860 SVDBs whch nclude 300 L25 SVDBs, 310 M25 SVDBs, and 250 H25 SVDBs. The data set 3 has 920 SVDBs whch nclude 300 L25 SVDBs, 350 M25 SVDBs, and 270 H25 SVDBs. The data set 4 has 970 SVDBs whch nclude 320 L25 SVDBs, 350 M25 SVDBs, and 300 H25 SVDBs. For each data set, we run the vdeo synopss applcaton and compare our two-stage data dstrbuton approach wth the default data dstrbuton approach n Hadoop and IDPA [10] whch assgns the task whch has the maxmum processng tme to the node whch has the mnmum load and does not consder the type of storage medum of each node n the cluster. We run the vdeo processng s n parallel by usng these three approaches respectvely. In ths experment, there are no other extra loads n the cluster except the vdeo processng s. For reducng the nfluence of the unstable system factors, we repeat the executon of each 6 tmes and measure the executon tme of each. The average executon tme of each data set for the vdeo synopss applcaton s gven n Fgure 3. As shown n the fgure, we can fnd that our strategy can sgnfcantly decrease the average executon tme compared wth the default data dstrbuton strateges n Hadoop and IDPA. In order to verfy the hgh SSD utlzaton of our data dstrbuton approach n the process of the vdeo processng, we collect and calculate the SSD utlzaton perodcally n the process of processng data set 3 and data set 4 n the node whose confguraton s the nstance 3 shown n table I based on our approach, the default data dstrbuton approach and (a) (b) Fgure 4. SSD utlzaton. (a) SSD utlzaton for data set 3. (b) SSD utlzaton for data set 4. IDPA respectvely. As shown n Fgure 4, compared wth the other two strateges, our method has hgher SSD utlzaton. V. RELATED WORK For the large-scale data processng n cloud computng platform, data placement becomes a fundamental research task due to the rapd ncrease of accessble large datasets. Kosar et al. [14] proposed a framework for dstrbuted computng systems, whch consdered both the data placement and computaton modules as two separate subsystems. In ths framework, the data placement module was proposed as an ndependent that can be scheduled and managed lke the computaton s. In [15], the workflows were modeled
by the hypergraph concept and a hypergraph parttonng technque, k-way parttonng, was appled to mnmze the cut sze. In that way, they clustered the workflow tasks as well as ther requred data n the same executon ste. The lterature [10] proposed a proactve vdeo data placement strategy for processng the large-scale vdeo data n the heterogeneous survellance vdeo processng system whch places the vdeo data on the approprate node, and t took the vdeo task features and the heterogeneous node processng capabltes nto account. However, they dd not consder the heterogenety of the storage medum n the cluster n whch each node was equpped wth both SSD and HDD. A few recent efforts [6], [9], [16] all focus on enablng support for the hybrd storage archtecture n Hadoop e- cosystem for mprovng the effcency of the dstrbuted data processng. They proposed to re-desgn HDFS nto a mult-tered storage system that seamlessly ntegrated the heterogeneous storage technologes, such as HDD and SDD, nto HDFS, and proposed the data placement and retreval strateges n order to explot the dfferent storage devces. However, they wll not work effectvely n the dstrbuted survellance vdeo computng envronment, because the features of the vdeo processng tasks are not taken nto account for placng the task data on the approprate nodes. VI. CONCLUSIONS In ths paper, for mprovng the effcency of the dstrbuted survellance vdeo s n the cloud platform wth hybrd storage, we propose a novel TSDDA whch takes the vdeo task features, the heterogeneous node processng and storage capabltes nto account. Frstly, a processng tme predcton model s presented to gude the approprate data assgnment n the cluster nodes wth the hybrd storage archtecture. Then, we present an ntal data placement algorthm whch s used to place the vdeo data on the approprate node n the process of the ntal data loadng and a dynamc data mgraton algorthm whch s used to mgrate the vdeo data from HDD to SSD durng the process of the executon. The expermental results show that the proposed two-stage data dstrbuton approach can mprove the SSD s utlzaton and the effcency of the survellance vdeo processng sgnfcantly compared wth the current data dstrbuton approaches. ACKNOWLEDGMENT Ths work s supported by the natonal key research and development plan (Grant No.2016YFC0801005); Natonal Natural Scence Foundaton of Chna (No. 61300013). REFERENCES [1] Y. H. Xong, S. Y. Wan, Y. He, and D. Su, Desgn and mplementaton of a prototype cloud vdeo survellance system, J. Adv. Comput. Intell. Intell. Informatcs, vol. 18, no. 1, pp. 40-47, 2014. [2] X. M. Zhao, H. D. Ma, H. T. Zhang, Y. Tang, and Y. Kou, HVPI: Extendng Hadoop to Support Vdeo Analytc Applcatons, IEEE Cloud 2015, pp. 789-796, 2015. [3] W. Tan, L. Fong, and Y. Lu, Effectveness Assessment of Sold-State Drve Used n Bg Data Servces, 2014 IEEE Internatonal Conference on Web Servces, pp. 393-400, 2014. [4] H. Sh, R. V. Arumugam, C. H. Foh and K. K. Khang, Optmal Dsk Storage Allocaton for Multter Storage System, IEEE Trans. on Magnetcs, vol. 49, no. 6, pp. 2603-2609, 2013. [5] X. Huang, Yzh Huang, Y. Lu, Renfa L and Xn Peng, A strp level data layout strategy for heterogeneous parallel storage systems, 11th Internatonal Conference on Natural Computaton, pp. 1085-1091, 2015. [6] N. S. Islam, X. Lu, M. Was-ur-Rahman, D. Shankar and D. K. Panda, Trple-H: A Hybrd Approach to Accelerate HDFS on HPC Clusters wth Heterogeneous Storage Archtecture, 15th IEEE/ACM Internatonal Symposum on Cluster, Cloud and Grd Computng, pp. 101-110, 2015. [7] S. Wu, G. Chen, K. Chen, F. L and L. Shou, HM: A Column- Orented MapReduce System on Hybrd Storage, IEEE Trans. on Knowledge and Data Engneerng, vol. 27, no. 12, pp. 3304-3317, 2015. [8] H. Roh, S. Park, S. Km, M. Shn, and S.-W.Lee, B+-tree ndex optmzaton by explotng the nternal parallelsm of flash-based sold state drves, Proc. VLDB Endowment, vol. 5, no. 4, pp. 286-297, Dec. 2011. [9] K. R. Krsh, B. Wadhwa, M. S. Iqbal, M. M. Rafque and A. R. Butt, On Effcent Herarchcal Storage for Bg Data Processng, 16th IEEE/ACM Internatonal Symposum on Cluster, Cloud and Grd Computng, pp. 403-408, 2016. [10] H. T. Zhang, B. Xu, J.Yan, L. J. Lu, H. D. Ma, Proactve Data Placement for Survellance Vdeo Processng n Heterogeneous Cluster, IEEE Cloudcom 2016, pp. 206-213, 2016. [11] H. T. Zhang, H. D. Ma, G. P. Fu, X. D. Yang, Z. Jang, Y. Y. Gao, Contaner based Vdeo Survellance Cloud Servce wth Fne-Graned Resource Provsonng, IEEE Cloud 2016, pp. 758-765, 2016. [12] ITU-T H.626 standard, Archtectural requrements for vsual of survellance, 2011. [13] hdparm, https://sourceforge.net/projects/hdparm/. [14] T. Kosar and M. Lvny, A Framework for Relable and Effcent Data Placement n Dstrbuted Computng Systems, Journal of Parallel and Dstrbuted Computng, vol.65, no. 10, pp. 1146-1157, 2005. [15] U. V. Catalyurek, K. Kaya and B. Ucar, Integrated Data Placement and Task Assgnment for Scentfc Workflows n clouds, The fourth nternatonal workshop on Data-ntensve dstrbuted computng, pp. 45-54, 2011. [16] S. Wu, G. Chen, K. Chen, F. L and L. Shou, HM: A Column-Orented MapReduce System on Hybrd Storage, IEEE Trans. on Knowledge and Data Engneerng, vol. 27, no. 12, pp. 3304-3317, 2015.