On Secure Distributed Data Storage Under Repair Dynamics

Transcription

1 On Secure Distributed Data Storage Under Repair Dynamics Sameer Pawar, Salim El Rouayheb, Kannan Ramchandran University of California, Bereley s: Abstract We address the problem of securg distributed storage systems agast passive eavesdroppers that can observe a limited number of storage nodes. An important aspect of these systems is node failures over time, which demand a repair mechanism aimed at matag a targeted high level of system reliability. If an eavesdropper observes a node that is added to the system to replace a failed node, it will have access to all the data downloaded durg repair, which can potentially compromise the entire formation the system. We are terested determg the secrecy capacity of distributed storage systems under repair dynamics, i.e., the maximum amount of data that can be securely stored and made available to a legitimate user with revealg any formation to any eavesdropper. We derive a general upper bound on the secrecy capacity and show that this bound is tight for the bandwidth-limited regime which is of importance scenarios such as peer-to-peer distributed storage systems. We also provide a simple explicit code construction that achieves the capacity for this regime. I. INTRODUCTION Data storage devices have evolved significantly sce the days of punched cards. Nevertheless, storage devices, such as hard diss or flash drives, are still bound to fail after long periods of usage, risg the loss of valuable data. To solve this problem and to crease the reliability of the stored data, multiple storage nodes can be networed together to redundantly store the data, thus formg a distributed data storage system. Applications of such systems are numerable and clude large data centers and peer-to-peer storage systems, such as OceanStore [], that use a large number of nodes spread widely across the Internet to store files. Codes for protectg data from erasures have been well studied classical channel codg theory, and can be used here to crease the reliability of distributed storage systems. Fig. illustrates an example where a maximal distance separable (MDS) code is used to store a file F of 4 symbols, (a,a,b,b ) F 4 5, distributively on 4 nodes, v,...,v 4, each of capacity symbols. The MDS code implemented here ensures that any user, also called data collector, connectg to any storage nodes can obta the whole file F. However, what distguishes the scenario here from the erasure channel counterpart is that when a storage node fails, it needs to be repaired or replaced by a new node order to mata a desired level of system reliability. A straightforward repair mechanism would be to add a new replacement node of capacity, and mae it act as a data collector by connectg This research was funded part by an AFOSR grant (FA ), a DTRA grant (HDTRA ), and an NSF grant (CCF ). File F a,a b,b v a,a v b,b v 3 a + b a + b v 4 a +b a + b v 5 a + a a +4a Fig.. An example of a distributed data storage system under repair. A file F of 4 symbols (a,a,b,b ) F 4 5 is stored on four nodes usg an MDS code. Node v fails and is replaced by a new node v 5 that downloads (b + b ), (a + a + b + b ) and (a +4a +b +b ) from v, v 3, and v 4 respectively to compute and store (a + a,a +4a ). Nodes v,...,v 5 form a new MDS code. The edges the graph are labeled by their capacities. The figure also depicts a data collector connectg to nodes v and v 4 to recover the stored file. to survivg nodes. The new node can then download the whole file (4 symbols) to construct the lost part of the data and store it. Another repair scheme that consumes less bandwidth is depicted Fig. where node v fails and is replaced by node v 5. When node v 5 connect to 3 nodes stead of, it is possible to decrease the total repair bandwidth from 4 to 3 symbols. Note that v 5 does not need to store the exact data that was on v ; the only required property is that the data stored on all the active nodes v,v 3,v 4 and v 5 form an MDS code. The above important observations were the basis of the origal wor of [] where the authors showed that there exists a fundamental tradeoff between the storage capacity of each node and the repair bandwidth. They also troduced and constructed regeneratg codes as a new class of codes that generalize classical erasure codes and permit the operation of a distributed storage system at any pot on the tradeoff curve. When a distributed data storage system is formed usg nodes widely spread across the Internet, e.g., Internet based peer-to-peer systems, dividual nodes may not be secure and can become susceptible to eavesdroppg. This paper focuses on such scenarios where an eavesdropper can ga access to a certa number of the storage nodes. The compromised distributed storage system is always assumed to be dynamic with nodes contually failg and beg repaired. Thus, the compromised nodes can belong to the origal set of storage

2 nodes that the system starts with, or even clude some of the replacement nodes added to the system to repair it from failures. Under this settg, we are terested determg how much data can still be stored the system with revealg any formation to any of the eavesdroppers. To answer this question, we follow the approach of [] and model the distributed storage system as a multicast networ that uses networ codg. Under this model, the eavesdropper is an truder that can access a fixed number of the networ nodes of her choice. This eavesdropper model is natural for distributed storage systems and comes contrast with the wiretapper model studied the networ codg literature [3], [4], [5] where the truder can observe networ edges, stead of nodes. We derive a general upper bound on the secrecy capacity as a function of the node storage capacity and the repair bandwidth. Motivated by system considerations, we defe an important operatg regime, that we call the bandwidth-limited regime, where the repair bandwidth is constraed not to exceed a given upper bound, while no limitation is imposed on the storage capacity of the nodes. For this important operatg regime, we show that our upper bound is tight and present capacity-achievg codes. This paper is organized as follows. In Section II we describe the system and security model. We defe the problem and give a summary of our results Section III. In Section IV we illustrate two special cases of distributed storage systems that are structive understandg the general problem. In Section V, we derive an upper bound on the secrecy capacity, and Section VI, we present a scheme that achieves this upper bound for the case of bandwidth-limited regime. We conclude Section VII. A. Distributed storage system II. MODEL A distributed storage system (DSS) is a collection of storage nodes that cludes a source node s, that has an compressible data file F of R symbols, or units, each belongg to a fite field F. The source node is connected to n storage nodes v,...,v n, each with a storage capacity of symbols, which may be utilized to save coded parts of the file F. The storage nodes are dividually unreliable and may fail over time. To guarantee a certa desired level of reliability, we assume that the DSS is required to always have n active, i.e., non-failed, storage nodes that are service. Therefore, when a storage node fails, it is immediately replaced by a new node with same storage capacity. The DSS should be designed a way to allow any legitimate user, that we also call data collector, that connects to any of the n active storage nodes available at any given time, to be able to reconstruct the origal file F. We term this condition as the reconstruction property of distributed storage systems. We assume that nodes fail one at a time, and we denote by v n+i the new replacement node added to the system to repair the i-th failure. The new replacement node connects then to some d nodes, chosen randomly, of the remag active n nodes and downloads γ units from them total, which corresponds to the repair bandwidth of the system. The repair degree d is a system parameter satisfyg d n. In this wor, we focus on the case of symmetrical repair where the new node downloads equal amount of data, say units, from each of the d nodes it connects to, i.e., γ = d. The process of replenishg redundancy to mata the reliability of a DSS is referred to as the regeneration or repair process. Note that a new replacement node may download more data than what it actually stores. Moreover, the stored data can possibly be different than the one that was stored on the failed node, as long as the reconstruction property of the DSS is retaed. A distributed storage system D is thus characterized as D(n, ). For stance, the DSS depicted Fig. corresponds to D(4, ) which is operatg at (, γ) =(, 3). B. Flow Graph Representation We adopt the flow graph model troduced [] which we describe here for completeness. In this model, the distributed storage system is represented by an formation flow graph G. The graph G is a directed acyclic graph with capacity constraed edges that consists of three ds of nodes: a sgle source node s, put storage nodes x i and put storage nodes x i and data collectors j for i, j {,,...}. The source node s has an formation S of which a specific realization is the file F. Each storage node v i the DSS is represented by two nodes x i and xi joed by a directed edge of capacity (see Fig. ), to account for the node storage constrat. The repair process is itiated every time a failure occurs. As a result, the DSS, and consequently the flow graph, are dynamic and evolve with time. At any given time, each node the graph is either active or active dependg on whether it has failed or not. The graph G starts with only the source node s beg active and connected to the storage put nodes x,...,xn by gog edges of fite capacity. From this pot onwards, the source node s becomes and remas active and the n put and put storage nodes become active. When a node v i fails a DSS, the correspondg nodes x i and xi become active G. If a replacement node v j jos the DSS the process of repairg a failure and connects to d active nodes v i,...,v id, the correspondg nodes x j and x j, with the edge (x j,xj ), are added to the flow graph G, and node x j is connected to the nodes xi,...,x i d by comg edges of capacity each. A data collector is represented by a node connected to active storage put nodes through fite capacity ls enablg it to reconstruct the file F. The graph G constitutes a multicast networ with the data collectors as destations. An underlyg assumption here is that the flow graph correspondg to a distributed storage system depends on the sequence of failed nodes. As an example, we depict Fig. the flow graph correspondg to the DSS D(4, ) of Fig., when node v fails. C. Eavesdropper Model We assume the presence of an truder Eve the DSS, who can observe up to, <,nodes of her choice among all the storage nodes, v,v,..., possibly at different time stances as the system evolves. In the flow graph model, Eve is an eavesdropper who can access a fixed number of nodes chosen from the storage put nodes x,x,... Notice that while a data collector observes put storage nodes, i.e., the

3 s x x x 3 x 4 v v v 3 v 4 x x x 3 x 4 = x 5 = Fig.. The flow graph model of the DSS D(4, ), with d =3, of Fig. when node v fails and is replaced by node v 5. Each storage node v i is represented by two nodes x i and xi connected by an edge (xi,xi ) of capacity representg the node storage constrat. A data collector connectg to nodes v and v 4 is also depicted. data stored on the nodes it connects to, Eve, has access to put storage nodes, and thus can observe, addition to the stored data, all comg messages to these nodes. We also assume that Eve has complete nowledge of the storage and repair schemes implemented the DSS. Thus, she can choose some of the nodes to be among the itial n storage nodes, or, if she deems it more profitable, she can choose to wait for failures and eavesdrop on a replacement node by observg its downloaded data. Eve is assumed to be passive, and only observes the data with modifyg it. v 5 x 5 III. PROBLEM STATEMENT AND RESULTS A. Secrecy Capacity Let S be a random vector uniformly distributed over F R q, representg the compressible data file at the source node with H(S) = R. Let V := {x,x,...} and V := {x,x,...} be the sets of put and put storage nodes G respectively. For a storage node v i, let D i and C i be the random variables representg its downloaded messages and stored content respectively. Thus, C i, represents the data that can be downloaded by a data collector when contactg node v i, while D i, with H(D i ) γ, represents the total data revealed to Eve when she accesses node v i. The stored data C i is a function of the downloaded data D i. Let V a be the collection of all subsets of V of cardality consistg of nodes that are simultaneously active at some stant time. For any subset B of V, defe C B := {C i : x i B}. Similarly, for any subset E of V, defe D E := {D i : x i E}. The reconstruction property, then, can be written as H(S C B ) = 0 B V a, () and the perfect secrecy condition implies H(S D E ) = H(S), E V and E. () Given a DSS D(n, ) with compromised nodes, its secrecy capacity, denoted by C s (, γ), is then defed to be the maximum amount of data that can be stored this system such that the reconstruction property and the perfect secrecy condition are simultaneously satisfied for all possible data collectors and eavesdroppers i.e., C s (, γ) := sup H(S C B )=0 H(S D E )=H(S) where B V a, E V and E. B. Results B E H(S) (3) First, we give the followg general upper bound on the secrecy capacity of a DSS: Theorem : [Upper Bound] For a distributed data storage system D(n, ), with a repair degree d, and <compromised nodes, the secrecy capacity is upper bounded as C s (, γ) m{(d i + ),}, (4) where γ = d. Next, we consider an important operational regime, namely the bandwidth-limited regime, where the repair bandwidth γ is constraed to a maximum amount Γ, i.e., γ Γ, while no constrat is imposed on the storage capacity at each node. The secrecy capacity this regime is defed as, C BL s (Γ) := sup γ Γ, 0 C s (, γ). (5) For a fixed Γ, when the parameter d is a system design choice, the upper bound of Theorem on the secrecy capacity can be further optimized, and attas a maximum for d = n. In section VI, we demonstrate that this upper bound can be achieved for d = n the bandwidth-limited regime. Thus, establishg the followg theorem: Theorem : [Bandwidth-Limited Regime] For a distributed data storage system D(n, ), <compromised nodes, the secrecy capacity for a bandwidth-limited regime, for d = n, is C BL s (Γ) = Γ (n i) n, (6) and is achieved with a storage capacity of =Γ. A. Static Systems IV. SPECIAL CASES A static version of the problem studied here corresponds to a DSS with ideal storage nodes that do not fail. Hence there is no need for any repair the system. The flow graph of this system is then the combation networ studied networ codg theory (see for e.g. [6, Chap. 4] ). Therefore, the static storage problem can be regarded as a special case of wiretap networs [3], [4], or equivalently, as the erasureerasure wiretap-ii channel studied [7]. The secrecy capacity for such systems is ( ), and can be achieved usg either nested MDS codes [7], or the coset codes of [8], [4]. Even though the above proposed solution is optimal for the static case, it can have a very poor secrecy performance when applied directly to dynamic storage systems with failures. For stance, a straightforward way to repair a failed node would be to download the whole file on the new replacement node,

4 d x n+ (d ) x n+ x n+ (d +) x n+ x n+ (d +) Fig. 3. Part of the flow graph correspondg to a DSS D(n, ), when nodes v,...,v fail successively, and are replaced by nodes v n+,...,v n+.a data collector connects to these nodes and wants to retrieve the whole file. Nodes v n+,...,v n+ shown with broen boundaries are compromised by Eve durg repair. and then generate the specific lost data. In this case, if Eve accesses the new replacement node while it is downloadg the whole file, it will be able to reconstruct the entire origal data. Hence, the secrecy rate for this scheme would be zero. However, Theorem suggests that for some systems we can achieve a positive secrecy capacity. This example highlights the fact that dynamical repair of the DSS renders it trsically different from the static counterpart, and one should be careful designg the repair scheme order to safeguard the whole stored data. x n+ x n+ B. Systems Usg Random Networ Codg Usg the flow graph model, the authors of [] showed that random lear networ codes over a large fite field can achieve any pot (, γ), on the optimal storage-repair bandwidth tradeoff curve with a high probability. Consider an example of random lear networ code used a compromised DSS D(4, 3), which stores R = 6 symbols and operates at d =3, =, and =3. In this case, each of the itial nodes v,...,v 4 stores 3 dependently generated random lear combations of these R =6symbols. Assume now that node v 4 fails and is replaced by a new node v 5 that connects to v,v, and v 3, and downloads from each one of them =random lear combation of their stored data. Assume that after some time, node v 5 fails and is replaced by node v 6 a similar fashion. Now, if =, and Eve accesses nodes v 5 and v 6 while they were beg repaired, it will observe 6 lear combations of the origal data symbols, which, with high probability are learly dependent. Therefore, she will be able to reconstruct the whole file. The above analysis shows that, when random networ codg is used, it is not possible to achieve a positive secrecy rate for this system, even with pre-processg at the source, usg for example Maximum Ran Distance (MRD) codes [5]. But accordg to Theorem, which we prove section VI, the secrecy capacity of the the above DSS D(4, 3) is equal to one unit when =. This is also contrast with the case of multicast networs with compromised edges stead of nodes [3], where, random networ codg can perform as good as x n+ any determistic secure code [5]. V. UPPER BOUND ON SECRECY CAPACITY In this section we derive the upper bound of Theorem. Consider a DSS D(n, ) with <. Assume that the nodes v,v,...,v have failed consecutively, and were replaced durg the repair process by the nodes v n+,v n+,...,v n+ respectively as shown Fig. 3. Now suppose that Eve accesses nodes E = {v n+,v n+,...,v n+ } while they were beg repaired, and consider a data collector connected to the nodes B = {v n+,v n+,...,v n+ }. The reconstruction property implies H(S C B )=0by Eq. (), and the perfect secrecy condition implies H(S D E ) = H(S) by Eq. (). We can therefore write H(S) =H(S D E ) H(S C B ) () H(S C E ) H(S C B ) () = H(S C E ) H(S C E,C B\E ) = I(S, C B\E C E ) H(C B\E C E ) = H(C n+i C n+,...,c n+i ) (3) m{(d i + ),}. Inequality () follows from the fact that the stored data C E is a function of the downloaded data D E, () from, C B\E := {C n++,...,c n+ }, (3) follows from the fact that each node can store at most units, and for each replacement node we have H(C i ) H(D i ) d, also from the topology of the networ (see Fig. 3). Note that each node x n+i is connected to each of the nodes x n+,...,x n+i by an edge of capacity. The upper bound of Theorem follows then directly from the defition of Eq. (3). VI. SECRECY CAPACITY IN THE BANDWIDTH-LIMITED REGIME A. Example Consider aga the DSS D(4, 3) with =3,d=3, =, and = of Section IV-B, for which the secrecy rate usg random lear networ codg was shown to be 0. The upper bound on the secrecy capacity of this system given by Theorem is. We provide a scheme that achieves this upper bound. The proposed code is depicted Fig. 4 and consists of the concatenation of an MDS coset code [8] with a special repetition code that was troduced [9] by Rashmi et al. for constructg exact regeneration codes. Let S F q denote the formation symbol to be securely stored on the system. S is encoded usg the er MDS code to a codeword (Z, K,K,...,K 5 ), where K,...,K 5 are dependent random eys uniformly distributed over F q and Z = S + 5 i= K i. The encoded symbols Z, K,...,K 5 are then stored on the nodes v,...,v 4 as shown Fig. 4, followg the special repetition code of [9]. It is easy to verify that any data collector connectg to 3 nodes, observes

5 Random eys K,K,...,K 5 Information symbol S MDS Z, K,...,K 5 coset code Node v Node v Node v 3 Node v 4 Z Z K K K K 3 K 5 K K 3 K 4 Fig. 4. Schematic representation of the optimal code for the DSS D(4, 3) with =3, =,d=3, and =that achieves the secrecy capacity of unit. An MDS coset code taes the formation symbol S and five dependent random eys K,...,K 5, as an put and puts a parity chec symbol Z = S + 5 i= K i, along with random eys systematic form. These symbols are then stored on the DSS usg the code structure of [9]. all the symbols Z, K,...,K 5, and can therefore decode S = Z 5 i= K i. However, an eavesdropper accessg any two nodes will only observe 5 symbols of 6, and cannot ga any formation ab S. Next, we generalize this construction to obta a capacity-achievg code for the bandwidth-limited regime. B. Code Construction Our approach builds on the results of [9] where the authors constructed a family of exact regeneratg codes for d = n. The exact property of these codes allows any repair node to reconstruct and store an identical copy of the data lost upon a failure. For simplicity, we will expla the construction for =, i.e., Γ=n. For any larger values of Γ, and turn of, the file can be split to chuns, each of which can be separately encoded usg the construction correspondg to =. Choose =Γ. From [] we now that M = i= (n i) is the capacity of the above DSS the absence of any adversary ( =0). Let R := (n i) be the number of formation symbols that we would lie to store securely on the DSS, and θ := n(n ). Let S = (s,...,s R ) F R q denote the formation file and K =(K,...,K M R ) F M R q denote M R dependent random eys each uniformly distributed over F q. Then, the proposed code consists of an er nested (θ, M) MDS coset code [7] which taes S and K as an put, and puts X =(x,...,x θ ), such that X = KG K + SG S, GK where G = is a generator matrix of a (θ, M) MDS G S code, and G K itself is a generator matrix for a (θ, M R) MDS code. The formation vector S effectively selects the coset of the MDS code generated by G K. This er (θ, M) MDS code is then followed by the special repetition code troduced [9] which stores the codeword X on the DSS. The procedure of constructg this ner code can be described usg an auxiliary complete graph over n vertices u,...,u n that consists of θ edges. Suppose the edges are dexed by the coded symbols x,...,x θ. The code then consists of storg on node v i the dices of the edges adjacent to vertex u i the complete graph. Consequently, every coded symbol x i is stored on exactly two storage nodes, and any pair of two storage nodes have exactly one distct coded symbol common, e.g., code Fig. 4 for n =4. This ner code transforms the dynamic storage system to K 4 K 5 an equivalent static pot-to-pot channel. First notice that = Γ, hence all the data downloaded durg the repair process, i.e., d = Γ, is stored on the new replacement node with any further compression. Thus, accessg a node durg repair process, i.e., observg its downloaded data, is equivalent to accessg it after the repair process, i.e., observg its stored data. Second, the exact regeneration codes restore a failed node with the exact lost data. So, even though there are failures and repairs, the data storage system loos exactly the same at any pot of time. Any data collector downloads M symbols of x,...,x θ by connectg to nodes. Moreover, any eavesdropper can observe µ = i= (n i) =M R symbols. Thus, the system becomes similar to the erasure-erasure wiretap channel-ii of parameters (θ, M, µ). Therefore, sce the er code is a nested MDS code, from [7] we now that it can achieve the secrecy capacity of M µ = M (M R) = R = (n i) of the correspondg erasure-erasure wiretap channel. This rate is achieved for every unit of. Thus, the total secrecy rate achieved for =Γ/(n ) is Γ (n i) VII. CONCLUSION n. In this paper we considered dynamic distributed data storage systems that are subject to eavesdroppg. Our ma objective was to determe the secrecy capacity of such systems, i.e., the maximum amount of data that these systems can store and deliver to data collectors, with revealg any formation to the eavesdropper. Modelg such systems as multicast networs with compromised nodes, we gave an upper bound on the secrecy capacity, and showed that it can be achieved the important bandwidth-limited regime where the nodes have sufficient storage capacity. Fdg the general expression of the secrecy capacity of distributed storage systems, and more generally of multicast networs with a fixed number of compromised nodes, remas an open problem that we hope to address future wor. REFERENCES [] S. Rhea, C. Wells, P. Eaton, D. Geels, B. Zhao, H. Weatherspoon, and J. Kubiatowicz, Matenance-free global data storage, IEEE Internet Computg, pp , 00. [] A. Dimais, P. Godfrey, Y. Wu, M. Waright, and K. Ramchandran, Networ codg for distributed storage systems, to appear IEEE Trans. Inform. Theory. [3] N. Cai and R. W. Yeung, Secure networ codg, Proc. IEEE Int. Symp. Inf. Theory (ISIT), 00. [4] S. El Rouayheb and E. Soljan, On wiretap networs II, Proc. IEEE Int. Symp. Inf. Theory (ISIT), (Nice, France), 007. [5] D. Silva and F. Kschischang, Security for wiretap networs via ranmetric codes, Proc. IEEE Int. Symp. Inf. Theory (ISIT), 008. [6] R. Yeung, S.-Y. Li, and N. Cai, Networ Codg Theory (Foundations and Trends Communications and Information Theory). Now Publishers Inc, 006. [7] S. Arunumar and S. W. Mclaughl, MDS codes on erasure-erasure wire-tap channel, arxiv: v, 009. [8] L. H. Ozarow and A. D. Wyner, Wire-tap channel-ii, AT&T Bell lab tech. journal vol. 63, no. 0, 984. [9] K. Rashmi, N. B. Shah, P. V. Kumar, and K. Ramchandran, Exact regeneratg codes for distributed storage, Allerton Conference on Control, Computg, and Communication, Urbana-Champaign, IL, 009. In the erasure-erasure wiretap channel-ii of parameters (θ, M, µ), the transmitter sends θ symbols. A legitimate receiver and an eavesdropper receive M and µ symbols respectively through dependent erasure channels [7].