Coding Techniques for Distributed Storage Systems

Transcription

1 Coding Techniques for Distributed Storage Systems Jung-Hyun Kim, Hong-Yeop Song Yonsei Univ. Seoul, KORE 3 rd CITW, Oct , 2013

2 Contents Introduction Codes for Distributed Storage Systems (DSS) Traditional Codes (Repetition code and MDS code) Regenerating Codes (MSR code and MR code) Codes with Local Regeneration (LRC) Performance Comparison dvanced Techniques 2

3 What s the Problem? Too big to manage! ig data storage Warehouse-scale data center Thousands of servers, Petabytes of disc space Google Data Center 3

4 What s the Problem? It needs repairing! t Facebook, it is quite typical to have 20 or more node failures per day. Number of failed nodes over a single month period in 3000 nodes M. Sathiamoorthy, M. steris, D. Papailiopoulos,. G. Dimakis, R. Vadali, S. Chen, D. orthakur, XORing Elephants: Novel Erasure Codes for ig Data, in Proc. of the 39th International Conf. on Very Large Data ases, ug

5 Traditional Codes for DSS How to regenerate failed nodes? Repetition codes have higher efficiency than MDS codes Node failure repetition code Node failure MDS code + +2 Source Distributed nodes 5

6 Traditional Codes for DSS How to regenerate failed nodes? MDS codes have higher reliability than repetition codes Node failure? Repair failure repetition code? Node failure MDS code Source Distributed nodes 6

7 Traditional Codes for DSS Which one is better? repetition code MDS code Tolerates all 1-failures and some 2-failures + Tolerates all 2-failures +2 Efficient Repair Low Reliability Inefficient Repair High Reliability Q. Tradeoff between efficiency and reliability? 7

8 Regenerating Codes for DSS Regenerating Codes?. G. Dimakis, P.. Godfrey, Y. Wu, M. Wainwright and K. Ramchandran, Network Coding for Distributed Storage Systems, IEEE Proc. INFOCOM, (nchorage, laska), May

9 Regenerating Codes for DSS Regenerating Codes Framework Storing n coded files of k original files at n distributed nodes C 1 F 1 C 2 F 2 S k Coding n M F k M/k C n α M/k 9

10 Regenerating Codes for DSS Regenerating Codes Framework In aspect of data collection C 1 C 2 α n α Data collector M n : # of storage nodes α k : # of storage nodes for data collection C n k α:storage size : data size. G. Dimakis, P.. Godfrey, Y. Wu, M. Wainwright and K. Ramchandran, Network Coding for Distributed Storage Systems, IEEE Transactions on Information Theory, Vol. 56, Issue 9, Sept

11 Regenerating Codes for DSS Regenerating Codes Framework In aspect of node repair C 1 β C 1 C 2 β n C n d β d : # of storage nodes for node repair β:download size dβ (= ) : repair bandwidth. G. Dimakis, P.. Godfrey, Y. Wu, M. Wainwright and K. Ramchandran, Network Coding for Distributed Storage Systems, IEEE Transactions on Information Theory, Vol. 56, Issue 9, Sept

12 Regenerating Codes for DSS Reducing storage size and repair bandwidth ased on the min-cut bound : Repair bandwidth d Storage-repair bandwidth trade-off curve Minimum Storage Regenerating (MSR) point min {,} Minimum andwidth Regenerating (MR) point Storage size per node < Tradeoff between storage size and repair bandwidth (M=7000, n=15, k=7, d=7) > N.. Shah, K. V. Rashmi, P. V. Kumar, and K. Ramchandran, Distributed Storage Codes With Repair-by-Transfer and Nonachievability of Interior Points on the Storage-andwidth Tradeoff, IEEE Trans. Inf. Theory, vol. 58, no. 3, pp , Mar MSR codes = = ( + 1) MR codes = = 2 (2 +1) 2 (2 + 1) 12

13 Regenerating Codes for DSS Minimum Storage Regenerating (MSR) codes Interference lignment method, Product-Matrix method, etc. C D + Interference lignment +C+D = = 4 2 =2 +C 2+D 2+C +D Linearly Indep. +C+D +C+D = ( + 1) 4 3 = 2( ) =3 N.. Shah, K. V. Rashmi, P. V. Kumar, and K. Ramchandran, Interference lignment in Regenerating Codes for Distributed Storage: Necessity and Code Constructions, IEEE Trans. Inf. Theory, vol. 58, no. 4, pp , pril K. V. Rashmi, N.. Shah, and P. V. Kumar, Optimal exact-regenerating codes for the MSR and MR points via a product-matrix construction, IEEE Trans. Inf. Theory, vol. 57, no. 8, pp , ug

14 Regenerating Codes for DSS Minimum andwidth Regenerating (MR) codes Repair-by-Transfer method, Product-Matrix method, etc. 1, 5 6, 7 1, 2 3, , 7 9, 10 2 = (2 + 1) = 3( ) = , 5 8, 9 8 3, 6 8, 10 2 = (2 +1) = 3( ) =4 K. W. Shum, and Y. Hu, Functional-Repair-by-Transfer Regenerating Codes, in Proc. of 2012 IEEE International Symposium on Information Theory, Cambridge, M, July K. V. Rashmi, N.. Shah, and P. V. Kumar, Optimal exact-regenerating codes for the MSR and MR points via a product-matrix construction, IEEE Trans. Inf. Theory, vol. 57, no. 8, pp , ug

15 Codes with Local Regeneration Locality? P. Gopalan, C. Huang, H. Simitci, and S. Yekhanin, On the Locality of Codeword Symbols, IEEE Trans. Inf. Theory, vol. 58, no. 11, pp , Nov

16 Codes with Local Regeneration Local Reconstruction Codes (LRC) High access efficiency for node repair m 1 m 2 m 3 m 4 P 1 (r, δ) locality : given that at the most (δ 1) symbols are erased can be deduced by reading at most r other surviving symbols. m 5 m 6 m 7 m 8 P 2 Local parity +1 1 ( 1) P 3 P 4 Global parity C. Huang, H. Simitci, Y. Xu,. Ogus,. Calder, P. Gopalan, J. Li, and S. Yekhanin, Erasure Coding in Windows zure Storage, 2012 USENIX nnual Technical Conference, S. Rawat, N. Silberstein, O. O. Koyluoglu, and S. Vishwanath, Optimal locally repairable codes with local minimum storage regeneration via rank-metric codes, in Information Theory and pplications Workshop (IT), San Diego, C, Feb

17 Performance Comparison Repair failure probability for different node failure probability Node failure prob. : the probability that a node is unavailable E T T E R Repair failure probability Repair failure prob. : the probability that any newcomer nodes can not repair the original data symbol from coded data symbols of surviving storage nodes REP(7,8) LRC(7,3,5) MDS(7,8) Node failure probability () Jung-Hyun Kim, Jin Soo Park, Ki-Hyeon Park, Inseon Kim, Mi-Young Nam, Hong-Yeop Song, Reliability Comparison of Various Regenerating Codes for Cloud Services, International Conference on ICT Convergence 2013 (ICTC2013), Jeju Island, Korea, October 13-16,

18 dvanced Techniques Reducing repair bandwidth using cooperation Individual regeneration Cooperative regeneration C D, C, D C D C D +C C D +C 2+D +C 2+D 2+C +D, C, D 2+C +D 2+C +D 2+D 2+C +D K. W. Shum, Cooperative Regenerating Codes for Distributed Storage Systems, in Proc. of 2011 International Conference on Communications, Kyoto, June K. W. Shum and Y. Hu, Cooperative Regenerating Codes, IEEE Trans. Inf. Theory, vol. 59, no. 11, pp , Nov

19 dvanced Techniques Handling erasures and errors during the data construction and node repair operations dditional erasure dditional erasure and error C X0 + C X C X C X C +2 +2C X C X0 + [3,2] MDS code in {a, b} Can correct one arbitrary erasure +3 +3C X0 + [4,2] MDS code in {a, b} Can correct one arbitrary error K. V. Rashmi, N. Shah, K. Ramchandran, and P. Kumar, Regenerating codes for errors and erasures in distributed storage, in Information Theory Proceedings (ISIT), 2012 IEEE International Symposium on, 2012, pp

20 20