Open-Xchange. Summit 2017 Brussels. Erik Julin Sr. Director, Technical Business Development

Transcription

1 Open-Xchange Summit 2017 Brussels Erik Julin Sr. Director, Technical Business Development

2 Introducing Scality (in 2 slides)

3 Scality Architecture OBJECT APPLICATIONS SCALITY SOFTWARE Object & File Access Local and Geo-Protection, Self-healing Advanced Routing, Object Storage FILE APPLICATIONS STANDARD LINUX / STANDARD x86 SERVERS Managemen t Scale-out access and capacity (millions of clients, billions of objects) Any-to-Any any client can access any object in parallel with no added latency Shared-nothing no master metadata node or SPOF for high availability Self-healing automated, fast repair and rebuild 100% Software complete hardware choice, flexible deployment, 100% available during SW/HW upgrades, no data migrations Scality Confidential 2

4 Scality Key Components Scality Software Infrastructure CONNECTOR NODES File, Object, Access STORAGE NODES Highly durable, scale-out object storage STORAGE SERVERS and Storage Nodes both run on the storage servers. Based on use case, s can also be run separately on physical or virtual servers SUPERVISOR Management GUI and CLI MANAGEMENT SERVER Can run on physical or virtual servers Scality Confidential 3

5 Scality for Optimized for In production since 2010 Dozens of deployments Hundreds of millions of accounts Scores of petabytes stored Trillions of objects 4

6 Scality / Dovecot Integration 5

7 Dovecot Pro / Scality: High Level Multi-site Architecture

8 Dovecot Pro / Scality: Connection to Site A and Sit B active concurrently. Users have affinity to a site. Dovecot Pro Backend uses sproxyd (HTTP/REST) API to communicate with. Backend servers with software LB like HAProxy or external LB. LB has pool of sproxyd connectors for local data center. Simple stateless load balancing. sproxyd connectors, co-located on storage servers, distribute objects to storenodes keyspace guarantees two copies per data center for Class of Service(3)

9 Sizing Methodology Enhancements 8

10 Sizing for Key Parameters for Sizing peak message delivery per second (post AA/AV) peak message retrieval per second peak message delete per second average message size message storage capacity Add data center topology SLAs few more relevant details... 9

11 Sizing Inputs Parameter Parameter Conversion (apply Magic Sauce) PUT/sec msg delivery/sec msg delivery + indices generated GET/sec msg retrieval/sec msg read + index object reads + HEAD operations DEL/sec msg del/sec msg del + index obj del average object size total usable capacity average msg size stored capacity weighted average of msg size post compression, all indices stored capacity + capacity of all indices + capacity of objects pending purge 10

12 Sizing Methodology worked well, but... tended to modestly overestimate loading which is much better than underestimating 11

13 New Methodology Now Dovecot supplies directly transaction rates peak object PUTs per second peak object GETs per second peak object DELs per second Removes assumptions mapping messages to objects Scality still validates metrics, applies our experience, especially for average object size and total capacity RESULT less time to BOM, reduction in cost, greater confidence 12

14 Introducing Stretched Failover Optimization for Dovecot (a.k.a. CoST) 13

15 Dynamic Failover Optimization a.ka. Class of Service Translator (CoST) Introduced in 6 (2016) as an optionally licensed feature to Reduce raw storage requirements Reduce IO requirements Improve data durability Invoked only during partition events, e.g. Loss of data center Loss of WAN connectivity Now available for Dovecot Pro using sproxyd API

16 Dynamic Failover Optimization Applicability Dynamic Failover Optimization feature applies only to Immutable objects all mail ISVs integrated with treat objects as immutable Protected via replication vast majority of deployments use replication typically lowest TCO lowest latency for multi-geo Deployed in multi-geo configuration

17 Background Self-healing operations Rebuild Recreates missing copies/chunks of objects. Repair optimized version of repair, where there is a list of missing objects, such as during a disk failure. If process dies or server reboots then a rebuild is necessary Rebuild is an all or none operation Rebuild every missing copy. By default rebuild is automatically disabled during site failure. If full rebuild needed, then additional raw capacity may be needed Repair operation Can occur during site failure

18 : CoS(3) PUTS on Fully Operational 17

19 : CoS(3) PUTS after Failed DC Data Center A Data Center B Replica 2 Replica 3 Replica 0 CoS(3): 4 Replicas to surviving DC Replica 1 18

20 : CoS(F) PUTS after Failed DC Data Center A Data Center B Replica 1 Replica 0 CoS(F): 2 Replicas per DC 19

21 Benefits of CoST Significant reduction in IO load during site failure Rebuilds during site failure without doubling raw capacity Much higher data durability during site failure Not nearly as good as normal operations Can save significant hardware CapEx (30-50%) and associated OpEx capacity license 20

22 Scality Deployed Architectures for 21 Scality

23 Single Site 22

24 Two-site Stretched 23

25 Three-site Stretched 24

26 Two-sites with Async Replication Data Center A Data Center B 25

27 Single Site pre-planned Two-site Growth 26

28 Single Site pre-planned multi-site 27

29 Thank You Erik Julin scality.com 555 California Street, Ste 3050, San Francisco, CA 94104, USA Telephone: +1 (650) Fax: +1 (650) Toll Free: +1 (855)