UH-Sky informasjonsmøte

Transcription

1 (XaaS, X I, B, ST, ) Cloud UH-Sky informasjonsmøte

2 XaaS X = infrastructure. At first.

3 Cloud

4 Cloud Sky-tjeneste

5

6 Cloud Platform

7 Cloud according to NIST

8 IaaS: Separation of Responsibilities

9 Separation of responsibilities Why?

10 Separation of responsibilities

11 Separation of responsibilities Economy: Job Specialization Programming: Separation of Concern

12 Service Stack Portal IaaS STaaS BaaS Self Service Portal VM Management OpenStack Client App Backup Client: TSM Provisioning API Bridge HTTPS S3/Swift HTTPS TSM TLS Virtual Machines OpenStack Object GW Backup Server: TSM Server Server IPnett RBD RBD librados NFS NFS Juniper EMC IBM Fast Storage Large Storage Storage Storage SuperMicro

13 Cloud

14 Cloud Sky-tjeneste

15 A secret: There is no

16

17

18 Our security focus - Secure by Design Physical Security Site Security Tier 3 Class DC (in country) High availability Logical Security Hardened OS Hardened application Continuous in-production patching Service security Transport encryption Data at rest encryption Client encryption

19 Cloud according to NIST

20 (XaaS, X I, B, ST, ) Cloud UH-Sky informasjonsmøte

21 Service Stack Portal IaaS STaaS BaaS Self Service Portal VM Management OpenStack Client App Backup Client: TSM Provisioning API Bridge HTTPS S3/Swift HTTPS TSM TLS Virtual Machines OpenStack Object GW Backup Server: TSM Server Server IPnett RBD RBD librados NFS NFS Juniper EMC IBM Fast Storage Large Storage Storage Storage SuperMicro

22 BaaS

23 Methods of doing backup Traditional method used by most other products Full backup every week Incremental backup every day Incremental forever the TSM method Full backup only first time All other backups are incremental 23

24 How much is stored in backup server? Example client: 1TB data, 1% Daily change, Save backup 90 days Traditional Incr forever Full backup 13 * 1TB 1 TB Incr backup 77 * 0.01 = 0.77 TB 89 * 0.01 = 0.89 TB Total TB 13.77TB 1.89 TB TSM generates as little as 1 7.3X of the data volume 24

25 BaaS is disk based As an extra bonus BaaS do not use tape. By using disk as storage media we can deliver restore performance without being limited of the number of tape drives. 25

26 Can we reduce the data even more? The BaaS service utilize two techniques to reduce storage need: Compression is enabled by default. Our experience is that typically the data is reduced by 50% Deduplication Deduplication only sends and stores the part of the file that are not already there. Typically the data is reduced by 30-60% 26

27 Deduplication details The file to be backuped is chunked The chunks are similar to blocks, but they have variable size. A checksum is calculated on each chunk. The client asks the server if a chunk with this checksum already exists, if so it says I have that too. Duplicate chunks are not stored, only pointers 27

28 Benefits of deduplication a word file that gets updated every day. Only changed chunks are transmitted/stored same file exists in several places.. Latest_hit.mp3 is in 100 copies, but only 1 copy sent/stored same file exists on many machines C:\Windows\... /usr/. 28

29 The example client Example client 1TB data, 1%, 90 days Traditional backup TSM backup TSM deduplication -50% TSM compression -50% TB 1.89 TB TB TB 1TB backuped over 90 days uses 0.48 TB backup storage 29

30 BaaS Demo: Auto-installation

31 BaaS main features Priced per fixed component + raw storage fee Based on IBM TSM and Cristie TBMR Incremental forever same object never transferred to server twice Compression compresses compressible data Deduplication Chunks up data and removes duplicates Optional: Client-side encryption All savings from these features transferred to user pay per used storage Savings are vast

32 BaaS additional features Self-service portal Packaged installations for Windows, Linux With silent deployment mode for easy roll-out on machines

33 BaaS Security & Performance Passwordless operations (managed through Portal/API) No shared common password on clients TLS always on Client encryption supported Multiple Gbps backup & restore to single client today: We want more - working with TSM engineers on improving large bandwidth & long distance throughput further Satisfaction occurs when the 10Gb WAN is the bottleneck for a singleserver restore (provided server isn t) SELinux profiles for TSM Server and soon client IBM hasn t done this yet in discussions with them over this as well

34 Backup as a Service - Examples Data volume on client (GB) File server Web server DB server DB server(*) Change rate 1% 5% 100% 100% Client side encryption No Yes No No Number of copies/day Deduplication ratio 75% 0% 75% 75% Number of days versions Estimated GB in service Price/month

35 Backup as a Service - Offering Price Fixed price 1005 / Customer / Month Storage cost / GB / Month

36 STaaS

37 Storage platforms, typical Typically, SAN:s and NAS:es Common typical bottleneck in IO-path: Head-ends typically active/passive or active/active Typically IO-path limiting drives can deliver more For small IOPS, head-ends are CPU limited Not inexpensive One vendor recommended to scale cloud at 25 IOPS/VM How to scale for thousands of VMs?

38 Storage platforms, us We deploy a full-scale cluster Hypervisors have direct access to the storage cluster Storage cluster consists of many drives, on many nodes, using many CPUs Harddrives talk to each other, many-to-many, without central point Hypervisors talk to entire cluster, in parallell Many-to-many IO using many CPU:s and many IO-paths simultaneously is possible Throughput: Limited primarily by network IOPS: Limited primarily by storage client CPU Both of these are natural limits Hypervisors use non-oversubscribed 40+40Gbps

39 Ceph Software defined storage open source called Ceph Strongly consistent object storage cluster technology Computes where data chunks are placed using multiple abstractions Fast computation No central service to query like GlusterFS etc. Powered by Math

40 Ceph RedHat owner of Inktank first company built around the technology Ceph started 2008 from 2006 PhD of Sage Weil Have known Sage since 2008 Most popular storage backend for OpenStack deployments today

41 Ceph - internals Two types of daemons: OSD Object Storage Daemon Typically one per physical HDD Monitor Paxos algorithm 5 maps: Odd number of daemons; 3 or 5 for production environment Responsible for keeping a set of maps and distribute to clients Also responsible for authenticating storage daemons as well as clients Monitor Object Storage Daemon (OSD) Placement Group (PG) CRUSH Map (placement calculator data) MDS Map (not used today)

42 Ceph internals - Replication

43 Ceph internals Erasure Coding

44 Ceph internals calculations

45 Ceph internals PG abstractions

46 Ceph internals - rebalancing

47 Ceph

48 Ceph and OpenStack

49 Storage as a Service Key points Remote and local access using Object Storage APIs Local access in cloud using block devices in VMs Elastic HDDs, resize at will Scale-out performance Tunable optimizations: Performance Capacity Something-in-between (e.g. regular OS drives) Free to use latest off-the-shelf storage components No meaningful limitations in the service Supports very large block devices and cluster sizes

50 Storage as a Service - Offering Storage tier Storage type Optimized towards Price Fast 100% flash storage Best Performance/price 0.55 /GB/mo Mix HDD-storage on replicated pools with flash-based cache tier VMs OS drives (Performance/Capacity tradeoff) Large Erasure Coding on HDDs Best Capacity/Price (Medium overwrite) 0.17 /GB/mo 0.04 /GB/mo Active Archive Erasure Coding on Active Archive HDDs Best Capacity/Price (Low overwrite) /GB/mo

51 IaaS

52

53

54 Who makes OpenStack?

55 Icehouse OpenStack Summit

56 OpenStack Summit

57 MULTI-TENANCY Physical Topology L3 Network L2 Network L2 Network L3 router L2 Network Logical Topology

58

59 Cloud & Legacy applications Typical enterprise IT work flows: Design infrastructure to fit a certain applications Modern cloud application deployment methods: Anti-thesis to the old work flow Fixed building blocks Fit the application to the infrastructure! New applications increasingly delivered in the modern way Tender defined the service in detail We proposed to address a mixture No local storage hypervisors patched continuously

60 IaaS (VMs) main features Storage virtualization on-demand capacity Network virtualization ease of use and security Fast VM instantiation seconds, not weeks, to running machine Self-service portal developers and application owners can mange VMs directly, no need to infrastructure team Elastic machines add/remove VMs to suit demand, automatically Hypervisor security hardening SELinux = Enforcing

61 IaaS (VMs) added features in tender - Import VMs Import VMs from VMware, Xen and KVM Majority of standard VMs and OSes works without hand involvement More complicated appliances or similar with non-standard setups more tricky Some OS:es or applications do licensing by hardware; change => reactivation Inevitable

62 IaaS (VMs) added features in tender Local network connection to the network Bandwidth free of charge Data retention for purpose of graceful service exit Options for VM reboot-on-failure Monitoring and notification features

63 Infrastructure as a Service - Offering VM Model vcpu RAM Price b.small 1 2 GiB /h /mo b.medium 2 4 GiB /h /mo b.large 4 8 GiB /h /mo h.medium 5 64 GiB 1,3720 /h /mo

64 Contact SWEDEN IPnett AB, Dalvägen 8, SOLNA Phone: Fax: info@ipnett.se NORWAY IPnett AS, Vollsveien 2 B, 1366 LYSAKER Phone: Fax: info@ipnett.no Oslo Stockholm DENMARK IPnett A/S, Gammel Køge Landevej 55, 4th floor, 2500 VALBY Phone: Fax: info@ipnett.dk Copenhagen Lund SWEDEN IPnett AB, Scheelevägen Lund Phone: Fax: info@ipnett.se