UH-Sky informasjonsmøte - PDF Free Download

(XaaS, X I, B, ST, ) Cloud 2015-04-16 UH-Sky informasjonsmøte

XaaS X = infrastructure. At first.

Cloud

Cloud Sky-tjeneste

Cloud Platform

Cloud according to NIST

IaaS: Separation of Responsibilities

Separation of responsibilities Why?

Separation of responsibilities

Separation of responsibilities Economy: Job Specialization Programming: Separation of Concern

Service Stack Portal IaaS STaaS BaaS Self Service Portal VM Management OpenStack Client App Backup Client: TSM Provisioning API Bridge HTTPS S3/Swift HTTPS TSM TLS Virtual Machines OpenStack Object GW Backup Server: TSM Server Server IPnett RBD RBD librados NFS NFS Juniper EMC IBM Fast Storage Large Storage Storage Storage SuperMicro

Cloud

Cloud Sky-tjeneste

A secret: There is no

Our security focus - Secure by Design Physical Security Site Security Tier 3 Class DC (in country) High availability Logical Security Hardened OS Hardened application Continuous in-production patching Service security Transport encryption Data at rest encryption Client encryption

Cloud according to NIST

(XaaS, X I, B, ST, ) Cloud 2015-04-16 UH-Sky informasjonsmøte

BaaS

Methods of doing backup Traditional method used by most other products Full backup every week Incremental backup every day Incremental forever the TSM method Full backup only first time All other backups are incremental 23

How much is stored in backup server? Example client: 1TB data, 1% Daily change, Save backup 90 days Traditional Incr forever Full backup 13 * 1TB 1 TB Incr backup 77 * 0.01 = 0.77 TB 89 * 0.01 = 0.89 TB Total TB 13.77TB 1.89 TB TSM generates as little as 1 7.3X of the data volume 24

BaaS is disk based As an extra bonus BaaS do not use tape. By using disk as storage media we can deliver restore performance without being limited of the number of tape drives. 25

Can we reduce the data even more? The BaaS service utilize two techniques to reduce storage need: Compression is enabled by default. Our experience is that typically the data is reduced by 50% Deduplication Deduplication only sends and stores the part of the file that are not already there. Typically the data is reduced by 30-60% 26

Deduplication details The file to be backuped is chunked The chunks are similar to blocks, but they have variable size. A checksum is calculated on each chunk. The client asks the server if a chunk with this checksum already exists, if so it says I have that too. Duplicate chunks are not stored, only pointers 27

Benefits of deduplication a word file that gets updated every day. Only changed chunks are transmitted/stored same file exists in several places.. Latest_hit.mp3 is in 100 copies, but only 1 copy sent/stored same file exists on many machines C:\Windows\... /usr/. 28

The example client Example client 1TB data, 1%, 90 days Traditional backup TSM backup TSM deduplication -50% TSM compression -50% 13.77 TB 1.89 TB 0.945 TB 0.4725 TB 1TB backuped over 90 days uses 0.48 TB backup storage 29

BaaS Demo: Auto-installation

BaaS main features Priced per fixed component + raw storage fee Based on IBM TSM and Cristie TBMR Incremental forever same object never transferred to server twice Compression compresses compressible data Deduplication Chunks up data and removes duplicates Optional: Client-side encryption All savings from these features transferred to user pay per used storage Savings are vast

BaaS additional features Self-service portal Packaged installations for Windows, Linux With silent deployment mode for easy roll-out on machines

BaaS Security & Performance Passwordless operations (managed through Portal/API) No shared common password on clients TLS always on Client encryption supported Multiple Gbps backup & restore to single client today: We want more - working with TSM engineers on improving large bandwidth & long distance throughput further Satisfaction occurs when the 10Gb WAN is the bottleneck for a singleserver restore (provided server isn t) SELinux profiles for TSM Server and soon client IBM hasn t done this yet in discussions with them over this as well

Backup as a Service - Examples Data volume on client (GB) File server Web server DB server DB server(*) 10000 50 50 50 Change rate 1% 5% 100% 100% Client side encryption No Yes No No Number of copies/day 1 1 1 1 Deduplication ratio 75% 0% 75% 75% Number of days 30 30 30 10 versions Estimated GB in service 3250 125 388 125 Price/month 319 12 39 13

Backup as a Service - Offering Price Fixed price 1005 / Customer / Month Storage cost 0.098 / GB / Month

STaaS

Storage platforms, typical Typically, SAN:s and NAS:es Common typical bottleneck in IO-path: Head-ends typically active/passive or active/active Typically IO-path limiting drives can deliver more For small IOPS, head-ends are CPU limited Not inexpensive One vendor recommended to scale cloud at 25 IOPS/VM How to scale for thousands of VMs?

Storage platforms, us We deploy a full-scale cluster Hypervisors have direct access to the storage cluster Storage cluster consists of many drives, on many nodes, using many CPUs Harddrives talk to each other, many-to-many, without central point Hypervisors talk to entire cluster, in parallell Many-to-many IO using many CPU:s and many IO-paths simultaneously is possible Throughput: Limited primarily by network IOPS: Limited primarily by storage client CPU Both of these are natural limits Hypervisors use non-oversubscribed 40+40Gbps

Ceph Software defined storage open source called Ceph Strongly consistent object storage cluster technology Computes where data chunks are placed using multiple abstractions Fast computation No central service to query like GlusterFS etc. Powered by Math

Ceph RedHat owner of Inktank first company built around the technology Ceph started 2008 from 2006 PhD of Sage Weil Have known Sage since 2008 Most popular storage backend for OpenStack deployments today

Ceph - internals Two types of daemons: OSD Object Storage Daemon Typically one per physical HDD Monitor Paxos algorithm 5 maps: Odd number of daemons; 3 or 5 for production environment Responsible for keeping a set of maps and distribute to clients Also responsible for authenticating storage daemons as well as clients Monitor Object Storage Daemon (OSD) Placement Group (PG) CRUSH Map (placement calculator data) MDS Map (not used today)

Ceph internals - Replication

Ceph internals Erasure Coding

Ceph internals calculations

Ceph internals PG abstractions

Ceph internals - rebalancing

Ceph

Ceph and OpenStack

Storage as a Service Key points Remote and local access using Object Storage APIs Local access in cloud using block devices in VMs Elastic HDDs, resize at will Scale-out performance Tunable optimizations: Performance Capacity Something-in-between (e.g. regular OS drives) Free to use latest off-the-shelf storage components No meaningful limitations in the service Supports very large block devices and cluster sizes

Storage as a Service - Offering Storage tier Storage type Optimized towards Price Fast 100% flash storage Best Performance/price 0.55 /GB/mo Mix HDD-storage on replicated pools with flash-based cache tier VMs OS drives (Performance/Capacity tradeoff) Large Erasure Coding on HDDs Best Capacity/Price (Medium overwrite) 0.17 /GB/mo 0.04 /GB/mo Active Archive Erasure Coding on Active Archive HDDs Best Capacity/Price (Low overwrite) 0.023 /GB/mo

IaaS

Who makes OpenStack?

Icehouse OpenStack Summit

OpenStack Summit

MULTI-TENANCY Physical Topology L3 Network L2 Network L2 Network L3 router L2 Network Logical Topology

Cloud & Legacy applications Typical enterprise IT work flows: Design infrastructure to fit a certain applications Modern cloud application deployment methods: Anti-thesis to the old work flow Fixed building blocks Fit the application to the infrastructure! New applications increasingly delivered in the modern way Tender defined the service in detail We proposed to address a mixture No local storage hypervisors patched continuously

IaaS (VMs) main features Storage virtualization on-demand capacity Network virtualization ease of use and security Fast VM instantiation seconds, not weeks, to running machine Self-service portal developers and application owners can mange VMs directly, no need to email infrastructure team Elastic machines add/remove VMs to suit demand, automatically Hypervisor security hardening SELinux = Enforcing

IaaS (VMs) added features in tender - Import VMs Import VMs from VMware, Xen and KVM Majority of standard VMs and OSes works without hand involvement More complicated appliances or similar with non-standard setups more tricky Some OS:es or applications do licensing by hardware; change => reactivation Inevitable

IaaS (VMs) added features in tender Local network connection to the network Bandwidth free of charge Data retention for purpose of graceful service exit Options for VM reboot-on-failure Monitoring and notification features

Infrastructure as a Service - Offering VM Model vcpu RAM Price b.small 1 2 GiB 0.0476 /h 20.55 /mo b.medium 2 4 GiB 0.0951 /h 41.10 /mo b.large 4 8 GiB 0.1902 /h 82.20 /mo h.medium 5 64 GiB 1,3720 /h 565.00 /mo

Contact SWEDEN IPnett AB, Dalvägen 8, 169 56 SOLNA Phone: +46 8 55 50 68 00 Fax: +46 8 55 50 68 01 info@ipnett.se NORWAY IPnett AS, Vollsveien 2 B, 1366 LYSAKER Phone: +47 67 20 10 10 Fax: +47 67 20 10 11 info@ipnett.no Oslo Stockholm DENMARK IPnett A/S, Gammel Køge Landevej 55, 4th floor, 2500 VALBY Phone: +45 48 10 75 00 Fax: +45 48 10 75 01 info@ipnett.dk Copenhagen Lund SWEDEN IPnett AB, Scheelevägen 27 223 70 Lund Phone: +46 8 55 50 68 00 Fax: +46 8 55 50 68 01 info@ipnett.se