FOSDEM 14 UNIFIED CLOUD STORAGE WITH VANGELIS KOUKIS, TECHNICAL LEAD, SYNNEFO

Transcription

1 UNIFIED CLOUD STORAGE WITH SYNNEFO + GANETI + ARCHIPELAGO + CEPH VANGELIS KOUKIS, TECHNICAL LEAD, SYNNEFO 1

2 p Runnin a public cloud: ~okeanos vkoukis@rnet.r History - Desin started late Production since July 2011 Numbers - Users: > VMs: > 7000 currently active - More than 250k VMs spawned so far, more than 70k networks 2

3 p Runnin a public cloud: ~okeanos vkoukis@rnet.r Our choices - Build own AWS-like service (Compute, Network, Storae) - Persistent VMs -Everythin open source - Production-quality IaaS - Super-simple UI How? 3

4 p Runnin a public cloud: ~okeanos vkoukis@rnet.r The touh stuff - Stability - Persistent VMs: VMs are not cattle, they are pets - Commodity hardware - Scalability - Manaeability: Gradual rollout of uprades and new features 4

5 p Runnin a public cloud: ~okeanos vkoukis@rnet.r Our approach - Synnefo - Goole Ganeti -DRBD -Archipelao - Ceph -OpenStackAPIs 5

6 Architecture 6

7 End-to-end workflow with unified storae 7

8 Live demo! Loin, view/upload files Unified imae store: Imaes as files View/create/destroy servers from Imaes on multiple storae backends on Archipelao, for thin, super-fast creation with per-server customization, e.., file injection View/create/destroy virtual networks Interconnect VMs, with NIC hotpluin Take a point-in-time snapshot of a VM s disk, in seconds Share it with collaborators, with fine-rained Access Control Create a virtual cluster from this Imae from the command-line, and in Python scripts 8

9 Goole Ganeti Mature, production-ready VM cluster manaement - used for Goole s corporate infrastructure Multiple storae backends out of the box -LVM, DRBD - Files on local or shared directory -RBD(Ceph/RADOS) External Storae Interface for SAN/NAS support Ganeti cluster = masterd on master, noded on nodes Easy to interate into existin infrastructure - Remote API over HTTP, pre/post hooks for every action! 9

10 Architecture 10

11 y Identity: Astakos Identity Manaement, Resource Accountin and SSO - Platform-wide service - Simple service- (Cyclades, Pithos) and user-facin APIs - Multiple authentication methods per user - Fine-rained per-user, per-resource quota A sinle dashboard for users - View/modify profile information and active authentication methods - Easy, interated t reportin of per-resource quotas - Project manaement: View/Join/Leave projects - Manae API access and retrieve authentication ti ti tokens 11

12 y Identity: Astakos Supported 3 rd -party providers - Shibboleth / AAI Federation - Goole - Twitter - LinkedIn 12

13 p y Compute/Network/Imae/Volume: Cyclades vkoukis@rnet.r Thin Compute layer over Ganeti - Python/Djano - Supports multiple Ganeti clusters, for scalin -OpenStackAPIs Networkin - No restrictions on deployment it s the Ganeti side - IPv4/IPv6 public networks, complete isolation amon VMs - Thousands of private networks, private L2 sements over sinle VLAN - Software-Defined Networkin, pilots with VXLAN interation 13

14 p y Compute/Network/Imae/Volume: Cyclades vkoukis@rnet.r 14

15 Interaction with Ganeti Support for all Ganeti storae templates External Storae Interface for SAN/NAS support Networkin = nt-network + snf-network (KVM ifup scripts) + nfdhcpd (custom NFQUEUE-based DHCP server) Asynchronous operation - Effect path: Receive API requests, enqueue requests over RAPI - Update path: Receive asynchronous notifications, update DB 15

16 Storae service: Pithos Exposes the OpenStack Object Storae (Swift) API - plus extensions, for sharin and syncin Rich sharin, with fine-rained Access Control Lists Content-based addressin for blocks Partial file transfers, deduplication, efficient syncin Backed by Archipelao - Provides a northbound endpoint for Archipelao -Implements the HTTP ateway - Exposes the Swift API to end users 16

17 p Archipelao overview Distributed Storae System - Powerin storae in clouds Decouples storae resources from storae backends - Files / Imaes / Volumes / Snapshots Unified way to provision, handle, and present resources Decouples loic from actual physical storae - Software-Defined Storae 17

18 p Archipelao loic Thin provisionin, with clones and snapshots - Independent from the underlyin storae technoloy Hash-based data deduplication Pluable architecture - Multiple endpoint (northbound) drivers - Multiple l backend (southbound) drivers Multiple storae backends - Unified manaement - with storae mirations 18

19 Unified view of resources Files - User files, with Dropbox-like syncin Imaes - Templates for VM creation Volumes - Live disks, as seen from VMs Snapshots - Point-in-time i ti snapshots of Volumes 19

20 Spawn my own Ubuntu Freeze 20

21 Clone Ubuntu + user data Snapshot 21

22 p The bi picture Archipelao Core Storae backend 1 Storae backend 2 Storae backend 3 (e.., Ceph cluster 1) (e.., Ceph cluster 2) (e.., NFS over NAS) 22

23 23

24 24

25 block I/O Linux block driver Northbound interface RADOS Volume Composer Monitor nodes Mapper Archipelao Core Storae Ceph/RADOS driver Southbound interface object I/O Object Storae nodes 25

26 Resource composition 26

27 p Archipelao interfaces GLUSTER DRIVER GLUSTER 27

28 p Runnin Archipelao 28

29 pp Flexible I/O pipeline Linux block drv Linux block drv Cacher Volume Composer Blocker Cacher Mapper Storae backend 29

30 p Experience Operations - Rollin hardware and software uprades - kernel, Ganeti, RADOS, Synnefo - with no VM downtime - Node evacuations with live VM mirations - Cross-datacenter move, Intel AMD, no VM downtime - On-the-fly miration from NFS-backed storae to RADOS - IP renumberin of all VMs 30

31 p Experience Scalability - From few physical hosts to multiple racks - dynamic addition of Ganeti clusters Diverse workloads - Different network and storae backends - Choice exposed to the user 31

32 y Try it out! 32

33 y Thank you! 33