Red Hat Storage Server for AWS

Size: px

Start display at page:

Download "Red Hat Storage Server for AWS"

Hester McDaniel
5 years ago
Views:

1 Red Hat Storage Server for AWS Craig Carl Solution Architect, Amazon Web Services Tushar Katarki Principal Product Manager, Red Hat Veda Shankar Principal Technical Marketing Manager, Red Hat

2 GlusterFS Community Project Community driven Open Distributed file system on top of local file system No metadata server with elastic hashing algorithm Spreads files across local file system directories called bricks Stores metadata in extended attributes of local file system Modular stackable architecture based on user-space FUSE

3 What is Red Hat Storage Open, Software Defined Storage Solution for On-Premise, Virtualized and Cloud environments Enterprise implementation of GlusterFS technology RHEL + XFS + GlusterFS Support Subscription 24x7 Premium Support

4 RED HAT STORAGE CONCEPTS VOLUME BRICK SERVER/NODES A namespace presented as a POSIX mount point and is comprised of bricks. The basic unit of storage, represented by an export directory on a server Contain the bricks

on filename Unique identifiers, similar to md5sum The Elastic Part

5 Elastic Hash Algorithm No central metadata No Performance Bottleneck Eliminates risk scenarios Location hashed intelligently on filename Unique identifiers, similar to md5sum The Elastic Part Files assigned to virtual volumes Virtual volumes assigned to multiple bricks

7 Why run RHS on AWS? You need shared access to a file system GlusterFS, NFS and CIFS You need a really large file system I ve created multiple 2PB clusters

8 Things to think about - Run at least replica 2 in two availability zone Single AZ = no durability commitments Dual AZ = 99.95% available Durability Backing store Performance Network interface EBS interface EBS performance

9 Backing stores EBS between 0.1% 0.5% AFR per volume Ephemeral hs1.8xlarge 48 terabytes of storage across 24 hard disk drives i2.8xlarge ~5.7 terabytes of storage across 8 SSDs

10 Running RHS on EC2 Use Reserved Instances! RHS is a perfect use case for reserved instances. Discounts of up to 65% Purely a billing construct, no technical implications. m3.xlarge is perfect for $.25/hr (1yr heavy RI)

11 VOLUME CONFIGURATION Global namespace Bricks are XFS filesystems Volumes are made of bricks Distributed Replicated Distributed and replicated Geo-replication

12 UNIFIED FILE AND OBJECT Object storage Objects Containers Accounts Standards based OpenStack Swi5 interface GlusterFS underneath RESTful APIs NAS protocol support NFS / CIFS / GlusterFS No translacon required

13 DATA FLOW: Gluster native client Clients talk directly to necessary nodes

14 Red Hat Storage User Perspective (distributed volumes) DISTRIBUTED VOLUME DISTRIBUTED VOLUME MOUNT POINT server1:/exp1 server1:/exp1 BRICK BRICK server2:/exp1 server2:/exp1 BRICK BRICK FILE 1 FILE 2 FILE 3

15 Red Hat Storage user perspective (distributed replicated volumes) DISTRIBUTED VOLUME MOUNT POINT server1 Replicated Volume 0 server2 server3 Replicated Volume 1 server4 BRICK (exp1) BRICK (exp2) BRICK (exp3) BRICK (exp4) FILE 1 FILE 2

16 Red Hat Storage Server for Public cloud Scale out performance, capacity, and availability Scale up capacity RED HAT STORAGE FOR PUBLIC CLOUD EC2 EBS GlusterFS Amazon Machine Images (AMIs) The only way to achieve high availability of Elastic Block Storage (EBS) Multiple EBS devices pooled POSIX compatible (no application to rewrite required to run on Amazon EC2) Scale out capacity and performance as needed SINGLE GLOBAL NAMESPACE

17 Red Hat Storage Use Cases in AWS EC2

18 Data Availability on Amazon Web Services Amazon East Zone A DNS Gateway Amazon East Zone B Amazon Region West Load-balancers Load-balancers Load-balancers Web and App Tier Web and App Tier Web and App Tier Red Hat Storage Synchronous Replication Red Hat Storage Red Hat Storage EBS EBS EBS Master Slave Replication

19 CONTENT SHARING WITH RHS Global Access Multiple Access Methods High Availability Disaster Recovery Scale with Growth Open Source EC2 West Region Distributed, Replicated RHS Vols EC2 East Region Content is available In RHS Volumes Geo Rep Object Access Posix Content Processing Web Servers MetaData Store Content Delivery Network Content Delivery Network Load Balancers Content Downloads Content Uploads

20 AD SERVING WITH RHS Map Reduce With RHS RHS Hadoop Plugin Compute Storage Co-residency High Availability Scale with Growth Open Source Distributed, Replicated RHS Vols Map Reduce Processing With RHS Hadoop Plugin Impression Log Info Ad Profile DB Click-Thru Logs App Servers Load Balancers Ad Serving Click Thru Requests

21 Demo

22 Demo Architecture

23 Check Out Other Red Hat Storage Activities at The Summit Enter the raffle to win tickets for a $500 gift card or trip to LegoLand! Entry cards available in all storage sessions - the more you attend, the more chances you have to win! Talk to Storage Experts: Red Hat Booth (# 211) Infrastructure Infrastructure-as-a-Service Storage Partner Solutions Booth (# 605) Upstream Gluster projects Developer Lounge Follow us on Twitter,

GlusterFS and RHS for SysAdmins

GlusterFS and RHS for SysAdmins An In-Depth Look with Demos Sr. Software Maintenance Engineer Red Hat Global Support Services FISL 7 May 2014 Introduction Name: Company: Red Hat Department: Global Support