Software Defined Data Infrastructure Management Cloud Edition Performance Availabity Capacity Economic Considerations MSP CMG - Industry Trends Perspectives Presented by Greg Schulz Server December 14, 2018 Nine Time Download PDF slides via www.storageio.com/cmg2018 Copyright 2018 Server StorageIO and UnlimitedIO LLC. All rights Rights reserved. Reserved www.storageio.com @StorageIO info@storageio.com
Industry Trends What s The Buzz, New, Trending, Trendy, Dejavu? Hybrid expansion from core (cloud and on-prem) to edge (fog, ROBO, not just IoT) Cross platform, cross cloud, hybrid management and insight tools o AWS, A10, Blue Medora, Datadog, Densify, Irods, Kubernetes, Loggly, Microsoft (ARM, SCM, WAC, MAP), Quest, SAS, Solarwinds, Splunk, VirtualMetric, ware many others Cloud storage is more than just object (or blob) o Block, file, queue, table, time series, data lakes, premium/fast, cold/archive, migration applainces New tiers of SSD compliment existing storage tiers (HDD & Tape still being enhanced) o Media SCM/PMEM (e.g. 3D XPoint/Everspin), various NAND/Flash tiers, NVRAM o Access interface Byte/DAX, block (SAS, SATA, Ne [various]), file & object o Packaging component (AIC, U.2/8639/Drive, M.2/NGFF, NVDIMM, EDSFF), systems Long growing list of vendors whose storage systems/software runs on cloud instances o Leveraging cloud s, bare metal, containers, block and on instance storage for primary o File, Object or Blob among other resources for protection, logs, snapshots, archive Download PDF slides via www.storageio.com/cmg2018
Industry Trends Data Infrastructures Focus of Data Infrastructures is to support and enable information services delivery Edge (Fog, ROBO) Source: StorageIOblog.com
Industry Trends All Application Workloads Have PACE Attributes PACE attributes vary by workload characteristics requiring different resources Application Workloads Metrics That Matter $$ Processors Performance Activity rates (IOP, TPS, Get, API calls) Data movement (Bandwidth, Throughput) Latency, Response Availability Data Protection Backup/Restore Archive, HA, BR BC, DR, Security Durability Capacity Resources CPU, Memory, I/O Storage, Network Ports, Services Software Licenses Economics Buy, Lease, Rent, Subscribe Upfront, Recurring Capex, Opex, Facilities HW, SW, Services Management PACE = Performance, Availability, Capacity, Economics + Management Source: Software Defined Data Infrastructure Essentials (CRC)
Industry Trends Enabling Business, Application & Data Growth All Available Data Infrastructure Resources Will Be Needed To Support Demand Workloads Data Storage SSD (Flash, SCM/PMEM), HDD, Tape Data Infrastructure Resources PACE Resource Demands Memory Time Compute X86/64, ARM GPU/ASIC/FPGA I/O Network PCIe, SAS, SATA, FC Ethernet, Gen Z DRAM, MRAM, PMEM, SCM 8 Resource Delivery Edge/ROBO (on prem and Via Public Cloud Core On Prem Data Center And Public Cloud (Conventional, BM, CI/HCI/CiB, Container, Scale out, Clusters, Stamps, Plexs) Data Infrastructures
Industry Trends Data Infrastructure and Services Layers Defining HW, SW, Services into Data Infrastructures and Related Services Metrics That Matter Insight and Awareness Storage (Ne, SSD, HDD, Tape) Hypervisor Software Defined Storage Software Defined Networks Adapters Physical Servers Operating Systems Hypervisors Adapters, I/O Networks Application Landscapes Containers Containers (Docker, K8, *nix, Win) AzureStack, Hyper V, K, ware, OpenStack, Xen ECS, EBS, vsan, S2D NSX and others Clouds AIX, Centos, Core, Nano, RHEL SUSE, Ubuntu, Windows Software Stacks Layers (Altitude) Core (Cloud or On Prem) Location Source: Software Defined Data Infrastructure Essentials (CRC) Value Increases Applications & Devices Layers / Altitude Edge
Industry Trends What Is The Cloud Depends on your views You Cant Manage and Optimize What You Don t Know or Have Insight Into Place or destination Product or Platform Protocol or process Paradigm or process Technology or tool HW, SW and Services For free or fee Services Resources o Compute o I/O & Network o Storage o Hardware o Software Different tiers, classes, categories of services Storage: Block, File, Object, Table, Queue Compute for example: o Lambda, AKS/EKS/PKS/K8s, ECS/Container o EC2/, Bare Metal/Dedicated/MaaS o CPU, GPU, ARM, Memory optimized o Demand, Spot, RI, BYOL, pay in advance Private Clouds Microsoft (Azure Stack, Win) OpenStack, Red Hat, ware CI / HCI / CiB Edge / Fog ROBO Public Clouds AWS, Azure, Google IBM, Alibaba and Many others Public Clouds Hybrid Gov Clouds
Industry Trends Some Cloud Resource Examples Everything is not the same in the Cloud however there are similarities Instance Type and Class Instance Limits (Container,, BM) Memory, I/O, Network # of CPU, Type, Speed On Instance Storage Metrics That Matter: Timely, Accurate, Reflective, Contextual JB, Database (SQL/NoSQL), Container and Application PACE Tuning Options Hypervisor BM Server Type CPU/Cores, Speed Memory, I/O Network, Storage CON CON CON CON CON CON MaaS Services AL/ML, IoT Imaging, Analytics General Resource limits API limits Other quotas Metrics That Matter Networking VPC/VNET Adapters, Speed Direct Connect Express Route Storage (Block, File, Object/Blob, Table, Queue) SCM/PMEM/SSD, HDD, Tape
Industry Trends Data Infrastructure Management Various layers and altitude across different locations Gaining Insight Data Infrastructure Management (more in depth on Wednesday) o Insight and awareness, avoid flying blind, metrics that matter o Orchestration, automation, remediation, node and cluster upgrades o Data and resource migration, cross platform, cross cloud Microsoft Various in box and add on tools, ARM, WAC/Honolulu, System Center, MAP, Migration Service, Azure Stack, Data Box Edge, IoT Edge among others ware vsphere, vcenter, Loginsghts & many others, keep an eye on Project Dimension Others AWS, Densify, A10, Splunk, Kubernetes, VirtualMetric, Irods, Datadog, Splunk, SAS, Loggly, Quest, SolarWinds and many others. Metrics That Matter Core (Cloud or On Prem) Location Applications & Devices Layers / Altitude Edge
Industry Trends Economic Financial Considerations What is the economic analysis of Consolidate, Cloud, Converge or other approaches? Metrics That Matter 36 MONTH CLOUD CONSOLIDATED $689,849 36 MONTH DISTRIBUTED FILE SERVERS $1,658,915 20 Sites each with 6TB used / allocated file server storage Assumes about 65% average utilization of storage per site Includes cost of managing (staffing, facilities, networks) Your costs will vary based on staffing and other factors Cloud compute and block storage as well as software costs Factors in network egress (transfer out) costs https://storageioblog.com/cloud file data storage consolidation and economic comparison model/
Industry Trends Data Infrastructure Related Topics Focus of Data Infrastructures is to support and enable information services delivery Microsoft Azure Data Box Disk Test Drive Review https://storageioblog.com/microsoft azure data box disk test drive review blogtobertech/ Cloud File Data Storage Consolidation and Economic Comparison Model https://storageioblog.com/cloud file data storage consolidation and economic comparison model/ Ten tips to reduce your cloud compute storage costs https://storageioblog.com/ten tips to reduce your cloud compute storage costs/ How I saved money storing more data on aws s3 simple storage service https://storageioblog.com/how i saved money storing more data on aws s3 simple storage service/ Dont Stop Learning Expand Your Skills Experiences Everyday https://storageioblog.com/dont stop learning expand your skills experiences everyday blogtobertech/ Vote for StorageIO blog in 2018 top vblog including independent blog and storage blog: https://topvblog.questionpro.com/ Download PDF slides via www.storageio.com/cmg2018
Closing comments, for now Everything is not the same, use right tool, technology technique for task at hand Expand your Data Infrastructure Tradecraft Experiences Everything is not the same, why treat everything the same? Avoid flying blind, have awareness into adjacent technologies Cost cutting is easy, removing cost without compromise is different Review software license particular under utilized licenses Review what your are subscribing to and how (pre pay vs. spot) Software Defined emphasis should be on Defined How things defined, composed, crafted, programmed for an outcome HW needs SW, SW needs HW, serverless and clouds needs HW & SW Where to learn more www.storageio.com (articles, videos & webcasts) www.picturesoverstillwater.com (for fun) Feel free to call, IM, tweet, or email greg@storageio.com www.storageio.com/book4.html Nine Time Download PDF slides via www.storageio.com/cmg2018