ISILON DATA LAKE 運用 ISILON 輕鬆建構企業資料湖 EDGE TO CORE TO CLOUD EMC Isilon 資深技術顧問 Frances Chien 簡君芳 1
Isilon 企業資料湖 Consolidated unstructured data storage infrastructure Scale-out architecture to support rapid data growth Supports multiple applications and workloads on a single platform Automated tiering to support varying performance requirements Extends from core data center to edge locations and the cloud Enables in-place data analytics 2
非結構性資料的成長 75% 78% 80% Nearly 2X data growth every 2 years 2015 2016 2017 71 EB 106 EB 133 EB Source: IDC: Total capacity shipped, worldwide % Of unstructured data 3
驅動非結構性資料成長的因素 Traditional and emerging workload examples Traditional workloads Emerging workloads Video Enterprise file data Archive Public records Social networks, user generated content Machine data Internet of Things Location data 4
建立一個企業資料湖 Consolidate data and eliminate storage silos Traditional workloads Emerging workloads NAS DAS SAN CLOUD TAPE OBJECT 5
建立一個企業資料湖 Consolidate data and eliminate storage silos Traditional workloads Emerging workloads NAS DAS SAN DATA LAKE Data Lake CLOUD TAPE OBJECT 6
Isilon 的水平擴充 scale-out 架構 With built-in multi-protocol support for multiple workloads and applications Powered by Intel Xeon Processors Windows Web Mac/iOS Apps Linux/Unix Cloud Archive Hadoop 7
Single volume 簡易的儲存系統管理 Isilon AutoBalance enables simple capacity growth Single volume and file system Directories and files striped across cluster nodes Automation NO manual intervention NO reconfiguration NO server or client mount point or application changes NO data migrations NO RAID 8
方便的儲存容量擴充 Isilon scale-out architecture with AutoBalance Balanced Empty Full Balanced AutoBalance Automated data balancing across nodes reduces costs, complexity, and risks for scaling storage Empty Full Balanced Empty Full Balanced Empty Full Balanced AutoBalance automatically moves content to new storage nodes Eliminates hot spots Enables unmatched storage capacity utilization of more than 80% Empty 9
Performance Isilon 的系列產品定位 Powered by Intel Xeon Processors S-Series Linear Scaling of Performance and Capacity High Performance Platform X-Series Highly Versatile Platform NL-Series HD-Series Nearline Platform High Density Platform Software Defined Your hardware Internal Cloud External Cloud Capacity 10
Performance 最佳化的自動分層 auto tiering Isilon SmartPools software Single point of management Single file system/single volume Multiple performance tiers Automatic data movement Policy-based tiering management Transparent reallocation NO application changes Optimize storage resources Automatically match storage resources with data requirements Eliminate data migration S-Series Performance X-Series Throughput NL-Series Nearline Capacity Reduced Costs HD-Series High Density 11
範例 : Isilon 的自動分層 auto tiering Isilon SmartPools software SmartPools Policy Example Isilon cluster at core data center <30 days 30 days- >30 days 1 year S210 NL410 > 1 year HD400 12
完善的資料保護功能 Backup and disaster recovery Data lake storage at core data center WAN/LAN Disaster recovery site Disk-to-disk backup Efficient snapshots with SnapshotIQ Remote archive and DR Data replication with SyncIQ Push-button simple failover/failback 13
安全性與法規相容性的選項 LEGAL IT ACCOUNTING MARKETING Roles-based access control (RBAC) Access Zones for secure isolation WORM Data Protection File System Auditing Data at Rest Encryption (DARE) with Self- Encrypting Drives (SEDs) Security and Technical Implementation Guide (STIG) hardening FIPS OpenSSL support 14
企業資料湖應用 直接資料分析 Scale-out storage with native Hadoop (HDFS) integration Enable in-place analytics Native integration speeds time to insight Eliminates need to copy and move data Data lake protection Snapshots for efficient backup and recovery Data replication for disaster recovery Lower costs No need for dedicated Hadoop infrastructure Increase flexibility Simultaneous support for any Apache-compliant Hadoop distribution Ambari integration for management, monitoring, and provisioning 15
Isilon 支援各種資料分析平台 NFS SMB NFS SMB SMB, NFS, HTTP, FTP, HDFS name node name node name node data node HDFS NFS name node MAP Reduce MAP Reduce MAP Reduce MAP Reduce MAP Reduce MAP Reduce 16
使用 Isilon 來達到資料倉儲 EDW 最佳化 Enterprise data warehouse offloading Increase efficiency with an active archive Offload cold data from your EDW Free up EDW resources for critical data analysis Data in active archive remains available to be queried, searched and analyzed Offload ETL and ELT processes Offload data wrangling and curation of data Gain increased EDW performance and accelerate time-to-results Harness the power of an Isilon Data Lake Lower costs and simplify management 17
延伸配置的企業資料湖 Storing, managing and protecting data beyond the core data center Powered by Intel Xeon Processors Enterprise Edge edge locations Core data Core center Cloud 44% of enterprises have 10-50 TB per branch office 4-10% of remote servers to support data collection for IoT Massively scalable capacity Reduce cost 18
Isilon 企業資料湖 Connecting edge-to-core-to-cloud Powered by Intel Xeon Processors Enterprise EDGE edge locations Core data CORE center CLOUD Cloud Software defined storage Non-disruptive upgrades Policy-based, automated tiering with roll-back option to the cloud 19
可延伸至企業子公司或小型辦公室 IsilonSD Edge software defined storage Product Features Software Defined scale-out NAS with OneFS features Runs on industry standard hardware Integrated with VMware Scales to 36 TB and 3-6 nodes Free & frictionless (non-production, community support) Key Benefits Simple, agile and cost-efficient Easy to manage with standard VMware tools Consolidate/distribute data from/to edge locations Other use cases: test/dev, small-medium businesses 20
可延伸至公有雲或私有雲 Isilon CloudPools software Product Features Cloud-enabled data lake Seamless policy-based data placement Simple to deploy and manage Encryption and compression Key Benefits Choice of cloud service providers Lower costs and optimize on-premise storage resources Extend data center with web-scale capacity Secure network transmission to the cloud Transparent to users and applications 21
至雲端的自動分層 auto tiering Isilon SmartPools and CloudPools Isilon cluster at core data center Cloud storage provider SmartPools with CloudPools Policy Example <30 days S210 30 days to 1 year NL410 1 year to > 1 year HD400 2 years Public, off-prem: >2 years Cloud Private, on-prem: Isilon or ECS 22
Isilon 企業資料湖 Edge-to-core-to-cloud EMC Scale-out storage Cloud-enabled Software-defined Edge Private Core Data Center Hosted Public 23
Isilon 企業資料湖的優勢 1 2 3 4 5 6 Eliminate inefficient islands of storage Simplify management and reduce costs Enable better information sharing Increase data protection and security Accelerate data analytics to gain new insight Support data-driven decision making 24
Isilon 企業資料湖提供的選項 25
相關的資源 For more information about Isilon products and solutions: www.emc.com/isilon To see Isilon products and solutions in our online store: https://store.emc.com/isilon To download the Isilon OneFS Simulator at no charge for non-production use, to create a simulated environment and get a feel for the interface and administration tasks available in the latest Isilon OneFS operating system software release: www.emc.com/getisilon To download a non-production version of IsilonSD Edge, the first software-defined storage solution with the power and flexibility of Isilon, at no charge: www.emc.com/getisilonsdedge Join the Isilon Community to access documentation, user guides, FAQs, and training. You can also join Community user discussions to get further insight about the latest OneFS features: https://community.emc.com/community/products/isilon 26
Appendix
企業應用場景 (HPC, MEDIA EDITING, BIG DATA, DR) Team A HPC Users Team B SmartConnect SmartPools SmartQuotas SnapshotIQ SmartPools Media Editing Users Team C SMB NFS HDFS SyncIQ CloudPool AWS or Azure Public Cloud File Sharing Users 10GbE Switch S210&X410 Cluster Production Site NL400&HD400 Cluster DR Site Hadoop 28
功能強大的監控軟體 INSIGHTIQ 完全掌握 USER 行為與歷史紀錄 29
ALL FLASH ISILON IS COMING! Project Nitro, released in mid 2017 Hardware: 4U chassis 4 nodes (CPU/RAM/Network) in 4U 60 drives, total 900TB with 15TB SSDs 40GbE x8 frontend and 40GbE/IB x8 backend 100+ chassis / 400+ nodes per cluster 15GB/s throughput and 250,000 OPS per 4U So total 1.5TB/s throughput and 25million OPS per cluster by x100 Latency much faster All enterprise features supported All flash competitors usually don t support Integrates into your existing cluster Protect your existing investment 30