ISILON DATA LAKE 運用 ISILON 輕鬆建構企業資料湖

Size: px

Start display at page:

Download "ISILON DATA LAKE 運用 ISILON 輕鬆建構企業資料湖"

Cathleen Elliott
6 years ago
Views:

1 ISILON DATA LAKE 運用 ISILON 輕鬆建構企業資料湖 EDGE TO CORE TO CLOUD EMC Isilon 資深技術顧問 Frances Chien 簡君芳 1

Isilon 企業資料湖 Consolidated unstructured data storage infrastructure Scale-out architecture to support rapid data growth Supports multiple applications and workloads on a

2 Isilon 企業資料湖 Consolidated unstructured data storage infrastructure Scale-out architecture to support rapid data growth Supports multiple applications and workloads on a single platform Automated tiering to support varying performance requirements Extends from core data center to edge locations and the cloud Enables in-place data analytics 2

非結構性資料的成長 75% 78% 80% Nearly 2X data growth every 2 years 2015 2016 2017 71 EB

3 非結構性資料的成長 75% 78% 80% Nearly 2X data growth every 2 years EB 106 EB 133 EB Source: IDC: Total capacity shipped, worldwide % Of unstructured data 3

4 驅動非結構性資料成長的因素 Traditional and emerging workload examples Traditional workloads Emerging workloads Video Enterprise file data Archive Public records Social networks, user generated content Machine data Internet of Things Location data 4

5 建立一個企業資料湖 Consolidate data and eliminate storage silos Traditional workloads Emerging workloads NAS DAS SAN CLOUD TAPE OBJECT 5

6 建立一個企業資料湖 Consolidate data and eliminate storage silos Traditional workloads Emerging workloads NAS DAS SAN DATA LAKE Data Lake CLOUD TAPE OBJECT 6

7 Isilon 的水平擴充 scale-out 架構 With built-in multi-protocol support for multiple workloads and applications Powered by Intel Xeon Processors Windows Web Mac/iOS Apps Linux/Unix Cloud Archive Hadoop 7

8 Single volume 簡易的儲存系統管理 Isilon AutoBalance enables simple capacity growth Single volume and file system Directories and files striped across cluster nodes Automation NO manual intervention NO reconfiguration NO server or client mount point or application changes NO data migrations NO RAID 8

方便的儲存容量擴充 Isilon scale-out architecture with AutoBalance Balanced Empty Full Balanced AutoBalance Automated data balancing across nodes reduces costs, complexity, and risks for scaling storage Empty

9 方便的儲存容量擴充 Isilon scale-out architecture with AutoBalance Balanced Empty Full Balanced AutoBalance Automated data balancing across nodes reduces costs, complexity, and risks for scaling storage Empty Full Balanced Empty Full Balanced Empty Full Balanced AutoBalance automatically moves content to new storage nodes Eliminates hot spots Enables unmatched storage capacity utilization of more than 80% Empty 9

10 Performance Isilon 的系列產品定位 Powered by Intel Xeon Processors S-Series Linear Scaling of Performance and Capacity High Performance Platform X-Series Highly Versatile Platform NL-Series HD-Series Nearline Platform High Density Platform Software Defined Your hardware Internal Cloud External Cloud Capacity 10

application changes Optimize storage resources Automatically match storage resources with data requirements Eliminate

11 Performance 最佳化的自動分層 auto tiering Isilon SmartPools software Single point of management Single file system/single volume Multiple performance tiers Automatic data movement Policy-based tiering management Transparent reallocation NO application changes Optimize storage resources Automatically match storage resources with data requirements Eliminate data migration S-Series Performance X-Series Throughput NL-Series Nearline Capacity Reduced Costs HD-Series High Density 11

12 範例 : Isilon 的自動分層 auto tiering Isilon SmartPools software SmartPools Policy Example Isilon cluster at core data center <30 days 30 days- >30 days 1 year S210 NL410 > 1 year HD400 12

with SnapshotIQ Remote archive and DR Data

13 完善的資料保護功能 Backup and disaster recovery Data lake storage at core data center WAN/LAN Disaster recovery site Disk-to-disk backup Efficient snapshots with SnapshotIQ Remote archive and DR Data replication with SyncIQ Push-button simple failover/failback 13

安全性與法規相容性的選項 LEGAL IT ACCOUNTING MARKETING Roles-based access control (RBAC) Access Zones for secure isolation WORM Data Protection File System

14 安全性與法規相容性的選項 LEGAL IT ACCOUNTING MARKETING Roles-based access control (RBAC) Access Zones for secure isolation WORM Data Protection File System Auditing Data at Rest Encryption (DARE) with Self- Encrypting Drives (SEDs) Security and Technical Implementation Guide (STIG) hardening FIPS OpenSSL support 14

15 企業資料湖應用直接資料分析 Scale-out storage with native Hadoop (HDFS) integration Enable in-place analytics Native integration speeds time to insight Eliminates need to copy and move data Data lake protection Snapshots for efficient backup and recovery Data replication for disaster recovery Lower costs No need for dedicated Hadoop infrastructure Increase flexibility Simultaneous support for any Apache-compliant Hadoop distribution Ambari integration for management, monitoring, and provisioning 15

16 Isilon 支援各種資料分析平台 NFS SMB NFS SMB SMB, NFS, HTTP, FTP, HDFS name node name node name node data node HDFS NFS name node MAP Reduce MAP Reduce MAP Reduce MAP Reduce MAP Reduce MAP Reduce 16

17 使用 Isilon 來達到資料倉儲 EDW 最佳化 Enterprise data warehouse offloading Increase efficiency with an active archive Offload cold data from your EDW Free up EDW resources for critical data analysis Data in active archive remains available to be queried, searched and analyzed Offload ETL and ELT processes Offload data wrangling and curation of data Gain increased EDW performance and accelerate time-to-results Harness the power of an Isilon Data Lake Lower costs and simplify management 17

center Cloud 44% of enterprises have 10-50 TB per branch office 4-10% of remote

18 延伸配置的企業資料湖 Storing, managing and protecting data beyond the core data center Powered by Intel Xeon Processors Enterprise Edge edge locations Core data Core center Cloud 44% of enterprises have TB per branch office 4-10% of remote servers to support data collection for IoT Massively scalable capacity Reduce cost 18

19 Isilon 企業資料湖 Connecting edge-to-core-to-cloud Powered by Intel Xeon Processors Enterprise EDGE edge locations Core data CORE center CLOUD Cloud Software defined storage Non-disruptive upgrades Policy-based, automated tiering with roll-back option to the cloud 19

20 可延伸至企業子公司或小型辦公室 IsilonSD Edge software defined storage Product Features Software Defined scale-out NAS with OneFS features Runs on industry standard hardware Integrated with VMware Scales to 36 TB and 3-6 nodes Free & frictionless (non-production, community support) Key Benefits Simple, agile and cost-efficient Easy to manage with standard VMware tools Consolidate/distribute data from/to edge locations Other use cases: test/dev, small-medium businesses 20

21 可延伸至公有雲或私有雲 Isilon CloudPools software Product Features Cloud-enabled data lake Seamless policy-based data placement Simple to deploy and manage Encryption and compression Key Benefits Choice of cloud service providers Lower costs and optimize on-premise storage resources Extend data center with web-scale capacity Secure network transmission to the cloud Transparent to users and applications 21

至雲端的自動分層 auto tiering Isilon SmartPools and CloudPools Isilon cluster at core data center Cloud storage provider SmartPools with CloudPools Policy

22 至雲端的自動分層 auto tiering Isilon SmartPools and CloudPools Isilon cluster at core data center Cloud storage provider SmartPools with CloudPools Policy Example <30 days S days to 1 year NL410 1 year to > 1 year HD400 2 years Public, off-prem: >2 years Cloud Private, on-prem: Isilon or ECS 22

23 Isilon 企業資料湖 Edge-to-core-to-cloud EMC Scale-out storage Cloud-enabled Software-defined Edge Private Core Data Center Hosted Public 23

Isilon 企業資料湖的優勢 1 2 3 4 5 6 Eliminate inefficient islands of storage Simplify management and reduce costs Enable better

24 Isilon 企業資料湖的優勢 Eliminate inefficient islands of storage Simplify management and reduce costs Enable better information sharing Increase data protection and security Accelerate data analytics to gain new insight Support data-driven decision making 24

25 Isilon 企業資料湖提供的選項 25

26 相關的資源 For more information about Isilon products and solutions: To see Isilon products and solutions in our online store: To download the Isilon OneFS Simulator at no charge for non-production use, to create a simulated environment and get a feel for the interface and administration tasks available in the latest Isilon OneFS operating system software release: To download a non-production version of IsilonSD Edge, the first software-defined storage solution with the power and flexibility of Isilon, at no charge: Join the Isilon Community to access documentation, user guides, FAQs, and training. You can also join Community user discussions to get further insight about the latest OneFS features: 26

27 Appendix

企業應用場景 (HPC, MEDIA EDITING, BIG DATA, DR) Team A

Azure Public Cloud File Sharing Users 10GbE Switch

28 企業應用場景 (HPC, MEDIA EDITING, BIG DATA, DR) Team A HPC Users Team B SmartConnect SmartPools SmartQuotas SnapshotIQ SmartPools Media Editing Users Team C SMB NFS HDFS SyncIQ CloudPool AWS or Azure Public Cloud File Sharing Users 10GbE Switch S210&X410 Cluster Production Site NL400&HD400 Cluster DR Site Hadoop 28

29 功能強大的監控軟體 INSIGHTIQ 完全掌握 USER 行為與歷史紀錄 29

30 ALL FLASH ISILON IS COMING! Project Nitro, released in mid 2017 Hardware: 4U chassis 4 nodes (CPU/RAM/Network) in 4U 60 drives, total 900TB with 15TB SSDs 40GbE x8 frontend and 40GbE/IB x8 backend 100+ chassis / 400+ nodes per cluster 15GB/s throughput and 250,000 OPS per 4U So total 1.5TB/s throughput and 25million OPS per cluster by x100 Latency much faster All enterprise features supported All flash competitors usually don t support Integrates into your existing cluster Protect your existing investment 30

BIG DATA READY WITH ISILON JEUDI 19 NOVEMBRE Bertrand OUNANIAN: Advisory System Engineer

BIG DATA READY WITH ISILON JEUDI 19 NOVEMBRE 2015 Bertrand OUNANIAN: Advisory System Engineer Unstructured Data Growth Total Capacity Shipped Worldwide % of Unstructured Data 67% 74% 80% 2013 37 EB 2015