Bridging Traditional Analytics with BigData - SAS on UCS

Transcription

1

2 Bridging Traditional Analytics with BigData - SAS on UCS Vadiraja Bhatt, Principal Engineer

3 Cisco Spark How Questions? Use Cisco Spark to communicate with the speaker after the session 1. Find this session in the Cisco Live Mobile App 2. Click Join the Discussion 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot#

4 Agenda Introduction Introduction to SAS SAS Analytics landscape UCS reference architecture UCS value add for SAS Conclusion

5 Cisco UCS BigData Solution UCS is a next gen Data Center Architecture Built from ground up for Virtualization and Cloud Programmable infrastructure Policy based application intent implementation Open standards with rich set of APIs UCS on BigData 4B revenue in just 7 years Big Data revenue to grow substantially Solution based approach Integrated and bundled solutions 5

6 SAS - A Company profile SAS originally stood for Statistical Analysis Software. Started in year 1976 by the SAS Institute. Privately Owned. Dr. James Goodnight CEO since inception employees worldwide 3.2B+ revenue ( 2016) HQ in Cary, NC 55K+ customers 6

7 What is SAS? SAS software can perform various operations on the data Data Management Statistical Analysis/Machine Learning Report formation with perfect graphics Business Planning Operations Research and project Management Quality Improvement Application Development Data extraction Data transformation Data updation and modification Has more than 200 components for various use cases. 7

8 SAS Analytics use case Analytical Categories and Target Usage 8

9 SAS Components (prominent) Base SAS It is a core component which contains data management facility and a programming language for data analysis. It is also the most widely used. SAS/GRAPH Create graphs, presentations for better understanding and showcasing the result in a proper format. SAS/STAT Perform Statistical analysis with the variance analysis, regression, multivariate analysis, survival analysis, and psychometric analysis, mixed model analysis. SAS/Enterprise Miner Data mining. 9

10 SAS Solution Landscape ( relevant) SAS 9.4 Foundation Stand alone Distributed SAS Grid with SAN ( For ex GPFS) For Power Users, Statisticians SAS VA/VS 7.3 Stand alone SAS In-Memory Analytics Cluster ( SAS LASR Analytic Server) HDFS to store data In-memory co-located it with Hadoop cluster For Analysts and decision makers Decision Tree/Machine Learning with VS 10

11 SAS - Solutions 11

12 SAS Business Intelligence Architecture 12

13 SAS 9.4 Foundation Genesis of all SAS products Variety of server side SAS products Single instance Available in Both Linux and Windows version Small to medium enterprise Typical 2 socket to 4 Socket system Licensing is based on number of cores More for Power users and Statisticians CPU performance is key 13

14 SAS 9 Deployment Architecture

15 SAS Foundation 9.4/SAS VA-VS 7.3 Connectivity Server Configuration Stand Alone 1 Cisco UCS C240 M5 Server with: 2 Intel Xeon Processor Scalable Family 6132 (2x14 cores and 2.6 GHz)(28 cores) 512-GB DDR4 memory Cisco UCS Virtual Interface Card (VIC) 1387 ( 40Gb ports) 4 x 1.6-TB Enterprise Value SATA SSD SFF(RAID0 for WORK and UTILLOC) ( 500MB/sec disks, 4 X 500 = 2GB/sec ) 22 x 600-GB 15Krpm SAS hard-disk drives (HDDs)(3GB/sec bandwidth) Data Storage Local File system.(posix) 15

16 Scale out SAS on UCS SAS Grid with UCS Blade server with SAN Storage Clustered file system like GPFS is preferred. SAS LASR analytics with HDFS Both LASR and Hadoop co-located on same cluster Distributed VA CVD released. SAS Grid with Hadoop cluster SAS calls into Hadoop cluster through SAS ACCESS for Hadoop 16

17 SAS Grid Traditional distributed SAS 17

18 SAS Grid Computing B200 M5 C240 M5 SAS ACCESS C240 M5 SAS ACCESS 18

19 When is SAS Grid preferred SAS applications are mission critical, and need to set up a highly available infrastructure. Many SAS users running multiple SAS applications, and need to implement a shared SAS analytic environment that allocates resources as needed. End of month or end of quarter SAS processing needs that have very tight SLAs and need to be guaranteed the computing resources to meet these SLAs. Establish a hardware and software infrastructure that can be scaled out to meet ever growing number of SAS users and the ever-growing data being analyzed by these SAS users. Can gain some performance improvements by using the new SAS High Performance Analytics processes or by taking existing SAS processes and converting them to a distributed processing format. 19

20 SAS Grid Infrastructure Characteristics Fits well for UCS Blades Requires SAN connectivity for shared file system Requires Clustered file system for SAS DATA Access IBM GPFS is popular choice For temporary data storage local SSDs or Flash drives SAN capacity planning is crucial Ensure enough ports and bandwidth to support Large sequential reads Project is underway benchmarking the SAS Foundation followed by Grid. 20

21 SAS Grid Computing on UCS Connectivity Server Configuration 2 X Cisco UCS UP 40-Port Fabric Interconnect 4 to 8 Cisco UCS B200 M5 Blade Servers, each with: 2 Intel Xeon Processor Scalable Family 6132 (2x14 cores and 2.6 GHz) 512-GB of DDR4 memory Cisco UCS VIC x 600-GB, 15,000-rpm SAS HDD CISCO UCS UP 2 Cisco UCS 5108 Blade Server chassis ( 4 server on each chassis) OS Data Storage RHEL 7.4 GPFS and any SAN 21

22 CISCO UCS 6248UP ID STAT CISCO UCS 6248UP ID STAT SAS Foundation on B200 M4 Server SAS 9.4 SAS 9.4 RHEL 7.4 RHEL 7.4 UCS B200 M5 UCS B200 M5 4 X 40Gb FCoE Cisco UCS FI UP Cisco UCS 5108 Blade Chassis Cisco UCS FI UP L2 Switch 10 GbE uplink 8 X 16/8 Gb FC (Through SAN Switch) SAN 22

23 Logical N/W Connectivity SAS 9.4 Foundation RHEL 7.4 SAS 9.4 network Boot A Boot B Data 1A Data 1B Data 2A Data 2B vnic0 vhba0 vhba1 vhba2 vhba3 vhba4 y vhba5 Cisco VIC 1340 Cisco UCS B200 M4 Failover path Cisco UCS UP A Cisco UCS UP B SAN Switch A SAN Switch B 10GbE network uplink 40GbE Converged Links 8G/16G FC links Controller A SAN Storage Controller B

24 Storage Layout /SASDATA /SASWORK XFS file system for SASDATA XFS file for SASWORK. UTILLOC /root Boot LUN LVM Vol For SASDATA 16 1 LVM Vol For WORK/UTALLOC 16 1 RHEL 7.4 Linux Multipath with 16 paths per LUN 16 Data LUNS 16 WORK/UTALLOC LUNS SAN Boot LUN LUN 16 LUN 16 LUN 1 Flash Storage Storage LUN 1 SAN Storage 24

25 SAS IO test with IBM FS 900 with GPFS Upto 19GB/sec across 4 nodes 7GB/sec read per node peak SAS IO test with GPFS 16Gb FC connectivity from FI to storage Direct Connect Storage connectivity Published record SPECFs number with this configuration Aggregate Read Throughput(MB/s) Aggregate Write Throughput(MB/s) Limited by Storage bandwidth. 0 1 Node 2 Node 3 Node 4 Node 25

26 SAS Visual Analytics server SAS Visual Analytics is primarily a web-based application that takes advantage of inmemory analytics Includes several components providing following broad range of functionality Analysts can use SAS Visual Analytics Explorer to explore data. Report designers can use SAS Visual Analytics Designer to create reports and dashboards Report designers can also use SAS Visual Analytics Graph Builder to create graph templates that can beused in multiple reports and shared with other users. Executives and other appropriate users can use the SAS Visual Analytics Viewer as well as native mobileviewers to surface reports and dashboards. Administrators can use SAS Visual Data Builder to prepare and load data into the environment. For Analysts and decision makers to directly visualize the data 26

27 SAS Visual Analytics server 27

28 SAS VA Non-distributed Workspace Server accepts incoming data and loads it into the LASR server LASR Server is where data is stored in memory and processed The Metadata Server makes sure that user s loading data have the correct permissions and mainains info about tables. 28

29 SAS LASR Analytics Server ( Distributed) The SAS LASR Analytic Server is an analytic platform that provides a secure, multi- user environment for concurrent access to data that is loaded into memory. Takes advantage of a distributed computing environment by distributing data and the workload among multiple machines and performing massively parallel processing. Supports the Hadoop Distributed File System (HDFS) as a co-located data provider HDFS is used because the server can read from and write to HDFS in parallel. HDFS provides replication for data redundancy. HDFS stores data as blocks in distributed form on the blades and the replication provides failover capabilities. SAS Visual Analytics uses SAS LASR Analytics for analysis 29

30 SAS In Memory Analytics Cluster with HDFS SAS LASR Analytic Server Data is loaded into the in-memory environment in parallel The in-memory data is distributed among multiple machines SAS keeps the data and computations massively parallel. The in-memory environment brings the analytics closer to the data

31 SAS VA- Distributed architecture Node 0 Node 1 Node 2 Noden

32 Component level deployment 32

33 SAS In Memory Analytic Cluster ( For Data) Connectivity 2 Cisco UCS Port Fabric Interconnects CISCO UCS UP Server Configuration 8 to 16 Cisco UCS C240 M5 Rack Servers, each with: 2 Intel Xeon Processor Scalable Family 6132 (2x14 cores and 2.6 GHz) 512-GB of DDR4 memory Cisco UCS VIC drives of 1.8-TB 10K SFF SAS HDD drives (Capacity) 12 x 1.6-TB Enterprise Value SATA SSD ( Performance) CISCO UCS C nodes Data Storage HDFS > 20 TB ( Raw) per node 33

34 SAS In Memory Analytic Cluster (cont ) Hadoop Management (Name node HA, SAS root node, Resource management) 3 X UCS C240 M5 Rack Servers, each with: 2 Intel Xeon Processor Scalable Family 6132 (2x14 cores and 2.6 GHz)) 256-GB of DDR4 memory Cisco UCS VIC x 1.6-TB Enterprise Value SATA SSD SAS Servers (Metadata, Workspace, WebApp) 3 X UCS C240 M5 Rack Servers, each with: 2 Intel Xeon Processor Scalable Family 6132 (2x14 cores and 2.6 GHz)) 256-GB of DDR4 memory Cisco UCS VIC x 1.6-TB Enterprise Value SATA SSD 34

35 UCS for VA distributed 16 Data Nodes + LASR Worker Nodes 2 Name Node 1 Hadoop Name node HA + VA Root node + HPMC 1 Name node HA + VA Root node installed( manual restart for Root node) 1 Resource Manager 3 node VA Infra 1 Meta data 1 Web App + Web 1 VA Server tier Provides Hadoop level HA 3 hadoop mgmt 3 VA Infra VA root node NN NN HA Hadoop Mgmt Hadoop Mgmt Web App VA server Tier? Worker node Worker node Worker node HDFS HDFS HDFS 16 Worker +DATA Node 35

36 SAS VA/VS Distributed design nodes design No HA at SAS LASR level SAS Web App+ SAS server + Meta data in single server is common deployment scenario SAS Client SAS Web App tier SAS VA Server tier SAS Metadata SAS Worker node + Data Node SAS Root Node + Name Node Name Node HA + Root node installed( for restart) Hadoop Mgmt node 3 nodes 16 nodes 3 nodes Root node is with Primary Name node HA at HDFS Name node level only Manual restart of root node is needed if Primary name node fails 36

37 SAS Sizing consideration Storage sizing is key for achieving optimal performance SAS DATA, SAS WORK and SAS UTILLOC are the three different storage partitions SAS DATA where actual data for processing is accessed. Large sequential Read (64k to 256k) MB/core is the typical processing power. SAN or Local storage SAS WORK and UTILLOC Temporary work tables and other bookkeeping storage space Large Sequential read and write Local storage preferred. SSD or FLASH opportunity 37

38 SAS Deployment options SAS Standalone VA/VS/Base SAS 2 socket C240 based server with sufficient storage C460 when more memory and compute power is required. SAS Grid Fits well with the Blade for compute If UTILLOC/SASWORK needs more bandwidth then C240 with local SSD/Flash storage I/O Bandwidth consideration is key B200 M4 with V4 processor can easily drive more than 5GB/sec with VIC 1340 SAN as the shared storage 38

39 SAS Deployment options ( contd) SAS VA Distributed LASR distributed across multiple nodes. Storage Compute Co-located HDFS co-located with VA C240 based CPA fits in very well. Preferred architecture for Cisco Storage and VA on separate cluster Storage can be on traditional SAS source. Loaded to LASR server through workspace server. Blade server may be a better fit just for LASR cluster. DI or DataLoader External Hadoop cluster UCS Blade for LASR Cluster BigData CPA for Hadoop storage Data Loader for Hadoop 39

40 SAS with Hadoop Deployment model SAS Embedded Process for Hadoop Allows access to Hadoop from existing SAS through SAS/ACCESS Interface to Hadoop SAS Embedded Process enables analysts to run programs in Hadoop. Eliminates data movement between Hadoop and SAS Perform the analytics where the data is stored and uses the distributed, parallel processing architecture of Hadoop for improved performance. SAS Grid with Hadoop SAS ACCESS to Hadoop Users can access data in Hadoop( like any other source) Computation is done in SAS SAS/ACCESS interface to data source Embedded Process Hadoop will be on C240 Based CPA S A S A C C E S S 40

41 SAS High Performance Analytics with Hadoop Provides the greatest potential for the fastest analytics on very large data sets from Hadoop. SAS LASR Analytic Server and SAS Plug-ins for Hadoop. that can take advantage of this environment SAS Visual Analytics and SAS Visual Statistics are the two products that uses it Two deployment options Hadoop as remote cluster SAS Embedded Process is deployed on the nodes of the Hadoop cluster to handle asymmetric, distributed workloads from remote Hadoop to the SAS analytics cluster. Hadoop co-located with Analytic cluster Hadoop cluster can be used to stage data in HDFS as SASHDAT tables. SASHDAT file format is memory efficient, supports SAS formats, and is optimal for use with in-memory processing. 41

42 SAS HPA with Hadoop C240 based CPA for both SAS and Hadoop Our current CVD implements it Best for optimal performance C240 based CPA for Hadoop B200 M4 or C240 M4 cluster for SAS SAS LASR may be preexisting 42

43 SAS Shared Environment Shareable components Meta Data server Middle tier Data Source 43

44 SAS BI Integrated platform SAS LASR Analytics clusters SAS Grid Connected to SAN Shared Metadata, Web App with HA UCS Rack-Mount Servers UCS Blade Servers SAN/NAS Arrays 44

45 SAS UCS Value add Simple to deploy Automated deployment through UCS SP for various SAS Components. Consistent design Network Path IO Path Open API to integrate with any Cloud orchestration tool Scalable Deploy as many servers based on base Service profiles. No manual intervention, hence less risky Configured in minutes Superior performance Converged fabric provides balanced CPU/Memory/IO configuration 40G Fabric to support heavy IO demand. Proven technology with benchmark leadership in BigData 45

46 SAS on UCS Reference Architecture SAS Solution SAS Foundation 9.4/SAS VA 7.3 SAS In-Memory Analytics cluster with HDFS (SAS LASR Analytic Server) SAS Grid with Traditional Shared- Storage Solution with Cisco UCS SAS LASR Analytics with external storage Hadoop Cluster for SAS LASR or SAS Grid storage Connectivity Stand Alone 2 Cisco UCS UP 32-Port Fabric Interconnects 2 Cisco UCS UP 32-Port Fabric Interconnects 2 Cisco UCS UP 32-Port Fabric 2 Cisco UCS UP 32-Port Interconnects Fabric Interconnects Server Configuration 1 Cisco UCS C240 M5 Rack Server with: 2 Intel Xeon Processor Scalable Family 6132 (2x14 cores and 2.6 GHz) 512-GB DDR4 memory Cisco UCS Virtual Interface Card (VIC) x 1.6-TB Enterprise Value SATA SSD SFF 22 x 600-GB 15,000-rpm SAS harddisk drives (HDDs) 22 UCS C240 M5 Rack Servers, each with: 2 Intel Xeon Processor Scalable Family 6132 (2x14 cores and 2.6 GHz) 512-GB of DDR4 memory Cisco UCS VIC drives of 1.8-TB 10K SFF SAS HDD drives or 12 x 1.6-TB Enterprise Value SATA SSD 8 Cisco UCS B200 M5 Blade Servers, each with: 2 Intel Xeon Processor Scalable Family 6132 (2x14 cores and 2.6 GHz)) 512-GB of DDR4 memory Cisco UCS VIC x 600-GB, 15,000-rpm SAS HDDs 8 Cisco UCS B200 M5 Blade Servers, each with: 2 Intel Xeon Processor Scalable Family 6132 (2x14 cores and 2.6 GHz)) 512-GB of DDR4 memory Cisco UCS VIC x 600-GB, 15,000-rpm SAS HDDs 19 UCS C240 M5 Rack Servers, each with: 2 Intel Xeon Processor Scalable Family 6132 (2x14 cores and 2.6 GHz) 512-GB of DDR4 memory Cisco UCS VIC drives of 1.8-TB 10K SFF SAS HDD drives or 12 x 1.6-TB Enterprise Value SATA SSD 16 for Data+worker node 3 for Hadoop management 3 for SAS Servers For existing SAS deployments requiring only hadoop Cluster for stroage and compute. 16 for Data Nodes 3 for Hadoop management Data Storage SAN/Local Storge with XFS HDFS SAN with Clustered File system like GPGS Data from external sources loaded to LASR Total Data storage(gb) In compute Total number of Cores Total memory(gb)

47 UCS BigData Solutions ( CVD) Designed Based on common customer use case Identify components which solves common use case Application, Platform, Infrastructure Not tied to any particular vertical Architected Design the plumbing between components Identify the control and data flow Establish the best practices across all the component in the stack Validated Describe the installation, configuration and tuning in detail Show validation results Functionality, performance, availability Supported Support as a solution One throat to choke 47

48 UCS BigData Solutions - Advantages Addresses key concerns of all the decision makers Application owners What all components required? How is it solving the problem? Are all components work as expected? IT What is the infrastructure need? Are all the components certified and inter dependencies taken care of? What is the sizing guidelines? CTO/CIO How quickly can solution be deployed? What is the TCO advantage? How is the support? 48

49 Resources SAS on UCS Solution brief SAS VA/VS with LASR analytics server on UCS (CVD) BigData Solutions on Cisco UCS 49

50 Cisco Spark How Questions? Use Cisco Spark to communicate with the speaker after the session 1. Find this session in the Cisco Live Mobile App 2. Click Join the Discussion 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot#

51 Please complete your Online Session Evaluations after each session Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt All surveys can be completed via the Cisco Live Mobile App or the Communication Stations Complete Your Online Session Evaluation Don t forget: Cisco Live sessions will be available for viewing on-demand after the event at

52 Continue Your Education Demos in the Cisco campus Walk-in Self-Paced Labs Tech Circle Meet the Engineer 1:1 meetings Related sessions 52

53 Thank you

54