Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
|
|
- Percival Franklin
- 6 years ago
- Views:
Transcription
1 VIRT1351BE New Architectures for Virtualizing Spark and Big Data Workloads on vsphere Justin Murray Mohan Potheri VMworld 2017 Content: Not for publication #VMworld #VIRT1351BE
2 Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitment from VMware to deliver these features in any generally available product. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined. 2
3 Agenda 1 Introductions 2 Existing and new Approaches in the Big Data World 3 Traditional Deployment Reference Architectures 4 New Architectures Changing the Paradigm 5 Proof of Concept: Testing in the VMware Solutions Lab 6 Introduction to Machine Learning 7 Conclusions 3
4 Why the Interest in Big Data? Enterprises want to get off existing costly data platforms Older data warehouse technology is not serving your needs Want to do queries and analytics against many different forms of data (structured, unstructured, streaming) Provide data access to our end customers Integrate systems that have been islands till now Single source of truth for the enterprise Exploit new application architectures for developer productivity Want to do data science, machine learning, deep learning VMworld 2017 Content: Not for publication 4
5 The Existing Hadoop Architecture Client ResourceManager Master Scheduler NameNode Master File System Index submit job Worker Node 1 Worker Node 2 Worker Node 3 Nodemanager AppMaster - 1 Datanode Workers Nodemanager Datanode Nodemanager Container - 2 Container - 3 Datanode HDFS Block 1 HDFS Block 2 HDFS Block 3 5
6 High Level View of Spark 6
7 The Spark Architecture Standalone Driver Job Worker Node 1 Worker Node 2 Worker Node 3 Executor JVM Executor JVM Executor JVM Executor JVM Executor JVM Executor JVM 7
8 The Spark Architecture (on YARN) Job Namenode Worker Node 1 Worker Node 2 Worker Node 3 Nodemanager AppMaster - 1 Datanode Resourcemanager Nodemanager Datanode Nodemanager Driver Executor Executor Container - 2 Container - 3 Datanode HDFS Block 1 HDFS Block 2 HDFS Block 3 8
9 Traditional Reference Architectures
10 Two Virtual Machines on a Host Server vsphere Host Server Hadoop Node 1 Virtual Machine Ext4 Nodemanager Ext4 Ext4 Datanode Ext4 Ext4 Ext4 Hadoop Node 2 Virtual Machine Ext4 Nodemanager Ext4 Datanode Ext4 Ext4 Ext4 Ext4 VMDK VMDK VMDK VMDK VMDK VMDK VMDK VMDK VMDK VMDK VMDK VMDK Local DAS disks/devices allocated to a Virtual Machine 10
11 Data/Compute Separation (with External Access to HDFS) Hadoop Virtual Node 1 Virtualization Host ResourceManager Ext4 Ext4 OS Image OS VMDK Image OS VMDK Image VMDK VMDK VMDK Hadoop Virtual Node 2 VMDK Temp NodeManager Ext4 Ext4 Hadoop Virtual Node 3 NodeManager HDFS requests Temp Ext4 NN NN NN NN NN NN Ext4 Isilon data node 11
12 Concerns with HDFS (The Hadoop Distributed File System) Difficult to separate compute from data storage concerns Three-way block replication for each 256MB data block (or 512MB block) Triples input data size at least - to achieve safety Re-balance of data when you add new data node processes Data must be ingested into HDFS from legacy systems (can be time consuming) Site-to-site replication not inherent NameNode process (which holds the central index of all files) can be sensitive to higher numbers of small files VMworld 2017 Content: Not for publication 12
13 Developers and Data Scientists Work on their code or on their data analysis model Don t need a multi-tenant cluster Don t care about job scheduling for other users Want to scale out to see the effect on their work Want to use the latest tools and newer versions (Python, R, Scala, ML kits) Experiment with different data models, code, algorithms, data sets Training the analysis model is separated from testing it interested in the time taken for each May not need the full Hadoop cluster set 13
14 New Architectures for Big Data
15 Key Trends in Big Data Infrastructure Decoupling of Compute and Storage Clusters Separate compute virtual machines from storage VMs Data is processed and scaled independently of compute Dynamic Scaling of compute nodes used for analysis from dozens to hundreds SPARK and other newer Big Data platforms can work with regular filesystems Newer platforms store and process data in memory New platforms can leverage Distributed Filesystems that can use local or shared storage Need for High Availability & Fault Tolerance for master components 15
16 Apache Spark Platform Capabilities Open-source cluster computing framework In Memory Data Processing Engine ETL, analytics, ML and graph processing Batch and streams processing Rich APIs for Scala, Python, Java, R, and SQL Distributed platform for complex multi-stage applications Reference: #VIRT1351BU CONFIDENTIAL 16
17 HDFS replacement needed for the next generation distributed file System What candidates present themselves? S3, Ceph, Gluster, etc. GlusterFS used in POC: Mature Solution Native GlusterFS filesystem for Linux Layers on top of any traditional storage Truly distributed and resilient distributed file system Supports many common client protocols 17
18 GlusterFS GlusterFS is a scale out distributed filesystem that can support thousands of clients File-system can run on DAS or Shared Storage Fault Tolerant Distributed File System. Provides multiprotocol support Native NFS CIFS HDFS S3 FTP 18
19 HDFS vs Ceph vs Gluster IOZONE Performance Comparison 19
20 SPARK with GlusterFS POC Architecture on Pure FC SAN Spark Master Spark Worker Spark Worker Gluster Node Spark Worker Spark Worker GlusterFS Gluster Node Spark Worker Gluster Node Spark Worker VMware vsphere VMware vsphere VMware vsphere VMware vsphere VMworld 2017 Spark Worker Spark Worker Content: Not for publication Pure M50 Storage on Fibre-Channel 20
21 SPARK with GlusterFS POC Architecture on Virtual SAN Spark Master Spark Worker Spark Worker VMware vsphere + VSAN Gluster Node Spark Worker VMware vsphere + VSAN Spark Worker GlusterFS Gluster Node Spark Worker Gluster Node VMware vsphere + VSAN Spark Worker VMware vsphere + VSAN Spark Worker Spark Worker Clustered VSAN Datastore 21
22 TPC-DS on SPARK on GlusterFS
23 TPC-DS with Spark-SQL and Apache SPARK IBM has helped integrate the TPC-DS Benchmark (v2), into the spark-sql-perf The 99 queries were generated using the TPC-DS query generator and are based on the 100- GB scale factor. The spark-sql-perf test kit can be used to evaluate and compare the infrastructure for its performance. We leveraged a subset of TPC-DS queries to evaluate our POC and Solution 23
24 Test Setup SPARK Nodes: 1 Master and 8 Slave Nodes with 16 vcpu and 128 GB each 3 Node GlusterFS cluster with 2 TB shared Filesystem mount across all SPARK nodes Storage: (Two Use Cases) 1. GlusterFS backed by Pure Storage LUNS (16 GBPS FC Fabric with Pure M50 Array) 2. GlusterFS backed by vsan (Western Digital NVMe Cache, High Capacity Flash for persistence) TPC-DS Data Sets 5 TB Queries Interactive TPC-DS Queries Set (q19, q42, q52, q55, q63, q68, q73 & q98) 24
25 Apache SPARK Web Console 25
26 SPARK Job Details 26
27 TPC-DS test results ( 5TB Data Set) Query Time Comparison between FC SAN and vsan 0 q19 q42 q52 q55 q63 q68 q73 q98 Pure VSAN 27
28 TPC-DS (vsan on Premises versus VMware Cloud on AWS) TPC-DS On Premises vs VMware Cloud on AWS 0 q19 q42 q52 q55 q63 q68 q73 q98 On-Prem VMware Cloud on AWS 28
29 Demo #VIRT1351BU CONFIDENTIAL 29
30 Section-Conclusion Modern Big Data platforms like SPARK are mostly memory resident GlusterFS provides a high performance distributed filesystem for SPARK and newer big data workloads GlusterFS supports a wide range of protocols that make it the ideal storage platform for data lakes Layering GlusterFS on top of shared storage or VSAN helps leverage all the vsphere platform features Dedicated HW with local storage is no longer required for modern big data applications. TPC-DS testing showed similar performance for SPARK-SQL on VSAN and FC. 30
31 Introduction to Machine Learning
32 32
33 What Is Machine Learning? Training Data (Big) Samples from History training New Sample Transaction Data testing Mathematical Model Mathematical Model Mathematical Model Classification or Prediction Machine Learning algorithms try to make predictions based on training data that is given to a mathematical model (e.g. a linear regression algorithm) Find the minimum the difference between the model s prediction and the already known outcomes (minimize the loss or objective function) 33
34 Example: Machine Learning Model for A Customer Applies for Credit Training Data (Big) A new application for credit Mathematical Model Mathematical Model Mathematical Model Training data contains many features that have each been given a numeric value (e.g. zip code = 99) Several models are used against the training data and the best one is chosen (minimal loss or error) One kind of outcome is a binary classification (a good credit application or bad) Classification or Prediction 34
35 Training Data Examples x i Acct Number Txn ID Txn Location Code Knowns Age Home Zip Code Balance Annual Salary Passed Valid Check Computed/Learned Model s Estimate as Valid Y N UK N Y Y Y 0 VMworld 2017 Content: Not for Error (Loss) publication Features or Feature Variables 35
36 Test Data Should Always Be Separated from Training Data Training Data Examples x i Test Data Acct Number Txn ID Txn Location Code Known Age Home Zip Code Balance Annual Salary Passed Valid Check Computed/Learned Model s Estimate as Valid Y N UK N Y Y Y 0 VMworld 2017 Content: Not for Error (Loss) publication Features or Feature Variables GOLDEN RULE : Don t TEST on your TRAINING DATA 36
37 Example: A Linear Classifier f (x i, W, b) = Wx i + b x: Example data W: weights b: bias Source: Stanford University class cs231n 37
38 Deployment Platform for Machine Learning Training Data (Big) Spark A new application for credit Spark Spark Mathematical Model Mathematical Model Mathematical Model Spark is the runtime platform for the models and ingestion of the training data Different Machine Learning algorithms available from MLlib library that comes with Spark Application and Data is distributed out to many nodes (virtual machines) Classification or Prediction 38
39 Introducing vsphere Scale-Out for Big Data and HPC Workloads New package that provides all the core features required for scale-out workloads at an attractive price point Features Packaging Hypervisor, vmotion, vshield Endpoint, Storage vmotion, Storage APIs, Distributed Switch, I/O Controls & SR- IOV, Host Profiles / Auto Deploy and more Sold in Packs of 8 CPU at a cost-effective price point Licensing EULA enforced for use w/ Big Data/HPC workloads only 39
40 Conclusions New architectures for big data are emerging beyond the existing documented ones Spark changes the profile of I/O and persistence for the newer applications This lends itself well to virtualization and separation of compute from data Traditional values in vsphere can be used in a big data context We would like to explore how these new architectural ideas will fit in your environment 40
41
42
43 BACKUP SLIDES NOT FOR PRESENTATION
44 Placeholder : Key Requirements for Big Data Architecture Subtitle Performance Scaling to dozens or hundreds of nodes (VMs) Robustness distributed file system, no one process is a single point of failure High Availability Fault Tolerance Capable of handling new workloads with new compute demands 44
45 Placeholder : Key Requirements for Big Data Architecture Can we use a distributed file system that is not HDFS? Use a lighter weight framework than full Hadoop e.g. Spark? Can we keep as much data in memory as possible and avoid I/O? Avoid spills Are shared file systems like VSAN useful? How to achieve the performance requirements without losing functionality? 45
46 vsan Optimization
47 Hardware Configuration All-Flash vsan (4) Node Dell R730XD (2) E5-2699V4 22-core 2.2GHz 1TB Memory (4) 10 Gb/s Ethernet connections PERC H730mini SDCard System Drive vsphere 6.5 Update 1 VMworld 2017 Content: Not for VSAN disk configuration (2) Disk groups per node (1) 1.6TB * Ultrastar SN100 cache drive (2) 3.84TB Optimus MAX capacity drive publication * 1TB=1,000GB, 1GB=1,000,000,000 bytes. Actual usable capacity less. 47
48 vsan Disk Group Configuration 48
49 vsan - Network Dual vsan VMKernel Adapters Port Group Virtual Switch Port Group VMworld 2017 These are not necessarily for redundancy (like an Air-Gap network with redundant physical interfaces routed to multiple VMKs) but for performance to pull from two physical interfaces at once. Content: Not for publication 49
50 vsan VMK Configuration 50
51 vsan Port Group Uplink Maps vds Contained 4 Uplinks 2 dedicated to normal operation 2 dedicated to vsan communication vds-comp01-private Active Uplink: dvuplink3 Standby Uplink: dvuplink4 vds-comp01-private2 Active Uplink: dvuplink4 Standby Uplink: dvuplink3 51
52 HCIBench Results Network 100% Read IOPs and Latency IOPs K 8K 32K 64K Block Size vsan Baseline Multiple vsan VMK 1500 MTU 10Gb Ethernet 10Gb Eth Multiple vsan VMK Baseline - Lat Multiple vsan VMK - Lat 1500 MTU - Lat 10Gb Ethernet - Lat 10Gb Eth Multiple vsan VMK - Lat MS 52
53 What Have We Seen so Far? We can use a different file system for big data to HDFS With the right storage, we can use the vmotion/drs/ha/ft features of vsphere VSAN can provide the storage underpinning big data (particularly for newer workloads) A number of different workloads were exercised on this new architecture Analytical queries, batch jobs and machine learning Testing is still in progress on all the above more to come 53
NEW ARCHITECTURES FOR APACHE SPARK TM AND BIG DATA WHITE PAPER NOVEMBER 2017
NEW ARCHITECTURES FOR APACHE SPARK TM AND BIG DATA WHITE PAPER NOVEMBER 2017 Contents Key Trends in Big Data... 4 Goal of the Study.... 4 Traditional Big Data Infrastructure in ware Virtualized Environments...
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
VIRT1445BU Extreme Performance: Fast Virtualized Hadoop and Spark on All-Flash Disks VMworld 2017 Dave Jaffe, Performance Engineering, VMware Justin Murray, Technical Marketing, VMware Content: Not for
More informationBig Data Performance on VMware Cloud on AWS
Big Data Performance on VMware Cloud on AWS Spark Machine Learning and IoT Analytics Performance On-premises and in the Cloud Performance Study - August 16, 2018 VMware, Inc. 3401 Hillview Avenue Palo
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
STO2591BU Amplifying VMware vsan Performance: A Deep Dive into Testing and Tuning (with Lessons Learned) by Western Digital Jonathan Flynn Virtualization Technologist, Western Digital Twitter: @levelrage
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
VIRT1983BU Making the Complicated Simple: Cycle Harvesting from the Virtual Desktop Infrastructure Estate for Financial Modeling and Simulation VMworld 2017 Content: Not for publication #VMworld #VIRT1983BU
More informationOverview. Prerequisites. Course Outline. Course Outline :: Apache Spark Development::
Title Duration : Apache Spark Development : 4 days Overview Spark is a fast and general cluster computing system for Big Data. It provides high-level APIs in Scala, Java, Python, and R, and an optimized
More informationAccelerating Digital Transformation with InterSystems IRIS and vsan
HCI2501BU Accelerating Digital Transformation with InterSystems IRIS and vsan Murray Oldfield, InterSystems Andreas Dieckow, InterSystems Christian Rauber, VMware #vmworld #HCI2501BU Disclaimer This presentation
More informationGain Insights From Unstructured Data Using Pivotal HD. Copyright 2013 EMC Corporation. All rights reserved.
Gain Insights From Unstructured Data Using Pivotal HD 1 Traditional Enterprise Analytics Process 2 The Fundamental Paradigm Shift Internet age and exploding data growth Enterprises leverage new data sources
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
SER2342BU What s New with vsphere? Mike Adams Sr. Director CPBU madams@vmware.com #VMworld #SER2342BU Disclaimer This presentation may contain product features that are currently under development. This
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
VIRT1052BE Extreme Performance Series: Monster VM Database Performance Todd Muirhead, VMware David Morse, VMware #VMworld #VIRT1052BE Disclaimer This presentation may contain product features that are
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
STO1515BU Extreme Performance Series: vsan Performance Troubleshooting Amitabha Banerjee and Suraj Kasi #VMworld #STO1515BU Disclaimer This presentation may contain product features that are currently
More informationBest Practices for Deploying Hadoop Workloads on HCI Powered by vsan
Best Practices for Deploying Hadoop Workloads on HCI Powered by vsan Chen Wei, ware, Inc. Paudie ORiordan, ware, Inc. #vmworld HCI2038BU #HCI2038BU Disclaimer This presentation may contain product features
More informationHadoop, Yarn and Beyond
Hadoop, Yarn and Beyond 1 B. R A M A M U R T H Y Overview We learned about Hadoop1.x or the core. Just like Java evolved, Java core, Java 1.X, Java 2.. So on, software and systems evolve, naturally.. Lets
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
SER3052BU How VMware vsphere and NVIDIA s Accelerate Your Organization Raj Rao, NVIDIA GRID Product Management Ziv Kalmanovich, vsphere ESXi Product Management #VMworld #SER3052BU Disclaimer This presentation
More informationMicrosoft SQL Server in a VMware Environment on Dell PowerEdge R810 Servers and Dell EqualLogic Storage
Microsoft SQL Server in a VMware Environment on Dell PowerEdge R810 Servers and Dell EqualLogic Storage A Dell Technical White Paper Dell Database Engineering Solutions Anthony Fernandez April 2010 THIS
More informationDATA SCIENCE USING SPARK: AN INTRODUCTION
DATA SCIENCE USING SPARK: AN INTRODUCTION TOPICS COVERED Introduction to Spark Getting Started with Spark Programming in Spark Data Science with Spark What next? 2 DATA SCIENCE PROCESS Exploratory Data
More informationThe Old School Cloud Is No More: Running Your Microsoft Applications on AWS
The Old School Cloud Is No More: Running Your Microsoft Applications on AWS Deji Akomolafe, VMware, Inc. #vmworld VAP2874BU #VAP2874BU Disclaimer This presentation may contain product features or functionality
More informationVMware Virtual SAN. Technical Walkthrough. Massimiliano Moschini Brand Specialist VCI - vexpert VMware Inc. All rights reserved.
VMware Virtual SAN Technical Walkthrough Massimiliano Moschini Brand Specialist VCI - vexpert 2014 VMware Inc. All rights reserved. VMware Storage Innovations VI 3.x VMFS Snapshots Storage vmotion NAS
More informationCloud Computing & Visualization
Cloud Computing & Visualization Workflows Distributed Computation with Spark Data Warehousing with Redshift Visualization with Tableau #FIUSCIS School of Computing & Information Sciences, Florida International
More informationHadoop 2.x Core: YARN, Tez, and Spark. Hortonworks Inc All Rights Reserved
Hadoop 2.x Core: YARN, Tez, and Spark YARN Hadoop Machine Types top-of-rack switches core switch client machines have client-side software used to access a cluster to process data master nodes run Hadoop
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
STO1206BU Interpreting performance metrics in your vsan environment Pete Koehler (@vmpete) Bradley Mott #VMworld #STO1206BU Disclaimer This presentation may contain product features that are currently
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
STO1500BU A Real-World Demonstration of Assessing and Sizing a Customer Environment for vsan VMworld 2017 Aparna Somaiah Product Manager Ankur Huralikoppi Sr. MTS Content: Not for publication #VMworld
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
SER2734BU Extreme Performance Series: Byte-Addressable Nonvolatile Memory in vsphere VMworld 2017 Content: Not for publication Qasim Ali and Praveen Yedlapalli #VMworld #SER2734BU Disclaimer This presentation
More informationStorage Strategies for vsphere 5.5 users
Storage Strategies for vsphere 5.5 users Silverton Consulting, Inc. StorInt Briefing 2 Introduction VMware vsphere is the number one hypervisor solution in the world with more than 500,000 customers worldwide.
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
STO1960BU vsphere Encryption for Virtual Machines and vsan Encryption Deep Dive VMworld 2017 Content: Not for publication Jase McCarty & Mike Foley #VMworld #STO1960 Disclaimer This presentation may contain
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
LHC2384BU VMware Cloud on AWS A Technical Deep Dive Ray Budavari @rbudavari Frank Denneman - @frankdenneman #VMworld #LHC2384BU Disclaimer This presentation may contain product features that are currently
More informationA Practitioner s Guide to Migrating Workloads to VMware Cloud on AWS
A Practitioner s Guide to Migrating Workloads to VMware Cloud on AWS Adam Osterholt, VMware, Inc. Paul Gifford, VMware, Inc. #vmworld HYP1496BU #HYP1496BU Disclaimer This presentation may contain product
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
STO1193BU A Closer Look at vsan Networking Design and Configuration Considerations Cormac Hogan Andreas Scherr VMworld 2017 Content: Not for publication #VMworld #STO1193BU Disclaimer This presentation
More informationRunning VMware vsan Witness Appliance in VMware vcloudair First Published On: April 26, 2017 Last Updated On: April 26, 2017
Running VMware vsan Witness Appliance in VMware vcloudair First Published On: April 26, 2017 Last Updated On: April 26, 2017 1 Table of Contents 1. Executive Summary 1.1.Business Case 1.2.Solution Overview
More informationAchieving Horizontal Scalability. Alain Houf Sales Engineer
Achieving Horizontal Scalability Alain Houf Sales Engineer Scale Matters InterSystems IRIS Database Platform lets you: Scale up and scale out Scale users and scale data Mix and match a variety of approaches
More informationWHITE PAPER SEPTEMBER VMWARE vsphere AND vsphere WITH OPERATIONS MANAGEMENT. Licensing, Pricing and Packaging
WHITE PAPER SEPTEMBER 2017 VMWARE vsphere AND vsphere WITH OPERATIONS MANAGEMENT Licensing, Pricing and Packaging Table of Contents Executive Summary 3 VMware vsphere with Operations Management Overview
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
SER1906BU VMware and Chef - Leveraging the vsphere API Together #VMworld #SER1906BU Disclaimer This presentation may contain product features that are currently under development. This overview of new
More informationData Processing at the Speed of 100 Gbps using Apache Crail. Patrick Stuedi IBM Research
Data Processing at the Speed of 100 Gbps using Apache Crail Patrick Stuedi IBM Research The CRAIL Project: Overview Data Processing Framework (e.g., Spark, TensorFlow, λ Compute) Spark-IO Albis Pocket
More informationBUSINESS DATA LAKE FADI FAKHOURI, SR. SYSTEMS ENGINEER, ISILON SPECIALIST. Copyright 2016 EMC Corporation. All rights reserved.
BUSINESS DATA LAKE FADI FAKHOURI, SR. SYSTEMS ENGINEER, ISILON SPECIALIST 1 UNSTRUCTURED DATA GROWTH 75% 78% 80% 2015 71 EB 2016 106 EB 2017 133 EB Total Capacity Shipped, Worldwide % of Unstructured Data
More informationVMware vsphere Customized Corporate Agenda
VMware vsphere Customized Corporate Agenda It's not just VMware Install, Manage, Configure (Usual VCP Course). For working professionals, just VCP is not enough, below is the custom agenda. At the outset,
More informationEmerging Technologies for HPC Storage
Emerging Technologies for HPC Storage Dr. Wolfgang Mertz CTO EMEA Unstructured Data Solutions June 2018 The very definition of HPC is expanding Blazing Fast Speed Accessibility and flexibility 2 Traditional
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
LHC1882BE Service Overview for VMware Cloud on AWS Marc Umeno #VMworld #LHC1882BE Disclaimer This presentation may contain product features that are currently under development. This overview of new technology
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
LHC2103BU NSX and VMware Cloud on AWS: Deep Dive Ray Budavari, Senior Staff Technical Product Manager NSX @rbudavari #VMworld #LHC2103BU Disclaimer This presentation may contain product features that are
More information2014 VMware Inc. All rights reserved.
2014 VMware Inc. All rights reserved. Agenda Virtual SAN 1 Why VSAN Software Defined Storage 2 Introducing Virtual SAN 3 Hardware Requirements 4 DEMO 5 Questions 2 The Software-Defined Data Center Expand
More informationDatabase Level 100. Rohit Rahi November Copyright 2018, Oracle and/or its affiliates. All rights reserved.
Database Level 100 Rohit Rahi November 2018 1 Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated
More informationIBM Spectrum NAS, IBM Spectrum Scale and IBM Cloud Object Storage
IBM Spectrum NAS, IBM Spectrum Scale and IBM Cloud Object Storage Silverton Consulting, Inc. StorInt Briefing 2017 SILVERTON CONSULTING, INC. ALL RIGHTS RESERVED Page 2 Introduction Unstructured data has
More informationApache Spark is a fast and general-purpose engine for large-scale data processing Spark aims at achieving the following goals in the Big data context
1 Apache Spark is a fast and general-purpose engine for large-scale data processing Spark aims at achieving the following goals in the Big data context Generality: diverse workloads, operators, job sizes
More informationPublic Cloud Leverage For IT/Business Alignment Business Goals Agility to speed time to market, adapt to market demands Elasticity to meet demand whil
LHC2386BU True Costs Savings Modeling and Costing A Migration to VMware Cloud on AWS Chris Grossmeier chrisg@cloudphysics.com John Blumenthal john@cloudphysics.com #VMworld Public Cloud Leverage For IT/Business
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
CNA1612BU Deploying real-world workloads on Kubernetes and Pivotal Cloud Foundry VMworld 2017 Fred Melo, Director of Technology, Pivotal Merlin Glynn, Sr. Technical Product Manager, VMware Content: Not
More informationNext Generation Storage for The Software-Defned World
` Next Generation Storage for The Software-Defned World John Hofer Solution Architect Red Hat, Inc. BUSINESS PAINS DEMAND NEW MODELS CLOUD ARCHITECTURES PROPRIETARY/TRADITIONAL ARCHITECTURES High up-front
More informationHedvig as backup target for Veeam
Hedvig as backup target for Veeam Solution Whitepaper Version 1.0 April 2018 Table of contents Executive overview... 3 Introduction... 3 Solution components... 4 Hedvig... 4 Hedvig Virtual Disk (vdisk)...
More informationIBM Data Science Experience White paper. SparkR. Transforming R into a tool for big data analytics
IBM Data Science Experience White paper R Transforming R into a tool for big data analytics 2 R Executive summary This white paper introduces R, a package for the R statistical programming language that
More informationIntroducing SUSE Enterprise Storage 5
Introducing SUSE Enterprise Storage 5 1 SUSE Enterprise Storage 5 SUSE Enterprise Storage 5 is the ideal solution for Compliance, Archive, Backup and Large Data. Customers can simplify and scale the storage
More informationEvaluation Report: HP StoreFabric SN1000E 16Gb Fibre Channel HBA
Evaluation Report: HP StoreFabric SN1000E 16Gb Fibre Channel HBA Evaluation report prepared under contract with HP Executive Summary The computing industry is experiencing an increasing demand for storage
More informationAccelerate Big Data Insights
Accelerate Big Data Insights Executive Summary An abundance of information isn t always helpful when time is of the essence. In the world of big data, the ability to accelerate time-to-insight can not
More informationModern Data Warehouse The New Approach to Azure BI
Modern Data Warehouse The New Approach to Azure BI History On-Premise SQL Server Big Data Solutions Technical Barriers Modern Analytics Platform On-Premise SQL Server Big Data Solutions Modern Analytics
More informationHewlett Packard Enterprise HPE GEN10 PERSISTENT MEMORY PERFORMANCE THROUGH PERSISTENCE
Hewlett Packard Enterprise HPE GEN10 PERSISTENT MEMORY PERFORMANCE THROUGH PERSISTENCE Digital transformation is taking place in businesses of all sizes Big Data and Analytics Mobility Internet of Things
More informationApache Hadoop 3. Balazs Gaspar Sales Engineer CEE & CIS Cloudera, Inc. All rights reserved.
Apache Hadoop 3 Balazs Gaspar Sales Engineer CEE & CIS balazs@cloudera.com 1 We believe data can make what is impossible today, possible tomorrow 2 We empower people to transform complex data into clear
More informationDell EMC Surveillance for IndigoVision Body-Worn Cameras
Dell EMC Surveillance for IndigoVision Body-Worn Cameras Functional Validation Guide H14821 REV 1.1 Copyright 2016 Dell Inc. or its subsidiaries. All rights reserved. Published February 2016 Dell believes
More informationFlash Storage Complementing a Data Lake for Real-Time Insight
Flash Storage Complementing a Data Lake for Real-Time Insight Dr. Sanhita Sarkar Global Director, Analytics Software Development August 7, 2018 Agenda 1 2 3 4 5 Delivering insight along the entire spectrum
More informationPerformance Testing December 16, 2017
December 16, 2017 1 1. vsan Performance Testing 1.1.Performance Testing Overview Table of Contents 2 1. vsan Performance Testing Performance Testing 3 1.1 Performance Testing Overview Performance Testing
More information#techsummitch
www.thomasmaurer.ch #techsummitch Justin Incarnato Justin Incarnato Microsoft Principal PM - Azure Stack Hyper-scale Hybrid Power of Azure in your datacenter Azure Stack Enterprise-proven On-premises
More informationCloud Computing and Hadoop Distributed File System. UCSB CS170, Spring 2018
Cloud Computing and Hadoop Distributed File System UCSB CS70, Spring 08 Cluster Computing Motivations Large-scale data processing on clusters Scan 000 TB on node @ 00 MB/s = days Scan on 000-node cluster
More informationAlbis: High-Performance File Format for Big Data Systems
Albis: High-Performance File Format for Big Data Systems Animesh Trivedi, Patrick Stuedi, Jonas Pfefferle, Adrian Schuepbach, Bernard Metzler, IBM Research, Zurich 2018 USENIX Annual Technical Conference
More informationVmware 3V VMware Certified Advanced Professional Data Center Virtualization Design.
Vmware 3V0-624 VMware Certified Advanced Professional 6.5 - Data Center Virtualization Design http://killexams.com/pass4sure/exam-detail/3v0-624 DEMO Find some pages taken from full version Killexams 3V0-624
More informationIBM Cloud for VMware Solutions
Introduction 2 IBM Cloud IBM Cloud for VMware Solutions Zeb Ahmed Senior Offering Manager VMware on IBM Cloud Mehran Hadipour Director Business Development - Zerto Internal Use Only Do not distribute 3
More informationvsan Mixed Workloads First Published On: Last Updated On:
First Published On: 03-05-2018 Last Updated On: 03-05-2018 1 1. Mixed Workloads on HCI 1.1.Solution Overview Table of Contents 2 1. Mixed Workloads on HCI 3 1.1 Solution Overview Eliminate the Complexity
More informationFlashGrid Software Enables Converged and Hyper-Converged Appliances for Oracle* RAC
white paper FlashGrid Software Intel SSD DC P3700/P3600/P3500 Topic: Hyper-converged Database/Storage FlashGrid Software Enables Converged and Hyper-Converged Appliances for Oracle* RAC Abstract FlashGrid
More informationLecture 11 Hadoop & Spark
Lecture 11 Hadoop & Spark Dr. Wilson Rivera ICOM 6025: High Performance Computing Electrical and Computer Engineering Department University of Puerto Rico Outline Distributed File Systems Hadoop Ecosystem
More informationEMC Business Continuity for Microsoft Applications
EMC Business Continuity for Microsoft Applications Enabled by EMC Celerra, EMC MirrorView/A, EMC Celerra Replicator, VMware Site Recovery Manager, and VMware vsphere 4 Copyright 2009 EMC Corporation. All
More informationIntroduction to Virtualization. From NDG In partnership with VMware IT Academy
Introduction to Virtualization From NDG In partnership with VMware IT Academy www.vmware.com/go/academy Why learn virtualization? Modern computing is more efficient due to virtualization Virtualization
More informationMapR Enterprise Hadoop
2014 MapR Technologies 2014 MapR Technologies 1 MapR Enterprise Hadoop Top Ranked Cloud Leaders 500+ Customers 2014 MapR Technologies 2 Key MapR Advantage Partners Business Services APPLICATIONS & OS ANALYTICS
More informationVMware vsphere 6.5 Boot Camp
Course Name Format Course Books 5-day, 10 hour/day instructor led training 724 pg Study Guide fully annotated with slide notes 243 pg Lab Guide with detailed steps for completing all labs 145 pg Boot Camp
More informationProcessing of big data with Apache Spark
Processing of big data with Apache Spark JavaSkop 18 Aleksandar Donevski AGENDA What is Apache Spark? Spark vs Hadoop MapReduce Application Requirements Example Architecture Application Challenges 2 WHAT
More informationVMware Virtual SAN Technology
VMware Virtual SAN Technology Today s Agenda 1 Hyper-Converged Infrastructure Architecture & Vmware Virtual SAN Overview 2 Why VMware Hyper-Converged Software? 3 VMware Virtual SAN Advantage Today s Agenda
More informationStorMagic SvSAN 6.1. Product Announcement Webinar and Live Demonstration. Mark Christie Senior Systems Engineer
StorMagic SvSAN 6.1 Product Announcement Webinar and Live Demonstration Mark Christie Senior Systems Engineer Introducing StorMagic What do we do? StorMagic SvSAN eliminates the need for physical SANs
More informationSpark Over RDMA: Accelerate Big Data SC Asia 2018 Ido Shamay Mellanox Technologies
Spark Over RDMA: Accelerate Big Data SC Asia 2018 Ido Shamay 1 Apache Spark - Intro Spark within the Big Data ecosystem Data Sources Data Acquisition / ETL Data Storage Data Analysis / ML Serving 3 Apache
More informationDell EMC. VxRack System FLEX Architecture Overview
Dell EMC VxRack System FLEX Architecture Overview Document revision 1.6 October 2017 Revision history Date Document revision Description of changes October 2017 1.6 Editorial updates Updated Cisco Nexus
More informationThe vsphere 6.0 Advantages Over Hyper- V
The Advantages Over Hyper- V The most trusted and complete virtualization platform SDDC Competitive Marketing 2015 Q2 VMware.com/go/PartnerCompete 2015 VMware Inc. All rights reserved. v3b The Most Trusted
More informationPERFORMANCE CHARACTERIZATION OF MICROSOFT SQL SERVER USING VMWARE CLOUD ON AWS PERFORMANCE STUDY JULY 2018
PERFORMANCE CHARACTERIZATION OF MICROSOFT SQL SERVER USING VMWARE CLOUD ON AWS PERFORMANCE STUDY JULY 2018 Table of Contents Executive Summary...3 Introduction...3 Test Environment... 4 Infrastructure
More informationNAS for Server Virtualization Dennis Chapman Senior Technical Director NetApp
NAS for Server Virtualization Dennis Chapman Senior Technical Director NetApp Agenda The Landscape has Changed New Customer Requirements The Market has Begun to Move Comparing Performance Results Storage
More informationCertified Big Data Hadoop and Spark Scala Course Curriculum
Certified Big Data Hadoop and Spark Scala Course Curriculum The Certified Big Data Hadoop and Spark Scala course by DataFlair is a perfect blend of indepth theoretical knowledge and strong practical skills
More informationVirtualization of the MS Exchange Server Environment
MS Exchange Server Acceleration Maximizing Users in a Virtualized Environment with Flash-Powered Consolidation Allon Cohen, PhD OCZ Technology Group Introduction Microsoft (MS) Exchange Server is one of
More informationDell Technologies IoT Solution Surveillance with Genetec Security Center
Dell Technologies IoT Solution Surveillance with Genetec Security Center Surveillance December 2018 H17435 Configuration Best Practices Abstract This guide is intended for internal Dell Technologies personnel
More informationEMC XTREMCACHE ACCELERATES VIRTUALIZED ORACLE
White Paper EMC XTREMCACHE ACCELERATES VIRTUALIZED ORACLE EMC XtremSF, EMC XtremCache, EMC Symmetrix VMAX and Symmetrix VMAX 10K, XtremSF and XtremCache dramatically improve Oracle performance Symmetrix
More informationRethink Storage: The Next Generation Of Scale- Out NAS
Rethink Storage: The Next Generation Of Scale- Out NAS Norbert Funke EMEA Product Marketing Manager, EMC Isilon 1 Agenda EMC Isilon Momentum What Is The Difference? EMC Isilon Versus Traditional Architectures
More informationVMware vsphere with ESX 4.1 and vcenter 4.1
QWERTYUIOP{ Overview VMware vsphere with ESX 4.1 and vcenter 4.1 This powerful 5-day class is an intense introduction to virtualization using VMware s vsphere 4.1 including VMware ESX 4.1 and vcenter.
More informationEMC Virtual Infrastructure for Microsoft Applications Data Center Solution
EMC Virtual Infrastructure for Microsoft Applications Data Center Solution Enabled by EMC Symmetrix V-Max and Reference Architecture EMC Global Solutions Copyright and Trademark Information Copyright 2009
More informationIOmark-VM. VMware VSAN Intel Servers + VMware VSAN Storage SW Test Report: VM-HC a Test Report Date: 16, August
IOmark-VM VMware VSAN Intel Servers + VMware VSAN Storage SW Test Report: VM-HC-160816-a Test Report Date: 16, August 2016 Copyright 2010-2016 Evaluator Group, Inc. All rights reserved. IOmark-VM, IOmark-VDI,
More informationvsan Remote Office Deployment January 09, 2018
January 09, 2018 1 1. vsan Remote Office Deployment 1.1.Solution Overview Table of Contents 2 1. vsan Remote Office Deployment 3 1.1 Solution Overview Native vsphere Storage for Remote and Branch Offices
More informationMOHA: Many-Task Computing Framework on Hadoop
Apache: Big Data North America 2017 @ Miami MOHA: Many-Task Computing Framework on Hadoop Soonwook Hwang Korea Institute of Science and Technology Information May 18, 2017 Table of Contents Introduction
More informationCIT 668: System Architecture. Amazon Web Services
CIT 668: System Architecture Amazon Web Services Topics 1. AWS Global Infrastructure 2. Foundation Services 1. Compute 2. Storage 3. Database 4. Network 3. AWS Economics Amazon Services Architecture Regions
More informationIBM Emulex 16Gb Fibre Channel HBA Evaluation
IBM Emulex 16Gb Fibre Channel HBA Evaluation Evaluation report prepared under contract with Emulex Executive Summary The computing industry is experiencing an increasing demand for storage performance
More informationEMC VSPEX END-USER COMPUTING
IMPLEMENTATION GUIDE EMC VSPEX END-USER COMPUTING VMware Horizon View 6.0 and VMware vsphere for up to 500 Virtual Desktops Enabled by EMC VNXe3200 and EMC Data Protection EMC VSPEX Abstract This describes
More informationWhat's New in vsphere?
INF1502 What's New in vsphere? Michael Adams, Vmware madams@vmware.com Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents
More information... IBM Advanced Technical Skills IBM Oracle International Competency Center September 2013
Performance benefits of IBM Power Systems and IBM FlashSystem for JD Edwards EnterpriseOne IBM Power 780 server with AIX and IBM FlashSystem 820 flash storage improves batch performance in a client proof
More information@joerg_schad Nightmares of a Container Orchestration System
@joerg_schad Nightmares of a Container Orchestration System 2017 Mesosphere, Inc. All Rights Reserved. 1 Jörg Schad Distributed Systems Engineer @joerg_schad Jan Repnak Support Engineer/ Solution Architect
More informationHPE Synergy HPE SimpliVity 380
HPE Synergy HPE SimpliVity 0 Pascal.Moens@hpe.com, Solutions Architect Technical Partner Lead February 0 HPE Synergy Composable infrastructure at HPE CPU Memory Local Storage LAN I/O SAN I/O Power Cooling
More informationATTACHMENT A SCOPE OF WORK IMPLEMENTATION SERVICES. Cisco Server and NetApp Storage Implementation
ATTACHMENT A SCOPE OF WORK IMPLEMENTATION SERVICES I. Executive Summary Superior Court of California, County of Orange (Court) is in the process of conducting a large enterprise hardware refresh. This
More informationDell EMC. VxBlock Systems for VMware NSX 6.3 Architecture Overview
Dell EMC VxBlock Systems for VMware NSX 6.3 Architecture Overview Document revision 1.1 March 2018 Revision history Date Document revision Description of changes March 2018 1.1 Updated the graphic in Logical
More informationReport on The Infrastructure for Implementing the Mobile Technologies for Data Collection in Egypt
Report on The Infrastructure for Implementing the Mobile Technologies for Data Collection in Egypt Date: 10 Sep, 2017 Draft v 4.0 Table of Contents 1. Introduction... 3 2. Infrastructure Reference Architecture...
More informationPerformance & Scalability Testing in Virtual Environment Hemant Gaidhani, Senior Technical Marketing Manager, VMware
Performance & Scalability Testing in Virtual Environment Hemant Gaidhani, Senior Technical Marketing Manager, VMware 2010 VMware Inc. All rights reserved About the Speaker Hemant Gaidhani Senior Technical
More informationVMware Cloud Provider Platform
VMware Cloud Provider Platform Enabling your journey to multicloud Winston Benjamin Systems Engineer Cloud Provider Program Disclaimer This presentation may contain product features that are currently
More informationThe Hadoop Ecosystem. EECS 4415 Big Data Systems. Tilemachos Pechlivanoglou
The Hadoop Ecosystem EECS 4415 Big Data Systems Tilemachos Pechlivanoglou tipech@eecs.yorku.ca A lot of tools designed to work with Hadoop 2 HDFS, MapReduce Hadoop Distributed File System Core Hadoop component
More informationBlended Learning Outline: Developer Training for Apache Spark and Hadoop (180404a)
Blended Learning Outline: Developer Training for Apache Spark and Hadoop (180404a) Cloudera s Developer Training for Apache Spark and Hadoop delivers the key concepts and expertise need to develop high-performance
More information