The BioHPC Nucleus Cluster & Future Developments

Size: px
Start display at page:

Download "The BioHPC Nucleus Cluster & Future Developments"

Transcription

1 1 The BioHPC Nucleus Cluster & Future Developments

2 Overview Today we ll talk about the BioHPC Nucleus HPC cluster with some technical details for those interested! How is it designed? What hardware does it use? How does this affect the work I need to run? Future Plans (2017 cluster upgrade and more!) 2

3 HPC Clusters HPC clusters consist of 3 major components Compute Nodes Powerful servers that run your jobs Some also contain GPU cards High-Speed Network Transfers data to/from compute nodes Carries communication for parallel code HPC Cluster High-Speed, High Capacity Storage Terabytes of storage for your research data 10s of GB per second bandwidth to feed nodes 3

4 Balance in Clusters Some clusters need much more storage than compute. Data intensive tasks (e.g. Next Generation Sequencing) Some clusters need very little storage, but a lot of compute. Compute intensive tasks (e.g. physical process modelling) Some clusters don t need very high performance networking Embarrassingly parallel tasks (no communication between tasks) Best solution depends on the workload of users Nucleus is a balanced, general purpose cluster Slight bias toward storage more storage than typical HPC system of its size 4

5 Compared to your PC Combined, the cluster is between 1,000 and 8,000x faster/larger than a typical PC Compute Nodes 8500 cores 45TB RAM (~2,000x Desktop) (~5,000x Desktop) High-Speed Network 5.5Tbps Throughput (~5,000x Desktop) High-Speed, High Capacity Storage >8PB Storage (~8000x Desktop) 90GB/s Throughput (~1000x Desktop) 4 Cores 8GB RAM 1TB HDD 5 openclipart.org -

6 Compute Nodes Nucleus has 196 Compute Nodes Based on standard servers, used by businesses: Lots of CPU cores: 32, 48 or 56 logical per server each physical core has 2 logical cores Lots of RAM: 128, 256, or 384GB per server Differences from business servers: Very little local storage Keep things on central storage systems High Speed Infiniband Network Much faster than normal business networking Possible to buy much individual faster machines But this is the sweet spot for price-performance of a cluster of machines. 6

7 Compute Nodes - Types We add nodes often, and buy newer, faster machines when they become available: Logical Cores 24 * 128GB Nodes Oldest Xeon E5 32 Cores 78 * 256GB Nodes Most nodes Xeon E5 v3 48 Cores 48 * 256GBv1 Nodes Fastest CPU nodes Xeon E5 v4 56 Cores 2 * 384GB Nodes Largest RAM Xeon E5 /v2 32/40 Cores Newer nodes have more cores Can be much faster if your work can use the extra cores Also have newer numerical features can speed up linear algebra a lot 7

8 How Does this Affect Me Cores and RAM? Most jobs that users run don t use compute nodes fully. 56 cores is a lot to fill up. Might be slower to split task into 56 parts due to overhead. Herzeel et. al Performance Analysis of BWA Alignment ExaScience Life Lab Combine smaller jobs run programs in parallel on fewer nodes Watch out for RAM usage 256GB / 56 cores is 4.5GB per core. You might need to run fewer than 56 tasks 8

9 How Does this Affect Me? CPU types Older nodes are often less busy shorter waits. If your code is not specifically optimized for new CPUs the older nodes are often not much slower. E.g. newest 256GBv1 node is often only 25% faster than oldest 128GB node on code not specifically optimized for many cores, and CPU numerical improvements. Running a test ChipSeq workflow (minimal 385MB test dataset) 32 Cores AVX Xeon E5 (v1) 128GB - 255s 56 Cores AVX2 Xeon E5 v4 256GBv1-194s 75% more cores 24% speedup astrocyte_example_chipseq workflow, run on a single node 9

10 How Does this Affect Me? CPU types Optimized numerical code will benefit from new CPUs but you must compile it for specific machines To compile for specific machine (fastest possible binaries) use: GNU gcc: -march=native Intel icc: -xhost 4096x4096 Element Matrix multiplication benchmark: 32 Cores AVX Xeon E5 (v1) 128GB - 507ms 56 Cores AVX2 Xeon E5 v4 256GBv1-168ms 75% more cores 3x speedup MKL sgemm mean time across 1000 replicate computations Intel 2016 compiler xhost O3 options for machine specific optimization 10

11 New Nodes for Low Memory Tasks Coming Soon Approx. 300 nodes will soon be added to the cluster (from TACC Stampede) 32GB RAM, 32 logical cores, similar to existing 128GB nodes Ideal for smaller RAM, interactive jobs. Will improve immediate availability of sessions. 11

12 GPU Nodes Nucleus has 20 GPU Compute Nodes Single or Multiple GPUs GPU NVIDIA Tesla K20 or K40 GPUv1 Dual NVIDIA Tesla P100 Differences vs Consumer GPUs Double Precision Arithmetic Performance Can be important for high accuracy work Reliability and Stability On well-suited tasks, 2x P100 GPUs can be 20x faster than using 56 CPU cores 12

13 Relion & Tensorflow Benchmarking New Dual P100 nodes are much faster than K40 nodes for GPU compute intensive software Relion CryoEM Classification Dual P100 approx. >6x faster than single K40 Speed-up on small benchmark limited by CPU initialization step TensorFlow AlexNet Benchmark Dual P100 approx. 4.3x faster than single K40 If you are using heavy GPU compute, the new GPUv1 nodes should be preferred Make sure your application can use, and is set to use 2 GPU cards! 13 GFDL,

14 K20 & K40 GPU Nodes Still Very Useful! Older K20 and K40 nodes are still ideal for: 3D Visualization very good 3D rendering performance Programs with limited GPU support (only 1 GPU, not much code GPU optimized) Please use them when they are appropriate, so P100 nodes are available for heavy computation 14

15 High Speed Network - Infiniband We use a normal Ethernet network to manage the nodes Just like the network connected to your desktop 1Gbps >125us latency for messages Your Jobs on Nucleus use the high-speed Infiniband network. 56Gbps connection per node 2:1 blocking each node guaranteed at least 28Gbps >0.7us latency for messages Supports RDMA - Remote Direct Memory Access Transfer data between RAM of nodes, without using CPU 15 David.Monniaux CC BY-SA 3.0,

16 How Does this Affect Me? - Infiniband Nodes have 2 network addresses x x - 1Gbps Ethernet - 56Gbps Infiniband Storage traffic, MPI traffic is setup to use the fast Infiniband network. Sometimes parallel programs (non-mpi) try to use the first network interface (1GbE) Must tell them to use Infiniband or things will be slow! 16

17 Storage Systems We use 2 main high-performance storage systems, plus others for lower-speed tasks They use large hard drives to give a lot of capacity per $ for your data A single hard drive in your desktop/laptop is slow Lots of hard drives (100s) working together can be very quick! 17

18 Project Storage System DDN SFA12K ExaScaler Lustre System 420 6TB drives in 40 disk pools 4 IO Servers, 2 Metadata Servers Redundancy in pools, gives 1.7PB usable space Each pool provides up to 1GB/s throughput Total Max throughput ~30GB/s Connected to cluster Infiniband Network 18

19 Work Storage System IBM Elastic Storage Server GL6 (SpectrumScale/GPFS) 712 8TB drives, and 4 IO Servers Redundancy in pools, gives 3.4 PB usable space Total Max throughput ~20GB/s* * Limited by network Located in Clements University Hospital Connected to cluster Infiniband network with 4 pairs of fiber under Harry Hines Boulevard 19

20 Data & Metadata Metadata is the information about a file or directory Name, dates, permissions, location of data Data is what s actually stored in your files On a single PC the data and metadata stay together on the disk On HPC storage we spread the data out over disk pools, so we can have fast parallel access to read and write Metadata has to be kept separately, and served to clients separately HPC filesystems have huge numbers of files = a lot of metadata to manage Difficult problem to do this when many clients could use same files 20

21 Data & Metadata How Does this Affect Me? HPC storage systems read and write data very quickly HPC storage systems handle metadata slowly Slow operations: Creating, deleting many files and folders Getting information about directories containing 1000s of files When writing code and workflows prefer using large files, instead of many small files. Use image stacks, instead of 1000s of individual TIFF files Use archives (tar, zip) to store small files you aren t working with Use node /local and /tmp space for large numbers of very small files 21

22 File Striping On HPC filesystems the data for a file is striped across the disk pools to achieve high speed. /work does this for you automatically /project does not stripe by default. Need to stripe very large files to get best speeds lfs setstripe -c 4 /project/department/myuser/bigfiles 1 Default Any file that doesn t fit the criteria below. Don t stripe small files! 2 Moderate size files 2-10GB that are read by 1-2 concurrent processes Moderate size files 2-10GB that are read by 3+ concurrent processes regularly 4 Large files 10GB+ that are read by 1-2 concurrent processes Large files 10GB+ that are read by 3+ concurrent processes regularly 8 Any very large files 200GB+ (to balance storage target usage) 22

23 Future Plans Nucleus is an excellent resource, built up over the past 4 years thanks to our contributing departments BioHPC is focused on an exciting future with new ways to use Nucleus, to advance your research 23

24 Update to Red Hat EL 7 A newer version of Linux (we currently use 6) Improved security, usability and compatibility with newer software Popular software that will work (again) Google Chrome Visual Studio Code Atom Editor 24

25 Full-featured Interactive Sessions More graphical tools: Modern desktop environment, web browsers, office suite, editors OpenGL (3D) support on all compute nodes, not just GPU nodes Use simple 3D software in webgui on any machine 25

26 Containers - Singularity Singularity will allow containers to be run on BioHPC Supports docker containers Support containers using GPUs Use software in a different environment e.g. Ubuntu Linux Direct access to 3045 tools from the biocontainers project Integrate with Astrocyte/Nextflow for reproducible workflows 26

27 New Nodes for Low Memory Tasks Approx. 300 nodes will soon be added to the cluster (from TACC Stampede) 32GB RAM, 32 logical cores, similar to existing 128GB nodes Ideal for smaller RAM, interactive jobs. Will improve immediate availability of sessions. 27

28 Xeon Phi (Knights Corner) Nodes from Stampede have Xeon Phi (Knights Corner) Coprocessors 61 Cores, 8GB RAM 3x faster than CPUs for numerical work Run standard code, unlike GPUs Can be used to speed up compute intensive, highly parallel code Will add function to portal to help launch code on the Xeon Phi MICs 28

29 Deep Learning with NVIDIA DIGITS Coming January On the new Nucleus cluster, you can launch an NVIDIA DIGITS session from the BioHPC Portal. Digits provides an easy-to-use, web-browser interface to deep learning tools. Easy to define models. Create and execute multiple runs, using GPU computation. Coming Soon! 29

30 Portal DIGITS, RStudio & Jupyter Coming 2018 Start & connect to dedicated Python, R, and DIGITS environments Directly from the BioHPC Portal 30

31 Distributed Computing, on Campus or in the Cloud - Planned Astrocyte will become a gateway to use resources beyond Nucleus 3 rd Party Cloud Nucleus Cluster BioHPC Cloud Workstations/Thin Clients ~500 cores 31

32 Workflow Designer Alpha version November Choose tools to create a workflow in your web browser Run analyses and share workflows with your lab, or wider 32

33 Workflow Visualization & Interactivity - Planned Downstream visualization of workflow results with interactive tools NGS Visualization apps Clinical / Microscopy 33

34 It s Your Cluster! Nucleus was built with your department contributions BioHPC is here to help you do your research What works well? What do you need? Let us know! Biohpc-help@utsouthwestern.edu Microsoft Teams: BioHPC General 34

Introduction to BioHPC New User Training

Introduction to BioHPC New User Training Introduction to BioHPC New User Training [web] [email] portal.biohpc.swmed.edu biohpc-help@utsouthwestern.edu 1 Updated for 2018-04-04 Overview Today we re going to cover: What is BioHPC? How do I access

More information

HPC Architectures. Types of resource currently in use

HPC Architectures. Types of resource currently in use HPC Architectures Types of resource currently in use Reusing this material This work is licensed under a Creative Commons Attribution- NonCommercial-ShareAlike 4.0 International License. http://creativecommons.org/licenses/by-nc-sa/4.0/deed.en_us

More information

LBRN - HPC systems : CCT, LSU

LBRN - HPC systems : CCT, LSU LBRN - HPC systems : CCT, LSU HPC systems @ CCT & LSU LSU HPC Philip SuperMike-II SuperMIC LONI HPC Eric Qeenbee2 CCT HPC Delta LSU HPC Philip 3 Compute 32 Compute Two 2.93 GHz Quad Core Nehalem Xeon 64-bit

More information

Isilon Performance. Name

Isilon Performance. Name 1 Isilon Performance Name 2 Agenda Architecture Overview Next Generation Hardware Performance Caching Performance Streaming Reads Performance Tuning OneFS Architecture Overview Copyright 2014 EMC Corporation.

More information

HPC and IT Issues Session Agenda. Deployment of Simulation (Trends and Issues Impacting IT) Mapping HPC to Performance (Scaling, Technology Advances)

HPC and IT Issues Session Agenda. Deployment of Simulation (Trends and Issues Impacting IT) Mapping HPC to Performance (Scaling, Technology Advances) HPC and IT Issues Session Agenda Deployment of Simulation (Trends and Issues Impacting IT) Discussion Mapping HPC to Performance (Scaling, Technology Advances) Discussion Optimizing IT for Remote Access

More information

Intel Enterprise Edition Lustre (IEEL-2.3) [DNE-1 enabled] on Dell MD Storage

Intel Enterprise Edition Lustre (IEEL-2.3) [DNE-1 enabled] on Dell MD Storage Intel Enterprise Edition Lustre (IEEL-2.3) [DNE-1 enabled] on Dell MD Storage Evaluation of Lustre File System software enhancements for improved Metadata performance Wojciech Turek, Paul Calleja,John

More information

Parallel File Systems. John White Lawrence Berkeley National Lab

Parallel File Systems. John White Lawrence Berkeley National Lab Parallel File Systems John White Lawrence Berkeley National Lab Topics Defining a File System Our Specific Case for File Systems Parallel File Systems A Survey of Current Parallel File Systems Implementation

More information

The Stampede is Coming Welcome to Stampede Introductory Training. Dan Stanzione Texas Advanced Computing Center

The Stampede is Coming Welcome to Stampede Introductory Training. Dan Stanzione Texas Advanced Computing Center The Stampede is Coming Welcome to Stampede Introductory Training Dan Stanzione Texas Advanced Computing Center dan@tacc.utexas.edu Thanks for Coming! Stampede is an exciting new system of incredible power.

More information

Introduction to BioHPC

Introduction to BioHPC Introduction to BioHPC New User Training [web] [email] portal.biohpc.swmed.edu biohpc-help@utsouthwestern.edu 1 Updated for 2017-01-04 Overview Today we re going to cover: What is BioHPC? How do I access

More information

Feedback on BeeGFS. A Parallel File System for High Performance Computing

Feedback on BeeGFS. A Parallel File System for High Performance Computing Feedback on BeeGFS A Parallel File System for High Performance Computing Philippe Dos Santos et Georges Raseev FR 2764 Fédération de Recherche LUmière MATière December 13 2016 LOGO CNRS LOGO IO December

More information

WELCOME! LIVE with ROBERT GREEN:

WELCOME! LIVE with ROBERT GREEN: WELCOME! LIVE with ROBERT GREEN: Select the Right Processor & RAM for CAD, Analysis & Visualization Workflows January 10, 2018 Robert Green CAD Management Expert Cadalyst Contributing Editor 113 TODAY

More information

Introduction to BioHPC

Introduction to BioHPC Introduction to BioHPC New User Training [web] [email] portal.biohpc.swmed.edu biohpc-help@utsouthwestern.edu 1 Updated for 2015-06-03 Overview Today we re going to cover: What is BioHPC? How do I access

More information

Visualization on BioHPC

Visualization on BioHPC Visualization on BioHPC [web] [email] portal.biohpc.swmed.edu biohpc-help@utsouthwestern.edu 1 Updated for 2015-09-16 Outline What is Visualization - Scientific Visualization - Work flow for Visualization

More information

Introduction to BioHPC

Introduction to BioHPC Introduction to BioHPC New User Training [web] [email] portal.biohpc.swmed.edu biohpc-help@utsouthwestern.edu 1 Updated for 2018-03-07 Overview Today we re going to cover: What is BioHPC? How do I access

More information

The Leading Parallel Cluster File System

The Leading Parallel Cluster File System The Leading Parallel Cluster File System www.thinkparq.com www.beegfs.io ABOUT BEEGFS What is BeeGFS BeeGFS (formerly FhGFS) is the leading parallel cluster file system, developed with a strong focus on

More information

System recommendations for version 17.1

System recommendations for version 17.1 System recommendations for version 17.1 This article contains information about recommended hardware resources and network environments for version 17.1 of Sage 300 Construction and Real Estate. NOTE:

More information

Lustre2.5 Performance Evaluation: Performance Improvements with Large I/O Patches, Metadata Improvements, and Metadata Scaling with DNE

Lustre2.5 Performance Evaluation: Performance Improvements with Large I/O Patches, Metadata Improvements, and Metadata Scaling with DNE Lustre2.5 Performance Evaluation: Performance Improvements with Large I/O Patches, Metadata Improvements, and Metadata Scaling with DNE Hitoshi Sato *1, Shuichi Ihara *2, Satoshi Matsuoka *1 *1 Tokyo Institute

More information

Using file systems at HC3

Using file systems at HC3 Using file systems at HC3 Roland Laifer STEINBUCH CENTRE FOR COMPUTING - SCC KIT University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association www.kit.edu Basic Lustre

More information

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or

More information

AN INTRODUCTION TO CLUSTER COMPUTING

AN INTRODUCTION TO CLUSTER COMPUTING CLUSTERS AND YOU AN INTRODUCTION TO CLUSTER COMPUTING Engineering IT BrownBag Series 29 October, 2015 Gianni Pezzarossi Linux Systems Administrator Mark Smylie Hart Research Technology Facilitator WHAT

More information

PART-I (B) (TECHNICAL SPECIFICATIONS & COMPLIANCE SHEET) Supply and installation of High Performance Computing System

PART-I (B) (TECHNICAL SPECIFICATIONS & COMPLIANCE SHEET) Supply and installation of High Performance Computing System INSTITUTE FOR PLASMA RESEARCH (An Autonomous Institute of Department of Atomic Energy, Government of India) Near Indira Bridge; Bhat; Gandhinagar-382428; India PART-I (B) (TECHNICAL SPECIFICATIONS & COMPLIANCE

More information

Chelsio Communications. Meeting Today s Datacenter Challenges. Produced by Tabor Custom Publishing in conjunction with: CUSTOM PUBLISHING

Chelsio Communications. Meeting Today s Datacenter Challenges. Produced by Tabor Custom Publishing in conjunction with: CUSTOM PUBLISHING Meeting Today s Datacenter Challenges Produced by Tabor Custom Publishing in conjunction with: 1 Introduction In this era of Big Data, today s HPC systems are faced with unprecedented growth in the complexity

More information

Monitoring and Trouble Shooting on BioHPC

Monitoring and Trouble Shooting on BioHPC Monitoring and Trouble Shooting on BioHPC [web] [email] portal.biohpc.swmed.edu biohpc-help@utsouthwestern.edu 1 Updated for 2017-03-15 Why Monitoring & Troubleshooting data code Monitoring jobs running

More information

LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance

LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance 11 th International LS-DYNA Users Conference Computing Technology LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance Gilad Shainer 1, Tong Liu 2, Jeff Layton

More information

WVU RESEARCH COMPUTING INTRODUCTION. Introduction to WVU s Research Computing Services

WVU RESEARCH COMPUTING INTRODUCTION. Introduction to WVU s Research Computing Services WVU RESEARCH COMPUTING INTRODUCTION Introduction to WVU s Research Computing Services WHO ARE WE? Division of Information Technology Services Funded through WVU Research Corporation Provide centralized

More information

IME (Infinite Memory Engine) Extreme Application Acceleration & Highly Efficient I/O Provisioning

IME (Infinite Memory Engine) Extreme Application Acceleration & Highly Efficient I/O Provisioning IME (Infinite Memory Engine) Extreme Application Acceleration & Highly Efficient I/O Provisioning September 22 nd 2015 Tommaso Cecchi 2 What is IME? This breakthrough, software defined storage application

More information

Intel Many Integrated Core (MIC) Matt Kelly & Ryan Rawlins

Intel Many Integrated Core (MIC) Matt Kelly & Ryan Rawlins Intel Many Integrated Core (MIC) Matt Kelly & Ryan Rawlins Outline History & Motivation Architecture Core architecture Network Topology Memory hierarchy Brief comparison to GPU & Tilera Programming Applications

More information

Dell EMC Ready Bundle for HPC Digital Manufacturing Dassault Systѐmes Simulia Abaqus Performance

Dell EMC Ready Bundle for HPC Digital Manufacturing Dassault Systѐmes Simulia Abaqus Performance Dell EMC Ready Bundle for HPC Digital Manufacturing Dassault Systѐmes Simulia Abaqus Performance This Dell EMC technical white paper discusses performance benchmarking results and analysis for Simulia

More information

Data Management. Parallel Filesystems. Dr David Henty HPC Training and Support

Data Management. Parallel Filesystems. Dr David Henty HPC Training and Support Data Management Dr David Henty HPC Training and Support d.henty@epcc.ed.ac.uk +44 131 650 5960 Overview Lecture will cover Why is IO difficult Why is parallel IO even worse Lustre GPFS Performance on ARCHER

More information

Workstations & Thin Clients

Workstations & Thin Clients 1 Workstations & Thin Clients Overview Why use a BioHPC computer? System Specs Network requirements OS Tour Running Code Locally Submitting Jobs to the Cluster Run Graphical Jobs on the Cluster Use Windows

More information

Parallel File Systems for HPC

Parallel File Systems for HPC Introduction to Scuola Internazionale Superiore di Studi Avanzati Trieste November 2008 Advanced School in High Performance and Grid Computing Outline 1 The Need for 2 The File System 3 Cluster & A typical

More information

R on BioHPC. Rstudio, Parallel R and BioconductoR. Updated for

R on BioHPC. Rstudio, Parallel R and BioconductoR. Updated for R on BioHPC Rstudio, Parallel R and BioconductoR 1 Updated for 2015-07-15 2 Today we ll be looking at Why R? The dominant statistics environment in academia Large number of packages to do a lot of different

More information

HPC File Systems and Storage. Irena Johnson University of Notre Dame Center for Research Computing

HPC File Systems and Storage. Irena Johnson University of Notre Dame Center for Research Computing HPC File Systems and Storage Irena Johnson University of Notre Dame Center for Research Computing HPC (High Performance Computing) Aggregating computer power for higher performance than that of a typical

More information

TACC s Stampede Project: Intel MIC for Simulation and Data-Intensive Computing

TACC s Stampede Project: Intel MIC for Simulation and Data-Intensive Computing TACC s Stampede Project: Intel MIC for Simulation and Data-Intensive Computing Jay Boisseau, Director April 17, 2012 TACC Vision & Strategy Provide the most powerful, capable computing technologies and

More information

Improved Solutions for I/O Provisioning and Application Acceleration

Improved Solutions for I/O Provisioning and Application Acceleration 1 Improved Solutions for I/O Provisioning and Application Acceleration August 11, 2015 Jeff Sisilli Sr. Director Product Marketing jsisilli@ddn.com 2 Why Burst Buffer? The Supercomputing Tug-of-War A supercomputer

More information

Using DDN IME for Harmonie

Using DDN IME for Harmonie Irish Centre for High-End Computing Using DDN IME for Harmonie Gilles Civario, Marco Grossi, Alastair McKinstry, Ruairi Short, Nix McDonnell April 2016 DDN IME: Infinite Memory Engine IME: Major Features

More information

Sharing High-Performance Devices Across Multiple Virtual Machines

Sharing High-Performance Devices Across Multiple Virtual Machines Sharing High-Performance Devices Across Multiple Virtual Machines Preamble What does sharing devices across multiple virtual machines in our title mean? How is it different from virtual networking / NSX,

More information

Introduction to BioHPC New User Training

Introduction to BioHPC New User Training Introduction to BioHPC New User Training [web] [email] portal.biohpc.swmed.edu biohpc-help@utsouthwestern.edu 1 Updated for 2019-02-06 Overview Today we re going to cover: What is BioHPC? How do I access

More information

DDN. DDN Updates. DataDirect Neworks Japan, Inc Nobu Hashizume. DDN Storage 2018 DDN Storage 1

DDN. DDN Updates. DataDirect Neworks Japan, Inc Nobu Hashizume. DDN Storage 2018 DDN Storage 1 1 DDN DDN Updates DataDirect Neworks Japan, Inc Nobu Hashizume DDN Storage 2018 DDN Storage 1 2 DDN A Broad Range of Technologies to Best Address Your Needs Your Use Cases Research Big Data Enterprise

More information

CIS 601 Graduate Seminar. Dr. Sunnie S. Chung Dhruv Patel ( ) Kalpesh Sharma ( )

CIS 601 Graduate Seminar. Dr. Sunnie S. Chung Dhruv Patel ( ) Kalpesh Sharma ( ) Guide: CIS 601 Graduate Seminar Presented By: Dr. Sunnie S. Chung Dhruv Patel (2652790) Kalpesh Sharma (2660576) Introduction Background Parallel Data Warehouse (PDW) Hive MongoDB Client-side Shared SQL

More information

Optimized Scientific Computing:

Optimized Scientific Computing: Optimized Scientific Computing: Coding Efficiently for Real Computing Architectures Noah Kurinsky SASS Talk, November 11 2015 Introduction Components of a CPU Architecture Design Choices Why Is This Relevant

More information

The Stampede is Coming: A New Petascale Resource for the Open Science Community

The Stampede is Coming: A New Petascale Resource for the Open Science Community The Stampede is Coming: A New Petascale Resource for the Open Science Community Jay Boisseau Texas Advanced Computing Center boisseau@tacc.utexas.edu Stampede: Solicitation US National Science Foundation

More information

Triton file systems - an introduction. slide 1 of 28

Triton file systems - an introduction. slide 1 of 28 Triton file systems - an introduction slide 1 of 28 File systems Motivation & basic concepts Storage locations Basic flow of IO Do's and Don'ts Exercises slide 2 of 28 File systems: Motivation Case #1:

More information

New User Seminar: Part 2 (best practices)

New User Seminar: Part 2 (best practices) New User Seminar: Part 2 (best practices) General Interest Seminar January 2015 Hugh Merz merz@sharcnet.ca Session Outline Submitting Jobs Minimizing queue waits Investigating jobs Checkpointing Efficiency

More information

IBM Emulex 16Gb Fibre Channel HBA Evaluation

IBM Emulex 16Gb Fibre Channel HBA Evaluation IBM Emulex 16Gb Fibre Channel HBA Evaluation Evaluation report prepared under contract with Emulex Executive Summary The computing industry is experiencing an increasing demand for storage performance

More information

Mission-Critical Lustre at Santos. Adam Fox, Lustre User Group 2016

Mission-Critical Lustre at Santos. Adam Fox, Lustre User Group 2016 Mission-Critical Lustre at Santos Adam Fox, Lustre User Group 2016 About Santos One of the leading oil and gas producers in APAC Founded in 1954 South Australia Northern Territory Oil Search Cooper Basin

More information

Small File I/O Performance in Lustre. Mikhail Pershin, Joe Gmitter Intel HPDD April 2018

Small File I/O Performance in Lustre. Mikhail Pershin, Joe Gmitter Intel HPDD April 2018 Small File I/O Performance in Lustre Mikhail Pershin, Joe Gmitter Intel HPDD April 2018 Overview Small File I/O Concerns Data on MDT (DoM) Feature Overview DoM Use Cases DoM Performance Results Small File

More information

Next-Generation NVMe-Native Parallel Filesystem for Accelerating HPC Workloads

Next-Generation NVMe-Native Parallel Filesystem for Accelerating HPC Workloads Next-Generation NVMe-Native Parallel Filesystem for Accelerating HPC Workloads Liran Zvibel CEO, Co-founder WekaIO @liranzvibel 1 WekaIO Matrix: Full-featured and Flexible Public or Private S3 Compatible

More information

Genius Quick Start Guide

Genius Quick Start Guide Genius Quick Start Guide Overview of the system Genius consists of a total of 116 nodes with 2 Skylake Xeon Gold 6140 processors. Each with 18 cores, at least 192GB of memory and 800 GB of local SSD disk.

More information

A Comparative Study of High Performance Computing on the Cloud. Lots of authors, including Xin Yuan Presentation by: Carlos Sanchez

A Comparative Study of High Performance Computing on the Cloud. Lots of authors, including Xin Yuan Presentation by: Carlos Sanchez A Comparative Study of High Performance Computing on the Cloud Lots of authors, including Xin Yuan Presentation by: Carlos Sanchez What is The Cloud? The cloud is just a bunch of computers connected over

More information

NVIDIA DGX SYSTEMS PURPOSE-BUILT FOR AI

NVIDIA DGX SYSTEMS PURPOSE-BUILT FOR AI NVIDIA DGX SYSTEMS PURPOSE-BUILT FOR AI Overview Unparalleled Value Product Portfolio Software Platform From Desk to Data Center to Cloud Summary AI researchers depend on computing performance to gain

More information

Intel Knights Landing Hardware

Intel Knights Landing Hardware Intel Knights Landing Hardware TACC KNL Tutorial IXPUG Annual Meeting 2016 PRESENTED BY: John Cazes Lars Koesterke 1 Intel s Xeon Phi Architecture Leverages x86 architecture Simpler x86 cores, higher compute

More information

November 7, DAN WILSON Global Operations Architecture, Concur. OpenStack Summit Hong Kong JOE ARNOLD

November 7, DAN WILSON Global Operations Architecture, Concur. OpenStack Summit Hong Kong JOE ARNOLD November 7, 2013 DAN WILSON Global Operations Architecture, Concur dan.wilson@concur.com @tweetdanwilson OpenStack Summit Hong Kong JOE ARNOLD CEO, SwiftStack joe@swiftstack.com @joearnold Introduction

More information

The rcuda technology: an inexpensive way to improve the performance of GPU-based clusters Federico Silla

The rcuda technology: an inexpensive way to improve the performance of GPU-based clusters Federico Silla The rcuda technology: an inexpensive way to improve the performance of -based clusters Federico Silla Technical University of Valencia Spain The scope of this talk Delft, April 2015 2/47 More flexible

More information

Benefits of 25, 40, and 50GbE Networks for Ceph and Hyper- Converged Infrastructure John F. Kim Mellanox Technologies

Benefits of 25, 40, and 50GbE Networks for Ceph and Hyper- Converged Infrastructure John F. Kim Mellanox Technologies Benefits of 25, 40, and 50GbE Networks for Ceph and Hyper- Converged Infrastructure John F. Kim Mellanox Technologies Storage Transitions Change Network Needs Software Defined Storage Flash Storage Storage

More information

MAHA. - Supercomputing System for Bioinformatics

MAHA. - Supercomputing System for Bioinformatics MAHA - Supercomputing System for Bioinformatics - 2013.01.29 Outline 1. MAHA HW 2. MAHA SW 3. MAHA Storage System 2 ETRI HPC R&D Area - Overview Research area Computing HW MAHA System HW - Rpeak : 0.3

More information

The Oracle Database Appliance I/O and Performance Architecture

The Oracle Database Appliance I/O and Performance Architecture Simple Reliable Affordable The Oracle Database Appliance I/O and Performance Architecture Tammy Bednar, Sr. Principal Product Manager, ODA 1 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

More information

SR-IOV Support for Virtualization on InfiniBand Clusters: Early Experience

SR-IOV Support for Virtualization on InfiniBand Clusters: Early Experience SR-IOV Support for Virtualization on InfiniBand Clusters: Early Experience Jithin Jose, Mingzhe Li, Xiaoyi Lu, Krishna Kandalla, Mark Arnold and Dhabaleswar K. (DK) Panda Network-Based Computing Laboratory

More information

Gateways to Discovery: Cyberinfrastructure for the Long Tail of Science

Gateways to Discovery: Cyberinfrastructure for the Long Tail of Science Gateways to Discovery: Cyberinfrastructure for the Long Tail of Science ECSS Symposium, 12/16/14 M. L. Norman, R. L. Moore, D. Baxter, G. Fox (Indiana U), A Majumdar, P Papadopoulos, W Pfeiffer, R. S.

More information

GPUs and Emerging Architectures

GPUs and Emerging Architectures GPUs and Emerging Architectures Mike Giles mike.giles@maths.ox.ac.uk Mathematical Institute, Oxford University e-infrastructure South Consortium Oxford e-research Centre Emerging Architectures p. 1 CPUs

More information

Deep Learning Frameworks with Spark and GPUs

Deep Learning Frameworks with Spark and GPUs Deep Learning Frameworks with Spark and GPUs Abstract Spark is a powerful, scalable, real-time data analytics engine that is fast becoming the de facto hub for data science and big data. However, in parallel,

More information

OpenPOWER Performance

OpenPOWER Performance OpenPOWER Performance Alex Mericas Chief Engineer, OpenPOWER Performance IBM Delivering the Linux ecosystem for Power SOLUTIONS OpenPOWER IBM SOFTWARE LINUX ECOSYSTEM OPEN SOURCE Solutions with full stack

More information

Parallel Visualization At TACC. Greg Abram

Parallel Visualization At TACC. Greg Abram Parallel Visualization At TACC Greg Abram Visualization Problems * With thanks to Sean Ahern for the metaphor Huge problems: Data cannot be moved off system where it is computed Large Visualization problems:

More information

Comet Virtualization Code & Design Sprint

Comet Virtualization Code & Design Sprint Comet Virtualization Code & Design Sprint SDSC September 23-24 Rick Wagner San Diego Supercomputer Center Meeting Goals Build personal connections between the IU and SDSC members of the Comet team working

More information

Resources Current and Future Systems. Timothy H. Kaiser, Ph.D.

Resources Current and Future Systems. Timothy H. Kaiser, Ph.D. Resources Current and Future Systems Timothy H. Kaiser, Ph.D. tkaiser@mines.edu 1 Most likely talk to be out of date History of Top 500 Issues with building bigger machines Current and near future academic

More information

Introduction to High-Performance Computing (HPC)

Introduction to High-Performance Computing (HPC) Introduction to High-Performance Computing (HPC) Computer components CPU : Central Processing Unit cores : individual processing units within a CPU Storage : Disk drives HDD : Hard Disk Drive SSD : Solid

More information

DDN. DDN Updates. Data DirectNeworks Japan, Inc Shuichi Ihara. DDN Storage 2017 DDN Storage

DDN. DDN Updates. Data DirectNeworks Japan, Inc Shuichi Ihara. DDN Storage 2017 DDN Storage DDN DDN Updates Data DirectNeworks Japan, Inc Shuichi Ihara DDN A Broad Range of Technologies to Best Address Your Needs Protection Security Data Distribution and Lifecycle Management Open Monitoring Your

More information

Sage 300 ERP. Compatibility Guide Version Revised: Oct 1, Version 6.0 Compatibility Guide i

Sage 300 ERP. Compatibility Guide Version Revised: Oct 1, Version 6.0 Compatibility Guide i Sage 300 ERP Compatibility Guide Version 2012 Revised: Oct 1, 2012 Version 6.0 Compatibility Guide i Overview 2 Sage ERP Accpac Contents Overview... 1 Version 2012 Compatibility... 2 All Environments...

More information

Isilon: Raising The Bar On Performance & Archive Use Cases. John Har Solutions Product Manager Unstructured Data Storage Team

Isilon: Raising The Bar On Performance & Archive Use Cases. John Har Solutions Product Manager Unstructured Data Storage Team Isilon: Raising The Bar On Performance & Archive Use Cases John Har Solutions Product Manager Unstructured Data Storage Team What we ll cover in this session Isilon Overview Streaming workflows High ops/s

More information

Our Workshop Environment

Our Workshop Environment Our Workshop Environment John Urbanic Parallel Computing Scientist Pittsburgh Supercomputing Center Copyright 2018 Our Environment Today Your laptops or workstations: only used for portal access Bridges

More information

Sun Lustre Storage System Simplifying and Accelerating Lustre Deployments

Sun Lustre Storage System Simplifying and Accelerating Lustre Deployments Sun Lustre Storage System Simplifying and Accelerating Lustre Deployments Torben Kling-Petersen, PhD Presenter s Name Principle Field Title andengineer Division HPC &Cloud LoB SunComputing Microsystems

More information

CONTAINERIZING JOBS ON THE ACCRE CLUSTER WITH SINGULARITY

CONTAINERIZING JOBS ON THE ACCRE CLUSTER WITH SINGULARITY CONTAINERIZING JOBS ON THE ACCRE CLUSTER WITH SINGULARITY VIRTUAL MACHINE (VM) Uses so&ware to emulate an en/re computer, including both hardware and so&ware. Host Computer Virtual Machine Host Resources:

More information

Fast packet processing in the cloud. Dániel Géhberger Ericsson Research

Fast packet processing in the cloud. Dániel Géhberger Ericsson Research Fast packet processing in the cloud Dániel Géhberger Ericsson Research Outline Motivation Service chains Hardware related topics, acceleration Virtualization basics Software performance and acceleration

More information

Veritas NetBackup on Cisco UCS S3260 Storage Server

Veritas NetBackup on Cisco UCS S3260 Storage Server Veritas NetBackup on Cisco UCS S3260 Storage Server This document provides an introduction to the process for deploying the Veritas NetBackup master server and media server on the Cisco UCS S3260 Storage

More information

G-WAN. Complete install process for Ubuntu (both for the 32 and the 64 OS versions).

G-WAN. Complete install process for Ubuntu (both for the 32 and the 64 OS versions). G-WAN Complete install process for Ubuntu 11.10 (both for the 32 and the 64 OS versions). G-WAN (Global Web Area Network) is both a web server (for static web pages) and a web application server (for rich

More information

CSE 124: Networked Services Lecture-17

CSE 124: Networked Services Lecture-17 Fall 2010 CSE 124: Networked Services Lecture-17 Instructor: B. S. Manoj, Ph.D http://cseweb.ucsd.edu/classes/fa10/cse124 11/30/2010 CSE 124 Networked Services Fall 2010 1 Updates PlanetLab experiments

More information

SMB Direct Update. Tom Talpey and Greg Kramer Microsoft Storage Developer Conference. Microsoft Corporation. All Rights Reserved.

SMB Direct Update. Tom Talpey and Greg Kramer Microsoft Storage Developer Conference. Microsoft Corporation. All Rights Reserved. SMB Direct Update Tom Talpey and Greg Kramer Microsoft 1 Outline Part I Ecosystem status and updates SMB 3.02 status SMB Direct applications RDMA protocols and networks Part II SMB Direct details Protocol

More information

(software agnostic) Computational Considerations

(software agnostic) Computational Considerations (software agnostic) Computational Considerations The Issues CPU GPU Emerging - FPGA, Phi, Nervana Storage Networking CPU 2 Threads core core Processor/Chip Processor/Chip Computer CPU Threads vs. Cores

More information

What You Need to Know When Buying a New Computer JackaboutComputers.com

What You Need to Know When Buying a New Computer JackaboutComputers.com If it s been a while since you bought your last computer, you could probably use a quick refresher on what you need to know to make a good purchase. Computers today are a much larger part of our life than

More information

RAIDIX Data Storage Solution. Clustered Data Storage Based on the RAIDIX Software and GPFS File System

RAIDIX Data Storage Solution. Clustered Data Storage Based on the RAIDIX Software and GPFS File System RAIDIX Data Storage Solution Clustered Data Storage Based on the RAIDIX Software and GPFS File System 2017 Contents Synopsis... 2 Introduction... 3 Challenges and the Solution... 4 Solution Architecture...

More information

Advanced Research Compu2ng Informa2on Technology Virginia Tech

Advanced Research Compu2ng Informa2on Technology Virginia Tech Advanced Research Compu2ng Informa2on Technology Virginia Tech www.arc.vt.edu Personnel Associate VP for Research Compu6ng: Terry Herdman (herd88@vt.edu) Director, HPC: Vijay Agarwala (vijaykag@vt.edu)

More information

SAS Enterprise Miner Performance on IBM System p 570. Jan, Hsian-Fen Tsao Brian Porter Harry Seifert. IBM Corporation

SAS Enterprise Miner Performance on IBM System p 570. Jan, Hsian-Fen Tsao Brian Porter Harry Seifert. IBM Corporation SAS Enterprise Miner Performance on IBM System p 570 Jan, 2008 Hsian-Fen Tsao Brian Porter Harry Seifert IBM Corporation Copyright IBM Corporation, 2008. All Rights Reserved. TABLE OF CONTENTS ABSTRACT...3

More information

IBM Power Systems HPC Cluster

IBM Power Systems HPC Cluster IBM Power Systems HPC Cluster Highlights Complete and fully Integrated HPC cluster for demanding workloads Modular and Extensible: match components & configurations to meet demands Integrated: racked &

More information

Designing Next Generation FS for NVMe and NVMe-oF

Designing Next Generation FS for NVMe and NVMe-oF Designing Next Generation FS for NVMe and NVMe-oF Liran Zvibel CTO, Co-founder Weka.IO @liranzvibel Santa Clara, CA 1 Designing Next Generation FS for NVMe and NVMe-oF Liran Zvibel CTO, Co-founder Weka.IO

More information

DDN s Vision for the Future of Lustre LUG2015 Robert Triendl

DDN s Vision for the Future of Lustre LUG2015 Robert Triendl DDN s Vision for the Future of Lustre LUG2015 Robert Triendl 3 Topics 1. The Changing Markets for Lustre 2. A Vision for Lustre that isn t Exascale 3. Building Lustre for the Future 4. Peak vs. Operational

More information

Linux multi-core scalability

Linux multi-core scalability Linux multi-core scalability Oct 2009 Andi Kleen Intel Corporation andi@firstfloor.org Overview Scalability theory Linux history Some common scalability trouble-spots Application workarounds Motivation

More information

EsgynDB Enterprise 2.0 Platform Reference Architecture

EsgynDB Enterprise 2.0 Platform Reference Architecture EsgynDB Enterprise 2.0 Platform Reference Architecture This document outlines a Platform Reference Architecture for EsgynDB Enterprise, built on Apache Trafodion (Incubating) implementation with licensed

More information

Sage Compatibility guide. Last revised: August 20, 2018

Sage Compatibility guide. Last revised: August 20, 2018 Sage 300 2019 Compatibility guide Last revised: August 20, 2018 2018 The Sage Group plc or its licensors. All rights reserved. Sage, Sage logos, and Sage product and service names mentioned herein are

More information

Habanero Operating Committee. January

Habanero Operating Committee. January Habanero Operating Committee January 25 2017 Habanero Overview 1. Execute Nodes 2. Head Nodes 3. Storage 4. Network Execute Nodes Type Quantity Standard 176 High Memory 32 GPU* 14 Total 222 Execute Nodes

More information

Windows Servers In Microsoft Azure

Windows Servers In Microsoft Azure $6/Month Windows Servers In Microsoft Azure What I m Going Over 1. How inexpensive servers in Microsoft Azure are 2. How I get Windows servers for $6/month 3. Why Azure hosted servers are way better 4.

More information

HPC Storage Use Cases & Future Trends

HPC Storage Use Cases & Future Trends Oct, 2014 HPC Storage Use Cases & Future Trends Massively-Scalable Platforms and Solutions Engineered for the Big Data and Cloud Era Atul Vidwansa Email: atul@ DDN About Us DDN is a Leader in Massively

More information

Technical guide. Windows HPC server 2016 for LS-DYNA How to setup. Reference system setup - v1.0

Technical guide. Windows HPC server 2016 for LS-DYNA How to setup. Reference system setup - v1.0 Technical guide Windows HPC server 2016 for LS-DYNA How to setup Reference system setup - v1.0 2018-02-17 2018 DYNAmore Nordic AB LS-DYNA / LS-PrePost 1 Introduction - Running LS-DYNA on Windows HPC cluster

More information

A Closer Look at SERVER-SIDE RENDERING. Technology Overview

A Closer Look at SERVER-SIDE RENDERING. Technology Overview A Closer Look at SERVER-SIDE RENDERING Technology Overview Driven by server-based rendering, Synapse 5 is the fastest PACS in the medical industry, offering subsecond image delivery and diagnostic quality.

More information

Introduction CPS343. Spring Parallel and High Performance Computing. CPS343 (Parallel and HPC) Introduction Spring / 29

Introduction CPS343. Spring Parallel and High Performance Computing. CPS343 (Parallel and HPC) Introduction Spring / 29 Introduction CPS343 Parallel and High Performance Computing Spring 2018 CPS343 (Parallel and HPC) Introduction Spring 2018 1 / 29 Outline 1 Preface Course Details Course Requirements 2 Background Definitions

More information

Maximizing Memory Performance for ANSYS Simulations

Maximizing Memory Performance for ANSYS Simulations Maximizing Memory Performance for ANSYS Simulations By Alex Pickard, 2018-11-19 Memory or RAM is an important aspect of configuring computers for high performance computing (HPC) simulation work. The performance

More information

BlueDBM: An Appliance for Big Data Analytics*

BlueDBM: An Appliance for Big Data Analytics* BlueDBM: An Appliance for Big Data Analytics* Arvind *[ISCA, 2015] Sang-Woo Jun, Ming Liu, Sungjin Lee, Shuotao Xu, Arvind (MIT) and Jamey Hicks, John Ankcorn, Myron King(Quanta) BigData@CSAIL Annual Meeting

More information

Architecting Storage for Semiconductor Design: Manufacturing Preparation

Architecting Storage for Semiconductor Design: Manufacturing Preparation White Paper Architecting Storage for Semiconductor Design: Manufacturing Preparation March 2012 WP-7157 EXECUTIVE SUMMARY The manufacturing preparation phase of semiconductor design especially mask data

More information

IBM B2B INTEGRATOR BENCHMARKING IN THE SOFTLAYER ENVIRONMENT

IBM B2B INTEGRATOR BENCHMARKING IN THE SOFTLAYER ENVIRONMENT IBM B2B INTEGRATOR BENCHMARKING IN THE SOFTLAYER ENVIRONMENT 215-4-14 Authors: Deep Chatterji (dchatter@us.ibm.com) Steve McDuff (mcduffs@ca.ibm.com) CONTENTS Disclaimer...3 Pushing the limits of B2B Integrator...4

More information

Enterprise print management in VMware Horizon

Enterprise print management in VMware Horizon Enterprise print management in VMware Horizon Introduction: Embracing and Extending VMware Horizon Tricerat Simplify Printing enhances the capabilities of VMware Horizon environments by enabling reliable

More information

Paperspace. Architecture Overview. 20 Jay St. Suite 312 Brooklyn, NY Technical Whitepaper

Paperspace. Architecture Overview. 20 Jay St. Suite 312 Brooklyn, NY Technical Whitepaper Architecture Overview Copyright 2016 Paperspace, Co. All Rights Reserved June - 1-2017 Technical Whitepaper Paperspace Whitepaper: Architecture Overview Content 1. Overview 3 2. Virtualization 3 Xen Hypervisor

More information