1 Preston Smith Director of Research Services RESEARCH DATA DEPOT AT PURDUE UNIVERSITY May 18, 2016 HTCONDOR WEEK 2016
2 Ran into Miron at a workshop recently.. Talked about data and the challenges of providing that service to campus. Miron: I d like to talk about that at HTCondor Week! Campuses and HTCondor sites all face challenges with storage, so.. (Dramatization)
3 COMMUNITY SECTION TITLE CLUSTERS A BUSINESS MODEL FOR COMPUTING AT PURDUE
4 COMMUNITY CLUSTERS THE BASIC RULES You get out at least what you put in Buy 1 node or 100, you get a queue that guarantees access up to that many CPUs But wait, there s more!! What if your neighbor isn t using his queue? You can use it, but your job is subject to a time limit You don t have to do the work Your grad student gets to do research rather than run your cluster. Nor do you have to provide space in your lab for computers. IT provides data center space, systems administration, application support. Just submit jobs!
5 COMMUNITY CLUSTERS VERSION 2: FURTHER REFINEMENT 5 Year cycle We build a cluster every year! Vendors provide 5 year warranty After 5 years, MOU with faculty says that the cluster will be retired Faculty get credit for the remaining market value of their nodes, towards the next cluster. Community clusters now appear to funding agencies as paying for a service not a capital purchase.
6 CHALLENGES CHALLENGES TO SMALLER COMMUNITIES HPC and HTC communities prefer different points to optimize the scheduler. Small but key communities (like large memory) lose benefits of standby queues when fewer nodes are spread between several clusters. HTC or large memory communities often have little need for HPC-specific optimizations Accelerators High-speed, low-latency networks Emerging communities often don t fit in existing model at all! Big Data Analytics Graphics Rendering Nontraditional platforms (Windows, cloud)
7 OBLIGATORY HTC/HTCONDOR CONTENT COMING
8 HTCONDOR We still run a fairly substantial HTCondor resource Serving CMS, opportunistic access for many OSG VOs, and several Purdue researchers Out of 240M hours delivered by our center last year, ~31M were High Throughput SEE, I LL TALK ABOUT IT!
9 DATA SECTION TITLE STORAGE INFRASTRUCTURE FOR RESEARCH DATA
10 SCRATCH HPC DATA Scratch needs are climbing Avg GB used (top 50 users) Avg GB used (top 50 users)
11 FORTRESS HPC DATA Archive usage is skyrocketing Fortress Archive Growth 4,500, ,035, ,000, ,500, ,273, ,000, ,500, ,109, ,000, ,500, ,002, ,000, , , , TB used
12 HPC STORAGE TIERS OF STORAGE Scr atc h Sto rag Working e space Archival Space Fast, large, purged, coupled with clusters, per-user for running jobs Medium speed, large, persistent, data protected, purchased, per research lab for shared data and apps High-speed, infinite capacity, highly-protected, available to all researchers for permanent storage
13 RESEARCH DATA ACROSS CAMPUS CURRENT STATE Working with researchers across campus, we encounter many different data solutions.. From something at the department/workgroup level:
14 RESEARCH DATA ACROSS CAMPUS To This CURRENT STATE
15 RESEARCH DATA ACROSS CAMPUS And This CURRENT STATE
16 RESEARCH STORAGE GAPS THE PROBLEM Many central storage options have not met all the needs that researchers care about Departmental or lab resources are islands and not accessible from HPC clusters. Most are individually-oriented, rather than built around the notion of a research lab. Scratch filesystems are also limited in scope to a single HPC system Some are not performant enough for research use Sometimes nothing is available for sale, despite faculty having funds
17 RESEARCH STORAGE GAPS COMMON QUESTIONS In the past, we ve heard lots of common requests: I need more space than I can get in scratch Where can I install applications for my entire research lab? I m actively working on that data/software in scratch: I have to go to great lengths to keep it from being purged. I shouldn t have to pull it from Fortress over and over Can I get a UNIX group created for my students and I? Is there storage that I can get to on all the clusters I use? I have funding to spend on storage what do you have to sell? I need storage for my instrument to write data into My student has the only copy of my research data in his home directory, and he graduated/went off the grid!
18 RESEARCH STORAGE GAPS SOME SOLUTIONS We ve addressed some of these with improving scratch: Everybody automatically gets access to HPSS Archive for permanent data storage Very large per-user scratch quotas More friendly purge policy based on the use of data, rather than when it was created.
19 DEPOT WHY A DEPOT? As a transport hub: a place where large amounts of cargo are stored, loaded, unloaded, and moved from place to place.
20 DATA GUIDING PRINCIPLES It s important to think of Depot as a data service not storage It is not enough to just provide infrastructure Here s a mountpoint, have fun Our goal: enabling the frictionless use and movement of data Instrument -> Depot -> Scratch -> Fortress -> Collaborators -> and back Continue to improve access to non-unix users
21 THE SERVICE FEATURES Purdue researchers can purchase storage! An affordable storage service for research to address many common requests: 100G available at no charge to research groups Mounted on all clusters and exported via CIFS to labs Not scratch: Backed up via snapshots, with DR coverage Data in Depot is owned by faculty member! Sharing ability Globus, CIFS, and WWW Maintain group-wide copies of application software or shared data
22 A TOOL-FRIENDLY STRUCTURE AS A STARTING POINT with POSIX ACLs to overcome group permission and umask challenges!
23 SELF-SERVICE MANAGE YOUR OWN ACCESS
24 A SOLUTION ADOPTION Well received! In just over a year, over 260 research groups are participating. Many are not HPC users!.75 PB in use since November 2014 A research group purchasing space has purchased, on average, 8.6TB.
25 A TESTIMONIAL SATISFIED FACULTY "It's really, really useful. It's actually one of the best things about this setup. In the past, with other HPC systems I've used, moving all the data around that these models output has always been a major pain. One of these high resolution simulations can produce terabytes of output and you've got to have somewhere you can put that in order to analyze it. The Research Data Depot is a place where I can put a large amount of data, I can mount it on local machines and access it. The Research Data Depot [is] large like the scratch but it's disk based and fast unlike the tape archive, so it's got the best of both world's, I think, and it's backed up! I think it's great. I use it a lot. I have 25 terabytes right now and I'd like to get more." Daniel Dawson, Assistant Professor of Earth, Atmospheric, and Planetary Science
26 THE TECHNOLOGY Approximately 2.25 PB of IBM GPFS Hardware provided by a pair of Data Direct Networks SFA12k arrays, one in each of two datacenters 160 Gb/sec to each datacenter 5x Dell R620 servers in each datacenter WHAT DID WE GET?
27 DESIGN TARGETS WHAT DO WE NEED TO DO? The Research Data Depot Can do: Depot Requirements Previous solutions At least 1 PB usable capacity >1 PB 40 GB/sec throughput 5 GB/sec < 3ms average latency, < 20 ms maximum latency Variable 100k IOPS sustained 55k 300 MB/sec min client speed 200 MB/sec max Support 3000 simultaneous clients Yes Filesystem snapshots Yes Multi-site replication No Expandable to 10 PB Yes Fully POSIX compliant, including parallel I/O No
28 GROWTH BY COLLEGE GROWTH AREAS 2015 New Groups 2014 Growth rate Liberal Arts % Education % HHS % Agriculture % Science (bio) % Pharmacy % Engineering % Science (non-bio) % Technology % Management % Vet School College 20% Life Science!
29 RELATED SECTION TITLE CHALLENGES BEYOND THE DATACENTER
30 IMPROVED NETWORKING WHAT DOES THIS MEAN FOR RESEARCHERS? As science gets more data-intensive researchers require increasing amounts of bandwidth (GigE on up) The last mile to their labs will be the key!
31 IMPROVED NETWORKING INSTRUMENTS Instruments are getting cheaper, more common, and generate more data. Researchers need high-speed (10Gb+) connections for labs and instruments to move data into clusters, Research Data Depot, and research WAN connections.
32 FIREWALLS The situation on campus is mixed, but researchers shouldn t have to choose. Watch for fast GigE links being throttled by 100Mb firewalls! We need to get researchers from DIY firewalls into the enterprise-grade one. SAFETY OR PERFORMANCE
33 GLOBUS IS KEY STATISTICS Data transferred in the last year: Volume: nearly 300 TB transferred! Avg 23 TB, 50 unique users each month Killer feature: sharing!
34 FUTURE WORK Need staff with expertise in the mechanics of working with data Work with Libraries to integrate with institutional repository How to centrally fund the storage?
Line Pouchard, PhD Purdue University Libraries Research Data Group Big Data infrastructure and tools in libraries 08/10/2016 DATA IN LIBRARIES: THE BIG PICTURE IFLA/ UNIVERSITY OF CHICAGO BIG DATA: A VERY
Science-as-a-Service The iplant Foundation Rion Dooley Edwin Skidmore Dan Stanzione Steve Terry Matthew Vaughn Outline Why, why, why! When duct tape isn t enough Building an API for the web Core services
WVU RESEARCH COMPUTING INTRODUCTION Introduction to WVU s Research Computing Services WHO ARE WE? Division of Information Technology Services Funded through WVU Research Corporation Provide centralized
Network Support for Data Intensive Science Eli Dart, Network Engineer ESnet Network Engineering Group ARN2 Workshop Washington, DC April 18, 2013 Overview Drivers Sociology Path Forward 4/19/13 2 Exponential
Cloud & Datacenter EGA The Stock Exchange of Thailand Materials excerpt from SET internal presentation and virtualization vendor e.g. vmware For Educational purpose and Internal Use Only SET Virtualization/Cloud
DELL EMC ISILON F800 AND H600 I/O PERFORMANCE ABSTRACT This white paper provides F800 and H600 performance data. It is intended for performance-minded administrators of large compute clusters that access
Cluster-Level Storage @ Google How we use Colossus to improve storage efficiency Denis Serenyi Senior Staff Software Engineer firstname.lastname@example.org November 13, 2017 Keynote at the 2nd Joint International
Conference 2017 The Data Challenges of the LHC Reda Tafirout, TRIUMF Outline LHC Science goals, tools and data Worldwide LHC Computing Grid Collaboration & Scale Key challenges Networking ATLAS experiment
Cloud Storage with AWS: EFS vs EBS vs S3 AHMAD KARAWASH Cloud Storage with AWS Cloud storage is a critical component of cloud computing, holding the information used by applications. Big data analytics,
1 DDN About Us Solving Large Enterprise and Web Scale Challenges History Founded in 98 World s Largest Private Storage Company Growing, Profitable, Self Funded Headquarters: Santa Clara and Chatsworth,
University of Colorado Boulder Research Computing PetaLibrary Storage Service MOU 1. INTRODUCTION This is the memorandum of understanding (MOU) for the Research Computing (RC) PetaLibrary Storage Service.
Data Movement and Storage 04/07/09 www.cac.cornell.edu 1 Data Location, Storage, Sharing and Movement Four of the seven main challenges of Data Intensive Computing, according to SC06. (Other three: viewing,
Georgia State University Cyberinfrastructure Plan Summary Building relationships with a wide ecosystem of partners, technology, and researchers are important for GSU to expand its innovative improvements
Cloud Computing CS 537 Fall 2017 What is cloud computing Illusion of infinite computing resources available on demand Scale-up for most apps Elimination of up-front commitment Small initial investment,
Cloud Bursting: Top Reasons Your Organization will Benefit Scott Jeschonek Director of Cloud Products Avere Systems Agenda Define Cloud Bursting Benefits of using Cloud Bursting Identify Cloud Bursting
Opendedupe & Veritas NetBackup ARCHITECTURE OVERVIEW AND USE CASES May, 2017 Contents Introduction... 2 Overview... 2 Architecture... 2 SDFS File System Service... 3 Data Writes... 3 Data Reads... 3 De-duplication
Deep Storage for Exponential Data Nathan Thompson CEO, Spectra Logic HISTORY Partnered with Fujifilm on a variety of projects HQ in Boulder, 35 years of business Customers in 54 countries Spectra builds
Accelerate your Azure Hybrid Cloud Business with HPE Ken Won, HPE Director, Cloud Product Marketing Mega trend: Customers are increasingly buying cloud services from external service providers Speed of
CLOUD MEETS BIG DATA Sujal Patel President, Isilon Storage Division EMC Corporation EMC s Mission To Lead Customers On Their Journey To Cloud Computing And Transforming IT... By Enabling Them To Store,
Storage PRESENTATION in the TITLE DIMM GOES HERE Socket Adrian Proctor Vice President, Marketing Viking Technology SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA unless
The Cirrus Research Computing Cloud Faculty of Science What is Cloud Computing? Cloud computing is a physical cluster which runs virtual machines Unlike a typical cluster there is no one operating system
Welcome to today s Webcast. Thank you so much for joining us today! My name is Michael Costa. I m a member of the DART Team, one of several groups engaged by HAB to provide training and technical assistance
1 Get More Out of Storage with Data Domain Deduplication Storage Systems David M. Auslander Sales Director, New England / Eastern Canada 2 EMC Data Domain Dedupe everything without changing anything Simplify
FlashSystem Family 2015 IBM FlashSystem IBM FLiP Tool Wie viel schneller kann Ihr IBM i Power Server mit IBM FlashSystem 900 / V9000 Storage sein? PiRT - Power i Round Table 17 Sep. 2015 Daniel Gysin IBM
Slashing Downtime from 24 Hours to 24 Minutes: Technology Advancements Make Warm-Site Disaster Recovery Possible Don Beyer Director, Technical Services Oakwood Healthcare Louie Caschera Chief Information
XSEDE New User Training Ritu Arora Email: email@example.com November 14, 2014 1 Objectives Provide a brief overview of XSEDE Computational, Visualization and Storage Resources Extended Collaborative
Simple Reliable Affordable The Oracle Database Appliance I/O and Performance Architecture Tammy Bednar, Sr. Principal Product Manager, ODA 1 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Enterprise Strategy Group Getting to the bigger truth. ESG Lab Validation Dell EMC Isilon All-Flash Scale-out All-flash Storage for Demanding Unstructured Data Workloads By Tony Palmer, Senior Lab Analyst
WHITE PAPER I JUNE 2010 LEVERAGING A PERSISTENT HARDWARE ARCHITECTURE How an Open, Modular Storage Platform Gives Enterprises the Agility to Scale On Demand and Adapt to Constant Change. LEVERAGING A PERSISTENT
ALICE Grid Activities in US 1 ALICE-USA Computing Project ALICE-USA Collaboration formed to focus on the ALICE EMCal project Construction, installation, testing and integration participating institutions
IBM TS4300 with IBM Spectrum Storage - The Perfect Match - Vladimir Atanaskovik IBM Spectrum Storage and IBM TS4300 at a glance Scale Archive Protect In July 2017 IBM The #1 tape vendor in the market -
IME (Infinite Memory Engine) Extreme Application Acceleration & Highly Efficient I/O Provisioning September 22 nd 2015 Tommaso Cecchi 2 What is IME? This breakthrough, software defined storage application
GET CLOUD EMPOWERED. SEE HOW THE CLOUD CAN TRANSFORM YOUR BUSINESS. Cloud computing is as much a paradigm shift in data center and IT management as it is a culmination of IT s capacity to drive business
Interface Trends for the Enterprise I/O Highway Mitchell Abbey Product Line Manager Enterprise SSD August 2012 1 Enterprise SSD Market Update One Size Does Not Fit All : Storage solutions will be tiered
TELSTRA CLOUD SERVICES VMWARE VCLOUD AIR PRICING GUIDE 20 NOVEMBER 2015 WELCOME TO VMWARE VCLOUD AIR SERVICES VMware vcloud Air is a cloud infrastructure offering from Telstra which includes a range of
The Future of Virtualization: The Virtualization of the Future Larry Rudolph, VMware 200 VMware Inc. All rights reserved Caveats The opinions expressed in this talk are Entirely my own Do not come with
1 Improved Solutions for I/O Provisioning and Application Acceleration August 11, 2015 Jeff Sisilli Sr. Director Product Marketing firstname.lastname@example.org 2 Why Burst Buffer? The Supercomputing Tug-of-War A supercomputer
Introduction to High Performance Parallel I/O Richard Gerber Deputy Group Lead NERSC User Services August 30, 2013-1- Some slides from Katie Antypas I/O Needs Getting Bigger All the Time I/O needs growing
DISK LIBRARY FOR MAINFRAME Mainframe Tape Replacement with cloud connectivity ESSENTIALS A Global Virtual Library for all mainframe tape use cases Supports private and public cloud providers. GDDR Technology
Data Movement & Storage Using the Data Capacitor Filesystem Justin Miller email@example.com http://pti.iu.edu/dc Big Data for Science Workshop July 2010 Challenges for DISC Keynote by Alex Szalay identified
SLATE Services Layer at the Edge Rob Gardner University of Chicago Shawn McKee University of Michigan Joe Breen University of Utah First Meeting of the National Research Platform Montana State University
CNM Master Plan Line # Master Plan Project Title Description Total Project Budget Section A - Proposed Facilities and IT Maintenance Projects for FY18 Projected spending for FY18 Potential Funding Source
Refining and redefining HPC storage High-Performance Computing Demands a New Approach to HPC Storage Stick with the storage status quo and your story has only one ending more and more dollars funneling
Data Staging: Moving large amounts of data around, and moving it close to compute resources PRACE advanced training course on Data Staging and Data Movement Helsinki, September 10 th 2013 Claudio Cacciari
How do I Manage my Disk and Profile Quotas? HTS controls the amount of disk space that each customer can use on its Windows servers. This is necessary to ensure efficiency and performance of the servers.
INTERVIEW TRANSCRIPT DDoS: Evolving Threats, Solutions Carlos Morales of Arbor Networks Offers New Strategies FEATURING: Characteristics of recent attacks; Gaps in organizations defenses; How to best prepare
Mainframe Backup Modernization Disk Library for mainframe Mainframe is more important than ever itunes Downloads Instagram Photos Twitter Tweets Facebook Likes YouTube Views Google Searches CICS Transactions
Data Staging: Moving large amounts of data around, and moving it close to compute resources Digital Preserva-on Advanced Prac--oner Course Glasgow, July 19 th 2013 firstname.lastname@example.org Definition Starting
Disaster Recovery Planning How to Ensure your IT systems are protected and your business keeps running should disaster strike. Benefits of Using Disaster Recovery as a Service DRaaS over Traditional Disaster
How to solve your backup problems with HP StoreOnce Andrew Dickerson Senior Manager Backup, Recovery and Archive June 25, 2014 Time is running out for legacy storage Cloud Volume 2005 2010 2012 2015 50X
High Performance Computing Management Philippe Trautmann BDM High Performance Computing Global Education @ Research HPC Market and Trends High Performance Computing: Availability/Sharing is key European
ProgressTestA Unit Vocabulary 1 Completethesentenceswithappropriate words.thefirstlettersofthewordshavebeen given. a Can you believe it? She s getting married to a man she has met on a s networking site!
Want More out of SQL Server? Consider Hyperconverged Infrastructure WHITE PAPER SQL Server is one of the industry s most successful software products. For more than two decades, Microsoft s flagship relational
Backtesting in the Cloud A Scalable Market Data Optimization Model for Amazon s AWS Environment A Tick Data Custom Data Solutions Group Case Study Bob Fenster, Software Engineer and AWS Certified Solutions
(RAPID) Landing Page Building A Practical Guide Presented by Thrive Themes Introduction Why RAPID is Better than Perfect This guide came about because of perfectionism. When we create landing pages, websites,
IBM Storage Software Strategy Hakan Turgut 1 Just how fast is the data growing? 128 GB/ person The World s total data per person No Problem I I think I can do this We have a problem 24 GB/ person 0.8 GB/
IBM Tivoli Storage Manager for Windows Version 7.1.8 Installation Guide IBM IBM Tivoli Storage Manager for Windows Version 7.1.8 Installation Guide IBM Note: Before you use this information and the product
WHICH HARD DRIVE IS RIGHT FOR YOU? Choosing the right desktop drive can be a challenge. Not only is it difficult to tell one small grey box apart from all the other small grey boxes, but there are so many
The LHC Computing Grid Visit of Finnish IT Centre for Science CSC Board Members Finland Tuesday 19 th May 2009 Frédéric Hemmer IT Department Head The LHC and Detectors Outline Computing Challenges Current
CDS and Sky Tech Brief Configuring Short RPO with Actifio StreamSnap and Dedup-Async Replication Actifio recommends using Dedup-Async Replication (DAR) for RPO of 4 hours or more and using StreamSnap for
Storage Optimization with Oracle Database 11g Terabytes of Data Reduce Storage Costs by Factor of 10x Data Growth Continues to Outpace Budget Growth Rate of Database Growth 1000 800 600 400 200 1998 2000
CASE STUDY: USING THE HYBRID CLOUD TO INCREASE CORPORATE VALUE AND ADAPT TO COMPETITIVE WORLD TRENDS Geoff Duncan, Senior Solutions Architect, Digital Fortress Brandon Tanner, Senior Manager, Rentsys Recovery
Picture yourself standing next to an empty tool box. If you are building a shed, you ll need to make sure that tool box contains a hammer, a saw, a level, and the host of other tools necessary to build
Data services for LHC computing SLAC 1 Xavier Espinal on behalf of IT/ST DAQ to CC 8GB/s+4xReco Hot files Reliable Fast Processing DAQ Feedback loop WAN aware Tier-1/2 replica, multi-site High throughout
Web Hosting Important features to consider Amount of Storage When choosing your web hosting, one of your primary concerns will obviously be How much data can I store? For most small and medium web sites,
September 26, 2014 XSEDE Campus Bridging Tools Jim Ferguson National Institute for Computational Sciences email@example.com Quick Advertisement: Student Programs Research Experience 12-18 students per summer,
Dell - Video Solutions Jeff Junker Technical Consultant Video Solutions Dell ESG Agenda Dell Forum 2011 Surveillance Overview What is the application or solution that needs to be deployed Solution Overview
EMC Solution for VIEVU Body Worn Cameras Functional Validation Guide H14533 01 Copyright 2015 EMC Corporation. All rights reserved. Published in USA. Published October, 2015 EMC believes the information
Solution brief Intel Storage Builders WekaIO Matrix Intel eon Processor E5-2600 Product Family Intel Ethernet Converged Network Adapter 520 Intel SSD Data Center Family Data Plane Development Kit Create
Key metrics for effective storage performance and capacity reporting Key Metrics for Effective Storage Performance and Capacity Reporting Objectives This white paper will cover the key metrics in storage
1 Academic Workflow for Research Repositories Using irods and Object Storage 2016 irods User s Group Meeting 9 June 2016 Randall Splinter, Ph.D. HPC Research Computing Solutions Architect RSplinter@ 770.633.2994
FLASHARRAY//M Business and IT Transformation in 3U TRANSFORM IT Who knew that moving to all-flash storage could help reduce the cost of IT? FlashArray//m makes server and workload investments more productive,
Was ist dran an einer spezialisierten Data Warehousing platform? Hermann Bär Oracle USA Redwood Shores, CA Schlüsselworte Data warehousing, Exadata, specialized hardware proprietary hardware Introduction
Brutus Above and beyond Hreidar and Gonzales Dr. Olivier Byrde Head of HPC Group, IT Services, ETH Zurich Teodoro Brasacchio HPC Group, IT Services, ETH Zurich 1 Outline High-performance computing at ETH
Boost your business with a more flexible phone system Cut costs and do more with your calls with BT Cloud Voice The phone system for businesses that are going places Cloud Voice could save you money and
Turning Object Open vstorage is the World s fastest Distributed Block Store that spans across different Datacenter. It combines ultrahigh performance and low latency connections with a data integrity that
See what s new: Data Domain Global Deduplication Array, DD Boost and more 2010 1 EMC Backup Recovery Systems (BRS) Division EMC Competitor Competitor Competitor Competitor Competitor Competitor Competitor
Advanced Research Compu2ng Informa2on Technology Virginia Tech www.arc.vt.edu Personnel Associate VP for Research Compu6ng: Terry Herdman (firstname.lastname@example.org) Director, HPC: Vijay Agarwala (email@example.com)
Redis Labs on POWER8 Server: The Promise of OpenPOWER Value Jeffrey L. Leeds, Ph.D. Vice President, Alliances & Channels Revolutionizing the Datacenter Join the Conversation #OpenPOWERSummit Who We Are
Malaysia's Computational Journey Towards Science DMZ Suhaimi Napis Technical Advisory Committee (Research Computing) MYREN-X Universiti Putra Malaysia firstname.lastname@example.org In the Beginning... Research on parallel/distributed