On-demand provisioning of HEP compute resources on cloud sites and shared HPC centers
|
|
- Dwain Jack Ferguson
- 10 months ago
- Views:
Transcription
1 On-demand provisioning of HEP compute resources on cloud sites and shared HPC centers CHEP San Francisco, United States of America Gunther Erli, Frank Fischer, Georg Fleig, Manuel Giffels, Thomas Hauth, Gunter Quast, Matthias Schnepf Institute of Experimental Nuclear Physics (IEKP), KIT, Germany Heese Katja Leppert, Javier Arnaez de Pedro, Rainer Strater Jorg 1&1 Internet SE, Montabaur, Germany
2 Motivation Modern high energy physics (HEP) research relies heavily on large computing resources: Simulation CPU intensive, moderate I/O Reconstruction I/O and CPU intensive Analysis I/O intensive, moderate CPU usage Other Compute-Intense Jobs like Limit Calculation, Theory Calculations mostly high CPU, low I/O Continuously growing demand for computing resources requires rethinking of traditional HEP-only clusters New ways to acquire computing capacity HPC Systems High Performance Computing (HPC) resources via bare-metal or virtualization Local Opportunistic Resources via bare metal, lightweight Docker or virtualization abstraction Commercial Clouds Pay commercial providers to handle peak loads or dedicated computation campaigns 12. October /12
3 From HPC Cluster to HEP Worker Node virtual infrastructure e.g. HEP Compute Node IaaS physical Infrastructure e.g. HPC node Virtual Host Cloud Software Host OS The Infrastructure-as-a-Service (IaaS) model Infrastructure (e.g. machines, network) is virtualized Decouples complexities of hardware maintenance and specific software setup The life cycle of this virtual infrastructure is managed by a Cloud system: Virtual machine images are managed The user can upload and start custom virtual machines HEP software is loaded on-demand via the CVMFS file system Input and output data is loaded via fast WAN links from HEP-specific storage sites with SRM or XRootD 12. October /12
4 Our Technology Choices Goals: off-the-shelf (if possible), experiment-independent Batch Server: HTCondor [1] Excels at handling dynamic resources Can easily integrate worker nodes beyond network zone boundaries ClassAd system and custom job routing allows adaption to specific use cases Resilient, scales to more than 100k jobs, open source Cloud Scheduler: ROCED [2] Support for multiple Cloud APIs (OpenStack, Amazon EC2 and other commercial providers) and batch systems Easily extendable thanks to modular design Parses HTCondor s ClassAds and boots VMs on cloud sites For System Abstraction: OpenStack and Docker 12. October /12
5 Job Submission and Workflow Management Goal: allow the Institute s user groups seamless and hassle-free access to both the Institute s local and the remote Cloud resources. jobs monitor jobs jobs Users can submit their jobs using the condor submit command The recommended workflow is to use the job submission tool grid control [3] By adding specific HTCondor ClassAds when submitting the job, users can either: submit only to local worker-nodes (with direct fileserver access) submit only to remote Cloud worker-nodes (with file access via Grid tools) submit to both at the same time 12. October /12
6 HPC System: shared hybrid Cluster Located at Freiburg University 150km south of Karlsruhe Shared by 3 diverse scientific user groups: Elementary Particle Physics, Neuroscience, Microsystem Engineering Full system recently installed with 16,000 Broadwell CPU cores OpenStack used as virtualization manager VM scheduling integrated into existing batch system to honor other user groups on cluster Hybrid HPC Cluster [4] Special ROCED-Adapter was developed to provision VMs via the HPC Cluster s batch system Important feature User jobs are automatically executed on this cluster, if local file system access was not explicitly requested by the users 12. October /12
7 HPC System: Results Jobs Slots Slots requested Slots available Slots draining Jobs available Time [days] Pre-production cluster (1000 cores) used for 6 month Some adaptations were required: more OpenStack management nodes, configuration of HTCondor collector (file handle limit and friends) Production system has been scaled up to virtualized cores In total, more than 5 million CPU hours of user jobs so far processed 12. October /12
8 Local Opportunistic: HTCondor+Docker New feature in HTCondor: run jobs in a Docker container, we call it HTConDocker Users submit job via HTCondor HTCondor runs on bare-metal system and starts a Docker container on demand More lightweight (esp. memory) compared to full virtualization Easier administration than OpenStack: only HTCondor and Docker Users may select a Docker container best suited for their needs Positive Experience so far Used successfully on our Institute s powerful Desktops (4 core, 16 GB RAM) to provide an additional 150 opportunistic job slots More than 10k CPU hours of computing over one weekend Satisfies the requirements of various user groups: AMS, Belle II, CMS 12. October /12
9 Commercial Cloud: 1&1 Internet Scope: 1&1 Internet SE is one of Europe s leading internet service provider with a strong global presence Jointly 1&1 and the KIT team started a pilot project to evaluate the Cloud Server product for HEP jobs The Cloud Server product offers dynamic provisioning of VMs and accurate billing depending on the machines uptime and configuration With the goal to make the results available to the scientific community, we adapted the 1&1 standard product to HEP use cases ROCED-Adapter for Cloud Server API [5] was developed Possibility to upload and deploy a custom VM-Image with Scientic Linux 6.7 (or other, if needed) and CVMFS and HTCondor support Configured dedicated CVMFS-Squid VM in 1&1 data center which gets automatically started by ROCED as soon as one worker VM is booted Possibility to run workloads across 1&1 s data centers 12. October /12
10 1&1 Internet Results Number of slots Slots requested Slots available Slots draining Time [days] Resulting preconditions: We took advantage of the fact that the load distribution is lower during night time (due to typical customer profile) Only jobs with a run-time smaller than 12h were scheduled to 1&1 data centers Results: Per night up to 800 job slots were provisioned, if enough user jobs had been queued Jobs of users from multiple experiments run reliably on the virtual WN API based scheduling without manual intervention is possible 12. October /12
11 Conclusion Our institute runs a flexible computing system which is able to leverage resources from multiple sources: Shared HPC System Local Opportunistic Resources 1&1 Internet Cloud Server The complexity and provisioning of these (remote) resources is hidden from the user with: HTCondor - for job submission and management ROCED - for provisioning of remote resources This dynamic model gradually replaces our Institute s private HEP-only cluster in the basement: both in ease of use and capacity 12. October /12
12 References I HTCondor website. ( ). ROCED website. ( ). Grid-control website. ( ). ACAT 2016: Dynamic provisioning of a HEP computing infrastructure on a shared hybrid HPC system. ( ). 1&1 Cloud Server SDK Python. ( ). 12. October /12
13 Additional Material 12. October /12
14 High [Performance and Throughput] Computing High Performance Computing (HPC) focuses on the efficient execution of compute intensive, tightly-coupled tasks.* High Throughput Computing (HTC) focuses on the efficient execution of a large number of loosely-coupled tasks.* Nearly all High Energy Physics workloads belong to the HTC category Property typical HPC jobs typical HEP jobs Interdependence interconnect between nodes each node independent Runtime runs up to weeks individual job runtime 24h HEP jobs on HPC Clusters HEP batch jobs can be placed anywhere (no fast interconnect between jobs) Backfill of HEP jobs can be run also in smaller quantities to fully load a partially occupied cluster * according to the European Grid Infrastructure (EGI) V1 12. October /12
15 Topology of Local and Remote Sites GridKa Storage KA Campus North WAN Link Batch Server HTCondor Compute Node nova-compute Controller Node Compute Node nova-api nova-api nova-scheduler nova-scheduler nova-conductor nova-conductor keystone-all keystone-all neutron-server neutron-server neutron-ovs-plugin neutron-ovs-plugin neutron-l3-agent neutron-l3-agent neutron-dhcp-agent neutron-dhcp-agent glance-api glance-api glance-registry glance-registry horizon horizon mysql-server rabbitmq-server openvswitch apache-server KA Campus South Remote Site 12. October /12
16 Data-transfer from and to Remote Site 400 concurrent VMs running at remote site to simulate typical work load Stage-In: Jobs copy files with the size 250 MB to 1 GB from the GridKa Tier-1 storage element Stage-Out: Synthetic jobs create random 1 GB files and transfer them to the GridKa Tier-1 storage element Benchmarks the full chain: Hardware virtualization, Firewalls, Routing and storage pools Successful outcome: The available bandwidth (10 GBit) can be saturated 12. October /12
Virtualization of the ATLAS Tier-2/3 environment on the HPC cluster NEMO
Virtualization of the ATLAS Tier-2/3 environment on the HPC cluster NEMO Ulrike Schnoor (CERN) Anton Gamel, Felix Bührer, Benjamin Rottler, Markus Schumacher (University of Freiburg) February 02, 2018
Grid Computing Activities at KIT
Grid Computing Activities at KIT Meeting between NCP and KIT, 21.09.2015 Manuel Giffels Karlsruhe Institute of Technology Institute of Experimental Nuclear Physics & Steinbuch Center for Computing Courtesy
Evolution of the ATLAS PanDA Workload Management System for Exascale Computational Science
Evolution of the ATLAS PanDA Workload Management System for Exascale Computational Science T. Maeno, K. De, A. Klimentov, P. Nilsson, D. Oleynik, S. Panitkin, A. Petrosyan, J. Schovancova, A. Vaniachine,
Tier 3 batch system data locality via managed caches
Journal of Physics: Conference Series PAPER OPEN ACCESS Tier 3 batch system data locality via managed caches To cite this article: Max Fischer et al 2015 J. Phys.: Conf. Ser. 608 012018 Recent citations
IN2P3-CC cloud computing (IAAS) status FJPPL Feb 9-11th 2016
Centre de Calcul de l Institut National de Physique Nucléaire et de Physique des Particules IN2P3-CC cloud computing (IAAS) status FJPPL Feb 9-11th 2016 1 Outline Use cases R&D Internal core services Computing
Conference The Data Challenges of the LHC. Reda Tafirout, TRIUMF
Conference 2017 The Data Challenges of the LHC Reda Tafirout, TRIUMF Outline LHC Science goals, tools and data Worldwide LHC Computing Grid Collaboration & Scale Key challenges Networking ATLAS experiment
HPC learning using Cloud infrastructure
HPC learning using Cloud infrastructure Florin MANAILA IT Architect florin.manaila@ro.ibm.com Cluj-Napoca 16 March, 2010 Agenda 1. Leveraging Cloud model 2. HPC on Cloud 3. Recent projects - FutureGRID
MOHA: Many-Task Computing Framework on Hadoop
Apache: Big Data North America 2017 @ Miami MOHA: Many-Task Computing Framework on Hadoop Soonwook Hwang Korea Institute of Science and Technology Information May 18, 2017 Table of Contents Introduction
Huawei FusionSphere 6.0 Technical White Paper on OpenStack Integrating FusionCompute HUAWEI TECHNOLOGIES CO., LTD. Issue 01.
Technical White Paper on OpenStack Integrating Issue 01 Date 2016-04-30 HUAWEI TECHNOLOGIES CO., LTD. 2016. All rights reserved. No part of this document may be reproduced or transmitted in any form or
Introduction to Neutron. Network as a Service
Introduction to Neutron Network as a Service Assaf Muller, Associate Software Engineer, Cloud Networking, Red Hat assafmuller.wordpress.com, amuller@redhat.com, amuller on Freenode (#openstack) The Why
DEEP DIVE: OPENSTACK COMPUTE
DEEP DIVE: OPENSTACK COMPUTE Stephen Gordon Technical Product Manager, Red Hat @xsgordon AGENDA OpenStack architecture refresher Compute architecture Instance life cycle Scaling compute
Corral: A Glide-in Based Service for Resource Provisioning
: A Glide-in Based Service for Resource Provisioning Gideon Juve USC Information Sciences Institute juve@usc.edu Outline Throughput Applications Grid Computing Multi-level scheduling and Glideins Example:
High Throughput WAN Data Transfer with Hadoop-based Storage
High Throughput WAN Data Transfer with Hadoop-based Storage A Amin 2, B Bockelman 4, J Letts 1, T Levshina 3, T Martin 1, H Pi 1, I Sfiligoi 1, M Thomas 2, F Wuerthwein 1 1 University of California, San
Introducing the HTCondor-CE
Introducing the HTCondor-CE CHEP 2015 Presented by Edgar Fajardo 1 Introduction In summer 2012, OSG performed an internal review of major software components, looking for strategic weaknesses. One highlighted
Cisco Enterprise Cloud Suite Overview Cisco and/or its affiliates. All rights reserved.
Cisco Enterprise Cloud Suite Overview 2015 Cisco and/or its affiliates. All rights reserved. 1 CECS Components End User Service Catalog SERVICE PORTAL Orchestration and Management UCS Director Application
NTT Com Press Conference March 1, 2016 #enterprisecloud
NTT Com Press Conference March 1, 2016 #enterprisecloud 1 Significant Enhancement of Enterprise Cloud - Realizing Digital Transformation - NTT Communications March 1, 2016 2 NTT Communications Initiatives
Road to Private Cloud mit OpenStack Projekterfahrungen
Road to Private Cloud mit OpenStack Projekterfahrungen Andreas Kress Enterprise Architect Oracle Sales Consulting DOAG Regio Nürnberg/Franken 20. April 2017 Safe Harbor Statement The following is intended
Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
LHC2384BU VMware Cloud on AWS A Technical Deep Dive Ray Budavari @rbudavari Frank Denneman - @frankdenneman #VMworld #LHC2384BU Disclaimer This presentation may contain product features that are currently
Taming your heterogeneous cloud with Red Hat OpenShift Container Platform.
Taming your heterogeneous cloud with Red Hat OpenShift Container Platform martin@redhat.com Business Problem: Building a Hybrid Cloud solution PartyCo Some Bare Metal machines Mostly Virtualised CosPlayUK
Demystifying the Cloud With a Look at Hybrid Hosting and OpenStack
Demystifying the Cloud With a Look at Hybrid Hosting and OpenStack Robert Collazo Systems Engineer Rackspace Hosting The Rackspace Vision Agenda Truly a New Era of Computing 70 s 80 s Mainframe Era 90
OPENSTACK: THE OPEN CLOUD
OPENSTACK: THE OPEN CLOUD Anuj Sehgal (s.anuj@jacobs-university.de) AIMS 2012 Labs 04 June 2012 1 Outline What is the cloud? Background Architecture OpenStack Nova OpenStack Glance 2 What is the Cloud?
Red Hat OpenStack Platform 10 Product Guide
Red Hat OpenStack Platform 10 Product Guide Overview of Red Hat OpenStack Platform OpenStack Team Red Hat OpenStack Platform 10 Product Guide Overview of Red Hat OpenStack Platform OpenStack Team rhos-docs@redhat.com
Network Function Virtualization Using Data Plane Developer s Kit
Network Function Virtualization Using Enabling 25GbE to 100GbE Virtual Network Functions with Cavium FastLinQ Intelligent Ethernet Adapters DPDK addresses key scalability issues of NFV workloads Cavium
Building a Virtualized Desktop Grid. Eric Sedore
Building a Virtualized Desktop Grid Eric Sedore essedore@syr.edu Why create a desktop grid? One prong of an three pronged strategy to enhance research infrastructure on campus (physical hosting, HTC grid,
Using Puppet to contextualize computing resources for ATLAS analysis on Google Compute Engine
Journal of Physics: Conference Series OPEN ACCESS Using Puppet to contextualize computing resources for ATLAS analysis on Google Compute Engine To cite this article: Henrik Öhman et al 2014 J. Phys.: Conf.
Baremetal with Apache CloudStack
Baremetal with Apache CloudStack ApacheCon Europe 2016 Jaydeep Marfatia Cloud, IOT and Analytics Me Director of Product Management Cloud Products Accelerite Background Project lead for open source project
bwcloud: cross-site server virtualization
bwcloud: cross-site server virtualization O. V. Dulov Steinbuch Centre for Computing, Karlsruhe Institute of Technology Hermann-von-Helmholtz-Platz 1, Building 441 / 236, 76344 Eggenstein-Leopoldshafen,
Alteryx Technical Overview
Alteryx Technical Overview v 1.5, March 2017 2017 Alteryx, Inc. v1.5, March 2017 Page 1 Contents System Overview... 3 Alteryx Designer... 3 Alteryx Engine... 3 Alteryx Service... 5 Alteryx Scheduler...
An Introduction to Red Hat Enterprise Linux OpenStack Platform. Rhys Oxenham Field Product Manager, Red Hat
An Introduction to Red Hat Enterprise Linux OpenStack Platform Rhys Oxenham Field Product Manager, Red Hat What is OpenStack? What is OpenStack? Fully open source cloud operating system Comprised of several
CouchDB-based system for data management in a Grid environment Implementation and Experience
CouchDB-based system for data management in a Grid environment Implementation and Experience Hassen Riahi IT/SDC, CERN Outline Context Problematic and strategy System architecture Integration and deployment
Image Management for View Desktops using Mirage
Image Management for View Desktops using Mirage Mirage 5.9.0 This document supports the version of each product listed and supports all subsequent versions until the document is replaced by a new edition.
OpenIO SDS on ARM A practical and cost-effective object storage infrastructure based on SoYouStart dedicated ARM servers.
OpenIO SDS on ARM A practical and cost-effective object storage infrastructure based on SoYouStart dedicated ARM servers. Copyright 217 OpenIO SAS All Rights Reserved. Restriction on Disclosure and Use
Virtualized Scientific Research Environments and the future role of Computer Centers
Virtualized Scientific Research Environments and the future role of Computer Centers University of Freiburg, escience dept. K. Meier, B. Grüning, C. Blank, M. Janczyk, D. v. Suchodoletz 31/05/2017 10th
Batch Services at CERN: Status and Future Evolution
Batch Services at CERN: Status and Future Evolution Helge Meinhard, CERN-IT Platform and Engineering Services Group Leader HTCondor Week 20 May 2015 20-May-2015 CERN batch status and evolution - Helge
Migrating SimCloud to HP Helion
Migrating SimCloud to HP Helion Francisco Oblea, Annabelle Eseo Votaw, Sunil Kothari, Jun Zeng HP Laboratories HPL-2014-36 Keyword(s): Cloud deployment; HP Helion; OpenStack; Software as a Service Abstract:
13th International Workshop on Advanced Computing and Analysis Techniques in Physics Research ACAT 2010 Jaipur, India February
LHC Cloud Computing with CernVM Ben Segal 1 CERN 1211 Geneva 23, Switzerland E mail: b.segal@cern.ch Predrag Buncic CERN E mail: predrag.buncic@cern.ch 13th International Workshop on Advanced Computing
Data Transfers Between LHC Grid Sites Dorian Kcira
Data Transfers Between LHC Grid Sites Dorian Kcira dkcira@caltech.edu Caltech High Energy Physics Group hep.caltech.edu/cms CERN Site: LHC and the Experiments Large Hadron Collider 27 km circumference
Docker and Oracle Everything You Wanted To Know
Docker and Oracle Everything You Wanted To Know June, 2017 Umesh Tanna Principal Technology Sales Consultant Oracle Sales Consulting Centers(SCC) Bangalore Safe Harbor Statement The following is intended
CernVM-FS beyond LHC computing
CernVM-FS beyond LHC computing C Condurache, I Collier STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot, OX11 0QX, UK E-mail: catalin.condurache@stfc.ac.uk Abstract. In the last three years
A Hands on Introduction to Docker
A Hands on Introduction to Docker Len Bass A Hands on introduction Introduction to to Docker May 2017 1 4, Len 2017 Bass 2017 Len Bass 1 Setting expectations This is an introduction to Docker intended
Reimagining OpenStack*
Reimagining OpenStack* Kristen Accardi *Other names and brands may be claimed as the property of others. 1 "Evaluators of OpenStack believe that complexity and difficulty of deployment are decreasing,
OpenStack and OpenDaylight, the Evolving Relationship in Cloud Networking Charles Eckel, Open Source Developer Evangelist
OpenStack and OpenDaylight, the Evolving Relationship in Cloud Networking Charles Eckel, Open Source Developer Evangelist Agenda Introduction OpenStack OpenDaylight OPNFV Putting it all Together Conclusion
Pegasus Workflow Management System. Gideon Juve. USC Informa3on Sciences Ins3tute
Pegasus Workflow Management System Gideon Juve USC Informa3on Sciences Ins3tute Scientific Workflows Orchestrate complex, multi-stage scientific computations Often expressed as directed acyclic graphs
Building Private Cloud Infrastructure
Building Private Cloud Infrastructure Matthias Wessendorf Consulting Systems Engineer 20.11.2014 Cloud == FOG?? 3 The Path to Data Center Transformation Application- Based Silos Zones of Virtualization
Scientific Workflows and Cloud Computing. Gideon Juve USC Information Sciences Institute
Scientific Workflows and Cloud Computing Gideon Juve USC Information Sciences Institute gideon@isi.edu Scientific Workflows Loosely-coupled parallel applications Expressed as directed acyclic graphs (DAGs)
PoS(EGICF12-EMITC2)081
University of Oslo, P.b.1048 Blindern, N-0316 Oslo, Norway E-mail: aleksandr.konstantinov@fys.uio.no Martin Skou Andersen Niels Bohr Institute, Blegdamsvej 17, 2100 København Ø, Denmark E-mail: skou@nbi.ku.dk
Case Study on Enterprise Private Cloud
Case Study on Enterprise Private Cloud A TCS Case Study Veerapuram Varadhan (SUSE) Murugan N (TCS) Agenda Tata Consultancy Services Company Overview Private Cloud Challenges Present infrastructure TCS
Enable Modern Work Styles with Microsoft VDI. Jeff Chin Windows Client Solutions Specialist
Enable Modern Work Styles with Microsoft VDI Jeff Chin Windows Client Solutions Specialist Empowering People-centric IT User and Device Management Access and Information Protection Microsoft Virtual Desktop
JELASTIC PLATFORM-AS-INFRASTRUCTURE
JELASTIC PLATFORM-AS-INFRASTRUCTURE Jelastic provides enterprise cloud software that redefines the economics of cloud deployment and management. We deliver Platform-as-Infrastructure: bringing together
HTCondor Week 2015: Implementing an HTCondor service at CERN
HTCondor Week 2015: Implementing an HTCondor service at CERN Iain Steers, Jérôme Belleman, Ulrich Schwickerath IT-PES-PS HTCondor Week 2015 HTCondor at CERN 2 Outline The Move Environment Grid Pilot Local
Data Center and Cloud Automation
Data Center and Cloud Automation Tanja Hess Systems Engineer September, 2014 AGENDA Challenges and Opportunities Manual vs. Automated IT Operations What problem are we trying to solve and how do we solve
Outline. ASP 2012 Grid School
Distributed Storage Rob Quick Indiana University Slides courtesy of Derek Weitzel University of Nebraska Lincoln Outline Storage Patterns in Grid Applications Storage
File Access Optimization with the Lustre Filesystem at Florida CMS T2
Journal of Physics: Conference Series PAPER OPEN ACCESS File Access Optimization with the Lustre Filesystem at Florida CMS T2 To cite this article: P. Avery et al 215 J. Phys.: Conf. Ser. 664 4228 View
Overview of ATLAS PanDA Workload Management
Overview of ATLAS PanDA Workload Management T. Maeno 1, K. De 2, T. Wenaus 1, P. Nilsson 2, G. A. Stewart 3, R. Walker 4, A. Stradling 2, J. Caballero 1, M. Potekhin 1, D. Smith 5, for The ATLAS Collaboration
Capacity Management for Hybrid IT
Software-Defined Infrastructure Control Define Demand. Optimize Supply. Automate. Capacity Management for Hybrid IT Dan Adirim SVP, Customer Management Capacity Planning Needs to Adjust to Hybrid Cost
HyperConverged Appliance
HyperConverged Appliance DATASHEET HCA Value Proposition StarWind HyperConverged Appliance is a complete turnkey solution designed to eliminate unreasonably high complexity and cost of IT infrastructures.
Application of Virtualization Technologies & CernVM. Benedikt Hegner CERN
Application of Virtualization Technologies & CernVM Benedikt Hegner CERN Virtualization Use Cases Worker Node Virtualization Software Testing Training Platform Software Deployment }Covered today Server
Layer-4 to Layer-7 Services
Overview, page 1 Tenant Edge-Firewall, page 1 LBaaS, page 2 FWaaS, page 4 Firewall Configuration, page 6 Overview Layer-4 through Layer-7 services support(s) end-to-end communication between a source and
Molecular dynamics simulations in the MolDynGrid Virtual Laboratory by means of ARC between Grid and Cloud
Molecular dynamics simulations in the MolDynGrid Virtual Laboratory by means of ARC between Grid and Cloud Andrii Salnikov * NorduGrid 2016 * manf@grid.org.ua MolDynGrid Virtual Laboratory Has been established
Top five Docker performance tips
Top five Docker performance tips Top five Docker performance tips Table of Contents Introduction... 3 Tip 1: Design design applications as microservices... 5 Tip 2: Deployment deploy Docker components
WHY COMPOSABLE INFRASTRUCTURE INSTEAD OF HYPERCONVERGENCE
WHY COMPOSABLE INFRASTRUCTURE INSTEAD OF HYPERCONVERGENCE WHO WE ARE GOAL: Composable Infrastruture HEADQUARTERS: Toronto - Canada COMPANY FOCUS: Composable infrastructure True Software Defined Datacenter
OpenStack Architecture and Pattern Deployment with Heat. OpenStack Architecture and Pattern Deployment using Heat Ruediger Schulze
OpenStack Architecture and Pattern Deployment using Heat Ruediger Schulze Agenda 1. OpenStack Architecture 2. Pattern Deployment with Heat 3. Summary 2 What is OpenStack? At its most basic level, OpenStackis
API s in a hybrid world. Date 28 September 2017
API s in a hybrid world Date 28 September 2017 So, What is this API Economy thing? A mutually beneficial ecosystem of suppliers & consumers of APIs. Suppliers use APIs to securely provide specific types
Computing for LHC in Germany
1 Computing for LHC in Germany Günter Quast Universität Karlsruhe (TH) Meeting with RECFA Berlin, October 5th 2007 WLCG Tier1 & Tier2 Additional resources for data analysis - HGF ''Physics at the Terascale''
Monitoring of large-scale federated data storage: XRootD and beyond.
Monitoring of large-scale federated data storage: XRootD and beyond. J Andreeva 1, A Beche 1, S Belov 2, D Diguez Arias 1, D Giordano 1, D Oleynik 2, A Petrosyan 2, P Saiz 1, M Tadel 3, D Tuckett 1 and
ISTITUTO NAZIONALE DI FISICA NUCLEARE
ISTITUTO NAZIONALE DI FISICA NUCLEARE Sezione di Perugia INFN/TC-05/10 July 4, 2005 DESIGN, IMPLEMENTATION AND CONFIGURATION OF A GRID SITE WITH A PRIVATE NETWORK ARCHITECTURE Leonello Servoli 1,2!, Mirko
Minimal OpenStack Starting Your OpenStack Journey
Minimal OpenStack Starting Your OpenStack Journey Sean Dague / Aug 19th, 2015 Twitter: @sdague Blog: dague.net The Mission of OpenStack "To produce the ubiquitous Open Source Cloud Computing platform that
Application Centric Microservices Ken Owens, CTO Cisco Intercloud Services. Redhat Summit 2015
Application Centric Microservices Ken Owens, CTO Cisco Intercloud Services Redhat Summit 2015 Agenda Introduction Why Application Centric Application Deployment Options What is Microservices Infrastructure
Hands-On Workshop bwunicluster June 29th 2015
Hands-On Workshop bwunicluster June 29th 2015 Agenda Welcome Introduction to bwhpc and the bwunicluster Modules - Software Environment Management Job Submission and Monitoring Interactive Work and Remote
The 5th Pan-Galactic BOINC Workshop. Desktop Grid System
The 5th Pan-Galactic BOINC Workshop Desktop Grid System AGENDA The problem LEGION Uses in PUCP Conclusions Future prospects The problem Needs Researchers (Physicists, Mathematicians, Sociologists, etc)
1/10/2011. Topics. What is the Cloud? Cloud Computing
Cloud Computing Topics 1. What is the Cloud? 2. What is Cloud Computing? 3. Cloud Service Architectures 4. History of Cloud Computing 5. Advantages of Cloud Computing 6. Disadvantages of Cloud Computing
Pulse Secure Application Delivery
DATA SHEET Pulse Secure Application Delivery HIGHLIGHTS Provides an Application Delivery and Load Balancing solution purposebuilt for high-performance Network Functions Virtualization (NFV) Uniquely customizable,
SUSE OpenStack Cloud Production Deployment Architecture. Guide. Solution Guide Cloud Computing.
SUSE OpenStack Cloud Production Deployment Architecture Guide Solution Guide Cloud Computing Table of Contents page Introduction... 2 High Availability Configuration...6 Network Topography...8 Services
UK Tier-2 site evolution for ATLAS. Alastair Dewhurst
UK Tier-2 site evolution for ATLAS Alastair Dewhurst Introduction My understanding is that GridPP funding is only part of the story when it comes to paying for a Tier 2 site. Each site is unique. Aim to
MATLAB. Senior Application Engineer The MathWorks Korea The MathWorks, Inc. 2
1 Senior Application Engineer The MathWorks Korea 2017 The MathWorks, Inc. 2 Data Analytics Workflow Business Systems Smart Connected Systems Data Acquisition Engineering, Scientific, and Field Business
Democratizing Machine Learning on Kubernetes
Democratizing Machine Learning on Kubernetes Joy Qiao, Senior Solution Architect - AI and Research Group, Microsoft Lachlan Evenson - Principal Program Manager AKS/ACS, Microsoft Who are we? The Data Scientist
High Performance Computing from an EU perspective
High Performance Computing from an EU perspective DEISA PRACE Symposium 2010 Barcelona, 10 May 2010 Kostas Glinos European Commission - DG INFSO Head of Unit GÉANT & e-infrastructures 1 "The views expressed
Springpath Data Platform with Cisco UCS Servers
Springpath Data Platform with Cisco UCS Servers Reference Architecture SPRINGPATH DATA PLATFORM WITH CISCO UCS SERVERS Reference Architecture 1.0 Introduction to Springpath Data Platform 1 2.0 Why Springpath
Transparent Throughput Elas0city for IaaS Cloud Storage Using Guest- Side Block- Level Caching
Transparent Throughput Elas0city for IaaS Cloud Storage Using Guest- Side Block- Level Caching Bogdan Nicolae (IBM Research, Ireland) Pierre Riteau (University of Chicago, USA) Kate Keahey (Argonne National
The Cisco WebEx Node for the Cisco ASR 1000 Series Delivers the Best Aspects of On-Premises and On-Demand Web Conferencing
. White Paper The Cisco WebEx Node for the Cisco ASR 1000 Series Delivers the Best Aspects of On-Premises and On-Demand Web Conferencing Executive Summary The Cisco WebEx Node for the Cisco ASR 1000 Series,
OpenStack hypervisor, container and Baremetal servers performance comparison
OpenStack hypervisor, container and Baremetal servers performance comparison Yoji Yamato a) Software Innovation Center, NTT Corporation, 3 9 11 Midori-cho, Musashino-shi, Tokyo 180 8585, Japan a) yamato.yoji@lab.ntt.co.jp
Human Centric. Innovation. OpenStack = Linux of the Cloud? Ingo Gering, Fujitsu Dirk Müller, SUSE
OpenStack = Linux of the Cloud? Ingo Gering, Fujitsu Dirk Müller, SUSE Introduction to OpenStack Software Defined Anything (SDx) Containers and container orchestration with OpenStack Magnum Providing Bare
INFS 214: Introduction to Computing
INFS 214: Introduction to Computing Session 13 Cloud Computing Lecturer: Dr. Ebenezer Ankrah, Dept. of Information Studies Contact Information: eankrah@ug.edu.gh College of Education School of Continuing
VMware vcloud Director for Service Providers
Architecture Overview TECHNICAL WHITE PAPER Table of Contents Scope of Document....3 About VMware vcloud Director....3 Platform for Infrastructure Cloud...3 Architecture Overview....3 Constructs of vcloud
Grid Computing at the IIHE
BNC 2016 Grid Computing at the IIHE The Interuniversity Institute for High Energies S. Amary, F. Blekman, A. Boukil, O. Devroede, S. Gérard, A. Ouchene, R. Rougny, S. Rugovac, P. Vanlaer, R. Vandenbroucke
Reference Architecture
vrealize Automation 7.0.1 This document supports the version of each product listed and supports all subsequent versions until the document is replaced by a new edition. To check for more recent editions
Aneka Dynamic Provisioning
MANJRASOFT PTY LTD Aneka Aneka 3.0 Manjrasoft 05/24/2012 This document describes the dynamic provisioning features implemented in Aneka and how it is possible to leverage dynamic resources for scaling
Installation of CMSSW in the Grid DESY Computing Seminar May 17th, 2010 Wolf Behrenhoff, Christoph Wissing
Installation of CMSSW in the Grid DESY Computing Seminar May 17th, 2010 Wolf Behrenhoff, Christoph Wissing Wolf Behrenhoff, Christoph Wissing DESY Computing Seminar May 17th, 2010 Page 1 Installation of
ATLAS Distributed Computing Experience and Performance During the LHC Run-2
ATLAS Distributed Computing Experience and Performance During the LHC Run-2 A Filipčič 1 for the ATLAS Collaboration 1 Jozef Stefan Institute, Jamova 39, 1000 Ljubljana, Slovenia E-mail: andrej.filipcic@ijs.si
#techsummitch
www.thomasmaurer.ch #techsummitch Justin Incarnato Justin Incarnato Microsoft Principal PM - Azure Stack Hyper-scale Hybrid Power of Azure in your datacenter Azure Stack Enterprise-proven On-premises
Interoperating AliEn and ARC for a distributed Tier1 in the Nordic countries.
for a distributed Tier1 in the Nordic countries. Philippe Gros Lund University, Div. of Experimental High Energy Physics, Box 118, 22100 Lund, Sweden philippe.gros@hep.lu.se Anders Rhod Gregersen NDGF
vrealize Business for Cloud User Guide
vrealize Business for Cloud User Guide vrealize Business for Cloud Standard 7.3 and vrealize Business for Cloud Advanced 7.3 vrealize Business 7.3 vrealize Business for Cloud 7.3 You can find the most
Containerization Dockers / Mesospere. Arno Keller HPE
Containerization Dockers / Mesospere Arno Keller HPE What is the Container technology Hypervisor vs. Containers (Huis vs artement) A container doesn't "boot" an OS instead it loads the application and
vrealize Business Standard User Guide
User Guide 7.0 This document supports the version of each product listed and supports all subsequent versions until the document is replaced by a new edition. To check for more recent editions of this
Overview. SUSE OpenStack Cloud Monitoring
Overview SUSE OpenStack Cloud Monitoring Overview SUSE OpenStack Cloud Monitoring Publication Date: 08/04/2017 SUSE LLC 10 Canal Park Drive Suite 200 Cambridge MA 02141 USA https://www.suse.com/documentation
EASILY DEPLOY AND SCALE KUBERNETES WITH RANCHER
EASILY DEPLOY AND SCALE KUBERNETES WITH RANCHER 2 WHY KUBERNETES? Kubernetes is an open-source container orchestrator for deploying and managing containerized applications. Building on 15 years of experience
OpenStack Magnum Pike and the CERN cloud. Spyros
OpenStack Magnum Pike and the CERN cloud Spyros Trigazis @strigazi OpenStack Magnum OpenStack Magnum #openstack-containers Kubernetes, Docker Swarm, Apache Mesos, DC/OS (experimental) aas Deep integration
OpenStack Manila An Overview of Manila Liberty & Mitaka
OpenStack Manila An Overview of Manila Liberty & Mitaka PRESENTATION TITLE GOES HERE October 7, 2015 SNIA Legal Notice! The material contained in this tutorial is copyrighted by the SNIA unless otherwise
The OpenCirrus TM Project: A global Testbed for Cloud Computing R&D
The OpenCirrus TM Project: A global Testbed for Cloud Computing R&D Marcel Kunze Steinbuch Centre for Computing (SCC) Karlsruhe Institute of Technology (KIT) Germany KIT The cooperation of Forschungszentrum
Intel, OpenStack, & Trust in the Open Cloud. Intel Introduction
Intel, OpenStack, & Trust in the Open Cloud Intel Introduction 1 Intel enables OpenStack Cloud Deployments 2 Intel Contributions to OpenStack Telemetry (Ceilometer) Object Store (Swift) Erasure Code Metrics