Allowing Users to Run Services at the OLCF with Kubernetes
|
|
- Jessie Fleming
- 5 years ago
- Views:
Transcription
1 Allowing Users to Run Services at the OLCF with Kubernetes Jason Kincl Senior HPC Systems Engineer Ryan Adamson Senior HPC Security Engineer This work was supported by the Oak Ridge Leadership Computing Facility (OLCF) at Oak Ridge National Laboratory (ORNL) for the Department of Energy (DOE) under Prime Contract Number DE-AC05-00OR ORNL is managed by UT-Battelle for the US Department of Energy
2 What is the Oak Ridge Leadership Computing Facility The Oak Ridge Leadership Computing Facility is charged with helping researchers solve some of the world s most challenging scientific problems with a combination of worldclass high-performance computing (HPC) resources and world-class expertise in scientific computing The OLCF is run by the National Center for Computational Sciences (NCCS) 2 Allowing Users to Run Services at the OLCF with Kubernetes
3 HPC Operations Tasked with keeping the OLCF leadership supercomputing systems running Compute Titan, Summit Storage Lustre, GPFS, HPSS Infrastructure Lots of industry standard services like DNS, DHCP, LDAP, Internal web applications and databases Monitoring and logging 3 Allowing Users to Run Services at the OLCF with Kubernetes
4 Our Users Users bring their scientific codes and run on the supercomputer Access to cluster resources is done with a batch scheduler Batch jobs have a start and end based on wallclock Initial use cases for user-run services were around scientific workflows 4 Allowing Users to Run Services at the OLCF with Kubernetes
5 Make The Case Users We were starting to see project needs for longer running services in addition to existing batch jobs databases, data portals, web services, Security constraints at OLCF for workflows Workflows need to be structured to run locally and reach out to pull jobs Want same guarantees as batch job Runs as regular user, job file that specifies work package, access to shared filesystems (Lustre, GPFS, NFS), access to batch scheduler (qsub) and continues to run as long as user has allocation 5 Allowing Users to Run Services at the OLCF with Kubernetes
6 Basic Workflow Requirements Need ways for users to manage their workflow system Diverse ecosystem of workflow systems makes it difficult for NCCS Operations to support every one At least 211 as of today[1] Upon surveying existing workflow systems we came up with the following requirements: Run a persistent service locally as a daemon that stays up Talk to batch submission system for current queue information and job submission Interact with files on GPFS/Lustre/NFS [1] 6 Allowing Users to Run Services at the OLCF with Kubernetes
7 Make the Case Staff New service requests can take a long time I have X application that we wrote, how can I get Ops to run it as a service? Wouldn t it be great if we offered X to our users? Lots of steps involved in standing up a new production operational service Even if we only shift the initial burden of standing up and testing service over to user for prototyping, still a big win 7 Allowing Users to Run Services at the OLCF with Kubernetes
8 Containers Exist only in the kernel Just cgroups and kernel namespaces (process, network, IPC, ) Unix processes, not lightweight virtual machines Root filesystem of container is an image image = application + dependencies Stateless, every time a container starts its in the state it was created 8 Allowing Users to Run Services at the OLCF with Kubernetes
9 Multiple Container Strategies in HPC Automate deploying, scaling, and operating application containers with Kubernetes Focused on framework for providing resources (cpu, memory, network, ) for running services and applications Uses own scheduler Helps users create and run persistent services HPC container runtimes with Singularity/Shifter Focused on using containers in a batch job Uses scheduler from batch job submission system Provides portable environment to our users for HPC resources Easier to run software that needs new libraries on outdated HPC resources 9 Allowing Users to Run Services at the OLCF with Kubernetes
10 Why not a VM infrastructure? Virtual machines are very powerful isolation abstractions, running entirely different operating systems completely isolated from host Isolation requires user to run all services related to running an operating system and manage those configurations (access and authentication, monitoring and logging, integration with other systems) Containers are simply processes with cgroups and namespaces which run in same kernel as host Generally our users don t require that level of isolation, just want to be able to run their application in userspace 10 Allowing Users to Run Services at the OLCF with Kubernetes
11 Platform Create a layer between the infrastructure and the application. This layer would manage the infrastructure resources and ensure applications are running as intended. It would provide a fully isolated container for each application to run independently of any other application running on the infrastructure. 11 Allowing Users to Run Services at the OLCF with Kubernetes
12 Kubernetes Kubernetes manages containerized applications across nodes and provides mechanisms for deployment, maintenance, and application-scaling. User self-service for allocating CPU, memory, data volumes just like batch scheduling It provides a common platform that is flexible enough for running ops and user services 12 Allowing Users to Run Services at the OLCF with Kubernetes
13 Kubernetes Architecture Configuration: YAML or JSON data that describes the application being deployed Configuration can define: Containers to run HTTP routes and network ports to expose outside of the cluster Mounting data volumes 13 Allowing Users to Run Services at the OLCF with Kubernetes
14 Kubernetes Pods Atomic unit of Kubernetes Made up of one or more containers deployed together on one host Pod lifecycle is defined, pod is assigned to run on a node and runs until the container(s) exit or it is removed for some other reason Volumes can be attached that do not share pod lifecycle for persistent data Each pod gets its own IP address that is accessible in the cluster 14 Allowing Users to Run Services at the OLCF with Kubernetes
15 Kubernetes Scheduling When a pod object is created, the scheduler is responsible for assigning the pod to a node in the cluster Pod is filtered through a configurable number of predicates to select the right node based on pod configuration and node status 15 Allowing Users to Run Services at the OLCF with Kubernetes
16 Kubernetes Replication Controllers Pod will not recreate itself if deleted for some reason such as cluster maintenance or quota limit exceeded A ReplicationController ensures desired number of pods is running in the cluster ex. thermostat in a room For example: I want to have three pods running nginx:1.10 image 16 Allowing Users to Run Services at the OLCF with Kubernetes
17 Kubernetes Services Service points to where application pods are running in the cluster Services get static cluster IP and DNS Can be implemented with type= NodePort or LoadBalancer for external connectivity 17 Allowing Users to Run Services at the OLCF with Kubernetes
18 Kubernetes Persistent Volumes Store stateful data Lifespan of data in volume is independent of lifespan of container Can be backed by many different options NFS Lustre/GPFS Host Disks (bind mount) 18 Allowing Users to Run Services at the OLCF with Kubernetes
19 Desired State and Implementation of Actual State The real power of offering Kubernetes as a service is in the implementation of actual state Ex. user requests X amount of storage and Kubernetes satisfies it with Y storage controller User does not need to know about topology of storage network, Kubernetes handles that 19 Allowing Users to Run Services at the OLCF with Kubernetes
20 Declarative vs Imperative Declarative Focuses on what Describes what needs to happen, how is left to system run two copies of this with <= 1 being down at any one time Imperative Focuses on how Explicitly state how to do something with expectation that desired outcome will result start this process on that server 20 Allowing Users to Run Services at the OLCF with Kubernetes
21 Flexible control over what users can request Processes in container run as a regular user (not root) Capabilities are stripped from process before it starts setuid sudo binary runs without setuid Most of kernel is namespace-aware but pieces that are not cannot be used from inside container All of these are configurable! 21 Allowing Users to Run Services at the OLCF with Kubernetes
22 Cluster Resources Resource allocation is different from the traditional core hours or node hours we use in HPC Quota system based on CPU and memory limits User defines what CPU and memory are required for each container, if container exceeds limits it is killed 22 Allowing Users to Run Services at the OLCF with Kubernetes
23 Exposing services OpenShift gives users the ability to expose services outside of the cluster For HTTP-based services, NCCS will handle initial authentication to ensure service is accessed only by members of that project 23 Allowing Users to Run Services at the OLCF with Kubernetes
24 Accessing NCCS resources All containers run as an automation user that is tied to a project and has access to the project s allocation and files like a regular user Batch job submission from container Users can base their container image off our NCCS golden image which comes with the tools to schedule batch jobs or get queue status Accessing shared filesystems (GPFS/Lustre/NFS) Shared filesystems can be mounted in the container by Kubernetes allowing access just like a login or compute node 24 Allowing Users to Run Services at the OLCF with Kubernetes
25 HPC Workflow Support Requirements Run a persistent service locally DONE: Kubernetes can run user services in NCCS Talk to batch submission system for current queue information and job submission DONE: Containers running on Kubernetes cluster in NCCS can run qsub/mshow commands to talk to Titan/Rhea/DTN cluster Moab Interact with files on GPFS/Lustre/NFS DONE: Containers running on Kubernetes cluster in NCCS can mount Lustre and NFS project and home areas 25 Allowing Users to Run Services at the OLCF with Kubernetes
26 NCCS Kubernetes Clusters Clusters are running Red Hat OpenShift distribution and are split by security domain Granite Cluster Ops cluster in our core services security enclave Built for Ops and Staff managed applications Some applications can run as root in the container Marble Cluster User-facing cluster in our moderate security enclave Integration in moderate enclave: Lustre mounted in container NFS home and project areas mounted in container Torque/Moab job submission in container All applications run as project automation user in the container Onyx Cluster Will be user-facing cluster in our open security enclave 26 Allowing Users to Run Services at the OLCF with Kubernetes
27 Questions? Jason Kincl Ryan Adamson
TEN LAYERS OF CONTAINER SECURITY
TEN LAYERS OF CONTAINER SECURITY Tim Hunt Kirsten Newcomer May 2017 ABOUT YOU Are you using containers? What s your role? Security professionals Developers / Architects Infrastructure / Ops Who considers
More informationRed Hat Atomic Details Dockah, Dockah, Dockah! Containerization as a shift of paradigm for the GNU/Linux OS
Red Hat Atomic Details Dockah, Dockah, Dockah! Containerization as a shift of paradigm for the GNU/Linux OS Daniel Riek Sr. Director Systems Design & Engineering In the beginning there was Stow... and
More informationTEN LAYERS OF CONTAINER SECURITY. Kirsten Newcomer Security Strategist
TEN LAYERS OF CONTAINER SECURITY Kirsten Newcomer Security Strategist WHAT ARE CONTAINERS? Containers change how we develop, deploy and manage applications INFRASTRUCTURE Sandboxed application processes
More informationOpenShift 3 Technical Architecture. Clayton Coleman, Dan McPherson Lead Engineers
OpenShift 3 Technical Architecture Clayton Coleman, Dan McPherson Lead Engineers Principles The future of *aas Redefine the Application Networked components wired together Not just a web frontend anymore
More informationKubernetes introduction. Container orchestration
Kubernetes introduction Container orchestration Container Orchestration Why we need container orchestration? Restart containers if they are not healthy. Provide private container network. Service discovery.
More informationInvestigating Containers for Future Services and User Application Support
Investigating Containers for Future Services and User Application Support JLAB CNI NLIT 2018 () Overview JLAB scope What is a container? Why are we interested? Platform-as-a-Service (PaaS) for orchestration
More informationCode: Slides:
Workshop Resources Code: https://github.com/beekpr/public-workshops Slides: https://tinyurl.com/yc2uo3wk Make sure minikube and kubectl is setup (labs/1-setup-cluster.md has some instructions) Kubernetes
More informationPreparing GPU-Accelerated Applications for the Summit Supercomputer
Preparing GPU-Accelerated Applications for the Summit Supercomputer Fernanda Foertter HPC User Assistance Group Training Lead foertterfs@ornl.gov This research used resources of the Oak Ridge Leadership
More informationIntroduction to HPC Parallel I/O
Introduction to HPC Parallel I/O Feiyi Wang (Ph.D.) and Sarp Oral (Ph.D.) Technology Integration Group Oak Ridge Leadership Computing ORNL is managed by UT-Battelle for the US Department of Energy Outline
More informationHPC Saudi Jeffrey A. Nichols Associate Laboratory Director Computing and Computational Sciences. Presented to: March 14, 2017
Creating an Exascale Ecosystem for Science Presented to: HPC Saudi 2017 Jeffrey A. Nichols Associate Laboratory Director Computing and Computational Sciences March 14, 2017 ORNL is managed by UT-Battelle
More informationSTATUS OF PLANS TO USE CONTAINERS IN THE WORLDWIDE LHC COMPUTING GRID
The WLCG Motivation and benefits Container engines Experiments status and plans Security considerations Summary and outlook STATUS OF PLANS TO USE CONTAINERS IN THE WORLDWIDE LHC COMPUTING GRID SWISS EXPERIENCE
More informationMicroservices. Chaos Kontrolle mit Kubernetes. Robert Kubis - Developer Advocate,
Microservices Chaos Kontrolle mit Kubernetes Robert Kubis - Developer Advocate, Google @hostirosti About me Robert Kubis Developer Advocate Google Cloud Platform London, UK hostirosti github.com/hostirosti
More informationOverview of Container Management
Overview of Container Management Wyn Van Devanter @wynv Vic Kumar Agenda Why Container Management? What is Container Management? Clusters, Cloud Architecture & Containers Container Orchestration Tool Overview
More informationImportant DevOps Technologies (3+2+3days) for Deployment
Important DevOps Technologies (3+2+3days) for Deployment DevOps is the blending of tasks performed by a company's application development and systems operations teams. The term DevOps is being used in
More informationContainer-Native Storage
Container-Native Storage Solving the Persistent Storage Challenge with GlusterFS Michael Adam Manager, Software Engineering José A. Rivera Senior Software Engineer 2017.09.11 WARNING The following presentation
More informationRed Hat OpenShift Roadmap Q4 CY16 and H1 CY17 Releases. Lutz Lange Solution
Red Hat OpenShift Roadmap Q4 CY16 and H1 CY17 Releases Lutz Lange Solution Architect @AtomicContainer OpenShift Roadmap OpenShift Container Platform 3.2 Kubernetes 1.2 & Docker 1.9
More informationA More Realistic Way of Stressing the End-to-end I/O System
A More Realistic Way of Stressing the End-to-end I/O System Verónica G. Vergara Larrea Sarp Oral Dustin Leverman Hai Ah Nam Feiyi Wang James Simmons CUG 2015 April 29, 2015 Chicago, IL ORNL is managed
More informationSoftware containers are likely to become a very important tool over the
MARK LAMOURINE Mark Lamourine is a senior software developer at Red Hat. He s worked for the last few years on the OpenShift project. He s a coder by training, a sysadmin and toolsmith by trade, and an
More information@briandorsey #kubernetes #GOTOber
1 Kubernetes Changing the way we think and talk about computing GOTO Berlin - December 2015 2 What is this talk? Container 3 Brian Dorsey Developer Advocate - Google Cloud platform +BrianDorsey @briandorsey
More informationCONTAINERS AND MICROSERVICES WITH CONTRAIL
CONTAINERS AND MICROSERVICES WITH CONTRAIL Scott Sneddon Sree Sarva DP Ayyadevara Sr. Director Sr. Director Director Cloud and SDN Contrail Solutions Product Line Management This statement of direction
More informationWHITE PAPER. RedHat OpenShift Container Platform. Benefits: Abstract. 1.1 Introduction
WHITE PAPER RedHat OpenShift Container Platform Abstract Benefits: Applications are designed around smaller independent components called microservices. Elastic resources: Scale up or down quickly and
More informationIntroduction to Kubernetes
Introduction to Kubernetes Neil Peterson @nepeters #ITDEVCONNECTIONS Session Topics - Quick primer on containers - Container mgmt solutions - Kubernetes basics - Kubernetes deeper dive - Kubernetes beyond
More informationAmir Zipory Senior Solutions Architect, Redhat Israel, Greece & Cyprus
Amir Zipory Senior Solutions Architect, Redhat Israel, Greece & Cyprus amirz@redhat.com TODAY'S IT CHALLENGES IT is under tremendous pressure from the organization to enable growth Need to accelerate,
More informationKubernetes 101: Pods, Nodes, Containers, andclusters
Kubernetes 101: Pods, Nodes, Containers, andclusters Kubernetes is quickly becoming the new standard for deploying and managing software in the cloud. With all the power Kubernetes provides, however, comes
More informationOracle Linux 5 & 6 Advanced Administration
Oracle University Contact Us: +27 (0)11 319-4111 Oracle Linux 5 & 6 Advanced Administration Duration: 5 Days What you will learn This Oracle Linux Advanced Administration training is ideal for experienced
More informationHigh Performance Containers. Convergence of Hyperscale, Big Data and Big Compute
High Performance Containers Convergence of Hyperscale, Big Data and Big Compute Christian Kniep Technical Account Manager, Docker Brief Recap of Container Technology Brief History of Container Technology
More informationConvergence of VM and containers orchestration using KubeVirt. Chunfu Wen
Convergence of VM and containers orchestration using KubeVirt Chunfu Wen chwen@redhat.com Agenda Context Introduction What Is Kubevirt And How It Feel Kubevirt Architecture And Design Demo FIRST A LITTLE
More informationDocker und IBM Digital Experience in Docker Container
Docker und IBM Digital Experience in Docker Container 20. 21. Juni 2017 IBM Labor Böblingen 1 What is docker Introduction VMs vs. containers Terminology v Docker components 2 6/22/2017 What is docker?
More informationRun containerized applications from pre-existing images stored in a centralized registry
Introduction This examination is based upon the most critical job activities a Docker Certified Associate performs. The skills and knowledge certified by this examination represent a level of expertise
More informationBackup strategies for Stateful Containers in OpenShift Using Gluster based Container-Native Storage
Backup strategies for Stateful Containers in OpenShift Using Gluster based Container-Native Storage Niels de Vos Senior Software Engineer Red Hat Storage Critical features for both Dev and Ops Self-Service
More informationHTCondor on Titan. Wisconsin IceCube Particle Astrophysics Center. Vladimir Brik. HTCondor Week May 2018
HTCondor on Titan Wisconsin IceCube Particle Astrophysics Center Vladimir Brik HTCondor Week May 2018 Overview of Titan Cray XK7 Supercomputer at Oak Ridge Leadership Computing Facility Ranked #5 by TOP500
More informationViryaOS RFC: Secure Containers for Embedded and IoT. A proposal for a new Xen Project sub-project
ViryaOS RFC: Secure Containers for Embedded and IoT A proposal for a new Xen Project sub-project Stefano Stabellini @stabellinist The problem Package applications for the target Contain all dependencies
More informationTHE STATE OF CONTAINERS
THE STATE OF CONTAINERS Engines & Runtimes in RHEL & OpenShift Scott McCarty Principal Technology Product Manager - Containers 10/15/2018 What if... I told you there is container innovation happening in
More informationRoot cause codes: Level One: See Chapter 6 for a discussion of using hierarchical cause codes.
Root cause codes: See Chapter 6 for a discussion of using hierarchical cause codes. Level one is where the cause is found, levels two and three are what the cause is, in increasing detail. While many level
More informationSimple custom Linux distributions with LinuxKit. Justin Cormack
Simple custom Linux distributions with LinuxKit Justin Cormack Who am I? Engineer at Docker in Cambridge, UK. @justincormack 3 Tools for building custom Linux Tools for building custom Linux Existing
More informationKubernetes 101. Doug Davis, STSM September, 2017
Kubernetes 101 Doug Davis, STSM September, 2017 Today's Agenda What is Kubernetes? How was Kubernetes created? Where is the Kubernetes community? Technical overview What's the current status of Kubernetes?
More informationKubernetes Integration with Virtuozzo Storage
Kubernetes Integration with Virtuozzo Storage A Technical OCTOBER, 2017 2017 Virtuozzo. All rights reserved. 1 Application Container Storage Application containers appear to be the perfect tool for supporting
More informationHacking and Hardening Kubernetes
SESSION ID: HT-W02 Hacking and Hardening Kubernetes Jay Beale CTO InGuardians, Inc @jaybeale and @inguardians Adam Crompton Senior Security Analyst InGuardians, Inc. @3nc0d3r and @inguardians Table of
More informationKubernetes The Path to Cloud Native
Kubernetes The Path to Cloud Native Eric Brewer VP, Infrastructure @eric_brewer August 28, 2015 ACM SOCC Cloud Na*ve Applica*ons Middle of a great transition unlimited ethereal resources in the Cloud an
More informationOPENSHIFT FOR OPERATIONS. Jamie Cloud Guy - US Public Sector at Red Hat
1 OPENSHIFT FOR OPERATIONS Jamie Duncan @jamieeduncan Cloud Guy - US Public Sector at Red Hat 20170504 ABOUT JDUNCAN I've been at Red Hat just over 5 years 2 This is my daughter Elizabeth. #cutestthingever
More informationBuilding Kubernetes cloud: real world deployment examples, challenges and approaches. Alena Prokharchyk, Rancher Labs
Building Kubernetes cloud: real world deployment examples, challenges and approaches Alena Prokharchyk, Rancher Labs Making a right choice is not easy The illustrated children guide to Kubernetes https://www.youtube.com/watch?v=4ht22rebjno
More informationDesigning MQ deployments for the cloud generation
Designing MQ deployments for the cloud generation WebSphere User Group, London Arthur Barr, Senior Software Engineer, IBM MQ 30 th March 2017 Top business drivers for cloud 2 Source: OpenStack user survey,
More informationAn introduction to Docker
An introduction to Docker Ing. Vincenzo Maffione Operating Systems Security Container technologies on Linux Several light virtualization technologies are available for Linux They build on cgroups, namespaces
More informationCloud & container monitoring , Lars Michelsen Check_MK Conference #4
Cloud & container monitoring 04.05.2018, Lars Michelsen Some cloud definitions Applications Data Runtime Middleware O/S Virtualization Servers Storage Networking Software-as-a-Service (SaaS) Applications
More informationVMware Integrated OpenStack with Kubernetes Getting Started Guide. VMware Integrated OpenStack 4.0
VMware Integrated OpenStack with Kubernetes Getting Started Guide VMware Integrated OpenStack 4.0 VMware Integrated OpenStack with Kubernetes Getting Started Guide You can find the most up-to-date technical
More informationPerformance Monitoring and Management of Microservices on Docker Ecosystem
Performance Monitoring and Management of Microservices on Docker Ecosystem Sushanta Mahapatra Sr.Software Specialist Performance Engineering SAS R&D India Pvt. Ltd. Pune Sushanta.Mahapatra@sas.com Richa
More informationRunning MarkLogic in Containers (Both Docker and Kubernetes)
Running MarkLogic in Containers (Both Docker and Kubernetes) Emma Liu Product Manager, MarkLogic Vitaly Korolev Staff QA Engineer, MarkLogic @vitaly_korolev 4 June 2018 MARKLOGIC CORPORATION Source: http://turnoff.us/image/en/tech-adoption.png
More informationWindows Azure Services - At Different Levels
Windows Azure Windows Azure Services - At Different Levels SaaS eg : MS Office 365 Paas eg : Azure SQL Database, Azure websites, Azure Content Delivery Network (CDN), Azure BizTalk Services, and Azure
More informationISLET: Jon Schipp, AIDE jonschipp.com. An Attempt to Improve Linux-based Software Training
ISLET: An Attempt to Improve Linux-based Software Training Jon Schipp, AIDE 2015 jonschipp@gmail.com, @Jonschipp, jonschipp.com About me: Security Engineer for the National Center for Supercomputing Applications
More informationA Container On a Virtual Machine On an HPC? Presentation to HPC Advisory Council. Perth, July 31-Aug 01, 2017
A Container On a Virtual Machine On an HPC? Presentation to HPC Advisory Council Perth, July 31-Aug 01, 2017 http://levlafayette.com Necessary and Sufficient Definitions High Performance Computing: High
More informationBright Cluster Manager
Bright Cluster Manager Using Slurm for Data Aware Scheduling in the Cloud Martijn de Vries CTO About Bright Computing Bright Computing 1. Develops and supports Bright Cluster Manager for HPC systems, server
More informationSUG Breakout Session: OSC OnDemand App Development
SUG Breakout Session: OSC OnDemand App Development Basil Mohamed Gohar Web and Interface Applications Manager Eric Franz Senior Engineer & Technical Lead This work is supported by the National Science
More informationThink Small to Scale Big
Think Small to Scale Big Intro to Containers for the Datacenter Admin Pete Zerger Principal Program Manager, MVP pete.zerger@cireson.com Cireson Lee Berg Blog, e-mail address, title Company Pete Zerger
More informationDocker A FRAMEWORK FOR DATA INTENSIVE COMPUTING
Docker A FRAMEWORK FOR DATA INTENSIVE COMPUTING Agenda Intro / Prep Environments Day 1: Docker Deep Dive Day 2: Kubernetes Deep Dive Day 3: Advanced Kubernetes: Concepts, Management, Middleware Day 4:
More informationLinux Containers Roadmap Red Hat Enterprise Linux 7 RC. Bhavna Sarathy Senior Technology Product Manager, Red Hat
Linux Containers Roadmap Red Hat Enterprise Linux 7 RC Bhavna Sarathy Senior Technology Product Manager, Red Hat Linda Wang Senior Eng. Manager, Red Hat Bob Kozdemba Principal Soln. Architect, Red Hat
More informationIntroduction to Containers
Introduction to Containers Shawfeng Dong Principal Cyberinfrastructure Engineer University of California, Santa Cruz What are Containers? Containerization, aka operating-system-level virtualization, refers
More informationLearn. Connect. Explore.
Learn. Connect. Explore. No More Storage Nightmares An Open Solution for Container Persistent Storage Learn. Connect. Explore. CONTAINERS vs VIRTUALIZATION Containers Abstracts OS Kernel Mostly Linux One
More informationThe four forces of Cloud Native
1 Aplicaciones Nativas En La Nube The four forces of Cloud Native Iñaki Borrero- Advisory vspecialist MEDI, Dell EMC @DellEMCForum 2 Containers Devops Confluence of 4 forces Agile 3 3 Microservices 4 4
More informationComparison of Scheduling Policies and Workloads on the NCCS and NICS XT4 Systems at Oak Ridge National Laboratory
Comparison of Scheduling Policies and Workloads on the NCCS and NICS XT4 Systems at Oak Ridge National Laboratory Troy Baer HPC System Administrator National Institute for Computational Sciences University
More informationScalable, Automated Characterization of Parallel Application Communication Behavior
Scalable, Automated Characterization of Parallel Application Communication Behavior Philip C. Roth Computer Science and Mathematics Division Oak Ridge National Laboratory 12 th Scalable Tools Workshop
More informationAn Introduction to Kubernetes
8.10.2016 An Introduction to Kubernetes Premys Kafka premysl.kafka@hpe.com kafkapre https://github.com/kafkapre { History }???? - Virtual Machines 2008 - Linux containers (LXC) 2013 - Docker 2013 - CoreOS
More information/ Cloud Computing. Recitation 5 February 14th, 2017
15-319 / 15-619 Cloud Computing Recitation 5 February 14th, 2017 1 Overview Administrative issues Office Hours, Piazza guidelines Last week s reflection Project 2.1, OLI Unit 2 modules 5 and 6 This week
More informationWhat s New in K8s 1.3
What s New in K8s 1.3 Carter Morgan Background: 3 Hurdles How do I write scalable apps? The App How do I package and distribute? What runtimes am I locked into? Can I scale? The Infra Is it automatic?
More informationS Implementing DevOps and Hybrid Cloud
S- Implementing DevOps and Hybrid Cloud Srihari Angaluri Lenovo Data Center Group Red Hat Summit // Outline DevOps and Containers Architectural Considerations Lenovo Cloud Technology Center Implementing
More informationSingularity: Containers for High-Performance Computing. Grigory Shamov Nov 21, 2017
Singularity: Containers for High-Performance Computing Grigory Shamov Nov 21, 2017 Outline Software and High Performance Computing: Installation/Maintenance of the HPC Software stack Why containers and
More informationIBM Spectrum Scale IO performance
IBM Spectrum Scale 5.0.0 IO performance Silverton Consulting, Inc. StorInt Briefing 2 Introduction High-performance computing (HPC) and scientific computing are in a constant state of transition. Artificial
More informationUtilizing Databases in Grid Engine 6.0
Utilizing Databases in Grid Engine 6.0 Joachim Gabler Software Engineer Sun Microsystems http://sun.com/grid Current status flat file spooling binary format for jobs ASCII format for other objects accounting
More information[Docker] Containerization
[Docker] Containerization ABCD-LMA Working Group Will Kinard October 12, 2017 WILL Kinard Infrastructure Architect Software Developer Startup Venture IC Husband Father Clemson University That s me. 2 The
More informationTRAINING AND CERTIFICATION UPDATE
TRAINING AND CERTIFICATION UPDATE Red Hat Enterprise User Group Twin Cities Steve Bonneville Manager, Curriculum Development / Red Hat November 11, 2015 RED HAT LEARNING SUBSCRIPTION One year access to
More informationOpenShift + Container Native Storage (CNS)
OpenShift + Container Native Storage (CNS) 31 May 2017 Michael Holzerland, Solution Architect OpenShift supports Persistent Storage GlusterFS Amazon EBS Azure Disk AUTOMATED CONFIGURATION SINGLE CONTROL
More informationDocker All The Things
OpenStack Services Docker All The Things and Kubernetes and Atomic OpenStack Summit Paris, November 2014 @brentholden @jameslabocki Agenda The Problem Current Solutions Tomorrow s Improvements Demonstration
More informationVMWARE PIVOTAL CONTAINER SERVICE
DATASHEET VMWARE PIVOTAL CONTAINER SERVICE AT A GLANCE VMware Pivotal Container Service (PKS) is a production-grade Kubernetes-based container solution equipped with advanced networking, a private container
More informationGenomics on Cisco Metacloud + SwiftStack
Genomics on Cisco Metacloud + SwiftStack Technology is a large component of driving discovery in both research and providing timely answers for clinical treatments. Advances in genomic sequencing have
More informationReal World CI with Red Hat Cloud Suite. Sim Zacks - Principal Quality Engineer Oded Ramraz - Manager QE Ops TLV
Real World CI with Red Hat Cloud Suite Sim Zacks - Principal Quality Engineer Oded Ramraz - Manager QE Ops TLV 29.6.2016 Who we are DevOps Engineers - experts in software engineering and sys admin QE Ops
More informationContainers, Serverless and Functions in a nutshell. Eugene Fedorenko
Containers, Serverless and Functions in a nutshell Eugene Fedorenko About me Eugene Fedorenko Senior Architect Flexagon adfpractice-fedor.blogspot.com @fisbudo Agenda Containers Microservices Docker Kubernetes
More informationTOSS - A RHEL-based Operating System for HPC Clusters
TOSS - A RHEL-based Operating System for HPC Clusters Supercomputing 2017 Red Hat Booth November 14, 2017 Ned Bass System Software Development Group Leader Livermore Computing Division LLNL-PRES-741473
More informationS INSIDE NVIDIA GPU CLOUD DEEP LEARNING FRAMEWORK CONTAINERS
S8497 - INSIDE NVIDIA GPU CLOUD DEEP LEARNING FRAMEWORK CONTAINERS Chris Lamb CUDA and NGC Engineering, NVIDIA John Barco NGC Product Management, NVIDIA NVIDIA GPU Cloud (NGC) overview AGENDA Using NGC
More informationX Grid Engine. Where X stands for Oracle Univa Open Son of more to come...?!?
X Grid Engine Where X stands for Oracle Univa Open Son of more to come...?!? Carsten Preuss on behalf of Scientific Computing High Performance Computing Scheduler candidates LSF too expensive PBS / Torque
More informationContainer Adoption for NFV Challenges & Opportunities. Sriram Natarajan, T-Labs Silicon Valley Innovation Center
Container Adoption for NFV Challenges & Opportunities Sriram Natarajan, T-Labs Silicon Valley Innovation Center Virtual Machine vs. Container Stack KVM Container-stack Libraries Guest-OS Hypervisor Libraries
More informationSAMPLE CHAPTER. Marko Lukša MANNING
SAMPLE CHAPTER Marko Lukša MANNING Kubernetes in Action by Marko Lukša Chapter 1 Copyright 2018 Manning Publications brief contents PART 1 OVERVIEW 1 Introducing Kubernetes 1 2 First steps with Docker
More informationRED HAT GLUSTER TECHSESSION CONTAINER NATIVE STORAGE OPENSHIFT + RHGS. MARCEL HERGAARDEN SR. SOLUTION ARCHITECT, RED HAT BENELUX April 2017
RED HAT GLUSTER TECHSESSION CONTAINER NATIVE STORAGE OPENSHIFT + RHGS MARCEL HERGAARDEN SR. SOLUTION ARCHITECT, RED HAT BENELUX April 2017 AGENDA Why OpenShift? The Journey So Far for OpenShift Storage
More informationToward Improved Support for Loosely Coupled Large Scale Simulation Workflows. Swen Boehm Wael Elwasif Thomas Naughton, Geoffroy R.
Toward Improved Support for Loosely Coupled Large Scale Simulation Workflows Swen Boehm Wael Elwasif Thomas Naughton, Geoffroy R. Vallee Motivation & Challenges Bigger machines (e.g., TITAN, upcoming Exascale
More informationContainer Management : First Looks
Container Management : First Looks John Hardy Senior Principal Product Manager jhardy@redhat.com 25th June 2015 Itamar Heim Senior Director, Software Engineering itamar@redhat.com Disclaimer This information
More informationWhat s New in K8s 1.3
What s New in K8s 1.3 Carter Morgan Background: 3 Hurdles How do I write scalable apps? The App How do I package and distribute? What runtimes am I locked into? Can I scale? The Infra Is it automatic?
More informationA REFERENCE ARCHITECTURE FOR DEPLOYING WSO2 MIDDLEWARE ON KUBERNETES
A REFERENCE ARCHITECTURE FOR DEPLOYING WSO2 BY FRANK LEYMANN MIDDLEWARE ON KUBERNETES BY IMESH GUNARATNE SENIOR TECHNICAL LEAD, WSO2 WSO2 REST API DESIGN GUIDELINES TABLE OF CONTENTS 1. An Introduction
More informationVMWARE PKS. What is VMware PKS? VMware PKS Architecture DATASHEET
DATASHEET VMWARE PKS AT A GLANCE VMware PKS is a production-grade Kubernetes-based container solution equipped with advanced networking, a private container registry, and full lifecycle management. VMware
More informationRDMA Container Support. Liran Liss Mellanox Technologies
RDMA Container Support Liran Liss Mellanox Technologies Agenda Containers 101 RDMA isolation Namespace support Controller support Putting it all together Status Conclusions March 15 18, 2015 #OFADevWorkshop
More informationWrite a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical
Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or
More informationPresented By: Gregory M. Kurtzer HPC Systems Architect Lawrence Berkeley National Laboratory CONTAINERS IN HPC WITH SINGULARITY
Presented By: Gregory M. Kurtzer HPC Systems Architect Lawrence Berkeley National Laboratory gmkurtzer@lbl.gov CONTAINERS IN HPC WITH SINGULARITY A QUICK REVIEW OF THE LANDSCAPE Many types of virtualization
More informationContainer Pods with Docker Compose in Apache Mesos
Container Pods with Docker Compose in Apache Mesos 1 Summary Goals: 1. Treating Apache Mesos and docker as first class citizens, the platform needs to seamlessly run and scale docker container pods in
More informationIntroduction to Container Technology. Patrick Ladd Technical Account Manager April 13, 2016
Introduction to Container Technology Patrick Ladd Technical Account Manager April 13, 2016 Container Technology Containers 3 "Linux Containers" is a Linux kernel feature to contain a group of processes
More informationTable of Contents 1.1. Introduction. Overview of vsphere Integrated Containers 1.2
Table of Contents Introduction Overview of vsphere Integrated Containers 1.1 1.2 2 Overview of vsphere Integrated Containers This document provides an overview of VMware vsphere Integrated Containers.
More informationGPFS for Life Sciences at NERSC
GPFS for Life Sciences at NERSC A NERSC & JGI collaborative effort Jason Hick, Rei Lee, Ravi Cheema, and Kjiersten Fagnan GPFS User Group meeting May 20, 2015-1 - Overview of Bioinformatics - 2 - A High-level
More informationOS Security III: Sandbox and SFI
1 OS Security III: Sandbox and SFI Chengyu Song Slides modified from Dawn Song 2 Administrivia Lab2 VMs on lab machine Extension? 3 Users and processes FACT: although ACLs use users as subject, the OS
More informationAirship A New Open Infrastructure Project for OpenStack
Airship A New Open Infrastructure Project for OpenStack Declaratively define your OpenStack & Kubernetes Infrastructure Airship A New Open Infrastructure Project for OpenStack Declaratively define your
More informationSIGHT. Benjamin Hernandez, PhD Advanced Data and Workflow(s) Group
SIGHT Benjamin Hernandez, PhD Advanced Data and Workflow(s) Group hernandezarb@ornl.gov ORNL is managed by UT-Battelle for the US Department of Energy name 1 Presentation This research used resources of
More informationMQ High Availability and Disaster Recovery Implementation scenarios
MQ High Availability and Disaster Recovery Implementation scenarios Sandeep Chellingi Head of Hybrid Cloud Integration Prolifics Agenda MQ Availability Message Availability Service Availability HA vs DR
More informationTHE AFS NAMESPACE AND CONTAINERS
THE AFS NAMESPACE AND CONTAINERS A SECURE DISTRIBUTED FILESYSTEM APPROACH TO SECURELY PROVIDE PERSISTENCE TO LINUX CONTAINERS LINUX AF_RXRPC AND KERNEL AFS BY DAVID HOWELLS (DHOWELLS@REDHAT.COM) DEMOS
More informationINTRODUCING CONTAINER-NATIVE VIRTUALIZATION
INTRODUCING CONTAINER-NATIVE VIRTUALIZATION Cats and Dogs Living Together Stephen Gordon Principal Product Manager Red Hat Fabian Deutsch Manager, Software Engineering Red Hat sgordon@redhat.com / @xsgordon
More informationBeyond 1001 Dedicated Data Service Instances
Beyond 1001 Dedicated Data Service Instances Introduction The Challenge Given: Application platform based on Cloud Foundry to serve thousands of apps Application Runtime Many platform users - who don
More information