Linux-CR: Transparent Application Checkpoint-Restart in Linux
|
|
|
- Brendan Wood
- 6 years ago
- Views:
Transcription
1 Linux-CR: Transparent Application Checkpoint-Restart in Linux Oren Laadan Columbia University Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
2 Application C/R Application Checkpoint/Restart a mechanism to save the state of running application(s) so that they can later resume execution from that point Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
3 Application C/R checkpoint restart checkpoint image original restored hierarchy hierarchy Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
4 What is it good for? Application roll back to the past Application suspend and resume Application migration Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
5 Application rollback Fault tolerance Effective debugging Fast application start-up Software testing Generic time-machine Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
6 Application rollback Fault tolerance long running applications cloud, HPC, at work, at home Effective debugging Fast application start-up Software testing Generic time-machine Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
7 Application rollback Fault tolerance Effective debugging Super-core-dump more details, multiple tasks re-run from checkpoint trace, profile, and instrument Fast application start-up Software testing Generic time-machine Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
8 Application rollback Fault tolerance Effective debugging Fast application start-up from default/previous state (ccache...) improve desktop boot time Software testing Generic time-machine Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
9 Application rollback Fault tolerance Effective debugging Fast application start-up Software testing repeat from specific point(s) distribute on multiple hosts Generic time-machine Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
10 Application rollback Fault tolerance Effective debugging Fast application start-up Software testing Generic time-machine revive old server/desktop state retry a move in a game Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
11 Application suspend/resume Improved OOM handling Better system utilization Suspend/resume a user's session Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
12 Application suspend/resume Improved OOM handling suspend applications, don't kill smart swap on embedded Better system utilization Suspend/resume a user's session Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
13 Application suspend/resume Improved OOM handling Better system utilization suspend application to reduce load Suspend/resume a user's session Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
14 Application suspend/resume Improved OOM handling Better system utilization Suspend/resume a user's session mobile desktop on USB key linux based VPS/VDI systems Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
15 Application Migration Load balancing / resource sharing Zero-downtime maintenance High availability Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
16 Application Migration Load balancing / resource sharing HPC (e.g. BlueWaters project) cloud environments linux-based VPS/VDI Zero-downtime maintenance High availability Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
17 Application Migration Load balancing / resource sharing Zero-downtime maintenance live migration of applications High availability Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
18 Application Migration Load balancing / resource sharing Zero-downtime maintenance High availability primary/backup in lock-step frequent incremental checkpoints Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
19 Application vs Virtual-Machine Application Virtual C/R Machine granularity specific operating system applications as a whole unit saved state application entire operating state only system state overhead none visible flexibility application operating system awareness is black box deployment linux only same arch family Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
20 Some examples HPC environments can extend linux-cr distributed-cr Cloud deployments using linux containers Light-weight clusters of ARMs combine LXC and linux-cr Linux Kernel Summit, November Linux Kernel Summit, November
21 Some concrete examples BlueWaters NCSA's most powerful supercomputer checkpointing based on linux-cr OpenVZ VPS hosting with migration capabilities Canonical / Ubuntu add LXC & linux-cr in UEC cluster stack Linux Kernel Summit, November orenl@cs.columbia.edu Linux Kernel Summit, November orenl@cs.columbia.edu
22 Who and Who Who is doing? Matt Helsley, Dan Smith, Serge Hallyn, Nathan Lynch, Sukadev Bhattiprolu, me Who is interested? IBM, Canonical, OpenVZ, HPC industry, Kerrighed, Google (?),... Who else does/did? OS: AIX, OpenVZ, IRIX, Cray... Systems: Moab, BLCR/Beowolf, Condor... Linux Kernel Summit, November Linux Kernel Summit, November
23 Linux-C/R design goals Transparency Reliability Security/safety Performance Maintainability Linux Kernel Summit, November Linux Kernel Summit, November
24 Linux-C/R design goals Transparency applications oblivious to operation allow notify of checkpoint or restart allow application awareness Reliability Security/safety Performance Maintainability Linux Kernel Summit, November Linux Kernel Summit, November
25 Linux-C/R design goals Transparency Reliability checkpoint succeeds restart succeeds report non-checkpoint-able reasons checkpoint is non-intrusive Security/safety Performance Maintainability Linux Kernel Summit, November Linux Kernel Summit, November
26 Linux-C/R design goals Transparency Reliability Security/safety ptrace capabilities to checkpoint reuse kernel code to reconstruct state Performance Maintainability Linux Kernel Summit, November Linux Kernel Summit, November
27 Linux-C/R design goals Transparency Reliability Security/safety Performance zero impact on performance reasonable code footprint Maintainability Linux Kernel Summit, November Linux Kernel Summit, November
28 Linux-C/R design goals Transparency Reliability Security/safety Performance Maintainability next slide... Linux Kernel Summit, November Linux Kernel Summit, November
29 Maintainability Placement of C/R code Extensive test-suite Positive experience so far Impact on developers Linux Kernel Summit, November Linux Kernel Summit, November
30 Maintainability Placement of C/R code generic code in kernel/checkpoint/... most c/r code with or near subsystem code so subsytem maintainers sees it c/r is well documented Extensive test-suite Positive experience so far Impact on developers Linux Kernel Summit, November Linux Kernel Summit, November
31 Maintainability Placement of C/R code Extensive test-suite test large list (>120) of scenarios test before/during/after behavior Positive experience so far Impact on developers Linux Kernel Summit, November Linux Kernel Summit, November
32 Maintainability Placement of C/R code Extensive test-suite Positive experience ( today) can ignore most kernel changes mainly need to add features minor changes to prior c/r code e.g. splice/pipe, syscalls #s, mm helpers Impact on developers Linux Kernel Summit, November Linux Kernel Summit, November
33 Maintainability Placement of C/R code Extensive test-suite Positive experience so far Impact on developers understand what may affect c/r code at least notify c/r people when needed awareness will grow with exposure Linux Kernel Summit, November Linux Kernel Summit, November
34 Design Summary Save/restore state in-kernel Checkpoint container/subtree/self Image holds user-visible state Userspace image conversion Detailed error reporting Linux Kernel Summit, November Linux Kernel Summit, November
35 Checkpoint (1) Freeze process hierarchy (2) Save global data (3) Save process hierarchy (4) Save state of all tasks (?) Filesystem snapshot (5) Thaw/kill process hierarchy in-kernel Linux Kernel Summit, November Linux Kernel Summit, November
36 Restart (1) Create container (?) Restore (stage) filesystem (3) Create process hierarchy (4) Restore state of all tasks (5) Resume execution in-kernel Linux Kernel Summit, November Linux Kernel Summit, November
37 Current State Supported subsystems: tasks (threads, signals, credentials, etc) namespaces (all but mounts-ns) sysvipc (shm, msg, sem) files, dirs (regular, fifos/pipes, epoll, event, simple devices) sockets (unix, ipv4, ipv6) security (smack, selinux labels) Linux Kernel Summit, November Linux Kernel Summit, November
38 Current State What's missing [reviewed] file locks, leases, owner [reviewed] unlinked files/dirs [wip] fanotify/inotify/dnotify [wip] mounts, mount-ns /proc filesystem ptraced tasks more devices Linux Kernel Summit, November Linux Kernel Summit, November
39 Current State Supported architectures: x86-32 x86-64 s390x PowerPC ARM Linux Kernel Summit, November Linux Kernel Summit, November
40 Current State Code: ~23K lines in total ~1200 lines documentation ~600 lines per arch (x5) ~8K lines in kernel/checkpoint/* (base) ~7K lines near-place files (*/checkpoint.c) ~2K lines in-place save/restore ~2K lines for LSM Linux Kernel Summit, November Linux Kernel Summit, November
41 Discussion... Concrete path to mainline (mm/next?) Exposure to subsystem maintainers? Image format tied to kernel version (userspace conversion tools) Many thanks to those who reviewed, tested, and provided suggestions! Web page: Git tree(s): git:// Linux Kernel Summit, November Linux Kernel Summit, November
Linux-CR: Transparent Application Checkpoint-Restart in Linux
Linux-CR: Transparent Application Checkpoint-Restart in Linux Oren Laadan Columbia University orenl@cs.columbia.edu Serge E. Hallyn IBM serge@hallyn.com Linux Symposium, July 2010 1 orenl@cs.columbia.edu
Making Applications Mobile
Making Applications Mobile using containers Ottawa Linux Symposium, July 2006 Cedric Le Goater Daniel Lezcano Clement Calmels Dave Hansen
An introduction to checkpointing. for scientifc applications
damien.francois@uclouvain.be UCL/CISM An introduction to checkpointing for scientifc applications November 2016 CISM/CÉCI training session What is checkpointing? Without checkpointing: $./count 1 2 3^C
Advanced Memory Management
Advanced Memory Management Main Points Applications of memory management What can we do with ability to trap on memory references to individual pages? File systems and persistent storage Goals Abstractions
Overview Job Management OpenVZ Conclusions. XtreemOS. Surbhi Chitre. IRISA, Rennes, France. July 7, Surbhi Chitre XtreemOS 1 / 55
XtreemOS Surbhi Chitre IRISA, Rennes, France July 7, 2009 Surbhi Chitre XtreemOS 1 / 55 Surbhi Chitre XtreemOS 2 / 55 Outline What is XtreemOS What features does it provide in XtreemOS How is it new and
An introduction to checkpointing. for scientific applications
damien.francois@uclouvain.be UCL/CISM - FNRS/CÉCI An introduction to checkpointing for scientific applications November 2013 CISM/CÉCI training session What is checkpointing? Without checkpointing: $./count
Alternatives to Solaris Containers and ZFS for Linux on System z
Alternatives to Solaris Containers and ZFS for Linux on System z Cameron Seader (cs@suse.com) SUSE Tuesday, March 11, 2014 Session Number 14540 Agenda Quick Overview of Solaris Containers and ZFS Linux
Operating System Virtualization: Practice and Experience
Operating System Virtualization: Practice and Experience Oren Laadan and Jason Nieh Columbia University {orenl,nieh}@cs.columbia.edu SYSTOR 2010, Haifa, Israel 1 orenl@cs.columbia.edu SYSTOR 2010, Haifa,
Travis Cardwell Technical Meeting
.. Introduction to Docker Travis Cardwell Tokyo Linux Users Group 2014-01-18 Technical Meeting Presentation Motivation OS-level virtualization is becoming accessible Docker makes it very easy to experiment
Linux Kernel Security Update LinuxCon Europe Berlin, 2016
Linux Kernel Security Update LinuxCon Europe Berlin, 2016 James Morris james.l.morris@oracle.com Introduction Who am I? Kernel security subsystem maintainer Started kernel development w/ FreeS/WAN in 1999
for Kerrighed? February 1 st 2008 Kerrighed Summit, Paris Erich Focht NEC
Virtualization for Kerrighed? February 1 st 2008 Kerrighed Summit, Paris Erich Focht NEC Why virtualization? Virtualization means many things! Multi-programming any UNIX is virtualizing resources to allow
Application Fault Tolerance Using Continuous Checkpoint/Restart
Application Fault Tolerance Using Continuous Checkpoint/Restart Tomoki Sekiyama Linux Technology Center Yokohama Research Laboratory Hitachi Ltd. Outline 1. Overview of Application Fault Tolerance and
Linux Kernel Security Overview
Linux Kernel Security Overview Linux Security Summit Europe 2018 Edinburgh, UK James Morris jmorris@namei.org $ whoami Linux kernel security subsystem maintainer Linux kernel engineer at Microsoft Previously
Veritas InfoScale Enterprise for Oracle Real Application Clusters (RAC)
Veritas InfoScale Enterprise for Oracle Real Application Clusters (RAC) Manageability and availability for Oracle RAC databases Overview Veritas InfoScale Enterprise for Oracle Real Application Clusters
Efficient and Large Scale Program Flow Tracing in Linux. Alexander Shishkin, Intel
Efficient and Large Scale Program Flow Tracing in Linux Alexander Shishkin, Intel 16.09.2013 Overview Program flow tracing - What is it? - What is it good for? Intel Processor Trace - Features / capabilities
Distributed Systems 27. Process Migration & Allocation
Distributed Systems 27. Process Migration & Allocation Paul Krzyzanowski pxk@cs.rutgers.edu 12/16/2011 1 Processor allocation Easy with multiprocessor systems Every processor has access to the same memory
Toward An Integrated Cluster File System
Toward An Integrated Cluster File System Adrien Lebre February 1 st, 2008 XtreemOS IP project is funded by the European Commission under contract IST-FP6-033576 Outline Context Kerrighed and root file
VERITAS Storage Foundation 4.0 TM for Databases
VERITAS Storage Foundation 4.0 TM for Databases Powerful Manageability, High Availability and Superior Performance for Oracle, DB2 and Sybase Databases Enterprises today are experiencing tremendous growth
Autosave for Research Where to Start with Checkpoint/Restart
Autosave for Research Where to Start with Checkpoint/Restart Brandon Barker Computational Scientist Cornell University Center for Advanced Computing (CAC) brandon.barker@cornell.edu Workshop: High Performance
Veritas Storage Foundation for Oracle RAC from Symantec
Veritas Storage Foundation for Oracle RAC from Symantec Manageability, performance and availability for Oracle RAC databases Data Sheet: Storage Management Overviewview offers a proven solution to help
Process. Heechul Yun. Disclaimer: some slides are adopted from the book authors slides with permission
Process Heechul Yun Disclaimer: some slides are adopted from the book authors slides with permission 1 Recap OS services Resource (CPU, memory) allocation, filesystem, communication, protection, security,
Chapter 5 C. Virtual machines
Chapter 5 C Virtual machines Virtual Machines Host computer emulates guest operating system and machine resources Improved isolation of multiple guests Avoids security and reliability problems Aids sharing
MCTS Guide to Microsoft Windows Server 2008 Applications Infrastructure Configuration (Exam # ) Chapter One Introducing Windows Server 2008
MCTS Guide to Microsoft Windows Server 2008 Applications Infrastructure Configuration (Exam # 70-643) Chapter One Introducing Windows Server 2008 Objectives Distinguish among the different Windows Server
Introduction to Container Technology. Patrick Ladd Technical Account Manager April 13, 2016
Introduction to Container Technology Patrick Ladd Technical Account Manager April 13, 2016 Container Technology Containers 3 "Linux Containers" is a Linux kernel feature to contain a group of processes
Scalable In-memory Checkpoint with Automatic Restart on Failures
Scalable In-memory Checkpoint with Automatic Restart on Failures Xiang Ni, Esteban Meneses, Laxmikant V. Kalé Parallel Programming Laboratory University of Illinois at Urbana-Champaign November, 2012 8th
DMTCP: Fixing the Single Point of Failure of the ROS Master
DMTCP: Fixing the Single Point of Failure of the ROS Master Tw i n k l e J a i n j a i n. t @ h u s k y. n e u. e d u G e n e C o o p e r m a n g e n e @ c c s. n e u. e d u C o l l e g e o f C o m p u
Kubernetes The Path to Cloud Native
Kubernetes The Path to Cloud Native Eric Brewer VP, Infrastructure @eric_brewer August 28, 2015 ACM SOCC Cloud Na*ve Applica*ons Middle of a great transition unlimited ethereal resources in the Cloud an
Data Sheet: Storage Management Veritas Storage Foundation for Oracle RAC from Symantec Manageability and availability for Oracle RAC databases
Manageability and availability for Oracle RAC databases Overview Veritas Storage Foundation for Oracle RAC from Symantec offers a proven solution to help customers implement and manage highly available
Spring 2017 :: CSE 506. Introduction to. Virtual Machines. Nima Honarmand
Introduction to Virtual Machines Nima Honarmand Virtual Machines & Hypervisors Virtual Machine: an abstraction of a complete compute environment through the combined virtualization of the processor, memory,
STING: Finding Name Resolution Vulnerabilities in Programs
STING: Finding Name Resolution ulnerabilities in Programs Hayawardh ijayakumar, Joshua Schiffman, Trent Jaeger Systems and Internet Infrastructure Security (SIIS) Lab Computer Science and Engineering Department
Microkernel Construction
Introduction SS2013 Class Goals Provide deeper understanding of OS mechanisms Introduce L4 principles and concepts Make you become enthusiastic L4 hackers Propaganda for OS research at 2 Administration
MIGSOCK A Migratable TCP Socket in Linux
MIGSOCK A Migratable TCP Socket in Linux Bryan Kuntz Karthik Rajan Masters Thesis Presentation 21 st February 2002 Agenda Introduction Process Migration Socket Migration Design Implementation Testing and
CXS Citrix XenServer 6.0 Administration
Course Overview View Course Dates & Register Today Students will learn to effectively install, configure, administer, and troubleshoot XenServer 6.0. Students will also learn how to configure a Provisioning
Android System Development Training 4-day session
Android System Development Training 4-day session Title Android System Development Training Overview Understanding the Android Internals Understanding the Android Build System Customizing Android for a
ECE 574 Cluster Computing Lecture 8
ECE 574 Cluster Computing Lecture 8 Vince Weaver http://web.eece.maine.edu/~vweaver vincent.weaver@maine.edu 16 February 2017 Announcements Too many snow days Posted a video with HW#4 Review HW#5 will
Lecture Topics. Announcements. Today: Threads (Stallings, chapter , 4.6) Next: Concurrency (Stallings, chapter , 5.
Lecture Topics Today: Threads (Stallings, chapter 4.1-4.3, 4.6) Next: Concurrency (Stallings, chapter 5.1-5.4, 5.7) 1 Announcements Make tutorial Self-Study Exercise #4 Project #2 (due 9/20) Project #3
Distributed File Systems. Directory Hierarchy. Transfer Model
Distributed File Systems Ken Birman Goal: view a distributed system as a file system Storage is distributed Web tries to make world a collection of hyperlinked documents Issues not common to usual file
Virtuozzo Containers
Parallels Virtuozzo Containers White Paper An Introduction to Operating System Virtualization and Parallels Containers www.parallels.com Table of Contents Introduction... 3 Hardware Virtualization... 3
GFS: The Google File System. Dr. Yingwu Zhu
GFS: The Google File System Dr. Yingwu Zhu Motivating Application: Google Crawl the whole web Store it all on one big disk Process users searches on one big CPU More storage, CPU required than one PC can
PBS PROFESSIONAL VS. MICROSOFT HPC PACK
PBS PROFESSIONAL VS. MICROSOFT HPC PACK On the Microsoft Windows Platform PBS Professional offers many features which are not supported by Microsoft HPC Pack. SOME OF THE IMPORTANT ADVANTAGES OF PBS PROFESSIONAL
FAULT TOLERANT SYSTEMS
FAULT TOLERANT SYSTEMS http://www.ecs.umass.edu/ece/koren/faulttolerantsystems Part 16 - Checkpointing I Chapter 6 - Checkpointing Part.16.1 Failure During Program Execution Computers today are much faster,
MEMBRANE: OPERATING SYSTEM SUPPORT FOR RESTARTABLE FILE SYSTEMS
Department of Computer Science Institute of System Architecture, Operating Systems Group MEMBRANE: OPERATING SYSTEM SUPPORT FOR RESTARTABLE FILE SYSTEMS SWAMINATHAN SUNDARARAMAN, SRIRAM SUBRAMANIAN, ABHISHEK
Checkpointing with DMTCP and MVAPICH2 for Supercomputing. Kapil Arya. Mesosphere, Inc. & Northeastern University
MVAPICH Users Group 2016 Kapil Arya Checkpointing with DMTCP and MVAPICH2 for Supercomputing Kapil Arya Mesosphere, Inc. & Northeastern University DMTCP Developer Apache Mesos Committer kapil@mesosphere.io
Embedded Linux kernel and driver development training 5-day session
Embedded Linux kernel and driver development training 5-day session Title Embedded Linux kernel and driver development training Overview Understanding the Linux kernel Developing Linux device drivers Linux
OS Virtualization. Linux Containers (LXC)
OS Virtualization Emulate OS-level interface with native interface Lightweight virtual machines No hypervisor, OS provides necessary support Referred to as containers Solaris containers, BSD jails, Linux
Using Linux Containers as a Virtualization Option
Using Linux Containers as a Virtualization Option Michal Svec Product Manager msvec@suse.com Mike Friesenegger Sales Engineer mfriesenegger@suse.com 2 Containers Linux Containers Virtualization OS Level
Virtualisation: The KVM Way. Amit Shah
Virtualisation: The KVM Way Amit Shah amit.shah@qumranet.com foss.in/2007 Virtualisation Simulation of computer system in software Components Processor Management: register state, instructions, exceptions
COEN-4730 Computer Architecture Lecture 3 Review of Caches and Virtual Memory
1 COEN-4730 Computer Architecture Lecture 3 Review of Caches and Virtual Memory Cristinel Ababei Dept. of Electrical and Computer Engineering Marquette University Credits: Slides adapted from presentations
Process Migration via Remote Fork: a Viable Programming Model? Branden J. Moor! cse 598z: Distributed Systems December 02, 2004
Process Migration via Remote Fork: a Viable Programming Model? Branden J. Moor! cse 598z: Distributed Systems December 02, 2004 What is a Remote Fork? Creates an exact copy of the process on a remote system
SMD149 - Operating Systems
SMD149 - Operating Systems Roland Parviainen November 3, 2005 1 / 45 Outline Overview 2 / 45 Process (tasks) are necessary for concurrency Instance of a program in execution Next invocation of the program
MPI versions. MPI History
MPI versions MPI History Standardization started (1992) MPI-1 completed (1.0) (May 1994) Clarifications (1.1) (June 1995) MPI-2 (started: 1995, finished: 1997) MPI-2 book 1999 MPICH 1.2.4 partial implemention
Chapter 5 B. Large and Fast: Exploiting Memory Hierarchy
Chapter 5 B Large and Fast: Exploiting Memory Hierarchy Dependability 5.5 Dependable Memory Hierarchy Chapter 6 Storage and Other I/O Topics 2 Dependability Service accomplishment Service delivered as
Distributed File Systems II
Distributed File Systems II To do q Very-large scale: Google FS, Hadoop FS, BigTable q Next time: Naming things GFS A radically new environment NFS, etc. Independence Small Scale Variety of workloads Cooperation
VM Migration, Containers (Lecture 12, cs262a)
VM Migration, Containers (Lecture 12, cs262a) Ali Ghodsi and Ion Stoica, UC Berkeley February 28, 2018 (Based in part on http://web.eecs.umich.edu/~mosharaf/slides/eecs582/w16/021516-junchenglivemigration.pptx)
HPC learning using Cloud infrastructure
HPC learning using Cloud infrastructure Florin MANAILA IT Architect florin.manaila@ro.ibm.com Cluj-Napoca 16 March, 2010 Agenda 1. Leveraging Cloud model 2. HPC on Cloud 3. Recent projects - FutureGRID
Namespaces and Capabilities Overview and Recent Developments
Namespaces and Capabilities Overview and Recent Developments Linux Security Summit Europe Edinburgh, Scotland Christian Brauner christian@brauner.io christian.brauner@ubuntu.com @brau_ner https://brauner.github.io/
Distributed Systems COMP 212. Lecture 18 Othon Michail
Distributed Systems COMP 212 Lecture 18 Othon Michail Virtualisation & Cloud Computing 2/27 Protection rings It s all about protection rings in modern processors Hardware mechanism to protect data and
The failure of Operating Systems,
The failure of Operating Systems, and how we can fix it. Glauber Costa Lead Software Engineer August 30th, 2012 Linuxcon Opening Notes I'll be doing Hypervisors vs Containers here. But: 2 2 Opening Notes
Devops, Docker and Security. John
Devops, Docker and Security John Willis @botchagalupe About Me https://github.com/botchagalupe/my-presentations One of the founding members of Devopsdays Co-author of the Devops Handbook. Author of the
CIS 5373 Systems Security
CIS 5373 Systems Security Topic 3.1: OS Security Basics of secure design Endadul Hoque Slide Acknowledgment Contents are based on slides from Ninghui Li (Purdue), John Mitchell (Stanford), Dan Boneh (Stanford)
I/O Systems. Jinkyu Jeong Computer Systems Laboratory Sungkyunkwan University
I/O Systems Jinkyu Jeong (jinkyu@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics Device characteristics Block device vs. Character device Direct I/O vs.
Linux Kernel Architecture
Professional Linux Kernel Architecture Wolf gang Mauerer WILEY Wiley Publishing, Inc. Introduction xxvii Chapter 1: Introduction and Overview 1 Tasks of the Kernel v -- 2 Implementation Strategies 3 Elements
Proactive Process-Level Live Migration in HPC Environments
Proactive Process-Level Live Migration in HPC Environments Chao Wang, Frank Mueller North Carolina State University Christian Engelmann, Stephen L. Scott Oak Ridge National Laboratory SC 08 Nov. 20 Austin,
CRFS: A Lightweight User-Level Filesystem for Generic Checkpoint/Restart
CRFS: A Lightweight User-Level Filesystem for Generic Checkpoint/Restart Xiangyong Ouyang, Raghunath Rajachandrasekar, Xavier Besseron, Hao Wang, Jian Huang, Dhabaleswar K. Panda Department of Computer
Distributed recovery for senddeterministic. Tatiana V. Martsinkevich, Thomas Ropars, Amina Guermouche, Franck Cappello
Distributed recovery for senddeterministic HPC applications Tatiana V. Martsinkevich, Thomas Ropars, Amina Guermouche, Franck Cappello 1 Fault-tolerance in HPC applications Number of cores on one CPU and
Efficient Memory Management on Mobile Devices
Efficient Memory Management on Mobile Devices Bartlomiej Zolnierkiewicz b.zolnierkie@samsung.com September 17, 2013 Issues on mobile systems: limited resources no physical swap need for custom Out-Of-Memory
How To Write a Linux Security Module That Makes Sense For You. Casey Schaufler February 2016
How To Write a Linux Security Module That Makes Sense For You Casey Schaufler February 2016 Casey Schaufler Unix 32 bit port - 1979 Smack Linux security module Security module stacking Why Would You Write
The Kernel Abstraction
The Kernel Abstraction Debugging as Engineering Much of your time in this course will be spent debugging In industry, 50% of software dev is debugging Even more for kernel development How do you reduce
IBM WebSphere MQ for HP NonStop Update
IBM WebSphere MQ for HP NonStop Update Gerry Reilly Development Director and CTO, IBM Messaging greilly@uk.ibm.com 5 th December 2013 2013 IBM Corporation Trademark Statement IBM, WebSphere and the IBM
MPI History. MPI versions MPI-2 MPICH2
MPI versions MPI History Standardization started (1992) MPI-1 completed (1.0) (May 1994) Clarifications (1.1) (June 1995) MPI-2 (started: 1995, finished: 1997) MPI-2 book 1999 MPICH 1.2.4 partial implemention
Process. Heechul Yun. Disclaimer: some slides are adopted from the book authors slides with permission 1
Process Heechul Yun Disclaimer: some slides are adopted from the book authors slides with permission 1 Recap OS services Resource (CPU, memory) allocation, filesystem, communication, protection, security,
LINUX CONTAINERS. Where Enterprise Meets Embedded Operating Environments WHEN IT MATTERS, IT RUNS ON WIND RIVER
Where Enterprise Meets Embedded Operating Environments WHEN IT MATTERS, IT RUNS ON WIND RIVER EXECUTIVE SUMMARY Flexible and connected platforms are core components in leading computing fields, including
A Global Operating System for HPC Clusters
A Global Operating System Emiliano Betti 1 Marco Cesati 1 Roberto Gioiosa 2 Francesco Piermaria 1 1 System Programming Research Group, University of Rome Tor Vergata 2 BlueGene Software Division, IBM TJ
OS Containers. Michal Sekletár November 06, 2016
OS Containers Michal Sekletár msekleta@redhat.com November 06, 2016 whoami Senior Software Engineer @ Red Hat systemd and udev maintainer Free/Open Source Software contributor Michal Sekletár msekleta@redhat.com
VERITAS NetBackup 6.0 Enterprise Server INNOVATIVE DATA PROTECTION DATASHEET. Product Highlights
VERITAS NetBackup 6.0 Enterprise Server INNOVATIVE DATA PROTECTION As the recognized leader for enterprise-class backup and recovery, VERITAS NetBackup Enterprise Server software is designed to help provide
TSM HSM Explained. Agenda. Oxford University TSM Symposium Introduction. How it works. Best Practices. Futures. References. IBM Software Group
IBM Software Group TSM HSM Explained Oxford University TSM Symposium 2003 Christian Bolik (bolik@de.ibm.com) IBM Tivoli Storage SW Development 1 Agenda Introduction How it works Best Practices Futures
Userfaultfd: Post-copy VM migration and beyond
Userfaultfd: Post-copy VM migration and beyond Mike Rapoport This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant
CSCE Operating Systems Interrupts, Exceptions, and Signals. Qiang Zeng, Ph.D. Fall 2018
CSCE 311 - Operating Systems Interrupts, Exceptions, and Signals Qiang Zeng, Ph.D. Fall 2018 Previous Class Process state transition Ready, blocked, running Call Stack Execution Context Process switch
CS510 Operating System Foundations. Jonathan Walpole
CS510 Operating System Foundations Jonathan Walpole File System Performance File System Performance Memory mapped files - Avoid system call overhead Buffer cache - Avoid disk I/O overhead Careful data
Kerrighed: A SSI Cluster OS Running OpenMP
Kerrighed: A SSI Cluster OS Running OpenMP EWOMP 2003 David Margery, Geoffroy Vallée, Renaud Lottiaux, Christine Morin, Jean-Yves Berthou IRISA/INRIA PARIS project-team EDF R&D 1 Introduction OpenMP only
An Introduction to GPFS
IBM High Performance Computing July 2006 An Introduction to GPFS gpfsintro072506.doc Page 2 Contents Overview 2 What is GPFS? 3 The file system 3 Application interfaces 4 Performance and scalability 4
Oracle Database 12c Administration I
Course Overview In this course, students will focus on configuration on supported systems. Also how the database fits in with other systems. Who Should Attend View Course Dates & Register Today This is
Linux Foundation Collaboration Summit 2010
Linux Foundation Collaboration Summit 2010 LTTng, State of the Union Presentation at: http://www.efficios.com/lfcs2010 E-mail: mathieu.desnoyers@efficios.com 1 > Presenter Mathieu Desnoyers EfficiOS Inc.
CS399 New Beginnings. Jonathan Walpole
CS399 New Beginnings Jonathan Walpole OS-Related Hardware & Software The Process Concept 2 Lecture 2 Overview OS-Related Hardware & Software - complications in real systems - brief introduction to memory
Virtualizaton: One Size Does Not Fit All. Nedeljko Miljevic Product Manager, Automotive Solutions MontaVista Software
Virtualizaton: One Size Does Not Fit All Nedeljko Miljevic Product Manager, Automotive Solutions MontaVista Software Agenda Linux and Automotive Challenges Solution: Virtualization Linux Containers Best
The Google File System
The Google File System Sanjay Ghemawat, Howard Gobioff and Shun Tak Leung Google* Shivesh Kumar Sharma fl4164@wayne.edu Fall 2015 004395771 Overview Google file system is a scalable distributed file system
New England Data Camp v2.0 It is all about the data! Caregroup Healthcare System. Ayad Shammout Lead Technical DBA
New England Data Camp v2.0 It is all about the data! Caregroup Healthcare System Ayad Shammout Lead Technical DBA ashammou@caregroup.harvard.edu About Caregroup SQL Server Database Mirroring Selected SQL
Three Steps Toward Zero Downtime. Guide. Solution Guide Server.
Three Steps Toward Zero Downtime Guide Solution Guide Server Server Solution Guide Three Steps Toward Zero Downtime Introduction Service uptime is a top priority for many business operations. From global
Shared File System Requirements for SAS Grid Manager. Table Talk #1546 Ben Smith / Brian Porter
Shared File System Requirements for SAS Grid Manager Table Talk #1546 Ben Smith / Brian Porter About the Presenters Main Presenter: Ben Smith, Technical Solutions Architect, IBM smithbe1@us.ibm.com Brian
Chapter 8: Virtual Memory. Operating System Concepts
Chapter 8: Virtual Memory Silberschatz, Galvin and Gagne 2009 Chapter 8: Virtual Memory Background Demand Paging Copy-on-Write Page Replacement Allocation of Frames Thrashing Memory-Mapped Files Allocating
EDGE COMPUTING & IOT MAKING IT SECURE AND MANAGEABLE FRANCK ROUX MARKETING MANAGER, NXP JUNE PUBLIC
EDGE COMPUTING & IOT MAKING IT SECURE AND MANAGEABLE FRANCK ROUX MARKETING MANAGER, NXP JUNE 6 2018 PUBLIC PUBLIC 2 Key concerns with IoT.. PUBLIC 3 Why Edge Computing? CLOUD Too far away Expensive connectivity
Virtual Memory. Reading. Sections 5.4, 5.5, 5.6, 5.8, 5.10 (2) Lecture notes from MKP and S. Yalamanchili
Virtual Memory Lecture notes from MKP and S. Yalamanchili Sections 5.4, 5.5, 5.6, 5.8, 5.10 Reading (2) 1 The Memory Hierarchy ALU registers Cache Memory Memory Memory Managed by the compiler Memory Managed
Distributed Filesystem
Distributed Filesystem 1 How do we get data to the workers? NAS Compute Nodes SAN 2 Distributing Code! Don t move data to workers move workers to the data! - Store data on the local disks of nodes in the
Virtual Memory. Lecture for CPSC 5155 Edward Bosworth, Ph.D. Computer Science Department Columbus State University
Virtual Memory Lecture for CPSC 5155 Edward Bosworth, Ph.D. Computer Science Department Columbus State University Precise Definition of Virtual Memory Virtual memory is a mechanism for translating logical
Linux Tiny Penguin Weight Watchers. Thomas Petazzoni Free Electrons electrons.com
Linux Tiny Penguin Weight Watchers Thomas Petazzoni Free Electrons thomas@free electrons.com Who am I? Since January 2008, works for Free Electrons Embedded Linux and Free software consulting company Before,
Network stack virtualization for FreeBSD 7.0. Marko Zec
Network stack virtualization for FreeBSD 7.0 Marko Zec zec@fer.hr University of Zagreb Network stack virtualization for FreeBSD 7.0 slide 1 of 18 Talk outline Network stack virtualization what, why, and
CS 333 Introduction to Operating Systems Class 2 OS-Related Hardware & Software The Process Concept
CS 333 Introduction to Operating Systems Class 2 OS-Related Hardware & Software The Process Concept Jonathan Walpole Computer Science Portland State University 1 Lecture 2 overview OS-Related Hardware
Storage and File Hierarchy
COS 318: Operating Systems Storage and File Hierarchy Jaswinder Pal Singh Computer Science Department Princeton University (http://www.cs.princeton.edu/courses/cos318/) Topics Storage hierarchy File system
GFS: The Google File System
GFS: The Google File System Brad Karp UCL Computer Science CS GZ03 / M030 24 th October 2014 Motivating Application: Google Crawl the whole web Store it all on one big disk Process users searches on one
vsphere Networking Update 1 ESXi 5.1 vcenter Server 5.1 vsphere 5.1 EN
Update 1 ESXi 5.1 vcenter Server 5.1 vsphere 5.1 This document supports the version of each product listed and supports all subsequent versions until the document is replaced by a new edition. To check