Outline. Cgroup hierarchies

Similar documents
Outline. Cgroup hierarchies

Control Groups (cgroups)

What s new in control groups (cgroups) v2

Chapter Two. Lesson A. Objectives. Exploring the UNIX File System and File Security. Understanding Files and Directories

Appendix A GLOSSARY. SYS-ED/ Computer Education Techniques, Inc.

The Classical OS Model in Unix

Manage Directories and Files in Linux. Objectives. Understand the Filesystem Hierarchy Standard (FHS)

Lecture Topics. Announcements. Today: Threads (Stallings, chapter , 4.6) Next: Concurrency (Stallings, chapter , 5.

Introduction to Linux. Woo-Yeong Jeong Computer Systems Laboratory Sungkyunkwan University

Checking Resource Usage in Fedora (Linux)

The Unix Shell & Shell Scripts

UNIT I Linux Utilities

Unix as a Platform Exercises. Course Code: OS-01-UNXPLAT

How many of you have never built a NetBSD kernel?

Unix Processes. What is a Process?

Welcome to getting started with Ubuntu Server. This System Administrator Manual. guide to be simple to follow, with step by step instructions

5/8/2012. Creating and Changing Directories Chapter 7

Introduction to Linux

Introduction to Linux

Slurm Support for Linux Control Groups

Roll No. :... Invigilator's Signature : UNIX AND SHELL PROGRAMMING. Time Allotted : 3 Hours Full Marks : 70

Linux Command Line Primer. By: Scott Marshall

Disks, Filesystems 1

Mid Term from Feb-2005 to Nov 2012 CS604- Operating System

(MCQZ-CS604 Operating Systems)

OS Containers. Michal Sekletár November 06, 2016

Understanding user namespaces

Unix API Books. Linux Kernel Books. Assignments and Exams. Grading for CSC 256. Operating Systems 8/31/2018. CSC 256/456 Fall

Introduction to Linux

UNIX System Programming Lecture 3: BASH Programming

CSE506: Operating Systems CSE 506: Operating Systems

User Namespaces. Linux Capabilities and Namespaces. Outline. Michael Kerrisk, man7.org c 2018 March 2018

CENG 334 Computer Networks. Laboratory I Linux Tutorial

Introduction to Linux. Roman Cheplyaka

Introduction to Linux

Perl and R Scripting for Biologists

CHAPTER 3 SHELL PROGRAMS: SCRIPTS

INSTALLING INSTALLING INSTALLING

Introduction to Operating Systems Prof. Chester Rebeiro Department of Computer Science and Engineering Indian Institute of Technology, Madras

IBM AIX Basic Operations V5.

APPLIED INFORMATICS Processes. Bash characteristics. Command type. Aliases.

COSC243 Part 2: Operating Systems

INSE Lab 1 Introduction to UNIX Fall 2017

CS 450 Operating System Week 4 Lecture Notes

Embedded Linux Systems. Bin Li Assistant Professor Dept. of Electrical, Computer and Biomedical Engineering University of Rhode Island

Operating System Labs. Yuanbin Wu

A Brief Introduction to Unix

UNIT I Linux Utilities and Working with Bash

Containers. Pablo F. Ordóñez. October 18, 2018

Introduction of Linux

Chapter 1 - Introduction. September 8, 2016

$Id: asg4-shell-tree.mm,v :36: $

Introduction to remote command line Linux. Research Computing Team University of Birmingham

Operating Systems. II. Processes

ECE 550D Fundamentals of Computer Systems and Engineering. Fall 2017

Scripting Languages Course 1. Diana Trandabăț

Operating Systems Design 12a. Special File Systems

Passthrough in QEMU/KVM on Linux

Chapter 4: Threads. Operating System Concepts. Silberschatz, Galvin and Gagne

4.8 Summary. Practice Exercises

Contents. Note: pay attention to where you are. Note: Plaintext version. Note: pay attention to where you are... 1 Note: Plaintext version...

Unix as a Platform Exercises + Solutions. Course Code: OS 01 UNXPLAT

An introduction to cgroups and cgroupspy tags = [ python, docker, coreos',

Use of interaction networks in teaching Minix

Docker A FRAMEWORK FOR DATA INTENSIVE COMPUTING

Chapter 4: Threads. Overview Multithreading Models Threading Issues Pthreads Windows XP Threads Linux Threads Java Threads. Operating System Concepts

CSE506: Operating Systems CSE 506: Operating Systems

System Programming. Introduction to Unix

Read the relevant material in Sobell! If you want to follow along with the examples that follow, and you do, open a Linux terminal.

Processes and Threads. Processes and Threads. Processes (2) Processes (1)

An introduction to Docker

CS 307: UNIX PROGRAMMING ENVIRONMENT FIND COMMAND

Disks, Filesystems Todd Kelley CST8177 Todd Kelley 1

Travis Cardwell Technical Meeting

Introduction to Linux

First of all, these notes will cover only a small subset of the available commands and utilities, and will cover most of those in a shallow fashion.

CSE 303 Lecture 2. Introduction to bash shell. read Linux Pocket Guide pp , 58-59, 60, 65-70, 71-72, 77-80

INSTALLING INSTALLING INSTALLING

CS246 Spring14 Programming Paradigm Notes on Linux

Capabilities. Linux Capabilities and Namespaces. Outline. Michael Kerrisk, man7.org c 2018 March 2018

5/20/2007. Touring Essential Programs

Unix/Linux Basics. Cpt S 223, Fall 2007 Copyright: Washington State University

Using bash. Administrative Shell Scripting COMP2101 Fall 2017

The Unix Environment for Programming (COMP433)

VIRTUAL FILE SYSTEM AND FILE SYSTEM CONCEPTS Operating Systems Design Euiseong Seo

SOFTWARE ARCHITECTURE 3. SHELL

Linux & Shell Programming 2014

Disks, Filesystems, Booting Todd Kelley CST8177 Todd Kelley 1

Introduction to Linux Part I: The Filesystem Luca Heltai

Processes & Threads. Recap of the Last Class. Microkernel System Architecture. Layered Structure

PROCESSES. Jo, Heeseung

Processes. Jo, Heeseung

UNIX Tutorial One

Reading Assignment 4. n Chapter 4 Threads, due 2/7. 1/31/13 CSE325 - Processes 1

A Design and Implementation of Universal Container

A2 Design Considerations. CS161, Spring

Question No: 1 In capacity planning exercises, which tools assist in listing and identifying processes of interest? (Choose TWO correct answers.

Presented by Bill Genske Gary Jackson

Project 3: An Introduction to File Systems. COP 4610 / CGS 5765 Principles of Operating Systems

Please choose the best answer. More than one answer might be true, but choose the one that is best.

Transcription:

Outline 4 Cgroups 4-1 4.1 Introduction 4-3 4.2 Cgroups v1: hierarchies and controllers 4-16 4.3 Cgroups v1: populating a cgroup 4-24 4.4 Cgroups v1: a survey of the controllers 4-38 4.5 Cgroups /proc files 4-53 4.6 Cgroups v2: background and introduction 4-56 4.7 Cgroups v2: enabling and disabling controllers 4-66 4.8 Cgroups v2: organizing cgroups and processes 4-79 4.9 Cgroups v2: delegation 4-88 Cgroup hierarchies Cgroup == collection of processes cgroup hierarchy == hierarchical arrangement of cgroups Implemented via a cgroup pseudo-filesystem Structure and membership of cgroup hierarchy is defined by: 1 Mounting a cgroup filesystem 2 Creating a subdirectory structure that reflects desired cgroup hierarchy 3 Moving processes within hierarchy by writing their PIDs to special files in cgroup subdirectories Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-18 4.2

Attaching a controller to a hierarchy A controller is attached to a hierarchy by mounting a cgroup filesystem: # mkdir -p / sys / fs/ cgroup / mem # Create mount point # mount -t cgroup -o memory none / sys / fs/ cgroup / mem Here, memory controller was mounted none can be replaced by any suitable mnemonic name Not interpreted by system, but appears in /proc/mounts Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-19 4.2 Attaching a controller to a hierarchy To see which cgroup filesystems are mounted and their attached controllers: # mount grep cgroup none on / sys /fs/ cgroup / mem type cgroup (rw, memory ) # cat / proc / mounts grep cgroup none / sys /fs/ cgroup / mem cgroup rw, relatime, memory 0 0 Unmounting filesystem detaches the controller: # umount / sys /fs/ cgroup / mem But..., filesystem will remain (invisibly) mounted if it contains child cgroups I.e., must move all processes to root cgroup, and remove child cgroups, to truly unmount Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-20 4.2

Attaching controllers to hierarchies A controller can be attached to only one hierarchy Mounting same controller at different mount point simply creates second view of same hierarchy Multiple controllers can be attached to same hierarchy: # mkdir -p / sys /fs/ cgroup / mem_cpu # mount -t cgroup -o memory, cpu none \ / sys /fs/ cgroup / mem_cpu In effect, resources associated with those controllers are being managed together Or, all controllers can be attached to one hierarchy: # mount -t cgroup -o all none / some / mount / point -o all is the default if no controller is specified Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-21 4.2 Creating cgroups When a new hierarchy is created, all tasks on system are part of root cgroup for that hierarchy New cgroups are created by creating subdirectories under cgroup mount point: # mkdir / sys /fs/ cgroup / mem /g1 Relationships between cgroups are reflected by creating nested (arbitrarily deep) subdirectory structure Meaning of hierarchical relationship depends on controller Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-22 4.2

Destroying cgroups An empty cgroup can be destroyed by removing directory Empty == last process in cgroup terminates or migrates to another cgroup and last child cgroup is removed Presence of zombie process does not prevent removal of cgroup directory Not necessary (or possible) to delete attribute files inside cgroup directory before deleting it Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-23 4.2

Outline 4 Cgroups 4-1 4.1 Introduction 4-3 4.2 Cgroups v1: hierarchies and controllers 4-16 4.3 Cgroups v1: populating a cgroup 4-24 4.4 Cgroups v1: a survey of the controllers 4-38 4.5 Cgroups /proc files 4-53 4.6 Cgroups v2: background and introduction 4-56 4.7 Cgroups v2: enabling and disabling controllers 4-66 4.8 Cgroups v2: organizing cgroups and processes 4-79 4.9 Cgroups v2: delegation 4-88 Files for managing cgroup membership To manage cgroup membership, each subdirectory in a hierarchy includes two automagically created files: cgroup.procs tasks Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-26 4.3

Tasks? Cgroups v1 draws distinction between process and task Task == kernel scheduling entity From scheduler s perspective, processes and threads are pretty much the same thing... (Threads just share more state than processes) Multithreaded (MT) process == set of tasks with same thread group ID (TGID) TGID == PID! Each thread has unique thread ID (TID) Here, TID means kernel thread ID I.e., value returned by clone(2) and gettid(2) Not same as POSIX threads pthread_t (But there is 1:1 relationship in NPTL implementation...) Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-27 4.3 Placing a process in a cgroup To move a process to a cgroup, write its PID to cgroup.procs file in corresponding subdirectory # echo $$ > / sys /fs/ cgroup / mem /g1/ cgroup. procs In multithreaded process, moves all threads to cgroup... Some shells don t check for error return from write(2) So we might not see error for an invalid PID... Use /bin/echo since it checks for errors from write(2) Write only one PID at a time Writing 0 to cgroup.procs moves writing process to cgroup Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-28 4.3

Viewing cgroup membership To see PIDs in cgroup, read cgroup.procs file PIDs are newline-separated Zombie processes do not appear in list List is not guaranteed to be sorted or free of duplicates PID might be moved out and back into cgroup or recycled while reading list Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-29 4.3 Placing a thread (task) in a cgroup Writing a PID to cgroup.procs moves all threads in thread group to a cgroup Each cgroup directory also has a tasks file... Writing a TID to tasks moves that thread to cgroup This feature goes away in cgroups v2... Reading tasks shows all TIDs in cgroup Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-30 4.3

Cgroup membership details Within a hierarchy, a task can be member of just one cgroup That association defines attributes / parameters that apply to the task Adding a task to a different cgroup automatically removes it from previous cgroup A task can be a member of multiple cgroups, each of which is in a different hierarchy On fork(), child inherits cgroup memberships of parent Afterward, cgroup memberships of parent and child can be independently changed Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-31 4.3 Cgroup release Consider the following scenario: We create a cgroup subdirectory Some processes are moved into cgroup Eventually, all of those processes terminate Who cleans up/gets notified when last process leaves cgroup? We might want cgroup subdirectory to be removed Manager process might want to know when all workers have terminated Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-32 4.3

Cgroup release release_agent in cgroup root directory Contains pathname of binary/script that is executed when cgroup becomes empty E.g., this program might remove cgroup subdirectory Release agent gets one command-line argument: pathname of cgroup subdirectory that has become empty Can also be specified as mount option mount -o release_agent =/ path / to/ executable notify_on_release in each cgroup subdirectory Should release_agent be run when cgroup becomes empty? (0 == no, 1 == yes) Initial setting for this file is inherited from cgroup parent Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-33 4.3 Mounting a named hierarchy with no controller Can mount a named hierarchy with no attached controller: # mount -t cgroup cgroup -o none, name = somename \ / some / mount / point Named hierarchies can be used to organize and track processes E.g., PIDs can be moved into cgroup.procs, and will automatically disappear on process termination (And we can use release_agent, etc.) systemd creates such a hierarchy for its management of processes Mounted at /sys/fs/cgroup/systemd Cgroups v1 only Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-34 4.3

Exercises 1 In this exercise, we create a cgroup, place a process in the cgroup, and then migrate that process to a different cgroup. If the memory cgroup is not already mounted, mount it: # cat / proc / mounts grep cgroup # Is cgroup mounted? # mkdir -p / sys /fs / cgroup / memory # mount - t cgroup - o memory none / sys / fs/ cgroup / memory # cd / sys /fs/ cgroup / memory Note: some systems (e.g., Debian) provide a patched kernel that disables the memory controller by default. If you find that you can t mount the memory controller, it may be necessary to reboot the kernel with the cgroup_enable=memory command-line option. Create two subdirectories, m1 and m2, in the memory cgroup root directory. Execute the following command, and note the PID assigned to the resulting process: # sleep 300 & Write the PID of the process created in the previous step into the file m1/cgroup.procs, and verify by reading the file contents. Now write the PID of the process into the file m2/cgroup.procs. Is the PID still visible in the file m1/cgroup.procs? Explain. Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-35 4.3 Exercises 2 In this exercise, we demonstrate the operation of a release_agent. Create a shell script, /tmp/release.sh, with following content: #! / bin /sh date > / tmp / release. log echo $* >> / tmp / release. log Make the script executable: chmod +x /tmp/release.sh If the memory cgroup is not already mounted, mount it # cat / proc / mounts grep cgroup # Is cgroup mounted? # mkdir -p / sys /fs / cgroup / mem # mount -t cgroup -o memory none / sys /fs/ cgroup / mem Change your working directory to the memory root cgroup and write the pathname of the script to the release_agent file: # cd / sys /fs/ cgroup / mem # echo / tmp / release.sh > release_agent Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-36 4.3

Exercises Create two subdirectories, x1 and x2, and write the value 1 to the notify_on_release file in x1: # mkdir x1 x2 # echo 1 > x1 / notify_on_release Execute the following command, and note the PID assigned to the resulting process: # sleep 300 & Write the PID of the process created in the previous step into the file x1/cgroup.procs. Now write the PID of the process into the file x2/cgroup.procs. Inspect the contents of the file /tmp/release.log. You should see one line containing the date and time, and a second line containing the pathname /x1. (If /tmp/release.log does not exist, review the above steps, kill the sleep process, and repeat the exercise.) Linux Security and Isolation APIs c 2016, Michael Kerrisk Cgroups 4-37 4.3