深 入解析 Docker 背后的 Linux 内核技术. 孙健波浙江 大学 SEL/VLIS 实验室

Transcription

1 深 入解析 Docker 背后的 Linux 内核技术 孙健波浙江 大学 SEL/VLIS 实验室

2 Agenda Namespace ipc uts pid network mount user Cgroup what are cgroups? usage concepts implementation

3 What is Namespace? Lightweight Process virtualization hostname IPC network stack PID1,PID2,. uid,gid,capabilities Isolation:Enable a process (or several processes) to have different views of the system than other processes. filesystem hostname IPC network stack filesystem PID1,PID2,. uid,gid,capabilities

4 namespaces There are currently 6 namespaces: uts (hostname) ipc (System V IPC) net (network stack) mnt (mount points, filesystems) pid (processes) user (UIDs)

5 /proc/[pid]/ns use mount to keep namespace alive

6 APIs Three system calls are used. clone() unshare() setns()

7 clone() namespace process clone() new namespace new process creates a new process and a new namespace

8 unshare() namespace creates a new namespace attaches the current process to it process new namespace unshare() process

9 setns() namespace A process setns() namespace B process joining an existing namespace.

10 UTS namespace struct task_struct *nsproxy static inline struct new_utsname *utsname(void){ return &current->nsproxy->uts_ns->name; } struct nsproxy *uts_ns *mnt_ns *net_ns *pid_ns *ipc_ns struct uts_namespace cee nodename sysname release version machine SYSCALL_DEFINE2(gethostname, char user *, name, int, len){ struct new_utsname *u;... u = utsname(); if (copy_to_user(name, u->nodename, i)) errno = -EFAULT;... }

11 IPC namespace the principle is the same more code

12 Network namespace logically another copy of the network stack use pipe to create veth pair to communicate container namespace A eth0 container namespace B eth0 veth veth Host Bridge: docker0 Physical Network Device

13 Mount namespace mount namespace /bin /lib /proc /root First namespace in history master share private unbindable Default to create a new copy instead of point to root namespace slave share private /bin /lib /proc share child namespace share /bin another namespace

14 PID namespace Same PID in different namespace can be nested up to 32 levels PID 1 = init process child reaping ignore SIGKILL

15 User namespace normal user Will be supported by Docker in future. user namespace (privileged user) Docker in Yarn pid mount network uts ipc

16 What are cgroups? Control Groups provide a mechanism for aggregating/partitioning sets of tasks, and all their future children, into hierarchical groups with specialized behaviour.

17 Usage of cgroups Resource Limitation:groups can be set to not exceed a configured limit Prioritization:some groups may get a larger share of CPU utilization or disk I/O throughput Accounting:measures how much resources certain systems use Control :freezing the groups of processes, their checkpointing and restarting

18 Concepts cgroup a group of tasks with shared characteristics subsystem a module that applies parameters to cgroups to control them in particular ways, typically for resource management hierarchy a set of cgroups organized in a hierarchical tree, plus one or more subsystems associated with that tree VFS -> API

19 Cgroups Example two hierarchy /cgroup /cgroup/memlimits (memory subsystem mount point & hierarchy) /cgroup/cpulimits (cpuset subsystem mount point & hierarchy) /cgroup/memlimits/student /cgroup/memlimits/teacher /cgroup/cpulimits/student /cgroup/cpulimits/teacher memory.limit=1g tasks=1,2,3,4,5 memory.limit=2g tasks= 6,7,8 cpuset.cpus=0-1 tasks=1,2,3,4,5 cpuset.cpus=0-3 tasks= 6,7,8

20 Parameters Examples cpuset subsystem cpuset.cpus: defines the set of cpus that the tasks in the cgroup are allowed to execute on echo 0-2 > /cgroup/cpuset/lab2/cpuset.cpus memory subsystem memory.limit_in_bytes: sets the maximum amount of user memory echo 1G > /cgroup/memory/lab1/memory.limit_in_bytes

21 Relationships Between Subsystems, Hierarchies, Control Groups and Tasks Rule 1

22 Relationships Between Subsystems, Hierarchies, Control Groups and Tasks Rule 2

23 Relationships Between Subsystems, Hierarchies, Control Groups and Tasks Rule 3

24 Relationships Between Subsystems, Hierarchies, Control Groups and Tasks Rule 4

25 Current subsystems used by Docker cpuset controls access to individual CPUs and memory nodes by a cgroup cpu schedules CPU access to cgroups cpuacct reports CPU resource usage by a cgroup memory controls access to memory resources and reports memory resource usage by a cgroup devices controls access to devices by a cgroup; e.g., gpus freezer suspends and resumes tasks in a cgroup blkio tracks I/O ownership, allowing control of access to block I/O resources

26 cgroups hooks task_struct css_set cg_cgroup_link cgroup list_head cgrp_link_list css_set *cgroups hlist_node hlist cgroup *cgrp cgroup_subsys_state *subsys[] list_head cg_list list_head tasks list_head cg_link_list list_head css_sets list_head cg_links css_set *cg cgroupfs_root *root cgroup_subsys_state *subsys[] cgroup_subsys_ state css_set_hash() cgroup *cgroup cgroup_subsys func create css_set_table func destroy cpuset cgroupfs_root func attach func fork func exit freezer blkio_cgrou p int hierarchy_id list_head root_list int subsys_id cgroupfs_root *root list_head subsys_list list_head sibling

27 References Red_Hat_Enterprise_Linux/6/html/ Resource_Management_Guide/

28 Thanks!