See Docker from the Perspective of Linux Process. Allen Hangzhou Docker Meetup

Transcription

1 See Docker from the Perspective of Linux Process Allen Hangzhou Docker Meetup

2 Agenda 1. Prerequisite Linux Process (do_fork / copy_process ) Namespaces 2. How Docker deals process dockerinit, ENTRYPOINT, CMD

3 syscall fork() Process A Parent - original PID fork() Child - new PID Process A continues SIGCHLD Process B execev() exit() executes a different program! wait() clean up ZOMBIE Reference:

4 do_fork copy_process check flags do_fork dup and init task_struct copy_process determine PID wake_up_new_task wait_for_completion check resource limit copy/share process details copy_semundo copy_namespaces set IDs, task relationships, etc. Reference:Mauerer W. Professional Linux kernel architecture[m] Figure 2-7 and Figure 2-8. John Wiley & Sons, 2010.

5 task_struct and namespaces struct task_struct Nsproxy proxies 5 kinds of namespace for a process. struct nsproxy *nsproxy 1.uts_namespace 2.mnt_namespace 3.pid_namespace 4.ipc_namespace 5.net struct nsproxy struct uts_namespace *uts_ns struct mnt_namespace *mnt_ns struct net *net_ns struct uts_namespace struct mnt_namespace struct net user_namespace is not in nsproxy! Based on Linux kernel 3.13

6 What is in namespaces? struct pid_namespace { struct task_struct * child_reaper; int level; struct pid_namespace *parent; }; struct uts_namespace { struct kref kref; struct new_utsname name; struct user_namespace *user_ns; } struct mnt_namespace { atomic_t count; struct mount *root; struct list_head list; }; struct new_utsname { char sysname[..]; char nodename[..]; char release[..]; char version[..]; char machine[..]; char domainname[..]; }; Based on Linux kernel 3.13

7 Docker? Where is Docker? Docker Client do_fork Docker Daemon fork! copy_process copy_namespaces Docker Container Docker Container do_execve Docker Container is born just by syscall fork and exec a process!

8 Difference (Docker s fork vs normal fork) Special flags used in syscall do_fork() flag name Linux kernel version CLONE_NEWNS CLONE_NEWUTS CLONE_NEWIPC CLONE_NEWPID CLONE_NEWNET CLONE_NEWUSER 3.8

9 Namespaces in Docker func init() { namespacelist = Namespaces { {Key: "NEWNS", Value: syscall.clone_newns, File: "mnt"}, {Key: "NEWUTS", Value: syscall.clone_newuts, File: "uts"}, {Key: "NEWIPC", Value: syscall.clone_newipc, File: "ipc"}, {Key: "NEWUSER", Value: syscall.clone_newuser, File: "user"}, {Key: "NEWPID", Value: syscall.clone_newpid, File: "pid"}, {Key: "NEWNET", Value: syscall.clone_newnet, File: "net"}, } } USER_NAMESPACE: not fully implemented in Docker NET_NAMESPACE: not used in network mode host and other container Based on libcontainer v1.2.0

10 What to Fork? Docker Client Docker Daemon fork with flags! fork Docker Container?? Docker Container == Process(es)?? Docker Container

11 What Process to Fork? Whatever! A process indeed. Process is just forked, not execed yet. Result is like below: task_struct namespaces other resources ready ready ready Process is still static, no program is running.

12 Then exec! exec what? Have you ever heard of dockerinit, ENTRYPOINT or CMD in Docker? name dockerinit ENTRYPOINT CMD description init thing that first runs inside a new namespace to setup mount, net namespaces and other things. An ENTRYPOINT allows you to configure a container that will run as an executable The main purpose of a CMD is to provide defaults for an executing container. Reference:

13 Dockerinit, ENTRYPOINT, CMD Docker Daemon init namespaces process exec fork new namespaces dockerinit ENTRYPOINT CMD the only process (same PID)

14 Docker Daemon and dockerinit parent Docker Daemon syncpipe Usage: coordnate the sequential of Docker Daemon and dockerinit. child dockerinit Dockerinit will be blocked if nothing read in syncpipe. Why?

15 How to coordinate? Docker Daemon 1.Create Command The executable in container(dockerint) 2.Create syncpipe 3.Pass pipe to Child 4. command.start() Fork and exec the command dockerinit fork, new PID! syncpipe(nothing) blocked 5. SetupCgroups syncpipe(nothing) blocked, controlled by cgroup 6. init network syncpipe(nothing) blocked, controlled by cgroup 7.Sync with Child syncpipe(has networkstate) read from syncpipe Based on libcontainer v1.2.0

16 How to coordinate? Docker Daemon 1.SetupNetwork 2.SetupRoute 3.Init Mount ns 4.Apply apparmor 5.execv Entrypoint dockerinit Setup devices, mount points and fs ENTRYPOINT exec, same PID! x. execv Cmd CMD exec, same PID! 8.command.wait() Finally, YOUR APP! Based on libcontainer v1.2.0

17 Docker Container Docker Daemon process exec fork init namespaces new namespaces cgroups applied dockerinit ENTRYPOINT CMD (your application) the only process (same PID) Docker Container process process process process

18 Why to Coordinate? 1. Docker Daemon needs to Synchronize with dockerinit. block dockerinit so no children of dockerinit can escape from cgroups. 2. Can not switch namespace in Go runtime. blocked until Docker Daemon transfers network details that will be used to setup network interface in newnet namespace.

19

20 Q&A

21 PRESENTATION TITLE THANK YOU! SPEAKER NAME 2014 / 12 /09 allen.sun@daocloud.io 莲子弗如清 webchat: shlallen