Dockerized 1 Tizen Platform Copyright 2017 Samsung. All Rights Reserved.
Abstract Tizen Pla.orm ECO System Container ECO System Build CI Management (Update) Cloud Monitoring Store Data (DB) Cloud 2 Cloud Infrastructure Dockeriza:on Docker update
Agenda Platform Management Docker Introduction Embedded Container Dockerization Demo Challenges 3
4 Why We Research? PLATFORM MANAGEMENT
Platform Mgt. Situation Development, Deployment & Operations Tizen Platform have life-cycles? Platform builder F/W upgrade Remote control 5
Platform Mgt. Challenges In IoT devices, The platform should be Simple as a single application Faster to create application Easy to distribute Support remote control (update, monitoring, ) Safe for system failure 6 Docker can be a solution?
7 What is Docker. DOCKER INTRODUCTION
Docker Introduced in 2013 8 h1ps://blog.docker.com
Docker Basic Concept Container - Similar to VM - but, based on Linux system call (no Virtual OS) - OCI (Open Container Initiative) - Isolated name space with executable packages 9 Docker (Container platform) - Build container image, Run container - ECO system for container image - Services (deploy, management) h1ps://www.docker.com/what- container
Docker Basic Workflow 10 h1ps://docs.docker.com/engine/docker- overview
Docker Extended Workflow Orchestration Management - Connection to cloud server - Device Clustering SERVICE Cloud Server SERVICE Node Cluster 11 NODE- A NODE- B NODE- C NODE- D replica9on POD- A container A container container B container C container D container E container E container E Docker Docker Docker Docker HW- A HW- B HW- C
Docker Services Monitoring - Host : CPU load, Memory, Disk Space, Running containers / Host UP time - Containers : CPU load, Memory, Disk I/O, Network I/O Container Deploying 12 - Rolling update, Rollback Logging - System log, Containers log Container Mgt. - Scaling, load balancing
13 Why We Use Docker. EMBEDDED CONTAINER
Embedded Container Concept Docker in embedded device Container has a initializer (/sbin/init instead of /bin/bash) Running container with privileged permission Full HW resources Embedded Pla.orm Container 14 Lightweight Host OS Docker Linux Kernel
Embedded Container Usage Exis:ng Usage for Server New Usage for Embedded Device Cloud Service U:lize Cloud Service 15 container container container A A A container container B container container container A A A container container B PlaHorm + App Container A PlaHorm + App Container A Docker Docker Lightweight Host OS Lightweight Host OS Server Infra structure Docker Docker Service oriented (regardless of physical device) Homogeneous app containers in server infra Device oriented Homogeneous app containers in different device Proper to IoT system
Embedded Container Tizen Platform Platform Managements with Docker Docker service features Build Deployment Update Docker- registry Pla.orm management tools CreaMon/modificaMon DistribuMon Upgrade PlaHorm store 16 Tizen Platform as a Embedded Container Container Tizen PlaHorm Linux Kernel Dockeriza:on Lightweight Host OS Docker Linux Kernel
17 What We Are Trying DOCKERIZATION
Overall Architecture [Tizen Pla.orm Containers] Cloud Server Mzen- headless kernel + Host Docker Registry [Container] Mzen- headless dockzen - agent update security monitor 18 Create images (+ fw) Mul:media fw Mzen- headless kernel + Host Create images (+App) Voice App Mzen- headless kernel + Host docker- client docker- engine docker- daemon container- ctr container- shim OCI::runc swarm containerd dockzen- launcher Network (Wi- Fi) ca- cermficate [Host OS] Mzen- minimal / bare- os Linux kernel
Dockerization Kernel Patches Kernel Has Docker Dependencies Container Host OS Docker & FW Kernel Enable cgroup iptables error roohs mount error FATA[0001] Error starmng daemon: Devices cgroup isn't mounted Fix : { CONFIG_CGROUP_DEVICE=y, CONFIG_CPUSETS=y, CONFIG_BLK_CGROUP=y} FATA[0002] Error starmng daemon: Error inimalizing network controller: Error creamng default "bridge" network: Failed to program NAT chain: Failed to inject docker in PREROUTING chain: iptables failed: iptables - - wait - t nat - A PREROUTING - m addrtype - - dst- type LOCAL - j DOCKER: iptables: No chain/target/match by that name. Fix : {CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=y } error=oci runmme error: container_linux.go:247: starmng container process caused "process_linux.go: 359: container init caused \"roohs_linux.go:54: mounmng \\\" to roohs \\\" at \\\" caused \\\ such device\\\"\" Fix : {CONFIG_POSIX_MQUEUE=y} 19 cgroup memory path error ERRO[0187] containerd: nomfy OOM events error=cgroup path for memory not found panic: standard_init_linux.go:175: exec user process caused "exec format error Fix : {CONFIG_MEMCG=y, CONFIG_MEMCG_SWAP=y, CONFIG_MEMCG_KMEM=y} Enable Overlayfs docker- runc keyring failure Fix : {CONFIG_OVERLAY_FS=y} error=oci runmme error: container_linux.go:247: starmng container process caused "process_linux.go:359: container in it caused \"could not create session key: funcmon not implement Fix : enable keyctl syscall compambility for 32bit with 64bit kernel {CONFIG_KEYS_COMPAT}
Container Dockerization Host OS Host OS Docker & FW Kernel Required Packages in Host-OS rootfs cgroup Network Certification Docker & Frameworks docker- client docker- engine docker- daemon container- ctr container- shim OCI::runc dockzen- agent swarm containerd update security monitor dockzen- launcher Network (Wi- Fi) ca- cermficate [Host OS] Mzen- minimal / bare- os 20 Linux kernel
Dockerization Host OS Container Host OS Docker & FW Kernel Two Candidates Using Tizen subset (Tizen minimal) Create New for docker (BareOS) Tizen minimal BareOS Arch type arm arm Size (ROM) 123MB 66M Size (RAM) 250MB (run dockerd : 311MB) 53M (run dockerd : 113M) Kernel version 4.4.19 4.4.19 Docker version v1.13.1 v1.13.1 Init system systemd sysvinit Package manager tpk None Filesystem ext4 ext4 docker : 52MB cermficate : 1MB wifi netconfig base : about 60MB docker : 52MB cermficate : 1MB wifi / base : about 13MB 21 Tizen minimal RAM Size (113 MB) BareOS RAM Size (66 MB)
Dockerization Dockzen-launcher Container Host OS Docker & FW Kernel Manage docker life-cycle Manage Container life-cycle Monitoring APIs dockzen- launcher dockzen- agent command test 22 API Service MainLoop state API parser json parser config file content device dockerd connect systemd docker engine
Dockerization Dockzen-agent Container Host OS Docker & FW Kernel Binding as a Container Connection to Cloud dockzen- agent Manage Device uuid Authentication Configure Update Policy web connecmon <<back- end>> API agent Server 23 converter connect dockzen- launcher
Containerization Initial Creation Platform Binaries to Tizen Container Image In Host PC 1. Download platform binaries (https://download.tizen.org/) 2. Loopback mount using mnt-img.sh $./mnt-img.sh mount tizen-common_xxx_common-wayland-3parts-armv7l-artik.tar.gz 3. Compress tarball $ sudo tar --xattrs -cvf../[tar-name]. In Target 4. Docker-import $ cat [tar-name] docker import [local-container-name] 5. Push into Docker-Hub $ docker tag [local-container-name] [dockerhub-id]/[image-name] $ docker push [dockerhub-id]/[image-name] Container Host OS Docker & FW Kernel 24
Container Containerization Re-Creation Docker-Build with Dockerfile 1. Install yum pkg-mgr Add yum into base container image v yum package files Host OS Docker & FW Kernel ### dockerfile for added yum_pkg and exampleapp ### FROM base- image # install yum # ADD yum/yum_pkg /usr/tmp/yum_pkg/ RUN rpm - Uvh - - nodeps - - force /usr/tmp/yum_pkg/*.rpm ADD yum/*.repo /etc/yum.repos.d/ 2. Case Study Add curl application à New Image ### install rpm pkg and exampleapp ### FROM base- image- yum # install packages # RUN yum install curl ### base_packages.repo [base_packages] name=base_packages type=rpm- md baseurl= h1ps://download.mzen.org/snapshots/mzen/base/latest/repos/arm/ packages enabled=1 gpgcheck=0 sslverify=false ### common_packages.repo [common_packages] name=common_packages type=rpm- md baseurl= h1ps://download.mzen.org/snapshots/mzen/common/latest/repos/ arm- wayland/packages enabled=1 gpgcheck=0 sslverify=false 25
Issues Smack Security Tizen uses Smack Security Extended attributes : security.smack64, security.capability Need to check xattr operations in docker patch#1 : Capability error Failure in Tizen Container running Occurred permission error checking CAP_MAC_ADMIN In OverlayFS, upper layer can t sync into lower layer as permission http://www.spinics.net/lists/linux-unionfs/msg00593.html patch#2 : xattr copy error Failure in docker commands (commit, push, ) Extended attribute lists doesn t be copied (in case of overlay, not overlay2) 26
Issues Privileged Container /sbin/init (systemd) vs. /bin/bash Much discussions about systemd in docker systemd requires privileged permission Initialize overall services regarding HW devices Necessary in Tizen container Patches adding -- privileged Docker-build Docker-service 27
Issues Union File System Union file system Handled by layer architecture Avoid duplication and isolation History Early 2013 : AUFS Late 2013 : Device Mapper Early 2017 : Overlay 28 Apply for Tizen OverlayFS Stability / mainline support Performance
Quality Inspection Security Need to minimize privileged permission Fail safe Robust Host-os Container can be recovered(reboot) Resource management Violation occurred in network resource CPU and memory is separated Disk is controlled by same journaling thread 29
30 What We Have Done. DEMO
Scenario Structure Build Tizen Container Image <3 rd Party Develop> Product Container Image Release Push New Image Docker Registry (official / public) Docker Registry (public / private) Developers Dash-board Update Monitoring Service Server Docker Registry Web UI Image Repository 31 Register Devices Embedded Device (ARTIK710) Update Images
Demo Video Bring up 32 Dashboard Update
Demo Structures Dash- board Web Docker- registry Web websocket PoC Server server registry- web 33 Target Device Container / Mzen- headless Container / others H1p Server H1p Server [dockzen- OS] base on Mzen- minimal docker api dockzen- launcher IPC dockzen- agent agent backend websocket container mgt. dockzen- backend registry Docker Registry docker- engine rest Linux kernel 4.4 ARTIK7
Development Packages Artik7 boot&kernel Host os docker-engine docker framework Tizen container image 34 Instructions Download boot&kernel Download host os Execute Tizen container image (only first time)
35 What We Try. CHALLENGES
Next Improvement Extend target device (raspi-3) Create Tizen 4.0 reference container images Optimize host-os embedded on Docker 36 Serviceability Service to support Tizen docker is in development 3 rd Party can deploy Tizen docker in the future
Contributing Github organization : https://github.com/dockzen Docker source-code (patched for tizen) https://github.com/dockzen/docker https://github.com/dockzen/containerd https://github.com/dockzen/runc Docker framework https://github.com/dockzen/dockzen-launcher https://github.com/dockzen/dockzen-agent Host-os : https://github.com/dockzen/dockzen-os Artik7 kernel : https://github.com/dockzen/linux-artik7-docker Docker-hub containers https://hub.docker.com/u/dockzen/ 37