Control Center Installation Guide

Transcription

1 Control Center Instlltion Guide Relese Zenoss, Inc.

2 Control Center Instlltion Guide Copyright 2016 Zenoss, Inc. All rights reserved. Zenoss nd the Zenoss logo re trdemrks or registered trdemrks of Zenoss, Inc., in the United Sttes nd other countries. All other trdemrks, logos, nd service mrks re the property of Zenoss or other third prties. Use of these mrks is prohiited without the express written consent of Zenoss, Inc., or the third-prty owner. Linux is registered trdemrk of Linus Torvlds. All other compnies nd products mentioned re trdemrks nd property of their respective owners. Prt Numer: Zenoss, Inc Four Points Drive Bldg 1 - Suite 300 Austin, Texs

3 Contents Aout this guide...5 Chpter 1: Instlling mster host... 7 Verifying cndidte host resources... 7 Mster host storge requirements...9 Disling consistent network device nming...11 Prepring the mster host operting system...12 Instlling Docker Engine Instlling Control Center Configuring Docker Engine...17 Optionl: Loding imge files...19 Creting the ppliction dt thin pool Chpter 2: Configuring nd strting the mster host Control Center mintennce scripts on the mster host...21 User ccess control Configuring the se size device for tennt dt storge Setting the host role to mster Optionl: Configuring offline use Mster host configurtion vriles Universl configurtion vriles...29 Strting Control Center for the first time...31 Adding the mster host to resource pool Chpter 3: Instlling delegte hosts...33 Verifying cndidte host resources Delegte host storge requirements...35 Disling consistent network device nming...35 Prepring delegte host...36 Instlling Docker Engine Instlling Control Center Configuring Docker Chpter 4: Configuring nd strting delegte hosts...43 Control Center mintennce scripts on delegte hosts...43 Enling use of the commnd-line interfce...43 Setting the host role to delegte...44 Setting internl services endpoints Optionl: Configuring offline use Delegte host configurtion vriles Universl configurtion vriles...48 Strting Control Center...49 Delegte host uthentiction

4 Chpter 5: Configuring ZooKeeper ensemle ZooKeeper nd Control Center Understnding the configurtion process Configuring the mster host s ZooKeeper node...54 Configuring delegte host A s ZooKeeper node...55 Configuring delegte host B s ZooKeeper node Strting ZooKeeper ensemle Updting delegte hosts...60 Appendix A: Strting nd stopping Control Center deployments Stopping Control Center (single-host deployment)...61 Strting Control Center (single-host deployment) Stopping Control Center (multi-host deployment)...62 Strting Control Center (multi-host deployment) Appendix B: Storge mngement utility...68 serviced-storge...68 Appendix C: Control Center configurtion vriles Best prctices for configurtion files Control Center configurtion file Appendix D: Prepring to instll without internet ccess...82 Downloding repository nd imge files Instlling the repository mirror...83 Stging Docker imge rchive files Appendix E: Configuring privte mster NTP server Configuring n NTP mster server Configuring NTP clients

5 Aout this guide Aout this guide Control Center Instlltion Guide provides detiled procedures for instlling nd configuring Control Center cluster. Plese review the Control Center Plnning Guide crefully efore using this guide. Zenoss customers: This guide does not include procedures for instlling high-vilility deployment. For dditionl informtion, plese contct your Zenoss representtive. Relted pulictions Title Control Center Relese Notes Control Center Plnning Guide Control Center Instlltion Guide Control Center Reference Guide Control Center Upgrde Guide Description Descries known issues, fixed issues, nd lte-reking informtion not included in other pulictions. Provides oth generl nd specific informtion out prepring to deploy Control Center cluster. Provides detiled procedures for instlling nd configuring Control Center cluster. Provides informtion nd procedures for mnging Control Center. This informtion is lso ville s online help in the Control Center rowser interfce. Provides detiled procedures for updting Control Center deployment to the ltest relese. Documenttion feedck To provide feedck out this document, or to report n error or omission, plese send n emil to docs@controlcenter.io. In the emil, plese include the document title nd prt numer, nd s much informtion s possile out the context of your feedck. The prt numer ppers t the end of the list of trdemrks, t the front of this guide. Supported clients nd rowsers The following tle identifies the supported comintions of client operting systems nd we rowsers. Client OS Windows 7 nd 8.1 Windows Server 2012 R2 Mcintosh OS/X 10.9 Uuntu LTS Supported Browsers Internet Explorer 11 (Enterprise mode only; comptiility mode is not supported.) Internet Explorer 10 (Comptiility mode is not supported.) Firefox 30 nd ove Chrome 30 nd ove Firefox 30 Chrome 36 Firefox 30 nd ove Chrome 36 nd ove Firefox 30 nd ove Chrome 37 nd ove 5

6 Control Center Instlltion Guide Client OS Red Ht Enterprise Linux 6.5, CentOS 6.5 Supported Browsers Firefox 30 nd ove Chrome 37 nd ove Chnge history The following list ssocites document prt numers nd the importnt chnges to this guide since the previous relese. Some of the chnges involve fetures or content, ut others do not. For informtion out new or chnged fetures, refer to the Control Center Relese Notes Add new est prctice guideline for mnging configurtion files Initil relese. 6

7 Instlling mster host Instlling mster host 1 This chpter descries how to instll Control Center on Red Ht Enterprise Linux (RHEL) or CentOS 7.1 or 7.2 host. The cndidte host must hve the CPU, RAM, nd storge resources required to serve s the mster host in Control Center cluster. For more informtion out mster host requirements, refer to the Control Center Plnning Guide. Verifying cndidte host resources Use this procedure to determine whether the hrdwre resources nd instlled operting system of host re sufficient to serve s Control Center mster host. 1 Log in to the cndidte host s root, or s user with superuser privileges. 2 Verify tht the host implements the 64-it version of the x86 instruction set. unme -m If the output is x86_64, the rchitecture is 64-it. Proceed to the next step If the output is i386/i486/i586/i686, the rchitecture is 32-it. Stop this procedure nd select different host. 3 Determine whether the CPU resources re sufficient. Disply the totl numer of CPU cores. ct /proc/cpuinfo grep -Ec '^core id' Compre the ville resources with the requirements for Control Center mster host. For more informtion, refer to the Control Center Plnning Guide. 4 Determine whether the CPU resources support the AES instruction set. ct /proc/cpuinfo grep -Ec '^flgs.*es' For optiml performnce, the result of the preceding commnds must mtch the totl numer of CPU resources ville on the host. If the result is 0, performnce is severely degrded. If the result is 0 nd the cndidte host is virtul mchine, the mnging hypervisor my e configured in Hyper-V comptiility mode. Check the setting nd disle it, if possile, or select different host. 5 Determine whether the ville memory nd swp is sufficient. 7

8 Control Center Instlltion Guide Disply the ville memory. free -h Compre the ville memory with the mount required for mster host in your deployment. For more informtion, refer to the Control Center Plnning Guide. 6 Ensure the host hs persistent numeric ID. Skip this step if you re instlling single-host deployment. Ech host in Control Center cluster must hve unique host ID, nd the ID must e persistent (not chnge when the host reoots). test -f /etc/hostid genhostid hostid Record the ID for comprison with other hosts in the cluster. 7 Verify tht nme resolution works on this host. hostnme -i If the result is not vlid IPv4 ddress, dd n entry for the host to the network nmeserver, or to /etc/ hosts. 8 Add n entry to /etc/hosts for loclhost, if necessry. Determine whether is mpped to loclhost. grep /etc/hosts grep loclhost If the preceding commnds return no result, perform the following sustep. Add n entry to /etc/hosts for loclhost. echo " loclhost" >> /etc/hosts 9 Updte the operting system, if necessry. Determine which relese is instlled. ct /etc/redht-relese If the result includes 7.0, perform the following susteps. Updte the operting system nd then restrt the host. The following commnds require internet ccess or locl mirror of operting system pckges. yum mkecche fst && yum updte -y && reoot 10 Updte the Linux kernel, if necessry. Determine which kernel version is instlled. unme -r If the result is lower thn el7.x86_64, perform the following susteps. Updte the kernel or the operting system, nd then restrt the host. The following commnds require internet ccess or locl mirror of operting system pckges. Updte only the kernel: yum mkecche fst && yum updte -y kernel && reoot 8

9 Instlling mster host Updte the operting system, including the kernel: yum mkecche fst && yum updte -y && reoot Mster host storge requirements This section provides quick reference for the minimum storge requirements of Control Center mster host. These requirements re in ddition to the storge required for the host's operting system. For more informtion, refer to the Control Center Plnning Guide. Control Center cluster hosts need either unformtted lock storge devices or prtitions, or free spce in one or more LVM volume groups. Enter the following commnd to disply informtion out lock storge: lslk -p --output=name,size,type,fstype,mountpoint Enter the following commnd to disply informtion out LVM volume groups: vgdisply Compre the output with the informtion in the following tle. Purpose Minimum size Description Docker dt 50GB (required) Locl, high-performnce storge. Control Center internl services dt 50GB (required) Locl, high-performnce storge formtted s n XFS filesystem. Appliction dt 200GB (suggested) Locl, high-performnce storge. Appliction dt ckups 150GB (suggested) A remote file server comptile with XFS or locl storge formtted s n XFS filesystem. If the storge is locl, ensure tht it does not cuse contention with the storge for Control Center internl services. The storge for Docker dt nd ppliction dt is configured s LVM thin pools in susequent procedures. The suggested minimum sizes for ppliction dt nd ppliction dt ckups should e replced with sizes tht meet your ppliction requirements. To clculte the pproprite sizes for these storge res, use the following guidelines: Appliction dt storge includes spce for oth dt nd snpshots. The defult se size for dt is 100GB, nd the recommended spce for snpshots is 100% of the se size. Adding the two yields the suggested minimum size of 200GB. For ppliction dt ckups, the recommended spce is 150% of the se size for dt. The following sections provide procedures for formtting nd mounting locl storge s XFS filesystems for Control Center internl services dt nd ppliction dt ckups. Creting filesystem for Control Center internl services This procedure requires one unused device or prtition. Use this procedure to crete n XFS filesystem on n unused device or prtition. 9

10 Control Center Instlltion Guide 1 Log in to the trget host s root, or s user with superuser privileges. 2 Identify the trget device or prtition for the filesystem to crete. lslk -p --output=name,size,type,fstype,mountpoint 3 Crete n XFS filesystem. Replce Storge with the pth of the trget device or prtition: mkfs.xfs Storge 4 Enter the following commnd to dd n entry to the /etc/fst file. Replce Storge with the pth of the device or prtition used in the previous step: echo "Storge /opt/serviced/vr/isvcs xfs defults 0 0" >> /etc/fst 5 Crete the mount point for internl services dt. mkdir -p /opt/serviced/vr/isvcs 6 Mount the filesystem, nd then verify it mounted correctly. mount - && mount grep isvcs Exmple result: /dev/xvd1 on /opt/serviced/vr/isvcs type xfs (rw,reltime,ttr2,inode64,noquot) Creting filesystem for ppliction dt ckups This procedure requires one unused device or prtition, or remote file server tht is comptile with XFS. Use this procedure crete n XFS filesystem on device or prtition, or to mount remote filesystem, for ppliction dt ckups. Note If you re using prtition on locl device for ckups, ensure tht the storge for Control Center internl services dt is not on the sme device. 1 Log in to the trget host s root, or s user with superuser privileges. 2 Optionl: Identify the trget device or prtition for the filesystem to crete, if necessry. Skip this step if you re using remote file server. lslk -p --output=name,size,type,fstype,mountpoint 3 Optionl: Crete n XFS filesystem, if necessry. Skip this step if you re using remote file server. Replce Storge with the pth of the trget device or prtition: mkfs.xfs Storge 4 Crete n entry in the /etc/fst file. Replce File-System-Specifiction with one of the following vlues: the pth of the device or prtition used in the previous step 10

11 Instlling mster host the remote server specifiction echo "File-System-Specifiction \ /opt/serviced/vr/ckups xfs defults 0 0" >> /etc/fst 5 Crete the mount point for ckup dt. mkdir -p /opt/serviced/vr/ckups 6 Mount the filesystem, nd then verify it mounted correctly. mount - && mount grep ckups Exmple result: /dev/sd3 on /opt/serviced/vr/ckups type xfs (rw,reltime,seclel,ttr2,inode64,noquot) Disling consistent network device nming The consistent network device nming feture cn crete device nmes tht re too long for use with virtul IPs. This step disles the feture so tht ll network device nmes re eth followed y n integer. 1 Log in s root, or s user with superuser privileges. 2 Edit the GRUB 2 generl settings file. c Crete ckup copy of the settings file. cp /etc/defult/gru /etc/defult/gru.k Open /etc/defult/gru with text editor. Add kernel oot rguments to the vlue of the GRUB_CMDLINE_LINUX vrile. The rguments to dd re iosdevnme=0 nd net.ifnmes=0. Mke sure the rguments re etween the delimiter chrcters ("). d Sve the file, nd then close the text editor. 3 Determine whether the host is configured for UEFI or legcy oot mode. find /oot -mxdepth 1 -type d If the output includes /oot/efi, the host is configured for UEFI oot mode. If the output does not include /oot/efi, the host is configured for legcy oot mode. 4 Recrete the GRUB 2 oot configurtion file. For UEFI oot mode: gru2-mkconfig -o /oot/efi/efi/centos/gru.cfg For legcy oot mode: gru2-mkconfig -o /oot/gru2/gru.cfg 5 Crete n override for the defult rules policy of the dynmic device mngement demon. ln -s /dev/null /etc/udev/rules.d/80-net-nme-slot.rules 11

12 Control Center Instlltion Guide 6 Chnge network device configurtions. Chnge directory to /etc/sysconfig/network-scripts. cd /etc/sysconfig/network-scripts List the network device configurtion files. c ls ifcfg-* grep -v ifcfg-lo The ifcfg-lo file is excluded ecuse it does not need to e chnged. Renme nd edit device files. For exmple, if you hve device file nmed ifcfg-eno , use the mv commnd renme it to ifcfg-eth0, nd then chnge the vlues of the NAME nd DEVICE vriles inside the file to eth0. 7 Reoot the host. Prepring the mster host operting system Note Zenoss customers: If you re creting deployment without internet ccess, perform the steps in Prepring to instll without internet ccess on pge 82, efore performing this procedure. Use this procedure to prepre RHEL/CentOS 7.1 or 7.2 host s Control Center mster host. 1 Log in to the cndidte mster host s root, or s user with superuser privileges. 2 Disle the firewll, if necessry. This step is required for instlltion ut not for deployment. For more informtion, refer to the Control Center Plnning Guide. Determine whether the firewlld service is enled. systemctl sttus firewlld.service If the result includes Active: inctive (ded), the service is disled. Proceed to the next step. If the result includes Active: ctive (running), the service is enled. Perform the following sustep. Disle the firewlld service. systemctl stop firewlld && systemctl disle firewlld On success, the preceding commnds disply messges similr to the following exmple: rm '/etc/systemd/system/dus-org.fedorproject.firewlld1.service' rm '/etc/systemd/system/sic.trget.wnts/firewlld.service' 3 Optionl: Enle persistent storge for log files, if desired. By defult, RHEL/CentOS systems store log dt only in memory or in ring uffer in the /run/log/ journl directory. By performing this step, log dt persists nd cn e sved indefinitely, if you implement log file rottion prctices. For more informtion, refer to your operting system documenttion. mkdir -p /vr/log/journl && systemctl restrt systemd-journld 4 Disle Security-Enhnced Linux (SELinux), if instlled. 12

13 Instlling mster host Determine whether SELinux is instlled. test -f /etc/selinux/config && grep '^SELINUX=' /etc/selinux/config c If the preceding commnds return result, SELinux is instlled. Set the operting mode to disled. Open /etc/selinux/config in text editor, nd chnge the vlue of the SELINUX vrile to disled. Confirm the new setting. grep '^SELINUX=' /etc/selinux/config 5 Enle nd strt the Dnsmsq pckge. The pckge fcilittes networking mong Docker continers. systemctl enle dnsmsq && systemctl strt dnsmsq If nme resolution in your environment relies solely on entries in /etc/hosts, configure dsnmsq so tht continers cn use the file: c d e f g h i j k Open /etc/dnsmsq.conf with text editor. Locte the line tht strts with #domin-needed, nd then mke copy of the line, immeditely elow the originl. Remove the numer sign chrcter (#) from the eginning of the line. Locte the line tht strts with #ogus-priv, nd then mke copy of the line, immeditely elow the originl. Remove the numer sign chrcter (#) from the eginning of the line. Locte the line tht strts with #locl=/loclnet/, nd then mke copy of the line, immeditely elow the originl. Remove net, nd then remove the numer sign chrcter (#) from the eginning of the line. Locte the line tht strts with #domin=exmple.com, nd then mke copy of the line, immeditely elow the originl. Replce exmple.com with locl, nd then remove the numer sign chrcter (#) from the eginning of the line. Sve the file, nd then close the editor. Restrt the dnsmsq service. systemctl restrt dnsmsq 6 Instll nd configure the NTP pckge. Note This procedure ssumes the host hs internet ccess. To instll nd configure NTP on host tht does not hve internet ccess, see Configuring privte mster NTP server on pge 84. Instll the pckge. yum instll -y ntp Set the system time. ntpd -gq 13

14 Control Center Instlltion Guide c Enle the ntpd demon. systemctl enle ntpd d Configure ntpd to strt when the system strts. Currently, n unresolved issue ssocited with NTP prevents ntpd from restrting correctly fter reoot. The following commnds provide workround to ensure tht it does. echo "systemctl strt ntpd" >> /etc/rc.d/rc.locl chmod +x /etc/rc.d/rc.locl 7 Add the Zenoss repositories to the host's repository list. Instll the repository pckge. The pckge dds the Zenoss stle, unstle, nd testing repositories to /etc/yum.repos.d. All three repositories re disled. rpm -ivh Clen out the yum cche directory. yum clen ll 8 Reoot the host. reoot Instlling Docker Engine Use one of the procedures in this section to instll Docker Engine. Instlling Docker Engine with internet ccess Use this procedure to instll Docker Engine on host tht hs internet ccess. 1 Log in to the host s root, or s user with superuser privileges. 2 Instll Docker Engine , nd then disle ccidentl upgrdes. Add the Docker repository to the host's repository list. ct <<EOF > /etc/yum.repos.d/docker.repo [dockerrepo] nme=docker Repository seurl= enled=1 gpgcheck=1 gpgkey= EOF Clen the yum cche nd updte repository metdt. yum clen ll && yum mkecche fst c Instll Docker Engine from the remote repository. yum instll -y docker-engine

15 Instlling mster host d Open /etc/yum.repos.d/docker.repo with text editor. e Chnge the vlue of the enled key from 1 to 0. f Sve the file, nd then close the text editor. Optionl: Instlling Docker Engine without internet ccess Zenoss customers: If you re instlling Docker Engine without internet ccess, perform the steps in Prepring to instll without internet ccess on pge 82 efore performing this procedure. Use this procedure to instll Docker Engine on host tht does not hve internet ccess. 1 Log in to the host s root, or s user with superuser privileges. 2 Instll Docker Engine from the locl repository mirror. Clen the yum cche nd updte repository metdt. yum clen ll && yum mkecche fst Instll Docker Engine. yum instll --enlerepo=zenoss-mirror -y docker-engine Instlling Control Center Use one of the procedures in this section to instll Control Center. Instlling Control Center with internet ccess Use this procedure to instll Control Center on host tht hs internet ccess. 1 Log in to the host s root, or s user with superuser privileges. 2 Instll Control Center Clen the yum cche nd updte repository metdt. yum clen ll && yum mkecche fst Instll Control Center from the remote repository. yum --enlerepo=zenoss-stle instll -y serviced Mke ckup copy of the Control Center configurtion file. Mke copy of /etc/defult/serviced. cp /etc/defult/serviced /etc/defult/serviced orig Set the ckup file permissions to red-only. chmod 0440 /etc/defult/serviced orig 4 Add drop-in file for the NFS service. This step is workround for n n unresolved issue. Crete directory for the drop-in file. mkdir -p /etc/systemd/system/nfs-server.service.d 15

16 Control Center Instlltion Guide Crete the drop-in file. ct <<EOF > /etc/systemd/system/nfs-server.service.d/nfs-server.conf [Unit] Requires= Requires= network.trget proc-fs-nfsd.mount rpcind.service Requires= nfs-mountd.service EOF c Relod the systemd mnger configurtion. systemctl demon-relod Optionl: Instlling Control Center without internet ccess Zenoss customers: Perform the steps in Prepring to instll without internet ccess on pge 82 efore performing this procedure. Use this procedure to instll Control Center on host tht does not hve internet ccess. 1 Log in to the mster host s root, or s user with superuser privileges. 2 Instll Control Center from the locl repository mirror. Clen the yum cche nd updte repository metdt. yum clen ll && yum mkecche fst Instll Control Center. yum instll --enlerepo=zenoss-mirror -y serviced Mke ckup copy of the Control Center configurtion file. Mke copy of /etc/defult/serviced. cp /etc/defult/serviced /etc/defult/serviced orig Set the ckup file permissions to red-only. chmod 0440 /etc/defult/serviced orig 4 Add drop-in file for the NFS service. This step is workround for n n unresolved issue. Crete directory for the drop-in file. mkdir -p /etc/systemd/system/nfs-server.service.d Crete the drop-in file. ct <<EOF > /etc/systemd/system/nfs-server.service.d/nfs-server.conf [Unit] Requires= Requires= network.trget proc-fs-nfsd.mount rpcind.service Requires= nfs-mountd.service EOF 16

17 Instlling mster host c Relod the systemd mnger configurtion. systemctl demon-relod Configuring Docker Engine Use this procedure to configure Docker Engine. 1 Log in to the mster host s root, or s user with superuser privileges. 2 Crete symolic link for the Docker temporry directory. Docker uses its temporry directory to spool imges. The defult directory is /vr/li/docker/tmp. The following commnd specifies the sme directory tht Control Center uses, /tmp. You cn specify ny directory tht hs minimum of 10GB of unused spce. Crete the docker directory in /vr/li. mkdir /vr/li/docker Crete the link to /tmp. ln -s /tmp /vr/li/docker/tmp 3 Crete systemd drop-in file for Docker Engine. Crete the override directory. mkdir -p /etc/systemd/system/docker.service.d Crete the unit drop-in file. ct <<EOF > /etc/systemd/system/docker.service.d/docker.conf [Service] TimeoutSec=300 EnvironmentFile=-/etc/sysconfig/docker ExecStrt= ExecStrt=/usr/in/dockerd \$OPTIONS TsksMx=infinity EOF c Relod the systemd mnger configurtion. systemctl demon-relod 4 Crete n LVM thin pool for Docker dt. For more informtion out the serviced-storge commnd, see serviced-storge on pge 68. Replce Storge with the pths of one or more lock devices or prtitions, or the nme of n LVM volume group: serviced-storge crete-thin-pool --size=50g docker Storge On success, the result is the device mpper nme of the thin pool, which lwys strts with /dev/mpper. 5 Configure nd strt the Docker service. Crete vrile for the nme of the Docker thin pool. 17

18 Control Center Instlltion Guide Replce Thin-Pool-Device with the nme of the thin pool device creted in the previous step: mypool="thin-pool-device" Crete vrile for the mster host. Replce Mster-Host with the hostnme or IPv4 ddress of the mster host: msterhost="mster-host" c d e The vlue of this vrile must mtch the vlue of the SERVICED_DOCKER_REGISTRY vrile in / etc/defult/serviced. Crete vriles for dding rguments to the Docker configurtion file. The --exec-opt rgument is workround for Docker issue on RHEL/CentOS 7.x systems. mydriver="--storge-driver devicempper" mylog="--log-level=error" myfix="--exec-opt ntive.cgroupdriver=cgroupfs" mymount="--storge-opt dm.mountopt=discrd" myflg="--storge-opt dm.thinpooldev=$mypool" myreg="--insecure-registry=$msterhost:5000" Add the rguments to the Docker configurtion file. echo 'OPTIONS="'$myLog $mydriver $myfix $mymount $myflg $myreg'"' \ >> /etc/sysconfig/docker Strt or restrt Docker. systemctl restrt docker The strtup my tke up to minute, nd my fil. If strtup fils, repet the restrt commnd. 6 Configure nme resolution in continers. Ech time it strts, docker selects n IPv4 sunet for its virtul Ethernet ridge. The selection cn chnge; this step ensures consistency. Identify the IPv4 sunet nd netmsk docker hs selected for its virtul Ethernet ridge. c ip ddr show docker0 grep inet Open /etc/sysconfig/docker in text editor. Add the following flgs to the end of the OPTIONS declrtion. Replce Bridge-Sunet with the IPv4 sunet docker selected for its virtul ridge: --dns=bridge-sunet --ip=bridge-sunet/16 For exmple, if the ridge sunet is , dd the following flgs: --dns= ip= /16 Note Use spce chrcter ( ) to seprte flgs, nd mke sure the doule quote chrcter (") delimits the declrtion of OPTIONS. 18

19 Instlling mster host d Restrt the Docker service. systemctl restrt docker Optionl: Loding imge files Zenoss customers: Perform the steps in Prepring to instll without internet ccess on pge 82 efore performing this procedure. Use this procedure to lod imges into the locl repository on host tht does not hve internet ccess. 1 Log in to the mster host s root, or s user with superuser privileges. 2 Chnge directory to /root. cd /root 3 Extrct the imges. for imge in instll-*.run do echo -n "$imge: " evl./$imge done Imge extrction egins when you press the y key. If you press the y key nd then Return key, the current imge is extrcted, ut the next one is not. 4 List the imges in the registry. docker imges The result should show one imge for ech rchive file. 5 Optionl: Delete the rchive files, if desired. rm -i./instll-*.run Creting the ppliction dt thin pool Use this procedure to crete thin pool for ppliction dt storge. The following Control Center configurtion vriles re used to specify ppliction dt thin pools: SERVICED_FS_TYPE Defult: devicempper The driver to mnge ppliction dt storge on the serviced mster host. Only devicempper is supported in production deployments. The only supported storge lyout for the devicempper driver is n LVM thin pool. To crete thin pool, use the serviced-storge utility. To specify the nme of the thin pool device, use the SERVICED_DM_THINPOOLDEV vrile. SERVICED_DM_THINPOOLDEV Defult: (none) The nme of the thin pool device to use with the devicempper storge driver. 19

20 Control Center Instlltion Guide Note This procedure does not include specific vlue for the size of the thin pool. For more informtion out sizing this resource, refer to the documenttion for your ppliction. Or, use the suggested minimum vlue, 200GB. You cn dd storge to n LVM thin pool t ny time. Perform these steps: 1 Log in to the mster host s root, or s user with superuser privileges. 2 Crete n LVM thin pool for ppliction dt. For more informtion out the serviced-storge commnd, see serviced-storge on pge 68. Replce Vlue with the fixed or reltive mount of spce to use for the thin pool; nd replce Storge with the pths of one or more lock devices or prtitions, or the nme of n LVM volume group: serviced-storge crete-thin-pool --size=vlue serviced Storge On success, the result is the device mpper nme of the thin pool, which lwys strts with /dev/mpper. Record the nme for use in the next step. 3 Edit storge vriles in the Control Center configurtion file. c d e Open /etc/defult/serviced in text editor. Locte the line for the SERVICED_FS_TYPE vrile, nd then mke copy of the line, immeditely elow the originl. Remove the numer sign chrcter (#) from the eginning of the line. Add SERVICED_DM_THINPOOLDEV immeditely fter SERVICED_FS_TYPE, nd set its vlue. Replce Thin-Pool-Nme with the device mpper nme of the thin pool for ppliction dt: SERVICED_DM_THINPOOLDEV=Thin-Pool-Nme Sve the file, nd then close the editor. Proceed to the next chpter nd configure the host. 20

21 Configuring nd strting the mster host Configuring nd strting the mster host 2 This chpter includes the procedures for configuring Control Center on the mster host, descries the configurtion options tht pply to the mster host, nd includes steps for strting the mster host for the first time. Perform the procedures in this chpter efore instlling Control Center on delegte hosts. Mny configurtion choices depend on ppliction requirements. Plese review your ppliction documenttion efore configuring Control Center on the mster host. This chpter includes synopses of the configurtion vriles tht ffect the mster host. For more informtion out vrile, see Control Center configurtion vriles on pge 72. Control Center mintennce scripts on the mster host The scripts in the following list re instlled when Control Center is instlled, nd re strted either dily or weekly y ncron. /etc/cron.dily/serviced This script invokes logrotte dily, to mnge the /vr/log/serviced.ccess.log file. This script is required on the mster host nd on ll delegte hosts. /etc/cron.weekly/serviced-fstrim This script invokes fstrim weekly, to reclim unused locks in the ppliction dt thin pool. The life spn of solid-stte drive (SSD) degrdes when fstrim is run too frequently. If the lock storge of the ppliction dt thin pool is n SSD, you cn reduce the frequency t which this script is invoked, s long s the thin pool never runs out of free spce. An identicl copy of this script is locted in /opt/ serviced/in. This script is required on the mster host only. /etc/cron.weekly/serviced-zenossdpck This script invokes serviced commnd weekly, which in turn invokes the dtse mintennce script for Zenoss ppliction. If the ppliction is not instlled or is offline, the commnd fils. This script is required on the mster host only. User ccess control Control Center provides rowser interfce nd commnd-line interfce. 21

22 Control Center Instlltion Guide To gin ccess to the Control Center rowser interfce, users must hve login ccounts on the Control Center mster host. In ddition, users must e memers of the Control Center dministrtive group, which y defult is the system group, wheel. To enhnce security, you my chnge the dministrtive group from wheel to ny non-system group. To use the Control Center commnd-line interfce (CLI) on Control Center cluster host, user must hve login ccount on the host, nd the ccount must e memer of the docker group. Memers of the wheel group, including root, re memers of the docker group. Pluggle Authentiction Modules (PAM) is supported nd recommended for enling ccess to oth the rowser interfce nd the commnd-line interfce. Adding users to the defult dministrtive group Use this procedure to dd users to the defult dministrtive group of Control Center, wheel. Performing this procedure enles users to gin ccess to the Control Center rowser interfce. Note Perform this procedure or the next procedure, ut not oth. 1 Log in to the host s root, or s user with superuser privileges. 2 Add user to the system group. Replce User with the nme of login ccount on the mster host. usermod -G wheel User Repet the preceding commnd for ech user to dd. Configuring regulr group s the Control Center dministrtive group Use this procedure to chnge the defult dministrtive group of Control Center from wheel to non-system group. The following Control Center vriles re used in this procedure: SERVICED_ADMIN_GROUP Defult: wheel The nme of the Linux group on the serviced mster host whose memers re uthorized to use the serviced rowser interfce. You my replce the defult group with group tht does not hve superuser privileges. SERVICED_ALLOW_ROOT_LOGIN Defult: 1 (true) Determines whether the root user ccount on the serviced mster host my e used to gin ccess to the serviced rowser interfce. Note Perform this procedure or the previous procedure, ut not oth. 1 Log in to the host s root, or s user with superuser privileges. 2 Crete vrile for the group to designte s the dministrtive group. In this exmple, the nme of group to crete is serviced. You my choose ny nme or use n existing group. GROUP=serviced 22

23 Configuring nd strting the mster host 3 Crete new group, if necessry. groupdd $GROUP 4 Add one or more existing users to the new dministrtive group. Replce User with the nme of login ccount on the host: usermod -G $GROUP User Repet the preceding commnd for ech user to dd. 5 Specify the new dministrtive group in the serviced configurtion file. Open /etc/defult/serviced in text editor. Locte the line for the SERVICED_ADMIN_GROUP vrile, nd then mke copy of the line, immeditely elow the originl. c Remove the numer sign chrcter (#) from the eginning of the line. d Chnge the vlue from wheel to the nme of the group you chose erlier. e Sve the file, nd then close the editor. 6 Optionl: Prevent the root user from gining ccess to the Control Center rowser interfce, if desired. Open /etc/defult/serviced in text editor. Locte the line for the SERVICED_ALLOW_ROOT_LOGIN vrile, nd then mke copy of the line, immeditely elow the originl. c Remove the numer sign chrcter (#) from the eginning of the line. d Chnge the vlue from 1 to 0. e Sve the file, nd then close the editor. Enling use of the commnd-line interfce Use this procedure to enle user to perform dministrtive tsks with the Control Center commnd-line interfce. 1 Log in to the host s root, or s user with superuser privileges. 2 Add user to the Docker group. Replce User with the nme of login ccount on the host. usermod -G docker User Repet the preceding commnd for ech user to dd. Configuring the se size device for tennt dt storge Use this procedure to configure the se size of virtul storge devices for tennts in the ppliction dt thin pool. The se size is used ech time tennt device is creted. In prticulr, the first time serviced strts, it cretes the se size device nd then cretes tennt device from the se size device. The following Control Center configurtion vrile is used to specify the se size device: SERVICED_DM_BASESIZE Defult: 100G The se size of virtul storge devices for tennts in the ppliction dt thin pool, in gigytes. The units symol (G) is required. This vrile is used when serviced strts for the first time, to set the initil size of tennt devices, nd when ckup is restored, to set the size of the restored tennt device. 23

24 Control Center Instlltion Guide The se size device is sprse device tht occupies t most 1MB of spce in the ppliction dt thin pool; its size hs no immedite prcticl impct. However, the ppliction dt thin pool should hve enough spce for twice the size of ech tennt device it supports, to store oth the dt itself nd snpshots of the dt. Since the ppliction dt thin pool is n LVM logicl volume, its size cn e incresed t ny time. Likewise, the size of tennt device cn e incresed, s long s the ville spce in the thin pool cn support the lrger tennt device plus snpshots. Perform these steps: 1 Log in to the mster host s root, or s user with superuser privileges. 2 Identify the size of the thin pool for ppliction dt. The size is required to set n ccurte vlue for the SERVICED_DM_BASESIZE vrile. lvs --options=lv_nme,lv_size grep serviced-pool 3 Edit storge vriles in the Control Center configurtion file. Open /etc/defult/serviced in text editor. Add SERVICED_DM_BASESIZE immeditely fter SERVICED_DM_THINPOOLDEV, nd set its vlue. Replce Fifty-Percent with the vlue tht is less thn or equl to 50% of the size of the thin pool for ppliction dt. Include the symol for gigytes, G: SERVICED_DM_BASESIZE=Fifty-PercentG c Sve the file, nd then close the editor. 4 Verify the settings. grep -E '^\*SERVICED' /etc/defult/serviced Setting the host role to mster Use this procedure to configure host s the mster host. The following configurtion vrile ssigns the host role: SERVICED_MASTER Defult: 1 (true) Assigns the role of serviced instnce, either mster or delegte. The mster runs the ppliction services scheduler nd other internl services. Delegtes run the ppliction services ssigned to the resource pool to which they elong. Only one serviced instnce cn e the mster; ll other instnces must e delegtes. The defult vlue ssigns the mster role. To ssign the delegte role, set the vlue to 0 (flse). This vrile must e explicitly set on ll Control Center cluster hosts. Perform these steps: 1 Log in to the host s root, or s user with superuser privileges. 2 Edit the Control Center configurtion file. c d Open /etc/defult/serviced in text editor. Locte the line for the SERVICED_MASTER vrile, nd then mke copy of the line, immeditely elow the originl. Remove the numer sign chrcter (#) from the eginning of the line. Sve the file, nd then close the editor. 24

25 Configuring nd strting the mster host 3 Verify the settings. grep -E '^\*SERVICED' /etc/defult/serviced Optionl: Configuring offline use Zenoss customers: If you re creting deployment without internet ccess, perform this procedure. Use this procedure to configure host to operte without internet ccess. The following configurtion vrile informs serviced tht internet ccess is not ville: SERVICED_OUTBOUND_IP Defult: (none) The defult strtup routines of serviced include ttempting to ping google.com. When vlue is set for this vrile, serviced does not ttempt the ping nd ssumes it does not hve internet ccess. Use this vrile to specify the IP ddress of network interfce other thn the defult, or to prevent serviced from ssuming it hs internet ccess. Note Setting the Docker HTTP_PROXY or HTTPS_PROXY environment vriles prevents ccess to the IP ddress defined with this vrile. To enle ccess, unset the Docker vriles, nd then reoot the host. Perform these steps: 1 Log in to the host s root, or s user with superuser privileges. 2 Identify the IPv4 ddress of the host. hostnme -i 3 Edit the Control Center configurtion file. Open /etc/defult/serviced in text editor. Locte the line for the SERVICED_OUTBOUND_IP vrile, nd then mke copy of the line, immeditely elow the originl. c Remove the numer sign chrcter (#) from the eginning of the line. d Chnge the vlue to the IPv4 ddress identified in the previous step. e Sve the file, nd then close the editor. 4 Verify the settings. grep -E '^\*SERVICED' /etc/defult/serviced Mster host configurtion vriles The tles in this section provide n overview of the serviced configurtion vriles tht pply to the Control Center mster host. Set these vriles s required for your environment or pplictions. Best prctices for configurtion files The Control Center configurtion file, /etc/defult/serviced, contins Bsh environment vriles tht re red y the serviced demon strtup script. The following list descries recommended est prctices for its use nd mintennce: 25

26 Control Center Instlltion Guide 1 When in dout, mke ckup. Mking ckup copy of the configurtion file efore editing it is lwys the sfest choice. 2 Copy vrile, then edit the copy. If you need to revert vrile to its defult vlue, you don't hve to leve the file to look it up. 3 Copy nd edit vrile only if the defult vlue needs to e chnged. It's esier to trouleshoot prolems when only non-defult vriles re copied nd edited. 4 Put the first chrcter of the vrile declrtion in the first column of its line. It's esier to grep for settings when ech one strts line. 5 Add customiztions to the top of the file. Customiztions t the end of the file or scttered throughout the file my e overlooked. Storge vriles The vriles in the following tle re set only on the mster host. Use one of the first two groups of vriles ut not oth. The defults of the third group my need to e chnged efore strting the mster host for the first time. Typiclly, the defults of the lst two groups of vriles re not chnged until Control Center hs mnged n ppliction for while nd need rises. The SERVICED_STORAGE_STATS_UPDATE_INTERVAL vrile sets the intervl for collecting kernel sttistics out the ppliction dt thin pool. Its defult vlue is unlikely to require chnge until need rises. Vrile Description Purpose SERVICED_FS_TYPE SERVICED_DM_ARGS SERVICED_DM_BASESIZE SERVICED_DM_THINPOOLDEV SERVICED_FS_TYPE SERVICED_DM_LOOPDATASIZE SERVICED_DM_LOOPMETADATASIZE SERVICED_ALLOW_LOOP_BACK The specifictions of devicempper-sed ppliction dt storge resource for production use. The specifictions of devicempper-sed ppliction dt storge resource for development use. Provide sic informtion out the dt storge resource. Provide sic informtion out the dt storge resource. SERVICED_ISVCS_PATH SERVICED_VOLUMES_PATH SERVICED_BACKUPS_PATH SERVICED_SNAPSHOT_TTL SERVICED_SNAPSHOT_USE_PERCENT SERVICED_MAX_DFS_TIMEOUT SERVICED_LOGSTASH_MAX_DAYS SERVICED_LOGSTASH_MAX_SIZE SERVICED_LOGSTASH_CYCLE_TIME The dt storge pths of seprte functionl components of Control Center internl services. The snpshot retention intervl, the percentge of the dt storge thin pool tht is unused, nd the snpshot ttempt timeout intervl. The vriles tht mnge the mount of spce used y the ppliction log storge service. Enle seprte storge res for one or more components. The defult instlltion process puts ll three components on the sme device. Prevent the cretion of snpshots tht re too lrge to fit the thin pool. Prevent ppliction logs from filling the storge device tht logstsh uses. 26

27 Configuring nd strting the mster host Internl services endpoint vriles The vriles in the following tle must e set identiclly on ll Control Center delegte hosts. On the mster host, the defult vlues of these vriles do not need to e chnged, unless the Docker registry is hosted on system other thn the mster. In tht cse, SERVICED_DOCKER_REGISTRY must e set explicitly on the mster host, too. The SERVICED_AUTH_TOKEN_EXPIRATION vrile ffects RPC, mux, nd internl services endpoint trffic. Vrile Endpoint Description SERVICED_DOCKER_REGISTRY SERVICED_ENDPOINT SERVICED_LOG_ADDRESS SERVICED_LOGSTASH_ES SERVICED_STATS_PORT Mster-Host:5000 The locl Docker registry for Control Center internl services imges nd ppliction imges. Mster-Host:4979 The serviced RPC server. The endpoint port numer must mtch the vlue of SERVICED_RPC_PORT. Mster-Host:5042 The logstsh service. Mster-Host:9100 The Elsticserch service for logstsh. Mster-Host:8443 The serviced metrics consumer service. SERVICED_AUTH_TOKEN_EXPIRATION (none) The length of time delegte uthentiction token is vlid. RPC service vriles The vriles in the following tle must e set identiclly on ll Control Center cluster hosts, except: SERVICED_RPC_PORT, set only on the mster SERVICED_MAX_RPC_CLIENTS, set only on delegtes By defult, serviced uses TLS to encrypt ll RPC trffic. The SERVICED_KEY_FILE nd SERVICED_CERT_FILE vriles identify the digitl certificte used for RPC, mux, nd HTTP trffic. The SERVICED_AUTH_TOKEN_EXPIRATION vrile ffects RPC, mux, nd internl services endpoint trffic. Vrile SERVICED_ENDPOINT SERVICED_MAX_RPC_CLIENTS SERVICED_RPC_PORT SERVICED_RPC_CERT_VERIFY SERVICED_RPC_DISABLE_TLS SERVICED_RPC_TLS_MIN_VERSION SERVICED_RPC_TLS_CIPHERS SERVICED_KEY_FILE Where to set Mster, delegtes Delegtes Mster Mster, delegtes Mster, delegtes Mster, delegtes Mster, delegtes Mster 27

28 Control Center Instlltion Guide Vrile SERVICED_CERT_FILE SERVICED_RPC_DIAL_TIMEOUT SERVICED_AUTH_TOKEN_EXPIRATION Where to set Mster Mster, delegtes Mster Multiplexer vriles The vriles in the following tle must e set identiclly on ll Control Center cluster hosts. By defult, serviced uses TLS to encrypt ll mux trffic. The SERVICED_KEY_FILE nd SERVICED_CERT_FILE vriles identify the digitl certificte used for RPC, mux, nd HTTP trffic. The SERVICED_AUTH_TOKEN_EXPIRATION vrile ffects RPC, mux, nd internl services endpoint trffic. Vrile SERVICED_MUX_PORT SERVICED_MUX_DISABLE_TLS SERVICED_MUX_TLS_MIN_VERSION SERVICED_MUX_TLS_CIPHERS SERVICED_KEY_FILE SERVICED_CERT_FILE SERVICED_AUTH_TOKEN_EXPIRATION Where to set Mster, delegtes Mster, delegtes Mster, delegtes Mster, delegtes Mster Mster Mster HTTP server vriles The vriles in the following tle re set only on the mster host, except the SERVICED_UI_PORT vrile, which must e set identiclly on ll cluster hosts. By defult, serviced uses TLS to encrypt ll HTTP trffic. The SERVICED_KEY_FILE nd SERVICED_CERT_FILE vriles identify the digitl certificte used for RPC, mux, nd HTTP trffic. Vrile SERVICED_UI_PORT SERVICED_TLS_MIN_VERSION SERVICED_TLS_CIPHERS SERVICED_KEY_FILE SERVICED_CERT_FILE Description The port on which the HTTP server listens for requests. The minimum version of TLS tht serviced ccepts for HTTP trffic. The list TLS ciphers tht serviced ccepts for HTTP trffic. The pth of digitl certificte key file. The pth of digitl certificte file. 28

29 Configuring nd strting the mster host Browser interfce vriles (mster host only) The vriles in the following tle re set only on the mster host. Vrile SERVICED_UI_POLL_FREQUENCY SERVICED_SVCSTATS_CACHE_TIMEOUT SERVICED_ADMIN_GROUP SERVICED_ALLOW_ROOT_LOGIN Description The numer of seconds etween polls from rowser interfce clients. The numer of seconds to cche sttistics out services. The group on the mster host whose memers cn use the rowser interfce. Determines whether root on the mster host cn use the rowser interfce. Tuning vriles (mster host only) Vrile SERVICED_ES_STARTUP_TIMEOUT SERVICED_MASTER_POOLID Description The numer of seconds to wit for the Elsticserch service to strt. The nme of the defult resource pool. This vrile is only used the first time serviced is strted. Universl configurtion vriles The tles in this section provide n overview of the serviced configurtion vriles tht pply to ll hosts in Control Center cluster. Set these vriles s required for your environment or pplictions. Role vrile Vrile SERVICED_MASTER Description Assigns the role of serviced instnce, either mster or delegte. The mster runs the ppliction services scheduler nd other internl services. Delegtes run the ppliction services ssigned to the resource pool to which they elong. Browser interfce vrile (ll hosts) Vrile SERVICED_UI_PORT Description The port on which the HTTP server listens for requests. Networking vriles Vrile SERVICED_STATIC_IPS Description A list of one or more sttic IP ddresses for IP ssignment. 29

30 Control Center Instlltion Guide Vrile SERVICED_OUTBOUND_IP SERVICED_VIRTUAL_ADDRESS_SUBNET SERVICED_DOCKER_DNS Description The IP ddress of the network interfce for serviced to use. When this vrile is not set, serviced uses the IP ddress of the defult network interfce nd ssumes it hs internet ccess. To prevent serviced from ssuming it hs internet ccess, set this vrile. The privte network for continers tht use virtul IP ddresses. The defult is /16, nd the network cn e unique on ech host. A /29 network is sufficient. A list of one or more DNS servers. The list is injected into ll Docker Engine continers. Deugging vriles Vrile SERVICED_LOG_LEVEL SERVICED_DEBUG_PORT SERVICED_DOCKER_LOG_DRIVER SERVICED_DOCKER_LOG_CONFIG Description The log level serviced uses when writing to the system log. The port on which serviced listens for HTTP requests for the Go profiler. The log driver for ll Docker continer logs. Docker --log-opt options. Tuning vriles (ll cluster hosts) Vrile GOMAXPROCS SERVICED_MAX_CONTAINER_AGE SERVICED_ISVCS_ENV_[0-9]+ SERVICED_SERVICE_MIGRATION_TAG SERVICED_OPTS SERVICED_CONTROLLER_BINARY SERVICED_HOME SERVICED_ETC_PATH SERVICED_VHOST_ALIASES Description The mximum numer of CPU cores tht serviced uses. The numer of seconds serviced wits efore removing stopped continer. Strtup rguments to pss to specific internl services. Overrides the defult vlue for the service migrtion imge. Strtup rguments for serviced. The pth of the serviced-controller inry. The pth of the home directory for serviced. The pth of the directory for serviced configurtion files. A list of hostnme lises for host; for exmple, loclhost. 30

31 Configuring nd strting the mster host Strting Control Center for the first time Use this procedure to strt the Control Center service (serviced) on mster host fter instlling nd configuring it. This procedure is vlid for single-host nd multi-host deployments, whether the deployment hs internet ccess or not, nd is only performed once. 1 Log in to the mster host s root, or s user with superuser privileges. 2 Verify the configurtion settings. grep -E '^\*SERVICED' /etc/defult/serviced 3 Strt the Control Center service. systemctl strt serviced To monitor progress, enter the following commnd: journlctl -flu serviced -o ct On hosts with internet ccess, the serviced demon invokes docker to pull its internl services imges from Docker Hu. The Control Center rowser nd commnd-line interfces re unville until the imges re instlled nd tgged, nd the services re strted. The process tkes out 5 minutes. On hosts without internet ccess, the serviced demon tgs imges in the locl registry nd strts its internl services. The Control Center rowser nd commnd-line interfces re unville for out 3 minutes. When the messge Host Mster successfully strted is displyed, Control Center is redy for the next procedure. Adding the mster host to resource pool Perform one of the procedures in this section. Adding the mster host to resource pool (single-host deployments) Use this procedure to dd the mster host to resource pool. For single-host deployments, the mster host is dded to the defult pool. 1 Log in to the mster host s root, or s user with superuser privileges. 2 Add the mster host to the defult resource pool. Replce Hostnme-Or-IP with the hostnme or IP ddress of the Control Center mster host: serviced host dd --register Hostnme-Or-IP:4979 defult If you use hostnme, ll of the hosts in your Control Center cluster must e le to resolve the nme, either through n entry in /etc/hosts or through nmeserver on the network. Adding the mster host to resource pool (multi-host deployments) Use this procedure to dd the mster host to resource pool. For multi-host deployments, the mster host is dded to its own, seprte pool. 1 Log in to the mster host s root, or s user with superuser privileges. 31