JSA KVM SUPPORT. Theodore Jencks, CSE Juniper Networks

Transcription

1 JSA KVM SUPPORT Theodore Jencks, CSE

2 KVM IMAGE SPECIFICATIONS Image is a compressed qcow2 image 16.5G in size when compressed Md5sum: Uncompressed the image is a 512G QCOW2 Ubuntu LTS w/ KVM and OVS was used to validate the image for KVM Contrail was validate to run with the image / Kilo / Ocata Openstack Versions For best IO performance image metadata pre allocation is recommended as a minimum. Full allocation of the disk is required for max performance and recommend for all installs on KVM. Image role and resource requirements can be found here: vrt_ap_reqs.html 2

3 INSTALL INSTRUCTIONS ON KVM Considerations for NUMA. It is recommended that JSA VM s be instantiated on the same NUMA as the disk controller / RAID Controller on the host system. This is to optimize Disk IO operations and avoid crossing QPI. It is also recommend that NUMA policy be set as strict for KVM so that memory and CPU resources are all allocated from the same NUMA. Copy the jsa7.3.1p6.qcow2 image to a safe location on the virtualization host. Clone the image: qemu-img convert -p -f qcow2 -O qcow2 /iso/jsa7.3.1p6-final.qcow2 /vm/vjsa-tacon.qcow2 -o preallocation=metadata Full allocation recommened: qemu-img convert -p -f qcow2 -O qcow2 /iso/jsa7.3.1p6.qcow2 /vm/vjsa-tacon.qcow2 -o preallocation=falloc Install with virt-install: virt-install --hvm --name jsa7.3.1-tacon --cpu host --cpuset 2-10, vcpus sockets=18,cores=1,threads=1 --memory memorybacking hugepages=yes --disk path=/vm/vjsatacon.qcow2,format=qcow2 --os-type linux --os-variant rhel7.4 --memballoon virtio --network network:default,model=virtio --graphics vnc --console pty,target_type=virtio -import -print-xml > vjsa-tacon.xml The above command will print the XML necessary to define the VM in KVM. In this case we are going to print the output to a file then edit the file to complete our static CPU pinning. 3

4 INSTALL INSTRUCTIONS ON KVM Next Edit the file and add the following to the XML: <cputune> <vcpupin vcpu='0' cpuset='11'/> <vcpupin vcpu='1' cpuset='51'/> <vcpupin vcpu='2' cpuset='12'/> <vcpupin vcpu='3' cpuset='52'/> <vcpupin vcpu='4' cpuset='13'/> <vcpupin vcpu='5' cpuset='53'/> <vcpupin vcpu='6' cpuset='14'/> <vcpupin vcpu='7' cpuset='54'/> <vcpupin vcpu='8' cpuset='15'/> <vcpupin vcpu='9' cpuset='55'/> <vcpupin vcpu='10' cpuset='16'/> <vcpupin vcpu='11' cpuset='56'/> <vcpupin vcpu='12' cpuset='17'/> <vcpupin vcpu='13' cpuset='57'/> <vcpupin vcpu='14' cpuset='18'/> <vcpupin vcpu='15' cpuset='58'/> <vcpupin vcpu='16' cpuset='19'/> <vcpupin vcpu='17' cpuset='59'/> </cputune> Now define the VM: virsh define vjsa-tacon.xml 4

5 INSTALL INSTRUCTIONS ON KVM Here is the example output that you should see: virsh dumpxml jsa7.3.1-tacon <domain type='kvm'> <name>vjsa-tacon</name> <uuid>957426e8-e ac8f-6a2b7a3236a1</uuid> <memory unit='kib'> </memory> <currentmemory unit='kib'> </currentmemory> <memorybacking> <hugepages/> </memorybacking> <vcpu placement='static' cpuset='11-19,51-59'>18</vcpu> <cputune> <vcpupin vcpu='0' cpuset='11'/> <vcpupin vcpu='1' cpuset='51'/> <vcpupin vcpu='2' cpuset='12'/> <vcpupin vcpu='3' cpuset='52'/> <vcpupin vcpu='4' cpuset='13'/> <vcpupin vcpu='5' cpuset='53'/> <vcpupin vcpu='6' cpuset='14'/> <vcpupin vcpu='7' cpuset='54'/> <vcpupin vcpu='8' cpuset='15'/> <vcpupin vcpu='9' cpuset='55'/> <vcpupin vcpu='10' cpuset='16'/> <vcpupin vcpu='11' cpuset='56'/> <vcpupin vcpu='12' cpuset='17'/> <vcpupin vcpu='13' cpuset='57'/> <vcpupin vcpu='14' cpuset='18'/> <vcpupin vcpu='15' cpuset='58'/> <vcpupin vcpu='16' cpuset='19'/> <vcpupin vcpu='17' cpuset='59'/> </cputune> 5

6 INSTALL INSTRUCTIONS ON KVM Now start the VM and console into it with the following commands: virsh start vjsa-tacon virsh console vjsa-tacon Login as root / no password should be required. Step through the install wizard for the virtual appliance type you are creating in this case a Threat Analytics Console or (all in one) 6

7 INSTALL INSTRUCTIONS ON KVM CONTINUED Login as root at the prompt no password should be required. If you are prompted for a password something has gone wrong with the install please contact Juniper Support. Accept the EULA license and proceed 7

8 INSTALL INSTRUCTIONS ON KVM CONTINUED 8

9 INSTALL INSTRUCTIONS ON KVM CONTINUED 9

10 INSTALL INSTRUCTIONS ON KVM CONTINUED 10

11 INSTALL INSTRUCTIONS ON KVM CONTINUED 11

12 INSTALL INSTRUCTIONS ON KVM CONTINUED 12

13 INSTALL INSTRUCTIONS ON KVM CONTINUED 13

14 INSTALL INSTRUCTIONS ON KVM CONTINUED 14

15 INSTALL INSTRUCTIONS ON KVM CONTINUED 15

16 INSTALL INSTRUCTIONS ON KVM CONTINUED 16

17 INSTALL INSTRUCTIONS ON KVM CONTINUED The below indicates that the installation has started. This process can take up to 6 hours depending on the speed of your system. Be patient and give the system an opportunity to complete, the system can seem hung or frozen at times when really the install just runs for a very long time. 17

18 INSTALL INSTRUCTIONS ON KVM CONTINUED The following output indicates a successful install of a Console system: Installing Qradar changes... psql: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.pgsql.5432"? Activating system with key 003V41-5T7A3E-077N7N-54512G. Appliance ID is Installing 'TA Threat Analytics "All-In-One" or Console' with id Configuring network... Setting time server to ntp.juniper.net. Synching time with server 'ntp.juniper.net' 8 Oct 16:42:41 ntpdate[10395]: adjust time server offset sec Restarting postgresql-qrd Running changeqradarpassword Stopping hostcontext Stopping httpd Stopping tomcat 1: waiting for port 7676 to start 2: waiting for port 7676 to start 3: waiting for port 7676 to start Mon Oct 8 16:43:39 PDT 2018 [setup-imq.sh] OK: IMQ Setup Completed Stopping httpd Stopping tomcat Updating db user password OK: Post Import Actions For Vulnerability Tables Are Successfully Completed.OK: Reseting Of Sequences Of Asset Related q_catalog Tables Is Successfully CompleteInstalling DSM rpms: done. Decompressing QidMap file /opt/qradar/conf/templates/ qidmap-import.xml.xz... Importing /opt/qradar/conf/templates/ qidmap-import.xml Finished updating QIDMap, took 1073 seconds to complete. 18

19 INSTALL INSTRUCTIONS ON KVM CONTINUED Known Issue: If during the Checking that tomcat is running and ready (attempt 0/30) phase goes past (attempt 10/30) it is recommended to use another SSH session to login to the systems IP during install and remove the imqbroker lock file as well as restart the imqbroker service as follows: If the install times out it is recommend to reboot the system and run through the setup for a second time. The second time through it should complete without issue. 19

20 KNOWN INSTALL ISSUES AND WORKAROUND Known Issue: If during the Checking that tomcat is running and ready (attempt 0/30) phase goes past (attempt 10/30) it is recommended to use another SSH session to login to the systems IP during install and remove the imqbroker lock file as well as restart the imqbroker service as follows: If the install times out it is recommend to reboot the system and run through the setup for a second time. The second time through it should complete without issue. 20

21 DISK IMAGE EXPANSION

22 DISK EXPANSION Currently the QCOW2 image is based on an image that is 512G in size. It may be desirable to increase the amount of storage allocated to a particular partition on the disk image. The following instructions cover the steps in order to expand the /store partition. First expand the qcow2 disk image. Please note the VM must be shutdown in order to perform this operation. Otherwise disk corruption of the qcow2 image file will occur. sudo qemu-img resize --preallocation=falloc vjsa-tacon.qcow2 +512G Image resized. Ensure the disk file is in good shape after the expansion. sudo qemu-img check -r all /vm/vjsa-tacon.qcow2 No errors were found on the image / = % allocated, 0.00% fragmented, 0.00% compressed clusters Image end offset: Now boot the disk image virsh start vjsa-tacon 22

23 DISK EXPANSION Now login via console or SSH virsh console vjsa-tacon Now run parted on the disk /dev/vda. Make sure to type fix when prompted parted /dev/vda GNU Parted 3.1 Using /dev/vda Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) print Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)? Fix/Ignore/Cancel? Fix Warning: Not all of the space available to /dev/vda appears to be used, you can fix the GPT to use all of the space (an extra blocks) or continue with the current setting? 23

24 DISK EXPANSION Now reboot so the new partition table is reflected reboot Now run parted to display free space and note beginning and ending blocks (parted) unit s print free Model: Virtio Block Device (virtblk) Disk /dev/vda: s Sector size (logical/physical): 512B/512B Partition Table: gpt Disk Flags: pmbr_boot Number Start End Size File system Name Flags 34s 2047s 2014s Free Space s 4095s 2048s bios_grub s s s xfs s s s xfs 24

25 DISK EXPANSION Now reboot to make sure the new partition fixes are reflected in the partition table create a new partition at the end of the existing partition (parted) mkpart Partition name? []? File system type? [ext2]? Start? End? (parted) unit s print free Model: Virtio Block Device (virtblk) Disk /dev/vda: s Sector size (logical/physical): 512B/512B Partition Table: gpt Disk Flags: pmbr_boot Number Start End Size File system Name Flags 25

26 DISK EXPANSION Now set the partition as an lvm partition (parted) set 7 lvm on (parted) unit s print free Model: Virtio Block Device (virtblk) Disk /dev/vda: s Sector size (logical/physical): 512B/512B Partition Table: gpt Disk Flags: pmbr_boot Number Start End Size File system Name Flags 34s 2047s 2014s Free Space s 4095s 2048s bios_grub s s s xfs s s s xfs s s s lvm 26

27 DISK EXPANSION Quit parted and add the new lvm partition as a physical volume to lvm sudo pvs PV VG Fmt Attr PSize PFree /dev/vda4 storerhel lvm2 a-- <387.99g 0 /dev/vda5 rootrhel lvm2 a g 0 pvcreate /dev/vda7 File descriptor 63 (pipe:[ ]) leaked on pvcreate invocation. Parent PID 8346: -bash Physical volume "/dev/vda7" successfully created. Now extend the volume group [root@vjsa-tacon ~]# vgextend storerhel /dev/vda7 File descriptor 63 (pipe:[ ]) leaked on vgextend invocation. Parent PID 8346: -bash Volume group "storerhel" successfully extended [root@vjsa-tacon ~]# vgs File descriptor 63 (pipe:[ ]) leaked on vgs invocation. Parent PID 8346: -bash 27

28 DISK EXPANSION Now that the physical volume group is extended, extend the logical volume df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/rootrhel-root 13G 3.0G 9.5G 24% / devtmpfs 32G 0 32G 0% /dev tmpfs 32G 20K 32G 1% /dev/shm tmpfs 32G 34M 32G 1% /run tmpfs 32G 0 32G 0% /sys/fs/cgroup /dev/mapper/rootrhel-tmp 3.0G 135M 2.9G 5% /tmp /dev/vda3 32G 4.8G 28G 15% /recovery /dev/mapper/storerhel-store 311G 21G 290G 7% /store /dev/mapper/rootrhel-opt 13G 4.3G 8.3G 34% /opt /dev/mapper/rootrhel-home 1014M 33M 982M 4% /home /dev/mapper/rootrhel-storetmp 15G 44M 15G 1% /storetmp /dev/mapper/storerhel-transient 78G 34M 78G 1% /transient 28

29 DISK EXPANSION Now that the logical volume is extended extend the file system: xfs_growfs /store meta-data=/dev/mapper/storerhel-store isize=512 agcount=4, agsize= blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=0 spinodes=0 data = bsize=4096 blocks= , imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=1 log =internal bsize=4096 blocks=39729, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 data blocks changed from to