OLT Us e r Guide for Or acle VM

Transcription

1 OLT Us e r Guide for Or acle VM Table of Contents 1.Introduction Hardware Requirements Planning the number of Vms Pre-Installation for OLT Configuration of Test Servers Configuration of Dom0 for OLT...4 OS Packages...4 System Configuration...4 Storage Configuration for Vms Template Setup for OLT Setup of VM templates Setup of Oracle templates OLT Setup and Execution OLT Configuration Execution of OLT Tests Scheduling tests Monitoring Tests Logs and Test Results Terminating tests Cleaning up all VMs Debugging OLT issues Migration on Oracle VM with OLT Migration Tests Troubleshooting OLT tests How to rerun a failed test Manual Configuration of OLT for OVM Appendix A OLT Utilities Appendix B- How-to edit olt-setup.env file for your storage configuration

2 1. Introduction This document describes the execution of the OLT tests with Oracle VM as a virtualization solution and the virtual machines that host the database instances. One of the important features of Oracle VM, is the support of the virtual machine live migration from server to server in case of the server crash. Live migration requires the virtual machine images to be located on shared storage which is accessible from all of the servers. We strongly recommend to test the virtual machine migration, and allocate for your tests at least two servers (see the hardware requirement section below). The steps to setup the kit and execute the tests on your system include Verification of Hardware Requirements Pre-Installation OS Configuration Storage Configuration OLT Installation and Configuration Execution of OLT tests- OVM tests Results Collation for Auditing 2. Hardware Requirements Listed below are the hardware requirements for executing OLT on Oracle VM Depending on your test requirement ( with or without migration ), you would require either Single VT Enabled Server with internal/local storage for tests without live migration; or single VT enabled server with minimum 2cpu, 4GB RAM. Internal storage or external storage such as SAN, iscsi or NAS [ refer to Storage requirements ] 2 VT Enabled Server with shared storage if you are testing/validating the configuration with live migration 2 VT enabled server with minimum 2cpu, 4GB RAM Shared storage such as SAN, iscsi or NAS [ refer to Storage requirements ] DHCP Server Setup on the test network Storage Requirements The following are the storage requirements for the OLT kit on Dom0 Location Space Purpose /opt/oracle/oltest 30GB Test Kit software /home/oracle 15GB Runtime work area /tmp 1GB Temporary files The following are the storage requirements for running VMs on Dom0 2

3 Location Space Purpose /OVS 70GB per VM To store VM image files In addition to the above requirements, each VM requires storage for the test database. The following are the requirements for the Test Data on a VM. Size 100GB per VM Purpose Test Data for VM If you are using a single server for testing (without migration ), you can use any of the following: ( NOTE: /OVS and the storage for the test database should be on the shared storage for validating with migration ) 3. Planning the number of VMs OLT executes tests on the virtualization server for the virtual machines to be tested. These tests include tests that verify a VMs basic functionality, oracle stress tests on the VMs and xm stress tests on the VMs. OLT has the capability of executing these tests on either a single or multiple VM depending on the resources of the Server (cpu and memory ) and storage. OLT decides on how many VMs to configure and schedule tests in parallel based on the following criteria: The number of VMs that needs to be tested based on your test requirements. The number of VMs that can be tested in parallel P - based on your hardware: CPU (minimum 2 cpus per VM ) Memory ( minimum 2GB per VM ) Storage ( as mentioned in the storage requirements section for a VM) i.e. The minimum of the above three parameters are selected. If the number of VMs determined in Step 1 is less than or equal to the number of VMs determined in Step 2 then the test run can be completed with a single configuration of OLT and execution of OLT ( OLT will configure and schedule tests across all VMs in parallel ) If the number of VMs determined in Step1 is greater than the number of VMs determined in 2, then OLT will configure and execute the tests in loops on each set of VMs that can be created in parallel, until all the required VMs are covered. Execute Migration tests as per Section- Migration on Oracle VM with OLT 3

4 4. Pre-Installation for OLT 4.1. Configuration of Test Servers Download Oracle VM( from zation/index.html ) and reinstall the Server with Oracle VM. Refer Storage Requirements for /OVS partition requirements and the directory/size requirements. Important: During installation, create a 50GB root partition (the default is 3GB). Thereafter, follow the steps below to setup the machine for OLT Configuration of Dom0 for OLT OS Packages The following rpms should be installed for OLT on the OVM Server Dom0. These packages are required only on Oracle VM Server 2.x. Install these additional packages on the OVM Server's Dom0 using up2date from the Oracle Linux 5 channel (ol5_u3_i386_base ) on ULN. Gcc expect To use up2date, Register to ULN. Subscribe to the required channels up2date -fi-channel ol5_u3_i386_base <package> to update your system with the required rpm. For Oracle VM Server 3.x, use channel ol6_u6_x86_64_base to configure the public-yum repository and install the following packages gcc expect syslinux System Configuration Network Edit /etc/hosts to include the fully qualified name first followed by the other names. Edit /etc/hosts to include localhost entry Setup your system such that hostname returns the fully qualified name( with domainname) Storage Configuration for VMs Configure and setup the storage for the test database to be run on each VM. Depending on the number of VMs that can be tested in parallel -P ( from Section 3 above ) follow the steps below depending on the type of storage. NAS: This can be configured for use in either of the following ways VM uses an nfs backed file image created on Dom0 ; or Repeat the step below for P Vms Create an image file each VMs using the command below 4

5 dd if=/dev/zero of=<mount point on Dom0 e.g:/ovm_test>/vm_testdata<n>.img oflag=direct bs="xxxx" count="yyy" where yyy=100gb/xxxx VM uses an nfs location directly which has the required space for each VM Create one exported file system on the nfs filer with size (P*100) for e.g: /ovm_test Create a directory for each VM on the nfs server for e.g: /ovm_test/vm1 to /ovm_test/vm<p> Shared storage iscsi/scsi/local disk Repeat the step below for P Vms Create a disk partition of size=100gb for each VM from Dom0 using fdisk Template Setup for OLT Download and setup the VM templates and Oracle templates for OLT as mentioned below Setup of VM templates Download the Oracle Linux PVM templates for the VMs to be tested from htt p://edelivery.oracle.com Use unzip to uncompress the file. Copy the extracted file templates to /OVS/seed_pool/ Setup of Oracle templates Download the Oracle database template ( the Oracle version to be tested ) from http: //edelivery.o ra cle.com Use unzip tool to uncompress the file. Copy the extracted file templates to /OVS/oracle_pool. 5. OLT Setup and Execution 5.1. OLT Configuration OLT is available in two formats: Single archived file ( for installation and execution ) Set of rpms ( hosted packages only ) Single archived file: Installation and Configuration Download the archive file ( OLT-<version>.tar.gz ) from oss.oracle.com/projects/olt to <Location>

6 cd <Location> 5 tar xzvf OLT-<version>.tar.gz Edit olt-setup.env ( mentioned below ) as per your requirements for VM related information (public network interface, template for each VM, oracle image for each VM, filesystem type for the test database on the VM, storage details for the test database on the VM[Refer Appendix B on how to setup olt-setup.env file for your storage configuration fs_type and fs<n> parameters ]) Note: OLT is setup in the context of "oracle" user. OLT supports only 1 public interface for VMs that is specified in olt-setup.env as public_if. <Example olt-setup.env file with entries for xen> export CONFIGURATION_MODE_SINGLE_INSTANCE=0 export CONFIGURATION_MODE_RAC=0 export CONFIGURATION_MODE_XEN=1 #OVM Specific #OVM Specific parameters # Set DOMAIN_SIZING=static/dynamic # DOMAIN_SIZING=static if we want to assign memory and cpu to each VM # DOMAIN_SIZING=dynamic if VM memory and cpu are calculated based on # system configuration. export DOMAIN_SIZING=static # Set DOMAIN_SETUP=auto/manual # DOMAIN_SETUP=auto if we want OLT to setup and create the VM's. # DOMAIN_SETUP=manual if VM's are already created and just want to run tests export DOMAIN_SETUP=auto # Set STORAGE_PER_DOM=<n> # space required on /OVS for a VM in GB. # minimum space required per VM is 70GB 6

7 export STORAGE_PER_DOM=70 # Set MEMORY_PER_DOM=<n>: Memory per VM in MB. # It is used in case of DOMAIN_SIZING=static. # else memory is be calculated by olt depending on the underlying hardware export MEMORY_PER_DOM=2048 # Set CPUS_PER_DOM=<n> : cpus per VM. # It is used in case of DOMAIN_SIZING=static. # else cpu will be calculated by olt depending on the underlying hardware. export CPUS_PER_DOM=2 # Set DOM0_PASSWORD, Domain0 root password export DOM0_PASSWORD=oracle #Set Agent PASSWORD export AGENT_PASSWORD=oracle # Set Public network interface on the dom0. # public_if=eth0, For OVM 2.2.* server releases # public_if=bond0, For OVM 3.* server releases export public_if=bond0 #Set TESTSET_TYPE=dom0/domu/dom0-domu/all/none # TESTSET_TYPE=dom0 to run only dom0 tests # TESTSET_TYPE=domu to run only domu tests # TESTSET_TYPE=dom0-domu tests to run only dom0-domu tests # TESTSET_TYPE=all to run all dom0,domu, and dom0-domu tests # TESTSET_TYPE=none will just configure olt on dom0 and on all domu's export TESTSET_TYPE=none #Set VALIDATED_CONFIG=1 to run tests for Validated Configuration export VALIDATED_CONFIG=0 #Set HW_OVM_CERTIFICATION=1 to run tests for Hardware Certification #else let it be set to 0 # for Hardware Certification dom0 tests will be run for all VM's and #domu feature tests will be run for VM's containing oracle DB. export HW_OVM_CERTIFICATION=0 # All the Oracle Linux (OL) templates should be under template pool 7

8 # location /OVS/seed_pool. if ol_template<n> is not set/provided, # by default all the OL templates will be taken from template pool location. #export ol_template1=ovm_el4u6_x86_64_pvhvm_4gb.tgz #export ol_template2=ovm_ol5u2_x86_64_pvm_4gb.tgz # Set db_template=all if you want to run more than one database on same # given OL templates put all databse templates in oracle pool location # i.e. /OVS/oracle_pool export db_template=none # when required to run domu based tests on a VM # All the DB templates should be put under oracle pool location # i.e. /OVS/oracle_pool # Set oradb_template only if you want to configure the corresponding VM # with oracle database. # If TESTSET_TYPE=dom0 is set,then the oradb_template setting is not required. # Set oradb_template1,oradb_template2,..,oradb_template<n>, # each with oracle image corresponding to each of the VMs #(ol_template1,ol_template2 etc) respectively. #export oradb_template1=ovm_ol5u2_x86_64_oracle11g_pvm.tgz #export oradb_template2=ovm_ol5u2_x86_64_oracle10g_pvm.tgz # Set fs_type=ext3/nfs/ocfs2 # File system on which we want to create the database export fs_type=ext3 # fs1,fs2,...the storage configured for the test database on the VM as seen # from Dom0 partition or image file for SAN,iSCSI, direct attached storage, or # nfs backed file image. - NFS mount point to used by the VM # E.g: # For partition, set fs1=<partition name> ; # for image file, set fs1=<full pathname of image file> # For nfs filesystem to be directly used by VM, # set fs1=<nfs server>:<dir to mount> format. # NOTE: If the partition or file image already has a filesystem created on it # then make sure to specify the same filesystem type in fs_type parameter. # The Storage space should meet the requirements as mentioned in OLT user guide. #export fs1=<partition name> #export fs2=<partition name> 8

9 5.2. Execution of OLT Tests Scheduling tests There are two ways to execute the tests from Dom0 as root Using oltinstaller [ first time installation/reinstallation of OLT on the system]./oltinstaller oltinstaller will install and setup your system for OLT tests; and then execute the tests. Using olt-xen-test If changing any VM settings, edit /opt/oracle/oltest/olt-schedule/domains.config to make the changes. To run All Tests: olt-xen-test -f domains.config -r -t all To Run subset of the Tests: olt-xen-test -f domains.config -r -t <dom0/domu/dom0-domu> Monitoring Tests To monitor the progress of tests executed specify olt-xen-test -f domains.config -s To monitor the progress on a subset of tests: olt-schedule -f dom0.cmd -s ( for dom0 only ) olt-schedule -f <domname>-domu.cmd -s ( for any one VM only ) olt-schedule -f dom0-domu.cmd s ( for tests running simultaneously on dom0 and VMs ) 5.3. Logs and Test Results The /OVS/olt-logs directory will have all the OVM tests logs after the same is completed. The /home/oracle/work directory on dom0 & VMs will have all the respective test logs and a run.suc file in the work area for the test indicates success. A run.dif in the work area for the test indicates failure The /OVS/olt-xen/logs/ directory will have test framework logs. The /opt/oracle/oltest/olt-schedule/log/olt-summary.csv has the collated results of all the respective test runs for dom0 and VMs. 9

10 5.4. Terminating tests olt-xen-test -f domains.config -k <cleanup-options> Below are the cleanup options: stop-test : kill the olt test running in all the VMs shutdown : kill the olt test and shutdown all the VMs cleanup : kill olt tests, shutdown VMs and delete all the VMs Cleaning up all VMs olt-xen-test -f domains.config -k cleanup The above command will shutdown and delete all the VMs. This is required in case of any abrupt termination of test or configuration Debugging OLT issues For detailed documentation please refer to DEBUG.txt & README-OLT-XEN under /opt/oracle/oltest/olt-schedule/doc/. Refer to Section Troubleshooting OLT Tests for troubleshooting and Appendix A. 6. Migration on Oracle VM with OLT This section provides the details of how to manually run migration tests on Oracle VM with OLT. These tests has to be executed after the OLT-OVM tests have been completed. Setup of Test Servers for Migration Server A This server should have already been setup for the previous automated tests. Server B(for migration purpose) This should have been setup such that: /OVS should be on shared storage such as nfs (NAS ), or ocfs2 (SAN or iscsi) Verify that the storage is visible on Server B, has the same device name, and has permissions to access. Synchronize the time with Server A using ntpd Migration tests to be executed The tests below are to be run on each VM 10

11 Note: In the table below, To schedule tests XX for a VM <VMname>, do the following In Dom0, Edit /opt/oracle/oltest/olt-schedule/<vmname>-domu.cmd to uncomment stmem-do m-migration testcase & also comment out the rest of the testcases. Start up the VMs using the following command in Dom0 # cd /opt/oracle/oltest/olt-schedule # olt-xen-test -f domains.config -r In Dom0, Schedule the tests using cd /opt/oracle/oltest/olt-schedule/ olt-schedule -f <VMname>-domu.cmd -r After each test (Tests 1-4, mentioned below ) completes, migrate the domain from Server B to Server A # xm migrate -l <domid> Server A In Expected Results Column,to verify Vm should migrate and systems should be up On Server A to verify that the VM is not present # xm list Server B to verify that the VM is present and in r state # xm list Connect to the VM through vnc console and through ssh to verify that the VM is functioning properly and is not unusable OLT tests should complete without issue On Server B, connect to the VM using vnc or using xm console. Verify the presence of run.suc file in /home/oracle/work/<testrun of OLT> 11

12 6.1 Migration Tests No Test Name Server A - Dom0 (Steps to be run) 1. Verify xmmigrate works 2. Verify Migration during OLT st-mem-dommigration test load. From Server A Dom0 execute xm migrate -l <domid> Server B Schedule test st-mem-dommigration.an hour into the test, execute the following command from Dom0 on Server A: xm migrate -l <domid> Server B Expected Results VM should migrate and the systems should be up VM should migrate and the systems should be up. OLT Test should complete without issues 3. OLT Dest with migrate (storage) 1. Schedule test st-mem-dom-migration 2. An hour into the test, execute the following command from Dom0 on Server A : xm migrate -l <domid> Server B 3. Disable storage on Server A for 10s 4. On Server B, Verify the VM status 5. Enable storage on Server A OLT tests should complete successfully 4. OLT Dest with migrate (network) 6. On Server B, verify the VM status 7. After the test completes, Migrate Back the Guest to Server A and rerun the test st-mem-dom- migration. 1. Schedule test st-mem-dom-migration 2. An hour into the test, execute the following command from Dom0 on Server A : xm migrate -l <domid> Server B OLT tests should complete successfully 3. Disable network on Server A for 10s 4. On Server B, Verify the VM status 5. Enable network on Server A for 10s 6. On Server B, verify the VM status 7. After the test completes, Migrate Back the Guest to Server A and rerun the test st-mem-dom- migration. 12

13 Result Logs Refer to the table above for expected behavior. Logs to be sent for auditing/for review: For OLT tests, refer to the Logs and Test Results Section in the Oracle Linux Test (OLT) User Guide Record the results as per the Expected results for each test. 7. Troubleshooting OLT tests 7.1. How to rerun a failed test Verify that the VM is running. If not, run the below command (as root user ) to manual setup/reconfigure VMs for OLT for OVM, # olt-xen-test -f domains.config -r Rerun a failed case. To rerun the failed dom0 testcases : Edit /opt/oracle/oltest/olt-schedule/dom0.cmd to comment out all test entries which have passed, retaining only the tests that require a rerun. Run the below command as "oracle" user to reschedule the required dom0 tests. #olt-schedule -f dom0.cmd -r To rerun the failed testcases on the VM (domu tests) : Edit /opt/oracle/oltest/olt-schedule/<vm>-domu.cmd all the VMs that have failed tests e.g.: OVM_OL5U3_X86_64_PVM_4GB-domu.cmd to comment out the test entries which have passed, retaining only the tests which requires rerun. Run the below command as "oracle" user to reschedule the required domu tests for each VM that has failed tests. # olt-schedule -f <VM->-domu.cmd -r To rerun the failed testcases run across Dom0 and VM (dom0-domu): Edit /opt/oracle/oltest/olt-schedule/dom0-domu.cmd to comment out all the test entries except for the failed test for the VMs being tested. For e.g: for xm-save- restore test across VM- OVM_OL5U3_X86_64_PVM_4GB <machinename>=xm-test-save-restore OVM_OL5U3_X86_64_PVM_4GB.<domainname>=xm-mem1-dbt2_4 Note: Each dom0-domu testcase has two parts, an xm operation related test running on Dom0 & a stress test running on VM. So the edited dom0-domu.cmd file should contain the correct test entries for dom0-domu testcases. On Dom0, execute the below command as "oracle" user to run the test # olt-schedule -f dom0-domu.cmd -r Note: Use the command on Dom0 to verify the status of the above scheduled tests, # olt-schedule -f <command file used for scheduling the tests> -s 13

14 7.2. Manual Configuration of OLT for OVM For reconfigurations, If there is any change in a VMs configuration mentioned above, then make the required changes to the VM config file, domains.config located at /opt/oracle/oltest/olt-schedule and run the below command to reconfigure & setup the VMs, olt-xen-test-f <VMs config file> -r More details at /opt/oracle/oltest/olt-schedule/doc/readme-olt-xen 8. Appendix A OLT Utilities This contains the list of OLT Utilities available in /opt/oracle/oltest/olt-schedule/utils olt-debug - [ No arguments ] This script collects system configuration and log files for debugging issues related to OLT test executions. Generates a tar file olt-debug-log-${time_stamp}.tgz in utils directory olt-logcleaner - [ No arguments] Clean up test logs for freeing up space. All QA directories older than 10 days will be deleted. No archive will be maintained start-qa / stop-qa - [ No arguments ] A script to manually start up/stop qamonitor without running OLT. Useful for collecting system logs during debugging olt-archive-logs [si/rac/ovm] This utility collects olt test log files, xen logs and system logs as an archive file - /home/oracle/work/olt-testlogs-`hostname`-${time_stamp}.tgz. 9. Appendix B- How-to edit olt-setup.env file for your storage configuration This describe how to setup olt-setup.env files for fs_type and fs<n> paramters fs_type: Set fs_type to reflect the filesystems for the Oracle Database datafiles on the VM For SCSI, iscsi, or NAS(using nfs file based image ) fs_type=ext3 or fs_type=ocfs2 14

15 For NAS: fs_type=nfs fs<n>: Map the 100GB storage partitions or files created during Storage Configuration to this setting. Note that the disk partition names and directory names mentioned below are examples and should be changed to reflect your storage configuration. Let P be the the number of VMs that can be tested in parallel -P ( from Section 3 above ) For SCSI, iscsi, specify the partition to be used by P VM fs1=/dev/sda2 fsp=/dev/sda<p+1> For NAS, specify the nfs exported filesystem to be used by each of the VM fs1=/ovm_test/vm1 fsp=/ovm_test/vm<p> For NAS using nfs file based image, specify the image file to be used for each VM fs1=/ovm_test/vm1_testdata.img fsp=/ovm_test/vm<p>_testdata.img Note: If the number of partitions or the exported nfs file system is less than the number of VMs, that can be tested in parallel (P), then these partitions/file systems will be reused for each loop of VMs that runs in parallel. 15