Who I am DARWIN IT-PROFESSIONALS. Master your VM s with Vagrant. Agenda. Per november 17: Oracle ACE. Quick Introduction to Virtualization

Transcription

1 DARWIN IT-PROFESSIONALS IT Driven Evolution Who I am Master your VM s with Martien van den Akker 08 B.V. Per november : Oracle ACE Cook a Base Box in your Kitchenette B.V. Quick Introduction to Virtualization AN INTRODUCTION TO VIRTUALIZATION VMWare first to introduce virtualization on Intel platform VirtualMachines Behave like real computer servers Highly movable because they re just a bunch of files Not dependend on hardware changes Allow for sharing hardware, better allocation and use of resources Started with VMWare Workstation and later Server and Player around 00, 00. Since 009 Oracle owns VirtualBox (Sun Microsystems acquisition)

2 Quick Introduction to Virtualization Quick Introduction to Virtualization Never install server software right on the laptop New laptop just means installing hypervisor and moving the VMs In 00/00 investigated and implemented Oracle BB: VMWare allows for multiple instances on same laptop. Or on different servers to mock different trading partners. Only install JDeveloper and SQL developer on host (laptop) VMWare, VirtualBox, etc. implement virtualization on HardWare level Docker, etc. implement virtualization on OS Level But: Creation of new VM s is a tedious job New OS update or new release of software asks for new VM installations Installations/configurations done by hand, hard to reproduce Multiple versions of VMs with different states to be archived/catalogued VMs can grow big After period of not-use: state of VM is unclear Every colleague creates a VM in his/her own way: lack of standardization 8 Cook a Base Box in your Kitchenette 9 0 : what? : why? Open-source software product for building and maintaining portable virtual software development environments Simplifies software configuration management of virtualizations To increase development productivity Creating and maintaining VM s is a tedious job: Upgrades of OS Upgrades of Software States of the configuration : Automates creation of VM s Automates provisioning of VM s Abstracts the configuration of the VM the installation/setup of the software within the VM

3 : How? Project Ingredients Base Box (Repo/Url/File) file Shell Scripts/ Cookbooks/ Recipies Provisioners Providers Chef Puppet Ansible Shell VirtualBox VMWare Docker Appliance Project folder Contains file with all the configuration BaseBox Downloadable from repo, or custom box Contains a VM, with OS installed in it user with sudo rights, and insecure key (replaced on startup) SSH daemon running NAT network adapter Architecture Components Building block concepts: Providers: VirtualBox Docker Vmware AWS Services to set up and create VMs Provisioners: Shell Ansible Puppet Chef Tools to customize the configuration of VM Cook a Base Box in your Kitchenette What is a base box? COOK A VAGRANT BASE BOX IN YOUR KITCHENETTE A base box is a template to build a VM Downloadable from Cloud: (Just provide a name) Downloadable from an URL Local file Added in local vagrant repo on first use 8

4 Base box requirements Base box build options An OS: I use Oracle Linux Update for this story A vagrant user public insecure key add vagrant to the sudoers list A started ssh daemon NAT Adapter as the first one VirtualBox GuestAdditions installed Password of root convention: vagrant recommends Packer Automates creation of base boxes ( Has its own requirements For full EE lifecycle Regularly building base boxes on a daily base (on nightly builds for instance) has a built in package command Incidentally creating base boxes I ll use this for the story 9 0 Base box build steps Download OS Base box build steps Create VM I used the latest OLU Download Oracle Linux from edelivery.oracle.com VirtualBox with Ext pack and assumed installed Create the VM 00 MB memory and a 8GB dynamically allocated virtual disk, CPUs Add downloaded ISO Start VM Base box build steps Install OS Base box build steps Install OS KaSaP principle: Keep as Slim as Possible Just select Server with GUI Chang the network hostname to darlin-vce.darwin-it.local Set root password to vagrant Create vagrant user with password vagrant Finish installation Answer licensing question Finish OS installation Answer licensing question Enable Automatically connect to this network when it is available on the General tab

5 Base box build steps Install Guest Additions Base box build steps Install Guest Additions Install Kernel Packages (more fun to do afterwards) Stop packagekit Insert Guest Additions Install the packages kernel-uek-devel kernel-uek-devel eluek.x8_ Base box build steps Configure user Base box build steps package the box Add line to /etc/sudoers file: As vagrant user create a.ssh folder in the vagrant home folder and create file authorized_keys in it: Insert the in secure key from Base box VM is ready: package it. You might want to create an OVA of it, so you can remove the base vm To package box, use vagrant package command: -- base: Name of VirtualBox VM to package -- output: file path to the box to package box to create 8 Manage your base boxes At startup the box is added to your local Repo List boxes with box list: Files can be found in.vagrant.d/boxes folder your user profile Can be removed using box remove: Cook a Base Box in your Kitchenette 9 0

6 Create a project structure BASE BOX READY? LET S CREATE A BOX! Create a project structure with boxes base boxes projects projects Stage Stage folder installation-binaries, for database, Weblogic, Java, FusionMiddleware and so on. Create a project folder for new VM Open a command window ( assumed installed) A clear project structure makes it easy to copy&paste and share projects and refer to components Create a project structure Initialize Project folder: With option -m or --minimal setting a just enough vagrant file is created, without comments Edit file file Globals (Recommended) Configuration (version ) Define base box to use, multi-machine naming SSH config Provider Configuration Set VM properties (Memory, CPU, etc) Toggle Bidirectional Clipboard/Drag&Drop Define and attach extra storage/media Provisioning Call shell script or Chef/Puppet/Ansible recipies/playbooks, etc. Configuration version relates to version of the configuration options Can have multiple configuration blocks Ends with end Place Global variables at the top of the file Easy tweaking/configuration Readable, meaningful settings Adds to auto-commenting project file Introduce globals VM Configuration Start with global variables for easy config: BOX_URL: link to the base box created in former section VMS_HOME: location where VM will be created. To have extra disk files in the same folder. Name the base-box using globals Add the lines: config.vm.box_url: directive refers to the base box used Just give a name to the box in the Cloud (see earlier) Or link to the boxes folder config.vm.define: allows to define multi-machine definitions in one project, like Middleware and Database in separate VMs.

7 Edit file: SSH vagrant user Provider config uses SSH to connect to VM and provision it. Can be configured using config.ssh.* properties: Provider configured using following block: Password should be optional, because of the injected insecure key. At first startup insecure key will be replaced by generated secure key. Config.ssh.port: port uses as a local port forwarded to the internal ssh port. 8 Provider config Provider config Create nd disk Properties in the block: Property vb.name vb.gui vb.memory Meaning Name of the VM to create. This is how the VM will appear in VirtualBox. Toggles the appearance of the UI of the VM This sets the memory available to the VM. In the global variables I have set it to GB (*0 MB) vb.cpus Number of CPU cores available to the VM. In the globals set to. vb.customize to an API to the VboxManage utility of VirtualBox: modifyvm: modify vm properties like --clipboard and --draganddrop to bidirectional createmedium: create a new disk: vb.customize [ "createmedium", "disk", "--filename", VM_DISK, "--format", "vdi", "--size", VM_DISK_SIZE, "--variant", "Standard" ] --format vdi: Virtual Disk Image --variant Standard: dynamically allocated file unless File.exists only create the file if it does not exist yet: 9 0 Provider config attach nd disk Shared/Synched folders vb.customize to an API to the VboxManage utility of VirtualBox: storageattach: attach the disk to the VM vb.customize [ "storageattach", :id, "--storagectl", "SATA", "--port", "", "--device", "0", "--type", "hdd", "--medium", VM_DISK] --storagectl SATA: storage controller to add the disk to --port, --device: port and device numbers on the controller --type hdd we re adding an harddisk here --medium: the medium to add :id: Special variable that refers to the VM that is created in VirtualBox by default creates a folder-link or shared folder, a so-called Synched Folder project folder (the one with file in it) mapped by default on /vagrant. For stage folder we add: Remember:

8 Provisioning Up, up and up it goes! allows for several provisioners like Puppet, Chef, Ansible, Salt. But a sample Shell snippet is provided in our file: If everything went alright, you re now ready to fire up your VM. Then simply issue the following command in the command window: I replaced it with: Only main steps, details in sub scripts Script is run as vagrant Can also use external scripts, even over URL Cook a Base Box in your Kitchenette A BIT MORE ADVANCED PROVISIONING Or actually, the question is: How do I provision using a provision user but not have it become the product owner? SO: Compare SYSTEM or DBA-user in database Compare Schema Owner user in database Cook a Base Box in your Kitchenette Provision but not owning 8 8

9 The provisioning/not owning dilemma Solution: run as another user The provisioning is done using the vagrant user. The vagrant user is in the sudoers list -> can run a script using the permissions of another user; sudo su - oracle -c "script does not do the trick -> vagrant still owns the files It is still the running vagrant user who owns the resulting files and folders. But oracle (or other install owner) is to own it. Option: create a new base box with another install user? Solution: run as another user Must be run as root, but that's no problem since vagrant is in the sudoers list Add following to the provisioning block: Script installjava.sh left unexplained Database, Fusion Middleware, etc. or other tools requiring other users can be installed this way THANK YOU FOR YOUR ATTENDANCE, PATIENCE AND ATTENTION Q & A HELPFUL COMMANDS B.V. 9

10 Some other helpful commands Some other helpful commands Basic vagrant commands: Command init package up halt suspend destroy Meaning Initializes current folder with an initial file Packages a VM from provider into a box Creates a vm from a box if non-existent, starts up the VM, and provisions it if not already done Stops the VM Suspends the VM Removes the VM Basic vagrant commands: Command Meaning ssh Starts a secure shell connection, logs on as vagrant user provision Reruns the provisioning of the VM box add This adds a box with the given address to local repo box list This command lists boxes in local repo box remove This command removes a box from local repo box update Updates a box from an updated base box in local repo Prepare Oracle Linux EXAMPLE PROVISION SCRIPTS Install required packages Install haveged to increase entropy for non-gui VMs Add security limits for oracle user Change kernel settings 8 Create filesystem Create Oracle user Create mountpoint Create logical group and volume using System Storage Manager Add mount point to fstab Function to read oracle user properties from file Create groups dba and oinstall Create the user with the groups Set the password from property file 9 0 0

11 Create Oracle user Add oracle to sudoers Create orainventory Create orainst.loc