Summary of Features For Ignite Version 3.4: o Backup, recovery, and copy data enhancements. o Disaster recovery orchestration via REST API o VDI enhancements with Windows unattended install and configuration capabilities o Live resize of disks with Windows and Linux instances o Smart disk placement across federated storage pools o Pre and Post snapshot processing scripts Additional Feature Notes: o Software based encryption Backup, recovery and copy-data enhancements Background In IT, there are countless situations in which it is desirable to create copies of your production data: to perform tasks such as malware analysis, application development and test (dev/test), user acceptance testing (UAT), analytics, data protection, archiving and provisioning of new VM s. It is not uncommon to see 7-10 copies of production data in enterprises today. Unfortunately, traditional copies don t come for free. They consume significant storage, server and network resources, and valuable administrator time and energy. As a result, many IT shops make do with fewer, and less up-to-date copies than they really need. The problem is so significant that entire companies have been created to attack this problem of copy data management. Cloudistics cloning allows IT administrators to support many data protection and availability use cases directly within the Cloudistics platform, all from only one golden copy of your production data. The application consistent clones can be created in seconds, are space efficient and available for quick access while also being deduplicated and compressed for backup or long-term retention. Clones allows IT administrators to access a single, point-in-time copy of your primary data. The basic operation is Clone and Attach which allows a consistent copy to be made and immediately attached to a new application that needs that copy. Clones may also be deleted once their need is over. A common use case for clones is malware analysis in a VDI environment. By using the Cloudistics API, active VM s disks can be cloned and attached to a VM with malware software installed that does scanning for vulnerabilities. The clone can then be deleted. This is accomplished during normal business operations and without disrupting the production VM. 1
Using Clone and Attach We added the ability to clone an individual disk (vdisk) in a snapshot and to permanently attach it to any application instance on the same virtual datacenter and storage pool as the snapshot. Attaching a vdisk to an instance from its own snapshot Method 1: 1. Select Add vdisk from the Actions drop-down menu or click Add vdisk in the Storage section of the instance's Profile page. 2. Click Add vdisk clone. 3. Select the snapshot and vdisk you wish to clone and attach to this instance. 4. Click Clone and Attach to Instance. Method 2: 1. On the instance's Local Snapshots page, click... beside the snapshot and click View vdisks. 2. Click... beside the vdisk you wish to clone and select Clone and attach to this instance. 3. Click Clone and Attach to this Instance. Attaching a vdisk to an instance from another instance's snapshot 1. On the instance's Local Snapshots or Disaster Recovery page, click... beside the snapshot and click View vdisks. 2. Click... beside the vdisk you wish to clone and select Clone and attach to. 3. Select the instance you wish to attach the vdisk to, and click Clone and Attach to Instance. After adding the disk in Windows 1. In the instance's console, open Disk Management 2. Right-click on the newly added disk and select Online. After adding the disk in Linux In this example, the new, clone disk is /dev/sdb with partition /dev/sdb1, which has an ext4 filesystem. 1. In the instance's console, run lsblk to verify the new, clone disk shows up. 2. Create a mount point: mkdir /disk1 3. Mount the clone disk to the mount point: mount /dev/sdb1 /disk1 4. Edit fstab so the new drive automatically mounts to /disk1 on reboot: o vi /etc/fstab o Enter ESC and type i to edit the file. o Add a line: /dev/sdb1 /disk1 ext4 defaults 1 2 o Enter :wq to write the changes and save the file. 2
The KB article is also updated on ZenDesk (https://support.cloudistics.com/hc/enus/articles/115003953853). REST API support for Clone and Attach All Cloudistics REST APIs are documented in Github. The Clone and Attach Disk API call fully supports the functionality referenced above. The process for using the Clone and Attach Disk API call is to: Obtain the UUID of the application / Virtual Machine you want to attach a disk clone to; this will be used in the REST end-point URL: Example Call curl -H "Authorization: Bearer [YOUR TOKEN]" \ -G https://manage.cloudistics.com/api/latest/applications \ -d 'limit-count=10' Example Results "uuid": "e696855c-186f-4c2a-a381-1637195bef3f", "name": "CentOS 7.5 Application", "description": "This application was created from a CentOS template.", "vcpus": 4, "memory": 1073741824, "categoryuuid": "a55bb4da-7cad-40cb-95e0-37db93c7aa5e", "tags": [ "uuid": "fa527c37-3582-41e5-afcb-4b181b8e39aa" } ], "datacenteruuid": "101552a2-e436-415a-a1cd-a11e5cb1e06e", Obtain the UUID of the disk snapshot you wish to attach to this VM. In the example below, we are querying the snapshots of the same VM we wish to attach to, and the result is a single snapshot of two disks: Example Call curl -H "Authorization: Bearer [YOUR TOKEN]" \ -G https://manage.cloudistics.com/api/latest/applications/e696855c-186f-4c2a-a381-1637195bef3f/snapshots \ -d 'limit-count=10' Example Results [ "uuid":"b576c8de-c70f-46d9-9a27-667416b2f788", "name":"snapshot One", 3
"createdtimestamp":"2016-11-15't'12:00:00", "size":1073741824, "type":"local", "generated":"manual", "disasterrecovery": "transferstatus":"transferring", "transferpercentage":50 }, "bootorder":[ "diskuuid":"1b35fadb-7c63-46a6-9011-1df2a4f34918", "name":"disk 1", "order":1 }, "vnicuuid":"bf6199ea-a94b-4768-b24d-8994bf9bdf88", "name":"vnic 0", "order":2 }, "diskuuid":"694c1484-ced4-4810-bcb3-a302ffb45f12", "name":"disk 2", "order":3 } ] } ] Clone the snapshot s disks individually as so: Example Call curl -H "Authorization: Bearer [YOUR TOKEN]" \ -H "Content-Type: application/json" \ -X PUT https://manage.cloudistics.com/api/latest/applications/e696855c-186f-4c2a-a381-1637195bef3f/clone-and-attach-disk \ -d '"uuid": "8292f853-1c47-41ac-941c-b8fad35e8497"}' Example Results "actionuuid": "71011d4e-a4f7-4ab1-95e0-9e8986fb3f2c", "objectuuid": e696855c-186f-4c2a-a381-1637195bef3f" } 4
Disaster recovery orchestration via REST API The Cloudistics REST API now provides API control for automation, batching, and sequencing capabilities for large-scale deployment of applications and managing a virtual networking infrastructure. Customers can execute fine-grained disaster recovery plans to match their business needs. REST API support for Disaster Recovery Automation Create, update, and delete Vnets. This allows customers, in the event of a disaster, to programmatically recreate existing Vnets on the disaster recovery site. Create an application instance from a snapshot. This allows customers to launch a new application instance from the latest available snapshot of an application. Our solutions team will offer code examples upon request. Please contact support for more information. VDI enhancements with Windows unattended install and configuration capabilities Cloudistics added the capability to do unattended installation and configuration of Windows instances. Application administrators are able to specify a sysprep answer file for creating and configuring Windows instances. An answer file allows administrators to automatically preconfigure instance settings such as joining Active Directory groups and accepting the EULA during creation and boot time for a fully unattended install. The steps are as follows: Create a gold master Windows 2016 Remote FX/RDSH instance. Create a template from that instance. During the template creation process: o Inject a sysprep answer file with settings for unattended install. Bulk deploy instances from that template. The answer file will be part of each instance. Resize storage disk attached to a running Windows instance In this release, we added the capability to hot-expand a disk attached to a Windows instance. Using Resize Disk How to resizing a disk which supports both Linux and Windows instances. 1. On the application instance's Profile page, click beside the vdisk you want to resize and click Edit Size. 2. Enter the new size of the disk. The new size must be larger than the current size. 3. Click Save. 5
On Windows a. Console into the Windows instance. b. Open Disk Management. The disk that was resized will show Unallocated space. c. To create a new volume in the Unallocated space: i. Right-click on the Unallocated space and select New Simple Volume. ii. Click Next and enter the volume size iii. Click Next and assign a drive letter to the new volume. iv. Click Next, select formatting options and assign a volume label. v. Click Next and Finish. d. To extend the current volume into the Unallocated space: i. Right-click on the volume and select Extend Volume. ii. Click Next and enter the amount of space to add to the volume. iii. Click Next and Finish. After resizing disk on Linux 1. Console into the Linux instance. 2. To create a new, ext4 partition /dev/sdb2 on disk /dev/sdb: 1. Check how many drives are installed: fdisk -l grep '^Disk' 2. Add a partition to the disk: fdisk /dev/sdb Type p to use the partitioner, n to create a new partition, and select your preferred settings when prompted. Type w to save the partition. 3. Format the disk: mkfs.ext4 /dev/sdb2 4. Create a mount point and mount the partition: mkdir /disk2 mount /dev/sdb2 /disk2 5. Edit fstab so the new partition automatically mounts to /disk2 on reboot: vi /etc/fstab Enter ESC and type i to edit the file. Add a line: /dev/sdb2 /disk2 ext4 defaults 1 2 Enter :wq to write the changes and save the file. 3. To expand an existing, ext4 partition /dev/sdc1 on disk /dev/sdc that's mounted to /disk3: 1. Unmount the partition: umount /disk3 2. Expand the partition by deleting and recreating it without deleting the data: fdisk /dev/sdc Print the current partitions: p Delete the partition: d If multiple partitions are present, select the desired partition when prompted Recreate the partition: Type p to use the partitioner, n to create a new partition, and select your preferred settings when prompted. Type w to save the partition. 3. Expand the ext4 filesystem: resize2fs /dev/sdc1 6
These instructions have also been added to the KB: https://support.cloudistics.com/hc/enus/articles/218442058 Additional Enhancements Storage enhancements with smart disk placement In this version we optimized storage utilization of storage pools with smart placement of an application instance's disks on multiple storage blocks when storage pools are federated, and capacity is available. This smart placement ensures that storage is utilized across blocks more efficiently. There are two options, it is important to understand the implications of each before implementing. 1. If the new disk is part of a VM template which has already been instantiated, then it is possible to use that same storage unit and create a clone of the disk. This is the quickest approach and requires the least amount of resources. However if the storage unit is using over 80% of its physical capacity, follow Item 2 below. 2. Determine the storage unit with the lowest Overallocation Exposure Factor (OEF). OEF is calculated as follows: OEF = Ratio of allocated virtual capacity (i.e. sum total of allocations of all disks on the storage block) to actual used physical capacity. This method favors spreading allocations across blocks as evenly as possible. If the Storage Unit with the lowest OEF is using over 80% of its physical capacity, utilize the Storage Unit with the next highest OEF or skip placement if there are none available. Added snapshot pre and post processing scripts In this version we added the ability for users to specify pre- and post- snapshot scripts that can be invoked in an application instance whenever a snapshot is taken. We also added the ability to specify a custom time interval for fine-grained specification of auto-snapshot policies. One use case for this feature is to ensure the effective backup of databases, such as flushing I/O caches and placing Oracle tablespaces into hot backup mode, and then taking the database out of backup mode after the snapshot is created. 7
Other Feature Notes Software Encryption Beginning with version 3.2 we use a hardware accelerated FIPS 140-2 certified software encryption engine and don t require SED drives, reducing flash drive costs. Since our software encryption engine is hardware accelerated by specialized encryption/decryption instructions, there is no performance penalty. Our implementation interfaces to external key managers - SafeNet and Vormetric - over KMIP to support special encryption use cases. Please see the online user guide: https://support.cloudistics.com/hc/enus/sections/115000396173-encryption-and-kmip or contact support for more information. Bug Fixes When accessing a share (import VM, create application from installer), the guidance message to the user "The share must be accessible by the storage controllers' Virtual IP", now includes the virtual IP in the message. The link labeled "Learn more about instance modes" at the bottom of the page when creating an instance has been corrected. Vnet creation is no longer prevented when a template description is NULL Stat file size no longer causes storage controller connection failures to the Ignite cloud controller. For customers who do not have Ignite SDN routers on UPS power, the SDN router no longer enters a reboot loop when trying to recover from a loss of power. The Cloudistics API now responds with JSON instead of HTML to GET migration-zones calls. The intermittent issue where importing template would fail after timeout - due to an image conversion failure - has been fixed. When creating a new virtual network with Virtual Networking mode, the Virtual Network select box is no longer empty. A syntax error no longer occurs when importing a VM from a directory with a parenthesis in its name. A small memory leak in the Cloudistics hypervisor kernel has been repaired. Known Limitations Compute block statistics will not be collected for applications running in compatibility mode. Restarting a controller service may sometimes result in the service not reconnecting to management. The workaround is to log onto the machine, stop the service, wait two minutes, and start the service again. In rare cases related to timing, a duplicate application template request may fail to be completed and take up to 24 hours to be canceled due to time-to-live request constraints. On Windows 10 applications in a Vnet, manual DHCP renewal may fail (ipconfig \renew). The workaround is to first release (ipconfig \release) and then (ipconfig \renew). 8