Hitachi Storage Adapter for Oracle ASM Storage Reclamation Utility User s Guide

Transcription

1 Hitachi Storage Adapter for Oracle ASM Storage Reclamation Utility User s Guide v FASTFIND LINKS Document Organization Software Version Getting Help Contents MK-92ADPTR036-05

2 2016 Hitachi, Ltd. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or stored in a database or retrieval system for any purpose without the express written permission of Hitachi, Ltd. Hitachi, Ltd., reserves the right to make changes to this document at any time without notice and assumes no responsibility for its use. This document contains the most current information available at the time of publication. When new or revised information becomes available, this entire document will be updated and distributed to all registered users. Some of the features described in this document might not be currently available. Refer to the most recent product announcement for information about feature and product availability, or contact Hitachi Data Systems Corporation at Notice: Hitachi, Ltd., products and services can be ordered only under the terms and conditions of the applicable Hitachi Data Systems Corporation agreements. The use of Hitachi, Ltd., products is governed by the terms of your agreements with Hitachi Data Systems Corporation. Hitachi is a registered trademark of Hitachi, Ltd., in the United States and other countries. Hitachi Data Systems is a registered trademark and service mark of Hitachi, Ltd., in the United States and other countries. Archivas, Essential NAS Platform, HiCommand, Hi-Track, ShadowImage, Tagmaserve, Tagmasoft, Tagmasolve, Tagmastore, TrueCopy, Universal Star Network, and Universal Storage Platform are registered trademarks of Hitachi Data Systems Corporation. AIX, AS/400, DB2, Domino, DS6000, DS8000, Enterprise Storage Server, ESCON, FICON, FlashCopy, IBM, Lotus, MVS, OS/390, RS6000, S/390, System z9, System z10, Tivoli, VM/ESA, z/os, z9, z10, zseries, z/vm, and z/vse are registered trademarks or trademarks of International Business Machines Corporation. All other trademarks, service marks, and company names in this document or website are properties of their respective owners. Microsoft product screen shots are reprinted with permission from Microsoft Corporation.

3 Contents Preface... 7 Intended Audience... 7 Release Notes... 7 Document Organization... 8 Referenced Documents... 8 Document Conventions... 9 Convention for Storage Capacity Values Getting Help Comments About Hitachi Storage Adapter for ASRU Product Overview Real-World Activities Architecture Functions hizpry.py display_storage Function hizpru.py reclaim_asm Function hizpru.py reclaim_storage Function hizpru.py storage_status Function Requirements and Prerequisites Hardware Requirements Software Requirements Server and OS Storage Model Oracle Software Hitachi Software Host Interface Volume Type Linux Packages HUVM Setup Requirements Contents 3

4 Supported Configurations RAID Manager Setup for Non-HUS Subsystems Storage Support SNM2 Command-Line Interface for HUS Subsystems Required Pre-operations Distribution Package Adapter Prerequisites Create LUs for the Required DP-VOLs Configure ASM Disks Set up the horcm<inst> Configuration File RAID Manager Setup Run RAID Manager Commands Using an In-band Method Required Account Privileges Set Ownership for Adapter Scripts Run RAID Manager Commands as Root Run SNM2 CLI Commands as Root Run sg3 Utilities as Root (Linux only) Installation, Deployment, and Uninstallation Summary of Steps Import an Adapter Set up the Software Library Synchronize EM CLI Client with an OMS Import the Adapter Archive Deploy an Adapter Create Target Instances Create Jobs Specify a Name Choose Parameters Set Credentials Set the Schedule Submit the Job Uninstall an Adapter Delete a Job Delete a Target Instance Undeploy an Adapter Delete an Adapter Operations Adapter Workflow Adapter Scripts Read and Invoke a Configuration File Display Reclaimable Storage (display_storage) Run Oracle ASM and Mark Zero Pages (reclaim_asm) Contents

5 Reclaim Zero Pages on the LUNS Subsystem (reclaim_storage) Display Current Storage Status (storage_status) Display Help Logging and Message Codes Logging Message Codes Abbreviations, Terms, and Acronyms Contents 5

6 6 Contents

7 Preface This document describes how to use the Hitachi Storage Adapter for Oracle ASRU. This preface includes the following information: Intended Audience Release Notes Document Organization Referenced Documents Document Conventions Convention for Storage Capacity Values Getting Help Comments Note The use of Hitachi Storage Adapter for Oracle ASRU and all other Hitachi Data Systems products is governed by the terms of your agreement(s) with Hitachi Data Systems. Intended Audience Release Notes This document is for system and database administrators, as well others who manage storage. You can find release notes on the documentation CD. Release notes contain requirements and more recent product information that this manual may not fully describe. Be sure to review the release notes before installation. Preface 7

8 Document Organization The following table provides an overview of the contents and organization of this document. Click the chapter name in the left column to go to that chapter. The first page of each chapter provides links to the sections in that chapter. Overview Chapter Requirements and Prerequisites Installation, Deployment, and Uninstallation Operations Logging and Message Codes Abbreviations, Terms, and Acronyms Description Provides an overview of Hitachi Storage Adapter for Oracle ASRU. Identifies the prerequisites for installing the Hitachi Storage Adapter for Oracle ASRU. Provides details on setup of the Hitachi Storage Adapter for Oracle ASRU. Describes the operation of the Storage Adapter for Oracle ASRU. Describes logging and message codes. Defines the terms, acronyms, and abbreviations used in this document. Referenced Documents The following Hitachi referenced documents can be found on the applicable Hitachi documentation CD: Hitachi Virtual Storage Platform User and Reference Guide, MK-90RD7042 Hitachi Command Control Interface Installation and Configuration Guide, MK-90RD7008 Hitachi Command Control Interface User and Reference Guide, MK-90RD7010. Hitachi Command Control Interface Command Reference Guide, MK-90RD Hitachi Storage AMS 2000 Family Dynamic Provisioning Configuration Guide, MK-09DF8201 Hitachi Storage Navigator Modular 2 Storage Features Reference Guide for AMS, MK-97DF8148 Storage Navigator Modular 2 Command Line Interface (CLI) User's Guide, MK-97DF8089 Hitachi Storage Navigator Modular 2 Advanced Settings User's Guide, MK-97DF8039 Oracle Database Backup and Recovery User's Guide, E Oracle Automatic Storage Management Administrator s Guide, E Preface

9 Document Conventions This document uses the following typographic conventions: Bold Italic Convention screen/code < > angled brackets [ ] square brackets Description Indicates text on a window, other than the window title, including menus, menu options, buttons, fields, and labels. Example: Click OK. Indicates a variable, which is a placeholder for actual text provided by the user or system. Example: copy source-file target-file Note: Angled brackets (< >) are also used to indicate variables. Indicates text that is displayed on screen or entered by the user. Example: # pairdisplay -g oradb Indicates a variable, which is a placeholder for actual text provided by the user or system. Example: # pairdisplay -g <group> Note: Italic font is also used to indicate variables. Indicates optional values. Example: [ a b ] indicates that you can choose a, b, or nothing. { } braces Indicates required or expected values. Example: { a b } indicates that you must choose either a or b. vertical bar Indicates that you have a choice between two or more options or arguments. Examples: [ a b ] indicates that you can choose a, b, or nothing. { a b } indicates that you must choose either a or b. This document uses the following icons to draw attention to information: Icon Label Description Note Calls attention to important and/or additional information. Tip Provides helpful information, guidelines, or suggestions for performing tasks more effectively. Caution Warns the user of adverse conditions and/or consequences (for example, disruptive operations). Preface 9

10 Icon Label Description WARNING Warns the user of severe conditions and/or consequences (for example, destructive operations). Convention for Storage Capacity Values We calculate physical storage capacity (for example, disk drive capacity) based on the values below. Physical Capacity Unit Value 1 KB 1,000 bytes 1 MB 1,000 2 bytes 1 GB 1,000 3 bytes 1 TB 1,000 4 bytes 1 PB 1,000 5 bytes 1 EB 1,000 6 bytes We calculate logical storage capacity (for example, logical device capacity) based on the values below. Logical Capacity Unit Value 1 KB 1,024 (2 10 ) bytes 1 MB 1,024 KB or 1,024 2 bytes 1 GB 1,024 MB or 1,024 3 bytes 1 TB 1,024 GB or 1,024 4 bytes 1 PB 1,024 TB or 1,024 5 bytes 1 EB 1,024 PB or 1,024 6 bytes 1 block 512 bytes Getting Help The Hitachi Data Systems Support Center staff is available 24 hours a day, seven days a week. Provisions for patches and fixes are restricted to normal business hours, 8 a.m. to 5 p.m. PST. To reach us, please visit the support Web site for current telephone numbers and other contact information: If you purchased this product from an authorized HDS reseller, contact that reseller for support. 10 Preface

11 Comments Before calling the Hitachi Data Systems Support Center, please provide as much information about the problem as possible, including: The circumstances surrounding the error or failure. The exact content of any error message(s) displayed on the host system(s). Please send us your comments on this document: Include the document title, number, and revision, and refer to specific section(s) and paragraph(s) whenever possible. Thank you! (All comments become the property of Hitachi Data Systems Corporation.) Preface 11

12 12 Preface

13 About Hitachi Storage Adapter for ASRU The Hitachi Storage Adapter for Oracle ASM Storage Reclamation Utility (ASRU) adapter compacts the Oracle Automatic Storage Management (ASM) disks that the Oracle database is using, writes zeroes to the space freed by compaction, and then resizes the ASM disks to their original size. When the Hitachi Adapter performs this process for an HDP/HDT Pool, the storage subsystem can use the reclaimed space for thin provisioning. Product Overview Real-World Activities Architecture Functions About Hitachi Storage Adapter for ASRU 13

14 Product Overview The Hitachi Storage Adapter for Oracle ASRU complements the ASRU utility that Oracle provides. A database administrator uses the ASRU utility to identify and mark any allocated unused storage space (disk blocks) in the Oracle database tablespaces. These disk blocks are created on top of Oracle ASM as possible candidates for reclamation and use by the host system and storage subsystem. A storage or system administrator can then run the Storage Adapter for Oracle ASRU script (hizpru.py). The Hitachi storage subsystem, uses its Dynamic Provisioning feature to reclaim the marked disk blocks for possible reallocation to another application or host. The Dynamic Provisioning feature remains completely transparent to the database application and works according to the thresholds that the administrator sets on the corresponding storage pool. Real-World Activities Using the Hitachi Storage Adapter for Oracle ASRU, you identify and mark any allocated unused storage space (disk blocks) in the Oracle database and then reclaim the marked disk blocks for possible reallocation to another application or host tablespace. The following table describes Adapter features and presents some scenarios where the Adapter can be used. Adapter Feature Display Storage Reclaim ASM Scenarios Feature. This feature allows you to obtain information from the Oracle ASM subsystem to determine how many megabytes would be freed on the LUNs when storage reclamation is complete for the target ASM disk group. Procedure. 1. Configure the Adapter by editing the HiASRU.conf file, including the name of the target ASM disk group. 2. Run the hizpru.py script with "display_storage" as the only parameter. Use Scenario. Joe the Oracle administrator wants to determine whether it is worthwhile to run the ASRU utility to reclaim unused space. Joe uses the "Display Storage" feature to determine how much space would be reclaimed. Feature. In this feature, the Oracle ASRU utility is invoked to analyze the database space used and then migrate the database data to the beginning of the ASM disk. The Adapter then fills 75% of the free space available with zeros using the zerofill utility. The zero-filled space is then available to the storage subsystem to reclaim and reuse elsewhere in the pool. Note: the storage subsystem must be told to reclaim the space, as described by the Reclaim Storage feature. 14 About Hitachi Storage Adapter for ASRU

15 Adapter Feature Reclaim Storage Scenarios Procedure. 1. Configure the Adapter by editing the HiASRU.conf file, including the name of the target ASM disk group. 2. Run the hizpru.py script with "reclaim_asm" as the only parameter. Use Scenario. Joe the Oracle administrator wants to return unused space in an ASM disk group to the Hitachi storage subsystem. Joe uses the "Reclaim Storage" feature to return the space. Feature. In this mode, space reclamation directives are sent to the storage subsystem corresponding to the subject ASM disk group using Hitachi RAID Manager. When the zero page reclamation for disk blocks is complete, the space is available to the HDP pool for thin provisioning. Procedure. 1. Configure the Adapter by editing the HiASRU.conf file, including the name of the target ASM disk group. 2. Run the hizpru.py script with "reclaim_storage" as the only parameter. Use Scenario. Joe the Oracle administrator wants to return unused space in an ASM disk group to the Hitachi storage system to use for other purposes. Joe uses the Reclaim Storage feature to return the space. About Hitachi Storage Adapter for ASRU 15

16 Architecture The following diagram shows a component view of the Hitachi Adapter Storage Adapter for Oracle ASRU. Figure 1 Hitachi Storage Adapter for Oracle ASRU Architecture Diagram The Hitachi Storage Adapter for Oracle ASRU complements the ASRU that Oracle provides. A database administrator uses the ASRU utility to identify and mark any allocated unused storage space (disk blocks) in the Oracle database tablespaces. These disk blocks are created on top of Oracle ASM as possible candidates for reclamation and use by the host system/storage subsystem. 16 About Hitachi Storage Adapter for ASRU

17 Functions A storage or system administrator can then run the Storage Adapter for Oracle ASRU function (hizpru.py). The Hitachi storage subsystem, via its Dynamic Provisioning feature, then reclaims the marked disk blocks for possible reallocation to another application or host. The Dynamic Provisioning feature remains completely transparent to the database application and works according to the thresholds that the storage administrator sets on the corresponding storage pool. Invoke the Hitachi Storage Adapter for Oracle ASRU using one of the following functions. Find detailed discussions about each function in the Operations chapter. hizpry.py display_storage Function This function allows you to obtain information from the Oracle ASM subsystem to determine how many megabytes will be freed when the storage reclamation is complete on the LUNs for the target ASM diskgroup. hizpru.py reclaim_asm Function This function invokes the Storage Adapter for Oracle ASRU utility with correct arguments and parameters as determined by the user values specified in the configuration file, HiAsru.conf. hizpru.py reclaim_storage Function This function invokes the Hitachi storage reclamation function on the LDEVs; the LDEVs correspond to the Oracle ASM disks in the Oracle ASM diskgroup where storage is reclaimed. hizpru.py storage_status Function This function lets you check the status of the LUNS/disks. If a reclaim_storage function has been issued, this function displays the status of the disks, so you know whether the reclamation process is complete. It also displays the current used/free disk space for each of the LDEVs that make up the ASM diskgroup. When this Adapter is run as an Oracle OEM Adapter, each of the functions for hizpru.py can be run as separate and independent jobs that can be scheduled on a periodic basis. Maximum benefit is achieved when the reclaim_asm job is immediately followed by the reclaim_storage job. This guarantees that the disk blocks are marked and made available by the Oracle ASM; then the Hitachi storage subsystem reclaims these disk blocks to be given back to the Hitachi Dynamically Provisioned (HDP) pool. About Hitachi Storage Adapter for ASRU 17

18 18 About Hitachi Storage Adapter for ASRU

19 Requirements and Prerequisites This section identifies the prerequisites for installing the Hitachi Storage Adapter for Oracle ASM Storage Reclamation Utility (ASRU). It also gives steps for a successful installation of the Adapter. This chapter includes the following sections: Hardware Requirements Software Requirements HUVM Setup Requirements RAID Manager Setup for Non-HUS Subsystems SNM2 Command-Line Interface for HUS Subsystems Required Pre-operations Distribution Package Adapter Prerequisites Requirements and Prerequisites 19

20 Hardware Requirements Note This release supports the following software and hardware. Hardware or software that is not listed in the table(s) below is not supported in this release. For the latest information, please consult the release notes. The zero page reclamation on an ASM disk configured using LDEVs from a Hitachi storage subsystem requires the following hardware. Item Minimum Requirements Note Hitachi Storage System CPU Memory Graphics HDD Capacity Network Hitachi VSP, VSP G1000, VSP Gx00, VSP Fx00, HUS, and/or HUS-VM 1.2 GHz or faster Intel or AMD processors 4 GB or more 1024x768, 24-bit color or more 4 GB Gigabit connection recommended H-UVM virtualization software also is required to support that functionality on VSP G1000/Gx00 systems MHz recommended: 50 CPUs recommended: 4 Software Requirements Server and OS OS version Red Hat Enterprise Linux 6.5 (x86_64) Red Hat Enterprise Linux 6.6 (x86_64) Red Hat Enterprise Linux 7.2 (x86_64) Only with CLI (Command Line Interface) Oracle Linux (OEL) 6.6 (x86-64) Oracle Linux (OEL) 7.2 (x86-64) Only with CLI (Command Line Interface) Supported x x x x x Storage Model Storage Model Microcode version Supported HUS 0981/B-S X HUS VM /00 X 20 Requirements and Prerequisites

21 Oracle Software Storage Model Microcode version Supported HUVM /02 X VSP /00 X VSP G /02 X VSP Gx /00 X VSP Fx /00 X VSP Gx00 Unified /00 X Oracle Database Software Supported Oracle 11g Release 2 ( & ) X Oracle 12c Release 1 ( ) X Oracle 11g Release 2 with Real Application Cluster ( & X ) Oracle 12c Release 1 with Real Application Cluster ( ) X OEM Software OEM12c Cloud Control ( & ) Supported X Hitachi Software Hitachi Storage Navigator Modular 2 CLI SNM2LIN_2733 Hitachi Command Control Interface /04 Supported X X Host Interface Interface FC iscsi Supported X X Volume Type Volume Type Hitachi Data Provisioning (HDP) Hitachi Dynamic Tiering (HDT) Hitachi Realtime Tiering (HRT) Supported X X X Requirements and Prerequisites 21

22 Linux Packages The table below identifies the prerequisite components that must be installed when the Linux operating system is used, along with the minimum versions required. Name of component Version Operating system version Bash el6_2 RHEL 6.5,6.6 OL 6.5,6.6 e2fsprogs el6 RHEL 6.5,6.6 OL 6.5,6.6 Python el6_2.2 RHEL 6.5,6.6 OL 6.5,6.6 sg3_utils el6 RHEL 6.5,6.6 OL 6.5,6.6 Sysfsutils el6 RHEL 6.5,6.6 OL 6.5,6.6 util-linux el6 RHEL 6.5,6.6 OL 6.5,6.6 Bash el7 RHEL 7.2 OL 7.2 e2fsprogs el7 RHEL 7.2 OL 7.2 Python el7 RHEL 7.2 OL 7.2 sg3_utils 1,28-4.el7 RHEL 7.2 OL 7.2 Sysfsutils el7 RHEL 7.2 OL 7.2 util-linux el7 RHEL 7.2 OL 7.2 Agreement type (OSS, OEM, End-User) OSS OSS OSS OSS OSS OSS OSS OSS OSS OSS OSS OSS License type (GPL, MPL, BSD) GPLv2+ GPL PSF GPLv2+ & BSD GPL distributable GPLv2+ GPL PSF GPLv2+ & BSD GPL distributable Ensure that you can access mount, SCSI commands, and systools from the command prompt without providing an absolute path to them. Many of the abovementioned packages are by default installed in the Linux operating system. 22 Requirements and Prerequisites

23 HUVM Setup Requirements The Hitachi Storage Adapter for Oracle ASM Storage Reclamation Utility supports Hitachi Universal Volume Manager (HUVM), which works with the VSP G1000 and VSP Gx00 platforms. When a system consists of multiple storage subsystem, a single host must usually connect to each subsystem. As a system administrator, you typically connect each system to the hosts, a process that may require several different types of procedures. With HUVM, you configure the connection from the host to the VSP G1000 or Gx00 storage system, then manipulate mapped volumes in an external storage system in the same way as volumes in the VSP G1000 or Gx00 storage system. Operations among storage systems can also involve varied procedures. But with Universal Volume Manager, you perform them with the same Hitachi software as when you use VSP G1000 and Gx00 systems. The RSG configurations in the following table work with virtual storage on VSP G1000 and Gx00 storage devices. # Configuration Supported 1 One RSG in VDKC-Box X 2 One RSG in respective multiple VDKC-Boxes X X = Supported Supported Configurations This section outlines the HUVM configurations currently supported by the Adapter. For further information, see Hitachi Virtual Storage Platform G100/Hitachi Universal Volume Manager User s Guide, MK-92RD8024. Requirements and Prerequisites 23

24 Figure 2. One Resource Group in VDKC Box Figure 3 One Resource Group in Respective VDKC Boxes RAID Manager Setup for Non-HUS Subsystems Storage Support RAID Manager runs on a server. By configuring the server to recognize a specific Logical Unit called a command device (CMD), you can administer and obtain Logical Unit status details via the CMD. VSP subsystems also support an alternative virtual command convention that does not require a dedicated LU, but instead uses the SVP IP and a special UDP port number for RAID Manager 24 Requirements and Prerequisites

25 connectivity. The Storage Adapter for Oracle ASRU must operate with all command device-interfacing techniques. Start the instance of the RAID Manager that corresponds to the storage subsystem containing the logical units (LDEVs) that have been presented to the host for Oracle ASM. On the storage subsystem, these logical units are associated with a DP-Vol that may or may not have dynamic tiering enabled. Use a HORCM configuration file to instantiate the RAID Manager. A sample basic HORCM configuration file horcmx.conf is included in the Storage Adapter for Oracle ASRU HiASRU directory. You can modify this file and change the X to match the HORCM instance number. SNM2 Command-Line Interface for HUS Subsystems The SNM2 command-line interface (CLI) runs on a server. By installing and configuring the SNM2 CLI for the server, you can administer and obtain Logical Unit status details. Install the SNM2 CLI, which corresponds to the storage subsystem containing the logical devices (LDEVs) that have been presented to the host for Oracle ASM. On the storage subsystem, these logical units are associated with a DP- Vol that may or may not have dynamic tiering enabled. Before registering the array, set the LD_LIBRARY_PATH environmental variable as follows: LD_LIBRARY_PATH=$STONAVM_HOME/lib:$LD_LIBRARY_PATH SNM2 registers the array; you cannot override the registration information. When registering an array into SNM2, use the auunitaddauto command to specify the unit name. Specify only the IP addresses, as in $STONAVM_HOME/auunitaddauto ip <Controller_ 0_IP_Address1> < Controller _1_IP_Address2> Required Pre-operations Perform the following operations before using the Hitachi Storage Adapter for Oracle ASRU. Task Product Program Licenses Configuration The licenses of required Product Programs are LUN Manager Account Authentication (optiona1) Dynamic Provisioning (default) Requirements and Prerequisites 25

26 Task Dynamic Provisioning pool(s) and volumes Set up RAID Manager Configure ASM disks Install Dynamic Provisioning License Set up the HORCM.conf file for host access to the storage subsystem Register an array into SNM2 CLI Configuration When Dynamic Provisioning is enabled, DP Pools and DP- VOLs must be defined in the HUS SNM2 GUI, HUS VM Storage Navigator, or VSP, VSP G1000, VSP Gx00 or VSP Fx00 storage navigators. See RAID Manager documentation for installation and setup of RAID Manager /04 or higher. Follow the Oracle Grid Infrastructure Installation Guide and Oracle ASM Administrator s guide. These are available on the Oracle web site. The Oracle ASM instance must be up and running before the Hitachi Storage Adapter for Oracle ASRU can be executed on the host machine. Dynamic Provisioning license (D_PROVISIONING) must be installed and enabled in the storage systems if the HDP feature is used. This is installed and enabled on VSP subsystems by default. Install RAID Manager on the database host. Choose the instance number for the installation and create the corresponding HORCM configuration file. For example, if the RAID Manager instance is 0, create horcm0.conf file and place under /etc/ directory. The HORCM instance should be up and running before the Storage Adapter for Oracle ASRU can be executed on the machine. Install the SNM2 CLI on the host. Register an array into SNM2 CLI and use the auunitaddauto command to specify the unit name. Distribution Package The Hitachi Storage Adapter for Oracle ASRU is distributed as Hitachi- Storage-Adapter-for-Oracle-ASRU zip, which contains: The scripts for Hitachi Storage Adapter for Oracle ASRU The Hitachi ASRU OEM Adapter archive file, _Hitachi.Adapter.ASRU_2000_0.opar. Adapter Prerequisites Create LUs for the Required DP-VOLs Create LUs on the Hitachi storage system that stores the Oracle database. Map these LUs to the host for the Oracle database. Note: ASRU cannot reclaim space on LDEVs that are used as a snapshot pair. The LUs map to the ASM disk groups as described below. 26 Requirements and Prerequisites

27 Configure ASM Disks You must have ASM disk groups on the production server if an Oracle database is to reclaim unused, thin-provisioned storage that has been allocated to the ASM for the database server. To create these ASM disk groups, assign the mapped LUs as ASM disks (using /etc/init.d/oracleasm), then assign one or more ASM disks to a single ASM disk group (using asmcmd or asmca). You can create the database directly on an ASM disk group, or an already-existing database can create database table spaces on the ASM disk group. Set up the horcm<inst> Configuration File A minimal horcm<inst>.conf is required for space reclamation of the thinprovisioned HDP/HDT storage. The horcm<inst>.conf identifies the HORCM instance to which the Hitachi RAID Manager should connect in order to access and manipulate the storage LUs on which the Oracle ASM instance disk group has been configured. Here is a sample of the horcm.conf config file used in the database host server: HORCM_MON #ip_address servicepoll(10ms) timeout(10ms) HORCM_CMD #dev_name dev_name dev_name /dev/sdd In the above sample configuration file, the IP address identifies the local host where the HORCM instance is running. The device /dev/sdd is the command device for the VSP storage. RAID Manager Setup RAID MANAGER runs on a server, which you configure to recognize a specific LU dedicated as a command device (CMD). The CMD is required to pass RAID MANAGER commands from the host. Refer to RAID Manager documentation for complete information. Requirements and Prerequisites 27

28 For installations of VSP, VSP G1000, VSP Gx00 and VSP Fx00, the command device requires the settings indicated in Figure 4. Figure 4. Command Device Settings Run RAID Manager Commands Using an In-band Method RAID Manager commands must be run in band, meaning that that commands transferring from the server to the command device share the same data path as the disk I/O. The exceptions are VSP arrays that also support virtual command devices via out-of-band transfers. The Hitachi Storage Adapter for ASRU scripts operates with VSP arrays configured for either physical or virtual command device methods. Refer to the sections on command execution by inband and out-of-band methods in the RAID Manager document for your storage system model. Required Account Privileges To execute the sqlplus commands, Oracle sysdba privileges are required. To execute ASM commands, Oracle sysasm privileges are required. The RAID Manager commands must be set up to run with OS root privileges. The SNM2 CLI commands must be set up to run with OS root privileges. Set Ownership for Adapter Scripts The Storage Adapter for Oracle ASRU ships with a standard set of default file/directory ownerships and permissions that you can change to suit your environment. If, for example, you want to change ownership to oracle with the user group dba, use the following command: $ chown R oracle.dba/{hitachi ASRU script directory, for example, /home/oracle/hiasru} 28 Requirements and Prerequisites

29 Run RAID Manager Commands as Root Use the root login to run RAID Manager commands as root: 1. Create a group for RAID Manager users, for example, horcm. # groupadd horcm 2. Discover the groups to which the Adapter user belongs. # id oracle 3. Add the Adapter user to the horcm group. # usermod G horcm, <other groups> oracle 4. Switch to the HORCM binary directory for example, the following assumes that the HORCM/RAID Manager is installed in directory /HORCM. # cd /HORCM/usr/bin 5. Examine the HORCM binary directory to verify that the RAID Manager commands are located there. # ls -al 6. Change ownership for the raidcom RAID Manager command. # chown root:horcm raidcom 7. Set permissions for the RAID Manager command. 8. Provide read and execute permissions to the user and the group # chmod ug+rx,o-wx raidcom 9. Provide setuid permission to the user. # chmod u+s raidcom Run SNM2 CLI Commands as Root Use the root login to run SNM2 CLI commands as root: 1. Create a group for SNM2 CLI users, for example, snm2cli. # groupadd snm2cli 2. Add the Adapter user to the snm2cli group. # usermod a -G snm2cli oracle Requirements and Prerequisites 29

30 3. Switch to the SNM2 CLI binary directory (the following assumes that the SNM2 CLI is installed in the directory /usr/stonavm and is installed with Oracle user). # cd /usr/stonavm # ls al 4. Change ownership for the required SNM2 CLI commands. # chown root:snm2cli audp* auunit* 5. Set permissions for the required SNM2 CLI commands. Provide read and execute permissions to a user and the group. # chmod ug+rx,o-wx audp* auunit* 6. Provide setuid permission. # chmod u+s audp* auunit* Run sg3 Utilities as Root (Linux only) Use the following method to run sg3 utilities as root for Linux. When running in the Linux operating system, the Hitachi ASRU Adapter must execute the sg3 utility /usr/bin/sg_inq (rpm - sg3_utils) to get storage vendor details from the LDEVs presented by the storage subsystem to the Linux host. This requires that the binary /usr/bin/sg_inq has groups execute permissions for you. This can be confirmed by using the commands above. Additionally, you must have group read permissions to the LDEVs. Verify the group ownership of the block devices in /dev/ directory for the LDEVs that are part of the ASM disk group in question. By default, these block devices have group disk as group owner. Use the usermod command to add this group membership for the current user. For example, to add the group disk to the list of groups for the current user, run this command: # usermod G disk oracle For these changes to take effect, log out and then log in again to the host machine. 30 Requirements and Prerequisites

31 Requirements and Prerequisites 31

32 Installation, Deployment, and Uninstallation This chapter explains the job-setup processes required for creating and running the Oracle Storage Adapter for ASRU using the OEM grid control console. Summary of Steps Import an Adapter Synchronize EM CLI Client with an OMS Uninstall an Adapter Installation, Deployment, and Uninstallation 32

33 Within the Hitachi Storage Adapter for Oracle ASRU folder is an OEM Adapter archive file called _Hitachi.Adapter.ASRU_2000_0.opar. This archive file contains one OEM Adapter. This Adapter contains the metadata information for deployment and un-deployment of the Adapter scripts as an Oracle OEM Adapter. This file also contains metadata information for running the Adapter scripts as OEM jobs. Each of the four functions of the storage Adapter are run as separate and independent jobs created from a single job type. An OEM user has the opportunity to create and schedule jobs built from this job type. Summary of Steps Using the Hitachi Storage Adapter for Oracle ASRU OEM Adapter involves the following main steps: 1. Import an Adapter 2. Deploy an Adapter 3. Create a target instance 4. Set preferred credentials 5. Create and schedule jobs Import an Adapter The following activities are required before importing the Adapter. Set up the Software Library This is a one-time setup procedure on the host. 1. Create a folder in the system where Enterprise Manager is installed. For example, /home/oracle/swlib1. 2. From the console, select Enterprise, then Provisioning and Patching, and then Software Library. 3. Click Actions, then Administration. 4. Click Add. 5. In the pop-up window, enter a name and location for the folder you set up in Step 1, for example, swlib1 and /home/oracle/swlib1. 6. Wait for the processing to finish. Installation, Deployment, and Uninstallation 33

34 Synchronize EM CLI Client with an OMS This procedure synchronizes an EM CLI client with an Oracle Management Service (OMS). (Note: this procedure is not supported on RHEL/OEL 7.2) Run emcli sync: <OMS-HOME>/bin/emcli sync If the synchronization fails due to a Session Expired error, log in again using: <OMS HOME>/bin/emcli login -username=sysman 7. Now run emcli sync again: <OMS-HOME>/bin/emcli sync After synchronization, all verbs and associated command-line help available to this OMS become available at the EM CLI client. Import the Adapter Archive 1. Import the Adapter into Enterprise Manager Cloud Control by using the emcli import_update command as follows: <OMS HOME>/bin/emcli import_update -file= path to the opar file omslocal 2. The -omslocal flag indicates that the Adapter archive is on the same system where you are running this command, and the path exists on this system. For example: <OMS HOME>/bin/emcli import_update - file="/home/oracle/sanjay/hiasru/ _hitachi.adapter.asru _2000_0.opar" omslocal After a successful import, the message should look like the following. 34 Installation, Deployment, and Uninstallation

35 3. When you go to Setup -> Extensibility -> Self Update, OEM shows a list of Adapters, as in the following figure. Deploy an Adapter 1. Select the ASRU Adapter and click the Deploy On button. 2. Choose the Management Servers option to deploy it on OMS. Installation, Deployment, and Uninstallation 35

36 3. Click Continue to proceed with the deployment on the management server. 4. Click Deploy to deploy the Adapter on the server. 36 Installation, Deployment, and Uninstallation

37 5. When the Adapter is deployed on the Management Server, add the Adapter to the desired agents by selecting Management Agent from the Deploy On drop-down list. 6. Click on Add to select the management agents to deploy the Adapter. 7. Click Select and continue to complete the deployment. 8. Click Show Status to monitor the deployment. 9. Similarly deploy all the agents. Installation, Deployment, and Uninstallation 37

38 Create Target Instances 1. From the Setup tab, choose Add Target and then choose Add Targets Manually as shown below. 2. Choose Add Non-Host Targets by Specifying Target Monitoring Properties and select the appropriate target type, Hitachi Storage Adapter for Oracle ASRU. 3. Select the Agent. 4. Click on Add Manually. 38 Installation, Deployment, and Uninstallation

39 5. Fill in the target name and the property values. The Hitachi ASRU Adapter location is the location of the root folder for the Hitachi Storage Adapter for Oracle ASRU installed on the agent host. 6. Provide information for the following fields and click OK: o Target Name o Hitachi ASRU Adapter Location 7. Confirm that the target was added to the list of monitored targets. Installation, Deployment, and Uninstallation 39

40 Create Jobs 1. From the OEM home page, go to Enterprise -> Jobs -> Library. Specify a Name 2. Create the Library Job by filling in the details as in the following example, which shows how to create a job to display_storage. Choose Parameters Possible parameters are: display_storage, reclaim_asm, reclaim_storage or storage_status. The parameters correspond to each of the possible modes for running the ASRU Adapter. 40 Installation, Deployment, and Uninstallation

41 Set Credentials Supply the credential values in this window. The system requires credentials for the host oracle user with DBA privileges on the host where the agent is. It also needs the credentials for the storage subsystem where the ASM disks are located; these allow update access to the storage subsystem in order to issue RAID Manager/raidcom directives on that subsystem. Set the Schedule Set the schedule for this job. It could be one time or repeating on a regular basis. In the current example screen shot, One-Time has been chosen. 1. Click Save to Library to create the job. 2. Confirm that the job was created successfully. Installation, Deployment, and Uninstallation 41

42 ` Submit the Job 3. Repeat these steps to create jobs for different modes of Adapter scripts. 1. Select the job from the library and set parameters such as Credentials, Scheduling details, Access, and Parameters from the appropriate tabs as shown in previous section. Then click Submit. 2. Click on the job ASRU_DISPLAY_STORAGE.1 to get details on the job output, as shown below. Similarly, jobs can be created and run for all the modes for the Storage Adapter for Oracle ASRU Adapter. 42 Installation, Deployment, and Uninstallation

43 Uninstall an Adapter Delete a Job Use the following steps to remove the Hitachi Storage Adapter for Oracle ASRU OEM Adapter from OEM grid control, and then delete the directory where the Hitachi Storage Adapter for Oracle ASRU was unzipped. 1. Launch a web browser and log in to Oracle Enterprise Manager grid control. 2. From the Home page, select Enterprise->Job->Library. 3. Select the Job and click Delete. Installation, Deployment, and Uninstallation 43

44 4. Confirm the deletion by clicking Yes. 5. Confirm the successful deletion message. Delete a Target Instance 1. Launch a web browser and login to Oracle Enterprise Manager grid control. 2. From the Home page, select Targets->All Targets from the tabs in the top level. 3. From the displayed windows, choose the appropriate target for deletion. 44 Installation, Deployment, and Uninstallation

45 4. On the Overview of the target page, click on the target name in the left top corner to get the drop down list. Select Target Setup -> Remove Target. 5. Confirm the deletion by clicking Yes. 6. Confirm the deletion in the information window that appears. Undeploy an Adapter 1. From the Home page, select Setup->Extensibility->Plug-ins in the top right corner. 2. On the Overview of the Plug-ins page, select the appropriate plug-in and click on Undeploy From. First Undeploy from the Agents and then from the Management Server. Installation, Deployment, and Uninstallation 45

46 3. Choose the agent from which to undeploy, and click the Undeploy button. 4. Continue and click Undeploy. Follow the same steps to undeploy from the Management Server. Delete an Adapter 1. Select Setup->Extensibility->Self Update. 46 Installation, Deployment, and Uninstallation

47 2. Choose the Adapter name and select Delete from the Actions tab. 3. Confirm the removal by clicking Delete. 4. Verify that the Adapter has been removed. Installation, Deployment, and Uninstallation 47

48 Operations This chapter provides details about the operation of the Hitachi Storage Adapter for Oracle ASRU. It discusses the high-level flow of control across the various sub-components of the Adapter and the different files that are included in the installation directory when the zip file is extracted. Included are the following section: Adapter Workflow Adapter Scripts Read and Invoke a Configuration File Display Reclaimable Storage (display_storage) Run Oracle ASM and Mark Zero Pages (reclaim_asm) Reclaim Zero Pages on the LUNS Subsystem (reclaim_storage) Display Current Storage Status (storage_status) Display Help Operations 48

49 Adapter Workflow The following illustration shows the Hitachi Storage Adapter for Oracle ASRU control workflow. Figure 5 Hitachi Storage Adapter for Oracle ASRU High-Level Design Operations 49

50 Adapter Scripts The top-level Adapter scripts are described in the following table. # Script Description 1 HiAsru.conf Storage Adapter for Storage Adapter for Oracle ASRU model configuration file. Modify copies of this file to match the database server environment. This file contains the required configuration parameters for a successful run of the Storage Adapter for Oracle ASRU and Hitachi zero page reclamation on the storage subsystem. 2 hizpru.py This is the main top-level function script for running the zero page reclamation. It performs these operations: Builds and manipulates data structures, functional logic Calls into validate routines for validation and logging subsystem for all logging Requests information from Oracle ASM and directs Storage Adapter for Oracle ASRU for ASM space reclamation Calls into the storage subsystem (Hitachi RAID Manager/RAID Manager) for status and updates to the LUs 3 hilogging.sh This script contains the logic for the logging subsystem as explained in detail in Logging. 4 hivalidate.py The script contains the logic for the validation subsystem. It provides functions to do all the validations that are needed. Does the following (and more): Validate config file Validate disk group, block devices Validate access rights 5 ASRU This is the wrapper shell script for calling ASRU. This script is provided by Oracle. It does initial checks on the command line arguments and environment. 6 ASRU.pl This Perl script calls into the Oracle ASM instance to execute the Oracle Automatic Storage Reclamation. It provides and updates Oracle ASM: Disks in disk group Total and used capacity This script helps you manage and manipulate ASM disks content and sizes. 7 Zerofill This is a script used by Oracle to blank out all the disk blocks that are then made available to the storage subsystem HDP Pool for zero page reclamation. 8 Message.txt Text file containing the list of messages that are displayed by the Storage Adapter for Oracle ASRU adapter. 9 Version.txt Text File containing the version information for the Adapter. 11 Hilicense Hitachi License Management utility for adding, listing, and deleting Adapter/plug-in license. 50 Operations

51 # Script Description _Hit achi.adapter.a SRU_2000_0.o par The Oracle Adapter Archive file for installing, deployment of the Adapter, and the creation of jobs. Read and Invoke a Configuration File The Hitachi Storage Adapter for Oracle ASRU requires a configuration file that the hizpru.py utility reads at the invocation time. This configuration file HiAsru.conf is placed in the same directory where the primary hizpru.py utility has been installed. This file contains all the parametric values that are needed for a successful run of the Storage Adapter for Oracle ASRU and Hitachi Zero Page Reclamation. The details of the HiAsru.conf file are as follows. HORCM_PATH HORCMINST Storage Adapter for Oracle ASRU Environment Variables HUS_AUTHENTICATION_ENABLED LOG_ARCNUM LOG_LEVEL LOG_SIZE ORACLE_ASM_DISKGROUP ORACLE_ASM_HOME ORACLE_ASM_SID STONAVM_HOME Installation directory for Hitachi CCI (RAID Manager). The HORCM instance that is configured to manage the storage subsystem on which the Oracle ASM diskgroup is configured. Account Authentication Status for HUS Subsystems - Yes, No Number of generations for log file archive. Level of messages to include in the Hitachi ASRU Adapter log file. The choices are: MINIMAL: Only error or fatal messages NORMAL: Warning, error or fatal messages VERBOSE: All messages, including debug messages. Defines the maximum size of the log file HiASRU.log before switching to the next file. ASM disk group for Space Reclamation. This parameter must be a single group name. Absolute path of the Oracle Grid Infrastructure home. System identifier of the ASM instance. Installation directory for Hitachi SNM2 CLI. Invoke the Hitachi Storage Adapter for Oracle ASRU using one of the following functions. hizpry.py display_storage hizpru.py reclaim_asm Operations 51

52 hizpru.py reclaim_storage hizpru.py storage_status Display Reclaimable Storage (display_storage) When the Adapter is invoked using the function display_storage, it runs in display mode. In this mode, the Adapter communicates with the Oracle ASM instance to find out the total space allocated to the logical disks in the ASM disk group. From the ASM instance, it also determines the actual space that is currently being used by the database on the ASM disks that are part of this ASM disk group. It then calculates the space that can be marked for reclamation to the HDP pool for thin provisioning. Finally, it presents you with this information. No space reclamation is actually done in this mode. The following example shows the Adapter running in this mode from the command line. Figure 6 Displaying storage to be reclaimed Run Oracle ASM and Mark Zero Pages (reclaim_asm) When invoked with function reclaim_asm, the Adapter runs in the ASM mode, where it simply runs the Storage Adapter for Oracle ASRU and exits. In this mode, the Oracle ASM instance analyzes the actual space used by the database and then migrates the database data to the beginning of the ASM disk. Storage Adapter for Oracle ASRU then allocates an additional buffer of 25 percent of currently used space. Via the zerofill utility, the rest of the ASM disk then becomes available to the storage subsystem for thin provisioning. The zerofill utility essentially zeroes out all the logical disk blocks that are not currently used. Because this process goes to all the disk blocks that can be marked for disk reclamation and updates them, this step can take a long time to finish, depending upon the size of the ASM disk group. 52 Operations

53 The following screens show an example of running the command via the command line. Figure 7 Beginning the reclamation process After the Adapter finishes running in this mode and updating the disk blocks, output similar to the following appears. Figure 8 Completing the process Operations 53

54 Reclaim Zero Pages on the LUNS Subsystem (reclaim_storage) Figure 9 Reclaiming space When the Adapter is invoked using the function reclaim_storage, the actual space reclamation directives reach the storage subsystem corresponding to the subject ASM disk group (via the Hitachi RAID Manager). When the zero page reclamation for disk blocks is complete, the space is available to the HDP pool for thin provisioning. The following screens show an example of running the Adapter on the command line. As shown in Figure 9, the Adapter function asks for the username and password for the storage administrator who has access privileges to the storage subsystem where the LDEVs associated with the ASM diskgroup are located. Once the username and password is provided, the Adapter function continues to display output similar to Figure 10. Figure 10 Storage status The STATUS column represents the current state of each of the LUNs. A status of NML means that the Storage LUN is working normally. The OPERATION 54 Operations

55 column displays any current operations on that LUN. An operation of NONE means that there is no current space reclamation running on the corresponding LUN. If the operation is in status ZPD, there is a zero-page reclamation currently running on this LUN. Once the reclamation function is issued, the operation may still show as NONE during the short time the actual zero-page reclamation is starting, If the Adapter function hizpru.py storage_status runs right after the reclaim_storage function, the operation status is ZPD toshow that the zeropage reclamation has been started on the LUNs. This is shown in the next section for storage_status. For a complete set of outputs shown for STATUS and OPERATION and their significance, please refer to the Hitachi Command Control Interface Command Reference. Display Current Storage Status (storage_status) The function storage_status displays the current status of the storage subsystem. When the Adapter is invoked using the function storage_status, it runs a status check on the LUs corresponding to the disks that are part of the subject ASM disk group. It reports the reclamation status on these LUs in an output similar to the reclaim_storage function, clearly showing the current used/free storage space. The following screens show an example of running the Adapter on the command line. Figure 11 Current status If the LUs are still being zero reclaimed from a previous reclaim_storage operation, the status column shows ZPD for the corresponding LUN id. Operations 55

56 Display Help When the Adapter is invoked using any function other than the ones listed above, the Adapter displays detailed help/usage information as shown below. Figure 12 Help information 56 Operations

57 Operations 57

58 Logging and Message Codes This chapter contains the following sections: Logging Message Codes Logging and Message Codes 58

59 Logging The Hitachi Storage Adapter for Oracle ASRU log files are located in the log subdirectory directly below the main Hitachi Storage Adapter for Oracle ASRU directory HiASRU. The LOG_ARCNUM configuration parameter in the configuration file HiAsru.conf dictates the generation of log files to keep. For example, if this value is 2, then the system will keep at most last 2 log files along with the current file. Therefore, in this case the log directory will contain HiASRU.log, HiASRU0.log, and HiASRU1.log. The configuration parameter LOG_SIZE determines the maximum allowed size (in bytes) of the log file HiASRU.log before switching to the next log file. The configuration parameter LOG_LEVEL determines the kind of messages that are recorded in the HiASRU log file. When the LOG_LEVEL is MINIMAL, only error and fatal messages are recorded in the log file. When the LOG_LEVEL is NORMAL, warning, error, and fatal messages are recorded. When the LOG_LEVEL is VERBOSE, all debug and informational messages are also recorded in the log file. Debug messages help in identifying the functional area and line number of the file that is being currently executed. Error messages are always recorded in the log file irrespective of the LOG_LEVEL. Below is sample output from the HiASRU.log file when the log level is VERBOSE. Logging and Message Codes 59

60 Figure 13 Output Log The output of the Storage Adapter for Storage Adapter for Oracle ASRU scripts is logged in the job activity page in the OEM cloud control. When the Storage Adapter for Storage Adapter for Oracle ASRU OEM Adapter component fails to load, import or deploy, refer to the respective OEM, WebLogic, and Oracle Database log files. A detailed description of these files is beyond the scope of this document. 60 Logging and Message Codes