Release Information FlexFrame Orchestrator V1.1A. Copyright 2015 Fujitsu

Transcription

1 Copyright 2015 Fujitsu Table of Contents General... 2 Ordering... 2 Delivery... 2 Documentation... 2 SW/HW Extensions / New Functionality... 3 Technical Information... 4 Resource Requirements... 4 Hardware Requirements / Support... 4 Installation, Configuration, Update... 4 Update an entire FlexFrame Landscape... 4 Product Use... 4 Obsolete Functions... 4 Incompatibilities... 4 Hints and Restrictions... 5 General... 5 Routes and default Gateway... 5 Application Nodes... 5 PRIMERGY BX2560M1 and BX2580M PRIMERGY RX2540M PRIMEQUEST PQ2800B and PQ2800B PRIMERGY RX4770M PRIMERGY RX4770M PRIMERGY BX900 switch blade hints... 6 Other PRIMERGY server hints... 6 VMware... 7 KVM-Hypervisor... 7 Management Tool... 8 Administration Commands... 8 FlexFrame Agents... 9 Upgrade... 9 Linux Operating System Control Nodes Networking Network Attached Storage SAN Support for database (sapdata and saplog volume) SAP Procedure in Case of Errors Edition: July 2015 The FlexFrame Orchestrator solution is a revolutionary approach to run complex SAP solutions with higher reliability and efficiency. At the same time some major changes to the configuration paradigms for infrastructures have been implemented. These changes are: A shared operating system booted over IP networks for the SAP servers. Decoupling of application software and operating system, called virtualization of SAP software. Shared Network Attached Storage from Network Appliance. Copyright 2015 Fujitsu 1

2 FlexFrame Agents providing a revolutionary mechanisms to implement high-availability functions without cluster software. General Ordering Delivery This Release Information is a summary of the major extensions, requirements, restrictions and operating information concerning FlexFrame Orchestrator V1.1A. Additionally the user is required to read the appropriate manuals, which describe the topics in detail. The use of names or product designations, etc., in this information does not constitute a general authorization to use such names or designations. In many cases they are legally or contractually protected names or designations even if they are not marked as such. FlexFrame Orchestrator is subject to the general terms and conditions of the software product use and service agreement. The FlexFrame Software consists of several DVDs: Table 1. Media DVD Name Version Description Delivery Control Node V1.1A Control Node Software Image and KVM Hypervisor Software Image Application Linux Documentation Node V1.1A Application Node Software Image Linux SLES4SAP 11SP3 x86_64 for PRIMERGY Server Service-DVD V1.1A FlexFrame Agents Suite, Management Tool, ESXi-Installer and Documentation Source-DVD V1.1A Sources of OS, HAE, SDK and PTCH and other used OSS component. The Service-DVD contains the following documentation titles: Manuals: Management Tool Installation of a FlexFrame Environment Administration and Operation Security Guide Network Design and Configuration Guide Installation Guide for SAP Solutions Upgrading FlexFrame Orchestrator 1.0A to 1.1A FlexFrame Agents - Installation and Administration With FlexFrame Orchestrator Software Kit With FlexFrame Orchestrator Software Kit With FlexFrame Orchestrator Software Kit DVD can be requested by to Expert.SAP@ts.fujitsu.com Copyright 2015 Fujitsu 2

3 FlexFrame LogAgent - Concept and Usage FlexFrame Messenger - Concept and Usage Installation and Configuration of LVM 2.1 Standard Edition Other technical documents: Technical White Paper Hardware Characteristics Quick Guides SW/HW Extensions / New Functionality FlexFrame Orchestrator V1.1A Service-DVD 07/2015 supports the following additional features: Virtual Control Center See further hints. [11] Support of VMware vsphere 5 with ESXi 5.5 Update 2 Support of SAP HANA V1.0 SPS09 New hardware: PRIMERGY RX4770M2 New hardware: PRIMEQUEST PQ2800B2 FlexFrame Orchestrator V1.1A Service-DVD 04/2015 supports the following additional features: Support NetApp cluster-mode (cdot) ONTAP 8.3 See cdot restrictions. [4] New hardware: PRIMERGY BX2560M1, BX2580M1 as Application Node and KVM Hypervisor New hardware: BX900 switch blade SBAX3 (as 10GbE-18/8-F) Brocade switch VDX6740 with 24-port license FlexFrame Orchestrator V1.1A supports the following additional features (compared to V1.0A). Support of NetApp cluster-mode (cdot) SAN support for database (sapdata and saplog volume) Support of Brocade switch ICX6430/6450 and VDX Support of Jumbo Frames Support of SAP HANA V1.0 SPS08 Support of SAP HANA System Replication Support VMware vsphere 5 with ESXi 5.5 Update 1 Support Sybase IQ V16 as near-line storage Support Sybase ASE V16 New hardware: PRIMERGY RX2540M1 as Control Node New hardware: PRIMERGY RX2540M1 as Application Node New filer models: FAS2552/FAS2554 and FAS8080EX Copyright 2015 Fujitsu 3

4 The new SLES for SAP Applications 11SP3 image includes the latest patch cluster available at the time of building. Please see the SAP related SLES hints. [14] Technical Information Resource Requirements The resource requirements are determined within the sizing process for SAP infrastructures from Fujitsu. Sizing requests can be sent to SAP requires that only servers listed in the Product Availability Matrix for SAP HANA are supported, e.g. RX300S8, RX2540M1, RX600S6, RX900S2, PQ2800B and RX4770M. Hardware Requirements / Support In the FlexFrame Orchestrator Support Matrix, available on the Fujitsu Service Partner Portal, you can find the latest information on supported models and the required firmware versions. General information about the configuration is available in the Configuration Guide on the Infrastructures Solutions Partner Portal. Installation, Configuration, Update The system is delivered preinstalled after having been assembled in a Staging Center taking the requirements of the Management Tool into consideration. The subsequent installation of SAP services is described in the Installation Guide for SAP Solutions Update an entire FlexFrame Landscape Product Use In contrast to earlier releases the Upgrade to FlexFrame Orchestrator V1.1A is possible from one base release only: FlexFrame Orchestrator V1.0A. You cannot upgrade from any FlexFrame (not even FlexFrame V5.3A)! Remember: You can extend a FlexFrame V5.3A to FlexFrame Orchestrator V1.0A by purchasing and installing the appropriate licenses. Any special release request ("Sonderfreigabe"), granted for an earlier version, will be voided with the update to FlexFrame Orchestrator V1.1A and must be reissued. The application of the product is described in the Document Administration & Operation (Service-DVD). Obsolete Functions Incompatibilities The option --op rename of the command ff_an_adm.pl is no longer supported. Use the option --op move of the command ff_an_adm.pl instead. The command ff_export.pl is no longer supported. Use the ff_nas_adm.pl --op export command instead. The command ff_esx_adm.pl is no longer supported. Use the command ff_hn_adm.pl instead. The command ff_esx_vm.pl is no longer supported. Use the command ff_vm.pl instead. ONTAP cdot: The use of ONTAP cluster-mode versions < 8.3 is not supported because the addition of pools or application nodes will fail due to incompatibilities between 8.2 and 8.3. If required to use ONTAP version cdot 8.2 the former ONTAP cdot 8.2 adapter module is available upon special release request only. Copyright 2015 Fujitsu 4

5 Cancelled CN HW: RX300S4 Cancelled AN HW: RX300S4, BX620S4, BX630S2 Dual/quad Definitively no new Application Node Image can be setup on these old server types. Cancelled Netapp Filer series: FAS3020, FAS3040, FAS3070, FAS6030 and FAS6070 Cancelled EMC Celerra models: all EMC Celerra models are cancelled. Hints and Restrictions General FlexFrame Orchestrator is a highly integrated product. It requires some knowledge about the dependencies of the components involved. Only people who have passed the FlexFrame training should plan, install, administer and/or maintain a FlexFrame landscape. Only certified FlexFrame specialists are allowed to install patch packages. The project and customer specific modifications of the system have to be checked and considered before a patch is installed. The upgrade of the software is subject to an additional charge and may only be installed by FlexFrame for SAP trained service engineers authorized by Fujitsu. We assume the people involved have read the full documentation which is shipped with the FlexFrame landscape. Most of the support issues are the result of not reading the manuals carefully enough, including the available online information. Routes and default Gateway Application Nodes The default gateway for Application Nodes will not be configured by using DHCP anymore. Instead, it is set in the /etc/sysconfig/network/routes file which is a symbolic link to /pool_img/etc/routes. Customers which have defined custom entries in /etc/sysconfig/network/routes will have to manually add those to the corresponding file in /FlexFrame/volFF/os/Linux/pool_img/pool-*/etc/routes. Entries which should only be available for a single Application Node can be configured by creating a new configuration file /etc/sysconfig/network/ifroute- <interfacename> on the Application Node. The syntax for this file is the same as the original /etc/sysconfig/network/ routes file. PRIMERGY BX2560M1 and BX2580M1 Blades BX2560M1 and BX2580M1 are supported by AN image FSC_1.1A SLES-11.x86_64 or newer. The former AN image FSC_1.1A SLES-11.x86_64 does not include the proper version of the Emulex be2net driver. Usage as pool-independent spare node: In case of failover the image version of the failed Application Node will be assigned to the BX25xxM1 spare. Therefore all Application Nodes in all pools have to use AN image FSC_1.1A SLES-11.x86_64 or newer. PRIMERGY RX2540M1 Disable BIOS USB Configuration / xhci Mode see HW_Characteristics RX2540M1_CN_Quickguide and HW_Characteristics RX2540M1_Quickguide. PRIMEQUEST PQ2800B and PQ2800B2 Definition PQ2800B is used for PRIMEQUEST 2800B Type1 while PQ2800B2 is used for PRIMEQUEST 2800B Type2. Copyright 2015 Fujitsu 5

6 The PXE-boot is supported via "Eth Ctrl 2x10Gbit PCIe x8 D2755". A PXE-boot via 1GbE connection is not supported within FlexFrame Orchestrator. The boot takes about minutes (depending on the number of DIMM-pairs). Therefore the usage as poolindependent spare node is not recommended. Use a pool-group specific spare node. Model PQ2800B2 currently requires a special network device order within the EFI Boot Manager. (Used network device not on top of the boot device list.) See HW_Characteristics_PQ2800B2_Quickguide. PRIMERGY RX4770M1 This server needs a dual 10GbE LAN connection. A 1GbE Data NIC connection is not supported within FlexFrame Orchestrator. The PXE-boot is supported via "Subm.LAN 2x10Gb optic" (S26361-F5295-E500) only. PRIMERGY RX4770M2 This server needs a dual 10GbE LAN connection. A 1GbE Data NIC connection is not supported within FlexFrame Orchestrator. The PXE-boot is supported via "Eth Ctrl 2x10Gbit PCIe x8 D2755" only. The "Subm.LAN 2x10Gb optic" is not supported within FlexFrame Orchestrator 1.1. SAP hardware check tool HanaHwCheck.py reports: "System Product Name PRIMERGY RX4770 M2 not supported". This message can be ignored. The model has been certified by SAP in the meanwhile and is fully supported by SAP. PRIMERGY BX900 switch blade hints FlexFrame Orchestrator currently supports the following BX900 switch blade models: Code-Name FFO switch-id HW_Characteristics (PDF) SB11 1GbE-36/8+2-Q HW_Characteristics_SWB_1GbE_36_8_2_Q_Quickguide SBAX2 10GbE-18/8-F HW_Characteristics_SWB_10GbE_18_8_F_Quickguide SBAX3 10GbE-18/8-F HW_Characteristics_SWB_10GbE_18_8_F_Quickguide With switch blade SBAX3 we currently use the SBAX2 functionality (no extended SBAX3 functionality) and therefore the switch-id 10GbE-18/8-F must be used for configuration. The use of the switch blade 10GbE-18/8-F (SBAX2) in fabric 1 within a BX900 chassis prohibits the operation of BX920 S1/S2 blades. The use of the switch blade 10GbE-18/8-F (SBAX3) within a BX900 chassis prohibits the operation of BX920/ BX924 S1/S2 blades in general. In a blade chassis equipped with 10GbE-18/8-F switch blades 10GbE Blade Servers can be run. When adding these blades they should be added as 10GbE Servers. It will not be checked or prevented if they are added as 1GbE blades. The blades BX920S3, BX924S3, BX920S4, BX924S4, BX2560M1 and BX2580M1 can also be used in a blade chassis equipped with 1GbE-36/8+2-Q (SB11) switch blades. The network interfaces will be throttled to 1Gbit. Other PRIMERGY server hints For the latest information about the supported PRIMERGY server models and their respectively supported infrastructure consult the FlexFrame Orchestrator Support Matrix. BX924S3, BX924S4, BX920S3 and BX920S4 related hints: When using AN image FSC_1.1A SLES-11.x86_64 you have to update the Emulex CNA controller firmware (e.g or newer) due to the new Emulex driver version within the new AN image. Copyright 2015 Fujitsu 6

7 Please proceed in the following order: - assign and boot the new AN image with proper Emulex driver - Download the Emulex CNA Controller Firmware (OCx_MC-CNA_V*.scexe) - flash the Emulex CNA firmware (see details below) In the FlexFrame Orchestrator environment it is supported to run the first boot with old CNA Firmware, but it is mandatory to flash the new CNA Firmware afterwards. Do not forget to reboot. If you do not follow this guideline the PRIMERGY BX-server is unsupported. To upgrade the Emulex CNA firmware of the onboard LAN controllers of the server models BX920S3, BX924S3, BX920S4, BX924S4, BX2560M1 and BX2580M1 execute the following steps: mkdir /tmp/firmware mount -o bind /tmp/firmware /lib/firmware hbacmd download <MAC> $PWD/<FW-File> e.g. /root/newfw/oc ufi [ the download takes some time ] umount /lib/firmware [ Reboot of the Application Node ] [ other supported commands ] hbacmd listhbas hbacmd hbaattributes <MAC> Another way is the offline flash by using the PRIMERGY Update-DVD. No additional PCI or USB devices may be attached to any Application Nodes. Extensions are only allowed in accordance to the Configuration Guide FFO V1.1A available through the Fujitsu Infrastructures Solutions Partner Portal. More network interfaces in addition to the 2 redundantly configured Gigabit interfaces (1 or 10 GbE) are not supported on the Application Node. The use of CIFS shares on the Application Nodes was not tested. FlexFrame is an NFS based solution. VMware KVM-Hypervisor To be configurable by FlexFrame scripts, PRIMERGY servers with vsphere ESXi must be licensed with vsphere or be in evaluation mode. The ESXi Single Server license (free ESXi), that comes with the embedded ESXi variant on PRIMERGY servers is not sufficient and must be replaced or removed before the FlexFrame specific ESXi configuration can be completed. New ff_ponpoff.pl --op power-on power-off support for Hypervisor Nodes. Prerequisite is a proper setup of Authentication Settings and Individual Node definition by FA_ConfigGui/configgui.sh. KVM-Hypervisor installation on BX2560M1 and BX2580M1: You have to install at least the new be2net driver from the Service-DVD before starting the installation otherwise the NIC selection screen looks like: [ ] eth0 (NIC does not exist) [ ] eth1 (NIC does not exist) Copyright 2015 Fujitsu 7

8 During the installation process one of the 'setup-screens' will ask for the kind of node (CN, KVM or ESX) you want to install. At this point please select <Cancel> to leave the installation. Then press 's' to get a shell command interface. Now you can mount the Service DVD and install the drivers. For more details see ChangedModules.txt. This driver update is also mandatory for BX920S3, BX924S3, BX920S4, BX924S4 and is recommended for other PRIMERGY models. After installation of the KVM Hypervisor Node (if applicable) the node does not contain the ServerView Agents and SV-RAID. Install and start the two ServerView products by running the following commands on the KVM Hypervisor Node: [ Login as user root into the KVM Hypervisor Node ] [ Insert the Service DVD ] # mount /dev/sr0 /mnt # cp /mnt/hynos/kvm/serverview_agents/fts_serverviewagents* /tmp # cd /tmp # unzip FTS_ServerViewAgents* # bash srvmagt.sh install # # cp /mnt/hynos/kvm/serverview_raid/fts_serverviewraidmanager* /tmp # unzip FTS_ServerViewRAIDManager* [ Note: you can ignore the '* appears to use backslashes as path separators' warning.] # rpm -U ServerView_RAID-*.rpm # umount /mnt [Repeat the steps above on the other KVM Nodes.] Management Tool Processor models with multiple threads per core: Logical CPUs listed by ff_hn_adm.pl -op list and assigned by ff_an_adm.pl --vcpus <no> are CPU threads. New ff_ponpoff.pl --op <power-on power-off> support for Hypervisor Nodes. Prerequisite is a proper setup of Authentication Settings and Individual Node definition by FA_ConfigGui/configgui.sh. The network segments of the various pools (Client LAN, Server LAN and Storage LAN) as well as the network segment of the Control LAN must have their own IP ranges. None of these network segments may be reused for a different purpose for instance a different pool. The host names of the virtual IP addresses of the SAP services are built by a predefined and fixed rule. They cannot be changed. However, it is possible to create aliases to each service after the installation has been completed. Note: The original predefined names still remain intact. The Management Tool can be used for the administration of a FlexFrame system. Please check the documentation for which tasks it can be used. Non ASCII characters must not be used in the following textboxes: Customer, Project Name, Person, Comments Filer in cluster-mode (cdot): When adding newly created NAS volumes in Administration Mode: showmount - e <mysvm>-co does not provide NFS export information. Therefore a manual input in the following format is required NAS <mysvm> volume e.g. Sapdata: NAS svm1 CL1data Saplog: NAS svm1 CL1log Administration Commands Please notice that pool names with a dash ("-") are not supported. Copyright 2015 Fujitsu 8

9 FlexFrame Agents DB2 reaction script: When using pools created with FFO 1.1A00-19 please change the Service_ReactionScript entry in the config file /opt/myamc/vff/vff_<pool>/config/myamc_fa.xml ### from old myamc_fa.xml settings: <configsection name="srv_db2">... <configentry name="service_reactionscript"> < SAP-script used from FF V5.0 (ff_service.sh) --> <value type="string">${scriptpath}/ff_service.sh -s ${SIDENT} -t ${monparam} ${BYKEYNIDENT} -a ${ACTION} <value> </configentry> ### to new myamc_fa.xml settings (repeat this change for all pools): <configsection name="srv_db2">... <configentry name="service_reactionscript"> < SAP-script used from FF V5.0 (ff_service.sh) --> <value type="string">${scriptpath}/ff_service.sh -s ${SIDENT} -t db ${BYKEYNIDENT} -a ${ACTION} <value> </configentry> To avoid sporadic event shutdown facility: shutdown could not work, because shutdown is invalid change config file /opt/myamc/vff/vff_<pool>/config/myamc_fa.xml ### from old myamc_fa.xml settings: <configentry name="shut_file_for_poff_sync"> <value type= String >/opt/myamc/vff/log/poff_sa_agt_cfg_files.log</value> </configentry> ### to new myamc_fa.xml settings (repeat this change for all pools): <configentry name="shut_file_for_poff_sync"> <value type= String >/opt/myamc/vff/vff_${vff}/data/fa/shutdown/poff_sa_agt_cfg_files.log</value> </configentry> Upgrade Run ff_ha_cmd.sh restart myamc.fa_ctrlagent to activate the myamc_fa.xml change. Run ff_ha_cmd.sh restart tomcat to load changed FA_WebGui/WEB-INF/lib/*jar files. Hint to usage of old FFO V1.0A Application Node images: - Only a temporary use is permitted. Switch to the FFO V1.1A image as soon as possible! - No assignment (--op os), no move (--op move), no create of an old FFO V1.0A-Image is possible. Copyright 2015 Fujitsu 9

10 - Only: reassigning (ff_new_an.sh -B), running and rebooting! - ff_hbaagt start exits with status message due to missing a old SAN command. - Jumbo frames are not enabled within old images. - Message "ff_group_by_name.sh: command not found" appears when stopping/starting a HANA DB. Please ignore this. Script ff_group_by_name.sh is part of the FFO V1.1A image only. Please note that the following exports will be set to their default permissions during the post phase of the upgrade: <volff>/os <volff>/flexframe <poolvolff>/pool-<poolname> If these were modified to allow external access, this modification will be reset and will have to be done again. Connections which rely on custom export rules will be interrupted during the post phase on CN1 of the upgrade. During upgrade the Management Tool is used to validate the running configuration. This requires a graphical interface. If there is no X11 forwarding possible use the -n option (none graphic mode) of the command i.e. ff_upgrade.sh -c -n The existing HANA db should be usable without migration, but when a new HANA SID should be added to an older pool it is mandatory to change the file structures/automounter specs (see ff_migrate_nonhrs2hrs.sh in the Service-DVD toolbox) PQ2800B: The former FFO V1.0A style 1GbE PXE boot helper is no longer needed and has to be removed. But NOTE this reconfiguration includes FW-Upgrade and Server downtime, but can already be done under FFO1.0A as a separate project, weeks or months before the actual upgrade. An existing ff_hardware_sfg.xml for a release lower than V1.1A is not valid for V1.1A and has to be requested again. Updating to the new release with the old ff_hardware_sfg.xml is not supported. The default Linux kernel setting net.ipv4.conf.all.rp_filter=1 interpretation has changed between SLES10 and SLES11. This may impact only customers with special routing configuration on the control nodes. Management Tool Control Center object: The former Type value "Control Node" is now set to default string "RX2540M1". Don't worry when running the upgrade with different control node hardware, the Type value is currently relevant and changeable in case of "New configuration" only. E.g. to enable the virtual Control Center support. The upgrade of the FlexFrame Agents has been moved to a later point in time. Please see the Upgrade Guide Chapter 2.2 for more details. FA_WebGui: Please check for duplicate vijava*.jar libraries because the new Transport Level Security (vijava55myamc_t should be used to collect Hypervisor information. cd /opt/myamc/fa_webgui/web-inf/lib/ mv vijava2u jar to vijava2u jar.bak Run ff_ha_cmd.sh restart tomcat to reload the jar files. Linux Operating System FlexFrame Orchestrator supports "SUSE Linux Enterprise Server for SAP applications" (SLES4SAP) alternatively to the standard "SUSE Linux Enterprise Server" (SLES). The main advantage of SLES4SAP in a FlexFrame for SAP configuration is the extended support period of 18 month after release of a new Service Pack. Copyright 2015 Fujitsu 10

11 Control Nodes The delivered operating system images for Control Nodes and Application Nodes are prepared to run either as SLES4SAP or as standard SLES. To make use of the advantages of SLES4SAP the customer has to order the corresponding SUSE Linux subscription & service packages, e.g. SUSE Linux ES4SAP 1-2 SOCK Server 3Y. Within one FlexFrame Orchestrator configuration, all subscriptions have to be either SLES4SAP or SLES, a mixture is not allowed. To order the SLES4SAP subscription & service packages please contact your local Fujitsu sales representative. As of FFO V1.1A the Control Center of FlexFrame can either consist of physical Control Nodes or of virtual Control Nodes. In case of a virtual Control Center both virtual machines are hosted on different physical PRIMERGY servers that run SUSE KVM as hypervisor. Before setting up a virtual Control Center refer to the following documentation: Support Matrix - Supported PRIMERGY models HW_Characteristics_*Quickguide - 4 LAN-interfaces, required disk-size and RAM-size Management_Tool manual - new Control Node Type KVMVM Installation_Guide - chapter "Installation of Hypervisor Nodes for virtual Control Center" - and the new ff_setup_vcn.sh command. Administration & Operation manual - chapter "virtual Control Center" - chapter "Site failover within virtual Control Center" NOTE: A switch from physical to virtual Control Center during the Upgrade process is not possible. When opening a FFO 1.0 config file the MgmtTool inserts Control Node Type "RX2540M1" no matter what type the current hardware is. If the real model is e.g. RX300S8, there is no need to change the default Type. The disk layout (partitioning) is created during the installation of the Control Node. The layout is predefined and cannot be changed by the user. The order of the partitions is likewise determined during the installation of the Control Node. The two unused partition created during this process can be used during a FlexFrame upgrade and may not be used for other purposes. No additional PCI cards or USB devices other than the ones installed on shipment may be used. Extensions are only allowed in accordance to the Configuration Guide available through the Fujitsu Infrastructures Solutions Partner Portal. It is not intended to have additional software packages on the Control Nodes. The backup procedure assumes that the Control Node can be installed using the initial installation DVD plus a handful of configuration files saved onto the NAS. There is no backup of any additional software. No manual changes of the Control Nodes configuration files or LDAP schemas are allowed, especially not in the configuration files which are generated by FlexFrame tools. They have an explicit warning in the beginning of each file. However, changes are allowed as they are based on the FlexFrame tools or procedures described in the Administration and Operations Manual or in the Installation Guide, or upon permission or instruction of Fujitsu support. The use of telnet, ftp or rsh to the Control Nodes is not intended nor configured. FlexFrame requires the usage of the ssh (secure shell) tools. The pool created with the internal attribute CN Pool = true (visible in the Management Tool) may not be removed by an administration command, since this pool is used to mount file systems from the NAS device. This attribute is usually assigned to the pool created first. Copyright 2015 Fujitsu 11

12 Networking SNMP trap forwarding changed from ServerView SMAWtrpsrvd setup to Linux native snmpd and snmptrapd. See forward entry in /etc/snmp/snmptrapd.conf and basic trap forwarding in chapter in FlexFrame_Messenger_Guide. Jumbo Frames: A new Installation automatically configures MTU size NAS interfaces of new pools are configured for MTU size But note the MTU size will not be changed during upgrade. See also Networking_Guide Chapter 2.3 Fallback to standard MTU size 1500: If one network component on the path to the NAS system does not support MTU size 9000 or by changing the Netapp Filer MTU size in etc/rc ifconfig storage-vlan... mtusize 9000 back to ifconfig storage-vlan... mtusize 1500 The VLANs with the IDs 1, , and 4094 are switch internally reserved and can therefore not be used for FlexFrame purposes. This is a restriction, which is imposed by the switches and is valid for all FlexFrame Orchestrator versions. The Cisco Catalyst 3750-E switches can also be configured with so-called TwinGig converters. Then the switch provides 4 SFP ports instead of the 2 10GbE ports and therefore basically offers the same connectivity as an Cisco Catalyst 3750G-24TS or -48TS model. This configuration is not generally supported in FlexFrame Orchestrator. Customer projects upon special release request are possible. Please note that only modules that are present in the SupportMatrix are supported for the switchtypes Cisco Catalyst 3750-X and Cisco Nexus 5010/5020 and 5548/5596. With more than 4 configured Cisco Catalyst 3750-E/-X switches a warning about the usage of the 10GbE ports is issued. Please contact Expert.SAP@ts.fujitsu.com about any throughput implications. The failure of the switch operating system or the reloading of a switch will make the whole switch stack unavailable. BX900 switch blade SBAX3 is supported in SBAX2 style as 10GbE-18/8-F. The generated switch config must be changed before activation. See line "interface range 0/1-0/34" in HW_Characteristics_SWB_10GbE_18_8_F_Quickguide. This patchlevel supports Brocade switch VDX6740 with 24 and 48 port license. When adding an additional (port on demand) licenses on model VDX use ff_swgroup_adm.pl --op migrate-sw --type vdx to enable the full port range. Network Attached Storage Netapp ONTAP is supported in 7-mode and cluster-mode (cdot). ONTAP 7-mode: After Upgrade to FFO V1.1A or new Installation of FFO V1.1A the FlexFrame Orchestrator internal folder structure is prepared for a later transition to cdot, but note: The Upgrade procedure does not include or require a transition to cdot. Transition from 7-mode to cdot is a separate optional step possible after Upgrade to FFO V1.1A. If applicable this must be planned together with company Netapp and is in the responsibility of certified Netapp technicians. FFO provides some commands to define the cdot Filer only. In cluster-mode FlexFrame Orchestrator expects access as user 'ff4s' on the SVM (Storage Virtual Machine) and optional access as user 'ff4s' on the cdotcluster. See HW_Characteristics_Netapp_SVM_Quickguide and Administration_Operation manual. If ssh access to the cdotcluster is not available, some commands e.g. ff_pool_adm.pl --op add will create a list of manual instructions. Concerning backup using snapshots there is a speciality with NetApp and cdot: NetApp offers a function to access the contents of snapshots by way of file-system, via the.snapshot directory. Copyright 2015 Fujitsu 12

13 Per default the.snapshot directory access is hidden within FlexFrame Orchestrator (volume modify option - snapdir-access false). See details and restrictions in Administration_Operation manual chapter 3.6. NetApp storage NICs: Either exactly two or four 10GbE NICs or none at all are supported by FlexFrame Orchestrator. SAN Support for database (sapdata and saplog volume) SAP Configuration and Administration of the SAN storage (RAID box) and of the SAN fabric (FC switches) is in the responsibility of the user. All Application Nodes within one pool-group must be configured equally: same number and type of HBAs, same OS image, same set of LUNs. ff_clone_sap.sh is not supported when using SAN volumes, because the clone feature is based on the Netapp feature FlexClone. Pool-independent spares are not supported for pool-groups using SAN volumes. Use a pool-group specific spare node. Application nodes using SAN volumes must not run on blade servers or on virtual machines, only native rackservers are allowed SAN storage can be assigned for sapdata and saplog volumes only. Warning: Limit the number of simultaneous FC logins (the limit depends on type of SAN storage): Use only two HBA ports with max. two logical connections and a limited number of ANs in the db-server pool-group. HANA: Test done with SPS08 and SPS09. If you plan to install or upgrade to SPS09 then use at least revision 97. HANA database authorization data is stored hostspecific in a Secure Storage File: /home_sap/<sidadm>/.hdb/<hostname>/ssfs_hdb.dat SAP_Installation_Guide chapter and includes hints for the hostspecific preparation of SSFS_HDB.DAT. The SAP installer sapinst cares for the installation host only! Sapbackup: This optional volume is handled by CLI only. See Administration_Operation manual. ff_sid_adm.pl --op list-all does not show the sapbackup volume. Use ff_sid_mnt_adm.pl --op list to get this information. The HANA Lifecycle Manager (HLM) is using a multiple user switch to <sid>adm which is generally forbidden with FlexFrame for security reasons. By Special Release Request the predefined group wheel can be used to bypass this restriction. Because of changes in SAP-Software (SAP Solution Manager/SAP Hostagent) FlexFrame will use hostspecific / etc/oratab instead of poolspecific one. The previous link '/etc/oratab' points to '/var/flexframe/etc/oratab' which contains only the SIDs currently running on specific application host. The older file '/oracle/ora-config/oratab' which contains the complete list of SIDs in the pool is reachable via the link '/etc/oratab.shadow'. Depending on the software stack used, the SAP-command startsap (called by ff_service.sh) shows a warning message Could not find virtual host in ifconfig list. That message can be ignored. BOBJ: In case of SAP Business Objects 4.0 FlexFrame Orchestrator supports the integrated (bundled) IBM DB2 database installation only. In case of SAP Business Objects 4.1 FlexFrame Orchestrator supports the integrated (bundled) SybaseSQL ANYWHERE database installation only. Copyright 2015 Fujitsu 13

14 DB2 V9.7FP3 - V10.5FP3: Sometimes (e.g. after switching the root image) the central instance does not start. R3trans fails due to missing DB2AUTH settings. This will be solved by DB2 V10.5FP4 and later. When using one of this older versions and fixpacks please use db2set DB2AUTH=OSAUTHDB as documented in the FF11A00_SAP_Installation_Guide chapter "Transparent LDAP" or setup the pool-specific global.reg as documented in chapter "Troubleshooting". Run db2stop and db2start to activate the change. With older versions of the SAP software provisioning manager (SWPM) a DB2 installation may fail with error 'Unable to determine whether account user="db2sid" uid="3600" has password, SAP case /2011' A workaround for this error is described in the SAP Installation Guide. There are two noteworthy notes from SAP in relation to SLES11: Note SAP JVM as replacement for Partner JDKs SAP developed its own Java Virtual Machine because of the end of support for Partner JDKs. Please see the SAP Note for more details. Note Linux-3.x-Kernel When trying to run SAP programs on a Linux Kernel 3.x an error message can appear ("can not determine current Linux kernel version --> exiting" / OS release Linux 3.x is not supported with this kernel). In the past only Linux 2.6 or 2.4 kernels where allowed. To run SAP on the new Linux kernel (which is integrated in SLES11 SP3), a SAP kernel update is required. Please check the SAP Note to determine which update fit your needs. The SAP Versions 4.0B - 4.6C and WebAS 6.20ITS are not supported with FlexFrame Orchestrator. The most recent tested DB versions are: MaxDB 7.9, Oracle 11g, IBM DB2 10.5, Sybase ASE 16, and HANA 1.0 SPS08. WebAS 6.20 with pure Java Instances (for portals) is not integrated and does not work. Mixed ABAP and Java Instances work well. Only those SAP solutions which SAP states to be adaptive are supported and may be used with FlexFrame Orchestrator. Known examples of SAP solutions not being supported in FlexFrame Orchestrator are: Request Bugseye and IPC with WebAS If needed, these solutions can be implemented on a single project basis upon special release request. The SID of a SAP System has to be identical to the corresponding SID of the database. This especially leads to the fact, that SAP's MCOD-Feature (Multiple Components in One Database) is not supported within FlexFrame Orchestrator. SAP NetWeaver services can use HANA as its database. A HANA database can serve multiple NetWeaver services and therefore the SID of the HANA database is different from the SID of NetWeaver service. Host-based SMD: When renaming or moving an Application Node with host-based SMD the LDAP definition will be adapted automatically but the SAP specific data (e.g. profiles) still need manual preparation. Procedure in Case of Errors Information about troubleshooting are included in chapter 12 of the document Administration & Operation. For all FlexFrame Orchestrator components (hardware and software), a service contract is mandatory. For service reachability, at least a 5 x 9 hours service contract has to be defined. This service contract assures support for error fixing of hard- and software components as well as delivery of updates and patches. Copyright 2015 Fujitsu 14