IBM InfoSphere Guardium Tech Talk: Roadmap to a successful V9 upgrade

Transcription

1 Vlad Langman - L3 Engineering Manager Abdiel Santos - Sr. L3 Engineer 14 Mar 2013 IBM InfoSphere Guardium Tech Talk: Roadmap to a successful V9 upgrade Information Management

2 Logistics This tech talk is being recorded. If you object, please hang up and leave the webcast now. We ll post a copy of slides and link to recording on the Guardium community tech talk wiki page: You can listen to the tech talk using audiocast and ask questions in the chat to the Q and A group. We ll try to answer questions in the chat or address them at speaker s discretion. If we cannot answer your question, please do include your so we can get back to you. When speaker pauses for questions: We ll go through existing questions in the chat Raise your hand in the SmartCloud meeting room if you want to ask a question verbally and we ll call your name You will need *6 to unmute phone line if you are dialed in 2 April 11, 2013 IBM InfoSphere Guardium Tech Talk

3 Reminder: Guardium Tech Talks Next tech talk: Take Control of your InfoSphere Guardium Appliance Speakers: Daniel Perlov and Abdiel Santos Date &Time: Thursday April 11, :30 AM Eastern Register here: Link to more information about this and upcoming tech talks can be found on the InfoSpere Guardium developerworks community: Please submit a comment on this page for ideas for tech talk topics. 3 April 11, 2013 IBM InfoSphere Guardium Tech Talk

4 Vlad Langman - L3 Engineering Manager Abdiel Santos - Sr. L3 Engineer 14 Mar 2013 IBM InfoSphere Guardium Tech Talk: Roadmap to a successful V9 upgrade Information Management

5 Agenda Enterprise Upgrade Strategy Appliance Upgrade Methods Appliance Upgrade Procedure Agent Upgrade Methods Known Issues and Limitations Troubleshooting Additional Information 5 March 14, 2013 IBM InfoSphere Guardium Tech Talk

6 Enterprise Upgrade Strategy Upgrade of large enterprise environments requires thorough planning and preparation. Upgrade strategy and logistics Change control management Required personal availability Contingency planning The upgrade process usually cannot be done simultaneously on all appliances and all S-TAPs. Therefore it requires multi-staged upgrade approach. During transition period, Guardium environment will operate in hybrid mode with Version 8.2 and Version 9.0 Guardium software. Upgrade IBM InfoSphere Guardium environment in Top-to-Bottom order 6 March 14, 2013 IBM InfoSphere Guardium Tech Talk Hybrid stage - Transition period during the upgrade, where customer have a hybrid Version 8.2 and Version 9.x Guardium solution. Top to Bottom - To allow continuous connectivity between different components of Guardium solution, the Guardium environment should be upgraded in Top to Bottom order, starting with Central Manager, Aggregators, Collectors and then all S-TAPs. 6

7 Enterprise Upgrade Strategy Top-To-Bottom Order of Upgrade Rollout Central Manager Appliance Aggregators Appliance Collectors Appliance Guardium agents Host DB Server 7 March 14, 2013 IBM InfoSphere Guardium Tech Talk Top-To-Bottom order of upgrade rollout is required by Guardium. This order is enforced to always keep corresponding higher level appliances on the same or higher level of Guardium software. Corresponding STAP agents should be upgraded last. 7

8 Enterprise Upgrade Strategy Typical Enterprise Upgrade Paths HORIZONTAL approach VERTICAL approach MIXED approach 8 March 14, 2013 IBM InfoSphere Guardium Tech Talk 8

9 Enterprise Upgrade Strategy Horizontal approach Central Manager level 1 2 Aggregators level Collectors level Guardium agents level 9 March 14, 2013 IBM InfoSphere Guardium Tech Talk Horizontal approach - You typically upgrade all the appliances at a specific level before moving on to the next level. For example, upgrade all Central Managers before moving on to the Aggregators. 9

10 Enterprise Upgrade Strategy Vertical approach Central Manager level 1 2 Aggregators level Collectors level Guardium agents level 10 March 14, 2013 IBM InfoSphere Guardium Tech Talk Vertical approach - You typically upgrade a single Central Manager (CM), followed by an aggregator managed by this CM, followed by all the collectors exporting to this aggregator. Then you move on to the next aggregator and its collectors until all the managed units are upgraded. This approach allows you to minimize the time that the CM and Collectors are version mismatched, for which there are some policy limitation (discussed on later slides). This approach also minimizes the time that Aggregators and Collectors are version mismatched, which should improve performance by avoiding the table structure conversions required in hybrid mode. 10

11 Enterprise Upgrade Strategy Mixed approach Central Manager level 1 2 Aggregators level Collectors level Guardium agents level 11 March 14, 2013 IBM InfoSphere Guardium Tech Talk Mixed approach incorporates both Vertical and Horizontal strategies and is the method most commonly used by customers. 11

12 Enterprise Upgrade Strategy Limitations - While "hybrid mode" is supported by Guardium, many functions are limited until all components are at the same version. (discussed in later slides) Appliances upgrade - Customers are strongly encouraged to choose the upgrade strategy which minimizes duration where appliances operate in mixed V8.2 and V9.0 versions. Downtime - Typical upgrade of Guardium appliance expected to last for several hours. During this time, the appliance might not be accessible and would not be performing any data collection activity. Duration - Upgrade process duration most contributing factors: Size, Usage and Data distribution of internal MySQL database tables Capacity of the appliance (Virtual Memory, CPU and so on) Purge of unnecessary data from the appliance. It will significantly decreases duration of the upgrade process. Pre-upgrade Health Check enforces DB size to be below 50% capacity. Our recommendations to keep databases under 20% threshold. 12 March 14, 2013 IBM InfoSphere Guardium Tech Talk Hybrid stage - The upgrade process usually cannot be done simultaneously on all appliances and S-TAPs (Central Manager, Aggregator, Collector and Managed Units). During the upgrade transition, customers will have a hybrid Version 8.2 and Version 9.x Guardium solution. Best practices Customers are encouraged to minimize the amount of time their appliances operate in hybrid mode. Approach consideration - In most cases, choice of enterprise upgrade approach depends on internal business constraints such as Change Control windows, main application and Host DB upgrades, resource availability and so on. We recommend to take a Vertical upgrade approach whenever possible. Downtime To avoid loss of data collection customers can redirect STAPs to different collectors. Keep in mind that this could result in data being exported to other aggregators. 12

13 Agenda Enterprise Upgrade Strategy Appliance Upgrade Methods Appliance Upgrade Procedure Agent Upgrade Methods Known Issues and Limitations Troubleshooting Additional Information 13 March 14, 2013 IBM InfoSphere Guardium Tech Talk

14 Appliance Upgrade Methods 1. Upgrade appliances using Guardium v8.x v9.0p02 upgrade bundle. 2. Rebuild appliances with v9.0 ISO image and restore the v8.2 backup 3. Build new separate v9.0 appliances and gradually retire the old ones. 14 March 14, 2013 IBM InfoSphere Guardium Tech Talk There are 3 typical upgrade methods currently used by customers: 1. Upgrade appliances using Guardium v8.x v9.0p02 upgrade bundle. 2. Rebuild appliances with v9.0 ISO image and restore their v8.2 backup 3. Build new v9.0 appliances and gradually retire v8.x appliances. We recommend using method 1 in most cases 1.Upgrade: Pros Cons Ensures complete data continuity. No major configuration changes required. Additional setup flexibility ( Ability to keep previous layout ) Requires more extensive planning Typically more downtime than other methods. 2.Rebuild and restore - typically used as a contingency option in case of an issue with method 1. Pros Less complex than Option 1 14

15 Appliance Upgrade Methods v8.01 v9.0p02 Upgrade (Standalone Appliances) Main Upgrade steps 1. Pre-upgrade Health Check patch for v801 v8.2 upgrade. 2. Upgrade appliance using v8.01 v9.0p02 upgrade bundle patch 3. Upgrade GIM / STAP agents to v9.0 Limitations: v8.01 STAP v9.0 appliance configuration is not supported. STAP agent needs to be simultaneously upgraded v9.0 or pointed to another v8.x base collector until it can be upgraded. Applicable for stand-alone (no CM) environments only Applicable for any GPU patching level 15 March 14, 2013 IBM InfoSphere Guardium Tech Talk 15

16 Appliance Upgrade Methods v8.01 v9.0p02 upgrade (Federated Environment) Main Upgrade steps 1. Pre-upgrade Health Check patch for v801 v8.2 upgrade. 2. Upgrade appliances to v8.2 using v8.01 v8.2 upgrade patch 3. Upgrade GIM / STAPs agents to v Upgrade appliances to v9.0p02 using v8.2 v9.0p02 upgrade bundle 5. Upgrade GIM / STAPs agents to latest v9.0 Limitations v8.01 v9.0p02 upgrade bundle can t be used in Federated environments. Applicable for any GPU patching level 16 March 14, 2013 IBM InfoSphere Guardium Tech Talk Two step upgrade method: upgrade from v8.01 to v8.2 upgrade from v8.2 to v9.0 Note: If upgrade v801 to v8.2 can t be immediately followed by upgrade to v9.0p02, pre-upgrade Health Check for v8.2-v9.0p02 upgrade would be required. 16

17 Appliance Upgrade Methods v8.2 v9.0p02 upgrade Main Upgrade steps 1. Pre-upgrade Health Check patch for v8.2 v9.0p02 upgrade. 2. Direct v9.0p02 upgrade using v8.2 v9.0p02 upgrade bundle Applicable for all Guardium architectures (stand-alone and federated environments). Applicable for any GPU patching level 17 March 14, 2013 IBM InfoSphere Guardium Tech Talk Direct upgrade using v8.2-v9.0p02 upgrade bundle. 17

18 Quiz question! What is one key thing you can do to reduce the amount of time to complete the appliance upgrade? 1. Keep current GUI layout 2. Drink lots of coffee 3. Purge the system to reduce size of internal database 4. None of the above The correct answer is 3.

19 Agenda Enterprise Upgrade Strategy Appliance Upgrade Methods Appliance Upgrade Procedure Agent Upgrade Methods Known Issues and Limitations Troubleshooting Additional Information 19 March 14, 2013 IBM InfoSphere Guardium Tech Talk Note: This portion of presentation focuses primarily on v8.2 to v9.0p02 upgrade. 19

20 Appliance Upgrade Procedure Patch download location IBM Fix Central site: Health Check patch for v8.2 v9.0 upgrade Upgrade bundle for v8.2 v9.0p02 upgrade 20 March 14, 2013 IBM InfoSphere Guardium Tech Talk 20

21 Upgrade Procedure Step-by-step upgrade 1. Check system requirements 2. Purge the systems 3. Run pre-upgrade backup for v Run pre-upgrade Health Check patch 5. If enabled, turn off high-availability 6. Decide on GUI layout 7. Stop Export process 8. Upload v8.2-v9.0p02 upgrade bundle patch 9. Install bundle patch 21 March 14, 2013 IBM InfoSphere Guardium Tech Talk Down time - Plan for at least two hours of down time for the upgrade. Duration Most contributing factors to upgrade duration: Size, Usage and Data distribution of internal MySQL database tables Capacity of the appliance (Virtual Memory, CPU and so on) System Req. Dell models R610 IBM models xseries 3550 M4 InfoSphere Guardium Patch Level v8.01(standalone) / 8.2 Memory 4 GB 16 GB Purge the systems - Upgrade requires appliance DB to be below 50% utilization, however its strongly recommended to go below 20% to speed up the process and minimize the risks related to the upgrade of large amounts of data. System Backup can be used as an alternative upgrade route. High-Availability System upgrades - If your unit is configured for highavailability then this functionality must be turned off via the CLI, and the unit 21

22 Upgrade Procedure V8.2 V9.0p02 bundle upgrade patch V8.2 V9.0p02 upgrade bundle contains the following patches: 1. V8.2 V9.0 upgrade patch 2. V9.0p02 GPU patch V8.2 V9.0p02 upgrade bundle is the recommended upgrade method. 22 March 14, 2013 IBM InfoSphere Guardium Tech Talk 22

23 Appliance Upgrade Methods 1. Upgrade appliances using Guardium v8.x v9.0 upgrade patch. 2. Rebuild appliances with v9.0 ISO image and restore the v8.2 backup 3. Build new separate v9.0 appliances and gradually retire the old ones. 23 March 14, 2013 IBM InfoSphere Guardium Tech Talk There are 3 typical upgrade methods currently used by customers: 1. Upgrade appliances using Guardium v8.x v9.0p02 upgrade bundle. 2. Rebuild appliances with v9.0 ISO image and restore their v8.2 backup 3. Build new v9.0 appliances and gradually retire v8.x appliances. We recommend using method 1 in most cases 1.Upgrade: Pros Cons Ensures complete data continuity. No major configuration changes required. Additional setup flexibility ( Ability to keep previous layout ) Requires more extensive planning Typically more downtime than other methods. 2.Rebuild and restore - typically used as a contingency option in case of an issue with method 1. Pros Less complex than Option 1 23

24 Appliance Upgrade Methods Procedure: Rebuild Appliance And Restore From Prev. Version 1.Create Data and Configuration Backup of Guardium v8.2 appliance 2.Install Guardium v9.0 ISO image. 3.Generate new v9.0 system backup. 4.Copy Data and Configuration backup file to appliance ( using CLI import file command ) 5.Restore and upgrade data from v8.2 backup ( using CLI restore db-from-prev-version ) 6.Apply most recent v9.0 GPU Maintenance patch 24 March 14, 2013 IBM InfoSphere Guardium Tech Talk Software location clarification: Upgrade bundles available on IBM FixCentral Infosphere Guardium v9.0 ISO image available on Passport Advantage 24

25 Appliance Upgrade Methods 1. Upgrade appliances using Guardium v8.x v9.0 upgrade patch. 2. Rebuild appliances with v9.0 ISO image and restore the v8.2 backup 3. Build new separate v9.0 appliances and gradually retire the old ones. 25 March 14, 2013 IBM InfoSphere Guardium Tech Talk There are 3 typical upgrade methods currently used by customers: 1. Upgrade appliances using Guardium v8.x v9.0p02 upgrade bundle. 2. Rebuild appliances with v9.0 ISO image and restore their v8.2 backup 3. Build new v9.0 appliances and gradually retire v8.x appliances. We recommend using method 1 in most cases 1.Upgrade: Pros Cons Ensures complete data continuity. No major configuration changes required. Additional setup flexibility ( Ability to keep previous layout ) Requires more extensive planning Typically more downtime than other methods. 2.Rebuild and restore - typically used as a contingency option in case of an issue with method 1. Pros Less complex than Option 1 25

26 Appliance Upgrade Methods New appliance: Build new appliance and gradually retire old one 1.Install Guardium v9.0 ISO image (from PA) 2.Generate new v9.0 system backup. 3.Apply most recent v9.0 GPU Maintenance patch 4.Configure STAPs to report to new appliance Note: All the reports/queries requiring pre-upgrade data need to be maintained on old appliances. 26 March 14, 2013 IBM InfoSphere Guardium Tech Talk 26

27 Appliance Upgrade Methods Import Definitions from v8.2 version to v9.0p02 GPU level appliance 27 March 14, 2013 IBM InfoSphere Guardium Tech Talk Limitations: Import definitions only applies for v8.2 -> v9. Importing definitions will not restore any Audit data. Import is only applicable for standalone appliances and CM units (in federated environment) This option could be relevant to copy/restore v8.2 definitions on v9.0p02 appliance and could be used in conjunction or instead of system restore. It s also handy to import report definitions from v8.2 appliances onto new v9.0p02 appliances when using Appliance Upgrade Method 3 (Build new separate v9.0 appliances and gradually retire the old ones.) Applicable for exports definitions on any v8.2 GPU level 27

28 Agenda Enterprise Upgrade Strategy Appliance Upgrade Methods Appliance Upgrade Procedure Agent Upgrade Methods Known Issues and Limitations Troubleshooting Additional Information 28 March 14, 2013 IBM InfoSphere Guardium Tech Talk 28

29 Agents Upgrade Methods GIM Upgrade GIM agent installation/upgrade need to be completed prior to installation of other Guardium agents. Typical GIM installation options: GIM shell installer GIM bundle upgrade (used on the system with GIM) GIM shell Installer options required on systems without GIM installed GIM bundle upgrade is applied through Infosphere Guardium appliance GUI GIM bundle upgrade applies only for systems with previous version of GIM. 29 March 14, 2013 IBM InfoSphere Guardium Tech Talk 29

30 Agents Upgrade Methods UNIX STAP Upgrade STAP installation options: STAP shell installer (from command line on DB server) GIM STAP bundle (from through Guardium Appliance UI) STAP shell installer 1.Live update via guard-stap-update utility (recommended) 2.Uninstall/reboot/install new version (required during the major upgrade of host operating system) GIM STAP bundle typically used with live update option (KTAP_LIVE_UPDATE=Y) which allows STAP to upgraded without rebooting the host DB server. 30 March 14, 2013 IBM InfoSphere Guardium Tech Talk Starting with version v8.x, Guardium supports live (boot-less) KTAP upgrade, which does not require reboot of DB Server after installation of newer version of STAP. The Live update mechanism is controlled through the GUI via the KTAP_LIVE_UPDATE parameter or BUNDLE-STAP/KTAP installers using guard-stap-update utility. guard-stap-update utility available as part of the STAP downloads on fix central. NOTE: Host DB Server OS major version upgrade 1. Uninstall all Guardium agents (GIM / STAP) 2. Reboot the system 3. Upgrade the operating systems. 4. Reinstall platform specific Guardium agents in following order 1) GIM 2) STAP 30

31 Agents Upgrade Methods WINDOWS STAP Upgrade Windows STAP installation options: STAP silent installer (from command prompt on DB server) GIM STAP bundle ( from Guardium Appliance UI) STAP silent installer 1.Live update via setup.exe /s /z UPGRADE (recommended) 2.Uninstall/reboot/install new version (required during the major upgrade of host operating system) GIM STAP bundle typically used with live update option which allows STAP to upgraded without rebooting the system. 31 March 14, 2013 IBM InfoSphere Guardium Tech Talk STAP silent installer allows bootless update of STAP 31

32 Agenda Enterprise Upgrade Strategy Appliance Upgrade Methods Appliance Upgrade Procedure Agent Upgrade Methods Known Issues and Limitations Troubleshooting Additional Information 32 March 14, 2013 IBM InfoSphere Guardium Tech Talk 32

33 Known Issues and Limitations 1.A remote policy installation from v9 CM on v8.2 Managed unit is not permitted. 2.Installation of Policy with v9 specific rules on v8.2 appliance (in Federated environment) will not work. 3.Error unable to locate DLL during Windows STAP upgrade. This can be fixed by using a later version of Java or downloading the DLL from the Microsoft site. 33 March 14, 2013 IBM InfoSphere Guardium Tech Talk 1. Remote Policy installation from CM is not permitted in mixed environments (policy can only be installed locally on the managed units). 2. There are a few IBM InfoSphere Guardium 9.0 rule actions that are not supported in IBM InfoSphere Guardium Version 8.2. If such rule actions are present, they will not work on the managed unit even if policy is installed locally. 3. During upgrade of Windows S-TAPs, for Java 1.6.0, an error may be generated by the JVM that indicates it is unable to locate the dynamic link library MSVCR71.dll. This error can be remedied by implementing one of the two workarounds: (1) use a different (another release) JVM (if one is available on the system) or (2) download the DLL from Microsoft and place it in the Windows system directory. 33

34 Agenda Enterprise Upgrade Strategy Appliance Upgrade Methods Appliance Upgrade Procedure Agent Upgrade Methods Known Issues and Limitations Troubleshooting Additional Information 34 March 14, 2013 IBM InfoSphere Guardium Tech Talk 34

35 Troubleshooting Pre-upgrade Health Check patch validation Appliance V9.0 upgrade process validation 35 March 14, 2013 IBM InfoSphere Guardium Tech Talk Pre-Upgrade Health Check is mandatory prerequisite for upgrade. Upgrade process will not run without Health Check running successfully. 35

36 Troubleshooting Pre-upgrade Health Check status validation Successful Health Check installation status through GUI Successful Health Check installation status through CLI 36 March 14, 2013 IBM InfoSphere Guardium Tech Talk 36

37 Troubleshooting Pre-upgrade Health Check status validation Health Check log file could be accessed through CLI fileserver. Content of Health Check directory through fileserver Health Check output log content 37 March 14, 2013 IBM InfoSphere Guardium Tech Talk Health Check output log locate under diag/current directory. Appliance is ready for upgrade at the end of the HC log indicate that user can proceed to next step of the upgrade process. 37

38 Troubleshooting V9 Upgrade and GPU V9.0p02 status validation Successful v9 Upgrade and GPU v9.0p02 installation statuses through GUI Successful v9 Upgrade and GPU v9.0p02 installation statuses through CLI 38 March 14, 2013 IBM InfoSphere Guardium Tech Talk 38

39 Troubleshooting V9 Upgrade status validation Upgrade installation log could be accessed through CLI fileserver. Content of patch installation log directory through fileserver Upgrade installation log content 39 March 14, 2013 IBM InfoSphere Guardium Tech Talk Upgrade patch installation log located under diag/depot directory. DONE: Installation Successful at the end of the installation log indicates successful competition of the V9 upgrade process. 39

40 Troubleshooting V9 Upgrade status validation Failed upgrade patch installation status through GUI Failed upgrade patch installation status through CLI 40 March 14, 2013 IBM InfoSphere Guardium Tech Talk 40

41 Troubleshooting Main upgrade installation log located in /diag/depot directory Search for ERROR: or fatal phrase End of the log file: 41 March 14, 2013 IBM InfoSphere Guardium Tech Talk Search for the ERROR phrase to identify installation error messages. 41

42 Troubleshooting Must Gather information Upgrade installation diagnostic data required to troubleshoot upgrade issues: 1.V9.0 Upgrade installation log patch-8.2p9000_upgrade_to_version_9.0_<timestamp>.log 2.V9.0p02 GPU patch installation log patch-9.0p02_gpu_october_2012_<timestamp>.log 3.Output of CLI Must Gather command support must_gather system_db_info 42 March 14, 2013 IBM InfoSphere Guardium Tech Talk Must Gather diagnostic information is required by Support to be provided with upgrade PMRs to speed up investigation process. 42

43 Agenda Enterprise Upgrade Strategy Appliance Upgrade Methods Appliance Upgrade Procedure Agent Upgrade Methods Known Issues and Limitations Troubleshooting Additional Information 43 March 14, 2013 IBM InfoSphere Guardium Tech Talk 43

44 Additional Information V8.2 to V9.0p02 Upgrade Best Practices V9.0 Detailed Release notes For new and enhanced features, see V9.0 Software Appliance Technical Requirements V9.0 System Requirements (Platforms Supported) 44 March 14, 2013 IBM InfoSphere Guardium Tech Talk 44

45 Reminder: Guardium Tech Talks Next tech talk: Take Control of your InfoSphere Guardium Appliance Speakers: Daniel Perlov and Abdiel Santos Date &Time: Thursday April 11, :30 AM Eastern Register here: Link to more information about this and upcoming tech talks can be found on the InfoSpere Guardium developerworks community: Please submit a comment on this page for ideas for tech talk topics. 45 April 11, 2013 IBM InfoSphere Guardium Tech Talk 45