IBM Security Guardium v9.5 Features and Updates Tech Talk

Transcription

1 IBM Security Guardium v9.5 Features and Updates Tech Talk Luis Casco-Arias Product Manager IBM Security Guardium Also with support from: Guy Galil, Lior Solomon and Oded Sofer 1

2 Logistics This tech talk is being recorded. If you object, please hang up and leave the webcast now. We ll post a copy of slides and link to recording on the Guardium community tech talk wiki page: You can listen to the tech talk using audiocast and ask questions in the chat to the Q and A group. We ll try to answer questions in the chat or address them at speaker s discretion. If we cannot answer your question, please do include your so we can get back to you. When speaker pauses for questions: We ll go through existing questions in the chat 2

3 Reminder: Upcoming Guardium Tech Talks April 7 th, 2015: Encrypting Data at Rest in IMS using InfoSphere Guardium Data Encryption Speaker: Dennis Eichelberger, IMS IT Specialist, IBM Register here! April 22nd: Part 2 Overview of InfoSphere Guardium Data Encryption Speaker: Ernie Mancill, Executive IT Specialist, IBM Register here! Link to more information about this and upcoming tech talks can be found on the InfoSphere Guardium developerworks community: Please submit a comment on this page for ideas for tech talk topics. 3

4 Guardium community on developerworks bit.ly/guardwiki Right nav 4

5 Information, training, and community InfoSphere Guardium Tech Talks at least one per month. Suggestions welcome! InfoSphere Guardium YouTube Channel includes overviews, technical demos, tech talk replays developerworks forum (very active) Guardium DAM User Group on Linked In (very active) Community on developerworks (includes discussion forum, content and links to a myriad of sources, developerworks articles, tech talk materials and schedules) Guardium on IBM Knowledge Center (was Info Center) Deployment Guide for InfoSphere Guardium Red Book Technical training courses (provided by IBM Business Partners) InfoSphere Guardium Virtual User Group. Open, technical discussions with other users. Not recorded! Send a note to krzeide@us.ibm.com if interested. 55

6 Agenda Guardium 9.5 Content New Platforms S-TAP automatic load balancer New GuardAPI commands Quick Search Infrastructure Update Enterprise Scope Investigation Dashboard Topology Navigator Outlier Detection enhancement Upgrade path recommendations Demo 6

7 InfoSphere Guardium Data Security & Privacy one-stop-shop to protect against unauthorized access to data, and reduce the cost of compliance Value Proposition Prevent data breaches Ensure data Privacy Reduce the cost of compliance Identifying security risks Enable secure sharing of data On Premise Data at Rest Stored (Databases, File Servers, Big Data, Data Warehouses, Application Servers, Cloud/Virtual..) On Cloud Data in Motion Over Network (SQL, HTTP, SSH, FTP, ,. ) Data Repositories Sensitive Documents 7

8 InfoSphere Guardium v9.5 (1Q15) IBM Security Expand Platform Coverage Enhance Activity Monitoring Scalability & Reduce Op. Costs Security Integrations Vulnerability Assessment * Adjust appliance pricing currency DAM pricing and packaging changes DAM Big Data new part DAM Data Warehouse new part New Platform Support: AsterDB, Pivotal, MariaDB, MS APD/PDW Updates: Teradata 15, GreenplumDB 4.3, Ubuntu 14 Entitlement Reporting included in base services Monitoring Teradata encrypted logon and traffic (A-TAP) Automatic STAP load balancing (phase 1) Quick Search analytics improvement Investigation Dashboard Outlier Detection performance and scoring New monthly based pricing Cloud deployment New GuardAPI commands VA parts consolidation 88

9 InfoSphere Guardium v9.5 New Platforms 9

10 Comprehensive support for structured and unstructured sensitive data: Databases, Data Warehouses, Big Data Environments and File Shares DATABASES Pure Data Analytics Optim Archival D A T A B A S E DB2 with BLU Acceleration InfoSphere BigInsights Master Data Management Exadata Data Stage DB HANA CICS Siebel, PeopleSoft, E-Business FTP Guardium z/os Datasets 10

11 New Platforms and Updates in v9.5 New Platform Support in v9.5 : Aster DB MariaDB MS APD/PDW Pivotal V 9.1 updates Platform support Updates in v9.5 Pivotal V9.1 Greenplum DB 4.3 Hortonworks HDP 2.2 Teradata 15 GreenplumDB SAP/HANA AsterDB GreenplumDB SAP/HANA V9p50 OS Support Updates in v9.5 HortonWorks GreenplumHD HortonWorks GreenplumHD Ubuntu 14 Cassandra CouchDB MongoDB Cassandra CouchDB MongoDB V9 Cloudera BigInsights Cloudera BigInsights Cloudera BigInsights V8 Teradata Netezza 2011 Teradata Netezza 2012 Teradata Netezza 2013 Teradata Netezza

12 InfoSphere Guardium v9.5 STAP Load Balancing (Phase 1) 12

13 Overview Installing an STAP on a DB server requires the customer to assign a managed unit appliance for STAP to connect to. On large sites (> 100 s of managed units), finding an available managed unit requires skill and time. Dedicating a collector to an STAP involves periodically consolidating several indicators (per collector). The 'STAP Load Balancer automates this process 13

14 Load Balancer Highlights Load Balancer Process Running on the Central Manager Collects and maintains a 'Load Map' of all the Managed Units (MU). Allocating least loaded Managed Unit to newly installed STAP(s). Periodic background activity MU 1 MU 2 Report load CM Load Balancer STEP 3 : Find least loaded MU Load Map MU 1 : Loaded MU 2 : Vacant MU 3 : Loaded STEP 2 : Allocate MU Request STEP 1 : Install STAP MU 3 STEP 4 : Connect to MU 2 DB Server *MU generally represents a Collector 14

15 Load Balancer Function - General Load Collection Mechanism Periodically collects load information from all the managed units. Load information relies mostly on 'UNIT UTILIZATION LEVELS' and 'STAP INFO' reports/tables. Load Map Specific load indicators are used to build a 'Load Map' in the Load Balancer memory. MU Allocation Upon receiving a request from STAP to allocate an MU, the Load Balancer will query the Load Map, and assign an MU for the requesting STAP. 15

16 Load Balancer Function Load Collection Default dynamic load collection Intervals. The more MU s, the bigger the load collection intervals. (Controlled by the parameter DYNAMIC_LOAD_CHECK_INTERVAL) Load indicators are used in order to estimate the load level on each MU over a period of time. Load indicators are defined by the STAP Load Balancer parameter LOAD_INDICATORS which is set by default with the following indicators: NO_OF_RESTARTS_UL (Number of Sniffer restart) ANALYZER_QUEUE_UL (Size of Sniffer analyzer queue) LOGGER_QUEUE_UL (Size of Sniffer logger queue) MYSQL_DISK_USAGE_UL (Disk usage) Load Sampling Interval is critical in order to get an accurate picture of the load per MU. Larger load sampling interval means we can capture a more accurate snapshot of load behavior (busy and non-busy hours) rather than capturing spikes. Load indicators are defined by the STAP Load Balancer parameter LOAD_SAMPLE_PERIOD which is set by default to 24 hours. 16

17 Load Balancer Function Load Map After evaluating the load from all load indicators, MU's are placed within the Load Map in 3 load levels : NOT-LOADED (LOAD_LEVEL_1), MEDIUM-LOADED (LOAD_LEVEL_2), VERY-LOADED (LOAD_LEVEL_3). In between load collections, the Load Map is dynamically updated upon each allocation of an MU to a requesting STAP. MU's will be replaced in a new load level if the number of STAPs per single MU exceeds the threshold defined by the Load Balancer parameters: MAX_STAPS_PER_MU_THRESHOLD1 (default : 20), MAX_STAPS_PER_MU_THRESHOLD2 (default :40) > grdapi get_stap_load_balancer_current_load_map LOAD_MAP: LOAD_LEVEL_1 MU's: MU=qa-vm18.guard.swg.usma.ibm.com_5: C_STAPS=1, I_STAPS=0, C_LOAD_LEVEL=1, I_LOAD_LEVEL=1--> MU=qa-vm18.guard.swg.usma.ibm.com_4: C_STAPS=1, I_STAPS=0, C_LOAD_LEVEL=1, I_LOAD_LEVEL=1--> LOAD_LEVEL_2 MU's: MU=qa-vm21.guard.swg.usma.ibm.com_4: C_STAPS=2, I_STAPS=2, C_LOAD_LEVEL=2, I_LOAD_LEVEL=1--> MU=qa-vm22.guard.swg.usma.ibm.com_3: C_STAPS=2, I_STAPS=2, C_LOAD_LEVEL=2, I_LOAD_LEVEL=2--> MU=qa-vm24.guard.swg.usma.ibm.com_6: C_STAPS=6, I_STAPS=6, C_LOAD_LEVEL=2, I_LOAD_LEVEL=2--> C_STAPS : Current # STAPS assigned to MU I_STAPS : Initial # STAPS assigned to MU C_LOAD_LEVEL: Current Load Level of MU I_LOAD_LEVEL : Initial # load level of MU LOAD_LEVEL_3 MU's: MU=qa-vm22.guard.swg.usma.ibm.com_4: C_STAPS=2, I_STAPS=2, C_LOAD_LEVEL=3, I_LOAD_LEVEL=3--> MU=qa-vm24.guard.swg.usma.ibm.com_4: C_STAPS=6, I_STAPS=6, C_LOAD_LEVEL=3, I_LOAD_LEVEL=2--> MU=qa-vm24.guard.swg.usma.ibm.com_5: C_STAPS=6, I_STAPS=6, C_LOAD_LEVEL=3, I_LOAD_LEVEL=3--> 17 STAP REQUESTS CACHE: =1 LAST MU USED PER STAP CACHE: =qa-vm18.guard.swg.usma.ibm.com_6

18 Load Balancer Function - Allocating MU's The Load balancer allows usage of groups in order to assign a pool of STAP(s) to a pool of MU. Guarantee that not every available MU can be assigned to a requesting STAP. Caveat: Have to define an STAP IP in some group prior to installing it on the DB server Upon receiving an MU allocation request from STAP, the load balancer will: 1. Check if it's assigned to some group of MU. 2. If it find such a group, it will look in the load map for the least loaded MU in that group. 3. Otherwise, the first available MU in the load map will be allocated. Once an MU is allocated, the load balancer caches the last allocated MU for the requesting STAP. If an additional request(s) comes from the same STAP, the load balancer will allocate a different MU (if there is one). Allocated MU's are re-positioned at the end of Load Map bucket queue. If the allocated MU is of LOAD_LEVEL 2 or 3 ( Medium/High Load ), an event is going to be recorded in GDM_EXCEPTION table. 18

19 STAP Load Balancer GRDAPI Assign MU group to an STAP group grdapi assign_stap_load_balancer_mu_tap_group mugroupname=<mu group> stapgroupname=<stap group> Get Current Load Map grdapi get_stap_load_balancer_current_load_map Get Load Balancer parameters grdapi get_stap_load_balancer_params Set load balancer parameter grdapi set_stap_load_balancer_param paramname=<param name> paramvalue=<param value> Un-assign STAP group from an MU group grdapi unassign_stap_load_balancer_mu_tap_group mugroupname=<mu group> stapgroupname=<stap group> 19

20 STAP Load Balancer GUI Associating Existing Groups All Groups of type 'STAP' will be automatically displayed All Groups of type 'Managed Unit' will be automatically displayed 20

21 STAP Load Balancer GUI Associating New STAP Group A List of available STAP hosts will be automatically available for selection. 21

22 STAP Load Balancer GUI Associating New MU Group All the managed units hosts will be automatically displayed 22

23 STAP Load Balancer GUI Associating completion 23

24 STAP Load Balancer GUI Disassociating Groups 24

25 InfoSphere Guardium v9.5 Quick Search Updates 25

26 Quick Search Infrastructure Update Continue to improve Quick Search Functionality in Collector environment, change infrastructure from Lucene to Solr Benefits : Real Time Distributed Search Performance Improvements Built-in Analytics functions Caveat : This engine change may consume extra memory (Requires 24GB of RAM then 16 GB previously) Investigation Dashboard Technology Preview Provides interrelated charts that help reveal patterns, anomalies, and relationships across your data Best-practice view includes data source-to-user behavior, data source-by-time behavior, data source-to-source program behavior, and other essential relationships. 26

27 Enhancing Quick Search for Enterprise Wide Scope Coming soon Search scope spanning Central Manager controlled environment No need to understand or concern about topology, aggregation, or load balancing schemes Search requests are sent to all nodes, results are gathered, consolidated, and sorted according to the request, and then the results are displayed to the user centrally Leverage familiar Quick Search glass Real-time Forensic New additional Investigation Dashboard Pivot like facility to extract data activity insights focus on any specific context :specific data source, user, date, etc reveal patterns, anomalies, and relationships across your data best-practice default views: data source-to-user behavior data source-by-time behavior data source-to-source program behavior other essential relationships. New topology navigation facilitator Narrowing of scope for search (local, distributed, selected sub-set) 27

28 Quick Search Enterprise scope 28

29 Topology Navigation 29

30 Active Filters in Quick Search 30

31 Investigation Dashboard Overview Business Goals Ability to grasp high-level view in a multi-dimensonal environment Quick way to inspect different aspects of a forensic case Browsing audit data related to a specific context Investigation Dashboard Solution A dashboard containing a combination of interactive heat maps Leverage fast indexing engine in Quick Search Each heat map offers two dimensional view, related to other two dimensional views Each change filters the other charts to drill down on a specific case Highly configurable 31

32 Investigation Dashboard X-axis contains DB users Y axis contains databases Intersection shows usage of DB by each user Color depth represents Intensity of usage Hover cells for details Click for interactive filtering 32

33 InfoSphere Guardium v9.5 Outlier Detection updates 33

34 Outlier Detection Enhancements related to : New Features New scorer Performance Improvement User clustering Multi-Threaded training New cleaning mechanism for old data from the internal MySQL tables Fixes Improved anomaly scoring Filtering inputs according to the filtering patterns specified by the user in the GUI Inputs collected during training are now analyzed 34

35 InfoSphere Guardium v9.5 Upgrade paths 35

36 Enterprise Upgrade Strategy Transition to Guardium V9.0p500 GPU Target 32-bit 32-bit 32-bit 64-bit 64-bit 64-bit Source V8.2 V8.2 to V9.0p150 bundle patch (IBM Fix Central) + V9.0p200 (or later) 32-bit GPU patch Rebuild with V9.0p200 (or later) 64-bit ISO (IBM Passport Advantage) N/A V9.0 (GA) V9.0p200 (or later) 32-bit GPU patch (IBM Fix Central) Rebuild with V9.0p200 (or later) 64-bit ISO (IBM Passport Advantage) N/A V9.0p02 V9.0p200 (or later) 32-bit GPU patch (IBM Fix Central) Rebuild with V9.0p200 (or later) 64-bit ISO (IBM Passport Advantage) N/A V9.0p50 V9.0p200 (or later) 32-bit GPU patch (IBM Fix Central) Rebuild with V9.0p200 (or later) 64-bit ISO (IBM Passport Advantage) V9.0p200 (or later) 64-bit GPU patch (IBM Fix Central) >V9.0p300 V9.0p300 (or later) 32-bit GPU patch (IBM Fix Central) Rebuild with V9.0p300 (or later) 64-bit ISO (IBM Passport Advantage) V9.0p300 (or later) 64-bit GPU patch (IBM Fix Central) Newly built appliance Install V9.0p180 32bit ISO + V9.0p200 (or later) 32-bit GPU patch N/A V9.0p200 (or later) 64-bit GPU patch (IBM Fix Central) 36

37 Enterprise Upgrade Strategy Upgrading 64-bit system V9.1p bit V9.5p500 (or later) 64-bit Central Manager level Upgrade Aggregators level Upgrade Collectors level Upgrade Guardium agents level Live Update 37

38 How to Access p500 GPU External link for information on Guardium p500 GPU Make the following selections on Fix Central: Product Group: Product: Information Management InfoSphere Guardium Installed Version: 9.0/9.5 Platform: Heading: Linux Appliance Patch (GPU and Ad-hoc) Click "Continue", then select "Browse for fixes" and click "Continue" again. 38

39 Search 39

40 Search Infrastructure Update Enterprise (and local) Search is based on Apache Solr A widely used, highly scalable, open source enterprise search platform from Apache. Solr runs as a separate web application under tomcat. Data is indexed and searched on shards. A zookeeper is responsible for distributing the indexing and the search queries to the relevant shards. Zookeeper runs on CM in a managed environment. 40

41 Data Flow On a Guardium collector, data is extracted every 2 minutes using the same mechanism (datamart) that was used in previous Quick Search. This data is indexed into 5 indexes. Indexing is performed through the zookeeper Note that if CM is down indexing is not performed. The actual index is local to the collector that collects the data. Old data is purged using the Purge Object mechanism, default age is 3 days. CLI command can be used to modify that age. 41

42 Transitioning to new Search On upgrade of a Guardium appliance to GPU500, if Quick Search is enabled, hardware requirements for Enterprise Search are met and old (Lucene) indexes are present there is a transition period. To prevent data loss in the transition period, data is indexed on both the old engine and the new one. Searches are performed using old engine. A clear message is displayed stating "Upcoming new search options, in ${date}." The data is calculated to be the date when old index data is purged and all data is indexed by new engine. 42

43 Hardware Requirements Enterprise Search is not supported on 32 bit Collectors. A 32 bit CM can be the zookeeper when it manages 64 bit collectors as long as it complies with the following HW requirements. The hardware requirements for Enterprise Search are at least 4 CPU cores and 24 Gig RAM on a collector On a CM 4 CPU cores and 24 Gig RAM are required. 43

44 Ports Solr uses port 8983 for communications. This port must be bidirectionally open between CM and MUs. The ports are opened in our internal firewall on registration to CM and closed on un-registration. The port is closed to the world by default to preserve the security of our system. search data. 44

45 CLI commands cli>grdapi enable_quick_search if hardware requirements are met then new search engine is enabled. cli>grdapi disable_quick_search Quick Search is disabled. A new optional parameter all if api function is invoked on CM and all=true then Quick Search is disabled on all managed units. 45

46 DEMO 46

47 Dziękuję Gracias Spanish Traditional Chinese Polish Thai Merci French Russian Obrigado Brazilian Portuguese Danke German Arabic Tack Swedish Simplified Chinese Japanese Grazie Italian 47