IBM Security QRadar Deployment Intelligence app IBM

Transcription

1 IBM Security QRadar Deployment Intelligence app IBM

2 ii IBM Security QRadar Deployment Intelligence app

3 Contents QRadar Deployment Intelligence app.. 1 Installing the QRadar Deployment Intelligence app. 1 Creating an authorized service token for QRadar Deployment Intelligence What's new in the QRadar Deployment Intelligence app Configuring QDI Ariel search priority Configuring graph time window Configuring graph data points Deployment Overview Advanced Health Querying Use cases for the QRadar Deployment Intelligence app Tuning QRadar Deployment Intelligence app Troubleshooting QRadar Deployment Intelligence. 11 iii

4 iv IBM Security QRadar Deployment Intelligence app

5 QRadar Deployment Intelligence app Use the QRadar Deployment Intelligence app to monitor the health of your QRadar deployment. QDI consolidates historical data on a per-host basis of: status, up-time, notifications, event and flow rates, system performance metrics, QRadar specific metrics and more. It is fully interactive: at first it shows an overview of the deployment and then the user can drill down and investigate specific hosts to see detailed health and status information at the application, middleware, and system level. Installing the QRadar Deployment Intelligence app Use the IBM Security QRadar Extensions Management tool to install the IBM Security QRadar Deployment Intelligence app on your QRadar Console. Before you begin Verify that you have IBM Security QRadar V7.2.8 or later installed. About this task You must have an IBM ID to access the App Exchange ( exchange.xforce.ibmcloud.com/) and download the app. You can register for an IBM ID at IBM id registration ( Note: The installation of apps does not void your IBM warranty for the QRadar product. Procedure 1. Open the Admin settings: a. In IBM Security QRadar V7.3.0 or earlier, click the Admin tab. b. In IBM Security QRadar V7.3.1, click the navigation menu ( ), and then click Admin to open the admin tab. 2. In the Extension Management window, click Add and select the QDI app archive to upload to the console. 3. Select the Install Immediately check box. Note: You might have to wait several minutes before you app becomes active. What to do next When the installation is complete, clear your browser cache and refresh the browser window before you use the app. 1

6 Creating an authorized service token for QRadar Deployment Intelligence QRadar Deployment Intelligence needs an Admin level SEC token to access REST API endpoints and to perform Ariel searches. After the app is installed, you are redirected to landing page that leads to the security token configuration page. Alternatively, the security token configuration page can be found under Deployment Intelligence Token Configuration in Admin settings in QRadar. Procedure 1. Open the Admin settings:. a. In IBM Security QRadar V7.3.0 or earlier, click the Admin tab. b. In IBM Security QRadar V7.3.1, click the navigation menu ( ), and then click Admin to open the admin tab. 2. Select Authorized Services in the User Management section. 3. In the Authorized Services window, click Add Authorized Service. 4. Add the relevant information in the following fields and click Create Service: a. In the Service Name field, type a name, up to 255 characters in length, for this authorized service. b. From the User Role list, select Admin. c. From the Security Profile list, select Admin. d. In the Expiry Date list, type or select a date that you want this service to expire. If you want uninterrupted data collection, select No Expiry. e. Click Create Service. 5. Click the row that contains the service you created, select and copy the token string from the Selected Token field in the menu bar, and close the Manage Authorized Services window. 6. On the Admin tab, click Deploy Changes. 7. On the app landing page click Configure. 8. Paste your SEC token in the pop-up and click Submit. Results After you submit your SEC token, QDI sets up the initial database and host information schemas from initial API calls and Ariel searches. QDI runs Daemon threads in the background to collect information about your deployment. After the initial information about the deployment is collected, the app redirects you to the QDI dashboard. Related concepts: IBM Security QRadar Operations app configuration settings Configure settings and preferences for the Operations app in the QRadar Operations window on the Admin tab. What's new in the QRadar Deployment Intelligence app Learn about the new features in each IBM Security QRadar Deployment Intelligence app release. 2 IBM Security QRadar Deployment Intelligence app

7 Version v Deployment overview that shows a consolidated view of the deployment health across all the hosts. v Enhanced chart widgets to allow customization. A chart widget can be closed and then added again. The time window of the metrics in a chart can be varied with a maximum of 24 hours. v Advanced tab in host-specific views that shows QRadar component-specific metrics. Configuring QDI Ariel search priority QRadar Deployment Intelligence runs searches by using the QRadar Ariel search API to retrieve health metrics data from QRadar. The searches are run every minute and the data is cached within the app for faster UI access. About this task Use Ariel Search Priority to balance application data retrieval performance and the application's impact on QRadar searches. High priority queries result in faster data retrieval, but have a negative impact on other running searches. Low priority queries result in the least impact of QRadar search performance, but slow down data availability in the app. Normal priority is default. Procedure 1. In the QDI app, click Configure Graphs on the header and go to the Ariel Search Priority section. 2. Select the search priority from the list and click Submit. Configuring graph time window Many of the metrics in QRadar Deployment Intelligence are visualized as a line or bar charts in the app dashboard. The time interval of the charts can be configured for better usability. About this task Configure the graph time window to set how long, in minutes that you want each time interval to be. Longer intervals show historic performance, while you can use shorter intervals to monitor real-time performance. Note: The graph time window is used to globally set the time window for all charts. Individual chart values can be set in the Chart widget. Procedure 1. In the QDI app, click Configure Graphs and go to the Change graph time window for all charts section. 2. Select the time window from the list and click Submit. QRadar Deployment Intelligence app 3

8 Configuring graph data points You can configure the number of data points that QRadar Deployment Intelligence displays on the graph. The number of data points on the chart are limited to this value, which is used as a threshold to sample the dataset for highly ranked data points to show the graph pattern. The higher the number of data points, the higher the chart resolution and the browser memory usage. Procedure Deployment Overview 1. In the QDI app, click Configure Graphs and navigate to the Number of data points to show on graph section.. 2. Enter the number of data points and click Submit. The Deployment Overview page in QRadar Deployment Intelligence shows the cumulative health details of the entire deployment. This gives a high-level view of the system health for monitoring and as a starting point for a top down analysis. The Deployment Overview page consists of two sections: v Application view v Performance view Application view The Application view contains dashboards that display QRadar metrics and statistics at the application level. Table 1. Application view Widget Deployment Health Notifications Status Feed Description The Deployment Health widget is a map of the host's health in the deployment. The health of the host is calculated based on two factors in the following priority order: 1. The status of the host. 2. The severity of notifications in the host. The Notifications widget displays QRadar notifications that are sorted by most recent occurrence. The Status Feed widget displays the host status changes that happened in the deployment in the last 7 days. The status feed is sorted in order of most recent occurrence. 4 IBM Security QRadar Deployment Intelligence app

9 Table 1. Application view (continued) Widget Host Status Overview User Activity Description The Host Status Overview widget shows the count of hosts in different QRadar states. The default states, which are listed as chart labels, are: v Active v Standby v Online v Online Note: If any hosts falls in a QRadar state other the default, they are dynamically loaded in the chart. The User Activity widget displays a high-level view of the QRadar application load. Three major activities are listed: User Session Count: The count of users and logged in sessions in QRadar. This metric is only available in QRadar v or later. Running Queries Count: The number of Ariel search queries running in QRadar. This metric is sampled every minute and does not account for short transient queries that run for less than 1 minute. Top Users Top Users By API Activity Cursor: the count of the Ariel search cursors that are present on the system. This widget displays search activity statistics for reach user. Two views can be used: v Search Activity by Users v Search Activity by User Groups The following search activity statistics are displayed: v Search Count v Running Count v Error Count v Canceled Count v Maximum Duration v Average Duration This widget shows QRadar REST API activity statistics for each user. Two views can be used: v API Activity by Users v API Activity by User Groups The following API activity statistics are displayed: v Successful Count v Failed Count QRadar Deployment Intelligence app 5

10 Table 1. Application view (continued) Widget License and Event/Flow Rate Top/Bottom N EPS/FPS License Utilization Top/Bottom N EPS/FPS Top/Bottom N Security Data Metrics Description This widget displays the total event and flow rate across all the hosts in the deployment. In QRadar V7.3.1 and later, this widget also displays the license limit and the license that is allocated to hosts. This widget displays the top and bottom hosts in the deployment by license utilization. This widget displays the top and bottom hosts in the deployment by events per second and flows per second. This widget displays the top and bottom security data elements across security artifacts in QRadar. The following security artifacts are analyzed: v Stored Events by Log Sources v Stored Events by Log Source Type v Unknown Events by Log Sources v Unknown Events by Log Source Type v Log Sources v Errored Log Sources v Rule Responses v Offense Updates v Asset, Vulnerabilities, Offense, Log Source, and Activity Rules counts Performance view The Performance view contains dashboards that display QRadar metrics and statistics at the pipeline and operating system level. Table 2. Performance view Widget Top/Bottom N Pipeline Utilization, Saturation, and Errors/Drops Expensive QRadar Application Artifacts Description This group of charts displays the top and bottom hosts by QRadar event pipeline utilization, saturation and drops. This widget shows the most recent expensive QRadar application artifact. The following application artifacts are monitored: v Expensive Custom Rules v Expensive Custom Properties v Expensive Log Sources 6 IBM Security QRadar Deployment Intelligence app

11 Table 2. Performance view (continued) Widget Top/Bottom N Hosts by System Metrics Deployment System Metric Averages Description This widget displays the top and bottom hosts in the deployment that are identified by different operating system metrics. The following metrics are used: v CPU Utilization v Memory Utilization v Disk I/O Throughput v Disk I/O Read Rate v Disk I/O Write Rate v Disk Read IOPS v Disk Write IOPS v Disk Await v Disk Space Utilization v Network Throughput v Network Read Throughput v Network Write Throughput These charts display the averages of system metrics across all hosts in the deployment. The following metrics are used: v CPU v Memory v IO Throughput v Network Throughput Host-specific view The Host-specific view displays metrics specific to a QRadar host. Each host has two subviews: v System view v Advanced view Table 3. System view Widget Status Uptime Description This widget is a pie chart that displays the percentage of time a host spends in different QRadar system states. The monitoring time that is taken for calculating this metric is shown in the title bar. Two pie charts are used for HA hosts, one for the HA active, and another for the HA standby host. QRadar Deployment Intelligence app 7

12 Table 3. System view (continued) Widget Host Information Notifications Process Monitor Component Status Feed Pipeline Utilization, Saturation, and Drops System Metrics Description This widget displays information about the host, including: v Private IP v Public IP v Virtual IP v HA Pair IP v Hostname v Serial Number v Appliance Type v Network Interface v HA Configuration v Encryption The Notifications widget displays QRadar notifications that are sorted by most recent occurrence and filtered for the host that you are monitoring. This widget displays the list of QRadar JVM processes, their status, and the last outage duration and start time. This widget displays changes in the status of QRadar JVM processes. This group of charts displays utilization, saturation and drops in the pipeline specific to the host. v Event Drop Count in CRE, DSM Parsing, Syslog v Event Collector Utilization in Flow Governor, Event Throttle, EC TCP TO EP Queues v Event Collector Saturation in Flow Governor, Event Throttle, EC TCP TO EP Queues This group of charts displays various operating system metrics specific to the host. The following metrics are used: v CPU Utilization v Load Average v Memory Utilization v Disk Usage v Disk IO Throughput v Disk IOPS v Disk Await v Disk Utilization v Network Usage Transmitted v Network Usage Received v Network Connection Stats 8 IBM Security QRadar Deployment Intelligence app

13 Table 4. Advanced view Category JVM Metrics ECS-EP Metrics ECS-EP Metrics Accumulator Metrics Data Node Metrics Metrics v Heap Usage v Process CPU Utilization v Garbage Collection Count v Garbage Collection Time v Total Compacts v Thread Count v Class Loading v Direct Memory v Mapped Memory v CRE CPU Utilization v CRE Thread States v CRE CPU Utilization v CRE Thread States v Ariel Writer CPU Utilization v Ariel Writer Thread States v Average Event Record Size v Average Event Payload Size v Preprocessor CPU Utilization v Preprocessor Thread States v Aggregation CPU Utilization v Aggregation Thread States v Accumulation Time v Average Event Record Size v Average Event Payload Size Advanced Health Querying You can use the Advanced Health Querying page to query for a health metric during any given time interval. Multiple charts can be loaded, stacked, closed and added. The data from these charts can be exported as CSV. To open the Advance Health Querying page, click Advanced Health Querying on the application header bar. The following fields are required to start a health query: Table 5. Advanced Health Querying fields Field Metric Name Hostname Metric Element Component Start Date Start Time QRadar Deployment Intelligence app 9

14 Table 5. Advanced Health Querying fields (continued) Field End Date End Time Note: Multiple items can be selected from the Hostname, Metric Element, and Component fields, with the multiplicity allowed only in one of the three fields at the same time. Use cases for the QRadar Deployment Intelligence app QRadar Deployment Intelligence displays Health monitoring, problem prevention, and troubleshooting QRadar Deployment Intelligence displays various metrics from QRadar at the application, middleware, and system level. These metrics are displayed as graphs. You can use these graphs to observe the health and functions of various QRadar components. You can monitor changes in application load across various users in real time, and you can also monitor the use of QRadar components. You can monitor components that are saturated, or users that are performing heavy load on QRadar. You can use the app to observe how the system operates at a low level in relation to the application load. You can use the overall view of this information in QDI to prevent performance and health-related QRadar outages, such as license oversubscription, slow searches, API bottlenecks, and memory issues. When an outage occurs, QDI performs a top-down analysis on the system with real-time graphs and Advanced Health Querying. Using QDI to perform this analysis is much faster than a traditional back-end investigation. Sizing QRadar You can use the various metrics that QRadar Deployment Intelligence monitors to effectively and properly size QRadar. The Top/Bottom N graphs display overused and underused systems in the deployment, which can help you to properly balance load across the QRadar deployment. You can use QDI to predict the future QRadar load by observing current trends, which can help you to be in proactive in sizing your deployment. Tuning QRadar Deployment Intelligence app You can improve the performance of your IBM Security QRadar Deployment Intelligence app by creating indexes in QRadar on Health Metrics log source properties. Procedure 1. Open the Admin settings:in IBM Security QRadar V7.3.0 or earlier, click the Admin tab.in IBM Security QRadar V7.3.1, click the navigation menu ( ), and then click Admin to open the admin tab. 2. In the System Configuration section, click the Index Management icon. 3. On the Index Management page, in the search box, enter Metric ID Category. 4. Select Metric ID Category Enable Index 10 IBM Security QRadar Deployment Intelligence app

15 Troubleshooting QRadar Deployment Intelligence A common issue in QRadar Deployment Intelligence is that the app does not show any health related data. This issue can occur for several reasons. 1. Ariel Server or Ariel Server API is not running. One way to identify this issue is by running a sample Ariel query using the Ariel API: select metric_id, value from events where LOGSOURCENAME(logsourceid) ilike %%metric_id%% last 10 minutes 2. If the query runs properly, check the resulting data from the query. If the query returns no data, then there is a possibility that the health metric events are not generated or there are issues in the pipeline to process the Health Metric Events. In this case, the QRadar has a Health Metric status on the app header that shows the status of Health Metric status. If there is a Health Metric outage, it could be an QRadar issue and Customer Support should be contacted. 3. If the Ariel query runs properly and returns proper data, yet app doesn't show graphs, this could be a QDI app issue in the polling process that gets the API data from QRadar. Use /store/log/poll.log log leas a starting point of investigation and further communication to Customer Support. QRadar Deployment Intelligence app 11