DomainTools App for QRadar

Transcription

1 DomainTools App for QRadar App Startup Guide for Version Updated November 1, 2017 Table of Contents DomainTools App for QRadar... 1 App Features... 2 Prerequisites... 3 Data Source Identification... 3 Data Source FQDN Field... 3 App Configuration... 4 QRadar User Account... 4 App Settings... 4 Log Sources... 5 App Log... 5 Reference Data... 6 Managing Reference Data... 6 DomainTools Reference Data Collections... 7 Sample AQL... 9 DomainTools App Area (c) 2017 DomainTools LLC 1

2 App Features The DomainTools App for QRadar populates reference data with DomainTools domain profile and risk scores for domain names observed in QRadar events. It also provides a DomainTools app area to research a single domain name to uncover domain ownership profiles, risk scores, and more. Key capabilities enabled by the app include: Create offenses using DomainTools proprietary proximity- based domain risk scores Investigate domain names in- context, without leaving QRadar Target threat hunting at key aspects of a domain name s registration profile (c) 2017 DomainTools LLC 2

3 Prerequisites Data Source Identification Before installing the app, first identify which data source(s) in your QRadar instance contain domain names. DomainTools data works best with web proxy log data, because the domain names are easy to extract, and the web traffic captures most of the interactions between end- user workstations on your network and potentially malicious domain names. Other less common but still effective log sources include DNS logs or logs from next- generation, layer 7 firewalls that also contain domain name data. Once you locate the list of data sources, take note of the log source names in QRadar. You will use it later when setting up the DomainTools app. Data Source FQDN Field For the DomainTools app to function optimally, your log source should provide a field that contains only a fully- qualified domain name, and if possible, it should be labeled FQDN. This documentation will assume the field name is FQDN unless otherwise noted. Here s why this is important. DomainTools provides Whois and risk scoring data on second- level domain names. Examples of a second- level domain names include domaintools.com, google.com, and bbc.co.uk. Most traffic on a network does not reference these second- level domains directly instead, logs will contain fully- qualified domain names (also known as FQDNs or hostnames) or even complete URLs. Examples of FQDNs include research.domaintools.com, or Those FQDNs must first be collapsed to only their domain name before a query is made to the DomainTools API to avoid making unnecessary requests. In most networks, this results in a 10x reduction in the volume of API queries, and it also improves performance by enabling effective caching. The task of extracting a second- level domain name from an FQDN or a complete URL is non- trivial, and cannot be performed effectively with regular expression matching. The optimal solution requires a list of domain extensions, and there are code libraries dedicated to solving the problem efficiently. QRadar does not provide a built- in mechanism to make that conversion, so the DomainTools app handles that for you. You may find it necessary to add a custom field to your data source to extract the FQDN from a URL or other unparsed field. Adding a custom field to a log source in QRadar is out of the scope of this documentation. (c) 2017 DomainTools LLC 3

4 App Configuration QRadar User Account The DomainTools app runs a process that queries your QRadar event logs for new events, finds domain names, and then populates reference sets with Whois and Risk Score data from DomainTools APIs. For this to work, the app needs a QRadar user account to sign in with and read those events. Create that account in QRadar, and then note the username and password so you can set that in the app settings page. App Settings Access the DomainTools App configuration page by first visiting the Admin settings page in QRadar, then scroll down to the DomainTools Configuration option. Click the DomainTools icon to open the settings page and enter the correct values for your environment. DomainTools application user name Password DomainTools host name API user name API user token Use HTTPS protocol to invoke DomainTools APIs Verify SSL certificate is used to invoke DomainTools APIs Max number of records to fetch from log source at a time. Max threshold value of reputation score User name of a QRadar user the app will use to read events and store reference data. Password for the QRadar user account. Must be set to api.domaintools.com DomainTools API username (contact your eval point of contact if you do not have an API username and API key) DomainTools API key. Whether to use SSL when accessing the DomainTools APIs. We strongly recommend setting this to false to get the most throughput and fastest response times from the server. API keys are still protected with HMAC signatures even when SSL is disabled. Some environments with SSL filtering require accepting an organization s CA, but that CA may not be loaded into the QRadar instance. Again, disable HTTPS queries whenever possible to avoid problems and improve throughput. Start with a value of 200 and adjust as needed. Domain names with a score higher than this threshold will be added to a special reference set. The score ranges from 0 to 100 with higher numbers indicating a riskier domain. (c) 2017 DomainTools LLC 4

5 Time interval to invoke the scheduler in minutes. After how many cycles the settings to be refreshed No. of records to be displayed in a page DomainTools recommends starting with a minimum value of 70. Set how frequently the job will run that extracts log data. Start with 10 minutes and adjust as needed. App settings are cached between successive runs of the enrichment job and are periodically refereshed. Start with a value of 1 while you are adjusting the settings, then increase to at least 4 for best performance. Adjust pagination for pivot data returned on the domain profile page. Start with 50 and adjust accordingly. Log Sources Access the DomainTools app configuration page, then click on Delete Log Source. The app installs with an example log source that you should remove once you familiarize yourself with the expected values for the log source name and domain column name. Next, click on Add Log Source to add one or more log sources that contain domain names (see Prerequisites above). Ensure the values in the fields match the data source name and column name, then click the Submit button. Repeat for as many data sources as you need. App Log Once the app is configured, the DomainTools App will run a job at the interval specified in the settings, query the logs, and fetch DomainTools data to populate in reference sets. A QRadar administrator can access application logs on the QRadar server to monitor this process and provide debugging information to DomainTools if problems arise. The logs are stored in one of these folders: /store/docker/vfs/dir/[container_id] /store/docker/containers/[container_id] The container_id portion of the path is not a predicable value, so it will require visiting each directory to find the one with the DomainTools log files. The correct folder will have a dtstore.db file and a log directory navigate to the log directory to find the app.log file. If you have command line access to the server, this command can help you locate the folder more quickly than trial- and- error: find /store -maxdepth 4 -name "dtstore.db" (c) 2017 DomainTools LLC 5

6 Reference Data Managing Reference Data QRadar supports several reference data collection types, but it only provides a UI to manage the contents of reference sets. There is no option in the QRadar admin interface to view reference maps or reference tables, both of which are used extensively by the DomainTools app. The only way to confirm these reference data were created properly, and to view their contents, is to use the API. Fortunately, QRadar provides interactive API documentation under the Help menu. To view a list of reference maps: Go to "Help" > "Interactive API for Developers" Navigate to the 7.0 tree, down to /reference_data Click on /maps Scroll down through the page that appears on right and click "Try it now" The Response Body will list details on each active reference map To view the contents of a reference map: Go to "Help" > "Interactive API for Developers" Navigate to the 7.0 tree, down to /reference_data Expand the /maps node and click /{name} Scroll down through the page that appears on right and locate the parameters section Enter the name of the reference map in the name field and click "Try it now" The Response Body will list details on each active reference map (c) 2017 DomainTools LLC 6

7 DomainTools Reference Data Collections Name Type Usage dt_fqdn_to_domain Reference Set Contains key / value pairs mapping fully- qualified domain names (FQDNs) to their second- level domain name. Provide a FQDN as the key to obtain a domain name. This reference set is also used to manage caching in the DomainTools app. Log entries that already have an entry in this reference set for the value in their FQDN field will be excluded from the enrichment job. Use this field in a custom AQL query to create a domain name column that can be used to lookup risk score and Whois data. For example: SELECT REFERENCESET('dt_fqdn_to_domain',FQDN) AS domain_name dt_domains_risk_score Reference Set Contains key / value pairs mapping second- level domain names to a DomainTools risk score. Provide a domain name as the key. Use this field in a rule with custom AQL to create offenses when domain names exceed a threshold. For example: REFERENCESET('dt_domains_risk_score', REFERENCESET('dt_fqdn_to_domain',FQDN)) >= 70 dt_whois_details Reference Table Contains a set of columns with parsed Whois data, indexed by the second- level domain name. Columns names include: Registrant Country Registrant Name Registrant Org Registrant Phone Registrar Name Created Date Expired Date Updated Date Use this data to enrich log searches or to create custom AQL rules based on attributes in the Whois record of a domain name. For example, this rule could alert on domains registered at a specific registrar: (c) 2017 DomainTools LLC 7

8 REFERENCETABLE('dt_whois_details', 'Registrar Name', REFERENCESET('dt_fqdn_to_domain',FQDN) ) = 'Evil Registrar Inc.' (c) 2017 DomainTools LLC 8

9 Sample AQL This AQL may be used to enrich a log source that contains an FQDN in the FQDN column. Adjust the LOG_SOURCE_NAME value to match the name of your log source. SELECT starttime, LOGSOURCENAME(logsourceid), FQDN, REFERENCEMAP('dt_fqdn_to_domain',FQDN) AS domain, REFERENCEMAP('dt_domains_risk_score',domain) AS dt_risk_score, REFERENCETABLE('dt_whois_details','Registrant Country',domain) AS dt_reg_country, REFERENCETABLE('dt_whois_details','Registrant Name',domain) AS dt_reg_name, REFERENCETABLE('dt_whois_details','Registrant Org',domain) AS dt_reg_org, REFERENCETABLE('dt_whois_details','Registrant ',domain) AS dt_reg_ , REFERENCETABLE('dt_whois_details','Registrar Name',domain) AS dt_registrar, REFERENCETABLE('dt_whois_details','Created Date',domain) AS dt_create_date FROM events WHERE LOGSOURCENAME(logsourceid)= 'LOG_SOURCE_NAME' AND domain IS NOT NULL (c) 2017 DomainTools LLC 9

10 DomainTools App Area When the app is installed, a new tab will appear on the QRadar navigation menu labeled DomainTools. Access that tab to view a dashboard focused on key threat hunting and risk metrics. You can adjust threshold and parameters for the dashboard panels by clicking the pencil icon next to the panel titles. (c) 2017 DomainTools LLC 10

11 To investigate a specific domain name, click the "Search" tab near the top of the dashboard and enter a domain name in the search box. The app loads risk score and Whois information on a single domain name from the DomainTools API. You may also click these elements to view additional related domains using DomainTools Reverse Whois, Reverse IP and Reverse Name Server datasets: Registrant, abuse and admin addresses Registrant name on the Domain Profile tab IP address Name servers (c) 2017 DomainTools LLC 11