DomainTools for Splunk

Transcription

1 DomainTools for Splunk Installation Guide version 2.0 January 2018 Solution Overview The DomainTools Technology Add-On (TA) for Splunk populates a whois index with DomainTools Whois and Risk Score data for each domain name observed in Splunk events. The Whois index can be used in Splunk searches to find events that match attributes of the domain s registration profile, including registrant name, , or preferred registrar. The DomainTools Risk Score can also be a useful attribute for event triage and malicious domain detection. The DomainTools TA for Splunk also provides a GUI for threat hunting, and ad-hoc domain profile lookups. For Splunk Enterprise Security users, it raises Noteable Events on risky domains with an integration into the Adaptive Response framework.

2 Prerequisites Identify Your Data Source The DomainTools TA needs a data source that contains domain names to function properly. Typically, web proxy logs provide the best source of these domains, usually in a dest or URL field. Alternative data sources such as DNS logs or next-gen firewall logs can also work, but preparing them for use usually requires some data cleanup, which is beyond the scope of this guide. The DomainTools TA must be configured with your own query to help the DomainTools app find the right events. The search must include a field that has either urls or domains in it and that field must be named, domain. The default search query is provided for example only: search url=* rename url AS domain. For best results, review your source data carefully to ensure it contains as few IP addresses as possible. The DomainTools solution provides data on domain names, not IPs, so any IPs sent to the API will not return useful data and consume rate limited resources. If you use a staging environment to test new Splunk apps, ensure the same data source you plan to use in production is also available to the Splunk search heads in the test environment. If they are not available, the DomainTools solution will not work properly, and it will be difficult to gauge the value of that solution if it cannot operate at production scale. Validate Compatibility The DomainTools TA is compatible with the following versions: Splunk Enterprise versions 6.2+, 7.0+ Optionally, the most current version of Splunk s Enterprise Security App Ensure Connectivity The DomainTools solution needs Internet access to query the DomainTools API, obtain a TLD suffix list, and optionally, the list of Alexa top 1 million most trafficked Internet web sites. Please ensure the Splunk search head or search head cluster you plan to install the TA on has Internet access, or that it has the proper HTTP and HTTPS proxy configuration for your network setup and that any firewalls or traffic filters allow api.domaintools.com outbound traffic on ports 80 and 443. Configuring Splunk proxy access is beyond the scope of this document; contact Splunk for help, or tell your DomainTools rep if you have unique network security requirements that cannot be fixed by proxy configuration alone.

3 Get API Key You will need a DomainTools Enterprise API username and API key to configure the app. If your org does not have access to the DomainTools API, contact your DomainTools representative to get a key, or EnterpriseSupport@domaintools.com. Obtain the TA You need the DomainTools TA to make the solution work. This certified app is available on Splunkbase at this URL: NOTE: DomainTools publishes pre-release versions of the app as a beta for customers interested in early access to the latest features. These beta apps are not certified by Splunk and should be tested in a lab environment before deploying to production. You can install the beta version of the DomainTools TA at this URL: Be sure to download the version of the app listed at the top of this document, or the version recommended to you by your DomainTools rep. If the version you need to install is not listed on Splunkbase, contact DomainTools for an alternative download location. Prepare Whois Index Most customers deploy the DomainTools Splunk Solution in a distributed environment, with multiple indexers and search heads. The DomainTools TA is fully compatible with this setup, but it is important to configure the environment properly to ensure the TA works as expected. Each indexer in a Splunk cluster must have the whois index configured on them. If you already use Splunk Enterprise Security, or have installed an earlier version of the DomainTools app, it is likely the index already exists - check first before continuing. If the whois index does not exist, you need to create it using the Splunk web UI or CLI. Creating indexes is beyond the scope of this document - consult the Splunk documentation or your Splunk representative for help.

4 App Setup Identify the deployment architecture that matches the desired installation using the diagrams below and follow the appropriate installation steps. Stand-Alone Deployment & Non-Clustered Search Head Deployment In a non-clustered search head environment, installation will occur on a single search head. Configuration Specific Installation Steps 1. Install TA-DomainTools on the Search Head using the web UI or CLI - see the Splunk documentation for details. 2. Configure API Credentials in installation.

5 Distributed Search - Clustered Search Head Deployment In a clustered search head environment, installation will typically be on the Deployer. IMPORTANT: If you are running the Splunk Enterprise Security (ES) app on some of your search heads, but not all of them, this is not the correct deployment for your environment. See the section on hybrid ES environments for more details. Configuration Specific Installation Steps 1. Install TA-DomainTools for Search Head deployment. Typically the application would be installed on the Deployer. Install using the web UI or CLI - see the Splunk documentation for details. 2. Configure API Credentials in installation. 3. Ensure Search Head cluster uses the same encryption key as the Deployer and then copy the app's installation to the deployment directory. 4. Deploy the application bundle to the Search Head cluster.

6 Distributed Search - Hybrid Enterprise Security Deployments In hybrid ES / non-es deployments, where some search heads have Enterprise Security but some do not, it is important that the DomainTools TA is not installed in both clusters. The DomainTools TA includes a populating search that uses the key/value store in Splunk to avoid making duplicate queries to the DomainTools API. Hybrid environments are likely to have multiple key/value stores. Selecting only one cluster, or the one search running Splunk ES, is a good way to avoid issues. In most environments, the DomainTools TA should be installed on the search head running ES, or the search head cluster where ES is installed.

7 Configuration Specific Installation Steps 1. Install TA-DomainTools on the Search Head with ES to enable ES functionality. Install using the web UI or CLI - see the Splunk documentation for details. 2. Configure API Credentials in installation. 3. If installed to clustered search heads, reference the Distributed Search - Clustered Search Head Deployment diagram and installation steps. DomainTools TA Configuration 1. Splunk will restart after the TA is installed. 2. Enter your API username and API key in the fields on the setup screen. 3. After setup, proceed with any additional installation steps required per the diagrams above. 4. Locate the DomainTools for Splunk app in the Splunk UI and open it to view the app s default page: DomainTools Threat Hunting dashboard. 5. Click Configure on the toolbar, then DomainTools App Configuration 6. The app will verify your API credentials and list the products you have access to. Ensure you have at least parsed-whois and either reputation or risk-score. 7. Click Search Configuration on the left and ensure Enable and Build Custom Search is selected. 8. Click on edit the search manually. 9. Enter the appropriate Splunk search string for your data source. Ensure the output of the search contains a field named domain which contains urls or domains. The default search string is search url=* rename url AS domain. 10. Click Settings on the left and note the status of the Enable Populating Search setting. a. If you use separate indexers, ensure the whois index exists on the indexers before you activate the populating search. Testing & Validation The quickest way to validate the solution is installed correctly is to perform an ad-hoc query on a domain name. Access the DomainTools for Splunk App in Splunk (usually under the Apps menu), and on the main page that appears, enter a domain name such as domaintools.com in the top search field and click Whois Lookup. If no errors appear, and you get data in most or all of the panels, the API credentials are working property. The next thing to test is whether the populating search is working correctly. You can check the job status for errors under the Search, or just wait about 15 minutes, and then try a search in Splunk for index=whois with a narrow time window. You should see a list of domain

8 ownership data and risk scores. If the index is empty, contact your DomainTools representative for help. Further Information We re here to help! For more information, please visit our website contact your DomainTools representative or