ActiveTrust Cloud Threats API

Transcription

1 QUICK START GUIDE ActiveTrust Cloud Threats API March Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 1 of 19

2 Contents Overview... 3 Prerequisites... 3 ActiveTrust Cloud Threats API... 3 API Calls... 3 Authentication... 3 Errors... 4 Request Parameters... 4 Threats API response... 5 Use Case Examples... 7 Sample script to pull and redistribute events... 7 atc_threats.py configuration... 7 Infoblox ActiveTrust Cloud Splunk application... 8 Integration with Carbon Black Server... 8 Integrations via Syslog/UDP... 9 Integration with Infoblox Reporting... 9 Integration with Splunk Integration with McAfee ESM References Annex A Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 2 of 19

3 Overview The ActiveTrust Cloud Threats API allows you to make RESTful API calls to gather DNS security data from ActiveTrust Cloud for SIEM (Security Information and Event Management) purposes. Based on your business needs, you can configure a SIEM system in your network to collect the DNS security data so you can filter the data and create custom reports. The document contains an overview and examples of using ActiveTrust Cloud Threats API. Prerequisites ActiveTrust Cloud subscription and an API key (an authorization token) are required to obtain access to the API and events. ActiveTrust Cloud Threats API API Calls The Threats API uses HTTPS (HTTP over SSL/TLS) as the transport mechanism. It only supports a single API call (i.e. dns_event) which allows you to obtain a list of blocked DNS requests. Table 1) API Calls HTTP Method URI Description GET Get list of the blocked DNS requests Authentication An API key (or authorization token) is used to authenticate requests to the portal. You can obtain an API key on the Cloud Service Portal (CSP, in the user's preferences. To obtain the authentication token: 1. Log in to the Cloud Services Portal. 2. At the upper right-hand corner, click your user name and select "User Preferences". 3. On the User Preferences page, click "Show" under "API key", and copy the API key. To authenticate requests, you should pass an HTTP "Authorization" header with the API key, and the prefix word "Token". For example. Command 1: Get RPZ hits from " :09:00 UTC" to " :00:00 UTC" in CEF format curl -H "Authorization: Token ac00eb99a55e4640be1eb899341cbd82" " Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 3 of 19

4 Command 2: Get all blocked requests to "example.com" for the last 5 minutes in JSON format t1=`date +%s`;t0=`expr $t1-300`;curl -H "Authorization: Token ac00eb99a55e4640be1eb899341cbd82" " Errors If the request was not authorized, the server returns a "401" response code and an error in JSON (regardless of the output format). Error Response 1: Unauthorized request HTTP/ Unauthorized Server: nginx/ Date: Mon, 29 Jan :19:38 GMT Content-Type: text/plain; charset=utf-8 Content-Length: 74 Connection: keep-alive X-Content-Type-Options: nosniff {"status_code": "401", "status_detail": {"errors": [ "Not authorized" ]}} If a request was executed successfully then the server will respond with a 200" response code. If the dataset is empty the response will not contain any data, the CEF format will be blank and the JSON format will contain an empty array. Error Response 2: An empty response in CEF format (including HTTP headers) HTTP/ OK Server: nginx/ Date: Mon, 29 Jan :36:00 GMT Content-Type: text/plain; charset=utf-8 Content-Length: 0 Connection: keep-alive Error Response 3: An empty response in JSON format (including HTTP headers) HTTP/ OK Server: nginx/ Date: Mon, 29 Jan :36:08 GMT Content-Type: text/plain; charset=utf-8 Content-Length: 36 Connection: keep-alive {"status_code": "200", "result": []} It is recommended to check the HTTP response code for the request result. Request Parameters It is possible for an API call to return a huge dataset. Currently, Infoblox limits the result dataset to records. It is recommended to narrow the search and execute it frequently to avoid polling a big dataset. The following table explains the supported parameters you can use in your API requests to filter those results Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 4 of 19

5 Table 2) Request Parameters Parameter Description Example t0 Start time in seconds (Unix timestamp). Query records on or after the start time. (required) t1 End time in seconds (Unix timestamp). Obtain records before (but not on) the end time. (required) source Enter rpz for RPZ events or rpz analytic for detection (threat insight) events. If omitted, data from both are provided. (optional) network Source Network (NAT case) / Some Network DFP name (DFP) / AT Endpoint (optional) qip Device IP that sent the DNS query (optional) qname Queried IP/Domain (optional) example.com policy_name Policy name (optional) Policy name threat_level Threat Level (optional) HIGH threat_class Threat category (optional) MalwareDownload _limit Number of rows to return 2 (optional) _offset Offset of first row to return, for 42 paging (optional) _format JSON (default) or CEF (Optional) cef Note: There is a short delay between a query being blocked and that information being available via Threats API. To avoid losing events it is recommended to not use a current timestamp as t1. Threats API response The API supports CEF and JSON (default) output formats. Events in JSON are listed in the "result" object array. In CEF format every event is located on a separate line. "New line" character is used as a delimiter. Response 1: Response in JSON Format { "status_code": "200", "result": [ { "qname": "pqgbrfxiwcyvdf.net.", "device": " ", "user": "unknown", "severity": "HIGH", "tclass": "MalwareC2", "tproperty": "CryptoLocker", "timestamp": "11e f cd79196", "event_time": " T17:28:01.000Z", "confidence": "HIGH", "rcode": "NXDOMAIN", 2018 Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 5 of 19

6 "rdata": "", "country": "unknown", "qtype": "A", "network": "Germany/Nürnberg", "feed_name": "antimalware", "feed_type": "DOMAIN", "policy_name": "Default Global Policy", "rip": "" } ] } Response 2: Response in CEF Format CEF:0 Infoblox ATC 2.0 dns_event 10 InfobloxAtcTimestamp= T17:28:01.000Z InfobloxAtcDevice= InfobloxAtcThreatClass=MalwareC2 InfobloxAtcThreatProperty=CryptoLocker InfobloxAtcRcode=NXDOMAIN InfobloxAtcUser=unknown InfobloxAtcNetwork=Germany/Nürnberg InfobloxAtcQname=pqgbrfxiwcyvdf.net. InfobloxAtcQtype=A InfobloxAtcConfidence=HIGH InfobloxAtcFeedName=antimalware InfobloxAtcFeedType=DOMAIN InfobloxAtcPolicyName=Default Global Policy InfobloxAtcRip= The following table describes the data fields that can appear in the output. Table 3) Data Output Fields JSON format CEF format Description Examples qname InfobloxAtcQname Destination example.com. Domain, Destination IP address device InfobloxAtcDevice Device IP or AT Endpoint name , "ATE" user InfobloxAtcUser User Info administrator severity In the CEF Severity field Threat Level HIGH, numeric in CEF format tclass InfobloxAtcThreatClass Malware category MalwareDownload, "CUSTOM" tproperty InfobloxAtcThreatProperty Threat property "Locky", LIST timestamp Timestamp in an UUID format "11e8-04bd bf e622352" event_time InfobloxAtcTimestamp Event time T17:28:01.000Z confidence InfobloxAtcConfidence Confidence (decoded, e.g., LOW, MEDIUM, HIGH) HIGH rcode InfobloxAtcRcode Response type (decoded, e.g. NXDOMAIN, NOERROR, etc.) NOERROR, "NXDOMAIN", "REDIRECT" rdata Response data " ", "" country Country code. Reflect location of the resolved IP "US", "unknown" 2018 Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 6 of 19

7 qtype InfobloxAtcQtype Query Type (decoded, e.g., A, AAAA, MX, etc.) network InfobloxAtcNetwork Source Network (NAT case) / DFP / ATE (decoded) TXT Some Network, AT Endpoint feed_name InfobloxAtcFeedName Feed name "ransomware", "CUSTOM" feed_type InfobloxAtcFeedType Feed type "DOMAIN", "IP- BASED" policy_name InfobloxAtcPolicyName Policy name "Default Global Policy" rip InfobloxAtcRip IP returned , "" Use Case Examples There are several use cases for the API sending ATC security events into a variety of security or analytical solutions. This document will be highlighting one such example use case focusing on integration with a SIEM. Additional configuration is required on the SIEM side to poll, aggregate and correlate events as well as create additional reports. The events already contain information about threats, and by which policy and feed request were blocked. Not every 3rd party software is capable of receiving events through REST API calls. However, syslog is commonly accepted by any SIEM or Syslog server/collector. This document includes a sample python script which will pull ATC events and redistribute the event to a list ofdestinations over Syslog/UDP. The following use case examples are covered in this document: "Infoblox ActiveTrust Cloud" application for Splunk Integration with Carbon Black Server Infoblox Reporting via Syslog Splunk via Syslog McAfee ESM via Syslog Sample script to pull and redistribute events "atc_threats.py" is a sample script which pulls events in CEF format and forwards them via Syslog/UDP. The script is available on GitHub ( and in the Annex A. Note that the script does not accept any CLI options. You will need to set parameters in the script itself. Refer atc_threats.py configuration chapter for the configuration details. Once you set the API key and destinations you can execute script by the following command: python atc_threats.py The script creates "atc_threats_ts.txt" file with the current timestamp in order to fetch only the new events. You can use cron or any other scheduler to periodically pull events. atc_threats.py configuration The script configuration is defined in the script itself. There are 4 configuration parameters: "ATC_APIKey" defines ActiveTrust Cloud API key. Refer to Authentication chapter for the details; 2018 Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 7 of 19

8 "Event_filter" defines event filter. The filter will be passed as GET HTTP request parameters. Refer to Request Parameters chapter for the details; "destinations" is an array of tuples which defines list of destinations. A tuple consists of: o "type" - should be "syslog"; o "host" - an IP address or hostname of a remote syslog receiver; o "port" - a UDP port on which a remote receiver listening the messages; o "facility" - a syslog facility. Only local facilities ("syslog0" - "syslog7") are supported; o "severity" - a syslog severity. sync_delay defines a delta in seconds from a current timestamp. Infoblox ActiveTrust Cloud Splunk application Infoblox ActiveTrust Cloud Splunk application allows: Collection of ActiveTrust Cloud logs into Splunk using the REST API; Filter it efficiently with full drill down support based on the time, threat property, threat class, source IP, domain name, query type and much more; Get additional context from the Infoblox Dossier threat investigation portal. The application is available on the splunkbase.splunk.com website: Integration with Carbon Black Server The Carbon Black Infoblox Secure DNS connector ingests reports via syslog from the Infoblox Secure DNS appliance or via REST against Infoblox ActiveTrust Cloud Services and correlates them against data in the connected Carbon Black server. The connector can then take one or more actions based on these 2018 Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 8 of 19

9 reports, including killing the offending process from the endpoint, isolating the system from the network, and creating an alert for future follow up. The integration was developed by Carbon Black. Documentation and the source code are available on GitHub: Integrations via Syslog/UDP Integration with Infoblox Reporting Infoblox appliances can accept and forward syslog messages from remote systems as well as Infoblox Reporting. This allows you to index and store internal and external syslog messages. The integration takes advantage of these capabilities and allows us to push events detected in ActiveTrust Cloud to a local reporting server using "atc_threats.py" script. Infoblox Reporting syslog destination in atc_threats.py script does not require any specific parameters. Please follow the procedure described in atc_threats.py configuration chapter. To configure NIOS and Reporting appliance: 1. In the grid member properties (can be any grid member), check the "Enable Syslog Proxy" checkbox, and define an appropriate access control list and listening ports Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 9 of 19

10 2. In the Grid Reporting Properties (Administration Reporting) on the "General" tab, enable "Syslog" index and set desired index percentage. 3. On the "Syslog Data" tab, set "Source" to "Any" or "External". Other settings are optional 4. Execute "atc_threats.py" python script on a remote server and wait for a few seconds after the script execution. 5. Navigate to Reporting Search and run the following search. index="ib_syslog" CEF:0\ Infoblox\ ATC 2018 Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 10 of 19

11 6. If everything was configured correctly you should see the events 7. Configure a remote server to execute the script on a schedule. Integration with Splunk Splunk is a SIEM solution which accepts events via different southbound interfaces. Infoblox ActiveTrust Cloud Splunk application is a preferred way to push events to Splunk. In the event that it is not possible to install additional applications, you can use Syslog which is natively supported by Splunk. The provided "atc_threats.py" script can send ActiveTrust Cloud events to Splunk via Syslog/UDP. Please refer to Splunk documentation on how to accept and process syslog data: Integration with McAfee ESM McAfee ESM does not support REST API as a source for the incoming events. You can configure a local receiver to accept syslog messages with an advanced syslog parser. The "atc_threats.py" script can be used to send ActiveTrust Cloud events to McAfee ESM Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 11 of 19

12 To configure the integration please follow the procedure described below. 1. Select a "Local Receiver" and click on the "Add Data Source" button. 2. Set "Data source vendor" to "Generic", "Data Source Model" to "Advanced Syslog Parser", check "Parsing", set "Name" to a desired value (e.g. ATC) and "IP Address" to that of a remote system where the script will be executed Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 12 of 19

13 3. Press "Ok" to save changes. 4. Select created syslog listener and click on the "Policy Editor" button Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 13 of 19

14 5. Add an advanced syslog parsing rule. You can use the rule provided below or create your own policy. any any any -> any any Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 14 of 19

15 6. The script retrieves events in CEF format so it is recommended to use CEF for data transformation. 7. Some fields like "DNS - Query" and "DNS - Type" are not available by default. You need to add them to the list using the "+" button Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 15 of 19

16 8. You must rollout any changes and new rules to the local receiver. 9. Open "View Streaming Events" and press "Start". 10. Generate sample events. 11. Execute the script Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 16 of 19

17 12. Check that McAfee ESM received Syslog messages and use the correct rule to parse events. 13. Configure a remote server to execute the script on a schedule. Please refer "McAfee Enterprise Security Manager. Writing Custom Parsing Rules in McAfee ESM" guide for other details and configuration options. References 1. ActiveTrust Cloud Administrator guide ( 2. McAfee Enterprise Security Manager. Writing Custom Parsing Rules in McAfee ESM. ( N/24000/PD24926/en_US/Writing_Custom_Parsing_Rules_in_McAfee_ESM.pdf) 3. Infoblox connector for Carbon Black Server ( 4. Splunk. Get data from TCP and UDP ports ( 5. "Atc_threats.py" script ( Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 17 of 19

18 Annex A import requests import time import re import socket import os import logging import urllib3 urllib3.disable_warnings() ATC_APIKey="PutYourAPIkeyHere" Event_filter="" #"&source=rpz" destinations=[{"type":"syslog", "host":" ", "port":"514", "facility":"local4", "severity":"warning"}, {"type":"syslog", "host":" ", "port":"514", "facility":"local4", "severity":"warning"}, ] sync_delay=60 t1=str(int(time.time())-60) try: with open('atc_threats_ts.txt', 'r+') as f: t0=f.read().replace('\n', '') f.seek(0) f.write(t1) except IOError, exc: if exc.errno == 2: t0=t1 with open('atc_threats_ts.txt', 'w') as f: f.write(t1) else: raise #Syslog RFCs # # class Facility: local0, local1, local2, local3, local4, local5, local6, local7 = range(16, 24) class Severity: emergency,alert,critical,error,warning,notice,informational,debug = range(8) class DST: def init (self, *initial_data, **kwargs): for dictionary in initial_data: for key in dictionary: setattr(self, key, dictionary[key]) for key in kwargs: setattr(self, key, kwargs[key]) if self.type == "syslog": self.socket = socket.socket(socket.af_inet, socket.sock_dgram) def send_msg_syslog(self,timestamp,msg): "send syslog" 2018 Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 18 of 19

19 facility=getattr(facility, self.facility) severity=getattr(severity, self.severity) print self.type, self.host, timestamp, msg, "\n" data = "<%d>1 %s %s atc-sync %s - - %s" % (severity + facility*8, timestamp, socket.gethostname(),os.getpid(), msg) while True: try: self.socket.sendto(data, (self.host, int(self.port))) break except IOError, exc: if exc.errno == 55: time.sleep(0.1) else: raise def send_msg(self,timestamp,msg): "send a message to a remote system" method = getattr(self, 'send_msg_'+str(self.type)) return method(timestamp,msg) dest_dst=[dst(x) for x in destinations] response = requests.get(' &_format=cef'+event_filter, headers={"authorization": "Token "+ATC_APIKey}) for msg in response.text.encode('utf-8').split('\n'): r = re.search('infobloxatctimestamp=([^\s]+)\s', msg) if r: for dst in dest_dst: dst.send_msg(r.group(1),msg) #print 'curl -k -H "Authorization: Token '+ATC_APIKey+'" " Infoblox Inc. All rights reserved. ActiveTrust Cloud Threats API Quick Start Guide March 2018 Page 19 of 19