IP Profiler. Tracking the activity and behavior of an IP address. Author: Fred Thiele (GCIA, CISSP) Contributing Editor: David Mackey (GCIH, CISSP)

Transcription

1 Security Intelligence June 2005 IP Profiler Tracking the activity and behavior of an IP address Author: Fred Thiele (GCIA, CISSP) Contributing Editor: David Mackey (GCIH, CISSP)

2 Page 2 Contents 3 Profiling IP addresses 5 Interpreting the data 7 Challenges 10 Case study 12 Conclusion Have you ever looked up an IP address at arin.net, whois.org, or samspade. org and wished that you could compile more information about it? What if you could deduce that a particular system using an IP address is an immediate threat to your organization? What if IP addresses could have profiles the same way people have profiles? This paper describes a tool that was designed to accomplish just that. The focus of this paper is IP Profiling, which profiles IP addresses with intrusion detection data. IP Profiling is a specific implementation of Security Profiling that utilizes security data across a suite of security services. Introduction Traditionally, security reporting and analysis tools have been concerned with aggregating simple data and creating some kind of pivot table or graphic. However, managed security service providers (MSSPs) can now gather an enormous amount of security data from a vast, global customer set. This breadth of information allows for the application of many data analysis techniques that were not possible before. One interesting analysis project involves gathering a set of attributes about an Internet Protocol (IP) address using security data generated by an array of security services (for example, intrusion detection, vulnerability scanning, and antivirus scanning). This project is called IP Profiler. An IP address is a unique name for a computer. Every computer connected to a network must identify itself when communicating with another computer. The IP address accomplishes this with a set of four numbers. Each of the numbers is between 0 and 254 and they are separated by a period (.). For example, is a valid IP address. For proper communication to occur, there is a source IP address from the originator and a destination IP address for the recipient. With advanced analysis techniques, IP Profiler can determine the maliciousness of the attacking host, thereby creating a profile of the attacker. The IP Profiler project is concentrated on analyzing source IP addresses that are attacking other machines. By looking at potential malicious network traffic and identifying its source, IP Profiler can store a history of activity originating from the source IP address. With advanced analysis techniques, IP Profiler can determine the maliciousness of the attacking host, thereby creating a profile of the attacker.

3 Page 3 Highlights The goal of the IP Profiler project at IBM is to perform high-level data analysis on large sets of security data. This idea is the same as more traditional methods, but we are choosing key data points to analyze and performing many calculations before the user ever analyzes a chart. If you wanted to see what kind of computers were attacking your network, you could employ an intrusion detection system (IDS) sensor. Each sensor contains a list of signatures (much like virus detection software) that is triggered when a computer performs an action that matches a known malicious behavior pattern. An IDS also logs several attributes about an attacking computer, such as source and destination IP address, source and destination port number, and any data that was sent across the network. The signatures and attributes logged by an IDS are the prime source of data for IP Profiler. Additionally, steps are being taken to integrate other data into the profiler engine, which will help improve its accuracy. Profiling IP addresses Many law enforcement agencies use the technique of profiling to sketch a composite of the perpetrator of crime. After gathering a large set of data about human behavior, physical traits and past criminal activity, law enforcement agents can apply techniques to correlate, aggregate, group and cluster the data. These transformations simplify the overall volume of data to display a report that outlines, for example, the profile of the suspected criminal. The act of profiling IP addresses works much the same as the process for profiling humans. The act of profiling IP addresses works much the same as the process for profiling humans. At IBM, large data sets from several security services are harvested from the vast IBM Managed Security Services (MSS) customer base. This data is stored in a data warehouse where high-level data analysis techniques are applied. IBM then transforms, or mines, the data to discover relationships that may not be obvious. For example, if several million IDS alarms are sent to a security operations center (SOC), there is no way an analyst can examine millions of events and find anything of use. Implementing data mining techniques assists an analyst in discovering relationships in the data. Data mining is a method of attaining business intelligence. Business intelligence is the process of interpreting large sets of data to make intelligent, informed business decisions.

4 Page 4 Highlights The IBM SOC is the primary consumer of the IP Profiler project. The operations center is constantly bombarded by IDS events. These events must be identified either as true attacks or false positives. The SOC analyzes millions of events per day, so higher-level decision tools must be in place to assist the SOC in these large-scale analyses. IP Profiler is the first attempt to provide this high-level decision-making and data analysis to the SOC. The IP Profiler tool applies data mining and business intelligence techniques to IBM security data to pull out information relating to individual IP addresses. The IP Profiler tool applies data mining and business intelligence techniques to IBM security data to pull out information relating to individual IP addresses. For example, it can identify when a low and slow scan is being used. A low and slow scan occurs when an attacker wishes to evade IDSs. An attacker scans a victim host over a long period of time, scanning, for example, 20 items over 3 days instead of 20 items in 3 seconds. This type of network scan is noticeable only with visual techniques such as the one shown in Figure 1. Figure 1 A sample low and slow scan Without a visual representation, an operations analyst would have to go through millions of lines of data to draw the same conclusion. This simply is not feasible from an operational decision-making standpoint.

5 Page 5 Highlights With IP Profiler, a security analyst can quickly look up the attacking IP address, view the historical data regarding that address, and make an informed decision about what action to take, even if only two or three alerts are observed in the IDS console. Interpreting the data Often, the people who are responsible for making operational security decisions do not have the breadth or depth of data to make an educated decision regarding malicious activity. This problem stems from the fact that data in many organizations is not retained long enough to provide historical context. One recommended practice is to retain historical data for a long period of time to monitor and analyze it to its full extent. For example, a security operations analyst monitors an IDS console and observes several alarms triggered for a particular customer. However, these two or three alerts are not enough for the analyst to determine whether to tell a client to batten down the hatches because of a malicious attack or ignore the attacks because they are benign. With IP Profiler, a security analyst can quickly look up the attacking IP address, view the historical data regarding that address, and make an informed decision about what action to take, even if only two or three alerts are observed in the IDS console. As illustrated in Figure 2, an extensible scoring framework is used to provide scores for certain attributes. These scores are rolled up in a hierarchical fashion, giving granularity and meaning to each scored attribute. Figure 2 Profiling attacker threat In Figure 2, the attacker score consists of three separate scores, each with its own algorithm and meaning. The scores are normalized such that the rolled-up score (in this case, attacker score), is on a scale of 1 to 100. As more methods to score an attacker become prevalent, the framework can be easily extensible to include any number of scores for an attacker. The more methods there are to score an attacker, the more meaningful the attacker score becomes.

6 Page 6 Highlights After interpreting the data provided by IP Profiler, analysts can make better decisions about how to respond to an attack. For example, if an IP address is triggering IDS alarms at a high rate (say, 300 alarms per second), an analyst can assume that this IP address is performing some type of mass scanning or mass exploit. If the analyst observes mindless activity such as automated, mass scanning, the attacker may be perceived as less of a threat than one that is manually targeting a server. Someone targeting specific machines with a manual process triggers an alarm to the SOC to investigate this behavior more closely. An attacker may have noticed a vulnerable service on the victim host and is attempting to exploit it. Time of attack is another attribute to examine. If a person is continuously scanning (that is, triggering IDS alarms Sunday through Saturday, at all hours of the day), this may be a clue that mass scanning tools are being run. However, if attacks are occurring during off-business hours, such as 9 p.m. to 3 a.m. eastern time (ET), it is possible to infer that the attacker has a day job and is performing attacks at night from a home machine. Based on these attributes, you could devise a score or profile for attackers that appear more malicious than others. For example, if a person who attacks 7 hours a day, on off hours, may be given a higher score than an attacker who triggers alarms during all hours of the day, which may point to an infected machine. If several attributes of the actions of an IP address are recorded, analysts can use them to derive more specific information that can help enterprises respond to an attack. If several attributes of the actions of an IP address are recorded, analysts can use them to derive more specific information that can help enterprises respond to an attack. For example, if an attacker threat has a 95 score, you can use that score, along with other scores and attributes, to make statements such as The attacker is a moderate threat to your enterprise environment, hails from The Netherlands and attacks banks in Switzerland during the hours of 18:00 to 22:00 coordinated universal time (UTC). Such insight into the behavior or activities of attackers can help operational decision-making.

7 Page 7 Challenges Profiling IP addresses is not without its challenges. A few challenges were encountered during the development of IP Profiler: Determining valid profile time frames. Identifying systems that are infected with malware and which have a human attacker behind them Building a robust infrastructure that can handle the volume of data and processing power associated with the data analysis Profile time frame is the amount of time that a profile is valid based on the Dynamic Host Configuration Protocol (DHCP) leases. Typically, home users connect to an Internet service provider (ISP) and receive an IP address for a certain length of time. This is a DHCP lease. Each ISP assigns different lengths of time to their DHCP leases, creating an interesting situation. For example, a home cable modem ISP may use one week for their DHCP leases, whereas a home DSL ISP may use one day as their lease time. Assembling a profile can be very difficult in these situations because a single IP address may be used by multiple systems. Without determining when DHCP addresses change, you may identify a malicious user whose IP address has already changed. To remedy this, IBM has developed two plans. The first is a simple solution of assigning a static time frame to the validity of a profile (for example, 7 days, 30 days, 90 days and so forth). The drawback is that every ISP likely will have a different DHCP lease value, so seven days may work for ISP A but may not work for ISP B. The second way to address the problem of dynamic IP address assignment involves developing an equation or heuristic that can review anomalies in traffic. In other words, when an IP address changes at the end of a DHCP lease, it creates a traffic pattern that can be identified. This method is being researched for possible future use. For now, a static value for length of profile is used.

8 Page 8 Highlights The second major challenge involves classifying a host as infected with a worm virus. Identifying metrics that can be used to profile worms and viruses is the likely solution. An initial study has shown that it may be possible to profile worms using the three numbers: unique signatures (from IDS) triggered, unique destination ports targeted and unique destination IPs targeted (Figure 3). These numbers are aggregated daily into box scores. These box scores hold the most promise as a method for positively identifying hosts infected with viruses, worms or both. Figure 3 Box scores Using hypothetical numbers, suppose Worm A triggers five unique IDS signatures, attacks two destination ports, and targets between 10,000 and 20,000 hosts each day. If research proves these numbers to be true, we have, in essence, profiled a Worm A infection and can provide feedback to operations analysts regarding the chance (as a percentage) that this host is infected. This approach is still under investigation. An extensive business intelligence infrastructure must be in place to handle backend data analysis. Many challenges also stem from the infrastructure that is required for performing data analysis on a large volume of data. An extensive business intelligence infrastructure must be in place to handle the backend data analysis that must take place. Figure 4 is an example.

9 Page 9 Data Mart Data Mart Vulnerability Scanning Intrusion Detection Data Mart Data Mart Data Mart Data Mart Data Warehouse Analysis Incident Management Virus Threat ETL Figure 4 BI infrastructure Several data feeds are gathered from different sources, followed by analysis and general cleanup of the data. This involves reorganizing the data into a format that can be understood by the data warehouse. Additionally, calculations can be performed on the incoming data (for example, percentages, standard deviation and so forth). After the data is cleaned and calculated, it is stored in the data warehouse. Data marts are deployed as a small subset of the data warehouse, so that individual applications such as IP Profiler can have dedicated data to work with. Additionally, an organization must have the skill sets necessary for running and maintaining the business intelligence infrastructure.

10 Page 10 Case study In this case study, we examine a typical screen capture (Figure 5) from IP Profiler and analyze the results. Several individual boxes, or components, are displayed. The idea behind the user interface of IP Profiler is to keep the tool as modular as possible so that, in future releases, users can organize the data in any format necessary. Figure 5 IP Profiler screen capture In the Global IP Address Information component, you can see that this is an IP address from the U.S. and that it is registered in the American Registry for Internet Numbers (ARIN) database. IP addresses are registered by geography in different databases around the world. The databases contain detailed information about who owns the IP address, where the owner is located and what block of IP addresses it belongs to. The Page Information component allows the user to see the IP address being profiled, the amount of time over which the profile was created and the last date the IP address was seen. In this example, we are viewing all information on this

11 Page 11 Highlights page as having been aggregated over 90 days, unless otherwise posted. In the Profile Period menu in the upper left, a user can change the time period of the data from which the profile is calculated. The Top 10 Signatures component (those are IDS signatures) shows the name and counts of each signature. The types of signatures triggered by an IP address can give analysts more insight into what types of attacks are being performed. If an attacker is triggering many signatures that are normally classified as false positives, it is likely that the attacks are benign. Four Attack Category charts are shown in Figure 5: Attacks by Hour, Attacks by Day, Attacks by Week and Attacks by Month. These charts are aggregates of data over the specified time period. For example, the Attacks by Hour chart shows the number of IDS alarms triggered, broken down by hour of the day, and aggregated over 90 days. With that in mind, for hour 0 (that is, 00:00-00:59 UTC), this attacker triggered approximately 50,000 events. The ability to predict the most likely time an attacker will strike a network can be quite beneficial. These charts are very helpful during the development of an initial profile of an attacker. When does this person normally attack? What days of the week do these attacks occur? The ability to predict the most likely time an attacker will strike a network can be quite beneficial. Also, analysts can see whether this attack pattern fits with that of a typical attacker. Studies have shown that hackers will perform the majority of their attacks during the late evening or early morning hours. The Box Scores component shows the most promise, thus far, for developing techniques for higher-level data analysis because Box Scores are more than just aggregated data. As mentioned earlier, the three box score numbers can theoretically be used to develop profiles of worm- or virus-infected hosts. The Attack Categories component shows the categories to which the Top 10 Signatures belong. IDS analysts classify signatures into categories so metrics can be pulled based on the type of signatures an attacker is triggering. For example, a number of signatures is grouped into a reconnaissance category to help analysts recognize that, when those signatures are alerted, reconnaissance activity is underway. Note: The Top Ports component is irrelevant in this profile. Normally it shows counts of source ports and destination ports used during an attack.

12 After a user has reviewed the information provided by IP Profiler, analysis can begin. The first thing that should catch an analyst s eye is the Attacks by Hour chart. This chart shows a distinctive attack pattern: most are occurring between 7 p.m. to 4 a.m. Eastern time (ET). This time falls within off-business hours in the U.S., so these attacks may be worth investigating. The next item that the analyst should see is that the majority of alerts are categorized as policy violations as follows: edonkey_activity: edonkey is a peer-to-peer file-sharing utility that is very popular. GotomyPC: Go to My PC is a utility used to access a home computer remotely from virtually anywhere using only HTTP as a transport protocol. Copyright IBM Corporation 2005 IBM Global Services Route 100 Somers, NY U.S.A. Produced in the United States of America All Rights Reserved IBM, the IBM logo and the On Demand Business logo are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. Other company, product and service names may be trademarks or service marks of others. References in this publication to IBM products or services do not imply that IBM intends to make them available in all countries in which IBM operates. At this point, the analyst can develop a statement such as: This person uses file-sharing utilities during the night, comes into work and uploads the files downloaded overnight to his or her work PC. This is obviously based on only two sets of data, but can be very powerful given the time it takes to come to this conclusion. When analysts keep these types of ideas in mind, more investigation into current alerts being seen and how to stop this kind of activity can be done. Conclusion Profiling IP addresses is both useful and feasible. As more data is integrated into the IP Profiler tool, you can dramatically increase your ability to identify false positive alerts and help operations center analysts make informed decisions. Future uses of the IP Profiler tool may include: Automated identification of infected hosts on a network Development of a blacklist outlining malicious IP addresses Automatic responses to ISPs with problematic attacking hosts in their networks G