The High Cost of Downtime:

Transcription

1 The High Cost of Downtime: Improving ROI with Better Log Management LogLogic, Inc B Zanker Road San Jose, CA United States US Toll Free: Tel: Fax: LogLogic EMEA Albany House Market Street Maidenhead, Berkshire SL6 8BE United Kingdom Tel: Fax: LogLogic APAC Suite 303, Tower B, Beijing Kelun Building 12A, Guang Hwa Lu Chaoyang District Beijing , China Office: Fax: loglogic.com blog.loglogic.com

2 Availability: it is the holy grail of all network and security managers, regardless of the size of the network or the nature of the business it serves. An unstable network, or one that s vulnerable to attack, is unlikely to be as available as is necessary for users to remain productive. Network outages and security incidents are a fact of life for IT departments, and according to research firm The Yankee Group, their frequency continues to rise. In 2003, the security threats and vulnerabilities to IT systems increased 300 percent over 2002 while the time for security events to compromise entire networks fell from days to just a few hours, according to analyst Matthew Kovar. With the rise of day zero attacks, when exploits and vulnerabilities emerge the same day and incidents spread around the world in seconds, preventing all incidents is truly impossible. More than ever, the threat of compromised network security underscores the need for companies to have good incident-response capabilities 1. As part of their incident response plan, organizations must invest in tools that will help to reduce the mean time to repair in the case of an incident. Without such tools, network and security managers will continue spending an inordinate amount of time troubleshooting network problems and validating policies, resulting in prolonged network outages that cost organizations potentially millions of dollars. The major time drain of an enterprise networking organization is troubleshooting and fixing problems, said analyst Jay E. Pulz at Gartner. Precious human resources can be freed to work on more strategic projects by improving infrastructure reliability. Figure 1: Scope of damage and proliferation speed of attacks are on the rise 1 Computer World, 12/11/03, LogLogic, Inc. All Rights Reserved 2

3 Expenses Rise as Downtime Increases Inevitably, downtime causes a loss of productivity for single users and workgroups. But sometimes, downtime leads to even more serious problems, affecting core applications, business processes or entire departments. Outage expenses can range from lost potential revenue to a damaged reputation with customers and in the marketplace, not to mention the labor charges for a team of technologists working to resolve the outages 2. The cost of downtime varies from industry to industry. For some, even an hour or two can be devastating. Gartner estimates the average cost of downtime is about $42,000 per hour, but companies that rely entirely on technology, such as online brokerages, trading platforms and e- commerce sites, face hourly downtime risks of $1 million or more 3, particularly if customers cannot complete online transactions. A recent survey by Forrest Research shows that viruses are now the biggest concern for IT managers 4. The FBI s annual survey confirms this finding, reporting that virus incidents were the most frequent form of attack, as cited by 82% of respondents 5. When a machine is infected by a worm or virus, it will typically initiate an abnormal number or outbound connections in an attempt to infect other machines. The problem is, system administrators usually aren t aware of the rise in connections until a user complains that his connection is slow. By that time, up to 30 percent of the desktops on the network might be infected. Working reactively, administrators may attempt to uncover the source of the virus by exporting log data from the individual firewalls to a flat text file. These files require Unix experts to decipher. They must write a Unix grep script, look for certain port activity, and manually sort and count the script s output to compose a list of suspect IP address. This process is extremely tedious and time consuming, taking eight hours or more. During this time, many more machines may become infected, rendering the original report useless. Consequently, most IT departments must update the anti-virus software on every machine rather than trying to pinpoint infected machines. Unfortunately, containing a virus attack machine by machine can take as long as 2,000 man hours for an organization with 6,000 desktops. Making matters worse, a company s firewall would succumb under the weight of abnormal traffic during the delay. When this happens, the organization will suffer expensive knowledge worker down time and loose revenues when customers are unable to complete transactions through the corporate fire wall. Log data can aid with segregation of duties and documentation because it can provide a complete, independent record of access, activity, and configuration changes for applications, servers, and network devices. Ideally the policy validation function of activity monitoring and change control audits is performed in real time and includes a complete audit trail of successful and unsuccessful logons, as well as successful and unsuccessful attempts to access files and directories. Whatever the cause, virtually every company faces the risk of IT interruptions that can grind business to a halt. An average company suffers 87 hours of down time per year 3. Consider the average hourly cost of $42,000 that can amount to $3,654,000 or more, a significant figure by anyone s standards. Clearly, the ability to proactively address network issues can minimize the risk of downtime. However, for those network problems that can t be curtailed, such as day zero attacks, rapidly isolating and resolving the threat will significantly reduce the cost of the outage. 2 Computerworld, April 19, Network World, 1/5/ Laura Koetzle, Security Analyst, Forrester Research, 5 CSI/FBI survey LogLogic, Inc. All Rights Reserved 3

4 LogLogic Infrastructure Speeds Response Time, Improves Availability LogLogic makes Log Management appliances that provide high-performance log data aggregation, analysis, and retention for enterprise IT departments. The scalable LogLogic network infrastructure significantly accelerates response time to data center security and performance events by giving IT managers real-time visibility into log data. The LogLogic LX appliances provide IT departments with timely information that can reduce the meantime to repair after a major virus or worm infection by 90%. Previously, it could cost our organization up to two thousand man hours to recover after a major virus outbreak. With early warnings and precise information from the LogLogic appliance, I am able to control the outbreak in one tenth that time, said one satisfied LogLogic customer. This is how it works: When an infected machine begins to generate an abnormal amount of outbound connections, the sudden surge in log data events triggers an alert configured in the LX appliance. It automatically notifies the IT manager that something is wrong. The IT manager can then run a denied IP or TCP distribution report that identifies which hosts initiated the suspect connections. The LX appliance generates this report in just seconds or minutes, instead of hours or days, providing a concise list of suspect host names or IP addresses that facilitates a swift and targeted cleanup effort. LogLogic customers have been able to reduce downtime significantly using the information generated and organized by the LX appliance. When the company s network was hit by the W32 Welchia Worm, the team relied on LogLogic s LX 1000 appliance to sort through the log information and deliver a concise report that made it easy to isolate the worm, said the IS director of a LogLogic customer. The network was suddenly receiving about 4,000 messages per second when the LX appliance sent out the alert. The team was able to view the information instantly, track the IP address back to the infected system, and remove the virus in less than 20 minutes. Using the LX appliance from Log Logic, companies can instantly pinpoint the infected systems and move more rapidly toward problem resolution. Rather than sifting through Unix code in flat files, IT managers can quickly review concise reports that are generated by the LX appliance in just minutes. The time saved enables them to limit the damage from viruses and other network threats, such as utilization risks LogLogic, Inc. All Rights Reserved 4

5 Cost savings leads to better ROI Statistics show that an organization can save close to one and a half million dollars annually with the LogLogic Log Management infrastructure instead of relying on a homegrown syslog server solution. Areas in which expenses are reduced include maintenance, remediation, productivity loss, and revenue loss. The following examples are based on data provided by a typical LogLogic customer: Reduced maintenance: A company can reduce maintenance costs by $24,000 per year by switching from a homegrown solution to the LogLogic infrastructure. When a company uses a homegrown solution, the IT staff typically must spend four hours per week on writing, improving, and using the syslog grep scripts, plus four hours per week compiling reports based on the scripts output. Additionally, companies typically spend two man-weeks per year on improving, tweaking, and expanding the homegrown solution. With the LogLogic infrastructure, these tasks are completely unnecessary. Improved remediation: A company can save 90 percent on remediation costs by pinpointing network trouble early on and by targeting remediation efforts to infected machines or to the root-cause of the problem. For example, a typical Fortune 1000 company with 6,000 desktops that experiences a single major virus outbreak annually can save roughly $135,000 per year. Without the LogLogic appliances, the company s IT department would have to clean up outbreaks machine by machine, a process that can take up to a half an hour per machine. Knowledge worker productivity gains: Potential savings in this area approach $462,000 in a company that has taken security prevention measures and experiences only one major virus outbreak and three minor outages per year. Without LogLogic appliances, a major outbreak would result in eight hours of downtime for knowledge workers, and the minor outages would cost one hour of downtime each. Each hour of downtime would cost an average of $42,000 in lost productivity. Preventing lost revenues: A company can prevent up to $800,000 in lost annual revenue with the LogLogic infrastructure, assuming that the cost of lost revenues is $100,000 per hour. The LogLogic solution prevents at least eight hours, or about one tenth, of critical downtime by providing early warning alerts and by getting to the rout-cause of critical problems faster LogLogic, Inc. All Rights Reserved 5

6 The savings outlined by the examples above do not include the savings realized by other applications that benefit from gaining real-time visibility into log data such as: improved network capacity planning more accurate performance management of networks and applications, including and transaction applications Figure 3: Examples of applications improved by immediate insight into log data LogLogic, Inc. All Rights Reserved 6

7 Time spent wisely Maintaining and supporting a complex network for a large organization continues to be a challenging task. However, with the right tools for isolating and taking action on network threats, high availability is an attainable goal. With the frequency of virus attacks increasing, IT departments no longer have the luxury of deciphering syslog data manually; they need automated and immediate access to specific data that allows them to zero in on network risks quickly and efficiently, and nip potential downtime in the bud. With advanced reporting and auto-alerting features, the LogLogic Log Management infrastructure enables IT departments to cut costs that have until now been considered unavoidable. By being able to respond more proactively and more rapidly to security and reliability risks, network administrators can redirect their time to more productive tasks, further improving the organization s ROI and increasing the perceived value of IT in the eyes of upper management. LogLogic is a trademark of LogLogic, Inc. All other products or services mentioned are the trademarks, service marks, registered trademarks or registered service marks of their respective owners LogLogic, Inc. All Rights Reserved 7