Monitoring individual traffic flows within the ATLAS TDAQ network
|
|
- Jasmin Lucinda Johns
- 5 years ago
- Views:
Transcription
1 Journal of Physics: Conference Series Monitoring individual traffic flows within the ATLAS TDAQ network To cite this article: R Sjoen et al 2010 J. Phys.: Conf. Ser View the article online for updates and enhancements. Related content - Integrated System for Performance Monitoring of the ATLAS TDAQ Network Dan Octavian Savu, Ali Al-Shabibi, Brian Martin et al. - The NetBoard : Network Monitoring Tools Integration for INFN Tier-1 Data Center D De Girolamo, L dell'agnello and and S Zani - The TDAQ Analytics Dashboard: a realtime web application for the ATLAS TDAQ control infrastructure Giovanna Lehmann Miotto, Luca Magnoni and John Erik Sloper This content was downloaded from IP address on 27/09/2018 at 00:46
2 Monitoring Individual Traffic Flows within the ATLAS TDAQ Network R Sjoen 1,4, S Stancu 2,3, M Ciobotaru 2,3, S M Batraneanu 1,2, L Leahu 1,2, B Martin 1 and A Al-Shabibi 1,5 1 CERN, 1211 Geneva 23, Switzerland 2 Politehnica University of Bucharest, Romania 3 University of California, Irvine, USA 4 Bergen University College, Norway 5 University of Heidelberg, Germany rune.velle.sjoen@cern.ch Abstract. The ATLAS data acquisition system consists of four different networks interconnecting up to 2000 processors using up to 200 edge switches and five multi-blade chassis devices. The architecture of the system has been described in [1] and its operational model in [2]. Classical, SNMP-based, network monitoring provides statistics on aggregate traffic, but for performance monitoring and troubleshooting purposes there was an imperative need to identify and quantify single traffic flows. sflow [3] is an industry standard based on statistical sampling which attempts to provide a solution to this. Due to the size of the ATLAS network, the collection and analysis of the sflow data from all devices generates a data handling problem of its own. This paper describes how this problem is addressed by making it possible to collect and store data either centrally or distributed according to need. The methods used to present the results in a relevant fashion for system analysts are discussed and we explore the possibilities and limitations of this diagnostic tool, giving an example of its use in solving system problems that arise during the ATLAS data taking. 1. Introduction We describe the concept of statistical sampling followed by a description of the system architecture. The developed system consists of a collector, a processor, a storage solution, a service exposing the data and a web interface. After describing the general architecture, we discuss every component in the system in more detail using a bottom-up approach. Finally there is a discussion of data presentation techniques and examples of typical use. 2. Statistical Sampling The concept of statistical sampling involves capturing and examining a relatively small subset of the total amount of data flowing through a network, and still being able to make fairly accurate assumptions and observations about the traffic. All the devices in the ATLAS data acquisition network support the sflow standard. One of the design goals of this standard is to be able to accurately monitor traffic in high speed networks, without being overwhelmed by the volume of data that needs to be processed and stored. c 2010 IOP Publishing Ltd 1
3 We can define an individual traffic flow in a network as a set of packets belonging to a single conversation between two nodes during a certain period of time. The ability to identify individual traffic flows in a network enables powerful techniques for debugging. Implementing statistical packet sampling within the ATLAS data acquisition network gives us the ability to identify and examine the causes of unknown traffic patterns. sflow[3] is an industry standard which enables an Ethernet switch to take a sample of the packets traversing it and send them to a collector for permanent storage. The packet samples are analyzed in software and conversations at different network layers can be individually traced. The sflow standard describes the mechanism used by the agents and allows implementation in hardware as well as software. sflow samples every n-th packet on average, the sampling process uses randomness in order to prevent capturing perodic patters. As every switch in ATLAS supports sflow in hardware, there is the potential to concurrently monitor over 4000 ports. Since even brief transactions can be important, we operate sflow at high sampling rates, up to one sample per 256 packets, which together with the large number of ports in the system generates a data handling problem of its own. There are other technologies for high-speed traffic monitoring such as NetFlow[4], which was originally developed by Cisco to collect IP traffic information, NetFlow looks at all the packets and adds their properties to the frame and byte counters of the flow records before sending these records to a NetFlow collector at regular intervals. Since NetFlow agents are implemented in software, this approach can be very computationally expensive for the devices, especially when there are many short-lived traffic flows. Closer to sflow and statistical sampling there is Sampled NetFlow, which samples every n-th packet where the value of n can either be deterministic or random. We tested several of the solutions that were already available[5], including commercial products such as InMon s [6] Traffic Sentinel, and non-commercial and open source products such as sflowtrend also from InMon and ntop[7]. While some of the tested solutions were simply not flexible and scalable enough, the more powerful commercial solutions did not offer the ability to provide tight integration with our other in-house developed tools due to the proprietary nature of the data storage formats. Developing our own solution also enables potential use of the network topology already discovered by other tools when analyzing the data. 3. System architecture The system we developed (Figure 1) consists of the following components s implemented on the devices we want to monitor, the agents sample packets and send them to the collector. No other packet processing is done by the device. receives and temporarily stores the samples received from the devices it monitors. The samples arrive packed in sflow datagrams. extracts information from the collected packet samples and prepares the data for permanent storage. MySQL database as a data storage service stores the processed data in a permanent location. service provides a clean API for accessing the data storage. interface front-end to the data which allows the user to make queries and display plots using sflow data. To increase flexibility in terms of resource allocation the collector and processor are modular components and can be decoupled if this proves necessary, an example is having multiple processors working for a single collector. The different models of distribution are presented in section
4 Figure 1. Overview of the system architecture 3.1. Data collection The process of collecting data involves receiving and temporarily storing samples until they are ready to be processed. The devices send flow samples and counter samples inside sflow datagrams (UDP). A datagram may contain multiple samples. We currently operate our devices at a sampling rate of 256 which can be considered a high sampling rate for high throughput networks. This effectively means that on average every 256 th packet passing through each interface on the sflow enabled devices is sampled. The flow samples received from the switches contain a copy of the first 128 bytes of the packet. We can use this data to extract information about the traffic flow the packet itself is a part of. All of our devices sample the raw header of the packets, so we have all the necessary information from each layer of the TCP/IP model. In addition to the packet header, a sample contains layer 2 switching information, i.e inbound and outbound physical interfaces used by the packet. The collector is a dual-threaded application written in C. The collecting thread is responsible for collecting all the samples and storing them into fixed-size dynamically allocated buffers. When a buffer is full, or an interval expires, this buffer is passed to the processing thread which processes the samples in the buffer and distributes the samples into POSIX shared memory objects, one object per agent address(device). The collector keeps an open handle to the memory object for the current interval. When a new interval starts the handle is closed and a message with information about the object is put into a POSIX message queue. The collector then creates a new object and starts collecting there. The interval size is configurable, allowing us to modify the granularity of the data we collect. This method will allow the collecting thread to store more samples than the processing thread 3
5 is able to save, for short periods of time, depending on the available memory. The maximum amount of memory to use by the collector is configurable. The processing daemon waits for a message from the collector by polling the message queue. The message contains the necessary information to process the samples received. Processing the temporarily stored samples is addressed in section 3.2. In order to get information about short-lived traffic flows, both in heavy and light load conditions we want to collect as many samples as we can. According to the sflow standard the agents may implement throttling features that permanently reduce the sampling rate in case the load gets too high. Not all agents write this into the logs and it is not necessarily easily detectable. We currently address this issue by resetting the sampling rates statically at regular intervals. At a later time we will explore the possibility of dynamically changing sampling rates Pushing vs pulling SNMP data In addition to packet sampling, the in the switch sends the values of the SNMP counters in the switch to the collector at regular intervals in the form of counter samples. It will do so using best-effort approach within a predefined maximum time limit and try to piggy-back these counter samples in the datagrams together with the flow samples. If this cannot be done it will send the counter sample in a separate datagram when the maximum time is reached. By using this method of collecting SNMP data from the switch we will be able to move away from the normal request-response scenario of pulling SNMP information from the switch by asking for it. In addition to significantly reducing the work needed to collect this information from the devices, this will also decrease the bandwidth used by monitoring tools in the network. There is no additional work required using this approach except receiving and storing the counter samples Data processing This process involves retrieving the temporarily stored samples from the collector, processing them, and extracting the information we are interested in. We define a conversation as a stream of frames and bytes between two nodes. On layer 2 the key for a unique conversation is a tuple that consists of the source and destination hardware address. This also goes for layer 3 except that the Internet address is used instead of the hardware address. On layer 4 a unique conversation is represented by a 4-tuple consisting of the source and destination address including the source and destination port. These keys, in addition to the ingress and egress interface id s, define the master key which we use to identify the conversation. This key is then used to sum up the bytes and frames from the sampled packets to their respective conversations. Currently we are only processing and storing byte and frame counters for each conversation, but this can be extended to include for example flag counters for TCP and protocol distribution counters for layer 2 and layer 3. Basically any data found in the first 4 layers of the TCP/IP model can be extracted and stored. By processing and grouping the samples into time intervals we can define the granularity of the data we collect. By doing this we can greatly reduce the amount of data that needs to be stored. This eliminates the direct proportionality between the sampling rate and the size of the stored data, allowing us to increase the sampling rate to gain more accurate information. We discuss the size of the stored data in more detail in section Storing data Sustained high sampling rates requires the sampled packets to be processed and stored in a timely fashion. We explored different methods of storing the data both sample-based and conversationbased. 4
6 Currently all the counter samples we receive are stored into the same database as the data extracted from the flow samples, but since the counter samples are sent at relatively fixed intervals we also have the option to decouple flow and counter samples and store the counter values in round robin databases (RRD) Sample based storage Storing the samples involves collecting the datagrams from the agents, unpacking them and storing the samples received from each agent in separate locations. No further processing is done until the data is needed. This is a simple but very inefficient way to store the data with regards to the total size. Because the data has not been processed in any way, additional delays will occur when it needs to be processed and displayed to the user. The total size of the data to store is dependent on two factors, the number of flow samples received and the number of ports which are sending counter samples. As the sampling rate is increased the effect of the counter samples becomes less significant and the size we need to store becomes directly proportional to the number of flow samples received Conversation based storage When storing conversations we do most of the processing in the before the data is requested by the user and this involves extracting the information we need at a pre-defined granularity. The granularity determines the detail at which we can examine the collected data. With a granularity of, say, 1 minute we collect data for that minute before processing it and extracting the information. This method of storage does not directly depend on the sampling rate, but on the expected number of conversations expected to be seen on every port in any single interval. There may be many samples within the time period but the number of conversations is only a function of the system, and not of the sampling. As a consequence the quantity of data that needs to be stored is significantly reduced in comparison to the sample based storage method Comparison between sample based and conversation based storage We can see that even in an almost idle network, where the traffic mostly consists of basic control and monitoring traffic (Figure 2), there is a significant improvement on the size of the data we need to store. When the network load increases, and the network is busy during data taking, this difference becomes even bigger (Figure 3). As traffic increases in the network, we will get more samples, but they will mostly belong to the conversations already seen, which means that they will not affect the data rate at which we store data. This means that as the traffic increases in the network, the data growth rates for the two storage types will diverge Storage solution The storage solution needs to be able to handle high speed continuous inserts. On the current solution, we are using an open source database, MySQL, as a backend for storing data. The configuration of the database server has been optimized for this purpose. The most important optimizations include the following: Increasing the maximum size for bulk inserts which is a very effective way of reducing the time and resources required to insert a large amount of rows. Instead of inserting every row as it becomes available we build a large query and do multiple insertions in the same query. This will also reduce the amount of open/close operations on the database table which are time consuming hard-disk access operations. Increasing the size of the key buffer allows the server to store table indexes in memory. By disabling the option to instantly flush transactions to disk we gain performance during write operations in exchange for data security. 5
7 90 80 Samples Conversations Samples Conversations Data rate (KB/s) Data rate (KB/s) Time (minutes) Time (minutes) Figure 2. A graph showing the data rate difference between storing raw samples and processed data in an almost idle network Figure 3. A graph showing the data rate difference between storing raw samples and processed data in a busy network The buffer pool determines how much data can be cached. Increasing this results in less disk I/O operations Data presentation This involves retrieving the stored data from the data storage facility, formatting it and presenting it in whatever form the user wants. The main goal is being able to perform post-mortem analysis of past events. We want to be able to examine who was using the bandwidth, contributing to the link utilization and how specific conversations developed over time. This information can be used to discover problems in the network. The information in the data collected can also be used to find links with errors, links operating at the wrong line speed and other information but these features can also be accomplished by using other standard tools. To make the collected data available to the end user we have implemented a web interface (Figure 4) that is part of the 3-tier architecture of the front-end system (Figure 1) where the database represents the data tier, the logic tier is represented by a web service providing access to the database while doing post-processing, formatting and other tasks. The presentation tier is the web interface itself. The interface gives the user the ability to select a switch or a single port, and define the TCP/IP layer and time interval to examine. After these values have been selected it shows the conversations which has been sampled during the selected interval. From this point the user can select between different ways of visually representing the data such as pie charts that show link utilization and line charts that illustrate how conversations develop over time. The web interface has been implemented using the Google Toolkit (GWT). This provides us with a powerful and cross-browser compatible interface with little extra effort. As a simple initial implementation for displaying graphs in the web interface we have used the Google Charts application programming interface. At a later time we will examine solutions to replace the current implementation with a more interactive graphing solution. The interface will get the data asynchronously via a Python [8] module served by Apache [9]. This module will take care of retrieving the data from the database and preparing it to be displayed in the web interface. This preparation involves different things such as resolving host and service names. This module exports a clean API to the world and makes it available for 6
8 Figure 4. Screen-shot of the web interface other applications to make use of the data. The data returned by the module is in JavaScript Object Notation (JSON) JSON format, which is a lightweight data interchange format that the Google Toolkit, by design, supports very well Distributing the system Because the system developed is inherently modular, it can be distributed in several different ways depending on the needs. The system in it simplest form with no distribution (Figure 5) has one collector feeding a processor which stores the data into a single location. It is also highly dependent on the performance of the system it is running on. At the moment we are operating with a nondistributed system and it has proven capable of handling the data rates during tests, however we are currenty operating at approximately 40% of the total number of nodes. In addition to adding more nodes, new nodes also have an increased number of cores leading to more processes running on each nodes which in turn leads to more conversations per node. Therefore we are looking at several different methods of distribution. We can distribute the collecting/processing process while still collecting to a centralized repository (Figure 6). This is the optimal solution when operating at high sampling rates with a limited number of possible conversations between each pair of nodes. In this case the work which is required to collect and process the samples outweighs the work required to insert the processed data into storage. In the Atlas TDAQ Network there are two switches and one local file server per rack, with each rack containing approximately 30 processors. We can run an /processor in each rack, resulting in distributed collection and processing. This will significantly reduce the bandwidth consumed by sflow since samples will not have to be aggregated to the single collector, but is processed in each rack leaving only the extracted information to be sent to the data storage. A fully distributed system (Figure 7) can also be implemented by giving each node its 7
9 Data storage Data storage service service interface interface Figure 5. A non distributed system Figure 6. A partially distributed system Data storage Data storage service interface Figure 7. A fully distributed system own database for storage and leaving the responsibility of requesting the correct data to the web service. If we need to store large amounts of data from a large number of devices with a long history, or the database server of the partially-distributed system cannot handle the load, this is a way to address that problem. The functionality required for the web service to enable full distribution will require extra application logic in the web service, and will be implemented if proven necessary. This distribution model will also increase the response time when communicating with the web service. 4. Example 1 By inspecting the plots generated from our network monitoring tools (Figure 8) we are able to identify unknown or unexpected traffic patterns but we have no way to extract more information about them. 8
10 Figure 8. Traffic plot from SNMP data By using the collected sflow data, we are able to extract information on the approximate bandwidth usage per host during the selected interval (Figure 9), as well as information about the TCP conversations made by each host. By examining the conversation plot we are able to determine which application is generating this traffic by looking for known port numbers. Figure 9. Bandwidth usage per host Figure 10. TCP conversations over time 5. Example 2 While checking for anomalous traffic a continuous traffic load on one port was noticed (Figure 11). The traffic led to machines that had nothing to do with data taking and tracking down the source across several hops of the general purpose network would have been extremely tedious. Figure 11. Unknown control core traffic Figure 12. Bandwidth usage per host However analysing the conversations across the busy core port with sflow immediately showed that a test PC was running some diagnostic procedure on local file servers (Figure 12). 9
11 6. Conclusion Network traffic sampling has been an industry technology for nearly two decades. It s typically employed with fairly low sampling rates (e.g. 1:8192) for continuous sampling and only specific ports would be subject to high rates (e.g. 1:512) for specific diagnostic purposes. However, the whole Atlas TDAQ network needs to be continuously monitored and the flows understood so as to achieve the maximum throughput under many different modes of operation and this in turns places a requirement for high rate sampling on every port. To achieve this we have developed a modular architecture that permitted both the rapid deployment of the prototype system and allows for staged task distribution to meet future load increases. Much attention has been paid to the GUI and event display as well as integration into the experimental monitoring system. As a result of this, the intuitive interface allows even non-expert users to easily obtain the desired results. The fact that we can continuously monitor every port means that all the statistics are on hand the moment a problem is detected and diagnosis can be performed immediately. References [1] S. Stancu, M. Ciobotaru, C. Meirosu, L. Leahu, and B. Martin, Networks for the ATLAS Trigger and Data Acquisition, Computing in High Energy and Nuclear Physics, Mumbai, India, [2] S.M. Batraneanu, A. Al-Shabibi, M. Ciobotaru, M. Ivanovici, L. Leahu, B. Martin and S. Stancu, Operational Model of the ATLAS TDAQ Network, Proc. IEEE Real Time 2007 Conference, Chicago, USA, May [3] P. Phaal, S. Panchen, and N. McKee, InMon Corporation s sflow: A method for monitoring traffic in switched and routed networks, T ech. rep., RFC 3176, Sept [4] Cisco IOS NetFlow. [Online]. Available: [5] sflow Collectors. [Online]. Available: [6] Inmon Corporation [Online]. Available: [7] Network Top (ntop) [Online]. Available [8] Python Programming Language. [Online]. Available: [9] Apache HTTP Server Project. [Online]. Available: 10
The TDAQ Analytics Dashboard: a real-time web application for the ATLAS TDAQ control infrastructure
The TDAQ Analytics Dashboard: a real-time web application for the ATLAS TDAQ control infrastructure Giovanna Lehmann Miotto, Luca Magnoni, John Erik Sloper European Laboratory for Particle Physics (CERN),
More informationsflow Elisa Jasinska
sflow Elisa Jasinska elisa.jasinska@ams-ix.net Agenda What is sflow? What is AMS-IX? AMS-IX requirements Existing software solutions Performance issues Software used at AMS-IX Privacy Results What is sflow?
More informationConfiguring sflow. Information About sflow. sflow Agent. This chapter contains the following sections:
This chapter contains the following sections: Information About sflow, page 1 Licensing Requirements, page 2 Prerequisites, page 2 Guidelines and Limitations for sflow, page 2 Default Settings for sflow,
More informationMonitoring ARC services with GangliARC
Journal of Physics: Conference Series Monitoring ARC services with GangliARC To cite this article: D Cameron and D Karpenko 2012 J. Phys.: Conf. Ser. 396 032018 View the article online for updates and
More informationsflow (http://www.sflow.org) Agent Software Description
sflow (http://www.sflow.org) sflow Agent Software Description This slide set is intended as a guide to InMon s example sflow agent software. The concepts and design choices are illustrated. The intention
More informationInterface Utilization vs. Flow Analysis
Interface Utilization vs. Flow Analysis Interface utilization is the calculated percentage utilization at the interface using SNMP polled data from the IF-MIB (Figure 2) and this is presented as inbound
More informationConfiguring sflow. About sflow. sflow Agent
About sflow This chapter describes how to configure sflow on Cisco NX-OS devices. This chapter includes the following sections: About sflow, on page 1 Licensing Requirements for sflow, on page 2 Prerequisites
More informationRaw Data Formatting: The RDR Formatter and NetFlow Exporting
CHAPTER 9 Raw Data Formatting: The RDR Formatter and NetFlow Exporting Revised: September 27, 2012, Introduction Cisco Service Control is able to deliver gathered reporting data to an external application
More informationThe AAL project: automated monitoring and intelligent analysis for the ATLAS data taking infrastructure
Journal of Physics: Conference Series The AAL project: automated monitoring and intelligent analysis for the ATLAS data taking infrastructure To cite this article: A Kazarov et al 2012 J. Phys.: Conf.
More informationMonitoring of large-scale federated data storage: XRootD and beyond.
Monitoring of large-scale federated data storage: XRootD and beyond. J Andreeva 1, A Beche 1, S Belov 2, D Diguez Arias 1, D Giordano 1, D Oleynik 2, A Petrosyan 2, P Saiz 1, M Tadel 3, D Tuckett 1 and
More informationUsing NetFlow Filtering or Sampling to Select the Network Traffic to Track
Using NetFlow Filtering or Sampling to Select the Network Traffic to Track First Published: June 19, 2006 Last Updated: December 17, 2010 This module contains information about and instructions for selecting
More informationRaw Data Formatting: The RDR Formatter and NetFlow Exporting
CHAPTER 8 Raw Data Formatting: The RDR Formatter and NetFlow Exporting Cisco Service Control is able to deliver gathered reporting data to an external application for collecting, aggregation, storage and
More informationConfiguring Application Visibility and Control for Cisco Flexible Netflow
Configuring Application Visibility and Control for Cisco Flexible Netflow First published: July 22, 2011 This guide contains information about the Cisco Application Visibility and Control feature. It also
More informationConfiguring NetFlow. Feature History for Configuring NetFlow. Release This feature was introduced.
Configuring NetFlow A NetFlow flow is a unidirectional sequence of packets that arrive on a single interface (or subinterface), and have the same values for key fields. NetFlow is useful for the following:
More information10Gbps Ethernet Solutions
Absolute Analysis Investigator 10Gbps Ethernet Solutions SFP+ Technology Provides Lower Cost, Longer Range with 10GBase-LRM Interface Compliance Support True 100% full line rate traffic capture with absolutely
More informationUsing NetFlow Sampling to Select the Network Traffic to Track
Using NetFlow Sampling to Select the Network Traffic to Track This module contains information about and instructions for selecting the network traffic to track through the use of NetFlow sampling. The
More informationBig Data Insights Using Analytics
Governors State University OPUS Open Portal to University Scholarship All Capstone Projects Student Capstone Projects Fall 2015 Big Data Insights Using Analytics Naga Krishna Reddy Muppidi Governors State
More informationAdvanced Application Reporting USER GUIDE
Advanced Application Reporting USER GUIDE CONTENTS 1.0 Preface: About This Document 5 2.0 Conventions 5 3.0 Chapter 1: Introducing Advanced Application Reporting 6 4.0 Features and Benefits 7 5.0 Product
More informationNoction Flow Analyzer
INSTALLATION & CONFIGURATION GUIDE Copyright 2018 Noction Inc. Table of Contents Introduction...3 What is Noction Flow Analyzer...3 System Requirements...3 Hardware requirements...3 Software requirements...3
More informationNetwork Operations Analytics
Network Operations Analytics Solution Guide Version 2.4.4 (Build 2.4.4.0.x) June 2016 Copyright 2012-2016 NetFlow Logic Corporation. All rights reserved. Patents Pending. Contents Introduction... 2 Solution
More informationIMC Network Traffic Analyzer 7.2 (E0401P04) Copyright 2016 Hewlett Packard Enterprise Development LP
Network Traffic Analyzer 7.2 (E0401P04) Copyright 2016 Hewlett Packard Enterprise Development LP Table of Contents 1. What's New in this Release 2. Problems Fixed in this Release 3. Software Distribution
More informationBest Practices. Deploying Optim Performance Manager in large scale environments. IBM Optim Performance Manager Extended Edition V4.1.0.
IBM Optim Performance Manager Extended Edition V4.1.0.1 Best Practices Deploying Optim Performance Manager in large scale environments Ute Baumbach (bmb@de.ibm.com) Optim Performance Manager Development
More informationMonitoring and Analysis
CHAPTER 3 Cisco Prime Network Analysis Module 5.1 has two types of dashboards: One type is the summary views found under the Monitor menu, and the other type is the over time views found under the Analyze
More informationFundamentals of IP Networking 2017 Webinar Series Part 4 Building a Segmented IP Network Focused On Performance & Security
Fundamentals of IP Networking 2017 Webinar Series Part 4 Building a Segmented IP Network Focused On Performance & Security Wayne M. Pecena, CPBE, CBNE Texas A&M University Educational Broadcast Services
More informationWLCG Transfers Dashboard: a Unified Monitoring Tool for Heterogeneous Data Transfers.
WLCG Transfers Dashboard: a Unified Monitoring Tool for Heterogeneous Data Transfers. J Andreeva 1, A Beche 1, S Belov 2, I Kadochnikov 2, P Saiz 1 and D Tuckett 1 1 CERN (European Organization for Nuclear
More informationConfiguring AVC to Monitor MACE Metrics
This feature is designed to analyze and measure network traffic for WAAS Express. Application Visibility and Control (AVC) provides visibility for various applications and the network to central network
More informationLightstreamer. The Streaming-Ajax Revolution. Product Insight
Lightstreamer The Streaming-Ajax Revolution Product Insight 1 Agenda Paradigms for the Real-Time Web (four models explained) Requirements for a Good Comet Solution Introduction to Lightstreamer Lightstreamer
More informationPerforming Path Traces
About Path Trace, page 1 Performing a Path Trace, page 13 Collecting QoS and Interface Statistics in a Path Trace, page 15 About Path Trace With Path Trace, the controller reviews and collects network
More informationIMC Network Traffic Analyzer 7.3 (E0504) Copyright 2015, 2017 Hewlett Packard Enterprise Development LP
Network Traffic Analyzer 7.3 (E0504) Copyright 2015, 2017 Hewlett Packard Enterprise Development LP Table of Contents 1. What's New in this Release 2. Problems Fixed in this Release 3. Software Distribution
More informationHow to Export sflow from a Cisco ASR 9k
LIVEACTION, INC. How to Export sflow from a Cisco ASR 9k CONFIGURATION LiveAction, Inc. 3500 Copyright WEST BAYSHORE 2016 LiveAction, ROAD Inc. All rights reserved. LiveAction, LiveNX, LiveUX, the LiveAction
More informationFlexible Netflow Configuration Guide, Cisco IOS Release 15S
Flexible Netflow Configuration Guide, Cisco IOS Release 15S Americas Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000 800 553-NETS
More informationUsing NetFlow Filtering or Sampling to Select the Network Traffic to Track
Using NetFlow Filtering or Sampling to Select the Network Traffic to Track Last Updated: December 7, 2011 This module contains information about and instructions for selecting the network traffic to track
More informationTrisul Network Analytics - Traffic Analyzer
Trisul Network Analytics - Traffic Analyzer Using this information the Trisul Network Analytics Netfllow for ISP solution provides information to assist the following operation groups: Network Operations
More informationCisco Security Monitoring, Analysis and Response System 4.2
Q&A Cisco Security Monitoring, Analysis and Response System 4.2 GENERAL Q. What is the Cisco Security Monitoring, Analysis and Response System? A. The Cisco Security Monitoring, Analysis and Response System
More informationplixer Scrutinizer Competitor Worksheet Visualization of Network Health Unauthorized application deployments Detect DNS communication tunnels
Scrutinizer Competitor Worksheet Scrutinizer Malware Incident Response Scrutinizer is a massively scalable, distributed flow collection system that provides a single interface for all traffic related to
More informationNETWORK PACKET ANALYSIS PROGRAM
NETWORK PACKET ANALYSIS PROGRAM Duration: 3 days (21 hours) Mode: 1. Instructor Led Class room Training and Labs 2. Online In this hands-on course, you will receive in-depth training on Protocol analysis
More informationOnline remote monitoring facilities for the ATLAS experiment
Journal of Physics: Conference Series Online remote monitoring facilities for the ATLAS experiment To cite this article: S Kolos et al 2011 J. Phys.: Conf. Ser. 331 022013 View the article online for updates
More information6.1 Internet Transport Layer Architecture 6.2 UDP (User Datagram Protocol) 6.3 TCP (Transmission Control Protocol) 6. Transport Layer 6-1
6. Transport Layer 6.1 Internet Transport Layer Architecture 6.2 UDP (User Datagram Protocol) 6.3 TCP (Transmission Control Protocol) 6. Transport Layer 6-1 6.1 Internet Transport Layer Architecture The
More informationAvaya Port Matrix: Avaya Proprietary Use pursuant to the terms of your signed agreement or Avaya policy.
Avaya Matrix: Release 3.0 Issue 2 April 2016 April 2016 Avaya Matrix: 3.0 1 ALL INFORMATION IS BELIEVED TO BE CORRECT AT THE TIME OF PUBLICATION AND IS PROVIDED "AS IS". AVAYA INC. DISCLAIMS ALL WARRANTIES,
More informationsflow Agent Contents 14-1
14 sflow Agent Contents Overview..................................................... 14-2 Flow Sampling by the sflow Agent........................... 14-2 Counter Polling by the sflow Agent...........................
More informationTHE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES
1 THE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES Vincent Garonne, Mario Lassnig, Martin Barisits, Thomas Beermann, Ralph Vigne, Cedric Serfon Vincent.Garonne@cern.ch ph-adp-ddm-lab@cern.ch XLDB
More informationUsing NetFlow Sampling to Select the Network Traffic to Track
Using NetFlow Sampling to Select the Network Traffic to Track Last Updated: September 17, 2012 This module contains information about and instructions for selecting the network traffic to track through
More informationTraditional network management methods have typically
Advanced Configuration for the Dell PowerConnect 5316M Blade Server Chassis Switch By Surendra Bhat Saurabh Mallik Enterprises can take advantage of advanced configuration options for the Dell PowerConnect
More informationDKT 224/3 LAB 2 NETWORK PROTOCOL ANALYZER DATA COMMUNICATION & NETWORK SNIFFING AND IDENTIFY PROTOCOL USED IN LIVE NETWORK
DKT 224/3 DATA COMMUNICATION & NETWORK LAB 2 NETWORK PROTOCOL ANALYZER SNIFFING AND IDENTIFY PROTOCOL USED IN LIVE NETWORK Lab #2 2 Lab #2 : Network Protocol Analyzer (Sniffing and Identify Protocol used
More informationNetFlow Traffic Analyzer
GETTING STARTED GUIDE NetFlow Traffic Analyzer Version 4.2.3 Last Updated: Wednesday, October 11, 2017 Retrieve the latest version from: https://support.solarwinds.com/success_center/netflow_traffic_analyzer_(nta)/nta_documentation
More informationInvestigator. SATA Solutions. Absolute Analysis Investigator Architecture. Revolutionary probe architecture overcomes SATA cabling limitations
Absolute Analysis Investigator Architecture Investigator SATA Solutions Revolutionary probe architecture overcomes SATA cabling limitations Only protocol analyzer entirely driven through a configurable
More informationConfiguring Data Collection Manager
This module describes the configuring of the Data Collection Manager feature. Table 1: Feature History for Release Release 5.2.2 Modification This feature was introduced Data Collection Manager, on page
More informationConfiguring QoS CHAPTER
CHAPTER 34 This chapter describes how to use different methods to configure quality of service (QoS) on the Catalyst 3750 Metro switch. With QoS, you can provide preferential treatment to certain types
More informationNetFlow Traffic Analyzer
ADMINISTRATOR GUIDE NetFlow Traffic Analyzer Version 4.4 Last Updated: Friday, June 15, 2018 2018 SolarWinds Worldwide, LLC. All rights reserved. This document may not be reproduced by any means nor modified,
More informationNew Persistent Back-End for the ATLAS Online Information Service
New Persistent Back-End for the ATLAS Online Information Service Igor Soloviev and Alexandru Sicoe The Trigger and Data Acquisition (TDAQ) and detector systems of the ATLAS experiment deploy more than
More informationConfiguring NetFlow. About NetFlow. This chapter describes how to configure the NetFlow feature on Cisco NX-OS devices.
This chapter describes how to configure the NetFlow feature on Cisco NX-OS devices. About NetFlow, page 1 Licensing Requirements for NetFlow, page 4 Prerequisites for NetFlow, page 4 Guidelines and Limitations
More informationAutomated load balancing in the ATLAS high-performance storage software
Automated load balancing in the ATLAS high-performance storage software Fabrice Le Go 1 Wainer Vandelli 1 On behalf of the ATLAS Collaboration 1 CERN May 25th, 2017 The ATLAS Experiment 3 / 20 ATLAS Trigger
More informationIBM Security QRadar Deployment Intelligence app IBM
IBM Security QRadar Deployment Intelligence app IBM ii IBM Security QRadar Deployment Intelligence app Contents QRadar Deployment Intelligence app.. 1 Installing the QRadar Deployment Intelligence app.
More informationRouter and ACL ACL Filter traffic ACL: The Three Ps One ACL per protocol One ACL per direction One ACL per interface
CCNA4 Chapter 5 * Router and ACL By default, a router does not have any ACLs configured and therefore does not filter traffic. Traffic that enters the router is routed according to the routing table. *
More informationProduct Overview. Overview CHAPTER
CHAPTER 1 This chapter provides a brief introduction to the Cisco TV Content Delivery System for an Interactive Services Architecture (ISA) environment. This chapter covers the following major topics:
More informationResource allocation in networks. Resource Allocation in Networks. Resource allocation
Resource allocation in networks Resource Allocation in Networks Very much like a resource allocation problem in operating systems How is it different? Resources and jobs are different Resources are buffers
More informationNetworking Quality of service
System i Networking Quality of service Version 6 Release 1 System i Networking Quality of service Version 6 Release 1 Note Before using this information and the product it supports, read the information
More informationZone-Based Firewall Logging Export Using NetFlow
Zone-Based Firewall Logging Export Using NetFlow Zone-based firewalls support the logging of messages to an external collector using NetFlow Version 9 export format. NetFlow Version 9 export format uses
More informationVXLAN Overview: Cisco Nexus 9000 Series Switches
White Paper VXLAN Overview: Cisco Nexus 9000 Series Switches What You Will Learn Traditional network segmentation has been provided by VLANs that are standardized under the IEEE 802.1Q group. VLANs provide
More informationOUTLINE. NSLS-II control system environment Monitoring goals Splunk and Splunk Apps Unix, Nagios, Snort sflow and Cacti Putting it all together
OUTLINE NSLS-II control system environment Monitoring goals Splunk and Splunk Apps Unix, Nagios, Snort sflow and Cacti Putting it all together NSLS-II CONTROL SYSTEM ENVIRONMENT Private network no email,
More informationConfiguring NetFlow. Information About NetFlow. Send document comments to CHAPTER
CHAPTER 11 Use this chapter to configure NetFlow to characterize IP traffic based on its source, traffic destination, timing, and application information, giving visibility into traffic transiting the
More informationApplication Level Protocols
Application Level Protocols 2 Application Level Protocols Applications handle different kinds of content e.g.. e-mail, web pages, voice Different types of content require different kinds of protocols Application
More informationIntroduction to Netflow
Introduction to Netflow Campus Network Design & Operations Workshop These materials are licensed under the Creative Commons Attribution-NonCommercial 4.0 International license (http://creativecommons.org/licenses/by-nc/4.0/)
More informationConfiguring IP Services
This module describes how to configure optional IP services. For a complete description of the IP services commands in this chapter, refer to the Cisco IOS IP Application Services Command Reference. To
More informationReport. Middleware Proxy: A Request-Driven Messaging Broker For High Volume Data Distribution
CERN-ACC-2013-0237 Wojciech.Sliwinski@cern.ch Report Middleware Proxy: A Request-Driven Messaging Broker For High Volume Data Distribution W. Sliwinski, I. Yastrebov, A. Dworak CERN, Geneva, Switzerland
More informationScrutinizer Flow Analytics
Scrutinizer Flow Analytics TM Scrutinizer Flow Analytics Scrutinizer Flow Analytics is an expert system that highlights characteristics about the network. It uses flow data across dozens or several hundred
More informationNetFlow Configuration Guide
Americas Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 527-0883 THE SPECIFICATIONS AND INFORMATION
More informationTraffic Flow Measurements within IP Networks: Requirements, Technologies and Standardization
Traffic Flow Measurements within IP Networks: Requirements, Technologies and Standardization Jürgen Quittek NEC Europe Ltd., Network Laboratories, Heidelberg, Germany Tanya Szeby, Georg Carle, Sebastian
More informationImplementing Access Lists and Prefix Lists
An access control list (ACL) consists of one or more access control entries (ACE) that collectively define the network traffic profile. This profile can then be referenced by Cisco IOS XR softwarefeatures
More informationAbsolute Analysis Investigator Architecture Fibre Channel Solutions
One Architecture. Many Products. Multiple Protocols. Absolute Analysis Investigator (AAI) features the most flexible, scalable and versatile instrument for protocol test and verification. The architecture
More informationOverview of the NetFlow FlowAnalyzer
CHAPTER 1 Overview of the NetFlow FlowAnalyzer NetFlow FlowAnalyzer Version 2.0 is a network analysis tool that you can use to display and analyze network traffic information collected from Cisco NetFlow-enabled
More informationSD-WAN Deployment Guide (CVD)
SD-WAN Deployment Guide (CVD) All Cisco Meraki security appliances are equipped with SD-WAN capabilities that enable administrators to maximize network resiliency and bandwidth efficiency. This guide introduces
More informationCCNA Exploration Network Fundamentals. Chapter 03 Application Functionality and Protocols
CCNA Exploration Network Fundamentals Chapter 03 Application Functionality and Protocols Updated: 27/04/2008 1 3.1 Applications: The Interface Between Human and Networks Applications provide the means
More informationCisco Service Control Business Intelligence Solution Guide,
CISCO SERVICE CONTROL SOLUTION GUIDE Cisco Service Control Business Intelligence Solution Guide, Release 4.1.x 1 Overview 2 Features Revised: December 23, 2013, OL-30603-01 Note This document supports
More informationStreamlining CASTOR to manage the LHC data torrent
Streamlining CASTOR to manage the LHC data torrent G. Lo Presti, X. Espinal Curull, E. Cano, B. Fiorini, A. Ieri, S. Murray, S. Ponce and E. Sindrilaru CERN, 1211 Geneva 23, Switzerland E-mail: giuseppe.lopresti@cern.ch
More informationNetFlow-based bandwidth estimation in IP networks
NetFlow-based bandwidth estimation in IP networks Rodrigo Alarcón-Reyes Department of Electrical and Computer Engineering McGill University March 3, 22 Outline. Introduction to NetFlow 2. Definition of
More informationNetwork Management and Monitoring
Network Management and Monitoring Introduction to Netflow These materials are licensed under the Creative Commons Attribution-Noncommercial 3.0 Unported license (http://creativecommons.org/licenses/by-nc/3.0/)
More informationIP Access List Overview
Access control lists (ACLs) perform packet filtering to control which packets move through a network and to where. The packet filtering provides security by helping to limit the network traffic, restrict
More informationUsing Diagnostic Tools
Using Diagnostic Tools The Tools System Diagnostics page on the INVESTIGATE view provides several diagnostic tools that help troubleshoot various kinds of network problems and process monitors. Tech Support
More informationThis chapter describes how to configure NetFlow Data Export (NDE).
51 CHAPTER This chapter describes how to configure NetFlow Data Export (NDE). Note For complete syntax and usage information for the commands used in this chapter, refer to these publications: The Cisco
More informationDelay Controlled Elephant Flow Rerouting in Software Defined Network
1st International Conference on Advanced Information Technologies (ICAIT), Nov. 1-2, 2017, Yangon, Myanmar Delay Controlled Elephant Flow Rerouting in Software Defined Network Hnin Thiri Zaw, Aung Htein
More informationQuestion Score 1 / 19 2 / 19 3 / 16 4 / 29 5 / 17 Total / 100
NAME: Login name: Computer Science 461 Midterm Exam March 10, 2010 3:00-4:20pm This test has five (5) questions. Put your name on every page, and write out and sign the Honor Code pledge before turning
More informationManaging Caching Performance and Differentiated Services
CHAPTER 10 Managing Caching Performance and Differentiated Services This chapter explains how to configure TCP stack parameters for increased performance ant throughput and how to configure Type of Service
More informationIMC Network Traffic Analyzer 7.1 (E0301P04) Copyright (c) 2015 Hewlett-Packard Development Company, L.P. All Rights Reserved.
Network Traffic Analyzer 7.1 (E0301P04) Copyright (c) 2015 Hewlett-Packard Development Company, L.P. All Rights Reserved. Table of Contents 1. What's New in this Release 2. Problems Fixed in this Release
More informationNetFlow Traffic Analyzer
GETTING STARTED GUIDE NetFlow Traffic Analyzer Version 4.5 Last Updated: Monday, December 3, 2018 GETTING STARTED GUIDE: NETFLOW TRAFFIC ANALYZER 2018 SolarWinds Worldwide, LLC. All rights reserved. This
More information15-441: Computer Networks Homework 3
15-441: Computer Networks Homework 3 Assigned: Oct 29, 2013 Due: Nov 12, 2013 1:30 PM in class Name: Andrew ID: 1 TCP 1. Suppose an established TCP connection exists between sockets A and B. A third party,
More informationThis document describes the sampled flow (sflow) feature and configuration steps to implement sflow.
This document describes the sampled flow (sflow) feature and configuration steps to implement sflow. Prerequisites for, page 1 Restrictions for, page 1 Information About sflow, page 1 How to Provision
More informationNetwork Design Considerations for Grid Computing
Network Design Considerations for Grid Computing Engineering Systems How Bandwidth, Latency, and Packet Size Impact Grid Job Performance by Erik Burrows, Engineering Systems Analyst, Principal, Broadcom
More informationDesign and Implementation of A P2P Cooperative Proxy Cache System
Design and Implementation of A PP Cooperative Proxy Cache System James Z. Wang Vipul Bhulawala Department of Computer Science Clemson University, Box 40974 Clemson, SC 94-0974, USA +1-84--778 {jzwang,
More informationOASIS: Self-tuning Storage for Applications
OASIS: Self-tuning Storage for Applications Kostas Magoutis, Prasenjit Sarkar, Gauri Shah 14 th NASA Goddard- 23 rd IEEE Mass Storage Systems Technologies, College Park, MD, May 17, 2006 Outline Motivation
More informationConfiguring NetFlow BGP Next Hop Support for Accounting and Analysis
Configuring NetFlow BGP Next Hop Support for Accounting and Analysis This document provides information about and instructions for configuring NetFlow Border Gateway Protocol (BGP) next hop support. This
More informationConfiguring QoS CHAPTER
CHAPTER 37 This chapter describes how to configure quality of service (QoS) by using automatic QoS (auto-qos) commands or by using standard QoS commands on the Catalyst 3750-E or 3560-E switch. With QoS,
More informationIntroduction to Routers and LAN Switches
Introduction to Routers and LAN Switches Session 3048_05_2001_c1 2001, Cisco Systems, Inc. All rights reserved. 3 Prerequisites OSI Model Networking Fundamentals 3048_05_2001_c1 2001, Cisco Systems, Inc.
More informationSystem Requirements. Things to Consider Before You Install Foglight NMS. Host Server Hardware and Software System Requirements
System Requirements This section contains information on the minimum system requirements for Foglight NMS. Before you can begin to download Foglight NMS, you must make sure that your computer meets the
More informationAppendix PERFORMANCE COUNTERS SYS-ED/ COMPUTER EDUCATION TECHNIQUES, INC.
Appendix E PERFORMANCE COUNTERS SYS-ED/ COMPUTER EDUCATION TECHNIQUES, INC 1 Default s for Commonly Used Objects Object Default Cache Data Map Hits % How often requested data is found in the cache This
More information-sflow. MRTG(Multi Router Traffic Grapher) MRTG NetFlow SNMP PNG HTML MRTG. Netflow. Statistic PC Collector. PC ( Statistic PC ) Analyzer.
TANET -sflow 1. MRTG Netflow MRTG(Multi Router Traffic Grapher) SNMP PNG HTML MRTG : Unix Windows MRTGPERL MRTG C PNG GD PNG MRTG Web Cisco Netflow (traffic) Cisco Netflow : Cisco Router flow-export function
More informationFlow Sampling for ASR1K
LIVEACTION, INC. Flow Sampling for ASR1K CONFIGURATION LiveAction, Inc. 3500 Copyright WEST BAYSHORE 2016 LiveAction, ROAD Inc. All rights reserved. LiveAction, LiveNX, LiveUX, the LiveAction Logo and
More informationInformation about Network Security with ACLs
This chapter describes how to configure network security on the switch by using access control lists (ACLs), which in commands and tables are also referred to as access lists. Finding Feature Information,
More informationStorage Networking Strategy for the Next Five Years
White Paper Storage Networking Strategy for the Next Five Years 2018 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 1 of 8 Top considerations for storage
More informationDesign and Implementation of Measurement-Based Resource Allocation Schemes Within The Realtime Traffic Flow Measurement Architecture
Design and Implementation of Measurement-Based Resource Allocation Schemes Within The Realtime Traffic Flow Measurement Architecture Robert D. allaway and Michael Devetsikiotis Department of Electrical
More information