Rethinking The Building Block: A Profiling Methodology for UDP Flows

Transcription

1 Rethinking The Building Block: A Profiling Methodology for UDP Flows 123 Jing Cai 13 Zhibin Zhang 13 Peng Zhang 13 Xinbo Song 1 Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China 2 Graduate University of Chinese Academy of Sciences, Beijing, China 3 National Engineering Laboratory for Information Security, Beijing, China caijing@software.ict.ac.cn Abstract With the increase of network bandwidth, more and more new applications such as audio, video and online games have become the main body in network traffic. Based on realtime considerations, these new applications mostly use UDP as transport layer protocol, which directly increase UDP traffic. However, traditional studies believe that TCP dominates the Internet traffic and previous traffic measurements were generally based on it while UDP was ignored. In view of this, we mainly discuss the profiling methodology of UDP flows in this paper. First, from the view of the network layer, we present a profiling methodology on the basis of the Claffy s parameter flow model. Next, due to the significant differences among the different applications, we found that using a unified methodology and ignoring the information of application layer is not appropriate for UDP flows. At last, we get the conclusion that for different applications, such as dns, qq[16], we must use corresponding profiling methodology based on their specific characteristics. I. INTRODUCTION The main purpose of the network traffic measurement is to enhance people s awareness about traffic characteristics. The traffic measurement that works based on the network layer started from the 198s. Earlier studies took the packet as the Building Block. But due to its small granularity, it could not meet the needs in many ways. According to the locality of network traffic in time and space, Claffy et al.[1][2] firstly proposed a parameters flow model. Since then, the traffic measurement based on flow had gradually become a hot issue in the field of network measurement. However, in the past, during the process of network traffic measurement, people generally believed that TCP traffic occupied the main body of the network traffic, and UDP traffic is negligible, and therefore ignored the measurements of the UDP flow. However, the situation has undergone tremendous changes at present. With the increase of network bandwidth, the traditional networking services based on images and text could no longer satisfy people s needs. More and more audio, video, and online games, have gradually become the main body of the network traffic. These applications mostly use UDP as their transport layer protocol[3], which directly results in the increase of UDP traffic. The organization of CAIDA[4] analyzed the trace collected in the period on several backbone links located in the US and Sweden, found the ratio between the UDP and TCP in packets, bytes, and flows have increased greatly. In China, as shown in Fig.1, after statistical analysis on the 24-hour data we collected from a backbone router, we found that the proportion of the UDP packets is around 4%. It could sometimes reach 8% at most. From the view of the proportion of UDP bytes, we also found that the proportion of the UDP bytes fluctuated greatly, but generally exceeded 4%, and sometimes was even above 8%. Since the increase of UDP traffic, more and more people have started to pay attention to the traffic of UDP. However, compared with the TCP, we found there at least exist two big differences. Firstly, TCP is a connection-oriented protocol, it has controlling flags such as FIN and RST to explicitly identify the end of flow. But for UDP, it is a connectionless protocol. In essence, there is no concept of flow in udp. At present, people commonly refer to UDP flows as a series of packets which are required to complete a data transfer from the view of the application layer. Therefore, Claffy s flow model is still applicable for UDP. But in this case, the parameters setting is not the same as before. The second, compared with TCP, the composition of UDP is more complicated. In the past, in measurement of TCP flows and from the view of the network layer, people do not care about the specific application services, so the parameters of flow model are generally identical. However in UDP, the characteristics of different applications often demonstrate significant differences. Using uniformed parameters to profile flow without considering the assumption on application layer is feasible for TCP flows, but will bring many uncertainties to the connectionless UDP flows. Due to these two great differences, earlier network measurement based on flows mostly focus the TCP flows, while UDP flows was ignored. The study on the UDP flows is nearly in the blank stage. In view of these, we mainly discuss the profiling methodology of UDP flow in this paper. To the best of our knowledge, we are the first to do so. There are two main contributions in our paper. Through our comprehensive considerations on each evaluation criteria, we built the appropriate profiling methodology on the basis of Claffy s parameter flow model from the view of the network layer. Taking the complexity of UDP into account, due to the significant differences between the application protocols, we want to know whether it is enough to use a unified

2 the ratio between the udp and ip The proportion of the udp s packets and bytes UDP pkt / IP pkt UDP byte / IP byte TABLE I: The basic information of the trace Id Begin time End time Bytes Packets I ,14: ,:3 275G 285(million) However, earlier studies believe that TCP dominates the Internet traffic, thus their study generally based on TCP flows while UDP flows was ignored. Therefore, we will present our profiling methodology of UDP flows on the basis of Claffy s parameter flow model in this paper..1 :5:57 4:25:57 11:15:57 14:35:58 18:5:58 21:35:58 time Fig. 1: The proportion of the UDP s packets and bytes profiling methodology to measure the characteristics of different applications. After our statistics and analysis, we found there indeed exist significant differences between the profiling methodologies of different applications. We were unable to use a unified methodology to process UDP flow as a whole. In contrast, we must use corresponding profiling methodology based on the characteristics of applications. The remainder of this paper is organized as follows: Section 2 presents some related work on the profiling methodologies of flows. In section 3, we discuss the profiling methodology from the view of the network layer. In section 4, from the view of the application layer, considering the great differences between the different applications, we want to know whether exists a unified profiling methodology to process UDP flows. At last, we conclude this paper and give some suggestions in Section 5. II. RELATED WORKS The studies of earlier traffic measurement took the packets as the Building Block. But as D.Clark[5]pointed out that, the traffic measurements based on packets can not reflect the relation between the packets and the higher-level information. Thus, it can not meet the need for understanding network traffic in many ways. Jain[6] proposed a model named Packet Train. In this model, a series of packets which have the same source and destination address will be identified as a packet train. If the interval between two packets exceeds a specified fixed timeout value, then these packets are said to belong to separate train. Claffy et al.[1][2] first propose a parameters flow model, in which flow was taken as a series of packets that was consistent with a specific flow specification and timeout constraint. By analyzing the common characteristics of the packets which belong to the specific flow, we can better understand and analyze network behavior from a higher layer, and also provide enough information support for network application designing. From then on, the network measurement based on flow had became more and more popular. III. PROFILING METHODOLOGY FROM THE VIEW OF THE NETWORK LAYER A. Date set We collected the data trace from a backbone router in China. The basic information of this trace is in Table I. The reason for not using the data set of CAIDA is that its payload information has been encrypted. And in this paper, we must use the payload information to classify the different applications. B. profiling methodology During the measurement based on flows, how to decide the boundary of flow is critical. There exist two main approaches. The first, controlling flags(etc., FIN and RST) are used to terminate the flow[7]. The second is the timeout strategy. If a flow become inactive beyond a given timeout, it is deemed to be ended and should be removed from memory. The timeoutbased method does not rely on the explicit protocol labels in packet header, thus it can deal with the TCP flows and UDP flows. In this case, although UDP is a connectionless protocol, Claffy s parameter flow model is still applicable. What we need to do is to find out the most appropriate parameters. We formally define UDP flows as a series of continuous arriving UDP packets which have the same five-tuple(source address, destination address, source port, destination port, transport protocol), and their packets inter-arrival do not exceed a specified fixed timeout. In this paper, we assume our scenario is located in a backbone router. Among the above six parameters, the selection of timeout is most difficult and important. The timeout determines the boundary of the flow, so it directly impact on the accuracy of flow measurement. If a smaller timeout value is selected, it can cause more long flows to be segmented into multiple short flows. If a larger timeout is selected, it may result in more unnecessary measurement overhead. For TCP, because it has controlling flags which can make up the impact of setting a inappropriate timeout. But for UDP, the selection of the timeout is critical. Claffy[1][2] proposed a method to solve the problem of selecting the fixed timeout value. The method was, on the same trace, under a fixed flow specification, using different timeout between 2-248s to profile flow, the differences between the flow set can entirely be determined by the timeout. Through experiments, Claffy[1][2] pointed out that for the host pair flows of the backbone network, at the timeout value of 64s, it seems reaching a optimal outcome because it performs well

3 in each indicators. Iannaccone s work[8] also shows that the timeout value in range of 6-12s will provide a reasonable estimation of flow numbers. Claffy s experiments result are widely recognized and referenced by a large number of subsequent papers. However, the network environment and the network bandwidth have undergone great changes since then. As we said earlier, the traffic of UDP has increased greatly. Therefore, is Claffy s result still fit for today s circumstances? In the next section, we try to answer this question. In order to better evaluate the flow set profiled by fixed timeout, we establish some evaluation criteria based on the considerations of various aspects. Through the comprehensive considerations of its performances on each evaluation criteria, we select the optimal timeout value. The evaluation criteria and their corresponding experiment results are as follows: C. Active avg and New avg We present two indicators Active avg, New avg. Active avg presents the average number of active flows staying in the measurement system per second. While New avg presents the average number of new flows staying in the measurement system per second. The measurement system are dynamic termination and recreation of UDP flows. Therefore, if the New avg is larger, it means that the measurement system must establish and destruct flows more frequently, which will consume a lot of computing resources and lead to the thrashing. While the Active avg is larger, it means that it needs much memory resources to maintain the flows information and bring the time used to find flow record increase. Therefore, for the application based on flows, its ideal state is that the Active avg and New avg are both small. If it is not possible to achieve, we should choose the timeout which makes the Active avg and New avg achieving a equilibrium state. Fig.2(a) reveals the relationship that the average(new avg ) and the 95th percentile number of new flows versus the timeout value per second. Fig.2(b) shows the relationship that the max and average(active avg ) number of active flows versus the timeout value per second. The New avg is a decreasing function of timeout value, while the Active avg is an increasing function of timeout value. Therefore, we can only choose a timeout value which makes the Active avg and New avg achieve the equilibrium state. Suppose the cost for a router to handle a new flow and a active flow are N and A respectively. In per second, the New avg and Active avg for a router to deal with are n and a respectively. Therefore, we get the total consumption: cost = N n + A a. (1) Define F = N/A. Fig.3 shows the relationship between the cost and the timeout value under several different F value. As Fig.3 shows, when timeout value is smaller than 64s, its total cost is a decreasing function of the timeout value. Conversely, when timeout value is larger than 64s, it turns to a increasing function of the timeout value. Therefore, its total cost get its minimum when the timeout value equal 64s. new flows active flows 1 5 (a)median and 95th percentile of new flows per second % new flows median new flows (b)median and Maximum of active flows per second max active flows median active flows tiemout(sec) Fig. 2: As a function of timeout value(a)average and 95th percentile of new flows per second(b)average and maximum of active flows per second Total cost versus the timeout value F = 2 F = 3 F = 4 flow Fig. 3: Total cost versus timeout value for various values of the ratio of flow setup cost to maintenance cost(f) D. Recreated avg We present this indication Recreated avg based on the consideration of the computing consumption. It presents the average number of recreated for one flow. Many studies based on flow information such as load-sensitive routing require larger proportion of the large flows (the packets in flow should be greater than 1 packets). Low Recreated avg is desirable to ensure no truncation in the long original flows. This indicator represent the burden of the computing consumption. Fig.4 shows the experiment results versus the timeout value. The Recreated avg is a decreasing function of timeout value. At the timeout value of two, each unique flow is set up and tore down almost three times on average. The smaller the timeout value, the higher the Recreated avg. This phenomenon is similar with our expectation that flows are more likely to be cut into multiple short flows when the timeout value is small.

4 ratio(recreated/unique) The average number of recreated versus the timeout value TABLE II: The classification of the UDP flows Category Representative Protocol SERVICE {dns,ntp,messengerservice} IM {qq,msn} DOWNLOAD {bittorrent,edonkey,xunlei,guntella,kazaa} STREAMING MEDIA {pplive,ppstream,sopcast,qqlive} OTHER {unknown}.5 Fig. 4: The average number of recreated versus the timeout value fhold factor The Fhold factor veruse the timeout value 1 Fig. 5: The F hold factor versus the timeout value E. F hold factor We introduced a F hold indicator presented by Ryu[9] based on the consideration of the memory consumption. Define F hold = D hold /D act, D hold represents the sum of the flow duration and the flow timeout, D act represents the flow duration. The key notion behind this indicator is that for each flow, it can not immediately end after processing the last packet, and it must wait for a specified timeout. Only after that, the flow was determined to the end and removed from memory. The smaller ratio between the time wasted in memory(the flow timeout) and the useful time (the flow duration) reveals that the measurement system is more efficient. And this is exactly what we look forward to. For router, F hold represents the burden of memory resource. For the flow set formed by N flows. The average F hold [9] is calculated as: F hold =( 1 N 1 N n= Fhold (n)) 1 (2) We use the inverse of F hold (n) and then take another inverse of the final result to take care of single-packet flows for which F hold (n) becomes infinite. In common sense, Fhold > 1., its ideal state is F hold =1. Fig.5 shows the F hold factor value versus the timeout value. As Fig.5 shows, at the timeout value of 16s, the F hold factor gets its minimum. The F hold factor for timeout 64s is 16.96, 5% higher than the minimum F hold factor, and ranking sixth. F. implication Through our comprehensive considerations, for UDP flows, we finally present its most appropriate profiling methodology in which the timeout value was set as 64s. The reason for selecting 64s as its timeout value is that its total cost spending on the maintenance,termination and creation of flow record is minimum, and its corresponding value of Recreated avg and the F hold factor are both minor. IV. PROFILING METHODOLOGY FROM THE VIEW OF THE APPLICATION LAYER A. Classification of the UDP Compared with TCP, the composition of UDP is more complicated, it contains many different data elements. In UDP, there not only exists some streaming media protocols such as ppstream[1], pplive which seem to be appropriate to profile flows, but also exists protocols such as dns which only involves question-answer and seems not to be suitable to profile flows. Due to this great differences among the applications, it is difficult to deal with it as a whole. Therefore, we divided the whole UDP flows into five categories according to its different characteristics. Table II shows the basic information of the classification. The whole UDP flows have been divided into five categories named {SERVICE, IM, DOWNLOAD, STREAMING ME- DIA, OTHER}. For common UDP flows(especially for some peer-to-peer applications), the traditional traffic identification method based on well known ports will lead to inaccurate judgement. Fortunately, some open source software such as L7-filter[12], OpenDPI provides features information of some popular application protocols. We draw this information to analyze the traffic of these most important applications in UDP. B. Profiling methodology for different applications 1) Active avg and New avg for different applications: As Table II shows, we divide UDP protocols into five categories. Among the five categories, we select xunlei[11], qq, dns, ppstream these four protocols as the representation protocols of their corresponding categories. We conducted experiments on these representation protocols respectively to find out their most suitable profiling methodologies and examine the difference between them. Fig.6 shows their optimal profiling methodologies from the view of the Active avg and New avg. It evidently shows that differences indeed exist among the different applications. For xunlei flows and ppstream flows, their suitable timeout value are almost the same as the timeout value drawn from the view

5 the average number of recreated versue tiemout value dns flows qq flows ppstream flows xunlei flows 1 2 the F(hold) factor versue the tiemout value dns flows ppstream flows qq flows xunlei flows ratio(recreated/unique) 1 F(hold) factor Fig. 7: The average number of recreated versus the timeout value for different applications 1 Fig. 8: The F hold factor versus the timeout value for different applications of the network layer, which is also set as 64s. But for qq flows and dns flows, their situations are completely different, and their suitable timeout vary widely. For dns flows, its most suitable timeout value is smaller than the timeout value drawn from the view of the network layer. As Fig.6(a) shows, its total cost stabilized when timeout between 2s and 32s. Once the timeout value is larger than 32s, its total cost increases greatly. Therefore, for dns flows, the timeout value between 2s-32s is a more reasonable result. For qq flows, its most suitable timeout value is larger than the timeout value drawn from the view of the network. As Fig.6(b) shows, when timeout value is smaller than 128s, its total cost decrease greatly with the increase of the timeout value. However, its total cost increases slightly when the timeout value is larger than 128s. Therefore, we get the conclusion that its total cost reaches its minimum when the timeout value between 128s-256s. 2) Recreated avg for different applications: Fig.7 shows the results of the Recreated avg over different applications, which is drawn in a log scale. Fig.7 reveals that for different applications, their Recreated avg is also a decreasing function of the timeout value. For dns flows, compared with other applications, its corresponding Recreated avg is always minimum. In most cases, its Recreated avg does not exceed.5, and its maximum of Recreated avg does not exceed.65. For ppstream flows and xunlei flows, their Recreated avg decrease rapidly when the timeout value is smaller than 64s. Their Recreated avg firstly fall below 1 at the timeout value of 64s. But for qq flows, its corresponding timeout value is 256s. At that timeout value, its Recreated avg firstly fall below 1. 3) Fhold factor for different applications: Fig.8 shows the results of F hold factor over different applications, which is also drawn in a log scale. For dns flows, its F hold factor is always greater than other application whatever the timeout is. In most cases, its F hold factor is around 55, and sometimes it even exceeds 8. For ppstream flows, its corresponding F hold factor is always minimum. In most case, its F hold does not exceed 3, and its maximum does not exceed 4. While for the qq flows and xunlei flows, their F hold factor value lies between the dns flows and the ppstream flows whatever the timeout value is. For ppstream, qq, xunlei flows, their F hold factor get theirs minimum at the timeout value of 64s, 128s, 32s separately. But for dns flows, the situation is completely different. When the timeout value is less than 64s, its F hold factor get its minimum at the timeout value of 8s. But to our surprise, the F hold factor is a decreasing function of timeout value when the timeout is larger than 64s. At the timeout value of 248s, its F hold reaches its minimum. We think this may determined by the nature characteristic of the dns protocol. C. implication From above experiment results, we get the following conclusions: The first, for dns flows, in most cases, it only involves one request packet and one response packet in one request and seems to be independent between many requests. Due to this characteristic, the majority of dns flows are short flows. Generally speaking, the phenomenon of thrashing caused by excessively frequent termination and creation of flows is evident for large flows and not obvious for short flows. This is the reason why the Recreated avg is always minimum and the F hold factor is always maximum. Therefore, its corresponding timeout value should be smaller. We get its optimal profiling methodology in which its timeout value is set as 8s. The second, for qq flows, due to its larger packets interarrival caused by intermittent chat, its corresponding timeout value should be larger. The unified profiling methodology is not appropriate for qq flows. We present its optimal profiling methodology in which its timeout is set as 256s. The third, for other applications, the profiling methodology of setting timeout value at 64s is appropriate. It can accurately measure the flows characteristics and not bring excessive overload to the measurement system. From above analysis, we get the conclusion that there indeed exists significant differences between the profiling methodology of different applications. Therefore, we can not use a unified profiling methodology to process UDP flows.

6 (a)dns protocol F = 2 F = 3 F = (b)qq protocol F = 2 F = 3 F = (c) xunlei protocol F = 2 F = 3 F = (d) ppstream protocol F = 2 F = 3 F = Fig. 6: Total cost versus timeout value for various values of the ratio of flow setup cost to maintenance cost(f) for different applications (a)dns protocol(b)qq protocol (c)xunlei protocol (d)ppstream protocol V. CONCLUSION In this paper, we mainly discussed the profiling methodologies of UDP flows. First, we give the definition of UDP flows on the basis of Claffy s parameter flow model. Among the many parameters, we mainly discuss the selection of the timeout value. From the view of the network layer, through the comprehensive considerations of related evaluation criteria such as Active avg, New avg, Recreated avg and F hold, we find out the most appropriate profiling methodology in which the timeout value was set as 64s. Secondly, when taking the complexity of UDP into account, due to the great differences between the different applications, we validate that using a unified profiling methodology is not enough for accurately measuring the different applications such as dns, qq. Therefore, we get the conclusion that we can not use a unified profiling methodology to process UDP flows. In contrast, we must establish appropriate profiling methodology based on the characteristics of applications. ACKNOWLEDGMENT Our work is supported in part by the National Basic Research Program 973 of China(Grant No.27CB3111) and the National Science Foundation of China(Grant No ). REFERENCES [1] K.C. Claffy, Internet traffic characterization. [Ph.D. Thesis],San Diego: University of California, [2] K.C. Claffy, H.W. Braun, and G.C. Polyzos, A Parameterizable Methodology for Internet Traffic Flow Profiling[J]. IEEE Journal on Selected Area in Communications, 1995,13(8): [3] K. Sripanidkulchai, B. Maggs, and H. Zhang, Analysis of Live Streaming Workloads on the Internet. In Proc. of IMC 4, October, 24, pp [4] CAIDA. [5] D.D. Clark, The design philosphy of the Darpa Internet protocols. In Proc. of ACM SICCOMM 88, Aug. 1988, pp [6] R. Jain and S.A. Routhier, Packet Trains-Measurements and a New Model for Computer Network Traffic. IEEE Journal on Selected Area in Communications, Vol. SAC-4, No. 6, Sep, 1986, pp [7] N. Hohn and D. Veitch, Inverting sampled traffic. In Proc. of the 3rd ACM SIGCOMM Conf. on Internet Measurement [8] G. Iannaccone, C. Diot, and I. Graham, Monitoring very high speed links. In Proc. of the First ACM SIGCOMM Workshop on Internet Measurement Workshop 21, San Francisco, California, USA, November 21, pp [9] B. Ryu, D. Cheney, and H.W. Braun, Internet flow characterization: adaptive timeout strategy and statistical modeling. In: Workshop on Passive and Active Measurement (PAM), 21. [1] ppstream. [11] xunlei. [12] l7-filter. [13] TRFC. [14] M. Rey, Transmission control protocol. RFC793,1981. [15] J. Mahdavi, and S. Floyd, TCP-Friendly Unicast Rate-Based Flow Control. Note sent to end2end-interest mailing list, Jan [16] qq.