A Novel Approach for Practical, Real-Time, Machine Learning Based IP Traffic Classification

Transcription

1 A Novel Approach for Practical, Real-Time, Machine Learning Based IP Traffic Classification Dissertation submitted in accordance with the requirements for the degree of Doctor of Philosophy Thuy T.T. Nguyen Centre for Advanced Internet Architectures Faculty of Information and Communication Technologies Swinburne University of Technology Melbourne, Australia February 2009

2 Declaration To the best of my knowledge and belief, this thesis contains no material previously published or written by any other person, except where due reference is made in the text of the thesis. This thesis has not been submitted previously, in whole or in part, to qualify for any other academic award. The content of the thesis is the result of work which has been carried out since the beginning of my candidature on March Melbourne, 23 rd February 2009 Thuy Nguyen 2

3 A Novel Approach for Practical, Real-Time, Machine Learning Based IP Traffic Classification

4 4 c Thuy Nguyen All rights reserved.

5 To my beloved family, especially my husband Uy Dzung and my little daughter Khiet Linh!

6 Acknowledgements Working towards this PhD was a long and challenging journey, and I would like to thank the following people for making it possible. First of all, I would like to express my sincere gratitude and appreciation to my first supervisor, Professor Grenville Armitage. Throughout the years he has been a great mentor to me. I have experienced both successful and frustrating experimental outcomes, sometimes losing my track, and it was his guidance, support, encouragement, enthusiasm and passion that helped me stay inspired and motivated. I feel so grateful to have had a supervisor who is willing to stand up for his students; who tries hard to provide a great working environment with all the necessary facilities and equipment for our experiments and research; and who creates opportunities for us to present our work and establish networking connections at both local and international conferences and workshops. Finally, I would like to thank him for all his patience during many long hours of discussions and experiments, for teaching me how to do good research, how to write a good paper, etc. All of these have really built a solid grounding for my future research career. I would like to thank Dr Philip Branch, my colleague and recently my second supervisor, who has always been willing to provide me with help, support and advice when needed. I am thankful for the encouragement he has given me since the early days of starting my thesis. I really appreciate all the time he spent helping me review my work and offering valuable suggestions and feedback. I would also like to thank Dr Jim Lambert (who was my second supervisor for the first two years of my candidature and is now retired) for his support of my work. I owe special thanks to my colleagues, Sebastian Zander and Nigel Williams, for the inspiration of their work, that ultimately led me to my thesis. My thanks to them for always being so helpful and kind to me over the years. I deeply thank Warren Harrop and Lawrence Stewart, for

7 7 their kindness and generosity in giving me the VoIP data trace collected at their home network to support my research. I would also like to thank Dragi Klimovski for giving me the opportunity to attend and study his Cisco CCNP class to widen my knowledge and gain experience which benefited my research. To my other colleagues at the Centre for Advanced Internet Architectures, I must say that I have been so lucky to have a chance to work in a great research environment, with such very smart, helpful and nice people thank you all! I would like to thank the Swinburne IT Services Department and the Centre for Astrophysics and Supercomputing for providing the laboratory equipment that facilitated my research. I would also like to thank the Centre for Advanced Internet Architectures, Cisco Systems Australia, and Swinburne University of Technology for awarding me the Swinburne University Postgraduate Research Award (SUPRA) and for providing funding support for the duration of my candidature. I would like to thank my husband Uy Dzung for walking with me on this journey with his infinite support, love, and encouragement. Thanks to my parents, my sisters and my parentsin-law for their patience and understanding. Completing this PhD would not have been possible without the friendship of many special people. This is a sincere thank you to all of them. Last but not least, this thesis is specially dedicated to Khiet Linh, my dear little daughter, who has been separated from mommy for many months so that I could complete my thesis!

8 Contents Acknowledgements 6 Abstract 14 Publications 17 Table of Acronyms 19 1 Introduction 21 2 Application Context for ML Based IPTC Introduction The importance of IP traffic classification QoS issues over Last Mile networks QoS provisioning Internet QoS standards QoS-enabled solutions from industry Automated QoS solution The role of IP traffic classification Internet pricing Lawful interception Traffic classification metrics Positives, negatives, accuracy, precision and recall Byte and flow accuracy Limitations of packet inspection for traffic classification Port-based IP traffic classification Payload-based IP traffic classification

9 CONTENTS Classification based on statistical traffic properties Conclusion A Brief Background on Machine Learning A review of classification with Machine Learning Input and output of an ML process Different types of learning Supervised learning The Naive Bayes algorithm The C4.5 Decision Tree algorithm Clustering EM algorithm Evaluating supervised learning algorithms Evaluating unsupervised learning algorithms Feature selection algorithms Imbalanced datasets problem The application of ML in IP traffic classification Training and testing a supervised ML traffic classifier Supervised versus unsupervised learning Challenges for operational deployment A deployment scenario The operational challenges Accuracy Timely and continuous classification Directional neutrality Efficient use of memory and processors Portability and Robustness Conclusion IP Traffic Classification Using Machine Learning Introduction Clustering approaches

10 10 CONTENTS Flow clustering using Expectation Maximisation Automated application identification using AutoClass TCP-based application identification using Simple K-Means Identifying HTTP and P2P traffic in the network core Supervised learning approaches Statistical signature-based approach using NN, LDA and QDA algorithms Classification using Bayesian analysis techniques GA-based classification techniques Simple statistical protocol fingerprint method Hybrid approaches Comparisons and related work Comparison of different clustering algorithms Comparison of clustering versus supervised techniques Comparison of different supervised ML algorithms ACAS: Classification using machine learning techniques on application signatures BLINC: Multilevel traffic classification in the dark Pearson s Chi-Square test and Naive Bayes classifier Limitations of the reviewed works Timely and continuous classification Directional neutrality Efficient use of memory and processors Portability and Robustness My research goal Conclusion Training Using Multiple Sub-Flows for Real-Time IPTC Introduction My proposal My experimental approach Flows and features

11 CONTENTS Machine Learning algorithms Some statistical properties of ET traffic Constructing training and testing datasets ET traffic Other traffic Training with full-flow, testing with four different sliding windows Training with full-flow instances of more than 25 packets (called filtered full-flow), testing with a sliding window of N = 25 packets Training with individual sub-flow, testing with a sliding window of N = 25 packets Training with multiple sub-flows, testing with a sliding window of N = 25 packets Data processing Results and analysis Training with full-flows, testing with four different sliding windows Training with filtered full-flows, testing with a sliding window of N = 25 packets Training with individual sub-flows, testing with a sliding window of N = 25 packets Training with multiple sub-flows, testing with a sliding window of N = 25 packets Discussion Conclusion Clustering For Automated Sub-Flow Selection Introduction My proposal Step 1 - Sub-flow identification Step 2 - Sub-flows selection An experimental illustration of my proposal Step 1 - Sub-flow identification

12 12 CONTENTS Step 2 - Sub-flow selection Evaluation of classifiers trained with sub-flows selected by EM Results and analysis Accuracy Computational performance Summary of results Sampling for faster clustering The problem Down-sampling for the clustering proposal Results and analysis Discussion and future work Conclusion Training Using Synthetic Sub-Flow Pairs Introduction Proposal using a synthetic sub-flow pairs approach Illustrating the Synthetic Sub-Flow Pairs Training Approach Experimental data Test methodology Results and analysis Classifying without training on SSP Training on SSP Option 1, classifying with a sliding window Training on SSP Option 2, classifying with a sliding window Conclusion Training Using SSP-ACT Introduction Evaluation of SSP-ACT in identifying VoIP traffic A brief background on ITU-T G.711 PCMU and GSM encoded voice traffic Data collection and research methodology Statistical properties of VoIP flows

13 CONTENTS Results and analysis Evaluation of SSP-ACT in the presence of additional packet loss Impact of packet loss on the classification of ET traffic Impact of packet loss on the classification of VoIP traffic Concurrent classification of multiple applications with SSP-ACT Discussion Conclusion Conclusion 214 Bibliography 218 List of Figures 240 List of Tables 240 A Traffic Characteristics of Selected Internet Applications 251 A.1 Asymmetric properties in bi-directional communication A.1.1 Client server ports asymmetry A.1.2 Statistical Properties Asymmetry A.2 Variation of traffic statistics during flow lifetime B Summary of ML-Based IP TC works in the Literature 258 B.1 A summary of key points for each reviewed work B.2 A qualitative evaluation of the reviewed works C Some Properties of Data Used for Training and Testing 265 C.1 Geographical distribution of ET traffic C.2 Traffic mix for training and testing D Characteristics of VoIP Traffic 271 D.1 VoIP data extraction D.2 Statistical properties of G.711 and GSM flows E Trade-offs in Cluster Quality and Classifier Performance 274 E.1 Accuracy E.2 Computational performance

14 Abstract Today s Internet does not guarantee any bounds on packet delay, loss or jitter for traffic traversing its networks. Uncontrolled networks can easily lead to bad user experiences for those emerging applications that have more stringent Quality of Service (QoS) requirements. This suggests there is a vital need for an effective QoS-enabled network architecture, in which the network equipment is capable of classifying Internet traffic into different classes for different QoS treatments. Beyond technology, there are other issues related to a practical QoS solution for the Internet, including the challenges of minimising the deployment cost of QoS technologies and simplifying users experiences. Like other services, the Internet is expected to be user-friendly, simple and easy to understand, stable and available on request, predictable and transparent, and not requiring users to understand its underlying architecture in order to use the service. With an awareness of these issues, my thesis focuses on the automation of the QoS control process, particularly by means of an automated, real-time IP traffic classification (IPTC) mechanism. Traditional techniques for the identification of Internet applications are based either on the use of well-known registered port numbers or on payload-based protocol reconstruction. However, applications can use unregistered ports or encryption to obfuscate packet contents; and governments may impose privacy regulations that constrain the ability of third parties to lawfully inspect packet payloads. Newer approaches, on the other hand, classify traffic by learning and recognising statistical patterns in externally observable attributes of the traffic (such as packet lengths and inter-packet arrival times). State-of-the-art approaches look closely at the application of Machine Learning (ML) a powerful technique for data mining and knowledge discovery to the classification of IP traffic. However, before I began publishing my work no ML-based approach to IPTC properly considered the constraints of being deployed in real-time operational networks. Most publications on the use of ML algorithms for classifying IP traffic have relied on bi-directional, full-flow

15 15 statistics (from start until finish or time-out), while assuming that flows have an explicit direction implied by the first packet captured, or a known client-server relationship. Some other studies have tried classification using the first few packets of a flow. In contrast, most if not all real-world scenarios require a classification decision well before a flow has finished, using statistics derived from a small number of recent packets rather than from the entire flow. Classifiers may also have missed an arbitrary number of packets from the start of a flow, and be unsure of the direction in which the flow started. To overcome these problems, I propose and evaluate novel modifications to the current MLbased approaches. My goal is to achieve classification by using statistics derived from only the most recent N packets of a flow (for some small value of N). Because a target application s short-term traffic statistics vary within the lifetime of a single flow, I propose training the ML classifier on a set of multiple short sub-flows, each sub-flow being a collection of N consecutive packets extracted from full-flow samples of the target application s traffic. The sub-flows are picked from regions of the application s flow that have noticeably different statistical characteristics. I further augment the training set by synthesising a complementary version of every sub-flow in the reverse direction, since most Internet applications exhibit asymmetric traffic characteristics in the client-to-server and server-to-client directions. Finally, I propose a novel use of unsupervised ML algorithms for the automated selection of appropriate sub-flow pairs when examples of traffic are given from applications that we wish to classify. I combine my proposals into a training approach that I call Synthetic Sub-flow Pairs with the Assistance of Clustering Techniques (SSP-ACT). I demonstrate my optimisation when applied to the Naive Bayes and C4.5 Decision Tree ML algorithms, for the identification of an online game Wolfenstein Enemy Territory (ET) and VoIP traffic. My experiments showed that for ET, being trained using SSP-ACT and classifying using a small sliding classification window of 25 packets (roughly corresponds to 0.5 of a second in real-time), the Naive Bayes classifier achieved 98.9% median Recall and 87% median Precision, and the C4.5 Decision Tree classifier achieved 99.3% median Recall and 97% median Precision. My results also confirmed that classification performance is maintained even when the classification is initiated at an arbitrary point within a flow and is independent of the direction of the first packet captured. For VoIP, being trained using SSP-ACT and classifying on a sliding window of 25 packets

16 16 (approximately 0.25 seconds in real-time when there is voice traffic in both directions), the Naive Bayes classifier achieved 100% median Recall and 95.4% median Precision, and the C4.5 Decision Tree classifier achieved 95.7% median Recall and 99.2% Precision. I also study the impact of packet loss on SSP-ACT s performance, with 5% synthetic, random and independent packet loss. For Wolfenstein Enemy Territory traffic, 5% packet loss only degraded the Recall and Precision of both the Naive Bayes and C4.5 Decision Tree classifiers by less than 0.5%. For VoIP traffic, 5% packet loss did not manifest noticeable degradation on the Naive Bayes classifier s Recall and Precision. However, it degraded the C4.5 Decision Tree classifier s Recall and Precision by 8.5% and 0.1% respectively. Despite this degradation, median Recall and Precision of the C4.5 Decision Tree classifier still remained above 87% and 99% for all the tested positions of the sliding window. Deeper investigation of the sensitivity of the Naive Bayes and C4.5 Decision Tree classifiers with regards to packet loss is left for future research. This work also can be expanded in future with other loss rates and loss models. I also demonstrate that SSP-ACT is effective in identifying both ET and VoIP traffic concurrently, by using a single common classifier or two separate classifiers in parallel, one for each application. My results reveal that using a common classifier provides better Precision and Recall, with a trade-off in the classification speed. It also has several pros and cons compared to the latter option of using two separate classifiers. How SSP-ACT could scale to classify a larger number of applications simultaneously is a question that requires further study. My results show that SSP-ACT is a significant improvement over the previous, published state-of-the art for IP traffic classification. My present work has focused on IPTC of an online game and VoIP, and revealed a potential solution to the accurate and timely classification of traffic belonging to other Internet applications.

17 Publications A number of peer-reviewed papers have been published based on material and discussion in this thesis, as listed below. Peer-reviewed Journal Papers: T.T.T. Nguyen and G. Armitage, A Survey of Techniques for Internet Traffic Classification using Machine Learning, IEEE Communications Surveys & Tutorials, Vol. 10, No. 4, 2008 J. But, T.T.T. Nguyen, G. Armitage, The Brave New World of Online Digital Home Entertainment, IEEE Communications, May 2005 T.T.T. Nguyen, G. Armitage, Evaluating Internet Pricing Schemes - A Three Dimensional Visual Model, ETRI Journal, Vol. 27, No. 1, February Peer-reviewed Conference Papers: T.T.T Nguyen and G. Armitage, Clustering to Assist Supervised Machine Learning for Real-Time IP Traffic Classification, in Proc IEEE International Conference on Communications, pp Beijing, China, May T.T.T. Nguyen, G. Armitage, Synthetic Sub-flow Pairs for Timely and Stable IP Traffic Identification, in Proc. Australian Telecommunication Networks and Application Conference, Melbourne, Australia, December T.T.T. Nguyen, G. Armitage, Training on Multiple Sub-flows to Optimise the Use of Machine Learning Classifiers in Real-world IP Networks, in IEEE 31st Conference on Local Computer Networks, Tampa, Florida, USA, November 2006.

18 18 S. Zander, T.T.T. Nguyen, G. Armitage, Automated Traffic Classification and Application Identification using Machine Learning, Proc. IEEE 30th Conference on Local Computer Networks (LCN 2005), Sydney, Australia, November 2005 S. Zander, T.T.T. Nguyen, G. Armitage, Self-learning IP Traffic Classification based on Statistical Flow Characteristics, Passive Active Measurement Workshop (PAM) 2005, Boston, USA, March/April T.T.T.Nguyen, G. Armitage, Experimentally Derived Interactions between TCP Traffic and Service Quality over DOCSIS Cable Links, Proc. Of Global Internet and Next Generation Networks Symposium, IEEE Globecom 2004, Texas, USA, November T.T.T. Nguyen, G. Armitage, Quantitative Assessment of IP Service Quality in b Networks, The 3rd Workshop on the Internet, Telecommunications and Signal Processing (WITSP 04), Adelaide, Australia, December T.T.T. Nguyen, G. Armitage, Quantitative Assessment of IP Service Quality in b and DOCSIS networks, The Australian Telecommunication Networks and Applications Conference (ATNAC 2004), Sydney, Australia, December T.T.T. Nguyen, G. Armitage, Pricing the Internet - A Visual 3-Dimensional Evaluation Model, Proc. of Australian Telecommunications Networks and Applications Conference (ATNAC), Melbourne, Australia, December 2003

19 Table of Acronyms CM DBSCAN DiffServ DNS DOCSIS DS ET FPS FTP HTTP IMAP IntServ IP IPTC ISP Kbyte LI Mbps Mbyte ML MPLS NTP QoS Cable Modem Density Based Spatial Clustering of Applications with Noise Differentiated Services Domain Name System Data Over Cable Service Interface Specifications Downstream Wolfenstein Enemy Territory First Person Shooter File Transfer Protocol HyperText Transfer Protocol Internet Message Access Protocol Integrated Services Internet Protocol IP Traffic Classification Internet Services Provider Kilo byte, which is equal to 1024 bytes Lawful Interception Mega bits per second Mega byte, which is equal to 1024 Kbytes Machine Learning Multi Protocol Label Switching Network Time Protocol Quality of Service

20 20 RTT SMTP SSP-ACT US HFC CMTS ACK P2P ICMP MTU FN FP TP TN MSS Round Trip Time Simple Mail Transfer Protocol Synthetic Sub-Flow Pairs with the Assistance of Clustering Techniques Upstream Hybrid Fibre Coaxial Network Cable Modem Termination System Acknowledgement Peer to peer Internet Control Message Protocol Maximum Transmission Unit False Negatives False Positives True Positives True Negatives Maximum Segment Size

21 Chapter 1 Introduction The Internet has now been a part of our lives for more than 30 years, since the first public demonstration of the ARPANET network technology in 1972 [1]. Its use has been growing rapidly over the years with increases not only in the number of users [2][3], hosts and servers [4], networks and autonomous systems [5], but also in volume and types of traffic [6]. Traditional Internet applications, such as electronic mail, file transfer, and static-content web sites, are being joined by newer services such as IP telephony, real-time interactive audio and video conferencing, streaming of multimedia content, online games, and electronic commerce. This creates a wide range of household [7][8][9] and business Internet uses [10][11][12][13][14][15][16]. This expanding trend is driven further by the rapid development of computing and communications in portable forms (e.g. laptop computers, PDAs and cellular phones), along with new modes of Internet access (e.g. from dial-up to broadband to possible optical access networks in the future [17][18]), which will potentially spawn more new applications and services. With these developing trends, parameters such as the timeliness of data delivery, packet loss and variability in end-to-end packet delay (jitter) become more important for Internet quality of service (QoS). Traditional non-interactive applications, such as bulk data transfer (FTP), backup operations or database synchronising, can span their operations over a long period of non-peak time as background activities [19]. On the other hand, emerging interactive applications such as business transactions and Web surfing are delay-sensitive; waiting times for these are tolerable only to the order of seconds [20]. Even less tolerant to delay are those applications that need to satisfy human requirements for interactivity, such as real-time voice communication and networked online games. The delay limits for these application types are a fraction of a second 21

22 22 CHAPTER 1. INTRODUCTION [21][22][23]. Similarly, video performance can suffer from jerky appearance due to jitter and frame distortion resulting from packet loss [24]. For voice applications, the loss of two or more consecutive voice samples may result in noticeable degradation of voice quality [19]. In various studies, online game applications have also been shown to be sensitive to network delay, loss and jitter ( [25][26][27][28][29]). Finding viable solutions for QoS-enabled Internet has attracted considerable research effort since the early 1990s, with the introduction of the Integrated Services (IntServ) [30], Differentiated Services (DiffServ) [31], and Multi Protocol Label Switching (MPLS) [32] architectures. However, the introduction of these architectures has yet to make a significant impact on the QoS perceived by Internet end users. Most networks and applications are still dominated by Best- Effort services, in which the network provides no guarantee on the bounds of packet delay, loss or jitter. One reason for the poor uptake of these QoS approaches is the lack of an effective mechanism that allows applications to signal their explicit QoS requirements to the underlying network [33]. One option is to leave this task to the applications or to the users. However, it might be unreasonable to expect software developers to be aware of the network issues or to understand the underlying technologies and explicit network requirements for providing QoS for their applications. Furthermore, tying an application to a particular standard for QoS provisioning, or requiring complicated user intervention or knowledge may restrict its options for deployment [33][34]. An alternative solution is to shift QoS signalling from the application to the network [33][35]. In this approach, the network is equipped with intelligent devices that can automatically classify traffic in terms of QoS demands, and prompt the ISP s QoS control system to provide appropriate QoS treatment. There are also other issues beyond those related to technology to be faced in order to achieve a successful Internet QoS solution [36]. These are the challenges of minimising the deployment cost of QoS technologies and simplifying users experiences. For ISPs, implementation and operational costs must not exceed the revenues likely to be gained by deploying any new QoS scheme. ISPs may also resist deploying a complex technology if there are questions as to its reliability and operational effort [37][38]. For Internet users, the Internet is expected to be userfriendly, simple to understand, stable and available on request, predictable and transparent, and

23 23 should not require that users understand the underlying architecture in order to use the service [20][37][39][40]. The work of this thesis is motivated by the desire to find a good solution for Internet QoS. My literature review on the QoS problem space suggests that a network based, robust and automated real-time IP traffic classification technique is an important component for implementing QoS across the Internet. IP traffic classification (which I will refer to as IPTC) is the process of identifying and classifying an individual Internet application or a group of applications of interest. It can serve as a core part of an automated QoS-enabled architecture, assist the QoS signalling process by quickly identifying the traffic of interest, and trigger an automated QoS control system for allocation of network resources for priority applications. Real-time IPTC allows network operators to know in good time what is flowing over their networks, so they can react quickly in support of their various business goals. It also has the potential to support class-based QoS accounting and billing. More importantly, it can be done automatically by the network providers, and does not require users intervention or specialist knowledge about the underlying technologies. It can help to bring QoS to consumers in a user friendly way. Furthermore, IPTC can assist in automated intrusion detection [41][42]. Recently, governments have also been clarifying ISP obligations with respect to lawful interception (LI) of IP traffic [43]. IPTC is an integral part of ISP-based LI solutions [44][45][46]. Traditional techniques for identifying Internet applications are typically based on the use of well-known registered port numbers or on payload-based protocol reconstruction. However, applications may use unregistered ports or encryption to obfuscate packet contents and governments may impose privacy regulations constraining the ability of third parties to lawfully inspect packet payloads. Newer approaches classify traffic by learning and recognising statistical patterns in externally observable attributes of the traffic (such as packet lengths and inter-packet arrival times). In particular, state-of-the-art techniques include the application of Machine Learning (ML) a powerful technique for data mining and knowledge discovery to IPTC. However, the literature of ML-based IPTC has not properly considered the constraints of being deployed in real-time operational networks. Most published work has primarily focused on the efficacy of different ML algorithms when applied to entire datasets of IP traffic. Classifica-

24 24 CHAPTER 1. INTRODUCTION tion models typically rely on flow statistical properties measured over full-flows (from their start until they finish or are timed out); some more recent work has attempted classification using the first few packets of a flow. Yet in real networks, traffic classifiers must reach decisions well before a flow has finished, so that network operators can react quickly to support their various business goals, for example, for flow QoS mapping and priority treatment. The classifier may start (or restart) at an arbitrary time and may not see the beginning of a flow. An application s statistical behaviour may change over its flow lifetime; in addition there may be thousands of concurrent flows, and the classifier needs to operate with finite CPU and memory resources. Further, although this has not always been clearly stated in the literature, directionality has been an implicit attribute of the features on which ML classifiers were built and used. Application flows in many cases are defined as bi-directional, and the application s statistical features are calculated separately in the forward and backward (reverse) directions. Most work assumes that the forward direction is indicated by the first packet of the flow (on the basis that it is commonly the initial packet from a client to a server). Subsequent evaluations assume that the classifier sees the first packet of every flow, in order to calculate features with the correct sense of direction. However, a real-world classifier cannot be sure whether the first packet it sees (of any bi-directional flow of packets) is heading in the forward (client-to-server) or backward (server-to-client) direction. Because, for many Internet applications, the traffic is asymmetric in the client-to-server and server-to-client directions, this can lead to degraded classification performance. In contrast to previously published work, I consider not only the timeliness of an ML traffic classifier, but also its sustainability in performance when monitoring traffic flows at any point in their lifetime, given the constraints of limited physical resources. This makes the contribution of my work novel and unique. I propose that practical real-time traffic classifiers must accurately classify traffic in the face of a number of constraints: The classifier should use statistical methods (such as ML algorithms), since TCP/UDP port numbers may be misleading, and packet payloads may be opaque to direct interpretation.

25 25 ML classification should be done over a small sliding window of the last N packets (to minimise memory requirements and perform classification in a timely manner). The classifier must only use features that require low processing/computation cost. Applications may change their network traffic patterns during the life of a flow. The classifier must recognise flows already in progress since the beginning of a flow may be missed. The classifier does not need to know the direction the original flow takes. It can assume the forward direction is the direction of the first packet of the most recent N packets it has captured, regardless of whether this is from client to server or server to client. My research question, therefore, is to investigate the possibility of building practical MLbased real-time traffic classifiers that address all of the above requirements. In this thesis I propose a novel approach to ML-based IPTC that I call the Synthetic Sub- Flow Pairs with the Assistance of Clustering Techniques (SSP-ACT) training method. Instead of using the statistical properties of a flow calculated over its whole lifetime, or from its first few packets, I train the ML classifier on a set of short sub-flows (each sub-flow contains a number of consecutive packets extracted from full-flow examples of the target application s traffic). This allows the classifier to properly identify an application, regardless of where within a flow the classifier begins capturing packets. Dealing with the directionality issues, SSP-ACT further augments the training set by synthesising a complementary version of every sub-flow in the reverse direction (hence the synthetic sub-flow pairs term). The first packet of a sliding window can alternatively represent traffic between a client to a server or a server to a client. SSP-ACT trains the classifier to recognise the application either way. A limited number of representative sub-flows that best capture distinctive statistical variations of the full-flows are selected to train the classifier. SSP-ACT makes use of unsupervised clustering ML techniques to automate the selection process. I demonstrate the effectiveness of SSP-ACT by constructing an ML classifier designed to identify highly interactive online game traffic mixed with thousands of unrelated interfering

26 26 CHAPTER 1. INTRODUCTION traffic flows. I chose a popular First Person Shooter (FPS) game application (Wolfenstein Enemy Territory (ET) [47]), the traffic characteristics of which can change significantly over the lifetime of each flow, and are asymmetric in the client-to-server and server-to-client directions. I evaluate the generality of SSP-ACT with the classification of another Internet application, Voice over IP (VoIP) traffic. The characteristics of VoIP traffic differ remarkably from ET traffic, to be more stable over a flow s lifetime, and more symmetric in the forward and backward directions. I also perform a preliminary investigation on the impacts of 5% random, independent packet loss on the classification of VoIP and ET traffic. The scalability of SSP-ACT for concurrent classification of multiple applications is also discussed. I demonstrate that SSP-ACT can significantly improve a classifier s performance using a small sliding window, regardless of how many packets are missed from the beginning of each flow and of the direction of the first packet of the most recent N packets used for the classification. The classifiers trained using SSP-ACT maintain their accuracy well with the presence of 5% random, independent synthetic packet loss. I also demonstrate that SSP-ACT is effective in identifying both ET and VoIP traffic concurrently, by using a single common classifier or two separate classifiers in parallel, one for each application. At the time of submitting this thesis, SSP-ACT has been implemented and used in an automated QoS-control system at Swinburne University of Technology [35], and has been demonstrated to provide sub-second real-time classification of online game traffic. My results show that SSP-ACT is a significant improvement over the previous, published state-of-the art for IP traffic classification. Although the experiments are confined to online game and VoIP applications, my results reveal a potential solution to the accurate and timely classification of traffic belonging to other Internet applications. The thesis is organised as follows. In Chapter 2 I provide the context for IPTC in IP networks, and highlight its importance in the areas of QoS provisioning, Internet accounting and charging, and lawful interception. I then review the traditional methods of traffic classification, and highlight the motivations for emerging ML-based IPTC techniques. ML-based IPTC is interdisciplinary involving the areas of networking and data mining techniques. It leverages data mining techniques to explore the large traffic statistical properties space

27 27 and to devise novel classification rules emerging from the data mining process. In Chapter 3, I summarise the basic concepts of ML and how they can be applied to IPTC. I discuss a number of key requirements for the employment of ML-based classifiers in operational IP networks, which act as guidelines for my research presented in subsequent chapters. In Chapter 4 I review significant works related to ML-based IPTC over the past. I discuss their limitations with regards to the operational challenges addressed in Chapter 3. This helps me define my research question with a justification of its originality and novelty and the reasons why it is worth pursuing. The chapter is concluded with the problem statement for my thesis. In Chapter 5 I present my novel modification to traditional ML training and classification techniques, using a multiple sub-flows training method. I demonstrate that the method optimises the classification of flows within finite periods of time, regardless of where within the flows lifetime the traffic is captured. My experiments are conducted on the Naive Bayes and C4.5 Decision Tree classifiers, with the goal to classify ET traffic against a number of other common Internet applications. In Chapter 6 I propose and demonstrate an automated approach based on the use of clustering ML techniques to choose appropriate, representative sub-flows, from which effective ML-based IP traffic classifiers may be trained. In Chapter 7, I demonstrate the directional issues when a classifier is trained based on an assumption of flow direction, which maybe wrong when classifying in real operational networks. I propose and demonstrate that training on synthetic sub-flow pairs allows that the classifier maintains its performance without relying on prior knowledge of inferred or actual directionality of a flow. Chapter 8 provides an evaluation of the overall SSP-ACT approach proposed. The effectiveness of SSP-ACT is demonstrated with VoIP application. My preliminary investigation on the impacts of 5% random, independent packet loss on the classification of VoIP and ET traffic is presented. I also propose two different implementation options for the concurrent classification of multiple applications, the pros and cons of which are discussed. Chapter 9 concludes the thesis with final remarks and suggestions for future work.

28 Chapter 2 Application Context for ML Based IP Traffic Classification 2.1 Introduction Real-time IP traffic classification (IPTC) has the potential to solve difficult network management problems for Internet service providers (ISPs) and their equipment vendors. Network operators need to know what is flowing over their networks promptly so they can react quickly in support of their various business goals. Traffic classification may be a core part of automated intrusion detection systems [48][42][49], used to detect patterns indicative of denial of service attacks, to trigger automated re-allocation of network resources for priority customers [33], or to identify customer use of network resources for accounting and billing purposes. More recently, governments have also been clarifying ISP obligations with respect to lawful interception (LI) of IP data traffic [50]. Just as telephone companies must support interception of telephone usage, ISPs are increasingly subject to government requests for information on network use by particular individuals at particular points in time. IPTC is an integral part of ISP-based LI solutions. Commonly deployed IPTC techniques have been based around direct inspection of each packet s contents at some point on the network. Successive IP packets that have the same five-tuple of protocol type, source address:port and destination address:port are considered to belong to a flow whose controlling application we wish to determine. Simple classification infers the controlling application s identity by assuming that most applications consistently use well known TCP or UDP port numbers (visible in the TCP or UDP headers). However, 28

29 2.1. INTRODUCTION 29 many applications are increasingly using unpredictable (or at least obscure) port numbers [51]. Consequently, more sophisticated classification techniques infer application types by looking for application-specific data (or well-known protocol behaviour) within the TCP or UDP payloads [52]. Unfortunately, the effectiveness of such deep packet inspection techniques is diminishing. Such packet inspection relies on two related assumptions: Third parties unaffiliated with either source or recipient are able to inspect each IP packet s payload (i.e. the payload is visible). The classifier knows the syntax of each application s packet payloads (i.e. the payload can be interpreted). Two emerging challenges undermine the first assumption customers may use encryption to obfuscate packet contents (including TCP or UDP port numbers), and governments may impose privacy regulations constraining the ability of third parties to lawfully inspect payloads at all. The second assumption imposes a heavy operational load commercial devices would need repeated updates to stay ahead of regular (or simply gratuitous) changes in every application s packet payload formats. The research community has responded by investigating classification schemes capable of inferring application-level usage patterns without deep inspection of packet payloads. Newer approaches (e.g. [53], [54], [55], [56], [57] and [58]) classify traffic by recognising statistical patterns in externally observable attributes of the traffic (such as typical packet lengths, interpacket arrival times, and flow duration and volume). The goal is to either cluster IP traffic flows into groups that have similar traffic patterns, or classify one or more applications of interest. A number of researchers are looking at the application of Machine Learning (ML) techniques (a subset of the wider Artificial Intelligence discipline) to IPTC (e.g. [59], [60], [61]). The application of ML techniques involves a number of steps. First, features are defined by which future unknown IP traffic may be identified and differentiated. Features are attributes of flows calculated over multiple packets (such as maximum or minimum packet lengths in each direction, flow durations or inter-packet arrival times). The ML classifier is trained to associate sets of features with known traffic classes (creating rules), and to apply the ML algorithm to

30 30 CHAPTER 2. APPLICATION CONTEXT FOR ML BASED IPTC classify unknown traffic using the previously learned rules. Every ML algorithm has a different approach to sorting and prioritising sets of features, which leads to different dynamic behaviours during training and classification. This chapter provides the rationale for IPTC in IP networks, reviews the traditional approaches to traffic classification, and highlights the motivations for emerging ML-based techniques for IPTC. The rest of this chapter is organised as follows. Section 2.2 justifies the importance of IPTC when reviewing the important networking areas of QoS issues and provisioning, Internet pricing and lawful interception. Section 2.3 follows with the introduction of a number of metrics for assessing classification accuracy. Section 2.4 discusses the limitations of traditional port- and payload-based classification techniques. This provides the basis for the motivation for statistical and ML based traffic classification approaches discussed in section 2.5. Section 2.6 concludes the chapter with some final remarks. 2.2 The importance of IP traffic classification The importance of IPTC may be illustrated by reviewing the important areas of IP QoS issues and provisioning, Internet pricing and Lawful Interception (LI) QoS issues over Last Mile networks Network capacity tends to be high in core (backbones) networks, low in access networks and high in home or enterprise LANs. Consequently the edge (the boundary between ISP and customer networks) tends to make a significant contribution to observed network queuing delay and jitter [62]. I conducted a number of experimental studies to observe the degree to which modern, high bandwidth access technologies still introduce uncontrolled latency fluctuations [63], [64] and [65]. I focused in particular on two common Internet access technologies: Data Over Cable Service Interface Specifications (DOCSIS) [66] networks and b [67] wireless local area networks. A typical DOCSIS access network is illustrated in Figure 2.1. In this scenario, the home user s equipment (used for various activities, such as Web browsing, data and movie downloading, or playing interactive online games and chat) is connected to the remote content or

31 2.2. THE IMPORTANCE OF IP TRAFFIC CLASSIFICATION 31 game servers through the DOCSIS cable network of an ISP. Conceptually, the user s traffic travels through the user s Cable Modem (CM), the Hybrid Fibre Coaxial Network (HFC) and the Cable Modem Termination System (CMTS) at the ISP site, and the remote links. Online Game VoIP Home Network Cable Modem HFC Network ISP DS US CMTS Web, P2P, SSH, SMTP Server Web, P2P, SSH, SMTP Game Server Figure 2.1: A typical DOCSIS cable network from ISP to home users I observed that when a client downloads content from an ISP-hosted server the DOCSIS link exhibits a significant spike in latency that impacts on all traffic concurrently sharing the DOCSIS link. (In my particular experiments [64], [63] the RTT jumped from 13ms when idle to over 100ms during long-lived TCP-based data transfers from a remote server to a home-based client 1.). Wireless LAN networks have become popular for interactive applications such as online gaming and videoconferencing. As with DOCSIS, I observed that consumer-grade b networks exhibited latency fluctuations in excess of 100ms during long-lived TCP-based data transfers [64] and [65]. These experiments confirmed my belief that modern access link technologies must deploy traffic prioritisation mechanisms to effectively isolate different classes of end-user traffic from each other. (With respect to my specific examples, better QoS control requires a CMTS, CM, AP and/or client that can discriminate between Internet applications, classes of traffic and customers with different needs.) QoS provisioning In responding to the problem of network congestion, a common strategy for network providers is under-utilising (over-provisioning) the link capacity. However, this is not necessarily an 1 The downstream (DS) and upstream (US) directions were capped to 2Mbps and 1Mbps respectively. This approximated a consumer-grade cable-modem downlink while also ensuring the upstream ACK rate was not a limiting factor. Further characterisation of the increase in RTT as a function of offered load is presented in [64], [63]