ABSTRACT 1. INTRODUCTION RESEARCH ARTICLE. Yao Shen 1,2 *, Liusheng Huang 1,2, Xiaorong Lu 1,2 and Wei Yang 1,2

Transcription

1 SECURITY AND COMMUNICATION NETWORKS Security Comm. Networks 2015; 8: Published online 11 August 2014 in Wiley Online Library (wileyonlinelibrary.com) RESEARCH ARTICLE A novel comprehensive steganalysis of transmission control protocol/internet protocol covert channels based on protocol behaviors and support vector machine Yao Shen 1,2 *, Liusheng Huang 1,2, Xiaorong Lu 1,2 and Wei Yang 1,2 1 School of CS & Tech., USTC, Hefei, , China 2 Suzhou Institute for Advanced Study, USTC, Suzhou, , China ABSTRACT Covert channels are malicious conversations disguised in legitimate network communications, allowing information leak to the unauthorized or unknown receiver. Various network steganographic schemes that modify the header fields of transmission control protocol/internet protocol (TCP/IP) have been proposed in recent years. People before conducted detection research based on the surface content of the header field and did not take into account the differences between the behavior characters of covert channels and the inherent behavior regularities of the header fields. Up to date, there is little comprehensive research on the steganalysis against the storage covert channels. In this paper, we focus on the detection of storage covert channels and introduce a novel comprehensive detection method based on the protocol behaviors. The protocol behavior characters are utilized to evaluate the regularities or correlations of header fields between adjacent packets according to the conventional use. First, the behavior features of the header fields in TCP/IP are extracted; a support vector machine is then applied to the behavior feature sets for discovering the existence of covert channels. Some recognized covert channel tools are detected in our detection experiment. Experimental results and discussion show that our detection method is of effectiveness. Copyright 2014 John Wiley & Sons, Ltd. KEYWORDS information hiding; network security; covert channels; comprehensive detection; protocol behaviors; SVM *Correspondence Yao Shen, Suzhou Institute for Advanced Study, USTC, Suzhou, , China. shenyao@mail.ustc.edu.cn 1. INTRODUCTION Steganography is a technique of hiding the very existence of covert communication by embedding secret messages in digital media such as text, image and audio [1]. However, in recent years, because of the booming development of Internet, the work centrality of steganography has gradually transferred to dynamic media, like network protocol packets. Because protocol packets are becoming ubiquitous in today s networks, massive protocol packets especially transmission control protocol/internet protocol (TCP/IP) packets are considered as ideal carriers for covert communications between secret parties. Network steganography is a synonym to covert channel, which was introduced by Lampson [2]. Covert channel is a malicious conversation within a legitimate network communication. Covert channels obviously violate the security policies laid down by the network environment, allowing the information leak to the unauthorized or unknown receiver. Various covert channels that use the header fields of IP [3,4] and TCP [3,5] are presented in literature. There are two main kinds of covert channels on the whole: storage covert channels and timing covert channels. Among the many TCP/IP covert channel schemes, the research direction of this paper is to detect various storage covert channels in TCP/IP headers. Steganalysis of covert channels has a high theoretical research value for network security and privacy protection, and a high practical value for the military field. Each header field in TCP/IP has its own properties and usages, which are the inherent behaviors of the header fields of TCP/IP. According to RFC791 [6], RFC793 [7] and the conventional use, they have their specific regularities or correlations between adjacent packets, which are the behavior characters of the header fields. For detection of covert channels, people before always conducted detection Copyright 2014 John Wiley & Sons, Ltd. 1279

2 A comprehensive steganalysis of TCP/IP covert channels Y. Shen et al. based on the surface values of the header fields and did not take into account the differences between the behaviors of the covert channels and the inherent behavior regularities of the header fields. The existing regularity-based detection measures alone fail to detect the presence of covert channels efficiently. This requires a more comprehensive and efficient methodology. The method not only should collect regularity features over the data packets but also can extract the correlation features between adjacent packets. But up to present, there is little comprehensive research on the steganalysis against the covert channels. This is the main reason that leads us to propose this work. In our research, we find that the information hiding in header fields of TCP/IP would change the regularities or correlations between adjacent packets. Then, we investigate and analyze the differences that the covert channels bring to the normal communication in terms of regularity and correlation. Hence, we can detect the existence of covert channels based on the differences between the behavior features of covert channels and the inherent regularity of the header fields in normal channels. Based on this fact, we have summarized a table of protocol behavior characters (PBC) to evaluate the regularities and correlations of the header fields between adjacent packets, so that the PBC can be used for extracting classification features to distinguish covert channels from normal. A support vector machine (SVM) classifier is finally applied to the feature sets for discovering the existence of covert channels. Experimental results and discussion show that the detection model based on the statistical characteristics of protocol behaviors can efficiently detect the covert channels in TCP/IP. The rest of this paper is organized as follows. In Section 2, the related work about the design and detection of storage covert channels is reviewed. Section 3 introduces the header fields that can be used for covert channels and describes the usages of SVM. Section 4 focuses on the description of the detection method based on statistical characteristics of protocol behaviors. Section 5 gives the experiments, test results, comparison and discussion. Finally, conclusions are presented in Section RELATED WORK In this section, we take an overview of techniques in TCP/IP steganography and steganalysis. Here, we should point out that the steganography mentioned in this paper does not contain the simple and crude covert channels that destroy TCP/IP communications or can be easily detected by intrusion detection systems (IDS) and packet-filtering firewall. We focus on the storage covert channels between a pair of users that can maintain normal communication TCP/IP steganography The TCP/IP is described in RFC791 [6] and RFC793 [7]. Various covert channels have been identified in TCP/IP. The first scientific paper of steganography was presented by Simmons [8] in 1983 who formulated it as the Prisoners problem. The problem is as follows: Two prisoners need to communicate to escape, but all the messages should pass through the warden who can detect any encrypted messages. They must find some technique of hiding their secret messages in an innocent-looking communication. A general survey of information-hiding techniques in the header fields of TCP/IP was described in [9,10]. Especially, Rowland [3] described three possible covert channels in the IP identification (IP ID) field, the initial sequence number (ISN) field and the TCP acknowledge sequence number field. He also programmed a proofof-concept implementation using a raw socket. Also, IP fragment offset field and time-to-live (TTL) field were used as covert channels in [11] and [12], respectively. The TCP header also contains different fields for covert communication. Murdoch et al. proposed using reversible transforms that mapped block cipher output onto TCP ISNs, indistinguishable from those generated by Linux [13]. Kumar et al. embedded the secret information in the TCP sequence number by adjusting the payload of TCP segments [14]. In [15], Giffin et al. developed a method for covert message through the timestamps field in TCP header. TCP flag bits, TCP urgent pointer and TCP options can also serve as a good medium for transmitting secret message over the Internet. The covert channels embedded in them can be found in literature [13] TCP/IP steganalysis Steganalysis, on the other hand, is a science to detect the existence of covert data in an innocent-looking communication. In general, steganalysis can be grouped into two categories: blind steganalysis and targeted steganalysis. Blind steganalysis is to detect a wide range of steganographic methods, and targeted steganalysis is intended for a specific steganographic method. In recent years, there are some papers about targeted steganalysis methods in TCP/IP. For example, a general description of targeted detection schemes for TCP/IP storage covert channels was provided by Sohn et al. [16] and Cabuk et al. [17]. In [13], the authors presented several methods to detect covert channels including TCP ISN-based hiding and IP headerbased hiding. A second-order detection scheme for packet length-based steganography was proposed in [18]. By contrast, few papers have been proposed to detect varieties of covert channels comprehensively. Up to date, there is little comprehensive and blind steganalytic research against the storage covert channels. Sohn et al. [16] provided a relatively comprehensive detection method. It proposed a detection method for covert channels using IP ID field and TCP sequence number field with SVM, which has excellent performance in pattern classification. Although the method achieved a good detection rate on some covert channels, it still had some drawbacks. For example, it only detected covert channels using the IP ID 1280 Security Comm. Networks 2015; 8: John Wiley & Sons, Ltd.

3 Y. Shen et al. A comprehensive steganalysis of TCP/IP covert channels and TCP sequence number fields, so that the method cannot detect the covert channels comprehensively. Another significant defect of the detection method was that it only selected the surface values of the header fields as SVM vector, and its feature sets were not enough to discover the detection-resistant covert channels from communication. It paid attention to the regularity but ignored the correlation between adjacent packets fields. That is, in principle, it is one regularity-based detection method without considering the correlation. Therefore, the detection method lets off those covert channels that have been improved in performance of anti-detection based on the correlation. 3. PRELIMINARIES In this section, we conduct a research on the header fields that can be used for covert channels in TCP/IP. Then, we describe the principles and usage of SVM, which plays an important role in our detection model The header fields used for covert channels The TCP/IP header has many fields for covert channels because of its redundancy. Figures 1 and 2 show the structure of IP and TCP header, respectively. We investigate the header fields in TCP/IP, which serve as ideal carriers for storage covert channels from related literature. The covert channels based on the following 10 header fields of TCP/IP not only can maintain normal communication and bypass monitoring of IDS and packetfiltering firewall but also can effectively transmit secret messages to outside. Hence, we research the properties and usages of the 10 common header fields in TCP/IP. The properties and usages of the 10 headers are described as follows: (1) Type of service (TOS): 8 bits. The TOS provides an indication of the abstract parameters of the quality of service desired. (2) IP ID: 16 bits. It is an identifying value assigned by the sender to aid in assembling the fragments of a datagram, which is increased from 0 to incrementally. (3) Flags: 3 bits. They are used to fragment and reassemble Internet datagrams, including the reserved bit, DF (Don t Fragment) bit and MF (More Fragment) bit. (4) Fragment offset: 13 bits. This field indicates where in the datagram this fragment belongs. (5) TTL: 8 bits. This field indicates the maximum time that the datagram is allowed to remain in the Internet. This field decreases by one whenever data packets pass through a router. Figure 1. IP header. Figure 2. TCP header. Security Comm. Networks 2015; 8: John Wiley & Sons, Ltd. 1281

4 A comprehensive steganalysis of TCP/IP covert channels Y. Shen et al. (6) ISN: 32 bits. Every time a TCP connection starts, the host automatically assigns an ISN. Because of the distribution of the randomness, it can be used to hide information. (7) Sequence number (seq): 32 bits. In TCP communication connection, each data byte transferred has a corresponding sequence number. The use of this hidden field is similar with that of ISN. (8) Acknowledgement number (ack): 32 bits. If the ACK control bit is set, this field contains the value of the next sequence number that the sender is expecting to receive. The use of this hidden field is similar with that of ISN and seq. (9) TCP control bits: 6 bits. The control bits indicate the status of TCP to communicate. (10) Urgent pointer: 16 bits. It points to the priority data followed. It is often used with the URG (urgent pointer) flag simultaneously for information hiding Support vector machine The SVM is a supervised learning model with associated learning algorithms that analyze data and recognize patterns, used for classification and regression analysis. SVM plots training vectors in high-dimensional feature space and classifies each vector by its class. SVM views the classification problem as a quadratic optimization problem. They combine generalization control with a technique to avoid the curse of dimensionality by maximizing the margin between the different classes. SVM classifies data by determining a set of support vectors, which are members of the set of training inputs that outline a hyperplane in feature space [19 21]. For classification [16], SVM provides a generic mechanism to fit the data within a surface of a hyperplane of a class through the use of a kernel function. The user may provide a kernel function, such as a linear, polynomial or sigmoid curve, to the SVM during the training process, which selects support vectors along the surface of the function. The primary advantage of SVM is binary classification and regression that it provides to a classifier with a minimal Vapnik Chervonenkis dimension, which implies low expected probability of generalization errors. In practice, there is one important point to note that the performance of SVM classifier is greatly influenced by the parameters, especially the kernel function. The four different kernel functions make the SVM classifier linear SVM, polynomial SVM, RBF (radial basis function) SVM and sigmoid SVM, respectively. Each kernel function has its applicability, so that users should choose the most appropriate kernel function according to practical situation. 4. DETECTION SCHEME From the previous preliminaries, we design the protocolbehavior-based detection (or PBBD for short) method based on the behavior features of the header fields; thus, we can distinguish the covert channels from normal. In this section, we first introduce the main framework of our detection method. Then, we analyze the behavior features of header fields in TCP/IP and obtain a table of PBC. Lastly, we introduce the classification features that are used in our experiment. More details are as follows. Figure 3. The main framework of PBBD Security Comm. Networks 2015; 8: John Wiley & Sons, Ltd.

5 Y. Shen et al. A comprehensive steganalysis of TCP/IP covert channels 4.1. Proposed detection method To detect the storage covert channels in TCP/IP comprehensively, we address our PBBD method. Figure 3 shows the main framework of PBBD method. There are three main modules employed: training module, testing module and classifying module. The arrowhead represents the work flow of data, among which the dashed line arrows represent the data flow of the training module and the solid line arrows represent the data flow of the testing module. The dashed rectangle indicates the whole detection system. Figure 3 graphically illustrates how the PBBD method works. The raw data set is processed in accordance with the data flow of the training and testing modules. Here, we introduce the work flow of our PBBD method briefly. While analyzing, both the training and testing data packet sets go through the data preprocessing, generating the processing data sets of each header field. Then, according to the behavior features in the PBC table, both the data sets go through the feature extraction, resulting in training and testing feature sets. Then, the training feature sets are used for the training of SVM classifier and generation of the classification model. The testing feature sets are used for the validation of classification model. The classification results indicate the detection performance of the PBBD method. It is important to note that the training process is required to execute before the testing process. More details about the detection method are described as follows Behavior features of TCP/IP header fields In the main framework of our PBBD method, we should talk about the behavior features of the headers before the training and testing. According to the behavior features, we determine how to extract classification features of the headers in TCP/IP, which is the foundation of PBBD method. Based on our investigation, each header field has its own properties and usages, which are the inherent behaviors of the header fields in TCP/IP. According to RFC791 [6], RFC793 [7] and the conventional use, the headers have their specific regularities or correlations between adjacent packets, which are the very behavior characters of the header fields. These also are the behavior features of normal communication, while the covert communication has its own behavior features influenced by the embedded covert channels. Compared with normal communication, the information hiding in header fields of TCP/IP would change the regularities or correlations between adjacent packets. Based on this fact, we can detect covert channels based on the differences between the inherent behavior characters of the headers and the behavior features of covert channels. To collect regularity and correlation information, we introduce two important rules: regularity rule and correlation rule. And if one covert channel has been hid in the communication, at least one of the two rules is broken compared with the normal communication. The rules are defined as follows. Rule 1: regularity rule, which is regularity of the field values that can be described by low-order statistics, for example, the mean, variance or distribution. If the behavior characters of fields comply with the regularity rule, we can detect the covert channels based on the surface values of header fields. Rule 2: correlation rule, which describes the trend between adjacent packets in one window, for example, increase or decrease. It can be depicted by the differences of field values between adjacent packets. If the behavior characters of fields comply with the correlation rule, a series of values can be converted to a sequence of differences, with which we can evaluate the interrelationships by means of high-order statistics, such as the entropy. According to the behavior features of the 10 headers, each header field complies with one of the two rules. And the header fields do not comply with both rules meanwhile. For example, the values of one field between adjacent packets increase along with time gone, it follows rule 2, and it has little or nothing to do with rule 1. Therefore, if the behavior characters of the field can be described by regularity rule, the field obeys rule 1; if the behavior characters of the field can be described by correlation rule, the field obeys rule 2. The rule that the header field follows determines that we are to obtain the regularity information or correlation information. We investigate and analyze the behaviors of the 10 header fields in TCP/IP. According to the definition of the two rules, the behavior features and our analysis of the 10 headers are listed as follows: (1) TOS: The TOS field is so rarely used in today s networks that it can serve as a carrier to embed secret information. Although TOS field is rarely used, the change of this value is very obvious, and in general, it always remains constant at a fixed value. If the field value changes frequently, it may be suspected of having a covert channel. The characteristics of building covert channels in TOS are destined to violate the normal interrelationships of adjacent packets based on rule 2. (2) IP ID: Under normal circumstances, IP ID shows an increasing trend in a single-communication link. The behavior characters of IP ID follow rule 2, because although the rate of increase varies because of different background traffic, the general trend is increasing. Then, we choose the differences between the field values to monitor IP ID. (3) Flags: Generally, the value of DF is 1 and remains unchanged, which can serve as the basis to detect the existence of covert channels. If the values of flags, for example, DF change frequently, the field values by Security Comm. Networks 2015; 8: John Wiley & Sons, Ltd. 1283

6 A comprehensive steganalysis of TCP/IP covert channels Y. Shen et al. Table I. The protocol behavior characters of transmission control protocol/internet protocol header fields. Header field Behavior rule Preprocessing mode TOS Rule 2 Differences of the header field between adjacent packets IP ID Rule 2 Differences of the header field between adjacent packets Flags Rule 1 Surface values of the header field Fragment offset Rule 1 Surface values of the header field TTL Rule 2 Differences of the header field between adjacent packets ISN Rule 1 Surface values of the header field seq Rule 2 Differences of the header field between adjacent packets ack Rule 2 Differences of the header field between adjacent packets TCP control bits Rule 1 Surface values of the header field Urgent pointer Rule 1 Surface values of the header field TOS, type of service; IP ID, Internet protocol identification; TTL, time to live; ISN, initial sequence number; seq, sequence number; ack, acknowledgement number; TCP, transmission control protocol. themselves are able to supervise whether the traffic is suspicious, which obeys rule 1. (4) Fragment offset: Normally, the value of offset is 0, because the DF bit is always 1. According to the conventional use, if its distribution is too stochastic, the communication is suspected to have a covert channel. It also complies with rule 1. (5) TTL: The TTL value decreases by one whenever data packets pass through a router. If this field contains the value zero, then the datagram must be destroyed. Additionally, the initial value of TTL is only associated with the type of system. Under normal circumstances, this value will not change in a short time for a single-communication link; if the value changes frequently in a short time, it indicates that there may be hidden channels. We can find the suspicious signs of covert channels based on rule 2. (6) ISN: According to many experimental results, ISNs follow a normal distribution; thus, rule 1 is available to detect the presence of ISN covert channels. (7) seq: 32 bits. On a single-data link, seq is gradually increased. Although the increase amplitude varies, it, on the whole, is increasing in a single communication. In order to describe the increasing trend, rule 2 is applied to it. (8) ack: The use of ack is similar with that of ISN and seq. Similarly, ack is gradually increased, but the increase amplitude varies. Also, rule 2 best fits the ack. (9) TCP control bits: The value range of control bits is limited, because of the limited states of communication transmission. So we can determine whether traffic is suspicious based on the field values and rule 1. (10) Urgent pointer: Generally, packets are transmitted according to the queuing order. Without special needs, the value of this field remains 0. If the value of this field is not 0 and has been changed frequently, this situation is not normal, because there cannot be so much priority packets to be sent. Given this, we choose the values of the header field to supervise the urgent pointer based on rule 1. Based on the previous analysis, a table of PBC of TCP/IP has been summed up, which is shown in Table I. The PBC table is the crucial basis of our detection method. Our raw data set of each header field is preprocessed according to the PBC table, and then, the classification feature sets can be extracted from the preprocessed data set Feature extraction and classification Having introduced the behavior features of the headers, we move to the description of feature extraction and classification. In Figure 3, the processing flows of the training and testing are similar with similar output. The key process of the two modules is the feature extraction, and the key element of feature extraction is the feature vector. After going through data preprocessing, the data set is extracted from packets and converted into the format that we need. Then, the data set is passed to the process of feature extraction based on the PBC table. The goal of feature extraction is to obtain the classification feature vectors of the data set. Feature vector V. In the experiment, the feature vector utilized by the SVM classifier is a three-dimensional vector. By the three statistical metrics from different angles, we can evaluate the regularity or correlation of the data set comprehensively. The feature vector is constituted of three features, which are described as follows: Feature 1: average (Avg). The Avg refers to the sum of a list of numbers divided by the size of numbers. Feature 2: variance (Var). The Var is a measure of the degree of dispersion of a set of data values. Feature 3: entropy (Ent). The Ent is a measure of uncertainty or information content in a random variable. Given a sample of sequential IDs amounting to W from a data flow, each ID can be mapped to one of a set of M possible values with the possibility of the i th given as P i. The Avg, Var and Ent of the sample are then calculated as follows: 1284 Security Comm. Networks 2015; 8: John Wiley & Sons, Ltd.

7 Y. Shen et al. A comprehensive steganalysis of TCP/IP covert channels WP ID i i=1 Avg(ID 1, :::, ID W )= W WP (ID i ) 2 i=1 Var(ID 1, :::, ID W )= W (1) (2) smaller it is, the worse is the detection performance. When the window size reaches a certain degree, the detection performance is not increasing anymore. In the experiment, we should determine the window size based on the practical situation. The details are described in the next section. 5. EXPERIMENTS AND ANALYSIS Ent(ID 1, :::, ID W )= MX P i log P i (3) where is the average of sequential IDs and equivalent of Avg in numeral. Equations (1), (2) and (3) describe the one-order, two-order and high-order metrics of the objects in window size of W, respectively. Based on the three equations, the three statistical features make up one threedimensional vector. Therefore, the classification feature vector is defined as follows: i=1 V =(Avg, Var, Ent) (4) From Equation (4), the feature vector V consists of average, variance and entropy, measuring the behavior characters of the header field comprehensively. After feature extraction, the data set would be converted into a series of three-dimensional feature vectors, which are depicted as follows: V 1 Avg 1 Var 1 Ent 1 V 2 B. A = Avg 2 Var 2 Ent 2 C (5).. A V m Avg m Var m Ent m where V i is the feature vector of window i, Avg i, Var i and Ent i are the statistical features of j window k i and N is the total number of data points, m = NW. Next, the SVM classifier can be trained and generates a classification model. But first of all, we should determine a suitable detection window size, W. The window size influences the detection performance of detection system. The 5.1. Data collection In our experiment, data collection is an important factor for our detection results. We should be aware that the experimental data set for the detection of covert channels in TCP/IP is composed of two parts: the training data set and the testing data set. The raw data are collected from the normal and covert communication. First, for the normal communication, we employ a packet capture tool, Wireshark, on the gateway of our department with about 500 computers running to collect the legitimate communications. For the covert communication, we build covert channels with the covert tools, for example, Covert_tcp, steg tunnel and one informationhiding tool, Steg_scc, which we implemented based on Covert_tcp. The Covert_tcp mainly employs the IP ID, ISN and ack fields of TCP/IP to transfer secret information. Steg tunnel builds covert channels on the ID and sequence number (seq) field of TCP/IP packets. The covert tool Steg_scc that we implemented can embed information into the fields such as TTL, TOS, offset and flags. Using the Wireshark, we can capture enough covert data sets from various covert communication. Second, we use the packet analysis tools implemented to extract the field values that we need from packets and convert them into a decimal. The data sets that we collect are shown in Table II. For each field in TCP/IP, the normal data sets and covert data sets in the training set are balanced in numbers, because if the numbers of the two sets are not balanced, the classification accuracy may be slightly influenced by the imbalance. In order to balance the scales for high detection performance, we set the number of normal data set and covert data set at the same number. Table II. The support vector machine training data set (each set is ). Header fields Normal data sets Covert data sets TOS Packets from normal communication Packets from steg_scc communication IP ID Packets from normal communication Packets from covert_tcp communication Flags Packets from normal communication Packets from steg_scc communication Fragment offset Packets from normal communication Packets from steg_scc communication TTL Packets from normal communication Packets from steg_scc communication ISN Packets from normal communication Packets from covert_tcp communication seq Packets from normal communication Packets from steg tunnel communication ack Packets from normal communication Packets from covert_tcp communication TCP control bits Packets from normal communication Packets from steg_scc communication Urgent pointer Packets from normal communication Packets from steg_scc communication TOS, type of service; IP ID, Internet protocol identification; TTL, time to live; ISN, initial sequence number; seq, sequence number; ack, acknowledgement number; TCP, transmission control protocol. Security Comm. Networks 2015; 8: John Wiley & Sons, Ltd. 1285

8 A comprehensive steganalysis of TCP/IP covert channels Y. Shen et al Training and testing In our experiment, we apply an SVM classifier [22,23] to classify the inherent behavior features of the headers from the behavior features of covert channels so as to detect covert communications. In practice, the SVM classifier that we use in our experiment is the LibSVM. LibSVM [22] is an integrated software for support vector classification, which was developed by Chih-Chung Chang and Chih-Jen Lin. It is convenient to use for classification, regression and distribution estimation and now is widely used in data mining. In order to detect the covert channels hid in the 10 headers, the classification processes are carried on separately, and the main processes are in common. In this subsection, we introduce the detailed process of the training and testing modules for each header Training. For each header field in TCP/IP, our training data set consists of data points: from normal communication and from covert communication. The normal training set of data points is extracted from the individual normal TCP/IP communication. The covert training set of data points are extracted from the covert communications that we build. First, we choose a detection window size, W, before feature extraction, and separate the TCP/IP packets into non-overlapping j k windows of size W packets. There are W windows each for the normal and the covert data points. In experiment, we do not determine the window sizes at a fixed value. We try different sizes from small to large and then choose one well-suited window size eventually according to the detection performance. Next, for each window i, we compute the average, variance and entropy of the window according to Equations (1), (2) and (3). So do other windows. In the same way, both the normal and covert feature vector sets are extracted from the normal data set and covert data set, respectively. Third, choosing one of the four kernel functions, the SVM classifier can be trained by both the feature vector sets. The SVM classifier generates the classification model eventually. Based on the classification model, we can carry on the testing work Testing. In the testing experiment, the testing data set consists of data points: for normal and for covert. With the same circumstance of training module, we carry on the testing experiment. The normal and the covert testing data points each has j W k windows. The feature extraction is the same with that of the training process. With the classification model, we apply the two-class SVM classifier to the feature sets extracted from the testing data set. Next, we averaged the detection results over 100 runs each time in testing. The results are shown in Table III and discussed in the next subsection Results and discussion In order to measure the experiment performance, we evaluate the efficacy of the detection method in terms of the false-positive rate (FP), the false-negative rate (FN) and the detection rate (DR), which are defined as follows. FP = N normal_as_covert N normal (6) FN = N covert_as_normal N covert (7) DR = N true N total (8) where N normal is the number of normal feature vectors, N covert is the number of covert feature vectors, N total is the total number of testing feature vectors, N normal_as_covert is the number of normal feature vectors identified as covert, N covert_as_normal is the number of covert feature vectors Table III. The average detection results of each field (W =200). Hidden fields Testing data volume FP (%) FN (%) DR (%) TOS (normal abnormal each ) IP ID (normal abnormal each ) Flags (normal abnormal each ) Fragment offset 20,000 (normal abnormal each ) TTL (normal abnormal each ) ISN (normal abnormal each ) seq (normal abnormal each ) ack (normal abnormal each ) TCP control bits (normal abnormal each ) Urgent pointer (normal abnormal each ) TOS, type of service; IP ID, Internet protocol identification; TTL, time to live; ISN, initial sequence number; seq, sequence number; ack, acknowledgement number; TCP, transmission control protocol Security Comm. Networks 2015; 8: John Wiley & Sons, Ltd.

9 Y. Shen et al. A comprehensive steganalysis of TCP/IP covert channels identified as normal and N true is the number of feature vectors identified as their true type. In the experiment, there are two important parameters that influence the performance of our detection method including SVM kernel function and detection window size. In order to obtain better experiment performance, we try enough choices in the training and testing processes and select the most appropriate kernel function and window size W according to the experimental results SVM kernel function. Given the detection window size W, the experimental results with the four different kernel functions are illustrated in Figure 4. It can be seen that when the detection window size is big enough at 400, the SVM classifier with RBF kernel function performs best and has lower FN and FP rates. The sigmoid SVM is suboptimal while the linear SVM and polynomial SVM have a poor performance under the same circumstance. Based on the results, we finally choose the RBF kernel function, which makes the SVM classifier work better than others in our experiment Detection window size W. According to previous analysis, we select the RBF function as our SVM kernel function. Given the kernel function, we choose the covert channel based on IP ID and TCP ISN fields as representative for determining one suitable detection window size. The experimental results of IP ID and TCP ISN with different window sizes from small to large are illustrated in Figures 5 and 6, respectively. As can be seen, the detection performance of IP ID and ISN covert channels is greatly influenced by the window size. When the detection window size is 200 packets or above, the detection accuracy of our method becomes very high and tends towards stability. The FP rate and FN rate are negligible. When the window size is below 200 packets, Figure 4. The average detection performance with four kernel functions when W = 400. Figure 5. The detection performance of IP ID channel with different window sizes. Figure 6. The detection performance of ISN channel with different window sizes. the detection accuracy is lowered, and FP rate and FN rate are perceptible, because if the window size is too small, there are not enough packets to extract the true features. In short, the detection performance improves as the window size increases. Therefore, we set the window size at 200, which keeps a good detection accuracy. In the testing experiment for covert channels based on other headers, the window size of 200 can also be satisfied to achieve a good detection accuracy. Hence, we suggest that the window size of our PBBD model is 200 so as to achieve the best detection performance for the 10 kinds of covert channels Detection results. With the two appropriate parameters, we continue our testing experiment using PBBD method based on the feature sets including the normal and the covert. For the covert channels based on the headers, we compute the FP rate, FN rate and DR rate according to Equations (1), (2) and (3) and then report the average FP, FN and DR rates for the covert channels in Table III. Security Comm. Networks 2015; 8: John Wiley & Sons, Ltd. 1287

10 A comprehensive steganalysis of TCP/IP covert channels Y. Shen et al. It can be seen that the detection accuracy of the covert channels on the header fields of TCP/IP keeps a pretty good accuracy rate. The first column of the table lists various covert channels embedded in TCP/IP header fields, which we can detect by our PBBD method. The second column is the testing data volume for verification of the method s performance. For each header field, the total number of testing data points is , and the normal and covert data sets each have data points. In the third, fourth and fifth columns of the table, we report the average FP rate, FN rate and DR rates for detecting the covert channels. From the detection accuracy of the covert channels, we can see that the detection performance of our PBBD method is pretty ideal Comparison and discussion. Experimental discussion is carried out with a comparison of the detection results of our scheme and other three detection schemes proposed in [13,16] and [24]. The three schemes are all targeted steganalysis of ISN covert channels. ISN covert channel obtains more attention because of its inherent behavior characters. Whenever a TCP session is established, sequence number (seq) is initialized with an ISN pseudorandom value and seq value is being incremented with a certain value during the session. Initial values of seq are not important, but according to many experimental results, the ISNs follow a normal distribution. Based on the fact, an attacker can use the ISN field to transfer secret information. Hence, we choose the detection of ISN-based covert channels as an example for discussion. In comparison experiment, we introduce two covert channel tools: covert_tcp [3] and NUSHU [24]. Both tools are implementations of steganography using TCP ISNs. Covert_tcp replaces the chosen field with data to be sent, while NUSHU is an improvement to covert_tcp. NUSHU uses TCP ISNs for encoding information and encrypts outgoing ISNs to hide the use of steganography. Both tools are classic implementation of ISN-based covert channel. Next, we are to use the previously mentioned three schemes and our PBBD scheme to detect the two covert channels. In [13], Murdoch et al. proposed using statistical characteristics of TCP ISNs to detect easy ISN covert channels and design more robust channels. Sohn et al. [16] provided a detection method with SVM to detect covert channels using identification fields of IP ID and ISN. And also, Tumoian et al. [24] used neural networks to learn statistical deviations of ISN network packet from the ISN model of normal communication and detect the ISN-based covert channel. The previous three schemes each have their characteristics and shortcomings as follows. Murdoch s scheme analyzed the characteristics of TCP ISN field and developed various tests to identify any anomalies that may indicate the use of steganography. But the detection tests were not put into practice without experimental results. The main aim of the paper was to develop a more robust covert scheme. Sohn s scheme used an SVM to learn and distinguish the abnormal from normal. But the Sohn s scheme can only identify simple features, so it cannot detect the complex structure present in these fields and their interdependencies. In principle, the Sohn s scheme focused on the regularity of the surface values of ISN field, so that it cannot detect the channels that mimic the regularities of normal communication. Tumoain s scheme employed neural networks to learn experimental ISN data and predict the successive ISN value based on the preceding ones, and classified the covert value from normal based on similarity measure. But practically, the volume of experimental data and the complexity of this method are undesirable. In principle, Tumoain s scheme focused on the correlation between consecutive packets, which is one of the two rules in our PBBD scheme. The comparison experiments are conducted with the same proper parameters described in each original paper. Murdoch s scheme only provided the testing method without implementation. Hence, we just get that the distribution test can be used for detecting the NUSHU covert channels, and nothing available for covert_tcp. Using the other two schemes and our PBBD scheme, the detection results for covert_tcp and NUSHU are illustrated in Figures 7 and 8, respectively. As can be seen in Figure 7, Sohn s, Tumoain s and our PBBD detection schemes have a good detection performance on the covert_tcp. Covert_tcp replaces the chosen field with data to be sent, so it can be detected either by observing the field values that do not meet the required overlap and uniqueness constraints or by comparing the data observed with statistical patterns of normal. In Figure 8, Sohn s scheme achieves a poor detection rate, because it selects surface values of ISN field as SVM vector, and its feature sets are not enough to distinguish Figure 7. The detection results of covert_tcp using Sohn s, Tumoain s and our PBBD schemes Security Comm. Networks 2015; 8: John Wiley & Sons, Ltd.

11 Y. Shen et al. A comprehensive steganalysis of TCP/IP covert channels from normal channels. The experimental results have shown that the detection method is highly effective and pretty ideal. In the future, our research work includes further development of the protocol behavior-based detection method, and we can extend it to other steganography methods for detection. We also can convert the detection platform into a real-time steganalytic tool, which can make the detection method much more practical. ACKNOWLEDGEMENT Figure 8. The detection results of NUSHU using Sohn s, Tumoain s and our PBBD schemes. the detection-resistant covert channels from communication. Tumoain s and our PBBD detection schemes both have a good detection performance, about 95%, among which the experimental complexity of Tumoain s scheme is undesirable. Tumoain s scheme uses neural networks to learn the similarity between consecutive packets and is able to discover the detection-resistant covert channels. Compared with Tumoain s scheme, ours has a slightly better DR and FN rates. Our PBBD scheme exploits data mining to extract the feature library considering both the regularity and correlation rules, while Sohn s and Tumoain s schemes focus on only one side. Therefore, our PBBD detection scheme not only can distinguish detection-resistant covert channels but also can detect the covert channels embedded in the 10 header fields comprehensively. Compared with the two schemes, the time complexity of our PBBD scheme is slightly higher than Sohn s scheme and less than Tumoain s scheme. The time complexity of our scheme is a function of data points N, window size W and the running time of the SVM classification algorithm C(.). The method performs classification per window. Hence, the running time for our detection method is O((N/W)C(W)). It is observed that the performance of PBBD is greatly influenced by the selection of detection window size W. Choosing appropriate size of the detection window contributes to the improvement of the performance of the experiment. 6. CONCLUSIONS In this paper, we have proposed an advanced detection method, PBBD, for steganalysis of the storage covert channels based on the protocol behaviors. The PBC, which evaluate the regularities or correlations of the header fields between adjacent packets, are the foundation for extracting classification features to distinguish covert channels This work was supported by the Natural Science Foundation of Jiangsu Province of China (No. BK ) and the Basic Perspective Project of SGCC (No. XXN ). REFERENCES 1. Wang H, Wang S. Cyber warfare: steganography vs. steganalysis. Communications of the ACM 2004; 47(10): Lampson BW. A note on the confinement problem. Communications of the ACM 1973; 16(10): C.H Rowland. Covert channels in the tcp/ip protocol suite. Technical Report 5, First Monday, Peer Reviewed Journal on the Internet, July Ahsan K. Covert Channel Analysis and Data Hiding in TCP/IP. Doctoral dissertation, University of Toronto: Toronto, Canada, K. Ahsan, D. Kundur. Practical Data Hiding in TCP/IP, in: Proc. Workshop on Multimedia Security at ACM Multimedia, French Riviera, 2002; Postel J. RFC 791: Internet protocol, Postel J. RFC 793: Transmission control protocol, September 1981, Status: Standard, Simmons GJ. The prisoners problem and the subliminal channel. In Advances in Cryptology. Springer: US, 1984; Petitcolas FAP, Anderson RJ, Kuhn MG. Information hiding a survey. Proceedings of the IEEE 1999; 87(7): Zander S, Armitage GJ, Branch P. A survey of covert channels and countermeasures in computer network protocols. IEEE Communications Surveys and Tutorials 2007; 9(1-4): Cauich E, Crdenas RG, Watanabe R. Data hiding in identification and offset IP fields. In Advanced Distributed Systems. Springer: Berlin Heidelberg, 2005; Zander S, Armitage G, Branch P. An empirical evaluation of IP Time To Live covert channels, ICON 2007, 15th IEEE International Conference on Networks, IEEE, Adelaide, SA, 2007; Security Comm. Networks 2015; 8: John Wiley & Sons, Ltd. 1289

12 A comprehensive steganalysis of TCP/IP covert channels Y. Shen et al. 13. Murdoch SJ, Lewis S. Embedding covert channels into TCP/IP. In Information Hiding. Springer: Berlin Heidelberg, 2005; Kumar VS, Dutta T, Sur A, et al. Secure network steganographic scheme exploiting TCP sequence numbers. In Advances in Network Security and Applications. Springer: Berlin Heidelberg, 2011; Giffin J, Greenstadt R, Litwack P, et al. Covert messaging through TCP timestamps. In Privacy Enhancing Technologies. Springer: Berlin Heidelberg, 2003; Sohn T, Seo JT, Moon J. A study on the covert channel detection of TCP/IP header using support vector machine. In Information and Communications Security. Springer: Berlin Heidelberg, 2003; Cabuk S, Brodley CE, Shields C. IP covert channel detection. ACM Transactions on Information and System Security (TISSEC) 2009; 12(4): Sur A, Nair AS, Kumar A, et al. Steganalysis of network packet length based data hiding. Circuits, Systems, and Signal Processing 2013; 32 (3): Cortes C, Vapnik V. Support-vector networks. Machine Learning 1995; 20(3): Burges CJC. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 1998; 2(2): Mukkamala S, Janoski G, Sung A. Intrusion detection using neural networks and support vector machines, IJCNN 02, Proceedings of the 2002 International Joint Conference on Neural Networks, IEEE, Honolulu, Hawaii, 2002; Chang CC, Lin CJ. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST) 2011; 2(3): Chen Z, Huang L, Meng P, et al. Blind linguistic steganalysis against translation based steganography. In Digital Watermarking. Springer: Berlin Heidelberg, 2011; Tumoian E, Anikeev M. Detecting NUSHU covert channels using neural networks, neural_networks_vs_nushu.pdf, (accessed on May 18, 2005), Security Comm. Networks 2015; 8: John Wiley & Sons, Ltd.