An approach of security testing for third-party component based on state mutation

Transcription

1 SECURITY AND COMMUNICATION NETWORKS Security Comm. Networks 2016; 9: Published online 23 January 2015 in Wiley Online Library (wileyonlinelibrary.com) SPECIAL ISSUE PAPER An approach of security testing for third-party component based on state mutation Jinfu Chen 1,2, Jiamei Chen 1, Rubing Huang 1,3 *, Yuchi Guo 1 and Yongzhao Zhan 1 1 School of Computer Science and Telecommunication Engineering, Jiangsu University, Zhenjiang, , China 2 Faculty of Information and Communication Technologies, Swinburne University of Technology, Hawthorn, Victoria, 3122, Australia 3 School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, , China ABSTRACT It is essential to study an effective approach of security testing for third-party component. In this paper, to effectively trigger implicit vulnerabilities of third-party components, an approach of security testing for third-party component is proposed based on state mutation. To start with, executable method sequences of components are transformed into extended finite state machine. Then, according to characteristics of condition conflict and behavior conflict, two test case generation algorithms are addressed, that is, Operations Conflict Sequences Generation Algorithm and Conditions Conflict Sequences Generation Algorithm, which are designed to generate inaccessible sequences of behavior and condition conflicts. These conflict sequences are run. Furthermore, the security detecting algorithms are addressed to detect implicit vulnerabilities of third-party components, and then, testing report of component security is obtained. In the end, some experiments are conducted on the basis of the proposed approach, and the experimental results show that the proposed approach can effectively detect security exceptions of third-party components. Copyright 2015 John Wiley & Sons, Ltd. KEYWORDS third-party component; security testing; method sequence; extended finite state machine; state mutation *Correspondence Rubing Huang, School of Computer Science and Telecommunication Engineering, Jiangsu University, Zhenjiang, , China. rbhuang@ujs.edu.cn 1. INTRODUCTION With the popularization of the third-party components, the research on security testing for third-party components is very important. Component security vulnerabilities mean the flaws in the aspects of component security including all the factors that are threatening and destroying component security. Component security vulnerabilities usually include explicit and implicit vulnerabilities. Explicit vulnerabilities commonly have some obvious security exceptions, which are caused by memory leak or buffer overflow when the software under test is run, whereas implicit vulnerabilities commonly have no some obvious security exceptions, which are usually caused by violating security requirement specification (RSF) of software under test. In our previous paper, an approach of component security testing based on extended chemical abstract machine was proposed [1], which provided the extended chemical abstract machine model to describe component information such as methods, transitions, and transition rules. With this model, the transition tree was generated, and the McCabe algorithm [2] was used to generate testing sequences based on the transition tree. Then, explicit exceptions were detected by testing these sequences based on fault injection technique, whereas implicit exceptions were detected by condition mutation approach and state mutation approach. It is a novel method to detect the explicit and implicit vulnerabilities. However, the test case generation algorithms for state and condition mutations was not very powerful, which are only suitable for testing some standard component object model (COM) components with enough type information and standard interface define language (IDL). In addition, we proposed a vulnerability testing approach based on precondition and parameter mutation [3], which extended security RSF in that some extra information was added, such as method precondition, post-condition, parameter value constraint, and parameters relation constraint, and effective mutation algorithms and test cases generation algorithms were also presented in this approach. However, this approach can only be used to detect explicit exceptions, which still cannot detect the implicit exceptions. To overcome the drawbacks of previous proposed approaches, in this paper, an approach of component security Copyright 2015 John Wiley & Sons, Ltd. 2827

2 State mutation-based security testing for third-party component J. Chen et al. testing is proposed to detect implicit security exceptions for general third-party components based on state mutation. The testing technique based on state machine in unified modeling language (UML) is frequently used to test components. Comparing with the finite state machine (FSM), extended finite state machine (EFSM, an extension of FSM), has been improved by adding some preconditions and actions of state transition into FSM. Therefore, some inaccessible sequences may be generated on the basis of EFSM [4,5]. In this paper, the mapping relations between EFSM and component method sequences were first considered. On the basis of testing sequences and security RSF, method sequences are transformed into the corresponding EFSM. Then, the concepts of condition conflict and behavior conflict are presented to help to design state mutation operators and test cases generation algorithms for generating inaccessible sequences. Finally, vulnerability detection algorithm based on state mutation is proposed to detect whether implicit exceptions exist. Main contributions in this paper are summarized as follows: (1) We further research the relationships between method sequences and EFSM, and then, a set of rules is designed to transform method sequences into EFSM. (2) State mutation operators are presented to mutate transitions and guard conditions in EFSM, and on the basis of these operators, test cases generation algorithm is addressed to generate the inaccessible sequences and method parameter values. In addition, security vulnerabilities detection algorithm based on state mutation is presented to detect component security. (3) A case study is conducted by describing the generation process of behavior and condition conflicts. Moreover, some experiments are also conducted. Compared with condition mutation and FUZZ testing techniques, experimental results show that our approach can detect more security exceptions. The remainder of the paper is organized as follows: some related component testing work is discussed in Section 2. The testing framework of state mutation and background of this paper are presented in Section 3. In addition, the transformation relations between method sequences and EFSM are also discussed in Section 3. The mutation operators, test cases generation algorithm, and security vulnerability detecting algorithm are addressed in Section 4. The case study and experiments are conducted in Section 5. The conclusions and the future work are described in Section RELATED WORK At present, the research on component security testing mainly focuses on functionality testing, security model description and assessment, and testing techniques based on fault injection and state mutation [5 26]. Ma et al. [5] proposed a component behavior model based on static information of UML component. This approach described behavior information table of interfaces, addressed interface operation mode, and its domain. Additionally, state generation and transition algorithm between different states were also presented to construct component behavior model, which was very meaningful for black-box testing of components. Zhang et al. [6] proposed a compatibility verification method of timing behavior in components, which also addressed the specification and verification tool for detecting timing behavior errors of components. Tang et al. [7] divided a web application into a set of functional components. An FSM was used to represent behavioral relationship and component interactions, and a directed graph was used to represent component structural relationship. In addition, the generation of test case set is based on the principles of complete executing sequences-coverage. On the basis of time behavior protocol and formal description methods of temporal behavior, Zhang et al. [8] proposed a verification method of behavior consistency for real-time component. This method presented a consistency verification algorithm, and the realtime component substitution theory was also discussed. Avila and Quinn et al. [9 11] proposed some approaches to construct functional test suites for software components and complex software systems composed by several components. The signature and behavior views of components that were used to describe static and dynamic information of components were also described in their papers. In addition, π calculus process was used to describe component behavior, and the corresponding algorithm that made the transfer automatic was proposed for detecting compatible errors during their composition. The aforementioned methods were all dependent on the detailed interface definition and specification, which were unsuitable for functionality behavior testing of third-party components. Chen et al. [12] proposed a quantitative assessment method to commercial off-the-shelf component security based on fault injection technology. The assessment framework, quantitative assessment algorithm, and formula are also addressed in their research on the basis of the internal factors of components. Maña et al. [13,14] proposed a security modeling framework for embedded component systems and web systems, which contained the definitions of security properties such as threats, attacks, cookie, and tests for managing security properties and specification. Nazir et al. [15] used analytic network process and ISO/IEC standard to evaluate security of components. Tang et al. [16] defined interface and behavior specifications, which described component interface syntax and functionality characteristics, respectively. Moreover, interface and behavior specifications, atomic requirement, and atomic service were also presented. On basis of the concepts, service-oriented component specification was proposed, which completely described business logic process. These methods introduced earlier were designed 2828 Security Comm. Networks 2016; 9: John Wiley & Sons, Ltd.

3 J. Chen et al. State mutation-based security testing for third-party component for describing security framework, specification, model, and security assessment, but the specific security testing techniques were not still proposed. Hashim et al. [17] used AspectJ technology to propose an improved testing technique based on interface fault injection technique, which took the advantages of crosscutting features without separate tool. Farj et al. [18] designed a tool for detecting errors in Web Services without getting application codes of Web Services through injecting incorrect values or timing faults. Hoijin et al. [19,20] applied fault injection and mutation analysis techniques to check customization and composition faults. Fugini et al. [21] analyzed the quality of composed services through injecting data faults and delays that perturbed the messages and data stored in databases used by services provider. Tao et al. [22] addressed a regression testing technique of components based on state testing. To support this testing technique, a re-test model was proposed, containing a state chart model and a test tree model. Lei et al. [23] proposed a state-based robustness testing technique for testing components. Lei regarded a state machine as a tuple SM = <S, S 0,, Guard, Tran>, where S represented a set of states, S 0 represented the initial state, and Σ represented the finite set of input symbols. To help component providers and consumers test component efficiently, Zhou et al. [24] presented a component test system by introducing UML state graph and object constraint language. Object constraint language was used to generate test data, but there were no specific testing algorithms proposed in this method. Chen et al. [1,3] proposed some related approaches to test third-party components. An approach of component security testing based on extended chemical abstract machine was proposed on the basis of extended chemical abstract machine model [1]. With this model, the transition tree and testing sequences were generated. Then, security exceptions can be detected by testing these sequences based on mutation technique. However, the test case generation algorithms for state and condition mutations was not very powerful, which are only suitable for testing some standard COM components with enough type information and standard IDL. In addition, a vulnerability testing approach based on precondition and parameter mutation was also proposed [3], which extended security RSF. And effective mutation algorithms and test cases generation algorithms were presented in this approach. However, this approach can only be used to detect explicit exceptions. Overall, previous research on component security testing was not still very sufficient. Especially, because the source codes of third-party and detailed design documents are hardly accessible, traditional testing approaches are not suitable for testing the security of third-party components. The state-based testing approach has been used to test component software, and method sequences of components have also higher code coverage rate than other test inputs. Therefore, on the basis of EFSM and method sequences, a security testing approach considering state mutation is proposed to address these problems in this paper. 3. SECURITY TESTING FRAMEWORK BASED ON STATE MUTATION 3.1. Background The research content in this paper is a part of our project, which focuses on the research on security testing of third-party components. The total framework of this project is shown in Figure 1. In this framework, the main functions modules include security specification mining module, execution sequence mining module, and component mutation module. The functions of modules are described as follows. First, on the basis of IDL document and methods information of tested component that is obtained by analyzing the component interfaces based on the packaging rules of third-party components, data mining algorithms including pattern recognition and frequent item mining algorithms are applied in specification mining module to obtain security RSF of third-party components. RSF is described by using XML schema, which consists of component name, method name, precondition and post-condition of component method, parameter name and type, return value type, parameter value constraint, and parameters relation constraint. Second, method execution sequences can be obtained by execution sequence mining module based on interface information, RSF, and log data set. Third, some mutation algorithms are applied to mutating and testing tested components in component mutation module, which is the research content of this paper. Security mutation testing of third-party components will be conducted based on RSF and executable method sequences. The mutation process is shown in the mutation testing module in Figure 1. Mutation testing is conducted based on preconditions, parameter constraints, and EFSM of tested components. An approach of component security testing based on precondition and parameters is proposed in the literature [3]. State mutation is the research focus of this paper. The method sequences are first transformed into associated EFSM by transformation algorithms. Then, EFSM is mutated to generate testing result set, and the security of the tested component will also be judged by security vulnerabilities detection algorithm Security testing framework In this paper, we mainly study the state mutation approach for testing component security. Therefore, in this section, the testing framework of state mutation is first proposed, and then, the mapping relationship between method sequences and EFSM is discussed. Moreover, state mutation operators and test case generation algorithm that generates Security Comm. Networks 2016; 9: John Wiley & Sons, Ltd. 2829

4 State mutation-based security testing for third-party component J. Chen et al. Figure 1. The project overview of proposed approach. unreachable sequences are also addressed. In the end, security vulnerabilities detection algorithm is applied to testing component security. The testing framework of state mutation is addressed to guide the test process to minimize the blindness of testing activity. State mutation testing framework is shown in Figure 2, from which key function modules and mutation flow are presented as follows. (1) EFSM transformation module The component method sequence is a collection of methods, and EFSM is a collection of states. In EFSM, transitions among states are essentially the execution sequence of methods, and the guard condition can be described by the precondition and post-condition of the tested component method. Therefore, EFSM transformation module is designed to transform method sequences into the associated EFSM according to their mapping relationship. (2) Mutants generator After transformation module transforms method sequence into EFSM, mutants generation module is applied to designing state mutation operators that mutate EFSM to inaccessible EFSM. These operators are applied to generating inaccessible conflict sequences. Figure 2. The testing framework based on state mutation Security Comm. Networks 2016; 9: John Wiley & Sons, Ltd.

5 J. Chen et al. State mutation-based security testing for third-party component (3) Test cases generator Test cases generation module uses Conditions Conflict Sequences Generation Algorithm (CCGA) and Operations Conflict Sequences Generation Algorithm (OCGA) to generate unreachable sequences of condition and operation conflict. After conflict sequences are generated, test data are obtained on the basis of constraint conditions of sequence, and then, each sequence is run and tested by vulnerabilities detection algorithm. Finally, testing report is obtained The transformation relationship between method sequences and EFSM Testing technique based on FSM is common; in the literatures [19 21], FSM is used to represent static and dynamic information of tested component. The EFSM is extended from FSM. Moreover, elements in EFSM, such as transition, state, guard condition, and trigger condition, are closely related to the method, execution sequence, and precondition and post-condition of the component method sequence. Before the relationship is discussed, some definitions of EFSM and method sequences are first given as follows. Definition 1. Extended finite state machine is a seven tuple: EFSM = <S, S 0,, Γ, P, T, V>, wheres represents state set, S 0 represents initial state, represents input set of EFSM, and Γ represents the set of trigger and guard conditions. Γ is a two tuple, Γ = <TC, GC>, wheretc represents the set of trigger conditions, whereas GC represents the set of guard condition. P represents action of transition, T represents transition set, and V represents variable set. V is a three tuple, V = <IV, OV, CV>, whereiv represents input variable set, OV represents output variable set, and CV represents environment variable set. The EFSM of tested component based on state mutation is shown in Figure 3. Definition 2. The RSF of component security is described by XML format according to some schema, which is Figure 3. EFSM of tested component based on state mutation. provided by developers or obtained through analyzing function description and IDL information by component users. Referring to requirement specification in the literature [27], some elements including method precondition, method post-condition, parameter value, and relation constraint of method parameters are added to RSF. Definition 3. Precondition is a series of constraint conditions, which must be true before the method can be invoked. Definition 4. Post-condition of a method decides its successive method and method execution order. Precondition and post-condition are both composed of relational expressions of input parameters and global variable. Definition 5. Parameter value constraint means that the parameter value is restricted in the certain scope. For example, index is the index of an array, and then, value constraint of index is expressed as index 0. Relation constraint means that constraint may exist between parameters that are described as the expression that is prone to be mistaken or be omitted. For instance, a method whose function is to judge the type of a triangle has three parameters, that is, a, b, and c to represent three edges of a triangle, and a programmer possibly makes a mistake or omits the judgment statement of non-triangle. Thus, relation constraint of the method is expressed as a+b> c && a+c> b && b +c> a. Definition 6. Method sequences are feasible execution sequences that can be generated by data mining technology or McCabe algorithm. In the literature [1], a chemical abstract model is presented, which is used to describe internal state and realization of a component. State transition tree is generated on the basis of the model, and then, McCabe algorithm is used to figure out the independent method sequences. Finally, these sequences are tested. On the basis of the aforementioned definitions and some literatures using state graph to describe components, the association rules between EFSM and method sequences of a component are shown as follows. Rule 1: Rule 2: Rule 3: A method sequence is represented by method set, which has an execution order. The EFSM is represented by state set S, which has some transitions. A method in a sequence is consistent with a state in an EFSM, and a method can be seen as a state in an EFSM. The method input parameters are consistent with input set in an EFSM. There is an association between execution order of methods and the transition in EFSM. Therefore, an execution order can be seen as a transition (t T) that is created from a state to another state. Security Comm. Networks 2016; 9: John Wiley & Sons, Ltd. 2831

6 State mutation-based security testing for third-party component J. Chen et al. Figure 4. Mapping relationships between the third-party component and EFSM. Rule 4: Rule 5: The precondition must be satisfied before its method is executed, which is associated with the trigger condition (TC) of a state. In addition, a post-condition of a method decides a successive method, and a transition takes place when a guard condition (GC) is met. An action in an EFSM can be expressed as an output sentence or an assignment sentence. These rules represent close relation between an EFSM and a method sequence. Their mapping relationship is shown in Figure 4. These rules can be applied to transforming a method sequence into an EFSM. guard condition; Dom op (p ti ) represents the domain of variable p for transition t i in the action method; head (t i ) represents source state of t i ;tail(t i ) represents destination state of t i. Definition 8. Behavior conflict: head(t j ) is the destination state of tail(t i ). Besides, the intersection of the domain of variable p for transition t i in the guard condition and the domain of variable p for transition t i in the operation method is empty, that is, Dom op (p ti ) Dom gc (p ti )=Φ, and then, behavior conflict appears between t i and t j. For example, in Figure 5, there exists behavior conflict between t 1 and t 2. It is obvious that t 1 and t 2 is interconnected, Dom op (p t1 ) is 0, and Dom gc (p t2 ) is greater than or equal to 2; therefore, their intersection is empty, and thus, t 1 and t 2 are a conflict sequence. 4. TEST CASE GENERATION ALGORITHM AND SECURITY VULNERABILITIES DETECTION ALGORITHM Some elements including method precondition and postcondition will be added to traditional state chart in the component EFSM, and the unreachable conflict sequences may be generated by mutating constraint factors of transitions. In this section, the definitions of condition and operation conflicts [4,28] are given as follows. Then two conflict sequence generation algorithms are proposed on the basis of two definitions, that is, OCGA and CCGA. Definition 7. Guard condition gc and action op are global variable of the method, and their variable p means the parameter of the method. Dom gc (p ti ) represents the domain of variable p for transition t i in the Definition 9. Condition conflict: head(t j ) is the destination state of tail(t i ). Besides, the intersection of the domain of variable p for transition t i and t j in the guard condition is empty, that is, Dom gc (p ti ) Dom gc (p ti )=Φ, and then, Figure 5. An example of EFSM Security Comm. Networks 2016; 9: John Wiley & Sons, Ltd.

7 J. Chen et al. State mutation-based security testing for third-party component Table I. State mutation operators. ID Operator Description 01 INT (increase the transition) A new transition t is increased, whose head state is a and tail state is b, and defaults of its trigger and guard conditions are True and default of action is null. 02 CST (change the head of the transition with other status) If head and tail states are a and b, CST operator changes a into c. However, the conditions (guard and trigger) and action of t keep constant. condition conflict appears between t i and t j. For example, in Figure 5, there exists a condition conflict between t 4 and t 5, which is because the intersection of Dom gc (p t4 ) and Dom gc (p t5 ) is empty. On the basis of the aforementioned definitions about condition and operation conflicts, some state mutation operators are designed, and then, condition conflict sequence generation algorithm and operation conflict sequence generation algorithm are also proposed as follows State mutation operators The purpose of state mutation is to construct method sequences of condition and operation conflict by changing the head of a transition or adding a new transition state based on associated mutation operators, and then, these conflict sequences are made unreachable. The definitions of state mutation operators are shown in Table I. In Table I, operator increase the transition (INT) is designed to increase a new transition when operation conflict or condition conflict is taken place between two transitions of EFSM. The head of the new transition is the tail of the first transition, and the tail of the new transition is the tail of the second transition. However, if there is a transition from the tail of the first transition to the tail of the second transition in original EFSM, the new transition cannot be added; otherwise, there will be two transitions that have same directions. In addition, operator change status transition (CST) changes the head of the second transition into the tail of the first transition, and the two transitions are linked directly Test case generation algorithms based on state mutation Operations Conflict Sequences Generation Algorithm is designed to generate unreachable sequences that meet the characteristics of behavior conflict based on the EFSM mutated by state mutation operators. Similarly, CCGA is designed to generate insecure sequences that meet the characteristics of condition conflict. The two algorithms are presented as follows. Operations Conflict Sequences Generation Algorithm is designed to generate operation conflict sequences based on EFSM. In the OCGA, first, the topology sequence is generated from EFSM [29], and then, the topology sequence will be tuned to transition sequence T, which guarantees that the final conflict sequence starts from the initial method. Then, each transition t i of T is orderly traversed, if the Security Comm. Networks 2016; 9: John Wiley & Sons, Ltd. 2833

8 State mutation-based security testing for third-party component J. Chen et al. action of t i contains an assignment sentence of global variable y, the value of y is computed after being transformed from t 0 to t i. In the meantime, if a transition t k that is after t i can be found, the intersection domain between y of t k and t i is empty, which shows {t 0,, t i, t k } is an operation conflict sequence; operator CST is applied to mutating EFSM and changing the head of t k into the tail of t i. In addition, operator INT is used to add a new transition from the tail of t i to the head of t k, and then, {t 0,, t i, t ik, t k } is part of condition conflict sequence. In the end, these conflict sequences are merged into the result set R. For analyzing the time complexity of OCGA, it is supposed that there are n transitions in the topology sequence and m states in EFSM. The order of time complexity of topology sequence generation algorithm is O (m + n), so the order of time complexity of OCGA is O(m + n + n (n 1)/2), that is, O(m + n 2 ). Condition Conflict Sequences Generation Algorithm is designed to mutate the EFSM of the tested component to generate condition conflict sequences set W. The process of CCGA is similar to CCGA, but the duty of CCGA is to judge whether there is an empty intersection between the two domains, namely the domain of the global variable y for the transition t i in the guard condition and the domain of the global variable y for the follow-up transition t k in the guard condition. If it is empty, it can assert that {t 0,, t i, t k } is a condition conflict sequence and this sequence will be merged into W. For analyzing the time complexity of CCGA algorithm, it is supposed that there are m states and n transitions in the EFSM. Because the process of CCGA is similar to OCGA, the order of time complexity of OCGA is O(m + n 2 ) Security vulnerabilities detection algorithm based on state mutation With the help of security vulnerabilities detection algorithm, two kinds of conflict sequence sets will be taken, namely behavior conflict sequence and condition conflict sequence, by the test case generation algorithm. If every method in the sequence has no exception faults, some methods in the sequence will be terminated owing to the behavior or condition conflict, after the appropriate test data was injected. On the contrary, if the conflict sequence is completely executed, there is at least a method that has exception faults in the sequence. Security vulnerability detecting algorithm based on sate mutation is presented as follows Security Comm. Networks 2016; 9: John Wiley & Sons, Ltd.

9 J. Chen et al. State mutation-based security testing for third-party component Table II. Component information of BankTransaction.dll and state information of EFSM. ID Method name Method description Parameter description 01 Open Open a bank account 02 Deposit Deposit operation Parameter a: deposit amount 03 Withdraw Withdraw operation Parameter b: withdraw amount 04 GetBalance Check account balance 05 Privilege1 if balance y is lower than 5000, Privilege1 is processed Global variable y: balance 06 Privilege2 If balance y is higher than 5000, Privilege2 is processed Security Vulnerability Detecting Algorithm Based on State Mutation calls OCGA and CCGA to generate separately operation conflict and condition conflict sequences. Two kinds of conflict sequences are then merged into T. For each conflict sequence, constraint equation set is constructed on the basis of guard and trigger conditions to create the value of each method parameter. Then, these values are assigned to the parameters and methods in each sequence. If each method in a conflict sequence is completely executed, it is shown that the tested component is insecure. As previously presented, the average order of time complexity of OCGA and CCGA are both O(m + n 2 ), where m represents the number of methods and n represents the number of transitions. In addition, the average time complexity of constraint equation set solution algorithm is O(v k), where v represents the parameter number of all methods in a sequence and k is the number of relational expressions. Therefore, the average order of time complexity of SVDASM is O(m + n 2 + t (v k)). 5. CASE STUDY AND EXPERIMENT ANALYSIS To describe the generation process of conflict sequences in detail, an EFSM example about simple bank business component is first given in this section. In addition, to verify the feasibility and effectiveness of the proposed approach, state mutation experiment is also conducted on the basis of several COM components Case study In this section, we will take an example of BankTransaction.dll to illustrate the generation process of conflict sequences and testing result. The bank business component includes six methods, which respectively implement different functions such as opening accounts, depositing money, withdrawing money, and checking balance. The detailed information about BankTransaction. dll is listed in Table II. This component uses a global variable to represent the balance in the bank card. On the basis of the function specification of the component, it can be inferred that some component methods contain assignment operation that is related with the global variable. As a result, there exist some arithmetic logic errors in the bank business component. The EFSM of the bank business component is shown in Figure 6. The transition topological sequence of the EFSM is{t 1, t 2, t 3, t 4, t 5, t 6, t 7, t 8 },which is described in Table III. According to the proposed OCGA, detailed steps are described as follows: (1) The transitions of the transition sequence {t 1, t 2, t 3, t 4, t 5, t 6, t 7, t 8 } are successively traversed. First, transition t 1 is taken as current transition, and the operation of t 1 contains the global variable operation y=0. Then, it is found that the guard condition of t 3 is 0 b y, which is among transitions after t 1. Therefore, when b is positive, the domains of y respectively in t 1 and t 3 have no intersection, which indicates that there exists operation conflict between t 1 and t 3. Then, operator CST is used to mutate t 3 into t 3, which is marked in red and shown in Figure 6, and then, it is obtained that {t 1, t 3, } is an operation conflict sequence. Similarly, when ID is t 8, the domain of y in guard condition does not include 0, so the intersection is empty. (2) The transitions are orderly traversed along the transition sequence, and t 2 is taken as current transition, and y is equal to a after t 1, and then, t 2 are traversed. Similarly, Figure 6. EFSM of the bank business. Security Comm. Networks 2016; 9: John Wiley & Sons, Ltd. 2835

10 State mutation-based security testing for third-party component J. Chen et al. Table III. Transition description of EFSM. ID Guard condition and its description Operation and its description t 1 True / y = 0 Analog assignment operation of y after open method is called t 2 a > 0,y 0 Deposit amount a is positive and balance y is non-negative y = y + a Analog assignment operation of y after deposit method is called t 3 0 b y Withdraw amount b is less than y and b is non-negative y = y b Analog assignment operation of y after withdraw method is called t 4 a > 0,y 0 Deposit amount a is positive and balance y is non-negative y = y + a Analog assignment operation of y after deposit method is called t 5 True / {} Null operation t 6 True / {} Null operation t 7 y < 5000 balance y < 5000 {} Null operation t 8 y 5000 Balance y 5000 {} Null operation according to previous steps, it need be judged whether there are operation conflicts between t 2 and one transition among {t 3, t 4, t 5, t 6, t 7, t 8 }. According to the aforementioned steps, operation conflict sequences are listed as follows. ID1: Init, (true/y = 0) Open, (y > = b > 0/y = y b) Withdraw, (a > 0, y > =0/y = y + a) Deposit, (true/{}) GetBalance, (y <5000/{}) Pri1 ID2: Init, (true/y = 0) Open, (a > = 5000/y = y + a) Deposit, (y <5000/{}) Pri1 ID3: Init, (true/y = 0) Open, (a b > =5000/y = y b) Withdraw, (y <5000/{}) Pri1 ID4: Init, (true/y = 0) Open, (a > 0, y > =0/y = y + a) Deposit, (a b < 5000/y = y b) Withdraw, (y > = 5000/{}) Pri2 Here, we only list four operation conflict sequences because of the limited space; the rest of the operation conflict sequences are omitted. Similarly, according to CCGA, transition t 2 is firstly taken as current transition because the guard condition of t 1 has no y. It can be found that there is no transition among {t 3, t 4, t 5, t 6, t 7, t 8 }, which has conflicts with t 2. Then, t 3 is orderly traversed; it can be observed that the guard condition of t 3 (5000 b y) has conflicts with the guard condition of t 7 (y < 5000). According to the aforementioned steps, condition conflict sequences generated are listed as follows: ID1: Init, (true/y = 0) Open, (a > 0,y > =0/y = y + a) Deposit, (y > = b > =0/y = y b) Withdraw, (a > 0,y > =0/y = y + a) Deposit, (true/{}) GetBalance, (y < 5000/{}) Pri1, (true/{}) GetBalance, (y > = 5000/{}) Pri2 ID2: Init, (true/y = 0) Open, (y > = b > =0/y = y b) Withdraw, (true/{}) GetBalance, (y < 5000/{}) Pri1, (true/{}) GetBalance, (y > = 5000/{}) Pri2 ID3: Init, (true/y = 0) Open, (a > 0,y > =0/y = y + a) Deposit, (y > = b > =0/y = y b), Withdraw, (true/{}) GetBalance, (y < 5000/{}) Pri1, (true/{}) GetBalance, (y > = 5000/{}) Pri2 Here, we only list three condition conflict sequences because of the limited space; the rest of the condition conflict sequences are omitted. The conflict sequences of operation and condition conflicts generated by previous steps are run to detect component security exceptions by vulnerabilities detection algorithm based on state mutation. First, conflict transition sequences are transformed into conflict method sequences, and the test data set is generated according to the constraint conditions mutated in conflict sequences. Then, test data are assigned to parameters, and the method is run. According to the definition of conflict sequence, if each method Table IV. The information of tested components. Component name Function description Method number Number of faults injected BankTransaction.dll Different bank businesses such as open account, 6 3 deposits, withdrawals, and balance inquiries. Examine.dll Chinese and math score statistics 6 2 Calculator.dll A calculator that realize addition and subtraction 9 4 Order.dll Order function according to menu 8 3 BeverageVending.dll A beverage vending machines that can drop coin, buy beverages, and return change Security Comm. Networks 2016; 9: John Wiley & Sons, Ltd.

11 J. Chen et al. State mutation-based security testing for third-party component in the conflict sequence is correct and sequences are run with test data that meet constraints, the sequence will be terminated in some method such as m 1 in the sequence because the global variables do not meet the constraint condition of m 1. However, if there are wrong operations about the global variable, the conflict sequence may be completely run without termination. In the end, it can be found that there exist some assignment errors and arithmetic logic errors about the global variable in the tested bank business component. Figure 7. The testing process of state mutation Experimental setup To evaluate the feasibility and effectiveness of the proposed approach, some experiments are conducted on the basis of state mutation testing approach. In the experiment, five subject COM components including BankTransaction.dll are conducted, which are listed in Table IV with component name, function description, method number, and number of faults injected. All tests are run on a machine having an Intel dual core i GHz processor, 2 GB of RAM, and running Windows XP. The development environment is Visual Studio The complete testing process is described as follows: (1) the IDL document of third-party components is analyzed and parsed to obtain static information such as interface method and parameter type; (2) the security RSF, which is described by XML schema, can be obtained on the basis of the simple function specification and the data set mined from monitoring log. The security RSF contains more component information including method name, method pre-condition and post-condition, parameter type, parameters value constraint, and parameters relationship constraint; (3) executable method sequences will be also mined on the basis of security RSF and log data set; (4) method sequences will be transformed into the corresponding EFSM; (5) OCGA and CCGA will be applied to generating unreachable sequences of behavior and condition conflicts; and (6) these insecure sequences will be run and tested, and then, security exceptions of Path Operation conflict transition sequences Table V. Operations conflict sequences of BankTransaction.dll. Operation conflict method sequences 1 t 1 t 3 t 4 t 6 t 7 init, (true/y = 0) Open, (b < = y &&b > 0/y = y b) Withdraw, (a > 0,G > =0/y = y + a) Deposit, (true/{}) GetBalance, (y < 5000/{}) Pri1 2 t 1 t 3 t 4 t 6 t 8 init, (true/y = 0) Open, (b < = y&&b > 0/y = y b) Withdraw, (a > 0,y > =0/y = y + a) Deposit, (true/{}) GetBalance, (y > = 5000/{}) Pri2 3 t 1 t 3 t 5 t 7 init, (true/y = 0) Open, (b < = y&&b > 0/y = y b) Withdraw, (true/{}) GetBalance, (y < 5000/{}) Pri1 4 t 1 t 3 t 5 t 8 init, (true/y = 0) Open, (b < = y&&b > 0/y = y b) Withdraw, (true/{}) GetBalance, (y > = 5000/{}) Pri2 5 t 1 t 18 t 8 init, (true/y = 0) Open, (true/{}) GetBalance, (y > = 5000/{}) Pri2 6 t 1 t 8 init, (true/y = 0) Open, (y > = 5000/{}) Pri2 7 t 1 t 2 t 3 t 4 t 6 t 7 init, (true/y = 0) Open, (a > 0,b > a/y = y + a) Deposit, (b < = y&&b > =0/y = y b) Withdraw, (a > 0,y > =0/y = y + a) Deposit, (true/{}) GetBalance, (y < 5000/{}) Pri1 8 t 1 t 2 t 3 t 4 t 6 t 8 init, (true/y = 0) Open, (a > 0,b > a/y = y + a) Deposit, (b < = y&&b > =0/y = y b) Withdraw, (a > 0,y > =0/y =y+a) Deposit, (true/{}) GetBalance, (y > = 5000/{}) Pri2 9 t 1 t 2 t 3 t 5 t 7 init, (true/y = 0) Open, (a > 0,b > a/y = y + a) Deposit, (b < = y&&b > =0/y = y b) Withdraw, (true/{}) GetBalance, (y < 5000/{}) Pri1 10 t 1 t 2 t 3 t 5 t 8 init, (true/y = 0) Open, (a > 0,b > a/y = y + a) Deposit, (b < = y&&b > =0/y = y b) Withdraw, (true/{}) GetBalance, (y > = 5000/{}) Pri2 11 t 1 t 2 t 7 init, (true/y = 0) Open, (a > = 5000/y = y + a) Deposit, (y < 5000/{}) Pri1 12 t 1 t 2 t 8 init, (true/y = 0) Open, (a < 5000/y = y + a) Deposit, (y > = 5000/{}) Pri2 13 t 1 t 2 t 3 t 7 init, (true/y = 0) Open, (a > 0,y > =0/y = y + a) Deposit, (b > 0,a > = b/y = y b) Withdraw, (y < 5000/{}) Pri1 14 t 1 t 2 t 3 t 8 init, (true/y = 0) Open, (a > 0,G > =0/y = y + a) Deposit,(b > 0,a < b +5000/y = y b) Withdraw, (y > = 5000/{}) Pri2 Security Comm. Networks 2016; 9: John Wiley & Sons, Ltd. 2837

12 State mutation-based security testing for third-party component J. Chen et al. Path Condition conflict transition sequences Table VI. Condition conflict sequences of BankTransaction.dll. Condition conflict method sequences 1 t 1 t 2 t 3 t 4 t 6 t 7 t 78 t 8 init, (true/y = 0) Open, (a > 0,y > =0/y = y + a) Deposit, (b < = y&&b > =0/y = y b) Withdraw, (a > 0,y > =0/y = y + a) Deposit, (true/{}) GetBalance, (y < 5000/{}) Pri1, (true/{}) GetBalance, (y > = 5000/{}) Pri2 2 t 1 t 2 t 3 t 5 t 7 t 78 t 8 init, (true/y = 0) Open, (a > 0,y > =0/y = y + a) Deposit, (b < = G&&b > =0/y = y b) Withdraw, (true/{}) GetBalance, (y < 5000/{}) Pri1, (true/{}) GetBalance, (y > = 5000/{}) Pri2 3 t 1 t 2 t 6 t 7 t 78 t 8 init, (true/y = 0) Open, (a > 0,y > =0/y = y + a) Deposit, (true/{}) GetBalance, (y < 5000/{}) Pri1, (true/{}) GetBalance, (y > = 5000/{}) Pri2 4 t 1 t 2 t 3 t 4 t 6 t 7 t 8 init, (true/y = 0) Open, (a > 0,y > =0/y = y + a) Deposit, (b < = y&&b > =0/y = y b) Withdraw, (a > 0,y > =0/y = y + a) Deposit, (true/{}) GetBalance, (y < 5000/{}) Pri1, (y > = 5000/{}) Pri2 5 t 1 t 2 t 3 t 5 t 7 t 8 init, (true/y = 0) Open, (a > 0,y > =0/y = y + a) Deposit, (b < = y&&b > =0/y = y b) Withdraw, (true/{}) GetBalance, (y < 5000/{}) Pri1, (y > = 5000/{}) Pri2 6 t 1 t 2 t 6 t 7 t 8 init, (true/y = 0) Open, (a > 0,y > =0/y = y + a) Deposit, (true/{}) GetBalance, (y < 5000/{}) Pri1, (y > = 5000/{}) Pri2 components will be detected by vulnerabilities detection algorithm, and in the end, security testing report will be obtained. The whole state mutation testing process is shown in Figure Experiment analysis In this paper, the analysis process of BankTransaction.dll is illustrated in detail. The analysis process of other components is similar to that of bank component. Each component may contain assignment and arithmetic logic errors about the global variable. Transition and method sequences of operation and condition conflicts for BankTransaction.dll are listed in Tables V and VI, and specific generation steps are also described as follows. Some conflict sequences of BankTransaction.dll are listed in Tables V and VI, and in the third column of Tables V and VI, method sequences are shown, which consist of mutated constraint conditions that are used to obtain data of method parameters. The domain of these conditions is the subset of original conditions. On the basis of these constraint conditions, test data are generated for parameters to make method called. Experimental results show that four sequences among 20 conflict sequences can be completely executed, which illustrates that BankTransaction.dll has implicit security exceptions. The information of five tested components is listed in Table IV. In addition, the results of state mutation testing for these five components are shown in Table VII, including the number of operation conflict sequence and condition conflict sequence. The number of effective sequences that can detect faults effectively and the sequence detection rate are also shown in Table VII. Additionally, the sequence detection ratio is equal to the percentage of number of effective sequence and the total number of conflict sequences. On the basis of the experimental data shown in Table VII, it can be seen that the more the number of component methods are, the more conflict sequences are generated; sequence detection rate shows that to some extent, state mutation approach can effectively detect implicit vulnerabilities and the proposed approach is effective. To further analyze the effectiveness of proposed approach, we compared it with the condition mutation method [2] proposed in the previous paper and FUZZ testing technique [1] based on five tested components. The comparison results for the three approaches are shown in Table VIII. Some comparison items including number of faults injected, number of test cases, effective test cases that can detect faults, and the detection ratio of test case are listed in Table VIII based on state mutation method, condition mutation method, and FUZZ testing technique. The Table VII. The testing result of state mutation. Component name Number of faults injected Number of operation conflict sequences Number of condition conflict sequences Number of effective sequences Sequence detection rate (%) BankTransaction dll Examine.dll Calculator.dll Order.dll BeverageVending. dll Security Comm. Networks 2016; 9: John Wiley & Sons, Ltd.

13 J. Chen et al. State mutation-based security testing for third-party component Table VIII. The comparison result of five components using three different testing approaches. Component name Testing approach Number of faults injected Number of test cases Number of effective test cases that find faults Effective rate of test cases (%) BankTransaction.dll State mutation Condition mutation FUZZ Examine.dll State mutation Condition mutation FUZZ Calculator.dll State mutation Condition mutation FUZZ Order.dll State mutation Condition mutation FUZZ BeverageVending.dll State mutation Condition mutation FUZZ test case detection ratio is equal to the ratio of the number of effective cases to the total number of test cases. From Table VIII, we can see that test case detection ration for Examine.dll using state mutation method is the highest with 37.5%, whereas that of Calculator.dll is the lowest compared with other components, and its ratio is only 26.3%. In addition, the ratios of each component tested by state mutation method are higher than those obtained using the other two methods. The histograms of test case ratio for five tested components based on three different methods are respectively shown in Figures In these figures, the first bar represents the ratio derived from state mutation method, the second bar represents the ratio derived from condition mutation, and the third bar represents the ratio derived from FUZZ testing technique. From Figures 8 12, it is obvious that the state mutation bar is higher than the other Figure 9. Effective rate of test cases for Examine.dll. Figure 8. Effective rate of test cases for BankTransaction.dll. Figure 10. Effective rate of test cases for Calculator.dll. Security Comm. Networks 2016; 9: John Wiley & Sons, Ltd. 2839

14 State mutation-based security testing for third-party component J. Chen et al. Figure 11. Effective rate of test cases for Order.dll. The comprehensive comparison results for three testing approaches are shown in Figure 13 to illustrate the overall effectiveness of test cases. The horizontal coordinate represents the number of test cases generated, whereas vertical coordinate represents the number of test cases that can detect faults. From Figure 13, we can see that with the increase of test cases, much more cases can detect faults. In addition, upward tendency of the line representing state mutation is more obvious than that of the line representing condition mutation method, whose upward tendency is more obvious than that of the line representing FUZZ testing technique. From the aforementioned figure, it is shown that state mutation method can effectively detect assignment and arithmetic logic errors related to global variables and can detect more such errors than condition mutation method and FUZZ testing technique Threats to validity Like any empirical evaluation, our study also has limitations that must be considered. Although we have experimented five subject components and some of them are downloaded from open source websites, some of them are not well known and a little smaller than integrated industry components. In addition, the faults used in our experiment are seeded manually by general mutation operators instead of derived from real programs. 6. CONCLUSIONS AND FUTURE WORK Figure 12. Effective rate of test cases for BeverageVending.dll. Figure 13. Comprehensive comparison results of five tested components. two bars. The ratio for Examine.dll tested by state mutation method is the highest, whereas the ratio of Examine.dll tested by FUZZ testing technique is the lowest. Generally speaking, source codes and specific specification of third-party components are difficult to obtain, which brings the great challenges for security testing of thirdparty components. To detect implicit vulnerabilities of components, a security testing approach based on state mutation is proposed in this paper. The method sequences are first transformed into associated EFSM, and then, state mutation operators are applied to mutating EFSM. As a result, the unreachable sequences of behavior and condition conflicts are generated. In the end, the component security is determined by security vulnerabilities detection algorithm. The advantages and disadvantages of proposed approach and the future research work are summarized as follows. (1) The security RSF proposed in this paper is extended, which not only records more information such as method name, parameter name, and parameter type but also contains method precondition, post-condition, parameter value, and parameters relationship constraints. In addition, to better guide the process of state mutation testing, the testing framework of state mutation is addressed. Additionally, according to the characteristics of the behavior and condition conflicts, OCGA and CCGA are proposed. In two algorithms, component EFSM is mutated by using state mutation operators, and 2840 Security Comm. Networks 2016; 9: John Wiley & Sons, Ltd.