Automatically Testing Interacting Software Components

Transcription

1 Automatically Testing Inteacting Softwae Components Leonad Gallaghe Infomation Technology Laboatoy National Institute of Standads and Technology Gaithesbug, MD 20899, USA Jeff Offutt Infomation and Softwae Engineeing Geoge Mason Univesity Faifax, VA 22030, USA ABSTRACT One goal of integation testing fo object-oiented softwae is to ensue high object inteopeability. Sent messages should have the intended effects on the states and subsequent actions of the eceiving objects. This is especially difficult when softwae is compised of components developed by diffeent vendos, with diffeent languages, and the implementation souces ae not all available. A pevious pape pesented a model of inte-opeating OO classes based on finite state machines. It addesses methods fo identifying the elevant actions of a test component to be integated into the system, tansfoms the finite state specification into a contol and data flow gaph, labels the gaph with all defs and uses of class vaiables, and pesents an algoithm to geneate test specifications as specific paths though the diected gaph. It also pesents empiical esults fom an automatic tool that was built to suppot this test method. This pape pesents additional details about the tool itself, including how seveal difficult poblems wee solved, and adds new capabilities to help automate the tansfomation of test specifications into executable test cases. The esult is a fesh appoach to automated testing. It follows accepted theoetical pocedues while opeating diectly on an objectoiented softwae specification. This yields a data flow gaph and executable test cases that adequately cove the gaph accoding to classical gaph coveage citeia. The tool suppots specification-based testing and helps to bidge the gap between theoy and pactice. Categoies and Subject Desciptos D.2.5 [Softwae Engineeing]: Testing and Debugging Testing tools The second autho is sponsoed in pat by National Institute of Standads and Technology (NIST), Softwae Diagnostics and Confomance Testing Division (SDCT). Pemission to make digital o had copies of all o pat of this wok fo pesonal o classoom use is ganted without fee povided that copies ae not made o distibuted fo pofit o commecial advantage and that copies bea this notice and the full citation on the fist page. To copy othewise, to epublish, to post on seves o to edistibute to lists, equies pio specific pemission and/o a fee. AST 06, May 23, 2006, Shanghai, China. Copyight 2006 ACM X/06/ $5.00. Geneal Tems Veification 1. INTRODUCTION Many object-oiented applications ae constucted fom a combination of peviously witten, extenally obtained components with some new components added fo specialization. Souce is often not available fo the peviously witten components, yet new objects must inteopeate via messages with objects in the existing components. This eseach is concened with ensuing that objects inte-opeate coectly, paticulaly when new objects ae added to existing components. In this pape, a class is the basic unit of semantic abstaction and each class is assumed to have state and behavio [2, 11]. A component is a closely elated collection of classes (possibly even a single class), and components inte-opeate to povide needed functionality. Each component is assumed to be a sepaate executable, theeby allowing asynchonous behavio. An object is an instance of a class. Each object has state and behavio, whee state is detemined by the values of vaiables defined in the class, and behavio is detemined by methods defined in the class that opeate on one o moe objects to ead and modify thei state vaiables. The behavio of an object is modeled as the effect the method has on the vaiables of that object (the state), togethe with the messages it sends to othe objects. Vaiables declaed by the class that have one instance fo each object ae called instance vaiables, and vaiables that ae shaed among all objects of the class (static in Java) ae class vaiables. This eseach is independent of pogamming language and the pape uses a mix of Java and C++ teminology. Behavio of objects is captued as a set of tansition ules fo each method and descibed as finite state machines [3, 4, 12]. A tansition is tiggeed by a call to a method with a paticula signatue, and is compised of a souce state, a taget state, an event, a guad, and a sequence of actions. Events ae epesented as calls to membe functions of a class. A guad is a pedicate that must be tue fo the tansition to be taken; guads ae expessed in tems of pedicates ove state vaiables (possibly fom multiple classes) and input paametes to the method. An action is pefomed when the tansition occus; actions ae usually expessed as assignments to class membe vaiables, calls sent to othe objects, and values that ae etuned fom the event method. A sequence of actions is assumed to be a block of statements in which all opeations ae executed if any one is executed.

2 Pe-conditions and post-conditions of methods can be deived diectly fom the tansitions. The pe-condition is a combination of the pedicates of the souce state and the guad; the post-condition is the pedicate of the taget state. Note that the post-condition deived fom a tansition is not the stongest post-condition. If the teste desied, state definitions could be moe efined, allowing stonge postconditions, which would yield lage gaphs and moe tests. Whethe to do so is a choice of ganulaity that esults in a cost vesus potential benefit tadeoff. A state tansition specification fo a class is the set of state tansition ules fo each method of the class. Given a state tansition specification fo each class, the goal of this eseach is to constuct test specifications that ae used to constuct an executable test suite. Hong et al. [8] developed a class-level flow gaph to epesent contol and data flow within a single class. Ou pevious pape [6] extended thei ideas to integation testing of multiple inteacting classes. The state tansition specification is stoed in a elational database. Tansitions that ae elevant to the class unde test ae used to ceate a component flow gaph, which includes contol and data flow infomation. Classical data flow test citeia ae applied to this gaph and conveted to test specifications in the fom of candidate test paths, and then to executable test cases. In taditional data flow testing [5], the teste is povided with pais of definitions and uses of vaiables (def-use pais), and then attempts to find tests to cove those def-use pais. This eseach stoes infomation about the specification in the database, epesents object behavio as banch choices in a diected gaph, povides the teste with full def-use paths instead of just def-use pais, and povides contol mechanisms to constuct calls of extenal methods that foce tavesal of the identified paths. 2. BEHAVIOR IN A DIRECTED GRAPH A combined class state machine epesents the vaiables, methods, paametes, states and tansitions of all state tansition specifications fo all classes in a system of inteopeating components. If a specific component is identified as the test component in this system, then tansitions fom the classes in the test component fom the basis of tansitions that ae elevant to that component. Relevant tansitions that call mutato methods in othe components of the system epesent outwad data flow, wheeas tansitions that call acto functions in othe components epesent inwad data flow. The collection of all tansitions in othe components that have a method so called by a elevant tansition become themselves elevant tansitions to the test component. The tansitive closue of this pocess defines the collection of elevant tansitions fo the test component. A component flow gaph epesents all data and contol flow elevant to a test component. It is a diected gaph deived fom the elevant tansitions as follows: Evey elevant tansition is a tansition node Evey state that is eithe a souce state o a taget state of a elevant tansition is a state node Evey non-tivial guad of a elevant tansition is a guad node Thee is a diected edge fom evey guad node to its coesponding tansition node S ss G t T t S ts Figue 1: Path Deived Fom a Relevant Tansition Thee is a diected edge fom evey tansition node to its taget state Thee is a diected edge fom the souce state node of a tansition to eithe the guad node of that tansition o to the tansition node itself if the guad is tivially tue In geneal, evey elevant tansition t poduces a path in the component flow gaph fom its souce state node, though the guad and tansition nodes, to the taget state node. This is shown in Figue 1. The path in Figue 1 epesents contol flow fom the state that an object is in when a method event is invoked, to the guad node that evaluates to tue, to the action of the tansition with that guad, to the taget state of the tansition. Actions on vaiables ae also epesented by tansitions with associated get and set methods. Tansitions with get (o acto) methods typically do not have non-tivial guads, and neve change the state of an object, so usually poduce paths in the component flow gaph only fom state nodes whee the method is defined though the tansition node and back to the same state node. In multi-class systems, state and guad pedicates ae allowed to call acto functions fom othe classes and the action of a tansition is allowed to invoke acto o mutato functions in othe classes. Contol and data flow esulting fom these actions is epesented by edges that connect nodes fom diffeent classes, as follows: If a state o guad pedicate, o the action of a tansition, calls an acto method in anothe class, thee will be an edge fom evey tansition node of the called method back to the node epesenting the calling state, guad o tansition. If the action of a tansition calls a mutato function in anothe class, thee will be an edge fom the tansition node that epesents that tansition to the souce state node of evey tansition of the called method in the othe class. In addition, if the mutato method etuns a value, thee will be an edge fom evey tansition node of the called method back to the tansition node of the action that makes the call. To illustate how the component flow gaph epesents data and contol flow acoss communicating objects, conside thee objects, obj1, obj2 and obj3, which ae instances of classes A, B and C (note that to make the example small, these components ae single classes, wheeas in geneal, the components can be much lage). Class A defines a tansition t 1, which epesents a mutato function f(), and whose action defines a vaiable v. This tansition as epesented in a component flow gaph is pesented in Figue 2. The tansition node is T 1, the guad node is G 1, and S a and S b ae the souce and taget state nodes. Tansition nodes that define a vaiable ae labeled, such as def v on node T 1. Uses of v

3 ae epesented by points u i on edges and inside tansition nodes. Both S a and S b use v (u 1 and u 3 ) in thei pedicate definitions. Function f() is invoked fom some othe tansition, which is epesented by tansition node T x. The get function fo v is epesented by tansition node T va if obj1 is in state S a when it s called, and by T vb if obj1 is in state S b. T x obj1 T va obj2 T y call f () u 1 S c call g () get v S a g () obj3 f () u 2 G 2 S e G 1 call h (p) u 4 h (p) T 1 T 2 u5 G 3 def v get v u 6 S d est S b T 3 u7 Figue 2: Potion of Component Flow Gaph Conside obj2, and a tansition t 2 (tansition node T 2 ). The guad pedicate fo t 2 (G 2) calls the get function fo v in obj1. This get is epesented by two edges fom obj1, both labeled by get v. G 2 uses v (u 4 ), and fo it to evaluate to tue, the mutato function g() must be called by some othe action (epesented by T y) when obj2 is in state S c. The action of T 2 uses v (u 5) and sends a message to obj3 by invoking the mutato function h(), passing the value of v as a paamete. Obj2 changes to state S d as the taget state of tansition t 2. Tansition t 3 handles the function call of h() when obj3 is in state S e, if the guad pedicate fo t 3 evaluates to tue. The guad of t 3 (G 3 ) uses the value of the incoming paamete p, which has the value of v, epesented by use u 6. The action of t 3 uses the value in a computation (u 7 ). Afte the action of t 3 is complete, obj3 changes to the taget state S f. In the figue, edges fom tansition nodes to state nodes that esult fom calling mutato functions in anothe class ae labeled with the name of that function (callf(), callg(), and callh()). Function names ae also put on edges fom tansition nodes to guad, state, and tansition nodes if the edge is a esult of a call to a get function in anothe class (get v). Edges fom souce state nodes to guad and tansition nodes ae also labeled with the function that is called (f(), g(), and h()). These labels ae used to access metadata about the functions when the component flow gaph is tavesed. In a geneal data flow gaph, uses of a vaiable in a state pedicate ae epesented by use labels on all edges leaving the coesponding state node (u 1, u 2, u 3); uses of a vaiable o paamete in a guad pedicate ae epesented by use labels on all edges leaving the coesponding guad node (u 4, u 6 ). These ae called pedicate uses. Uses of a vaiable u 3 T vb S f in the action of a tansition ae called computational uses (u 5, u 7 ). Uses of a vaiable in a passed paamete ae called paamete uses (u 6, u 7). To summaize, Figue 2 epesents the data and contol flow esulting fom definition of vaiable v by a call of function f in obj1, the pedicate and computational use of v s value in obj2, the passing of that value as a paamete to function h(p), and the esulting pedicate and computational paamete use of v in the guad and action of t 3 in obj3. This is a flexible model that can and should be adapted by the develope. Fo example, if things such as countes ae impotant to the FSM, they can be captued in the state pedicates (we ae cuently employing this technique fo anothe application). How complete ou model is will depend on how complete the design model is. If the initial specification coves all possible behavios, then they will be captued in the database. An advantage of this appoach is that once the methods and possible states ae in the database, we can do some athe igoous testing to ensue that all possible combinations ae addessed. Space equiements and limitations ae discussed in the pevious pape [6]. 3. COVERAGE CRITERIA A numbe of diffeent coveage citeia can be defined on data flow gaphs, including all-defs, all-uses and allpaths. These have been discussed and compaed extensively in the liteatue [5, 7, 9, 10, 13]. The object-oiented testing methodology descibed hee follows the lead of othe eseaches and focuses on defs and uses of class vaiables in each object. This allows a teste to focus on testing citeia that equie tavesal of a def-use path in the component flow gaph fom a tansition node that defines a vaiable to a node o edge that uses the vaiable, with no e-definition of the vaiable at any node along the path. The all-uses testing citeion applied to a component flow gaph equies tests to execute at least one path fom each definition of a vaiable at a tansition node to each eachable use at anothe node o edge. In Figue 2, applying the alluses citeion to vaiable v in class A equies tests that will foce execution of def-use paths fom tansition node T 1 to each of the uses u 1 to u 7. In the potion of the component flow gaph pictued, one sees that uses u 1 and u 2 ae not eachable fom T 1 and that any path fom T 1 to u 7 will include subpaths fom T 1 to each of the uses u 3, u 4, u 5 and u 6. In geneal, it is desiable to find two diffeent kinds of paths in the component flow gaph. One is a def-use path fom the definition of a vaiable to a use that includes as many othe uses as possible. Anothe is an ext-int path fom an extenal tansition that can be executed by a teste, which in tun foces execution of othe tansitions that need to be executed as pat of a def-use path. Both types of paths can be seen in Figue 2. The path T 1 : S b : T wb : G 2 : T 2 : S e : G 3 : T 3 is of the def-use type and paths T x : S a : G 1 : T 1 and T y : S c : G 2 : T 2 ae of ext-int type. In the def-use path, execution of tansition T 1 defines v and moves obj1 to the est state S b. Then execution of tansition T 2 gets the value of v fom obj1 and foces tavesal of the emainde of the path. If the methods of T 1 and T 2 ae intenal functions that cannot be executed diectly by a teste, then one must be able to find extenal functions that can be executed by a teste in a sequence that foces tavesal of the def-use path.

4 S 2 F 2 T 5 est S y F 5 T y g S 3 def est S c v () T 1 S b T vb g G 2 T 2 F 1 T 4 h S e h G 3 use T 3 est G 1 f S a f T x est S F x 4 T 6 F 3 S1 Figue 3: Extenal Function Calls to Tavese a Def-Use Path Constuction of def-use paths must satisfy the following popeties: Paths must espect the ode of execution of statements in the action of a tansition Function labels on adjacent tansition-to-state-to-guad edges must be identical Guad pedicates must be feasible with espect to passed paametes and cuent state Paths that exit and then e-ente a class must satisfy a state compatibility ule to ensue that the state afte exit is identical to the state of e-enty The state compatibility ule equies that if a path leaves a tansition node T 1 fom class A to go to a node deived fom some othe class, and if the path late etuns to the same o anothe tansition node T 2 of class A, then the taget state of T 1 must be equal to the souce state of T 2. This equiement ensues that the action that causes class A to change state is captued as pat of the path. Constuction of ext-int paths must satisfy all of the popeties of def-use paths, but in addition must not have any est states that would equie additional use actions to tavese the path. This ensues that execution of the identified extenal tansition tigges successive actions that esult in execution of the identified intenal tansition. Ou pevious pape [6] includes an algoithm that allows constuction of both types of paths. Given the set of all def-use pais in a component flow gaph it is possible to patition that set as follows: Pais fo which a def-use path can be constucted Pais in which the use is povably not eachable fom the def Pais that emain unesolved and fo which it is still unknown whethe thee exists a feasible, def-fee, path fom the def to the use The geneated def-use paths ae test specifications fo the all-uses citeion ove the component flow gaph. Using the ext-int paths, it is then possible to geneate executable test cases that esult in tavesal of the test specifications. Coveage is detemined by the numbe of def-use pais that can be esolved by a def-fee tavesal of a test specification fom the identified def to its use. 4. TEST SPECIFICATION COVERAGE Given test specifications fo the all-uses citeion ove a component flow gaph, it is desiable to constuct a sequence of extenally executable functions that when executed will cause tavesal of as many of the undelying def-use paths as possible. A teste begins with each object of the softwae system in some initial state. The initial states detemine which extenal tansitions may be executed by calling vaious extenal functions. As an illustation of the methodology, conside the thee objects that detemined the potion of the component flow gaph in Figue 2. To develop a test sequence fo the def-use path T 1 : S b : T vb : G 2 : T 2 : S e : G 3 : T 3, a teste begins with objects 1, 2 and 3 in some initial states S 1, S 2 and S 3. The goal is to execute a sequence of extenal functions {F i i = 1, 2,...} that will popely place each object into states that suppot execution of the ext-int paths T x : S a : G 1 : T 1 and T y : S c : G 2 : T 2. The desied sequence of actions and effects can be seen in Figue 3. Figue 3 is called a feathe gaph, because thee is a pimay path (fom the def at T 1 to the use at T 3 ), and extenal paths ae needed to put the objects epesented by the pimay path into the pope states. These extenal paths feathe in to the pimay path and ae essential to ensue contollability [1] of the system unde test. Fist, if obj3 is not aleady in state S e, some extenal function F 1 must be executed to move it fom initial state S 3 to state S e though some tansition T 4. Next, if obj2 is not aleady in the souce state S y fo tansition T y, some extenal function F 2 must be executed to move it fom initial state S 2 to state S y though some tansition T 5. This must be done while keeping obj3 in state S e. Next, if obj1 is not aleady in the souce state S x fo tansition T x, some extenal function F 3 must be executed to move obj1 fom its initial state S 1 to state S x though some tansition T 6. Then extenal function F 4 can be executed while obj1 is in state S x to invoke tansition T x, which in tun calls function f, which esults in tansition T 1, which defines vaiable v and puts obj1 in state S b. State S b is defined to be a est state fo the given def-use path; anothe extenal method must be invoked by the teste to foce continued tavesal of the path. Rest states fo this and othe paths ae labeled in the figue. Next, extenal function F 5 can be executed while obj2 is in state S y to invoke tansition T y, which in tun calls function g that esults in tansition T 2. The action of tansition

5 T 2 gets the value of v fom obj1 and then sends a message to obj3 that calls function h and completes tavesal of the desied def-use path fom T 1 to T 3. Note that the def-use path T 1 : S b : T vb : G 2 : T 2 : S e : G 3 : T 3 has only one est state S b ; the othe state node in this path, S e, is a est state fo a diffeent path, i.e. S 3 : T 4 : S e. The ecognition of est states fo a given path in the component flow gaph is an impotant pat of test case development. The geneated test sequence is F 1, F 2, F 3, F 4, F 5. Cae must be taken to choose input paametes caefully to ensue that guad pedicates will be satisfied and that the desied tansitions will be executed. We have developed tools to automate the constuction of test sequences. These tools ae descibed futhe in the following sections. 5. TEST SEQUENCE GENERATION If an object-oiented softwae system is epesented by a combined class state machine, and if a specific component of that system has been identified fo integation testing, then the pocesses descibed in Sections 2 and 3 can be used to constuct a component flow gaph, a set of def-use candidate test paths, and a set of ext-int tansition tiggeing paths. Section 4 gives an example of constucting a single sequence of extenal function invocations to foce the tavesal of a single def-use path. This section descibes a moe geneal appoach fo constucting test sequences that foce coveage of as many def-use paths as possible, theeby detemining coveage esults fo integation testing of the identified component with the emainde of the softwae system. Let CTP be the set of all def-use paths and EXT the set of all ext-int paths geneated fom a component flow gaph. CTP is the set of all candidate test paths that detemine the test specifications fo integation testing of the selected component. Let F be the set of all extenal functions defined in the combined class state specification that could be called in black box testing. The goal of this eseach is to identify a sequence of functions F i fom F that when executed in the identified ode will cove as many paths in CTP as possible. Given a softwae system consisting of multiple objects, its system state at a given point in time is defined to be the state of each object at that time. Suppose a pogam is in some initial system state SS 0 and suppose an extenal function F 1 is executed with some choice of values fo any of its input paametes. Depending on the object states identified by SS 0, and depending on the values chosen fo any input paametes, tavesal of a subset of paths in EXT will be initiated. Since none of the paths in EXT have any est states, each path will be tavesed to its ending tansition node. But, by constuction of such paths, this node may be the head definition node of a numbe of def-use paths in CTP, each of which will be initiated and tavesed up to its fist est state. If a path in CTP has no est states, then the entie def-use path will be completed and one can conclude that execution of F 1 coves all such paths. At the completion of all tiggeed tansitions identified by these paths, the softwae system will be in a new system state SS 1. The teste could then execute a second extenal function F 2 to initiate anothe subset of ext-int paths fom EXT. Some of these paths may then, in tun, initiate a set of new def-use paths fom CTP. In addition, evey execution of a tansition in the middle of an ext-int path, o in the middle of a leg between est states of a def-use path, may call functions that tigge the next tansition in a peviously initiated def-use Extenal Functions F F 1 F 2 F n System States SS SS 0 SS 1 SS n-1 ext-int Paths EXT t t t def-use Paths CTP Figue 4: Geneating an Extenal Test Sequence path that is cuently in a est state. Figue 4 pesents the possibilities of choosing successive functions fom F to initiate new EXT paths, which in tun initiate new CTP paths o foce tavesal along existing CTP paths to the next est state. Rest state nodes in a def-use path ae labeled, and tansition nodes that foce tavesal fom a est state to a new leg in a def-use path ae labeled t. An extenal function F i may initiate multiple paths in EXT and each EXT path may initiate o extend multiple paths in CTP. Many of these paths will be mutually inconsistent. Some will have contadictoy guad pedicates, some will violate the state compatibility ule fo paths, and some will e-define a vaiable that has aleady been defined by a definition node at the head of a def-use path. It will be necessay to analyze, vey caefully, all initiated paths one tansition at a time. This analysis will emove fom futhe consideation at this step in test sequence geneation paths that poduce inconsistencies in the paallel tavesal of all initiated paths. At each step in this pocess, the following actions must be taken: Identify one guad pedicate to be satisfied when a choice of guad nodes appea in simultaneous paths Delete all paths with guad nodes fo the same function that diffe fom the selected guad Identify and delete any combined ext-int-def-use paths that violate the state compatibility ule Delete any paths cuently in a est state whose next action is inconsistent with any tiggeed tansition up to this point in the analysis Identify and delete any combined ext-int-def-use paths that ae cuently at a est state, but fo which the tiggeed actions of the cuent extenal function esult in e-definition of the def vaiable Deleted def-use paths ae not lost foeve. They may eappea late afte execution of a subsequent extenal function,

6 possibly with diffeent input paamete values, and with selection of new EXT paths that may e-initiate the desied path. This time, subsequent actions may cause complete tavesal to the use node of this def-use path. We have developed a tool that helps a human teste make the above choices in a way that allows maximal coveage of untavesed def-use paths. The human teste meely esponds to questions about which guad pedicates ae to be satisfied at each step of the pocess; the tool pesents only feasible choices and handles all othe aspects of the pocess. Conside the set of ext-int-def-use paths that have been initiated by execution of pevious functions in a test sequence and emain undeleted and uncompleted afte the tiggeed effects of function F i. Call this set ATP fo active test paths. The algoithm fo test sequence geneation consides evey extenal function that could be executed duing the cuent system state, SS i, and counts the numbe of feasible, untested, paths in EXT x CTP that might be initiated, and the numbe of paths in EXT x ATP that might be pogessed along the next leg. With knowledge of these counts, a human teste can choose the next extenal function F i+1 that has the best possibility of maximizing these counts towad coveing the most emaining untavesed def-use paths. Futhe eseach may help to automate this existing human ole in test sequence geneation. Afte completion of the analysis of the tiggeed effects of each extenal function call F i, the tool ecods two pieces of infomation: The set of def-use paths coveed by F i The expected system state SS i, afte stepping though all intemediate tansitions in the identified paths. The constuction of test sequences continues until as many def-use paths as possible have been completely tavesed fom def to use. In some cases this may involve constuction of multiple test sequences, each stating with a diffeent initial system state. In othe cases, a single test sequence may suffice to cove all def-use paths in CTP. In eithe case the set of executable test sequences geneated by this pocess detemines a test suite fo integation testing of the test component with the emainde of the system. When an implementation claiming to implement the specification is tested, each test sequence in a test suite is executed against the implementation. Afte the tiggeed effects of each F i, the implementation should be in the system state pedicted by SS i. If an implementation fails to be in the pedicted system state afte each F i, it fails the test sequence. An implementation passes the test suite if each test sequence in the test suite is completed with no test sequence failues. 6. TEST ARCHITECTURE Conside the testing achitectue pesented in Figue 5. The combined class state machine of the softwae system specification is epesented in a database of six elational tables. The tables epesent the Classes, Vaiables, Functions, Paametes, States, and Tansitions of the combined class state machine. Upon choosing a component of the system fo integation testing, the Test Sequence Geneato constucts the component flow gaph and follows the pocesses descibed in Sections 2 though 5 to geneate one o moe sequences {(F i, SS i ) i = 1,..., n} of executable extenal functions and pedicted system states, both epe- DB Rep of Spec Test Suite Geneato Java Rapid Pototype Machine Executable Test Sequences Java Imp of Spec Figue 5: Test Tool Achitectue Test Haness Gaphical Use Inteaface sented by Executable Test Sequences in the figue. The Test Haness impots an executable test sequence with pedicted system states, executes each test against a claimed implementation of the specification, and detemines if the implementation passes o fails the test sequence by checking the actual system states against the pedicted states. The Java Rapid Pototype Machine eads a database epesentation of a finite-state specification, and poduces a geneic test simulato witten in Java. The machine consists of a simple kenel that is able to wait fo, queue, and pocess input tasks fom eithe a use o fom the Test Haness. An input task is codified as an instance of a Java wappe class that stoes data fields taditionally associated with objectoiented pogamming, such as an objects identity, state, and behavio, applied to that object (a function). The object in question is an instance of a class defined by the tables of the database epesentation. An intepete evaluates an input task and queies the database epesentation to simulate the actions of each defined tansition. The esulting Java efeence implementation povides an optional Gaphical Use Inteface fo test wites to add visual components fo simulation puposes. The Test Haness is designed to suppot testes who want to un a sequence of test cases unde the efeence implementation, o against a eal implementation inseted into the testing achitectue in place of the efeence implementation. 7. CONCLUSIONS This pape has pesented technical details about an automated tool to suppot integation testing of object-oiented softwae. The assumed test scenaio is that a new o modified collection of classes (a component) is being integated into an existing collection of classes (such as a component o system). The classes ae epesented as inteacting finite state machines, which model the infomation that is nomally included in design models such as UML statechats, and augmented with details about definitions and uses of class vaiables. This augmentation is cuently done by hand, but this infomation could be obtained by a detailed analysis of the implementation. The combined class state machine and component flow gaph ae new to this eseach, and this pape descibes in detail how the behavio of OO softwae is epesented in

7 them and details fo how they ae constucted. The test citeia used ae based on taditional data flow citeia, but applying them to these types of models is new and intoduces numeous complexities. This pape descibes how this model woks with specific examples. The use of a database to stoe definition/use infomation simplifies the constuction of full def-use paths fom definitions to uses. Stoing and manipulating complete path pedicates fo taditional code-based data flow is impactical due to size. The database allows these potentially lage pedicates to be managed efficiently. As with any automated test systems, undecidable poblems pohibit complete solutions. In this eseach, some candidate test paths cannot be esolved into executable test cases. This sometimes happens because esolving the test paths is too complicated, and sometimes because they epesent tuly undecidable potions of the poblem space. This poblem is common to all automatic test data geneation techniques. One advantage of this wok is that potential concuent actions among objects is fully captued in the component flow gaph and in the geneation of candidate test paths though that gaph. The model assumes that eceipt of asynchonous messages is handled by queuing and that any one of the eceived messages could be the next to execute. This ensues that all possible executions ae consideed by the candidate test paths. If cetain concuent executions ae not feasible, that infomation will be detected duing testing and the infeasible path segments can be emoved in subsequent test path geneation. Moe details of the concuency and asynchonous aspects of this wok ae in the pevious pape [6]. Futue eseach will focus on additional automation of existing manual steps in this methodology. One cuent diection is to povide automation fo tanslation of softwae specifications into the database epesentation. If the specifications ae in a state-machine epesentation (such as UML statechats), then much of the metadata can be captued automatically. Howeve, UML statechats do not specify the details of the tansition actions. Indeed, this infomation is seldom included in design models, so this may still need to be supplied by the human. Anothe possibility is to extact this infomation fom the implementation, if one exists. Anothe taget of moe automation is the identification of infeasible path segments in the component flow gaph, which should be avoided in def-use and ext-int path constuction. Additional teste suppot fo choosing the best extenal method to call in test sequence constuction is also desiable. 8. ACKNOWLEDGMENTS We would like to thank Tony Cincotta fo implementing the apid pototype machine and the test haness, and Julie Zanon, Hankim Ngo and Bois Etho-Assoumou fo diffeent featues of a web application that implements the algoithms of the test suite geneato. 9. REFERENCES [1] B. Binde. Testing Object-oiented Systems. Addison-Wesley Publishing Company Inc., New Yok, New Yok, [2] G. Booch. Object-Oiented Design With Applications. Benjamin-Cummings Publishing Co. Inc., Reading, MA, [3] M.-H. Chen and M.-H. Kao. Testing object-oiented pogams - an integated appoach. In Poceedings of the 10th Intenational Symposium on Softwae Reliability Engineeing, pages 73 83, Boca Raton, FL, Novembe IEEE Compute Society Pess. [4] T. Chow. Testing softwae designs modeled by finite-state machines. IEEE Tansactions on Softwae Engineeing, SE-4(3): , May [5] P. G. Fankl and E. J. Weyuke. An applicable family of data flow testing citeia. IEEE Tansactions on Softwae Engineeing, 14(10): , Octobe [6] L. Gallaghe and A. J. Offutt. Integation testing of object-oiented components using finite state machines. The Jounal of Softwae Testing, Veification, and Reliability. In pess - published online Januay [7] P. Heman. A data flow analysis appoach to pogam testing. Austalian Compute Jounal, 8(3):92 96, Novembe [8] H. W. Hong, Y. R. Kwon, and S. D. Cha. Testing of object-oiented pogams based on finite state machines. In Poceedings of the Asia-Pacific Softwae Engineeing Confeence, pages , Bisbane, Austalia, [9] J. Laski. On data flow guided pogam testing. Sigplan Notices, 17(9), Septembe [10] S. Rapps and E. J. Weyuke. Data flow analysis techniques fo test data selection. In Softwae Engineeing 6th Intenational Confeence. IEEE Compute Society Pess, [11] J. Rumbaugh, M. Blaha, W. Pemelani, F. Eddy, and W. Loensen. Object Oiented Modeling and Design. Pentice Hall, [12] C. D. Tune and D. J. Robson. The state-based testing of object-oiented pogams. In Poceedings of the 1993 IEEE Confeence on Softwae Maintenance (CSM-93), pages , Monteal, Quebec, Canada, Septembe [13] H. Zhu, P. A. V. Hall, and J. H. R. May. Softwae unit test coveage and adequacy. ACM Computing Suveys, 29(4): , Decembe 1997.