An Approach to a Test Oracle for XML Query Testing

Transcription

1 An Approach to a Tet Oracle for XML Query Teting Dae S. Kim-Park, Claudio de la Riva, Javier Tuya Univerity of Oviedo Computing Department Campu of Vieque, /n, (SPAIN) kim_park@li.uniovi.e, claudio@uniovi.e, tuya@uniovi.e Abtract. XML querie are broadly ued in Web environment, but the exiting approache toward oftware quality baed on teting have not deeply addreed them. Although there are ome work oriented to generate tet input for teting XML querie, the evaluation of expected output againt the actual output reulting from the tet ha not been tackled a far a we are concerned. In thi paper, a reearch propoal i preented to deal with the abence of the expected output when teting XML querie, focuing the effort on the definition of a feaible tet oracle. 1 Introduction and motivation In recent year, Web Engineering ha emerged a a new dicipline motivated by the wift growth of the World Wide Web. Thi growth ha been accompanied by an increaing complexity of Web application (WebApp) a we can clearly ee, for example, in today WebApp formed by compoition of Web Service created, in turn, with heterogeneou fat-evolving technologie. The development of thee complex WebApp, like other oftware ytem, hould involve quality criteria to guarantee a certain degree of reliability baed on teting, verification and validation activitie. Regarding teting, current approache on WebApp are commonly centered in checking tatic and dynamic navigational path [9][15][11]. However, much functionality of WebApp depend on data acce operation, for example, to retrieve and manipulate data requeted by the uer or by a oftware component in execution. For thi reaon, it i expected that teting data acce operation may have ome impact on the improvement of the quality of WebApp. There are well-known technologie intended to carry out data acce operation, uch a the SQL language for relational databae, but XML-baed technologie for data querying are becoming popular a i the ue of XML-baed format for data repreentation and interchange in the Web. The exiting approache and tool to tet data acce operation on XML-baed application are commonly conceived to tet data repoitorie or query engine [2][1], while the querie utilized to retrieve the data (XML querie) are generally ignored, although being prone to fault [7]. In previou work [7][4][5], we entailed thi concern by defining a technique to automatically generate the input for teting XML querie. However, the expected output for the

2 2 Dae S. Kim-Park, Claudio de la Riva, Javier Tuya tet were dimied depite they are a key factor to determine the correctne of tet execution. To addre the lack of the expected output, ome reearcher tate that there hould be an entity, called oracle, capable to determine whether or not a program under tet ha behaved a expected during execution. The definition of uch oracle derive in the o called oracle problem, a oracle are uually difficult or even impoible to obtain [8]. In order to overcome thi problem, it i aumed that a human can act a an oracle by manually checking the correct behaviour of the program under tet, but thi approach may be unfeaible in ome circumtance. Particularly, a human oracle for XML query teting raie ome problem becaue (1) the tet input may be large, (2) there may be many querie to tet, and (3) there may be numerou tet cae to conider. In thi reearch propoal we preent a line of work intended to tackle ome of the problem concerning teting of querie for data acce in XML-enabled environment (uch a the Web), with a focu on the definition of a tet oracle oriented to olve the abence of the expected output during teting. The paper i tructured a follow. In ection 2 the aim and objective of the reearch are outlined. Section 3 briefly decribe the current work and, finally, in ection 4 the expected contribution of the reearch and plan for future work are preented. 2 Reearch hypothee and objective The current reearch activity in the field of tet oracle i not ignificant in contrat to the huge variety of the exiting approache involving tet input generation. Amongt the work encompaing tet oracle, partial oracle eem to be the bet alternative to XML query teting. Such partial oracle are capable to determine whether an actual output of a tet cae i incorrect while ignoring the correct output [8]. Thu, they do not need to infer the expected output of a tet in order to provide a diagnotic about the correctne of the target program. In XML query teting, thi i a remarkable advantage conidering the large volume of XML data that may be provided a tet input and/or output. With thee conideration, our reearch hypothei i intended to prove that a partial oracle i a viable alternative when dealing with large volume of data for teting. In particular, we are aiming to prove that a partial oracle may be uitable to tet data acce operation baed on querie in XML-enabled environment. To cover the hypothei, the following objective are in cope: 1. Define a feaible partial oracle to tet XML querie. The automation of the oracle will be one of the main point to conider. 2. Validate the propoal to prove it effectivene in a real teting cenario, which may include developing a proof of the concept to how the practical application of the propoal.

3 An Approach to a Tet Oracle for XML Query Teting 3 3. Find poible alternative to the propoed oracle if required. The propoed oracle may not be uitable in every cae. Becaue of that, other alternative hould be tudied, pecially the exiting one that could be adapted to XML query teting, uch a metamorphic teting [3]. 3 Propoed approach The complexity of the XML querie i a concern ince there are many broadly ued alternative oriented to query XML data. XML querie are commonly expreed uing the XPath [12] and XQuery [13] language, but alo other programming technologie can be ued to cover the imilar functionalitie, uch a the Java language upported by the SAX/DOM API [10][14]. To overcome thi heterogeneity, we abtracted the repreentation of the querie under tet by treating them a black-box program, each of which receive an XML document a input, and output an XML document fragment (a et of XML tructure not necearily well-formed) reulting from the query operation. Formally, let q : D F be the query under tet, where D i the et of all poible input (XML document), and F i the et of all poible output (XML document fragment). Then, q i a black-box query program under tet that take an XML document I D and produce the actual output q( I ) F. With thi abtraction, we propoe the ue of a partial oracle to tet the query program according to a pecification compoed by the following two element: Behavioural requirement, which etablih a pecification about the correct behaviour of a determined target query program under tet. Thi pecification i looe, which mean that it need not be complete, and Oracle contraint, which define invariant propertie of the expected output that mut be atified by every correct query program. Each oracle contraint i intended to check the preence a known type of fault. Since behavioural requirement depend on the target program, the teter mut provide them manually. In contrat, oracle contraint are invariant; thu, they could be embedded in the partial oracle internal and their evaluation could be automated. Fig. 1 how the tructure of the oracle and it integration in the teting environment. A it can be een, the oracle receive a input the behavioural requirement, a well a the tet input and the actual output of the tet, and yield a Pa or Fail repone a a reult of the oracle contraint evaluation. Becaue oracle contraint pecify general propertie of the expected output, prior to their evaluation, they are particularized to the target program under tet by mean of the behavioural requirement. Whenever an oracle contraint i not atified given a et of behavioural requirement, it i undertood that the partial oracle ha detected a fault, in which cae, the partial oracle hould repond with a Fail meage.

4 4 Dae S. Kim-Park, Claudio de la Riva, Javier Tuya Fig. 1. Partial tet oracle in the teting environment. In the paper in [6] we detail the partial oracle approach with a et of behavioural requirement and oracle contraint, including a cae tudy. Some of the propoed behavioural requirement and oracle contraint are preented in the next ection. 3.1 Behavioural requirement In order to eae the manual pecification of the query program under tet, behavioural requirement are looe, which mean that they do not comprie a complete pecification of the target program. However, the preciion of the partial oracle will increae a the teter pecifie thee requirement cloer to the complete pecification. Thi provide flexibility to balance the relationhip between the cot and the preciion of the oracle. The behavioural requirement are given by the tuple S = < A, D, t >, where: A i a et of XML node (XML element or attribute) that are expected to be in the actual output of the target program, but are not included in the tet input, D i a et of XML node that are expected to be in the input, but not in the actual output. t i a relaxed pecification-compliant query. It i defined a a function with the ame domain and image a the query under tet q. Then, the property q ( I ) t ( I) mut hold for the definition of t, where q denote the correct implementation of the query that meet the pecification, alo in the domain and image of q, and q (I) i the expected output for the tet input I. Note that q it i not available for the teter. In practice, t i expected to be a le expreive query than q, and hence, it hould be le error-prone and eaier to pecify. For example, uppoe that we need to obtain the title of an XHTML Web page only if it body contain the tring value "Content". The pecification-compliant query, q would be repreented with the XPath yntax a /html/head/title[../body = "Content"]. Then, t could be

5 An Approach to a Tet Oracle for XML Query Teting 5 defined a the query /html/head/title, which alway return the title of the page, independently of the body content. A een, t hould be impler and eaier to code than q. The element of the behavioural requirement tuple need not be defined a a complete pecification. It i not required to pecify every poible node in A or D, and the query t doe not need to be a expreive a q in order to detect ome type of error. In the next ection, the ue of thee behavioural requirement i hown. 3.2 Oracle contraint Oracle contraint et neceary condition for the correct behaviour of the querie under tet. Since thee contraint are intended to deal with operational apect of the querying procee, they are domain-independent, and thu, can be embedded in the oracle (a hown in Fig. 1). Some of the oracle contraint we currently propoe are preented and decribed below. q ( I )I A = A. Thi contraint etablihe that the XML node pecified in the et A mut be contained in the actual output, q (I ). q( I )I D = Ø. The contraint etablihe that the actual output mut not contain node from D. q( I ) t ( I). It check that the actual output i contained in the relaxed pecification-compliant query. If thi contraint i atified, it mean that data election fault (uch a mitaken predicate or bad node reference) were not detected in the query under tet, q. Note that each contraint make ue of a different behavioural requirement from the tuple in Section 3.1, and every behavioural requirement could be defined with a cutom preciion by the teter. After a tet execution, if any of the contraint doe not hold for a given query program, then a fault ha been detected, and the oracle could notify the caue of that fault with a human-readable meage for the teter. 4 Expected contribution and plan for future work The expected contribution of the reearch to Web Engineering conit on improving the quality of WebApp by mean of teting XML querie. For thi purpoe, we focu on an approach to a tet oracle whoe main characteritic are outlined below: 1. Allow the teter to balance the cot and the effectivene of the oracle. The teter can provide a looe pecification (behavioural requirement) of the program under tet, intead of giving a complete pecification, which would be more cotly and complex. 2. The propoed oracle i highly automatable. The mechanim of the oracle depend on the rule defined by oracle contraint whoe evaluation could be automated.

6 6 Dae S. Kim-Park, Claudio de la Riva, Javier Tuya At thi moment, the propoed oracle can detect a relatively mall et of fault. For future work, we plan to add/modify behavioural requirement and oracle contraint (a hown in the example in Section 3) to enhance the fault detection capabilitie of the oracle. It i alo neceary to define a ytematic method for the evaluation of oracle contraint a thi will reult in the automation of the oracle. Beide, the oracle propoal hould be validated to prove it effectivene. We are planning to do the validation by uing experimental technique uch a mutation, and by providing real/indutrial cae tudie. 5 Acknowledgement Thi work wa partially funded by the Department of Education and Science (Spain) and ERDF fund within the National Program for Reearch, Development and Innovation, project Tet4SOA (TIN C06-01) and the RePRIS Software Teting Network (TIN E). 6 Reference [1] D. Barboa, A.O. Mendelzon, Declarative Generation of Synthetic XML Data, Software Practice and Experience, vol 36, pp , [2] A. Bertolino, J. Gao, J. Marchetti, A. Polini, Automatic Tet Data Generation for XML Schema-baed Partition Teting, Proceeding of Automation of Software Tet, pp. 4-11, [3] T.Y. Chen, F.-C. Kuo, T.H. Te, Z.Q. Zhou, Metamorphic Teting and Beyond, Proceeding of the Eleventh Annual International Workhop on Software Technology and Engineering Practice, 00: 94:100, [4] D.S. Kim-Park, C. de la Riva, J. Tuya, J. García-Fanjul, Generating Input Document for Teting XML Querie with ToXgene, Proc. of the 3rd IEEE Teting: Academic and Indutrial Conference, Fat Abtract Track, [5] D.S. Kim-Park, C. de la Riva, J. Tuya, J. García-Fanjul, Synthetic Data Generation for XML Query-aware Teting, 6 th Workhop on Sytem Teting and Validation, [6] D.S. Kim-Park, C. de la Riva, J. Tuya, A Partial Tet Oracle for XML Teting, Teting: Academic and Indutrial Conference Practice and Reearch Technique, [7] C. de la Riva, J. García-Fanjul, J. Tuya, A Partition-Baed Approach for XPath Teting, Proceeding of the International Conference on Software Engineering Advance, Wahington, DC, USA, [8] E.J. Weyuker, On Teting Non-tetable Program, The Computer Journal, 25(4): , [9] F. Ricca, P. Tonella, Analyi and Teting of Web Application, Proceeding of the 23rd International Conference on Software Engineering, [10] SAX Project, [11] S. Elbaum, S. Karre, G. Rothermel, Improving web application teting with uer eion data, Proceeding of the 25th International Conference on Software Engineering, [12] World Wide Web Conortium, XML path language 2.0 (XPath 2.0),

7 An Approach to a Tet Oracle for XML Query Teting 7 [13] World Wide Web Conortium, XQuery 1.0. An XML query language, [14] World Wide Web Conortium, Document Object Model (DOM), [15] Y. Deng, P Frankl, J. Wang, Teting web databae application, SIGSOFT Software Engineering Note, 29(5), 2004.