Evaluation of Different Fitness Functions for the Evolutionary Testing of an Autonomous Parking System

Transcription

1 Evaluatio of Differet Fitess Fuctios for the Evolutioary Testig of a Autoomous Parkig System Joachim Wegeer 1 ad Oliver Bühler 2 1 DaimlerChrysler AG, Research ad Techology, Alt-Moabit 96 a, D Berli, Germay Joachim.Wegeer@DaimlerChrysler.com 2 STZ Softwaretechik, Im Gaugemaier 20, D Esslige-Zell, Germay Oliver.Buehler@stz-softwaretechik.de Abstract. The method of evolutioary fuctioal testig allows for the automatio of testig by trasformig test case desig ito a optimizatio problem. To this ed it is ecessary to defie a suitable fitess fuctio. I this paper two differet fitess fuctios are compared for the testig of a autoomous parkig system. The autoomous parkig system is executed with the test scearios geerated, the fitess for each test sceario is calculated o the basis of a evaluatio of the quality of the parkig maeuver calculated by the autoomous parkig system. A umerical aalysis shows, that the proposed area criterio supports a faster covergece of the optimizatio compared to the proposed distace criterio ad that the proposed area criterio describes a efficiet method for fidig fuctioal errors i the system i a automated way. 1 Itroductio Electroic cotrol uits (ECUs) i cars are takig over icreasigly complex tasks. New applicatios such as autoomous parkig systems, itelliget cruise cotrol systems, which track the distace to precedig vehicles, or emergecy breakig systems rely o the sesor-based calculatio of distaces to other objects. For such applicatios, errors i the ECU's software ca result i high costs. Therefore, the aim is to fid as may errors as possible by testig the systems before they are released. I practice, dyamic testig is the aalytical quality assurace method most commoly used. Usually, a complete test is ifeasible because of the huge umber of possible iput situatios. Therefore, test cases have to be selected accordig to test hypotheses, e.g. each requiremet should be tested at least oce or every program brach should be executed durig the test. I most cases, test case desig is performed maually, requirig a cosiderable part of the project s resources. The evolutioary fuctioal testig method facilitates the geeratio of test cases i order to detect fuctioal errors durig a directed search. The method trasforms the test case desig process ito a optimizatio problem. Automated test result evaluatio is a prerequisite for this process. The evaluatio is carried out by meas of the fitess fuctio which assigs a umerical quality value to a test result. K. Deb et al. (Eds.): GECCO 2004, LNCS 3103, pp , Spriger-Verlag Berli Heidelberg 2004

2 Evaluatio of Differet Fitess Fuctios for the Evolutioary Testig 1401 This paper evaluates two differet approaches to the defiitio of fitess fuctios for the fuctioal testig of a autoomous parkig system. The fitess fuctios defied represet a quality metric ad ca automatically evaluate a parkig maeuver calculated by the parkig system, i.e. they retur a umerical value which describes the quality of the parkig maeuver. Both approaches are compared by meas of umerical experimets for a prototype implemetatio of a autoomous parkig system. The results show that of both the criteria proposed i this paper, the area criterio ca idetify critical parkig maeuvers better tha the distace criterio itroduced i [1]. The area criterio provides a more efficiet method of error detectio i the parkig system. The structure of the paper is as follows: after a brief itroductio toevolutioary testig i the secod sectio, the autoomous parkig system is itroduced ad the applicatio of evolutioary testig to its fuctioal testig is explaied i the third sectio. Sectio 4 describes the differet fitess fuctios, whereas sectio 5 shows the experimetal results achieved by applyig these fitess fuctios. The paper closes with a short summary i the sixth sectio. 2 Evolutioary Testig Testig aims to fid errors i the system uder test ad create cofidece i its correct behavior by executig the system with selected iput situatios. Of all the test activities, test case desig is assiged decisive importace. Test case desig determies the type, scope ad thus the quality of the test [2]. If relevat test cases are forgotte, the probability of detectig errors i the system drops. Due to the cetral importace of test case desig, a umber of testig methods have bee developed to help the tester with the selectio of appropriate test data. Oe importat weakess of the testig methods available is that they caot be automated i a straightforward way. Maual test case desig, however, is time-itesive ad error-proe. The test quality depeds o the performace of the tester. I order to icrease the effectiveess ad efficiecy of the test ad thus to reduce the overall developmet ad maiteace costs for systems, a test should be systematic ad extesively automatable. Both these objectives are addressed by the evolutioary testig method [3]. I order to trasform a test aim ito a optimizatio task a umerical represetatio of the test aim is ecessary, from which a suitable fitess fuctio for the evaluatio of the test data geerated ca be derived. Depedig o which test aim is pursued, differet fitess fuctios emerge for test data evaluatio. If, for example, the temporal behavior of a applicatio is beig tested, the fitess evaluatio of the idividuals created by evolutioary testig will be based o the executio times measured for the test data [3]. For safety tests, the fitess values are derived from pread post-coditios of modules [4], ad for robustess tests o fault-tolerace mechaisms, the umber of cotrolled errors ca form the startig poit for the fitess evaluatio [5]. Applicatios of evolutioary testig to structural testig result i differet fitess fuctios agai [6], [7], [8], [9]. A overview of the differet applicatios of evolutioary testig ca be foud i [10].

3 1402 J. Wegeer ad O. Bühler If a appropriate fitess fuctio ca be defied, the evolutioary test proceeds as follows. The iitial populatio of test data is usually geerated at radom. Each idividual withi the populatio represets a test datum with which the system uder test is executed. For each test datum the executio is moitored ad the fitess value is determied with respect to the test aim defied. Next, populatio members are selected with regard to their fitess ad subjected to combiatio ad mutatio processes to geerate ew offsprig. These are the also evaluated by executig the system uder test with the correspodig test data. A ew populatio is formed by combiig offsprig ad paret idividuals, accordig to the survival procedures laid dow. From here o, the process repeats itself, startig with selectio, util the test objective has bee fulfilled or aother give stoppig coditio has bee reached. 3 The Evolutioary Testig of the Autoomous Parkig System As a automobile maufacturer, DaimlerChrysler is cotiuously developig ew systems i order toimprove vehicle safety, quality, ad comfort. Withi this cotext, prototypical vehicle systems have bee developed which support autoomous vehicle parkig a fuctio that is likely to be itroduced oto the market i some years time. I order to describe the applicatio of evolutioary testig for the fuctioal testig of autoomous parkig systems, we shall first provide a brief descriptio of the fuctioality of a autoomous parkig system. We shall the explai the applicatio of the evolutioary test to the parkig system. The test eviromet developed for testig the parkig systems ad the test data geerator are also described. 3.1 The Autoomous Parkig System The autoomous parkig systems dealt with i this paper are iteded to automate parkig legthways ito a parkig space, as show i Fig.1. For this purpose, the vehicle is equipped with evirometal sesors which register objects surroudig the vehicle. Whe drivig alog, the system ca recogize sufficietly large parkig spaces ad ca sigal to the driver that a parkig space has bee foud. If the driver decides to park i the parkig space detected, the vehicle ca do this automatically. Fig.2 shows the system eviromet ad the iteral structure of the autoomous parkig system. The iputs are sesor data, cotaiig iformatio o the state of the vehicle, e.g. vehicle speed or steerig positio, ad iformatio from the evirometal sesors which register objects o the left ad right had side of the vehicle. As regards output, the system possesses a iterface to the vehicle actors, where the vehicle's velocity ad steerig agle are set. The parkig space detectio compoet processes the data obtaied from the evirometal sesor systems ad delivers the recogized geometry of a parkig space if it has bee idetified as sufficietly large. The parkig cotroller compoet

4 Evaluatio of Differet Fitess Fuctios for the Evolutioary Testig 1403 Fig. 1. Fuctioality of a Autoomous Parkig System uses the geometry data of the parkig space together with the data from the vehicle sesors to steer the vehicle through the parkig procedure. For this purpose, velocity ad steerig agle are set for the vehicle actors. The parkig cotroller has to guaratee that the collisio area is ot etered by the vehicle i order toavoid a high probability of causig damage to adjacet parked vehicles or objects ad the vehicle itself. Autoomous Parkig Cotroller vehicle sesors evirometal sesors vehicle data evirometal data Parkig Space Detectio parkig space geometry Parkig Cotroller actordata vehicle actors Fig. 2. System Eviromet ad Sub-Compoets of the Autoomous Parkig System 3.2 Applyig Evolutioary Testig to the Autoomous Parkig System Comprehesive ad efficiet testig is essetial before releasig a system such as the automatic parkig system. As may tests as possible must be performed i a systematic ad efficiet way. Maual testig of the complete system is cost-itesive ad time-cosumig because every test case ivolves settig up a parkig sceario with real cars ad the maual drivig of each maeuver. Furthermore, it is difficult to achieve a exact reproductio of the tests because the details of the test executio vary. I cotrast, automated tests have the potetial to perform a large umber of test cases with less effort i a reproducible maer. Therefore, automated fuctioal tests performed i a cotrolled simulatio eviromet should be itegrated ito the quality assurace process of the autoomous parkig system. Evolutioary fuctioal testig provides a way of automatig fuctioal tests as a complete process. Istead of selectig the test cases maually, a search for iterestig test cases is performed automatically. This is doe by traslatig test case selectio ito a optimizatio problem. The possible iput situatios of the system uder test are mapped to the search space. O the oe had, the mappig should keep the size of the search space as small as possible, ad o the other had, the mappig should be

5 1404 J. Wegeer ad O. Bühler able to produce all possible iput data for the system. If oe cosiders the whole iput rage durig the desig of the test data geerator, it does ot mea that all the test cases i this rage will actually be tested but it does provide the possibility of geeratig ay test data required. A appropriate model has tobe desiged for this purpose. Both compoets of the autoomous parkig system have to be tested thoroughly. I the case of the parkig space detectio compoet, we must make sure that suitable parkig spaces are idetified precisely whereas parkig spaces too small for a parkig maeuver have to be rejected. Usually, the parkig space detectio has to be tested i atural eviromets sice reliable simulatio models for the evirometal sesors are ot available. Therefore, it is ot cosidered i the subsequet sectios of this paper. Nevertheless, we pla to test the parkig space detectio i the future by geeratig evirometal data. For the testig of the parkig cotroller, test cases describig differet parkig scearios are geerated. The evaluatio of the test cases is carried out by the fitess fuctio. I order to test the automatic parkig system, the fitess fuctio calculates a umerical fitess value for each parkig sceario geerated. This fitess value represets the quality of the correspodig test case ad aims to lead the evolutioary search ito a directio of iput situatios which result i a parkig cotroller error. Therefore, the fitess fuctio is desiged to assig good fitess values to parkig scearios which cause the system to eter the collisio area or ed up i a iadequate parkig situatio. Bad fitess values are assiged to scearios which result i a good parkig positio with eough distace to the collisio area. 3.3 Test Eviromet The test eviromet of the automatic parkig system (Fig. 3) comprises the simulatio eviromet, a evolutioary algorithm toolbox, a implemetatio of the fitess fuctio ad the test data geerator which traslates idividuals ito actual parkig scearios. The test object is the cotrol uit of the vehicle with the implemetatio of the automated parkig system iside. The GEA toolbox for Matlab ( was used to implemet the evolutioary algorithms used to test the autoomous parkig system. The simulatio eviromet (built up o a Matlab R12.1 platform) simulates the properties of the vehicle as well as the surroudig eviromet. It rus with the "i-the-loop" cotrol uit, meaig that the simulatio eviromet calculates the sesor data for the vehicle ad presets it tothe parkig cotroller iside the cotrol uit. The cotrol uit processes this sesor data ad reacts to it with cotrol data for the simulatio eviromet. This loop simulates a complete parkig maeuver for the parkig space sceario geerated. The parameters describig a parkig space sceario for the simulatio, such as the positio of the car ad the size of the parkig space, are outputs of the test data geerator (see 3.4.). After the simulatio of a parkig maeuver the fitess value is calculated, usig the fitess fuctios described i the subsequet sectio, ad is the assiged to the idividual geerated.

6 Evaluatio of Differet Fitess Fuctios for the Evolutioary Testig 1405 fitess Fitess Fuctio parkig maeuver Evolutioary Algorithm Toolbox parkig space geometry idividuals Simulatio Eviromet sesor data actor data Cotrol Uit Test Data Geerator Fig. 3. Desig of the Test Eviromet 3.4 Test Data Geerator The geometric data which characterizes a parkig space comprises six poits P0 to, ad is referred to as parkig space geometry. The poits defie the border betwee the drivable ad impassable area of the parkig situatio. The model for the geeratio of this parkig space geometry is show i Fig.4. It is a simplified model because the borders of the parkig space are always rectagular. The shape of the parkig space ca oly vary i legth ad depth. This model takes the values of five idepedet variables ad calculates the parkig space geometry from them. The idepedet variables defie legth ad width of the parkig space. I additio, the startig positio with the distace of the car to the parkig space ( dist2space ), the agle psi ad the gap betwee the vehicle ad the collisio area o the right had side of the car are part of the parkig space scearios geerated. distace to space P0 P1 space legth space width g a p p si collisio area Fig. 4. Model for the Geeratio of Parkig Space Geometry 4 Defiitio of Fitess Fuctios This sectio describes the defiitio of two differet strategies for the evaluatio of the fitess of a parkig sceario. Oe strategy uses the distace betwee the vehicle ad the collisio area as a measure of the evaluatio of fitess [1], the other strategy

7 1406 J. Wegeer ad O. Bühler works with the area betwee the vehicle ad the collisio area. Both strategies separate the parkig space ito two parts: (1) collisio with the precedig vehicle ad (2) collisio at the parkig side. While the vehicle is pullig ito the parkig space its freedom of movemet is limited ad thus, if a collisio with the precedig vehicle or the parkig side occurs, either the right rear edge or the right frot edge of the car will be ivolved. Depedig o the positio of the collisio areas the lies betwee - ad - have to be observed i order to idetify a collisio with the frot vehicle. To idetify a collisio at the side, the lie - has to be observed. The defiitio distiguishes betwee the observatio of a corer, defied by three poits, ad the observatio of a edge, defied by two poits. The fitess fuctio is iteded toassess a parkig sceario ad to assig a adequate fitess value to it. The fitess value should correspod to the quality of the parkig sceario. From the testig perspective it is a good parkig sceario if a collisio with other parked cars occurs or the cotrolled vehicle touches the parkig side. Sice we are miimizig the fitess values durig the search process, the fitess value assiged to a parkig sceario decreases with the quality of the sceario. A iterestig parkig sceario achieves a smaller value tha a parkig sceario for which the automatic parkig system performs well. The fitess becomes egative whe a collisio occurs. 4.1 The Distace Criterio Fitess Fuctio The distace criterio cosiders the closest distace betwee a vehicle edge ad the collisio border durig the parkig maeuver. I separate evaluatios the smallest distace of the collisio corer -- ad the smallest distace from the collisio side - are calculated. The followig subsectios describe, how the evaluatio of the collisio corer ad the collisio side is carried out. Fig. 5. Selectio of Smallest Distace Evaluatio of a Collisio Corer The collisio corer is defied by three poits --. All distaces are calculated as polar coordiates with as their origi (Fig. 5). The evaluatio ofthe collisio corer observes the sectio defied by -- ad the sectio diagoally opposite. Oly the poits withi these sectios are cosidered. As a quality measure for the evaluatio of the parkig maeuver, the smallest distace betwee the vehicle path positios ad poit is take. The value of the distace is positive, if the path is

8 Evaluatio of Differet Fitess Fuctios for the Evolutioary Testig 1407 outside the collisio corer. The value is set to zero, if the path crosses. The value is measured as a egative value, if the path rus through the collisio corer (Fig. 6). D1>0 D2=0 D3<0 Fig. 6. Siged Distace Values This strategy aims to esure, that the closer the path gets to the collisio corer, the lower the assiged fitess value becomes. For differet paths which cotiuously move ito the collisio corer, the correspodig fitess values become cotiuously lower, as show i Fig.7, where D1 > D2 > D3. Evaluatio of a Collisio Side The collisio side is defied by the straight lie betwee -. The distaces calculated are betwee this lie ad the path positios of the car. I this calculatio, oly path poits whose x-values are withi the rage of ad are take ito accout. The selectio is carried out by comparig the x-coordiates of the path poits with ad. D<0 D>0 D=0 Fig. 7. Distace from Lie with Positive, Zero ad Negative Value The distace is calculated positive, whe the path is above the lie, ad is set to zero, whe the path touches the lie. The distace is calculated as a egative value, whe the path rus through the collisio area. From all the distace values calculated, the miimum is take as the fitess value for the parkig space sceario geerated. 4.2 The Area Criterio Fitess Fuctio The area criterio fitess fuctio cosiders the area betwee the path of the vehicle ad the parkig geometry. Here, the areas icluded i the collisio corer ad the collisio side are calculated i separate evaluatios. The followig subsectios describe how the evaluatio of the collisio corer ad the collisio side is carried out with the area criterio.

9 1408 J. Wegeer ad O. Bühler Evaluatio of a Collisio Corer The area icluded betwee the corer lies ad the vehicle path is take as a measure of the evaluatio of the parkig sceario. I order to calculate this area, it is separated ito smaller segmets, appropriate to the poits of the vehicle s path through the corer, as show ifig.8. The overall area A icluded i the collisio corer is the sum of all the segmets together. A A Fig. 8. Separatio of Icluded Area ito Small Segmets For a effective ad fast calculatio of the fitess value, a approximate descriptio of the area of each segmet ca be provided by a triagle. Whe the distaces betwee two poits of the path T ad T +1 is sigificatly smaller tha the distace to the agle alpha is very small. T + T + al p h al p h d T r T Fig. 9. Approximatio of Segmet by a Triagle For small agles of alpha, the ratio d to r is approximately the agle i radias. The area of oe segmet which is approximately described by a triagle is: 1 A = r d 2 the area of a segmet ca be calculated thus: A 1 2 ad with d r ( th th ) 2 r (1) ( th th ). (2) The agle th ad the radius r for each path poit T ca be easily obtaied, by trasferrig the path positios from Cartesia coordiates ito polar coordiates with as the origi of the coordiate system. The overall area of the corer is the sum of all the segmets withi the corer A. (3) corer = A

10 Evaluatio of Differet Fitess Fuctios for the Evolutioary Testig 1409 To lead the optimizatio towards a collisio, the corer -- symmetrical to poit is also take ito cosideratio. The path of the vehicle has topass the poit ad the aim of the optimizatio is to brig that path ito the collisio area. The idea is to rate a area icluded i the opposite corer as a positive value ad a area icluded ithe collisio corer as a egative value. Whe the vehicle path crosses through the corer poit the correspodig value is set to zero. Thereby, the areas show i Fig.10 become A1 > A2 > A3 whe the vehicle path keeps o shiftig ito the collisio corer. This leads to a gradual improvemet of the fitess value, depedig o how ear the vehicle path passes by the collisio corer or crosses ito it. A1>0 A2=0 A3<0 Fig. 10. Siged Area Values Evaluatio of a Collisio Side The evaluatio of a collisio side takes the area icluded betwee the vehicle path ad the straight lie - as a measure. The calculatio of the area takes oly those poits ito cosideratio which have their x-coordiate i the rage betwee ad. The calculatio is carried out usig a approximatio, with the rectagles defied by the path positios. The area of each rectagle ca be easily calculated by x ad the distace betwee path ad lie y. With a sufficiet umber of path poits i the rage betwee ad, a good approximatio of the area icluded ca be achieved. A<0 A>0 A=0 Fig. 11. Area with Positive, Zero ad Negative Value Fig.11. (1) shows whe the path is above the lie, (2) the path touches the lie or (3) the path crosses the straight lie ito the collisio area. I the first case, the area icluded has a positive value, i the secod case the value for the area is set to zero. I the third case the area below the lie has a egative value. The distictio draw betwee the three cases prevets a larger area above the lie from compesatig for small areas below the lie. It also prevets a situatio whe the lie is touched from beig cocealed by adjacet areasso that it caot be observed.

11 1410 J. Wegeer ad O. Bühler 5 Experimets The aim of the experimets is to aalyze ad compare the suitability of the fitess fuctios described i the previous sectio for the testig of autoomous parkig systems. Aumber of experimets were performed to aalyze the fitess ladscapes for both fitess fuctios. For each experimet, two variables from the test data geeratio iput vector were varied withi a defied rage ad a defied umber of samples. The remaiig three variables were kept costat resultig i twodimesioal variatio plots of the fitess ladscapes. I order to calculate the fitess value for each parkig sceario, a parkig maeuver carried out by the autoomous parkig system was simulated as described i sectio 3.2. Fig. 12 shows the results for the experimet i which the variables dist2space ad psi were varied, ad i which the legth ad width of the parkig space as well as the gap to the right of the vehicle are kept costat: legth was set to 8.0 m, width to 2.5 m ad the gap to 0.7 m. The axis to the right shows the distace to the parkig space i the rage of 0.0 m to 7.0 m, with a resolutio of 70 poits. The axis at the bottom shows the agle psi from +10 o to 10 o, with a resolutio of 40 poits. Both fuctios retur egative fitess values whe agle psi reaches +10 o ad dist2space goes towards 7 m. A cosiderable differece betwee the shapes of both surfaces is that for fitess values greater tha zero the distace criterio returs costat small values where the area criterio lowers its values towards the border to zero. This appears as a flat plateau i the distace criterio surface (o the right had side of Fig. 12) where the area criterio surface slopes cotiuously (left had side of Fig. 12). Fig. 12. Fitess Ladscapes for Area Criterio (left) ad Distace Criterio (right) as a Fuctio of Agle psi ad Distace to Parkig Space Comparable results were foud i most experimets, e.g. Fig. 13 shows the plots for the variatio of dist2space ad parkig space legth. The rage of the legth, show o the axis to the right, is betwee 5 ad 10 m. Dist2space is show o the axis to the left ad rages from 0 to 7 m. The remaiig variables are kept costat: width of parkig space was set to 2.5 m, the gap to the right of the vehicle was set to 0.7 m ad the agle psi was set to 0 o. As i the precedig example, the surfaces for

12 Evaluatio of Differet Fitess Fuctios for the Evolutioary Testig 1411 the area criterio ad the distace criterio differ. The distace criterio surface has a flat plateau of equal fitess values for small distaces to the parkig space ad loger parkig spaces. I cotrast, the area criterio shows a slopig characteristic for that domai. Fig. 13. Fitess Ladscapes for Area Criterio (left) ad Distace Criterio (right) as a Fuctio of Distace to ad Legth of the Parkig Space 6 Coclusio The experimets have show that both fitess fuctios have a slopig characteristic for collisio maeuvers, where their fitess value is less tha zero. The slope of the distace criterio fuctio is steeper i that domai tha that of the area criterio fuctio. Furthermore, the distace criterio fuctio shows a agle at the poit where the values eter the egative rage whereas the area criterio fuctio has a smooth trasitio for fitess values aroud zero. Both fitess fuctios guide the search towards scearios i which a collisio occurs. The area criterio fuctio provides a slopig characteristic for maeuvers with positive fitess values, i cotrast to the distace criterio fuctio, which returs costat fitess values resultig i plateaus of equal fitess i the search space for scearios without a clash. It does ot differetiate betwee good ad better maeuvers. As a cosequece, the area criterio fuctio is better suited to directig the search towards collisio maeuvers tha the distace criterio fuctio. It helps us to fid test cases more quickly for which the autoomous parkig system does ot react correctly. Therefore, the area criterio will be used for the evolutioary testig of autoomous parkig systems i the future. Ackowledgmets. The work described was carried out as part of the SysTest project., fuded by the EC uder the 5th framework programme (GROWTH, project ref. G1RD-CT ).

13 1412 J. Wegeer ad O. Bühler Refereces 1. Buehler, O., Wegeer, J.: Evolutioary Fuctioal Testig of a Automated Parkig System. Proceedigs of the Iteratioal Coferece o Computer, Commuicatio ad Cotrol Techologies (CCCT '03) ad the 9th. Iteratioal Coferece o Iformatio Systems Aalysis ad Sythesis (ISAS '03), Florida, USA (2003). 2. Grochtma, M., Grimm, K.: Classificatio-Trees for Partitio Testig. Software Testig, Verificatio & Reliability, vol. 3, o. 2, pp (1993). 3. Wegeer, J., Grochtma, M.: Verifyig Timig Costraits of Real-Time Systems by Meas of Evolutioary Testig. Real-Time Systems, vol. 15, o. 3, pp (1998). 4. Tracey, N., Clark, J., Mader, K.: The Way Forward for Uifyig Dyamic Test Case Geeratio: The Optimisatio-Based Approach. Proceedigs of the IFIP Iteratioal Workshop o Depedable Computig ad Its Applicatios, South Africa, pp (1998). 5. Schultz, A., Grefestette, J., Jog, K.: Test ad Evaluatio by Geetic Algorithms. IEEE Expert, vol. 8, o. 5, pp (1993). 6. Joes, B., Sthamer, H., Eyres, D.: Automatic Structural Testig Usig Geetic Algorithms. Software Egieerig Joural, vol. 11, o. 5, pp (1996). 7. Pargas, R., Harrold, M., Peck, R.: Test-Data Geeratio Usig Geetic Algorithms. Software Testig, Verificatio & Reliability, vol. 9, o. 4, pp (1999). 8. Michael, C., McGraw, G., Schatz, M.: Geeratig Software Test Data by Evolutio. IEEE Trasactios o Software Egieerig, vol. 27, o. 12, pp (2001). 9. Tracey, N., Clark, J., Mader, K., McDermid, J.: A Automated Framework for Structural Test-Data Geeratio. Proceedigs of the 13th IEEE Coferece o Automated Software Egieerig, Hawaii, USA (1998). 10. McMi, P.: Search-based Software Test Data Geeratio: A Survey. To appear i Software Testig, Verificatio & Reliability & Reliability (2004).