Functional Testing of Digital Systems

Transcription

1 Functonal Testng of Dgtal Systems Kwok- Woon La Bell Laboratores Murray Hll, New Jersey Danel P. Seworek Carnege-Mellon Unversty Pttsburgh, Pennsylvana ABSTRACT Functonal testng s testng amed at valdatng the correct operaton of a dgtal system wth respect to ts functonal specfcaton. We have desgned and mplemented a practcal test generaton methodology that can generate tests drectly from a system's hgh-level specfcaton. Solutons adopted nclude mult-level fault models and mult-stage test generaton. Tests generated from the methodology were compared aganst test programs suppled by a computer manufacturer and were found to detect more faults wth much better effdency. The experment demonstrated that functonal testng can be both practcal and ej~cent. Automatc generaton of desgn valdaton tests s now closer to realty. 1. ntroducton Functonal testng s testng amed at valdatng the correct operaton of a dgtal system wth respect to ts functonal specfcaton. Conventonal testng methodologes, by contrast, generate tests solely based on the physcal ~structure (e.g. nterconnectons, components used) of a system. Ther am s to dagnose hardware falures and they offer lttle assurance about whether a system has been desgned correctly n the frst place. 1.1 Desgn Faults Desgn faults are non-physcal faults caused by mperfectons ntroduced nto a system durng ts desgn stage. Our orgnal motvaton for research nto automatc test generaton (ATG) at the functonal level was the problem of archtecture valdaton - valdatng that a prototype correctly mplements the nstructon set of a specfed computer archtecture. A second motvaton s the hgh cost of desgn errors n VLS crcuts, where feld engneerng changes are no longer possble. Currently, the functonal correctness of most new systems are beng checked by ad hoc methods. Test programmers are * R~ear.da eonduetes:l - whle prncpal author was a graduate student ar t...amegae-mellon Unversty. Ths research was sponsored by the Defense Advanc,~ a~eare~. Project's Asency (DOD), ARPA Order No. ~/, tuomturen oy me A~r F6rce Avaoncs Laboratory under contrac~ F C The vews and conclusons contaned n ths document are those of the authors and thould.no= b.e nte]!pr.exed ~ representng the offcal poltes, earner expressen or maplen, ot me /Jefense Advanced Research vroject~ Agency or the U.S. Government. often gven the tedous task of wrtng valdaton programs wth vague gudelnes such as "exercse each nstructon at least once" and left to ther own ngenuty. For a new member of an exstng computer famly, these programs may be supplemented by operatng systems and applcaton programs whch have not been wrtten wth testng n mnd. t s hardly surprsng that such ad hoc approaches often do not gve satsfactory qualty assurance. Good evdence demonstratng the nadequaces of present testng technques s expensve desgn errors dscovered n the feld for many computer famles. A functonal testng methodology s very much needed. 1.2 Physcal Fault Detecton The usefulness of functonal testng, however, goes beyond desgn valdaton. t may even be able to supplant tradtonal crcut testng n detectng physcal faults. Wth the number of components per chp doublng every one to two years, tradtonal test generaton methods are apparently comng to a dead end. Many ATG systems based on crcut testng technques have become unusable due to combnatoral explosons n ther computer run tmes. When users generate tests for ther systems pecemeal, they fnd that the complexty has lowered fault coverage whle requrng ever larger numbers of test vectors. Ths growng complexty can be made more manageable by specfyng a system at the functonal level n a herarchcal manner and generatng tests at ths level usng effcent heurstcs. Functonal tests may actually be more effcent than tests generated from a crcut dagram usng tradtonal approaches. Whereas mplementaton-specfc tests are more thorough n detectng ndvdual physcal faults, functonal tests exercse a system at a hgher level, perhaps actvatng many physcally dstant components smultaneously, and can potentally detect more faults n shorter tme than tests that focus on one spot of the hardware at a tme. As n many other areas of computer scence, global effcency may prove to be far more mportant than local effcency. 1.3 Advantages of Functonal Testng Automatc test generaton at the functonal level offers the followng advantages over ad hoc approaches: Better qualty - tests would be generated accordng to scentfcally establshed fault models rather than the personal judgment of test programmers. The tests should be much more thorough. Automaton- test generaton tme and cost can be drastcally reduced, freeng sklled manpower. Tests can X/83/0000/ EEE 20th Desgn Automaton Conference 207

2 be generated as soon as the functonal specfcaton of the system becomes avalable, even before logc desgn s underway, and keep up wth any changes. Such an automatc brdge between specfcaton and mplementaton s extremely useful n development and debuggng and can sgnfcantly shorten the development cycle. Project managers can count on tests of guaranteed qualtes beng generated on schedule, rather than lve n the constant fear of unsatsfactory tests and schedule delays. 1.4 Revew of Lterature Akers ll have proposed the use of bnary decson dagrams n generatng "experments" for smple dgtal devces such as flp-flops and gates. Hs work s among the frst attempts to generate tests based on formal functonal descrptons other than boolean equatons and flow tables. Bnary decson dagrams, however, are not adequate for the descrpton of more complcated dgtal systems. Breuer and Fredman [21 have extended tradtonal crcut testng approaches to handle "functonal level prmtves" such as adders and shft regsters. Ther man concern s hardware fault dagnoss and tests are stll generated from the crcut dagram of the hardware to be tested. Because the structure of ther algorthms are the same as n the orgnal methods, ther extenson nherts crcut testng's problem of combnatoral exploson. Thatte and Abraham TM have proposed a methodology of testng mcroprocessors. Ther chef concern s agan hardware fault detecton. Ther model descrbes data transfer among regsters and the man memory but does not nclude control and data transformaton functons. Based on ths model they have proposed a number of test generaton algorthms. Other researchers nterested n hardware fault detecton have also looked nto the problem of ATG for mcroprocessors. n the nterest of space, ther work wll not be dscussed here. 2. Methodology Our methodology s an ambtous one. t generates tests drectly from the functonal specfcaton of a dgtal system usng mult-level fault models and mult-stage test generaton. Major components of the methodology are shown n Fgure 1. Functonal analyzer s the heart of the test generaton system. ts nputs are the dgtal system's graph descrpton and graph level fault models chosen by the user. For each fault model and each graph prmtve covered by the model, t generates a parameterzed test that detects the fault at that prmtve. The test case syntheszer then substtutes the formal parameters by bt patterns obtaned from the test pattern database. These test vectors can then be mapped by the test program syntheszer nto test program segments that would actually carry out the tests. Any practcal functonal testng methodology must necessarly be a compromse between generalty and practcalty. Functonal approaches that assume nothng about how a system s mplemented would requre astronomcal numbers of tests to cover every possblty. For test generaton to be feasble, one has to make assumptons about the mplementaton. Snce the choce of fault models drectly determnes the cost of test generaton and the effectveness of the tests generated, our approach s to let users decde for l" "-, Functonal Specfcaton,!, (Graph Descrpton),. Parameterzed Tests Test Case Syntheszer Test Cases Test Program Syntheszer Test Program :... Fa_~_ t M_od_els_... " Prmtve Level Fault Models 1 l~ata Base Fgure 1. Components of the Methodology themselves - they can choose predefned fault models or defne ther own. 2.1 Graph Descrpton Language A functonal specfcaton language sutable for test generaton was needed, so we desgned an extensble graph descrpton language called state transformaton graphs (STG). STG can descrbe dgtal systems at levels of detal rangng from user-defned prmtves of arbtrary complexty down to logc gates. Whle an STG graph serves as a system's functonal specfcaton, t also facltates the expresson of structural assumptons about the system whch are vtal for practcal test generaton. Ths s a tradeoff between generalty and practcalty. By allowng the user to nclude as few mplementaton assumptons as s practcal, as much generalty as possble n the tests generated s preserved. A user nterested n physcal fault detecton s free to construct a descrpton the nodes and paths of whch have a one to one correspondence wth physcal components n the system. Wth such a descrpton and approprate fault models, tests generated by our methodology would actually be dentcal to those generated by a tradtonal method from the system's crcut dagrams. n other words, our methodology actually contans tradtonal crcut testng as a specal case. We should emphasze, however, that STG s ntended to be a functonal specfcaton language used by desgners to specfy ther systems. Assocated wth each STG graph s a set of state varables whch can be read and wrtten by read/wrte operators, together they represent a fnte state machne. Even though STG s ntended to be a functonal specfcaton language used by desgners, t can also descrbe arbtrary 208

3 frnge 1 ~ll <" Data Operator Decder Wrte Cell ceu-> l arrayt 1-> J~ay[l <- Read Cell Read Memory Wrte Memory... Decode Juncton... Demultplexer Data Flow Gates Fgure 2. Some STG Prmtves combnatonal and sequental crcuts. An STG graph s a drected graph n whch paths represent (logcal) data/control paths whle nodes represent data transformaton operators. Data s passed along n the form of tokens, wth at most one token per path. A node can.fre when t has the requred tokens on ts nput paths. The frng of a node results n the removal of nput tokens and the placement of new tokens on ts outputs. Every graph has two specal nodes: begn and end. At the begnnng of each cycle, a token s fred from the begn node whch subsequently actvates other nodes dependng on the current state. Eventually a token arrves at the end node sgnallng the end of the cycle. Some basc STG prmtves are shown n Fgure 2. As an example, the top level graph descrpton of the PDP-8 mncomputer s gven n Fgure 3 and the graphs for two of ts nstructons, AND (logcal and) and JMS (jump subroutne), are shown n Fgure 4. Detaled defnton of STG can be found n [6]. 2.2 Graph Level Fault Models Graph level fault models can be vewed as "macro" fault models that model faults at the graph level and are therefore functonal fault models. Each model specfes the knd of graph prmtves covered by the model, and gven such a prmtve, where the test data should be appled and where the test results can be found. The sngle-path model models faults n the data paths of a graph descrpton wth the assumpton that at most one data path can be faulty at any one tme (note that a path s a logcal entty and may not correspond to any physcal wres). t covers any fault that affects a path's ntegrty. A path can be tested by applyng test data to ts nput and examnng the result at ts output. Durng test generaton, symbols representng test data and test result are propagated backward and forward respectvely. The model also specfes fault t p- <11.9> /0 1 r and tad L L ]-> PC+ page J <8> bt bt addr ~ dea tm mp lot L J. mtrucdma Executon "- 77.-,;7.. ~ 1 Fgure 3. STG graph of PDP-8 nstructon Decodng and effectve ~c'~ Mtl "~ '-., / ~D ^el<" effectve +1 PC+ Mt]<- Fgure 4. PDP-8 nstructons : AND, JMS domnance and fault equvalence relatonshps among nputs and outputs of each node type. Smlar to the sngle-path model, the sngle-node model models faults n the nodes of a graph. Ths model specfes that test parameters are to be appled to all nputs of a node wth the node's outputs then carryng the test results to be nspected. The double-path model assumes that up to two paths may be faulty at the same tme. For each par of paths selected for test generaton, a test parameter s appled to the begnnng of each path wth the correspondng test result comng out at the end of the path. Both parameters are backward propagated whle both results are forward propagated. The 209

4 double-path model s useful for detectng nterference between two dat~ 2 paths. The number of tests requred s proportonal to n, where n s the number of paths n the descrpton. One can go even further and allow any number of paths to be faulty at the same tme. The number of tests to be generated then becomes o(2n). A graph level fault model can specfy any combnaton of paths and nodes - the possbltes are endless. The more parameters and results to propagate, the hgher the probablty that a test ether does not exst or cannot be generated by the heurstcs. Practcal experence n other felds of testng has shown that multple faults whch cannot be detected by tests generated for sngle faults occur nfrequently. Wth costs much hgher than the sngle-fault models, the potental beneft of multple-fault models s probably margnal. 2.3 Prmtve Level Fault Models Prmtve level fault models model faults at the level of ndvdual graph prmtves. At ths level, assumptons about the mplementaton are nevtable unless exhaustve or random testng s to be used. For each lkely mplementaton of the same functonal prmtve, a dfferent model may be developed and entered nto the test pattern database. A user can choose a sutable fault model or smply use all avalable models to cover as wde a range of mplementatons as possble. Ths s a tradeoff between cost and generalty. f a model assumes a specfc crcut realzaton, tradtonal crcut testng methods can be called upon to generate test patterns based on that realzaton. f the graph prmtve s n turn defned n terms of lower-level prmtves, the functonal test generaton system can be called upon recursvely to generate'tests for the hgher-level prmtve. Ths s made possble by the extensble graph language. n each case, patterns generated are entered nto the test pattern database under the name of the prmtve along wth the mplementatons assumed and the number of test vectors requred. Example : The sngle-path model makes no assumpton about what the actual fault n the data path may be. The task of selectng test data from the 2 w possbltes (where w s the wdth of the data path) s left to the prmtve level fault model. To test for sngle-bt stuck-at faults n the data path, for example, t s only necessary to use a par of test vectors that turn each bt on and off at least once e.g & , & To test for shorts between adjacent bts, however, would requre more than two test vectors 2.4 Functonal Analyzer The functonal analyzer frst consults the graph level fault model chosen to select a set of prmtves. For each case, test generaton s performed n four sequental stages (Fgure 5): parameter ntroducton, backward propagaton, forward propagaton, and justfcaton. Each stage has ts own objectve and condtons are mposed on the graph as necessary to meet that objectve. By successvely constranng the graph, a parameterzed test whch detects the fault s generated at the end. Each stage s further subdvded nto test generaton routnes that work on one prmtve at a tme. These routnes are called recursvely untl success. f a step fals, the state of the graph s rolled back to the pont where a choce was last made and the next choce s attempted. Localzed heurstcs whch only consder a prmtve's mmedate neghbors n makng a decson are used whenever possble, so the number of prmtves consdered at each step s not affected by the sze of the graph, although the number of steps,,, l.,rapn-level, Graph Descrpton,,,,, Fault Models, J n ] Parameter ntroducton ' ' _! Backward Propagaton ', 1 : Pr Graph tve : Forward Propagaton Database,..J " -! Justfcaton [! t Parameterzed Tests Fgure 5. Operaton of the Functonal Analyzer needed n each stage may grow proportonally wth the number of paths n a graph. At present, only loop-free graph descrptons can be handled by the functonal analyzer. A constrant s a predcate functon assocated wth a path. A path can only carry tokens that satsfy the constrant. A path can have any number of constrants as long as they are mutually consstent. Non-overlappng bt felds wthn a path each can have ts own set of constrants. Cells n the machne state and symbols created durng test generaton may also have constrants. Symbols are used to represent test data, test results, bt felds, and a number of other thngs. Constrants nvolvng symbols and ther manpulaton play a central role n the test generaton process. Durng test generaton, constrants are added to paths, cells, and symbols to create the condtons needed to generate a successful test. Constrant resoluton s the process of determnng whether a set of constrants s consstent and then possbly smplfyng the set. Constrants are usually added one at a tme to a set of constrants that has already been determned to be consstent. Constrant resoluton can be an extremely nvolved problem f arbtrary predcate functons are to be consdered. Fortunately, the descrptons of most computers requre only smple predcate functons such as =, ~, <, ~<, >, >/. As a result, most constrants that arse n practce are very smple and can be resolved usng relatvely smple algorthms. 210

5 Parameter ntroducton - ths phase ntroduces symbols representng test data parameters and test results nto the graph and sets the stage for subsequent phases. The graph level fault model selected s consulted to dentfy the next set of prmtves for whch a test s to be generated. Symbols representng the requred parameters and results are then added to the graph. mplcaton - each tme a path receves a new constrant, t may have mplcatons on other paths connected to the same nodes. At each test generaton step, paths that have receved new constrants are noted and the mplcaton routne s called at the end of the step to check f any of the changes results n a contradcton, ndcatng falure of that step Backward Propagaton For test data to be appled to the requred places, the machne state pror to the test cycle must contan the test data or some transformaton of them. The objectve of the backward propagaton phase s to set up the condtons necessary for the applcaton of test data parameters to requred places. Ths s done by backward propagatng each parameter to a read node n the graph. Then by ntalzng the machne state accordngly pror to the test cycle, t s guaranteed that the parameters would be appled to the correct spots. Each step wthn the backward propagaton phase consders only one path at a tme. t tres to mpose the necessary condtons on the nputs of the path's source node so that the current value of the path s assured. Snce the value to be backward propagated s almost always a symbolc parameter or a symbolc expresson, a one to one mappng of that value must appear somewhere on the nputs of the source code. Ths concept s formalzed n the followng defnton. Defnton: A constant vector (Co,C 1... c.l,c Cn. 1) s a backward propagaton vector of the n-nput functon F ff F(c 0... c.1, g(x), c Cn.1) = x -1 for all x n the output doman of F and ff g exsts. The functon g s called the transformaton functon of the vector. By mposng a backward propagaton vector on the nputs of the source node, the objectve of the current backward propagaton step s satsfed. The backward propagaton r[]~tne s then recursvely called for the path connected to the nput of the node F. The process s successful f a read node s eventually reached Forward Propagaton Test results must be saved n the machne state at the end of the test cycle so that they can be checked for correctness n subsequent cycles. They must therefore ether appear n the contents or the addresses of the new machne state. The objectve of the forward propagaton phase s to brng these results to "wrte" nodes n the graph to ensure ther observablty after the test cycle. Forward propagaton s smlar to backward propagaton n operaton except for ts forward drecton. The concept of propagaton vectors can smlarly be defned. Defnton: A constant vector (c0,c... C.l,C Cn. 1) s a forward propaptlon vector for the th nput of the n- nput functon F ff F(c0,c 1... C.l, x, c Cn.1) = g(x) for all x n the th nput doman of F and ff g'l exsts. The functon g s called the transformaton functon of the vector. Propagaton s successful f the value s passed along to the output wth no loss of nformaton. For a multple-output node represented by the set of functons {F0., F.,,..., F -1} where m s the number of our0uts, the above deftmto.nthcan~e extended - a vector s a propagatontl~ector for the nput f t can propagate the value at the nput to any one of the m outputs. get names of all nodes connected to path any node left? ~.. select n~ node ~ no obtan propagaton vectors ] for node from database / no/ any vector left?.~,,,, yes select next vector nod. propagate symbol thru mplcaton ok? no / yes Propagaton Successful ) Propagaton Faled Fgure 6. Flow Chart for Forward Propagaton Propagaton vectors for a functon can usually be determned smply by nspecton. The most obvous and smple ones can be entered nto the graph prmtve database whch contans all defntons and heurstcs assocated wth each type of graph prmtve. Fgure 6 shows a flow chart of the forward propagaton phase. The backward propagaton phase works n essentally the same manner. Justfcaton - Durng the course of backward and forward propagaton, when necessary condtons for the test generaton to be successful are assgned, many paths are often left unspecfed. Justfcaton flls n the unspecfed values so that a parameterzed test can actually be generated. 211

6 The test case syntheszer brngs together parameterzed tests produced by the functonal analyzer and test patterns that have been developed through prmtve-level fault models. t substtutes the formal parameters by actual test data and expected results obtaned from the test pattern database. The user can specfy the characterstcs and cost constrants he wants to mpose. Only patterns meetng the requrements wll be chosen from the database. Test Program Syntheszer - f the system beng tested s a computer, ts nstructons can usually be grouped by ther operand fetchng modes and members of the same set can usually be tested wth smlar sequence of nstructons. Test program templates can be developed for each group. For each actual test case gven, the syntheszer selects the applcable templates and tres to fll n the necessary "blanks" whenever possble, producng ready-to-run program segments and automatng the test programmng process. ZS Desgu Comderatlom Separaton of ssues - a major goal of the research s to untangle the ssues nvolved n functonal testng, dvdng them nto smaller problems that can be attacked one at a tme. Ths enables better focusng on the ndvdual ssues, makes the functonal testng problem as a whole more manageable, and permts the development of more effcent solutons. Ths s done throughout the methodology whenever possble. The solutons adopted nclude: 1. Mult-level fault models - fault models are dvded nto graph and prmtve levels. 2. Mult-stage test generaton - tests are generated n two stages. Parameterzed tests can be repeatedly used by the test case syntheszer to generate tests for dfferent requrements. Modularty and Flexblty - the methodology s desgned to facltate evolutonary changes and ncremental mprovement n every one of ts buldng blocks. The followng solutons were adopted: * Test generaton s performed n stages even wthn the functonal analyzer, a heurstc can be "unplugged" and replaced by a new one wth ease. Whenever possble, nformaton s grouped nto databases that can be easly modfed by the user. The databases nclude all the fault models, the test pattern database, test program templates, and defntons of the prmtves used n graph descrptons. User Conlrol - the user s gven maxmum control over the whole test generaton process because each applcaton has ts own fault dstrbuton and cost consderatons and t s best to let the user specfy the pertnent nformaton rather than forcng unrealstc assumptons upon the user. A user can exert control over the test generaton process by : 1. Selectng the level of detal at whch the dgtal system or parts of the system s specfed. 2. Defnng new graph prmtves. 3. Selectng or defnng fault models. 4. mposng cost and other restrctons durng test generaton. 3. Experment A fault smulaton experment utlzng SPS [4], a regstertransfer level hardware descrpton language, was conducted to evaluate the effectveness of our methodology. Tests were generated for the DEC PDP-8 mncomputer and compared aganst test programs suppled by the computer's manufacturer. An SPS descrpton of the PDP-8 was smulated wth sngle stuck-at faults n all the data paths represented n the descrpton, for a total of 1,438 faults. Desgn faults were not chosen because of ther much larger fault space and t mpossble to select a subset wthout usng consderable subjectve judgment. On the other hand, sngle stuck-at faults have long been accepted as the startng pont of many testng strateges at both SPS Descrpton Fanlt Routnes Generat~ [ SPS Compler dependent faults SS Smulator [ Expermental Results Fgure 7. Desgn of Experments Database hgh and low levels. t s the closest thng to a quamtatve yardstck by whch tests generated by varous methodologes are measured. t s generally accepted that a test that detects a hgher percentage of sngle stuck-at faults than another test s probably the better test, even though many of the faults that actually occur n practce are not sngle stuck-at faults. Sngle stuck-at faults were therefore selected for our experment as the most objectve and practcal measure avalable. The desgn of the experment s shown n Fgure 7. An STG graph of the PDP-8 was developed based on the computer's handbook and has 158 nodes and 235 paths. The graph was run through our system to produce parameterzed tests usng the sngle-path fault model. The heurstcs succeeded n generatng tests for 97% of the paths selected by the model. The test patterns used n test case synthess were smple: each bt of every parameter wthn a parameterzed test need to be turned on and off at least once. The test cases were gven to a test programmer who sem-mechancally translated them nto PDP-8 assembly code (a test program syntheszer was not wrtten for the PDP-8). The test programmng effort was straghtforward and no n-depth understandng of the system 212

7 beng tested was requred at all. The same translaton could have been performed automatcally by smple test program synthess technques. The test programs suppled by DEC exerdse the machne at a functonal level and halts whenever an error s dscovered, pnpontng the nstructon at fault. Our functonal tests actually provde better dagnostc resoluton snce each test s desgned to detect faults n a functonal prmtve. Results of the experment are summarzed n Table. Our tests outperformed the manufacturer's programs by a substantal margn - we detected 98.5% of the stuck-at faults compared to 95.5%. Most surprsngly, tests produced by our methodology acheve the hgher fault detecton rate wth far fewer nstructons. Test Program nstructons Executed Faults Detected Manufacturer's > 10, % Ours % References S.B.Akers, "Functonal Testng wth Bnary Decson Dagrams", n Prec. 8th nt'l Conference on Fault-Tolerant Computng (FTCS-8), pp 75-82, EEE Computer Socety, June M.A. Breuer and A.D. Fredman, "Functonal Level Prmtves n Test Generaton", EEE Transactons on Computers C-29(3),pp , March S.M.Thatte and J.A. Abraham, "Test Generaton for Mcroprocessors", EEE Transactons on Computers C- 29(6), June M.R.Barbacd, G.E.Barnes, R.O.Cattell, and D.P.Seworek, "Symbolc Manpulaton of Computer Descrptons: The SPS Computer Descrpton Language", CS Dept. TR CMU-CS , Carnege-Mellon U., August Kwok-Woon La, "Functonal Testng of Dgtal Systems", PhD thess, Computer Scence Dept. TR CMU-CS , Carnege-Mellon Unversty, Dec Table 1. Results of the Experment Frst verson of the test generaton system had over 5,000 lnes of LSP code and requred 600K bytes to run. Only the graph descrpton language and the functonal analyzer were fully mplemented n the frst verson. Test generaton for the PDP-8 took only 21 mnutes of CPU llme runnng wth nterpreted LSP code on a DEC Complaton of the LSP code alone can speed thngs up about fve tmes. 4. Condml~a We have taken a practcal approach towards the problem of functonal testng at the system level. We lad down the groundwork for a systematc assault on the problem and also provded a framework for dvdng the overall problem nto smaller, more manageable problems. An ambtous functonal testng methodology whch generates tests drectly from the functonal specfcaton of a dgtal system has been desgned, mplemented, and evaluated. Automatc generaton of desgn valdaton tests s now closer to realty. Test cases that would have taken test programmers man-months and hard-earned experence to develop can now be generated automatcally n a matter of mnutes. Although the scope of the evaluaton experment s lmted, ts results are very promsng. The qualty and the effcency of the tests generated further underscore the promse of functonal testng. On the other hand, there are stll many thngs that we cannot begn to understand untl more practcal experence s ganed. Large scale experments are currently beng planned. mprovement of the system wll contnue as w contnue to learn. 1. The nterested reader wll fnd a m~e detaled dscus~rm c~ our results n my thess [5]. 213