FROG and TURTLE: Visual Bridges Between Files and Object-Oriented Data y

Transcription

1 FROG and TURTLE: Vsual Brdges Between Fles and Object-Orented Data y Vashnav Anjur Yanns E. Ioannds z Mron Lvny Department of Computer Scences, Unversty of Wsconsn, Madson, WI fvash,yanns,mrong@cs.wsc.edu Abstract The problem of translatng database objects nto a flat format to be wrtten out n a flat Asc fle or, conversely, translatng the contents of a fle nto a complex database object arses n several applcatons. It s especally mportant n scentfc database applcatons, where fle-based communcaton wth external programs (e.g., vsualzaton packages or model smulatons) s very common. We ntroduce Frog, a vsual tool that can be used to specfy translatons between database objects and flat fles, requrng no programmng by the user. The tool can deal wth objects of arbtrary complexty, wthout the object complexty beng drectly reflected n the complexty of the correspondng vsual nteracton. Based on the vsual actons of the user, the tool stores enough nformaton n a map-fle, whose contents are used at runtme by another tool, Turtle, to translate any chosen database object nto the approprate fle layout. The tool has been developed as part of the ZOO desktop Experment Management Envronment and has been used by a few expermental scentsts wth success. 1 Introducton Many applcatons exst that requre an Object-Orented Database System (OODBMS) to be communcatng wth other, external, software, sendng data to t or recevng data from t. For example, database data may be sent out for some complex computatons to be performed on t (possbly by legacy software) wth the results comng back to be stored n the database as well. Very often ths communcaton s acheved through Asc fles (or Asc byte streams) because of the heterogenety of the overall system. For example, the external software may have been bult ndependently of the OODBMS and may support a fle-based I/O nterface; or t may smply support an nterface that s dfferent from that of the OODBMS and fles are a convenent ntermedate representaton for the data to be communcated. Hence, the OODBMS must have the ablty to translate any complex database object ntoa flat structure to be wrtten out nto a fle, or to read n a fle and translate ts flat contents nto a complex object. Ths ablty s also necessary n order for legacy data Work supported n part by the Natonal Scence Foundaton under Grant IRI y We would lke to acknowledge Tom Wang for hs sgnfcant effort n mplementng Frog and Turtle. z Addtonally supported n part by the Natonal Scence Foundaton under Grant IRI (PYI Award) and by grants from DEC, IBM, HP, AT&T, Oracle, and Informx. to be loaded nto a database so that future access to the data can be done through the OODBMS. The above functonalty s especally crtcal for scentfc databases, snce they are assocated wth applcatons that have all the above characterstcs. For example, data stored n scentfc databases often needs to be sent out to specalzed vsualzaton programs to be vsualzed, or to statstcal software for analyss, or to modelng packages for smulaton, wth the results (of the last two types of software) stored back nto the database. Experment management, whch s our emphass and man motvaton, also generates smlar scenaros. An experment management system needs to communcate wth the expermentaton envronment (be t a model smulaton program, an automated assembly of nstruments, or even a human that manually operates on nstruments) to send requests for experments wth specfc nput and to later receve the experment output for storage. Translaton between database data and fles s often hardcoded nto exstng systems, e.g., bulk loadng facltes n most commercal OO and relatonal DBMSs. Ths s undesrable, however, because t restrcts the layout of the fles that can be generated or loaded, whle n scentfc database applcatons, one wants genercty so that communcaton wth several dverse programs s possble. An alternatve s for the OODBMS to provde hooks so that dfferent translaton modules may be bult, as applcaton code. Such codng, however, could be rather laborous snce t has to be repeated for each external software program expectng or producng fles of dfferent layouts; t has to deal wth database objects of nontrval structural complexty, e.g., sets or ndexed-sets, and to produce the correct layout of the complex objects parts, e.g., usng the correct delmter between set elements or the correct set termnator. t has to deal wth format conversons between database values and ther correspondng entres n fles, e.g., between floatng pont and decmal values. A thrd alternatve s for the OODBMS to provde a hgherlevel mechansm, e.g., a declaratve language, for descrbng how specfc translatons from database objects to fles must be done, e.g., as n relatonal report wrters. Then, the OODBMS ether generates translaton code based on the descrpton or nterprets the descrpton at run-tme to perform the necessary translatons. In ths paper, we follow the thrd alternatve n ts nterpreted verson. Our task s slghtly more complex than that of report wrters though. Whereas they are prmarly used

2 to desgn the layout of fles (reports), we need to translate database objects nto gven fle layouts, requred by the external programs. We ntroduce Frog, a vsual tool that we have developed and s used off-lne to map (.e., specfy translatons from) database objects to flat fles. (For smplcty, our entre presentaton focuses on translatons from database objects to fles only, but the reverse drecton s handled followng essentally dentcal prncples.) It generates a fle of ts own, a map-fle, whch contans all the necessary nformaton for object translatons. At run-tme, ths fle s consulted by another tool, Turtle, whch nterprets ts contents to translate any chosen database object nto the approprate fle layout. We present the vsual mechansms ncorporated nto Frog and the underlyng technques that make these mechansms work. These are suffcently general to allow translatons of objects of arbtrarly complex structure, e.g., sets of sets, as well as translatons of objects n dfferent branches of sa herarches (wth dfferent structures). We descrbe the contents of the map-fle that s generated by Frog and also how Turtle uses them for specfc translatons. Ths whole effort has been carred out wthn the context of the ZOO Desktop Experment Management Envronment that we have been developng [8]. Hence, we also dscuss some prelmnary experences that we have had wth the tools as they have been used by some expermental scentsts n ther database-to-fles translatons. 2 The ZOO Experment Management Envronment Expermental studes n essentally every scentfc dscplne go through a smlar lfe-cycle of several phases: desgn, expermentaton, and data analyss. In most cases, the current state of the art forces scentsts to use dfferent tools n each phase of that cycle, makng the whole process dffcult to manage. We are nvolved n the development of the ZOO desktop Experment Management Envronment that aspres to brng state-of-the-art management tools to the desk of expermental scentsts [8]. ZOO s an ntegrated software package that wll enable such scentsts to manage ther experments and assocated data from ther desk va a unform nterface. Ths work s done n collaboraton wth researchers throughout the Unversty of Wsconsn - Madson campus, especally the Departments of Sol Scences, Bochemstry, and Physcs. ZOO has the ablty to communcate wth several external expermentaton envronments, e.g., laboratory equpment, smulaton programs, statstcal analyss tools, etc. Each envronment operates on an approprate nput fle and produces an output fle. An nput fle s constructed by a ZOO module called Turtle, whch receves the od(s) of the object(s) that capture the envronment s nput as well as the name of a map-fle that has been generated off-lne by another module called Frog. Turtle nterprets the map-fle, uses the gven object od(s) to extract the needed data from the approprate database under ZOO, and eventually constructs the fle. In a smlar fashon, when the external processng s over, the output fle produced s translated nto database objects. Turtle and Frog are the focus of ths paper, so they are descrbed extensvely n subsequent sectons. Understandng ther operaton requres some famlarty wth Moose and Fox [12], the data model and query language of the database server of ZOO, whch are brefly descrbed below. There are three knds of object classes n Moose: prmtve, tuple, and collecton. The prmtve classes are nteger, real, boolean, and character-strng. Objects n tuple classes consst of a prespecfed number of other objects, called parts, dentfed by labeled relatonshps. Objects n collecton classes consst of an arbtrary number of other objects, all from a sngle elements class. Collecton classes are further subdvded nto set, multset (bag), sequenced-set (lst or array), and ndexed-set classes. An ndexed-set s a generalzaton of a sequenced-set. Whereas the members of a sequenced-set are ndexed by the set of consecutve ntegers f1; : : :; ng, for some n, the elements of an ndexed-set are ndexed by (the elements of) an arbtrary collecton object. Ths collecton object s called the keyset for the ndexed-set. There are fve knds of bnary object relatonshps n Moose. An arbtrary number of has-part relatonshps, each pontng to a sngle object, defnes the structure of tuple classes. A sngle set-of relatonshp defnes the structure of collecton classes, except for ndexed-set classes whose structure s defned by a sngle set-of and a number of ndexed-by relatonshps equal to the dmensonalty of the ndexed-set. Assocaton relatonshps do not defne any structure but smply connect ndvdual objects n two arbtrary classes. Fnally, an sa relatonshp between two classes dentfes one of them as a specalzaton of the other and mples that every object n the subclass belongs n the superclass as well. A path expresson n Fox may traverse any of these knds of relatonshps. It starts at a source class, and ts result s the set of objects that are (transtvely) related to the objects n the source class va the relatonshps n the path expresson. Usng graphs to represent Moose schemas s very ntutve. Each class s a node: oval for prmtves (abbrevated as for ntegers, r for reals, b for booleans, and c character strngs) and rectangle for all others. Each relatonshp s an arc n the graph, wth dfferent brush patterns ndcatng dfferent knds. Each relatonshp has a label n each drecton, whch f unspecfed, s equal to the name of the target class of the relatonshp n that drecton. 3 An Example Fgure 1 shows a smple Moose schema that s used as an example throughout the rest of the paper. It represents a (smplfed) sol-scence study to determne the total yeld and qualty of a crop dependng on the Weather and on how varous types of plants are dstrbuted n a large feld dvded

3 has part setof assocaton s a ndexed by date c Input Experment (D) Output yeld qualty ranfall temperature Weather wnd_speed c name Plant_communty Zones wnd_drecton c Wndy Dry humdty name Plant heght zone_name c Dsaster wdth Fgure 1: Sample Moose schema of Sol Scences experment nto Zones. Each experment s modeled as a complex object, wth sub-objects representng ts Input and ts Output. Its output s a par of the total yeld and qualty of the harvest. Its nput conssts of the Weather and a Plant communty, whch s an ndexed-set of plants ndexed by the set of feld Zones so that the zone where each plant s grown s recorded. The weather s captured by ranfall, temperature, and wnd-speed values, and may be Wndy (wnd-speed > 30 mph), n whch case wnd-drecton becomes mportant as well, Dry (ranfall < 2 n), n whch case ar humdty becomes mportant as well, or Dsaster, whch combnes the two. We assume that the output part of an experment s the result of runnng an external program on ts nput part. Fgure 2 shows an example nput fle for that program, whch contans the necessary nformaton plus other tems as well (the background patterns under some tems wll be explaned shortly). Rows WEATHER PLANT COMMUNITY Z1 corn, 5, 45 Z2 rye, 7, 30 Z3 corn, 5, 45 Z4 wheat, 4, 38 SOIL phosphorus 13 Sample Input Fle constant area ranfall temperature wnd_speed constant area Zone obsolete area Fgure 2: An example nput fle name, wdth, heght 4 General Operaton of FROG Before descrbng the basc operaton of Frog, we frst clarfy some basc notons and ntroduce some termnology. Gven a complex database object, one can wrte n a fle only ts parts, elements, or assocated objects that belong to prmtve classes,.e., leaf classes n the schema graph. For example, one cannot wrte the od of a plant object because t s meanngless outsde the database system. Thus, when we use the terms mappng a complex class (for Frog) or translatng/prntng a complex object (for Turtle), we refer to the prmtve, alphanumerc, objects related to t. Also, n prncple, there may be multple ndependent classes n the schema whose objects correspond to the fle to be generated. Nevertheless, for smplcty of presentaton, we assume that there s only one such class. It s called the source class, and any object n t that Turtle must translate s called a source object. Generalzng to mappng multple classes to a sngle fle s straghtforward. Based on the above, we turn to a descrpton of how Frog operates. The users of Frog are desgners who are famlar wth both the Moose database schema and the layout of the nput fle to the external program. A sample nput fle for the external program s brought on the man wndow of Frog. The sample fle could exst from earler uses of the external program or could have been constructed just so that t can be used n the mappng process. The Moose schema for the experment concerned s also brought n graph form on another wndow managed by the Opossum schema manager [5]. The entre mappng task proceeds by repeatng the followng step sequence. 1. The desgner chooses an area n the sample fle (by hghlghtng t wth the mouse). Ths sgnfes that mappng

4 specfcatons for that part of the fle wll be gven. The area remans hghlghted for the rest of the sesson wth Frog and even across sessons. At ths pont, only areas that cover all the way to the end of a fle lne before movng on to the next lne are supported. 2. The desgner chooses a class n the schema shown by Opossum (by a mouse clck). The path expresson from the source class to the chosen class s sent to Frog. Ths sgnfes that the area dentfed on the sample nput fle n step 1 s mapped from the contents of the specfed class. 3. Dependng on the knd of class chosen n step 2, Frog presents another wndow wth varous entres consttutng a prntng specfcaton,.e., how an object of the class should be prnted. Usually, only a subset of the entres s enough, but the system presents all those that could possbly be used. Based on the precse contents of the area chosen n step 1, some of the entres are already flled up by best guesses of Frog. The desgner may accept those or modfy them approprately. For example, assume that the fle n Fgure 2 s used as the sample nput fle for the mappng process. To map the ranfall attrbute n the schema of Fgure 1 to the area of the sample nput fle contanng 40, the desgner would frst hghlght 40 and then clck on the oval ponted to by the arc labeled ranfall, whch represents the class of ntegers, where ranfall values belong. Opossum generates the path expresson Experment.Input.Weather.ranfall and transmts t to Frog. The prntng specfcaton wndow then appears wth entres related to ntegers, e.g., maxmum length (number of dgts) occuped by any nteger (Secton 5.1). Ths entry wll contan the value 2 as a guess, whch s the length of 40. If the external program does not need ranfall values prnted so that they occupy a prespecfed maxmum length, or f the database may contan ranfall values that are longer, then the user wll change the estmated value approprately (n the frst case, t wll make the entry empty). The step sequence outlned above deals wth the areas of the sample nput fle that are ndeed mapped to from data n the database, called varable areas. There could be areas, however, that should appear n no nput fle generated by Turtle. These may be found n legacy sample fles that no longer correspond exactly to the nput requred by the external program. Such areas are called obsolete. They are dentfed as n step 1 above wth Frog beng n a specal mode that nterprets step 1 dfferently: the area s hghlghted n dfferent color from that of varable areas, and there s no contnuaton nto steps 2 and 3. In Fgure 2, the areas n rows 8 and 9 are obsolete, as ndcated by the darker background. In addton to the above, there could also be areas n the sample nput fle that should have the same value n all nput fles generated by Turtle and do not correspond to any class n the schema. These are called constant areas. Any area not hghlghted through step 1 s consdered constant. In Fgure 2, the areas n rows 1 and 3 are constant, as they have not been hghlghted at all. 5 Basc Translaton Specfcaton by FROG For clarty of presentaton, we frst deal wth mappng a restrcted form of Moose schemas to fles, and then remove the restrcton n Secton 6. In partcular, we consder mappng (sub-)schemas that contan only tuple and prmtve classes, and only has-part and assocaton relatonshps. That s, when appled on a sngle source object, any path expresson to a class that must be mapped results n a sngle object as well. In ths case, all (prntable) nformaton about the source object s n ts leaves. Hence, for the purposes of mappng to a fle, the entre schema of nterest can (although does not have to) be seen as equvalent to a smple 1-level-deep schema wth all leaves hangng drectly under the (tuple) source class. 5.1 Vsual Language In the restrcted case above, the man concern s mappng the leaves of nterest to the fle n the approprate order. The source class maps to the entre sample fle, and ts leaves n the schema to the ndvdual elements n the sample fle. To express the above, desgners nteract wth Frog as follows. Brngng the sample nput fle on the screen s equvalent to choosng the entre fle area n step 1 of the step sequence of Secton 4. The class ndcated n step 2 becomes the source class of all path expressons to be generated by Opossum later on. Gven the restrctons of ths secton, ths must be a tuple class, so n step 3, the prntng specfcaton wndow for tuples pops up, contanng the followng entres: Delmter between the parts of the tuple object Termnator for the tuple object Maxmum number of elements per lne Maxmum number of characters per lne Default value to be prnted f database entry s null Consder the example of Fgures 1 and 2. If we only concentrate on row 2 of the sample fle and the Weather part of the Input class n the schema, then we are dealng wth an nstance of the restrcted case dscussed n ths secton. By specfyng the delmter entry of the prntng specfcaton wndow to be <space> and the termnator entry to be <return> (the newlne character), and leavng the subsequent two entres unspecfed, a desgner provdes enough nformaton for Turtle to know how to prnt an entre tuple object n a fle (all parts prnted n the same lne). After ths, desgners enter the step sequence of Secton 4 as many tmes as there are varable areas n the sample fle. Independent of whch prmtve class the correspondng leaf n the schema s, the prntng specfcaton wndow that pops up always contans the followng entres: Length (n number of characters)

5 Default value to be prnted f database entry s null If the correspondng leaf n the schema s the class of reals, then the prntng specfcaton wndow also contans the followng addtonal entres: Precson (n number of decmal dgts) Notaton (scentfc or decmal) Consder agan row 2 of the sample fle n Fgure 2. The followng table presents collectvely prntng specfcatons that desgners may gve as they map each of the three prmtve parts of the Weather class: Entry ranfall temperature wnd-speed length default As mentoned above, for each entry, Frog ntally offers a guess for a value based on what the desgner hghlghted on the sample nput fle. (For lack of space, the detals of ths guessng as mplemented n Frog are not presented here.) 5.2 Map-Fle The result of a sesson wth Frog s a map-fle. For each varable and constant area dentfed through Frog, the map-fle contans a block wth all the necessary nformaton to populate the area n an nput fle. There are dfferent types of blocks for the dfferent types of areas and/or knds of mapped classes. Some blocks are prntable, capturng areas of ndvdual values that may be prnted. Other blocks are structural, capturng larger areas that contan smaller (prntable or structural) areas and enclosng the blocks correspondng to these contaned areas. Each block begns wth a begnfxg statement and ends wth an endfxg statement, where X sgnals the type of the block, e.g., tuple, set, nteger, strng, constant. Between the two enclosng statements of blocks correspondng to constant areas, one fnds a sngle lne wth the constant to be prnted. For blocks of varable areas, one fnds n separate lnes the path expresson correspondng to the block (step 2), all the entres consttutng a prntng specfcaton for the type of the block (all those n the wndow of step 3), and possbly other blocks n the requred order. For the restrcted case of ths secton, the outermost block has X always equal to tuple, and encloses nner blocks wth X equal to constant, nteger, real, or strng, whch enclose no further blocks. Contnung on wth the prevous example, the map-fle generated through Frog for a fle that conssts of only row 2 of Fgure 2 s gven below: \begn{tuple} pathexp = nput delmter = <space> termnator = <return> maxelms = maxchars = default = NULL \begn{nteger} pathexp = Input.Weather.ranfall length = 3 default = -1 \end{nteger} \begn{nteger} pathexp = Input.Weather.temperature length = 4 default = -999 \end{nteger} \begn{nteger} pathexp = Input.Weather.wnd-speed length = 2 default = -1 \end{nteger} \end{tuple} In the above, we assume that f the nput od that Turtle receves at run-tme s null, then the strng NULL wll be prnted. Lkewse, f any of the Weather parts s null, then the values -1, -999, and -1 are prnted n place of ranfall amount, temperature, and wnd-speed, respectvely. Note how the nformaton gven as prntng specfcaton s repeated n the map fle. Also note that, even f the desgner specfed mappngs n an arbtrary order, n the end, Frog rearranges blocks to reflect the order of the correspondng areas n the sample nput fle. In readng the map-fle, Turtle wll generate an nput fle n the order t fnds prntable blocks, whch should be dentcal to the spatal order of the correspondng areas n the sample nput fle, e.g., frst ranfall, then temperature, then wnd-speed. 6 General Translaton Specfcaton by FROG In ths secton, we remove the restrcton of Secton 5. In partcular, we descrbe how to deal wth mappng set or ndexedset classes, and also how to deal wth s-a relatonshps, whch are features found n many applcatons. Even n the most general case, all (prntable) nformaton about the source class remans n ts leaves, but now the ntermedate structure s mportant for the overall layout of the fle, capturng larger areas of t, and cannot be flattened out as n the restrcted case of Secton 5. Ths ntermedate structure generates somorphc herarches n the hghlghted areas of the sample nput fle as well as n the correspondng map-fle blocks. Tuple classes are treated exactly as descrbed n the prevous secton for the source class, except that step 1 corresponds to explctly dentfyng a fle area correspondng to a tuple object. Hence, we do not elaborate on them any further. We dscuss each of the remanng three advanced features n a separate subsecton below. In all cases, the desgner must follow the step sequence descrbed earler, wth dfferences only n the prntng specfcaton wndow and the resultng map-fle block. In general, the areas hghlghted n step 1

6 may form a contanment herarchy n the sample nput fle. Each such area s hghlghted n dfferent color from ts mmedately enclosng area and s nterpreted wthn the latter s context. Lkewse, path expressons returned for an area at step 2 by Opossum are checked for havng the correct prefx based on the path expresson returned for the mmedately enclosng area. 6.1 Sets Vsual Language The prntng specfcaton wndow that pops up n step 3 of the sequence contans the followng entres related to sets: Delmter between the elements of the set object Number of elements n the set object Termnator for the set object Maxmum number of elements per lne Maxmum number of characters per lne Default value to be prnted f set n database s null Note that the only dfference from the entres for tuple objects s an addtonal entry for the number of elements n the set object, somethng that s not needed for the parts of a tuple object as ther number s prespecfed n the schema. After ths, desgners enter the step sequence agan once to dentfy one element of the set. Dependng on the type of the element (prmtve, tuple, set, etc.), the sequence wll proceed accordngly, based on our descrptons n the correspondng subsecton. Note that ths does not have to be repeated for every set element n the sample nput fle, as sets are unform objects, and descrbng how to prnt one of them s enough. The example of Fgures 1 and 2 contans no set classes that are to be mapped on ther own to the sample fle. The only set class, Zones, s the keyset for Plant-communty and must be mapped as part of the ndexed-set. The whole process for sets, however, s extremely smlar to that for ndexed-sets, so the example that we gve n Secton 6.2 llustrate the stuaton for sets as well Map-Fle The block nserted n a map-fle for a set begns wth a begnfsetg statement and ends wth an endfsetg statement. Between these two statements, one fnds n separate lnes the path expresson correspondng to the block, all the entres consttutng a prntng specfcaton for sets, and a block for the elements. 6.2 Indexed-Sets The followng dscusson deals wth one-dmensonal ndexedsets,.e., ndexed by a sngle collecton. The generalzaton to mult-dmensonal ndexed-sets s straghtforward, as one may vew them recursvely as one-dmensonal ndexed-sets of ndexed-sets wth one fewer dmenson Vsual Language Agan, the prntng specfcaton wndow that pops up n step 3 contans exactly the same entres as that of sets, except that now one deals wth key-element pars of the ndexed-set nstead of smply elements. (A key-element par conssts of a member of the keyset (key) and the correspondng member of the ndexed-set (element).) In Fgure 1, Plant-communty s an ndexed-set class, ndexed by the Zones set class. To specfy the approprate mappng, desgners would frst hghlght the large lght-gray area of the fle, as shown n Fgure 2, then clck on the Plantcommunty node on the schema, and fnally possbly update the entres n the prntng specfcaton wndow. In ths case, except for the ever-present default value, all that s needed s to put <return> n the delmter entry, sgnfyng that each key-element par occupes a sngle lne. After ths, desgners enter the step sequence agan once to dentfy one key-element par of the ndexed-set. Due to the specal nature of ndexed-sets, a devaton from the norm s necessary here: step 2 s skpped as there s no schema class correspondng to key-element pars. Snce the area dentfed n step 1 s wthn the ndexed-set area, however, Frog has the necessary nformaton to skp step 2, and smply proceed to step 3 by presentng a prntng specfcaton wndow. Ths s dentcal to the wndow for a tuple, as the key and the element n the par are essentally two ndvdual enttes. After that, desgners wll have to enter the step sequence twce more, once for the key and once for the element. Each case wll proceed normally, based on the types of the key and the element, respectvely. Contnung on wth the example of Fgures 1 and 2, the key-element par s dentfed n the fle by hghlghtng the frst such par, the lne startng wth Z1. In Fgure 2, ths s shown wth angled strped background. Frog wll then go drectly to step 3 to allow the desgner to specfy how the par wll be prnted. After that, the desgner wll enter the step sequence once by hghlghtng the key Z1 (lght-gray area) and clckng on the character strng class of members of Zones, and once by hghlghtng the correspondng element corn,5,45 (whte area) and clckng on the plant class, whch s the members of Plant-communty class. Fnally, the desgner wll enter the step sequence agan, once for each part of the plant tuple class (small lght-gray areas under corn, 5, and 45). The prntng specfcatons approprate for ths fle are captured n the generated map-fle below, so they are not repeated here. Note that one needs to map to a sngle keyelement par of the ndexed-set, as ndexed-sets are homogeneous collectons and each par wll be prnted n the same way. Also note that the precse shade of the background that each area receves s not mportant, as long as t s dfferent from the mmedately enclosed and the mmedately enclosng areas. Only obsolete areas have a standard shadng to dstngush them from all varable areas, whch n our example s dark gray.

7 If the ndexed-set s of the sequenced-set knd (.e., ndexed by the set of ntegers), then Frog treats t as a regular set snce for prntng purposes there s no dfference between the two. Thus, when an area s hghlghted wthn an ndexed-set area n step 1, Frog does not enter the specal process of skppng step 2 so that a key-element par s dentfed. Instead, t enters step 2 as usual, watng for the element class of the sequenced-set to be dentfed through Opossum Map-Fle The block nserted n a map-fle for an ndexed-set begns wth a begnfndexedsetg statement and ends wth an endfndexedsetg statement. Between these two statements one fnds nformaton dentcal to that of a set block. For a general ndexed-set, the enclosed block for the elements s a tuple block, whch n turns encloses two further blocks, one for the keys and one for the elements. For a sequenced-set, the enclosed block s of the type correspondng to the elements of the sequenced-set. Contnung on wth the prevous example, the map-fle block generated through Frog for the Plant-communty ndexedset based on the sample fle of Fgure 2 s gven below: \begn{ndexedset} pathexp = Input.Plant-communty delmter = <return> noelms = termnator = maxelms = maxchars = default = \begn{tuple} pathexp = delmter = <space> termnator = maxelms = maxchars = default = \begn{strng} pathexp = Input.Zones.zone-names length = 2 default = NULL \end{strng} \begn{tuple} pathexp = Input.Plant-communty.Plant delmter =, termnator = maxelms = maxchars = default = NULL \begn{strng} pathexp = Input.Plant-com.Plant.name length = default = unknown \end{strng} \begn{nteger} pathexp = Input.Plant-com.Plant.wdth length = default = -1 \end{nteger} \begn{nteger} pathexp = Input.Plant-com.Plant.heght length = default = -1 \end{nteger} \end{tuple} \end{tuple} \end{ndexedset} 6.3 Is-A Herarches Consder an object class that s the root of an s-a herarchy. When an object n that class s translated nto fle format, the resultng fle layout s dfferent dependng on the partcular subclass where the object belongs. How all the possble layouts are captured n a sngle map-fle s the topc of ths subsecton. The key problem s how to avod remappng areas n the fle that are common to objects of all subclasses. Durng the mappng process, desgners may need to use several sample nput fles, as many as there are dfferent fle layouts n the worst case. In the followng dscusson, we assume that the same object can be n two classes only f one s a subclass of the other, or f they are both superclasses of a thrd class n whch the object belongs as well (multple nhertance) Vsual Language Desgners choose to start workng on one of the sample nput fles. The step sequence for the root class of the s-a herarchy proceeds on that fle as descrbed above based on the knds of the classes mapped. Let the root area be the fle area correspondng to the root class object n the fle. When an area enclosed n the root area s hghlghted n step 1 and then assocated wth a subclass of the root class n step 2, the comparson of the two path expressons reveals the relatonshp to Frog, whch generates the approprate specal block n the map-fle. 1 Ths may be repeated recursvely for an entre s-a path from the root class down to ncreasngly specalzed subclasses. To capture a dfferent fle layout, correspondng to a dfferent path n the s-a herarchy, desgners brng up another sample fle. That second fle could be a full-fledged nput fle, or smply a fle that contans just the areas that are dfferent from the orgnal fle, or anythng n between. Step 1 of the frst sequence wth ths fle dentfes the area that has dfferent layout from the frst fle and needs mappng, whle step 2 dentfes the subclass that corresponds to ths area. Beyond ths pont, the process contnues as before. The above process s most effcent when all the elements of a superclass are mapped n assocaton to the superclass, and that mappng s shared wth all the subclasses. However, the ablty to do that depends prmarly on what s common 1 Full path expressonsn Fox nclude explctly traversals of s-a relatonshps as well, beng the connectve ndcatng such a relatonshp.

8 among the varous fle layouts. If the common elements are placed close together and n the same layout for objects n the dfferent subclasses, then they can be mapped only once. If they are nterleaved wth elements that dffer from subclass to subclass, or they are lad out dfferently for objects of dfferent subclasses, then the common part may become mnmal or even nonexstent. Fnally, for multple nhertance, mappng for the common subclass must repeat the mappng for the elements of all but one of ts mmedate superclasses. As an example, we present a case that s relatvely smple, whch can be dealt wth wthout usng many of the features of the general algorthm descrbed above and n fact requres a sngle sample nput fle. Consder agan the schema n Fgure 1, where the Weather class s the root of an s-a herarchy that ncludes multple nhertance. The orgnal Fgure 2 has a general Weather object, snce t ncludes none of the specalzed attrbutes. Fgure 3 has a Dsaster Weather object, snce t ncludes all fve of the possble Weather attrbutes. Desgners could use the general algorthm to map each subclass separately to an approprate nput fle. However, n ths case, all mappngs may be specfed by smply usng the fle n Fgure 3, whch contans a Dsaster Weather object. Because the subschema that deals wth Weather contans only a tuple class wth has-part relatonshps, as n Secton 5, mappng may proceed drectly wth the leaf classes. After mappng ranfall, temperature, and wnd-speed, as before, the desgner may enter the step sequence by hghlghtngthe area that contans NW and then clckng on the character-strng class connected to the Wndy class. The path expresson returned ndcates that ths s for a subclass of Weather, whch provdes enough nformaton for Turtle to do the approprate translaton n each case. The desgner may proceed smlarly to do the mappng for humdty, by hghlghtng the area that contans 52 n the sample nput fle Map-Fle The block nserted n a map-fle for any subclass begns wth a begnfsag statement and ends wth an endfsag statement. It s always wthn the block of a superclass of t (most probably ts mmedate superclass), whch may also be an sa block. The blocks of sblng subclasses (.e., wth a common mmedate superclass) are enclosed n the same superclass block but not n each other. Contnung on wth the prevous example, f we only concentrate on row 2 of the fle n Fgure 3, the map-fle block generated through Frog for the entre Weather s-a herarchy s the one shown n Secton 5.2, but wth the followng addtonal blocks placed mmedately before the fnal endftupleg. \begn{strng} pathexp = Input.Weather@Wndy.wnd-drecton length = 2 default = \end{strng} \begn{nteger} pathexp = Input.Weather@Dry.humdty length = 3 default = \end{nteger} Note that by havng the default value for wnd-drecton and humdty be empty, f Turtle deals wth a Wndy object, the path expresson leadng to humdty wll return a null object and therefore nothng wll be prnted, as t should. Smlarly for the wnd-drecton attrbute f Turtle deals wth a Dry object. The layouts of the nput fles have permtted the desgner here to work wth a sngle sample nput fle and generate no sa blocks. 7 General Operaton of TURTLE When the external program needs to be executed wth a specfc nput expressed as a Moose object, an nput fle wth the approprate layout has to be generated and flled wth the rght values from the database object. Turtle accomplshes ths based on the map-fle that has been generated for that purpose and the od of the nput object. As mentoned earler, the blocks n the map fle are placed n the order ther correspondng areas appear n the fle, except for sblng s-a blocks, whch appear n an arbtrary order between them wthn the same superclass block. For every block, Turtle sends to the database server a query that essentally processes the block s path expresson, retrevng the object(s) that correspond to the fle area related to the block, f any. If the path expresson leads to a prmtve class, the retreved value(s) should actually be placed n the nput fle n that area. Ths placement s done by Turtle nterpretng the contents of the prntng specfcaton wndows of the area and all ts enclosng areas, and dervng the approprate prntng format for the value(s). When s-a blocks are nested one nsde another, they are processed as all other blocks, the outer before the nner. When they are sblngs, they are processed n the order they appear. In the absence of multple nhertance, at most one of the sblng blocks may have ts path expresson return a nonempty answer, snce objects cannot appear n multple classes wthout a common subclass. In the presence of multple nhertance, multple sblng blocks may return nonempty answers, n whch case Turtle must look further nsde for the block correspondng to the common subclass to determne whch of the sharng superclass blocks should be followed. 8 Status, Experence, and Other Uses Prelmnary versons of Frog and Turtle have been mplemented n C++ n the context of the ZOO system. They do not deal wth ndexed-sets or s-a herarches but support all the remanng features descrbed above. The mssng tems are currently under mplementaton. These early versons have

9 Rows WEATHER constant area NW 52 ranfall temperature wnd_speed wnd_drecton humdty PLANT COMMUNITY constant area Z1 corn, 5, 45 Z2 rye, 7, 30 Z3 corn, 5, 45 Z4 wheat, 4, 38 SOIL phosphorus 13 Zone obsolete area name, wdth, heght Sample Input Fle Fgure 3: An example nput fle for Dsaster Weather been used by scentsts n the Sol Scences Department and the Bochemstry Department of the Unv. of Wsconsn, to automate generaton of nput fles to ther experments. Ther nput fles are very large (contanng hundreds of parameters and many constants), and the use of tools lke Frog and Turtle has sgnfcantly smplfed the task of generatng these fles. In addton to communcatng wth external programs, Frog and Turtle can also be used as reportng tools. For nstance, users may lke to prnt copes of detals on each plant specmen they studed. A sample fle wth the desred layout s frst created, Frog s then used to map the correspondng database classes to the fle elements, and fnally Turtle generate the requred copes. Frog and Turtle can also be used for data mgraton between heterogeneous DBMSs. Data from the source system can be frst transferred n a fle based on the layout expected by the target system, whch can then read and translate the fle nto ts nternal representaton. 9 Related Work Most commercal DBMSs provde utltes (such as report wrters) to transfer database data nto fles n user-specfed layouts. There are also numerous tools that enable nformaton exchange between dfferent systems va fles. Although often smlar n functonalty to Frog and Turtle, these tools usually work n restrcted domans and do not emphasze vsual nteracton lke Frog and Turtle. We frst menton such tools that have been developed n the context of scentfc databases, as ths has been our prmary motvaton as well. The Computatonal Chemstry Database project [3] provdes nter-operablty between computatonal chemstry applcatons by encapsulatng the applcatons and data n an object-orented database. The system s based on the framework developed by Cushng et al. [2] for nterfacng experments wth a database and has been demonstrated to serve the needs of computatonal chemsts. The system translates data between databases and fles that have well defned layouts. The layout of the fles s specfed usng CCIL and CCOL (Computatonal Chemstry Input/Output Language). These are declaratve report-wrtng languages that are talored towards the requrements of chemstry applcatons. Chpperfeld et al. address the ssues of storng genome data nto a relatonal database, mgratng data from other sources nto that database, and retrevng data from that database [1]. The database can be nterfaced wth applcatons that process the data n well-defned formats for the genome. Both the above tools were developed to provde nteroperablty between specfc applcatons, so they work prmarly wthn the specfc domans for whch they were bult. Our system, on the other hand, s general-purpose and hence well-suted to a broader set of applcatons. In addton, t offers a vsual user nterface that sgnfcantly affects usablty. Smlar efforts exst outsde the scentfc database realm as well. In some approaches, the contents of an Asc fle are descrbed by means of a grammar. The OBST persstent object management system [11] (developed as part of the STONE [9] envronment) supports a tool called the Unversal Structurer and Flattener (STF). If the structure of a fle can be descrbed by a context-free grammar, STF can generate a parser (structurer) and unparser (flattener) for the fle. The structurer parses the contents of the fle accordng to the grammar and creates an object structure and the flattener does the nverse operaton. The grammar fle resembles the mapfle generated by Frog. The dsadvantage of ths approach s that the scentst has to understand the language n whch grammars are specfed and then desgn the grammar desred. Also, the grammar s requred to be context-free, whch mposes restrctons on the fles that can be translated. The Ontos OODBMS has mechansms to provde a un-

10 fed vew of data resdng n dfferent databases. In partcular, Ontos allows product descrptons to be transferred between manufacturng systems and Ontos databases. Followng the STEP standard, the structure (schema) of product data s specfed usng the EXPRESS language [4, 10], whch s translated nto an Ontos schema. Any product data fle s an nstance of a gven EXPRESS schema. Users annotate such a fle wth path expressons on the correspondng schema, whch can then be translated nto an Ontos object. Although adequate and/or desrable n manufacturng applcatons, havng the users annotate every sngle data fle wth the approprate schema nformaton would have been a serous problem n the ZOO envronment. That s why we followed the Frog/Turtle approach. Fnally, SAS [6] and SPSS [7] are establshed statstcal tools that allow fle structures to be descrbed by FORTRANlke commands for statstcal analyses, report wrtng, and data-fle buldng. They are more powerful than Frog/Turtle because they can handle arbtrary processng of data as t s translated. On the other hand, as wth all the tools mentoned n ths secton, they are not as easy to use as Frog/Turtle, snce they do not offer a vsual style of nteracton. 10 Conclusons We have presented Frog and Turtle, two tools that can be used to facltate the translaton of objects n an OODBMS nto fle format, so that fles can be generated and sent as nput to external programs. Frog s used off-lne to map classes n OO schemas to areas of sample nput fles and specfy how database values are to be lad out and prnted n a fle. The result s a map-fle wth all the approprate nformaton. Turtle s used at run-tme to translate gven database objects based on the map-fle generated by Frog. Although we dd not descrbe ths at all n ths paper, the same mechansms are used for mappng (sub)-schemas to fles that are to be read n by the OODBMS and translated nto database objects. The only dfference s that, n ths case, we deal wth readng specfcatons (not prntng), whch occasonally contan more entres, e.g., for set objects. There are several ssues that are part of our current and future work. Frst, the mplementaton of the second verson of the system contnues. Second, as descrbed n ths paper, all blocks n the map fle contan path expressons rooted at the source class beng mapped. Ths s qute neffcent, snce these path expressons share long prefxes, whch are processed agan and agan. We are currently nvestgatng varous ways to speed up the entre process. Thrd, we plan to nvestgate f prntng specfcatons could potentally contan nconsstences, n whch case, algorthms should be developed for checkng map-fles for such nconsstences. References [1] M. Chpperfeld et al. Growth of data n the genome data base snce ccm92 and methods for access. In Proc. Human Genome Mappng, pages 3 5, [2] J. Cushng et al. Object-orented database support for computatonal chemstry. In H. Hnterberger and J.C. French, edtors, Proc. 6th Internatonal Workng Conference on Statstcal and Scentfc Database Management, Zurch, Swtzerland, September [3] J. Cushng, D. Maer, M. Rao, D. Abel, D. Feller, and D. DeVaney. Computatonal proxes: Modelng scentfc applcatons n object databases. In Proc. 7th Internatonal Conference on Statstcal and Scentfc Database Management, Charlottesvlle, VA, September [4] ISO Group. ISO 10303: Gudelnes for the development and approval of STEP applcaton protocols, verson 1.0. Techncal Report ISO TC184/SC4/WG4NS4(P5), ISO, February [5] E. Haber, Y. Ioannds, and M. Lvny. Opossum: Desktop schema management through customzable vsualzaton. In Proc. 21st Internatonal VLDB Conference, pages , Zurch, Swtzerland, September [6] SAS Insttute Inc. SAS Language Reference - Verson [7] SPSS Inc. SPSS User s gude, 3rd edton [8] Y. Ioannds, M. Lvny, and S. Gupta. The ZOO desktop experment management envronment, February Submtted for publcaton. [9] C. Lewerentz and E. Casas. STONE: A short overvew. Techncal Report FZI.040.1, Forschungszentrum Informatk (FZI), Karlsruhe, Germany, May [10] D. Schenck. Exchange of product model data - part 11: The Express language. Techncal Report TC184/SC4 Document NC64, ISO, July [11] J. Uhl et al. The object management system of STONE - OBST release 3.2. Techncal Report FZI.027.1, Forschungszentrum Informatk (FZI), Karlsruhe, Germany, October [12] J. Wener and Y. Ioannds. A Moose and a Fox can ad scentsts wth data management problems. In Proc. 4th Internatonal Workshop on Database Programmng Languages, pages , New York, NY, August 1993.