Brad A. Myers Human Computer Interaction Institute Carnegie Mellon University Pittsburgh, PA

Transcription

1 PAPERS CHI APRIL 1998 Scripting Graphica Appications ABSTRACT Writing scripts (often caed macros ) can be hepfu for automating repetitive tasks. Scripting faciities for text editors ike Emacs and Microsoft Word have been widey used and avaiabe. However, for graphica appications, scripting has been tried many times but has never been successfu. This is mainy due to the data description probem of determining how to generaize the particuar objects seected at demonstration time. Previous systems have mosty tried to sove this using inferencing, but this has a number of probems, incuding guessing wrong and providing appropriate feedback and contro to users. Therefore, the Topaz framework does not use inferencing and instead aows the user to specify how the appropriate objects shoud be found, This is achieved by recording changes to which objects are seected and searches for objects, so that scripts can be written with respect to the seected object, in the same way as Emacs keyboard macros. Furthermore, a vaues can be expicity generaized in a number of ways, and scripts can be invoked as a resut of other commands. By everaging off of Amuet s command object architecture, programmers get these capabiities for free in their appications. The resut is that much more sophisticated scripting capabiities avaiabe in appications with no extra work for programmers. Keywords: Scripting, Macros, Programming by Demonstration (PBD), Command Objects, Tookits, User Interface Deveopment Environments, Amuet INTRODUCTION Creating scripts (aso caed macros ) for textua appications ike text editors and spreadsheets has a ong and very successfu history. These scripts are important for automating repetitive tasks that are so common in direct manipuation interfaces. They can aso be usefu for creating new commands and for customizing generic appications to be more effective for speciaized tasks (such as making a drawing program ike MacDraw more efficient for creating charts). Permission to make digit&hard copies of a or part of this mr&ria for persona or cassroom use is granted without f= provided that the copks.are not made or diibuted for profit or commercia advanuge. the copyright notice. the tite of the pubication and its date appear, and notice is given that copyright is by permission of the ACM, Inc. To copy othewise, to repubish, to post on servem or to rediibute to Iis& requires specific permission antior fee. CHI 98 Los Angees CA USA copyi!# s5.00 Brad A. Myers Human Computer Interaction Institute Carnegie Meon University Pittsburgh, PA bam@cs.cmu.edu by Demonstration To create a script, the user typicay goes into record mode, then performs some commands, which operate normay in addition to being recorded. The recorded script can then be re-executed ater in different contexts. For exampe, the keyboard macro faciity of the Emacs text editor [13] is easy to use and extremey usefu for manipuating the text. Many Emacs users caim that such faciities are essentia to use the editor efficienty, especiay when making repetitive changes. Scripting faciity aso exist in Microsoft Word using Visua Basic, and spreadsheets have a ong history of creating scripts by exampe. However, for graphica appications, such scripting faciities have mosty been unsuccessfu. Graphica appications here refers to programs such as drawing editors, CAD programs, graphica programming anguages, and iconic manipuation programs such as visua shes which are graphica interfaces to the fie system (ike the Macintosh Finder or the Windows Desktop). We have anayzed the fundamenta features that aow text editors to be successfuy scripted by demonstration and then incorporated these features into a graphica editor framework caed Topaz. Topaz stands for panscripts of programs activated with zea. The resut is that powerfu and sophisticated scripts can be created by exampe in any graphica program. This is in contrast to specia-purpose scripting anguages ike Chimera [3] that ony work for a drawing editor. With Topaz, the user is abe to: change which object is seected by moving forwards and backwards through the objects in a variety of ways, recording the change of seection in a script, search for objects by matching on various properties or by ocation, and cause the found object to become seected (this is a generaization of graphica search and repace [4]), execute subsequent commands with respect to the currenty seected objects, so that the next time the script runs, it wi operate on newy seected objects, generaize the parameters of the operations (coors, ocations, numbers, strings, etc.) in a variety of ways, so the vaues can be computed at run-time, execute the script a specified number of times or continuousy unti an error occurs (such as a search faiing), specify that the script shoud be automaticay invoked before or after other commands are executed. 534

2 Insert a new node in a row of nodes and move a of the nodes that are on the right of the insert point further to the right to make room. Create a bar chart by making a row of rectanges whose heights depend on a given a ist of numbers. Whenever a string is created, create a rectange that is 10 pixes bigger than the string and center the rectange behind the string. Perform the same edits to a set of graphics fies, for exampe to repace a uses of an od ogo picture with a new picture, and move a of the other objects a few pixes to the right and down to make room. And many more.... Figure 1. A drawing program created using Topaz showing the resut of a script which subdivides a triange into 3 smaer trianges, appied 13 times. This is caed a Sietpinski Gasket Figure 6 shows the code of the script. Of course, a particuar appication might have some of these commands buit-in, but the goa here is to aow end users, who wi not necessariy know how to program, to construct these kinds of scripts by demonstration when the appication does not have the buit-in command that they need. We have impemented the Topaz framework using the Amuet tookit [12], everaging off of Amuet s command object architecture []. The resut is that graphica appications get these sophisticated scripting faciities without any extra code in their appications. At this point, these faciities have been tested with a drawing program and a circuit editor, and we are working on a visua she. WHY ARE TEXT EDITORS EASIER TO SCRIPT THAN GRAPHICAL APPLICATIONS? Figure 2. A sampe circuit design program created using Topaz showing a circuit. The inset picture is the resut of a script that converts an And gate and a Not gate into two Not gates and an Or gate and reconnects the wires. The resut is that users can create scripts by demonstration that perform such actions as repacing objects with patterns, performing repetitive edits, and creating graphica abbreviations. Specific exampes of scripts created with Topaz are to: e Buid interesting patterns ie the Sierpinski Gasket of Figure 1. Repace an And gate and a Not gate with the equivaent circuit according to DeMorgan s aw, connecting a the wires appropriatey, as shown in Fi,aure 2. Put a drop-shadow beneath any type of seected object Put an arch inside each rectange as in Mondrian [a. The user has fu contro over the whether the width of the side pieces and the height of the top is proportiona to the size of the rectange or constant. In a text editor ike Emacs and Microsoft Word, most operations are performed with respect to the cursor, and there are a wide variety of ways to move the cursor using the keyboard and the mouse. These incude moving the cursor forward and backward by characters, by words, by ines, or by sentences. The movement operations work in text documents because the content is an ordered sequence of characters, so forward and backwards are meaningfu. Furthermore, for text with any kind of structure (incuding program code, content ists, etc.) moving forward by a ine or by a certain number of words wi correspond to moving by a semanticay meaningfu unit. Scripts for repetitive actions take advantage of this ordering. A very common idiom used in Emacs keyboard macros is to move or search to the next occurrence of a pattern, and then perform operations on it. This script can then be repeated unti a occurrences are processed. It is interesting to note that many powerfu scripts can be written in this way without using conditionas or iterations, other than repeating the entire script unti a search or move fais. There are no equivaent operations in graphica programs. Just recording the ow-eve input events, which works we for keyboard events, does not work for mouse events since the specific ocation of the mouse is recorded when the buttons are cicked, and when the script is repayed, often 535

3 PAPERS CHI APRIL1998 the wrong object is at that ocation. Indeed Microsoft Word turns off the use of the mouse to move the cursor whie recording a script. Many attempts to provide scripting in graphica programs therefore try to infer the meaning of the mouse ocations (that is, what is at this mouse ocation, and why was that object cicked on?). Other programs try to match the objects that were used in different executions of the script to try to create generaizations. These programs must use heuristics which means that the system can guess incorrecty, and often wi not infer the correct program., \ Figure 3. Circe 1 is seected. Moving the seection to the right wi seect rectange 2, but then shoud ova 3 or 8 be seected? Currenty, when moving to the right, Topaz seects objects in the order 1,2,3,4,5. But then moving the seection eft from 5 seects 5,4,6,7. The approach taken in Topaz is to aow the user to specify how to find the correct objects using capabiities simiar to those found in text editors, rather than trying to infer the generaizations. In graphics programs, the seection (often shown by back squares around objects as in Figure 3) corresponds to the cursor in text editors, and most commands operate on the seected set of objects. The innovation in Topaz is that users can change which objects are seected in graphica appications in a variety of ways, and have these recorded in a script. Other innovations in Topaz are that there are various ways that the user can expicity generaize a script so it wi work in new contexts, and the abiity to invoke a script automaticay before or after other commands execute, as proposed in [2]- MOVING THE GRAPHICAL SELECTION Text has a natura order, so commands ike forward and backward are meaningfu. However, in a graphica appication, it is not so obvious what the order for a objects woud be. The requirements for the ordering is that it have a we-defined first and ast object, moving forward from the first object shoud go through every object exacty once, and moving backwards from the ast object shoud seect a the objects in the reverse order as forwards. It woud aso be good if the order made sense to users. However, in many cases, the particuar order does not matter-it is often ony important that each object be visited exacty once. Our first idea was to go top-to-bottom, eft-to-right, but this is not we-defined for some ayouts of objects, and the backwards order is often not the reverse of the forwards order (see Figure 3). Therefore, we decided to make the primary order for traversing objects be the dispay Z order from back to front. This has a number of advantages: it is we defined, reversibe, and usuay corresponds to the chronoogica order in which objects were created (oder objects are further back uness the user has expicity changed the order with a To- Top or To-Bottom command). A sophisticated user can aso take advantage of this order in scripts. For exampe, to make sure that a script that creates arch inside of rectanges does not create arches inside the arches themseves, the script might start at the end of the ist and move backwards. Since new rectanges are aways created after the current end of the ist, the seection wi never get to the rectanges of the arch. The current user interface for moving the seection is that the HOME key on the keyboard seects the first object, the END key seects the ast object, and the eft and right arrow keys seect the next and previous objects in the Z order. If the shift key is hed down, then the new object is added to the seection, in the same way as the shift key works with the arrow keys for text seection in Microsoft Word. A of the seection moving operations beep when a movement is not possibe because there are no more objects on which to operate, and this stops the execution of the script. There are some cases where it is important to find the next object in a graphica direction, so there are commands to seect the next object to the eft, right, up, down, inside or outside of the current object. These operations find the nearest object in that direction to the seected object that overaps in the other dimension, so the reverse direction does not necessariy choose the same objects, as shown in Figure 3. These operations are avaiabe as buttons at the bottom of the search window (Figure 4). SEARCHING FOR OBJECTS Often it is important for scripts to ony execute on specific objects. Previous demonstrationa graphica systems have often tried to infer the properties from one or more exampes, which is error-prone. Instead, Topaz aows the user to search for the appropriate object. This graphica search was introduced in 1988 [4] but sti is not avaiabe even in sophisticated graphics editors ike Adobe Iustrator, athough it is provided by some CAD programs. Topaz extends this searching to make it work in any graphica appication, not just a drawing program. A objects in Amuet have a we-defined protoco for querying their properties. Topaz uses this to find what types of objects can be created and what parameters are avaiabe for each type. This does not require any new code in the appication. Topaz automaticay constructs a diaog box that aows the user to choose which properties shoud be used 536

4 C-II APRL 1998 PAPERS for a search, as shown in Figure 4. Of course, this diaog box coud be made to ook much nicer if it was designed by hand, but we wanted to minimize the amount of custom code needed in each appication, so we used a straightforward automatic mechanism to generate the search diaog. This is why it uses interna names (iie FILL-STYLE) and a ndive ayout ~~Sekct &mmand to Undo or Rqxac Figure 5. The undo history diaog box, in which previous commands can be seected for undo, repeat or scripting. Recording of seections and scroing can aso be turned on and off. Here, the commands for the script to create the trianges of Figure 1 are seected. Figure 4. The search (find) diaog box automaticay created by Topaz for the graphica editor of F&we 1. The vaues from an arc (ova) have been oaded. To perform a search, the user can type in a vaue, or oad the vaues from an existing object There are eight ways to search for the next object, using the buttons at the bottom of the window. Find Next and Previous search in the 3 order, and the others find in graphica ayout order. A searches start Corn the current seection, or if nothing is seected, then from the first item in that order. For exampe, when searching to the Right, the first item is the eft-most. Find Inside finds the back-most (in 2 order) object on top of the seected object that is entirey inside it. This is usefu for finding encosed objects, ike the abe inside a box. Find Outside finds the front-most object behind the seection and competey surrounding it, and is usefid for going Corn a abe to its box. If nothing is seected, Find Inside starts from the back-most object, and Find Outside starts from the front-most When searching, Topaz ony matches on the properties that have their checkboxes seected, so if nothing is checked, the search is through a objects. Seecting various properties supports more compex searches, such as find the wire which is down from the seected object which was needed for the script for Figure 2. Figure 6. The commands of Figure 5 dispayed in the scripting window, with some of the parameters aready generaized into pacehoders. USER INTERFACE FOR SCRIPTING Unike most other demonstrationa scripting faciities where the user has to think ahead that the next operations shoud be in a script, Topaz aows scripts to be created by seecting the desired commands from the ist of commands dispayed in the undo diaog box (see Figure 5). This ist shows a the previous commands that have been executed. As reported earier [ 11, the user can seect any command in this ist for seective undo or seective repeat. 537

5 PAPERS CHI APRIL 1998 The new feature added by Topaz is the abiity to seect a sequence of commands to be incuded in a script. (Seecting previous commands aso was avaiabe for the graphica histories in Chimera [3].) Topaz aows noncontiguous sequences of commands to be seected (using the standard shift- and contro- cicking), so that unike other systems, the sequence of commands for the script do not have to be executed fawessy without errors. Once seected, the commands are expanded to show a their parameters, and are dispayed in the scripting window (Figure 6). Now, the user can edit the script in various ways (discussed beow) How the script wi be invoked can be specified (the script of Figure 6 wi be executed when the user hits the P8 keyboard key). When the script is ready, it can be executed, saved, or removed (deeted). EDITING AND DEBUGGING SCRIPTS Many previous programming-by-demonstration systems seem to have assumed that a scripts woud work the first time and never need to be changed, since there was no way to edit the recorded script (a notabe exception was Chimera [3], which had nice editing faciities). In fact, some systems do not even have a reasonabe representation of the recorded script that can be viewed. Topaz provides fu editing of the script, incuding seecting the commands and deeting, cutting, copying, and pasting them. Aso, commands in the undo diaog box (Figure 5) can be seected and inserted before or after any statement in the script to add new commands. In order to debug the script, the user can seect specific commands and execute just those commands, or singe step through the script one command at a time. The resuts of the commands wi be visibe in the main appication window, so the user can check if they are operating correcty. Because a commands can be undone, incuding the execution of the script itsef, it is easy to back out of any operations that are incorrect. An important capabiity not avaiabe in other systems that is this editing makes possibe, is the abiity to demmstrute new commands to be added to the midde of a script. The user can singe step the script to the appropriate point, then execute the desired new commands, which wi appear in the undo diaog box. These commands can be seected and inserted into the script at the current point. Then the user can continue singe stepping the script to make sure that the subsequent script commands sti work correcty. GENERALIZATION OF PARAMETERS An important feature of Topaz is the abiity to generaize the parameters of operations. This is important to have a different object, position, or vaue used when the script is run, rather than the specific constant object used when the script was demonstrated. The user can seect any vaue dispayed in the script window and doube-cick (or use the generaize menu item in the Edit menu) to bring up a diaog box. After generaizing, the vaue dispayed in the script is repaced with a descriptive pacehoder. Bringing up the diaog box for a pacehoder wi aow it to be edited, or changed back into a constant. This is reated to the data description property sheets in the SmaStar visua she [], but here they are domain independent. If an appication has a specia type of vaue, the programmer can add a new diaog box to hande it, but the buit-in diaog boxes seem sufficient for many appications. Aso, the diaog boxes have a custom option that wi in the future ink to a programming subsystem where any expression for computing the vaues can be entered in a anguage ike JavaScript or Visua Basic. However, we have found the buit-in options to be sufficient for most scripts. Figure 7. The diaog box for generaizing objects, with the scrint window in the background. There are three buit-in diaog boxes: one for generaizing objects, one for positions, and one for a other vaues. The diaog box for generaizing objects (see Figure 7) ets the user pick how to get the object when the script runs: Use a constant object (which defauts to the origina object, but the user can type a new vaue). Use whatever objects are seected at that point in the script. Often the user wi arrange for the appropriate object to be seected before executing an operation. Pause the script and ask the user to seect some objects. Use the object or objects that are returned or operated on by a previous operation in the script. For exampe, a resize operation might be generaized to operate on the object returned by a previous create command. The user specifies which previous command to use by seecting the command in Figure 6 and cicking the Load button in Figure 7. There are two options for this, because many commands return a variabe number of objects (e.g., paste, dupicate or seect-a) and there must be a way to specify that subsequent commands operate on a of the objects, no matter how many, or on a particuar object (e.g., the first one) from the set. Figure 7 shows that the cicked on object is used in three paces in this script. The one the user actuay cicked on is shown in green, and a other uses are shown in yeow. 538

6 CH APRIL 1998 The top of the generaizing diaog box aows a of these uses to be repaced at the same time, or just the specific one that the user cicked on. Topaz performs one generaization automaticay. When a set of commands is brought into the script window, Topaz first checks to see if any of the commands are the kind that create a new set of objects. This incudes create commands (incuding Paste and Dupicate) as we as seection commands that define a set of objects. Topaz searches for any subsequent uses of these objects, and if found, repaces the occurrences with pacehoders that refer to the resuts of the create commands. This repacement is made automaticay because in amost every script, if an object is created and then manipuated, when the script is run, the manipuation shoud operate on the newy created object and not on the origina constant object. For exampe, a of the object gen-,eraizations for the script of Figure 6 were automaticay performed by this mechanism. If the user reay wants to use tie origina, constant object, the automaticay generated pacehoder can be edited using the standard diaog box (Figure 7) back to the constant vaue, but this has never been necessary so far. PAE EFiS the other s width). When an option is chosen that depends on another object, the user can seect which object to use. The norma choice is whatever is seected when this command runs. A other types of vaues use a diaog box which aows the vaue to be constant, the current vaue from the paette if there is a paette registered for this type of vaue (ike the coor paette), a vaue chosen from a ist, a vaue that the user types, or the vaue computed by a previous command. One use for the ist of vaues is to create a script that wi process a number of fies. The fie Open and Save-As commands have the fiename as their parameter, so the user can record a script with open and save in them, and then generaize the fiename parameter to either be a ist of fienames or to ask the user for the fienames. Generaizing the vaues integrates we with the Search diaog box, since the search command uses as its parameters the vaues that were searched for. The user can therefore generaize parameters for a search in the same way as any other command, for exampe to search for the next object whose coor is the same as the seected object, rather than some constant coor. INVOCATION OF SCRIPTS Topaz supports a variety of ways to invoke a script. First, the Execute button on the script window (bottom of Figure 6) can be used. Second, Topaz supports the conventiona ways to invoke scripts, using a keyboard acceerator or by putting the script into a menu of the appication. In the future, scripts wi be abe to be executed at a certain time (for exampe to ceanup a disk every night). - Figure S. The diaog box for generaizing ocations, aong with the pop-up menus for setting the eft, top, width and height. The picture in the ower eft of the diaog box shows the resut of the Left and Top seections. Figure 8 shows the diaog box for generaizing ocations. The user can cick on the check boxes at the eft of the diaog box to choose which parameters of the ocation to generaize. The options for the eft coordinate are to be constant, a difference from the od vaue, a vaue specified by the user at run time either by cicking in the picture or typing, or a vaue computed with respect to a diierent object either by being to the right outside, right inside, center, Iefi inside or eft outside. If the center is seected, then the text input fied is for the percent of the way across (the defaut is 50% which is in the center, but the user coud type another vaue such as 33% to make the object be /3 of the way across). For a other reationships, the text box is the offset from the other object The top properties are simiar. For the width and height, the bottom two icons are to make it depend on another object by offset (e-g., my width is the other object s width t- 10) or percent (my width is 50% of Figure 9. The diaog box for specifying how to invoke scripts. The Text-Create command description was automaticay entered when the user cicked on the Load hutton. Topaz aso supports a nove way to invoke scripts: before or after any other command is executed. This idea was proposed earier [2], but it has never previousy been impemented in any system. The idea is that the user not ony can demonstrate the script to be executed, but aso which commands the script shoud run after or before. For exampe, after demonstrating a script to surround an object with 539

7 PAPERS CHI APRIL 1998 a rectange, the user brings up the invocation diaog box of Figure 9. Next, the user seects the After option, then demonstrates a Text-Create operation and seects it in the main undo diaog box (Figure 5) (or ese the user can just seect any previousy executed Text-Create operation). Finay, the user hits the Load button in Figure 9 which oads the command description into the fied. By defaut, a the parameters to the command are generaized so the script wi be invoked whenever any Text-Create command is executed. This wi put a rectange behind any strings created. Aternativey, the user can seect any parameter and use the origina vaue of the exampe or any of the other generaizations discussed in the previous sections. This provides the abiity to create many intriguing scripts. For exampe, graphica abbreviations can be defined such as whenever the text ogo is created, deete it and put in the ogo picture read from a fie. Since scroing operations can be recorded, a script to scro down after a search coud be created. To further the anaogy with Emacs, scripts can be executed a specified number of times (using the count fied at the bottom of Figure 6). The user can type a vaue or, ike in Emacs, U wi mutipy the current count vaue by 4. For many scripts, the user wi pick some reahy arge number of times to execute, because, aso ie Emacs, a scripts are stopped if anything causes a beep (the ow-eve beep function was modified to set a goba fag if a beep happens, and Topaz cears the fag before executing any script and checks the fag before each operation). Error diaogs aways beep, and searches and movements of the seection causes a beep if they fai, and nothing beeps when everything is OK, so this is a good heuristic for stopping script execution. Scripts are represented as a command ike any other command, and are isted in the undo diaog box, so the execution of a script can be undone, repeated, or even incuded in other scripts. The count of the number of times to execute is the parameter for a script (which can be generaized), so a recursive script coud even be written that woud stop when the count got to zero or when there was a beep. IMPLEMENTATION An important feature of Topaz is that it requires amost no work from the appication deveoper to use scripting, if the appication is created using the Amuet framework Amuet requires that a operations be encoded into the methods and data vaues of a command object [ 11. Due to the way that these command objects are designed, they aready provide Topaz with most of the information it needs. Command objects have a Do method that performs the operation, and an Undo method for undoing. They aso contain methods to support seective-repeat and seective-undo which are when the user seects the command in the ist of Figure 5 and asks for it to be repeated or undone in the current context. The scripting mechanism uses this seectiverepeat to execute each command of the ist. Seective-undo is used when the script itsef is undone-it just undoes each of the component commands in the script. The seectiverepeat has a companion method that tests whether the command can be executed in the current context, and this is used by the script before each command is executed to verify that it can be executed, and if not, an appropriate error message is provided. This makes it safe for Topaz to aow the user to insert or deete arbitrary commands in the script, since at run-time there wi be a check to make sure that each command can execute. Command objects aso adhere to a standard protoco for describing their parameters and generated vaues, so Topaz can inspect, dispay and generaize the vaues of any command. Using the seective repeat mechanism, instead of just invoking the origina Do method again, has a number of advantages. The Do method does not take any parameters, since the vaues to be used come from the seected object, the paette, and pop-up diaog boxes that ask the user. This is a very annoying feature of the scripts in Microsoft Word-they keep popping up the diaog boxes when the script is run. Pursuit [9] used me&diaog boxes to aow the user to specify whether the diaog box shoud appear at run time, and if so, what parts shoud be fied in by the user, but this required that Pursuit parse and understand the diaog boxes. By using the seective repeat method, the parameters can be passed to the operation directy since usuay Topaz computes the vaues. Of course, it woud aso be usefu to et the user pop up the origina diaog boxes when desirabe, by using the origina Do method of the command rather than the seective-repeat. The command object architecture aso heps Topaz record the scripts at the appropriate eve. Scripts recorded at the mouse-movement-eve fai because objects are not at the same pace the next time. Appications written in Amuet must aready encode the semantics of their operations into command objects, so recording at this eve aows Topaz to create robust, repeatabe and generaizabe scripts without requiring Topaz to try to infer the meaning of the operations or of mouse events. Aso, users can invoke commands in any way that is convenient (from the mouse, menus or keyboard acceerators) and these are recorded the same way in scripts, since they a use the same command object. Amuet s buit-in save and oad mechanism aows appications to save and oad their fies with a minimum of code. A that is necessary is to register each of the main types that the user can create, and the important parameters of those types. Topaz takes advantage of this information to automaticay construct the Search window, so again no extra work is required of the programmer. Most graphica appications use Amuet s seection handes widget, and so the seection moving operations and searching come for free when this widget is used. Note that a Repace operation as in [4] coud not be added without new code in the appication, because Topaz woud not know how to create new objects. In the current design, Topaz can 540

8 inspect a of the graphica objects, but it can ony change the objects by re-executing commands that have aready been demonstrated by the user. f01 extra code in the appication is optiona. The programmer shoud register the paettes in the appication, so that the vaue generaization can te if the seected vaue can be computed from a paette. Aso, the main menubars shoud be registered with Topaz so it wi know how to add scripts to the menus if requested by the user (Figure 9). STATUS AND FUTURE WORK Topaz is mosty working, and has been integrated with a drawing program and a sma circuit design program. The main hoe is the ack of a genera-purpose pro@unming anguage for the custom generaization options. We woud aso ie to investigate writing scripts that work across mutipe appications. As we.get more experience with users and test Topaz with a wider variety of appications, we wi continuousy refine the interface. RELATED WORK The main infuence on Topaz is Emacs [13], and we have tried to provide the key features of Emacs to graphica editors, which has not been done previousy- Vmacs [SJ was an eary attempt to appy Emacs ideas to graphics, but it did not support any scripting by exampe, and instead concentrated on parsing of freehand drawings. The idea of diaog boxes to generaii parameters in scripts was used in the SmaStar [1] visua she, and Topaz provides this capabiity for any graphica appication. Graphica search was introduced in 141 for a graphics editor, and Topaz aows it to be used in scripts and for mutipe types of appications. Topaz is aso the first to aow the generaization of the parameters of the search. Chimera [3] supported graphica histories of operations in a graphics editor, and aowed commands to be seected for scripts, but supported ony rudimentary generaizations. There are many previous exampes of scripting by demonstration in graphica appications, incuding Peridot [o], h{etamouse 171, Mondrian [6], Pursuit [9], Gamut [S], etc., but most of these concentrated on inferencing agorithms rather than providing sophisticated contros to users. CONCLUSIONS Topaz aows the user to construct sophisticated scripts by demonstrating the desired commands and then expicity generaizing the parameters, usuay without the need for conditionas and embedded iterations. This is achieved by generaizing the cursor movement capabiities from Emacs to a graphica domain. Topaz aso aows scripts to be executed before or after other commands. These capabiities are provided to users with amost no effort on the part of the appication deveoper. We hope that these capabiities wi be generay usefu, and that they wi appear in many more kinds of appications. ACKNOWLEDGMENTS For hep with this paper, I woud ike to thank Bemita Myers, Rob Mier, Rich McDanie, and Bruce Kye. This research was partiay sponsored by NCCOSC under Contract No. N C 6037, Arpa Order No. B326, and partiay by NSF under grant number IRI The views and concusions contained in this document are those of the authors and shoud not be interpreted as representing the officia poicies, either expressed or impied, of the U.S. Government. REFERENCES 1. Habert, D.C. SmaStan Progranuning by Demonstration in the Desktop Metaphor, in Watch What I Do: Programming by Demonstration Cambridge, MA: MIT Press. pp Kosbie, D.S. and Myers, B.A., PBD Invocation Techniques: A Review and Proposa, in Watch What I Do: Programming by Demonstration, A. Cypher, Editor 1993, MIT Press. Cambridge, MA. pp Kurander, D. Chimera: Exampe-Based Graphica Editing, in Watch What I Do: Programming by Demonstration Cambridge, MA: MIT Press. pp Kurander, D. and Bier, E.A. Graphica Search and Repace, in Proceedings SIGGRAPH 88: Computer Graphics Atanta, GA: 22. pp, L&in, F., et a., The Eectronic Design Notebook: Performing Medium And Processing Medium. Visua Computer: Intemationa Journa of Computer Graphics, (4): pp Lieberman, H. Dominos and Storyboards: Beyond Icons on Stings, in IEEE Computer Society: 1992 IEEE Workshop on Visua Languages Seatte, WA: pp Mausby, D.L. and Witten, I.H. Inducing Procedures in a Direct-Manipuation Environment, in Proceedings SIGCHI 89: Human Factors in Computing Systems Austin, TX: pp McDanie, R.G. and Myers, B.A. Buiding Appications Using Ony Demonstration, in I998 Internationa Conference On Intehigent User Interfaces San Francisco, CA: To appear. 9. Modugno, F., Corbett, A.T., and Myers, B.A., Graphica Representation of Programs in a Demonstrationa Visua She -- An Empirica Evauation ACM Transactions on Computer-Human Interaction, (3): pp Myers, B.A., Creating User Interfaces Using Programmingby-Exampe, Visua Programming, and Constraints. ACM Transactions on Programming Languages and Systems, (2): pp Myers, B.A. and Kosbie, D. Reusabe Hierarchica Command Objects, in Proceedings CHI 96: Human Factors in Computing Systems Vancouver, BC, Canada: pp Myers, B.A., et a., The Amuet Environment: New Modes for Effective User Interface Software Deveopment. IEEE Trcmsactions on Sojiware Engineering, (6): pp Staman, R-M., Emacs: The Extensibe, Customizabe, Sef- Documenting Dispay Editor. MIT Artificia Inteigence Lab,