Upgrading the JCMT Control System to Platform Independence

Upgrading the JCMT Control System to Platform Independence Mary Fuka a, Dennis Kelly b, and Richard Prestage a a Joint Astronomy Centre, 660 N. A ohoku Place, Hilo, HI 96720 b Royal Observatory, Blackford Hill, Edinburgh, EH9 3HJ, Scotland ABSTRACT The James Clerk Maxwell Telescope (JCMT) is escaping from its dependence on VAX/VMS legacy software by incrementally upgrading its control system using platform-independent tools. The first stage of the upgrade includes a GUI, written in Java and modeled on project planning software, for defining complex observing recipes. Another GUI serves as an editor for observation definition files describing the details of an individual observation. The recipe designer also serves as an observation subsystem controller, executing observing scripts containing combinations of low-level and high-level commands (recipes), using the parameters from the observation definition file. The next stage of the upgrade includes a Java-based queue manager. The DRAMA system developed at the Anglo Australian Observatory provides a convenient bridge between the existing VAX/ VMS-based instrument tasks and the new controller and queue manager. Keywords: Control Systems, Observing Recipes, Java 1. INTRODUCTION The James Clerk Maxwell Telescope, a 15-meter radio telescope optimized for sub-millimetre wavelength astronomy, celebrated its tenth anniversary in autumn 1997. Its control system has evolved through planned incremental upgrades into a mature, stable system. However, this very stability has drawbacks: it is becoming increasingly difficult to add new instruments and improved observing modes to the existing system. In order to provide more flexibility for adding instruments and observing modes and modifying existing modes, the JCMT controls group is implementing a major control system upgrade. Goals of the observatory control system upgrade project include: improving functionality and flexibility for implementing new observing modes while retaining those currently provided; maximizing performance by performing non-conflicting tasks in parallel; reducing operating system dependencies that limit choices for designing new instruments and observing modes; facilitating flexible queue-based scheduling of both heterodyne and continuum instruments. This paper will describe the new JCMT observation controller and explain how it contributes to the above goals. 2. EXISTING CONTROL SYSTEM DESIGN AND LIMITATIONS Observing with JCMT requires cooperation among several subsystems, each subsystem controlled by a separate task for ease of maintenance. Fast communication among tasks is provided by noticeboards stored in shared global sections, while slower inter-task communication is accomplished using the ADAM message system. 1 Even simple observations taken with the JCMT require synchronization of activities among several subsystems. Typical heterodyne (position switched) samples consist of several data-taking cycles, each cycle requiring: slewing the telescope between source and reference positions, updating secondary mirror position, checking that the receiver is phase-locked, a. Email: M.Fuka@jach.hawaii.edu b Email: D.Kelly@roe.ac.uk a Email: R.Prestage@jach.hawaii.edu

integrating the data at each telescope position, reading out the data when each integration is finished, Performing an FFT and other preliminary data reduction steps on the latest data read, updating the system temperature using the reference position for sky data, calibrating the latest data and adding it into accumulators, storing and displaying the accumulated data. These steps are performed for each data cycle until enough data is collected to achieve the desired signal-to-noise ratio. For efficiency, data readout and reduction and cycle setup activities such as phase-lock checking are done while the telescope is slewing between source and reference. The other single-point sample modes are similar to position-switching, with some type of telescope movement or frontend frequency adjustment being the main inter-cycle activity to be coordinated with integration, readout and data reduction. Coordination of the sub-systems is the responsibility of the heterodyne control task, an ADAM task which contains the hardcoded logic to sequence and control these low-level activities as required. Map-making is conceptually similar to single-point observations, but with the added complexity that the telescope must be commanded to cover a series of sky positions. The current control system allows only two mapping modes for heterodyne instrument observations: grids and raster-maps. Grids are evenly-space single-point observations. Rasters are accomplished by slewing the telescope while repeatedly integrating across a whole row of the map, then integrating on a single reference position to calibrate data for the row. The existing control task handles the current observing modes quite capably, as there are relatively few overlapping activities. However, both for single position and map observations, we frequently receive requests to allow modifications to the observing process. For maps, some options desired are to allow repeat cycles of several source positions with one reference position, or to take a full set of calibration data periodically during a map. In principle it would be possible to modify the current control task to support these new observing modes, although each one requires modification to the VMS Fortran in the control task. Other requests, however, such as adding support for heterodyne polarimetry, are almost impossible to support within the framework of the existing system. In addition, new instrumentation, such as the multi-element array system under development, will require even more new functionality not supported by the existing control task. These considerations have led us away from the idea of a control task which has pre-defined observing modes hard-coded within it, to that of a flexible observation controller. This will be capable of executing recipes describing the sequence of lowerlevel activities required. The provision of new observing modes will then be a simple matter of modifying the existing recipes, or writing new ones, as required. An added benefit is that the recipe will clearly separate the specification of the control flow for a specific observation from the mechanisms used to cause the observation to be executed. SCUBA, the bolometer array, is controlled by its own Fortran control task, with the assistance of a queue manager, its own private scripting language, and observation definition files containing observing parameters for a particular observation. Figure 1 diagrams the interaction among tasks in the existing control system. Bringing both heterodyne and continuum observing under a single observation controller will simplify operations and make software maintenance less complicated. Efficient observing with the JCMT requires several actions to be performed in parallel at various stages of the observation. The ADAM environment provides the ability to execute actions in parallel, but the logic becomes tedious to follow if more than a small amount of parallelism is implemented. The control task programmer must keep track of those actions active at each stage of the observation, and those which must be completed before going on to the next stage. The command line interface to the existing control system has been well-received, so much so that the menu-oriented interface was rarely used, and has therefore been discarded. Nevertheless, controlling an observation is rapidly becoming more complex and the telescope support specialists have to keep track of several interacting subsystems during the course of an observation. A graphical interface showing the progress of an observation and state of the observatory will make this easier.

ICL Procedures Heterodyne Instruments Control Task SCUBA Control Task SCUBA Queue Manager TEL TMU Heterodyne Instrument Tasks Storage, Display, Logging SMU SCUBA Tasks Figure 1: Current Control System Software Configuration 3. CONTROL SYSTEM UPGRADE OVERVIEW The new observation controller will replace both control tasks and the command line control interface with a graphical observation driver coded primarily in Java. Features that led to the choice of Java include: Simplicity of porting to new hardware platforms: Java s claim to be write once, run anywhere was especially appealing after years of being locked into hardware platforms that support VMS. Java s multithreading provides a convenient way to run several control flows at the same time, simplifying the handling of actions executing in parallel. Java has a large and growing library of components for building the graphical user interfaces required for the recipe design tool and observing control interface. No effort is available to replace the existing instrument tasks, so for now they will remain VMS ADAM tasks. Drama, 2,3 the software environment developed at the Anglo-Australian Observatory to support the requirements of real-time distributed systems, has built into it the ability to communicate with VMS ADAM tasks. The Java observation controller uses an auxiliary Drama task to communicate with these ADAM tasks as well as with newer tasks, such as the Portable Telescope Control System Suite (PTCS), coded in the Drama environment. Drama is the preferred JCMT software environment for future instruments, although we do not want to rule out other options that may present themselves. Observation Definition Tool Observing Recipes Observation Definition Files Telescope Observation Designer / Driver Observation Queue Manager Heterodyne Storage / Display Heterodyne Instrument Tasks TEL TMU SMU SCUBA Tasks Figure 2: Upgraded Control System Configuration and Command Flow

The observation controller, called Todd (Telescope Observation Designer and Driver), is also used for creating observing recipes and for testing their execution in simulation mode, a convenient way for the recipe designer to debug observing modes. Another auxiliary program, called toddle, provides an interface from any available scripting language to the observation controller. SCUBA s observing queue will be replaced with a new queue manager, talking directly to the Todd, and capable of controlling both heterodyne and continuum observations. Interaction among the queue manager, the Todd, auxiliary tasks and instrument tasks is diagrammed in Figure 2. 4. USING THE RECIPE DESIGNER A Todd observing mode recipe is constructed of three visual components, FoldBox, Command, and Pause. FoldBoxes can have other components, including FoldBoxes, placed inside them. Command components represent actual commands that can be executed by the Todd. There are two types of command components: built-in Todd commands that manipulate Todd internal variables, and action commands to be sent to tasks. Action commands correspond to the ADAM / Drama commands to obey or cancel a task action or to get or set a task parameter. Examples of typical commands are: Obeyw DAS integrate - to start an integration Obeyw TEL slew - to slew the telescope Obeyw RXA frequency 230.538 - to set frontend frequency Pause allows the programmer to set points where the observation may be paused if the observer wishes to do so. FoldBoxes have loop variables associated with them: maxloop for specifying the number of iterations, and countname for counting iterations. A ConditionBox, a special type of FoldBox, allows conditional execution of recipe elements depending on evaluation of an expression associated with it. Each Todd component has a status level variable that can be set to success, abort or fatal to control whether or not the component executes under each status condition. This permits the recipe designer to specify which parts of the recipe must be executed under abort or fatal error conditions. For example, the observation may be terminated immediately when a fatal error occurs, but files should be closed and the telescope reset to its state at the beginning of an observation. A less serious error or an abort command from the observer may require the current observing cycle to be finished and the data reduced before terminating. A simulation flag and time is associated with each FoldBox and command for convenience in testing recipes. A FoldBox can be labelled with an appropriate label and folded away Figure 3: FoldBox Properties Window to conceal observation sequence details. On start-up, the Todd reads in two files, one containing a set of often-used commands and one containing default values for a number of Todd variables. These files can be edited to add commands and variables as new instruments and observing modes are added. Commands and variables that can be included in a recipe are not limited to those in the start-up files. To create a recipe, the observation designer starts up the Todd. Three windows appear, the main Todd window, containing the TopBox, an object properties window and a ToolBox. Objects are placed on the main Todd window by clicking on the desired element in the ToolBox window, then clicking on the TopBox or the FoldBox where you want to place the object. Clicking on an element with the right mouse button brings up its properties in the Properties window. The properties can then be edited. Figures 3 and 4 show the main Todd window, the ToolBox, and the Properties windows for FoldBoxes and commands. Note that FoldBox properties include countname and maxloop for loop iterations, while command properties include command and params entries for selecting and editing commands. Clicking on the command entry point brings up a scroll-box of frequently used standard commands. Clicking on the params entry point brings up a window displaying the entered command, or the default if no command has been entered, which can then be edited. Foreground and background colors for each object may also

be changed from the default gray-scale choices. For now, we prefer to use color conservatively, to highlight the parts of an observation as they execute or to call attention to alarm and warning conditions. Figure 4: Todd Main Window, ToolBox, and Command Properties. The pull-down edit menu is used to sequence components to indicate the following causal relationships: execute B when A completes, execute B when A has started or execute B on a trigger signalled by A. Elements are re-arranged on the Todd window to show these relationships, as shown in Figure 4. The edit menu also provides cut, copy and paste options for adding or changing recipe elements, and link and unlink options for re-arranging the causal relationships. Recipes or portions of recipes contained in a FoldBox are saved using the File pull-down menu. This menu also provides options for loading previously saved recipes, for clearing the main Todd window, and for testing recipes by selecting reset, then start. Any portion of a recipe contained in a FoldBox may be saved as a separate entity; this enables the designer to build up libraries of reusable parts for ease in creating complete observation recipes. 5. USING THE OBSERVATION DEFINITION TOOL A recipe by itself is not sufficient to define an observation. It describes the sequence of actions to be performed, but does not include the parameters for those actions, for example, rest frequency and velocity for setting the frontend observing frequency. Observation Definition Files (ODF s) are text files containing variable names paired with variable values. The Todd reads these to fill in values for any variables invoked in a recipe s commands. At start-up, the Todd reads in a file containing defaults for all of the commonly referenced variables, but a recipe may use as many variables as it requires. Variables included in an ODF override the Todd s default values. A prototype version of the observation definition tool has been coded in Tcl/Tk. It brings up only the parameters necessary for each type of observation, showing system defaults or last-used values for each parameter. A saved ODF file may be read in to replace the default values. Parameters are then edited as needed, and the ODF file is saved. A set of ODF files for typical observations is available to make it easier to add and edit ODF s for particular observations.

Figure 5: Prototype Observation Definition Tool. The Observe Modes button brings up a list of available observing modes, for example, single-point sample, grid, focus or fivepoint. Switch Modes similarly brings up a menu of switching modes and Instruments lists the backend instruments available. Frontend instruments do not affect the observing mode, so are edited as parameters. A carriage return after entering a value causes the tool to check whether the value is within range. The? button brings up a brief description of the parameter and the acceptable range of values. 6. DRIVING AN OBSERVATION In addition to its recipe creation function, the Todd also serves as an observation driver. When used in this mode, it expects to receive commands via two higher-level components, a queue manager and a scripting language. An auxiliary task, called toddle, provides an interface to the Todd from any scripting language that runs on a Unix workstation. In observing mode, the Todd depends on an auxiliary task, called dramatodd, to pass Drama or ADAM commands on to tasks running on any networked workstation. Before observing, the Todd and its auxiliary task dramatodd are run up on the operator interface workstation. Device control tasks are loaded on the networked VMS and Unix workstations as required. Drama network communications are started on all workstations. The Todd is sent the observe command with the name of an ODF file, either interactively, from a script, or from the queue manager. The Todd reads in values for the variables specified in the ODF file, including the name of the observation mode recipe. It then checks to see if the recipe is already in its cache; if not, it reads it from disk. It displays the recipe on its main window and immediately starts executing it. As the observation proceeds, currently active parts are highlighted by changing color. Recipes are usually saved with FoldBoxes in folded-up state, but any FoldBox can be opened during the course of the observation to give a more detailed view of how it is proceeding. Figure 6 shows a beam-switched cycle recipe with one of its nod-and-monitor FoldBoxes open to show the details. Parameters in the ODF file can be overridden by direct commands to the Todd, even during the execution of an observation. For example, the number of cycles can be set high before an observation, then reduced when the desired signal-to-noise is reached. Pause points can be included in a recipe, initially turned off by setting pausecount to zero, then turned on by setting pausecount to one. Error conditions may also toggle the pausecount variable, providing a point where the observation stops and prompts the telescope support specialist to adjust some hardware, after which the observation can finish successfully.

Figure 6: Beam-switched cycle recipe with one nod-and-monitor FoldBox open. The observer can also command the Todd to override some of its variables, then repeat the last observation. This is convenient when an observer wants to perform a sequence of similar observations with minor changes to the ODF variables, keeping the same instrument configuration but changing frequency, for example. The scripting interface will provide the ability to override Todd variables on the observe command line. The Todd implements this feature by reading in an ODF file, then modifying variables as requested before executing the observation. 7. DETAILS OF THE RECIPE DESIGNER / DRIVER 5 The Todd is based on extensive modifications of Sun s BeanBox. Extensive information on JavaBeans is available on Sun Microsystem s web pages: http:// java.sun.com/beans. JavaBeans are reusable software components that can be manipulated visually in a builder tool 6. Each of the Todd recipe builder components, FoldBox, Command, and Pause, is a Java class, and an instance is created for each component appearing in a recipe. Each Todd component is an independent Java Object with its own internal variables. Each has properties which are displayed in the properties window when the component is selected. The Todd itself is a frame containing a TopBox and a ScrollPane. TopBox contains any number of FoldBox, Command and Pause objects. FoldBox objects contain other FoldBox objects as well as Command and Pause objects. Figure 8 shows these relationships. Inheritance among the classes is illustrated in Figure 7. JCMTBean is the base class which implements much of the functionality inherited by TopBox, FoldBox, RawCommand, Command and Pause. It manages scheduling connections to other objects and has a parallel thread, JCMTCommand, which carries out the activity represented by its associated JCMTBean. JCMTCommand waits on semaphores signalling that a command should be started or has completed, then performs the relevant

calls to the methods in the JCMTBean. It also keeps track of the repeat count for the command and updates the variable associated with the count. When the command has completed, it calls the JCMTBean method that informs other objects of its completion. Panel Todd JCMTBean TopBox ScrollPane TopBox Command RawCommand Pause Command FoldBox FoldBox Pause = a kind of = a part of Figure 7: JCMTBean Inheritance Figure 8: Todd Object Aggregation To manage sequencing, each component contains a list of the components that must be informed of changes in its state. Each component also keeps a list and a count of the objects it has to wait for. When it receives notification from one of its objects, it decrements the count of the items it is waiting for, and when the count goes to zero, it starts executing its operation. There is no global sequencing knowledge; it is scattered among the components, each of which understands only its own relationships with the other objects. The Todd uses TCP/IP sockets for connecting upwards to the scripting system and downwards to the auxiliary Drama task that interprets and passes on Drama and ADAM commands. When it starts up, it announces itself using two TCP/IP port numbers. If nothing connects to the auxiliary task port, it functions only as a recipe designer, echoing commands to its stdout instead of passing them on to the Drama system. Figure 9 illustrates the communications flow among tasks when the Todd is used as an observation driver. The DramaComms class handles downwards communication. It contains threads to perform the built-in commands for setting Todd internal variables, to simulate commands by echoing to stdout, and to send commands to tasks and receive responses. The auxiliary Drama task, dramatodd, connects to the Todd s down port when it starts up. It accepts commands coming in on the socket and calls Drama user interface subroutines to send these commands on to the specified Drama and/or ADAM tasks. A structure is created specifying the completion, error, and information message handlers for each command sent. These handlers in turn send messages back to DramaComms where the receiving thread takes action depending on the type of message.

Drama Task Scripting Language Interface Todd (Java) dramatodd (Drama C-code) Drama Message Handling ADAM Task ADAM Task Drama Task Figure 9: Communications Flow Communication upwards is handled by the CommandServer class, which can service up to three simultaneous connections, normally from the toddle scripting interface program. Each command is handled by a thread that waits for the command to complete before returning the completion message through the socket and closing the connection. While most messages coming in through this socket will be observe commands, CommandServer can handle any primitive command that can be entered into a Command button, thus providing a great deal of flexibility for modifying observation parameters interactively or from scripts. 8. PROJECT STATUS AND FUTURE PLANS We are now in the process of creating and testing observation recipes, first on our simulated system in Hilo, then on the summit system. A few modifications to existing tasks will be required before the upgrade can be fully tested. Enhancements being added to the Todd include an operator display screen to relay messages sent in response to task commands and a status display showing the current state of the observation. We plan to run the Todd on a Sun workstation. However, it has been successfully tested running from Windows NT, so we have flexibility in choosing future operator interface workstations. We are discussing options for implementing the queue manager. We hope to provide a scripting language interface tied more closely to at least one of the popular languages. The observation definition tool needs to be re-worked, and will probably be recoded in Java. Our first release will replicate observing modes available in the existing system, but with improved efficiency because of the increased level of parallel action execution. A full test and evaluation is planned for late summer or early autumn 1998. 9. REFERENCES 1. A. Bridger, R. Prestage, M. A. Fuka, Cooperating Software Tasks for the Control of Large Telescopes, Nuclear Instruments and Methods in Physics Research Vol. A293, pp 148-153, North-Holland, Amsterdam, 1990 2. T. J. Farrell, K. Shortridge, and J. A. Bailey, DRAMA: An environment for instrumentation software, Bull. Amer. Astr. Soc. 25, pp. 954-957, 1993. 3. J. A. Bailey, T. J. Farrell, and K. Shortridge, DRAMA: An environment for distributed instrumentation software, in Telescope Control Systems, P. T. Wallace, ed., Proc. SPIE 2479, pp 62-68, 1995. 4. D. Kelly, Alpha-2 Release of the Todd, JCMT OCS System Note 15.1, 1998. 5. D. Kelly, The Todd Internals Manual, JCMT OCS System Note 13.1, 1998. 6. G. Hamilton, ed., Sun Microsystems JavaBeans TM, Sun Microsystems, Inc., 1998.