A PROTOCOL DATA UNIT FOR TRAINING SIMULATED COMMAND FORCES

Transcription

1 A PROTOCOL DATA UNIT FOR TRAINING SIMULATED COMMAND FORCES Kenneth E. Frosch, Elizabeth L. White, J. Mark Pullen C 3 I Center and Department of Computer Science George Mason University 4400 University Drive Fairfax, VA kfrosch@cne.gmu.edu KEYWORDS Protocol Data Unit, Command Forces, Knowledge Acquisition ABSTRACT The emergence of command forces (CFORs) as a simulated entity within a DIS simulation requires effective interfaces between existing learning systems and simulation applications. Existing methods utilize programmers and knowledge engineers to encode the knowledge of a subject matter expert (SME) into a knowledge base. However, there are no interfaces that integrate existing learning systems with a DIS application that would provide effective and efficient knowledge acquisition by allowing the SME to directly instruct the learning system. We have integrated Disciple, a machine learning system, with ModSAF to provide this direct interface for the SME. This integration extends the familiar graphical interface of ModSAF to include the tools necessary for the SME to instruct an agent through demonstration. A new experimental PDU was developed to provide the communication mechanism and common data format for the two applications to communicate. An overview of this integration, the design issues and architecture of the interface, and the format and protocol of the new PDU are discussed in this paper. 1.0 INTRODUCTION As CGF technology has advanced sufficiently to allow CFORs[4] to be created and deployed in a simulation, learning and knowledge acquisition have become critical issues. The high degree of human behavior that the CFOR must represent increases these needs. While a lot of work is currently being performed in the area of learning and human behavior for CGF, it has been found to be very difficult to interface these learning systems to existing DIS applications. Individually, however, learning systems and simulation applications are readily extensible. While existing approaches use knowledge engineers to encode the expertise of an SME, our approach utilizes a common simulation interface to build and adapt a suitable knowledge base through teaching by demonstration.[2] This method involves a SME that interfaces directly with the CFOR agent in order to instruct it, without the intervention of a knowledge engineer. This allows the instructor to use an interface that is familiar and the same techniques to teach the computerized agent as he/she would for a human student. The emphasis for this approach is on apprenticeship learning rather than knowledge acquisition. Apprenticeship learning adapts knowledge that has already been given or learned to increase the knowledge base. Our system, Virtual Commander (VCDR), is an interface between an apprenticeship learning system, Disciple, and ModSAF, a DIS application. The combined system, pictured in Figure 1, presents an efficient manner for teaching complex behavior to an agent through demonstration. The VCDR approach provides a rich interaction between the SME and the agent, which is necessary to create more powerful agents required for CFORs. The SME uses the GUI interface of ModSAF enhanced with a teaching interface as a front-end for the Disciple learning system.

2 Workstation A Workstation B Subject Matter Expert ModSAF Agent Libraries Editor Libraries Agent-Disciple Problem Solving Functions Learning Functions Interface Libraries Interface Functions Figure 1: Virtual Commander Design This interface between Disciple and ModSAF was implemented using an experimental PDU. The PDU provides the communications mechanism and the data format necessary to have the application communicate effectively in a distributed fashion. This paper describes the development of this PDU and the protocol used between the two applications. Section 2.0 begins by giving an overview of the design and architecture of Virtual Commander. In Section 3.0, the requirements for the Virtual Commander interface and the PDU are discussed. The format and protocol of the VCDR PDU are detailed in Section 4.0. Finally, an example of the interaction of ModSAF and Disciple using the new PDU is given in Section VIRTUAL COMMANDER The Virtual Commander system is an approach to solving the problem of knowledge acquisition for CFORs. VCDR is an instructable agent which utilizes programming by demonstration[5] and machine learning techniques to allow a SME to teach an agent. Programming by demonstration systems give an end user the ability to create programs by demonstrating their action. The implementation of Virtual Commander is an interface between ModSAF and Disciple[6]. Disciple is a machine learning system written in Lisp that runs on a typical Unix workstation. It employs multistrategy learning, combining explanation-based, analogical, and example-based methods with knowledge acquisition to learn from an SME. Figure 1 depicts the design and architecture of Virtual Commander. The system consists of several major components residing on distributed workstations. These parts include ModSAF, libdisciple, SME editors, Disciple, a DIS front-end, and the VCDR API. Since Disciple is written in Lisp strictly for learning purposes, it did not have the capability to send or receive network traffic. Using the foreign function interface of Lisp, a DIS front-end was written in C that implemented a network interface to handle DIS PDUs. An API was developed on top of the foreign function interface for Disciple to access the functions in the DIS front-end. For ModSAF, a series of GUI editors were created for the SME to use as an interface to instruct Disciple. The appropriate data is then passed through the new libdisciple library and transported in the VCDR PDU. Responses from Disciple are caught by ModSAF and passed to the appropriate Network VCDR Editor libdisciple ModSAF DIS PDUs Disciple VCDR API DIS Front-end DIS PDUs Figure 1: Virtual Commander Architecture

3 PDU handler in libdisciple, which would update the editors for the SME or instantiate entities and represent them on the plan view display (PVD). 3.0 INTERFACE DESIGN ISSUES AND PDU REQUIREMENTS The method to be chosen as an interface between this DIS application and learning system was determined based on the analysis of several important factors.[7] Our design and implementation is driven by three significant factors: real-time performance, data representation, and control of the learning cycle. The initial concern addressed in this implementation was the difficulty of integrating an application that relies on real-time delivery of data (ModSAF) and one that does not (Disciple). The performance or latency issue was important since ModSAF, a DIS application, typically requires real-time transmission of data. The ModSAF/Disciple interface does not require this type of service, however. In fact, due to the algorithms involved in the various learning phases of Disciple, it would be difficult to achieve real-time response from Disciple. Therefore, the concern about realtime performance is how to construct a non-real-time communication that does not cause perceptible delay in ModSAF. Due to the fact that both applications are computationally intensive it made sense to distribute their execution across multiple machines. This introduces the need to either develop a proprietary protocol or use an existing standard protocol. Using DIS PDUs as the communications mechanism between ModSAF and Disciple is effective for several reasons. First of all, it makes use of the existing communications functions within ModSAF. By doing so, the processing of the VCDR PDUs is handled in the same way as other existing PDUs. To do this a definition of the PDU structure was added to ModSAF s libpduapi library and callback function was registered that would pass the VCDR PDUs to the proper function for processing.[3] Using these functions from ModSAF provides a seamless integration of the new PDU. It permits ModSAF to execute non-blocking network operations to Disciple and allows it to continue operation without noticeable interruption. A client-server Remote Procedure Call (RPC) would not provide this. A second issue was the format in which the data would be communicated between the applications. While ModSAF stores its data in a Persistent Object (PO) database, Disciple represents its data in the form of a semantic network. In order for the two systems to communicate, a common format was constructed as the VCDR PDU. The message within the PDU is data extracted from either the PO database or from Disciple s semantic network. The experimental PDU format was developed and introduced into ModSAF. By utilizing DIS PDUs to perform the messaging for the VCDR system, we can use a messaging infrastructure already established for DIS. The final issue was the need to move the control interface from Disciple to ModSAF. Originally, the steps taken by the SME to teach Disciple were selected from a menu presented by the Disciple software in Lisp, based on which phase of the learning process was being executed. When introducing the GUI editor into ModSAF, the control mechanism was transferred to ModSAF. A new library, libdisciple, was built into ModSAF to implement the control features needed. The resulting interface is much more natural to its intended user. The interaction of the SME with Disciple is governed by a protocol that is based on the different phases of the learning by demonstration cycle. 4.0 VCDR PDU FORMAT AND PROTOCOL The PDU resulting from the requirements discussed above is pictured in Table 1. This is the format as it is defined in ModSAF. In this section, the format and function of the PDU and its functions are described The DIS 2.03 header that is at the beginning of the VCDR PDU contains elements such as version number and PDU kind. The PDU kind is a number associated with each DIS PDU. For example, the entity state PDU (ESPDU) has a PDU kind of 1. For Description Size (bits) DIS 2.03 header 96 Message ID 32 Fragment number 32 Learning phase 32 Message type 32 Message length 32 Message 8*length Table 1: Virtual Commander Experimental DIS 2.03 PDU Format

4 the VCDR PDU an arbitrary number was chosen, 172. This number is used to reference the PDU format within the libpduapi database. The fields of primary interest in the PDU are the message, message type, and learning phase. All data sent between ModSAF and Disciple is composed in the message field as an ASCII string. This string forms the common data format shared by the two applications. The message length field simply denotes how long the message is. Depending on the action performed by the SME messages may contain examples, variables, explanation, or rules. The combination of message type and learning phase fields specifies what the contents of the message field represent. The contents of the phase and type fields of the VCDR PDU describe the request-reply protocol used to implement control of the learning process. Table 2 shows these values which govern the interaction between Disciple and ModSAF. The phases correspond to separate functions within the learning by demonstration process. The message type further describes the PDU and is used to trigger a response in the learning functions of Disciple. Evennumbered types indicate a message originating from ModSAF sent to Disciple, while odd-numbered types denote Disciple sending a message to ModSAF. Some of the types simply represent acknowledgment that the previous PDU was received. Typically, though, a response from Disciple implicitly is an acknowledgment of the prior request. Their is also a need to denote negative acknowledgments, NAKs, when a message missing. Adding this functionality to the interface is currently under development. In this case, the message type field will contain a negative value. A message is recognized as missing when two messages are received when their message IDs are not consecutive. When this occurs the last message will be dropped and a NAK sent to the sender with the missing message ID. The sender will resume sending packets beginning with the missing message. To avoid the re-execution of duplicate requests by Disciple the protocol will adhere to an at most once semantic. In other words, both applications will be ensured that Disciple will fulfill the request only once, or if an error occurs, then not at all. The message ID, in conjunction with the fragment number field, is also used in the fragmentation and assembly of VCDR PDUs that contain large Specification of Initial Scenario Phases Phase 0 - Give Initial Example Type 0 - Select Example Template Type 1 - Send Acknowledgement of Template Type 2 - Select Initial Example Type 3 - Send Acknowledgement of Initial Example Phase 1 - Give Initial Explanations Type 0 - Select Variable to Generate Explanations from Type 1 - Send List of Explanations Type 2 - Select Explanation(s) Type 3 - Send Acknowledgement of Explanations Phase 2 - Quit Initial Signal Phases Type 0 - Quit Specification of initial Type 1 - Send Acknowledgement of Quit Learning through Experimentation Phases Phase 3 - Experimentation Search Parameters Type 0 - Select Variable(s) to Fix Type 1 - Send Acknowledgement Fixed Type Quit Search Phase Type Send Acknowledgement of Quit Phase 4 - Generate Experimentation Example Type 0 - Request Example to be Generated Type 1 - Send Example Phase 5 - Give Experimentation Example Type 0 - Specify Example Type 0 - Send Acknowledgement of Example Phase 6 - Classify Experimentation Example Type 0 - Classify Example Type 1 - Send Acknowledgement of Classification Phase 7 - Explain Mistake in Experimentation Type 0 - Select Variable to Generate Explanations from Type 1 - Send List of Explanations Type 2 - Select Explanation(s) Type 3 - Send Acknowledgement of Explanations Type 4 - Select Variable to Blame Type 5 - Send Acknowledgement Type Quit Explanation Phase Type Send Acknowledgement Phase 8 - Quit Experimentation Signal Phases Type 0 - Quit Experimentation Type 1 - Send Acknowledgement of Quit Table 2: Virtual Commander Learning by Demonstration Protocol

5 typedef struct vcdr_pdu { uint32 msg_id; uint32 fn; VCDR_PHASE phase; int32 type; uint32 msg_length; char message[max_length]; } VCDRPDU; Table 3: C-style Header for VCDR PDU messages. Since the strings shared between Disciple and ModSAF may be very long, it may need to be broken into several parts before sending, and then reassembled after receiving them. In ModSAF, there is a global variable that represents the largest PDU size, pa_mtu. If a VCDR PDU is larger than that value, the message field is segmented into separate PDUs. The message ID will remain the same for each PDU of a message fragment, but the fragment number will be incremented for each. A sentinel value is used in the fragment number field to represent that is the last PDU, or the only one, for a given message. The corresponding ANSI C structure that is used to manipulate the PDU contents is shown in Table 3. For this prototype, 32-bit integers are used for all numeric fields. However, for most of those fields msg_id, fn, type, and msg_length 32 bits is not necessary. The VCDR_PHASE enumerated type is a definition of the values used in the phase field that correspond to the phases described in Table VCDR PDU EXAMPLE With Virtual Commander, an SME teaches an agent through the ModSAF GUI. The actions taken by the SME are transported to the Disciple learning system via the newly defined VCDR PDU. To illustrate the interaction of the SME and the Virtual Commander system as well as the underlying protocol of the VCDR PDU an example teaching session is given in this section. Figure 2 gives a detailed representation of this request-reply conversation highlighting the use of the phase and type fields as well as how the negative acknowledgment is used to recover from lost messages. To begin the learning process, the SME demonstrates to the VCDR agent how to perform a new task or carry out a new mission. Using the existing unit and task editors within ModSAF as the SME normally would to create a sequence of tasks, the SME programs the agent. This created task is given as the initial example of the mission to the learning system. The libdisciple library then extracts from the PO database of ModSAF the data relevant to the mission. This data is stored as an ASCII string and transported to Disciple in the message field of the VCDR PDU. The corresponding phase and message type is 0 and 2, respectively. After processing this message, Disciple will then respond with a list of variables that it generated from the mission. The variables may refer to things such as companies, areas of responsibility, or other aspects of the mission. This response also serves as an implicit acknowledgment of the request. In all responses from Disciple it will use the same message ID as the incoming request. Likewise, the phase value is the same as well. The message type field will be set to 3 for this response. The teaching process continues by the SME explaining the important features of the mission. The SME does so by repeatedly selecting variables for which Disciple will generate a list of explanations. The corresponding phase and field values for the SME request to select variables are 1 and 0. Disciple will then respond with a list of explanations about that variable using the same phase but a type of 1. From the received list of explanations, the SME will select those which are relevant for the mission. A VCDR PDU is sent with this input, a phase of 1, and a type of 2, to Disciple. Disciple processes that input and returns an explicit acknowledgment. At this point the SME may continue the explanation process by selecting addition variables or send a quit message to signal Disciple that the specification phase is complete. The phase for the quit signal is 2 with the types for the ModSAF message and the Disciple acknowledgment being 0 and 1, respectively. The learning through experimentation phases (3-8) continue using the phase and type fields in the same manner. In these phases, the learning system may be asked by the SME to attempt to perform the same mission on a different terrain perhaps. For each computer generated solution, the SME is asked to evaluate and classify it as correct or incorrect. Using the ModSAF GUI, the SME can correct the agent if it does not perform the mission appropriately. The

6 Msg 1, Phase 1, Type 0 ModSAF Disciple Msg 0, Phase 0, Type 2 Time Msg 0, Phase 0, Type 3 Message Lost Next input from SME, but no response yet from Disciple Msg 1, Phase 1, Type NAK Msg 1, Phase 1, Type 1 Resends lost message Figure 2: Protocol Interaction Between Applications SME does so by again selecting variables and explanations as to why the solution was incorrect. These explanations are incorporated by Disciple to refine the plausible version space for valid solution. After the teaching session is complete, the VCDR agent will have learned how to perform this type of mission. The learning system will have created a rule specifying how to select tasks and assign task parameters for this mission. The learning system will then be able to perform the learned mission when asked by the SME, without the SME programming the behavior. In the case where a message is lost, the receiving application will use a NAK to signal the sending application to resend the missing packet. In Figure 2, a message containing a list of explanations, in response to a selected variable from the SME, from Disciple has been lost, possibly due to a network problem. In the meantime the SME may have already input the next action to the learning system. In this instance, ModSAF will send a NAK for the retransmission of that missing message. Disciple will then reissue the last response again without reexecuting the operation, adhering to the at-mostonce policy. 6.0 SUMMARY To represent more complex human behaviors in DIS simulations, it is well known that effective learning and knowledge acquisition will be necessary. In addition, the ability for a SME to directly build this knowledge base without the need for a knowledge engineer to translate the information into the system is a effective method. The Virtual Commander Protocol Data Unit provides the interprocess communication mechanism for learning system to interoperate with a DIS application. Extensions to ModSAF provide a familiar interface for the SME to instruct the learning system efficiently. Although this interface presents a communication method using DIS PDUs, this combination of Disciple and ModSAF could use the High Level Architecture's Run-Time Infrastructure. The goal of the RTI is to provide services to create interoperable components such as DIS applications and C 4 I systems. In any case the need for knowledge acquisition for intelligent agents will require an

7 integration of a learning system to the DIS simulation. A direct interface for an SME, like the VCDR PDU, presents an effective solution for this interface. 7.0 ACKNOWLEDGMENT This research was conducted in the Center for Excellence in Command, Control, Communications and Intelligence and the Computer Science Department at George Mason University. The authors wish to thank Dr. Michael Hieb, currently with SAIC, for his support and assistance with Disciple. 8.0 REFERENCES [1] Cypher, A. (Ed.), "Watch What I Do: Programming by Demonstration," MIT Press, Cambridge, MA, [2] Hieb, M.R., Tecuci, G., Pullen, J.M. (1996). Virtual Commander An Instructable ModSAF Agent, 6 th Conference on Computer Generated Forces and Behavioral Representation. [3] Loral Advanced Distributed Simulation (1995). ModSAF 2.0 Developer s Kit. [4] Salisbury, M.R., Booker, L.B., Seidel, D.W. and Dahmann, J.S., Implementation of Command Forces (CFOR) Simulation, 5 th Conference on Computer Generated Forces and Behavioral Representation, [5] Tambe, M., Johnson, W. L., Jones, R.M., Koss, F., Laird, J.E, Rosenbloom, P.S., and Schwamb, K., Intelligent Agents for Interactive Simulation Environments. AI Magazine, 16(1), Spring, [6] Tecuci, G., "DISCPLE: A Theory, Methodology, and System for Learning Expert Knowledge," Ph.D. Thesis, University of Paris South. [7] White, E.L., Frosch, K.E., Laviano, V.P., Hieb, M.R., Pullen, J.M. (1996), Interfacing External Decision Processes to DIS Applications, 6 th Conference on Computer Generated Forces and Behavioral Representation. 9.0 ABOUT THE AUTHORS Ken Frosch is a Graduate Research Assistant in the C 3 I Center at George Mason University. He is currently on leave of absence from Lockheed Martin Federal Systems where he has been a Systems Engineer since He received his B.S. in Computer Science and Mathematics from the University of Wisconsin - Madison in Elizabeth White is an Assistant Professor of Computer Science at GMU. She received M.S. and Ph.D. degrees in Computer Science from the University of Maryland in 1990 and 1996, respectively, and a B.S. degree from the College of William and Mary in J. Mark Pullen is Associate Professor of Computer Science and member of the C 3 I Center at GMU. Dr. Pullen was with the Defense Advanced Research Projects Agency (DARPA) from 1986 to 1992, where he was Program Manager for Advanced Computing, Networking and Distributed Simulation.