Optimize Performance in the Presence of Time Delays 1. J. K. Yook, D. M. Tilbury, N. R. Soparkary. The University of Michigan

Transcription

1 A Design Methodology for Distributed Control Systems to Optimize Performance in the Presence of Time Delays 1 J. K. ook, D. M. Tilbury, N. R. Soparkary The University of Michigan Mechanical Engineering and Applied Mechanics yelectrical Engineering and Computer Science Ann Arbor, MI 819 fjyook, tilbury, soparkarg@engin.umich.edu Abstract When a control system is implemented in a distributed fashion, with multiple processors communicating over a network, both the communication delays associated with the network and the computation delays associated with the processing time can degrade the system's performance. In this case, the performance of the system may depend not only on the performance of the individual components but also on their interaction and cooperation. The approach taken in this paper assumes that the control has been designed without taking into account the network architecture. A theoretical framework is presented which allows the eect of time delays on the mechanical performance of the system to be precisely modeled, and these models are used to determine the optimal network architecture for the given control system. A design example of a two-axis contouring system is presented. 1Introduction and Motivation From monolithic, centralized control algorithms, there is an increasing trend towards decentralized, distributed realtime control implementations. This trend may be observed in application areas that range from robotic machining to autonomous vehicle control. The changes are dictated by pragmatic considerations that include inherent physical distribution of systems, control performance, and even economic factors. In distributed systems, modularity can be used to manage complexity. If the physical system to be controlled has natural points of division, a convenient way to modularize the control scheme would be to allocate one control module for each physical component of the system. However, this decoupled control strategy can only provide good overall performance if there is little physical coupling between the system components, or if the desired performance does not inherently depend on the coordination of the system components. In many cases, the mechanical system performance metrics (such as speed of response, trajectory following error, etc.) will be closely linked to the computing system performance metrics (processor speed, network bandwidth, laxity, quality of service, etc.). In our research, we are working to explicitly characterize the 1 This research was supported in part by the NSF under the grants EEC and CMS performance of the control system, integrating the mechanical system performance metrics with the computing system performance metrics, and thereafter, to employ quality of service (QoS) approaches from computer science to improve the overall performance. In this paper, we assume that traditional techniques are used to design a controller for the coupled mechanical system, ignoring any implementation eects. We then show how the best implementation for a given control system can be determined. Future work will address mechanisms for designing or re-designing a control system to take into account the limitations of the underlying computing infrastructure. Background Traditionally, control algorithms have been designed without consideration of their implementation details. However, when networks are used to carry feedback signals for a control system, the limited bandwidth induces unavoidable communication delays. In order to successfully analyze the behavior of a networked control system, the type and location of these delays must be characterized, and their effect on the performance of the control system understood. Although some parts of this problem have been studied previously, in this paper we study how the control architecture the allocation of control tasks to processing nodes on a network aects the mechanical performance..1 Control network eects When a network is used to transmit data, the delay between the initiation time of the message and the delivery time is inuenced not only by the speed of the network (bitrate) and message size but also by the amount of other traf- c on the network and the communication protocol used. The communication protocol also aects the reliability and fault-tolerance of the networked system through its priority assignment and error-correction schemes. In [5], communication protocol requirements for real-time systems were discussed in detail. A study of the delay characteristics of popular control networks can be found in [6]. It is well-known that excessive delays in a feedback loop can destabilize a control system []. Thus, in a distributed control system, the eect of the network induced delays on the stability of the system should be analyzed. Robust control [3, 9], state prediction control scheme [1], and recursive information ow system [8] have been proposed to compensate for

2 the eect of time delays. However, these approaches do not allow a system with well-characterized and predictable time delays to be optimized. Our work assumes that the network time delays can be characterized, and focuses on determining the eects of these types of delays.. Control architecture eects The mechanical performance of a distributed control system depends on the control algorithms used and on the hardware implementation of those algorithms. The eects of synchronization [1], sampling frequency [], and delay [1] on the performance have been considered by other researchers. However, most of the existing work has focused on the single-input, single-output case, with one actuator, one sensor, and one processor. Even when multi-input, multi-output plants are considered, the control is typically centralized with the sensor and actuator data perhaps sent over a network, but there is no distributed coordination of physically separated mechanical systems. After control engineers develop block diagrams that describe the system behavior, the system integrator must answer fundamental implementation questions. The choice of sampling rates, processors, and communication protocol, will introduce different communication and computation delays in a system which will inuence the overall performance of the system signicantly. Not only the magnitude but also the location of the delay aects the system's performance. In the absence of delay, the performance of any system architecture should be identical. However, since delay is unavoidable in a distributed system, the architecture which most eciently arranges the delays should be chosen. Existing studies on distributed control systems typically assume a predetermined distributed architecture. The analysis of a distributed control systems has to overcome the complexity inherent in a sampled system, and there is a lack of nonlinear systems control theory to analyze or synthesize such sampled control laws with respect to output performance indexes. For these reasons, no rigorous systematic approach to increase the performance of the system by proper implementation has been developed, although guidelines and simulation studies are presented in [7, 1]. These preliminary investigations are useful to show the feasibility of distributed real-time control systems, but they are inadequate in terms of general principles that can be used to guide the design of distributed control systems. In this paper, we propose an analytical approach to characterize the eects of the control architecture on the mechanical performance, given a predened control algorithm. This will allow the system integrator to quickly evaluate many dierent scenarios and choose the best possible one. It will also enable future work whereby the control architecture will be designed together with the control algorithm. 3 Mathematical framework In this section, we dene the performance criteria for mechanical systems, and then the performance degradation function associated with dierent time delays. In Section, we present preliminary analytic, simulation, and experimental results which validate this framework. 3.1 Performance criteria Within the context of this paper, the performance of a mechanical control system will be dened by how closely the system tracks a given reference trajectory. That is, given a desired reference trajectory r(t) for the system, the performance is the dierence between the actual system output y(t) and the reference, P = ky,rk. Depending on the physical system and the application domain, one of many dierent norms may be used, including the maximum deviation from the trajectory, the average error along the trajectory, or the endpoint error. The focus of this work is on developing methodologies for implementing control laws using distributed computing systems, not on developing new control laws. Thus, the baseline performance will always be taken to be the expected value of the performance criteria with no time delay. This characterization allows us to isolate the eect of the time delay from the control design. 3. Performance degradation functions Consider a control system with and without time delay. Let r(t) be the reference, y (t) be the output of the system without time delay, and y(t) be the output of the system with the time delay. The nominal performance criteria is given by P = ky, rk We assume that the controller has been designed well, and that P is the \best" possible performance that we can achieve. With time delay, the performance criteria becomes P = ky, rk = ky, y + y, rk ky, y k + ky, rk = kk + P where represents the degradation in performance due to the time delay. In general, there may be many time delays which contribute to the overall performance degradation function. Consider a given distributed control architecture with n nodes (sensors, actuators, control modules) in the control system, and communication and computation delays i;j as follows: i;i = computation time at ith node i;j = communication delay from ith to jth node The performance degradation function will depend on all of the time delays i;j. Its Taylor series expansion without the higher order terms can be written as ( 1;1; 1;;::: ; n;n) = (; ;::: ;) 1;1 1; + (1) n;n Since (; ;::: ;) = by denition, the rst term on the right hand side is zero. Thus, to a rst-order approximation, the degradation in performance due to multiple time delays is equal to the sum of the performance degradation due to each time delay. In addition, the performance degradation due to a particular time delay is a linear function of

3 REFERE TRAJECTOR GENERATION reference delay τ r Σ PI TRACKING CONTROLLER T b(z+1) B 1+ s (z-1) [ ] τ τ y sensing delay computation delay c PLANT K s(s+a) error caused from Tr Error caused from input delay Tr =.1 sec Tr =.5 sec Tr =.9 sec 1 3 error caused from Tc Error caused from computational delay..1.1 Tc =.5 sec Tc =.1 sec Tc =.9 sec. 1 3 error caused from Ty Error caused from sampling delay. Ty =.1 sec Ty =.5 sec.1.1. Ty =.9 sec 1 3 Figure 1: Possible delays in a control system with two control modules. the magnitude of the time delay. We dene the performance dierential function,, to distinguish from the performance degradation function,. The performance dierential function i;j is given by i;j and represents the performance degradation due to a unit time delay between nodes i and j. We also use i;j to represent the performance degradation function for a single time delay between nodes i and j. Thus, the performance degradation function is given by the expression = 1;1 + 1; + + n;n n n = 1;1 1;1 + 1; 1; + + n;n n;n () = i;j i;j i=1 j=1 The performance dierential functions can be computed analytically for linear systems and numerically for nonlinear systems. The performance degradation function associated with a control architecture can be found by summing the products of the expected time delays with the performance dierential functions as in equation (), and this value can be used to compare with other control architectures. Framework Validation In the context of a very simple mechanical system, we have performed basic analysis, simulation studies and experiments to verify the mathematical framework presented in Section 3..1 Analytic results: Performance degradation For the study, we considered a single degree-of-freedom system, consisting of a motor which moves a mass along an axis. Delays may occur in sending the reference and sensor data over a network, or in computing the control algorithm. Time delays in sending the actuator command over a network occur in the same place in the loop as the computational delay, and are not considered separately here. The block diagram of the system, with the potential time delays in the loop, is shown in Figure 1. To compute the performance degradation functions for each of the three time delays, the modied Z-transform was used to compute the eects of delays []. Using this method, we can account for time delays which are not multiples of the sampling period. Figure shows the performance degradation functions as functions of time for the three individual time delays; three dierent magnitude of time delay are shown for each delay location. As predicted by the Taylor expansion in Section 3., these functions r; c and y are linear functions of the magnitude of the time delay. a. r b. c c. y Figure : Performance degradation functions for dierent time delay locations and magnitudes. error caused from Ty & Tc Error caused from Ty & Tc Ty & Tc=.1 sec Ty & Tc=.5 sec Ty & Tc=.9 sec 1 3 error caused from Tr & Tc Error caused from Tr & Tc Tr & Tc=.1 sec Tr & Tc=.5 sec Tr & Tc=.9 sec 1 3 error caused from Ty & Tr Error caused from Ty & Tr Ty & Tr =.1 sec Ty & Tr=.5 sec Ty & Tr=.9 sec 1 3 a. c & y b. c & r c. y & r Figure 3: Performance degradation functions for dierent combinations of delays. The performance degradation functions for combinations of dierent types of time delays in the system are shown in Figure 3. Comparing these results with those of Figure, it can be seen that the eects of time delays are additive, validating our hypothesis that the performance degradation function for the entire system can be computed by adding the individual performance degradation functions: = r + c + y = r r + c c + y y. Simulation results: Two-axis system The eect of control architectures and communication delays on the performance of the two-axis contouring system was evaluated through simulation studies and experiments on a small x-y table. This system contains two copies of the previous single-axis system and a cross-coupled controller. Although both axes are linear, the cross-coupled controller has a non-linear term for estimation of contour error and the performance criteria; the contour error [11] is a nonlinear function of the axis-level errors and the reference trajectory. Hence, we use simulations and experiments to validate the hypotheses. Although the control algorithms used were very simple and would, in practice, almost always be implemented on a single processor, a distributed implementation was considered in order to study the eects of delays on the control performance. We only consider one distributed implementation containing two delays: x;y and y;x, namely the x-position data to y-axis system and vice versa. Figure shows that the performance degradation function is the sum of x;y and y;x although the -axis contouring system is non-linear. Figure 6(a) also shows that the performance degradation function is a linear function of time delay..3 Experimental results: Two-axis system A small two-axis table with DC motors was used for the experiment. A picture of the experimental set-up is shown in

4 1ms Φ x,y x 1-3ms 5ms 7ms Φ y,x x 1-1ms 3ms 5ms 7ms Φ (Φ x,y + Φy,x ) 1 x ms 3ms 5ms 7ms a. x;y b. y;x c., ( x;y + y;x) Figure : A comparison between and ( x;y + y;x). data bank 6 6,5 C/C controller 3,5 1 servo 5 5, servo,5 5,1 3,1, 3 enc. 1,7,8 enc. x axis table 7 8 y axis table x y Figure 7: Two axis contouring system block diagram showing the eight basic modules and nine potential communication delays. Figure 5: The experimental set-up, with two computers and a two-axis table. Figure 5. Matlab/Simulink/Real-Time Workshop was used to implement the controllers on personal computers. Synchronization between the two computers was accomplished using a hardware switch which was ipped to start the experiments. Network communication was done using the bidirectional serial port, and the results were used to compare with the centralized architecture. In order to examine the eects of dierent magnitudes of delay, we needed to be able to specify the communication delay. For that reason, only one computer was used for the experiment with the time delay intentionally implemented in the control algorithm. As shown in Figure 6(b), the performance degradation function for small values of time delay is a linear function of the time delay, as predicted by the derivation in Section Design Example In this section, we show how the performance degradation function framework can be used to evaluate and compare several dierent architectures for a distributed control system. A two-axis contouring system is considered again since it is relatively low-dimensional yet has tight coupling between the axes. It is also nonlinear, showing the broad applicability of the techniques. 5.1 Identication of basic modules The rst step in determining the architecture is to dene the basic modules of the control system. A module can be a system component such as a sensor or actuator, an algorithm such as a servo controller, or a subsystem such as an axis. The inputs and outputs of each module must be dened, and their connections determined. The number of modules will determine the amount of communication that must be considered. By choosing the basic modules to be as small as possible in this step, the best performance can be guaranteed; however, smaller modules require more combinations to be examined. For the example system, we have chosen eight basic modules as shown in Figure 7 with all possible delays. The basic modules are numbered from 1 to 8, and the time delay between modules i and j is denoted i;j. For this logical control architecture with eight basic modules, there are nine possible time delay locations. 5. Computation of performance dierentials After the basic modules and time delay locations have been identied, the performance dierential functions associated with each time delay can be found either analytically or by simulation as discussed in Section 3. Computation of i;i is not necessary since computation delay aects the system in the same way as communication delay. Thus, i;i can be computed using the following equation. i;i = i;j (3) j6=i Equivalently, the computation delay at node i could be added to all communication delays emanating from node i. llφll: (.1 inch) llφll vs. delay in communication delay llφave ll llφ abs ll llφ rms ll llφ max ll communication delay: (msec) (a) Simulation llφll: (.1 inch) llφll vs. delay in communication delay llφave ll llφ abs ll llφ rms ll llφ max ll communication delay: (msec) (b) Experiment Figure 6: Performance degradation functions for dierent time delays and dierent performance metrics. For a system with n basic modules, the maximum possible number of the performance dierential functions that must be computed is n. These performance dierential functions can be put into a matrix = [ i;j]. Each i;j can be stored as a vector; the exact functional expression is not needed to compute the performance degradation function. Note that some of the i;j will be zero, indicating that there is no coupling between the corresponding modules. In addition, i;i is zero if there is no computational task at the ith node. The performance dierential function matrix for the example system is shown in Figure 8a. Note that a number ofentries in the performance dierential function matrix are zeros indicating that there is no communication between the respective modules.

5 a. b. Figure 8: The performance dierential functions i;j and time delays i;j of the system in Figure 7 are shown in matrix formation. 1 3 SERVO SERVO MOTOR MOTOR CONT. INPUT CONT. INPUT 3 SERVO MOTOR SERVO MOTOR CONT. INPUT SERVO MOTOR SERVO MOTOR 3 a. Completely b. Lumped c. Lumped distributed servo/motor axis control SERVO MOTOR SERVO MOTOR 3 CONT. INPUT CONT. INPUT SERVO SERVO C/C CONT. REF. INPUT MOTOR MOTOR d. Independent axes e. Centralized Figure 9: Several possible network control architectures. Computation of the performance dierential functions i;j for the example system is done by simulation. To compute each i;j, the system is simulated with a delay i;j and with no delay. The dierence between the two simulation results divided by the magnitude of i;j is i;j. Associated with each nonzero performance dierential function i;j, there will be a time delay i;j. The matrix is shown in Figure 8b. The magnitude of i;j will be determined by the architecture of the control system and network and choice of computing hardware and software. While depends on the logical architecture (block diagram) of the system, it does not change with dierent implementation architectures. On the other hand, depends only on the implementation architecture of the system (number of processing nodes, network protocol, operating system, etc.) The combination = Pi;j i;ji;j determines the overall performance of the system. 5.3 Determination of potential architectures With n basic modules, there are Pn n! i=1 possible architectures. For each potential architecture, the corre- i!(n,i)! sponding time delay matrix needs to be found. The performance degradation function for all cases can then be easily compared and the best architecture chosen. In this design example, we tested the ve architectures shown in Figure 9. The two extreme cases completely distributed and centralized were chosen as candidates, along with three other natural groupings in which the servo controller is together with the motor. 5. Determination of time delays The delay matrix contains the magnitudes of the communication and computation delays. These delays depend onanumber of factors related to the system implementation. Communication delay mainly depends on the type of network, protocol used, data size, and number of nodes in the system. Computation delay depends on the processor power, operating system, and number of computing tasks assigned to each node. Finding accurate characterizations for computation and communication delay is another challenging problem and will not be addressed seriously in this paper (see [6] for some work in this area). For the computation delays, we assumed that each basic module has a task to be computed which takes t tk =:8msec, the time required to switch between tasks is t sw =:3msec, a motor module requires no computation time, and the computation delay is the time required for all other computation tasks on the node to be completed. For the communication delay, we assumed that the message transmission time is t tx =:5msec regardless of the size of the message, and that the communication delay is the time required for all other nodes on the network to transmit their messages. Let N be the number of nodes on the network, and n k be the number of basic modules assigned to node k. Using these assumptions, the communication delays are equal to i;j = t tx(n, 1), and the computation delay of a basic module i on node k is i;i = t tkn k +(n k, 1)t sw. Using these values, the time delay matrix can be computed for the ve dierent control architectures shown in Figure 9. Even though a simplied model was used to estimate, well dened communication and computation delay characteristics, if available, can be employed to more accurately determine the time delay matrix. In this case, each different priority assignment for the same architecture would result in a dierent time delay matrix. 5.5 Computation of and analysis After computing and, can be computed using equation (). One advantage of using the proposed performance degradation function framework is that does not need to be recomputed or modied for dierent architectures. In fact, only must be recomputed for each dierent architecture which is considered. The performance degradation function for each architecture is computed and shown in Figure 1, along with the individual components i;j. Note that (solid line) in Figure 1 is of a similar magnitude for each architecture tested. This is due to the fact that the two-axis system is symmetric, and the communication delays are all equal, causing the terms i;j to largely cancel with each other. This phenomenon was noted in Section.1. If the delays are not equal, this cancellation will not occur. Thus, when choosing an architecture, both the overall performance degradation function aswell as its components i;j should be as small as possible. Thus, for the given assumptions on the magnitudes of the time delays, the lumped servo/motor architecture (b) would be the best implementation of the system. If the reference trajectory is well-characterized (such as a step, ramp, or the circular contour considered here), then the Fourier transforms of the performance dierential func-

6 x x x a. Completely b. Lumped c. Lumped distributed servo/motor axis control x x d. Independent axes e. Centralized Figure 1: The performance degradation function for each architecture shown in Figure 9 is plotted. The dotted line indicates i;j and the thick solid line indicates. tions can be computed to extract more information about the best 's to minimize the performance degradation function. In this case, not only the impact of the magnitude of each delay to the performance of the system but also the interrelation between the delays can be seen algebraically. This information can be used to choose a communication protocol which can guarantee the desired relationships between dierent time delays. 6 Conclusions and Future Work Although more and more control systems are being implemented in a distributed fashion with networked communication, the unavoidable time delays in such systems impact the achievable performance. In this paper, wehave presented a mathematical framework for analyzing the eect of time delays on the mechanical performance of distributed control systems. Once a control algorithm has been designed, the eect of dierent implementations of this algorithm can be determined using this framework. By dening the performance degradation function as the dierence between the performance of the system with and without time delay, we are able to isolate the eect of the individual time delays from the eect of the controller. Once the individual performance dierential functions have been computed, either analytically (for linear systems) or through simulation (for nonlinear systems), the overall eect of the time delays on a distributed system can be computed in a straightforward manner. Many dierent implementation architectures can be evaluated quickly using these techniques, allowing the system integrator to choose the best one. In addition, information gained in this analysis procedure can also be used to choose a control network protocol which can guarantee the desired relationship between dierent time delays in the system. In the future, we plan to examine new ways to design control algorithms for distributed systems which can take fully utilize their advantages but minimize the impact of the unavoidable time delays. References [1] H. Chan and U. Ozguner. Closed-loop control of systems over a communication network with queues. International Journal of Control, 6:93{51, [] G. F. Franklin, J. D. Powell, and A. Emani-Naeini. Feedback Control of Dynamic Systems. Addison-Wesley, Reading, Massachusetts, third edition, 199. [3] F. Goktas, J. M. Smith, and R. Bajcsy. -synthesis for distributed control systems with network-induced delays. In Proceedings of IEEE Conference ondecision and Control, pages 813{81, [] P. K. Khosla. Choosing sampling rates for robot control. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), [5] H. Kopetz. A communication infrastructure for a fault-tolerant distributed real-time system. Control Engineering Practice, 3(8):1139{116, August [6] F.-L. Lian, J. M. Moyne, and D. M. Tilbury. Performance evaluation of control networks for manufacturing systems. In Proceedings of the ASME International Mechanical Engineering Congress and Exposition (Dynamic Systems and Control Division), [7] U. Ozguner. Decentralized and distributed control approaches and algorithms. In Proceedings of IEEE Conference ondecision and Control, pages 189{19, December [8] F. Paganini. A recursive information ow system for distributed control arrays. In Proceedings of the American Control Conference, volume 6, pages 381{385, June [9] A. Ray. Output feedback control under randomly varying distributed delays. AIAA Journal of Guidance, Control, and Dynamics, 17():7{711, 199. [1] D. Simon, E. Castaneda, and P. Freedman. Design and analysis of synchronization for real-time closed-loop control in robotics. IEEE Transactions on Control Systems Technology, 6:5{61, July [11] K. Srinivasan and P. K. Kulkarni. Cross-coupled control of biaxial feed drive mechanisms. ASME Journal of Dynamic Systems, Measurement, and Control, 11:5{3, 199. [1] J. ook, D. Tilbury, K. Chervela, and N. Soparkar. Decentralized, modular real-time control for machining applications. In Proceedings of the American Control Conference, pages 8{89, 1998.