Softness Comparison of Stabilization Control in Remote Robot System with Force Feedback

Transcription

1 Softness Comparison of Stabilization Control in Remote Robot System with Force Feedback Qin QIAN Graduate School of Engineering Nagoya Institute of Technology Nagoya , Japan Yutaka ISHIBASHI Graduate School of Engineering Nagoya Institute of Technology Nagoya , Japan Pingguo HUANG Faculty of Business Administration Seijoh University Tokai Japan Yuichiro TATEIWA Graduate School of Engineering Nagoya Institute of Technology Nagoya , Japan Hitoshi WATANABE Department of Management Engineering Tokyo University of Science Tokyo Japan Kostas E. PSANNIS Department of Applied Informatics University of Macedonia Thessaloniki, Greece Abstract In this paper, we deal with four types of stabilization control for a remote robot system in which a user remotely operates an industrial robot arm with a force sensor by using a haptic interface device while watching video. The four types of control include the stabilization control with filters, the stabilization control by viscosity, the reaction force control upon hitting, and the switching control. To make a comparison of them quantitatively, we carry out the quality of experience (QoE) assessment in which we treat work of pushing four balls with different softness by a metal rod attached to the tip of the industrial robot arm. Index Terms remote robot system, force feedback, stabilization control, QoE assessment, softness I. INTRODUCTION Recently, remote robot systems with force feedback have been actively studied [1]-[3]. In an instance of the systems, a user at a master terminal controls an industrial robot arm at a slave terminal by operating a haptic interface device while watching video [4]. The information about shape, weight, and softness of a remote object can be transmitted through the haptic interface device. Therefore, we can greatly improve the efficiency and accuracy of work by operating the remote robot. However, the operability of the haptic interface device may seriously be degraded due to instability phenomena in the remote robot system. Also, when the remote operation is performed over a network like the Internet, which does not guarantee the quality of service (QoS) [5], the quality of experience (QoE) [6] may largely deteriorate owing to the network delay, delay jitter, and packet loss. It has become more important to solve the above problems. To this end, several types of stabilization control [4], [7]-[10] and QoS control [11] have been proposed so far. In this paper, we mainly focus on the stabilization control. In [7], control using the phase control filter in combination with the wave filter [12], [13] (called the stabilization control with filters in this paper) is applied to a remote robot system with force feedback for achieving stabilization of bilateral control. By experiment in which a user pushes a soft object with a metal rod attached to an industrial robot arm, it is demonstrated that the control can stabilize the system without large dependence on the network delay and delay jitter. However, there is a vibration problem that the robot arm jumps when the metal rod hits a hard object. Then, in [8], an improvement method under the stabilization control with filters is proposed. The effectiveness of the proposed method is demonstrated by carrying out an experiment in which balls with different softness are pushed. As a result, it is found that the improvement method is effective for some hard objects, but the method is not effective for soft objects. In [9], the stabilization control by viscosity is proposed to suppress the instability phenomena in the remote robot system. Also, in [4], the reaction force control upon hitting is proposed to prevent the industrial robot arm from jumping when the arm hits a hard object (that is, to solve the problem as mentioned earlier). Furthermore, in [10], based on results of a preliminary experiment that the stabilization control by viscosity outperforms the reaction force control upon hitting for soft objects, and the latter control is superior to the former control for hard objects, the authors propose the switching control, which switches between the stabilization control by viscosity and the reaction force control upon hitting depending on object softness. The four types of stabilization control mentioned above have their own characteristics. We need to illustrate the applicability of each control by comparison to use them effectively. Nonetheless, only in [4], QoE assessment is carried out to make a softness comparison between the reaction force control upon hitting and the stabilization control with filters. We need

2 Fig. 1. Configuration of remote robot system with force feedback. Fig. 2. Configuration of industrial robot arm. to make softness comparisons for the other combinations of the control. In this paper, the four types of stabilization control are dealt with for the remote robot system with force feedback, and we carry out QoE assessment in which we do work of pushing four balls with different softness by a metal rod attached to the tip of the industrial robot arm. The organization of this paper is as follows. Section 2 describes the remote robot system with force feedback. In Section 3, details of the four types of stabilization control are introduced. Then, the assessment method is explained in Section 4, and assessment results are presented in Section 5. We conclude the paper in Section 6. II. REMOTE ROBOT SYSTEM WITH FORCE FEEDBACK A. System Configuration Figure 1 shows the configuration of the remote robot system with force feedback. The master terminal is composed of PC for a haptic interface device (Geomagic Touch [14]) and PC for video. The two PCs are connected to each other by a switching hub. The slave terminal is composed of PC for an industrial robot and PC for video which are connected with each other through a switching hub. PC for the industrial robot is directly connected to the industrial robot via an Ethernet cable. PC for video has a Web camera (Microsoft Corp.) whose resolution is pixels. The industrial robot consists of a robot arm (RV-2F-D [15] by Mitsubishi Electric Corp.), a robot controller, a force sensor, and a force interface unit. The force sensor is attached to the surface of the flange of the robot arm, and a metal rod is fixed to the force sensor (see Fig. 2). As shown in Fig. 2, the metal platform is set at the slave terminal, and four holding jigs are installed on the metal platform to fix an object (a soft tennis ball is fixed in Fig. 2, and there are three other kinds of balls used in our assessment next to the ball) B. Remote Operation The industrial robot arm at the slave terminal can be operated remotely when a user at the master terminal presses the stylus button of the haptic interface device. The master terminal acquires the position information from the haptic interface device every millisecond, calculates the reaction force, and outputs it through the device. Then, the information about the position information is transmitted to the slave terminal by UDP. The slave terminal employs the real-time control function [16] to get the position information of the industrial robot arm and the real-time monitor function [16] to get the information about the force sensor from the robot controller every 3.5 milliseconds. The two types of information are transmitted as different packets between the robot controller and PC for the industrial robot by UDP. Then, PC for the industrial robot sends the position information of the industrial robot arm and the information about the force sensor to the master terminal.

3 At the master terminal, the reaction force F (m) t outputted at time t (ms) (t 1) against the haptic interface device is calculated as follows: F (m) t = K scale F (s) t 1 (1) where F (s) t is the force received from the slave terminal at time t. K scale is the mapping ratio about the force between the slave terminal and the master terminal (K scale = 1 [4] in this paper). At the slave terminal, the industrial robot arm is operated on the basis of the position information transmitted from the master terminal. The position vector of the industrial robot arm S t (t 1) is calculated as follows: S t = 0.5M t 1 (2) where M t is the position vector of the haptic interface device that is received from the master terminal at time t. In this paper, the mapping ratio from the work space of the industrial robot arm at the slave terminal to that of the haptic interface device at the master terminal is 1:2 [8]. Thus, M t 1 is multiplied by 0.5 to obtain S t as shown in Eq. (2). III. STABILIZATION CONTROL A. Stabilization Control with Filters The phase control filter in combination with the wave filter is used in the stabilization control with filters [12], [13]. The control can make the remote robot system with force feedback stable against any network delay. The reader is referred to [12] and [13] for details of the theory. B. Stabilization Control by Viscosity The stabilization control by viscosity employs the following calculation method of position vector S t instead of Eq. (2): S t = 0.5M t 1 C d (0.5M t 1 S t 1 ) (3) where C d is a coefficient related to viscosity for restricting the movement distance of the industrial robot to some extent [9]. The value of C d is set to 0.95 [9] in this paper. C. Reaction Force Control upon Hitting In the reaction force control upon hitting, the following calculation method of reaction force F (m) t is employed: K scale (F (m) F (m) t 1 + K if th ) t = (if F (m) t 1 K scalef (s) t 1 > F th ) (4) K scale F (s) t 1 (otherwise) where the threshold F th is set to N/ms. If F (m) t 1 K scale F (s) t 1 > F th, F (m) t is gradually increased by adding K i F th to F (m) t 1. The initial value of i is 0, and it increases by 1 every millisecond. If F (m) t 1 K scalef (s) t 1 > F th again, the value of i is reset to 0 and increases by 1 every millisecond. K i is set as follows: K 0 = 1.000, K 1 = 1.001, K 2 = 1.002, (i.e., K i = i). Otherwise, the calculation method is the same as Eq. (1). D. Switching Control The switching control switches between the stabilization control by viscosity and the reaction force control upon hitting. In the switching control, the stabilization control by viscosity is carried out for soft objects, and the reaction force control upon hitting is performed for hard objects [10]. We use a variable V flag to mark an object depending on whether the object is hard or soft. The initial value of V flag is set to 0; this means that the object is soft. As in Eq. (4), if F (m) t 1 K scalef (s) t 1 > F th, another threshold F (s) th (set to 0.06 N/ms [10]) is used. Then, F (s) (s) t 1 F t 2 is checked; if F (s) t 1 F (s) (s) t 2 > F th, we set V flag = 1 (this means that the object is hard). After checking, if V flag = 1, the calculation methods of F (m) t and S t in the reaction force control upon hitting are employed. Otherwise, we set V flag = 0 and use the calculation methods of F (m) t and S t in the stabilization control by viscosity. If F (m) t 1 K scalef (s) t 1 F th, the control is switched into the stabilization control by viscosity. IV. ASSESSMENT METHOD We carried out QoE assessment in which we did work of pushing four balls with different softness by a metal rod attached to the tip of the industrial robot arm to investigate effects of the four types of stabilization control on object softness, and then we compare the effects. In this paper, we performed the following four assessments of object softness. One is the assessment for comparison between the stabilization control by viscosity and the stabilization control with filters. Another is that between the reaction force control upon hitting and the stabilization control with filters [4]. The others are that between the switching control and the stabilization control with filters, and that between the stabilization control by viscosity and the reaction force control upon hitting. Each subject answered which stabilization control between the two types of stabilization control produced the closer feeling to that when pushing the ball with hand via the stylus of the haptic interface device in each assessment. The four balls with different softness are a sponge ball, a soft tennis ball, a rubber ball, and a hard tennis ball (see Fig. 2); the balls are harder in this order. We used a network emulator (NIST Net [17]) to add a constant delay to each packet transmitted between the master and slave terminals. Practices were done before each assessment. Firstly, each subject held each ball with hand, pushed the ball by the tip of the stylus of the haptic interface device to confirm the softness of the ball. Then, the subject practiced how to remotely operate the industrial robot arm for pushing the soft tennis ball and the hard tennis ball while watching video. In our assessment, each subject pushed each ball five times

4 for about 10 seconds. Then, the subject selected one answer among the following three answers for two types of stabilization control: The first time is closer than the second time, the second time is closer than the first time, and the first time is almost the same as the second time. In the assessment, we randomized the order which we presented combinations of the additional delay and ball for each assessment. The two types of stabilization control were also selected in random order. For simplicity, the industrial robot arm was set to move only in the vertical direction (the y-axis). There were 15 subjects whose ages were between 24 and 30, and the total assessment time per subject was about 60 minutes as in [4]. V. ASSESSMENT RESULTS A. Comparison of Stabilization Control by Viscosity and Stabilization Control with Filters We show assessment results of the stabilization control by viscosity and the stabilization control with filters in Fig. 3. The figure reveals that the stabilization control by viscosity outperforms the stabilization control with filters for soft objects (i.e., the sponge ball and the soft tennis ball), and the stabilization control with filters is superior to the stabilization control by viscosity for hard objects (i.e., the rubber ball and the hard tennis ball). In addition, as the additional delay becomes larger, both the percentages of respondents in Figs. 3 (a) and (b) tend to decrease. All the percentages are small as the additional delay is about 400 ms. In this case, it is difficult for the subjects to perceive differences among them. Therefor, it is necessary to improve the softness quality under the control when the network delay is large. B. Comparison of Reaction Force Control upon Hitting and Stabilization Control with Filters Assessment results of the reaction force control upon hitting and the stabilization control with filters are shown in Fig. 4. From the figure, we observe that for the sponge ball and the soft tennis ball, the percentages of respondents who answered that the reaction force control upon hitting is closer are large. Also, the stabilization control with filters is closer than the reaction force control upon hitting for the rubber ball and the hard tennis ball. Moreover, both the percentages in Figs. 4 (a) and (b) tend to decrease as the additional delay becomes larger. When the additional delay is about 400 ms, all the percentages are small. Thus, the quality of softness is also needed to be improved under the control. C. Comparison of Switching Control and Stabilization Control with Filters Figures 5 (a) and (b) show the percentage of respondents in the assessment with the switching control and the stabilization control with filters. From Fig. 5, we see that a number of subjects answered that the switching control is closer than the stabilization control with filters for the sponge ball and (a) Percentage of respondents who answered that stabilization control by viscosity is closer (b)percentage of respondents who answered that stabilization control with filters is closer Fig. 3. Percentage of respondents in assessment with stabilization control by viscosity and stabilization control with filters. the soft tennis ball. Conversely, for the rubber ball and the hard tennis ball, the stabilization control with filters is closer. In Fig. 5 (a), we note that the results are different from those in Subsection V.B; that is, the percentage of respondents who answered that the switching control is closer in this subsection is higher than the percentage of respondents who answered that the reaction force control upon hitting is closer in Subsection V.B for the rubber ball; note that the switching control selects the reaction force control upon hitting for the rubber ball [10]. The reason is that since the rubber ball is softer than the hard tennis ball, the switching control selects the stabilization control by viscosity immediately after the industrial robot arm hits the ball, and then switches to the reaction force control upon hitting.

5 (a) Percentage of respondents who answered that reaction force control upon hitting is closer (a) Percentage of respondents who answered that switching control is closer (b)percentage of respondents who answered that stabilization control with filters is closer Fig. 4. Percentage of respondents in assessment with reaction force control upon hitting and stabilization control with filters. D. Comparison of Stabilization Control by Viscosity and Reaction Force Control upon Hitting We show the percentages of respondents in the assessment of object softness with the stabilization control by viscosity and the reaction force control upon hitting in Fig. 6. In the figure, we find that the percentages of respondents who answered that the stabilization control by viscosity is closer are large for the sponge ball and the soft tennis ball. Besides, for the rubber ball and the hard tennis ball, the reaction force control upon hitting is closer than the stabilization control by viscosity. Based on these results, we can confirm the validity of the switching control; note that the switching control employs the stabilization control by viscosity for soft objects and the reaction force control upon hitting for hard objects. In addition, the figure reveals that the percentages of respondents are 0 when the additional delay is larger than about 300 ms. This (b)percentage of respondents who answered that stabilization control with filters is closer Fig. 5. Percentage of respondents in assessment with switching control and stabilization control with filters. means that all the subjects answered the first time is almost the same as the second time. As mentioned earlier, we need to improve the quality of softness under the control. VI. CONCLUSION In this paper, we compared the effects of the following four types of stabilization control on object softness for a remote robot system with force feedback by QoE assessment: The stabilization control with filters, the stabilization control by viscosity, the reaction force control upon hitting, and the switching control. As a result, we found that the switching control is the most effective among the four types of stabilization control for soft objects, and the stabilization control with filters is the most effective for hard objects. As the next step of our research, we need to improve the softness quality under the four types of control when the network delay is large. Furthermore, we plan to study combination use of QoS control and the four types of stabilization control.

6 (a) Percentage of respondents who answered that stabilization control by viscosity is closer (b)percentage of respondents who answered that reaction force control upon hitting is closer Fig. 6. Percentage of respondents in assessment with stabilization control by viscosity and reaction force control upon hitting. ACKNOWLEDGMENT The authors thank Takafumi Rikiishi for his help for our assessment. This work was partly supported by JSPS KAK- ENHI Grant Number 18K11261 and the Telecommunications Advancement Foundation. [3] Y. Maeda, K. Suzuki, Y. Ishibashi, and S. Fukushima, Influence of network delay on work efficiency in remote robot control with force feedback, (in Japanses), IEICE Technical Report, CQ , Jan [4] R. Arima, P. Huang, Y. Ishibashi, and Y. Tateiwa, Softness assessment of objects in remote robot system with haptics: Comparison between reaction force control upon hitting and stabilization control, (in Japanese), IEICE Technical Report, CQ , Jan [5] ITU-T Rec. I. 350, General aspects of quality of service and network performance in digital networks, Mar [6] TU-T Rec. G. 100/P. 10 Amendment 1, New appendix I - Definition of quality of experience (QoE), [7] P. Huang, T. Miyoshi, and Y. Ishibashi, Stabilization of bilateral control in remote robot system, (in Japanese), IEICE Technical Report, CQ , Mar [8] P. Huang, Y. Toyoda, E. Taguchi, T. Miyoshi, and Y. Ishibashi, Improvement of haptic quality in stabilization control of remote robot system, (in Japanese), IEICE Technical Report, CQ , Nov [9] T. Rikiishi, Y. Ishibashi, P. Huang, T. Miyoshi, H. Ohnishi, Y. Tateiwa, K. E. Psannis, and H. Watanabe, Effect of stabilization control by viscosity in remote robot system, (in Japanese), IEICE Technical Report, MVE , Sep [10] Q. Qian, Y. Toyoda, Y. Ishibashi, P. Huang, and Y. Tateiwa, Switching between stabilization control by viscosity and reaction force control upon hitting in remote robot system with haptics, IEICE Global Conference, BS-2-14, Mar [11] K. Suzuki, Y. Maeda, Y. Ishibashi, and N. Fukushima, Improvement of operability in remote robot control with force feedback, in Proc. The 4th IEEE Global Conference on Consumer Electronics (GCCE), pp , Oct (The 3rd Prize of Student Paper Award). [12] T. Miyoshi, K. Terashima, and M. Buss, A design method of wave filter for stabilizing non-passive operating system, Proc. IEEE CCA, pp , Oct [13] M. D. Duong, T. Miyoshi, K. Terashima, and E. J. Rodriguezseda, Analysis and design of position-force teleoperation with scattering matrix, Proc. 17th IFAC World Congress, pp , July [14] Geomagic, [15] RV-2F-D Series: Standard Specifications, co.jp/dl/fa/members/document/manual/robot/bfp-a8899/ bfp-a8899u.pdf. [16] CRn-500 Series: Ethernet Interface Instruction Manual, [17] M Carson and D Santay, NIST Net - A Linux-based network emulation tool, ACM SIGCOMM, vol 33, no 3, pp , July REFERENCES [1] K. Ohnishi, Real world haptics : Its principle and future prospects, (in Japanese), IEEJ, vol. 133, no. 5, pp , Mar [2] T. Kawai, Haptics for surgery, (in Japanese), IEEJ, vol. 133, no. 5, pp , Mar