A Calligraphy Robot - Callibot: Design, Analysis and Applications

Transcription

1 Proceeding of the IEEE International Conference on Robotics and Biomimetics (ROBIO) Shenzhen, China, December 13 A Calligraphy Robot - Callibot: Design, Analysis and Applications Yuandong Sun and Yangsheng Xu Abstract Combining functions with aesthetics, characters with drawings, Chinese calligraphy is a unique type of traditional art in China. To mimic the special writing skills, we develop a calligraphy robot which consists of one 6-DOF robot arm, one linear rail and one paper conveyor. This calligraphy robot, called Callibot, has several features. Firstly, it has sufficient degrees of freedom and very few singularities which improve the manipulability of the robot. Secondly, Callibot could mimic real motion of human writing because playback from demonstration approach is adopted. The encoders will record the joint positions while demonstrating. Then Callibot filters out the noises (error readings in serial port communication) and repeats the motion. Thirdly, the workspace of the robot arm is largely expanded due to the linear rail and the paper conveyor. The experimental results show that Callibot is capable of writing a large piece of aesthetic calligraphic work. I. INTRODUCTION Chinese calligraphy is an attractive traditional art in China with a history of more than four thousand years [1]. It is an art of writing Chinese characters with a brush, thereby expressing the aesthetic and emotion of the author. Due to the complexity of Chinese characters and flexibility of the hairy brush, it is an extremely tough task for human to write aesthetic calligraphy. Usually, training calligraphic skills is divided into three stages [1]. The first stage is to learn from demonstration. The student s hand is held by a teacher and guided by the teacher to write calligraphy. Then the student could feel the motion of the brush and establish the basic mapping from the calligraphic strokes or characters to the brush motion. The second stage is to imitate from copybook. Without the help of the teacher, the student imitates the calligraphy in the copybook based on the mapping established in the first stage. The third stage is to write in a particular style. Once he has learned a style from a copybook or created his own style, the student is capable of writing any characters in a particular style. Three calligraphy robots were developed before. They were all designed to write calligraphy in the second stage. F. Yao et al. [][3] modeled the trajectory of regular script and seal script so that the 5-DOF calligraphy robot is able to replicate calligraphy. K. Zhang et al. [4] proposed a sensor management method based on fuzzy decision tree (FDT) which can help control the 4-DOF calligraphy robot. Differs Yuandong Sun is with Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong. ydsun@mae.cuhk.edu.hk Yangsheng Xu is with Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong, and Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China. ysxu@cuhk.edu.hk from the arm-type robot which are used in [] - [4], the Intelligent Control Systems Laboratory of the Chinese University of Hong Kong developed a robotic drawing platform containing a 5-DOF manipulator which can move in x, y and z translational directions and two additional rotation joints. These five degrees of freedom can be controlled independently so as to avoid kinematic problem. K. W. Lo et al. [5] described a system for brush footprint acquisition. Through analysis of captured footprint and the characters, the robot platform is able to determine the required (x, y, z) position of the brush tip. K. W. Kwok et al. [6] improved the approach proposed in [4] by applying genetic algorithm to generate required footprints. J. H. M. Lam et al. [7] proposed an approach to generate stroke trajectories for 5-DOF robot which is compatible with the robot platform. There are some common problems of these three calligraphy robots. Firstly, the degrees of freedom of the robot are not sufficient. Unlike pen or pencil whose tip is solid, brush tip is soft so that different orientations of the brush would create different footprints. Therefore, 6-DOF is indispensable for writing aesthetic calligraphy. Secondly, while planning the motion, most of them did not consider the orientation of the brush. All of them skip the first stage and move directly to the second stage. Since it is not intuitive to acquire the orientation of the brush from character images, the robots in [] - [6] all keep the brush vertical and only control x, y and z positions. However, orientation of the brush is even more important than position for aesthetic calligraphy. So in [7], J. H. M. Lam et al. proposed an approach to generate 5-DOF motion. However, this motion still does not perfectly match with the real motion of human writing. Thirdly, the workspace is limited. In this paper, we propose a calligraphy robot, called Callibot, to tackle these problems. A 6-DOF robot arm is developed. Its joint configuration is different from common 6-DOF robot arm (such as PUMA) which contributes to very few singularities. We apply playback from demonstration approach. Different from learning from demonstration, only one demonstration is provided. The encoders will record the joint positions while demonstrating. Then the robot arm filters out the noises (error readings in serial port communication) and repeats the motion of writing. The robot arm is installed on a linear rail and a paper conveyor is developed. Therefore, the workspace is expanded extensively so that Callibot is capable of writing a large piece of aesthetic calligraphic work. The rest of the paper is organized as follows. Section II describes the design and properties of Callibot. Section III shows the preliminary applications. Some issues are /13/$ IEEE 185

2 Fig.. Assignment of the coordinate frames. L 1 =.718m, L =.5m, L 3 =.1133m, L 4 =.94m, L 5 =.115m. θ 1 = 8 8, θ = 1 1, θ 3 = 8, θ 4 = 18 18, θ 5 = 1 1, θ 6 = TABLE I D-H PARAMETERS OF THE ROBOT (a) i α i 1 a i 1 d i θ i 1 θ 1 L 1 θ 3 9 L θ L 3 θ θ θ 6 Fig. 1. (b) Callibot: (a) prototype and (b) sketch discussed in Section IV. Section V concludes the paper. II. ROBOT DESIGN AND ANALYSIS A. System Structure The prototype and the sketch of Callibot are shown in Fig. 1. Callibot consists of one 6-DOF robot arm, one linear rail and one paper conveyor. The linear rail and the paper conveyor are only used for expanding the workspace of the robot arm. Currently, the robot arm will not move until the linear rail and the paper conveyor stop at a proper position. Therefore, different from those redundant robot, this robotic system can still be regarded as a 6-DOF robot arm. B. Forward Kinematics The assignment of the coordinate frames is shown in Fig. (the subscript bt is short for brush-tip). And the D-H parameters [8] are shown in Table I. The link transformations are 1T = cos θ 1 sin θ 1 sin θ 1 cos θ 1 1, (1) 1 T = 3T = 3 4T = 4 5T = 5 6T = cos θ sin θ L 1 sin θ cos θ 1 sin θ 3 cos θ 3 L 1 cos θ 3 sin θ 3 cos θ 4 sin θ 4 1 L 3 sin θ 4 cos θ 4 cos θ 5 sin θ 5 1 sin θ 5 cos θ 5 cos θ 6 sin θ 6 1 sin θ 6 cos θ 6 Therefore, the forward kinematics is C. Inverse Kinematics, (), (3), (4), (5). (6) 6T = 1T 1 T 3T 3 4T 4 5T 5 6T. (7) Refer to Fig., given the target position and orientation of brush-tip as follows: P = [ p x p y p z, (8) d 1 = [ d 1x d 1y d 1z, (9) d = [ d x d y d z, (1) 186

3 Fig. 3. Joint 1 and Joint Fig. 4. Joint 4 and Joint 5 the position of Q is Q = P L 4 d 1 + L 5 d = [ q x q y q z. (11) The procedure of solving inverse kinematics is divided into four steps. 1) Step 1 (Solving θ 3 ): The position of Q is independent of the rotation of the last three joints. And the z coordinate is only related to the rotation of Joint 3. Therefore, or θ 3 = arcsin q z L 3, (1) θ 3 = π + arcsin q z L 3. (13) ) Step (Solving θ 1 and θ ): Joint 1 and Joint are sketched in Fig. 3. From geometric relation between Fig. and Fig. 3, L = L + L 3 cos θ 3. (14) Therefore, if L + L 3 cos θ 3 >, θ 1 = atan (q y, q x ) α, (15) Fig. 5. Joint 6 3) Step 3 (Solving θ 4 and θ 5 ): Joint 4 and Joint 5 are sketched in Fig. 4. The orientation vector d 1 which is expressed in the world coordinate frame can be derived as d 1 = 1R 1 R 3R 3 d 1, () where i 1 i R is the rotation matrix which has been derived from forward kinematics (the upper left 3 3 matrix of i 1 i T) and 3 d 1 is the orientation vector expressed in Frame{3}. Therefore, 3 d 1 = 3R T 1 R T 1R T d 1 = [ ] d 1x d 1y d T 1z. (3) or where θ = α + β, (16) θ 1 = atan (q y, q x ) + α, (17) θ = (α + β), (18) α = arccos L 1 + ( ) qx + qy L, (19) L 1 qx + qy β = arccos L + ( qx + qy) L 1. () qx + qy L Take the direction of rotation into consideration, or θ 4 = atan (d 1z, d 1x), (4) ( ) θ 5 = atan d 1x + d 1z, d 1y, (5) θ 4 = π + atan (d 1z, d 1x), (6) ( ) θ 5 = atan d 1x + d 1z, d 1y. (7) 4) Step 4 (Solving θ 6 ): Joint 6 is sketched in Fig. 5. Similar to Step 3, 5 d = 4 5R T 3 4R T 3R T 1 R T 1R T d = [ ] d x d y d T z, (8) If L + L 3 cos θ 3 <, { θ1 = θ 1 θ = θ + π. (1) θ 6 = atan ( d z, d x). (9) Theoretically, there are eight analytic solutions in total. While in practice, (13) will not happen according to the 187

4 joint limit. (6) and (7) are fairly redundant, because the configuration of the robot under these conditions is the same as under (4) and (5). Actually, two solutions to the inverse kinematics are meaningful in practice. D. Jacobian and Singularities A similar approach is proposed by F. T. Cheng et al. [9] for solving the singularities of PUMA. The two Jacobian matrices that relate joint rates and the velocities at P and Q have the following form (each block is a 3 3 matrix): [ ] J11 J J P = 1, (3) J 1 J [ ] J J Q = (31) J 1 J The singularities happen when det (J P ) =. Now we are going to prove that det (J P ) = det (J Q ) = det ( J 11) det (J ) (3) so as to simplify the calculation. From Fig., P = Q + L 4 d 1 L 5 d. Assume only Joint i is rotating, we have and Therefore, d j = θ i z i d j, i = 1,..., 6, j = 1, (33) d j = d j θ i θ i, i = 1,..., 6, j = 1,. (34) d j θ i = z i d j = d j,z d j,y d j,z d j,x d j,y d j,x z i. (35) Then we will have [ ] P J11 J 1 = θ = Q θ + L d 1 4 θ L d 5 θ = [ J ] [ ] (36) A J1 J, where L 4 d 1z L 5 d z L 5 d y L 4 d 1y A= L 5 d z L 4 d 1z L 4 d 1x L 5 d x. L 4 d 1y L 5 d y L 5 d x L 4 d 1x (37) Hence [ ] [ ] [ ] J11 J J P = 1 I3 A J = J 1 J 3 3 I 3 J 1 J [ ] (38) I3 A = J 3 3 I Q. 3 Therefore, det (J P ) = det (J Q ) = det ( J 11) det (J ). (39) The two sub-matrices J 11 and J are presented in (4) and (41), where c i = cos θ i, s i = sin θ i, c ij = cos (θ i + θ j ) and s ij = sin (θ i + θ j ). Omit the process of calculating determinant, the results are shown below: det ( J 11) = L1 L 3 s c 3 (L + L 3 c 3 ), (4) Fig. 6. Translational velocity ellipsoid when θ = 5 det (J ) = s 5. (43) The singularities happen when θ =, θ = 18, θ 3 = ±9, θ 3 = ± arccos( L /L 3 ), θ 5 = and θ 5 = 18. Refer to the caption of Fig., only θ = (boundary singularity) and θ 5 = (wrist singularity) will happen due to the range limits of the joints. Therefore, it is possible to easily avoid these two singularities while doing demonstrations and planning the trajectories. The manipulability of Callibot near singularities can be measured as follows. P. Corke [1] gives the detail description. Assume the joint velocities have a unit norm θ T θ = 1. From the relations between Cartesian velocities ẋ and joint velocities θ, ẋ T ( J Q (θ) J Q (θ) T ) 1 ẋ = 1, (44) which is the equation of a 6-dimensional ellipsoid. If one of the radii is very small (small eigenvalue of J Q (θ) J Q (θ) T ), the robot cannot achieve velocity in the corresponding direction (the eigenvector corresponding to the eigenvalue). We plot the ellipsoid corresponding to translational velocity (corresponding to the upper left 3 3 matrix of J Q (θ) J Q (θ) T ) near the boundary singularity at θ = [ 5 3 ] T in Fig. 6. We can see that the radius in x direction is very small which means the velocity in x direction is difficult to achieve. We check the wrist singularity at θ = [ 3 ] T. The [ eigenvector corresponding ] to zero eigenvalue is T This means the velocity in this direction cannot be achieved. Therefore, while planning the motion, Callibot needs to avoid moving through the singularities in the corresponding directions. III. PRELIMINARY APPLICATIONS A. Verification of Inverse Kinematics and Jacobian Although it is not taken into much consideration since we are at the first stage (learning from demonstration), solving inverse kinematics and Jacobian matrix is of great importance 188

5 J11 L1 s1 s1 (L + L3 c3 ) s1 (L + L3 c3 ) L3 c1 s3 c1 (L + L3 c3 ) L3 s1 s3 = L1 c1 + c1 (L + L3 c3 ) L3 c3 J c1 c3 = s1 c3 s3 c1 s3 s4 s1 c4 s1 s3 s4 + c1 c4 c3 s4 (4) c1 s3 c4 s5 + c1 c3 c5 s1 s4 s5 s1 s3 c4 s5 + s1 c3 c5 + c1 s4 s5 c3 c4 s5 s3 c5 (a) (41) (b) Fig. 8. Calligraphic character of Water : (a) the demonstrated one and (b) the playback one (a) (b) Fig. 7. Calligraphic characters (short for The Chinese University of Hong Kong) written by solving inverse kinematics: (a) without velocity constraints and (b) with velocity constraints is provided which means no learning process applied. The experimental results are shown in Fig. 8 (the character of Water ). The playback one (right) is very similar to the demonstration one (left). After that, we are confident about writing some complicated work using this approach. The experimental results meaning Science and Art are shown in Fig. 9 (the size is about 35cm 45cm). when we move to the second and third stage. Therefore, we have conducted two basic experiments to verify the correctness of equations derived in Section II. We only use the start and the end of a stroke as the target position and orientation to solve inverse kinematics first. The experimental results are shown in Fig. 7(a) (the word is short for The Chinese University of Hong Kong). Without constraints on the process of writing, the strokes may be curved which affect the beauty of the characters. Then we use Jacobian to keep the Cartesian velocity constant during writing one stroke. We periodically read the joint positions of the robot and use (45) to determine the required joint speed θ (x is constant while writing one stroke). The experimental results are shown in Fig. 7(b) (same word as Fig. 7(a)). The strokes are much better than those in the first experiment. 1 θ = JP (θ) x. (45) Consequently, the equations derived in Section II are correct and Callibot is capable of writing calligraphic characters. Fig. 9. Playback results of a large piece of calligraphic work meaning Science and Art B. Playback from Demonstration Other experiments are based on playback from demonstration approach. The encoders will record the positions of the motors in every 6ms while we are holding the brush and writing. Thereafter, Callibot filters out the noises (error readings in serial port communication) and repeats the motion according to the recorded positions. Different from learning from demonstration, only one demonstration IV. DISCUSSIONS A. Features of Callibot 1) Sufficient DOFs and Few Singularities: The DOFs of Callibot are sufficient for writing calligraphy which is a big improvement compared with all the previous calligraphy robots. One of the most famous calligraphist in China, Mi Fu 189

6 ( ), was so proud of his calligraphy and named his calligraphy brushing calligraphy (just like brushing paint on the wall) [11]. Therefore, 6-DOFs is the minimum requirement for producing various orientations of the brush. Moreover, Callibot encounters very few singularities (one boundary singularity and one wrist singularity), which benefits significantly for the process of demonstration and motion planning. ) Vivid Motion: Playback from demonstration approach is applied. Therefore, the motion of the robot is as vivid as human writing. This not only contributes to better visual effect, but also more aesthetic calligraphic work. The biggest difference between robot and human is that robot could record the demonstrating motion exactly. Therefore, although without any learning process, Callibot is capable of writing aesthetic calligraphy. 3) Expanded Workspace: Benefited from the linear rail and paper conveyor, the workspace of Callibot is largely expanded. So it is capable of writing a large piece of calligraphic work which is shown in the previous section. 4) Analytic Solutions to Inverse Kinematics: As stated by K. W. Lo et al. [5], one of the advantages of the calligraphy robot is that all the five degrees of freedom are independently controlled in order to avoid kinematics problem. In our design, benefited from the adoption of spherical wrist, solving inverse kinematics becomes intuitive and analytic solutions exist. Therefore, the kinematics will not be a problem in Callibot. B. Future Work Although the playback results are good, we believe that learning from several demonstrations will improve the performance. Then through the learning process, if we could establish a mapping from the strokes to the robot motion, we will be able to move to the second stage. Combining with our previous work [1] which has the potential of synthesizing calligraphy in various styles, we will be able to move to the third stage. This will come out a real robotic calligraphist. Fund, CUHK/41781 of General Research Fund, and RFD 1/13 in the Centre for Research on Robotics and Smartcity, CUHK/SC. The authors would like to thank Mr. Yong Yang for his advise on the design of calligraphy robot and Mr. Chi Zhang for his help in assembling the robot. REFERENCES [1] L. L. Y. Chang and P. Miller. Four Thousand Years of Chinese Calligraphy. University of Chicago Press, 199. [] F. Yao, G. Shao and J. Yi. Extracting the Trajectory of Writing Brush in Chinese Character Calligraphy. Engineering Applications of Artificial Intelligence, 17(6): , 4. [3] F. Yao and G. Shao. Modeling of Ancient-style Chinese Character and Its Application to CCC Robot. Proceedings of IEEE International Conference on Networking, Sensing and Control, 7-77, August 6. [4] K. Zhang and J. Su. On Sensor Management of Calligraphic Robot. Proceedings of the International Conference on Robotics and Automation, , April 5. [5] K. W. Lo, K. W. Kwok, S. M. Wong and Y. Yam. Brush Footprint Acquisition and Preliminary Analysis for Chinese Calligraphy using a Robot Drawing Platform. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, , October 6. [6] K. W. Kwok, K. W. Lo, S. M. Wong and Y. Yam. Evolutionary Replication of Calligraphic Characters by A Robot Drawing Platform Using Experimentally Acquired Brush Footprint. Proceedings of the International Conference on Automation Science and Engineering, , October 6. [7] J. H. M. Lam and Y. Yam. Stroke Trajectory Generation Experiment for a Robotic Chinese Calligrapher Using a Geometric Brush Footprint Model. Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, 315-3, October 9. [8] J. J. Craig. Introduction to Robotics: Mechanics and Control (3rd Edition). Prentice Hall, 4. [9] F. T. Cheng, T. L. Hour, Y. Y. Sun and T. H. Chen. Study and Resolution of Singularities for a 6-DOF PUMA Manipulator. IEEE Transactions on Systems, Man and Cybernetics Part B: Cybernetics, 7(), , [1] P. Corke. Robotics: Vision and Control. Springer, 11. [11] þ ÖxÑ. 5{ Ö{Ø À6(Treatises on Calligraphy in Successive Dynasties). þ ÖxÑ. þ, 9. [1] Y. Sun, N. Ding, H. Qian and Y. Xu. A Robot for Classifying Chinese Calligraphic Types and Styles. Proceedings of IEEE International Conference on Robotics and Automation, , May 13. V. CONCLUSIONS In this paper, we propose a calligraphy robot, called Callibot, which consists of one 6-DOF robot arm, one linear rail and one paper conveyor. With sufficient degrees of freedom (6-DOF), Callibot is feasible for writing calligraphy. Due to its configuration, Callibot encounters very few singularities (one boundary singularity and one wrist singularity). This benefits significantly for the process of demonstration and motion planning. The workspace of the robot arm is largely expanded due to the linear rail and the paper conveyor. Playback from demonstration approach is adopted. Therefore, Callibot could repeat the real motion of human writing. From the experimental results, Callibot is capable of writing a large piece of aesthetic calligraphic work. VI. ACKNOWLEDGMENTS This research is jointly supported by the projects GH- P/9/11GD from Hong Kong Innovation and Technology 19