FLEXIBLE robots, such as wire (or tendon or cable) driven

Transcription

1 IEEE SENSORS JOURNAL, VOL. 15, NO. 8, AUGUST Electromagnetic Positioning for Tip Tracking and Shape Sensing of Flexible Robots Shuang Song, Zheng Li, Member, IEEE, Haoyong Yu, Member, IEEE, and Hongliang Ren, Member, IEEE Abstract Wire-driven flexible robots are efficient devices for minimally invasive surgery, since they can work well in complex and confined environments. However, the real-time positional and shape information of the robot cannot be well estimated, especially when there is an unknown payload or force working on the end effector. In this paper, we propose a novel tip tracking and shape sensing method for wire-driven flexible robots. The proposed method is based on the length of each section of the robot as well as the positional and directional information of the distal end of each section of the robot. For each section, an electromagnetic sensor will be mounted at the distal end to estimate the positional and directional information. A reconstruction algorithm, which is based on a three-order Bézier curve, is carried out utilizing the positional and directional information along with the length information of the section. This method provides the advantage of good tracking results and high shape reconstruction accuracy with limited modification to the robot. Compared with other reconstruction methods, no kinematic model is needed for reconstruction. Therefore, this method works well with an unknown payload that applied at the tip of the robot. The feasibility of the proposed method is verified by simulation and experimental results. Index Terms Bézier curve, electromagnetic tracking, wire-driven flexible robot, shape sensing. I. INTRODUCTION FLEXIBLE robots, such as wire (or tendon or cable) driven manipulators [1] [4] and concentric tube robots [5], [6] have been widely studied for the use in minimally invasive surgery [7]. During a surgical operation, flexible robots may interact with tissue. Tissue will affect the position and shape of the robots, which need to be detected in real time to avoid damage to the tissue. Additionally, real time position and shape information are necessary to provide feedback to the controller to perform accurate maneuvering. Therefore, it is very important to provide shape information of the robot. One drawback of the flexible robot is that the joints rotations Manuscript received March 15, 2015; accepted April 13, Date of publication April 17, 2015; date of current version June 23, This work was supported by the Singapore Academic Research Fund under Grant R , Grant R , and Grant R The associate editor coordinating the review of this paper and approving it for publication was Dr. Patrick Ruther. (Corresponding authors: Hongliang Ren and Haoyong Yu.) S. Song, H. Yu, and H. Ren are with the Department of Biomedical Engineering, National University of Singapore, Singapore ( biesshua@nus.edu.sg; bieyhy@nus.edu.sg; ren@nus.edu.sg). Z. Li is with the Institute of Digestive Disease and Chow Yuk Ho Technology Centre for Innovative Medicine, Chinese University of Hong Kong, Hong Kong ( lizheng@cuhk.edu.hk). Color versions of one or more of the figures in this paper are available online at Digital Object Identifier /JSEN cannot be controlled independently, therefore the backbone deformation cannot be controlled as desired and consequently the actual joints rotations are unknown. As a result, the tip position and shape information of the robot cannot be well estimated when there is an external payload or force working on the end effector. Usually, backbone deformation is estimated by kinematic modeling with some assumptions, such as the piecewise constant curvature assumption [8], [9]. This method can only estimate the shape of the backbone without a payload. A more accurate shape estimation is to use the Cosserat Rod Theory [10] and incorporate the statics model. Xu and Simaan [11] proposed a method using elliptical integrals to achieve the shape restoration with a known external load. Trivedi et al. [12] presents a new approach for modeling soft robotic manipulators which incorporates the effect of material nonlinearities and distributed weight and payload. The model is geometrically exact for the large curvature, shear, torsion, and extension that often occur in these manipulators. In [13], a model based on a Rayleigh-Ritz formulation is proposed. By using the transversal tip force and distributed load as inputs, the needle deflection can be predicted. The drawback of these model based methods is that the forces or payload applied to the backbone needs be known in advance. In a real application, these forces or loads are usually unknown, therefore these model based shape estimation methods are of limited use. An alternate way is to use a sensor-based method for tip tracking and shape sensing. Medical image based methods are often used, such as Ultrasound [14] and Magnetic Resonance Imaging (MRI) [15]. Another popular technology that has been well studied is the Fiber Bragg Grating (FBG) based method [16], [17]. Usually, a number of FBG sensors are mounted in the robot. The FBG sensors can measure the axial strain of the placed position, which enables the computation of the needle curvature. The three-dimensional (3-D) robot shape can then be reconstructed from the curvature. Besides, the Electromagnetic Tracking (EMT) [18] is also studied to detected the bending characteristics of a multi-segment continuum robot in contact with the environment. Compared with other sensor based methods, the EMT method can directly provide positional and directional information. It is easy to setup and has no line-of-sight problems. In this paper, a novel tip tracking and shape sensing method for a wire-driven flexible robot is proposed. The proposed method is based on the positional and directional information of the distal end of each section of the robot along X 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See for more information.

2 4566 IEEE SENSORS JOURNAL, VOL. 15, NO. 8, AUGUST 2015 Fig. 1. Multi-section wire-driven flexible robot. The manipulator comprises a flexible backbone and a number of controlling wires. For each section, a EM sensor is mounted at the distal end. Each section will be reconstructed with two sensors. with the length of the robot. As shown in Fig. 1, the tip tracking and shape sensing system includes two parts, the multi-section wire-driven flexible Robot and the EMT system. The manipulator comprises multi sections. Each section can be controlled to bend independently and two basic shapes, a C shape or a S shape, can be achieved. For each section, an electromagnetic sensor will be mounted at the distal end to estimate the positional and directional information of the section s tip. A reconstruction algorithm, which is based on a three order Bézier Curve, is carried out utilizing the positional and directional information along with the length information of the section. This method provides the advantage of good tracking results and high shape sensing accuracy with limited modification to the robot. Compared with other shape sensing methods, no kinematic model is needed for reconstruction. Therefore, the method works well with an unknown payload. The feasibility of this method is shown by simulation and experimental results. The primary contributions of this work are summarized as follows: A shape sensing method based on Bézier Curve fitting and LM algorithm is proposed. This method only needs the positional and directional information of the distal end of each section. Compared with kinematic and statics based models, no prior payload or force information is needed. Compared with other sensor based methods, only one sensor is needed for each section, and thus very few modifications are required to the robot. The rest of this paper is organized as follows. In Section II, the electromagnetic tracking method for the tip tracking will be introduced. In Section III, the shape reconstruction method will be presented in detail. In Section IV, the simulation and experimental results will be shown. Finally, conclusions will be made in Section V. II. EMT METHOD FOR TIP TRACKING As shown in Fig. 1, an uniaxial EM sensor is mounted at the distal end tip of each section of the robot. EMT will be used to track the positional and directional information of each Fig. 2. Electromagnetic tracking system consisting of six uniaxial transmitting coils, which are stimulated sequentially. Uniaxial sensing coil is used to serve as the tracking target. Localization coordinate system is established based on the first transmitting coil. sensor and provide 5 Degree-of-Freedom (DoF) information: 3DoF position and 2DoF direction. Generally, a magnetic tracking technique uses one or more permanent magnets or electromagnetic coils as the excitation source, which generates a magnetic field that can be measured by magnetic sensors and then the position and orientation can be estimated [19] [22]. Based on these magnetic sources, there are two types of magnetic tracking, permanent magnet based tracking [19], [23] and quasi-static electromagnetic based tracking [20], [24] [26]. In contrast to other state-of-the-art tracking technologies such as mechanical optical tracking or ultrasonic tracking, magnetic tracking is emerging to provide an occlusion-free tracking scheme [27] [30]. Compared with optical tracking techniques, this occlusion-free feature brings substantial benefits for intracorporeal applications, which typically lack direct line-of-sights between the base frames and the tracked targets [31], [32]. The tracking method used in this paper is the EMT method. Most EMT technologies are based on accurate mapping of a 3D magnetic field generated by transmitting coils and computing from the field mapped the position and orientation parameters of the sensors relative to the source [20], [25], [26], [33] [35]. Compared to the permanent magnet based method, EMT has the advantages of anti-interference and a larger working space. As shown in Fig. 2, the uniaxial sensing coil is used as the target and senses the magnetic field that is generated by the six transmitting coils. These transmitting coils are stimulated sequentially. The position and orientation information of the sensing coil can then be estimated based on the sensing signals. As shown in Fig. 2, assuming that the positional and direction parameters of the sensing coil in the tracking coordinate system is (x, y, z, m, n, p), where (x, y, z) is the position information, (m, n, p) is the direction vector and m 2 + n 2 + p 2 = 1. Therefore the degree of freedom of the direction is 2.

3 SONG et al.: EM POSITIONING FOR TIP TRACKING AND SHAPE SENSING OF FLEXIBLE ROBOTS 4567 For the i-th transmitting coil, the sensing magnetic field is V i = k(mb xi + nb yi + pb zi ) (1) where k is a constant relating to the turns of transmitting coils and sensing coil. Based on the magnetic dipole model, (B xi, B yi, B zi ) is represented as follows where B xi = Q i(x a i ) R 5 i B yi = Q i(y b i ) R 5 i B zi = Q i (z c i ) R 5 i m i R 3 i n i R 3 i p i R 3 i (2) Q i = 3[m i (x a i ) + n i (y b i ) + p i (z c i )] (3) (a i, b i, c i ) is the position of the i-th transmitting coil, (m i, n i, p i ) is the direction of the i-th transmitting coil and R i is the distance between the sensing coil and the i-th transmitting coil R i = (x a i ) 2 + (y b i ) 2 + (z c i ) 2 In our setup, for each transmitting coil, (m i, n i, p i ) = (0, 0, 1). Therefore (2) can be simplified as follow B xi = 3(x a i)(z c i ) Ri 5 B yi = 3(y b i)(z c i ) R 5 (4) i B zi = 3(z c i) 2 Ri 5 1 Ri 3 We define the error evaluation function Err as follows N Err = (V i k(mb xi + nb yi + pb zi )) 2 (5) i=1 where N is the number of transmitting coils. Therefore (x, y, z, m, n, p) can be estimated with an optimization algorithm if N 5. In our EMT system, N = 6andthe LM algorithm is used to solve this least square problem. As shown in Fig. 1, a sensor is mounted at the distal end of the last section. Therefore it can provide the tip s position and direction. Thus tip tracking is realized. Note that since the uniaxial sensor is used for the tracking, we can have a 5DoF tracking result, 3DoF position and 2DoF direction. The self rotation information is missed. Therefore, torsion may not be detected. III. CURVE SHAPE RECONSTRUCTION METHOD By applying this tracking method in a robotic system, the positional and directional information of the distal end of each section can be estimated. A curve fitting method can be utilized in order to reconstruct the shape of the robot with an estimated result. This curve fitting problem needs an appropriate curve equation. When there is a large payload, the backbone will bend, and the constant curvature assumption Fig. 3. Cubic Bézier curve that used to perform the shape sensing for the flexible robot. Two kinds of curves: (a) Bézier curve with C shape and (b) Bézier curve with S shape can work well for the reconstruction of the robot with unknown payload. In the both curves, P 0 is the start point and P 3 is the end point. P 1 and P 2 are the two control points, which provide directional information for the curve. S 01 and S 23 are the length of P 0 P 1 and P 2 P 3. H 0 is the direction vector from P 0 to P 1 and H 3 is the direction vector that from P 2 to P 3. The curve starts at P 0 going toward P 1 and arrives at P 3 coming from the direction of P 2. may not stand anymore. Based on the known parameters, the Bézier curve will be used in the curve fitting, for its good performance of modelling smooth curves. In the following part, the Bézier curve and the shape reconstruction method will be introduced. A. Bézier Curve A Bézier curve is a parametric curve frequently used to model smooth curves. Take a cubic Bézier curve as an example, which will be used in the following part. As shown in Fig. 3, the cubic Bézier curve can have two kinds of space curves: the C shape curve and the S shape curve. For each curve, P 0 is the start point and P 3 is the end point; P 1 and P 2 are the control points; the curve starts at P 0 going toward P 1 and arrives at P 3 coming from the direction of P 2.

4 4568 IEEE SENSORS JOURNAL, VOL. 15, NO. 8, AUGUST 2015 Usually, it will not pass through P 1 or P 2 ; these two control points are only there to provide directional information. The distance between P 0 and P 1 determines how long the curve moves into direction P 2 before turning towards P 3. The explicit form of the curve is B(t) = (1 t) 3 P 0 + 3(1 t) 2 tp 1 + 3(1 t)t 2 P 2 + 3t 3 P 3 (6) Generally, the formula for the Bézier curve of order n can be expressed explicitly as follows: n B(t) = b i,n (t)p i (7) i=0 where t [0, 1] and b i,n is defined as follow: ( ) n b i,n = t i i (1 t) (n i) (8) From the EMT system, we can have the positional and directional information of the distal end of each section. For each section, the distal end can be seen as the start joint of the next section. Therefore, for each section, the start and end joints position and direction are known. A Bézier Curve is determined by two kinds of points, the start and end points, and the control points. The control points are located on the tangent lines of both end joints. Therefore, for a three order Bézier curve, only two unknown parameters (S 01 and S 23 that shown in Fig. 3) need to be solved. B-spline curves are often used to fit a curve. Compared to the B-spline curves, the Bézier curve has the advantages as follows: 1) the positional information of the sensors can be directly used as the start point and end point of the Bézier curve; 2) the directional information of the sensors can be directly used to express the control points with the two unknown parameters. The above advantages make the Bézier curve a very good choice for shape reconstruction for EMT based shape sensing. Besides, there is no other known position or direction information of the points between the start point and end point, therefore B-spline curves may not be a good choice for this research. B. Shape Reconstruction Method For each section in the robot, the same shape reconstruction method will be carried out. Therefore, we will use the k-th section as an example to introduce the method. The distal end of the (k 1)-th section is used as the start point of the curve; the distal end of the k-th section will be used as the end point of the curve. As shown in Fig. 4, define the parameters as follows: S 01 = P 0 P 1 S 23 = P 2 P 3 P 0 P 1 H 0 = H 3 = S 01 P 2 P 3 S 23 Fig. 4. Bézier curve. P 0 is the start point and P 3 is the end point. P 1 and P 2 are the control points. S 01 and S 23 are the length of P 0 P 1 and P 2 P 3. H 0 is the direction vector from P 0 to P 1 and H 3 is the direction vector that from P 2 to P 3. P 0, P 3, H 0 and H 3 are measured from the EM sensors. B i relates to the i-th joint of the robot. where H 0 and H 3 are the tangent vectors of the curve at point P 0 and P 3. Positional and directional parameters of the sensing coil mounted at the (k 1)-th section can be estimated from the EMT method and the results are (x 1, y 1, z 1, m 1, n 1, p 1 ). Position and direction parameters of the sensing coil mounted at the distal end of the k-th section are (x 2, y 2, z 2, m 2, n 2, p 2 ) based on the EMT result. The relationship between the cubic Bézier curve and the position and direction parameters can be established as follows ( ) x 1 m 1 x 2 m 2 P0 H 0 P 3 H 3 = y 1 n 1 y 2 n 2 (9) z 1 p 1 z 2 p 2 The relationship between P 0, P 3 and P 1, P 2 can be seen as follows: { P 1 = P 0 + S 01 H 0 P 2 = P 3 S 23 H 3 (10) Therefore, to find a cubic Bézier curve that is described with (6), the start point P 0, end point P 3 and two control points P 1 and P 2 need to be identified. P 0 and P 3 can be obtained from (9). The relationship between (P 0, P 3 ) and (P 1, P 2 ) can be seen in (10). The reconstruction problem then leads to solving the two length parameters S 01 and S 23. In order to solve the two length parameters S 01 and S 23, an error evaluation function needs to be established. The flexible robot can only bend during the operation and no stretching and contraction movement will be achieved. Based on this character, the length information of the robot can be used to establish the objective function for the optimization.

5 SONG et al.: EM POSITIONING FOR TIP TRACKING AND SHAPE SENSING OF FLEXIBLE ROBOTS 4569 Define L i as the length of the i-th vertebra and L ci as the corresponding curve length. The objective function can then be defined as L i = L ci (11) where L ci is estimated as follows: L ci = B i B i 1 = B( i n ) B(i 1 ) (12) n where n is the number of points that are used to estimate the curve length on the cubic Bézier curve. Here we define n equal to the joint number of the reconstruction section of the robot. Define the error estimation function f as follows n f = (L i L ci ) 2 (13) i=1 The parameters S 1 and S 2 can then be estimated by minimizing the objective function f. Here the LM algorithm is used to perform the optimization. Based on the curve reconstruction result, each joint s position can then be estimated based on (6) with the following equation (x i, y i, z i ) = B i = B( i n ) (14) Therefore, each joint on the robot has an estimation value and the shape reconstruction of the robot is achieved. Fig. 5. Simulation results of the single section curve. In the simulation, five curves are generated based on the kinematic model. Blue lines and circles represent the results that generated by simulation and red lines and squares represent the reconstruction results. TABLE I PARAMETERS SETUP FOR SIMULATION IV. SIMULATION AND EXPERIMENTAL RESULT In this part, first we will show some simulation result for a multi-section robot without external force to test the shape sensing method. After that, the simulation and experimental results for a single section robot with external payload will be shown. The reconstruction is carried out independently for each section. Therefore, the reconstruction result of one section will not affect the results of other sections. The only factor that would affect the results is the position and orientation results from the EMT system. Therefore, in the simulation test with external force and the real experimental test with external payload, only a single section robot will be used. A. Simulation For a Multi-Section Robot Without External Force First we will do some simulations for a multi-section robot without external force to test the reconstruction method. Three kinds of robots, from single section robot to a robot that has three sections, have been simulated. The simulation data is based on the kinematic model as shown in [36]. Fig. 5 shows the simulation result of the single section curve. Fig. 6 shows the simulation result of the two-section curve. Fig. 7 shows the simulation result of the three-section curve. For each section, it is deformed without external force or payload. From the simulation results, it can be seen that the proposed shape sensing method works well for the multi-section robot. B. Simulation For a Single Section Robot With External Force In this simulation, a single section robot with 10 joints is used to perform the test. The length of the robot in the simulation is 135mm. 6 different shapes under different external forces are generated based on a kinematic model. These forces are all applied at the distal tip position of the robot, as from the beam theory, the deformation of the beam under distributed external load is equivalent to the deformation under a lumped force and a pure moment at the backbone distal end [37], [38]. It is also the position for external tools to be mounted and the most contacted position. The parameters settings are shown in Tab. I. In the first simulation, no external load is applied. The robot suffers only gravitational force and forces from the wires when there is no external payload. The payload model is based on the kinematic model mentioned in [38]. The curve reconstruction algorithm is then used to fit the simulation data based on (10)(11). Fig. 8 shows the simulation results of each curve reconstruction, where blue lines represent the results generated by the simulation and red lines represent the reconstruction result. From Fig. 8 we can see that reconstruction results are very good. The joints of the robot all fit well. Error evaluation will be drawn in the following section.

6 4570 IEEE SENSORS JOURNAL, VOL. 15, NO. 8, AUGUST 2015 Fig. 6. Simulation results of the two-section curves. Blue lines and circles represent the results that generated by simulation. Red lines represent the reconstruction results for the first section and green lines represent the results for the second section. Fig. 7. Simulation results of the three-section curves. Blue lines and circles represent the results that generated by simulation. Red lines, green lines and purple lines represent the reconstruction results for the first, second and third section, respectively. C. Algorithm Performance Evaluation Method From Fig. 5, Fig. 6, Fig. 7 and Fig. 8 we can see that the reconstruction method performs well. For the error evaluation, the distance between the joint on the robot and the related position on the curve is used, which is shownin(15). dis = (x r i, yr i, zr i ) B i (15) where is (x r i, yr i, zr i ) is the position of the i-th joint on the robot in the simulation and B i isshownin(14).

7 SONG et al.: EM POSITIONING FOR TIP TRACKING AND SHAPE SENSING OF FLEXIBLE ROBOTS 4571 Fig. 8. Simulation results with external forces added. In the simulation, 6 curves are generated based on the kinematic model with external forces. The forces are shown in Tab. I. Blue stars and circles represent the data generated by simulation and red lines and squares represent the reconstruction results. Fig. 10. Flexible robot shape sensing system setup. An uniaxial electromagnetic sensing coil is mounted at the distal end of the robot. Two cameras are used to capture the position of each joints. The EMT system contains a system board that generates the transmitting signals and samples the sensing signals. Six transmitting coils are stimulated sequentially to generate the required magnetic filed. Fig. 9. Mean error and the standard deviation of the simulation results. The whole mean error is 1.26mm. Fig. 9 shows the errors of the simulation results of each curve reconstruction with external forces. The whole mean error is 1.26mm. D. Experiment As mentioned in the pervious part, the reconstruction method is carried out independently for each section and the reconstruction result of one section will not affect the results of other sections. Therefore, the experiments are carried out on a single section robot to validate the performance of the shape reconstruction method with an external force. As shown in Fig. 10, the length of the robot is 135mm. An uniaxial electromagnetic sensor (Aurora Shielded and Isolated 5DOF Sensor, mm) has been mounted on distal end of the robot to provide the positional and directional information of the tip, along with the shape sensing result of the robot. Two cameras are also used to capture the position of each joints. The EMT device consists of six uniaxial transmitting coils, which are stimulated sequentially with sinusoidal signal. The position information of these coils in the electromagnetic tracking coordination system are shown in Table II. The sensing data is sampled with a USB AD card and then sent to a PC for data processing, position and direction estimation and shape reconstruction. The tracking range of the uniaxial electromagnetic sensor is about 30cm 20cm 30cm. Note that the start joint of the robot is fixed, therefore P 0 and H 0 are determined by the system setup. Here in the experiment, we define P 0 and H 0 as { P 0 = (0 00) T H 0 = (1 00) T (16)

8 4572 IEEE SENSORS JOURNAL, VOL. 15, NO. 8, AUGUST 2015 TABLE II POSITION INFORMATION OF THE TRANSMITTING COILS Before the reconstruction procedure, the translation and rotation parameters between the transmitting coils and the flexible robot base are fixed. The EM sensor is first mounted on the right side of the base of the robot along with the X-axis of the robot, the reading result is (x 0, y 0, z 0, m x, n x, p x ). The EM sensor is then mounted on the same position but along with the Y-axis of the robot, the reading result is (x 0, y 0, z 0, m y, n y, p y ). The mounted position in robot coordinate system is (t, 0, 0). Therefore, the homogeneous transformation matrix from EMT coordinate system to robot coordinate system can be defined as following ( R R TE R = E te R ) (17) 0 1 Fig. 11. Experimental results with load. In the experiments, six different shape are generated and estimated. There are ten joints in the experimental robot. Blue lines represent the position results from the camera and red lines represent the reconstruction results. where and t R E m x m y m z RE R = n x n y n z (18) p x p y p z can be estimated as following x 0 t te R = y 0 RR E 0 (19) 0 z 0 where (m z, n z, p z ) T = (m x, n x, p x ) T (m y, n y, p y ) T. Therefore, during the reconstruction, the position and orientation of the tip in robot coordinate system (P 3, H 3 ) can be estimated from the results (x e, y e, z e, m e, n e, p e ) from EMT system using the following equation x e m e x e m e ( ) P3 H 3 = T E y e n e R z e p e = (TR E ) 1 y e n e z e p e (20) Six different cases that under different external forces have been the tested. Fig. 11 shows the reconstruction result of each case, where blue lines represent the position results from the camera and red lines represent the reconstruction results. Fig. 12 shows the average errors and standard deviation of the experimental results of each shape curve reconstruction in the experiments. The whole mean error is 3.02mm. The loading condition is shown in Table III. Fig. 12. Mean error and the standard deviation of the experiment results with load. The whole mean error is 3.02mm. TABLE III PARAMETERS SETUP FOR EXPERIMENTS V. DISCUSSION In this section, we will discuss some advantages and disadvantages with the proposed method.

9 SONG et al.: EM POSITIONING FOR TIP TRACKING AND SHAPE SENSING OF FLEXIBLE ROBOTS 4573 Fig. 13. Reconstruction result for a circle arc using cubic Bézier curve. The blue stars are the arcs and the red line represent the cubic Bézier curves. The first arc is 10 π and the increment for each arc is also π/10. The results get worse and worse after the radian surpassing 3π/2. A. Advantages The cubic Bézier curve is determined by four points, the starting point, the ending point and two control points. As shown in Fig. 3, the curve is a space curve and these four points are not restricted to be in the same plane. Therefore, this method can still work if a section is deformed to different planes. The reconstruction method for each section is separate from any other section. Therefore the reconstruction result of one section will not affect the results of other sections. As a result, there will be no accumulative error for a multi-section robot. Note that although the proposed method is used in the wire-driven flexible robot, there are other applications where the method can also be used. One example is concentric tube robots. Taking the advantage of the direct position and orientation estimation, the tip tracking method can be effectively used in the minimally invasive surgical tools tracking, in which optical tracking will lose efficacy because of the line-of-sight problem. B. Disadvantages For a bending section, it s better for the radian of the curve less than π. Otherwise, the method may be invalid, which can be seen in Fig. 13. The first arc is π 10 in Fig. 13 and the increment for each arc is also 10 π. For each arc, a cubic Bézier curve is used to fit the curve. From the figure we can see that the results get worse and worse after the radian surpassing 3 2 π. In the real experiments, the maximum radian that a single section can bend is no more than π 2 thanks to its structure. Therefore, the proposed method is expected to have a good reconstruction result no matter how the robot bends. VI. CONCLUSION In this paper, a novel shape reconstruction method for the wire-driven flexible robot is proposed. Electromagnetic sensors are used to provide positional and directional information of the proximal and distal tips of the bending section in the flexible robot. Then, the reconstruction algorithm, which is based on a three order Bézier Curve, is carried out utilizing the position and direction information of the sensors, together with the length information of each section of the robot. A few experiments have been performed under different payload conditions and their results have proved the feasibility of the method. The mean distance error of the joints between the reconstruction results and the camera results is 3.02mm. This method can be used to track the tip and detect the shape

10 4574 IEEE SENSORS JOURNAL, VOL. 15, NO. 8, AUGUST 2015 of the flexible robot and provide feedback information during a transoral or transnasal surgery. In the future, we will reduce the size of the robot and apply this method in a real operating room to test feasibility of the method for minimally invasive surgery. REFERENCES [1] Z.Li,R.Du,M.C.Lei,andS.M.Yuan, Designandanalysisof a biomimetic wire-driven robot arm, in Proc. Int. Mech. Eng. Congr. Expo., 2011, pp [2] G. Palli and S. Pirozzi, A miniaturized optical force sensor for tendon-driven mechatronic systems: Design and experimental evaluation, Mechatronics, vol. 22, no. 8, pp , [3] G. Palli and C. Melchiorri, Friction compensation techniques for tendon-driven robotic hands, Mechatronics, vol. 24, no. 2, pp , [4] M. Gouttefarde, D. Daney, and J.-P. Merlet, Interval-analysis-based determination of the wrench-feasible workspace of parallel cable-driven robots, IEEE Trans. Robot., vol. 27, no. 1, pp. 1 13, Feb [5] P. E. Dupont, J. Lock, B. Itkowitz, and E. Butler, Design and control of concentric-tube robots, IEEE Trans. Robot., vol. 26, no. 2, pp , Apr [6] D. C. Rucker, B. A. Jones, and R. J. Webster, A geometrically exact model for externally loaded concentric-tube continuum robots, IEEE Trans. Robot., vol. 26, no. 5, pp , Oct [7] H. Ren et al., Computer-assisted transoral surgery with flexible robotics and navigation technologies: A review of recent progress and research challenges, Critical Rev. Biomed. Eng., vol. 41, nos. 4 5, pp , [8] D. C. Rucker, R. J. Webster, III, G. S. Chirikjian, and N. J. Cowan, Equilibrium conformations of concentric-tube continuum robots, Int. J. Robot. Res., vol. 29, no. 10, pp , [9] R. J. Webster, III, and B. A. Jones, Design and kinematic modeling of constant curvature continuum robots: A review, Int. J. Robot. Res., vol. 29, no. 13, pp , [10] D. C. Rucker and R. J. Webster, Statics and dynamics of continuum robots with general tendon routing and external loading, IEEE Trans. Robot., vol. 27, no. 6, pp , Dec [11] K. Xu and N. Simaan, Analytic formulation for kinematics, statics, and shape restoration of multibackbone continuum robots via elliptic integrals, J. Mech. Robot., vol. 2, no. 1, p , [12] D. Trivedi, A. Lotfi, and C. D. Rahn, Geometrically exact models for soft robotic manipulators, IEEE Trans. Robot., vol. 24, no. 4, pp , Aug [13] R. J. Roesthuis, M. Abayazid, and S. Misra, Mechanics-based model for predicting in-plane needle deflection with multiple bends, in Proc. 4th IEEE RAS EMBS Int. Conf. Biomed. Robot. Biomechatron. (BioRob), Jun. 2012, pp [14] E. M. Boctor, M. A. Choti, E. C. Burdette, and R. J. Webster, III, Three-dimensional ultrasound-guided robotic needle placement: An experimental evaluation, Int. J. Med. Robot. Comput. Assist. Surgery, vol. 4, no. 2, pp , [15] Y.-L. Park et al., Real-time estimation of 3-D needle shape and deflection for MRI-guided interventions, IEEE/ASME Trans. Mechatronics, vol. 15, no. 6, pp , Dec [16] K. Henken, D. Van Gerwen, J. Dankelman, and J. Van Den Dobbelsteen, Accuracy of needle position measurements using fiber Bragg gratings, Minimally Invasive Therapy Allied Technol., vol. 21, no. 6, pp , [17] B. Kim, J. Ha, F. C. Park, and P. E. Dupont, Optimizing curvature sensor placement for fast, accurate shape sensing of continuum robots, in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), May/Jun. 2014, pp [18] A. Bajo and N. Simaan, Kinematics-based detection and localization of contacts along multisegment continuum robots, IEEE Trans. Robot., vol. 28, no. 2, pp , Apr [19] C. Hu, M. Li, S. Song, W. Yang, R. Zhang, and M. Q.-H. Meng, A cubic 3-axis magnetic sensor array for wirelessly tracking magnet position and orientation, IEEE Sensors J., vol. 10, no. 5, pp , May [20] C. Hu, S. Song, X. Wang, M. Q.-H. Meng, and B. Li, A novel positioning and orientation system based on three-axis magnetic coils, IEEE Trans. Magn., vol. 48, no. 7, pp , Jul [21] V. Schlageter, P. Drljaca, R. S. Popovic, and P. Kučera, A magnetic tracking system based on highly sensitive integrated Hall sensors, JSME Int. J. C, Mech. Syst., Mach. Elements Manuf., vol. 45, no. 4, pp , [22] S. Song et al., 6-D magnetic localization and orientation method for an annular magnet based on a closed-form analytical model, IEEE Trans. Magn., vol. 50, no. 9, Sep. 2014, Art. ID [23] W. Yang, C. Hu, M. Q.-H. Meng, S. Song, and H. Dai, A six-dimensional magnetic localization algorithm for a rectangular magnet objective based on a particle swarm optimizer, IEEE Trans. Magn., vol. 45, no. 8, pp , Aug [24] E. Paperno, I. Sasada, and E. Leonovich, A new method for magnetic position and orientation tracking, IEEE Trans. Magn., vol. 37, no. 4, pp , Jul [25] S. Song, W. Qiao, B. Li, C. Hu, H. Ren, and M. Q.-H. Meng, An efficient magnetic tracking method using uniaxial sensing coil, IEEE Trans. Magn., vol. 50, no. 1, Jan. 2014, Art. ID [26] S. Song, C. Hu, B. Li, X. Li, and M. Q.-H. Meng, An electromagnetic localization and orientation method based on rotating magnetic dipole, IEEE Trans. Magn., vol. 49, no. 3, pp , Mar [27] H. Ren, K. Wu, and X. Kang, Surgical navigation and planning with minimum radiation in orthopaedic interventions, Bone Joint J. Orthopaedic Proc. Suppl., vol. 95, no. SUPP 28, pp , [28] H. Ren, W. Liu, and S. Song, Towards occlusion-resistant surgical instrument tracking, Bone Joint J. Orthopaedic Proc. Suppl., vol. 95-B, no. SUPP 28, p. 52, [29] H. Ren, W. Liu, and A. Lim, Marker-based surgical instrument tracking using dual Kinect sensors, IEEE Trans. Autom. Sci. Eng., vol. 11, no. 3, pp , Jul [30] A. Hamed et al., Advances in haptics, tactile sensing, and manipulation for robot-assisted minimally invasive surgery, noninvasive surgery, and diagnosis, J. Robot., vol. 2012, 2012, Art. ID [31] J. Stoll, H. Ren, and P. E. Dupont, Passive markers for tracking surgical instruments in real-time 3-D ultrasound imaging, IEEE Trans. Med. Imag., vol. 31, no. 3, pp , Mar [32] H. Ren and P. Kazanzides, A paired-orientation alignment problem in a hybrid tracking system for computer assisted surgery, J. Intell. Robot. Syst., vol. 63, no. 2, pp , [33] S. Song, H. Ren, and H. Yu, An improved magnetic tracking method using rotating uniaxial coil with sparse points and closed form analytic solution, IEEE Sensors J., vol. 14, no. 10, pp , Oct [34] H. Ren and P. Kazanzides, Investigation of attitude tracking using an integrated inertial and magnetic navigation system for hand-held surgical instruments, IEEE/ASME Trans. Mechatronics, vol. 17, no. 2, pp , Apr [35] W.-F. Loke, T.-Y. Choi, T. Maleki, L. Papiez, B. Ziaie, and B. Jung, Magnetic tracking system for radiation therapy, IEEE Trans. Biomed. Circuits Syst., vol. 4, no. 4, pp , Aug [36] Z. Li and R. Du, Design and analysis of a bio-inspired wire-driven multi-section flexible robot, Int. J. Adv. Robot. Syst., vol. 10, pp. 1 11, Apr [37] B. A. Jones, R. L. Gray, and K. Turlapati, Three dimensional statics for continuum robotics, in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Oct. 2009, pp [38] Z. Li, R. Du, H. Yu, and H. Ren, Statics modeling of an underactuated wire-driven flexible robotic arm, in Proc. 5th IEEE RAS EMBS Int. Conf. Biomed. Robot. Biomechatron. (BioRob), Aug. 2014, pp Shuang Song received the B.S. degree in computer science and technology from North Power Electric University, Baoding, in 2007, the M.S. degree in computer architecture from the Chinese Academy of Sciences, in 2010, and the Ph.D. degree in computer application technology from the University of Chinese Academy of Sciences, China, in He is currently a Research Fellow with the Department of Biomedical Engineering, National University of Singapore (NUS), and the Singapore Institute for Neurotechnology, NUS. He mainly focuses on the area of biomedical robotics. His research interests include magnetic tracking and actuation method for bioengineering applications, such as surgical robot and micromanipulation.

11 SONG et al.: EM POSITIONING FOR TIP TRACKING AND SHAPE SENSING OF FLEXIBLE ROBOTS 4575 Zheng Li (S 11 M 13) received the B.Eng. and M.Eng. degrees in mechanical engineering from Beihang University, Beijing, China, in 2007 and 2010, respectively, and the Ph.D. degree in robotics from the Chinese University of Hong Kong, Hong Kong, in He is currently a Research Assistant Professor with the Institute of Digestive Disease and the Chow Yuk Ho Technology Centre for Innovative Medicine, the Chinese University of Hong Kong. His research interests include design, kinematic modeling, sensing and control of flexible robots for surgical applications, and biomimetic miniature robots, such as swimming robot and active capsule endoscope. Haoyong Yu (M 10) received the B.S. and M.S. degrees from Shanghai Jiao Tong University, Shanghai, China, in 1988 and 1991, respectively, and the Ph.D. degree from the Massachusetts Institute of Technology, Cambridge, MA, USA, in 2002, all in mechanical engineering. He was a Principal Member of the Technical Staff with DSO National Laboratories, Singapore, until He is currently an Assistant Professor with the Department of Biomedical Engineering and a Principal Investigator with the Singapore Institute of Neurotechnology, National University of Singapore. His areas of research include medical robotics, rehabilitation engineering and assistive technologies, system dynamics, and control. Dr. Yu was a recipient of the Outstanding Poster Award at the IEEE Life Sciences Grand Challenges Conference He served on a number of IEEE Conference Committees. Hongliang Ren received the Ph.D. degree in electronics engineering with a specialization in biomedical engineering from the Chinese University of Hong Kong, in After graduation, he was a Research Fellow with the Laboratory for Computational Sensing and Robotics and the Department of Biomedical Engineering and the Department of Computer Science, Engineering Center for Computer-Integrated Surgical Systems and Technology, The Johns Hopkins University, Baltimore, MD, USA, from 2008 to In 2010, he joined the Pediatric Cardiac Biorobotics Laboratory, Department of Cardiovascular Surgery, Children s Hospital Boston and Harvard Medical School, USA, for investigating the beating heart robotic surgery system. Prior to joining the National University of Singapore (NUS), he was involved in a collaborative computer integrated surgery project with the Surgical Innovation Institute, Children s National Medical Center, USA, in He is currently an Assistant Professor and leading the Research Group on Medical Mechatronics with the Biomedical Engineering Department, NUS. He is an affiliated Principal Investigator with the Singapore Institute of Neurotechnology and the Advanced Robotics Center, NUS. His main areas of interest include biomedical mechatronics, computer-integrated surgery, and dynamic positioning in medicine.