Force Feedback for a Spine Biopsy Simulator with Volume Graphic Model

Size: px

Start display at page:

Download "Force Feedback for a Spine Biopsy Simulator with Volume Graphic Model"

Ethelbert Walsh
5 years ago
Views:

1 Proceedngs of the 2001 IEEE/RSJ Internatonal Conference on Intellgent Robots and Systems Mau, Hawa, USA, Oct Nov. 03, 2001 Force Feedback for a Spne Bopsy Smulator wth Volume Graphc Model K-Uk Kyung*, Dong-Soo Kwon*, Sung-Mn Kwon**, Heung Sk Kang***, Jong Beom Ra** * Dept. of Mech. Eng. KAIST (Tel : ; Fax : ; E-mal: kwonds@me.kast.ac.kr) ** Dept. of Electrcal Eng. & Computer Scence. KAIST (Tel : ; E-mal: jbra@ee.kast.ac.kr) *** Dept. of Dagnostc Radology, Seoul Natonal Unversty College of Medcne (E-mal: kanghs@radcom.snu.ac.kr) Korea Advanced Insttute of Scence and Technology (KAIST) 373-1, Kusong-dong, Yusong-gu, Taejon, , Korea Abstract Ths paper shows how to mplement haptc renderng for a needle nserton problem wth volume graphc model. The target s a spne bopsy smulator for tumor nspecton by needle nserton. Smulated force s calculated from the relatonshp between voxel data and the orentaton and poston of the needle. At frst, t s generated usng PHANToM TM. The needle puncturng resstve force s calculated from a stffness model and the correcton force to gve physcal constrant to the needle s derved usng a pvot. For more realstc force feedback voxel based modelng s consdered. MRI T1 value and CT densty can be used to gve physcal propertes to voxel. Soft tssue can be modeled as a damper usng MRI value n the needle nserton problem, and the stffness of hard tssue s derved from CT densty. Skn deformaton s modeled wth Kelvn s vscoelastc model. Snce the hgh stffness and large dampng need to be smulated, t s attempted to generate force usng actve actuators and passve devces together. 1. Introducton Medcal engneerng technology has kept apace wth remarkable developments n computer hardware. Such development of medcal technology has resulted n many mprovements for the publc welfare and n medcal educaton. Especally there have been large strdes n the development of medcal haptc smulators that help n the acqurng of medcal skll and knowledge. Watson et al. developed stereoscopc haptc vrtual envronments [1]. Reng et al. studed real-tme vsually and haptcally accurate surgcal smulaton. In ths study, they mplemented an algorthm to produce the sensaton of surgcal cuttng [2]. The applcaton area of force feedback s also becomng larger and larger and many basc research projects have been performed. Hemenz, Bae and Maaβ performed experments to determne the physcal characterstcs of dfferent tssues [3,4,10]. Hopkns analyzed the bomechancal characterstcs of muscle and bone [5]. Margaret developed a 3 DOF (degree of freedom) joystck that provdes force feedback. Kwon studed a 6 DOF haptc controller for a telesurgery system [6]. Sngh and Popa measured the force from the needle durng a needle bopsy and suggested a scheme to generate a smlar force profle n a needle bopsy smulaton [7,8]. Masse developed the PHANToM TM devce that has 6 DOF moton and can produce 3 DOF force [9]. Cleary and Greco appled the PHANToM TM devce to a 2D spne needle bopsy smulator [11]. A spne needle bopsy s a useful non-nvasve operaton to detect and verfy a spne tumor. For ths operaton, tssue s extracted from the spne of a lvng body usng a hollow needle. However, snce there are many crtcal organs near the spne, the spne needle bopsy operaton requres complcated and accurate procedures. Hence, much tranng s essental for relable operatons and the haptc smulator can be a valuable tool n the tranng process. In ths paper, the haptc renderng scheme for a 3D spne needle bopsy smulator wth a volume graphc model s proposed. From tme that Salsbury frst defned haptc renderng as the force relatonshp between the vrtual object and the haptc devce [16], much research has been conducted on the subject. Zlles proposed God-Object as a reference pont usng a Lagrange multpler to calculate nteracton force for a surface rendered object [17], and varous haptc surface renderng schemes have been studed through PUG (PHANToM User s Group) and others. However, lttle research have addressed haptc renderng for a volume vsualzed object. Gbson suggested a voxel-based object n haptc renderng for a volume rendered graphc model [18]. Avla showed the method usng the densty and the opacty value of voxel to determne ts physcal propertes [19]. McNeely proposed Pont-shell to make user feel the surface clearly for a voxel-based surface [20]. Ths paper s organzed as follows. We brefly ntroduce a surgcal smulator for spne needle bopsy n Secton 2. In Secton 3, we present mplementaton detals for force reflecton scheme usng PHANToM TM. In Secton 4, another mplementaton s shown for force feedback usng a new devce for 1 DOF. Fnally, we present our results and state our conclusons n Secton /01/$ IEEE 1732

Ths system s composed of three parts: a 3D human model, a vsual feedback part and a force feedback part. Fgure 1 shows the Spne Bopsy Smulator System. 2.

2 Fgure 1. The Spne Needle Bopsy Smulator 2. Spne Needle Bopsy Smulator In ths secton, we ntroduce a spne needle bopsy smulator that provdes realstc vsual and force feedback to a tranee n the PC envronment [12]. Ths system s composed of three parts: a 3D human model, a vsual feedback part and a force feedback part. Fgure 1 shows the Spne Bopsy Smulator System D Human Model A 3D human model plays an mportant role n the spne needle bopsy smulator because t s nvolved n most parts of the system. CT (Computer Tomography) mage or MRI (Magnetc Resonance Image) can be regenerated nto a 3D human volume model. The voxel sze s determned by mage resoluton and each voxel has ts own densty (CT: Hounsfeld Unt, MRI: T1, T2 value). The data set was provded by Seoul Natonal Unversty Hosptal. In order to produce segmented 3D organ models, a homemade semautomatc segmentaton tool s used. The target organs are selected as organs that should be needed n the procedure of smulatng a spne needle bopsy. Doctors n the Department of Dagnostc Radology at Seoul Natonal Unversty Hosptal determned these organs. For the lumbar vertebra regon, bone, lung, esophagus, artery, skn, muscle, and fat are selected as the target. For the thoracc vertebra regon, ven and kdney are addtonally chosen. Therefore a total of nne organs are modeled n our 3D human model. functons of a real CT system. In the 2D vrtual CT console, a tranee can select a leson that s a suspected of spnal tumor. Then he smulates scannng CT mages around the leson. Many knds of analytcal tools are provded to the tranee n ths console, such as fndng the dstance between two ponts, measurng the angle between two lnes, zoomng a specfc CT mage, etc. The second vsual feedback part s a 3D vew. In ths vsual feedback part, a 3D path plannng tool s provded for the needle path from the skn to the leson. Unlke prevous 2D smulators, t supports realstc 3D volume renderng vews nteractvely for 3D path plannng. It provdes axal, sagttal, coronal and 3D vsualzaton mages around the regon also. Especally for the 3D vsualzaton mage, a tranee selects one mage from among the color volume renderng result, gray volume renderng result, MIP (maxmum ntensty projecton), and summed voxel projecton. 2.3 Force Feedback Part Force feedback ncreases the realty n the smulaton system. The PHANToM TM haptc devce (SensAble Technologes Inc.) s used and the spne bopsy needle replaces a stylus pen n order to mplement drectonal and rotatonal force wthout any addtonal mechancal calculaton. The haptc devce s used to provde force feedback to the tranee through the bopsy needle durng smulaton. For proper force feedback generaton, the 3D human model data that contans the force feedback parameters of organs near the spne s used and the haptc devce s kept operatng at 1kHz. The tranee then nserts the spne bopsy needle nto the hollow dummy on the table and feels the force n a realstc envronment. We provde further detals n secton Smulaton Ths system s mplemented by attachng a 3 DOF PHANToM TM devce to a PC that has 600MHz Pentum III Dual CPUs and 512Mbyte RAM. A tranee smulates takng 2.2 Vsual Feedback Part Vsual feedback ncludes all the GUI (graphc user nterface) of the smulaton system. Through the GUI of the whole procedure, a tranee has a vsual experence of performng the spne bopsy. In our smulator, there are two types of vsual feedback. One s a 2D vrtual CT console and the other s a 3D vew. The 2D vrtual CT console box mmcs the GUI and Fgure 2. The physcal user nterface of the developed spne bopsy smulator. 1733

F = F + F + F (1) R C G Equaton (1) shows that the total force ( F ) has three components: 1) the force requred to penetrate tssue ( F ); R 2) a correcton force to keep the needle along the drecton

Graphc user nterface CT mages of a patent and selects a leson that s a canddate for a spne tumor n the 2D vrtual CT console.

3 F = F + F + F (1) R C G Equaton (1) shows that the total force ( F ) has three components: 1) the force requred to penetrate tssue ( F ); R 2) a correcton force to keep the needle along the drecton of movement ( F ); and 3) an ambent force to compensate C for gravty ( F ). Ths secton explans how to calculate G these forces. 3.1 Tssue Penetraton Resstve Force Fgure 3. Graphc user nterface CT mages of a patent and selects a leson that s a canddate for a spne tumor n the 2D vrtual CT console. In the 3D vew, the 256x256x256 3D human model s rendered so that the tranee can plan the approprate needle path from the skn to the leson so as not to touch the other crtcal organs. After plannng, he performs a needle bopsy vrtually usng the haptc devce. When he moves the needle, the haptc devce captures the movements of the needle and these captured movements are rendered over the 3D human model. Accordng to the poston of the needle, the tranee feels dfferent force reflecton. When he touches the leson wth the needle tp, the smulaton s over. 3. Implementaton of Force Reflecton In ths secton, force reflecton mechansm s shown usng a stffness model. The force s generated by PHANToM TM. Fgure 4 llustrates the force reflecton mechansm for the needle bopsy smulator. Ths force s smulated from the pont of ntal skn puncture untl bone s touched. In our smulator, expermental data from experments performed by Sngh was used to develop a tssue model. Sngh reported expermental results for needle punctures and developed an nteractve lumbar puncture smulator wth tactle feedback [7,8]. The drecton of the tssue penetraton resstve force depends on the needle drecton. The upper lmt of ths force s set to 6N because the PHANToM can only generate 8.5N as the peak force. Fgure 5 shows Sngh s expermental data whch was collected from a needle nserted at a constant velocty of 1 cm/sec [3]. We modeled each tssue as a sprng wthout dampng. Snce poston resoluton at a needle tp s low and the dampng dsplay of a haptc devce s poor [9], dampng s not consdered. To smulate a realstc force profle, stffness was determned wth respect to the depth of the nserted needle and the mage segmentaton value. Although the mage segmentaton classfes tssues as ar, skn, muscle, lung, ven, artery, kdney and bone, the smulated force s derved from only ar, skn, fat and bone because these are the tssues penetrated by the needle n most spne bopses. If the bopsy path touches a lung or a kdney, the force falls to zero and a red warnng sgn appears on the screen. Fgure 6 shows the algorthm used to calculate the tssue penetraton force. Xr s a reference poston to calculate force and ths pont s located on the needle. Xp s ts tp poston. k s the stffness of the tssue and vel s the speed and d s the depth of the needle nserton. Xr s determned wth the nserted needle drecton and the tssue Fgure 4. Force Reflecton Mechansm Fgure 5. Needle Puncturng Resstve Force [7] 1734

4 force, we assumed the followng: The needle s touchng only muscle or fat. The skn punctured poston s a pont and does not change. The rotatonal angle s much smaller than 1 radan. One voxel layer perpendcular to the needle drecton s homogeneous. Fgure 6. Force Generaton Algorthm that s punctured by needle. Varables, Xd, Xd2 are used to decde the tme to change Xr. It s assumed that the skn s punctured when the needle s nserted to some determned depth. Fgure 7 shows smulaton results of the resstve force. Note the result of the model s smlar to the expermental data of Fgure Correcton Force to Smulate Tssue Constrant In a real needle bopsy, the moton of the needle s constraned after nserton by the nternal tssue. Movng the needle along the drecton of the moton s not dffcult, but t s very dffcult to move to an orthogonal drecton. Snce the haptc devce used can only generate haptc translatonal forces (and not torques), an addtonal force s requred to constran the needle once t s nserted nto the dummy. To solve ths problem we make a hole on the surface of the dummy whch s used as a pvot pont to create rotatonal forces to constran the needle along the nserted drecton. Frst, the drecton vector of the needle s computed after the skn puncture. For calculaton of the correcton The frst assumpton s reasonable because the skn s very thn and there s only muscle and fat between the skn and the bone. The second assumpton s trval. Snce the sde surface of the needle s not sharp lke the tp and ts moton s constraned by nternal tssues except n the movement of the nserted drecton, the thrd one s acceptable. The forth assumpton s reasonable from the frst one. Usng these assumptons, the correcton force s calculated usng the torque derved from the layered Kelvn s Vscoelastc model. Fgure 8 shows the mechansm to calculate correcton force. b k F F 1 + = k x + b (1 + ) x (2) k k For the th voxel layer, the correcton force F s derved as equaton (2). x s a devaton from needle nserted poston, b s dampng, and k 1, k 2 are stffness at th layer. Equaton (3) shows total correcton force. C F = 0 2 F = (3) The upper lmt of ths force s 2N snce the haptc devce can only generate 8.5N and the lmt of the tssue penetraton force s 6N. From the expermental results, we verfed that ths correcton force helps to keep the needle along the ntal nserted drecton. Fgure 9 shows the expermental result. We tlted and released the needle twce after the needle was nserted. The fgure shows that the needle returned to the orgnal poston stably. 4. Implementaton of Puncturng Resstve Force from Voxel-based Physcal Propertes Fgure 7. Smulated Resstve Force Fgure 8. Puncture Resstve Force over Tme 1735

s determned. If t can be assumed that the soft tssue resstance s almost dampng, a voxel s dampng s as shown n Eq. 4. C 1 and C 2 are coeffcents that depend on the expermental condtons.

5 s determned. If t can be assumed that the soft tssue resstance s almost dampng, a voxel s dampng s as shown n Eq. 4. C 1 and C 2 are coeffcents that depend on the expermental condtons. If the needle penetrates k voxels, the puncture resstve force s as follows n Eq. 5. The dscontnuty of force can be lnear nterpolated. Fgure 9. Correcton Force usng Vscoelastc model Fgure 5 shows that the puncture resstve force s composed of skn puncture force, soft tssue puncture force and bone contact force. In Secton 3, we smulated the puncture resstve force usng a commercal haptc devce wth 3 motors. Snce the tssue s modeled as sprng, the needle jumps out when the user release t durng nserton and the user feels a pushng force whle pullng the needle out. In addton, when the needle touches the bone, vbraton occurs. Actually, the major component of real puncturng resstve force s frcton. Moreover, the used devce has lmtatons n smulatng hgh stffness. 4.1 Physcal Property of Voxel Most 3D volume mages for medcal applcaton are regenerated from CT slce mages or MRIs. Snce nner space of a volume mage s flled wth voxels, volume renderng s well-suted for a spne bopsy smulator. A voxel s a small hexahedron and the poston resoluton of a graphc model depends on the sze of voxel. If each voxel has ts own mechancal propertes and there s a devce to smulate them, a more realstc and more stable haptc volume renderng can be realzed. The skn puncture force can be modeled wth a nonlnear vscoelastc model. Soft tssue puncture force s derved from MRI value. Hemenz, Chu and Rutt studed the relatonshp between MRI value and physcal propertes [4, 21]. They showed the MRI T1 value and puncturng resstance of needle have an exponental relatonshp. Snce each voxel has ts own T1 value, ts own puncture resstance b C2T1 = C1 e (4) F R k = k = 0 b x, > (5) Bone contact force can be derved from CT value (Hounsfeld Unt). It s known that there s a lnear relatonshp between CT value and Young s modulus for the bone [10]. From the applyng of Hook s law and the solvng the smple equatons, the bone contact force can be derved. 4.2 Implementaton of Force Feedback To mplement ths proposed scheme, a haptc devce wth a hybrd actuator consst of actve actuator (motor) and passve devce (MR Brake) s used. There s a sample tssue whch has 20mm of skn deformaton and 80mm to bone. The nonlnear skn deformaton coeffcent s determned from the bomechancs of the skn, MRI T1 value of soft tssue along the needle and the CT value of the bone s determned from real measured values. The result (Fgure 11) shows stable and realstc smulated force to the needle nserted depth. Soft tssue puncture force and bone contact force s produced from the passve devce. There s no jumpng out of the needle and no needle vbraton. The tme delay at 20mm s caused from the magnetzaton tme of the MR Brake. 5. Concluson and Dscusson Fgure 10. Needle Insertng n MRI Volume Data Fgure 11. Resstve Force to Needle Inserted Depth 1736

6 Smulated force for a spne needle bopsy s calculated from the relatonshp between volume mage data and the orentaton and poston of the needle. For more realstc force reflecton n the spne needle bopsy smulator, the drectonal force of the needle was generated by a tssue model. The rotatonal force s generated usng a pvot to keep the needle n the ntal nserted drecton after puncturng the skn. We have also proposed a voxel based haptc renderng scheme for a more realstc spne bopsy smulator. Each voxel can be modeled wth mechancal parameters from MRI or CT values. Wth these approaches the stable and realstc force feedback have been obtaned. For a more advanced system two ssues need to be solved. Frst, a more generalzed haptc volume renderng scheme s requred. Snce voxels not only exst on the surface but also nsde the space object, lnkage modelng of voxels needs to be consdered. Second, a haptc devce whch has a large achevable mpedance range s needed to smulate soft organ, hgh frcton or hghly stff bone. Acknowledgements Ths research was supported by KAIST Vrtual Realty Research Center(VRRC) and the Korean Mnstry of Informaton and Communcaton as an nternatonal jont project wth ISIS(Image Scence and Informaton Systems) Center at Georgetown Unversty Medcal Center, USA. REFERENCES [1] Watson K. et al., Development of stereoscopc-haptc vrtual envronments, 12th IEEE Symposum on Computer-based medcal systems, pp , [2] Karl D. Reng et al., Real-tme Vsually and Haptcally Accurate surgcal smulaton, [3] Maaß H.and Kuhnapfel U., Nonnvasve Measurement of Elastc Propertes of Lvng Tssue, 13th Internatonal. Congress on Computer Asssted Radology and Surgery (CARS '99), Pars, France, June 23-26, pp , [4] L. Hemenz and A. Ltsky, Puncture Mechancs for the Inserton of an Epdual Needle, Amercal Socety of Bomechancs 21th Annual Meetng, [5] Davd A. Hopkns, Musculoskeletal Bomechancs Research, dahste/dahste.htm, [6] Dong-Soo Kwon, K Young Woo, and Hyung Suck Cho, Haptc Control of the Master Hand Controller for a Mcrosurgcal Telerobot System, IEEE Int. Conf. on Robotcs and Automaton, pp. 1722~1727, [7] Sunl K. Sngh and Mkael Bostrom, Desgn an Interactve lumbar Puncture Smulator wth Tactle Feedback, IEEE Internatonal Workshop on Robot and Human Communcaton, [8] Dan O. Popa et al., Creatng Realstc Force Sensatons n a Vrtual Envronment: Expermental System, Fundamental ssues and Results, Proc. of the IEEE Internatonal Conference on Robotcs and Automaton, [9] Thomas Masse, Desgn of a Force Reflectng Fngertp Stmulator, Bachelor s Thess, Department of Electrcal Engneerng and Computer Scence, MIT, [10] T. S. Bae, Estmaton of Young s Modulus and Apparent Densty of Human Trabecular Bone by Sphercal ndentaton Test and Computed Tomography, Master Thess, Department of Mechancal Engneerng, Sogang Unversty, Korea, [11] Kevn Cleary and Robert Greco, Development and Evaluaton of a Spne Bopsy Smulator, Proceedng of Medcne Meets Vrtual Realty, vol. 50, pp , [12] J. Km et al, Development of a Spne Needle Bopsy Smulator wth Vsual and Force Feedback, Proceedng of Congress on Computer Asssted Radology and Surgery (CARS) 99, pp. 1052, Pars, France, June 23-26, [13] Edward J. Colgate and Gerd G. Schenkel, Passvty of a Class of Sampled-Data Systems: Applcaton to Haptc Interface, Amercan Control Conference, pp , Baltmore [14] Paul Gorman, Thomas Krummel et al, A Prototype Haptc Lumbar Puncture Smulator, Proceedng of Medcne Meets Vrtual Realty 2000, IOS Press, vol. 70, pp , [15] S. M. Kwon, J. K. Km, and J. B. Ra, A Block-Based Volume Renderng Scheme for the Spne Bopsy Smulator, Proc. IEEE ICIP 1999, vol. 2, pp , Kobe, Japan, [16] Salsbury K., Brock D., T. Masse, N. Swarup, C. Zlles, Haptc Renderng: Programmng Touch Interacton wth Vrtual Objects, Proceedng of ACM symposum on Interacton 3D Graphcs, pp , [17] Zlles C., Salsbury K., A Conatrant-Based God- Object Method for Haptc Dsplay, IEEE Internatonal Conference on ntellgent Rorots and Systems, pp , [18] Sarah F. Gbson, Beyond Volume Renderng: Vsualzaton, Haptc Exploraton, Physcal Modelng of Voxel-based objects, Techncal Report 95-4, Mtsubsh Electrc Research Laboratores Cambrdge Research Center, [19] Avla R.S., Soberajsk L.M., Haptc Interacton Method for Volume Vsualzaton, Proceedng of IEEE Vsualzaton, pp , 1996 [20] Wllam A. McNeely, Kevn D., James J., Sx Degreeof-Freedom Haptc Renderng Usng Voxel Samplng, SIGGRAPH 99 Proceedng of the 26 th Annual Conference on Computer Graphcs, ACM, pp , [21] Chu K.C., Rutt B.K., An Ideal Phantom Materal for MR Studes of Arteral Flow and Elastcty, Journal of Magnetc Resonance n Medcne, Vol. 37, pp ,

DESIGN OF A HAPTIC DEVICE FOR EXCAVATOR EQUIPPED WITH CRUSHER

DESIGN OF A HAPTIC DEVICE FOR EXCAVATOR EQUIPPED WITH CRUSHER Kyeong Won Oh, Dongnam Km Korea Unversty, Graduate School 5Ga-1, Anam-Dong, Sungbuk-Gu, Seoul, Korea {locosk, smleast}@korea.ac.kr Jong-Hyup