APPLICATION OF PREDICTION-BASED PARTICLE FILTERS FOR TELEOPERATIONS OVER THE INTERNET

APPLICATION OF PREDICTION-BASED PARTICLE FILTERS FOR TELEOPERATIONS OVER THE INTERNET Jae-young Lee, Shahram Payandeh, and Ljljana Trajovć School of Engneerng Scence Smon Fraser Unversty 8888 Unversty Drve, Burnaby, BC V5A S6, Canada jla55@sfu.ca, shahram@cs.sfu.ca, ljlja@cs.sfu.ca ABSTRACT In ths paper, we present the predcton-based partcle flter approach for processng moton and force data n teleoperaton over the Internet. We frst ntroduce the predcton-based partcle flter algorthm, one of the Sequental Monte Carlo methods based on the recursve Bayesan predcton. The predcton algorthm s appled to dynamc models of the moton and force data flows n the state-space formulaton. It s appled to the moton data transmtted to the slave controller and to the reflectng force data receved at the master controller. Experments are performed usng the haptc devce wthn a vrtual 3D graphcal envronment. In each experment, the moton and reflectng force data extracted from the haptc devce are used to verfy the predcton performance of the proposed method. KEY WORDS Teleoperaton, Internet, Partcle flter, Sequental Monte Carlo method, Bayesan predcton. Introducton Internet-based teleoperaton s an nteractve applcaton where a human user transmts moton data from a master controller whle smultaneously recevng the reflectng force data from a slave controller statoned n a remote envronment. Unle most other Internet applcatons that requre relable data transmsson, for stable operaton nteractve applcatons need to mantan a constant frequency of data transmsson. Hence, n a teleoperaton, both the tme delay and the relable data transmsson should be consdered. The end-to-end Internet tme delay conssts of the propagaton delay, transmsson delay, processng delay, and queung delay. Unle the frst three delay components, the queung delay vares wth tme due to the Internet traffc condtons. The transport control protocol (TCP) and user datagram protocol (UDP) are two wdely used Internet transport protocols. TCP, whch provdes relable data transmsson, often ntroduces large varable delay due to ts retransmsson scheme and congeston control. Thus, t has been suggested that UDP be employed for teleoperaton even though t does not guarantee relable data transmsson and may lead to data losses []. Many approaches have been proposed to solve the tme delay and data loss ssues n teleoperaton over the Internet. Varous control systems approaches have been suggested, ncludng the wave-varable transformaton and ts extensons [2], [3]. Predcton-based sgnal processng approaches that perform moton and force predctons have been also proposed [4], [5]. The Kalman flter method, whch provdes a recursve soluton to the lnear predcton and estmaton, was proposed as a predcton-based approach [6]. The moton and reflectng force data are mpared by the presence of the Internet delay. Hence, these sgnal processng approaches are expected to compensate for the delays that vary over tme. Nevertheless, the moton and reflectng force data are often dffcult to predct f they nvolve nonlnear and non-gaussan system characterstcs. For example, hand movement patterns from a master controller can be hghly nonlnear and the tradtonal Kalman flter may fal to provde accurate predcton. The reflectng force data may be even more dffcult to predct snce the data need to be sent at relatvely hgh frequences n order to ensure realstc and contnuous force. The partcle flter algorthm, also nown as the bootstrap flter or the Condensaton flter, s a Sequental Monte Carlo (SMC) method that provdes suboptmal solutons to the recursve Bayesan approach [7], [8]. Due to ts robust predcton and estmaton performances n nonlnear and non-gaussan envronments, the algorthm has been wdely used n communcatons, mage and speech sgnal processng, and robotcs [9]-[2]. The partcle flter method may be appled to any nonlnear dynamc model usng a state-space framewor, and, hence, t can be appled to the dynamc

models of the moton and force data n the state-space formulaton. In ths paper, we present the predcton-based partcle flter algorthm to predct the moton and reflectng force data that suffer from the Internet delays that vary over tme. In Secton 2, we ntroduce the generc partcle flter algorthm wth the predcton-based formulaton. In Secton 3, we dscuss the nonlnear state-space models of the moton and reflectng force data, and address the ssues dealng wth predcton of data usng the partcle flter algorthm. Expermental results wth the mplemented predcton-based partcle flter algorthm are gven n Secton 4. We conclude wth Secton 5. 2. Predcton-Based Partcle Flter Algorthm A dscrete-tme dynamc system may be represented usng a state-space model, where unnown states of the system are predcted or estmated based on avalable nosy observatons. The partcle flter method performs suboptmal predcton and estmaton wthn the recursve Bayesan approach n case when the dynamc system s nonlnear and non-gaussan. Usng the partcle flter method, the true posteror densty n such nonlnear and non-gaussan dynamc systems can be approxmated by a smulaton-based approach. The dscrete tme nonlnear state-space model can be expressed as: x g ( x, u ), () x h ( x, v ) where x and x are respectvely the state and the observaton system at tme, g and h are nonlnear state and observaton transton functons, and u and v are state and observaton nose sequences, whch may be non-gaussan. In a state-space model, the predcton of the true state at tme can be obtaned based on the current state x and avalable observatons x :. Based on the recursve Bayesan approach, the optmal predctor of the true state at tme can be expressed by the condtonal means: xˆ x p( x x ) dx, (2) : where x ˆ denotes the one-step-ahead predcton of the state x gven avalable observatons x:. Accordng to the Bayesan approach, the posteror densty should be evaluated recursvely solvng two densty functons [8]: p( x x ) p( x x ) p( x x ) dx (3) : : p( x x ) : p( x x ) p( x x ) p( x x ) :. (4) : Equatons (3) and (4) are respectvely the predcton and update procedures for fndng the optmal soluton. They are not computatonally tractable due to the ntegral forms. Hence, as a suboptmal soluton, the partcle flter method s used to approxmate the posteror denstes. Based on the predcton-based partcle flter algorthm, (3) can be approxmated as [3], [4]: N s : p( x x ) w ( x x ), (5) where N s s the number of partcles, (.) s the Drac delta functon, and computed as: w w s the mportance weght that can be p( x x ) p( x x ). (6)! w q( x x, x ) The mportance densty q (.) may be chosen to be equal to the pror densty n order to mnmze the varance of the mportance weghts: q( x x, x ) p( x x ). (7) Hence, the mportance weght can be smplfed as: w w p( x x ). (8) Pror to performng the resamplng step, the mportance weghts (8) should be normalzed so that w. The smplfed llustraton of the partcle flter algorthm ncludng the resamplng step s shown n Fgure. After N s number of partcles s randomly dstrbuted n the frst teraton, the mportance weghts are computed for each partcle n order to obtan the approxmaton of p( x x : ). The resamplng step s then performed to regenerate the predcted samples based on the weghted samples. In the resamplng process, the partcles wth small weghts are elmnated whle the partcles wth hgh weghts are concentrated. The large number of partcles gves more accurate stochastc approxmaton, whch n general provdes the reasonable predcton performance. However, an effcent number of partcles may be selected to avod computatonal burden. The predcton-based partcle flter algorthm s descrbed n Table.

,..., Ns partcles { x, Ns } ( a) { x, w } ( b) { x, Ns } ( c) { x, Ns } ( d) { x, w } Fgure. Graphcal representaton of the partcle flter algorthm [8]. Startng from the ntal state (a), llustrated are the weghted measure (b), resamplng (c), and predcton of next state (d). Step. Intalzaton Table Predcton-based partcle flter algorthm. Draw the ntal state randomly and defne ntal parameters. Step 2. Predcton Draw x ~ p( x x ), N Step3. Update s Evaluate mportance weghts accordng to (8) and normalze the weghts. Step 4. Resamplng Multply/suppress samples wth hgh/low mportance weghts. Step 5. Iteraton Increase tme step and go bac to Step 2. 3. Predcton of Moton and Force Usng the Partcle Flter Algorthm The moton data transmtted from the master controller may be mpared by the Internet delay. Hence, the recevng moton data at the slave controller need to be predcted n order to compensate for the delay. The predcted moton data, consstng of postons over tme samples, may cause a contact wth an object or a surroundng envronment, whch n turn generates reflectng force data. The force data that feeds nto the master controller may also suffer from a smlar Internet delay. Therefore, the reflectng force data also need to be predcted n order to compensate for the delay. The moton and reflectng force data sequences may be formulated as state-space models. The state-space formulatons of the moton and reflectng force data n a teleoperaton scenaro are shown n Fgure 2. 3. Moton Predcton The moton data transmtted from the master controller consst of postons over tme samples. As shown n Fgure 2, the moton data, whch may be nonlnear or non-gaussan, are formulated as a state-space model. The true moton data are then predcted usng the predcton-based partcle flter algorthm based on avalable observatons, whch are mpared by the Internet delays. For a sngle degree of freedom (DoF) teleoperaton system, the true poston x at tme s transmtted va the Internet and t s delayed by n tme steps. The mpared observaton receved at the slave controller s denoted as x n. In a state-space model, the predcton of the true poston at tme n can be obtaned based on the current state x n and avalable observatons x: n. Hence, as n (2), x ˆ n n represents the one-step-ahead predcton of the state x n gven avalable observatons x: n. Usng the predcton-based partcle flter method, x ˆ n n can be computed by approxmatng the posteror densty functon p( x n x: n) based on (5)-(8) evaluated at tme n. In order to mplement the partcle flter algorthm to the moton data predcton problem, each step ntroduced n Secton 2 should be executed. After the ntalzaton step that randomly defnes the ntal state of the moton model, the predcton step s performed to obtan samples x n from the pror densty p( x n x n ), where N s. In the update step, the new state x n s assgned by usng mportance weghts (8). Snce the mportance densty q (.) was chosen to be the pror densty and the state and observaton nose was assumed Gaussan, the evaluaton of the mportance weghts can be smplfed as: 2 x n x n 2 w n e. (9) Equaton (9) gves the mportance weghts of the th partcle at tme n and needs to be normalzed for the resamplng. For the resamplng step, the new set of states x n s determned based on the mportance weghts. After cumulatve dstrbuton functons (CDFs) of the normalzed weghts are constructed, each element of the CDFs s compared wth the unformly dstrbuted functon to determne the low or hgh weghts. The new set of states s then regenerated based on the hgh weghted samples.

Delayed networ x x n Partcle flter xˆ n Master controller Slave controller fˆ n Partcle flter f n f Fgure 2. State-space formulatons of moton and reflectng force data n the teleoperaton scenaro and the proposed predcton scheme usng the partcle flter method. 3.2 Force Predcton Predcton of the reflectng force data can be acheved by a smlar approach. The force data generated by a contact wth an object or a surroundng envronment are fed nto the master controller. The transmsson of the force data over the Internet can also be formulated as a nonlnear or non-gaussan state-space model, as shown n Fgure 2. The true force data are then predcted by the predcton-based partcle flter algorthm gven the avalable observatons. Snce the force data must be sampled at a relatvely hgh frequency rate (above,000 Hz) n order to acheve realstc and contnuous force, they are relatvely dffcult to predct. Let the true force data generated from the slave controller at tme be f n a sngle DoF teleoperaton system. Ths s the feedbac force data delayed n tme steps and transmtted over the Internet. The force data at master controller receved through the Internet s vewed as the mpared observaton and t can be expressed as f n. Smlar to the poston predcton problem, the one-step-ahead predcton of the state f n gven avalable observatons f: n can be represented as ˆ f n n. Based on the predcton-based partcle flter approach, the posteror densty functon p( f n f: n) can be approxmated by computng the mportance weghts based on (5)-(8). The mplementaton of the partcle flter algorthm to the force data can be acheved by applyng steps shown n Table. The dynamc model of the reflectng force data n the state-space framewor and ts predcton approach usng the partcle flter method are llustrated n Fgure 2. 4. Experments In order to verfy the proposed predcton-based partcle flter method for a teleoperaton scenaro, we performed experments usng the PHANTOM Destop haptc devce. In conjuncton wth vrtual 3D graphcal envronment, the haptc devce provdes postonng nput whle recevng feedbac force by a 6-DoF manpulaton. In ths expermental scenaro, the PHANTOM Destop was used as a master controller where a human operator provdes moton data. A 3D vrtual teleoperator model was programmed as a slave controller usng the C++ and OpenGL lbrares. The vrtual teleoperator based on the 4-DoF Selectve Complance Assembly Robot Arm (SCARA) model was desgned. Accordng to the movement from the master controller, the contact force that feeds nto the master controller s generated when the tp of the SCARA model colldes wth objects. Note that the postons of the 4-DoF SCARA model are mapped to the master controller so that the PHANTOM Destop s only capable of manpulatng 4-DoF. The expermental scenaro consstng of the PHANTOM Destop haptc devce nterfaced wth the 3D vrtual teleoperator s shown n Fgure 3. Master controller Moton predcton Internet Force predcton Slave controller Fgure 3. Predcton-based teleoperaton expermental scenaro: The PHANTOM Destop haptc devce and the vrtual 3D graphcal representaton are used for master and slave controllers, respectvely. In the expermental scenaro, we smulated the Internet tme delay model between the master and slave controllers. TCP provdes relable data transmsson between the controllers. However, due to the TCP retransmsson and congeston control mechansm, large fluctuatons of tme delay may be ntroduced. Hence, t has been suggested that UDP may be used for the teleoperaton because t ntroduces

Delay (ms) relatvely low tme delay varatons. The expermental scenaro provded the smulated Internet tme delay model typcally observed n the UDP transmsson [], [6]. The model that ntroduces fluctuaton and jtter s shown n Fgure 4. The maxmum and average tme delay was 32 ms and 63 ms, respectvely, over a fve-second nterval. In ths experment, we assumed that the moton data transmtted to the slave controller and the force data receved at the master controller experenced dentcal delay, shown n Fgure 4. 200 80 60 40 20 00 80 60 40 20 0 0 2 3 4 5 Tme (s) Fgure 4. The Internet tme delay model wth varatons. 500 partcles were used. For smplcty, the state and observaton noses were assumed to be Gaussan wth zero means and unt varances. The predcted moton data and reflectng force data are shown n Fgure 6. In general, the large number of partcles gves mproved performances n the moton and force predctons. However, the large number of partcles ntroduces computatonal complexty, whch may ntroduce tme delay. Snce the teleoperaton s an nteractve applcaton that requres the real-tme operaton, such computatonal requrements may adversely affect performng real-tme operatons. Hence, t may be necessary to effcently select the number of partcles. The mean square error (MSE) of the moton and reflectng force predctons based on the selected number of partcles s shown n Table 2. The number of partcles can be effcently selected whle mantanng the MSE performances. Number of partcles Table 2 MSE vs. number of partcles. Moton error (mm) Force error (Newton) 00 4.423 0.46 200 4.366 0.406 300 4.037 0.373 400 4.027 0.358 500 4.009 0.357 To verfy the predcton performance of the partcle flter method, one-dmensonal moton and reflectng force data at each stage were extracted over a fve-second nterval. The expermental scenaro s based on the 3D vrtual representaton. The samplng rate of the moton data was 50 Hz to enable the human eye to perceve contnuous moton. The samplng rate of the reflectng force data rendered by the PHANTOM Destop haptc devce was,000 Hz n order to mantan realstc and contnuous force. It s advsed that the moton data and force data should be sampled at no less than 30 Hz and,000 Hz, respectvely. The one-dmensonal true moton and reflectng force data obtaned from the master and slave controllers are shown n Fgure 5. The observed moton and force data that are delayed based on the tme delay model shown n Fgure 4 are also shown n Fgure 5. The delayed moton and force data were mpared by the tme delay model due to the fluctuaton of tme delay. The predcton-based partcle flter method was mplemented n order to predct the moton and reflectng force data. In both moton and force predcton cases, 00 to 5. Concluson In ths paper, we consdered the predcton of moton and force for teleoperaton over the Internet by usng the partcle flter algorthm. The predcton-based partcle flter was ntroduced and was appled to the moton and reflectng force predctons n a tme-varyng networ such as the Internet. The experments were performed usng the PHANTOM Destop haptc devce n a vrtual 3D graphcal envronment. The experments showed that the predcton-based partcle flter algorthm successfully performed the one-step-ahead predctons of the moton and force data. Ths paper addressed one of the sgnal processng approaches to overcome the Internet delay n teleoperaton scenaros. Sgnal processng solutons may also need to be combned wth relable control systems n order to mprove blateral teleoperaton systems. More effcent partcle flter algorthms also need to be desgned to address the complexty and operaton tme ssues by adaptvely selectng the number of partcles [5].

Axs Poston (mm) Axs Force (N) Axs Poston (mm) Axs Force (N) 70 60 True poston Delayed poston 3 True force Delayed force 50 2 40 30 20 0 0 00 90 - -2 80 2 3 4 5 Tme (s) -3 2 3 4 5 Tme (s) Fgure 5. True and delayed moton data (left) and true and delayed feedbac force data (rght) at the master and slave controllers over a fve-second nterval. 70 60 True poston Predcted poston 3 True force Predcted force 50 2 40 30 20 0 0 00 90 - -2 80-3 2 3 4 5 2 3 4 5 Tme (s) Tme (s) Fgure 6. Predcted moton data (left) and feedbac force data (rght) at the slave and master controllers over a fve-second nterval. References [] E. Kamran, H. Momen, and A. Sharafat, Modelng Internet delay dynamcs for teleoperaton, IEEE Internatonal Conference on Control Applcatons, Toronto, ON, Canada, Aug. 2005, pp. 528-533. [2] G. Nemeyer and J. J. E. Slotne, Desgnng force reflectng teleoperators wth large tme delays to appear as vrtual tools, IEEE Internatonal Conference on Robotcs and Automaton, Albuquerque, NM, USA, Apr. 997, pp. 222 228. [3] K. Kawashma, K. Tadano, G. Sanaranarayana, and B. Hannaford, Blateral teleoperaton wth tme delay usng modfed wave varables, IEEE Internatonal Conference on Intellgent Robots and Systems, Nce, France, Sept. 2008, pp. 424-429. [4] S. Clare, G. Schllhuber, M. Zach, and H. Ulbrch, Predcton-based methods for teleoperaton across delayed networs, Sprnger-Verlag, Multmeda Systems, vol. 3, no. 4, Oct. 2007. [5] S. Clare, G. Schllhuber, M. Zach, and H. Ulbrch, The effects of smulated nerta and force predcton on delayed telepresence, Presence, vol. 6, no.5, pp. 543-558, Oct. 2007. [6] S. Munr and W. Boo, Internet-based teleoperaton usng wave varables wth predcton, IEEE Transactons on Mechatroncs, vol. 7, no. 2, pp. 24-33, June 2002. [7] M. Sanjeev Arulampalam, S. Masell, N. Gordon, and T. Clapp, A tutoral on partcle flters for onlne nonlnear/non-gaussan Bayesan tracng, IEEE Transactons on Sgnal Processng, vol. 50, no. 2, pp. 74-88, Feb. 2002. [8] A. Doucet, N. de Fretas, and N. Gordon, An ntroducton to sequental Monte Carlo methods, n Sequental Monte Carlo Methods n Practce, A. Doucet, N, Fretas, and N. Gordon, Eds. New Yor: Sprnger-Verlag, 200, pp. 4-4. [9] Y. Lu and S. Payandeh, Cooperatve hybrd mult-camera tracng for people survellance, IEEE Canadan Journal of Electrcal and Computer Engneerng, vol. 33, no. 3, pp. 45-52, 2008. [0] R. Mottag and S. Payandeh, Coordnaton of multple agents for probablstc object tracng, IEEE Canadan Conference on Computer and Robot Vson, Vctora, BC, Canada, May 2005, pp. 62-67. [] R. Mottag and S. Payandeh, An overvew of a probablstc tracer for multple cooperatve tracng agents, IEEE Internatonal Conference on Advanced Robotcs, Seattle, WA, USA, July 2005, pp. 888-894. [2] D. Schulz, and D. Fox, Bayesan color estmaton for adaptve vson-based robot localzaton, IEEE Internatonal Conference on Intellgent Robots and Systems, Senda, Japan, Sept. 2004, vol. 2, pp. 884-889. [3] G. Ktagawa and S. Sato, Monte Carlo smoothng and self-organsng state-space model, n Sequental Monte Carlo Methods n Practce, A. Doucet, N, Fretas, and N. Gordon, Eds. New Yor: Sprnger-Verlag, 200, pp. 77-95. [4] F. Desbouvres and B. At-El-Fquh, Drect versus predcton-based partcle flter algorthm, IEEE Worshop on Machne Learnng for Sgnal Processng, Cancun, Mexco, Oct. 2008, pp. 303-308. [5] D. Fox, Adaptng the sample sze n partcle flters through KLD-samplng, Internatonal Journal of Robotcs Research, vol. 22, no. 2, pp. 985-003, 2003.