Hybrid Signal-based and Geometry-based Prediction for Haptic Data Reduction

Transcription

1 Hybri Signal-base an Geomery-base Preicion for Hapic Daa Reucion Xiao Xu, Julius Kammerl, Rahul Chauhari an Eckehar Seinbach nsiue for Meia Technology, Technische Universiä München, Germany {xiao.xu, kammerl, rahul.chauhari, Absrac Hapic aa reucion schemes aress he high packe-rae requiremens of neworke hapics. Percepionriven preicive coing approaches enable srong packe rae reucion while keeping he inrouce isorion below human hapic percepion hreshols. The performance of preicive coing is srongly influence by facors such as human behavior, sysem characerisics, geomeric an impeance properies of he environmen, ec. n his paper, we firs escribe a novel surface geomery-base preicion approach for hapic aa reucion where local objec surface feaures are approximae wih he help of simple geomeric moels. Seconly, we presen a hybri framework ha combines signal-base an geomerybase preicion. Psychophysical experimens are performe o valiae his framework. The resuls of he propose geomerybase preicion show an improvemen in hapic aa reucion of abou 54% as compare o he signal-base preicion (linear preicor). Furhermore, he presene hybri preicion echnique allows for an aiional gain of 15%.. NTRODUCTON A elepresence an eleacion (TPTA) sysem is compose of wo main pars: a human operaor (OP) / maser sysem an a eleoperaor (TOP) / slave sysem [1]. The TOP is conrolle by he OP while ineracing wih he remoe environmen. The wo subsysems exchange hapic aa over he nework, as visualize in Fig. 1. The user movemen (posiion / velociy) capure by a hapic evice a he OP sie commans a slave robo a he TOP sie. The slave follows hese commans an reurns he force (an orque) feeback signals sense uring is ineracion wih he remoe environmen. The hapic evice isplays he force an orque o he OP hrough he hapic evice which allows him o hapically immerse ino he remoe environmen. For real-ime hapic eleoperaion, hapic signals on eiher sie of he communicaion channel nee o be sample (an packeize immeiaely) wih a rae of 1 khz. This is necessary for sabiliy as well as ransparency reasons []. When running a TPTA sysem across a packe-base communicaion nework, such a high packe rae as well as he aiional aa overhea ue o he ransmission of packe heaer informaion lea o inefficien communicaion. Early approaches aressing hapic aa compression can be foun in [3]. These approaches, however, concenrae on he saisical properies of hapic signals. The firs percepual eaban-base approach for real-ime hapic aa compression has been presene in [4], [5], [6] an invesigae in OP/Maser Posiion Velociy Local loop Nework Local loop Force Torque TOP/Slave Fig. 1. Srucure overview of he TPTA sysem (aape from [1]). mage source: hp:// erms of sabiliy crieria in [7] an [8]. n [9], a linear preicor has been ae o furher reuce he hapic velociy an force-feeback packe raes. Signal-base preicion mehos in combinaion wih he eaban approach o furher reuce he hapic aa rae are presene in [10] an [11]. n [1], a moel-meiae approach is presene o rener he hapic signal locally wih a geomeric moel of remoe objecs for a TPTA sysem wih significan communicaion elay. n [13], a geomery moel-base preicive coing exension for he eaban approach has been briefly menione, which preics he signal samples base on he surface geomeric srucure an impeance of he remoe objecs. This allows for local renering of he remoely generae or sense hapic forcefeeback values. n his paper, we make wo conribuions. Firsly, we provie a eaile escripion of he surface geomery-base preicion approach for hapic aa reucion which was briefly escribe earlier in [13]. n his approach, local objec surface feaures are esimae by simple geomeric primiives. This allows for local renering of he hapic signals an furher reuces he packe rae of he hapic aa in comparison o signal-base preicion. Seconly, we combine signal-base an geomery-base preicion in a hybri framework ha swiches beween he preicors aapively. The hybri preicion meho enhances he subjecive qualiy along wih a furher reucion of packe rae wih respec o each iniviual preicor. The res of his paper is organize as follows: Secion reviews briefly he percepual eaban hapic aa reucion approach. Secion 3 eails various preicive coing algorihms incluing signal-base an geomery-base preicors.

2 The srucure of he propose hybri preicion framework is explaine in Secion 4. Secion 5 escribes he psychophysical ess conuce o valiae he hybri preicion framework. The experimenal resuls are iscusse an analyze in Secion 6. We conclue his paper in Secion 7 wih a summary of he resuls an ieas for fuure work. A. Weber s Law. PERCEPTUAL HAPTC DATA REDUCTON Weber s law escribes he perceivabiliy of he change of a simuli in a pairwise comparison of he simuli ([14], [15]). The size of he ifference hreshol, or jus noiceable ifference (JND), is expresse as a linear funcion of simulus inensiy. This relaionship can be represene by he following equaion: = k = consan where is he simulus inensiy, is he change in simulus inensiy which is perceivable jus as ofen as i is no an k is a consan calle he Weber facor. Wih some variaion, Weber s law has been foun o apply o mos of he human senses incluing he hapic sense [15]. B. Percepual Deaban Coing Weber s law has been exploie for hapic aa reucion by he lossy eaban coing approach in [6]. The eaban encoer oupus a hapic sample for ransmission only when i excees he JND wih respec o he las ransmie sample value. Oherwise, nohing is ransmie. On he ecoer sie, he incoming hapic signal is reconsruce in he following manner. When a sample value is receive, i is sen for isplay o he hapic evice. A oher sampling insans when no upae is receive, he las receive hapic sample is hel. The principle of he eaban approach is illusrae in Fig.. The black fille circles represen sample values ha require ransmission accoring o he eaban coing algorihm. The hapic samples represene by empy circles, on he oher han, o no require ransmission, as hey o no consiue perceivable changes in he hapic signal.. PREDCTVE CODNG n orer o furher reuce he number of ransmie packes, a preicion scheme for hapic signals can be employe. Only if he preice hapic sample values iffer from he incoming signal value by more han he JND, he preicion error nees o be communicae [9], [6]. n his paper, we iscuss wo ifferen preicion mehos, a signal-base an a novel geomery-base preicion meho. A. Signal-base Preicion For he signal-base preicion, we eploy wo preicion moes: he zero-orer-hol (ZOH) moe an he linear firs orer moe. The zero orer preicion is escribe in Secion -B. The linear preicor [9] works as follows, { fnew new value arrive f i = f i 1 + fnew 1 fnew new 1 new ( i i 1 ) else (a) (b) Fig.. 1-DoF eaban approach as escribe in [6]. The inpu signal (a) is ownsample an only he values represene wih black fille circles are ransmie. n (b), he oupu signal is upsample using he zero-orer-hol meho. npu Signals Preice Signals Fig. 3. Principle of linear preicion. The re values are ransmie an use o preic he curren hapic value. f he preice value iffers by more han he JND from he acual value, a new value is ransmie an he preicor is upae. where {f i, f i 1, f i,... } are he mos recen preicion force values given by he preicor an { i, i 1, i,...} are he corresponing ime insances. {f new, f new 1, f new,... } an { new, new 1, new,...} are he las receive forces an he corresponing ime insances, respecively. Accoring o he formula above, he preice force value lies on a sraigh line eermine by he las wo receive force values (Fig. 3). Once he ifference beween he preice value an he acual value is larger han he JND, a new value f new is ransmie an he parameers of he preicor are upae accoringly. B. Geomery-base Preicion While ouching a surface, we perceive imporan informaion such as posiion, siffness, fricion, ec. We may use his informaion o also preic conac poins on he same surface ha we have no ouche ye. nspire by his observaion, we propose a geomery-base preicion meho which preics fuure signal samples base on a locally vali geomery moel which is buil from previous conac poins wih he

3 Slave Environmen Nework HS Human n f s f s h r f p f e f s f s T c Peneraion eph h O T c (a) (b) f h f m f p Fig. 5. Device posiion an is peneraion eph for he plane moel (a) an sphere moel (b). p h p m OP Moel Fig. 4. Principle of geomery-base preicion. Force-posiion pairs are use o esimae preicion moels escribing geomeric an impeance properies of he environmen. The moel is use o escribe he combine impeance of he nework an he slave in conac wih he environmen. remoe objec. This moel is solely base on ransmie hapic signals, wihou he nee for ransmiing aiional sie informaion. By combining absolue posiion an remoely receive force-feeback informaion, we are able o moel geomeric an physical properies of he eleoperaor conac wih he remoe environmen. Similar o he concep of moelmeiae elemanipulaion [1], his allows for preicing he hapic signals by locally renering he conac evens. n his conex, he parameer values of such local moels shoul be esimae as quickly as possible in orer o ensure conrolloop sabiliy an ransparency. These requiremens eman simple moels wih small parameer ses. Our propose scheme is illusrae in Fig. 4. On he maser sie, p h enoes he posiion comman by he human an p m he resuling maser posiion. On he slave sie, p s = p m enoes he esire slave posiion, while enoes he acual resuling slave posiion. On he oher han, f e enoes he conac force wih he remoe environmen, while f s enoes he force sensor oupu. fm = f s enoes he esire force feeback, wih f h enoing he acual force applie o he human operaor. The eploye moel preics he force feeback signal f p base on he incoming posiion signal p s (=p m ) a boh, he maser an he slave sie. A he slave sie, he moel preicion is use o evaluae he correcness of he preicion. Only if he ifference beween he preice force f p an he acual force f s excees he applie percepion hreshols (ecie by he eaban verificaion - DBV), we ransmi hapic signals (conrolle by he rigger signal T c ). Accoringly, he maser can isplay he preice samples as long as no hapic upaes are receive. p m Moel esimaion & hapic renering n orer o locally preic remoe force feeback base on he sense posiion signal p s, we nee o moel he geomeric an impeance properies of he environmen. Noe ha he moele impeance properies represen he combine impeance of he nework an he slave in conac wih he environmen (see Fig. 4). n he following, we show ha by combining he esire slave posiion p s an he corresponing measure force feeback signal f s, we are able o esimae he parameers of our preicion moels. To minimize he amoun of samples necessary for esimaing remoe objecs, our moels only focus on he isplaye siffness an geomeric surface srucure of noneformable objecs. We neglec surface fricion, slave ineria an nework elay. Furhermore, he geomeric characerisics of he environmen are moele wih he help of simple planar an spherical moels for escribing planar, convex an concave surface srucures. n he following, we escribe hese moels in more eail. Plane moel A plane surface in 3D space can be expresse as follows: ax + by + cz + = 0 (1) where n = (a, b, c) is he normal vecor of he plane (see Fig. 5 (a)). n absence of surface fricion, he irecion of he acual force f s = (f x, f y, f z ) is ienical o he plane normal n. So n = ( fx f, fy s f, fz s f ). s Furhermore, assuming ha we know he siffness of he slave in conac wih he environmen, we can use Hooke s law o esimae he surface conac poin = f s /s + p s Where p s = (p x, p y, p z ) an s is he siffness. Combine wih (1), we ge ( f x f s + f ( ) y f s + f z 1/s f s, 1) = ( f xp x f s + f yp y f s + f zp z f s ) n his equaion he only unknown variables are he siffness s an plane parameer. Therefore, by collecing a leas wo

4 force-posiion pairs, an solving a sysem of linear equaions, we can efine he plane moel (Fig. 6 (lef)). n orer o use he moel for local hapic renering, he peneraion eph h can be compue as follows: h = ap x + bp y + cp z + a + b + c f h > 0, he curren evice poin is ousie he objec. So he preice force shoul be 0, oherwise he preice force can be compue by Hooke s law: { h 0 fp = 0 no ouche h < 0 f p = s h ouche Fig. 6. Geomeric plane (lef) an sphere (righ) moels are buil from he receive force-posiion pairs. Wih hese moels he preicor a boh he TOP an he OP sies can rener he hapic force locally, if he JND hreshol is no violae. Sphere moel A sphere is efine by he cener o = (o x, o y, o z ) an is raius r (Fig. 5 (b)). The poin = o + r shoul lie on he sphere s surface. Uner he assumpion of no surface fricion, he acual force f s poins from he cener o of he sphere, raially ouwar. Taking Hooke s law ino accoun, we can ge he force sample ino he equaion: where r = r fs o + r f s /s = p s. n marix-vecor noaion form, f x f x f y f x f z f x o x o y o z = r 1/s Here we have five unknown variables an hree equaions. Therefore, o solve his sysem of linear equaions, we again nee a leas wo force-posiion pairs. n orer o use he sphere moel for force preicion, we nee o eermine he peneraion eph h an he force irecion. f s > 0, he surface is convex, oherwise i is concave. So epening upon wheher we are in conac wih a virual objec or no, an wheher we have eermine a convex or a concave surface moel, he preice force f p can be compue as: p x p y p z p p s o r s s o p s o if p s o < r, s > 0 f p = p o p s o r s s o p s if p s o > r, s < 0 0 else Fig. 6 (righ), shows an example of a local sphere moel of he objec surface. V. HYBRD PREDCTOR So far we have inrouce four kins of preicors (ZOH, linear, plane-base an sphere-base). These preicors show ifferen performance in ifferen siuaions. For example, if he remoe objec exhibis simple geomeric surfaces, e.g. a wall, a esk or a rigi ball or ege, ec, he geomeric preicors ouperform he signal-base preicion. On he oher sie, if he surface consiss of complex srucures, he signal-base preicion performs beer. n aiion, he user s exploraion sraegy (apping, moving along he surface, pressing, ec.) can also srongly affec he performance of he preicors. n orer o selec he bes preicion scheme which woul allow bes possible aa reucion performance, we propose a hybri preicion framework. n our hybri preicion scheme, nework ransmission is only riggere if a new hapic signal upae nees o be ransmie. The hybri preicion runs all 4 preicors in parallel. is base on a ecision funcion combine wih a hol-lasselece-preicor sraegy. The ecision funcion is expresse as, w i = (f i p f s ) / f s where w i is he error value for he ih preicor, f i p is he preice force from he ih preicor an f s is he acual force from he TOP. This enables us o selec he jh preicor, which has he smalles preicion error w j. Wih he hol-las-selece-preicor sraegy, he curren preicion scheme is use unil he preice signal violaes he applie percepion bouns. Wih he ransmie packe, all he preicors are upae an he new preicion scheme is selece accoring o he ecision funcion. V. SUBJECTVE TESTS n orer o evaluae he performance of he propose preicion mehos, we evelope a virual environmen for simulaing he TPTA sysem. n psychophysical experimens wih his virual environmen, we invesigae he subjecive qualiy an performance of he presene percepual hapic aa reucion echniques.

5 A. Subjecs TABLE RATNG SCHEME Descripion Raing no ifference 5 percepible, bu no isurbing 4 slighly isurbing 3 isurbing srongly isurbing 1 compleely isore 0 A oal of foureen subjecs paricipae in he psychophysical ess, ranging in age from 3-3. All of hem were righhane. Ten of hem were males, while he res were females. Ten of hem ha never use a hapic evice before an he remaining four ha use such a evice on a regular basis. B. Experimenal proceure n orer o sanarize ess across subjecs, uring he experimen, specific insrucions regaring he seaing posure an he han-evice configuraion were given. The compuer screen isplaying he VE was place in fron of he paricipan while he hapic evice was place on he righ. A carboar screene he hapic evice from visual observaion by he subjecs. The paricipans also wore heaphones playing music o mask he noise emanaing from he moor of he hapic evice. This was one in orer o ensure ha he subjecs respone only o hapic simuli, while giving psychophysical raings. The es sofware uses he CHA3D library ( The simulae environmen conains a 3D moel (see Fig. 6) which can be hapically explore. The SensAble PHANTOM DESKTOP TM hapic evice is use for he experimens. n he experimens, we use Weber facors of.5%, 5%, 10%, 15%, 0%, 30% for configuring he percepual hapic aa reucion. During he experimen, subjecs shoul give a raing from level 0 o 5 (Table 1) accoring o he qualiy of he isplaye hapic sensaions. A he beginning of he experimen, a raining session was conuce wih he subjecs o familiarize hem wih he experimenal seings an he ask o be performe. The subjecs were guie o recognize he isorion arifacs inrouce by he eaban coing scheme an is combinaion wih various preicion moes. Here he bes possible hapic feeback (0% Weber facor, esignae level 5), an inermeiae qualiy hapic feeback(1.5% Weber facor, ZOH preicion, esignae level 3) an a ba qualiy hapic feeback (5% Weber facor, linear preicion, esignae level 1) were isplaye o he subjecs as references. n he experimenal phase, each subjec was require o perform 30 ess (six ifferen Weber facors for each one of he five preicion mehos: ZOH, linear, plane, sphere an hybri). The parameers of he preicors an he Weber facors were chosen ranomly. n each es, he subjec was allowe a ime frame of 10 secons o ap or move across a cerain region of he virual objec. Afer each es, he paricipan gave a feeback raing for his es. The whole experimen lase aroun 30 minues (incluing inroucion an raining). Afer every 10 ess here was a break for he subjecs. V. RESULTS The packes rae vs. Weber facors for various preicion moes are shown in Fig. 7. The subjecive raing resuls for he various preicors across he consiere range of Weber facors are shown in Fig. 8. The hybri an geomery-base preicors perform beer an show a higher compression raio in comparison o he signal-base preicion for he same Weber facor. Especially, compare o he ZOH preicor, a a Weber facor of 10% we observe a aa reucion of abou 31% for he plane-base preicion, 18% for he spherebase preicion an 4% for he hybri preicor while he subjecive qualiy are all abou 30% higher. n Fig. 8, here is no significan subjecive qualiy ifference beween he hybri preicion an he geomery-base preicion. Bu hese preicion mehos have higher qualiies han he signal-base preicion, especially for large Weber facors (see Weber facors 0% an 30%). n our observaion, he subjecive qualiy is aversely affece by isurbances cause by suen force-changes ue o he upaing of preicors. During he experimen we observe ha uner large Weber facors he hybri an geomery-base preicion generae reuce isurbances; heir upae raes are much lower han hose for he signal-base preicion. n Fig. 9, he subjecive qualiy vs. packe rae curves (Q- R curve) are shown, where he hybri preicion consisenly ouperforms he oher preicion mehos. n erms of subjecive qualiy, he hybri preicion performs bes an always gives a beer subjecive feeling a comparable packe raes. The hybri preicion also achieves a higher compression raio in comparison o he oher iniviual preicors. A he subjecive qualiy level 4 (goo qualiy), by using geomery-base preicion he hapic aa reucion is abou 54% an 60% as compare o he linear an ZOH preicors, respecively. Furhermore, he hybri preicion shows an aiional gain of abou 15% in comparison o he geomery-base preicion meho. V. CONCLUSON AND FUTURE WORK This paper eails our geomery-base preicion for eaban-base hapic aa reucion an combines i wih a previously propose signal-base preicor in a hybri framework. The geomery-base preicion allows for local renering of remoe hapic signals. The hybri preicion scheme ynamically selecs he bes preicor in erms of subjecive qualiy an packe rae reucion. Conuce experimens show significan improvemens in performance when using he geomery-base preicion moels. Aiional gains can be achieve wih he hybri preicion framework. n fuure work, we will inclue aiional moel parameers such as fricion an ynamic environmens. n aiion, we

6 Packe rae [%] ZOH Linear Plane Sphere Hybri Weber facor Fig. 7. The packe rae curve for all 5 preicion moes. The hybri preicion moe leas o he lowes packe rae. Subjecive Evaluaion( 0 5 ) ZOH Linear Plane 1.5 Sphere Hybri Packe rae [%] Fig. 9. The Q-R curve. The hybri preicion performs always beer han he oher 4 preicion moes. Subjecive Evaluaion( 0 5 ) ZOH Linear Plane Sphere Hybri Weber facor Fig. 8. Mean subjecive raings an sanar eviaion for all preicion moes. There is no significan ifference beween he hybri preicion an he bes one (sphere-base preicion). will invesigae real-worl TPTA scenarios wih communicaion elay. ACKNOWLEDGMENT This work has been suppore, in par, by he European Research Council uner he European Unions Sevenh Framework Programme (FP7/ ) / ERC Gran agreemen no , an in par, by he German Research Founaion (DFG) uner he projec STE 1093/4-1. REFERENCES [1] W. R. Ferrell an T. B. Sherian, Supervisory conrol of remoe manipulaion, EEE Specrum, vol. 4, no. 10, pp , Ocober [] J. E. Colgae, P. E. Grafing, M. C. Sanley, an G. Schenkel, mplemenaion of siff virual walls in force-reflecing inerfaces, in Proc. of he Virual Realiy Annual nernaional Symposium, pp. 0 08, Seale, WA, Sepember [3] C. Shahabi, A. Orega, an M. R. Kolahouzan, A comparison of ifferen hapic compression echniques, in Proc. of he EEE nernaional Conference on Mulimeia & Expo (CME), vol. 1, pp , Lausanne, Swizerlan, Augus 00. [4] P. Hinerseer, E. Seinbach, S. Hirche, an M. Buss, A novel, psychophysically moivae ransmission approach for hapic aa sreams in elepresence an eleacion sysems, in Proc. of he EEE nernaional Conference on Acousics, Speech an Signal Processing (CASSP), vol., pp , Philaelphia, PA, USA, March 005. [5] P. Hinerseer an E. Seinbach, A psychophysically moivae compression approach for 3D hapic aa, in Proc. of he nernaional Symposium on Hapic nerfaces for Virual Environmen an Teleoperaor Sysems (HAPTCS), pp , Arlingon, VA, USA, March 006. [6] P. Hinerseer, S. Hirche, S. Chauhuri, E. Seinbach, an M. Buss, Percepion-base aa reucion an ransmission of hapic aa in elepresence an eleacion sysems, EEE Trans. on Signal Processing, vol. 56, no., pp , February 008. [7] S. Hirche, P. Hinerseer, E. Seinbach, an M. Buss, Transparen aa reucion in neworke elepresence an eleacion sysems. par 1: Communicaion wihou ime elay, Presence: Teleoperaors & Virual Environmens, vol. 16, no. 5, pp , Ocober 007. [8] M. Kuschel, P. Kremer, S. Hirche, an M. Buss, Lossy aa reucion mehos for hapic elepresence sysems, in Proc. of he nernaional Conference on Roboics an Auomaion (CRA), pp , May 006. [9] P. Hinerseer, E. Seinbach, Moel-base aa compression for 3D virual hapic eleineracion, in Proc. of he nernaional Conference on Consumer Elecronics (CCE), pp. 3 4, Las Vegas, NV, USA, January 006. [10] S. M. Clarke, G. Schillhuber, M. F. Zaeh, an H. Ulbrich, Preicionbase mehos for eleoperaion across elaye neworks, Mulimeia Sysems, vol. 13, no. 4, pp , 008. [11] N. Sakr, J. Zhou, N. Georganas, J. Zhao, an X. Shen, Preicionbase hapic aa reucion an compression in ele-menoring sysems in elemenoring sysems, EEE Trans. on nsrumenaion an Measuremen, vol. 58, no. 5, pp , May 008. [1] P. Mira an G. Niemeyer, Moel meiae elemanipulaion, nernaional Journal of Roboics Research, vol. 7, no., pp. 53 6, February 008. [13] E. Seinbach, S. Hirche, J. Kammerl,. Viorias, an R. Chauhari, Hapic aa compression an communicaion, EEE Signal Processing Magazine, EEE, vol. 8, no. 1, pp , January 011. [14] E. Weber, Die Lehre vom Tassinn un Gemeingefuehl, auf Versuche gegruene, Braunschweig, Germany: Vieweg, [15] G. A. Gescheier, Psychophysics, Lawrence Erlbaun, 1985.