Tracking Articulator Movements Using Orientation Measurements

Transcription

1 Tracking Articulator Movement Uing Orientation Meaurement An Ji, Michael T. Johnon, and Jeffrey Berry Marquette Univerity Department of Electrical and Computer Engineering Speech and Signal Proceing Laboratory P.O. Box 1881 Milwaukee, WI {an.ji, mike.johnon, Abtract Thi paper introduce a new method to track articulator movement, pecifically jaw poition and angle, uing 5 degree of freedom (5 DOF) orientation data. The approach ue a quaternion rotation method to accomplih thi jaw tracking during peech uing a ingle enor on the mandibular incior. Data were collected uing the NDI Wave Speech Reearch Sytem for one pilot ubject with variou peech tak. The degree of jaw rotation from the propoed approach i compared with traditional geometric calculation. Reult how that the quaternion baed method i able to decribe jaw angle trajectory and give more accurate and mooth etimation of jaw kinematic. 1. Introduction The tracking of jaw movement tracking peechi of ignificant interet to a large number of dicipline, including occupational therapy, orthodontic, pychology, peech language pathology and acoutic peech proceing. A number of technologie have been ued for recording articulator movement. X-ray cinematography [1-] i effective, but the radiation to the ubject head i a concern. Cine MRI can provide dynamical 3D meaurement of the vocal tract but it i cumberome and expenive [3-4]. In contrat, the ultraound technique i able to capture the urface of the tongue [5-6] but noie, echo artifact and refraction may affect the reult. High peed optical tracking i an attractive option for jaw movement but thi approach require much more complex recontruction algorithm, and the inacceibility of mandible i a challenge. Recently, electromagnetic articulography(ema) ening ytem have been developed to meaure articulatory movement. Thi ytem i able to track both poition and orientation of the enor, which can include both 5 DOF (planar orientation) and 6 DOF (full orientation). Thi enable detailed tudy of the relationhip between acoutic data and articulator movement. However, mot of the publihed work in acoutic and articulator inverion [7-9] only utilize the 3D poition data to extract articulator movement feature. The orientation information conit of quaternion data and ha thu far not been incorporated in mot kinematic EMA tudie. Jaw movement during peech ha generally been decribed a a pure tranlation and pure rotation of a ingle point on the jaw. Wetbure and Edward develop a two-dimenional rigid-body model of jaw movement during peech [10-11]. Thi model decompoe peech-related jaw movement into three component: rotation degree around the tranvere axi located approximately through the condyle, and horizontal and vertical tranlation of thi axi in the mid-agittal plane. The purpoe for tracking jaw movement here i to find appropriate feature to decribe the degree of jaw openne for the acoutic to articulatory mapping, with a focu on the rotation angle of the jaw in the following experiment. Thi paper i a preliminary tudy of applying quaternion orientation data to the tak of jaw movement tracking during peech. The kinematic data of jaw movement were recorded bythe NDI Wave Speech Reearch Sytem which i a motion capture ytem pecifically deigned for tracking peech related articulatory or ofacial movement. A noted, the kinematic data tore both poition and orientation information for each enor attached on articulator. Since the 6DOF enor are ignificantly bulkier and can thu create articulation difficulty, it i beneficial to be able to ue 5DOF enor to capture the neceary orientation information.. Quaternion orientation repreentation /1/$ IEEE 9 ICALIP01

2 In computer viualization and animation, quaternion format i a commonly ued method to repreent rotation and orientation [1].A quaternion i a 4-D unit vector q= [ q0, qx, qy, qz] atifying the following equation: q0 + qx + qy + qz = 1 (1) A quaternion rotation thu lie on the 4-D unit hyperphere. The key application of quaternion to tracking jaw movement during peech in thi paper lie in their ue to repreent rotation. A quaternion can repreent a rotation by an angle θ around an unit axi v. θ θ θ θ q = (co( ),in( ) vx,in( ) vy,in( ) vz) () θ where the vector part in( ) v = [ qx, qy, qz] define the θ rotate axi direction, and the calar part co( ) = q0 define the degree of rotation. To rotate a point whoe poition i repreented by the vector p by an angle θ along the axi v to a new poition p final we apply the quaternion multiply operation p final = QPQ (3) Where Q [co( θ θ = ),in( ) v], P = [0, p] In the EMA ytem, each enor orientation i repreented at each data point by a ingle quaternion vector, repreenting the rotation that would be required to move the enor from it original baeline orientation to it current orientation. In thee experiment the quaternion data i ued to calculate jaw movement by tracking the enor orientationchange over time, to determine jaw angle.the global coordinate ytemi referenced to a fixed head orientation with origin at the front upper incior. The mid-agittal plane i the x-y plane in the ytem. The 5DOF enor bae orientation i parallel to the x-y plane, o in thi orientation the quaternion data how no rotation. The initial enor plane can be defined a two orthogonal bae vector [1 0 0] and [0 1 0].The quaternion data can be applied to thee bae vector to calculate the final orientation poition of the enor plane at any ampling time. 3. Method 3.1. Sytem et up In the NDI Wave Speech Reearch Sytem, 5-DOF enor allow tracking of x, y, and z patial coordinate, a well a angular coordinate characterizing rotation about the tranvere axi (pitch) and anterior poterior axi (roll). 6-DOF enor have the added capacity for tracking angular coordinate characterizing rotation about the inferior uperior axi (yaw). The tandard NDI Wave (ued for the current work) ha eight input channel and record enor movement with a 400-Hz ampling rate. Figure1 how the ytem et up and the ubject head orientation relative to the field generator. A ubject it at the right ide of the field generator within the range of the magnetic field. Figure 1 Coordinate ytem direction and enor placement Figure Bite-plate with enor poition 3.. Senor placement and data collection In order to locate the maxillary occlual plane which will form the x-z plane of the local coordinate ytem, a calibration bite-plate recording i done initially.. One 6DOF enor i attached at the center of forehead and two 5DOF enor are placed on the bite-plate, one at the maxillary central incior (OS) and one along the mid-agittal plane at the biection between the back molar (MS) (See figure below for more detail). To create the bite-plate, two piece of wax were oftened in warm water and molded onto a tongue depreor. Thi oftened wax i then placed into the ubject mouth and an impreion of the bite i taken. Immediately afterward (while the wax i till oft), experimenter meaure the midpoint between the back molar and create an indentation for the placement of the MS enor. The OS enor i placed directly in front of the central maxillary incior. Thee enor are preed into the wax until they are 93

3 ecured and the bite-plate i replaced in the ubject mouth for the bite-plate recording. In normal recording, the bite-plate wax i taken out and 5DOF enor are placed at the deired recording point. For thi experiment a imple configuration wa ued with two 5DOF enor placed on the mandibular incior and back molar. The 6DOF enor remain attached in the ame forehead poition which i ued to a a reference to calibrate all the other enor data to eliminate the error of heat movement. The two 5DOF enor are ued for tracking jaw movement Speech material Initially the ubject wa aked to do wide range jaw motion. Thi large cale jaw motion repetition i ued to evaluate the propoed rotation calculation method baed on quaternion. In the reiterant peech tak, the ubject repeat the target three vowel [i], [a], [ea] 4 time each. The election of vowel create a wider range of movement amplitude. Of thee, the vowel [ea] i aociated with the larget amplitude jaw motion, wherea the vowel [i] i aociated with mallet amplitude motion. The final peech tak i a hort word and entence repetition uing a normal peed pattern Experiment 1 (baeline):calculate the jaw angle by uing the poition of two 5DOF enor and the bite plate record. Calculating the jaw angle from the poition data of the above two 5DOF enor during normal recording require traightforward geometric manipulation. In the bite-plate recording the poition of OS and MS can be located by taking the average value of x, y, z coordinate of OS and MS enor. The two point OS and MS form vector mo. During normal recording, the mandibular incior (MI) and back molar (BM) enor poition alo give a vector bmmi. The jaw angle i calculated uing the following equation moo( x, y), bmmi( x, y) θ = moo( x, y) bmmi( x, y) (4) where moo( x, y) repreent the vector projection in the x yplane, and repreent moo, bmmi the dot product of thee two vector. The magnitude of lateral motion of the jaw in thi tak i not conidered in thee experiment Experiment :Calculate the jaw angle uing a ingle enor on the mandibular incior, uing quaternion orientation information Since teeth are rigid, the enor orientation change during jaw movement, in direct correpondence to the angle of the jaw. Thi experiment ue the MI enor to calculate the jaw angle by uing the quaternion orientation data. Figure 3 illutrate the approach for the enor attached on the front lower incior. For the initial poition, which i alway the jaw cloed condition, the enor quaternion data i ued to calculate an original orientation poition. The ame method i ued to obtain the jaw orientation over time, for each data point. vector1 and vector in Figure 3repreent the normal vector perpendicular to the initial and final enor plane repectively. The angle between thee two vector i the jaw rotation angle. The initial and final enor plane can be defined by vector, i1 and i f 1 f i1 = qbaeq i 1 i (5) i = qbae i qi (6) f 1 = qfbae1qf (7) f = qf baeqf (8) where q i and q f i the quaternion orientation of the initial and final poition, repectively. bae 1 = [1,0,0] and bae = [0,1,0] are the vector defining the plane, a previouly introduced. Following thi, vector1 and vector, the normal vector perpendicular to the initial and final enor plane are determined: v = (9) 1 i1 i f 1 f v = (10) where repreent cro product of two vector. The angle i then calculated by v1( x, y), v( x, y) α = (11) v1( x, y) v( x, y) where v1( x, y) and v( x, y) repreent the vector projection in the x y plane. Figure 3 x-y plane projection of the jaw angle calculation 94

4 4. Reult and dicuion In thi ection, we preent the mid-agittal plane jaw motion path and evaluate jaw rotation by both the baeline and new method. The goal i to ae the degree of jaw rotation in different peech behavior and compare the quaternion method with the traditional geometrical approach. Figure 4 how a typical jaw angle trajectory for the initial wide jaw movement pattern. The path of the enor attached to the mandibular incior and back molar teeth diplay the jaw motion. From thi plot, the maximum angle of jaw openne can be etimated a approximately 14 degree. Figure 5 how the correponding jaw angle. The top plot i calculated from one enor uing quaternion data, and the bottom plot i calculated from two enor uing poition data only. Thee peak are aociated with five jaw movement repetition. The moothne and conitency of the quaternion-baed plot indicate that the new approach i effective at tracking the motion of the jaw over time. Figure 6 Rotation angle for vowel repetition Figure 7 Rotation angle for word repetition Figure 4 jaw trace for large range movement Figure 8 Rotation angle for entence repetition Figure 5 Rotation angle for large range jaw movement In order to further evaluate the quaternion method, Figure6, 7, and 8 how jaw movement for different peech tak: vowel, word and entence repetition. By carefully examining and comparing the angle from thee two method, the amplitude of the jaw openne doe not match preciely. From thee plot, we can ee that the two method return imilar reult. There i a mall deviation in peak value, a well a a ignificant difference in term of the moothne of the angle contour, with the quaternion approach yielding a much moother trajectory than the poition-baed method. Thi may be due to the ue of two enor meaurement rather than one, with two eparate error ource, or alternatively due to difference in meaurement error range for poition veru orientation data from the enor. To ummarize, we have hown that jaw movement can be accurately etimated uing orientation data from a ingle enor on the mandibular incior, and that thi approach give a more accurate decription of jaw openne during peech.thi approach i uitable aa 95

5 feature to characterize jaw motion in future application, uch a acoutic-to-articulatory inverion. Continued work in thi area include ytem calibration baed on bite plate orientation and calculation of other articulatory meaure from the poition and orientation data. Thi reearch i upported by grantiis from the National Science Foundation. Reference [1]R. Houde. A Study of Tongue Body Motion during Selected Cononant Sound, PhD thei, Univerity Microfilm, Ann Arbor, Mich []K.G. Munhall, E. Vatikioti-Bateon, and Y. Tohkura. X-ray film databae for peech reearch, Journal of the Acoutical Society of America, pp. 1 14, [3]Maaki, S., Tiede, M.K., Honda, K., Shimada, Y., Fujimoto, I., Nakamura, Y., and Ninomiya, N., MRI-baed peech production tudy uing a ynchronized ampling method, Journal of the Acoutical Society of America, vol. 0, no. 5, pp , [4]Narayanan, S., Nayak, K., Lee, S., Sethy, A., and Byrd, D., An approach to real-time magnetic reonance imaging for peech production, Journal of the Acoutical Society of America, vol. 115, no. 4, pp , 004. [5]Stone, M.L., Sonie, B.C., Shawker, T.H., Wei, G., and Nadel, L., Analyi of real-time ultraound image of tongue configuration uing a grid-digitizing ytem, Journal of Phonetic, vol. 11, no. 3, pp , [6]Kaburagi, T. and Honda, M., An ultraonic method for monitoring tongue hape and the poition of a fixed-point on the tongue urface, Journal of the Acoutical Society of America, vol. 95, no. 4, pp , [7]Richmond, K., "Etimating articulatory parameter from the acoutic peech ignal". Edinburgh, Doctor of Philoophy, Univerity of Edinburgh, 001. [8]Toda, T., A. Black and K. Tokuda, "Acoutic-articulatory inverion mapping with gauian mixture model".international Conference on Spoken Language Proceing, Jeju Iland, Korea, 004. [9]Hiroya and Honda, 004 S. Hiroya and M. Honda, Etimation of articulatory movement from peech acoutic uing an HMM-baed peech production model, IEEE Tranaction of Speech Audio Proce. 1 (), pp , 004. [10] Wetbury, J. R. Mandible and hyoid bone movement during peech, Journal of Speech and Hearing Reearch, 31, pp [11] Edward J. Mandibular rotation and tranlation during peech, Doctoral diertation, City Univerity of New York, [1] John C. Hart, George K. Franci, Loui H. Kauffman, Viualizing quaternion rotation, ACM Tranaction on Graphic, v.13 n.3, pp ,