PIECEWISE LINEAR DYNAMICAL MODEL FOR HUMAN ACTIONS CLUSTERING FROM INERTIAL BODY SENSORS WITH CONSIDERATIONS OF HUMAN FACTORS

Transcription

1 PIECEWISE LINEAR DYNAMICAL MODEL FOR HUMAN ACTIONS CLUSTERING FROM INERTIAL BODY SENSORS WITH CONSIDERATIONS OF HUMAN FACTORS Jiaqi Gong, Philip Asare,, John Lach, Yanjun Qi Charles L. Brown Department of Electrical and Computer Engineering Department of Computer Science University of Virginia {jgong, pka6qz, jlach, ICST BodyNets, September 9 th, 4

2 Motivation Dividing motion data into discrete segments is useful Training Diagnosis Segmentation can be at different granularities Walking vs. not-walking Where in the gait cycle Left Wrist Left Ankle Right Wrist Right Ankle Walking Jumping Running Punching

3 Problems 3 Typical BSN Processing Pipeline Assumes repeatability of signals S S Sk Raw Data Y t = (y, y y k ) y (t) R p y (t) R p y (t) R p y k (t) R p Preprocessing Y (t) = (y, y y k ) y (t) R p y k (t) R p Segmentation W= {w, w w m } w w m Feature Extraction X i [,m] = {x i, x i, x i f } x x f x x f x m x m f Classification P i (s X i, θ) p (s X ) p (s X ) p (s c X ) Class s s m t Samples t Samples m Segments m Segments C Classes Credit: A. Bulling, U. Blanke, and B. Schiele, A Tutorial on Human Activity Recognition Using Body-Worn Inertial Sensors, ACM Computing Surveys (CSUR), vol. 46, no. 3, pp. 33, 4.

4 Problems 4 Human Factors Physical placement affects signal Chien et al. (3)*: Model-based estimation Heel Strike Event o * Gyroscope data of inertial BSNs mounted insecurely o * Gyroscope data of inertial BSNs mounted incorrectly *C. C. Chien, J.Y. Xu, H-I Chang, X. Wu and G. J. Pottie, Model Construction for Human Motion Classification using Inertial Sensors, IEEE Workshop on Information Theory and Applications, San Diego, Feb Gyroscope data of inertial BSNs mounted correctly

5 Addressing Problems 5 Our Approach S S Sk Raw Data Y t = (y, y y k ) y (t) R p y (t) R p y (t) R p y k (t) R p Preprocessing Y (t) = (y, y y k ) y (t) R p y k (t) R p Segmentation W= {w, w w m } w w m Feature Extraction X i [,m] = {x i, x i, x i f } x x f x x f x m x m f Body Motion Modeling Classification P i (s X i, θ) p (s X ) p (s X ) p (s c X ) Class s s m t Samples t Samples m Segments m Segments C Classes Motion Stimulus X i Human Body Linear Dynamical Transition Inertial Sensors Nonlinear Observation Sensor Data Y t X(t- ) S(t- ) X(t) S(t) X(t+) S(t+) Y(t- ) Y(t) Y(t+)

6 Our Approach 6 Basic Approach: Piecewise Linear Dynamical Modeling human action (motion stimulus/input) dynamics/ properties S(t+)=AS(t)+Bx(t) y(t)=cs(t)+p noise sensor data (observations) System generally non-linear, but approximately piecewise linear (over short time segments)

7 z Our Approach 7 Identifying Motion Stimulus Cur gyro = 3 ( ( (y(t, g()), y(t, g()),y(t, g(3))))) Left Wave Right Wave (a) - Point Point Transformation of 3D curvature (a) y (b) Scalar time- series curvature x Toe off Heel Strike 4 x 4 3 (b) (c)

8 Our Approach 8 Identifying Motion Stimulus 5 Curvature of gyroscope data (insecure mounting) 4 3 Left Wave Right Wave 4 x Curvature of gyroscope data (mounting error) (a) Curvature of gyroscope data (correct mounting) Toe off Heel Strike (b)

9 Overall Algorithm 9 S S S3 S4 Preprocessing Raw Data Y t = (y (t), y (t) y 4 (t)) Curvature Calculation y t, g, y t, g, y t, g 3 x 5 5 x 4 Motion Stimulus Detection (Peaks of Calculated Curvature) 3 x 6 6 x 5 Hierarchical Temporal Selection of Data Pieces based on detected moments of motion stimulus Clustering.8. PLDM Features X i [,m] = {x i, x i, x i f } EM Algorithm ) Initialize a random sparse input Χ satisfying the constraints: x t t ) Repeat: (a) Given Χ: Identify a LDS: A, B, C, S o ; (b) Given A, B, C, S o : minimize X (Cost Function) x t t 3) Calculate histogram ofx into bins after recursion Data Piece Y(t) i < t < t l

10 What does this buy us? Coarse-grain segmentation PLDM Insecure Mounting Error Mounting Start minute minute 3 minute 4 minute Start minute minute 3 minute 4 minute Correct Mounting Start minute minute 3 minute 4 minute Ground Truth Correct Mounting Error Mounting Insecure Mounting ACA Start minute minute 3 minute 4 minute Start minute minute 3 minute 4 minute Start minute minute 3 minute 4 minute Start minute minute 3 minute 4 minute Walk Straight Run Jump Side Walk Punching Body Rotation

11 What does this buy us? Fine-grained segmentation Heel-strike and toe off detection No. of pairs of events Expected range Humans walk at about two steps per second (one per leg) so we expect about 36 event pairs per leg with some variance Method in [5] PLDM [5] S. Chen, C. L. Cunningham, J. Lach, and B. C. Bennett, Extracting Spatio-Temporal Information from Inertial Body Sensor Networks for Gait Speed Estimation, IEEE International Conference of Body Sensor Networks (BSN), pp. 7-76,.

12 Recap Segmentation is important for BSNs Human factors can be a problem Linear dynamical systems modeling can help for Fine grained Coarse grained

13 Future Work 3 Reduce Computation Complexity Optimize clustering process Other Applications Surgery education data analysis Head impact identification (in sports)