Terry Taewoong Um. Terry T. Um and Dana Kulić. University of Waterloo. Department of Electrical & Computer Engineering

Transcription

1 AN UNSUPERVISED APPROACH TO DETECTING AND ISOLATING ATHLETIC MOVEMENTS Terry T. Um and Dana Kulić University of Waterloo Department of Electrical & Computer Engineering Terry Taewoong Um Terry Taewoong Um

2 Motivation Scenario We want to analyze athletic movements from a long sequence of skeleton data CMU motion capture dataset ( Procedure Data collection Segmentation (Extracting athletic movements) Movement analysis B. Tekin. et., Direct Prediction of 3D Body Poses from Motion Compensated Sequences (2016) 1 / 12

3 What are athletic movements? There can be various ways to define athletic movements Universal principles that apply to all sports tasks (R. Bartlett, Introduction to sports biomechanics: Analysing human movement patterns. Routledge, 2007) - Use of the stretch shortening cycle of muscle contraction. - Minimisation of energy used to perform the task. - Control of redundant degrees of freedom in the segmental chain. In this research, we attempt to detect stretch-shortening cycles from a long sequence of skeleton data 2 / 12

4 What are athletic movements? We will detect flexed poses followed by a explosive and coherent movement prestretch Pre- Stretch Storing potential energy (Usually) flexing body limbs Stretch Stretching body limbs explosively & coherently stretch 0 T proposed measure Pre-Stretch Stretch Stretch followed by pre-stretch detect! pre-stretch stretch 3 / 12

5 Detection of Athletic Movements I will measure the extent of Pre-Stretch & Stretch, and blend them for detecting athletic movement 9 / 12

6 Detection of Pre-Stretch Phase Manipulability (T. Yoshikawa, 1985) The robot arm s ability to change the position or orientation of its endpoint in each direction pre-stretch stretch From the Jacobian J of the limb, pre-stretch close to 1 stretch close to 0 4 / 12

7 Detection of Stretching Phase In the stretching phase, all joints should move coherently Different joint movements make a coherent kinematic synergy at the limb s endpoint. problem2 problem1 Prob. 1) How can we represent kinematic synergy of joints? Forward Kinematics in products of exponentials (POE) formula (by Lie group formulation) joint axis joint angle Kinematic Synergy An approximate abstraction of all joints instantaneous movements 5 / 12

8 Detection of Stretching Phase Baker Campbell Hausdorff (BCH) formula If A and B is enough small, If we keep merging POEs one after another, ω BCH R 3 for rotations, R 6 rotations & translations Kinematic Dimensionality Reduction (KDR) Note that ω i q i is small enough if we set q i as instantaneous changes of joint angles 6 / 12

9 Detection of Stretching Phase Prob. 2) How can we detect coherency from the kinematic synergy, ω BCH? Displaying the trajectory of kinematic synergy values (ω BCH R 3 ) ω BCH (Right leg) ω BCH (Left leg) 7 / 12

10 Detection of Stretching Phase Prob. 2) How can we detect coherency from the kinematic synergy, ω BCH? stretching part In the stretching part, ω BCH travels long distance toward a certain direction. Or we can say ω BCH momentarily forms a submanifold (lower dimensional manifold). [eigenvalue ratio for detecting submanifold] [Scaling factor for measuring distance] Coherency 8 / 12

11 Detection of Athletic Movements 1. Use fixed size (e.g. 0.25sec) sliding window (Let s assume that the window contains 2m+1 data from x i m to x i+m ) 2. Calculate manipulability at (i m) for detecting pre-stretches μ (i m) 3. Calculate scaled coherency from x (i m i+m) for detecting stretches 4. Blend them with a ratio β 5. Report athletic movements when υ i > υ threshold 9 / 12

12 Experiments The proposed method is evaluated on the CMU motion capture dataset We experimented with 4 athletic motions (which has about 60 trials in total), which are jumping, soccer kicking, baseball pitching, and golf, and 2 long sequences of random motions (walking, hand waving, squat, etc.) All codes are available from publications/#em BC / 12

13 Results Measure Pre-stretch Stretch / 12

14 Results : Test with long sequences of activities (a) (b) 11 / 12

15 Remarks No prior knowledge for movements is required No machine learning techniques have been applied, that is, no training data or training time is required. You can enhance the detection performance with the combination of machine learning techniques. Or, you can use the proposed representation (kinematic synergy) for machine learning tasks, e.g., human activity classification In the future work, we will verified the proposed concepts in machine learning tasks An Unsupervised approach to Detecting and isolating athletic movements, T. T. Um and D. Kulić, EMBC / 12

16 Summary An approach to detect athletic movements of body parts is proposed Pre-stretching motions are captured by using manipulability We proposed kinematic dimensionality reduction (KDR) method for representing kinematic synergy of joint movements Stretching motions are captured by detecting submanifold in the kinematic synergy By detecting sequential pre-stretching and stretching motions, we can detect athletic movements of the body parts The proposed approach has been verified with CMU mocap dataset (The Matlab code is available from An Unsupervised approach to Detecting and isolating athletic movements, T. T. Um and D. Kulić, EMBC / 12