Internal Organ Modeling and Human Activity Analysis

Transcription

1 Internal Organ Modeling and Human Activity Analysis Dimitris Metaxas CBIM Center Division of Computer and Information Sciences Rutgers University

2 3D Motion Reconstruction and Analysis of the Left andright Ventricles From Tagged MRI Idith Haber, Dimitris Metaxas and Leon Axel

3 Anatomy From Atlas of Human Anatomy, Netter, 93

4 Overview Image acquisition Contour segmentation Tag tracking initial time all times RV-LV finite element mesh generation 3D motion reconstruction Validation Motion analysis

5 Short-Axis Long-Axis MRI-SPAMM Tissue Tagging Parallel Tagging Planes Image Plane RV LV RV LV

6 Image Acquisition for 3D Motion Information Tag plane (dark) and image plane orientation Possible tag motion in image plane Representative images

7 Geometry Short-axis contours Finite Element Mesh Shaded Endocardial Walls

8 3D Motion Reconstruction Deformable Modeling Approach Geometric model is fit to image-derived data Allows for inclusion of a priori geometric information

9 3D Motion Reconstruction Deformable Model Dynamics: dq/dt = F e + F i where q = displacement at nodes F e : External, spring-like forces Tags (SPAMM forces) Contours F i : Internal, smoothing forces Forces from all 3 directions were applied simultaneously

10 Free Wall Regional Comparisons Separated free wall and septum into regions using anatomical landmarks Base Mid-ventricle Outflow Tract Apex

11 RESULTS

12 Validation: Short-axis End-diastole End-systole

13 Validation: Long-axis End-diastole End-systole

14 Normals: Displacement

15 Normals: Displacement Color plot on endocardial wall mm Paths of mid-wall points

16 RVH: Displacement mm Paths of mid-wall points

17 RVH: E E3 direction

18 Conclusions Developed the first volumetric 3D motion reconstruction technique for the RV Validation: good agreement between model and original images Obtained consistent results for 5 normal volunteers Found notable differences in deformation for RVH patients

19 Human Activity Analysis: Why is it Important? Monitoring and Security Applications Airport security Military security applications Interrogation intelligence applications Better Understanding of how People Interact (relationship between speech and gesture)

20 Why is Dynamic Activity Recognition Hard? 3D/2D tracking from monocular images Statistical variations in the input data Activities happen in parallel Semantics of recognized data Shape and motion events Computational complexity

21 Objective Human gait recognition Recognition of specific person from gait Tracking of faces, arms etc for other security applications

22 Parts of Framework Obtaining accurate 3D data with 3D computer vision methods Modeling by breaking motions into their constituent elementary parts (emphasis on representation) Capturing statistical variations with Hidden Markov Models

23 Obtaining accurate 3D data 3D tracking based on single/multiple cameras Physics-based modeling: Q = FQ Picture shows the tracking of a person s arms

24 What parts we can track/uniqueness Face Arms Single/multiple people What is unique in our methodology: Physics-based framework Emphasis on representation and not on training data Use of statistical methods for activity recognition Use of methods from computational biology

25 Face Tracking

26 Face/hands Tracking

27 Hand tracking

28 People Tracking (Indoor Scenes)

29 People Tracking (Outdoor Scenes)

30 Gait recognition Identify people from the way they walk Important for surveillance and intrusion detection How to cope with even/uneven terrain What are good features for identifying a person? i.e., what features are person-specific?

31 Background Sagittal plane - divides body into left and right halves Limb segment - a vector between two sites on a particular limb

32 Elevation Angles

33 Elevation Angles In the past, we have used sagittal elevation angles for recognizing the type of gait Reported to be invariant across people Recognize level walk, upward walk,... Any person-specific aspects to elevation angles? Yes, based on preliminary results

34 The trajectories of the sagittal elevation angles are invariant across different subjects. As a consequence, person-independent gait recognition will require less training data. (Borghese et al., 1996)

35 The cyclogram Elevation angles trace curve in a 4D space Curve is called cyclogram Cyclogram lies in a 2D plane Well, almost Hypothesis: deviation of cyclogram from plane is person-specific

36 Cyclogram example Curve is cyclogram projected into best-fit plane Green points are real points of cyclogram Red lines trace the deviation of points from plane (exaggerated scale)

37 Deviation features Possible measure of deviation: Cartesian distance from plane Unfortunately, not suitable for identification Too little variation across persons Graphs on the next slide show similarity between two people Need a different measure...

38 Similarity of Distance

39 Vector-based measure Find cyclogram plane with PCA Use vectors to measure deviation: Let: e = cyclogram point, e' = projected point Let: n 1, n 2 = smallest principal components n 1, n 2 are normal to the plane e = e' + s n 1 + t n 2 Solve for s and t

40 Vector-based measure (s, t) forms a vector in the plane spanned by the point e', and the vectors n 1, n 2 The signs of s and t show rather interesting properties Map signs of (s, t) to letters, as follows: (+, +) -> A (+, -) -> C (-, +) -> G (-, -) -> T

41 Cyclogram sequence Deviation from cyclogram plane can be represented as a sequence e.g., CCCGTTTTATATTTTTAAAAGCCGGTAAATTAGGGG Compare sequences between people via longest common subsequence (LCS) matching Well-known dynamic programming algorithm, used in computational biology

42 LCS Dynamic programming algorithm, runs in O(nm) time AGTATTCCAGCA GAAATTTTATCTA LCS: GATTACA

43 Exploit LCS Let l = normalized length of LCS over total length of strings On the average, l for pairs between people is smaller than l for pairs from the same person Deviation of cyclogram from plane seems to contain some person-specific elements Need to test with large data sets from many people

44 Relevant Publications Harold Sun and D. Metaxas: Animating Human Gait. Procs. Siggraph Harold Sun, Ambarish Goswami, Dimitris Metaxas, and Janice Bruckner. CYCLOGRAM PLANARITY IS PRESERVED IN UPWARD SLOPE WALKING Congress of International Society of Biomechanics. Christian Vogler and Dimitris Metaxas. Adapting Hidden Markov Models for ASL recognition by using three-dimensional computer vision methods. SMC 97. Christian Vogler and Dimitris Metaxas. ASL recognition based on a coupling between HMMs and 3D motion analysis. ICCV 98. Christian Vogler and Dimitris Metaxas. Toward scalability in ASL recognition: Breaking down signs into phonemes. Springer Lecture Notes on Artificial Intelligence 2000, Proceedings of the Gesture Workshop'99, Gif-sur- Yvette, France. Christian Vogler and Dimitris Metaxas. Parallel Hidden Markov Models for American Sign Language Recognition. ICCV 99.