Human motion analysis: methodologies and applications

Transcription

1 Human motion analysis: methodologies and applications Maria João M. Vasconcelos, João Manuel R. S. Tavares CMBBE th International Symposium on Computer Methods in Biomechanics and Biomedical Engineering 27th February 1st March, Porto, Portugal

2 Contents: Introduction; Methodologies; Applications; Conclusions; Future work. 2

3 Introduction: The main goal of this work is to present a review about the leading computational techniques used in human motion analysis and some of their applications. This review is the first step of a project which will focus in the study, development and implementation of computational techniques capable of segment, track and analyse the movement of objects in image sequences, particularly for human motion analysis. 3

4 Introduction: The study of human motion in image sequences usually follows a general framework: 1. Feature extraction Identification of the objects characteristics to be analysed in the image frames 2. Feature correspondence Where the problem of matching features between two consecutive image frames is approached 3. High level processing Such as recognition of human movements, activities or poses 4

5 Feature extraction: There are two typical approaches to the motion analysis of human body depending on whether a priori shape models are employed or not; In each approaches, varying models of increasing complexity can be considered; It is possible to use simple models, such as stick figures or more complex models involving 2D contours or even 3D volumes. Aggarwal, J. and Q. Cai. Human Motion Analysis: A Review. In Computer Vision and Image Understanding, vol. 73 (3), p ,

6 Articulated motion without a priori shape models: Stick figures - Consists of line segments linked by joints where its motion provides the key for motion estimation. - Johansson (1975) first showed that the human eyes can interpret a moving human-like structure with moving light displays (MLD), their relative movement creates a vivid impression of a person walking, running, dancing. - So attempts have been made to recover a connected human structure with a projected MLD by assuming that points belonging to the same object have higher correlations in projected positions and velocities. Johansson, G.. Visual motion perception. In Sci American, vol. 232(6), p , Rashid, R R.F.. Towards a system for the interpretation i of moving light display. In IEEE Trans. On PAMI, vol. 2(6), p ,

7 Articulated motion without a priori shape models: 2D contours In this representation, the human body segments are analogous to 2D ribbons or blobs. Kurakake (1992) attempted to obtain the joint locations in images of walking humans by establishing correspondence between extracted ribbons. The joints were identified as the centre of the area where two ribbons overlaps. Kurakake, S. Description and tracking of moving articulated objects. In 11 th Intl. Conf. On Pattern Recognition, p ,

8 Articulated motion with a priori shape models: Stick figures Is based on the observation that human motion is essentially the movement of the human skeleton brought about by the attached muscles. Is often used to recover the 3D configuration of a moving object according to its projected 2D image. Wang, g J. and S. Singh. Video Analysis of Human Dynamics - A Survey. In Real-time Imaging g Journal, Rius, I. et al. Automatic Learning of 3D Pose Variability in Walking Performances for Gait Analysis. In International Journal for Computational Vision and Biomechanics,

9 Articulated motion with a priori shape models: 2D contours These models are directly associated with the projection of the human figure in images. 2D ribbons are the most commonly used models and are comprised of two components: -the basic human body model (outlines the structural and shape relationships between the body parts); -the extended body model (the support posture model, the side view kneeling model and side horse motion model). Aggarwal, J. and Q. Cai. Human Motion Analysis: A review. In Computer Vision and Image Understanding, Leung, M.K. and Y.H. Yang. First sight: A human body outline labelling system. In IEEE Trans on PAMI,

10 Articulated motion with a priori shape models: Volumetric models (3D) - Elliptical cylinders These models can also be used to model articulated and self-occluding objects such as fingers Mixture of models -Stick models with volumetric models Where the first provides the basic shape and the second define the outer appearance of a person Aggarwal, J. and Q. Cai. Human Motion Analysis: A review. In Computer Vision and Image Understanding, Wang, J. and S. Singh. Video Analysis of Human Dynamics - A Survey. In Real-time Imaging Journal,

11 Feature correspondence: Methods which do not use a priori shape models The correspondence between successive frames is based upon prediction or estimation of features related to position, velocity, shape, texture and colour. Methods which assume a priori shape models Feature correspondence is automatically achieved once the matching between the images and the model data is established. Aggarwal, J. and Q. Cai. Human Motion Analysis: A Review. In Computer Vision and Image Understanding, vol. 73 (3), p ,

12 Applications: Clinical studies Study of Spondylolisthesis or Parkinson disease, for example Gait classification Human motion analysis to differentiate between male and female walkers and their performances; or recognize gait patterns of old and young people Git Gait analysis Recognition of human movements, activities or poses Biomechanical study of athletes In order to improve their performances 12

13 Human motion applications: Example of motion tracking using markers (point data to be analysed) Pinho et al. Human Movement Tracking and Analysis with Kalman Filtering and Global Optimization Techniques. ICCB 2005, Lisbon, Portugal, Pinho et al. Correspondência entre Pontos no Seguimento de Movimento em Imagens. 6APAET 2005, Azores, Portugal,

14 Human motion applications: Example of pedestrian tracking Baumberg, A. and D. Hogg. Generating spatiotemporal models from examples. In Image and Vision Computing, vol. 14, p ,

15 Human motion applications: Biomechanical i study of athletes Jesus, R.M. et al. Tracking of Human Motion using Multiple Predictors. In International Workshop on Articulated Motion and Deformable Objects, Palma de Mallorca, Spain,

16 Conclusions: A review about the computational techniques used for human motion analysis and some of their applications was presented; Human motion analysis is a complex, non-linear and time variant subject; The study of human motion in image sequences starts with feature extraction, ti then feature correspondence is established and ends with high h level l processing task; There are several human motion applications such as human recognition in surveillance systems, motion analysis in clinical studies or gait classification. 16

17 Future work: Develop methodologies for human motion analysis in 2D/3D: Develop methodologies to segment persons in image frames using modelization and update of images background Develop methodologies to track humans without using marks, treating in a robust away problems of occlusion, blobs b merging and splitting Using non-linear filters for tracking, like particle filters or unscented Kalman filter Develop methodologies to automatically build models for the motion and shape involved Develop mechanisms to automatically switch between the most appropriate models to be used 17

18 Acknowledgements: The first author would like to thank the support of the PhD grant SFRH/BD/28817/2006 from FCT Fundação para a Ciência e Tecnologia from Portugal; This work was partially done in the scope of the project Segmentation, Tracking and Motion Analysis of Deformable (2D/3D) Objects using Physical Principles, reference POSC/EEA-SRI/55386/2004, financially supported by FCT. 18

19 Human motion analysis: methodologies and applications Maria João M. Vasconcelos, João Manuel R. S. Tavares CMBBE th International Symposium on Computer Methods in Biomechanics and Biomedical Engineering 27th February 1st March, Porto, Portugal