Development and Research on College Physical Supporting System for Kinect-based Motion Capture

Transcription

1 Development and Research on College Physical Supporting System for Kinect-based Motion Capture Abstract Lijun Tang Institute of Physical Education, Shanghai Normal University, Shanghai , China Olympic Games focus on "Higher, Swifter and Stronger" purposes. The athletes increase the speed and strength by constant challenge to records and going beyond themselves. With successful sponsorship of 2008 Olympic Games in Beijing, more and more people realize physical exercise is beneficial to physical and psychological health, and the concept of the mass physical is gradually accepted by Chinese people. Chinese administrations gradually reached the consensus on the concept that the "physical power" was based on the improvement in the quality of all-people motion. The improvement in the quality of all-people motion relies on the guidance on professional physical, and the attention should be initiated from students. However, presently systematic and professional technical guidance is lacked when viewed from physical teachings and instructions in universities and colleges of China, causes students suffering from damages during exercises. For this purpose, Kinect motion capture-based teaching assistant training system adapting to the demands of students in colleges and universities is designed in this paper, and the capture technology is utilized to provide solutions corresponding to possible figure posture distortion problems to drive virtual characters and to represent the character animations of testers, providing systematic references for the trainings of other auxiliary physical. Keywords: Kinect Technology, System Construction, College Physical, Assist Training. 1. RESEARCH BACKGROUND 1.1 Literature review While mass physical undergo rapid development, the information technology represented by the computer technology also make progress by leaps and bounds like rapid expansion in the personal computer and fast increase in the computing speed; the folks also conduct new exploration in the field of physical while buzzing the intelligence and cheapness. (Zhang and Wang, 2015). Since it is impossible to produce a large number of qualified physical professional teachers to fill substantial teaching requirements of colleges and universities within short period of time, why not designing exercise teaching system providing professional technical guidance to supplement or replace artificial teaching methods mainly including the human eye-based observation and experience-based training analysis method. Such operation system can not only overcome the limitation of space of human eye observation to guarantee the objectivity of the evaluation and guidance of the motion but also it may be extended in the time and dimension, like the playback, slow and fast plays of the actions. These advantages are the value and the center researched in this topic. Compared to ideal requirements, there is substantial space for existing teaching system. (Xu, 2016), in most cases the existing system is based on high resolution video capture and analysis-based human motion measurement method and human motion simulation- based human motion analysis method. Such system is not only capable of greatly observing the action of college students themselves and conducting comparative analysis to the target motion in the aspect of function. With respect to the price, merely a set of hundreds of thousands of high precisions video capture system determines the system can only be applied in the field of competitive physical rather than being widely used in the physical teaching for colleges and universities. Appropriate physical teaching system for colleges and universities still require further development of computer aided motion system. 1.2 Research purposes Theoretically, the motion of human body is closely related to the distribution of the human body's skeleton, namely states of skeletons, the three-dimensional reconstruction of the human body will be provided with an important basis if the distribution of skeletons is acquired in real-time and proper manner, and the height, body 706

2 size and other auxiliary information are to be resolved afterwards. The progress of research on this subject may inspire theory and research methods for other subjects. It is easily to set up the model of human body by integrating the information of such aspects. The motion capture and analysis of three-dimensional human body includes the image processing, computer vision, skeletal animation and other knowledge and also demonstrates the theory for pattern recognition and artificial intelligence is a multi-disciplinary cross-over research orientation. With respect to practical application (Yang, 2016), not only academic results of college students can be scientifically and efficiently improved but also professional athletes are also provided with auxiliary motion. The computer is utilized to replace human eyes traditionally used to track and record three-dimensional information of the athletes in the games, achieving the improvement in terms of accuracy and controllability and providing the guarantees for the development of scientific training plan for coaches and rapid improvement in results of athletes. 2. OVERVIEW OF KINECT DEVELOPMENT PLATFORM 2.1 Kinect structure Kinect contains multiple components, including the motor, USB interface, infrared transmitter, infrared receiver, RGB camera, microphone array, chips, etc. The structure diagram of the device is shown in Figure 1. Figure 1. Schematic Diagram of Kinect Structure The middle of two sets of cameras of Kinect is RGB color camera, which is capable of capturing two-dimensional standard color image data stream, the maximum resolution is 1280 * 960, and the infrared transmitter and infrared receivers are respectively located on both left and right sides, constituting the depth camera together, which is capable of capturing the depth data stream, namely recording device the distance between the recording unit and the object to be tested, the maximum resolution hits 640 * 480.(Zhang and Liu, 2016). Kinect involves two models including Kinect for Xbox360 and Kinect for windows, close shot work mode for more advanced technical supports is adopted in more Kinects for windows bold in developing functions, Kinect for windows is adopted to develop this system research, and best interaction depth distance ranges from 1.2 to 3.5 m as shown in Figure 2. The microphone array is capable of collecting the audio data and conducting the positioning for students according to the time difference of the audio data collected audio signal by audio signal processing technology makes more accurate, is advantageous to the realization of speech recognition and positioning of solidarity, and other functions.(wang, 2014) drive motor can access visual Angle, by adjusting the best attitude to use the position of the students, to pretend students beyond the scope of activities. USB block is used to connect the device and the computer, and it empowers the chip, and additional power supply is required to support other part. The core component of Kinect is the PS1080 chip of prime sense. The data is calculated and processed by adjusting the parts to control and image coding for projection, the generation of infrared spectroscopy and generation of depth image, and the depth data stream, color data stream, audio data stream of sensors are synchronized and passed back to the computer. 707

3 Figure 2. Kinect Interaction Distance 2.2 Operating principle of Kinect The depth sensor for Kinect acquires three types of raw data, which are the color video stream, depth data stream respectively, and original audio stream respectively and with correspond to three processing courses including the identity identifying, skeleton tracking, speech recognizing. The developers may conduct the research and development according to three data streams as shown in Figure 3. Figure 3. Research and Development of Kinect Data Stream Principle of Kinect acquiring the depth data is based on the theory for optical encoding technology, it is different from traditional optical measurement technology and TOF technology, optical encoding technology is a technology coded for three-dimensional space by use of light source. Kinect infrared emitter emits an invisible laser light which is capable of uniformly distributing the space, this type of light goes through the ground glass and infrared filter and cover the space object surface within the visual range, forming "laser speckles" with shapes, and each spot of this type of speckle is of uniqueness, equivalently marking the space.(wu and Duan, 2014) CMOS infrared light-sensitive chips collects the lights of speckle figure in the space with the rate of 30 frames per second, and decode the data operation, thereby acquiring the depth information of the target object. 3. DEVELOPMENT OF KINECT MOTION-BASED CAPTURE SUPPORTING SYSTEM FOR COLLEGE PHYSICAL 3.1 System framework design The design for functions, internal structure and development content of current system is planned mainly for purpose of meeting the requirements of practical applications of the college students, and overall framework of 708

4 the system may be established by four functions, the details are shown in Figure 4. Figure 4. University Physical Auxiliary Training System Frame The design for video teaching module mainly provides the students with physical item standard action as the video guide to have the students learn by watching video and choose to the chapter and contents of physical section to be learnt. (Wang, 2014). The action acquisition module includes the data acquisition, data filtering noise reduction processing and skeleton data saving. The movement data of college students is saved after the processing of data filtering noise reduction, and the standard action database is set up. The skeleton data of the tester is saved after filtering noise reduction processing to wait for the phase of test score. The auxiliary scoring module mainly conducts the angle feature extraction on standard skeleton data of action sampling module. The angle feature of the test skeleton data is extracted, DTW algorithm matches the corresponding frame. (2017) Ma and Gui, and then two groups of skeleton data undergo the speed feature extraction respectively, auxiliary judgment basis is provided to the test module by combined with the angle feature data extracted previously, the test skeleton data and standard data are compared, the scores are evaluated, the advices for college physical trainings are proposed, achieving the goal of assistant training. The three-dimensional animation display module mainly assigns the movement document developed from physical exercise data of the tester to the role model built, drivea virtual characters, and represents the animation of students' exercises. (Hu, 2016) making the students get rid of traditional training based on the experience, increasing the interest of the training. 3.2 Standard database design The first step of college physical exercise assistant training system is the data collection. Considering motion capturer worn under normal circumstance makes the college physical teachers or coaches are restricted by the age and motion, being adverse to the presentation standard, thus this system acquires the motion data through the human body without wearing body feeler without wearing other equipment. (Yang, 2016) In this way physical education teachers can freely demonstrate standard motions, afterwards the data of the athletes collected is recorded in the form of virtual data. The establishment of physical item standard database directly determines the profession of the system throughout the teaching evaluation. 3.3 Teaching video design According to the characteristics of the physical, it is required to follow college physical coaches, thereby improving the training level in the course of repeated learning. (Fu and Zhao, 2013). For purpose of this system, one of the main functions is to provide the demonstration of standard physical item motions. The teaching modules of this system involve traditional video teaching, its applicability is wide, and the students may learn by only needing the computer or mobile client in the form of video playback. The flow chart is shown in the 709

5 Figure 5 below: Figure 5. Physical Video Teaching in Colleges and Universities The video sources in the flow chart are teaching resources provided by the teaching modules of this system, which is provided with scalability and rich contents, including the video guide for physical standard motions and related culture of assisted learning physical. Standard motion video guide is to include a full range of motions including the physical invent and standard motion essentials subdividing the teaching behaviors. The following practice sport item is not only long and repeated training, and its process is a kind of accomplishment. So the teaching of this system introduces the physical culture in the form of video play, including cultural background, celebrity introduction, theoretical knowledge, origin and use method relating to physical exercises. (Liu, 2013). The students can choose learning content according to the demand, the control parts are sent with demands with through the buttons of the system interface design, the program controls the play, pause, fast forward, fast backward and other screen controls by students entering the command controls, achieve the goal of learning. 3.4 Joint feature extraction The three-dimensional coordinate information of 20 space frame joint points may be obtained by Kinect. The contribution degrees of 20 joint points on the descriptions of motion states are different. For purpose of the extraction angle feature, the skeleton model for human body not only includes the shape information but also contains the information of skeleton structure. According to characteristics of physical training, the space site of limbs decided by the rotation characteristics of joints plays key role. And Four key points of human body parts in such process basically show a straight line, it is believed that contribution of such four key points on the state description are not significant, and hands, feet and other details are ignored. (Zhao, 2015). This paper focuses on joint and angle characteristics of eight main points including the shoulder, elbow, hip and knee joints under static posture. The eight key points may be divided into two classes shown in Figure 6 according to the degree of freedom: Freedom Rotate 360 Figure 6. Schematic Diagram of Joint Angle 710

6 These four joints including the left elbow, right elbow, left knee and right knee the four can only rotate in the same plane. These four joints including the left shoulder, right shoulder, left hip, right hip may be rotated in multiple directions in the space. For four joint points including the elbow and knee joint points, their joint angle value ranges from 0 to 180, hence the joint point may be calculating the angle between two vectors. Take the right shoulder and right arm for examples ( CEF), when the arms are forward vertically or straight down, the angle value of the shoulder joint point value is 90. Therefore, the display is not reasonable if the state of right shoulder is described from the angles of FE and EC. The three-dimensional space coordinate system is established in this paper in directions of AC, E1, two sectors and normal vector. The right shoulder joint point may be used to represent the states of skeletons in the combinations of Angle FEX, Angle FEY and Angle FEZ (i.e: x, Y and Z, respectively represent the directions of AC, E1 and cross product, the normal vector of plane). (Gao, 2015) It may be derived by space geometry the angle between the direction of the line can be uniquely identified can be known if the angles between a straight line and three non-collinear lines are known. The angle characteristics of left shoulder and hip joint points may be calculated with similar method s. 3.5 DTW matching algorithm DTW algorithm itself is provided with the advantages of simplicity and effectiveness, it is widely used in speech recognition, text data matching, and other fields, and it is a kind of template matching optimization algorithm. The thought of that algorithm calculates the proximity between the test sample and standard sample by mainly extending and shortening the time series, and set up the time calibration matching path between the test sample and standard sample. And solve the path with minimum cumulative distance between two samples in the process of matching as the optimal path. (Zheng and Luo, 2013) The rate of Kinect acquiring the data hit approximate 30 frames per second, a series of action sequence may be viewed as the acquirement of continuous frame data. AndTwo action sequences U=(U 1,U 2, U m) and V=(V 1,V 2, V m) are defined, V denotes the test sequence and V denotes the standard sequence, and their lengths are m frame and n frame respectively. U i means the angle characteristics of the ith frame of human body posture in the action sequence u. m n is satisfied under normal circumstances, it is required to build a matrix grid, and the matrix D is defined as follows: U1, V1 ) U2, V1 ) Um, V1 ) U, V ) U2, V ) U 1 2, V ) 2 Um, Vn ) U, ) m Vn Um, Vn ) 2 D (1) where, U i,v j )-- U i,v j denotes the euclidean distance of corresponding angle characteristics. The objective is to find out the shortest path going through a number of grid points in the grid matrix, the grid points passes through are the points of the frame corresponding to two sequences. The shortest path is orderly put continuous matrix elements in Wk={w 1,w 2, w k} to map the relations of angle characteristics of action sequences U and V, and the (k-1)th and k elements were are respectively defined to be W k 1 = (i, j ) k 1, W k = (i, j) k.this optimal path has to satisfy the following constraints. (1) Boundary constraint: w 1=D(1,1), w k=d(m,n). (2) Boundedness: max(m,n) k m + n 1. (3) Continuity: w i=d(i,j), wk-1=d(i,j ), i-i 1, j-j 1. m 711

7 (4) Monotonicity: i-i =1, j-j =1 are not allowed to appear synchronously. Figure 7. DTW Searching for Optimal Matching Path According to the above constraints, optimal path of distance matrix, V j and V j are corresponding frames if certain frame U i in Sequence U and a frame V j e in Sequence V, corresponds the same element of the optimal path. The following formula is the posture difference formula for human body: Euler angle formula (Euler angle is a relatively intuitive angle representation method, corresponding Euler angle also changes with the time with the rotation of the body coordinate system) is used in the D Angle (A, B) = 1 n n i 1 D Angle(B i, B i ), the angle vectors of the human body are set to be R(θ), R(φ), R(γ): Then the rotation of the space coordinate system relative to the geographical coordinate system can be expressed as the function of the angular velocity vector: R () 0 cos( ) sin( ) (2) 0 sin( ) cos( ) cos( ) 0 sin( ) R ( ) (3) sin( ) 0 cos( ) cos( ) sin( ) ( ) sin( ) cos( ) R (4) where it is the differential equation for Euler Angle, according to measured angular velocity vector, the joint rotation angle information of the human body may be obtained by resolving the equation. 4. CONCLUSION This paper, based on the motion capture technology and virtual reality technology, designs and develops 712

8 auxiliary training system for Kinect based motion capture -based college physical education to assist physical exercise teachers in training, and judge the postures and motions are standard and timely correct. As an appropriate complement to traditional teaching, it can not only help students learn according to video learning modules provided by themselves, and can help students evaluate and score their own motions, improves the interests of students in learning, realize autonomous learning and self-assessment, effectively solves the inevitable problems of traditional physical education learning, and provides reliable basis for scientific training in the sports teachings of colleges and universities, thereby improve the efficiency of training, casting a strong practicability. Practical reference value is delivered on the application of Kinect in the sports, virtual display, animation production, body feeling, and other games. ACKNOWLEDGMENTS The primary and secondary school physical education curriculum implementation effect of empirical research in China, Humanities and Social Sciences project of Ministry of Education of China, number, 15YJC890033, Youth fund projects, REFERENCES Fu Q, Zhao H.Q. (2013). Application of motion capture technology in sports simulation, Journal of Shanxi Datong University (Natural Science Edition), 29 (05), Gao X. (2015). Research on motion capture technology based on Kinect and after effects, China New Communication, 17 (14), 112. Hu W. (2016). Application of 3D motion capture system in physical education. automation and instrumentation, (12), Liu Z. (2013). Research on the application of human motion capture technology based on somatosensory in military sports, Journal of Military Physical Education, 32 (04), Ma G.T., Cui J.W. (2017). Application of practical motion capture technology in teaching, Mechanical Engineers, (02), Wang W.J. (2014). Application of sports biomechanics in motion capture technology and sports system simulation analysis of, Testing, (13), Wang Z.Z., Wang H.Y. (2014). Research on human work simulation method based on, Kinect Journal of Zhejiang Sci-Tech University, 31 (01), Wu C.M., Duan W.J. (2014). 3D motion capture application of. in the teaching of college sports system, Chinese education information, (08), Xu J.P. (2016). Research on college physical education training based on virtual technology, Journal of Xi'an University of Arts and Science (Natural Science Edition), 19 (05), Yang Z.Q. (2016). Research on teaching demonstration system of physical education based on Kinect, Automation, Automation and Instrumentation, (10), Yang Z.Q. (2016). Research on teaching demonstration system of physical education based on, Kinect Automation and Instrumentation, (11), Zhang T.L., Wang X.H. (2015). The motion capture technology in sports in the application of. electronic test, (24), Zhang Z.Y., Liu K.Z. (2016). Design and implementation of motion capture system based on Kinect, Guangdong Communication Technology, 36 (10), Zhao M. (2015). Construction of sports teaching and training system based on motion capture technology, Electronic Test, (02), Zheng L.G., Luo J.L. (2013). Implementation of motion capture system based on Kinect, Journal of Jilin University (Engineering Edition), 43 (S1),