Framework Study on the Three-dimensional Long-distance Running Sport Training Based on the Markerless Monocular Videos

Transcription

1 Framework Study on the Three-dimensional Long-distance Running Sport Training Based on the Markerless Monocular Videos Bin Feng 1, Wenbao Li 2, Jian Shi 1, Zengyang Li 3 1 Tianjin Collage, University of Science & Technology Beijing, Tianjin301830, China 2 Jilin Sport University, Jilin130022, China 3 Sichuan Institute of Sports Skills, Sichuan , China Abstract With constant improvement of the science and technology level and gradual maturity of the computer vision theory, 3D human body motion has gradually become one of the key aspects in the computer vision study. 3D human body motion cannot only play an important role in the virtual reality technology, film and television animation, and game production, robot motion design, but also play an important supporting role in the physical training. Traditional computer vision technology mainly uses the monocular video or camera system with marker, which however have certain restrictions in the 3D human body motion. So the 3D human body motion will be usually made by using the markerless monocular video. However, such mode still has its defects. In this article, the markerless monocular video has been specifically optimized by constructing the human body motion model from static and dynamic aspects, raising the constructing strategy of 3D human body gesture estimation, then proposing the application method of the markerless monocular video in the long-distance sport training. This has served as the experience and reference for improving the study and development level of the assisted physical training system in China. Keywords:Markerless, Monocular video, Sport recovery. 1. OVERVIEW OF THE STUDY 1.1 Background for the study Computer vision is an important branch of the artificial intelligence development, and its core idea lies in enabling the computer to simulate the personal vision through the algorithm and procedure, for observation of the static or moving objects in the objective world, and thus for the extraction of the information that cannot be extracted with the personal eyes, for deeper observation and understanding of the world. In the background with high-speed development of science and technology, the computer vision-related technology has been gradually mature and widely used in various aspects for the life and development of human beings, while with respect to the physical sports, the psychologist Johansson thought in 1973 that in the human vision, it was very difficult to extract the valuable information for the static joint or gesture, while if the joint or gesture is in the motion, it is possible to distinguish the walking and running movements according to the movement trend, and even under the premise that other conditions are not given, it is possible to distinguish the gender of the other party from the movement. In the 1980s, Marr systematically proposed the computer vision theory, thinking that vision is a intrinsically distinguishable process, which can be classified into three phases, first, low-level vision, that is, extracting the color, boundary and texture of the image by using the testing approach; second, intermediate-level vision, that is, the information of 2.5D, such as 3D scenario depth; third, high-level vision, that is, improving this information into the 3D information based on the intermediate-level vision. Such computer vision theory has become the fundamental theory in the development of computer vision. 1.2 Literature overview The human body motion vision based on the computer vision technology not only has wide space for application in virtual reality technology, intelligent monitoring technology, advanced user interface, motion analysis and motion assistant system, but also has huge potential economic value. Nowadays it has become the important 454

2 direction for the computer scientific research. However, the current computer vision technology is not so mature, the equipment is expensive, and also certain technological defects. For example, when the computer vision technology is used to express the high-dimensional status parameter space in the three-dimensional long-distance running, it will generate a huge computing quantity, while in the motion, when passing other object or due to the self-occlusion, it will result in the ambiguity problem during the estimation of the human body motion vision. Therefore, there is still very large research and development space for the human body motion vision (Li et al, 2012). The three-dimensional human body motion analysis has rich theoretical system and is widely used. For the time being, the three-dimensional human body motion analysis mainly consists of motion examination, motion estimation and motion behavior identification. The research of three-dimensional human body motion test centers on the motion object, and the contour region is locked, even the simple and regular-shaped graphical region, such as rectangular, and oval. The research of three-dimensional human body motion estimation focuses on the determination of the status of every joint or location in the motion for the motion object. The three-dimensional human body motion identification focuses on the analysis of the possible semantic behavior from the motion behavior for the motion object (Dai et al, 2012). In the traditional physical training, usually the trainer gives the instruction on the athletes by experience, while the athletes will receive the repeated exercise of the skilled movement, thus improving the motion skills. But such mode has low training efficiency, difficult to meet the training demand of the athletes. In the physical training, the three-dimensional human body motion analysis is integrated, capable of analyzing the motion trail and motion trend. Such information data will have an important assisting effect in the physical training, and enjoy wide application prospect (Zhao, 2012). 2. CONSTRUCTION OF THE HUMAN BODY MODEL 2.1 Brief introduction of the human body model Static human body model The three-dimensional human body motion analysis is intrinsically the process for the estimation of the posterior probability under the framework of high-dimensional space. In order to improve the accuracy of estimation, it usually requires the introduction of Bayesian network for the reasoning under uncertainty (RUU), which is an important method reflecting the causal information (Chen and Wu, 2015). Figure 1. Upper Body Model of Human Body Fig. 1 is the upper body model of human body, linking the body mainly by the nodes, where every segment of the body has one status vectorx i and image observation Y i. Based on the above model diagram, the propagation algorithm with circle of trust has been chosen, thus resulting in the three-dimensional final shape model of human body (Chen et al, 2015), as shown in Fig. 2, 455

3 Figure 2. Final Model of Human Upper Body There are many methods for the expression of static human body model. In addition to the above expression methods, it is possible to use Markoff network for expression, with the structural diagram as shown in Fig. 3. Figure 3.Markoff Network Human Body Model Where, X=[X 1,, X M ] mainly represents the hinged human body gesture, while the special node X K will be expressed with Z K, therefore, the observed node images shall be expressed with Z=[Z 1,, Z M ] (Chen and Zhuang, 2016). So in the computer vision technology, for the target of the sport object tracking, it has become the posterior probability of X K to Z K in Fig.3: P(X Z ). Compared with the above method, this method has its advantages and disadvantages, that is, static human body simulation method is more intuitive and simple, while Markoff network is more detailed.the method should be chosen according to the study contents (Zhang et al, 2010) Dynamic model of human body Human movement is mainly achieved through the motion of the skeleton and joint. From the human dynamic aspect, dynamic model mainly refers to the human body structure model for the description according to the principle for the motion of human skeleton or joint within certain range. Generally speaking, in thehuman body dynamic structure model, the skeleton tree structure is formed by some fixed limbs through the movable connecting point, that is, joint, according to the human skeleton structure (Wu and Tian, 2010). Though every joint point has high flexibility, there is certain interconnectivity, and based on the judgment on the movement mode of one joint, it is possible to smoothly analyze the movement status of the related joint points. This makes the resultant skeleton tree structure form the herringbone-structure model complying with the human movement standard. With regard to the composition and use, it is possible to use two-dimensional human body model and three-dimensional human body model. The two-dimensional human body model is shown in Fig. 4 below, 456

4 Figure 4. Two-dimensional Cardboard Man Model Such two-dimensional cardboard man model usually only exists in the plane. The two-dimensional cardboard man model is composed of the limb segments of the plane, including some parameters. By adjusting these parameters, it is possible to adjust the pattern of the two-dimensional cardboard man model, and track the two-dimensional plane dynamic model of the motion object. If these cupboard model will be elaborated according to the bone and joint structure of human being. It is possible to obtain more exquisite two-dimensional cardboard man model, meeting the motion models of different dimensions in the restriction of two-dimensional space. The two-dimensional cardboard man model has simple structure, and cannot accurately describe the behavior of the sport object in the long-distance running sport. So it is required to construct the three-dimensional human body model, as shown in Fig. 5, Figure 5. 3D Hinged Human Body Model From the motion principle of the human joints, it may be known that every joint can turn to three directions at most, and according to the actual research directions, there is relatively large difference for the human body gesture based on the 3D human body model. For example, in case of the study singly on the motion of the upper human body, the joint point of the gesture has about 10 degrees of freedom, and in case of the whole-body motion, it is required to input at least 50 degrees of freedom for the study. Thus the long-distance running sport is analyzed with the three-dimensional human body model, which would generate huge computing quantity. Though it is limited for the scope of human motion, the computing quantity is very huge and it is one of the key problems difficult for resolving. 2.2 Construction of human body model From the above contents, it may be known that two-dimensional human body model and three-dimensional human body model have their respective advantages and disadvantages. So it is possible to choose the most favorable model according to the actual situations. While in the three-dimensional long-distance run training, 457

5 both models have the major defects. So they may be combined to propose a new human body model construction method Human dynamics model Human dynamics model is constructed mainly with the hinged structure, consisting of 14 joints connecting the limbs, as shown in Fig. 6. Figure 6. Human Dynamics Model The major characteristic of such human body mechanical mode is not subject to the two-dimensional or three-dimensional restriction. In case of the study in the two-dimensional aspect, 14 joint points are distributed according to the coordinates X k t = (u, v)(k = 1, 2,, 14). When at a time spot t, the human gesture is X t = (X t 1, X t 2,, X t 14 ). When the three-dimensional study is carried out, 14 joints are expressed according to the three-dimensional coordinatesx t = (x k, y k, z k )(k = 1, 2, 14) (Zou et al, 2010) Body appearance model Now, the tracking of human body gesture is mainly realized with the method from up to down, that is, taking separate consideration from the limbs of the motion object, and then combining the from down to up into the final human body pattern. In the above contents, one simplified human body model has been constructed through thetwo-dimensional cardboard man model, where there is no expression of the trunk part, mainly because there is little change that can be generated by the trunk part, and when the three-dimensional coordinates for other joints have been defined, it is very easy to estimate the three-dimensional coordinates of the trunk. So this portion can be directly omitted, thus effectively reducing the computing capacity. In the body appearance model, in case of t=0, the initial position of every joint point is X t o (k = 1, 2,, 14). In the actual observation, the change of the light ray has certain impact on the body appearance model. To eliminate such influence, HS channel in HSV may be used as the template, to address the influence from the change of the light ray Body graph model In the above body graph model, there are 14 joint points, that is, forehead, neck, left and right shoulders, left and right wrists, left and right buttocks, left and right knees, left and right ankles. In addition, there is one point not explicitly marked, that is, sternal angle. Sternal angle is the center of mass of human, and is unchanged. So it 458

6 will not be calculated and tracked, but can be obtained from looking for the center of mass in the framework formed by the left and right shoulders, and left and right buttocks. The human body graph model is shown in Fig. 7, 3. 3D HUMAN BODY GESTURE ESTIMATION Figure 7. Body Graph Model According to the above contents, the two-dimensional coordinates of 14 joint points are X k = (u k, v k )(k = 1, 2,, 14). The three-dimensional coordinates are the output values, and will be implanted in certain proportion into the projection model, thus obtaining the three-dimensional coordinates of 14 joint points X k = (x k, y k, z k )(k = 1, 2, 14), to build the human body motion model in the three-dimensional space. Therefore, it is possible to obtain the following relation formula, x k = u k y k = v k s s (1) From the actual observation, it is possible to obtain the known quantity u k, v k. To obtain x k, y k, it is only necessary to obtain the value of s (Liu, 2014). Supposing that l mn represents the length of the limb segment mn projected on the plane, it is possible to draw the following conclusion in combination with the equation for the distance between two points in the space, So the relative depth between two points is: 2 l mn = (x m x n ) 2 + (y m y n ) 2 + (z m z n ) 2 (2) z = z m z n = l2 mn ( u m u 2 n + (v m v n ) 2 ) s 2 (3) In actual measurement, the length of the limbs is unchanged. So the relative depth in the above formula can be considered as the function of the scale factor s. In the above contents, the length of the lower limbs is obtained from the initialized operation. Supposing that the pixel is the unit and the initialized length is d mn, then it is possible to obtain from the formula, L mn =d mn /s(4) z = z m z n = d2 mn ( u m u 2 n + (v m v n ) 2 ) s (5) In the above formulas, the scale factor s plays a critically important role. Besides, for mn between two points, (Wang et al, 2011). For this purpose, suppose that the reference point is X root = (x root, y root, z root ), and place the reference point in the plane (z root = 0), since there are certain correlations between different joint points, 459

7 these correlations can link these joint points in the three-dimensional coordinate system, thus recovering to the shape of the human. 4. APPLICATION FORMULA OF MARKERLESS MONOCULAR VIDEOS IN THE LONG-DISTANCE TRAINING 4.1 Acquire the human body motion parameters In the traditional long-distance run training, the trainer instructs the students mainly by means of experience and visual inspection. Such means has certain advantage, but comparatively speaking, it is blind, and often has the problems, such as low efficiency and poor training effect. While in the markerless monocular video, it is capable of acquiring the sport parameters of the sport object in a more effective manner, and also plays the important role for study and reference when the trainer customizes the training plan and changes the traditional training model (Li et al, 2016). 4.2 Contour extraction of the athlete It is always difficult to quickly and accurately extract the contour of the motion objects. In actual capture, due to the influence of objective environment change, scene background, shadow, camera movement and blocking by the external object, it will result in inaccurate extraction of the athlete contour. While in the context, the study has been conducted on the contour of the motion object on the basis of the markerless monocular video, achieving the optimizing effect. The motion contour of the athlete is captured with the computer vision, capable of enabling the trainer to capture whether the skilled actions of the athletes in the long-distance running are reasonable, and whether there is the space for optimization, so as to give the targeted guide and adjustment (Xia et al, 2014). In the high-speed development of computer vision, it is possible to place the motion contours of the athletes under the same background for comparison, thus exquisitely distinguish the athletes in skilled actions, and effectively improve the long-distance running skills of the athletes. 4.3 Acquire the sport data One important advantage of markerless monocular video is to extract the true sport data of the athletes through the capture of the athlete contour and behavior (Zhao and Yin, 2010). Such sport data is quite precious resource for the trainer to understand the physical ability, explosive effort, and staying power of the athletes, thus customize thecorresponding training plan for the athletes, and maximally meet the personalized training demand of every athlete. The sport data is mainly acquired through the sport capturing sensor on the body of the athlete, so that the related information can be transmitted into the computer during the sports of the athletes, forming the virtual printed image and obtaining the related sport data. Such data is adjusted entirely according to the physical activities of the athletes, having very high accurateness, and cannot only be used in the long-distance running, and but also havecertain effects in the sport events with highly complex skilled actions (Liu and Du, 2013). REFERENCES Chen J., Wang W.C., Wu E.H. (2015). Unmarked human motion tracking in monocular videos, Journal of computer aided design and computer graphics,(09), Chen J, Wu E.H. (2015). Iterative optimal estimation of human 3D motion in monocular videos, Journal of computer aided design and computer graphics,(07), Chen X.J., Zhuang Y.T. (2016). Monocular video 3D human motion, recovery, Journal of computer aided design and computer graphics,21 (08), Dai Y.C., He M.Y., Li H.D., Li B. (2012). Smooth constraint method for 3D reconstruction of monocular moving target trajectory, Chinese Science: Information Science, 42 (08), Li P.R., Huang W.J., Tao X.B. (2016). Research on 3D reconstruction of monocular video based on moving objects with multiple rigid bodies, Computer and digital engineering, 44 (10), Li Y., Sun Z.X., Chen S.L., Li Q. (2012). Method of human pose analysis in monocular video based on annealing particle swarm optimization, Journal of automation,38 (05), Liu L. (2014). Research on scene modeling method of intersection based on monocular video, Journal of Chifeng University (NATURAL SCIENCE EDITION), 30 (20),

8 Liu S.H., Du K. (2013). Calculation method of human body measurement parameters under monocular video, Journal of Naval University of engineering,25 (03), Wang W.Z., Wang Z.Q., Deng X.M., Hong H., Qiu X.J. (2011). Human motion tracking in monocular video based on local prior model, Computer aided design and computer graphics, 23 (09), Wu Y., Tian O. (2010). 3D human animation monocular video motion tracking, Micro computer application,(05), Xia A.M., Wang Y.S., Zhang H.Z. (2014). Design of monocular video aisle number statistics system, Electronic technology and software engineering,(15), Zhang L., Lin H.P., Jin G.C. (2010). Research on head 3D motion simulation algorithm based on monocular video, Information technology, 33 (09), Zhao W.S., Yin S.G. (2010). Research on 3D surface reconstruction, Computer and modernization, monocular image sequences, (02), Zhao Y. (2012). Research on attitude recovery technology based on single shot video by annealing particle filter, Journal of Ningxia University (natural science edition), 33 (02), Zou B.J., Chen S., Peng X.N. (2010). The history of operation. No marked 3D human motion tracking, Journal of computer aided design and computer graphics for monocular video,(08),