A Distance Computer Vision Assisted Yoga Learning System

Transcription

1 38 JOURNAL OF COMPUTERS, VOL. 6, NO., NOVEMBER 0 A Distance Computer Vision Assisted Yoga Learning System Chen-Chiung Hsieh, *Bo-Sheng Wu, and Chia-Chen Lee Department of Computer Science and Engineering, Tatung University, Taipei, Taiwan. Abstract As computers and networks have been developed vigorously, distance learning could be integrated with computer vision techniques for the purpose of better learning effects. In this paper, we developed a distance yoga learning system for people to learn/play through the internet. The main point of the interactive learning system essentially consists in that the gesture performed by player, segmented by computer vision techniques, should possess the same silhouette for a given yoga posture. For better accuracy, the learning score is calculated by matching the distance transformation of the player silhouette with stored standard yoga posture. In the experiments, 3 postures were defined and six persons were invited to do each posture three times. About 86% of the difference between computer scores and the scores given by a yoga teacher falls within -.5~.5. Index Terms Distance learning, interactive learning, yoga learning, computer vision, distance transformation, posture recognition. I. INTRODUCTION A famous video sport system Wii Fit [] is very popular for body-shaping and weight-losing by balancing one s self mass center. There are a lot of sport games for Wii Fit and the Yoga game attracted the most attention from the very beginning when Wii Fit was launched in the market. Since the network infrastructure is well developed, multi-user game playing systems [-] unrestricted by location could be achieved by utilizing networks and cameras. Captured images of two users performing yoga postures according to a given posture could be sent over the network for reification. Therefore, the goals of this paper are to accurately detect the silhouette of yoga learners from internet under variety of environmental lighting and to recognize the yoga posture in real time. The rest of this paper is organized as follows. In Section, we briefly introduce related technologies. In Section 3, we present the detail of proposed video based yoga learning system. Section 4 gives the experimental results and conclusions are made in the last section. II. RELATED WORKS In Wii Fit games, the mass center of a user could be detected by a balancer as shown in Fig.. The central * The corresponding author. zzz736@yahoo.com.tw. The authors are with the Department of Computer Science and Information Engineering, Tatung University, No. 40, Sec. 3, ZhongShan N. Rd. Taipei, 04, Taiwan, ROC. area around the body in the figure is the range in which the mass center of the user has to keep. Wii Fit's balance board, based on pressure sensors, is used to detect the body center of gravity. The more the number of pressure sensing points, the more the accuracy. However, the disadvantage is the higher cost. By computer vision, human silhouette could be extracted with lower cost and higher accuracy. Advantages and disadvantages about Wii Fit and video based game system were given in Table. Distance learning systems are a promising option to deliver education to children who lived in rural area. The main problem with the large scale deployment of such system either relies on broadband network or interaction content for students and teachers. Lai et al. [3] proposed a software architecture that turns the global postal system into a generic digital communication channel that enables relatively sophisticated distance learning systems for disadvantaged areas. Their method explores techniques to compliment the communication channel with storage units and the internet. Thus, it could achieve a high bandwidth and low latency hybrid network on top of which a wide variety of applications could be built. Fig.. Scenario of the Wii Fit yoga game. TABLE. COMPARISONS OF WII FIT BALANCER AND VIDEO BASED GAME SYSTEM. System Wii Fit s multipoint pressure detection system Approach balancer webcam Advantages Disadvantages -quick response -resist ant to environmental changes -expensive -large size -single user Video based Yoga learning system -low price -less size -convenient in carrying-multiusers -affected by environmental changes doi:0.4304/jcp

2 JOURNAL OF COMPUTERS, VOL. 6, NO., NOVEMBER Puranam et al. [4] propose a monocular gesture recognition system that focuses on recognizing a set of arm movements commonly used for traffic control, aircraft landing, and communication over long distances. This is an attempt towards implementing gesture recognition systems that require full body tracking. They used distance transform techniques to track the movements by fitting the parameters of a defined model in every frame of the captured video. The tracker s output is fed to a state-machine which identifies the made gestures. Their gesture vocabulary contains only arms moving up and down, clapping, and arms waving. Jiang et al. [5] presented a novel human body gesture recognition method using a linear programming based matching scheme. Instead of attempting to segment an object from the background, they developed a novel successive convexification linear programming method to locate the target by searching for the best matching region based on a graph template. By matching distance transformations of edge maps, the proposed scheme is able to match figures with large appearance changes. However, only several pictures are tested. Microsoft is going to announce a new game system Project Natal [6] which is a revolutionary new way to play game without holding any controller. User could have fun by shaking their hips or just speaking. The only needed is life experience. According to the released news from Microsoft, Natal not only could track user s full body movement in 3-D, but also could recognize user s face and facial expressions. It could even detect emotion changes in user voice. Hence, user can connect to Xbox LIVE without remote controller. This is the future of a whole new user interface! III. YOGA LEARNING SYSTEM Background [, 8] subtraction is the most traditional and popular method for detecting the prospects. The concept of it consists in that a pixel value difference is acquired by subtracting an input image from the previously established background image. Advantages of background subtraction are that it is an easy method capable of quickly finding out the difference between the input image and the stored background image and there are several good adaptive background modeling methods. Therefore, background updating is deployed in our system. A. System Architecture Before playing the yoga system, background modelling has to be conducted so that the silhouette could be extracted by background subtraction. Given a yoga posture, the user has to complete the appointed posture within a predefined time period. Distance transformation [8, 9] and template matching [0, ] are then deployed to calculate the score. The system flow chart is as shown in Fig.. Fig. System flow chart. To reduce the lighting effects, the images captured from the webcam are transformed into YCbCr by Eq.() for better lighting resistant. Fig. 3 gives an example of the background subtraction. By calling cvaddweighted in the Opencv library, components of subtraced YCbCr are respectively multiplied by different weight Wi as in Eq.() to reduce the illuminance Y. Thus, Y will not be so sensible to the light changes as shown in Fig. 3. R = G B Y Cb Cr 8 FGi ( x, y) = Wi ( I i ( x, y) BGi ( x, y)), i = Y, Cb, and C r () where FG, I, and BG are the foreground, input, and background images. B. Normalization and Distance Transformation The extracted silhutte is normalized to fit different people doing the same posture at different distance. Then, distance transformation is used to reduce the error in template matching. Distance transformation [8, 9] is applied to the detected silhouette image to get a grayscale representation of object. Different from the common grayscale image, the intensity of each object pixel is the distance from the nearest object boundary to ()

3 384 JOURNAL OF COMPUTERS, VOL. 6, NO., NOVEMBER 0 (a) (b) (c) Fig. 3 Foreground detection by weighted YC bc r background subtraction. (a) The background. (b) A person therein. (c) Extracted silhutte after noise removal. (a) (b) (c) Fig. 4 (a) Euclidean transformation image. (b) City block transformation image. (c) Chessboard transformation image. that object pixel. The distance transformation can be calculated by three different ways: () Euclidean distance DE culid = ( p, p) = ( x x) + ( y y) () City block distance: = ( p, p = x x + y y D City ) (3) Chessboard distance DC hess = ( p, p ) = max( x x y y β (, y and P = ( x, y ) Where = x ) In Fig. 4 Euclidean distance transformation has the lowest accuracy and the most internal area of the Chessboard transformation is not obvious. Thus, Cityblock transformation is selected to conduct the transformation. The object edges for each layer are finer and the most internal layer is larger than the other types. C. Template Matching By template matching [0-] a set of suitable and fixed standard templates were previously stored for a subsequent comparison. The similarity of the observed image to the referential template was decided by the standardized mutual correlation coefficient in statistics. The higher the coefficient is, the more similar the observed object is to the template. The advantages of ) (3) (4) (5) template matching is simple and easy to implement. On the other hand, its disadvantage consists in that it can not tolerate object rotation and scaling. Before utilizing this method, we have considered such defects and avoid trapping into problems., There is no worry about its unbearable object rotation for most yoga actions are face forwand. The change in size will be normalized before conducting template matching. By distance transformation, the lighting effect could he reduced. Fig. 5 shows two examples of template matching on the distance transformation maps. Fig. 5(a) are the stored postures and Fig. 5(b) are the extracted user silhuttes. After distance transformation, we have the maps as shown in Fig. 5(c). By template matching as in Eq.(6), the larger the value calculated, the more they are of similar postures. The score computed normalizing DM to 0~00. DM[ i, j] M M = u = 0 v= 0 I [( i + u) mod N,( j + v) mod N ]* T [ u, v],0 i, j < N IV. EXPERIMENT RESULT Six persons were invited to test the system including adults and children. All the persons to be tested are in clothes having different colors from background. Table shows the average score for each posture. (6)

4 JOURNAL OF COMPUTERS, VOL. 6, NO., NOVEMBER (a) (b) (c) (d) (e) scores 75 scores 65 Fig. 5. Two postures for user to learn. (a) Standard postures in the database. (b) Binary silhuttes of the users. (c) Distance transformation images of the users. (d) Resulting images after template matching. (e) The calculated scores. (a) (b) (c) Fig. 6 (a) Communication between the two parties. (b) and (c) are the screenshots of the loser and the winner, respectively. TABLE. THE AVERAGE SCORE FOR EACH POSTURE. No score No score Fig. 6 shows the screenshots of proposed yoga learning system. Table 3 shows the Yoga posture database which was created by ourselves. Table 4 shows the binary image, distance transform and template matching results. If the clothes are of similar color with background, the extracted silhutte may be incomplete and these user may not easy to get high scores. V. CONCLUSIONS AND FUTURE WORKS After surveying most video games in the market, two users playing the game together, they have to be at the same site to utilize the same system and to watch the same TV at the same time. On-line game overcomes the limitation of single user and allows more persons to play at the same time. However, all the players of an on-line game have to communicate with each other through an avatar. With the most popular MSN messenger, people in communication can see each other through the webcam. They can use words or microphone to communicate but they can not play a game jointly. Thus, we decided to design a game through which the players can see each other, and make an interaction with each other by their own body/gesture. As to the functional performance, our system can be improved from aspects of processing speed and graphic user interface. Video based game systems are getting more popular for the advamed of computer vision and hardware cost down. Now, camera has become an indispensable peripheral product of a computer. Even a notebook contains a built-in webcam. The resolution of webcam will be getting higher and higher, and the capture frame rate is getting faster and faster, not only the distances among people has been shortened but also a bridge between people and machine was built. Thus, if more games are developed on our system platform, we believe that we could have a Natal prototype.

5 386 JOURNAL OF COMPUTERS, VOL. 6, NO., NOVEMBER 0 TABLE 3. YOGA POSTURE DATABASE. No. Graph No. Graph No. Graph No. Graph

6 JOURNAL OF COMPUTERS, VOL. 6, NO., NOVEMBER TABLE 4. TEMPLATE MATCHING RESULT FOR EACH GESTURE. No. Binary image Distance transformation Template matching No. Binary image Distance transformation Template matching

7 388 JOURNAL OF COMPUTERS, VOL. 6, NO., NOVEMBER 0 REFERENCES [] J. E. Deutsch, D. Robbins, J. Morrison, P. G. Bowlby,; Wiibased compared to standard of care balance and mobility rehabilitation for two individuals post-stroke, Virtual Rehabilitation International Conference, pp. 7 0, 009. [] Vuong, B.; McConville, K.; Use of recurrence quantification analysis in virtual reality training: A case study, Science and Technology for Humanity (TIC-STH), 009 IEEE Toronto International Conference Digital pp , Object 009. [3] J. Lai, E. Ziskind, F. Zheng, Y. Shao, C. Zhang, M. Zhang, N. Garg, S.Sobti, R. Wang, and A. Krishnamurthy, Distance Learning Technologies for Basic Education in Disadvantaged Areas Proceedings of the 8th Global Chinese Conference on Computers in Education, pp , June 004. [4] Puranam, Muthukumar B.; Towards Full-Body Gesture Analysis and Recognition, The Graduate School University of Kentucky Master Thesis, 005. [5] H. Jiang, Z. N. Li, Mark S. Drew, Linear Programming for Matching in Human Body Gesture Recognition, IEEE International Workshop on Analysis and Modeling of Faces and Gestures, pp , 005. [6] Natal, [7] Jin-Quan Lin, The Behavior Analysis and Detection of Falling, Computer Science and Information Engineering Master Thesis, NCU, page 9, 004. [8] Distance Transform, [9] Kuo-Ming Chen, 3-D Registration and Fusion of CT and MRI Images of Lumbar Vertebra. [0] Engineering Science Master Thesis, NCKU, pp. 7-, 003. [] R. Brunelli, and T. Poggio, Template Matching: Matched Spatial Filters And Beyond, MIT AI Memo 549, July 995. [] Tzu-Thieh Chang-Chien, The FPGA Realization by Using CMAC for Vehicle License Plate Recognition, Industrial Education Master Thesis, NTNU, pp. 4-6, 003. [3] Chen-Chiung Hsieh received his B. S., M. S., and Ph.D. degrees all in the Department of Computer Science and Information Engineering, National Chiao- Tung University, Hsinchu, Taiwan, in 986, 988, and 99, respectively. During Dec. 99 to Jan. 004, he was with the Institute for Information Industry (III) as a vice director. From Dec. 004 to Jan. 006, he joined Acer Inc. as a senior director. He is presently an assistant professor in the Department of Computer Science and Engineering at Tatung University, Taipei, Taiwan. His research area is mainly focused in image and multimedia processing. Bo-Sheng Wu received the B.S. in Department of Computer Science and Engineering from Tungnan Technology University, Taipei, Taiwan, in 006 and M.S. degree in Graduate School of Mathematical science at Aletheia University, Taipei, Taiwan, 008. Currently he is working toward his Ph.D. degree in Department of Computer Science and Engineering at Tatung University, Taipei, Taiwan. His research interests are min the area of video-on-demand, media streaming, multimedia information network and Interactive TV. Chia-Chen Lee received his B.S. degree in the Department of Computer Science and Engineering, Tatung University, Taipei, Taiwan in 00.