A Supporting System for Beginners of Billiards Based on Predicting the Path of the Balls by Detecting the Cue Stick

Transcription

1 A Supporting System for Beginners of Billiards Based on Predicting the Path of the Balls by Detecting the Cue Stick Akira SUGANUMA Department of Electronics and Information Engineering, National Institute of Technology, Ariake College, 150 Higashi-hagio, Omuta, Fukuoka , Japan Fumiya TAKATA Advanced Production and Information Systems Engineering Course, National Institute of Technology, Ariake College, 150 Higashi-hagio, Omuta, Fukuoka , Japan Abstract- Billiards is a sport that every player can enjoy. However, beginners cannot enjoy a game of billiards as much as they likes, because they cannot shoot a ball to his satisfaction frequently. Therefore, the authors have developed a system supporting a beginner player of Billiards by using OpenCV, which is a library to make a system for the image processing, so that they can enjoy a game of billiards. Our system has a USB camera and a projector which are mounted over the billiards table. The system distinguishes among the balls and detects the direction of the cue stick from the image captured by the camera. It detects the straight line of the both sides of the cue stick using the Hough transformation. The system estimates the moving path of the white ball and the color balls which are hit by the other balls, and projects the path each ball on the table by the projector. Thus, the beginners can predict visually the path of the balls and shoot the ball. Keywords USB camera, projector, image processing, OpenCV, Hough transform I. INTRODUCTION Billiards is known as a mental sport played in the room. The summary of the game of the billiards is to hit the white ball (cue ball) with the long stick called cue stick and to drop some colored ball (numbered ball) in one of the pockets at the four corners and the two sides of the billiards table. Billiards is played and enjoyed by a wide range of players because his physical strength and his body-build do not matter. On the other hand, many players may feel difficult to play billiards. They, especially the beginner players, can t imagine the course of the white and color balls when they want to pocket the colored ball. Furthermore, it is hard for them to forecast the course of the ball reflected at the cushion. For the reasons mentioned above, the beginners cannot fully enjoy the game of billiards. Accordingly, this study has focused on supporting the beginner players. We have developed a system with an image processing using OpenCV and tried to perform visual support for them by using a camera and a projector[1,2,3]. In concrete terms, the system captures an image by the camera located over the billiards table and then processes it to make the support information such as the path of some balls. The system realizes visual support by reflecting it on the table by the projector located over the table. This study is conducted for three years back. Hence, the old version of the system distinguishes all balls on the billiards table after they have stopped, and calculates the easiest course of the cue ball in order to contact the lowest numbered ball and to pocket it. The method of distinguishing the balls has been the SVM (Support Vector Machine). The accuracy of this method is about 97.3%[3]. This system have been indeed practical, bat the player cannot always hit the cue ball along the displayed course. Although studies have been made on path calculation from the placement of the ball to a pocket, there has been no study that tried to detect the cue stick and obtain the path of the ball. We tried, therefore, to detect the direction of the cue stick by the straight line detection technique. As a result, once the player puts the cue stick for the cue ball, our system is able to display the path along which the cue ball and the colored ball will go if the player shoots the cue ball without changing the direction of the cue stick. The beginners will be able to predict the path of the cue ball and the colored balls by our system. Special Issue on ACEIAT & JTSTE - Thailand ISSN:

2 II. SUMMARY OF SYSTEM A. Processing in the System This system has been built for beginners to play billiards. It offers visual support to them. Figure 1 shows the appearance of the system. Figure 2 shows a result of processing by our system. We describe below a flow of the processing in the system. (1) Capture a state of the billiards table as a background image by the camera. The camera is installed right above the center of the billiards table. The height is 2.56 meters from the floor. The system gets the overall image of the billiards table by this camera. (2) Process an image with a computer. As a beginning the position of the cushion is established by a user clicking a mouse. Next, the computer detects the position of the balls and the direction of the cue stick. Finally, the computer predicts the path of the ball using this information. Figure 2 shows the result that predicted the path of the ball. First, our system distinguishes the cue ball and other balls. Then, the cue stick is detected when the player is pointing the cue stick to the cue ball. Third, our system calculates the path of the balls in consideration of the rebounding with the cushion and the contacting other balls. (3) Correspond to the size of billiards table by coordinate transformation. The system draws the path of the ball on the image and performs coordinate transformation of it in order to correspond to the size of the billiards table. (4) Display the supporting information on the table by the projector. The projector reflects the image of the monitor. The projection range corresponds to the area of billiards table. The path of the ball which is estimated by computer is shown as a line on the table. This system assists beginner players to play the billiards by the procedure described above. They may be able to stroke a cue ball to their satisfaction with this system. By such a constitution, we enable visually to support the beginner players in real time. B. OpenCV This system performs the image processing using OpenCV (Open Source Computer Vision Library) which is the open source computer vision and machine learning software library[4]. The library has more than 2500 optimized algorithms. It supports various functions helping image processing. For example, edge detection, geometry conversion of the image, and so on. We adapt the function of the image processing for billiards to distinguish the balls on the table, detect the cue stick, and describe the path of the ball. Figure 1. Appearance of system Figure 2. Processing of system III. DETAILE OF THE STUDY A. Detection of the Cue Stick We need the direction of the cue stick to calculate the path of the ball. We have focused on shape of the cue stick. Both sides of the outline of the cue stick are formed by a straight line. Special Issue on ACEIAT & JTSTE - Thailand ISSN:

3 First of all, it is necessary to extract a contour of only cue stick in order to detect the straight lines of cue stick. Our system acquires a foreground image by the background subtraction technique. This difference is based on the brightness of the pixels in the image. The system applies the binarization processing to the foreground image to extract a clearer outline. Thereafter, it extracts the contour by the edge detection technique. Consequently, contours of the cue stick appeared in the image. Figure 3 shows the flow of the image processing before detecting the edge of the cue stick. Figure 3. Processing the image In the next stage, our system detects straight lines from the edge image. We have used the Hough transformation for the detection method of the straight line. The Hough transform is a technique which can be used to isolate features of a particular shape within an image. A principle of Hough transform in the straight line detection is to represent every point of the image in another parameter space and to check whether it constitutes a straight line. Our system has gotten the segments of line on both sides of the cue stick by using this technique. However, other straight lines exist as well as the straight lines of the cue stick if the method mentioned above detects all segmented lines from the ordinary image of playing billiard. When the system detects the straight lines from the image of Figure 4-(a), the outline of the arm and edges of the billiards table are extracted, as shown in Figure 4-(b). Thus, it is necessary to sort out the only straight lines that along the outline of the cue stick from the some straight lines in the image. We paid attention to the position of the cue ball and the relations with the straight line of the cue stick. Moreover, we thought that two straight lines passing through the inside of the cue ball were more likely the straight lines on both sides of the cue stick. For the reasons mentioned above, our system calculates the distance between each straight lines and the center of the cue ball and detects also only two straight lines which close each other. Our system is selected straight line on both sides of the cue stick by this way. The result of the process is shown in Figure 4-(c). After that, our system calculates the straight line passing the center of the both sides of the cue stick. It detects the direction of the cue stick by comparing the coordinates of both ends of the line segments and the coordinates of the center of the cue ball. One of the end points of the line segment, which is closer to the cue ball, is likely the tip of the cue stick. In addition, we assume that a player hits the cue ball at the center of cue ball. Our system has to derive its path along which it always seems to go. The direction of the cue stick is moved parallel to go through the center of the cue ball. Consequently our system detects the trajectory of the cue ball from the direction of the cue stick. Special Issue on ACEIAT & JTSTE - Thailand ISSN:

4 Figure 4. Result of the straight line detection B. Path Calculation Our system estimates the path of the cue ball from the vector calculated on the detection of the cue stick. It is necessary to calculate the path of some colored balls, when the cue ball contacts them. The path of the ball turns at the contact point to the cushion. Therefore we prepared two functions. One function works when the cue ball contacts with another colored ball. Our system calculates its trajectory before the player hits the cue ball. The other function works when the moving ball contacts with a cushion and reflects. Our system calculates its trajectory. These functions are graphically shown in Figure 5 and Figure 6[5] When the cue ball contacts another colored ball, the colored one will move along the impact line shown in Figure 5. The contact point of the cue ball and the colored one is searched when a colored ball exists around the trajectory of the cue ball. The imaginary ball is a circle touched the colored ball at the contact point. It is illustrated by the dotted circle in Figure 5. The impact line is calculated with the contact point by connecting the center of the imaginary ball and the center of the colored ball. Accordingly our system derives the imaginary ball, and calculates the trajectory of the colored ball. It calculated path of the cue ball with the direction of the cue stick, and derives the path of the other flicked balls in this function. Figure 8 illustrates the path of the cue ball that bounces off the cushion. It is called the bank-shot. Our system virtually draws the imaginary line at a distance of the radius of the ball from the cushion. It calculates the point where the imaginary trajectory crosses the imaginary line as the bank point. The rebound angle and the approach angle are equal in the case of no-rotating ball[6]. Finally, the system calculates the reflection vector. The system judges a contact with a cushion or a ball using the vector of movement of the cue ball or the colored ball hit by the cue ball. Finally, it is able to predict the paths of the cue ball and the colored one by above two functions mentioned above. Figure 7 shows the result of executing our system. Special Issue on ACEIAT & JTSTE - Thailand ISSN:

5 Figure 5. Contact with balls Figure 6. Path of the bank-shot Figure 7. Executing the system. IV. EVALUATION In this section, we describe the evaluation of detection of the cue stick. Our system uses the probabilistic Hough transformation, which is a function of OpenCV to detect line segments. We have evaluated the accuracy of detection of the cue stick with its width. We have measured the average, variance, and standard deviation of the width of the cue stick. Figure 8 shows the definition of the width of the cue stick. The P m is the midpoint of line segment P 1 P 2, which is detected by our system. The Q H is the point at which the perpendicular line intersects another straight line, which is detected by our system. We regard the distance between P m and Q H as the width of cue stick. The value of W is the width of cue stick in Figure 8. Figure 8. Definision of the width of the cue stick Measurement method has the following two situations. We captured 1,000 pieces of image in each situation and calculated the average, variance and standard deviation of the width of the cue stick. (1) We do not move a cue stick. (2) We rotate the cue stick around the cue ball. (Figure 9) Special Issue on ACEIAT & JTSTE - Thailand ISSN:

6 Table 1 shows the result of measurement. The unit of value is pixel. Figure 9. Measurement method situation (2) Table 1. Measurement Result of the width of the cue stick Average variance standard deviation situation (1) situation (2) The ratio between the standard deviation and the average value is approximately 20%. It means that the width of the cue stick is not stable. Accordingly this result shows that the detection of the line segment of the both sides of the cue stick is not so good if no improvement. We have investigated the images of the error detection. The main reason of error is that our method has detected different line segments on only one side of the cue stick. Figure 10 shows examples of the detected line segments in the error case. These examples have two line segments which is extract from the same edge of the cue stick. Therefore we have provided the threshold in order for our system to decide two straight lines as the both edge of the cue stick. The threshold has been set based on the result of the above measurement. When a width of two detected line segments exceeds the threshold, they are estimated as both sides of the cue stick. Table 2 shows the result of measurement in the case of using the threshold. The variance and standard deviation was consequently smaller. The ratio between the standard deviation and the average value is approximately 8.5%. In this way, it is possible to stably detect each side of the cue stick. The system extracts the only line segments of the both sides of cue stick from the plural straight lines in the image. Figure 10. False detection in line segment detection Table 2. Measurement Result of the width of the cue stick with threshold average variance standard deviation situation (1) situation (2) Special Issue on ACEIAT & JTSTE - Thailand ISSN:

7 Figure 11. Result of the straight line detection of the cue stick V.CONCLUSION In conclusion, the beginners may predict the movement of the balls by visual support of our system. This system calculates the path of each ball if the cue ball is hit by the cue stick reflecting in the image captured by the camera. Once a player replaces the cue stick near the cue ball on the billiards table, our system shows an appropriate path of the balls. The previous system can display only limited path in one distribution pattern. On the other hand, the new system we proposed in this paper can display various paths which a player wants to aim in one distribution pattern. Compared with previous one, the later one has improved flexibility. However, some faults occur as well as a good aspect. One of them occurs when the cue stick is moving slightly. The straight line detection is sensitive to the movement of the cue stick. In other words, the indication by a projector may flicker finally. Our system sometimes makes an error detection of the cue stick direction when the cue stick in the image captured by the camera is very short. If the detected line segment is very short, the derived direction has a computing error. Therefore the straight line is not detected near the cushion of the Billiard table. It is not also detected in a situation that the player s arm or body overlaps with the cue stick. Some future works are existed. We have a plan to make our detection method of the direction of the cue stick more exact and stable. The flicker indication is not so good psychically. Our system has been demanded to expressly display the information on the Billiard table. REFERENCES [1] Yusuke Ogata, Akira Suganuma, Atsushi Shimada, Daisaku Arita, and Rin-ichiro Taniguchi, "A Billiard Instruction System based on Mixed Reality Technique," Proc. of 14th Korea-Japan Joint Workshop on Frontiers of Computer Vision, (2008). [2] Akira Suganuma, Yusuke Ogata, Atsushi Shimada, Daisaku Arita and Rin-ichiro Taniguchi, "Billiard Instruction System for Beginners with a Projector-Camera System," Proc. of the International Conference on Advances in Computer Entertainment Technology, (2008). [3] Noriyuki Watanabe and Akira Suganuma, "Development of Billiards Support System for Beginners by Image Processing," Proc. of the International Technical Conference on Circuits/Systems, Computers and Communications, (2013). [4] "About OpenCV," [5] David G.Alciatore, The Illustrated Principles of Pool and Billiards, Sterling Publishing, (2004). [6] Jack H.Koehler, The Science of Pocket Billiards, Sportlogy Publications, (1995). Special Issue on ACEIAT & JTSTE - Thailand ISSN: