Pawel Cieslewski Robo-Retriever Formal Report

Transcription

1 Pawel Cieslewski Robo-Retriever Formal Report EEL 4665 Intelligent Machines Design Laboratory University of Florida Department of Electrical and Computer Engineering TAs: Andy Gray, Josh Weaver, Nick Cox Instructors: Dr. A. Antonio Arroyo Dr. Eric M. Schwartz

2 Table of Contents 3 - Abstract 4 - Executive Summary 5 - Introduction 6 - Integrated System 7 - Mobile Platform 8 - Actuation 9 - Sensors 10 - Behaviors 11 - Experimental Layout and Results 12 - Conclusion 13 - Documentation 14 - Appendices 2

3 3. Abstract Robo-Retriever is a robot inspired by the behavior of a Black Labrador named Jake. It is every exciting to play with this dog and it would be interesting to try to mimic this behavior in a robot. Robo-Retriver is a robot that does exactly this. The purpose of this robot is to try to play with a person by not only avoiding obstacles but by picking up tennis balls. Making use of various sensors and a camera is this possible. The robot is capable of using sonars to avoid hitting itself and looking at a video feed to identify tennis balls. Once the robot locks on to a ball, it grabs it, and returns it. 4. Executive Summary Robo-Retriever, more specifically, is an autonomous robot with several goals. These goals are to be able to prevent the robot from running into objects and to use image processing to identify objects in a picture. The Robo-Retriever uses sonars to judge the distance to the nearest object based on the time it takes for the sound to return to the robot after is has been emitted. The sonars must be controlled individually in order to prevent their sounds from interfering with each other. Infrared encoders on the wheels are also used to be able to judge the if the robot is traveling in the correct direction. The encoders work by counting the gaps in the inside of the wheel and toggling a digital value. An interrupt on the microcontroller counts these toggles in order to create feedback from motors that would otherwise not have any. This robot in particular is looking for tennis balls, due to their brightly colored nature. Using a library called Open Computer Vision, the robot is able to efficiently process an incoming video stream to judge if a tennis ball has actually been seen. To accomplish this the robot checks for several layers of criteria. First it thresholds the hue, saturation, and value of each frame. Using this threshold, it is able to see blobs. Filters are applied to these images to reduce noise and the only the largest blobs remain. We then check if the blobs are round enough to constitute a tennis ball. 5. Introduction While Robo Retriever s mission is not a serious one it is interesting and complex. This project attempts to mimic to behavior of a dog. The purpose of building this robot is to learn from it. The scope of this entire project is educational one. Each portion of Robo Retriever represents an interesting challenge to explore. The first major problem is mechanical assembly. The robot needed to be designed and manufactured. The second problem was electrical design. The systems of the robot needed to be able to function properly, each with their own operational demands being met. Aside for this, these components needed to communicate to create a coherent entity. This was extremely 3

4 challenging, not only from the point of design, but also organization. The other major problem is software. In order to make all of the mechanical and electrical systems to work, it is necessary to write software for the microcontroller and onboard computer to control all of these components. 6. Integrated System Robo-Retriever uses two main circuit boards for controlling the system. For data acquisition and low-level control a Arduino Uno R3 (ATmega16U2 Architecture) is used. This board was chosen because of the simplicity of use and large support. Since this board is natively programmed in C with convenient software, it is easier to focus on robot design and functionality. This board is used to control all of the sensors on board, the motor driver, and the servos. All traveling is handled directly by the Arduino. For higher level processing a Raspberry Pi board is used. Running a 700 MHz ARM1176JZF-S core CPU, the raspberry pi is able to handle signal processing type applications. This board is connected to the Raspberry Pi camera. The raspberry pi will use image processing to find objects and then relay this information to the Arduino. 4

5 7. Mobile Platform The mobile platform of the robot is used to house all of the electronics and protect the robot from harm. I was able to design a majority of my robot in solidworks and use hacked T-Tech to cut the platform for me. The T-Tech s real purpose was to etch PC boards. However, by pressing the drill through balsa wood, it turns out it can also manufacture robots. This is a picture of my Solidworks model for the robot itself. Comparing this model to the real version, they are very similar. I believe that using Solidworks to design the robot was a good choice and dramatically simplified the thinking and manufacturing. While the manufacturing of the model went very well, it cannot be said that all of my mechanical decisions were good. The robot ended up being slightly tilted because the caster wheels were slightly taller than the height that the wheels. Furthermore the planning of the mechanical gripping mechanism was nothing more than a disaster. It was very difficult for me to think of a proper way I could grip the tennis ball, so the mechanism I settled for was very simple and unreliable. 5

6 8. Actuation For actuation the robot uses two micro gear motors with a gear ratio of 150:1 pictured below. The reason that these motors were chosen is because they have a powerful amount of torque for a relatively low amount of current. They are rated at 45oz/in of torque with a stall current of only 1.6 amps and a no load current of 70ma. These motors are high quality and since the power consumption is low, the motor board does not have to be very expensive. The motor board to run these motors is a TB6612FNG Dual Motor Driver Carrier pictured below. This motor board is very inexpensive however the specs are well within the range. The driver is able to control two motors at the same time with a continuous current of 1A on each channel and a peak current of 3A. The motor driver is rated well above the motor rating so overheating or failure should not be a concern. To be able to drive the vehicle, the Pololu wheel and encoder set have been chosen. 6

7 The vehicle needs an encoder to be able to track the distance the robot has traveled but also to ensure that the robot is capable of driving in a straight line. The wheels measure 42x19mm with a thick rubber tire. The tires provide a firm grip of the floor to prevent the robot from slipping. 9. Sensors The sensors used on Robo-Retriever deal with obstacle avoidance and tennis-ball detection. The robot uses two Ultrasonic Range Finders. These range finders are used for object avoidance, especially on the edges of the robot. This model was chosen because it has a semi-wide beam angle and an analog output so data acquisition is very simple. It is important to note however that I realized late in the project that these sonars actually interfere with each other. I needed to reintegrate them with the system in order to prevent getting false values. The special sensor that this robot possesses is a video camera, specifically a Raspberry Pi Camera Module. This camera was chosen over a webcam because of its integration with the raspberry pi. The connection is integrated directly into the board rather than using a serial port. More importantly however, the GPU on board the raspberry pi assists with processing data. Since computing power is extremely important with this processor, being able to relieve the processor, even slightly, is important. 7

8 10. Behaviors Robo-Retriever has four primary behaviors. The first behavior is obstacle avoidance. During this behavior the microcontroller is heavily monitoring all sensors to prevent the robot from hitting anything. This is the only behavior in which the robot travels to different locations. During this behavior the robot also listens to the raspberry pi data checking if the camera happened to notice a tennis ball. The second behavior is rotation. This behavior is started after the robot has spent some time traveling around in the obstacle avoidance mode. In this mode the robot stops, and then begins to rotate in a circle. During his rotation he is spending a large amount of time making sure there is not a tennis ball in the frame. The third mode is adjustment. After a ball has been seen in any mode, the robot attempts to adjust. What is meant by this is it tries to center the ball in its field of vision. If the ball is centered within a certain threshold, it begins to inch forward, constant calling the adjust function. Once the ball has been found and its radius is a certain length, the robot will do its final approach. In this mode, the robot opens its arms, goes forward, and closes its arms around the tennis ball. 11. Experimental Layout and Results There were many challenges in implementing the special sensor. Although the Rasberry Pi Camera Module is a very high quality camera, it does have its drawbacks. Since the connection from the raspberry pi to the camera is proprietary, Open CV does not support it by default. However, I was able to implement a library that allowed the use of the camera and importing it into an Open CV format. Once this was working I was able to approach the problem very systematically. I needed to first threshold the image to find pixels that could be part of the tennis ball. I started off with a video feed such as the following. 8

9 I was then able to change the picture from a Red, Green, Blue color space to Hue, Saturation, Value space. This allowed me to filter the colors based on the hue. If I did not do this, shadows on the ball would be a lot more difficult to manage. After some calibration I was able to get the following image. However, I needed the ball to be one large mass so I would be able to apply some other filters on it. Occasionally, noise would enter the picture and cause speckles of white dots all around the screen. I was able to remove the noise by using OpenCV s built in erode and dilate. By doing this, I was able to filter the background and remove the letters from the ball itself. 9

10 Once this picture was attained, it was a matter of applying a circle finding algorithm to finish the job. I tried to use Hough Circles at first. I had a rather unsuccessful attempt at this. Using this algorithm I had many false positives that would destroy the functionality of my robot. At close distances to the robot, I would find many circles rather than just one. I decided to try a different approach. I took the binary representation of my image and first discarded all blobs that did not meet my size requirements. I did this to reduce noise. For example, all small dots were immediately removed. I also knew for a fact that the tennis ball would never be in point blank range so larger items were discarded as well. After this step I would draw a encompassing circle around the blob and I would measure the area of this circle. If the blob that was encompassed did not fill in a majority of the encompassing circle I knew that the object was not sudo-round. So objects such as squares would not trip the algorithm. This greatly reduced my false positive count. Actually it worked so well that I very rarely even encountered an instance where noise would create false tennis balls. My results are shown in the picture below. 10

11 The issues that arose however were difficult to fix. It was clear that the lighting conditions were extremely sensitive. Even as the time of the day changed I need to change my HSV filtering values. In order to solve this issue, I mounted a light source on top of the robot. It shines directly in front of the robot in order to illuminate the tennis balls and keep the color values constant. This helped greatly; however, it introduced a new problem. Since the light source was placed off center of the camera, the tennis ball would be illuminated slightly off center. This had a large impact on being able to center on the ball and I was forced to correct my centering algorithm. Once the special sensor was working testing the other functions of the robot consisted of tweaking values. I laid my robot on a box and observed the OpenCV values and adjusted the microcontroller behavior. CLOSING 12. Conclusion The work that I accomplished on this robot was very successful in my opinion. The purpose of the robot was to learn how to build a robot from the ground up. While all of my components were not perfect, they each taught valuable lessons. In the beginning I was not capable of using solidworks and after the mechanical design portion of this robot I can now create models that can work with each other in solidworks. I am able to materialize my thoughts in this software to greatly facilitate manufacturing. 11

12 The hardware portion of my robot taught me about integration of sensors and motors. It also made me familiarize myself with power distribution and providing working conditions for all my sensors. My favorite learning experience, however, was learning to use the camera on this robot. I never used image processing before and I am now able to apply elementary OpenCV algorithms to detect objects and colors. Of course it is important to note that I was very ambitious with regards to my initial intentions with the robot. I wanted this robot to be able to detect face and listen to sounds. Working with limited hardware and more importantly limited time put these objectives outside of the scope of the project. One of the things that I could have improved on is definitely behavior of the robot. One of the issues that I ran into with the microcontroller is that I was having difficulty parallelizing code without instantiating a lot of timers. Having the ability to use threads on an operating system makes this so simple, but on a microcontroller it is non-trivial. I wish that I spent more time programming with a clear organized end goal in mind. I feel that my code had a lot of hacked patches that would hinder development of this robot much further than his current state. Another thing that I would have liked to fix is adding a behavior to return the ball to the user somehow. I could have made a glove of a particular color for the robot to find but this would have been a simple extension of my current code. One of my initial idea was to try to use dead reckoning to return to my original position, but after seeing how much the robot slips, this seems impossible. For future students I would pass along a lot of information. One of the most important things to consider when doing image processing knowing that you have full control of your camera. One of the things that I noticed is that my camera would automatically change exposure settings which would ruin everything I attempted with the OpenCV code. Furthermore, if you choose to use a raspberry pi, it is important to know that the WiringPi library exists. It allows for communication with the GPIO pins and serial ports using a library written in C++. This is a bonus because most likely the code you will use for OpenCV will be written in C++ as well. If I was to start the project from the beginning, I would definitely pay more attention to the camera. I would make sure that I am getting consistent images that have the same exposure settings. I would also make sure that I think about OpenCV and coding the microcontroller a lot earlier. I would have preferred it if my functions were coordinated and thoughtful, rather than hundreds of lines shoved together to make a working product. 13. Documentation WiringPi Library for hardware interfacing with the Raspberry Pi in C

13 *It is important to note that libserial does not compile on the Raspberry Pi which makes this library even more useful. Robidouille s Library to use Raspberry Pi camera with the Raspberry Pi OpenCV Documentation for example code Grzegorz Cieslewski Computer Engineering Graduate from the University of Florida 13

14 14. Appendices Microcontroller: #include <Servo.h> #define FORWARD 0 #define BACKWARD 1 #define LEFT 2 #define RIGHT 3 int currentdir = 0; //Declares for motor direction switches. I am guessing H-Bridge. int ain1 = 52; int ain2 = 53; int bin1 = 50; int bin2 = 51; //Led on board. static const int led = 13; //PWM pins to control motor int motorpwm1 = 2; int motorpwm2 = 3; //Pins to read in Sonar Values int rightsonar = A0; int leftsonar = A1; int rightsonarval = 300; int leftsonarval = 300; int rightsonarstrobe = 41; int leftsonarstrobe = 39; //Values for OpenCV infro int ballradius = 0; int ballx = 0; int bally = 0; bool ballisleft = false; bool ballisright = false; bool balliscentered = false; bool seeball = false; bool inchforward = false; bool canbepicked = false; bool seeballduringavoid = false; bool haveball = false; bool raspionline = false; Servo leftarm; Servo rightarm; //Servo cameraservo; int cameraservopin = 6; int leftarmpin = 5; int rightarmpin = 4; int leftservpos = 10; int rightservpos = 45; //int cameraservpos = 135; //closed : left:10 right:45 //open: left:120 right:155 //First run? bool firstrun = true; void setup() { SerialUSB.begin(9600); Serial.begin(9600); 14

15 //Serial.begin(9600); pinmode(led, OUTPUT); pinmode(ain1, OUTPUT); pinmode(ain2, OUTPUT); pinmode(bin1, OUTPUT); pinmode(bin2, OUTPUT); pinmode(motorpwm1, OUTPUT); pinmode(motorpwm2, OUTPUT); int rightsonar = A0; int leftsonar = A1; pinmode(leftsonar, INPUT); pinmode(rightsonar, INPUT); pinmode(leftsonarstrobe, OUTPUT); pinmode(rightsonarstrobe, OUTPUT); setdirection(forward); analogwrite(motorpwm1, 0); analogwrite(motorpwm2, 0); servoinit(); closearms(); waitforpi(); setdirection(forward); void loop() { roamandlook(); if(!seeball){ rotate(); else{ SerialUSB.flush(); while(seeball){ seeball = false; ballisleft = false; ballisright = false; inchforward = false; canbepicked = false; balliscentered = false; for(int i=0; i<4; i++){ getballinfo(); if(seeball){ break; if(canbepicked){ openarms(); setdirection(forward); delay(6000); closeonball(); closearms(); seeball = false; ballisleft = false; ballisright = false; inchforward = false; canbepicked = false; 15

16 balliscentered = false; delay(1000); setdirection(right); delay(5000); delay(500); movearms(120,45); closearms(); break; adjust(); void analyze(){ digitalwrite(leftsonarstrobe, HIGH); delay(1); digitalwrite(leftsonarstrobe, LOW); delay(50); digitalwrite(rightsonarstrobe, HIGH); delay(1); digitalwrite(rightsonarstrobe, LOW); delay(50); leftsonarval = analogread(leftsonar); rightsonarval = analogread(rightsonar); //Serial.println(rightSonarVal); //delay(200); if( (leftsonarval < 25) && (currentdir == FORWARD) ){ //Serial.println(currentDir); delay(500); setdirection(right); delay(500); //This ensures that the vehicle turns at least some finite amount. else if( (rightsonarval < 25) && (currentdir == FORWARD) ){ //Serial.println("R"); delay(500); setdirection(left); delay(500); //This ensures that the vehicle turns at least some finite amount. else if( (rightsonarval >= 35) && (leftsonarval >= 35) && (currentdir!= FORWARD) ){ //Serial.println("F"); delay(250); setdirection(forward); void rotate(){ //This method spins the robot in a circle trying to find the tennis ball. delay(500); setdirection(left); for(int i = 0; i<14; i++){ delay(800); 16

17 for(int j = 0; j<2; j++) getballinfo(); if(seeball){ return; delay(1000); void analyzeandlook(){ digitalwrite(leftsonarstrobe, HIGH); delay(1); digitalwrite(leftsonarstrobe, LOW); delay(50); digitalwrite(rightsonarstrobe, HIGH); delay(1); digitalwrite(rightsonarstrobe, LOW); delay(50); leftsonarval = analogread(leftsonar); rightsonarval = analogread(rightsonar); //Serial.println(rightSonarVal); //delay(200); getballinfofast(); if( (leftsonarval < 15) && (currentdir == FORWARD) ){ //Serial.println(currentDir); delay(500); setdirection(right); delay(500); //This ensures that the vehicle turns at least some finite amount. else if( (rightsonarval < 15) && (currentdir == FORWARD) ){ //Serial.println("R"); delay(500); setdirection(left); delay(500); //This ensures that the vehicle turns at least some finite amount. else if( (rightsonarval >= 25) && (leftsonarval >= 25) && (currentdir!= FORWARD) ){ //Serial.println("F"); delay(250); setdirection(forward); 17

18 void setdirection(int dir){ currentdir = dir; if(dir==forward){ digitalwrite(ain1, HIGH); digitalwrite(ain2, LOW); digitalwrite(bin1, LOW); digitalwrite(bin2, HIGH); else if(dir==backward){ digitalwrite(ain1, LOW); digitalwrite(ain2, HIGH); digitalwrite(bin1, HIGH); digitalwrite(bin2, LOW); else if(dir==left){ digitalwrite(ain1, HIGH); digitalwrite(ain2, LOW); digitalwrite(bin1, HIGH); digitalwrite(bin2, LOW); else if(dir==right){ digitalwrite(ain1, LOW); digitalwrite(ain2, HIGH); digitalwrite(bin1, LOW); digitalwrite(bin2, HIGH); void go(){ analogwrite(motorpwm1, 75); analogwrite(motorpwm2, 82); void sto(){ analogwrite(motorpwm1, 0); analogwrite(motorpwm2, 0); void getballinfo(){ String recieved = ""; char temp = 'z'; SerialUSB.write('q'); delay(400); if(serialusb.available() > 0){ while(serialusb.available() > 0){ temp = SerialUSB.read(); if(temp == 'n'){ seeball = false; ballisleft = false; ballisright = false; inchforward = false; canbepicked = false; balliscentered = false; else if(temp == 'y'){ seeball = true; else if(temp == 'l'){ ballisleft = true; ballisright = false; balliscentered = false; else if(temp == 'r'){ ballisright = true; ballisleft = false; balliscentered = false; 18

19 else if(temp == 'f'){ inchforward = true; else if(temp == 'g'){ canbepicked = true; else if(temp == 'c'){ balliscentered = true; ballisright = false; ballisleft = false; return; else{ seeball = false; ballisleft = false; ballisright = false; inchforward = false; canbepicked = false; balliscentered = false; void adjust(){ delay(25); if(seeball){ if(balliscentered){ setdirection(forward); delay(200); return; else if(ballisleft){ setdirection(left); delay(100); return; else if(ballisright){ setdirection(right); delay(100); return; void waitforpi(){ char temp; while(1){ if(serialusb.available() > 0){ temp = SerialUSB.read(); if(temp == 'o'){ raspionline = true; return; 19

20 void servoinit(){ leftarm.attach(leftarmpin); rightarm.attach(rightarmpin); leftarm.write(leftservpos); rightarm.write(rightservpos); delay(1000); void openarms(){ while(leftservpos!= 120 rightservpos!= 155){ leftarm.write(leftservpos); rightarm.write(rightservpos); if(leftservpos < 120) leftservpos++; else if(leftservpos > 120) leftservpos--; if(rightservpos < 155) rightservpos++; else if(rightservpos > 155) rightservpos--; delay(20); void closearms(){ while(leftservpos!= 10 rightservpos!= 45){ leftarm.write(leftservpos); rightarm.write(rightservpos); if(leftservpos < 10) leftservpos++; else if(leftservpos > 10) leftservpos--; if(rightservpos < 45) rightservpos++; else if(rightservpos > 45) rightservpos--; delay(20); void movearms(int angleft, int angright){ while(leftservpos!= angleft rightservpos!= angright){ rightarm.write(rightservpos); leftarm.write(leftservpos); if(leftservpos < angleft) leftservpos++; else if(leftservpos > angleft) leftservpos--; if(rightservpos < angright) rightservpos++; else if(rightservpos > angright) rightservpos--; delay(15); void closeonball(){ movearms(80,115); movearms(80,60); movearms(25,45); void pickup(){ openarms(); delay(25); delay(1000); //Tweak for distance. closearms(); delay(1000); void roam(){ setdirection(forward); 20

21 unsigned long timeelapsed = 0; unsigned long startmillis = millis(); while(timeelapsed < 20000){ timeelapsed = (millis() - startmillis); analyze(); delay(500); void roamandlook(){ setdirection(forward); unsigned long timeelapsed = 0; unsigned long startmillis = millis(); while(timeelapsed < 20000){ timeelapsed = (millis() - startmillis); analyzeandlook(); if(seeball){ break; delay(500); bool checkifhaveball(){ delay(25); setdirection(backward); delay(1300); setdirection(forward); for(int i=0; i<2; i++){ getballinfo(); if(seeball){ return false; return true; void getballinfofast() String recieved = ""; char temp = 'z'; SerialUSB.write('q'); if(serialusb.available() > 0){ while(serialusb.available() > 0){ temp = SerialUSB.read(); if(temp == 'n'){ seeball = false; ballisleft = false; ballisright = false; inchforward = false; canbepicked = false; 21

22 balliscentered = false; else if(temp == 'y'){ seeball = true; else if(temp == 'l'){ ballisleft = true; ballisright = false; balliscentered = false; else if(temp == 'r'){ ballisright = true; ballisleft = false; balliscentered = false; else if(temp == 'f'){ inchforward = true; else if(temp == 'g'){ canbepicked = true; else if(temp == 'c'){ balliscentered = true; ballisright = false; ballisleft = false; return; else{ seeball = false; ballisleft = false; ballisright = false; inchforward = false; canbepicked = false; balliscentered = false; 22

23 OpenCV Code: #include <cv.h> #include <highgui.h> #include <iostream> //#include "RaspiCamCV.h" //#include <wiringserial.h> using namespace std; using namespace cv; //Tweak Values float fraction =.7; int radius_min = 20; IplImage* GetThresholdedImage(IplImage* img); void findthecontours(iplimage* bin); int ballx = 0; int bally = 0; int ballradius = 0; bool seeball = false; String ballxs; String ballys; String ballradiuss; String convertint(int number){ stringstream ss; ss << number; return ss.str(); IplImage* image; IplImage* imagethreshed; IplImage* imghsv; int main(){ /* int fd; if( (fd = serialopen("/dev/ttyacm0", 9600)) < 0 ){ cout << "Unable to open serial communication." << endl; */ //RaspiCamCvCapture* capture = raspicamcvcreatecameracapture(0); CvCapture* capture = cvcapturefromcam( 0 ); cvnamedwindow("camtest", 1); //I cant get rid of this line? image = cvqueryframe( capture ); //serialputchar(fd, 'o'); //cout << "Raspi Online. Sending o." << endl; do { image = cvqueryframe( capture ); //image = raspicamcvqueryframe(capture); imagethreshed = GetThresholdedImage(image); cverode( imagethreshed, imagethreshed, 0, 1 ); cvdilate( imagethreshed, imagethreshed, 0,5 ); cvshowimage("camtest", imagethreshed); 23

24 findthecontours(imagethreshed); cvreleaseimage(&imagethreshed); ballxs = convertint(ballx); ballys = convertint(bally); ballradiuss = convertint(ballradius); //cout << ballxs << endl; /* serialputchar(fd, 'r'); serialputs(fd, ballradiuss.c_str()); serialputchar(fd, 'e'); serialputchar(fd, 'x'); serialputs(fd, ballxs.c_str()); serialputchar(fd, 'e'); serialputchar(fd, 'y'); serialputs(fd, ballys.c_str()); serialputchar(fd, 'e'); */ while (cvwaitkey(10) < 0); cvdestroywindow("camtest"); cvreleasecapture(&capture); //raspicamcvreleasecapture(&capture); return 0; //This method needs to be changed based on light conditions. IplImage* GetThresholdedImage(IplImage* img) { // Convert the image into an HSV image IplImage* imghsv = cvcreateimage(cvgetsize(img), 8, 3); cvcvtcolor(img, imghsv, CV_BGR2HSV); //Create another image to store the threshed image IplImage* imgthreshed = cvcreateimage(cvgetsize(img), 8, 1); //thresh cvinranges(imghsv, cvscalar(18, 90, 90), cvscalar(50, 255, 255), imgthreshed); //free mem cvreleaseimage(&imghsv); return imgthreshed; void findthecontours(iplimage* threshed) { //data structures vector<vector<point> > contours; vector<vec4i> hierarchy; //change iplimage into a new Mat image Mat threshold_output(threshed); //find the contours findcontours( threshold_output, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, Point(0, 0) ); //data structures vector<vector<point> > contours_poly2( 1 ); Point2f center2; float radius2; 24

25 //find the largest contour float largest_area = 0; int largest_contour_index = 0; //int next_largest_index = 0; for( int i = 0; i< contours.size(); i++ ) { float a = contourarea(contours[i],false); if(a>largest_area) { largest_area = a; largest_contour_index=i; Mat drawing = Mat::zeros( threshold_output.size(), CV_8UC3 ); //So if we have any contours if(contours.size()!= 0) { //Draw a circle around the largest one. approxpolydp( Mat(contours[largest_contour_index]), contours_poly2[0], 3, true ); minenclosingcircle( (Mat)contours_poly2[0], center2, radius2 ); //if the tennisball is approx round and has a radius of at least this... THen update the values if( (largest_area > ((fraction)*(3.14)*((int)radius2)*((int)radius2))) && ((int)radius2 > radius_min) ){ ballradius = (int)radius2; ballx = center2.x; bally = center2.y; seeball = true; else{ Scalar color = Scalar( 0, 0, 255 ); circle( drawing, center2, (int)radius2, color, 2, 8, 0 ); ballradius = 0; ballx = 0; bally = 0; seeball = false; cout << "Radius: " << ballradius << endl; cout << "Center X: " << ballx << endl; cout << "Center Y: " << bally << endl; //Draw on window if we need it. For testing //Scalar color = Scalar( 0, 0, 255 ); //circle( drawing, center2, (int)radius2, color, 2, 8, 0 ); /// Show in a window namedwindow( "Contours", CV_WINDOW_AUTOSIZE ); imshow( "Contours", drawing ); 25