CONCEPTUAL CONTROL DESIGN FOR HARVESTER ROBOT

Transcription

1 CONCEPTUAL CONTROL DESIGN FOR HARVESTER ROBOT Wan Ishak Wan Ismail, a, b, Mohd. Hudzari Razali, a a Department of Biological and Agriculture Engineering, Faculty of Engineering b Intelligent System and Robotics Laboratory, Institute of Advanced Technology Universiti Putra Malaysia, UPM Serdang, Selangor, Malaysia wiwi@putra.upm.edu.my, hud47@hotmail.com ABSTRACT The previous development of the robot eye system for agriculture arm machine was successfully designed and fabricated to predict the actual distance of the object target. Stereo pair of Videogrammetry technique and triangulation was used to determine the distance measurement of object target. By a click on the images displayed on developed graphical user interface created by Visual Basic V6, the three-dimensional target distance was generated and robot arm able to move and grab the selected target automatically. The enhancement done with the application of camera vision system for recognizing the matured cocoa fruit in the field. The training image/ benchmark concept was applied into developed graphical user interface software. The user only click for make the training image/ benchmark data, then the threshold of the target image produced and the software decided for image recognition. The user needs to update the training image data by click to training image due to sunlight variances. This system was theoretically successful for color-based recognitions application in real plantation of agriculture product. Keywords: Control, Design, Harvester, Robot. 1. INTRODUCTION The previous development of the robot eye system for agriculture arm machine was successfully designed and fabricated by Hudzari (2002). The concept of robot eye is to predict the actual distance of the object target. Stereo pair of Videogrammetry technique and triangulation was used to determine the distance measurement of object target. By a click on the image of displayed developed graphical user interface as shown in Figure 1, the three-dimensional target distance was generated and robot arm able to move and grab the selected target automatically. The image matching eliminates the sunlight-changing problem by given the reference color of the target. The system was compared the real time reference color appear on image using clicked into the image required on left window then the threshold image produced on right window as target image. Ishak and Ismail (2003) mentioned that after the target image was defined, the computer determined the location of target by using videogrammetry method and the 3-dimensional measurement was generated which refer from calculated robot base datum through target location. The D-H kinematic formula was use to manipulate the distance of target and the mathematical model was developed to define the suitable angle of joint and make the robot end effectors reach the targeted. The mathematical model which developed by Omrane (1999) was based on Denavit-Hertenberg kinematics solutions for determine the joint relationship of robot arm for producing the robot movement with real time simulation. Two digital cameras used and fixed at parallel stage for applying the Videogrammetry concept of coordinate detection. The user need to key in the computer for sending the signal to pneumatic solenoid valves of robot through controller card and produce proper output to locate robot arm. The actual movement of robot was determined before fabrication process for safety and cost reduction by applying simulation process. Sapuan et al. (2007) also was mentioned the simulated process was useful in design and study for the correct fabrication and accuracy of the final product. The main objective of the project is to Design and Development of Automated Harvester System for Cocoa Pod Plantation. There are the sub-objectives that relate to achieve the main objective: - 1. To design and develop outdoor machine vision system. 2. To design and develop the intelligent robot arm harvester 3. To apply D-H Kinematic solution to actuate the robot arm 4. To apply the videogrammetry system for positioning of robot arm. This paper describes the development of the robot eye system for cocoa harvester robot that can recognize the target fruit on real plantation environment and can predict the actual distance of the object target. Stereo pair of Videogrammetry technique and triangulation was used to determine the distance measurement of object target. By a click on the image of displayed developed graphical user interface created by Visual Basic Version 6, the three-dimensional target distance was generated and robot arm able to move and grab the selected target automatically. 14

2 Figure 1: The Complete User Interface Software 2. METHODOLOGY The image matching eliminates the sunlight changing problem by given the reference color of the target. The system was compared the real time reference color appeared on image using clicked into the image required on left window then the threshold image produced on right window as target image (Figure 2). After the target image was defined, the computer automatically determine the location of target by using videogrammetry method and the 3-dimensional measurement was generated which refer from calculated robot base datum through target location (Figure 3). The D-H kinematic formula was used to manipulate the distance of target and the mathematical model was developed to define the suitable angle of joint to make the robot end effectors reach the target (Figure 4). The system consists the integration system, structure of robot arm, and the communication system. 2.1 Integration System All developed application programs like development of robot eye machine vision, videogrammetry method and D-H kinematic was used, download and store in the computer. Whenever the robot was operating, the machine vision system was recognized the matured fruit using the training image concept, the FFB coordinate was determined using videogrammetry concept and the movement requires for robot arm by dynamic calculation method Robot Eye Machine Vision This system was developing using visual basic V6 programming language that enhanced by API (application programming interface) that store in window application for color recognition method. The file called Dynamic Link library (DLL) is used and function to extract the pixel value of Red, Green and Blue color component on image and manipulate to recognize the matured FFB that refer to the training image as a benchmark. These codes below are functions that were automatically do RGB color extraction on every point clicked: Dim R as Byte, G as Byte, B as Byte Dim Color as Long Color = PictureBox.Point(0, 0) R = ExtractR(Color) G = ExtractG(Color) B = ExtractB(Color) This extraction value of RGB is used to store and set as reference value of color target. To update the image, the refresh function is used and delayed within range 3 second. After that, the image threshold was show the area and position of the target image (fruit) was defined on real time mode. The user needs to update the reference color for fruit detection because of the changing of light intensity in outdoors environment. This is disadvantages of the system and further explains how to solve will continue on discussion session below. The machine vision system can also predict the actual distance of the object target. Stereo pair of Videogrammetry technique and triangulation was used to determine the distance measurement of object target. Figure 3 shows the technique of triangulation for target detection. 15

3 . Figure 2: Development of Graphical User Interface for Outdoor Vision (captured during run time) Initial Point, (0,0,0) X Z Actual Target Position (X,Y,Z) Target Positioning on Stereo Image Xa Ya Y Xb Yb f Y B Figure 3: Triangulation Technique for Distance Detection 16

4 After image recognition process was finished, the threedimensional target distance was generated and robot arm able to move and grab the selected target automatically. Two digital cameras used and fixed at parallel stage for applying the Videogrammetry concept of coordinate detection Two rectangular show the image area on computer that had been taken by digital camera and the focal Len is f, so the coordinates for axis: - X = BXa / (Xa Xb) Y = Bf / (Xa - Xb) Z = BYa / (Xa Xb) Mathematical Model The development of mathematical model for robot arm was based on Denavit-Hertenberg kinematics solutions that used to develop joint relationship of robot arm for producing the robot movement with real time simulation. Using Inverse Kinematics model, the information of length on each robot link and the position of target point are known (from videogrammetry method) than the appropriate angles of each joint needed to obtain that position was found. The length on each robot link measured during fabrication stage while the target positions were get using Videogrammetry method. The machine vision system was triggered the computer for sending the signal to pneumatic solenoid valves of robot through controller card and produce proper output to locate robot arm. In D-H representation each coordinates frame determined and established on the basis of three rules as shown in Figure 4: 1. The Z i-1 axis lies along the axis of motion of the i th joint. 2. The X i axis is normal to the Z i-1 axis, and pointing away from it. 3. The Yi axis completes the right-handed coordinates system as required. The D-H representation depended on four geometric parameters associated with each link as shown in Figure 5. i : is the joint angle from x i-1 axis to the x i axis about z i-1 axis using the right hand rule. d i : is the distance from the origin of the (i-1)th coordinate frame to the The relationship between two adjacent links is given by: A (i-1,i) = T(z, d i )*T(z, i )*T(x, a i )*T(x, i ) and in form of matrix as; C i -C i S i S i S i a i C i z i-1 y i i z i S i C i C i -S i C i a i S i 0 S i C i d i Link i a i x i Related to Adjacent Links d i y i-1 Z 3 X 2 Z 2 i L 2 L 1 Y 2 Y 3 X 3 Link i-1 x i-1 X 1 Z 0 L 0 Figure 4: D-H Coordinate Frame Parameters Z 1 Y 1 X 0 Y 0 Figure 5: D-H Coordinate frame Parameters Related to Adjacent Link 17

5 2.2 Communication System The communication system was send and receive the signal from camera (for image recognition), computer (for decision making) and mechanical harvester system (final movement and output feedback). Figure 6 show the illustrated of basic communication system. 3. ROBOT HARDWARE The robot was designed to cut the frond of cocoa fruit and then grab it before unloading process. There are 4 pneumatic cylinders for linear movement and 1 pneumatic motor for robot turning. It s also including the solenoid valve, air pressure pump, pneumatic wiring and personal computer with digital camera. The computer was triggered the input/output card by 5 volts power and then this card was pass through 24 volts from power source to trigger solenoid valve to release air pressure for robot movements. Refer Figure 7 show the diagram and picture for complete robot system. 4. RESULTS AND DISCUSSION The software was able to recognize color of the cocoa pod and the location is defined location using the triangulation method and simulate the robot movements. The user needs to update the reference color for fruit detection because of the changing of light intensity in outdoors environment. From the study review, this is possible to model the outdoor condition system to control the image color of the target. Shashi (2002) was mentioned that under outdoor conditions, a surface's apparent color is a function of the color of the incident daylight, the surface reflectance and surface orientation, among several other factors. The trial outdoors image modeling was initially collected and shows that there is the linear changing on the pixel value of component RGB versus time (light intensity) as shown in Figure 8. Table 1 was stated the results of the application for the D-H approach which shown in diagram of Figure 5. Table 1: Results of the D-H Parameters No. of link i a i d i i l < 1 < l < 2 < l < 3 < The conceptual design model was theoretically capable to control the system which always can determine the target images instead of light changing. The light intensity changing was measure using light meter (flux unit) and the fruit images were captured synchronously to relate and model the relationship within these factors. For working in 24 hours harvester operation, the artificial lighting with shading system was conceptually can to be apply not only for working in night shift but to control illumination supply for target detection. Otherwise the light is main tools for sighting, even for human being, without light we can t see anymore as mentioned by Allah in Al-Quran sura Ash-Shams, verse 3-4 By the day as it shows up (the sun's) brightness, By the night as it conceals it (the sun) (al-quran, 2009). Figure 8: Developed Software for Color Modeling Analysis 5. CONCLUSIONS The development of the conceptual outdoor vision system for agriculture was achieved. The developed software can recognize the cocoa fruit on any outdoor condition after appropriate training image is given. The system was able to retrieve real time dynamic video scene and from there the 3 coordinate axes of object targets were generated using manipulation of image threshold by color extraction. The coordinate was calculated using triangulation principle based on the video scene of two different locations of cameras. These 3-axis coordinates were measured from the Cartesian robot coordinate that was taken as reference point to calculate a mathematical model for robot simulation and kinematics. The workspace in the simulation software also was calibrated to be same with the real robot workspace. During simulation process, the embedded programmed will produce electrical signal to trigger the solenoid valve to move the robot to the selected targets. 18

6 Figure 6: Flow Chart for Basic Communication System. Figure 7: The Complete Diagram and Picture of Cocoa Pod Harvester 19

7 Acknowledgement The author would like to thank the Malaysian Organization of Science, Technology, Innovation (MOSTI) for providing the funding for Design And Development Of Intelligent Machine Vision System for Agriculture (Oil Palm) and also for sponsoring other research. The author is grateful for contributions from all the research group members, especially Associate Professor Dr. Abd. Rahman Ramli and Associate Professor Dr. Md. Nasir Sulaiman as well as other colleagues from ITMA and UPM. Finally, the author wishes to give great thanks to Allah S.W.T which provided health and strength. REFERENCES Mohd. Hudzari Razali, Videogrammetry Technique For Arm Positioning of Bio-Production Harvester, MSc Thesis, Universiti Putra Malaysia Bouketir Omrane, Vision System Interfacing Three DOF Agriculture Robot. MSc Thesis, University Putra Malaysia S.M. Sapuan, S.S.S. Imihezri, S. Sulaiman, M. Hamdan and E.S. Zainudin, Comparison Of Simulated And Actual Product Of Polymer Composite Automotive Clutch Pedal, International Journal Of Mechanical And Materials Engineering (IJMME), Vol. 2, No. 1, Shashi D. Buluswar, Color-Based Models For Outdoor Machine Vision, Phd Thesis, The University Of Massachusetts Amherst Al-Quran Al-Karim, Accessed on 15 November Wan Ishak Wan Ismail, Development of Vision System For Mechanical Harvester Oil Palm FFB. PIPOC Ogos Proceedings of Agricultural Conference PIPOC2003 Palm Oil: The Power House for the Global Oils, Fats and Economy, MPOB. 20