Camera kinetic model applied to crop row tracking and weed detection

Transcription

1 Camera kinetic model applied to crop row tracking and weed detection Gonzalo Ruiz-Ruiz *, Luis M. Navas-Gracia, Jaime Gómez-Gil 2 Department of Agricultural and Forest Engineering, University of Valladolid, Avda. Madrid 44, Campus La Yutera, Palencia, E34004, SPAIN 2 Department of Communications and Signal Theory and Telematics, University of Valladolid, Cmno. del Cementerio, Campus Miguel Delibes, Valladolid, E470, SPAIN *Corresponding author. gruiz@iaf.uva.es Abstract Computer vision can be used within precision farming to carry out crop row tracking and weed detection, looking for objectives such as vehicle steering and selective weed removing. Previous work used static parametric search areas to look for similar crop lines between consecutive frames in a sequence. This new work improves that method by introducing a camera kinetic model which generates dynamic search areas, based on configurable parameters of the vehicle which mounts the camera/s. These new search areas are much more realistic than static areas, therefore crop row detection and tracking are improved because of the efficient utilization of processing resources. Key words: Row tracking, weed detection, image analysis, kinetic model, precision farming.. Introduction Computer vision is one of the tools which are used in precision farming: by processing digital images and analysing the information extracted from them, farm tasks can be made more efficiently and product quality can be improved. In most of the cases, computer vision has been used for remote sensing with cameras mounted on aerial platforms: satellites, aircrafts or observation balloons. When cameras are mounted on terrestrial vehicles such as tractors, instead of aerial platforms, information extracted from images can also be applied in real time. Two objectives of precision farming involving computer vision are crop row tracking and weed detection. First one can be employed in tractors steering, either automatic or assisted. Guidance of vehicles is usually performed by satellite-based systems, mainly GPS (Global Positioning System), which offer a very good precision using additional technologies such as RTK (Real Time Kinematic), but sometimes it is necessary to guide vehicles detecting local objects, such as crop plants themselves. Detecting the distribution of plants by image analysis makes possible to improve the satellite-based guidance or even to create an autonomous guidance system. (Marchant & Brivot, 995; Marchant, 996; Southall, et al., 998; Astrand & Baerveldt, 2002; Leemans & Destain, 2006; Leemans & Destain, 2007). Crop row tracking can also be used for weed detection, previously detecting crop plants inside rows. Finally, the last objective of automatic weed detection consists of removing weeds in a selective way, using chemical or mechanical methods. This work presents an advance in previous algorithms for row tracking and weed detection by introducing a kinetic model of cameras mounted in any type of terrestrial vehicle. The objective of this model consists of improving the efficiency and performance of sequence correlation developed in previous work. In that work, a line searching method was based on patterns with static parameters instead of the dynamic parameters of this new work.

2 2. Materials and methods 2.. Row tracking and sequence correlation Previous work consisted of tracking crop rows detecting individual plants (Ruiz-Ruiz et al., 200a) and correlating sequence frames (Ruiz-Ruiz et al., 200b). The algorithms followed the diagram shown in Fig., and were applied to images of crops sowed in lines, such as peas, sunflowers, maize and sugar beet. Images were acquired using an automatic system mounted in the shovel of a tractor. The novel algorithm for crop rows tracking had two parts. In the first step, potential crop rows were detected for each individual image whereas, in the second step, lines from consecutive frames were filtered and correlated to improve crop row detection. FIGURE : General diagram of the algorithms for crop row tracking and sequence frames correlation (left) in images corresponding to linear crops, peas (a,b) and sunflowers (c,d). The correlation of detected lines in consecutive frames is based on the fact that a certain region of soil appears in these consecutive images and all of them represent the same crop rows. Analyzing the persistence of detected lines in consecutive images, the rest which do not appear in continuous way can be removed as detection noise, as shown in Fig. 2.

3 FIGURE 2: Three consecutive frames in a sequence. Non-vertical detected line in middle frame (b) is removed since it not detected in previous (a) and following (c) frames. Inter-image lines correlation is based on search areas, which are regions in the image with a specific geometry around well detected lines (Fig. 3). For a given line with parameters (θ 0, ρ 0 ) (), the search area is made by rotating the line between two limits [θ 0 α, θ 0 + α] and shifting horizontally these limit lines (±d). In this case, the rotation point p 0 = (x 0, y 0 ) is the middle point of each line in the y-axis, since crop rows should appear following y-axis and similar directions. Thus, a search area is defined by a central line (θ 0, ρ 0 ) and two configurable parameters, α and d (Fig. 3). x cos y sin () FIGURE 3: Parametric search areas for row correlation between consecutive frames in a sequence Given a new line (θ, ρ ), it is considered similar to (θ 0, ρ 0 ) if the first one belongs to the search area of the second one. First, to fulfil this condition, θ must be between θ 0 α and θ 0 + α. Once it is true, ρ must be between ρ A and ρ B, that is, ρ values for two lines which go through p A and p B points (Fig. 3) and which angle is θ. Both conditions are defined in (2). (θ, ρ ) belongs to (θ 0, ρ 0 ) search area if x cos y sin x cos y sin A A 0 0 B B (2)

4 A search area is assigned to each well detected crop row. These search areas will be used to look for the same crop rows in the following frames Camera kinetic model The advance of the present work consists of limiting and determining the camera position between consecutive frames in a sequence, so the parameters of the search area (Fig. 3) can be defined dynamically. The camera movement model is obtained from the movement model of the farm vehicle in which the camera is mounted. In this case the model represents a typical farming tractor with front steering wheels. Table list the parameters relating to the vehicle, including both static and dynamic parameters. TABLE : Kinetic model parameters and units Static Vehicle parameters Dynamic Crop parameters Camera parameters - Front axle width (m) - Rear axle width (m) - Distance between axles (m) - Steering angle (rad) - Driving wheels speed (m/s) - Number of lines - Gap between lines (m) - Lines angle (rad) - Relative X camera position (m) - Relative Y camera position (m) - Rotation angle (rad) - Frame area width (m) - Frame area height (m) Also some parameters describe crop lines: number of represented lines, distance between them and angle to a general coordinate system (Table ). Finally, the camera position is described by the distance to the middle point of the front axle of the vehicle in order to transfer the tractor movements to the camera. The camera description is completed with the frame area, considering a rectangular frame area (Table ). From a known point, the vehicle movement is computed for short time increments. Then the position of the camera is computed by moving its position from the reference point of the vehicle, in this case the middle point of the front axle. If necessary, the model allows the camera to be rotated. Finally, the camera position and orientation is moved to the ground through the field of view of the camera, using two parameters: distance between optics and ground and view angle in relation to ground view. The camera model is placed on top of a static model of an in-line crop (Fig. 4a). Changing to the camera view (Fig. 4b), it is possible to store the position of the model crop lines along the time. These lines make an area representing the searching areas for the tracking algorithm, so the shape and other parameters of these areas can be easily computed.

5 (a) FIGURE 4: (a) Camera and crop row models in a general coordinate system. (b) Crop row model in the camera coordinate system (black-dashed line) and lines from previous frames for one row (red-continuous line). (b) 3. Results and conclusion Several tests were performed in some types of crops, sunflower, sugar beet and maize. The results were satisfactory, improving the rate of well detected crop rows related to previous work using static search areas. In this way, the correlation between lines of consecutive frames of an image sequence adapts to the specific conditions at any time, such as position, steering angle and speed. Also, the computing time and resources are optimised because the searching areas are not overestimated or underestimated. Moreover, realistic search areas have not a sandglass (crossed trapezium) shape, as supposed in previous work, but a simple trapezium shape. Acknowledgements Thanks to Grants of the University of Valladolid for PhD studies, to Consejería de Educación of the Junta de Castilla y León for the support in projects VA064A08 and VA008B08, and to Escuela de Capacitación Agraria Viñalta in Palencia, Spain. References Astrand, B., & Baerveldt, A. J. (2002). An Agricultural Mobile Robot with Vision-Based Perception for Mechanical Weed Control, Autonomous Robots. 3, Leemans, V., & Destain, M. (2007). A computer-vision based precision seed drill guidance assistance, Computers and Electronics in Agriculture,. 59, -2. Leemans, V., & Destain, M. (2006). Application of the Hough Transform for Seed Row Localisation using Machine Vision, Biosystems Engineering. 94, Marchant, J. A., (996). Tracking of row structure in three crops using image analysis, Computers and Electronics in Agriculture. 5, Marchant, J., & Brivot, R., (995). Real-Time Tracking of Plant Rows Using a Hough Transform, Real-Time Imaging., Ruiz-Ruiz, G., Gómez-Gil, J., Navas Gracia, L.M., & Silva Junior, M.C., (200a). Crop row tracking by detecting individual plants using computer vision to guide farming vehicles.

6 Proceedings of XVII World Congress of the International Commission of Agricultural and Biosystems Engineering (CIGR200), Quebec City, Canada, June 3-7. Ruiz-Ruiz, G., Navas-Gracia, L.M., Gómez-Gil, J., & Silva Junior, M.C., (200b). Sequence Correlation for Row Tracking and Weed Detection using Computer Vision. Proceedings of the II CIGR Workshop on Image Analysis in Agriculture, August, 200, Budapest, Hungary. Southall, B., Marchant, J., Hague, T., & Buxton, B., (998). Model based tracking for navigation and segmentation. Computer Vision - ECCV'98, Springer Berlin - Heidelberg, pp