Glas robotprojecten + (PicknPack, Phenobot, Trimbot & Sweeper) Erik Pekkeriet

Transcription

1 1

2 Glas robotprojecten + (PicknPack, Phenobot, Trimbot & Sweeper) Erik Pekkeriet

3 PicknPack3

4 4

5 Globale strategie Yield & quality Stress Gezonde planten Oogst prognose Groei snelheid(homogeneity) Rijheid Onkruid (management) Geometrische kenmerken(2d/3d) Gestresste planten Photosynthese (CF) Geometrische kenmerken Spectral information Welke maatregelen neem je als je stress vindt? Disease Zieke planten Spectrale analyse & patroon herkenning Volg ontwikkelingen/maatregelen in de tijd 5

6 Phenobot6

7 SpySee (EU project Spicy) 4* IR, Colour, Range (ToF) cameras

8 3D Light Field Camera Technology* 3D reconstruction and extended depth-of-field based on only one snapshot and a single-lens camera - calibration-free monocular camera - robust & space-saving setup - down to micron resolution - extended depth-of-field by software re-focus - captures fast moving objects by single shot Light field engine 4D light field raw image data Micro-lens array (MLA) optics perspectiv e shift 3D 3D depth map in total focus 3D view re-focus plane on eye re-focus plane on nose US-Pat.-No.: 2012/ A1, CHIP-Award 2012: Innovation of the year Copyright 2013 by Raytrix GmbH, Germany. All rights reserved. Design, features, and specifications are subject to change without notice.

9 Robot Trolley Adapted spraying robot from Berg Hortimotive

10 Ziekte en plagen detectie

11 PhenoBot Greenhouse measurements

12 PhenoBot Greenhouse measurements

13 PhenoBot Greenhouse measurements Example data

14 PhenoBot Greenhouse measurements Find red tomatoes based on colour only

15

16 Individual tomato detection watershed transform and sobel edge detection are used

17 Image stitching Harris points detector is used to recognise keypoints in the images RANSAC algorithm is used to select the best point matchings between the images An accurate pre- segmentation of the tomato clusters is the key to the proper functionality of RANSAC algorithm

18 Hyperspectral imaging Image/Features Spectral image of four roses. PCA analysis from image to features

19 Hyperspectral imaging Image/Features Spectral image of four roses. PCA analysis from features to image

20 Mapping back to pixels: concentration images (chemical imaging) G. Polder, G.W.A.M. van der Heijden, and I.T. Young. Tomato sorting using independent component analysis on spectral images. Real-Time Imaging, 9(4): , 2003.

21 Ripening of tomatoes 1 0 Scatter plot of feature analysis of the RGB and spectral images. Classes 1-5 represent the ripeness stages of a tomato during the five days after day 1 day 2 day 3 day 4 day G. Polder, G.W.A.M. van der Heijden, and I.T. Young. Spectral image analysis for measuring ripeness of tomatoes. Transactions of the ASAE, 45(4): , harvest day 1 day 2 day 3 day 4 day 5

22 Rob2Pheno Phenotypic trait Feature (Possible method, to be discussed) Breeders' Priority Average Imaging Experts' Challenge Average Delivery Date (M) Juvenile stage First growth plant length cm from base M12 leaf area cm² from base without stems first truss of flowers mm from base treshold blob yellow color scale second truss of flowers mm from base treshold blob yellow color scale (template) number of flowers yet undefined Mature stage Full tomato plant full length (top to ripe truss) cm top to ripe truss M12 tortuosity/curvature max span width or radius (?) number of trusses and leaves number from first flower to ripe tomato truss thickness of the stem mm diametr in predefined points Top of the tomato plant shape of the head match of predefined templates thickness of the head mm on predefined point in the head M12 size of the head mm² from predefined point angle of the head yet undefined size of the flowertruss stem yet undefined size of the flowerbuds yet undefined Tomato ripening area * Stem and leaves internode length mm between leaves and trusses leaf angle maximum angel between stem and first x mm of leaf * Bottom tomato truss number of fruits in a truss Number of spheres M12 size and weight of fruits in a truss Size of sphere x specific gravity ripening of fruits in a truss color scales beginning and end of truss color of the ripe tomatoes color scale of first tomatoes at the point of harvest Thickness of the stem of the truss predefined point x mm from stem truss angle on the stem maximum angel between stem and first x mm of truss Breeders' Priority Imaging Experts' Challenge

23 Trimbot 24

24 Trimbot2020

25 26

26 27

27 28

28 29

29 Resutaten Met lidar en kleurencamera omgeving 3D in kaart brengen: lukt Leren naar plantsoorten en elementen lukt, maar niet gevalideerd op andere tuinen Robot succesvol getest, maar nog niet in combinatie met vision 30

30 Sweeper 31

31 Overview EU project Sweeper Jos Balendonck, Jochen Hemming Business Unit Greenhouse Horticulture of Wageningen University & Research Centre The Netherlands

32 CROPS: FP7 EU project within Theme NMP: Nanotechnologies, Materials and new Production Technologies Call: Automation and robotics for sustainable crop and forestry management Start date: Oct. 1st 2010 End date: Sept. 30th 2014 Universal modular robot platform for different applications

33 Harvesting sweet-pepper Unstructured Delicate Limited space Occlusion Fruit clustering Economic feasible Sensing Target and non-target Ripeness and quality 3d localization

34 Oogst robots - fundamentals

35 Sweet-pepper robot Follow-up project of CROPS ( ) Horizon 2020 ICT use case project 6 partners from 4 countries (The Netherlands, Belgium, Sweden and Israel). The project focusses on technology transfer rather than R&D Science + Industry + End Users Expert Team + Exploitation Committee + Growers Advisory Board Greenhouse experiments

36 Modules Patented end-effector concept End-effector (grasp, cut) + Camera (location, distance and ripeness detection) + Illumination (better detection under alternating conditions) Deep-learning (to avoid obstacles) Robot arm (to search, move to fruit, and convey fruit) Platform (to move robot in the greenhouse) Logistics (to convey picked fruits)

37 Advanced camera sensor Only one camera, placed in the end-effector Strobed active LED illumination Combined 3D data and colour image

38 Deep-Learning for plant part localization in images Large annotated datasets on a per-pixel level. Synthetic dataset used to bootstrap the model. Trained network deployed for real-time obstacle detection and to determine best end-effector alignment.

39 Target approach using visual servoing A modular software framework was developed and implemented for eye-in-hand sensing and robot motion control. Laboratory testing at Umeå University

40 In resultaten 95% van de oogstbare paprika s wordt gedetecteerd Oog hand coördinatie werkt naar tevredenheid Voldoende rijpheid niet gevalideerd in praktijksituatie Oogstpercentages nog steeds erg laag. End effector voldoet nog niet aan de verwachtingen 41

41 Greenhouse harvesting experiments currently ongoing