RECONSTRUCTION OF TREE STRUCTURE FROM MULTI-SCALE MEASUREMENT DATA

Transcription

1 International Journal of Complex Systems Computing, Sensing and Control Vol. 1, No. 1-2, pp , 2013 Copyright 2013, TSI Press Printed in the USA. All rights reserved RECONSTRUCTION OF TREE STRUCTURE FROM MULTI-SCALE MEASUREMENT DATA SHENGLIAN LU, XXINYU GUO *, JIANJUN DU, WEILIANG WEN Beijing Research Center for Information Technology in Agriculture Beijing , China * guoxy@nercita.org.cn 1. INTRODUCTION ABSTRACT Two methods are proposed for three-dimensional (3D) reconstruction of tree structure from multi-scale measurement data. The proposed reconstruction methods include reconstruction from digitized characteristic points of the tree and reconstruction from scanned point cloud of a whole tree. Both methods used real measurement data as such multi-scale data needed to be acquired from the targeted tree. At organ scale, laser scanner was used to obtain point cloud data from the surface of an organ. At individual tree scale, 3D digitizer and large-scale 3D scanner was used to measure the data of a whole tree. While at colony scale, several morphological characteristics were measured from the plant colony. Methods for reconstructing the 3D structure of a tree from these multi-scale measuring data were described in detail. A quantitative evaluation combing reconstructed accuracy and time consuming for data measurement was given to compare different reconstruction strategies. Key Words: 3D reconstruction, plant modeling, digitizing and scanning, digital plant The rapid and automatic reconstruction of plant model is an interesting and challenging topic both in computer graphics and agronomic research. Recently digitizing data from real objects have been used extensively for creating 3D models with the increasing availability and power of digitizers and laser scanners, which enabled us to measure larger and more complex objects, e.g. buildings and modern machines. Plants, however, pose special problems for reconstruction from scanned data. Unlike buildings and machines, which have relatively big and regular outside shape, and this make it can be measured easily by laser scanner, plant have numerous small organs which are hard to be scanned. It is difficult to obtain accurate and enough point cloud of fine branches from the canopy with thick leaves by using laser scanner, because of occlusion within the canopy and limited scanning resolution. As a result, fine branches, including the small petiole (that links the leaf and the branch or stem) are often missing from the scanned data. Standard mesh generation techniques used in commercial software often fail to reconstruction a complete polygonal model of plant because of this inadequate data. In most applications of 3D modeling for plants, an entire and detail mesh model is expected to enable further use (e.g. calculating light intersection of canopy, demonstrating the difference between varieties from outside appearance). The aim of this study is to test different reconstructing strategies for 3D plant models based on digitizing dada and 3D scanning data, with consideration both on reconstructed accuracy and time consuming for data 97

2 98 International Journal of Complex Systems measurement. By this way we wish to highlight that the best applicable 3D reconstruction method for plant need to consider its structure. 2. PREVIOUS WORK The modeling of plant structure has a long history. Pioneering work on plant modeling by Lindenmayer and later by Prusinkiewicz gave a general method known as L-systems to generate the structure and topology of plants [1]. L-system-based plant modeling packages, L-studio, has been developed to leverage the power of L-system to model the structures and development processes of plants with visual results [2]. However, due to its highly abstract, significant skilled user is expected to create pleasing plant models by using this tool. While some methods wish to produce realistic plant structure by direct interactive approaches [3, 4]. But these methods are also difficult to reconstruct the structure from real existing plant. With the successful applications of various measure devices in industry, more and more people reproduce virtual 3D plant models from measured data [5-7]. Measurements are presently considered to be the most accurate approach to quantitatively represent the 3D plant architecture, because the actual features of plant geometry are taken into account. Presently two kinds of digitizing devices are used extensively. Electromagnetic digitizers were used earlier to measure the spatial position and orientation of stems and leaves for giving a quantitative assessment to the tree geometry [5-7]. This kind handheld device is robust for capture the spatial characteristics of objects. However, it is a tedious and time-consuming job in digitizing plant (especially for tree which often with complex branch structure and crowned leaves) by using electromagnetic digitizers, and often not enough precise for accurately capture the detailed organ geometry, e.g. measuring the detail leaf surface [8]. Another kind of widely-used 3D capture devices is non-contact laser scanners, which have been used for various plant measurement and reconstruction [9-12]. Laser scanners enable us to rapid quantify the surface of an object as a dense set of points. But if an organ or part of an object is invisible to the scanner, its information will be missed in the captured point cloud. The missing information can be estimated by using existing or statistic knowledge about morphological structure of plants [13]. But this will lose accuracy to the measured plant. Another drawback of using laser scanners is the point cloud may contain considerable noise points coming from background and other plants, these noise information is needed to clearly distinguish subsets of points related to the plant organs such as leaves and stems. Presently this still maintains an open problem expected to be addressed [11]. In this paper we wish to test different reconstruction strategies by using multi-scale measurement data, and to demonstrate the effect of different reconstruction methods, with a detail discussion according to different application situations. 3. DATA MEASUREMENT We measure data from the tree at three scales: organ scale, individual tree scale and colony scale respectively. At organ scale, a hand-held laser scanner was used to obtain point cloud data from the surface of an organ. At individual tree scale, 3D digitizer and large-scale 3D scanner was used to measure the data of a whole tree. While at colony scale, several morphological characteristics were measured from the plant colony. 3.1 Capturing Data at Organ Scale A laser scanner (Polhemus FastSCAN Scorpion) was used to digitize organ surface data (we only consider leaf in this study). The scanner is capable of returning very large data sets from small

3 Reconstruction of Tree Structure from Multi-Scale Measurement Data 99 leaf surfaces, and can be classified as a multiple-point digitizer. We sampled several leaves with different ages (locate at different position at a same branch) to scan directly from the tree (Figure 2a). Since there may be hundreds and thousands leaves in a tree, and the shape of each leaf is distinct to others leaves, but it is impracticable to scan all the leaves. We choose ten sampled leaves to scan in this study. A triangular mesh model could be generated from the scanned point cloud of a sampled leaf, as Figure 1 shows. Figure 1. Two triangular mesh models of apple tree leaves. 3.2 Capturing Data from Individual Tree Scale Two kinds of digitizers were used in this step. And different capturing strategies results in different reconstruction method. Firstly we used a mechanical digitizer (a stylus option in the FastSCAN Scorpion laser scanner) to digitize characteristic points from trunks and branches of a tree. To reduce the time consumption and complexity for digitizing the tree, we only obtain the necessary characteristic points from the tree. Generally only two points were selected for a straight trunk and branch. The inflection point was also captured if the branch is bent. The digitizing order was from the root to the top of the tree, as Figure 2 shows. In which each number means a selected digitizing characteristic point. Figure 2. The diagram of digitizing order for a tree.

4 100 International Journal of Complex Systems The digitized points for each tree were output as an ASCII file. During digitizing, the diameter of each trunk and branch were manually measured with a vernier caliper, and these data were used later for the 3D reconstruction of the tree. Secondly we used a large-scale laser scanner (Leica ScanStation 2) to obtain the 3D point cloud from a whole tree. This kind scanner enables quick capturing detail 3D points from very big tree and with less additional artificial labors. 3.3 Capturing Data at Colony Scale At this scale some morphological characteristics (including mean length of different types of branch, branching angle, leaves number at different types of branch, leaf inclination angle, leaf azimuth and so on) were collect from the tree. These morphological characteristics were measured manually with ruler and protractor from the tree colony. In this study we choose 5 tree stands to collect these data. These morphological characteristics would be used for creating statistical models which are expected to guideline the later 3D reconstruction. Table 1 gives an example of the statistical characteristics of leaves in different type of branches of apple tree. Table 1. Leaf characteristics of different type of branches in apple tree Type of Mean leaf Mean leaf Mean leaf stalk branch length(cm) width(cm) length(cm) long shoot medium shoot short shoot leafage shoot D RECONSTRUCTION We test two 3D reconstruction methods for tree structure from the above measuring data. 4.1 Reconstruction from Digitizing Characteristic Points In this reconstruction method we wish to combine digitizing characteristic points and leaf blade mesh scanned by laser scanner. Using an apple tree for example, firstly we used the characteristic points digitized from the trunks and branches to reconstruct a skeleton model of the tree (as Figure 3b shows), by connecting the measured points from the same trunk or branch as a line. The reconstructed skeleton includes trunks and branches, and its detail degree depends on the measured data. Secondly each line in the skeleton was replaced with a B-spline curve as such the branch looking more smooth (see Figure 3c). Then a mesh model (see Figure 3d) could be generated from this skeleton by using the diameter information of each trunk and branch measured in Section 3.2. Lastly the sampled scanned leaf meshes were attached to the mesh model of tree. The statistical models (created in Section 3.3) about the morphological characteristics of this kind tree were used to decide the leaf number, length and width for each branch. And texture mapping also could be used to the final 3D model (see Figure 3e).

5 Reconstruction of Tree Structure from Multi-Scale Measurement Data 101 (a) (b) (c) (d) (e) Figure 3. The process and results reconstruction from digitizing characteristic points: (a) the real apple tree; (b) the digitizing characteristic points; (c) the reconstructed skeleton with B-spline curve description; (d) mesh model; (e) the final 3D model with leaves mesh and texture mapping. 4.2 Reconstruction from Scanned Point Cloud of a Whole Tree In this reconstruction strategy we wish to extract a skeleton of the tree directly from the scanned point cloud (mentioned at Section 3.2). A constrained Laplacian smoothing method [14] was used to extract the skeleton from scanned point cloud. But we found, in this study, the skeleton extracted directly from our scanned point cloud was not very satisfactory, because there were many scattered points in the origin scanned points (coming from leaves and little shoots, see Figure 4a). A distance based noise points deleting method was used to remove these scattered points, and this could resulted in a simplified point cloud (see Figure 4b). Then we generated a skeleton model from the simplified point cloud by using the constrained Laplacian smoothing method (as Figure 4c shows). The little shoots and leaves were missing in this extracted skeleton, and then need to be restored for a complete reconstruction. Currently we used a knowledge-driven strategy to repair the

6 102 International Journal of Complex Systems missing little shoots and leaves. Concretely, we used distribution models (created in section 3.3) about different types of little shoots how locating on branch to supply little shoots to the extracted skeleton model, then we got a mesh model of the tree with little shoots (as Figure 4d shows). Finally the sampled scanned leaf mesh could be added to the mesh model by using the method mentioned in Section 4.1, and finished the reconstruction (see Figure 4e). (a) (b) (c) (d) (e) Figure 4. The process and results reconstruction from scanned point cloud of a whole tree: (a) the origin scanned point cloud; (b) the simplified point cloud; (c) the extracted skeleton curve; (d) mesh model with little shoots; (e) the final 3D model with leaves mesh and texture mapping. 5. RESULTS AND DISCUSSION Based on the above mentioned method, we had chosen a five year old apple tree(see Figure 3a) to measure data, and the digitizing characteristic points can be found in Figure 3b (red points), the scanned point cloud of the whole tree can be found in Figure 4a. While some others morphological characteristics were also collected at that time, and the relevant statistical models were derived from these morphological characteristics.

7 Reconstruction of Tree Structure from Multi-Scale Measurement Data 103 Two reconstruction methods were proposed by combing the above multi-scale measuring datas. Both methods can generate pleasure 3D tree model (see Figure 3e and Figure 4e respectively) compared to the real tree. Additionally, the leaf characteristics of different type of branches and sampled mesh of different-age leaves were collected and used in the reconstruction, thus the resulted model have a very detailed in the canopy. Another six year old citrus tree (see Figure 5a) was also chosen to measure data and test the proposed methods. The reconstructed results by using the two proposed methods were shown in Figure 5, with a visual comparison to the original real tree. (a) (b) (c) (d) (e) Figure 5. Reconstruction of a citrus tree: (a) the real citrus tree; (b) the digitizing characteristic points; (c) the final 3D model reconstructed from digitizing characteristic points; (d) the origin scanned point cloud; (e) the final 3D model reconstructed from scanned point cloud. We also wish to consider the time and artificial labors for data measurement and reconstruction, and to make the reconstruction more practicable. It took between 2 and 3 hours to digitize the characteristic points from the apple tree. This digitizing process needs two people to do. In other words, the total time for characteristic points measuring may be 4 to 6 hours. On the other hand, the time for scanning point cloud of the whole

8 104 International Journal of Complex Systems tree is 0.5 hours with two people. To collect the morphological characteristics from 5 apple tree stands, we took 8 hours with two people. In other words the time for measuring morphological characteristics is 16 hours totally. While in the reconstructed steps, 3 hours was needed in method of reconstruction from digitizing characteristic points, including processing the characteristic points, generating the skeleton, and reconstructing the final 3D model with leaves mesh and texture mapping. But we needed to spend 5 hours to finish the reconstructing by using the method of reconstruction from scanned point cloud of a whole tree. Because we need to simplify the origin point cloud and repair the missing little shoots. The times consuming for measuring and reconstructing citrus tree is similar to apple tree. Totally, in our current study, the time (including data measurement and reconstruction) for method of reconstruction from digitizing characteristic points may be 8 to 12 hours, because only part of the collected morphological characteristics information needed to be used in this reconstruction. While the time for reconstruction from scanned point cloud of a whole tree may up to 20 hours. However, we need to know that, both the structure of the used apple tree and citrus tree are simple relatively, as such the operation of digitizing characteristic points can be carry out easily (for example, we don t need to use a scaffold to digitize the branch in the top of the tree). The larger-scale laser scanner will take its advantage if we wish to reconstruct a big and high tree. 6. CONCLUSION We proposed two methods for reconstruction 3D structure of tree by combing multi-scale measurement data, these datas were measured from real trees from organ, individual plant and colony scale respectively. Hand-held electromagnetic digitizer, hand-held laser scanner and largescale laser scanner were used to measuring data. We wish to find a practicable method for the rapid and accurate 3D reconstruction of trees with the consideration of time and artificial labors both for data measurement and reconstruction processing. In our current study (where the tree has a small and simple canopy, enable the easy measuring data), conclusion can be achieved that both method could create pleasure 3D model of the targeted tree, from the viewpoint of visual effect. From the view point of time and labors for data measurement and 3D reconstruction, the method of reconstruction from digitizing characteristic points may has obvious advantage. However, method reconstruction from scanned point cloud will be more effective when reconstructing tree with big and high canopy. Automatic 3D reconstruction of whole plant from laser scanned data points is presently still an open problem. The key challenge may be the segmentation of the 3D data set, as suitable automatic algorithms are for the moment unavailable. Another difficulty is that scanner beams only hit the plant organs making the plant hull, preventing one to get information inside the plant volume. And many organs would be hided in the canopy, especially in plants with high foliage density. All these problems are expected to be furthered and maintain the reconstruction of plants a challenging topic. ACKNOWLEDGEMENTS This work is supported by National High-tech R&D Program of China (No. 2013AA102405), Beijing Science and Technology Project (No. D ) and Beijing Nova Program (No. XX ).

9 Reconstruction of Tree Structure from Multi-Scale Measurement Data 105 REFERENCES 1. P. Prusinkiewicz, Modeling and visualization of biological structures, Proceeding of Graphics Interface '93. Toronto, Ontario. 1993, pp R. Karwowski and P. Prusinkiewicz, The L-system-based plant-modeling environment L- studio 4.0, Proceedings of the 4th International Workshop on Functional-Structural Plant Models. Viala Montpellier, France. 2004, pp B. Lintermann and O. Deussen, Interactive modeling of plants, IEEE Computer Graphics and Applications, 1999, 19(1): J. Wither, F. Boudon, M.P. Cani and C. Godin, Structure from silhouettes: a new paradigm for fast sketch-based design of trees, Computer Graphics Forum, 2009, 28(2): H. Sinoquet and P. Rivet, Measurement and visualisation of the architecture of an adult tree based on a three-dimensional digitising device, Trees: Structure and Function, 1997, 11: H. Sinoquet, S. Thanisawanyangkura, H. Mabrouk and P. Kasemsap, Characterization of the light environment in canopies using 3D digitising and image processing, Annals of Botany, 1998, 82: G. Sonohat, H. Sinoquet, V. Kulandaivelu, D. Combes and F. Lescourret, Threedimensional reconstruction of partially 3D digitised peach tree canopies, Tree Physiology, 2006, 26: B. Loch, J. Belwar, and J. Hanan, Application of surface fitting techniques for the representation of leaf surfaces, International Congress on Modelling and Simulation. MODSIM Press, Melbourne. 2005, pp E. Kaminuma, N. Heida, Y. Tsumoto, N. Yamamoto and N. Goto, Automatic quantification of morphological traits via three-dimensional measurement of Arabidopsis, The Plant Journal, 2004, 38: S. K. Rice, C. Gutman and N. Krouglicof, Laser scanning reveals bryophyte canopy structure, New Physiologist, 2005, 166: T. Dornbusch, P. Wernecke and W. Diepenbrock, A method to extract morphological traits of plant organs from 3D point clouds as a database for an architectural plant model, Ecological Modelling, 2007, 200: T. Dornbusch, S. Lorrain, D. Kuznetsov, A. Fortier, R. Liechti, I. Xenarios and C. Fankhauser, Measuring the diurnal pattern of leaf hyponasty and growth in Arabidopsis a novel phenotyping approach using laser scanning, Functional Plant Biology, 2012, 39(11): H. Xu, N. Gossett and B. Chen, Knowledge and heuristic-based modeling of laser-scanned trees, ACM Transaction on Graphics, 2007, 26(4): Z. X. Su, Y. Zhao, C. J. Zhao, X. Y. Guo and Z. Y. Li, Skeleton extraction for tree models, Mathematical and Computer Modelling, 2011, 54: ABOUT THE AUTHORS S. L. Lu received his Ph.D. degree in Mechanical Engineering from Shanghai Jiao Tong University in He joined the National Engineering Research Center for Information Technology in Agriculture (NERCITA) at the same year. Currently he serves as an associate research fellow in computer graphics. His research interests include simulating, modeling and visualizing of complex biological objects; physically-base modeling and image processing.

10 106 International Journal of Complex Systems X. Y. Guo received his Ph.D. degree in agricultural science from China Agricultural University in Currently he is a research fellow at the National Engineering Research Center for Information Technology in Agriculture, China; and director of the department of cartoon & animation. His current research focuses on digital agriculture, plant structure-function modelling and virtual reality. J. J. Du received his Ph.D. degree in Computer Science in 2010 from the Beijing University of Technology. Currently he is an assistant researcher in China National Engineering Research Center for Information Technology in Agriculture and his research interests include 3D reconstruction from microimages, image processing, modelling greenhouse, etc. W. L. Wen received his B.S. degree in Computational Mathematics in 2008 from the Dalian University of Technology. Currently he is an engineer in China National Engineering Research Center for Information Technology in Agriculture. His research interests focus on geometric modeling and its applications in agriculture.