A RECOGNITION METHOD FOR AIRPLANE TARGETS USING 3D POINT CLOUD DATA

Transcription

1 A RECOGNITION METHOD FOR AIRPLANE TARGETS USING 3D POINT CLOUD DATA Mei Zho*, Ling-li Tang, Chan-rong Li, Zhi Peng, Jing-mei Li Academy of Opto-Electronics, Chinese Academy of Sciences, No.9, Dengzhang Nanl, Beijing, China Commission III, WG III / KEY WORDS: LiDAR, Point clod data, Target recognition, Target segmentation, KD-Tree, Moment invariants ABSTRACT: LiDAR is capable of obtaining three dimension coordinates of the terrain and targets directly and is widely applied in digital city, emergent disaster mitigation and environment monitoring. Especially becase of its ability of penetrating the low density vegetation and canopy, LiDAR techniqe has sperior advantages in hidden and camoflaged targets detection and recognition. Based on the mlti-echo data of LiDAR, and combining the invariant moment theory, this paper presents a recognition method for classic airplanes (even hidden targets mainly nder the cover of canopy) sing KD-Tree segmented point clod data. The proposed algorithm firstly ses KD-tree to organize and manage point clod data, and makes se of the clstering method to segment objects, and then the prior knowledge and invariant recognition moment are tilized to recognise airplanes. The otcomes of this test verified the practicality and feasibility of the method derived in this paper. And these cold be applied in target measring and modelling of sbseqent data processing. 1. INTRODUCTION LiDAR (Light Detection and Ranging) is an active remote sensing system which can qickly provide three dimensional information of earth srface and object. Crrently it has been sed in many fields, sch as 3D city models, rban planning, design of telecommnication networks, vegetation monitoring and disaster management, etc. Based on the characteristic of LiDAR that can penetrate vegetation cover, this paper in particlar focses on the research on the method of target extraction by sing mlti-echo point clod data of LiDAR. Considering the application of three-dimensional target recognition sing LiDAR data, it mainly focses on airplane, bilding, power lines, etc. So far many methods were proposed bt there are still some problems. The first one is abot how to express and recognize objects of arbitrary shape while most of the recognition methods restrict the objects shape at present. Secondly, objects are sally in complex backgrond while many methods can only apply to a single object withot considering srronding. Thirdly, it s difficlt to recognize object with ncompleted information. De to the limitation of the crrent eqipment, the distance between point clods is fairly long, so we cold only get information on large objects withot details. Considering there are many kinds of object with a complex backgrond in one area, artificial interpretation wold cost a lot of time and work. So, it s necessary to design reasonable algorithm on targets recognition. Crrently, bare objects orientated recognition and abstraction based on LiDAR point data mainly focs on bilding, road and power line abstraction. A relative smaller body of literatres address disgised objects abstraction, also mainly concentrating on camoflage net disgised objects detecting (Marino et al., 005; Bck et al., 007). In general, most of those recognition methods are based on region, otline or featre points (Prokhorov, 009; Golovinskiy et al., 009). Considering the airplane targets in LiDAR point clod data, the direction of airplanes may be arbitrary, even the otline or size is different, while their shapes have certain similarities, so the method mst consider both deformation and scale invariance. The moment invariants are sitable to solve the problem mentioned above. Atomatic recognition of aircrafts based on moment invariants from binary television image was described (Sahibsingh et al., 1977). In this paper, the profile and bondary of targets was extracted from binary television images before the moment transformation, and then classification experiments were carried ot base on a Bayes decision rle and a distance-weighted k- NN rle. Zhongliang et al. (199) proposed a new method for atomatic ship classification sing sperstrctre moment invariants. In those papers, the classification methods of aircrafts or ships extracted from images by moment cold be also applied on LiDAR point clod data, bt point clod data is discrete, there are some differences between the distance image and binary image, especially in the detail of the target otline. Hans-Gerd et al. (1999) worked on invariant moments in raw laser altimetry data to extract model parameters of standard gable roof type hoses, sch as location, length, width and height of a bilding as well as its orientation, roof type and roof slope. Jinhi et al. (009) proposed a new approach of target recognition based on LiDAR point clod data by affine invariable moment, the experiment was based on data acqired by terrestrial laser scanner. Firstly the target images were generated by distance vale of point clod data, and then throgh the process of filtering, thresholding and region labelling, the affine moments of target region were extracted as featres, finally BP network and SVM algorithm were sed for target classification and recognition. There wold be still * Corresponding athor. 199

2 frther research of this method on the application of airborne LiDAR point clod data. In addition, the experiment of hidden targets extraction was not mentioned in these papers. This method is based on LiDAR point clod data organized by KD-tree, after target segmentation, H invariable moments and flight characteristics cold be also combined to improve the airplane target recognition, even hidden airplanes mainly nder the cover of canopy. The first step of hidden targets detection is target segmentation. Compared to other image segmentation methods, the segmentation based on KD-tree can deal with 3D point clod data directly. The proposed algorithm firstly ses KD-tree to organize and manage point clod data, and then tilizes clstering method to segment objects. Based on the reslt of segmentation, the prior knowledge and invariant recognition moment is tilized to identify target of interest. The otcomes of this test verified the practicality and feasibility of the method derived in this paper. The accracy rate is 0.937, error ratio is The approach in this paper cold also be applied in other kinds of covered targets extracting. These cold be applied in target measring and modelling of sbseqent data processing.. SEGMENTATION BASED ON KD-TREE In data processing, target segmentation is to classify points have same properties (sch as height, intensity, etc.) into one class, and it is also the premise of target recognition, fitting and measrement. The proposed algorithm firstly ses KD-tree to organize and manage point clod data, and then ses clstering method to segment objects..1 KD-tree This paper employs KD-tree to store and manage LiDAR point clod data. In this way, each point s neighbor-finding cold be qeried by KD-tree which is actally a K-dimensional binary tree of every node (Moore, 1991). In this section, KD-tree is three dimension, and bilt as follows: choose the splitting dimension as the longest dimension of the crrent hyperrectangle, and then choose the point closest to the middle of the hyperrectangle along this dimension. This strategy gives consideration to both tree s balance and segmented nits reglarity. Moreover, in order to avoid empty branch, the tree s top floors cold still follow the standard rles.. Targets Clstering After organizing point clod data by KD-tree, qery to search points within effective distance among neighborhood has been established. Aig at target segmentation, points in certain distance are classified into one class. We adopt the nearest neighbor qery method since finding proximal points to create local srface patch rapidly is the key step (Bin, 008). When given point p and qery distance d, the nearest neighbor qery method is to lookp points in the sphere with p as center and d as radis. Obviosly, the core algorithm is to find all nits intersecting with the sphere. The nit of KD-tree is called hyperrectangle (as rectangle in two-dimensional, rectanglar parallelepiped in three-dimensional), and can be defined with two arrays: the imm and imm of coordinate vale of each point. On prpose of estimating whether there is an intersection between the hyperrectangle hr and the sphere, the point t in hr nearest to p mst be fond, its expression is: hri pi hri (1) ti = pi hri < pi < hri hri pi hri where hr and hr is the imm and imm vale of hr i i in the ith dimension. Hyperrectangle satisfies condition that distance between p and t no more than d wold be looked p, and the range qery terates when all points are checked. In the clstering method, the choice of a reasonable distance threshold has an important inflence on the reslts of target segmentation. Therefore, in this paper, the distance threshold is set to times point spacing. 3. MOMENT INVARIANTS Considering that the featre of airplane is distinct when overlooked, so the target recognition method in image is sed for airplane recognition in LiDAR data. Moment is sally sed to characterize the distribtion of random qantity in statistics. If we take binary image or gray image as two-dimensional density distribtion fnction, the moment method can be applied to image analysis. 3.1 Basic Concept of Moment For digital image f (, i j ) of two-dimensional (N M), moment with p+q order can be defined as (Zhongliang et al., 199): M 1N 1 p q m i j f(, i j) () = j= 0 i= 0 where i, j is the coordinate in the image, and the central moment of f (, i j ) with p+q order is: M 1N 1 p q = ( x x) ( y y) f( x, y) j= 0 i= 0 It s easy to prove central moment is translation invariance, if making it with scale invariance, normalized central moment shold be sed and defined as: = or r = (4) ( p+ q+ )/ ( 0 + 0) ( 00) where r = ( p+ q+ )/4; p+ q=,3, H s Moment Invariants Lower order moment cold express a certain distribtion or target s basic geometric properties. Main reglar moments are zeroth order moment, first-order moment, second-order moment, third-order moment. H s moment invariants theory (H, 196) is a nonlinear combination of 7 parameters by normalized central moments, defined as: Moment invariant 1: 1 = (5) Moment invariant : = ( 0 0) (6) Moment invariant 3: = ( 3 ) + (3 ) (7) (3) 00

3 Moment invariant 4: 4 = ( ) + ( 1+ 03) (8) Moment invariant 5: 5 = ( )( ) [( ) 3( 1+ 03) ] (9) + (3 1 03)( ) [3( ) ( ) ] Moment invariant 6: 6 = ( 0 0)[( ) ( 1+ 03) ] (10) ( )( ) Moment invariant 7: 7 = (3 1 03)( ) [( ) 3( 1+ 03) ] (11) + (3 1 03)( ) [3( ) ( ) ] Semi-major axis a: 1/ [( 0-0) +4 11] 1/ a = [ ] (1) μ00 / Semi-major axis b: 1/ [( 0-0) +4 11] 1/ b = [ ] (13) 00 / Based on the above parameters, ratio ϕ of semi-major axis and semi-or axis is: ϕ = a/ b (14) Radiometric F of ellipse is: F = / π ab (15) 00 As the same with H s 7 moment invariants, in actal data processing, ϕ and F shold be expressed by normalized central moments. 4. AIRPLANE RECOGNITION METHOD In LiDAR data, the direction of airplane is arbitrary, the otline and size are sally different, bt they are similar in appearance, so the method mst consider deformation and scale invariance. We se prior knowledge and invariant recognition moment to recognise airplanes (even hidden targets mainly nder the cover of canopy). Figre 1 shows the processing procedre as follows: 1. Filter and classify point clod data into grond points and non-grond points;. Organize non-grond points by KD-tree; 3. Clster the organized point clod data and calclate the different targets; 4. Transfer each object into depth image. Assign the imm and imm of the targets height as 0 and 55, so the height range can be extended to 0-55; 5. Refine depth image; 6. Remove the incorrect reslts based on prior knowledge sch as the targets length, width, height and the ratio of length to width; 7. Calclate seven invariant moments of airplanes; 8. Identify target by Eclidean distance between the target and template which was bilt before. Figre 1. Flowchart of airplane recognition 5. EXPERIMENTAL RESULTS AND ANALYSIS In this section, we experimented on the LiDAR point clod data to confirm the recognition method for airplane targets proposed in this paper. The point clod data to be identified is shown in Figre (the red and green points respectively represent grond and non-grond points), which contains 318 points and with a density of.4 points/m, and the reslts of target recognition are shown in Figre 3 (the green and red points respectively represent airplane target and other target points). Figre. Point clod data to be identified 01

4 Figre 6. Depth images of the clstering reslts Figre 3. Reslt of airplane recognition In order to demonstrate the precision and robstness of the proposed method, another LiDAR point clod data is shown in Figre 4 (the red and green points respectively represent grond and non-grond points), which contain points and with a density of 1.7 points/m, and the reslts are shown in Figre 5 (the green and red points respectively represent airplane target and other target points). In addition, in order to verify the effectivity of moment invariants, we carried ot an experiment tilizing the match template method (Bin, 008) to recognize airplanes in the same depth image. Table 1 lists the performance comparison. The reslts show that the moment invariants sed in this paper is sperior to the other. Reslt of this paper Reslt of paper (Bin, 008) Total nmber of targets Nmber of correct reslts Nmber of incorrect reslts 1 4 Nmber of miss targets 1 3 Accracy rate (%) False alarm rate (%) Table 1. Performance comparison 6. CONCLUSIONS Figre 4. Another point clod data to be identified Airplane recognition based on LiDAR point clod data is a brand new application domain. Taking advantage of KD-tree and Moment Invariants, this paper presents a novel method to recognize airplane targets. And by carrying ot tests we validated its feasibility and practicality. Considering many other factors, e.g. canopy density, canopy thickness, and LiDAR hardware properties, cold inflence the effect of disgised objects detecting by sing point clod data, the frther research work shold be considered the above isses. In addition, the approach of this paper cold also be applied to other kinds of targets (even hidden targets) recognition. And we are now working on the algorithm evalation and perfection, as well as analyzing the factors that affect targets recognition and researching the better method for neighbor targets segmentation. Figre 5. Reslt of airplane recognition In this paper, we firstly se KD-tree to organize and manage point clod data, and make se of the clstering method to segment objects, and then the prior knowledge and invariant recognition moment are tilized to recognise airplanes. Some of depth images obtained from reslts of the clstering method are shown in Figre 6 (for displaying, in the depth images, the grey range is 17-55, and the points of no vale are set to 0). The reslts in top row are correct, while the ones in below row are incorrect, bt can be eliated by the invariant recognition moment. 7. REFERENCES Bin, X., 008. Research on object extraction and measrement based on LiDAR data, Master Thesis, Academy of Opto- Electronics, Chinese Academy of Sciences, Beijing, China. Bck, J., Malm, A., Zakel, A., Krase, B., Tiemann, B., 007. High-resoltion 3D coherent laser radar imaging. Laser Radar Technology and Applications XII. Proc. of SPIE, Vol. 6550, pp Golovinskiy, A., Kim, V., Fnkhoser, T., 009. Shape-based recognition of 3D point clods in rban environments. 0

5 Compter Vision, 009 IEEE 1th International Conference on, Kyoto, Japan, pp Hans-Gerd, M., George, V., Two algorithms for extracting bilding models from raw laser altimetry data. ISPRS Jornal of Photogrammetry & Remote Sensing 54, 1999, pp H, M., 196. Visal pattern recognition by moment invariants. Information Theory, IRE Transactions, 8(), pp Jinhi, L., Zhoxn S., 009. New Approach of Imagery Generation and Target Recognition Based on 3D LIDAR Data. The Ninth International Conference on Electronic Measrement & Instrments, 009, pp Marino, R. and W. Davis, 005. Jigsaw: a foliage-penetrating 3D imaging laser radar system. Lincoln Laboratory Jornal, 15(1). Moore, A., An intodctory ttorial on kd-trees. PhD Thesis, Efficient Memory-based Learning for Robot Control, Compter Laboratory, University of Cambridge, Cambridge, England. Prokhorov, D.V., 009. Object recognition in 3D LiDAR data with recrrent neral network. Compter Vision and Pattern Recognition Workshops 009, California, USA, pp Sahibsingh, A., Kenneth, J., Robert, B., Aircraft Identification by Moment Invariants. IEEE Transactions on Compters, Vol. C-6, No , pp Zhongliang, Q., Wenjn, W., 199. Atomatic ship classification by sperstrctre moment invariants and two-stage classifier. Singapore ICCS/ISITA '9, 199, pp ACKNOWLEDGEMENTS The athors wold like to thank for spporting of the project from the National Natral Science Fondation of China (Grant No ) and International Cooperation Project of the Ministry of Science and Technology of the People s Repblic of China (Grant No.010DFA790). 03