6th WSEAS International Conferene on CIRCUITS, SYSTEMS, ELECTRONICS,CONTROL & SIGNAL PROCESSING, Cairo, Egypt, De 9-31, 007 65 Detetion and Reognition of Non-Oluded Objets using Signature Map Sangbum Park, Youngjoon Han, Hernsoo Hahn College of Information Tehnology Soongsil University Sangdo-5Dong, Dongjak-Gu, Seoul Republi of KOREA http://visionlab.ssu.a.kr Abstrat: - For onstruting a flexible bin piking system where parts an be provided with arbitrarily staked in a workspae, detetion and reognition of non-oluded objets are essential proess. For implementing suh proess, this paper proposes a new algorithm whih determines whether an objet is oluded or not and at the same time whih objet it is in the DB of objet models. It is based on a signature map whih is onstruted by deteting the objets in an input image and drawing the signatures of the whole image with referene to individual objets. Thus the number of signature maps is equal to the number of the objets deteted in the input image. A signature map shows the outer ontour and inside edge features. Olusion by other objets appears as distortions in the outer ontour of the signature map. The inside edge features are used for diserning the objets having the same outer ontour by different inside shape. To make the manipulator pik up a seleted part, a pose estimation method for elliptial objets is also proposed. The performane of the proposed algorithm has been tested with the task of piking the top or non-overlapped objet from a stak of arbitrarily loated objets. In the experiment, a reognition rate of 98% has been ahieved. Key-Words: - Bin-piking, Signature map, Contour traing, Objet detetion, Objet reognition 1 Introdution In many automati assembly systems, the known parts are palletized and thus reognition proess is not needed. However, if the parts an be delivered with unsorted, the palletizing devies are not needed but a omplex reognition and pose estimation proesses are required. For this reason, how effiiently represent 3D objets using the features that an be easily extrated has drawn a great attention from many researhers working for developing automati assembly systems. However, detetion of the top objet on a stak of arbitrarily loated objets still remains as the ore problem in onstrution of the bin piking system without pallets.[1,] Many researhers have proposed different solutions for bin-piking problem that a manipulator grasps an objet loated in a pallet onstraining the pose of the objet or on a onveyer belt without onstraints. To exatly loate the objet s pose, an objet representation sheme whih may provide suh information from its D image should be developed. Most objet representation shemes proposed for this purpose onstrut have been based on the shape from ontour method.[3,4] The features extrated from the D image are used for mathing with the objet models in the reognition module. When objets are overlapped in the image, the approahes based on the shape from ontour method generate many problems. One of them is that it requires an exessive omputation time sine all models in the database should be ompared with a test objet. The other one is that inorret mathing results may our if some parts of the objet features are oluded.[5] The approahes using T-juntion analysis for reognizing overlapped objets are suitable for diserning the objets having the linear ontours. When ontours are urved ones then it does not work appropriately.[6] For solving these problems, this paper propose a new signature map tehnique for extrating geometrial features of 3D objets using a D image to detet and reognize the top objet on a stak of 3D objets. The proposed method onsists of two parts: the generation of signature maps of an input image and the detetion and loalization of the top objet. To generate the signature map, we use an effetive and robust edge detetor to redue the noise or to reonstrut the lost portion of edges. For this reason, a ontour traing method whih is based on a ubi spline interpolation
6th WSEAS International Conferene on CIRCUITS, SYSTEMS, ELECTRONICS,CONTROL & SIGNAL PROCESSING, Cairo, Egypt, De 9-31, 007 66 is used in the preproessing module. It is in order to inlude urved edges as well as linear edges in the appliation domain of the proposed method. Based on the proposed representation sheme, this paper proposes a new algorithm to find overlapped objet using bending point information. The algorithm memorizes the number of bending points of the objet models and ompares it with that of a test objet. If that of a test objet is bigger than that of the objet model, then the objet model is seleted as the model of the test objet. Finally, the depth information inluded on the signature map is extrated for estimating the pose of the objet. The depth information of the objets in the signature map is generated from ontours of objet. This signature map also desribes the objet features loated inside the boundary. This information is very useful to disern the objet having the same outer boundary but different inner shapes. The rest of this paper is onstruted as follows: Setion explains how to generate a signature map and how to use it for objet reognition. Setion 3 illustrates how to detet overlap of objets and selet the top objet. Setion 4 shows how to determine the pose of a deteted objet and Setion 5 illustrates the performane of the proposed algorithm by the experiments. Objet Reognition using Signature Map.1 Constrution of a signature map The signature of a 3D objet is generated from its D image. It draws the distanes of the boundary points from the enter of the objet in a D plane by inreasing the angle to the ounter-lok diretion as shown in Fig. 1. Thus the auray of a signature depends on how learly the boundary is deteted. To extrat the boundary for this purpose, ontour traing or boundary traking method is used in general. These methods are basially edge traking tehniques. They searh all neighboring pixels around the referene edge to test the onnetivity with it. Thus, they are very sensitive to noise or disonnetion of an edge sine suh things generate unertainty in determining the diretion to follow. To solve this problem, the signature map proposed in this paper desribes the internal and external edge features. Thus, even in the ase where there are objets having the same outer ontour shape but different internal edge features as shown in Fig. 1, the proposed algorithm an separate them learly. Fig. 1(a) shows the ase of having irular and Fig. 1(b) does the ase of having retangular arves, inside the outer boundary. Although both objets have the same outer ontour in signature map, the proposed method easily diserns one objet from another using the internal edge features. (a)cirular holes inluded (b)retangular arves inluded Fig. 1 The signature maps for the objets having the same outer ontour but different inside edge features When multiple objets are given in an image, the signature maps orresponding to individual objets are generated as shown in Fig.. If an objet is not oluded by any other objet, then the signature map generated with referene to the enter of this objet may generate the same signature as that of the objet model as shown in Fig. (a). However, if an objet is oluded by other objets as shown in Fig. (b), then the signature map generated with referene to the objet beomes distorted and it also shows whih objet is oluding the referene objet. (a)not-oluded ase (b)oluded ase Fig. The signature maps in the ases of oluded and not-oluded If a signature map is used, then a reognition task an be defined as a proess of testing if the signature of a test objet is mathing with whih objet model s signature. In order to simplify the signature omparison proess, the proposed algorithm uses the signature features suh as the number of poles(n p ), the angles between the poles(θ i ), the number of ontours(l) and the distane between ontours(d i ).
6th WSEAS International Conferene on CIRCUITS, SYSTEMS, ELECTRONICS,CONTROL & SIGNAL PROCESSING, Cairo, Egypt, De 9-31, 007 67. Feature extration from a signature map The signature basially shows the shape of an objet. Therefore, a pole in a signature orresponds to a onvex vertex of an objet in a D image and the number of poles tells the number of orners. The pole is deteted analyzing the variation of signature as shown in Fig. 3(a). For a given signature, a hain ode is used to find a pole. However, a onventional hain ode representation is easily ontaminated by noise[7]. Therefore, this paper uses a false pole elimination tehnique using the forward and bakward detetion window given in Fig. 3(b). (a) Pole detetion by testing urvature variation (b) Bakward and forward detetion window () Pole andidate detetion (d) False pole elemination Fig. 3 Pole extration from a signature Let s assume that a signature as shown in Fig. 3() is given and the traing has begun from the leftmost top pixel shaped in the figure to generate an index sequene 30030030030030. In the sequene, the pixel marked as B are onsidered as a pole andidate sine the index hange ours at the pixels. However a new edge is not generated from pixel B and the pattern is repeated, they are onsidered as a part of the edge and ontinues until pixel C. Sine a new index is appeared at pixel C, it is onsidered as a pole andidate and a new traing starts from it. If a new sequene is long enough to be onsidered as an edge, then pixel C is determined as a pole pixel. However, if the length of a new edge is too short as shown in Fig. 3(d) then it is onsidered as a part of an edge. Bakward detetion window is used to find a branhing point. When the poles are deteted in the signature map, the angle between the neighboring poles is same as the distane between poles in the signature map and it should be larger than a threshold Fig. 4 The internal features of the objet Differently from the outer ontour, the detetion of inside edge features requires more attention to eliminate false features generated by noise. Shadows or refletion aused by light soures may generate noisy features. To eliminate suh features, every feature is tested if it has the area larger than a threshold. If it is smaller than the threshold, then it is onsidered as a noisy region..3 Model registration and deision rule for mathing The signature map of eah model(m i ) is registered in the database as a feature vetor, M i ={ontour feature(f out ), internal feature(f in )}. F out and F in are defined as F out ={the number of pole(n p ), the angle between poles(θ i ), the number of ontour(l), the distane between ontours(d i )} and F in ={f in f i in= {Shape(S i ), Position(P i ), average intensity (I avg )}}, respetively. The mathing funtion T in Eq.(1) is utilized to deide for if two feature vetors A and B are oinide. The Boolean funtion T returns TRUE(1) if the differene between two features A and B is smaller than the error threshold(α). Otherwise, it returns FALSE(0). The error threshold (α) is deided by the experiments. T( A, B, α ) = BOOL 0< A α A B Using T, the deision rule omparing two feature vetors is defined as Eq. () whih tests the math of individual elements of the feature vetors. ( θ, θ, α) ( μ, μ, α) (,, α) (, d, α ) T = T T T L L m out m m m T d m In Eq. (), the subsript m and are the indies of the model and test objets. If the number of model objets mathed with the test objet is more than, the internal features are ompared using Eq.(3). ( ( ), ( ), α ) (,, α ) (, I, α ) T = T CS CS T p p m in m m T I m (1) () (3)
6th WSEAS International Conferene on CIRCUITS, SYSTEMS, ELECTRONICS,CONTROL & SIGNAL PROCESSING, Cairo, Egypt, De 9-31, 007 68 In Eq. (3), C is the funtion of testing how lose to a irle, defined as Eq. (4). ( ) C S np = (4) π s ( L /) where n p is the number of pixel in the area S, L s is the distane of long axis 3 Overlap Deision and Reognition 3.1 Contour traing in the signature map Sine the ontour of a irular objet does not hange sharply, overlap of objets an be easily deteted by ontour traing. Fig. 8 illustrates the ontour deteting method used in this paper when the ontour is ontaminated by noise or disonnetion. If a ontour has a disonneted part, the algorithm estimates the diretion of ontour and alulates onnetive strength to apply the spline interpolation. This spline interpolation assumes that the ontour is differentiable at every points and its urvature does not hange a lot. Therefore, the algorithm selet those points whih are rendering a urvature smaller than a threshold among the neighboring points. For example, in Fig. 5, when disonneted point is deteted at the n-th pixel in setion A, the algorithm searhes the nearest edge in the shaded region. If found, then a setional polynomial-funtion whih is differentiable in the setion [x 0, x n ] is inserted to onnet the ontour given in setion A and the newly found edge. The onneted edge region is defined as a new setion A. In the same way, this ontour traing ontinues until it annot find the onneted edges any more. Setion B shows the reonstruted region by the interpolation proess urvature hanges abruptly. In the proess of ontour traing, a andidate for an olusion point is seleted by testing the following onditions given Eq. (5) whih tests the urvature variation range. d p p d + min max min max d p p d 0 α α max In Eq. (5), p p ± is a linear distane between two points, α [-π, π] is the angle between p and p ±, d min, d max and α max are the referene values to be used as the thresholds. α ( p) at an olusion point an be represented as the inlination between the edges, β (p)=π - α (p). Sine d min and d max define the angle differene, they are determined to be inversely proportional to the objet size (a) Cirular objets (b) Ellipse objets Fig. 6 Distorted signature by oluding objets 4 Pose Estimation of The Top Objet To ontrol a manipulator to pik up the target objet, its pose must be estimated with error smaller than the tolerable error. Fig. 7 shows onfiguration of the pose estimation proess where O is the world frame, C is the amera frame, and T is the objet frame. (5) Fig. 5 The ontour traing method 3. Overlap estimation If the target objet is overlapped, the signature beomes distorted as shown in Fig. 6. This distortion an be deteted by searhing the points where the Fig. 7 The onfiguration of the pose estimation proess To ontrol the manipulator, the relation, o H, between the world frame and the amera frame should be known. One o H is known, then the objet frame in the world frame an be extrated through measuring
6th WSEAS International Conferene on CIRCUITS, SYSTEMS, ELECTRONICS,CONTROL & SIGNAL PROCESSING, Cairo, Egypt, De 9-31, 007 69 the objet frame with referene to the amera frame. Then, o H an be expressed by Eq. (6). HT = HC HT osθ sinθ 0 a (6) sinθ osα osθosα sinα sinαd HT = sinθsinα osθsinα osα osαd 0 0 0 1 O O C O Under these relationships among the frames, as shown in Fig. 8(a), the objet pose an be represented by two rotation angles about X axis and Z axis and translation in the (X,Y) plane. The position of the objet frame with referene to the amera image frame (x,y) is determined first. After loating the amera so that the origin of the amera image frame and the objet enter, the amera is rotated about Z axis of the amera image frame by θ degrees to oinide the X axes of the amera image frame and the objet frame. Then it is rotated one more about the X axis of the amera image frame by α degrees to oinide the Z axes of the amera image frame and the objet frame. α an be by Eq.(7) using the lengths of the irular objet measured from Fig. 8(b). 1 ll α = os (7) l S By applying these pose parameters to Eq. (6), the pose of the objet in the world frame an be extrated. (a) Pose parameters (b) Definition of α Fig. 8 Definition of pose parameters 5 Experimental Results To evaluate the performane of the proposed algorithm, the algorithm has been implemented on the 6-DOF manipulator having a single amera mounted on the gripper. It is asked to detet and reognize and pik up the top or non-oluded objet among arbitrarily staked multiple objets. Six different types of objets are used to make a stak of objets. Their shapes and signature features are given in Table 1. 5.1 Reognition and detetion of the top objet To test the reognition auray, eah objet model is loated at an arbitrary position in a D plane and the algorithm is applied. This test has been repeated 100 times per eah objet model. Sine the result will be same if an objet is loated separately from other objets and it is segmented suessfully, the reognition performane is tested in this way. The experimental results testing the reognition proess are summarized in Table. The proess extrats the signature map with referene to the test objet and mathes with those of the 6 objet models, to find the best mathing objet model. Table1 The DB of the 6 objet models. Type External N p /θ i 48/8 0/0 0/0 L/d i /6,36 /10,38 3/3,0,38 Internal S i 0.0 0.6 1.0 p i 0 180 90 I avg 0 7 8 Type External N p /θ i 0/0 6/60 0/0 L/d i /10,38 /18,7,4/1,17,5,7 Internal S i 0.35 0.0 0.0 p i 90 0 0 I avg 68 0 0 Table The reognition rate [%]. Type Deision A B C D E F Corret 99 98 100 98 99 99 Inorret 0 1 0 1 0 0 Not-available 1 0 0 1 1 1 As an be notied, the reognition rate depends on the shape of a test objet. In the ases of objet model B and D having the same outer ontour, reognition errors are ourred when the math proess is performed with only the outer ontours. However, if the inner features are also used, they an be suessfully diserned. In the ases of objet model A, E and F whih have distintive shapes of outer boundaries, the reognition is also suessful but there are hane to be failed sine there exists no inside edge feature. In ontrast, the objet model C whih has the
6th WSEAS International Conferene on CIRCUITS, SYSTEMS, ELECTRONICS,CONTROL & SIGNAL PROCESSING, Cairo, Egypt, De 9-31, 007 630 inside edge features an be easily reognized even when other objets have the same outer boundary, sine it has enough inside edge features. The top objet detetion proess is tested with the 100 images in whih 3 to 10 objets of 6 types are arbitrarily staked. The task is to segment all not-oluded objets in the image. It is onfirmed by the experiments that the suessful detetion rate is over 98%. 5. Pose estimation of the top objet To test the auray of the pose estimation algorithm, the poses of the deteted not-oluded objets are estimated. Here the pose resolution of the manipulator is assumed less than 1.0[mm]. The rotation is expressed by the angles rotated with referene to the X and Z axes of the manipulator, and the position of the objet enter is expressed by the (X,Y) oordinates in the manipulator frame. The experimental results are summarized in Table 3. The error inluded in rotation estimation about the X axis inreases as rotation angle gets larger, but the error inluded in rotation estimation about the Z axis does not hange depending on the rotation angle, as an be expeted. Translation error also inreases if the objet is loated far from the origin of the amera image frame. As shown in Table 3, both rotation and translation errors are small enough to be used for guiding the manipulator to pik up an objet. Table 3 The errors inluded in the pose estimations. Contents Average Error +30 3.1 X axis +15.34-15.9 Rotation -30.94 [degree] +90.1 Z axis +45.30-45. Translation [mm] -90.51 100 50.50 50 100.71 50 50 1.53-100 -50.76-50 -100.81-50 -50 1.7 6 Conlusions We have presented a new algorithm for detetion and reognition of the top objet on a stak of arbitrarily loated objets, using a signature map. A signature map is the signature of whole image generated with seleting the enter of a referene objet in the image as the enter of the signature. In a signature map, sine the signature of the referene objet is not distorted if it is not oluded, it an be determined of the referene objet is on the top or not. Of ourse, if an objet is loated without olusion, it is also onsidered as the top objet that an be grasped. The experimental results have show that the proposed method has several advantages in deteting and reognizing the top objet. The first one is that the method simultaneously detets and reognizes the top objet by analyzing the signature map. The seond one is that the method an disern those objets having the same boundaries but having different inside features. Above all, the an detet and reognize the objet even when the boundary of a referene objet is not ompletely onneted. To obtain these advantages, one weakness that must be enhaned is that the time omplexity inreases as the number of objets in the input image inreases. Referenes: [1] Berger. M, Bahler. G, Sherer. S, "Vision Guided Bin Piking and Mounting in a Flexible Assembly Cell," IEA/AIE 000, pp.109-118, 000. [] Dmitry Chetverikov, "A Simple and Effiient Algorithm for Detetion of High Curvature Points in Planar Curves," Computer Analysis of Image and Pattern 003, pp.746-753, 003. [3] Adnan A.Y. Mustafa, "Boundary Signature Mathing for Objet Reognition," VI001 Vision Interfae Annual Conferene, pp.7-79, 001. [4] Adnan A.Y. Mustafa, "Mathing Inomplete Objet Using Boundary Signatures," Proeedings of the 4th International Workshop on Visual Form, pp.563-57, 001. [5] Ovidiu Ghita, Paul F. Whelan, and John Mallon, A bin piking system based on depth from defous," Journal of Eletroni Image, Vol.14, Issue, pp.34-44, 003. [6] Kirkegaard.J, Moeslund.T.B, "Bin-Piking based on Harmoni Shape Contexts and Graph-Based Mathing," ICPR06, pp.581-584, 006. [7] Fu Chang and Chun-Jen Chen, "A Component -Labeling Algorithm Using Contour Traing Tehnique, IEEE Doument Analysis and Reognition, pp.741-745, 003.