A Partial Curve Matching Method for Automatic Reassembly of 2D Fragments

Transcription

1 A Partial Curve Matching Method for Automatic Reassembly of 2D Fragments Liangjia Zhu 1, Zongtan Zhou 1, Jingwei Zhang 2,andDewenHu 1 1 Department of Automatic Control, College of Mechatronics and Automation, National University of Defense Technology, Changsha, Hunan, , P.R. China dwhu@nudt.edu.cn 2 Hunan Supreme People s Court, Changsha, Hunan, , P.R. China Abstract. An important step in automatic reassembly of 2D fragments is to find candidate matching pairs for adjacent fragments. In this paper, we propose a new partial curve matching method to find the candidate matches. In this method, the fragment contours are represented by their turning functions. The matching segments between two fragment contours are found by analyzing the difference curve between two turning functions directly. The performance of our method is illustrated with randomly shredded document fragments. 1 Introduction Automatic reassembly of 2D fragments to reconstruct original objects is an interesting problem with applications in forensics[1], archaeology[2,3], and other disciplines. The fragments are often represented by their boundary curves and candidate matches between different fragments are usually achieved by curve matching. Since matching between two fragments usually occurs over a fraction of their boundaries, partial curve matching is needed. The 2D fragments reassembly problem is similar to the automatic reassembly of jigsaw puzzles, which has been widely studied [4,5]. However, those solutions exploiting some specific features or a priori knowledge, e.g. puzzle pieces have smooth edges and well-defined corners, are impractical in many real applications. More generally, the fragments reassembly problem can be considered as a special case of partial curve matching problem. Researchers have proposed many solutions to this problem with different applications. Those solutions can be roughly divided into two kinds as to whether the fragment contour is sampled uniformly or not. One is string-matching based methods that represent fragment contours with uniformly sampled points. In [2], the curvature-encoded fragment contours are compared, at progressively increasing scales of resolution, using an incremental dynamic programming sequence-matching algorithm.wolfson [6] proposed an algorithm that converts the curves into shape signature strings and applies string matching techniques to find the longest matching substrings. This is also a curvature-like algorithm. However, the calculation of numerical curvature is not a trivial task as expected when noise exists [7]. The other is D.-S. Huang, K. Li, and G.W. Irwin (Eds.): ICIC 2006, LNCIS 345, pp , c Springer-Verlag Berlin Heidelberg 2006

2 646 L. Zhu et al. feature-based matching methods. In [3], fragment contours are re-sampled using polygonal approximation and the potential matching pairs are found through optimizing an elastic energy. However, a difference in the relative sampling rate of aligned contour segments can affect the optimal correspondence and the match cost [8]. In this paper, we propose a partial curve matching method to find the candidate matching fragment pairs. The fragment contours are represented by their turning functions and the matching segments are found by analyzing the difference curve between two turning functions directly. The curve similarity is evaluated as the residual distance of corresponding points after optimal transformation between two matching segments. This paper is organized as follows: Section2 presents our partial curve matching method. We present some experimental results in Section 3, and draw our conclusions in Section 4. 2 Partial Curve Matching Based on Turning Functions We assume that the fragment contours have been extracted successfully from the scanned fragments image. The method of comparing two fragment contours can be formulated as follows. 2.1 Contour Representation We first build the turning function θ(s) for eachfragmentcontour, asin [6]. Then, all θ i (s), i=1:n are sampled with the same space δ and stored as character strings C i,i=1:n in clockwise order. Note that the common segments of two matched fragments traverse in opposite directions. 2.2 Histogram Analysis on Δθ Suppose the two fragment contours to be compared are C A =(a 1,a 2,,a m ) and C B =(b 1,b 2,,b n )withm n. At a moment, C A is shifted by d positions (d is an integer) to become CA d = (a 1+d,a 2+d,,a m+d ) = Δ (a d 1,ad 2,,ad m ), the corresponding turning function becomes θa d = θ A(s i + dδ), i =1:m. The difference between θa d and θ B is defined as Δθ d AB = θ B θ d A Δ =(b 1 a d 1,b 2 a d 2,,b m a d m ) (1) At this moment, if there exist two sufficiently similar segments on CA d and C B, the corresponding part on ΔθAB d will almost be a constant. Draw the histogram of ΔθAB d to calculate the number of points lies in each sampling interval [iλ, (i +1)λ],i =0:t n, there must be a peak on the histogram corresponding to the matching segments. t n is the number of sampling intervals that determined by t n = Δθd AB (m) Δθd AB (1) (2) λ

3 A Partial Curve Matching Method for Automatic Reassembly 647 Denote the indices of start and end points of each segment by start and end respectively. We only check the peaks with its height H > H max 2 and end start > m t for candidate pairs of start and end points, where H max is the l maximum of the histogram and t l is an integer. m t is the parameter controlling l the minimum length of the permitted matching segments between contour A and B. An example of the relation between Δθ(s) and the histogram is given in Figure 1. The dashed dot lines mark the mean value of the selected segment. Fig. 1. The relation between Δθ(s) and the histogram This is just a primary selection for finding the correct pairs of start and end points. The candidate match pairs are selected according to the following decision rule. Decision rule: For a segment (Δθ start,,δθ end ) on ΔθAB d, compute the standard deviation std, average deviation avd and angle change number acn as std = end (Δθ i mean) 2 end start (3) avd = end sin(δθ i mean) end start (4) where acn = end 1 acn i (5) end Δθ i mean = end start, acn i = { 1, 0, if Δθ i+1 Δθ i >t 0 otherwise (6) If (1) std < t 1 ;and(2)avd < t 2 ;and(3)acn > t 3 then the corresponding segments are selected as candidate matches.

4 648 L. Zhu et al. The conditions (1) and (2) reflect the fact that if two segments are sufficiently similar, then the overall angle turning tendency will almost be the same; condition (2) means that the difference curve of two well matched segments should be distributed near uniformly around its mean value; and condition (3) is used to avoid matching an almost straight segment with another segment. Other constraints can also be added to these conditions. One or more segments may be found each time when shift the shorter contour one step further. For comparing any two different fragment contours, we have to shift the shorter contour C B n times, where n is the number of samples on contour C A. For computing ΔθAB d for each shift d, the total number of comparisons is m, where m is the number of samples on contour C B. Hence, the complexity of histogram analysis is O(mn). 2.3 Recovery the Transformation and Similarity Given a pair of start points and end pints, we compute the appropriate matching contour segments in the (x, y) plane. Denote these contour segments by X and Y, then the optimal transformation E opt between those two segments will minimize the l 2 distance between EX and Y E opt X Y 2 =min EX Y E 2 (7) As in [9], transform X with E opt in the (x, y) plane to get the transformed segment X.ThenX and Y are evenly sampled and represented by two sequence {u i } and {v j }. The curve similarity is evaluated by S = m d(u i Y )+ n d(v j X ) i=1 j=1 (min(l 1,l 2 )) 2,d(u i,y)= min v j Y u i v j (8) Here, m and n are the number of points in X and Y, l 1 and l 2 are the length of each segment respectively. 3 Experimental Results We used the randomly shredded document fragments to test the algorithm. The algorithm was implemented on a Windows platform, and the programming language was C#. An AGFA e50 scanner was used as the image acquisition device. The fragments had been digitized in 150 dpi. Figure2(a) shows the image of the scanned fragments and its size is The scanned image was thresholded in RGB space to get a binary image. The contour of each fragment was extracted from this binary image. Figure2 (b) shows the extracted contours. In the test, the number of fragments is N = 16. The parameters were set as δ = 3.57, λ =0.2, t l = 15, t 0 =0.05, t 1 =0.3, t 2 =0.1, t 3 =3andt s =1. In comparing any two different fragment contours, we may get several possible matches with the curve similarity smaller than t s. In this case, we only select the

5 A Partial Curve Matching Method for Automatic Reassembly 649 (a) (b) Fig. 2. (a) The image of scanned fragments, (b) extracted contours Fig. 3. The first 24 candidates returned by our partial curve matching method. The similarity S of each candidate match is showed on the left bottom of each grid. The true matches are marked with star( ). Table 1. Comparison between our Method and Stolfi s Method[2] Object Resolution T R Recognition Rate Ours Document 150dpi % Stolfi s Ceramic 300dpi % most similar one as the candidate match. In this test, there were 24 true matches in the original document; let T denote this set and R denote the recognized true matches from T. The algorithm started with 128 initial possible matches, and returned 30 matches with S<1, of which 16 were true. Figure 3 shows the first 24 candidate matches, in order of increasing S. Note that candidates 1-10 and 12-13, 15, 17, 18, 20 are all correct. Table 1 shows the comparison results between our method and Stolfi s method. It is hard to mark a strict comparison between the performance of these two

6 650 L. Zhu et al. methods because the test fragments are different. However, one thing to note is that our method depends much less on the scan resolution. 4 Conclusions and Future Work A turning function based partial curve matching method has been proposed to find candidate matches for automatic reassembly of 2D fragments. The accuracy of the method was verified by our experiment. Finding the candidate matches is only the first step to reassemble the original objects. We are now working on solving the global reconstruction problem to eliminate the ambiguities resulting from the partial curve matching. Our recent results will be reported in the not remote future. Acknowledgement This work is supported by the Distinguished Young Scholars Fund of China ( ), National Science Foundation ( ), Ministry of Education of China (TRAPOYT Project), and Specialized Research Fund for the Doctoral Program of Higher Education of China ( ). References 1. De Smet, P., De Bock, J., Corluy,E.: Computer Vision Techniques for Semiautomatic Reconstruction of Ripped-up Documents. Proceedings of SPIE (2003) Leitão, H.C.G., Stolfi, J.: A Multiscale Method for The Reassembly of Twodimensional Fragmented Objects. IEEE Transactions on Pattern Analysis and Machine Intelligence. 24 (2002) Kong, W., Kimia, B.B.: On Solving 2D and 3D Puzzles Using Curve Matching. Proceedings of Computer Vision and Pattern Recognition. 2 (2001) Burdea, C., Wolfson, H.J.: Solving Jigsaw Puzzles by A Robot. IEEE Transactions on Robotics and Automation. 5 (1989) Yao, F.H., Shao, G.F.: A Shape and Image Merging Technique to Solve Jigsaw Puzzles. Pattern Recognition Letters. 24 (2003) Wolfson, H.J.: On Curve Matching. IEEE Transactions on Pattern Analysis and Machine Intelligence. 12 (1990) Calabi, E., Olver, P., Shakiban, C., Tannenbaum, A., Haker, S.: Differential and Numerically Invariant Signature Curves Applied to Object Recognition. International Journal of Computer Vision. 26 (1998) Sebastian, T.B., Klein, P.N., Kimia, B.B.: On Aligning Curves. IEEE Transactions on Pattern Analysis and Machine Intelligence. 25 (2003) Pajdla, T., van Gool, L.: Matching of 3-D Curves Using Semi-differential Invariants. Proceeding of International Conference, Computer Vision. (1995)