1/19 Robust and efficient 2D Skeleton Shape Representation using Shock Graphs Chung, In Young Lee, Kang Eui 1. Introduction An important approach to representing the structural shape of a plane region is to reduce it to a graph. This reduction may be accomplished by obtaining the skeleton of the region via a thinning (or skeletonizing) algorithm. There have been lots of algorithms on obtaining skeleton. However, there are some drawbacks in the performance and time complexities of these algorithms [1]. There has recently been significant interest in using representations based on abstractions of Blum s skeleton into a graph for qualitative shape matching. The application of these techniques to large databases of shapes hinges on the availability of numerical algorithms for computing the medial axis. Unfortunately, this computation can be extremely subtle [2]. Approaches based on Voronoi techniques preserve topology, but heuristic pruning measures are introduced to remove unwanted edges. Methods based on Euclidean distance functions can localize skeletal points accurately, but often at the cost of altering the object s topology [3]. The new algorithm for computing subpixel skeletons which is robust and accurate, has low computational complexity, and preserves topology has been proposed by P. Dimitrov, C. Phillips and K. Siddiqi. The key idea is to measure the net outward flux of a vector field per unit area, and to detect locations where a conservation of energy principle is violated. This is done in conjunction with a thinning process applied in a rectangular lattice. However, there are, also, some disadvantages in this algorithm [2]. In this project, we attempt to modify and advance this algorithm and define a new one using Shock Graphs. In this report, we implement the new method proposed by P. Dimitrov, C. Phillips and K. Siddiqi and compare its result with that of Blum s method
2/19 2. Approach and Work Performed A. Blum s method In recent years there has been significant interest in using graph-based abstractions of Blum s skeleton for qualitative shape recognition [4]. The application of such methods to large image databases hinges on the availability of robust and efficient algorithms for computing the medial axis, or approximations to it. Blum proposed that the skeleton of a region may be defined via the medial axis transformation (MAT). The MAT of a region R with border B is as follows. For each point P in R, we find its closest neighbor in B. If p has more than one of closest dep on the definition of a distance. The MAT of a region has an intuitive definition based on the so-called prairie fire concept. Consider an image region as a prairie of uniform, dry grass and suppose that a fire is lit along ints border. All fire fronts will advance into the region at the same speed. The MAT of the region is the set of points reached by more than one fire front at the same time [1]. Following figure shows the resulted skeleton image using Blum s method. (a) Original image
3/19 (b) Resulted Skeleton image using Blum s method Figure 1. (a) shows the original image and the bottom one (b) shows the resulted image. B. New method a. Euclidean Distance Transformation (EDT) The Euclidean distance between the points p and q with their coordinates ( x, y), ( s, t), respectively, is defined as [1] D 2 ( p, q) = [( x s) + ( y t ) 2 ] 1 2 b. Computing Vector Field. Consider Blum s grassfire flow C = N t acting on a 2D closed curve C, such that each point on its boundary is moving with unit speed in the direction of the inward normal N. This formulation leads to a Hamilton-Jacobi equation on the Euclidean distance function to the initial curve [12]. In physics, such equations are typically solved by looking at the evolution of the phase space of the equivalent Hamiltonian system. Since Hamiltonian systems are conservative, the locus of skeletal points coincides with locations where a conservation of energy principle is violated. This loss of energy can be used to formulate a natural criterion for detecting singularities of the distance function.
4/19 More specifically, let D be the Euclidean distance function to the initial curve C 0. The magnitude of its gradient D is identical to 1 in its smooth regime. With q = ( x, y), p = ( Dx, Dy), the Hamiltonian system is given by H H p = = ( 0, 0), q = = ( Dx, Dy) q p 2 2 2 with an associated Hamiltonian function H = 1 D x + D ) [12]. It is ( y straightforward to show that all Hamiltonian systems are conservative and hence divergence free. Conversely, when trajectories of the system intersect, a conservation of energy principle is violated. This suggests a natural approach for detecting the skeleton: compute the divergence of the gradient vector field q and detect locations where it is not zero [2]. 1 Figure 2. Computed vector field
5/19 c. Computing Divergence (Flux) The divergence is defined as the net outward flux per unit area, as the area about the point shrinks to zero: div( q ) lim Δa 0 L < q, N) dl Δa Here Δ a is the area, L is its bounding contour and N is the outward normal at each point on the contour. Via the divergence theorem div ( q ) da = < q, N > a L In other words, the integral of the divergence of the vector field within a finite area gives the net outward flux through the contour which bounds it. Locations where the flux is negative, and hence energy is lost, correspond to sinks or skeletal points of the interior. Similarly, locations where the flux is positive correspond to sources or skeletal points of the exterior [2]. dl
6/19 Figure 3. Computed Flux
7/19 d. Thinning Some definitions Removable point; A point can be removed if and only if its 3x3 neighborhood graph, with cycles of length 3 removed, is a tree. A straightforward way of determining whether or not a graph is a tree is to check that its Euler characteristic V E (the number of vertices minus the number of edges) is identical to 1. 1 1 1 0 P 1 0 0 1 Figure 4. Example of a tree End point; A point could be an point of a 1 pixel thick digital curve if, in a 3x3 neighborhood, it has a single neighbor, or it has two neighbors, both of which are 4- adjacent to one another. P P Figure 5. Example of an point The thinning process can be made very efficient by observing that a point within the object which does not have at least one background point as an immediate neighbor cannot be removed, since this would create a hole. Therefore, the only potentially removable points are on the border of the object. Once a border point is removed, only
8/19 its neighbors may become removable. This suggests the implementation of the thinning process using a heap [2]. Algorithm FOR each border point P IF (P is removable) Insert(P,Heap) with Flux(P) END {For} WHILE (Heap.size>0) P=MaxHeapExtract(Heap) IF (P is removable) IF P is not an point Flux(P)>T remove(p, Heap) FOR each neighbor, Q of P IF (Q is removable) Insert(Q, Heap) END {FOR} ELSE Mark P as a Skeletal Point END {WHILE} Algorithm 1. Thinning algorithm
9/19 (a) Original image (b) Resulted subtle Skeleton image using divergence (Threshold = -0.3) (c) Resulted Skeleton image using new algorithm Figure 6. Original and Resulted Skeleton image
10/19 3. Results (a) Resulted Skeleton image using Blum s method (b) Resulted Skeleton image using new algorithm Figure 7. Comparison of the skeleton images using Blum s and new method Figure 7 shows the results of the skeleton images. As can be seen in the figure 7 a), some edges are lost and some topology has been changed. Red rectangle which located in the left upper side of the image (a) represents the change in the topology. The right representation of the topology is shown in the image (b). Also, the red box in the right side of the image (b) shows unfinished edge and this is modified in the new algorithm. However, in the result of the new algorithm, there is also incorrect result and that is unfinished edge which is located in the ear part of the rabbit.
11/19 4. Discussion The essential idea of the new algorithm for skeletonizing is to combine a divergence computation on the gradient vector field of the Euclidean distance function with a thinning process that preserves topology. The identification of points and branch points allows the result to be interpreted as a graph. The digital skeleton is then shifted to be within arbitrary precision of the true skeleton. However,, whereas results of this algorithm are good, there is one problem that should be solved. That is choice of the threshold is not obvious. 5. References [1] Rafael C. Gonzalez, Richard E. Woods, Digital Image Processing, 2E, Prentice Hall. [2] K. Siddiqi, A. Shokoufandeh, S. J. Dickinson, and S. W. Zucker, Shock graphs and shape matching. International Journal of Computer Vision, 35(1):13 32, 1999. [3] R. L. Ogniewicz. Discrete Voronoi Skeletons. Hartung-Gorre, 1993. [4] T.-L. Liu, D. Geiger, and R. V. Kohn. Representation and self-similarity of shapes. In Sixth International Conference on Computer Vision, 1998. [5] P. Dimitrov, C. Phillips and K. Siddiqi, Robust and Efficient Skeletal Graphs, Proc. Computer Vision and Pattern Recognition Conf., Vol. 1, 2000, 417-423.
12/19 Appix I. Source Code -Main routine clear all, close all; [FileName,PathName] = uigetfile('*.*','select image file'); I=imread([PathName '/' FileName]); %Change the type of I into double for computation I=double(I); %Calculate the size of input image [Ys Xs]=size(I); %Change the max of the input image into 255 if max(max(i))==1 I=255.*I; %Show the input image imshow(i); reply = input('which represents the background? White[1] or Black[0]: ','s'); close all; if isempty(reply) reply = '0'; if reply=='0' I=xor(I,ones(Ys,Xs));
13/19 [D L]=bwdist(I); sigma=0.5; kernel_size=[3,3]; G=fspecial('gaussian',kernel_size,sigma); SD=imfilter(D,G); TOL=[0 1]; SD1 = imadjust(sd,stretchlim(sd,tol),[]); figure; subplot(1,2,1); imshow(double(d)); title('original Image') subplot(1,2,2); imshow(sd); title('smothed Image'); xlabel(['kernel size = ' num2str(kernel_size(1)) 'x' num2str(kernel_size(2)) ' with sigma = ' num2str(sigma)]) [r c]=ind2sub([ys Xs], L); deldx=zeros(ys,xs); deldy=zeros(ys,xs); [X Y]=meshgrid(1:1:Xs,1:1:Ys); for i=1:1:ys for j=1:1:xs if SD(i,j) ~= 0 deldx(i,j)=-1*(c(i,j)-j)/sd(i,j); deldy(i,j)=-1*(r(i,j)-i)/sd(i,j);
14/19 figure; contour(x,y,sd1), hold on, quiver(x,y,deldx,deldy); FluxP=zeros(Ys,Xs); FluxP=divergence(X,Y,delDx,delDy); figure; surface(fluxp); Result=255.*ones(Ys,Xs); for i=1:ys for j=1:xs if I(i,j)==0 Result(i,j)=128; for i=1:ys for j=1:xs if FluxP(i,j) < -0.3 Result(i,j)=0; else %do nothing figure; imshow(uint8(result)); wewewe
15/19 % Proposition 1 A 2D digital point P can be removed if and % only if its 3x3 neighborhood graph, with cycles of length 3 % removed, is a tree. % % A straightforward way of determining whether or not a % graph is a tree is to check that its Euler characteristic % V - E (the number of vertices minus the number of % edges) is identical to 1. % Proposition 2 A point P could be an point of a 1 pixel % thick digital curve if, in a 3x3 neighborhood, it has a single % neighbor, or it has two neighbors, both of which are 4-adjacent % to one another. %Algorithm % For each border point P % if (P is removable) % insert(p, Heap) with FluxP Heap = struct('flux',0,'y',0,'x',0); flag=0; for i=2:ys-1 for j=2:xs-1 if Result(i,j)==0 if (Result(i-1,j-1)==1) (Result(i-1,j)==255) (Result(i-1,j+1)==255) (Result(i,j-1)==255) (Result(i,j+1)==255) (Result(i+1,j-1)==255) (Result(i+1,j)==255) (Result(i+1,j+1)==255) temp=result(i-1:i+1,j-1:j+1); eul=bweuler(temp,8); if eul==1 if flag==0 Heap_size=size(Heap); Heap(1,Heap_size(2)).flux=FluxP(i,j);
16/19 Heap(1,Heap_size(2)).y=i; Heap(1,Heap_size(2)).x=j; flag=1; else Heap_size=size(Heap); Heap(1,Heap_size(2)+1).flux=FluxP(i,j); Heap(1,Heap_size(2)+1).y=i; Heap(1,Heap_size(2)+1).x=j; % Proposition 2 A point P could be an point of a 1 pixel thick digital % curve if, in a 3x3 neighborhood, it has a single neighbor, or it has two % neighbors, both of which are 4-adjacent to one another. T = -0.5; offset=[-1-1; -1 0; -1 1; 0-1;0 1; 1-1; 1 0; 1 1]; skeletal=struct('y',0,'x',0); count=0; while size(heap,2) > 1 [a,b]=max([heap.flux]); Ycoord=[Heap(b).y]; Xcoord=[Heap(b).x]; if (Result(Ycoord-1,Xcoord-1)==255) (Result(Ycoord-1,Xcoord)==255) (Result(Ycoord-1,Xcoord+1)==255) (Result(Ycoord,Xcoord-1)==255) (Result(Ycoord,Xcoord+1)==255) (Result(Ycoord+1,Xcoord-1)==255) (Result(Ycoord+1,Xcoord)==255) (Result(Ycoord+1,Xcoord+1)==255) temp=result(ycoord-1:ycoord+1,xcoord-1:xcoord+1); eul=bweuler(temp,8); if eul==1 if ~ispoint(ycoord,xcoord,result) FluxP(Ycoord,Xcoord) > T
17/19 tempheap=[heap(1,1:b-1) Heap(1,b+1:size(Heap,2))]; Heap=tempHeap; for i=1:8 temp=result(ycoord+offset(i,1)- 1:Ycoord+offset(i,1)+1,Xcoord+offset(i,1)-1:Xcoord+offset(i,2)+1); eul=bweuler(temp,8); if eul==1 Heap_size=size(Heap); Heap(1,Heap_size(2)+1).flux=FluxP(Ycoord+offset(i,1),Xcoord+offset(i,1)); Heap(1,Heap_size(2)+1).y=Ycoord+offset(i,1); Heap(1,Heap_size(2)+1).x=Xcoord+offset(i,1); else skeletalsize=size(skeletal); skeletal(1,skeletalsize(2)+1).y=ycoord; skeletal(1,skeletalsize(2)+1).x=xcoord; tempheap=[heap(1,1:b-1) Heap(1,b+1:size(Heap,2))]; Heap=tempHeap; skeletalresult=255.*ones(ys,xs); for i=2:size(skeletal,2) tempx=[skeletal(i).x]; tempy=[skeletal(i).y]; skeletalresult(tempy,tempx)=0; figure; imshow(uint8(skeletalresult));
18/19 -Sub routine function eval=ispoint(y,x,im) % Proposition 2 A point P could be an point of a 1 pixel thick digital % curve if, in a 3x3 neighborhood, it has a single neighbor, or it has two % neighbors, both of which are 4-adjacent to one another. eval=0; offset=[-1-1; -1 0; -1 1; 0-1;0 1; 1-1; 1 0; 1 1]; sum=0; for i=1:8 temp=im(y+offset(i,1),x+offset(i,2)); sum=sum+temp; if sum==255*1 eval=1; if sum==255*2 j=1; for i=1:8 temp=im(y+offset(i,1),x+offset(i,2)); tempx(j)=x+offset(i,2); tempy(j)=y+offset(i,1); j=j+1; if (abs(tempx(1)-tempx(2))+abs(tempy(1)-tempy(2)))==1 eval=1; return;
19/19 Contribution Distribution Chung, In Young Lee, Kang Eui Literature Review 40% 60% Testing 50% 50% Matlab Programming 40% 60% Report Composition 50% 50% Project Presentation 70% 30% Overall 50% 50% Chung, In Young Signature Date Lee, Kang Eui Signature Date