Geometric-Edge Random Graph Model for Image Representation

Transcription

1 Geometric-Edge Random Graph Model for Image Representation Bo JIANG, Jin TANG, Bin LUO CVPR Group, Anhui University /1/11, Beijing Acknowledgements This research is supported by the National Natural Science Foundation of China (No )

2 Content Random geometric graph Generalized edge random graph G-E random graph (GERG) model Image representation based on GERG Random Dot Product Graph (DPRG) for GERG Image matching Image recognition

3 Spatial domain image modeling Polygon, skeleton, chain code, run length code, pyramid, and graphs Advantage of graph model : Image matching and recognition graph matching and recognition Capable of describing structural relationships between image units Exist solid problem solving schemes from mathematics.

4 Problems of image graph model Imperfect image segmentation or key point extraction Extra and missing nodes, and edge structure may not be stable Random graph model

5 Tradition graph based image representation Delaunay Graph/ K-NN graph/ Geometric graph MST Region adjacent graph Skeleton graph/ Shock graph ARG

6 Tradition graph based image representation Delaunay Graph KNN Graph

7 Tradition graph based image representation Geometric Graph Skeleton Graph

8 Motivation Geometric graph - can be used to extract structure information of an image Edge stability problem Two geometric graphs are generated based on two point-sets (the right image is obtained by adding Gaussian noise to the point positions of the points in the left image). The red edges denote the structure variation between two graphs

9 Geometric graph Let. be some norm (such as Euclidean norm) on R d, and let r be some positive parameters. Given finite sets X, Y R d, A geometric graph G(X; r) is an undirected graph with vertex set X and with undirected edges connecting all pairs {X, Y} if they satisfy Y-X r.

10 Random geometric graph Let X=(X 1, X X n ) be iid d-dim variables with common density f G(X; r) = {G(Y; r) Y belongs to X} Property: 1. Given X, the parameter r (contains the relation between graph nodes) determines the graph structure. The randomness lies in the structures attached to the vertices (vertex random graph); E-R random graph: p is a constant matrix

11 Generalized edge random graph (GERG) Given vertex set V={1, n} and a symmetric probability function p: [n] [n] [,1] The edge probability: p ij = p (i, j) The probability measure P p (G) on G n Pp G p(, i j) 1 p(, i j) i j, ij E i j, ij E

12 Generalized edge random graph (GERG) Properties 1. The randomness lies in the structures attached to the edges. The edges are independent 3. GERG can be represented by a symmetric matrix (we can consider the algebraic method, Spectral method)

13 G-E Random graph Main idea Random Geometric graph + GERG Structure Algebraic information Representation

14 G-E Random graph Given vertex set V={1, n} and X = (X 1, X X n ) for each node, We determine the symmetric function p G-E as p G-E (;, ri j) f( u) d where u ij = X i -X j and f ij is the pdf of random variable u ij. r ij ij u ij

15 G-E Random graph Properties 1. Edge random graph. Convey the vertex randomness to the edges 3. Way to represent structure information of node random variables

16 Measurement Expected average degree and maximum degree n n n 1 E( k ) p( i, j) E( k ) max p( i, j) a n i 1 j 1 Expected number of edges Dynamic evolution n E( n ) ( i, j) e n p i 1 j 1 G (p G-E, r n ) a i j 1

17 GERG image representation Noncentral chi-square distribution Y N i k i ~ ( i, i ),( 1,..., ) The pdf of chi-square distribution: Y k /4 1/ 1 ( x )/ x Y k / 1 k i ( ) ~ ( k, ) i 1 i f ( xk ;, ) e I ( x) I ( y) ( y/) a a j j ( y /4) j! ( a j 1)

18 GERG image representation The cdf of chi-square distribution: ( /) Pxk e Qxk j j! j / ( ;, ) ( ; ) j ( k /, x/) Qxk ( ; ) ( k /) ( k, z) is the low incomplete Gamma function

19 GERG image representation Graph nodes: 1. key points of the image. For node i, a random attribute vector y i = (y i1, y i ) is allocated 3. Assume y ~, ij N xij

20 GERG image representation Graph edge probability: p d ~ k G-E (; rkl,) Pr ( ;, ) 1 xk1 xl1 xk xl

21 Application: Image matching Initial matching embedding followed by alignment Idea: given edge probability from the GERG, find the best n fitting feature vectors xi attached to each graph node, i 1 which is achieved by Random Dot Product Graph, traditional graph matching method like Hungarian or SVD could be used to find the matches. Ref. Jin Tang, Bo Jiang, Bin Luo, Graph matching based on dot product representation of graphs, Proceedings of the 8th IAPR-TC-15 International Workshop on Graph- based representations in Pattern Recognition(GBRPR11), 11,

22 Application: Image matching Consistent check To remove initial mis-matching, and leave most positive correspondences. Idea: Enforce coherent spatial relationships of corresponding nodes between graphs.

23 Application: Image matching Association graph

24 Application: Image matching Co-embedding We use the positive correspondences obtained from the above consistency check process and integrate the embedding processes for G1 and G at the same time. Missing value or unmatched nodes can be recovered by, T min f (X) X X (X X) X T P, M PAG M AG where X [ X, X, X ], M is the missing data label matrix. I I I 1

25 Application: Image matching CMU data Ref. Bin Luo, E.R. Hancock, Structural graph matching using the EM algorithm and singular value decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.3, No.1, 1,

26 Application: Image matching CMU (Compare with EM SVD) Images Points methods Correct False No correspondences correspondences correspondences EM house1 3 DPRG Our method EM 9 1 house 3 DPRG (5 degree) Our method house3 (1 degree) 3 house4 (15 degree) 31 house5 ( degree) 3 house6 (5 degree) 3 EM DPRG Our method 8 EM 3 5 DPRG 5 3 Our method 7 1 EM DPRG 4 4 Our method 8 EM DPRG Our method 5 4 1

27 Application: Image matching York data

28 Application: Image recognition Flow chart Image 1 Image Image Image n Harris Corner detector Corner Points G-E Random graph G-E random graph feature extraction G-ECRI 1 G-ECRI G-ECRI n Similarity of images Euclidean Distance

29 Application: Image recognition Image data Ref. Bin Luo, Richard C.Wilson, Edwin R. Hancock. Spectral embedding of graphs[j]. Pattern Recognition,3,36:13 ~ 3

30 Application: Image recognition Clustering Graph Index Graph Index 1 Second eigenvector MDS Second eigenvector PCA (a) Distance matrix (b) MDS embedding (c) PCA embedding Fig. * Expected average degree clustering result

31 Application: Image recognition Clustering Graph Index Graph Index 1 Second eigenvector PCA Second eigenvector MDS (a) Distance matrix (b) MDS embedding (c) PCA embedding Fig.* Expected edge number clustering result

32 Application: Image recognition Synthetic house model sequence

33 Application: Image recognition Object view structure Third eigenvector Second eigenvector Third eigenvector Second eigenvector Second eigenvector (a) PCA embedding (b) MDS embedding (c) Laplacian embedding Fig. * Expected average edge number based embedding for the model-house.1

34 Application: Image recognition Object view structure Third eigenvector Second eigenvector Third eigenvector Third eigenvector Second eigenvector Second eigenvector (a) PCA embedding (b) MDS embedding (c) Laplacian embedding Fig. * Expect edge number based embedding for model-house

35 Application: Image recognition Object view structure Third eigenvector Second eigenvector Third eigenvector Second eigenvector Third eigenvector Second eigenvector (1) PCA embedding () MDS embedding (3) Laplacian embedding Fig. * Expert average degree based embedding for York-House

36 Application: Image recognition Object view structure Third eigenvector Second eigenvector Third eigenvector Second eigenvector Third eigenvector Second eigenvector (1) PCA embedding () MDS embedding (3) Laplacian embedding Fig. * Expect edge number based embedding for York-House

37 Application: Image recognition Object view structure Third eigenvector Second eigenvector Third eigenvector Second eigenvector Third eigenvector Second eigenv (1) PCA embedding () MDS embedding (3) Laplacian embedding Fig. * Expert average degree embedding for MOVI-House

38 Application: Image recognition Object view structure Third eigenvector Second eigenvector - -5 Third eigenvector Second eigenvector Third eigenvector Second eigenvector (1) PCA embedding () MDS embedding (3) Laplacian embedding Fig. 15 Expect edge number embedding for MOVI-House

39 Conclusion Contribution 1: We propose a Geometric-Edge (G-E) Random Graph Model for image representation Contribution : We cast image matching into G-E Random Graph matching by using the random dot product graph based matching algorithm The proposed method is more robust to graph node position disturbance. When used for graph matching, it has been shown the advantages of correct matching rate.

40 Potential applications of the proposed model Video analysis (structured data) Motion analysis (graph evolution)?

41 Thank you!