Image Content Representation

Transcription

1 Image Conen Represenaion Represenaion for curves and shapes regions relaionships beween regions E.G.M. Perakis Image Represenaion & Recogniion 1

2 Reliable Represenaion Uniqueness: mus uniquely specify an image oherwise dissimilar images migh have similar represenaions Invariance: mus be invarian o viewing condiions: ranslaion, roaion, scale invarian, viewing angle changes and symmeric ransformaions Efficiency: compuaionally efficien E.G.M. Perakis Image Represenaion & Recogniion 2

3 More Crieria Robusness: resisance o moderae amouns of deformaion and noise disorions and noise should a leas resul in variaions in he represenaions of similar magniude Scalabiliy: mus conain informaion abou he image a many levels of deail images are deemed similar even if hey appear a differen view-scales (resoluion) E.G.M. Perakis Image Represenaion & Recogniion 3

4 Image Recogniion Mach he represenaion of an unknown image wih represenaions compued from known images (model based recogniion) Maching akes in wo represenaions and compues heir disance he more similar he images are he lower he heir disance is (and he reverse) E.G.M. Perakis Image Represenaion & Recogniion 4

5 Shape Represenaion The shapes are represened by conours or curves Conour: a linked lis of edge pixels open or closed region boundaries much space overhead Curve: he mahemaical model of a conour polygons splines E.G.M. Perakis Image Represenaion & Recogniion 5

6 Shape Maching The algorihm akes in wo shapes and compues heir disance he correspondences beween similar pars in he shapes E.G.M. Perakis Image Represenaion & Recogniion 6

7 Polylines Sequence of line segmens approximaing a conour joined end o end Polyline: P={(x 1,y 1 ),(x 2,y 2 ),.,(x n,y n )} for closed conours x 1 =x n, x n =y n Verices: poins where line segmens are joined E.G.M. Perakis Image Represenaion & Recogniion 7

8 Polyline Compuaion Sar wih a line segmen beween he end poins find he furhes poin from he line if is disance > T a new verex is insered a he poin repea he same for he wo curve pieces E.G.M. Perakis Image Represenaion & Recogniion 8

9 Hop along algorihm Works wih liss of k pixels a a ime and adjuss heir line approximaion 1. ake he firs k pixels 2. fi a line segmen beween hem 3. compue heir disances from he line 4. if a disance > T ake k < k pixels and go o sep 2 5. if he orienaion of he curren and previous line segmens are similar, merge hem o one 6. advance k pixels and go o sep 2 E.G.M. Perakis Image Represenaion & Recogniion 9

10 B-Splines guided polygon spline Piecewise polynomial curves beer aesheic represenaion analyic properies coninuiy of firs and second derivaives E.G.M. Perakis Image Represenaion & Recogniion 10

11 Types of Splines n=2: quadraic spline n=1: linear spline n=3: cubic spline The spline varies less han he guided polygon The spline lies beween he convex hull of groups of n+1 consecuive poins n is he degree of he inerpolaive polynomial cubic splines are frequenly used E.G.M. Perakis Image Represenaion & Recogniion 11

12 Spline Inerpolaion The inerpolan hough a se of poins x i,i=1,2, n is a vecor piecewise polynomial funcion x() n 1 + x( ) = i= 0 vc ( ) v i s are he verices of he guided polygon here are n+2 coefficiens he addiional 2 coefficiens are deermined from boundary condiions by seing curvaure = 0 a end poins we ge v 1 =(v 0 +v 2 )/2, v n =(v n-1 +v n+1 )/2 i i E.G.M. Perakis Image Represenaion & Recogniion 12

13 x( ) n 1 = + = i 0 vc ( ) i i E.G.M. Perakis Image Represenaion & Recogniion 13

14 E.G.M. Perakis Image Represenaion & Recogniion 14 Spline Base Funcions ) ( ) ( ) ( 6 ) ( = = + = = = C C C C x() in (i,i+1) is compued as C i (s): piecewise cubic polynomials ) ( ) ( ) ( 1) ( ) ( i C v i C v i v C C v x i i i i = + +

15 Chain Codes Recording of he change of direcion along a conour (4 or 8 direcions) sar a he firs edge and go clockwise he derivaive of he chain code is a roaion invarian E.G.M. Perakis Image Represenaion & Recogniion 15

16 Commens Independency of saring poin is achieved by roaing he code unil he sequence of codes forms he minimum possible ineger Sensiive o noise and scale Exension: approximae he boundary using chain codes (l i,a i ), (Tsai & Yu, IEE PAMI 7(4): , 1985) l i : lengh a i : angle difference beween consecuive vecors E.G.M. Perakis Image Represenaion & Recogniion 16

17 Maching Chain Codes Ediing disance D(A,B) D(A,B): minimum cos o ransform A o B operaions: inser, delee, change coss: cos of changing o is 2 cos of changing o is 1 cos of deleion is 2 cos of inserion is 2 E.G.M. Perakis Image Represenaion & Recogniion 17

18 a a a a a a b b c A c d d b B c c c d b b C c d d A: aabbccdd B: aaabcccd C: a bbc dd C: a bbb dd D(A,C) = 4 D(B,C) = 4 E.G.M. Perakis Image Represenaion & Recogniion 18

19 Maching Algorihm Compue D(A,B) #A, #B lenghs of A, B 0: null symbol R: cos of an edi operaion D(0,0) = 0 for i = 0 o #A: D(i:0) = D(i-1,0) + R(A[i] 0); for j = 0 o #B: D(0:j) = D(0,j-1) + R(0 B[j]); for i = 0 o #A for j = 0 o #B { 1. m 1 = D(i,j-1) + R(0 B[j]); 2. m 2 = D(i-1,j) + R(A[i] 0); 3. m 3 = D(i-1,j-1) + R(A[i] B[j]); 4. D(i,j) = min{m 1, m 2, m 3 }; } E.G.M. Perakis Image Represenaion & Recogniion 19

20 i j a b b c d d a a a b c c c d iniializaion cos oal cos E.G.M. Perakis Image Represenaion & Recogniion 20

21 Ψ-s Slope Represenaion Plo of angen Ψ versus arc lengh s Ψ-s is a represenaion of he shape of a conour line segmens horizonal line segmens in Ψ-s circular arcs oher line segmens in Ψ-s use Ψ-s o segmen a curve ino lines and arcs for a closed conour Ψ-s is periodic Ψ-s is ranslaion and scale invarian he derivaive of Ψ-s is also roaion invarian E.G.M. Perakis Image Represenaion & Recogniion 21

22 Ψ-s Examples From Ballard and Brown 84 a) riangular curve b) regions of high curvaure c) resulan segmenaion E.G.M. Perakis Image Represenaion & Recogniion 22

23 E.G.M. Perakis Image Represenaion & Recogniion 23 Fourier Represenaion Fourier ransform of conour represenaion u(n) = x(n) + j y(n), n = 0,1,2,N-1 or u(n) = Ψ(n) 2πn/L (subracs rising componen) for closed curves u(n) is periodic N kn j N n N k N kn j e n u N k a N n e k a n u π π ) ( 1 ) ( 1,0 ) ( ) ( = = = =

24 Fourier Descripors (FDs) The complex coefficiens a(k) are called Fourier Descripors (FD) of he boundary use he lower order FD s if only he firs M coefficiens are used M u( n) = 1 k= 0 a( k) e j2πkn N,0 u(n) is an approximaion of u(n) he approximaion depends on M 1 Shape disance: disance beween vecors of FD s E.G.M. Perakis Image Represenaion & Recogniion 24 n M

25 FDs and Invariance Simple geomeric ransformaions: ranslaion : u(n) + a(k) + δ(κ) roaion : u(n)e jθ a(k)e jθ scaling : su(n) sa(k) saring poin : u(n - ) a(k) e j2πk/n FDs of Ψ-s: invarian FDs of (x(n),y(n)): see Wallace & Winz Efficien 3-D Aircraf Recogniion using FDs, CGIP, 13:99-126, 1980 E.G.M. Perakis Image Represenaion & Recogniion 25

26 FDs and Occluded Shapes FDs of derivaive of Ψ-s: roaion invariance Ignoring a(0): ranslaion invariance Normalizing all FD s by a(1) : scale invariance Vecor of M coefficiens saring from a(2) E.G.M. Perakis Image Represenaion & Recogniion 26

27 Momen Invarians [Hu 62] An objec is represened by is binary image R A se of 7 feaures can be defined based on cenral momens m pq = x y m, x = y = p q ( x, y) R m00 m00 p q µ pq = ( x x ) ( y y ), p,q = 0,1,2... ( x,y ) R E.G.M. Perakis Image Represenaion & Recogniion 27 m

28 Cenral Momens [Hu 62] Invarian o ranslaion and roaion Use η pq =μ pq /μ γ 00 where γ=(p+q)/2 + 1 for p+q=2,3 insead of μ s in he above formulas o achieve scale invariance E.G.M. Perakis Image Represenaion & Recogniion 28

29 Scale-Space Descripions (SSD) Shape maching using represenaions a various level of deail (resoluion) Mehod: Scale-Space Descripions and Recogniion of Planar Curves, F. Mokharian and A. Mackworh, IEEE PAMI 1986 SSDs were originally proposed by Wikin E.G.M. Perakis Image Represenaion & Recogniion 29

30 Zero Crossings SSD: represenaion of zero-crossings of he curvaure k for all possible values of σ Curve: {x(), y()}, in [0,1] Curvaure: k = dφ/d=1/ρ k = xy yx + ) 2 2 ( x y y' = x = x = E.G.M. Perakis Image Represenaion & Recogniion 30 3/ 2 ρ 2 dy d y, y' ' = 2 dx dx dx dy, y = d d 2 2 d x d y, y = 2 2 d d

31 E.G.M. Perakis Image Represenaion & Recogniion 31 Curvaure Compue k on x(),y() a various levels of deail convolve wih... ), ( ) ( ), ( 2 1 ) ( ), ( ) ( ), ( 2 1 ), ( ) ( 2 = = = = = σ σ π σ σ σ π σ σ σ σ g y y du e u x g x x e g u

32 Smoohing a Curve κ(,σ) E.G.M. Perakis Image Represenaion & Recogniion 32

33 SSD of Africa E.G.M. Perakis Image Represenaion & Recogniion 33

34 Maching SSDs Varian of A* algorihm 1. compare he wo higher curves firs 2. compare he curves included saring from he wo higher curves ec. unil all included curves are mached 3. compare he curves nex o he wo higher Some curves may be missing assign a cos for missing curves E.G.M. Perakis Image Represenaion & Recogniion 34

35 Maching Scale-Space Curves σ A ll 1 σ B ll 2 d h 2 1 lr 1 d1 h 2 lr 2 D(A,B) = h 1 - h 2 + ll 1 - ll 2 + lr 1 - lr 2 Trea ranslaion and scaling: compue (d,k) = k + d, k = h 1 /h 2, d = d 1 - d 2, σ = κσ mormalize A, B before maching Cos of maching: leas cos maching E.G.M. Perakis Image Represenaion & Recogniion 35

36 Relaional Srucures Represenaions of he relaionships beween objecs Aribued Relaional Graphs (ARGs) Semanic Nes (SNs) Proposiional Logic May include or combined wih represenaions of objecs E.G.M. Perakis Image Represenaion & Recogniion 36

37 Aribued Relaional Graphs (ARGs) Objecs correspond o nodes, relaionships beween objecs correspond o arcs beween nodes boh nodes and arcs may be labeled label ypes depend on applicaion and designer usually feaure vecors recogniion is based on graph maching which is NP-hard E.G.M. Perakis Image Represenaion & Recogniion 37

38 ARG for Cup direced graph Node feaure: compacness = area/perimeer 2 Arc feaures: bigger (area 1 /area 2 ) adjacency (percenage of common boundary), disance beween ceners of graviy E.G.M. Perakis Image Represenaion & Recogniion 38

39 ARG for Face Node feaure: perimeer l Arc feaures: relaive disance r, angle wih he horizonal a E.G.M. Perakis Image Represenaion & Recogniion 39

40 ARG for Doll Connecion graph represening he connecions beween he pieces of an objec E.G.M. Perakis Image Represenaion & Recogniion 40

41 Semanic Nes (SNs) Generalizaion of ARG SNs represen informaion a he high semanic level e.g. a represenaion of chairs around a able E.G.M. Perakis Image Represenaion & Recogniion 41

42 Hierarchical SNs (HSN) Informaion organized in ISA and PART-OF hierarchies ISA: generalizaion hierarchy, generalizaion of classes and relaionships beween insances and classes PART-OF: relaionships beween pars and whole Inheriance: lower level classes inheri properies of he higher level classes House Building Camel Mammal E.G.M. Perakis Image Represenaion & Recogniion 42

43 Example HSN op of a ransisor SN of a silicon chip ransisor silicon chip E.G.M. Perakis Image Represenaion & Recogniion 43

44 Long / Shor Term Memory Long Term Memory (LTM): schema or model represenaion of an image a high - semanic level virual classes in C++ Shor Term Memory (STM): represenaion of insances o LTM objecs insances o virual classes E.G.M. Perakis Image Represenaion & Recogniion 44

45 E.G.M. Perakis Image Represenaion & Recogniion 45

46 Proposiional Represenaions Collecion of facs and rules in informaion base new facs are deduced from exising facs ransisor(region 1 ) ransisor(region 2 ) greaer(area(region 1 ), 100.0) & less(area(region 1 ), 4.0) & is-conneced(region 1,region 2 ) & base(region 2 ) ransisor(region 2 ) E.G.M. Perakis Image Represenaion & Recogniion 46

47 Commens Pros: clear and compac expandable represenaions of image knowledge Cons: non-hierarchical no easy o rea uncerainy and incompaibiliies complexiy of maching E.G.M. Perakis Image Represenaion & Recogniion 47

48 Maching Relaional Srucures Maching beween A, B is ransformed o a graph or sub-graph isomorphism problem graph isomorphism: one o one mapping of nodes and arcs beween he srucures of A, B (NP-hard!) sub-graph isomorphism: isomorphism of a subgraph of A and B (harder!!) double sub-graph isomorphism: isomorphism of a sub-graph of A and a sub-graph of B (even harder!!!) E.G.M. Perakis Image Represenaion & Recogniion 48

49 (a), (b) are isomorphic (a), (c ) have many sub-graph isomorphisms (a), (d) have many double sub-graph isomorphisms E.G.M. Perakis Image Represenaion & Recogniion 49

50 Maching Algorihm Find all sub-graph isomorphisms Branch and bound search wih backracking a each sep expand a parial soluion a all possible direcions when search fails (a parial soluion can be expanded) backrack o an earlier parial soluion he cos of complee soluion is an upper bound o prune he expansion of non-promising soluions wih greaer cos keep he leas cos complee soluion E.G.M. Perakis Image Represenaion & Recogniion 50

51 he graph of (a) has o be mached wih he graph of (b) arcs are unlabeled differen shapes denoe differen shape properies: differen shapes canno be mached parial maches complee maches E.G.M. Perakis Image Represenaion & Recogniion 51

52 Maching Cos Cos of maching: he maching wih he leas cos cos: disance beween heir feaure vecors cos of maching nodes plus o he cos of maching arcs plus he cos of missing nodes and arcs (someimes exra or missing nodes or arcs in model graph are ignored) E.G.M. Perakis Image Represenaion & Recogniion 52

53 query model E.G.M. Perakis Image Represenaion & Recogniion 53

54 Leas Cos Maching E.G.M. Perakis Image Represenaion & Recogniion 54