Image Segmentation EEE 508

Transcription

1 Image Segmetatio Objective: to determie (etract) object boudaries. It is a process of partitioig a image ito distict regios by groupig together eighborig piels based o some predefied similarity criterio. It ca be viewed as a classificatio techique that forms regios of similarities i the image. The similarity criterio is determied usig specific properties or features of piels represetig objects i the image. Much more difficult with o-uiform backgroud as compared to uiform backgroud.

2 Image segmetatio Broad classificatio of methods: Edge-based methods The edge iformatio is used to determie boudaries of objects. The boudaries are the aalyzed ad modified to form closed regios belogig to the objects i the image. Piel-based direct classificatio methods Heuristics or estimatio methods derived from the histogram statistics of the image or clusterig algorithms are used to form closed regios belogig to the objects i the image. Regio-based methods Piels are aalyzed directly for a regio growig process based o a pre-defied similarity criterio to form closed regios belogig to the objects i the image.

3 Edge-based image segmetatio Geeral procedure: Detect edges based o first-order or secod-order gradiet iformatio. Track ad lik relevat edges based o gradiet iformatio. Edge detectio The Gradiet magitude commoly used to detect edges 3

4 Edge Detectio Oe basic tool: Gradiet operator The Gradiet magitude is commoly used for edge detectio ( ) ( ) ( ) ( ) ( ) ( ) ; g g g ( ) iput image gradiet: poits i directio of greatest chage (locally at a piel) ( ) ( ) ( ) g 4

5 Edge Detectio Questio: How do we realize (compute) the gradiet i discretedomai? Eample: Robert s Gradiet Iterpretatio: this is like a cross-wise D first backward differece. ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) g g ( ) ( ) ( ) ( ) ( ) g 5

6 Edge Detectio I geeral: g g ( ) ( ) h ( ) ( ) ( ) h ( ) ( ) g ( ) g ( ) g Robert s Gradiet h ( --) h ( - - ) () () (-) (-) 6

7 Edge Detectio Sobel s Gradiet h ( - - ) h ( - - ) (-) () (-) (-) (-) (-) () (-) () () () () h ( ) h ( ) () (-) () () () () (-) () (-) (-) (-) (-) 7

8 Edge Detectio Prewitt s Gradiet h ( - - ) h ( - - ) (-) () (-) (-) (-) (-) () (-) () () () () 8

9 Edge Detectio Note: Edge detectio methods based o computig some form of gradiet or based o computig a differece are sesitive to oise. Eample: Backgroud oise appears as isolated edge poits oise smoothig ca be applied before edge detectio (recommeded). 9

10 Edge Detectio Note: Edge detectio methods based o computig some form of gradiet or based o computig a differece are sesitive to oise. Eample: Backgroud oise appears as isolated edge poits oise smoothig ca be applied before edge detectio (recommeded). Laplacia operator Edge is detected ot oly whe g( ) is large but also zero crossigs are cosidered. Laplacia: d derivative i D case. Gradiet: d derivative i D case. Laplacia: L ( ) ( ) ( ) ( ) 0

11 Edge Detectio How do we compute Lapalcia? Several approimatios possible: approimate ad with a forward differece Similarly: ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) g g g ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) g g g ( )

12 Edge Detectio L ( ) ( ) ( ) ( ) ( ) ( ) 4( ) ( ) h( ) h( ) () () (-4) () () Edges detected at zero-crossig of L ( ) But if there is o checkig o magitude of gradiet a lot of edge poits ad false edge cotours may be geerated.

13 Edge Detectio Solutio: At zero-crossig check local variace: If large variace edge. No edge otherwise. Note: Edge detectio methods are used for detectig object/regio boudaries for segmetatio ad image aalysis. 3

14 Edge Detectio First- order gradiet operators: Sobel Robert Prewitt Cay First-derivative: Gradiet Edge is detected at the highest poit Secod-derivative: Laplacia Edge is detected at the zerocrossig poit 4

15 Cay Edge Detector iput image Horizotal Gradiet Vertical Gradiet Gradiet Magitude Ad Gradiet directio Nomaimal Suppressio Compute the high ad low thresholds Hysteresis Thresholdig edge map.calculatig horizotal gradiet G ad vertical gradiet G y at each piel locatio by covolvig with partial derivatives of a D Gaussia fuctio give as F y σ σ y) e e e σ σ y σ ( y y y σ σ σ Fy ( y) e ye e G y I Fy σ σ.computig the gradiet magitude G ad directio θ G at each piel locatio. 3.Applyig No-maimum suppressio (NMS) to thi edges. 4.Computig high ad low thresholds based o the histogram of the gradiets of the etire image. 5.Performig hysteresis thresholdig to determie the edge map. G I F 5

16 Cay Edge Detector Effect of Noise Cosider a sigle row or colum of the image Plottig itesity as a fuctio of positio gives a sigal Where is the edge? 6

17 Cay Edge Detector Solutio : Smooth first 7

18 Cay Edge Detector Solutio : First derivative of the Gaussia fuctio 8

19 Edge-based image segmetatio Boudary trackig Objective: assemble meaigful edges to form closed regios. Usually based o piel-by-piel search to fid coectivity amog edge segmets based o similarity criterio amog edge piels. Geometrical proimity ad topographical properties used to improve edge likig operatios for oisy or occluded piels. Also probabilistic approaches graphs ad rule-based methods for model-based segmetatio have bee used. 9

20 Edge-based image segmetatio Neighbor search method: Assume the edge-detectio operatio produces a edge magitude e(y) ad a edge orietatio φ(y) iformatio. List of edge piels ca be produced from gradiet image obtaied from edge detectio procedure. Assume the first edge piel is a boudary piel b j. A successor boudary piel b j ca be foud i the 4-coected or 8- coected eighborhood if the followig coditios are satisfied: e(b j ) >T ad e(b j ) >T ad e(b j )-e(b j ) <T ad φ(b j )-φ(b j ) mod π < T 3 If there is more tha oe eighborig piel satisfyig these coditios the piel that miimizes the differece is selected. Apply the algorithm recursively util all eighbors are searched. If o eighbor is satisfyig the coditios search is stopped ad procedure starts with ew edge piel. Note: this algorithm might leave may edge piels ad partial boudaries ucoected. 0

21 Edge-based image segmetatio Graph-based search method: It attempts to fid the path betwee the start ad ed odes i a costructed graph by miimizig a cost fuctio based o a distace measure ad trasitioal probabilities. A edge map with magitude ad directio iformatio A graph derived from the edge map with a miimum cost path (darker arrows) betwee the start ad ed odes.

22 Edge-based image segmetatio A edge map with magitude ad directio iformatio A graph derived from the edge map with a miimum cost path (darker arrows) betwee the start ad ed odes.

23 Piel-based direct classificatio methods Use histogram statistics to defie sigle or multiple thresholds to classify a image piel-by-piel. A simple approach: Bimodal histogram set the threshold to gray value correspodig to the deepest poit i the histogram value. Multimodal histogram partitio the image usig some heuristics the determie the threshold for each regio usig histogram of the regio. Classify every piel by comparig its gray level to the selected threshold. g ( y) 0 if if f ( y) > T f ( y) T 3

24 Piel-based direct classificatio methods Origial image Image histogram Thresholded T Thresholded T66 Thresholded T5 4

25 Piel-based direct classificatio methods Optimal global thresholdig To determie the optimal global threshold apply parametric distributio-based methods to the histogram. Assume histogram cosists of two Gaussia distributios belogig to two respective classes such as backgroud ad object. p ( z) P p ( z) P p ( z) P P ; Classificatio is doe accordig to: - class probabilities g ( y) Class Class if if f ( y) T f ( y) > T 5

26 Piel-based direct classificatio methods Defie error probabilities of misclassifyig piels: ( T ) p( z dz ad ( T ) E ) T T E p( z) dz probability to classify a class piel to class probability to classify a class piel to class ( T ) P E ( T ) P E ( T ) - the overall probability of error E For Gaussia distributed Class ad Class : p ( z) P e πσ ( zµ ) ( zµ ) σ P e πσ σ where σ i ad µ i are stadard deviatio ad mea for classes 6

27 Piel-based direct classificatio methods Determie optimal T by fidig a geeral solutio that miimizes E(T). Usig the miture distributio p(z) solutio ca be foud by solvig: where If the variaces of both classes are equal to σ : Note: If both classes are with equal likelihood T (µ µ )/. 0 C BT AT ( ) l ; ; P P C B A σ σ σ σ σ µ σ µ σ µ µ σ σ σ l P P T µ µ σ µ µ 7

28 Piel-based direct classificatio methods Piel classificatio through clusterig Clusterig is the process of groupig data poits with similar feature vectors together. Data poits that are close to each other i the feature space are clustered together. The similarity of feature vectors ca be represeted by a appropriate distace measure such as Euclidea or Mahalaobis distace. Each cluster is represeted by its mea (cetroid) ad variace (spread) associated with the distributio of the correspodig feature vectors of the data poits i the cluster. The formatio of clusters is optimized with respect to a objective fuctio ivolvig pre-specified distace ad similarity measures alog with additioal costraits such as smoothess. Clusterig may produce disjoit regios with holes postprocessig algorithm might be eeded such as regio growig ad others 8

29 Piel-based direct classificatio methods k-meas clusterig (similar to LBG for VQ) It partitios d-dimesioal data ito k clusters such that a objective fuctio providig the desired properties of the distributio of the feature vectors i terms of similarity ad distace measures is optimized. Procedure:. Select the umber of clusters k with iitial cluster cetroids v i ; i k.. Partitio iput data poits ito k clusters by assigig each data poit j to the closest cluster cetroid v i usig the selected distace measure e.g. Euclidea distace: d ij j vi where X {... } is the iput data set 9

30 Piel-based direct classificatio methods 3. Compute a cluster assigmet matri U represetig the partitio of the data poits with the biary membership value of the j-th data poit to the i-th cluster such that: U u ij where uij { 0} k i 4. Re-compute the cetroids usig the membership values as v ij j i u j ij u ij j for all i 0 < u for all i j u for all j ad for all i j ij < 5. If cluster cetroids or the assigmet matri does ot chage from the previous iteratio stop; otherwise go to Step. 30

31 Piel-based direct classificatio methods The k-meas clusterig method optimizes the sum-of-squarederror based objective fuctio: J w ( U v) k i j j v i Note: The k-meas clusterig method is quite sesitive to the iitial cluster assigmet ad the choice of the distace measure additioal criterio such as withi-cluster ad betwee-cluster variaces ca be icluded i the objective fuctio as costraits to force the algorithm to adapt the umber of clusters k. 3

32 Regio-based image segmetatio Regio-growig based segmetatio Eamie piels i the eighborhood based o a pre-defied similarity criterio. Neighborhood piels with similar properties are merged to form closed regios for segmetatio. Mergig cotiues util there is a isufficiet umber of eighborhood piels to be added i the curret regio. Two criteria eeded: A similarity criterio for iclusio of piels i the regio. A stoppig criterio for stoppig the growth usually based o the miimum umber of eighborhood piels required to satisfy the similarity criterio. 3

33 Regio-based image segmetatio Eample: Assume the piel i the ceter is the origi of the regio-growig process. Assume stoppig criterio: miimum umber of similar eighborhood piels 30% of the regio to be icluded. Iteratio : 8 piels i 3 3 eighborhood (00%) satisfy similarity criterio iclude the 8 piels. Iteratio : 9 piels i 5 5 eighborhood (56%) satisfy similarity criterio iclude the 9 piels. Iteratio 3: 6 piels i 7 7 eighborhood (5%) satisfy similarity criterio iclude oe. Stop regio-growig. Segmeted regio 33

34 Regio-based image segmetatio Eample: Origial image Segmeted regio 34

35 Regio-based image segmetatio Regio-splittig Eamie the heterogeeity of a predefied property of the etire regio i terms of its distributio ad the mea variace miimum ad maimum values. If the regio is heterogeeous split ito two or more regios. Cotiue splittig util all regios satisfy homogeeity criterio idividually. Geerated R i i regios satisfy the followig coditios:. Each regio R i i is coected.. i R i R 3. Ri R j 0 for all i j ; i j 4. H(R i ) TRUE for i 5. H(R i R j ) FALSE for i j where H( ) is a logical predicate for the homogeeity criterio. 35

36 Regio-based image segmetatio Regio-splittig methods ca also be implemeted by rule-based systems ad quadtrees. I the quadtree-based method regios are represeted by odes i the quadtree. Eample: Regios R ad R 3 satisfy homogeeity criterio. Regios R ad R 4 failed homogeeity criterio ad are further split. R R R R R 3 R 4 R R R 3 R 4 R 4 R 4 R 3 R 43 R 44 R R R 3 R 4 R 4 R 4 R 43 R 44 36