12. Segmentation. Computer Engineering, i Sejong University. Dongil Han

Transcription

1 Computer Vson 1. Segmentaton Computer Engneerng, Sejong Unversty Dongl Han Image Segmentaton t Image segmentaton Subdvdes an mage nto ts consttuent regons or objects - After an mage has been segmented, the aggregate of segmented pxels s represented and descrbed and recognzed. One of the most dffcult tasks n mage processng - Industral nspecton applcaton : control the envronment for easy segmentaton - If control s mpossble : focus on selectng the type of sensor - Infrared magng to detect heat sgnatures n mltary applcatons Segmentaton accuracy determnes the eventual success or falure of analyss procedures /59

2 Image Segmentaton t Example 3/59 Image Segmentaton t Example 4/59

3 Image Segmentaton t Segmentaton Cues Smlar Color Smlar Texture Combnaton of prevous segmentaton results Segmentaton methods Use dscontnuty - Edge based segmentaton Use smlarty - Based on thresholdng, regon growng, regon splttng, regon mergng g 5/59 Image Segmentaton t Dscontnuty based segmentaton Pont detecton Lne detecton Edge detecton - Gradent operator - Laplacan operator => Stage for connectng edge segments(edge lnkng) s requred 6/59

4 Image Segmentaton t Edge lnkng and Boundary detecton Edge detecton methods generate spurous ntensty dscontnutes - Because of nose, nonunform llumnaton, and other effects edge detecton algorthms - Gradent operators : Roberts, Prewtt, Sobel, etc. - Laplacan, LoG Edge detecton algorthms are followed by edge lnkng procedures Edge lnkng methods Local processng Hough transform Graph-theoretc technque 7/59 Edge lnkng Local processng Analyze the characterstcs of pxels n a small neghborhood All ponts that are smlar accordng to a set of predefned crtera are lnked Usually 3x3, 5x5 wndow s used Two prncpal propertes foe establshng smlarty of edge pxel - Strength of the response of the gradent operator - Drecton of the gradent vector 8/59

5 Gradent operator Edge lnkng The gradent of an mage f(x,y) at locaton (x,y) s defned as the vector f = G x G y The magntude of ths vector = f f x f y f = mag( f ) = G x + G y or G x + Gy Lt Let α(x,y) ( ) represent tthe drecton angle of fthe vector at t( (x,y) G α ( x, y ) = tan 1 ( G 9/59 y x ) Edge detecton and lnkng Edge lnkng A pont n the predefned neghborhood of (x, y) s lnked to the pxel at (x, y) f the followng two crtera are satsfed f ( x, y) f ( x, y ) 0 0 α ( x, y ) α ( x, y ) < 0 0 A E E, A : nonnegatve thresholds Ths process s repeated at every locaton n the mage 10 /59

6 Edge lnkng 11 /59 Edge lnkng Local processng vs. Global processng Local processng - Smple and fast - Edge lnkng results depend on the sze of wndow - Edge lnkng results depend on the selecton of threshold value - Senstve to nose Global processng - Complex and slow - Exact and robust to nose - Popular technques 1. Hough transform. Graph-theoretc technque 1 /59

7 Hough Transform Concept Global edge detecton n the parameter space - Consder a pont (x,y ) - General equaton of a straght lne n slope-ntercept form y = ax + b - Infntely many lnes pass through (x, y ) for varyng values of a and b - If we consderng the ab-plane b = x a + y - Here, ab-plane s called parameter space 13 /59 Hough Transform Concept All ponts contaned on one lne have lnes n parameter space that ntersect at the same pont (a, b ) 14 /59

8 Hough Transform Implementaton Subdvde the parameter space nto accumulator cells - Expected ranges of slope and ntercept values (a max, a mn ), (b max, b mn ) - Determne the resoluton of accumulator cells A(, j) - Intally accumulator cells are set to zero - For every pont (x k, y k ) n the mage plane, ncrease the accumulator cells whch satsfy the followng equatons b = x a + k y k - The value of each accumulator cell A(, j) presents the number of ponts lyng on the lne y = a x + b j n xy-plane 15 /59 Hough Transform Accumulator cell example 16 /59

9 Hough Transform Selecton of Parameter space ab-plane - The slope approaches nfnty as the lne approaches the vertcal - The sze of accumulator cell s unequal ρθ-planeρ p ρ : the smallest dstance between the lne and orgn θ : the angle of the locus vector from the orgn to ths closest pont x cosθ + y snθ = ρ - Loc are snusodal curves n the ρθ-plane - Wdely used 17 /59 Hough Transform accumulator cell example wth ρθ-plane 18 /59

10 Hough Transform Hough Transform Example 19 /59 Hough Transform Edge lnkng wth Hough Transform Compute the gradent of an mage Threshold h t to obtan a bnary mage Specfy subdvsons n the ρθ-plane Hough transform for edge pxels Examne the counts of the accumulator cells for hgh pxel concentratons Examne the relatonshp between pxels n a chosen cell 0 /59

11 Hough Transform Lne detecton wth Hough Transform 1 /59 Hough Transform Generalzaton Smlar transform can be used for fndng any shape whch can be represented by a set of parameters Possble form of Generalzed Hough transform g( v, c) = 0 Here, v: coordnate vector, c: parameter vector Example : detecton of crcle ( x c = c 1 ) + ( y c) 3-dmensonal parameter space s used 3 /59

12 Graph-Theoretc Technque Global processng va graph-theoretc technque Represents edge segments n the form of graph and searchng the graph for edge lnkng Low cost path => sgnfcant edge Robust to nose Procedure s consderably complcated Requres more processng tme 3 /59 Graph-Theoretc Technque Edge element edge element : the boundary between two pxels p and q If we let xy-coordnate d a e of pont p and dqq be (x p,y p ), (x q,y q ), the edge element s defned by ( x, y )( x, y ) ( p p q q Edge : a sequence of connected elements => requres the decson of edge element and edge lnkng process 4 /59

13 Graph-Theoretc Technque Detecton of edge element The cost functon defned by pxels p and q c ( p, q ) = H [ f ( p ) f ( q )] H : Hghest gray-level value n the mage f(p), f(q) : the gray-level lvalues of p and q 5 /59 Graph-Theoretc Technque Lnkng of Edge element Edge lnkng : Fndng a mnmum cost path 6 /59

14 Graph-Theoretc Technque Nosy mage and edge boundary 7 /59 Graph-Theoretc Technque Nosy mage and edge boundary 8 /59

15 Image Segmentaton t Smlarty based segmentaton Thresholdng : smple and wdely used - Global thresholdng - Adaptve thresholdng - Local thresholdng Regon based segmentaton : fndng the regons drectly - Regon Growng : group pxels or subregons nto larger regons based on predefned df dcrtera - Regon splttng and mergng : subdvde an mage ntally nto a set of arbtrary, dsjonted regons, and then merge/splt the regons to satsfy the predefned condtons 9 /59 Concept Thresholdng Sngle threshold : a threshold T separates and extract the objects from the background f f(x,y) > T then object pont else background pont Multple threshold : regon growng s more proper algorthm f T1 < f(x,y) <= T then object1 else f f(x,y) > T then object else background 30 /59

16 General form of threshold value T Thresholdng T = T [ x, y, p ( x, y ), f ( x, y )] f(x, y) : gray level of pont (x, y) p(x, y) : some local property of pont (x, y) ex) average gray level of a neghborhood centered on (x, y) A thresholded mage g(x,y) s sdefned edas g ( x, y ) = 1 0 f f ( x, y ) > T f f ( x, y) T 31 /59 Thresholdng Global thresholdng T = T [ f ( x, y )] Local thresholdng T = T[ p( x, y), f ( x, y)] Dynamc thresholdng, adaptve thresholdng T = T[ x, y, p( x, y), f ( x, y)] 3 /59

17 Thresholdng Role of llumnaton 33 /59 Global thresholdng Thresholdng Hghly controlled envronment s requred : ndustral nspecton applcatons Smplest Approach 34 /59

18 Thresholdng Selecton of global thresholdng value T heurstc approach automatc approach automatc approach 1. Select an ntal estmate for T. Segment the mage usng T. Ths wll produce two groups of pxels : pxels n G 1 > T, pxels n G T 3. Compute the average gray level values,,μμ 1, μ for the pxels n regons G 1, G 4. Compute the new threshold value : 1 T = ( μ + 1 μ 5. Repeat steps through 4 untl the dfference n T n successve teratons s smaller than a predefned parameter T 0 ) 35 /59 Thresholdng automatc thresholdng result 36 /59

19 Thresholdng Adaptve thresholdng Uneven llumnaton : global threshold can not segment perfectly Dvde orgnal mage nto submages and utlze dfferent threshold to segment each submage 37 /59 Adaptve thresholdng Thresholdng Faled submage further subdvded nto much smaller submages - adaptve 38 /59

20 Thresholdng Optmal thresholdng a method for estmatng thresholds that produce the mnmum average segmentaton error. 39 /59 Optmal Thresholdng Assumpton : an mage contans only two prncpal p gray-level regons => Determne a optmal threshold(n terms of mnmum error) for segmentng the mage nto the two dstnct regons Prelmnary z: gray level value => random quantty Image hstogram can be consdered an estmate of ther probablty densty functon p(z) Ths overall densty functon s the sum of two denstes : lght and dark regons of mage If the form of densty s known or assumed, t s possble to determne an optmal threshold 40 /59

21 Optmal Thresholdng Optmal thresholdng Assume that the overall densty functon s the sum of two densty functons p ( z) = P1 p1( z) + P p( z) Here, p 1 (z), p (z) are probablty densty functons of object and background P 1, P are the probabltes of occurrence, respectvely and we can get P + P 1 = 1 If probablty densty functons(pdf) s Gaussan densty functon p( z) = P1 e πσ 1 ( z μ 1) / σ1 ( z μ ) / + P e πσ σ 41 /59 Optmal Thresholdng Optmal thresholdng The probablty of erroneously classfyng a background as an object, E 1 (T) T E 1( T ) = T p( z) dz The probablty of erroneously classfyng a object as an background, E (T) E ( T ) = p1 ( z ) dz T The overall probablty of error E(T) E ( T ) = P E ( T ) + PE ( T ) ( 1 1 T 4 /59

22 Optmal thresholdng Optmal Thresholdng Mnmum error : dfferentatng E(T) wth respect T and equatng the result to 0 P 1 p1( T ) = P p( T ) If P 1 = P, the optmal threshold h s where the curves p 1 () (z) and p () (z) ntersect t If PDF of p 1 (z) and p (z) are Gaussan densty functon AT + BT + C = 0 where 1 A = σ σ B = ( μ σ 1 μ σ 1 ) C = σ 1μ σ μ1 + σ 1 σ l( ln( σ P1 / σ1 P ) 43 /59 Optmal thresholdng Optmal Thresholdng Quadratc equaton : two optmal soluton s possble If the varances are equal BT + C = 0 We can get sngle threshold T T = μ1 + μ σ P + ln( ) μ μ P 1 P1 If P 1 = P,or σ = 0, the optmal threshold h s the average of the means 44 /59

23 Optmal Thresholdng Estmaton of densty functon and parameters Instead of assumng a functon form, estmaton of exact functon form and parameters(average, standard devaton, etc) gves better results Common approach : mnmum mean square error approach e ms = 1 n n = 1 [ p( z ) h( z )] h(z ) : mage hstogram p(z ) : estmated densty functon 45 /59 Optmal Thresholdng Example Outlne automatcally the boundares of heart ventrcles n cardoangograms ( 심장혈관조영도의좌심실윤곽결정예 ) 46 /59

24 Optmal Thresholdng Example Outlne automatcally the boundares of heart ventrcles n cardoangograms Image s subdvded nto 7x7=49 regons Unmodal hstograms are rejected Remanng hstograms are ftted by gaussan densty curves and optmal thresholdng s appled for extractng the boundary 47 /59 Local Thresholdng Requrements for obtanng ggood threshold Hstogram peaks are tall and narrow Hstogram s symmetrc Hstogram s separated by deep valleys Consder only pxels that le on/near the edges between objects and background Hstogram s less dependent on the relatve sze of objects/background Resultng hstogram have peaks of approxmately the same heght/probablty => Satsfy f the requrements for obtanng good threshold hld Use of gradent/laplacan operators Produces the hghly desrable deep valleys 48 /59

25 Determnaton of edge regon Local Thresholdng 0 f f < T s ( x, y ) = + f f T and f 0 f f T and f < 0 49 /59 Local Thresholdng Segmented mage by local thresholdng 50 /59

26 Smlarty based segmentaton Image Segmentaton t Thresholdng : smple and wdely used - Global thresholdng - Adaptve thresholdng - Local thresholdng Regon based segmentaton : fndng the regons drectly - Regon Growng : group pxels or subregons nto larger regons based on predefned df dcrtera - Regon splttng and mergng : subdvde an mage ntally nto a set of arbtrary, dsjonted regons, and then merge/splt the regons to satsfy the predefned condtons 51/46 /59 Basc formulaton Regon-Based Segmentaton t Let R represent the entre mage regon. Segmentaton s a process that parttons R nto n subregons, R 1, R,,R R n, such that n ( a) U R ( b) ( c) ( d ) ( e) = 1 R R s = R R a connected j = for P ( R ) = TRUE P( R R j ) = all regon, = 1,,..., n and j, for = 1,,..., n FALSE for j j Here, P(R ) s a logcal predcate defned over the ponts n set R 5 /59

27 Regon Growng A procedure that groups pxels or subregons nto larger regons based on predefned crtera Several topcs Sl Selectng a set of seed ponts Selecton of smlarty crtera and predcate for regon growng Use of connectvty, adjacency nformaton Formulaton of stoppng rule Regon-based approach s more effectve f the hstogram of mage s complex 53 /59 Regon Growng X-ray mage of weld contanng several cracks 54 /59

28 Regon Growng Hstogram of X-ray mage 55 /59 Regon Splttng and Mergng Subdvde an mage ntally nto a set of dsjonted regons and then merge and/or splt the regons n an attempt to satsfy the predefned condtons Splttng technque example : quadtree 56 /59

29 Regon Splttng and Mergng Splttng and merge process Splt nto four dsjont quadrants any regon R for whch hp(r )= Fl False Merge any adjacent regons R j and R k for whch P(R j R k ) = True Stop when no further mergng or splttng s possble 57 /59 Regon Splttng and Mergng Splttng and merge example P(R ) = True f at least 80% of the pxels n R have the followng property z j m σ z j : gray level of the jth pxel n R m : mean gray level lof regon R σ : standard devaton of the gray levels n regon R 58 /59

30 Summary Image segmentaton s an essental prelmnary step n most automatc pattern recognton and scene analyss problems Choce of one segmentaton e technque over another s dctated mostly by the pecular characterstcs of the problem beng consdered Gradent operator, Edge lnkng, Hough transform, Thresholdng technques, Regon growng : wdely used dtechnques n real applcatons 59/59