15/12/2017. Image segmentation: discontinuities. Image segmentation: discontinuities. Image segmentation: discontinuities

Transcription

1 5//07 Image segmentaton Toy problems & kds problems Image analyss: Frst step: Segmentaton,.e. subdvson of the mage nto ts consttuent parts or obects. Autonomous segmentaton s one of the most dffcult tasks n mage processng! Segmentaton algorthms are based on two basc propertes of graylevel values: Dscontnuty: the mage s parttoned based on abrupt changes n gray level. Man approach s edge detecton. Smlarty: the mage s parttoned nto homogeneous regons. Man approaches are thresholdng, regon growng, and regon splttng and mergng. Copyrght notce: Most mages n these sldes are Gonzalez and Woods,, Prentce-Hall 3 basc types of dscontnutes n dgtal mages: Ponts, Lnes, Edges. SNR-optmal lnear flter n..d. Gaussan nose: matched flter, a.k.a. template matchng, a.k.a. cross-correlaton approach Pont detecton Thn lne detecton The output of the convoluton wll be stronger where a one-pxel-wde lne s present n the correspondng drecton. Note: zero-sum masks ( second-order drectonal dervatve) e.g. detect a tny hole n a turbne blade (dark pxel wthn the brght zone below) (c): g = flter(f) (d): g >T, wth T= 0.9*max( g )

2 5//07 Edge: boundary between two regons wth sgnfcantly dstnct gray levels Edge models, (also for roof edge): f f ( x ) f ( x) x Frst-order dervatve has nonzero phase response -D case f x [ f ( x ) f ( x)] [ f ( x) f ( x )] f ( x ) f ( x ) f ( x) Typcal real-world problems: edges wth dfferent slopes obects wth dfferent szes scale-space operators? uneven llumnaton not sgnfcant (?) detals nose Ideal ramp edge plus nose havng std = 0.,, 0 gray levels (out of 56); frst- and second-order dervatves -D case Gradent f f f, x y f f f ; x y tan f f / y x Laplacan f f f x y Sobel H/V or 45/35 deg.

3 5//07 Edges from dagonal masks Thresholded mages Edge detecton based on zero crossngs of second-order dervatve (Marr-Hldreth operator) Standard mplementaton of a Laplacan: Its magntude produces double edges Unable to detect the edge drecton Very senstve to nose use Laplacan of Gaussan nstead: G( r) exp( r ) ; r G( r) exp( r r x y, K x, y K ) ; r G( r) exp( r )

4 5//07 "Mexcan hat" To detect the zero crossngs of the LoG mage: center a 3x3 mask on each pxel p(x,y) of the LoG mage check all pars (p,p ) of opposte neghbors (l/r, u/d, 45, 35) p s edge f at least n one par pxels have dfferent sgn to reduce nose", consder only {p: abs(p -p )>thresh.} Qualty crtera small false alarm rate, small mssed detecton rate good localzaton one-pxel-wde edges Canny edge detector Comparson between Sobel (before thresholdng) and zero crossngs of LoG Edges n LoG are thnner and tend to form loops; obects sze s altered Canny operator A good approxmaton of an deal detector for -D nosy stepedges s the dervatve of the Gaussan x G( x) exp( x ) In D, t should be appled orthogonally to the edge crcularly symmetrc lowpass Gaussan flter, followed by computaton of the gradent G(x,y) shows thck patterns non-maxma suppresson:. determne the quantzed drecton d k of the gradent. f G(x,y) < at least one of ts neghbours along d k then set t to zero Apply a threshold to reduce false alarms

5 5//07 Canny operator: quantzed edge drectons Canny operator: hysteress thresholdng reduces both false alarms and mssed detectons. Set two thresholds: T L and T H, wth T H 3 T L. Generate two bnary mages: G H = G(x,y) > T H (strong edges) G L = G(x,y) > T L (strong and weak edges) 3. Elmnate from G L all strong edge pxels (pxels that are nonzero n G H ): G L = G L - G H 4. Label all pxels n G H as edge 5. Fll edge gaps: a. Vst each nonzero pxel p n G H and mark as edge all pxels n G L that are 8-connected to p b. Reset all unmarked pxels n G L c. Fnal edge mage = G H + G L T L = 0.04, T H = 0.0 (normalzed pxel values) = 4, mask sze =5x5 T L = 0.05, T H = 0.5 =, mask sze =3x3

6 5//07 Image segmentaton: edge lnkng Image segmentaton: edge lnkng - local processng: mage ponts wth smlar gradent Jon the detected edge pxels accordng to ther smlarty (e.g., smlar ampltude and drecton of the gradent), and form a boundary K = 30 d = 45 deg. L = 5 px. E.g.: lookng for rectangles calculate mage gradent G(x,y) scan G(x,y) along rows and buld bnary edge mage, settng pxels where G > K% G max and G angle = ±90±d deg. re-scan by rows and fll gaps shorter than L do the same by columns, G angle = 0±d, or 80±d deg. add the two resultng mages Note detected lcense plate Image segmentaton: edge lnkng - global processng: the Hough transform Image segmentaton: edge lnkng - global processng: the Hough transform A more effcent method to detect straght lnes Can be generalzed to curves. Generc lne through a pont n the mage ( x, y ) : y ax b. In the parameters space (a,b), ( x, y ) defne a lne b x a y 3. Take a second pont n the mage along the same generc lne; ts representaton n the parameters space s: ( x, y ) : y ax b b x a y 4. Let (a,b ) be the coordnates at whch the two lnes ntercept n the parameters space 5. a,b are the slope and ntercept of the specfc lne through x, y ), ( x, y ) 6. All ponts located on such a lne n the mage plane have lnes n the parameters space whch ntersect at (a,b ) [ndeed, the lne n the mage plane sets a well-defned par of (slope,ntersect) values] ( mage space parameter space 6. All ponts located on such a lne n the mage plane have lnes n the parameters space whch ntersect at (a,b ) [ndeed, the lne n the mage plane sets a well-defned par of (slope,ntersect) values]

7 5//07 Image segmentaton: edge lnkng - global processng: the Hough transform Image segmentaton: edge lnkng - global processng: the Hough transform Subdvde the parameters space nto a matrx A(a,b) of cells and reset t For each edge pont ( x, y ) n the mage For each value of a solve b x a y ncrement A(a,b) x cos y sn r The value Q n A(a,b) ndcates that Q edge ponts n the mage le on a lne of slope a and ntersect b Brght ponts n A show the parameters of the man edges n the mage. Problem: vertcal edges are dffcult to represent snce a tends to nfnty Soluton: use the normal form (trgonometrc form) of a lne: Ponts n the mage space now defne a snusod n the parameters space Let ( x, y ), ( x, y ) defne two snusods that ntersect n (r',q ') these are the parameters of a lne through x, y ), ( x, y ) n the mage ( Defne a matrx A(r,q ) and reset t. For each edge pont n the mage For each value of q ; fnd r ; ncrement A(r,q ) Brght ponts n A show the parameters of the man edges n the mage Image of sze 0x0; orgn n pt.; Image segmentaton pt.: snq = r pt.4: 00 cosq + 0 = r pt.3: 50 cosq + 50 snq = r x cos y sn r Ponts, 5 n (x,y) are mapped to snusods wth ampltude r,3,5 are algned; snusods ntersect at (0,45) (A),3,4 are algned; snusods ntersect at (70.7, -45) (B) r reflects specularly at q =90, q =-90 (Q,R,S): edges are both vertcal

8 5//07 Image segmentaton: edge lnkng Image segmentaton: snakes - global processng: the Hough transform Note: The HT can be used n prncple for any edge shape, represented by a functon of the type g(v,coef)=0, where v s a vector of coordnates and coef s a vector of coeffcents. E.g.: lookng for crcular obects: (x-a)^ + (y-b)^ = c^ Three parameters (a,b,c), 3-D parameter space, cube-lke cells, accumulator takes the form A(,,k). Procedure:. Increment a and b. Solve for c 3. Update the accumulator assocated wth (a,b,c) - Actve (elastc) contour models: Snakes slde: z0_snakes e Matlab: z0_snakes_matlab.zp t.mathworks.com/help/mages/hough-transform.html Thresholdng Thresholdng (global) nose g( x, y) 0 f f f ( x, y) T f ( x, y) T (edge-preservng nose smoothng preprocessng may be useful)

9 5//07 Thresholdng (global) llumnaton x reflectance Thresholdng (global) (Morphologcal or Retnex preprocessng may be useful) A smple algorthm: Select a frst value for T=T0; threshold the mage; evaluate the average gray-levels Ga, Gb of the two groups; set T=(Ga+Gb)/; repeat untl T(k)-T(k-) < e Ths approach s acceptable only for vsbly bmodal hstograms Thresholdng (global, optmal) Thresholdng (global, optmal) Consder a smple case (Mng Jang 5..) More formally: A bmodal hstogram can ndcate the presence of two obects n the mage,.e. t can be the weghted sum of two unmodal denstes (one for lght, one for dark areas): p(z) = P p(z) + P p(z) The parameters (probabltes P and P, wth P+P=) are proportonal to the areas of the pcture of each brghtness. If a mathematcal expresson for the denstes p(z), p(z) s known or assumed, determnng an optmal (e.g. MMSE) threshold s possble.

10 5//07 Thresholdng (global, optmal) Thresholdng (global, optmal) Above threshold Below threshold TP: # true postve FP: # false postve TN: # true negatve FN: # false negatve Assume the mage conssts of the obects and background, where the obects occupy P of the pxels (P+P=). Assume both obects and background are subect to a Normal dstrbuton; by the total probablty rule, the mage has the followng densty functon: P ( z ) p z) exp( Let T be the threshold. The ms-segmentaton takes place n two cases: * Background pxels ms-classfed nto obect pxels (FP): the error probablty (or the number of errors) s E * Obect pxels ms-classfed nto background pxels (FN): the error probablty (or the number of errors) s E T ( T ) p ( z) dz ; E ( T ) p P ( z ) ) exp( ( The total ms-segmentaton error s E(T) = P E(T) + P E(T) T E ( z) dz ) Thresholdng (global, optmal) The total ms-segmentaton error s E(T) = P E(T) + P E(T) The optmal threshold s T* = arg mn{e(t)}. Dfferentatng E(T): P E (T) + P E (T) = 0 P p(t) + P p(t) = 0 Substtutng the exact formula for the Gaussan nto the above equaton, we have P ( T ) P ( T ) exp( ) exp( ) ( T ) Two specfc examples: If the s.d. are the same: If the s.d. are the same and P=P=/: ( T ) P log P P T* log P T* For any (sub)set of gray levels (K K), defne CDF, mean, varance: Thresholdng (Otsu) (no hypotheses on the dstrbuton) K K K K P ( ) / ; ; K n K N P K (They all are functons of K,K, omtted) ( ) P ; Let class be formed by all pxels whose gray level s <= a threshold T; class by pxels > T; The between-class varance s the varaton of the mean values for each class from the global (G) ntensty mean of all pxels: G B ( G ) ( G ) ( ) K

11 5//07 Thresholdng (Otsu) Thresholdng (Otsu) Otsu: All possble thresholds are evaluated, and the one (T*) that maxmzes the between-class varance s chosen. BTW, ths s equvalent to fndng the mnmum ntra-class varance Qualty of result s gven by the normalzed between-class varance, called separablty, measured at T* ( T) ( T ) / B G 0 ( T ) 0 s attanable only by mages wth a sngle unform gray level s attanable only by -valued mages wth gray levels 0 and L- Otsu s method can be extended to multple thresholds T = 69 T* = 8 H = Regon growng == Defectve weld. Select seed regons (e.g. values n the upmost % of the dstrbuton) Regon growng. Detect all connected components, then erode them to one pxel 3. Defne a predcate for the pxels of the mage. E.g.: «the lumnance dfference wrt the average of the orgnal seed area s below a threshold; and the pxel s 8- connected to at least one pxel n the regon» 4. Sequentally append to the eroded seeds all the pxels that satsfy the predcate

12 5//07 Regon splttng and mergng Regon splttng and mergng - example P(R): «at least 80% of the pxels n the regon R dffer from the average of the regon by less than s.d.: z m» If P(R)=true graylevel output: set pxels n the regon to the average m of the regon bnary output (actual segmentaton): set above pxels to Defne a predcate P [e.g., usng descrptors (Ch. dp)], and subdvde the mage n regons for whch P s satsfed. More precsely:. Splt nto four quadrants any regon R for whch P(R)=false. Stop when a gven mn. sze s reached (e.g. x) a quadtree s created. Merge any adacent regons R, R for whch P(R U R)=true. Stop when no further mergng s possble Orgnal splt and merge thresholdng Note mssng parts when smple thresholdng s used: stem of the leaf, and shadng (f the latter s consdered a part of the obect!) Watersheds - The mage s treated as f t were a topographc map: gray level = heght - Watershed lnes dvde catchment basns - Floodng s appled (a hole s punched n each local mnmum, and water enters from below) - Dams are bult to prevent mergng between basns Watersheds - The fnal dams are the desred segmentaton result (fg. h) - Watershed lnes form a connected path contnuous boundares Watershed segmentaton s often appled to the gradent of the mage

13 5//07 Watersheds Watersheds A dam s bult on a bnary mage (thresholded flooded mage) usng morphologcal dlaton, wth two rules: - Dlaton s constraned to the connected regon formed by the mergng of the two basns - Dlaton s not performed on a pont f ths would cause the two regons to merge Ths operaton s repeated for each gray level Vdeo frame segmentaton [Refers to both spatal frame segm. and temporal shot segm.] Vdeo frame segmentaton MOTION s a useful cue for segmentaton, even for humans. - The trval way: compare two successve frames, pxel by pxel, and search pxels for sgnfcant changes; dfference mage takes value n postons where frame n (x,y) frame n-k (x,y) > T (k>=) Ths s senstve to nose, spatal msregstraton (camera moton or shake), varatons of llumnaton - Accumulatve Dfference Image (ADI): each pxel s a counter, ncremented every tme a sgnfcant dfference s found between that locaton n a frame of the sequence and the same locaton n a reference frame. The reference frame can be the frst one of the sequence. - Absolute ADI: ncrement f ref(x,y) frame n (x,y) > T - Postve ADI: ncrement f ref(x,y) frame n (x,y) > T - Negatve ADI: ncrement f ref(x,y) frame n (x,y) < -T E.g.: BRIGHT rectangular obect movng rght/downwards on a DARK background: A-ADI P-ADI N-ADI - Nonzero area n P-ADI = obect area (f sequence s long enough) - P-ADI shows the obect locaton n the reference frame - P-ADI stops ncreasng when obect s wholly dsplaced wrt ref. frame - Drecton and speed can be obtaned by A-ADI and N-ADI

14 5//07 Vdeo frame segmentaton Vdeo frame segmentaton Determnaton of the reference mage s not trval Example: buld a statc reference mage usng ADIs when the whte car has moved completely out of ts poston n the ref. frame, copy the correspondng background n the present frame nto the ref. frame. repeat for all movng obects. Example: ntruson detecton on a bus n a garage: acqure a frame once per second and compare to the prevous one dvde each frame nto 8x8 blocks and 3x3 macroblocks (MBs) compute the SAD (sum of absolute dfferences) for each block n the same poston n the two frames calculate N : for each MB, number of blocks wth SAD > T calculate N : number of MBs wth N > T f N > T 3 alarm Vdeo frame segmentaton T = 0 T = 0 MBs wth at least 6 blocks wth movement (T = 6 )