Moivaion Image segmenaion Which pixels belong o he same objec in an image/video sequence? (spaial segmenaion) Which frames belong o he same video sho? (emporal segmenaion) Which frames belong o he same aciviy? (spaioemporal segmenaion) Mos segmenaion algorihms are askoriened Segmenaion is essenial for recogniion purposes. Definiions Image segmenaion=pariion of an image ino a se of regions ha cover i Goal: regions mus represen well meaningful areas of an image Example: foress, waer, urban areas in saellie images Objecive Segmenaion decomposes he image ino pars for furher analysis Example: background subracion in human moion analysis Once he region of ineres is segmened, he represenaion space can be changed (from imagespace o feaure space) 3 Circumscribed (benign) lesions in digial mammography Spiculaed (malignan) lesions in digial mammography 4 A classificaion of segmenaion echniques Inensiy-based segmenaion: Thresholding Assumpions for hresholding. he inensiy values are differen in differen regions. wihin each region, which represens he corresponding objec in a scene, he inensiy values are similar. Edge-based segmenaion Region-based segmenaion 5 6
Inensiy-based hresholding Image hresholding classifies pixels ino wo caegories: Those o which some propery measured from he image falls below a hreshold, and hose a which he propery equals or exceeds a hreshold. Thresholding creaes a binary image : binarizaion e.g. perform cell couns in hisological images Choosing a hreshold is a criical ask. Fixed versus dynamic hresholding In fixed (or global) hresholding, he hreshold value is held consan hroughou he image: g(x,y) = f(x,y)<t f(x,y)>=t n=imread( nodules.jpg ); figure(); imshow(n); n=imbw(n,.35); n=imbw(n,.75); figure(), imshow(n); figure(3), imshow(n); 7 Local (or dynamic hresholding): depends on he posiion in he image. The image is divided ino overlapping secions which are hresholded one by one. 8 Threshold deecion mehods P-ile hresholding Opimal hresholding Mixure modelling Adapive hresholding P-ile mehod a priori informaion: objec is brigher/darker han background and occupies a cerain known percenile /p from he oal image area (example: prined ex shee) We se he hreshold by finding he inensiy level such ha /p image pixels are below his value We use he cumulaive hisogram g c( g) = h( k) k= n h( k) = k n T verifies he equaion c(t)=/p (for a dark foreground) c(t)=-/p (for a brigh foreground) 9 Finding modes Hisogram shape analysis Foreground pixels form one peak Background pixels form he second peak Inuiively: he hreshold is se as he gray level ha has a minimum value beween wo maxima Problem: noisy hisograms (sal-and pepper noise) Opimal hresholding Idea: he hisogram of an image is approximaed using a weighed sum of wo or more probabiliy densiies wih normal disribuion Threshold: overlapping poin of hese disribuions (corresponds o he minimum probabiliy beween he maxima of disribuions) Problem: disribuions are unknown
Comparison beween convenional and opimal hresholding Opimal hresholding by clusering Simples case: segmenaion ino wo classes (objec/background). The inensiies in each class will be our clusers. We wan o find a hreshold so ha: 3 4 Ieraive opimal hreshold selecion. Selec an iniial esimae for T (usually average inensiy). Segmen he image using T. This produces groups: G pixels wih value >T and G, wih value <T Ieraive K-Means Clusering Algorihm m()=6.83, m()=539. m()=39.37, m()=45.65 m(3)=5.9, m(3)=98.63 m(4)=54.7, m(4)=6.8 m(5)=55.4, m(5)=7.4 m(6)=55., m(6)=7.44 m(7)=55., m(7)=7.44 3. Compue µ and µ, average pixel values of G and G 4. New hreshold: T=/(µ+µ) 5. Repea seps o 4 unil T sabilizes. 5 6 Opimal hresholding : he Osu mehod The Osu mehod Opimal hresholding mehods selec he hreshold based on he minimizaion of a crierion funcion. The crierion for Osu is he minimizaion of he wihin-group variance of he wo groups of pixels separaed by he hreshold. 7 8
The Osu mehod The beween class variance is obained by subracing he wihin-class variance from he oal variance of he combined disribuion: where σ is he variance of he combined disribuion μ and σ are no dependen on hreshold T, hus minimizing he wihin-class variance is he same as maximizing he beween-class Osu s mehod For every possible :. Pick a.. Calculae wihin group variances. probabiliy of group. probabiliy of group 3. deermine mean of group 4. deermine mean of group 5. calculae variance for group 6. calculae variance for group 7. calculae weighed sum of group variances and remember which gave rise o minimum. variance 9 probabiliy of being in each group mean of individual groups q q () = p() i max () = p() i + μ μ () = i p() i / q () max () = i p() i / q () + σ σ variance of individual groups () = [ i μ () ] p() i / q () max () = [ i μ() ] p() i / q() + weighed sum of group variances W ( ) = q ( ) σ ( ) q ( ) σ ( ) σ + Calculae for all s and minimize. { ( ) max} min σ W 3 4
Mixure modelling Assumpion: region inensiies are each normal disribuions (Gaussians) 5 6 Mixure modelling (con d) Thresholding and illuminaion Each of he Gaussian disribuions has a mean and sandard deviaion independen of he hreshold ha we choose Foreground/background case: We need o esimae 6 parameers Evaluaion of how well he sum of he disribuions approximae he hisogram The parameers will be chosen such as o minimize he error F 7 8 Adapive hresholding Esimaing hresholds along boundaries s( x, y) = + if f < T if f T if f T and f and f < Esimaing hresholds along boundaries Ligh background/dark objec: ( ) (-,+) ( or +) (+,-) ( ) pixels ha are no on an edge are labeled pixels on he dark side of an edge are labeled + pixels on he ligh side of an edge are labeled The hisogram is sampled only near where he boundary probabiliy is high. 9 3
Thresholding: Summary Advanages: Simple o implemen Fas (especially if repeaing on similar images) Good for some kinds of images (e.g., documens, conrolled lighing) Disadvanages: No guaranees of objec coherency may have holes, exraneous pixels, ec. (incomplee) soluion: pos-processing wih morphological operaors 3 3 Nex lecure Edge-based segmenaion Region-based segmenaion 33