Binary Image Proc: week 2

Transcription

1 Binary Image Proc: week 2 n Imaging devices n Getting a binary image n Morphological filtering n Connected components extraction n Extracting numerical features from regions CSE803 Fall 207

2 Imaging Devices Summary of several common imaging devices covered (Chapter 2 of Shapiro and Stockman) MSU/CSE Fall 207 2

3 Major issues What radiation is sensed? Is motion used to scan the scene or are all sensing elements static? How fast can sensing occur? What are the sensing elements? How is resolution determined? -- in intensity -- in space MSU/CSE Fall 207 3

4 CCD type camera Array of small fixed elements Can read faster than TV rates Can add refracting elts to get color in 2x2 neighborhoods 8-bit intensity common MSU/CSE Fall 207 4

5 Color CCD Bayer Filter CSE803 Fall 207 5

6 Blooming Problem with Arrays Difficult to insulate adjacent sensing elements. Charge often leaks from hot cells to neighbors, making bright regions larger. MSU/CSE Fall 207 6

7 8-bit intensity can be clipped Dark grid intersections at left were actually brightest of scene. In A/D conversion the bright values were clipped to lower values. MSU/CSE Fall 207 7

8 Lens distortion distorts image Barrel distortion of rectangular grid is common for cheap lenses ($50). Pin cushion opposite. Precision lenses can cost $000 or more. Zoom lenses often show severe distortion. Barrel distortion Pincushion distortion Mustache distortion MSU/CSE Fall 207 8

9 Other important effects/problems Discrete pixel effect: signal is integrated Mixed pixel effect: materials mix Thin features can be missed Measurements in pixels have error Region or material B Region or material A MSU/CSE Fall 207 9

10 Orbiting satellite scanner View earth pixel at a time (through a straw) Prism produces multispectral pixel Image row by scanning boresight All rows by motion of satellite in orbit Scanned area of earth is a parallelogram, not a rectangle MSU/CSE Fall 207 0

11 Human eye as a spherical camera 00M sensing elts in retina Rods sense intensity Cones sense color Fovea has tightly packed elts, more cones Periphery has more rods Focal length is about 20mm Pupil/iris controls light entry Eye scans, or saccades to image details on fovea 00M sensing cells funnel to M optic nerve connections to the brain MSU/CSE Fall 207

12 Surface data (2.5D) sensed by structured light sensor MSU/CSE Fall Projector projects plane of light on object Camera sees bright points along an imaging ray Compute 3D surface point via line-plane intersection REF: Minolta Vivid 90 camera

13 Structured light MSU/CSE Fall 207 3

14 2.5D face image from Minolta Vivid 90 scanner in the CSE PRIP Lab MSU/CSE Fall 207 4

15 Freehand Laser Scanning Using Mobile Phone CSE803 Fall 207 5

16 LIDAR also senses surfaces Single sensing element scans scene Laser light reflected off surface and returned Phase shift codes distance Brightness change codes albedo watch?v=ebucgxzq_xg MSU/CSE Fall 207 6

17 What about human vision? Do we compute the surfaces in our environment? Do we represent them in our memory as we move around or search? Do we save representations of familiar objects? (See David Marr, Vision, 982; Aloimonus and Rosenfeld 99.) MSU/CSE Fall 207 8

18 3D scanning technology 3D image of voxels obtained Usually computationally expensive reconstruction of 3D from many 2D scans (CAT computer-aidedtomography) More info later in the course. MSU/CSE Fall 207 9

19 Magnetic Resonance Imaging Sense density of certain chemistry S slices x R rows x C columns Volume element (voxel) about 2mm per side At left is shaded 2D image created by volume rendering a 3D volume: darkness codes depth MSU/CSE Fall

20 Single slice through human head MRIs are computed structures, computed from many views. At left is MRA (angiograph), which shows blood flow. CAT scans are computed in much the same manner from X-ray transmission data. MSU/CSE Fall 207 2

21 Other variations Microscopes, telescopes, endoscopes, X-rays: radiation passes through objects to sensor elements on the other side Fibers can carry image around curves; in bodies, in machine tools Pressure arrays create images (fingerprints, butts) Sonar, stereo, focus, etc can be used for range sensing (see Chapters 2 and 3) MSU/CSE Fall

22 Summary of some digital imaging problems. Mixed pixel problem: mixed material in the world/scene map to a single pixel Loss of thin/small features due to resolution Variations in area or perimeter due to variation in object location and thresholding Path problems: refraction, dispersion, etc. takes radiation off straight path (blue light bends more or less than red light going through lens?) MSU/CSE Fall

23 Quick look at thresholding n Separate objects from background. n 2 class or many class problem? n How to do it? n Discuss methods later. CSE803 Fall

24 Cherry image shows 3 regions n n n n Background is black Healthy cherry is bright Bruise is medium dark Histogram shows two cherry regions (black background has been removed) Use this gray value to separate CSE803 Fall

25 Choosing a threshold n Common to find the deepest valley between two modes of bimodal histogram n Or, can level-slice using the intensities values a and b that bracket the mode of the desired objects n Can fit two or more Gaussian curves to the histogram n Can do optimization on above (Ohta et al) CSE803 Fall

26 Example red blood cell image n Many blood cells are separate objects n Many touch bad! n Salt and pepper noise from thresholding n How useable is this data? CSE803 Fall

27 Cleaning up thresholding results n Can delete object pixels on boundary to better separate parts. n Can fill small holes n Can delete tiny objects n (last 2 are salt-andpepper noise) CSE803 Fall

28 Removing salt-and-pepper n Change pixels all (some?) of whose neighbors are different (coercion!): see hole filled at right n Delete objects that are tiny relative to targets: see some islands removed at right CSE803 Fall

29 Simple morphological cleanup n Can be done just after thresholding -- remove salt and pepper n Can be done after connected components are extracted -- discard regions that are too small or too large to be the target CSE803 Fall

30 Dilation & Erosion Basic operations Are dual to each other: Erosion shrinks foreground, enlarges Background Dilation enlarges foreground, shrinks background Slides contributed by Volker Krüger & Rune Andersen, University of Aalborg, Esbjerg %2FCLASS_479%2Flectures_202_479%2F20.% %2520A._Introduction_to_Morphological_Operators.ppt&ei=_fMtUpvQNYbwyAGN0IAo&usg= AFQjCNGJat2gHPKxO_WlYkw3gzv6XxEJIQ&sig2=5McelJco9akafAA7xCB6Q

31 A first Example: Erosion Erosion is an important morphological operation Applied Structuring Element:

32 Example for Erosion Input image Structuring Element Output Image 0

33 Example for Erosion Input image Structuring Element Output Image 0 0

34 Example for Erosion Input image Structuring Element Output Image 0 0 0

35 Example for Erosion Input image Structuring Element Output Image

36 Example for Erosion Input image Structuring Element Output Image

37 Example for Erosion Input image Structuring Element Output Image

38 Example for Erosion Input image Structuring Element Output Image

39 Example for Erosion Input image Structuring Element Output Image

40 Another example of erosion White = 0, black =, dual property, image as a result of erosion gets darker

41 Counting Coins Counting coins is difficult because they touch each other! Solution: Binarization and Erosion separates them!

42 Example: Dilation Dilation is an important morphological operation Applied Structuring Element:

43 Example for Dilation Input image Structuring Element Output Image

44 Example for Dilation Input image Structuring Element Output Image 0

45 Example for Dilation Input image Structuring Element Output Image 0

46 Example for Dilation Input image Structuring Element Output Image 0

47 Example for Dilation Input image Structuring Element Output Image 0

48 Example for Dilation Input image Structuring Element Output Image 0

49 Example for Dilation Input image Structuring Element Output Image 0

50 Example for Dilation Input image Structuring Element Output Image 0

51 Another Dilation Example Image get lighter, more uniform intensity

52 Edge Detection Edge Detection. Dilate input image 2. Subtract input image from dilated image 3. Edges remain!

53 Opening & Closing Important operations Derived from the fundamental operations Erosion next dilation -> opening Dilation next erosion -> closing Usually applied to binary images, but gray value images are also possible Opening and closing are dual operations

54 Connected components n Assume thresholding obtained binary image n Aggregate pixels of each object n 2 different program controls n Different uses of data structures n Related to paint/search algs n Compute features from each object region CSE803 Fall

55 CC analysis of red blood cells n 63 separate objects detected n Single cells have area about 50 n Noise spots n Gobs of cells CSE803 Fall

56 More control of imaging n More uniform objects n More uniform background n Thresholding works n Objects actually separated CSE803 Fall

57 Results of pacmen analysis n 5 objects detected n Location known n Area known n 3 distinct clusters of 5 values of area; 85, 45, 293 CSE803 Fall

58 Results of coloring objects n Each object is a connected set of pixels n Object label is color n How is this done? CSE803 Fall

59 Extracting white regions n n n n Program learns white from training set of sample pixels. Aggregate similar white neighbors to form regions. Regions can then be classified as characters. (Work contributed by David Moore, CSE 803.) (Right) output is a labeled image. (Left) input RGB image of a NASA sign CSE803 Fall 207 6

60 Extracting components: Alg A n Collect connected foreground pixels into separate objects label pixels of same object with same color n A) collect by random access of pixels using paint or fill algorithm CSE803 Fall

61 paint/fill algorithm n Obj region must be bounded by background n Start at any pixel [r,c] inside obj n Recursively color neighbors CSE803 Fall

62 Events of paint/fill algorithm n PP denotes processing point n If PP outside image, return to prior PP n If PP already labeled, return to prior PP n If PP is background pixel, return to prior PP n If PP is unlabeled object pixel, then ) label PP with current color code 2) recursively label neighbors N,, N8 (or N,, N4) CSE803 Fall

63 Recursive Paint/Fill Alg: region n Color closed boundary with L n Choose pixel [r,c] inside boundary n Call FILL FILL ( I, r, c, L) If [r,c] is out, return If I[r,c] == L, return I[r,c] ß L // color it For all neighbors [rn,cn] FILL(I, rn, cn, L) CSE803 Fall

64 Recall different neighborhoods n 4-connected: right, up, left, down only n 8-connected: have 45 degree diagonals also n If the pixels [r,c] of an image region R are 4- connected, then there exists a path from any pixel [r,c] in R to any other pixel [r2,c2] in R using only 4 neighbors n The coloring algorithm is guaranteed to color all connected pixels of a region. Why? CSE803 Fall

65 Correctness of recursive coloring n n n n If any pixel P is colored L, then all of its object neighbors must be colored L (because FILL will be recursively called on each neighbor) If pixels P and Q are in connected region R, then there must be a path from pixel P to Q inside of R. If FILL does not color Q, let X be the first uncolored pixel on a path from P to Q. If a neighbor N of X is colored, then so must be X! This contradiction proves the property all of the path must be colored. Pixels P and Q are in a connected region. CSE803 Fall P Q N X P is colored; Q is not; N is colored; X is not. But if N is colored, X must be too!

66 Connected components using recursive Paint/Fill n Raster scan until object pixel found n Assign new color for new object n Search through all connected neighbors until the entire object is labeled n Return to the raster scan to search for another object pixel CSE803 Fall

67 Extracting 5 objects CSE803 Fall

68 Outside material to cover n Look at C++ functions for raster scanning and pixel propagation n Study related fill algorithm n Discuss how the recursion works n Prove that all pixels connected to the start pixel must be colored the same n Runtime stack can overflow if region has many pixels. CSE803 Fall

69 Alg B: raster scan control Visit each image pixel once, going by row and then column. Propagate color to neighbors below and to the right. Keep track of merging colors. CSE803 Fall 207 7

70 Have caution in programming n Note how your image tool coordinates the pixels. n Different tools or programming languages handle rows and columns differently n Text uses EE-TV coordinates or standard math coordinates col Yi row Xi CSE803 Fall

71 Events controlled by neighbors n n n n If all Ni background, then PP gets new color code If all Ni same color L, then PP gets L If Ni!= Nj, then take smallest code and make all same See Ch 3.4 of S&S Processing point CSE803 Fall

72 Merging connecting regions Detect and record merges while raster scanning. Use equivalence table to recode (in practice, probably have to start with color = 2, not ) CSE803 Fall

73 Alg A versus alg B n n n n n n Visits pixels more than once Needs full image Recursion or stacking slower than B No need to recolor Can compute features on the fly Can quit if search object found (tracking?) visits each pixel once Needs only 2 rows of image at a time Need to merge colors and region features when regions merge Typically faster Not suited for heuristic (smart) start pixel CSE803 Fall

74 Outside material n More examples of raster scanning n Union-find algorithm and parent table n Computing features from labeled object region n More on recursion and C++ CSE803 Fall

75 Computing features of regions Can postprocess results of CC alg. Or, can compute as pixels are aggregated CSE803 Fall

76 Area and centroid CSE803 Fall

77 Bounding box: BB n Smallest rectangle containing all pixels n r-low is lowest row value; c-low is lowest column value n r-high, c-high are the highest values n Bounding box has diagonal corners [rlow, c-low] and [r-high, c-high] n BBs summarize where an object is; they can be used to rule out near collisions CSE803 Fall

78 Second moments These are invariant to object location in the image. CSE803 Fall

79 Contrast second moments n For the letter I n Versus the letter O n Versus the underline _ I r c CSE803 Fall 207 8

80 Perimeter pixels and length How do we find these boundary pixels? CSE803 Fall

81 Circularity or elongation Computing P is more trouble than computing A. CSE803 Fall

82 Circularity as variance of radius CSE803 Fall

83 Axis of least (most) inertia gives object oriented, or natural, coordinate system passes through centroid axis of most inertia is perpendicular to it concept extends to 3D and nd CSE803 Fall

84 Approach (in Cartesian/math coordinates) n First assume axis goes through centroid n Assume axis makes arbitrary angle Θ with the horizontal axis n Write formula for the angular momentum (inertia) to rotate a single unit of mass (pixel) about the axis n Write formula for the sum over all pixels n Take derivative of formula to find Θ for extreme values CSE803 Fall

85 Rotational inertia of pixel Y d Θ + 90 [x,y] Θ Axis of inertia Centroid of pixels X CSE803 Fall

86 Overall steps n d = [x, y] projected along normal to axis = -x sin Θ + y cos Θ n Inertia of single pixel is d squared n Inertia of entire region is sum over all pixels İ = ( x sin Θ y cos Θ)^2 = sin^2 Θ x^2 2sin Θ cos Θ xy + cos^2 Θ y^2 = sin^2 Θ a 2sin Θ cos Θ b + cos^2 Θ c where a, b, c are the second moments! CSE803 Fall

87 Final form n By differentiating İ with respect to Θ and setting to 0 n 0 = a tan 2 Θ -2b c tan 2 Θ n tan 2 Θ = 2b/(a-c) pretty simple n What if all pixels [x,y] lie on a circle? n What if they are all on a line? CSE803 Fall

88 As in text with centroid CSE803 Fall

89 Formula for best axis n use least squares to derive the tangent angle θ of the axis of least inertia n express tan 2θ in terms of the 3 second moments n interpret the formula for a circle of pixels and a straight line of pixels CSE803 Fall 207 9

90 Applications An iterative approach for fitting multiple connected ellipse structure to silhouette, PRL, 200 CSE803 Fall