Real-time Corner and Polygon Detection System on FPGA. Chunmeng Bi and Tsutomu Maruyama University of Tsukuba

Transcription

1 Real-time Corner and Polygon Detection System on FPGA Chunmeng Bi and Tsutomu Maruyama University of Tsukuba

2 Outline Introduction Algorithms FPGA Implementation Experimental Results Conclusions and Future Work

3 Introduction Corner is a basic step frequently used in many image processing applications such as object recognition. Polygon can further help the recognition of the objects in the images.

4 Polygon Detection Introduction

5 Polygon Detection Introduction

6 Algorithms of Corner Detection SUSAN: weak in the blurred image input image SUSAN

7 Algorithms of Corner Detection SUSAN: weak in the blurred image Harris: low localization rate input image Harris(13143)

8 Algorithms of Polygon Detection Hough Transform: requires significant computation and large memory, and it is difficult to achieve real-time processing. Shape Matching: uses the shape description for detecting polygons but it is difficult to define a descriptor which works well for all images.

9 Our Algorithm edge edge thinning corner polygon

10 Our Corner Detection Algorithm 1.For all pixels in the image, a filter whose size is (2L+1) (2L+1) is applied. In this filter, the number of the edges are counted up along the radial direction and 8L sums are calculated in total.

11 Our Corner Detection Algorithm 1.For all pixels in the image, a filter whose size is (2L+1) (2L+1) is applied. In this filter, the number of the edges are counted up along the radial direction and 8L sums are calculated in total. 2. A histogram is obtained by calculating the 8L sums.

12 Our Corner Detection Algorithm 2. A histogram is obtained by calculating the 8L sums. 3. The peaks in the histogram are detected.

13 Our Corner Detection Algorithm 3. The peaks in the histogram are detected. 4. If the peak is larger than a given threshold T line, it is considered that there exists a line segment on the radial direction.

14 Our Corner Detection Algorithm 3. The peaks in the histogram are detected. 4. If the peak is larger than a given threshold T line, it is considered that there exists a line segment on the radial direction. From the histogram for a pixel p, it is expected that two line segments (one of them passes through p and another one stops at p) are crossing at p.

15 Our Corner Detection Algorithm input image our algorithm(473) Our approach detects fewer corners and can tell the exact position of the corners as well as the gradients of all the line segments crossing at the corners.

16 Our Polygon Detection Algorithm 1. Let S c be the set of the corners and a corner(c s ) in S c is chosen. S c : C s, C u, C v, C w, C x, C y

17 Our Polygon Detection Algorithm 1. Let S c be the set of the corners and a corner(c s ) in S c is chosen. S c : C s, C u, C v, C w, C x, C y

18 Our Polygon Detection Algorithm 1. Let S c be the set of the corners and a corner(c s ) in S c is chosen. 2. Find the corners which can be matched with (k a, k b ). C s : k a, k b,-k a, -k b d p (direction pair) : {k a, -k b } t r (trace) :{C s } S c : C s, C u, C v, C w, C x, C y

19 Our Polygon Detection Algorithm 3.Scan the corners in S c and choose the closest connectable corner (C u ) to C s in distance from the list. S c : C s, C u, C v, C w, C x, C y

20 Our Polygon Detection Algorithm 3.Scan the corners in S c and choose the closest connectable corner (C u ) to C s in distance from the list. 4. C u has four directions but k f is chosen, because it is the closest to -k a and k b. S c : C s, C u, C v, C w, C x, C y C u : k a, k f, -k a, -k f d p :{k f, k a } t r :{C u, C s }

21 Our Polygon Detection Algorithm 3.Scan the corners in S c and choose the closest connectable corner (C u ) to S c in distance from the list. 4. C u has four directions but k f is chosen, because it is the closest to - k a and k b. C u : k a, k f, -k a, -k f d p :{k f, k a } t r :{C u, C s } 5. This tracking is continued for this new d p from step 3. S c : C s, C u, C v, C w, C x, C y

22 Our Polygon Detection Algorithm C v :-k f, k g d p :{k g, k f } t r :{C v, C u, C s } S c : C s, C u, C v, C w, C x, C y

23 Our Polygon Detection Algorithm C w : k g, k h, -k g, -k h d p :{k h, k g } t r :{C w, C v, C u, C s } S c : C s, C u, C v, C w, C x, C y

24 Our Polygon Detection Algorithm C x : k b, k h, -k b, -k h d p :{-k b, k h } t r :{C x, C w, C v, C u, C s } S c : C s, C u, C v, C w, C x, C y

25 Our Polygon Detection Algorithm C s : k a, k b, -k a, -k b d p :{k a, -k b } t r :{C s, C x, C w, C v, C u, C s } 5.Finally, C s is chosen again. Then, a polygon is detected and the corners in t r fix the shape of the polygon. S c : C s, C u, C v, C w, C x, C y

26 Our Polygon Detection Algorithm 6. Repeat the same procedure for other angles of C s (eg.(k a, -k b ) (-k a, -k b )). S c : C s, C u, C v, C w, C x, C y

27 Our Polygon Detection Algorithm 6. Repeat the same procedure for other angles of C s (eg.(k a, -k b ) (-k a, -k b )). 7. C s is deleted from S c to prevent multiple of the same polygon. S c : C u, C v, C w, C x, C y

28 Our Polygon Detection Algorithm 6. Repeat the same procedure for other angles of C s (eg.(k a, -k b ) (-k a, -k b )). 7. C s is deleted from S c to prevent multiple of the same polygon. Our algorithm can detect polygons of any shapes and any number of corners.

29 FPGA Implementation Edge Detection(Prewitt filter) Horizontal direction hx Vertical direction hy Horizontal& Vertical hxy ( hx hy )

30 edge FPGA Implementation

31 FPGA Implementation the edges are binarized: Edge thinning:each edge is compared with its left, right, up and down pixels and if the edge is not the peak along the x nor y axis, its value is masked to 0. edge

32 FPGA Implementation edge thinning edge

33 FPGA Implementation corner edge thinning edge

34 FPGA Implementation corner The binarized edges are held on a register array whose size is (2L+1) (2L+1) and summed up along the radial lines which are drawn from the center to the 8L pixels on the edges. edge thinning edge

35 FPGA Implementation corner The binarized edges are held on a register array whose size is (2L+1) (2L+1) and summed up along the radial lines which are drawn from the center to the 8L pixels on the edges. edge thinning edge

36 FPGA Implementation corner The binarized edges are held on a register array whose size is (2L+1) (2L+1) and summed up along the radial lines which are drawn from the center to the 8L pixels on the edges. edge thinning edge

37 FPGA Implementation corner edge thinning The binarized edges are held on a register array whose size is (2L+1) (2L+1) and summed up along the radial lines which are drawn from the center to the 8L pixels on the edges. 8L data are counted up at the same time. edge

38 FPGA Implementation corner peak detector edge thinning edge

39 FPGA Implementation peak mask unit corner edge thinning edge

40 FPGA Implementation peak mask unit corner edge thinning edge

41 FPGA Implementation peak mask unit corner edge thinning edge

42 FPGA Implementation peak mask unit corner edge thinning edge

43 FPGA Implementation corner corner generator edge thinning (x,y): coordinates k i : gradient r i : the distance from the origin to the line whose gradient is k i, and which goes through (x, y). edge

44 FPGA Implementation corner edge thinning (x,y): coordinates k i : gradient r i : the distance from the origin to the line whose gradient is k i, and which goes through (x, y). edge

45 FPGA Implementation corner polygon edge thinning edge

46 FPGA Implementation polygon We use 6 units to trace all directions(up to 6) of the line segments crossing at the corner at the same time.

47 FPGA Implementation 1 corner polygon We use 6 units to trace all directions(up to 6) of the line segments crossing at the corner at the same time.

48 FPGA Implementation 2 corner polygon We use 6 units to trace all directions(up to 6) of the line segments crossing at the corner at the same time.

49 FPGA Implementation 3 corner polygon We use 6 units to trace all directions(up to 6) of the line segments crossing at the corner at the same time.

50 FPGA Implementation 4 corner polygon We use 6 units to trace all directions(up to 6) of the line segments crossing at the corner at the same time.

51 FPGA Implementation Data for a corner are read out from the buffer and put into the register set d p in the 6 units. polygon

52 FPGA Implementation k 0 k 0 k 1 k 1 polygon In another sub-unit in this unit, k 1 and -k 0 are put to d p instead of k 0 and k 1 to trace the other side of the corner.

53 FPGA Implementation polygon A: Comparing the addresses of the new corners detected in both subunits by the loop detector. B: The address of the new corner is also compared with the previous one in another sub-unit.

54 FPGA Implementation polygon C s Register set R c : keep the closest corner to C s

55 FPGA Implementation polygon C s Suppose that a corner C c is now held on the register set R c as the closest connectable corner to C s of the corners that have been detected already.

56 FPGA Implementation polygon C c is replaced by C u if the following conditions are satisfied 1 connectable unit: connectable to C s

57 FPGA Implementation polygon C c is replaced by C u if the following conditions are satisfied 1 connectable unit: connectable to C s 2 calc.dist unit: choose the closest one

58 FPGA Implementation In our method, all corners in the buffer are scanned, and all angles in each corner is compared with the angle on d p in parallel. By managing the corners using their k i or r i, we can reduce the number of the corners which should be compared with d p, but we have chosen not to use those sophisticated approaches, because of the following two reasons. polygon 1. We can still achieve more than 30fps for pixel images if the number of the corners is less than With our corner detector, the corners which are unrelated to the polygons are rarely detected.

59 FPGA Implementation After comparing d p with all angles of each other, one of the angles of the corner on R c is chosen according to the two angles on d p and put into d p. At the same time, the angle on d p is moved to the shift register to keep the trace of the tracking. polygon

60 Experimental Results -1 Experimental environment: Xilinx XC4VLX160 Image size: X 1024 Circuit size and the performance

61 Experimental Results -2

62 Conclusions and Future Work We have described a corner algorithm, and the polygon system based on the algorithm. We can reduce the number of the corners in the images with our algorithm and obtain the correct angles of the corners. This makes it possible to detect polygons on hardware systems efficiently.

63 Conclusions and Future Work Future Work: Develop a technique for excluding false corners which are detected around shadows.

64 Thank you for your kind attention.