Carmen Alonso Montes 23rd-27th November 2015

Transcription

1 Practical Computer Vision: Theory & Applications 23rd-27th November 2015

2 Wrap up Today, we are here 2

3 Learned concepts Hough Transform Distance mapping Watershed Active contours 3

4 Contents Wrap up Object Recognition Practical application in industry: code target recognition Biometrics Recognition Practical application in authentication applications using medical images Summary 4

5 Object Recognition 5

6 Object Recognition Object recognition is a process for identifying a specific object in a digital image or video. Object recognition algorithms rely on matching, learning, or pattern recognition algorithms using appearance-based or feature-based techniques Humans recognize a lot of objects in images with little effort even when the image may vary in view points, sizes and scales, translated or rotated. partially obstruction 6

7 Challenges The main challenges is: Images are big Viewing conditions are infinite Computers are finite Objects are surrounded by other objects 7

8 How does the machine recognise? Once the objects are detected (segmented, edges, skeletons, etc) it is the moment to recognise Highly dependent on the application Real-time? Offline processing Do we have a database with reference images? How much is the variability of the images? Do I need any similarity measure? As engineers, we can go for sophisticated approaches (neural networks) or more simple ones template matching or image registration Usually techniques from pattern recognition are useful in this area. q=tbn:and9gctdhvmls7x7b5avbwy_aznaf8kd5bn9pwvo6dq2xhidhp5hvumiaa 8

9 9

10 Template matching Template matching is a technique in digital image processing for finding small parts of an image which match a template. It can be used in manufacturing as a part of quality control, a way to navigate a mobile robot, or as a way to detect edges in images

11 Optical Character Recognition (OCR) Optical character recognition(optical character reader) (OCR) is a classical example of pattern recognition. Hand made characters shall be recognised by the computer and provided a digital version. First test and approaches were developed to recognise Chinese characters due to its singularity Nowadays, it is a popular technique thanks to initiatives to put in digitall format old printed books OCR is a field of research in pattern recognition, artificial intelligence and computer vision. Resources: Stanford data set: Some popular techniques Matrix matching involves comparing an image to a stored glyph on a pixel-by-pixel basis; it is also known as "pattern matching", "pattern recognition", or "image correlation". Feature extraction decomposes glyphs into "features" like lines, closed loops, line direction, and line intersections. These are compared with an abstract vector-like representation of a character, which might reduce to one or more glyph prototypes. (e.g. k-nearest neighbors algorithm) Software such as Cuneiform and Tesseract use a two-pass approach to character recognition. The second pass is known as "adaptive recognition" and uses the letter shapes recognized with high confidence on the first pass to recognize better the remaining letters on the second pass. 11

12 Robotics: Obstacle Avoidance Obstacle avoidance: nonintersection or non-collision The willow garage project ROS: Robot operating system R3-COP - Resilient Reasoning Robotic Co-operating Systems 12

13 Collision Avoidance 13

14 Practical case in industry 14

15 Practical Industrial Case Problem: Automatic raw part alignment for machining process by means of artificial Coded Targets (CT) within a Phogrammetry-based Application Published: Miguel Fernandez-Fernandez, Carmen Alonso-Montes, et al.: Industrial Non-intrusive Coded-Target Identification and Decoding Application. IbPRIA 2013:

16 What do we need to do as CV Engineers? Analyze the environment Where will the camera be placed? Light conditions? Pollution? External factors that can affect my system? Analyze the elements you have The camera properties and capacities Crucial: Calibration phase The images Do I have a set of training images? Is this training set realistic? Must I have real time processing? How do I integrate with the other systems? 16

17 Our environment: Schema Wireless communication among camera & PC Processing must be done in the camera or the image must have lower resolution 17

18 Coded Target (CT) The reliability of CT for industrial applications comes from its structure invariant to rotation, scale, and distortion The central circle target is surrounded by the code band, which is divided into equally spaced angular intervals corresponding to bit positions (meaning 1 for a white interval and 0 for a black one). The code band is read anticlockwise from a predefined starting point extracting the associated bit pattern, and consequently the associated number. The bit pattern is which corresponds to the associated number 190. The exact raw part geometry correspondence is guaranteed thanks to the uniquity of the CT decoded numbers. 18

19 Let's start We already have an black and white image High resolution image Objective: read it and get the associated number Challenges: Light conditions Pollution and dust Occlusions and changes of perspective depending on where is placed 19

20 The algorithm We have split up the system in 2 phases Phase 1: aims to locate approximately the valid Cts Phase 2: To read the selected CT from Phase 1 20

21 Phase 1: Where are my Cts? The bilateral filtering is an edge preserving and noise reducing smoothing filter, which performs specifically well in poor illumination conditions and the pollution type of this industrial raw part images. An adaptive threshold algorithm (mean value of all pixels in its 5x5 neighbourhood) is applied to binarise the image. Then, erosion and dilation operations are performed in order to remove the salt-and-pepper noise. 21

22 Identification of the potential CT Search for the CT and to compute the surrounding area (ROI). The CT identification is done based on the analysis of the detected contour structures. valid or non-valid CT structures according to their conformance with an ideal CT pattern Circularity factor : minimum enclosing technique to distinguish those contours that are actually circles Three concentric circles Ratio descriptors: distance ratio among circles and diameter ratio of the central target 22

23 Phase 2: My identified CTs, let's read them Step 2. Image denoising Noise will be removed by means of the bilateral filter. Image is binarised by the adaptive threshold and then, Erosion and dilation operations are performed to remove noise. Step 3 performs a contour identification. Minimum enclosing circle technique. The CT center coordinates must be the same for all the circles. In order to compute the exact center, the momentum of the image is used, in particular, the centroid or center of mass of the CT The Step 4 will perform the analysis of the CT structure for its number decoding. 23

24 Examples 24

25 Biometrics 25

26 Biometrics Biometrics refers to metrics related to human characteristics. Biometrics authentication (or realistic authentication) is used in computer science as a form of identification and access contro, to identify individuals in groups that are under surveillance. Biometric identifiers are the distinctive, measurable characteristics used to label and describe individuals fingerprint, palm veins, face recognition, DNA, palm print, hand geometry, iris recognition, retina and odour/scent. biometric identifiers unique to individuals more reliable in verifying identity Challenge: personal privacy concerns about the ultimate use of this information. 26

27 Fingerprint Recognition 27

28 Face Recognition 28

29 Example 29

30 Iris Recognition %2Brecognition%2Bsystem%2Balgorithm.gif 30

31 Practical case: biomedical authentication based on retina pattern 31

32 Retinal Authentication 32

33 Practical case: Retinal Authentication Minority Report Future or reality? 33

34 The problem We want to use the retinal vessel tree to identify persons using the retina vessel pattern. 34

35 Thinking about the problem Images Segmentation issues Image acquisition Vessel Diseases Central reflex How do I get the minutae? How do I register and get the authentication? Validation of the system. How well do I perform? 35

36 My schema 36

37 Processing the retinal images Retinal images can be obtained by several methods We are only interested in those points of the vessels corresponding to junctions Authentication using images the same philosophy than fingerprints We only need crossing points 37

38 Challenges (I) Central reflex Small vessels 38

39 Challenges (II) Pathologies can make that the images under study does not comply with our minimum requirements Optic disk usually is not considered in global retinal process due to its particular brigthness 39

40 Our specific needs 40

41 Vessel skeleton Stage 1 is devoted to clean the vessels from noise, and to homogeneize vessel locations The results will be used to compute the initial contours and the external energy needed by the active contours Pixel Level Snakes are a specific active contours technique were all the pixels of the contours evolve No nodes are needed in this case 41

42 Phase 1. Vessel pre-estimation Diffusion acts as a kind of smoothing kernel Goal: remove noise due to the acquisition device The thresholded image is obtained through adaptive segmentation due to the high variability in the different areas of the retina 42

43 Phase 2. Computing the input for the Active Contours Active contours need Initial contour External Potential Initial contour must be close to vessel locations External Potential must guide the contours towards the real edges 43

44 Computing the External Energy Simple operations were used since only addition, substraction, multiplication and neighbourhood operations were available in the SCAMP device Distance mapping to the real edges was computed with several dilations over the detected edges 44

45 The evolution of the active contours was splitted in 2 steps Fast evolution (mainly guided by external energy only) Hole filling to remove noisy regions Slow evolution to fit the details 45

46 Retinal vessel skeleton 46

47 Computing the Skeleton 47

48 Minutae in Retina Skeletonised image barely is optimal Discontinuities usually appears Two types discontinuities Union of the same vessel Junctions 48

49 Results No possibility of fraud No false positives 49

50 Conclusion Segmentation techniques are in the core of a good range of computer vision applications, and in particular for object recognition Different images from industrial and from medical domain benefit from segmentation for Normalizing intensities Segmenting important objects for their applications Usually, computer vision engineers needs to thing in the big picture before start doing Think in your needs and in what you have Go for a simple approach, and then try to improve it moving towards more sophisticated algorithms to fulfill your needs After this course, you have a minimum knowledge about classical techniques used in the literature Try to play with the classical ones and move forward to more exciting approaches 50