Computer Vision at Cambridge: Reconstruction,Registration and Recognition

Transcription

1 Computer Vision at Cambridge: Reconstruction,Registration and Recognition Roberto Cipolla Research team

2 Cognitive Systems Engineering Cognitive Systems Engineering

3 Introduction New applications Computer vision has now found a place in consumer products Mobile phones and PDAs Games Cars Image and video search Internet and shopping

4 Smart erase on a mobile phone Introduction

5 I. Reconstruction: Recovery of accurate 3D shape from uncalibrated images Cipolla and Blake 1992 Cipolla and Giblin 1999 Mendonca, Wong and Cipolla Vogiatzis, Hernandez and Cipolla Hernandez, Brostow and Cipolla 2007

6 Digital Pygmalion the myth

7 Digital Pygamlion Project

8 Scanning technologies Laser range finders Very accurate Very expensive Complicated to use Minolta Michelangelo project MENSI 3D Scanner

9 3D models We need a way to get them that is practical fast non-intrusive low cost

10 Stereo vision 3D point

11 3D Models

12 1. Camera motion and multiview stereo Cipolla and Blake 1992 Cipolla and Giblin 1999 Mendonca, Wong and Cipolla Hernandez, Schmidt and Cipolla 2007 Vogiatzis, Hernandez and Cipolla 2007

13 Input Images

14 Recovery of camera motion Input images Feature extraction Feature matching Bundle adjustment

15 2. Probabilistic 3D segmentation using graph cuts Vogiatzis, Hernandez and Cipolla 2007

16 Probabilistic 3D segmentation

17 3D models

18 3D models

19 Advantages Low cost Non intrusive Accurate Simple Can work with about 5-30 images Fast Approximately minutes of computation for these models We believe we can bring this down to minutes

20 Application requirements In order of importance 1. Simplicity 2. Cost 3. Visual accuracy 4. Intrusiveness 5. Speed 6. Robustness 7. Geometric accuracy ~25 images, no calibration Digital camera + PC Sub-pixel Completely non-intrusive Recently down to 15min tranparency, deformations 1 part in 1000

21 3.Multiview photometric stereo Vogiatzis, Hernandez and Cipolla 2006

22 Untextured objects Almost impossible to establish correspondence

23 Untextured objects Changing lighting uncovers fine geometric detail Assumptions: Single, distant light-source Silhouettes can be extracted No texture, single colour

24 Image acquisition setup

25 Surface Evolution: 3D Mesh Evolve mesh until it is predicted appearance under recovered illumination matches images

26 3D Models

27 3D Models

28 3D Models

29 3D Models

30 3D Models

31 3D Models

32 II. Registration: Target detection and pose estimation

33 4. Registration: Where am I? What am I looking at? Johansson and Cipolla 2002 Robertson and Cipolla 2004

34 Where I am? Determine pose from single image by matching

35 Register database view

36 Registration of input image

37 Matching

38 Localisation of query view

39 Image-based localisation

40 Image-based localisation

41 Image-based localisation

42 5. Finding 2D shapes and applications to HCI Stenger, Thayananthan, Torr and Cipolla 2003 Williams, Blake and Cipolla 2003 and 2006 Ramanan, Fitzgibbon and Cipolla

43 Matching shape templates Oriented Oriented Canny Distance Edge Transform Detector

44 Matching shape templates Oriented Chamfer Matching

45 Hand detection system

46 Tracking - 3D mouse

47 Real-time visual controller for Dasher

48 People and pose detection

49 People and pose detection

50 III. Object recognition and machine learning Shotton, Blake and Cipolla Kim, Kittler and Cipolla 2006 Johnson and Cipolla 2007

51 road Overview image classification categorical object detection horses airplanes background semantic segmentation tree bicycle building grass dog car sky building road

52 6. Using interest points and visual words Johnson and Cipolla 2007

53 Image matching

54 7. Using contour and shape Shotton, Blake and Cipolla

55 Supervised learning Learn to recognise images of a particular class, localised in space and scale i.e. find the horse/cow/car etc! Desired Results

56 Cognitive Systems Engineering Learning and Adaptability : : } : :

57 Object Model F σ p x F = (T, p,,,, a, b)

58 Retraining Approach Dataset Training set Test set Build Codebook Learn Classifier Test Classifier Results

59 Unsegmented Segmented Dictionary of contour fragments

60 Unsegmented Segmented Shape (a) (b) (c) (d) (e) (f) Exemplar Centroid votes Exemplar Centroid votes Exemplar Centroid votes

61 Segmented Unsegmented Shape Exemplar Centroid votes Exemplar Sub-cluster members Centroid votes

62

63

64

65 8. Using texture and contour Shotton, Blake and Cipolla

66 Texture-based segmentation

67 Extracting textons clustering and assignment input image filter bank texton map (colours texton indices)

68 Extracted texton dictionary

69 Use with contour shape model sr 2 sr 3 sr 1 (x, s) t 1 t 2 textons

70 Learned contour and texture (1) (2) (3) (4) (5) (6) (7) (8) (9) (24)

71 True Positive Rate Performance ROC AUC Canny BEL False Positive Rate

72 Recall Performance RP AUC Canny BEL False Positives Per Image

73 Performance

74 9. Detection and tracking of people Brostow and Cipolla 2006 Bucciarelli and Cipolla 2007

75 Pedestrian detection

76 Tracking people in crowds Cognitive Systems Engineering

77 10. Recognition in video using CCA Arandjelovic and Cipolla 2006 Kim, Wong and Cipolla

78 Face recognition Overcome appearance variations due to: Lighting condition Scale, pose, motion pattern

79 Automatic cast listing Problem difficulties

80 : Automatic cast listing Simple clustering results

81 Action recognition

82 Action recognition Boxing Hand clapping Hand waving Jogging Running Walking

83 Summary Image registration and matching 3D shape from uncalibrated images. Object detection and tracking