EECS 442 Computer Vision fall 2011

Transcription

1 EECS 442 Computer Vision fall 2011 Instructor Silvio Savarese Office: ECE Building, room: 4435 Office hour: Tues 4:30-5:30pm or under appoint. (after conversation hour) GSIs: Mohit Bagra Murali Telaprolu Class Time & Location Tu Th 3:00PM - 4:30PM -- G906 COOL Conversation hour [it s part of the course!] Wed 3:30PM - 4:30PM EECS

2 EECS 442 Computer Vision fall 2011 Text books: [FP] Computer Vision, A Modern Approach, by D.A. Forsyth and J. Ponce, Prentice Hall, [HZ] Multiple View Geometry in Computer Vision, by R. Hartley and A. Zisserman, Academic Press, 2002 Also: Computer Vision: Algorithms and Applications, R. Szeliski, Springer, 2011 If you plan to audit this class, please signup your name on the mailing list

3 Agenda Administrative Grading policy Project What is computer vision? Syllabus

4 Grading policy Homeworks: 40% 5 homeworks Mid term exam: 10% [end of October] Course project: 45% progress report 5% final report 30% presentation 10% Attendance and class participation: 5% Questions, answers, remarks Late policy home works: If 1 day late, 50% off the grade for that homework Zero credits if more than one day. Late policy project: If 1 day late, 25% off the grade for the project If 2 days late, 50% off the grade for the project Zero credits if more than 2 days Collaboration policy Read the student code book, understand what is collaboration and what is academic infraction. Discussing project assignment with each other is allowed, but coding must be done individually Home works or class project coding policy: using on line code or other students/researchers code is not allowed in general. Exceptions can be made and individual cases will be discussed with the instructor.

5 Course Project Replicate an interesting paper Comparing different methods to a test bed A new approach to an existing problem Original research Write a 8-page paper summarizing your results Release the final code Give a presentation We will introduce projects in two weeks Important dates: look up class schedule

6 Course Project Form your team: 1-3 people the quality is judged regardless of the number of people on the team be nice to your partner: do you plan to drop the course? Evaluation Quality of the project (including writing) Final ~15 minutes project presentation in class students will vote your presentation! For final code and paper due dates please consult webpage

7 Agenda Administrative Grading policy Project What is computer vision? Syllabus

8 There was a table set out under a tree in front of the house, and the March Hare and the Hatter were having tea at it. The table was a large one, but the three were all crowded together at one corner of it From A Mad Tea-Party Alice's Adventures in Wonderland by Lewis Carroll Illustration by Arthur Rackham

9 Computer vision Image/video Object 1 Object N - semantic -semantic

10 Computer vision Image/video Object 1 Object N - semantic -geometry -semantic -geometry

11 Computer vision Image/video Object 1 Object N - semantic -geometry -semantic -geometry spatial & temporal relations

12 Computer vision Image/video Object 1 Object N - semantic -geometry -semantic -geometry spatial & temporal relations Scene -Semantic - geometry

13 Computer vision Computer vision studies the tools and theories that enable the design of machines that can extract useful information from imagery data (images and videos) toward the goal of interpreting the world Scene Objects People Actions Sensing device Computational device Extract information Interpretation Information: visual cues, 3D structure, motion flows, etc Interpretation: recognize objects, scenes, actions, events

14 Semantic Actions, Events Have we reached humans? not yet computer vision is still no match for human perception but catching up, particularly in certain areas Categorization Object Recognition 3D modeling 3D scenes Physical attributes

15 Successful Applications Finger prints recognizer Sources: L. Fei-Fei

16 Medical Imaging Sources: L. Fei-Fei

17 Special effects movies - videogames Sources: L. Fei-Fei

18 Consumer applications

19

20 Auto Stitch

21 Toy & Robots

22 Applications of computer vision Factory inspection Assistive technologies Surveillance Autonomous driving, robot navigation Sources: K. Grauman, L. Fei-Fei, S. Laznebick Driver assistance (collision warning, lane departure warning, rear object detection) Security

23 Automatic control Robotics Robot vision Signal processing Compression Non linear SP Multi-variate SP Neurobiology Biological vision Visual Psychophysics Data mining Image retrivial Computer vision Visual pattern recognition Machine learning Artificial intelligence Statistics Geometry Optimization Applied math Optics Smart cameras Acquisition methods Physics Imaging Computer graphics

24 EECS 442 course overview 1. Geometry 2. Low & Mid-level vision 3. High level vision

25 EECS 442 course overview 1. Geometry 2. Low & Mid-level vision 3. High level vision Geometry: - How to extract 3d information? - Which cues are useful? - What are the mathematical tools?

26 Visual cues: texture shading contours shadows reflections

27 Visual cues: texture shading contours shadows reflections

28 Visual cues: texture shading contours shadows reflections

29 Vision techniques Visual cues: texture shading contours shadows reflections

30 Visual cues: texture shading contours shadows reflections

31 Vision techniques Visual cues: texture shading contours shadows reflections Number of observers: monocular multiple views camera

32 Vision techniques Visual cues: texture shading contours shadows reflections Number of observers: monocular multiple views camera 1 camera 2 camera N

33 Stereo Epipolar geometry Tomasi & Kanade (1993) Structure from motion Image sources: S. Laznebick Projective structure from motion: Here be dragons!

34 Structure from motion Courtesy of Exford Visual Geometry Group

35 Vision techniques Visual cues: texture shading contours shadows reflections Number of observers: monocular multiple views Active lighting: laser stripes structured lighting patterns camera Laser/projector/light

36 3D Scanning Scanning Michelangelo s The David The Digital Michelangelo Project BILLION polygons, accuracy to.29mm Courtesy of Stanford computer graphics lab

37 3D Digital models Architecture Entertainment Medicine

38 The yellow line in superbowls

39 The yellow line in superbowls

40 Course overview 1. Geometry 2. Low & Mid-level vision 3. High level vision Mid-level vision: - Extract useful building blocks - Region segmentation - Motion flows

41 Extract useful building blocks

42 Mid-level vision Extract useful building blocks Alignment Extract planar regions Object segmentation

43 Image enhancement

44 Automatic Panorama Stitching Sources: M. Brown

45 Automatic Panorama Stitching Sources: M. Brown

46 Tracking and 3D modeling Courtesy of Jean-Yves Bouguet Vision Lab, California Institute of Technology

47 Camera tracking and V.R. insertions Courtesy of Exford Visual Geometry Group

48 Course overview 1. Geometry 2. Low & Mid-level vision 3. High level vision High level operations Recognition of objects and people Places Actions & events

49 Object recognition and categorization Building Downtown chicago clock person car Pedestrians crossing street

50 Challenges: viewpoint variation Michelangelo slide credit: Fei-Fei, Fergus & Torralba

51 Challenges: illumination image credit: J. Koenderink

52 Challenges: scale slide credit: Fei-Fei, Fergus & Torralba

53 Challenges: deformation

54 Challenges: occlusion Magritte, 1957 slide credit: Fei-Fei, Fergus & Torralba

55 Challenges: background clutter Kilmeny Niland. 1995

56 Challenges: object intra-class variation slide credit: Fei-Fei, Fergus & Torralba

57

58 Face recognition

59 Recognizing scenes

60 Sorting out millions of images/videos Personal photo albums Movies, news, sports Surveillance and security Medical and scientific images Credit slide: S. Lazebnik

61 Choi & Shahid & Savarese WMC 2010 Choi & Savarese, ECCV 2010 Detecting and tracking people

62 Recognizing human activities Biking Walking Walking Credit slide: Song & Perona

63 There was a table set out under a tree in front of the house, and the March Hare and the Hatter were having tea at it. The table was a large one, but the three were all crowded together at one corner of it From A Mad Tea-Party Alice's Adventures in Wonderland by Lewis Carroll

64 Syllabus See webpage

65 Next lecture - Review of linear algebra for multi-view geometry - Basic image transformations

66 Homework 0.1: Who painted this? Or what is a smart way to search an image on the web?

67

68 Homework 0.1: Who painted this? Or what is a smart way to search an image on the web?