Computer Vision EE837, CS867, CE803

Transcription

1 Computer Vision EE837, CS867, CE803

2 Introduction Lecture 01 Computer Vision

3 Prerequisites Basic linear Algebra, probability, calculus - Required Basic data structures/programming knowledge - Required Working knowledge of MATLAB - Required Knowledge and understanding of basic image processing - Preferable

4 Text and Reading Class slides, research papers, tutorials and supplemental material Linda G. Shapiro and George Stockman, Computer Vision, Upper Saddle River, NJ: Prentice Hall, David A. Forsyth and Jean Ponce, Computer Vision A Modern Approach, 2nd edition, Prentice Hall, Inc., Richard Szeliski, Computer Vision: Algorithms and Applications, Springer; 2011 edition, Made available online by the author: Extra List of CV books

5 Broader course topics Camera geometry and basic transformations Camera calibration and camera- parameters estimation Sources, shadows, shading and shape from shading Feature Extraction Texture Synthesis Template Matching and Image Registration Segmentation Vision based Tracking Multiple view geometry

6 Grading policy Homework Mostly programming assignments: Midterm/Hourly: 15% Surprise quizzes/attendance: 10% Final Project: 30% Final exam: 30% 15%

7 Grading policy Homework/programming assignments: Reports should be type-written Code and program output are required Final Project: Brain storm on project ideas Project highlights 10 minute each group Individual or a group of max two with individual roles clearly defined Type-written report upto 10 pages in CVPR format, with additional pages for commented codes as appendix. Project presentation Late Policy: No credit for late submissions

8 Grading policy Plagiarism is strictly prohibited Cite the source Negative marking will be done, where found

9 Material citations Dr George Stockman Professor Emeritus, Michigan State University Dr Mubarak Shah Professor, University of Central Florida The Robotics Institute Carnegie Mellon University

10 Any queries? By appointment only: Cdr Dr Hammad PG 111 Preferably Tuesday and Wednesday11AM Noon

11 Lets start!!!

12 What is an image? What we see What a computer sees

13 What is an image? What we see What a computer sees

14 Where is the Sun?

15 Image Processing Fourier Transform Sampling, Convolution Image enhancement Feature detection

16 What is Computer Vision? Inverse Optics Intelligent interpretation of Imagery Building a Visual Cortex Part of the cerebral cortex responsible for processing visual information No matter what your definition is Vision is complex.but is FUN!!!

17 Difference between CV and IP Image processing: Process the output of sensors. Computer vision: Relates the output of the sensors to real world. Image processing: The output is a transformed image. Computer vision: The output is usually a decision. Image Processing: Signal processing. Computer Vision: Artificial Intelligence. Defect detection or automatic driving relates to? Enhancing an image relates to?

18 Components of a Computer Vision System Camera Image acquisition Lighting Scene Computer Scene Interpretation

19 Video clip

20 Sequence of images 16 images in succession that shows motion

21 Shape from shading Shade deceives human visual system Changes the 3D shape Gradual variation of the shading gives 3D information

22 Shape from texture (1,0,1) (-1,1,1) (-1,-1,1)

23 3D from Shading Shape from Shading

24 Shape from texture Same shape (circles) repeated, forms texture Circles become ellipses at some places Gives 3D cue Texture can be used to recover 3D

25 Shape from motion Cannot understand just from dots that what it is Humans have this capability to understand motion

26 Shape from motion

27 Optical flow Sequence Color wheel Completing pixel wise motion Raw optical flow

28 Optical Flow

29 Microsoft photosynth Panorama stitching Can capture in amazing resolution and full 3D. For anyone with a D-SLR (Single Lens Reflex) or a point-and-shoot camera.

30 Video clip and mosaic Stitching images together

31 Applications of Computer Vision Face Recognition Object Recognition Video Surveillance and Monitoring Object detection, tracking and behavior analysis Remote Sensing: UAVs Robotics Computer Graphics And more.

32 Face Recognition Principle Components Analysis (PCA) Fisher Linear Discriminant (FLD)

33 Face recognition

34 Facial expression Surprised Smiling

35 Detecting driver alertness

36 Human detection Left UAV image Bounding boxes Will learn basic techniques on how we can track these moving objects

37 Video surveillance and monitoring Object detection Object tracking Object classification Automated surveillance systems Detection and tracking Activity recognition

38 Airport surveillance

39 Aerial imagery - UAVs Drones Military use Instead of drones many want to brand the technology as "Unmanned Aerial Systems" (UAS) in preference over "drones. Aerial surveying of crops Acrobatic aerial footage in filmmaking Search and rescue operations Inspecting power lines and pipelines Counting wildlife Delivering medical supplies to inaccessible regions

40 Aerial imagery Camera motion compensation Feature based + gradient Motion detection Object tracking Frame differencing + background modeling + object segmentation Kernel tracking + blob tracking + occlusion Event detection and tracking

41 Aerial imagery Registration results

42 Aerial imagery Detection results

43 Aerial imagery Tracking results

44 Wide area surveillance

45 Wide area surveillance

46 Tracking results

47 Unmanned Ground Vehicle Comes under Robot vision Google Self driving car The system combines information from Google Street View with artificial intelligence software that combines input from: Video cameras inside the car Identifying pedestrians and moving obstacles LIDAR sensor on top of the vehicle For 3D map Radar sensors on the front of the vehicle Position of distant objects Position sensor attached to one of the rear wheels Locate the car's position on the map.

48 Unmanned Ground Vehicle Defense Advanced Research Projects Agency (DARPA) urban challenge

49 Human activity recognition Involves Events Actions Activities Different datasets available for analysis

50 Human activity recognition - datasets Weizmann action dataset 10 actions 09 actors per action KTH Data Set 06 categories 25 actors 04 instances 600 clips

51 Human activity recognition - datasets UCF Sports dataset 9 actions 142 videos Bench swing Kick IMAX multi-view dataset Dive Lift Swing Ride Golf swing Run Skate

52 Human activity recognition - datasets UCF 50

53 Stereo Regular camera lose 3D information Microsoft Kinect sensor game changer Gives direct 3D information + RGB image 50,000 different gestures Challenge is that can you identify all/some of these IR LED Emitter 3D depth sensors Tilt motor Array of microphones RGB Camera

54 Binocular Stereo

55 Stereo Regular camera lose 3D information

56 Range Scanning and Structured Light

57 High density crowded scenes Tracking required for: Crowd management Public space design Virtual environments Visual surveillance Intelligent environments And more!!!

58 High density crowded scenes Can we do tracking in this kind of crowd? Political Rallies Religious Festivals Marathons High Density Moving Objects

59 High density crowded scenes Can we do tracking in this kind of crowd? Average chip size 14 x 22 pixels 492 Frames Selected 199 athletes for tracking Successfully tracked 143 athletes

60 High density crowded scenes Can we do tracking in this kind of crowd?

61 High density crowded scenes Can we do tracking in this kind of crowd? Average chip size 14 x 17 pixels 453 Frames Selected 50 athletes for tracking

62 High density crowded scenes Can we do tracking in this kind of crowd?

63 Behaviors in crowded scenes Can we identify the behavior of the crowd?

64 Image localization Input Output Location in terms of Longitude ( ) Latitude ( ) Image compared with database of images

65 Geospatial trajectory extraction Sequence of images compared with database

66 Computer graphics CV used for movies like Harry Potter, Avatar, Matrix etc

67 Layer based image composition Segmentation method Green Chroma key screen Green and blue differ the most in hue from skin colors Virtual studio y

68 Layer based video composition Segmentation method

69 Layer based video composition Segmentation method

70 Industrial robots vs low skilled workers