Basilio Bona DAUIN Politecnico di Torino

Transcription

1 ROBOTICA 03CFIOR DAUIN Politecnico di Torino

2 Mobile & Service Robotics Sensors for Robotics 3

3 Laser sensors Rays are transmitted and received coaxially The target is illuminated by collimated rays The receiver measures the time of flight (back and forth) It is possible to change the rays direction (2D or 3D measurements) D D Transmitter L Receiver l = c f ( L + D ) + 2 D = ( L + D ) + q 2p l 3

4 Laser sensors l plitude Amp 0 q Transmitted Reflected Phase 4

5 Laser sensors METHODS Pulsed laser: direct measurement of time of flight: flight: one shall be able to measure intervals in the picoseconds range Beat frequency between a modulating wave and the reflected wave Phase delay dl It is the easiest implementable method 5

6 Laser sensors l c = ; D = L+ 2D = L+ f q 2p l c = speed of light f = frequency of the moduling wave D = total distance f = 5 MHz; l = 60 m Theconfidence on distance estimation is inversely proportional to the square value of the received signal amplitude 6

7 Laser sensors A typical image from a rotating mirror laser scanner. Segment lengths are proportional to the measurement uncertainty t 7

8 Triangulation Triangulation i is the process of determining i the location of an object by measuring angles from known points to the object at either end of a fixed known baseline The point can be chosen as the third point of a triangle with one known side and two known angles In practice: Light sheets (or other patterns) are projected on the target Rfl Reflected tdlight is captured dby a linear or 2D matrix ti light sensor Simple trigonometric relations are used to compute the distance 8

9 Triangulation Triangulation concepts l baseline = d d ; d l tan a + tan b = tan a tan b 9

10 Triangulation sin a sin b sin g = = BC AC AB AC AB sin b = ; BC = sin g AB sin sin g a RC = AC sin a RC = BC sin b RC = AB sin a sin g sin b RC = AB sin a sin sin( a + b ) b 10

11 Triangulation f D Transmitter L x D = L f x f 11

12 Structured light 12

13 Structured light H = Dtan a 13

14 Structured light Monodimensional i case D f cota -u f u x = f Du cot a -u a z = f Df cota -u 14

15 Vision Vision is the most important sense in humans Vision includes three steps Data recording and transformation in the retina Data transmission through the optical nerves Data elaboration by the brain 15

16 Natural vision Retina 16

17 Natural vision fmri shows the brain areas interested by neural activity associated to vision Optic chiasm 17

18 Artificial vision Camera = retina Frame grabber = nerves CPU = brain 18

19 Vision sensors: hardware CCD (Coupled Charge Device, light sensitive, e discharging capacitors of 5 to 25 micron) CMOS (Complementary Metal Oxide Semiconductor technology) 19

20 Artificial vision Projection from a 3D world on a 2D plane: perspective projection (transform matrix) Discretization effects due to transducer pixels (CCD or CMOS) Misalignment errors Parallel lines Converging lines Pixel discretization 20

21 Artificial vision p p F 3D object p Optical axis Reversed image plane Focal Plane Principal image plane 21

22 Artificial vision Geometric parameters P R Optical axis m x m O m xc Cc R c f x i i i O i x i t c i i i R i j i P Focal plane p F R R i O i j i p Image plane 22

23 Artificial vision T A R m c P Several rigid and perspective transformations are involved T B p i Rescaling i Optical correction 23

24 Artificial vision x x x c i c = z = f c z f x c i k c A i c z P z c C c x i x c P p pf C i p P f f 24

25 Artificial vision i i x x y y 25

26 Artificial vision Image parameters O i p x j i p y i i t c i i C i j i 26

27 Artificial vision Aberration types Pincushion distortion i Barrel ldistortioni Radial distortion Non radial distortion (tangential) Radial distortion is modelled by a function D(r) that affects each point v in the projected plane relative to the principal point p, where D(r) is normally a non linear scalar function and p is close to the midpoint of the projected image. Barrel projections are characterized by a positive gradient of the distortion function, whereas pincushion by a negative gradient v = D ( v- p ) v + p d 27

28 Artificial vision Image errors Errors are due to the imperfect alignment of pixel elements 28

29 Vision sensors Distance sensors Depth from focus Stereo vision Motion and optical flow 29

30 Depth from focus The method consists in measuring the distance of an object evaluating the focal length adjustment necessary to bring it in focus Short distance focus Medium distance focus Far distance focus 30

31 Depth from focus = + f f D e D L L D (,, xyz) image plane ( x, y ) i i focal plane L ( d + e ) D d e bx ( ) = f ( d + e) s( x) blur radius shape 31

32 Depth from focus Near focusing Far focusing 32

33 Stereo disparity ( x, y, z) left lens f z x right lens image plane ( x, y ) ( x, y ) r r b baseline (known) 33

34 Stereo disparity Idealized camera geometry for stereo vision x /2 x x + b r x - b /2 =, = f z f z x - x r b = f z ( x + x )/2 r x = b x - x r ( y + y ) r )/2 y = b y - y r z b f = x - x f ( ) Disparity between two images Depth computation r 34

35 Stereo vision Distance is inversely proportional to disparity closer objects can be measured more accurately Disparity is proportional to baseline For a given disparity error, the accuracy of the depth estimate increases with increasing baseline b However, as b is increased, some objects may appear in one camera, but not in the other A point visible from both cameras produces a conjugate pair Conjugate pairs lie on epipolar line (parallel to the x axis for the arrangement in the figure above) 35

36 Stereo points correspondence Th i are corresponding: di These two points how do you find them in the two images? Left image Right image Disparity Right Ri ht Left 36

37 Epipolar lines P corresponding points stay on the epipolar lines p t t 1 2 q q 2 C 1 epipolar lines e 1 Rt, e 2 these two points are known and fixed (they are called epipoles) C 2 37

38 Stereo vision Depth calculation The key problem in stereo vision is how to optimally solve the correspondence problem Corresponding points lie on the epipolar lines Gray Level Matching Match gray level features on corresponding epipolar lines Zero crossing of Laplacian of Gaussians is a widely used approach for identifying the same feature in the left and right images Brightness = image irradiance or intensity I(x,y) is computed and used as shown below 38

39 Laplacian The Laplacian is a 2D isotropic measure of the 2 nd spatial derivative of an image The Laplacian of an image highlights regions of rapid intensity change and is often used for edge detection The Laplacian is often applied to an image that has first been smoothed hdwith ihsomething approximating i a Gaussian smoothing filter, in order to reduce its sensitivity to noise The operator normally takes a single gray level image as input and produces another gray level image as output 39

40 Laplacian The Laplacian L(x,y) of an image with pixel lintensity it values I(x,y) is given by: Lxy (, ) 2 2 I I = + x y L = P ÄI 2 2 é0 1 0ù é 1 2 1ù P = 1 - G = Convolution 16 ê ú ë û operator ê ë ú û é 1 1 1ù P 2 = êë úû 40

41 Convolution Convolution is a simple mathematical operation which is fundamental to many image processing operators Convolution multiplies together two arrays of numbers, generally of different sizes, but of the same dimensionality, to produce a third array of numbers of the same dimensionality This can be used in image processing to implement operators whose output pixel values are simple linear combinations of certain input pixel values In an image processing, one of the input arrays is normally just the gray level image. The second array is usually much smaller, and is also two dimensional (although it may be just a single pixel lthick), and is known as the kernel 41

42 Convolution 42

43 Convolution matrix Iij (, ) IMAGE Kij (, ) KERNEL If the image has M rows and N columns, and the kernel has m rows and n columns, then the size of the output image will have M - m + 1 rows, and N - n + 1 columns (6-2+ 1) (9-3+ 1) =

44 Convolution product k11 k12 k13 k21 k22 k23 k11 k12 k13 k21 k22 k23 Oi (, j ) m n åå Oij (, ) = Ii ( + k- 1, j+ l-1) Kkl (, ) k= 1 l= 1 i = 1,, ( M - m + 1); j = 1,, ( N - n + 1) 44

45 Stereo vision Zero crossing of Laplacian of Gaussian Identification of features that are stable and match well Laplacian of intensity image Step/edge detection ec of noisy image: filter through Gaussian a smoothing 45

46 Edge detection 46

47 Stereo vision L R VERTICAL FILTERED IMAGES Confidence image Depth image 47

48 Optical flow Optical flow or optic flow is the pattern of apparent motion of objects, surfaces, and edges in a visual scene caused by the relative motion between an observer (an eye or a camera) and the scene Optical flow techniques such as motion detection, object segmentation, time to collision and focus of expansion calculations, motion compensated encoding, and stereo disparity measurement utilize this motion of the objects surfaces, and edges 48

49 Optical flow The optical flow methods try to calculate the motion between two image frames which are taken at times t and t + δt at every voxel position. These methods are called differential since they are based on local ltaylor series approximations of the image signal, i.e., they use partial derivatives with respect to the spatial and temporal coordinates Ixyt (,,) = Ix ( + dxy, + dyt, + dt) I I I = Ixyt (,, ) + d x + d y + d t +... x y t I I I dx + dy + dt = x y t 0 Avoxel(volumetric + pixel) is a volume element representing a value on aregular grid in 3D space 49

50 Optical flow I I V + V + I = 0 x y x y t T I V =- I t This problem is known as the aperture problem of the optical flow algorithms There is only one equation in two unknowns and therefore cannot be solved To find the optical flow another set of equations is needed, given by some additional constraint. All optical flow methods introduce additional conditions for estimating the actual flow. 50

51 Optical flow Lucas Kanade Optical Flow Method A two-frame differential methods for motion estimation The additional constraints needed for the estimation of the flow are introduced in this method by assuming that the flow V, V is constant x y in a small window of size with m > 1 which is centered at pixel (, xy) 2 Numbering the pixels as a set of equations can be found 1,, n = m I V + I V =-I ü é x1 x y1 y t I I ù é I ù x1 1 t 1 y 1 I V + I V =-I I I V I x2 x y2 y t ú é ù 2 x2 y2 x t2 ý ï úê =- úêv y úêë úû I V + I V =-I I I I xn x yn y t xn yn t nïþ ï êë úû ë ê n ú û 1 ( T - T Ax = b x = AAA ) Ab 51