Robert Collins CSE486, Penn State. Robert Collins CSE486, Penn State. Image Point Y. O.Camps, PSU. Robert Collins CSE486, Penn State.

Transcription

1 Stereo Vision Inerring depth rom images taken at the same time b two or more s. Lecture 08: Introduction to Stereo Reading: T&V Section 7.1 Scene Point Image Point p = (,,) O Basic Perspective Projection Scene Point Perspective Projection Eqns P = (,,) Basic Perspective Projection Perspective Projection Eqns P = (,,) Image Point p = (,,) Scene Point p = (,,) O p = (,,) O O.Camps, PSU P = (,,) k k k k Image Point Wh Stereo Vision? Perspective Projection Eqns P = (,,) O.Camps, PSU Basic Perspective Projection O O.Camps, PSU Fundamental Ambiguit: An point on the ra OP has image p O.Camps, PSU 1

2 Wh Stereo Vision? Wh Stereo Vision? P ~63mm p OL OR our two ees orm a stereo sstem The right and let ees see the world rom slightl shited vantage points. A second can resolve the ambiguit, enabling measurement o depth via triangulation. Ke Concepts or Toda Do-it-oursel Paralla Demo Paralla Anaglphs Random Dot Stereograms Mathematics o Simple Stereo Show: Points at dierent depths displace dierentl Nearb points displace more than ar ones A Hitchhiker s Guide to Paralla Paralla = apparent motion o scene eatures dierent distances General Idea o Stereo Iner distance to scene points b measuring paralla. INFER Ver distant mountain peak Ver small displacement Ver small displacement Far Midrange More distant tree Smaller displacement Smaller displacement Large displacement Large displacement Nearb guardrail Close 2

3 Anaglphs Anaglphs are a wa o encoding paralla in a single picture. Two slightl dierent perspectives o the same subject are superimposed on each other in contrasting colors, producing a three-dimensional eect when viewed through two correspondingl colored ilters Put red ilter over let ee 3

4 How Anaglphs Work Making an Anaglph Take a grescale stereo pair. Cop the let image to the red channel o a new image (the anaglph image) Close right ee, then close let. What do ou observe? Red ilter selectivel passes red color, and similarl or can ilter and can color. Stereo Pschophsics Cop the right image to the green and blue channels o the anaglph image (note: green+blue = can) Now when ou view with red-can glasses, the let ee sees onl the let image, and the right ee sees onl the right image. The brain uses to orm 3D. Higher-level Depth Cues How does stereo depth perception work? In particular, at what level in the visual sstem does it occur at? An earl debate: do we iner depth rom higher-level inormation like perspective and contours, or does it occur at a much lower level? "The basis o this three-dimensional perception was hotl debated between Wheatstone and ellow phsicist Sir David Brewster. (Though it ma seem odd or phsicists to concern themselves with the phsiolog o optics, this was elt to be a natural etension o the stud o the phsics o optics.) Brewster opined that perspective was the source o the apprehension o an object's shape. Wheatstone insisted that the images in the each ee had identiiable landmarks that were combined to assign depth to the landmarks. -- Ralph M. Siegel Cho ices: The Science o Bela Jules Perspective (vanishing points) 4

5 Higher-level Depth Cues Similar sied objects appear smaller at a distance (this is also related to perspective) Stereo Pschophsics Obviousl perspective and contours are important, (particularl or monocular depth perception), but are the necessar or binocular stereo depth perception? Higher-level Depth Cues Occluded contours (perceptual completion) Jules Random-Dot Eperiment Generate a random dot pattern using a computer Bela Jules answered this question in 1960 with his eperiments with random dot stereograms. In 1960, Bela's eperiment with what eventuall became known as Jules random dot stereograms unambiguousl demonstrated that stereoscopic depth could be computed in the absence o an identiiable objects, in the absence o an perspective, in the absence o an cues available to either ee alone. -- Ralph M. Siegel Cho ices: The Science o Bela Jules Jules Random-Dot Eperiment Clip out a square region and shit it to the let e.g. im = roicolor(rand(300,300), 0.5, 1); B deinition, this is just noise, so there are obviousl no monocular depth cues here. Jules Random-Dot Eperiment Clip out a square region and shit it to the let Fill in the hole let behind with more random dots. 5

6 Jules Random-Dot Eperiment Original dot image Dot image with shited square Now view as a stereo pair. Jules used a special viewer, but we will displa as an anaglph (get our glasses!) Make our Own %make an image with random dots im = roicolor(rand(300,300),.5,1); %second image starts as a cop o that im2 = im; %shit a square o piels to the right im2(100:200,110:210) = im(100:200,100:200); %ill in the "hole" with more random dots im2(100:200,100:110) = roicolor(rand(101,11),.5,1); %encode image2 in red channel o a color image ana = 255*im2; %encode image1 in blue and green channels ana(:,:,2) = 255*im; ana(:,:,3) = 255*im; %take a look (remember to wear our red/can glasses!) image(uint8(ana)) Stereograms Another method o encoding paralla in a single image. Subtle shits o repeated teture encode disparit o depths in a scene (a technique made amous under the Magic Ee brand name). Unlike anaglphs, ou don t need special glasses to see these, just some practice ocusing our ees behind the page. Tr this: what happens when ou shit the square to the let instead o to the right? Stereograms Give our ees a break beore we move on

7 A Simple Stereo Sstem A Simple Stereo Sstem Top Down View ( plane) let (0,0,0) P=(,,) T right (T,0,0) Let? l Right r Right is simpl shited b T units along the ais. Otherwise, the s are identical (same orientation / ocal lengths) T Translated b a distance T along ais (T is also called the stereo baseline ) Camps, PSU A Simple Stereo Sstem T right (T,0,0) Image coords o point (,,) in Let Camera: -T let (0,0,0) (,,) T What are image coords o that same point in the Right Camera? right (T,0,0) Insight: translating to the right b T is equivalent to leaving the stationar and translating the world to the let b T. Camps, PSU (-T,, ) (,,) let (0,0,0) A Simple Stereo Sstem Camps, PSU A Simple Stereo Sstem (-T,, ) Let -T let (0,0,0) (,,) T Stereo Disparit Right right (T,0,0) Stereo Disparit depth baseline disparit Important equation! Camps, PSU 7

8 Stereo Disparit Let Stereo Disparit / Paralla Tie in with Intro: or our purposes Disparit = Paralla Right Disparit/Paralla inversel proportional to depth Note: Depth and stereo disparit are inversel proportional this is wh near objects appear to move more than ar awa ones when the translates sidewas depth disparit Important equation! 8