Derivation of perspective stereo projection matrices with depth, shape and magnification consideration

Transcription

1 Derivatio of perspective stereo projectio matrices with depth, shape ad magificatio cosideratio Patrick Oberthür Jauary 2014 This essay will show how to costruct a pair of stereoscopic perspective projectio matrices for a computer visualizatio eviromet like OpeGL. These matrices eed to be i such a maer, that the whole visualizatio chai delivers correct ad predictable images. The chai goes from object-space through left ad right chaels image-space ad fially percepted through the eyes ad calculated back by brai to the viewig-space, the space which we relet as a three dimesioal space. Primarily the backgroud was to adapt a existig 3D computer visualizatio to a stereoscopic system. Therefore I treat all geometric issues ecessary for this adaptio. If you ot iterested i the backgrouds, oe ca fid both fial fuctios with all parameters i sectio 5. Techical details of the visualizatio systems are ot discussed here, but ca be foud o the project-homepage of the visualizatio[bvsim]. Importat issues for stereo chais regardig to geeral computer visualizatios are viewed i the last chapter. 1

2 U U U U 1 Problem Sice the applicatio already has a scee ad all objects of a existig visualizatio eviromet are put i place, it is about to replace the sigle-image projectio matrix as described i sectio 2 with two differet projectio matrices. Object Space O P³ (w, q, f,, t, h) M l M r Image Space S l R² Image Space S r R² both image Spaces have the same properties (e, V, s) T cbd 2014 Patrick Oberthür Viewig Space i R³ T will extract relative depth values from horizotal differeces i both images (parallax) Figure 1: trasformatio chai So the mai problem S is about the search of a suitable pair of projective fuctios (perspective projectio matrices) M l : O P 3 R 4 S l R 2 M r : O P 3 R 4 S r R 2. where S l, S r are the scree surfaces of the viewig system called Image Space ad P 3 the three dimesioal projective Space. O is the Object space, a subset of the Projective Space where we ca describe poits, lies ad surfaces. The eyes ad brai are processig both surfaces together to the viewig space I via Trasformatio T. The problem has restrais i all spaces ad parameters from the visualized models also. These are packed together i Ω. So we eed to solve C : Ω R 3 R 3 T : S l S r R 3 so that objects O i the chai seems to be uchaged: 2

3 S(O, Ω) O. This makes a good uderstadig of the Trasformatio T ecessary, sice S depeds o it: S(O, Ω) = T (M l, M r ). Sectio 3 shows us how T works. 2 A mooscopic projectio ad simple stereo approach Sog Ho Ah has a very good derivatio [Ah] of the OpeGL Projectio Matrix for Trasformig object coordiates ito scree space. It gives a correct perspective Projectio ad NDC-Trasformatio with all parameters leadig i symmetric case to M = r t (f+) 2f. I geeral this matrix could be traslated or rotated to chage the viewpoit. But there are good reasos ot to do so as described i 6. A first approach for a stereoscopic setup, so a left ad right projectio is just to apply a traslatio after the matrix-operatio to the left as to the right i the same distace d to simulate two eyes i distace 2d: M l = r t (f+) d 2f, M r = r t (f+) d 2f. This works a little, but there is either a predicted outcome, or the distacepaes are correct. Sice this maual setup ad playig aroud with parameters is ot very efficiet, it is about to aalyze stereoscopic image perceptio to kow what happes here. 3

4 cbd 2014 Patrick Oberthür q x f -w -1 -q object to image-space trasformatio 0-1 y dc qM rasterizatio 2wM Figure 2: OpeGL dc-perspective projectio ad scalig rasterizatio 3 The stereoscopic projectio ad perceptio model The first cosideratio of stereoscopic imagig ad part of every paper i stereoscopy is doe by Joh T. Rule [Rule]. Also based o this, a ew work criticizig the iterative process of set up the two matrices or cameras for every ew scee as ot efficiet ad expesive by Smith ad Collar [SmiCo] will be used for further discussios. It aalyze the complete chai from object to image-space for a camera setup. They searched for the depth magificatio ad shape-ratios i this setup. Good ews is, that the geometric study of the perceptio part, what we call T is idepedet from the part of creatig the images ad it is therefore for every pair of images the same. We will first cosider the filmig eviromet. We eed to trasfer it from filmig setup to computer-visualizatio eviromet. There the image sesor area is replaced with the surface-area S r ad S l also from behid the focal poits to frot of them. So we still use the two matrices from last sectio. The differece here is that we got o sesor offset h respectively it is zero i this setup. Our projectio-matrix-horizotal shift d is oly the iteraxial 4

5 z cbd 2014 Patrick Oberthür depth rage C f h -h left ad right imagers (ormalized device buffers) z=0 -t/2 t/2 x Figure 3: stereo image creatio, chaged fig.1 i [SmiCo] from camera setup to computer-visualizatio eviromet camera separatio. So first we eed to fid the projectio matrix based also o the sesor shift. Istead of calculatig it, just look of what is happeig i the camera setup: first the camera is moved (T c ) ad after the camera trasformatio(m) we Traslated the sesor (T s ). So everythig we eed is to calculate T s (M(T c )) where T s is alog X-axis with distace h ad ad T c is alog X-Axis ad distace t/2 = d. Be aware that whe movig from back to frot T s is applied reverse. Ufortuately a sesor shift is ot doe this easy. It is about chagig the left ad right parameters i both matrices, so we eed to go a step backward i Sog Ho Ahs work ad use l l, r l as left ad right parameters for both matrices i a o-symmetric i x, still symmetric i y (t = q, b = q)) projectio: 5

6 M l = 2 r l l l q r l +l l r l l l (f+) 2f, M r = 2 r r l r q r r+l r r r l r (f+) 2f. The appliace of x-shift o l ad r is easy doe by l l = w/2 + h r l = w/2 + h l r = w/2 h r r = w/2 h where w is the width of the viewport. With these the quotiets i M l ad M r ad the fial camera movemet we got both matrices complete as M l = 2 w 2h w t/2 q (f+) 2f, M r = 2 w q 2h w t/2 (f+) 2f. We ca derive a covergece distace of C = t 2h This is the essetial distace where all objects appears o the distace of the scree. With that you ca predict whether a object will appear o, behid or i frot of the scree. So this will become more importat, whe 2D elemets,like i our case overlays or meus, are used ad stay i the same absolute positio for both chaels. The iterferece with 3D cotet will give the costrai, that the ear-pae will at least be behid this distace. ad we ca use ow all formulas of Smith ad Collar s work i their otatio. There we ca fid for the secod trasformatio T the virtual depth Z i i the viewig eviromet, as the virtual X i value with 6

7 Z i = V e e P = X i = e(x Sl + X S r) 2(e P ) V ez O Mft Z O (2Mh e) where P is the distace of a related poit pair, the left ad right scree poits X Sl, X Sr of the same object. The they replaced it with the values from the object to scree values to V ez O Z i = Mft Z O (2Mh e) MefX O X i = Mft Z O (2Mh e) ad this gives us the opportuity to look, how distaces i x ad z chages durig the complete trasformatio. The ext step would be to derive all the relatios betwee the ear-pae, far-pae f, covergece distace C, les-focal-legth, image-sesor-offset h also called horizotal image Traslatio (HIT) ad iter-axial-camera-separatio t. 4 Scee settig, ear- ad farpae issues The les-focal-legth i Smith ad Collar used to be f. Sice our viewig port begis with the ear-pae which is idetical to the sesor-width, their focal legth f will be replaced with our ear-pae distace. Istead of defiig width or height of the projectio-widow (ear-pae properties) it is more commo to use a fixed diagoal opeig agle 2ω ad derive width ad height through the aspect ratio r = q/w because it is more similar to a les-setup ad oe has a idea of what the outcome will look like. So you have to deploy all these i w = ta (ω) r q = w r 7

8 ad solve it. It is still to clarify how to choose the ear ad far pae. The ear pae is, as we said the view port ad therefore a mai factor for magificatio, i moo as i stereo setup. It is already visible i all matrices, that the larger gets, the larger objects, respectively vertical distaces becomes, as it is a output-factor for x ad y. The far pae does ot have a ifluece o the magificatio, it oly effects z-buffer issues, as described i sectio 6, so it should be as small as possible so that all object appear i the coe. Also we eed to regard it later i the shape-ratio discussio. So ow lets look at the remaiig stereo parameters h ad t. Therefore we eed iformatio about scree-to-eye-distace V, scree-sizew s ad eyedistacee. Sice we do t kow them, we will keep them set able i the program or use statistical values. Maily our goal is to keep all objects udistorted but scalable. A x-y-distortio is preveted by applyig the correct screeshape-ratio. But as Smith ad Collar have show there could be a distortio i depth-directio z. They calculated the shape-ratio S ad request that it should be hold early at oe: 1 S = M D M W 3D = V t Mt Z O (2Mh e) Ufortuately the depth magificatio is either costat or liear ad depeds o the object-depth Z O. It chages with Z but ca be hold early costat. So it eeds to be cosidered aroud a certai depth pae of a Object Z 0. Figure 1 shows the Shape-ratio with differet Parameter-sets. Whe we defie a = V t b = Mt c = 2Mh e 8

9 S becomes S(z) = a b zc ad that shows, that to reach S 1 we should keep c early zero ad the a b what meas V M e 2Mh. Sice M depeds o ad V is a property of the viewig scee, it is about to chage the ear-pae. Iterestig is, that chagig the iteraxial cameradistace wo t affect the shape ratio. Also the eye-distace is ot chageable, o average 63mm, we got a Fuctio for the Horizotal image traslatio (HIT): h = e 2M Also whe it is useful to itesify the depth-impressio, mostly doe by icreasig t till it hurts, because it more complicated to fid similar surfaces to match ad orietate, our goal is, that all objects appear i proper shape. So the better way to icrease depth-impressio is to move earer to objects or icrease their size ad hold their positio i the Z S=1 pae. Sice our objects have a measurable size, we have i our object-space a referece ad therefore a idea of the image sesor size w O. This delivers us the sesor to scree magificatio M = w S w O = s w Sice shape ratio of image sesor ad scree is the same, we ca use the width of ear-pae w ad the width of the scree s ad fialy we got Still left is the ear pae, which is ow h = ew 2s. (1) 9

10 = V M = V w s. (2) As already told, the iteraxial camera distace wo t affect the shape ratio issue. But sice fulfillig 1 ad 2 is ot possible i all cases, but still to keep shape ratio it is recommeded to icrease t. But at a certai poit there will be too less area o both chaels to match, depedig also o the opeig agle ω. 5 Fully parameterized matrices With our fuctios for h ad we got ow the two Matrices with M l = V s e s t/2 V w sq (f+) 2f, M r = V s e s t/2 V w sq (f+) 2f. The last two rows are ot iterestig for the projectio. Sice both are ot ecessary for the projectio, oly for z-bufferig, we do t replace there. For a better overview here comes a list of all parameters i this setup: e.. eye distace (average 63 mm) V.. view distace (scree to viewer) s.. scree width w.. width of the ear pae (= w(ω, r)) q.. height of ear pae (= q(ω, r)) ω.. opeig-agle r.. shape-ratio (= w q ) f.. far pae distace (see sectio 6) t.. iteraxial camera separatio Sice w ad q are fuctios of ω ad r they are exchageable i the matrices. 10

11 cbd 2014 Patrick Oberthür Figure 4: stereo image creatio 6 importat scee restrictios ad otes There is always the questio about to apply a traslatio o the projectio matrix to simulate the movig of a camera, but this commo mistake will lead to certai issues. Sice our stereo applicatio is tued to the scee we will crash i z-buffer-fightigs easier ad these errors o two differet chaels will become eve more disturbig ad ot fie eough for more distat objects. Also we use i our scee sphere-mappig for reflectio effects. There the ormal iformatio of surfaces will be used o a cetered sphere for texturatio. But with a moved projectio the sphere will ot be cetered aymore ad especially i the stereo-case the mirrorig-effect will ot fit. So the reflectios for both eyes would ot matchig the parallax-effect. So if you wat to move through the scee with your camera, it is about to 11

12 apply the iverse trasformatio o all objects, the geometric outcome is the same, but the computed will be more precise. Refereces [Ah] Sog Ho Ah OpeGL Projectio Matrix [BvSim] Almuth Sührma, Patrick Oberthür BVSim - aget based bio-modellig framework - Project-homepage wwwpub.zih.tu-dresde.de/~s /b_v_sim/ 2011 [SmiCo] Michael D. Smith ad Bradley T. Collar Perceptio of size ad shape i stereoscopic 3D imagery Proc. SPIE 8288, Stereoscopic Displays ad Applicatios XXIII, 82881O (February 9, 2012) [Rule] Joh T. Rule The Geometry of Stereoscopic Projectio J.O.S.A. 31 (April, 1941):