Coordinate Transformations & Homogeneous Coordinates

CS-C31 (Introduction to) Computer Graphics Jaakko Lehtinen Coordinate Transformations & Homogeneous Coordinates Lots of slides from Frédo Durand CS-C31 Fall 216 Lehtinen

Outline Intro to Transformations Useful Classes of Transformations Representing Transformations, Homogeneous Coordinates Combining Transformations Hardcore: Transforming normals CS-C31 Fall 216 Lehtinen 2

Modeling/Viewing Pipeline Recap 1.Model the geometry Here a triangle mesh. Also, specify materials. CS-C31 Fall 216 Lehtinen 3

Modeling/Viewing Pipeline Recap 1.Model the geometry Here a triangle mesh. Also, specify materials. Object coordinates CS-C31 Fall 216 Lehtinen 4

Modeling/Viewing Pipeline Recap 1.Model the geometry 2.Place the objects in world space Each object has its own object space Only one world space Object Object coordinates World coordinates Another Object CS-C31 Fall 216 Lehtinen World Origin 5

Modeling/Viewing Pipeline Recap 1.Model the geometry 2.Place the objects in world space 3.Pick viewing position and direction Camera position and orientation Object coordinates World coordinates View coordinates CS-C31 Fall 216 Lehtinen 6

Modeling/Viewing Pipeline Recap 1.Model the geometry 2.Place the objects in world space 3.Pick viewing position and direction 4.Transform objects to view space and project to image plane Compute shading CS-C31 Fall 216 Lehtinen Object coordinates World coordinates View coordinates Image coordinates 7

Modeling/Viewing Summary Today we ll look at the transformations that take us between these coordinates. Object coordinates World coordinates View coordinates Image coordinates CS-C31 Fall 216 Lehtinen

Questions? CS-C31 Fall 216 Lehtinen 9

What is a Transformation? A function that maps point x to point x': Applications: animation, deformation, viewing, projection, real-time shadows, From Sederberg and Parry, Siggraph 1986 CS-C31 Fall 216 Lehtinen 1

Simple Transformations? CS-C31 Fall 216 Lehtinen 11

Simple Transformations Can be combined Are these operations invertible? Yes, except scale = CS-C31 Fall 216 Lehtinen 12

Questions? CS-C31 Fall 216 Lehtinen 13

Outline Intro to Transformations Useful Classes of Transformations This is the stuff you use for most modeling/viewing Representing Transformations, Homogeneous Coordinates Combining Transformations Transforming Normals CS-C31 Fall 216 Lehtinen 14

Rigid-Body / Euclidean Transforms What properties are preserved? Rigid / Euclidean Translation Identity Rotation CS-C31 Fall 216 Lehtinen 15

Rigid-Body / Euclidean Transforms Preserves distances Preserves angles Rigid / Euclidean Translation Identity Rotation CS-C31 Fall 216 Lehtinen 16

Similitudes / Similarity Transforms Preserves angles The shapes are the same, just at different scale, orientation and location Similitudes Rigid / Euclidean Translation Identity Rotation Isotropic Scaling CS-C31 Fall 216 Lehtinen 17

Linear Transformations Similitudes Rigid / Euclidean Linear Translation Identity Rotation Isotropic Scaling Scaling Reflection Shear CS-C31 Fall 216 Lehtinen 18

Linear Transformations L(p + q) = L(p) + L(q) L(ap) = a L(p) Similitudes Rigid / Euclidean Linear Translation Identity Rotation Isotropic Scaling Scaling Reflection Shear CS-C31 Fall 216 Lehtinen 19

Linear Transformations L(p + q) = L(p) + L(q) L(ap) = a L(p) Similitudes Rigid / Euclidean Linear Translation Identity Rotation Isotropic Scaling Scaling Reflection Shear??? CS-C31 Fall 216 Lehtinen 2

Linear Transformations L(p + q) = L(p) + L(q) L(ap) = a L(p)??? Translation Similitudes is not linear: Linear Rigid / Euclidean Scaling Identity f(ap) = ap+t a(p+t) Isotropic Scaling = a f(p) Reflection Rotation Shear Translation f(p) = p+t f(p+q) = p+q+t (p+t)+(q+t) = f(p) + f(q) CS-C31 Fall 216 Lehtinen 21

Affine Transformations What is preserved..? Affine Similitudes Rigid / Euclidean Linear Translation Identity Rotation Isotropic Scaling Scaling Reflection Shear CS-C31 Fall 216 Lehtinen 22

Affine Transformations Preserves parallel lines Affine Similitudes Rigid / Euclidean Linear Translation Identity Rotation Isotropic Scaling Scaling Reflection Shear CS-C31 Fall 216 Lehtinen 23

Projective Transformations Preserves lines: lines remain lines (planes remain planes in 3D) (Planar) Projective Affine Similitudes Rigid / Euclidean Linear Translation Identity Rotation Isotropic Scaling Scaling Reflection Shear Perspective CS-C31 Fall 216 Lehtinen 24

What s so nice about these? What s with the hierarchy? Why have we grouped types of transformations with and within each other? (Planar) Projective Affine Similitudes Rigid / Euclidean Identity Translation Rotation Isotropic Scaling Linear Scaling Reflection Shear Perspective CS-C31 Fall 216 Lehtinen 25

What s so nice about these? They are closed under concatenation Means e.g. that an affine transformation followed by another affine transformation is still an affine transformation Same for every subgroup, e.g. rotations, translations.. Very convenient! Projections are the most general Others are its special cases Translation (Planar) Projective Rigid / Euclidean Affine Similitudes Identity Rotation Linear Isotropic Scaling Perspective Scaling Reflection CS-C31 Fall 216 Lehtinen 26 Shear

Name-dropping Fancy name: Group Theory Remember algebra? A group is a set S with an operation f that takes two elements of S and produces a third: (and some other axioms) s, t S, f(s, t) =u u S These transformations are group(s) and subgroups The transformations are the set S, concatenation of transformations is f CS-C31 Fall 216 Lehtinen 27

Transforms are Groups Why is this useful? You can represent any number of successive transformations by a single compound transformation Example The object-to-world transformation, the world-to-view transformation, and the perspective projection (view-to-image) can all be folded into a single projective object-toimage transformation (OpenGL: Modelview, projection) CS-C31 Fall 216 Lehtinen Disclaimer: Not ANY transformation, but the types just introduced Object coordinates World coordinates View coordinates Image coordinates 28

Questions? CS-C31 Fall 216 Lehtinen 29

More Complex Transformations.....can be built out of these, e.g. Skinning Blending of affine transformations We ll do this later! And you will code it up! :) Ilya Baran CS-C31 Fall 216 Lehtinen 3

More Complex Transformations Harmonic coordinates (link to paper) Pixar Object enclosed in simple cage, each object point knows the influence each cage vertex has on it Deform the cage, and the object moves! CS-C31 Fall 216 Lehtinen 31

Questions? CS-C31 Fall 216 Lehtinen 32

Outline Intro to Transformations Useful Classes of Transformations Representing Transformations, Homogeneous Coordinates Combining Transformations Transforming Normals CS-C31 Fall 216 Lehtinen 33

Interlude: ImmersiveMath.com If your matrices and vectors are a little rusty, and even if they aren t, read the first chapters from this really neat interactive linear algebra book! CS-C31 Fall 216 Lehtinen 34

How are Linear Transforms Represented? Linear transformations include rotation, scaling, reflection, and shear CS-C31 Fall 216 Lehtinen 35

How are Linear Transforms Represented? x' = ax + by y' = dx + ey Linear transformations include rotation, scaling, reflection, and shear CS-C31 Fall 216 Lehtinen 36

How are Linear Transforms Represented? x' = ax + by y' = dx + ey Linear transformations include rotation, scaling, reflection, and shear x' y' = a b d e x y p' = M p CS-C31 Fall 216 Lehtinen 37

How are Linear Transforms Represented? x' = ax + by y' = dx + ey Linear transformations include rotation, scaling, reflection, and shear Note that the origin always stays fixed. x' y' = a b d e x y p' = M p CS-C31 Fall 216 Lehtinen 38

How are Affine Transforms Represented? x' = ax + by + c y' = dx + ey + f Affine transformations include all linear transformations, plus translation x' y' = a b d e x y + c f p' = M p + t CS-C31 Fall 216 Lehtinen 39

Can we use matrices for affine transforms? x' = ax + by + c y' = dx + ey + f x' y' = a b d e x y p' = M p? CS-C31 Fall 216 Lehtinen 4

The Homogeneous Coordinate Trick This is what we want: x' = ax + by + c y' = dx + ey + f Affine formulation Homogeneous formulation x' y' = a b d e x y + c f x y 1 p' = M p + t CS-C31 Fall 216 Lehtinen 41

The Homogeneous Coordinate Trick This is what we want: x' = ax + by + c y' = dx + ey + f Affine formulation Homogeneous formulation x' y' = a b d e x y + c f x' y 1 = a b d e c f 1 x y 1 p' = M p + t p' = M p CS-C31 Fall 216 Lehtinen 42

Homogeneous Coordinates Add an extra dimension in 2D, we use 3-vectors and 3 x 3 matrices In 3D, we use 4-vectors and 4 x 4 matrices Makes affine transformations linear in one higher dimension (can be represented by a matrix) Each point has an extra value, w You can think of it as scale For all transformations except perspective, you can just set w=1 and not worry about it x' y 1 = a b d e CS-C31 Fall 216 Lehtinen 43 c f 1 x y 1

Homogeneous Coordinates x' y' z' 1 = a e i b f j c g k d h l 1 x y z 1 If we multiply a homogeneous point by an affine matrix, w is unchanged Affine matrix means the last row is ( 1) This form encodes all possible affine transformations! What happens when you multiply an affine matrix with another affine matrix? CS-C31 Fall 216 Lehtinen 44

More on w As mentioned, you can think of it as scale. Let s see: x' a b c x = y' d e f y 1 1 1? = a b d e c f 1 2x 2y 2 CS-C31 Fall 216 Lehtinen 45

More on w As mentioned, you can think of it as scale. Let s see: x' a b c x = y' d e f y 1 1 1 (DUH!) 2x' 2y' 2 = a b d e c f 1 2x 2y 2 CS-C31 Fall 216 Lehtinen 46

More on w As mentioned, you can think of it as scale. Let s see: x' a b c x = y' d e f y 1 1 1 (DUH!) 2x' 2y' 2 = a b d e c f 1 2x 2y 2 It s tempting to divide by w, because this will get us back to the result above. CS-C31 Fall 216 Lehtinen 47

Projective Equivalence in 1D (x,w) w=3 w=2 w w=1 x (,) w= CS-C31 Fall 216 Lehtinen 48

Projective Equivalence in 1D We can get into a canonical form by dividing by w. This projects (x, w) onto the line w=1 yielding (x/w, 1). w (x,w) (x/w,1) w=3 w=2 w=1 x (,) w= CS-C31 Fall 216 Lehtinen 49

Projective Equivalence in 1D We can get into a canonical form by dividing by w. This projects (x, w) onto the line w=1 yielding (x/w, 1). We say that all points on dashed line are identical because they project to the same point on the line w=1. w (x,w) (x/w,1) w=3 w=2 w=1 x (,) w= CS-C31 Fall 216 Lehtinen 5

Projective Equivalence More mathematically, all non-zero scalar multiples of a point are considered identical So, in fact, we are saying that points in ND are identified with lines through the origin in (N+1)D x y z w = ax ay az aw w!= = x/w y/w z/w 1 a!= CS-C31 Fall 216 Lehtinen 51

Homogeneous Visualization in 2D Divide by w to normalize (project) (,,) (,, 1) = (,, 2) = (7, 1, 1) = (14, 2, 2) = (4, 5, 1) = (8, 1, 2) = w = 1 w = 2 CS-C31 Fall 216 Lehtinen 52

Homogeneous Visualization in 2D Divide by w to normalize (project) w =? Points at infinity (directions) (,,) (,, 1) = (,, 2) = (7, 1, 1) = (14, 2, 2) = (4, 5, 1) = (8, 1, 2) = w = 1 w = 2 CS-C31 Fall 216 Lehtinen 53

Projective Equivalence Why? For affine transformations, adding w=1 in the end proved to be convenient. The real showpiece is perspective. CS-C31 Fall 216 Lehtinen 54

Projective Equivalence Why? For affine transformations, adding w=1 in the end proved to be convenient. The real showpiece is perspective. From the camera s point of view, all 3D points on the line fall on the same 2D coordinate in the image. In that sense, all those 3D points are identical. CS-C31 Fall 216 Lehtinen 55

Questions? CS-C31 Fall 216 Lehtinen 56

Translate (tx, ty, tz) y Translate(c,,) Why bother with the extra dimension? Because now translations can be encoded in the matrix! p c p' x x' y' z' 1 = 1 1 1 tx ty tz 1 x y z 1 CS-C31 Fall 216 Lehtinen 57

What s Actually Happening? x' = x + c x x c CS-C31 Fall 216 Lehtinen 58

What s Actually Happening? x' = x + c Let s first add the 2nd dimension w and set coordinates to 1. What does this mean..? (x,1) (x,1) c CS-C31 Fall 216 Lehtinen 59

What s Actually Happening? x x x' = x + c w=2 w=1 (x,1) (x,1) w w= x (,) CS-C31 Fall 216 Lehtinen 6

What s Actually Happening? x x x' = x + c w=2 w=1 w w= x (,) CS-C31 Fall 216 Lehtinen 61

What s Actually Happening? x x x' 1 = 1 c 1 x 1 w=2 w=1 w w= x (,) CS-C31 Fall 216 Lehtinen 62

What s Actually Happening? w x x' 1 = 1 c 1 x 1 It s a 2D shear! w=2 w=1 w w= x (,) CS-C31 Fall 216 Lehtinen This line stays put, and origin stays still 63

Questions? CS-C31 Fall 216 Lehtinen 64

Scale (sx, sy, sz) y Scale(s,s,s) p' Isotropic (uniform) scaling: sx = sy = sz p q q' x x' y' z' 1 = sx sy sz 1 x y z 1 CS-C31 Fall 216 Lehtinen 65

Rotation About z axis y ZRotate(θ) θ p' p z x x' y' z' 1 = cos θ sin θ -sin θ cos θ 1 1 x y z 1 CS-C31 Fall 216 Lehtinen 66

Rotation y Rotate(k, θ) About (kx, ky, kz), a unit vector on an arbitrary axis (Rodrigues Formula) z θ k x x' y' z' 1 = kxkx(1-c)+c kykx(1-c)+kzs kzkx(1-c)-kys kxky(1-c)-kzs kyky(1-c)+c kzky(1-c)+kxs kxkz(1-c)+kys kykz(1-c)-kxs kzkz(1-c)+c 1 x y z 1 where c = cos θ & s = sin θ CS-C31 Fall 216 Lehtinen 67

Rotations and Matrices Rotations are represented by orthonormal matrices, i.e., matrices such that M T M = I This implies det(m) = ±1 (why?) Furthermore, to rule out reflections, require det(m)=1 Rotations are a group of their own Can you prove this based on the above properties? CS-C31 Fall 216 Lehtinen 68

Questions? CS-C31 Fall 216 Lehtinen 69

A Word of Warning In regular 3D, adding a displacement d to a point p is simple p x + d x p = p + d = p y + d y p z + d z What about homogeneous coordinates? p = p x p y p z 1 + d x d y d z? CS-C31 Fall 216 Lehtinen 7

A Word of Warning In regular 3D, adding a displacement d to a point p is simple p x + d x p = p + d = p y + d y p z + d z What about homogeneous coordinates? p = p x p y p z 1 + d x d y d z = p x + d x p y + d y p z + d z 1 CS-C31 Fall 216 Lehtinen You can t add homogeneous points to each other like in 3D. 71

A Word of Warning In 3D, px' adding a 1 displacement dx d to px a point p is simple py' 1 p dy x + d py = x pz' p = p + d = 1 p dz y + d pz y 1 1 1 p z + d z What about homogeneous coordinates? p = p x p y p z 1 + d x d y d z = p x + d x p y + d y p z + d z 1 CS-C31 Fall 216 Lehtinen w stays the same! You can t add homogeneous points to each other like in 3D. 72

The Same for Scaling ap x ap= ap y ap z a Remember projective equivalence CS-C31 Fall 216 Lehtinen 73

The Same for Scaling ap x ap= ap y ap z 1 You also leave w fixed. CS-C31 Fall 216 Lehtinen 74

Summary Important! Affine n-d transformations are encoded by linear transformations in (n+1)d that move corresponding homogeneous points in the w=1 (hyper)plane Technically, w=any constant works too, just have to scale translation distances as well. CS-C31 Fall 216 Lehtinen 75

Questions? CS-C31 Fall 216 Lehtinen 76

Why Bother II: Perspective This picture gives away almost the whole story. w = 1 w = 2 CS-C31 Fall 216 Lehtinen 77

Perspective in 2D Camera at origin, looking along z, 9 degree f.o.v., image plane at z=1 x=-z p=(x,z) x=z z x p =(x/z,1) (,) z=1 z= CS-C31 Fall 216 Lehtinen 78

Perspective in 2D x=-z The projected point in homogeneous coordinates (we just added w=1): x/z p = 1 1 x=z p=(x,z) z x p =(x/z,1) (,) z=1 z= CS-C31 Fall 216 Lehtinen 79

Perspective in 2D x=-z x/z p = 1 1 x=z x z z p=(x,z) Projectively equivalent z x p =(x/z,1) (,) z=1 z= CS-C31 Fall 216 Lehtinen 8

How do you do this with a matrix? 1 x @ za = @? 1 A 1 x @ za z 1 CS-C31 Fall 216 Lehtinen 81

Perspective in 2D We ll just copy z to w, and get the projected point after homogenization! x=-z p x 1 x z = 1 z z 1 1 x=z p=(x,z) z x p =(x/z,1) (,) z=1 z= CS-C31 Fall 216 Lehtinen 82

Extension to 3D Trivial: Just add another dimension y and treat it like x z is the special one, it turns into w Different fields of view and non-square image aspect ratios can be accomplished by simple scaling of the x and y axes. x y z w = 1 1 1 1 x y z 1 CS-C31 Fall 216 Lehtinen 83

Caveat These projections matrices work perfectly in the sense that you get the proper 2D projections of 3D points. However, since we are flattening the scene onto the z=1 plane, we ve lost all information about the distance to camera. Not a big deal for ray tracers, but GPUs need distances for Z buffering, i.e., figuring out what is in front of what. CS-C31 Fall 216 Lehtinen 84

The View Frustum in 2D (In 3D this would be a truncated pyramid.) z=far p q z x (,) z=1 z=near z= CS-C31 Fall 216 Lehtinen 85

The View Frustum in 2D We can transform the frustum by a modified projection in a way that makes it a square after projection and homogenization (division by w). z x x z w 1 = f+n f n 1 2 f n f n CS-C31 Fall 216 Lehtinen z /w x /w x z, 1 These z /w values go into the Z buffer. 86

Details in Handout in MyCourses Understanding Projections and Homogenous Coordinates How you get the matrix from previous slide Fun demonstration: Print the following page out. Holding the paper as flat as possible, try to see if you can hold it at such an angle in front of you so that the grid looks like a square with a regular grid in it. CS-C31 Fall 216 Lehtinen 87

CS-C31 Fall 216 Lehtinen 88

Cool Application Video Structure from Motion -algorithms SfM if a branch of computer vision that tries to understand the 3D structure of the scene from pictures taken from different viewpoints. It s all based on projective geometry and homogeneous coordinates. Examples: http://phototour.cs.washington.edu/ http://phototour.cs.washington.edu/findingpaths/ CS-C31 Fall 216 Lehtinen 89

Want to Know More? Take Ville Kyrki s Machine Perception (AS-84.3126) to know more! CS-C31 Fall 216 Lehtinen 9

CS-C31 Fall 216 Lehtinen 91

Phase 3: Profit All these are linear in homogeneous coordinates! Projective Affine Similitudes Rigid / Euclidean Linear Translation Identity Rotation Isotropic Scaling Scaling Reflection Shear Perspective CS-C31 Fall 216 Lehtinen 92

Outline Intro to Transformations Useful Classes of Transformations Representing Transformations, Homogeneous Coordinates Combining Transformations Transforming Normals CS-C31 Fall 216 Lehtinen 93

How are transforms combined? Scale then Translate (,) (1,1) Scale(2,2) (,) (2,2) Translate(3,1) (3,1) (5,3) Use matrix multiplication: p' = T ( S p ) = TS p TS = 1 3 2 2 3 1 1 2 = 2 1 1 1 1 Caution: matrix multiplication is NOT commutative! CS-C31 Fall 216 Lehtinen 94

Non-commutative Composition Scale then Translate: p' = T ( S p ) = TS p (,) (1,1) Scale(2,2) (,) (2,2) Translate(3,1) (3,1) (5,3) Translate then Scale: p' = S ( T p ) = ST p (8,4) (,) (1,1) Translate(3,1) (3,1) (4,2) Scale(2,2) (6,2) CS-C31 Fall 216 Lehtinen 95

CS-C31 Fall 216 Lehtinen 96 TS = 2 2 1 1 1 3 1 1 ST = 2 2 1 1 3 1 Non-commutative Composition Scale then Translate: p' = T ( S p ) = TS p 2 2 3 1 1 2 2 6 2 = = Translate then Scale: p' = S ( T p ) = ST p 1 1 1

Questions? CS-C31 Fall 216 Lehtinen 97

Outline Intro to Transformations Useful Classes of Transformations Representing Transformations, Homogeneous Coordinates Combining Transformations Transforming Normals CS-C31 Fall 216 Lehtinen 98

Normal Surface Normal: unit vector that is locally perpendicular to the surface CS-C31 Fall 216 Lehtinen 99

Why is the Normal important? It's used for shading makes things look 3D! object color only Diffuse Shading CS-C31 Fall 216 Lehtinen 1

Visualization of Surface Normal ± x = Red ± y = Green ± z = Blue CS-C31 Fall 216 Lehtinen 11

How do we transform normals? n WS n OS Object Space World Space CS-C31 Fall 216 Lehtinen 12

Transform Normal like Object? translation? rotation? isotropic scale? scale? reflection? shear? perspective? CS-C31 Fall 216 Lehtinen 13

Transform Normal like Object? translation? rotation? isotropic scale? scale? reflection? shear? perspective? CS-C31 Fall 216 Lehtinen 14

What class of transforms? Projective Affine Similitudes Rigid / Euclidean Linear Translation Identity Rotation Isotropic Scaling Scaling Reflection Shear Perspective a.k.a. Orthogonal Transforms CS-C31 Fall 216 Lehtinen 15

Transformation for shear and scale Incorrect Normal Transformation Correct Normal Transformation CS-C31 Fall 216 Lehtinen 16

More Normal Visualizations Incorrect Normal Transformation Correct Normal Transformation CS-C31 Fall 216 Lehtinen 17

So how do we do it right? Think about transforming the tangent plane to the normal, not the normal vector n OS n WS v OS Original Incorrect Correct v WS Pick any vector v OS in the tangent plane, how is it transformed by matrix M? v WS = M v OS CS-C31 Fall 216 Lehtinen 18

Transform tangent vector v v is perpendicular to normal n: Dot product n OS T v OS = n T OS (M -1 M) v OS = (n T OS M -1 ) (M v OS ) = (n T OS M -1 ) v WS = v WS is perpendicular to normal n WS : n WS n WS T = n OS T (M -1 ) n OS v OS v WS n WS = (M -1 ) T n OS n T WS v WS = CS-C31 Fall 216 Lehtinen 19

Digression n WS = (M -1 ) T n OS The previous proof is not quite rigorous; first you d need to prove that tangents indeed transform with M. Turns out they do, but we ll take it on faith here. If you believe that, then the above formula follows. CS-C31 Fall 216 Lehtinen 11

Comment So the correct way to transform normals is: n WS = (M -1 ) T n OS Sometimes denoted M -T But why did n WS = M n OS work for similitudes? Because for similitude / similarity transforms, (M -1 ) T =λ M e.g. for orthonormal basis: M -1 = M T i.e. (M -1 ) T = M CS-C31 Fall 216 Lehtinen 111

Connections Not part of class, but cool Covariant : transformed by the matrix e.g., tangent Contravariant : transformed by the inverse transpose e.g., the normal a normal is a co-vector Google differential geometry to find out more CS-C31 Fall 216 Lehtinen 112

Questions? CS-C31 Fall 216 Lehtinen 113

That s All for Today Further Reading Projection handout (MyCourses) Other Cool Stuff Algebraic Groups http://phototour.cs.washington.edu/ http://phototour.cs.washington.edu/findingpaths/ Free-form deformation of solid objects Harmonic coordinates for character articulation CS-C31 Fall 216 Lehtinen 114