An Introduction to Computer Graphics Joaquim Madeira Beatriz Sousa Santos Universidade de Aveiro 1
Topics What is CG Brief history Main applications CG Main Tasks Simple Graphics system CG APIs 2D and 3D Visualization Illumination and shading
Computer Graphics The technology with which pictures, in the broadest sense of the word, are Captured or generated, and presented Manipulated and / or processed Merged with other, non-graphical application data It includes: Integration with other kinds of data Multimedia Advanced dialogue and interactive technologies 3
Computer Graphics Computer Graphics deals with all aspects of creating images with a computer Hardware Software Applications [Angel] 4
Computer Graphics: 1950 1960 Earliest days of computing Pen plotters Simple calligraphic displays Issues Cost of display refresh Slow, unreliable, expensive computers 5
Computer Graphics: 1960 1970 Wireframe graphics Draw only lines! [Angel] 6
Computer Graphics: 1960 1970 Ivan Sutherland s Sketchpad PhD thesis at MIT (1963) Man-machine interaction Processing loop Display something Wait for user input Generate new display [http://history-computer.com] 7
Sketchpad (Ivan Sutherland, 1963) http://www.youtube.com/watch?feature=endscreen&nr=1&v=usyot_ha_ba 8
Computer Graphics: 1970 1980 Raster graphics Allows drawing polygons First graphics standards Workstations and PCs 9
Raster graphics Image produced as an array (the raster) of picture elements (pixels) in the frame buffer [Angel] 10
Raster graphics Drawing polygons Illumination models Shading methods [Angel] 11
Gouraud shading 1971 [Wikipedia] 12
Gouraud shading Poor highlight Very high polygon count http://en.wikipedia.org/wiki/gouraud_shading 13
Phong shading 1973 [Wikipedia] 14
Phong reflection model 1973 [Wikipedia] 15
Can you see the differences? [Foley, Van Dam] 16
Computer Graphics: 1980 1990 The quest for realism Smooth shading Environment mapping Bump mapping [Angel] 17
Ray-Tracing example http://radsite.lbl.gov/radiance/book/img/plate10.jpg 19
https://en.wikipedia.org/wiki/ray_tracing_(graphics)
Vermeer s Studio Wallace & Cohen, 1987: Radiosity and Ray-Tracing 22
Radiosity Radiosity https://en.wikipedia.org/wiki/radiosity_(computer_graphics)
Radiosity Without radiosity With radiosity [Burdea] 24
Texture mapping Smooth shading Environment mapping Bump mapping [Angel] 25
Computer Graphics: 1980 1990 Special purpose hardware Industry-based standards PHIGS RenderMan (https://renderman.pixar.com/) Human-Computer Interaction 27
Computer Graphics: 1990 2000 OpenGL API First successful computer-generated featurelength animation film: Toy Story New hardware capabilities 28
Oscar winner 2017- Piper https://renderman.pixar.com/stories https://www.youtube.com/watch?v=lkqte0wdo2k http://oscar.go.com/news/winners/piper-is-the-2017-oscar-winner-for-short-film-animated
Computer Graphics: 2000 Photorealism Graphics cards for PCs dominate the market Nvidia ATI Game boxes / players determine the market CG is routine in the film industry 30
The first two waves of CG research Product design and manufacturing improvements drove research from the 1960s through the 1980s Arts and entertainment became the dominant driver in the mid-1980s, but has been decreasing And now? Kasik, D. J. (2011). The third wave in Computer Graphics and Interactive Techniques. IEEE Computer Graphics And Applications, 31(4), 89 93.
CG Application areas Entertainment Computer games Animation films Special effects Engineering / Architecture Computer-Aided Design (CAD) Data Visualization Medicine Visualization 32
Games Lara Croft 1996 2007 2013 [Wikipedia] 33
Animation films Pixar Toy Story 1995 Ratatouille 2007 Toy Story 2014 [www.pixar.com] 34
Special effects ILM 1991 1994 1999 [Wikipedia] 36
Special effects ILM 2005 2009 2015 [Wikipedia] 37
Computer-Aided Design CAD mockup [Wikipedia] 38
CAD Simulation [Wikipedia] 39
Virtual and Augmented Reality
Virtual Reality examples Industry Jaguar/ Land Rover http://www.youtube.com/watch?v=j3qcnvgvlnk 41
Medicine http://www.youtube.com/ watch?v=zjmrcem-uva 43
Entertainment Oculus Rift 2014; ~300 USD http://www.oculusvr.com/ 44 http://www.youtube.com/watch?v=n8uudt5ayts
CG is not alone Visualization Core areas: CG, IP, CV and HCI Satellite areas: Geometric Modeling Data and Information Visualization What is common? CV CG, IP : image file formats, color models, CG, CV : 3D model representations, IP, CV : noise removal, filters, IP CG HCI Geometric Modeling 45
Example Medical Imaging Processing pipeline Noise removal Segmentation Generating 2D / 3D models Data visualization User interaction [www.mevislab.de] RVA - 2014/2015 47
CG Main Tasks Modeling Construct individual models / objects Assemble them into a 2D or 3D scene Animation Static vs. dynamic scenes Movement and / or deformation Rendering Generate final images Where is the observer? How is he / she looking at the scene? 48
Basic Graphics System [Angel] Image formed in the frame buffer 49
Computer Graphics APIs Create 2D / 3D scenes from simple primitives OpenGL Rendering No modeling or interaction facilities Direct 3D Microsoft VTK 3D CG + Image processing + Visualization Three.js 50
API contents Functions for specifying / instantiating Geometric primitives Materials Light sources Viewer / Camera Functions for simple user interaction Input from devices: mouse, keyboard, etc. 51
Geometric Primitives Simple primitives Points Line segments Polygons Geometric primitives Parametric curves / surfaces Cubes, spheres, cylinders, etc. 52
Lights and materials Types of light sources Point vs distributed light sources Spot lights Near and far sources Color properties Material properties Absorption: color properties Scattering: diffuse and specular Transparency 53
Camera specification Position and orientation Lens image size Orientation of image plane [Angel] 54
OpenGL Multi-platform API for rendering 2D and 3D computer graphics Interaction with the GPU to achieve hardware-accelerated rendering Application areas CAD Virtual reality Scientific and Information Visualization 55
OpenGL OpenGL ES Subset for use in embedded systems and portable devices WebGL JavaScript API based on OpenGL ES 2.0 Rendering interactive 2D and 3D graphics on any compatible browser, without the use of plug-ins 56
Three.js Cross-browser JavaScript library/api used to create and display animated 3D computer graphics in a web browser. Uses WebGL https://threejs.org/
Thee.js first example 1. Defining the scene, the camera and where the scene is rendered var scene = new THREE.Scene(); var camera = new THREE.PerspectiveCamera( 75, window.innerwidth / window.innerheight, 0.1, 1000 ); var renderer = new THREE.WebGLRenderer(); renderer.setsize( window.innerwidth, window.innerheight ); document.body.appendchild( renderer.domelement ); 58
2.Creating an object and camera position var geometry = new THREE.BoxGeometry(1,1,1); var material = new THREE.MeshBasicMaterial( { color: 0x00ff00 } ); var cube = new THREE.Mesh( geometry, material ); scene.add( cube ); camera.position.z = 5; 59
3. Scene rendering function render() { requestanimationframe(render); renderer.render(scene, camera); } render(); 4. Scene animation cube.rotation.x += 0.1; cube.rotation.y += 0.1; 60
Adding lights and shading var material = new THREE.MeshPhongMaterial({ ambient: '#006063', color: '#00abb1', specular: '#a9fcff', shininess: 100 }); 61
2D Visualization Define a 2D scene in the world coordinate system Select a clipping window in the XOY plane The window contents will be displayed Select a viewport in the display The viewport displays the contents of the clipping window 62
World -> display y y Display Coordinates x World Coordinates x
Coordinate mapping World Coordinates Screen coordinates 64
Coordinate mapping If the aspect ration is not the same in both situations the result is distortion 65
World -> screen y y Screen Coordinates x The aspect ration is not the same in both situations: distortion! World Coordinates x
3D Viewing 68
3D Viewing Where is the observer / the camera? Position? Close to the 3D scene? Far away? How is the observer looking at the scene? Orientation? How to represent as a 2D image? Projection? 69
Obtaining an image of the scene using perspective View frustum image [Interactive 3D Graphics, Udacity]
Light and Rendering In the real world the light emits rays that are reflected by objects and seen by the eye Computing this is too time consuming {Interactive 3D Graphics, Udacity]
Reversing the process in CG In CG simplifying assumptions may be made Start from the camera No shadows Only the rays that matter are processed
3D visualization pipeline Instantiate models of the scene Position, orientation, size Establish viewing parameters Camera position and orientation Compute illumination and shade polygons Perform clipping Project into 2D Rasterize 73
3D visualization pipeline Each object is processed separately Typically 3D triangles (e.g. a cube or a sphere are made of triangles) Triangles are modified by the camera view of the world Is the object inside the view frustum? (No -> next object!) Yes -> compute the location on the screen of each triangle (rasterization) Compute the color of each pixel 74
3D scene Geometry Material Light (animation) + Camera {Interactive 3D Graphics, Udacity]
Projection Parallel Projection (allows measures) Perspective Projection (more realistic images) 76
Projections Parallel Projection Perspective Projection 77
Parallel Projections Ortographic / Axonometric projection Oblique projection Ortographic/ Multiview projection 78
Perspective Projections 79
Perspective Projections 80
How to limit what is observed? Clipping window on the projection plane View volume (frustum) in 3D Clipping plane Clipping plane Clipping window 82
Lighting Compute surface color based on Type and number of light sources Illumination model Phong: ambient + diffuse + specular components Reflective surface properties Atmospheric effects Fog, smoke Polygons making up a model surface are shaded Realistic representation 85
Phong reflection model Empirical model of the local illumination of points on a surface It describes the way a surface reflects light as a combination of the diffuse reflection of rough surfaces with the specular reflection of shiny surfaces and a component of ambient light [Wikipedia] 86
Phong Model Ambient illumination Constant illumination component for each model Independent from viewer position or object orientation! Take only material properties into account! 87
Phong Model Ambient illumination 88
Phong Model Diffuse reflection Model surface is an ideal diffuse reflector What does that mean? Independence from viewer position! Unit vectors!! 89
Phong Model Specular reflection To light Normal Reflection To viewer Important for shiny model surfaces How to model shininess? Take into account viewer position! Unit vectors! 91
Phong Model Specular reflection 92
Phong Model Specular reflection and or 93
More than one light source 94
Illumination and shading How to optimize? Fewer light sources Simple shading method BUT, less computations mean less realism Wireframe representation Flat-shading Gouraud shading Phong shading 96
Flat shading For each polygon: Applies the illumination model just once All pixels have the same color smooth objects seem blocky It is fast
Gouraud shading Apply the illumination model at vertices For each triangle: Applies the illumination model at each vertex Interpolates color to shade each pixel It provides better results than flat shading But highlights are not rendered correctly highlight
Gouraud shading [Wikipedia] Same object with a much higher n. of faces 99
Phong shading Interpolates normals across rasterized polygons computes pixel colors based on the interpolated normals It provides better results than Gouraud shading But is more time consuming highlight
Flat shading vs Phong shading [Wikipedia] 101
Test these examples in three.js: Material and light properties https://threejs.org/examples/#webgl_geometry_teapot
Wire frame Flat shading Gouraud shading Phong shading https://threejs.org/examples/#webgl_geometry_teapot
https://threejs.org/examples/?q=phong #webgl_materials_variations_phong
Physically accurate lighting https://threejs.org/examples/#webgl_lights_physical
Examples in three.js (with code) Orthogonal camera; Ambient light + direction light; Lambertian objects https://threejs.org/examples/#canvas_camera_orthographic
(uses Gouraud shading)
Clipping https://threejs.org/examples/#webgl_clipping
Basic 2D Transformations p = (x, y) original point p = (x, y ) transformed point Basic transformations: P = P = x y x y - Translation Vector notation - Scaling - Rotation 114
Translation It is a rigid body transformation (it does not deform the object) To apply a translation to a line segment we need only to transform the end points To apply a translation to a polygon we need only to transform the vertices 115
Translation It is necessary to specify translations in x and y x = x + tx y = y + ty P = x y P = x y T = tx ty P = P + T transformation matrix
Rotation To apply a rotation we need to specify: - a point (center of rotation) (x r,y r ) - A rotation angle θ (positive - counter-clockwise) Positive rotation 117
Rotation around the origin The simplest case: x = r cos (Φ + θ) = r cos Φ cos θ r sin Φ sin θ y = r sin (Φ + θ) = r cos Φ sin θ + r sin Φ cos θ r sin (Φ + θ) Polar coordenates of the original point: x = r cos Φ y = r sin Φ Replacing: r cos (Φ + θ) x = x cos θ y sin θ y = x sin θ + y cos θ 118
2D Rotation r sin (Φ + θ) r cos (Φ + θ) 119
Scaling Modifies the size of an object; we need to specify scaling factors: s x and s y x = x. s x y = y. s y rasformation matrix P = S. P Transforming a square into a larger square applying a scaling s x =2, s y =2
2D Transformations Matrix representation Homogeneous coordinates!! Concatenation = Matrix products Complex transformations? Decompose into a sequence of basic transformations 121
Homogeneous coordinates Most applications involve sequences of transformations For instance: - visualization transformations involve a sequence of translations and rotations to render an image of a scene - animations may imply that an object is rotated and translated between two consecutive frames Homogeneous coordinates provide an efficient way to represent and apply sequences of transformations 122
It is possible to combine in a matrix the multiplying and additive terms if we use 3x3 matrices All transformations may be represented by multiplying matrices Each point is now represented by 3 coordinates ( x, y ) (x h, y h, h), h = 0 x = x h / h y = y h / h ( x.h, y.h, h) 123
2D Translation 124
2D Rotation r sin (Φ + θ) r cos (Φ + θ) 125
2D Scaling 126
Concatenation 127
Concatenation 128
Arbitrary Rotation Translation + Rotation + Inverse Translation 129
Arbitrary Scaling Translation + Scaling + Inverse Translation 130
Order is important! 131
3D Transformations Translation Scaling 132
3D Rotation Rotation around each one of the coordinate axis Positive rotations are CCW (counter clock wise)!! 133
Rotation around ZZ 134
Rotation around XX 135
Rotation around YY 136
Other useful 3D Transformations Shears Reflections Shear in ZZ 137
How to apply Projections? Also by matrix multiplication Example: Matrix of the orthographic projection on the xy plane in homogeneous coordinates: zz coordinates are discarded y z 138 x
Transformations in three.js https://threejs.org/docs/#api/math/matrix4
Some reference books D. Hearn and M. P. Baker, Computer Graphics with OpenGL, 3 rd Ed., Addison-Wesley, 2004 E. Angel and D. Shreiner, Introduction to Computer Graphics, 6 th Ed., Pearson Education, 2012 J. Foley et al., Introduction to Computer Graphics, Addison-Wesley, 1993 Hughes, J., A. Van Dam, et al., Computer Graphics, Principles and Practice, 3rd Ed., Addison Wesley, 2013 149
Acknowledgments Thanks are due to Paulo Dias Samuel Silva 150