Introduction to Computer Graphics. April 6, 2017 Kenshi Takayama

Transcription

1 Introduction to Computer Graphics April 6, 2017 Kenshi Takayama

2 Lecturers Kenshi Takayama (Assistant Prof., NII) Toshiya Hachisuka (Junior Associate Prof.) Ryoichi Ando (Assistant Prof., NII) Ag3RwxUAAAAJ&hl=en TA:Seung-Tak Noh (Igarashi Lab) aknoh/home 2

3 Course overview Modeling Animation Rendering Image processing 2~3 lectures per topic, 12 lectures in total Rendering part by Prof. Hachisuka Fluid animation part by Prof. Ando 3

4 Grading Programming assignments only No exam, no attendance check Two tyes of assignments: Basic & Advanced Basic 1 assignent per topic (4 in total), very easy Advanced For motivated students Deadline: The end of July Evaluation criteria 1 assignemnt submitted C (bare minimum for the degree) 4 assignments submitted B or higher Distribution of S & A will be decided based on the quality/creativity of submissions and the overall balance in the class More details explained later 4

5 References Course website Famous textbooks (not used in the class) Fundamentals of Computer Graphics ( ) Computer Graphics: Principles and Practice in C ( ) 5

6 Lecturers research topics 6

7 Coordinate transformations 7

8 Linear transformation In 2D: x y = a b c d x y In 3D: x y z = a b c d e f g h i x y z Intuition: Mapping of coordinate axes a c = a b c d 1 0 0, 1 b, d b d Origin stays put a b = c d 0 1 1, 0 a, c 8

9 Special linear transformations Rotation Scaling Shearing (X dir.) Shearing (Y dir.) cos θ sin θ sin θ cosθ s x 0 0 s y 1 k k 1 9

10 Linear transformation + translation = Affine transformation x y = a b c d x y + t x ty x y 1 = a b t x c d t y x y 1 Homogeneous coordinates: Use a 3D (4D) vector to represent a 2D (3D) point Can concisely represent linear transformation & translation as matrix multiplication Easier implementation 10

11 Combining affine transformations Just multiply matrices Careful with the ordering! R = T = cos θ sin θ 0 sin θ cos θ t x 0 1 t y x = R T x x = T R x 11

12 Another role of homogeneous coordinates: Perspective projection Objects apparent sizes on the screen are inversely proportional to the object-camera distance Orthographic projection Perspective projection 12

13 Another role of homogeneous coordinates: Perspective projection When w 0, 4D homogeneous coordinate x, y, z, w represents x a 3D position, y, z w w w Camera at the origin, screen on the plane Z=1 p x, p y, p z is projected to w x, w y = p x, p x p z p z X Z= Projection matrix p x p y p z 1 = p x p y p z + 1 p z p x /p z p y /p z 1 + 1/p z 1 w x w y w z Z w z (depth value) is used for occlusion test Z-buffering 13

14 Orthographic projection Objects apparent sizes don t depend on the camera position Simply ignore Z coordinates Orthographic Perspective Frequently used in CAD 14

15 Viewing pipeline Projection trans. (4x4 matrix) Modelview trans. (4x4 matrix) Projected coord. system Camera coord. system Viewport trans. (x, y, w, h) w h y x Window coord. system Object (world) coord. system Local coord. system 15

16 Classical OpenGL code glviewport(0, 0, 640, 480); Viewport transform glmatrixmode(gl_projection); glloadidentity(); gluperspective( Projection transform 45.0, // field of view 640 / 480, // aspect ratio 0.1, 100.0); // depth range glmatrixmode(gl_modelview); glloadidentity(); glulookat( 0.5, 0.5, 3.0, // view point Modelview transform 0.0, 0.0, 0.0, // focus point 0.0, 1.0, 0.0); // up vector glbegin(gl_lines); glcolor3d(1, 0, 0); glvertex3d(0, 0, 0); glvertex3d(1, 0, 0); glcolor3d(0, 1, 0); glvertex3d(0, 0, 0); glvertex3d(0, 1, 0); glcolor3d(0, 0, 1); glvertex3d(0, 0, 0); glvertex3d(0, 0, 1); glend(); Output Scene content 16

17 Z-buffering 17

18 Hidden surface removal Without hidden surface removal With hidden surface removal Classic problem in CG 18

19 Painter s algorithm Sort objects according to distances to camera, then draw them in the back-to-front order Fundamentally ill-suited for many cases Sorting is also not always straightforward 19

20 Z-buffering For each pixel, store distance to the camera (depth) More memory-consuming, but today s standard 20

21 Typical issues with Z-buffering: Z-fighting Multiple polygons at exact same position Impossible to determine which is front/back Strange patterns due to rounding errors 21

22 Typical issues with Z-buffering: Simultaneous drawing of faces and lines Dedicated OpenGL trick: glpolygonoffset Without polygon offset With polygon offset 22

23 Typical issues with Z-buffering: Depth range gluperspective( 45.0, // field of view 640 / 480, // aspect ratio 0.1, ); // znear, zfar znear= zfar =1000 Fixed bits for Z-buffer Typically, 16~24bits Larger depth range Larger drawing space, less accuracy Smaller depth range More accuracy, smaller drawing space (clipped) znear=50 zfar =100 23

24 Rasterization vs Ray-tracing Purpose Idea Real-time CG (games) Per-polygon processing High-quality CG (movies) Per-pixel (ray) processing One polygon updates multiple pixels One ray interacts with multiple polygons Hidden surface removal Z-buffering (OpenGL / DirectX) By nature More details by Prof. Hachisuka 24

25 Quaternions 25

26 Rotation about arbitrary axis Needed in various situations (e.g. camera manipulation) about X-axis about Y-axis about Z-axis about arbitrary axis Problems with matrix representation Overly complex! u x, u y, u z : axis vector Degree of Freedom Should be represented by 2 DoF (axis direction) + 1 DoF (angle) = 3 DoF Can t handle interpolation (blending) well 26

27 Geometry of axis-angle rotation v u: axis (unit vector) u v θ: angle v: input position u(u v) θ v v : output position u O v = v u u v cos θ + u v sin θ + u u v 27

28 Complex number & quaternion Complex number i 2 = 1 c = a, b a + b i c 1 c 2 = a 1, b 1 a 2, b 2 = a 1 a 2 b 1 b 2 + a 1 b 2 + b 1 a 2 i Quaternion i 2 = j 2 = k 2 = ijk = 1 ij = ji = k, jk = kj = i, ki = ik = j q = a, b, c, d a + b i + c j + d k Not commutative! q 1 q 2 = a 1, b 1, c 1, d 1 a 2, b 2, c 2, d 2 = a 1 a 2 b 1 b 2 c 1 c 2 d 1 d 2 + a 1 b 2 + b 1 a 2 + c 1 d 2 d 1 c 2 i + a 1 c 2 + c 1 a 2 + d 1 b 2 b 1 d 2 j + a 1 d 2 + d 1 a 2 + b 1 c 2 c 1 b 2 k 28

29 Notation by scalar + 3D vector q = a + b i + c j + d k a + b, c, d = a + v q 1 q 2 = a 1 a 2 b 1 b 2 c 1 c 2 d 1 d 2 + a 1 b 2 + b 1 a 2 + c 1 d 2 d 1 c 2 i + a 1 c 2 + c 1 a 2 + d 1 b 2 b 1 d 2 j + a 1 d 2 + d 1 a 2 + b 1 c 2 c 1 b 2 k = a 1 a 2 v 1 v 2 + a 1 v 2 + a 2 v 1 + v 1 v 2 29

30 Rotation using quaternions Note: u is a unit vector = v u u v cos α + u v sin α + u u v Interesting theory behind (cf. Wikipedia) 30

31 Rotation interpolation using quaternions Linear interp + normalization (nlerp) nlerp q 1, q 2, t normalize 1 t q 1 + t q 2 less computation, non-uniform angular speed nlerp(t ) q 2 q 1 1 t t Spherical linear interpolation (slerp) Ω = cos 1 q 1 q 2 slerp q 1, q 2, t sin 1 t Ω sin Ω q 1 + sin tω sin Ω q 2 more computation, constant angular speed slerp(t ) q 2 1 t t q 1 Animating rotation with quaternion curves. Shoemake, SIGGRAPH

32 Signs of quaternions Quaternion with angle θ: v q q v q = cos θ 2 + u sin θ 2 Quaternion with angle θ 2π: cos θ 2π 2 + u sin θ 2π 2 = q u When interpolating from q 1 to q 2, negate q 2 if q 1 q 2 is negative Otherwise, the interpolation path becomes longer 32

33 How to work on assignments 33

34 Choices for implementing real-time CG Two kinds of APIs for using GPU Different API designs (slightly?) Both supported by most popular programming languages Many choices for system- & langualge-dependent parts GUI management, handling images,... Many libraries: GUI: GLUT (C), GLFW (C), SDL (C), Qt (C++), MFC (C++), wxwidgets (C++), Swing (Java),... Images: libpng, OpenCV, ImageMagick Often quite some work to get started 34

35 = JavaScript + Runs on many (mobile) browsers HTML-based can easily handle multimedia & GUI No compiling! Quick trial & error Some performance concerns Increasingly popular today 35

36 for Embedded Systems Hurdle in WebGL development: OpenGL ES No support for legacy OpenGL API Reasons: Less efficient Burden on hardware vendors Immediate mode Polygonal primitives Light & material Transform. matrices Display list Default shaders glbegin, glvertex, glcolor, gltexcoord GL_QUADS, GL_POLYGON gllight, glmaterial GL_MODELVIEW, GL_PROJECTION glnewlist Allowed API: Prepare arrays, send them to GPU, draw them using custom shaders Shaders Shader variables Arrays Drawing glcreateshader, glshadersource, glcompileshader, glcreateprogram, glattachshader, gllinkprogram, gluseprogram glgetattriblocation, glenablevertexattribarray, glgetuniformlocation, gluniform glcreatebuffer, glbindbuffer, glbufferdata, glvertexattribpointer gldrawarrays 36

37 C / OpenGL 1.x #include <GL/glut.h> void disp( void ) { float f; glclear(gl_color_buffer_bit); glpushmatrix(); for(f = 0 ; f < 1 ; f += 0.1) { glcolor3f(f, 0, 0); glcalllist(1); } glpopmatrix(); glflush(); } void setdisplist( void ) { glnewlist(1, GL_COMPILE); glbegin(gl_polygon); glvertex2f(-1.2, -0.9); glvertex2f(0.6, -0.9); glvertex2f(-0.3, 0.9); glend(); gltranslatef(0.1, 0, 0); glendlist(); } int main(int argc, char ** argv) { glutinit(&argc, argv); glutinitwindowsize(400, 300); glutinitdisplaymode(glut_rgba); glutcreatewindow("kitty on your lap"); glutdisplayfunc(disp); setdisplist(); glutmainloop(); } WebGL <html><head> <title>learning WebGL lesson 1</title> <script type="text/javascript" src="glmatrix min.js"></script> <script id="shader-fs" type="x-shader/x-fragment"> precision mediump float; void main(void) { gl_fragcolor = vec4(1.0, 1.0, 1.0, 1.0); } </script> <script id="shader-vs" type="x-shader/x-vertex"> attribute vec3 avertexposition; uniform mat4 umvmatrix; uniform mat4 upmatrix; void main(void) { gl_position = upmatrix * umvmatrix * vec4(avertexposition, 1.0); } </script> <script type="text/javascript"> var gl; function initgl(canvas) { gl = canvas.getcontext("experimental-webgl"); gl.viewportwidth = canvas.width; gl.viewportheight = canvas.height; } function getshader(gl, id) { var shaderscript = document.getelementbyid(id); var str = ""; var k = shaderscript.firstchild; while (k) { if (k.nodetype == 3) { str += k.textcontent; } k = k.nextsibling; } var shader; if (shaderscript.type == "x-shader/x-fragment") { shader = gl.createshader(gl.fragment_shader); } else if (shaderscript.type == "x-shader/x-vertex") { shader = gl.createshader(gl.vertex_shader); } gl.shadersource(shader, str); gl.compileshader(shader); return shader; } var shaderprogram; function initshaders() { var fragmentshader = getshader(gl, "shader-fs"); var vertexshader = getshader(gl, "shader-vs"); shaderprogram = gl.createprogram(); gl.attachshader(shaderprogram, vertexshader); gl.attachshader(shaderprogram, fragmentshader); gl.linkprogram(shaderprogram); gl.useprogram(shaderprogram); shaderprogram.vertexpositionattribute = gl.getattriblocation(shaderprogram, "avertexposition"); gl.enablevertexattribarray(shaderprogram.vertexpositionattribute); shaderprogram.pmatrixuniform = gl.getuniformlocation(shaderprogram, "upmatrix"); shaderprogram.mvmatrixuniform = gl.getuniformlocation(shaderprogram, "umvmatrix"); } var mvmatrix = mat4.create(); var pmatrix = mat4.create(); var vertices = [ 0.0, 1.0, 0.0, -1.0, -1.0, 0.0, 1.0, -1.0, 0.0 ]; gl.bufferdata(gl.array_buffer, new Float32Array(vertices), gl.static_draw); trianglevertexpositionbuffer.itemsize = 3; trianglevertexpositionbuffer.numitems = 3; squarevertexpositionbuffer = gl.createbuffer(); gl.bindbuffer(gl.array_buffer, squarevertexpositionbuffer); vertices = [ 1.0, 1.0, 0.0, -1.0, 1.0, 0.0, 1.0, -1.0, 0.0, -1.0, -1.0, 0.0 ]; gl.bufferdata(gl.array_buffer, new Float32Array(vertices), gl.static_draw); squarevertexpositionbuffer.itemsize = 3; squarevertexpositionbuffer.numitems = 4; } function drawscene() { gl.viewport(0, 0, gl.viewportwidth, gl.viewportheight); gl.clear(gl.color_buffer_bit gl.depth_buffer_bit); mat4.perspective(45, 1.5, 0.1, 100.0, pmatrix); mat4.identity(mvmatrix); mat4.translate(mvmatrix, [-1.5, 0.0, -7.0]); gl.bindbuffer(gl.array_buffer, trianglevertexpositionbuffer); gl.vertexattribpointer(shaderprogram.vertexpositionattribute, trianglevertexpositionbuffer.itemsize, gl.float, false, 0, 0); setmatrixuniforms(); gl.drawarrays(gl.triangles, 0, trianglevertexpositionbuffer.numitems); mat4.translate(mvmatrix, [3.0, 0.0, 0.0]); gl.bindbuffer(gl.array_buffer, squarevertexpositionbuffer); gl.vertexattribpointer(shaderprogram.vertexpositionattribute, squarevertexpositionbuffer.itemsize, gl.float, false, 0, 0); setmatrixuniforms(); gl.drawarrays(gl.triangle_strip, 0, squarevertexpositionbuffer.numitems); } function webglstart() { var canvas = document.getelementbyid("lesson01-canvas"); initgl(canvas); initshaders(); initbuffers(); gl.clearcolor(0.0, 0.0, 0.0, 1.0); gl.enable(gl.depth_test); drawscene(); } </script></head> <body onload="webglstart();"> <canvas id="lesson01-canvas" style="border: none;" width="500" height="500"> </canvas> </body> </html> 37

38 Libraries for easing WebGL development Many popular ones: three.js, O3D, OSG.JS,... All APIs are high-level, quite different from legacy OpenGL API Good for casual users, but maybe not for CS students (?) <script src="js/three.min.js"></script> <script> var camera, scene, renderer, geometry, material, mesh; function init() { scene = new THREE.Scene(); camera = new THREE.PerspectiveCamera( 75, 640 / 480, 1, ); camera.position.z = 1000; geometry = new THREE.BoxGeometry( 200, 200, 200 ); material = new THREE.MeshBasicMaterial({color:0xff0000, wireframe:true}); mesh = new THREE.Mesh( geometry, material ); scene.add( mesh ); renderer = new THREE.WebGLRenderer(); renderer.setsize(640, 480); document.body.appendchild( renderer.domelement ); } function animate() { requestanimationframe( animate ); render(); } function render() { mesh.rotation.x += 0.01; mesh.rotation.y += 0.02; renderer.render( scene, camera ); } init(); animate(); </script> High-level API three.js 38

39 legacygl.js Developed by me for this course Demos & tutorial Assignemnts sample codes will be mostly using this Try playing with it and see how it works 39

40 WebGL development using Mozilla Thimble A free web space for putting js/html/css Online editing and quick previewing 40

41 How to work on assignemnts Implement your solution using WebGL, upload it to the web, the URL to the TA Thimble is recommended, but other means (e.g. your own server) is also OK Include some descriptions/discussions/etc in the HTML page OK to use other WebGL libraries (e.g. three.js) Other programming languages (e.g. C++) are also acceptable Should compile and run on typical computing systems Include source+binary+doc in a single ZIP file If you have any questions, don t hesitate to contact TA or me! 41

42 Shaders Vertex shader Vertex shader: per-vertex processing Per-vertex data passed by glbufferdata Vertex position, color, texture coordinate,... Mandatory operation: Specify vertex location on the screen after coordinate transformation (gl_position) Fragment shader: per-pixel processing Do something with rasterized (=linearly interpolated) data Mandatory operation: Specify pixel color to be drawn (gl_fragcolor) Fragment shader GLSL (OpenGL Shading Language) codes passed to GPU as strings compiled at runtime 42

43 Shader variables uniform variables Readable from vertex/fragment shaders Passed to GPU separately from vertex arrays (gluniform) Examples: modelview/projection matrices, flags attribute variables Readable only from vertex shaders Vertex array data passed to GPU via glbufferdata Examples: XYZ position, RGB color, UV texcoord uniform mat4 u_modelview; uniform mat4 u_projection; attribute vec3 a_vertex; attribute vec3 a_color; varying vec3 v_color; void main(void) { gl_position = u_projection * u_modelview * vec4(a_vertex, 1.0); v_color = a_color; } Vertex shader varying variables Written by vertex shader, read by fragment shader Per-vertex data linearly interpolated at this pixel (Grammar differs slightly with versions) precision mediump float; varying vec3 v_color; void main(void) { gl_fragcolor.rgb = v_color; gl_fragcolor.a = 1.0; } Fragment shader 43

44 Tips for JavaScript beginners (=me) 7 types: String / Bool / Number / Function / Object / null / undefined Unlike C++ Number: always double precision No distinction between integer & floating point Object: associative map with string keys x.abc is equivalent to x["abc"] (as if a member ) { abc : y } is equivalent to { "abc" : y } Non-string keys are implicitly converted to strings Arrays are special objects with keys being consecutive integers With additional capabilities:,,.pop(), Always pass-by-value when assigning & passing arguments No language support for deep copy When in doubt, use.length.push().foreach() console.log(x) 44

45 References OpenGL Official spec WebGL/JavaScript/HTML5 Learning WebGL Official spec Mozilla Developer Network An Introduction to JavaScript for Sophisticated Programmers Effective JavaScript 45

46 References