OpenGL & Visualization

Transcription

1 OpenGL & Visualization Martin Ilčík Institute of Computer Graphics and Algorithms Vienna University of Technology

2 Motivation What is OpenGL How to use OpenGL Slices with OpenGL GPU raycasting Martin Ilčík 2

3 What is OpenGL? Low-level API for 3D graphics Cross-platform High extensible Open source OS independent Headers for all common prog. languages Lots of tutorials and resources on the web Very simple to use Martin Ilčík 3

4 What can it do? HW acceleration for Rendering geometry Texture lookups Drawing buffers GPU programming Programmable pipeline What can t it do? 3D engine stuff GUI Math Window managing Input processing Martin Ilčík 4

5 What can we use it for? (1) Slices Stored as texture(s) Quick switching and manipulation Arbitrary sliceplane Transfer function Again a texture Comfortable and easy changes Fast application to data Martin Ilčík 5

6 What can we use it for? (2) Raycasting with 2.0 shaders partially on GPU with 3.0 shaders completely on GPU for others at least pleasant drawing Other HW accelerated volume rendering techniques discussed at lectures also in EG 06 tutorial not required for the LU Martin Ilčík 6

7 Let s start! Now I really want to use all the great benefits of OpenGL! How? Martin Ilčík 7

8 OpenGL resources (1) News, Links, Forums (great response time) OpenGL 1.x/2.x specs, GLSL specs best help & reference The red book old but good Biggest and best tutorial series all over the net NeHe auf deutsch Featured articles on general programming Martin Ilčík 8

9 OpenGL resources (2) GLinfo (for win) find out your graphics HW capabilities Many resources, also for 3D graphics Very good forums Many papers and presentations Documentation for company specific features Martin Ilčík 9

10 Principles of OpenGL Structured, not object oriented State machine with a state-vector 1.x core is old, but regulary extended ARB, EXT extensions NV, ATI extensions 2.x common standard supported by HW Functionality of 2.0 = Functionality of 1.5 Function names start with gl Type of main parameter as suffix: i, f, us, v Martin Ilčík 10

11 OpenGL setup (1) Application Initialization Create the window Attach the rendering surface to the OpenGL Set it s pixel format and buffers OpenGL initialization Setup basic parameters and states Extensions availability check Window resizing handling Load resources Martin Ilčík 11

12 OpenGL setup (2) Rendering loop Clear the buffer (Reset transformations) Draw Setup can be done Self-made SDL GLUT All samples and frameworks at NeHe Basecode Martin Ilčík 12

13 Hello Triangle! Type this into the rendering loop glbegin(gl_triangles); glvertex3f(-1.0f, 0.0f, 0.0f); glvertex3f(1.0f, 0.0f, 0.0f); glvertex3f(0.0f, 1.0f, 0.0f); glend(); More in NeHe Lesson 2 Martin Ilčík 13

14 Hello colorful triangle! glbegin(gl_triangles); glcolor3f(1.0f, 0.0f, 0.0f); glvertex3f(-1.0f, 0.0f, 0.0f); glcolor3f(0.0f, 1.0f, 0.0f); glvertex3f(1.0f, 0.0f, 0.0f); glcolor3f(0.0f, 0.0f, 1.0f); glvertex3f(0.0f, 1.0f, 0.0f); glend(); More in NeHe Lesson 3 Martin Ilčík 14

15 Hello texture! (1) Enable the texturing (only 2 K textures) glenable(gl_texture_2d); Allocate memory for texture data unsigned char* data = new unsigned char[size*size*3]; Create alias for our texture GLuint texture; Genrate a new texture in the graphics memory glgentextures(1, &texture); Martin Ilčík 15

16 Hello texture! (2) Send the texture data to the graphics card glbindtexture(gl_texture_2d, texture); glteximage2d(gl_texture_2d, 0, GL_RGB, size, size, 0, GL_RGB, GL_UNSIGNED_BYTE, data); delete[] data; Set the bilinear interpolation (tent filter)!!! gltextureparameteri(gl_texture_2d, GL_TEXTURE_MIN_FILTER, GL_LINEAR); gltextureparameteri(gl_texture_2d, GL_TEXTURE_MAG_FILTER, GL_LINEAR); Martin Ilčík 16

17 Hello textured quad! //don t forget to bind a texture before glbegin(gl_quads); gltexcoord2f(0.0f, 0.0f); glvertex3f(-1.0f, -1.0f, 0.0f); gltexcoord2f(1.0f, 0.0f); glvertex3f(1.0f, -1.0f, 0.0f); gltexcoord2f(1.0f, 1.0f); glvertex3f(1.0f, 1.0f, 0.0f); gltexcoord2f(0.0f, 1.0f); glvertex3f(-1.0f, 1.0f, 0.0f); More in NeHe Lesson 6 Martin Ilčík 17

18 Black hole? You ll see nothing, unless you fill the texture data with some values This has to be done before calling glteximage Texture data just a pointer to some memory block copied to the graphics memory Try some constant color or just random noise Our volumetric data is just a 3D image even in raw format Related stuff in NeHe lesson 33 Martin Ilčík 18

19 Slices as textures Generate separate texture for each slice for each axis Statically precomputed (needs memory) Dynamically generated on demand Save all the volume data in one 3D texture Easy slicing for arbitrary plane HW accelerated lookups (incl. interpolation) Can be used as input for GPU raycasting glteximage3d, gltexcoord3f, GL_TEXTURE_3D Martin Ilčík 19

20 Reading the dataset FILE* indata; indata = fopen(filename, rb ); unsigned short dimx, dimy, dimz; fread(&dimx, 1, 2, indata); //one entry fread(&dimy, 1, 2, indata); //two bytes long fread(&dimz, 1, 2, indata); unsigned short* imdata = new unsigned short[dimx*dimy*dimz]; fread(imdata, dimx*dimy*dimz, 2, indata); Martin Ilčík 20

21 Slices as 2D textures (1) ReadData(); //let the dataset be stored in data[] unsingned shot* slicedata = new unsigned short[size*size]; k = 10; for (int i = 0; i < dimx; i++) for (int j = 0; j < dimy; j++) slicedata[(size*j)+i] = data[(dimx*dimy*k) + (dimx*j) + i]; Martin Ilčík 21

22 Slices as 2D textures (2) GLuint slice; glgentextures(1, &slice); glbindtexture(gl_texture_2d, slice); glteximage2d(gl_texture_2d, 0, GL_LUMINANCE, dimx, dimy, 0, GL_LUMINANCE, GL_UNSIGNED_SHORT, slicedata); //don t forget to set the filters delete[] slicedata; for arbitrary data dimensions GL_TEXTURE_RECTANGLE_ARB or use glubuild2dmipmaps gltexsubimage2d Martin Ilčík 22

23 Slices as 3D textures (1) ReadData(); unsingned shot* slicedata = new unsigned short[dimx*dimy*dimz]; //we convert the 12-bit values to 16-bit for(int i = 0; i < dimx*dimy*dimz; i++) slicedata[i] = data[i]*16; Martin Ilčík 23

24 Slices as 3D textures (2) GLuint slices; glgentextures(1, &slices); glbindtexture(gl_texture_3d, slice); glteximage3d(gl_texture_3d, 0, GL_LUMINANCE, dimx, dimy, dimz, 0, GL_LUMINANCE, GL_UNSIGNED_SHORT, slicedata); //don t forget to set the filters delete[] slicedata; for non 2 k data dimensions GL_texture_non_power_of_two or use glubuild3dmipmaps gltexsubimage3d Martin Ilčík 24

25 Slices as 3D textures (3) k = 10.0f; glbegin(gl_quads); gltexcoord3f(0.0f, 0.0f, k/(float)dimz); glvertex3f(-1.0f, -1.0f, 0.0f); gltexcoord3f(1.0f, 0.0f, k/(float)dimz); glvertex3f(1.0f, -1.0f, 0.0f); //add the 2 remaining vertices in the same way glend(); Martin Ilčík 25

26 Transfer Function Interactive enhancement of the data by coloring [Intensity] [RGBA] Values defined at arbitrary points Interpolation What about trying to use L*a*b* Martin Ilčík 26

27 Transfer Texture a 1D texture X axis intensity [x] [r,g,b,a]; <0,1> <0,1> 4 Easy to show to the user Can be applied to slices/volumes in a shader Automatic interpolation of colors Must be regenerated after change Sampling errors Take a big texture (e.g. dimx == 4096) Martin Ilčík 27

28 Intermediate OpenGL Martin Ilčík 28

29 Resources Read more tutorials Download the OpenGL specs Get the latest headers (e.g. gl.h, glu.h, glext.h) Google Ask me :) Martin Ilčík 29

30 Transformations Matrices included in OpenGL state Modelview glmatrixmode(gl_modelview); Projection glmatrixmode(gl_projection); Reset the current matrix glloadidentity(); Matrix stack glpushmatrix(); glpopmatrix(); Transformations gltranslatef( ), glrotatef( ), glscalef( ) glmultmatrix( ); More in this slides (Ed Angel, UNM) Martin Ilčík 30

31 Camera stuff Camera position and viewing direction glulookat(eye,center,up); Projection type, clipping planes glperspective(angle, ratio, znear, zfar); glortho(left, right, bottom, up, znear, zfar); Should be updated by viewport changes More in this slides by Ed Angel, UNM Martin Ilčík 31

32 Memory management Objects allocated by OpenGL calls should be cleaned up, when no more needed Video RAM Stores textures, shaders, displaylists, vertexarrays, VBO s, FBO s When filled full, swapping to RAM SLOW! By visualization applications easily overflowed Use gldeletetextures( ) etc. Martin Ilčík 32

33 Multitexturing glactivatetexture(gl_texturen); Switches all texturing commands to the unit n Now texture compositing modes come to play gltexenv(gl_texture_env, GL_TEXTURE_ENV_MODE, mode) GL_NONE, GL_REPLACE,GL_MODULATE Deactivate texture units when no more used gldisable(gl_texture_2d); glmultitexcoord(gl_texturen, s, t ); More in OpenGL specs Martin Ilčík 33

34 Basic shaders stuff Low level GPU assembler (no more used) NVidia s Cg Crossplatform (OpenGL, DirectX) Different versions GLSL Integrated in OpenGL 2.0 Compiled and linked at runtime Works only with shaders 2.0 and higher CUDA Martin Ilčík 34

35 GLSL Shaders initialization Read source from a text file Create vertex/fragment shader object glcreateshaderobjectarb(shader_type); glshadersourcearb( ); glcompileshaderarb(shader); Create GPU program object glcreateprogramobject(); glattatchobjectarb(prog, shader); gllinkprogramarb(prog); Martin Ilčík 35

36 Enabling GLSL shaders gluseprogramobjectarb(program); passing parameters to variables inside of the shader glgetuniformlocationarb(program, name); gluniformarb(location, value); sampler variables must be set this way n = glgetuniformlocationarb(prog, sampler_name); gluniform1iarb(n, texture_unit_number); More in this article at NeHe (by Florian Rudolf) Martin Ilčík 36

37 Transfer function in GLSL (1) Vertex shader just as fixed pipeline void main() { } gl_texcoord[0] = glmultitexcoord0; gl_texcoord[1] = glmultitexcoord1; gl_position = gl_modelviewprojectionmatrix * gl_vertex; Martin Ilčík 37

38 Transfer function in GLSL (2) Fragment shader uniform sampler2d slice_sampler; uniform sampler1d transfer_sampler; void main() { } vec4 intensity = texture2d(slice_sampler, vec2(gl_texcoord[0]) ); gl_fragcolor = texture1d(transfer_sampler, intensity.x); Martin Ilčík 38

39 GPU Raycasting Advanced OpenGL Martin Ilčík 39

40 How it works (1) Martin Ilčík 40

41 How it works (2) Implementation One fragment (pixel) = one ray HW makes it simpler Implementation in shaders Use direction textures for the ray Iterate all the steps in a fragment shader Data storage Render directions to textures Store data in one or more 3D textures Store the transfer function also in a texture Martin Ilčík 41

42 Direction textures Lowpoly envelope around the dataset Take simply a box Map world coordinates to it s primary color Render Front faces for entry points of rays Back faces for exit points of rays Store in two textures Martin Ilčík 42

43 Hardware ray initialization Rendering in 3 passes 1 st... get texture with ray-entry coordinates 2 nd... get texture with ray-exit coordinates 3 st... actual raycasting In the fragment shader Compute ray directions Sample trough the dataset iterate steps until opacity ~ 1.0 or ray outside Martin Ilčík 43

44 Rendering to texture Frame buffer object New extension in OpenGL 1.5 Finally fast access to offscreen buffers API feature, independent from hardware Backwards compatible, depends on drivers Interface similar to multitexturing More on LWJGL Wiki Martin Ilčík 44

45 Creating texture storage glgentextures(1, FBOTextureFront); glbindtexture( GL_TEXTURE_RECTANGLE_ARB, FBOTextureFront); glteximage2d( GL_TEXTURE_RECTANGLE_ARB, 0, GL_RGBA8, ScreenX, ScreenY, 0, GL_RGBA, GL_UNSIGNED_BYTE, 0); Martin Ilčík 45

46 Creating and linking a FBO glgenframebuffersext(1, &FBufferFront); glbindframebufferext( GL_FRAMEBUFFER_EXT, FBufferFront); glclearcolor(0, 0, 0, 0); glframebuffertexture2dext( GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT, GL_TEXTURE_RECTANGLE_ARB, FBOTextureFront, 0); Create a FBO for the back faces just the same way Martin Ilčík 46

47 Rendering direction textures (1) Bind the front faces buffer and draw them glbindframebufferext( GL_FRAMEBUFFER_EXT, FBufferFront); glclear(gl_color_buffer_bit); gluseprogramobjectarb(prog1st); drawboundingbox(); Martin Ilčík 47

48 Rendering direction textures (2) Bind the back faces and draw them glcullface(gl_front); glbindframebufferext( GL_FRAMEBUFFER_EXT, FBufferBack); glclear(gl_color_buffer_bit); drawboundingbox(); unbind the texture and render to screen glbindframebufferext( GL_FRAMEBUFFER_EXT, 0); glcullface(gl_back); Martin Ilčík 48

49 Rendering direction textures (3) Martin Ilčík 49

50 Raycasting in fragment shader Check if the ray crosses the bounding box Read the entry and exit point from textures Determine ray direction Iterate until opacity ~ 1.0 or outside Read the intensity at this sample Apply transfer function Compositing Next sample Martin Ilčík 50

51 How it works once again Martin Ilčík 51

52 Fragment shader (1) Input samplers uniform sampler2drect tex_entry; uniform sampler2drect tex_exit; uniform sampler3d tex_intensity; uniform sampler1d tex_transfer; Martin Ilčík 52

53 Fragment shader (2) void main() { const float dz = 0.005; const int maxrange = 347; vec4 entry_point = texture2drect(tex_entry, vec2(gltexcoord[0])); vec4 exit_point = texture2drect(tex_exit, vec2(gltexcoord[0])); float dist = distance(entry_point, exit_point)/dz; int maxiter = int(floor(dist)); vec3 diff = (exit_point.xyz entry_point.xyz)/dist; Martin Ilčík 53

54 Fragment shader (3) if (entry_point.w == 0.0) discard; else { float numsamples = 0.0f; for(int = 0; i < maxrange; i++) { // see next slide } glfragcolor = Result; } Martin Ilčík 54

55 Fragment shader Compositing (4) intensity = texture3d(tex_intensity, point); transferred = texture1d(tex_transfer, intensity.x); Result.xyz += (1.0 Result.w) * transferred.w * transferred.xyz; Result.w += (1.0 Result.w) * transferred.w; point +=diff; if ((Result.w >= 1.0) (i >= maxiter)) break; Martin Ilčík 55

56 Fragment shader Averaging (5) intensity = texture3d(tex_intensity, point); transferred = texture1d(tex_transfer, intensity.x); Result.xyz += transferred.w * transferred.xyz; Result.w += transferred.w; point +=diff; if (i >= maxiter) { Result /= float(maxiter); break; } Martin Ilčík 56

57 Conclusions Use hardware rendering Use the best API for you Store data in textures Shaders make things much simpler and faster Feel free to ask: ilcik at cg.tuwien.ac.at Thank you for your attention! Martin Ilčík 57