WebGL and GLSL Basics. CS559 Fall 2016 Lecture 14 October

Transcription

1 WebGL and GLSL Basics CS559 Fall 2016 Lecture 14 October

2 Review

3 Hardware Rasterization For each point: Compute barycentric coords Decide if in or out.7,.7, , 0, -.1.9,.05,.05.33,.33,.33

4 1988: The Personal Iris

5 A Pipeline 1 transf light project raster shade write

6 Applica>on Program The pipeline (1988) Graphics Driver Command Buffer (Triangle Queue) Queue Processing (TCL) Cache Assembly Triangle Processing Rasterize Pixel Queue Pixel Processing Pixel Tests shading Geometry Shading Pixel Shading Texture Memory Render to texture Frame Buffer

7 Applica>on Program The pipeline (2006-current) Graphics Driver Command Buffer (Triangle Queue) Queue Processing (TCL) Cache Assembly Triangle Processing Rasterize Pixel Queue Pixel Processing Pixel Tests shading Geometry Shading Pixel Shading Texture Memory Render to texture Frame Buffer

8 Applica>on Program Graphics Driver The parts you have to program Now (in addition to above) Command Buffer (Triangle Queue) Queue Processing (TCL) Cache Assembly Triangle Processing Rasterize Pixel Queue Pixel Processing Pixel Tests shading Geometry Shading Pixel Shading Texture Memory Render to texture Frame Buffer

9 A Triangle s Journey

10 A Program to Draw a Triangle The complete WebGL thing we need Doing each necessary steps Just one triangle

11 Just a Triangle

12 HTML like you are used to

13 A Lot of Code

14 Look at the process inside-out We ll start with the end of the pipeline And work backwards

15 Applica>on Program Graphics Driver The parts you have to program Now (in addition to above) Command Buffer (Triangle Queue) Queue Processing (TCL) Cache Assembly Triangle Processing Rasterize Pixel Queue Pixel Processing Pixel Tests shading Geometry Shading Pixel Shading Texture Memory Render to texture Frame Buffer

16 I warned you: Applica>on Program This step will get to be prexy exci>ng Graphics Driver Command Buffer (Triangle Queue) Queue Processing (TCL) Cache Assembly Triangle Processing Rasterize Pixel Queue Pixel Processing Pixel Tests shading Geometry Shading Pixel Shading Texture Memory Render to texture Frame Buffer

17 We have to program this step! Applica>on Program Graphics Driver The program is called the: Fragment Shader Command Buffer (Triangle Queue) Queue Processing (TCL) Cache Assembly Triangle Processing Rasterize Pixel Queue Pixel Processing Pixel Tests shading Geometry Shading Pixel Shading Texture Memory Render to texture Frame Buffer

18 Pixel in à Pixel out, each independent Applica>on Program All we do is change its values Graphics Driver Command Buffer (Triangle Queue) Queue Processing (TCL) Cache Assembly Triangle Processing Rasterize Pixel Queue Pixel Processing Pixel Tests shading Geometry Shading Pixel Shading Texture Memory Render to texture Frame Buffer

19 Coming axrac>ons Applica>on Program This step will get to be prexy exci>ng Graphics Driver Command Buffer (Triangle Queue) Queue Processing (TCL) Cache Assembly Triangle Processing Rasterize Pixel Queue Pixel Processing Pixel Tests shading Geometry Shading Pixel Shading Texture Memory Render to texture Frame Buffer

20 The Pixel Shader Given information about the pixel Compute color Optionally, compute other things

21 How to program it Use a Shading Language We ll use a language called GLSL Compiler built into WebGL Language specifics as we go

22 A (boring) Fragment Shader void main(void) { gl_fragcolor = vec4(0.0, 1.0, 1.0, 1.0); }

23 A (boring) Fragment Shader void main(void) { Shaders define a main func>on that take no arguments return no values gl_fragcolor = vec4(0.0, 1.0, 1.0, 1.0); }

24 A (boring) Fragment Shader void main(void) { GLSL Shaders operate by side effects on special variables (they look like globals) gl_fragcolor = vec4(0.0, 1.0, 1.0, 1.0); }

25 A (boring) Fragment Shader void main(void) GLSL has types useful in graphics Like 4 vectors { gl_fragcolor = vec4(0.0, 1.0, 1.0, 1.0); } This is opaque yellow (even colors are 4- vectors)

26 Processing Applica>on Program Graphics Driver We have to program this part too Also in GLSL Command Buffer (Triangle Queue) Queue Processing (TCL) Cache Assembly Triangle Processing Rasterize Pixel Queue Pixel Processing Pixel Tests shading Geometry Shading Pixel Shading Texture Memory Render to texture Frame Buffer

27 Process each vertex independently Applica>on Program Graphics Driver Transform compute x and n Clip Light compute c Command Buffer (Triangle Queue) Queue Processing (TCL) Cache Assembly Triangle Processing Rasterize Pixel Queue Pixel Processing Pixel Tests shading Geometry Shading Pixel Shading Texture Memory Render to texture Frame Buffer

28 What data about vertices? Inputs: Position Other Stuff A/ributes from applica>on Outputs: Position Other Stuff varying proper>es to framgment Shaders (remember interpola>on)

29 The simplest vertex shader attribute vec3 pos; void main(void) { } gl_position = vec4(pos, 1.0); }

30 The simplest vertex shader attribute vec3 pos; void main(void) { Shaders define a main func>on that take no arguments return no values gl_position = vec4(pos, 1.0); } }

31 The simplest vertex shader attribute vec3 pos; Shaders output by side effects: seang special variables void main(void) { } gl_position = vec4(pos, 1.0); }

32 The simplest vertex shader attribute vec3 pos; Shaders get input by reading special variables void main(void) { } gl_position = vec4(pos, 1.0); }

33 Special Variables Built in (magic) gl_position output of vertex shader gl_fragcolor output of frag shader User Defined attributes inputs to vertex shader varying output from vertex to fragment uniform constant over triangle group

34 The simplest vertex shader attribute vec3 pos; We are defining our own special variable void main(void) { } gl_position = vec4(pos, 1.0); }

35 The simplest vertex shader attribute vec3 pos; void main(void) { Cool GLSL feature: type conversions gl_position = vec4(pos, 1.0); } }

36 No Transformation? I will assume the position is already in the right coordinate system. The rasterizer (and everything else) works in Normalized Device Coordinates (NDC) -1 to 1 in each dimension

37 Applica>on Program Graphics Driver Start here Setup modes (window, ) Setup transform, lights Draw a triangle Posi>on, color, normal Command Buffer (Triangle Queue) Queue Processing (TCL) Cache Assembly Triangle Processing Rasterize Pixel Queue Pixel Processing Pixel Tests shading Geometry Shading Pixel Shading Texture Memory Render to texture Frame Buffer

38 In JavaScript using WebGL

39 The beginning

40 The beginning This should look like HTML5 Canvas You can have multiple contexts (draw canvas over WebGL)

41 Now about those shaders Get them into strings Use a library to read them from resources

42 Run the compiler!

43 Error Checking Here I checked for errors (since I often have syntax errors) You should check for errors everywhere

44 Run the compiler again! Need to compile both shaders

45 Link the shaders together Shaders always work in pairs Need to connect them

46 Setup the special variables Important to communicate with shaders

47 The simplest vertex shader attribute vec3 pos; Javascript needs to connect to the pos variable void main(void) { } gl_position = vec4(pos, 1.0); }

48 Communicating an attribute We give it an array of attributes Assign it to a position We have to ask which position

49 OK, Now for our triangle How do we get this data to the hardware? Need to do a block transfer

50 Applica>on Program Graphics Driver Need to get the ver>ces to the hardware fast! (normally more than 3) Command Buffer (Triangle Queue) Queue Processing (TCL) Cache Assembly Triangle Processing Rasterize Pixel Queue Pixel Processing Pixel Tests shading Geometry Shading Pixel Shading Texture Memory Render to texture Frame Buffer

51 Key Idea: Buffer Create a buffer buffer = a block of memory on the GPU Copy the data into the buffer Must be a special JavaScript object: Float32Array (array of fixed types)

52 Now to draw First we have to clear the screen Notice that color is a 4-vector I don t really need the z-buffer

53 Now we actually draw the triangle Notice that we use the shaders and the buffer

54 All that for a triangle!

55 Is it really 100 lines of code? Not really lots of comments Build wrappers to be more concise you do the same thing over and over But there are lots of steps and you should understand them

56 Two triangles Can you see where these triangles will go? (remember they are in NDC)

57 Change the array sizes

58 Two triangles

59 How do we color them differently?

60 Color per vertex Add an attribute for each vertex so we can pass a color for each Have the vertex shader output the color varying variable for fragment shader Have the fragment shader input the color

61 A (boring) Fragment Shader void main(void) { gl_fragcolor = vec4(0.0, 1.0, 1.0, 1.0); }

62 A (less boring) Fragment Shader precision highp float; varying vec3 outcolor; Our own magic variable! void main(void) { gl_fragcolor = vec4(outcolor, 1.0); }

63 A (less boring) Fragment Shader precision highp float; varying vec3 outcolor; Required so the shaders can talk void main(void) { gl_fragcolor = vec4(outcolor, 1.0); }

64 Connecting Shaders varying variables connect shaders the output of a vertex shader becomes the input to a fragment shader The 3 vertices of a triangle are interpolated

65 The simplest vertex shader attribute vec3 pos; void main(void) { } gl_position = vec4(pos, 1.0); }

66 The (almost) simplest vertex shader attribute vec3 pos; varying vec3 outcolor; void main(void) { gl_position = vec4(pos, 1.0); } outcolor = vec3(1.0,0.0,1.0); }

67 Two purple triangles hxp://jsbin.com/wecaci/edit?js,output

68 Make color an input as well attribute vec3 pos; attribute vec3 incolor; varying vec3 outcolor; void main(void) { gl_position = vec4(pos, 1.0); } outcolor = incolor; }

69 Remember We can t pass values directly to a fragment we don t even know what they will be! We pass attributes of vertices which can then pass them to fragments

70 Now to connect to JavaScript

71 Colors per vertex

72 Put them in a buffer

73 When we draw, use 2 buffers

74 Two triangles hxp://jsbin.com/digupi/edit?js,output

75 Apply a transformation One transformation for the triangle group It is constant over the drawarrays call This is a uniform variable hxp://jsbin.com/>rapu/19/edit?js,output

76 Simplifying the Code There is stuff you do over and over and Write it once and use it often Or let someone else write it once This is where twgl comes in

77 Compile two vertex programs For each run the compiler check for errors Link them together Attach to the attributes Set up to specify the uniforms

78 Do it by hand Do it with twgl var shaders = twgl.createprograminfo(gl, ["vs", "fs"]); Yes, one line And it grabs the string from script tags so they are separate from your JS program. But the documentation is terrible.

79 How about those shaders They do very specific things you need to understand the pipeline They have 3 kinds of weird variables you need to understand the model They are written in a cool language you ll pick it up quickly The language has a bunch of useful stuff look at the quick reference card

80 Learning Shader Programming Connecting your program to shaders is hard So, don t bother (yet) Use a Shader IDE that lets you focus on shaders Gives you an object, a program,

81 Some things about GLSL Very strongly typed float x = 1; // error! integer and float Cool sub-vector access: vec3 v; v.xy (a 2-vector) vec4(v,1) (a 4-vector) vec4(v.xy, v.zx)

82 More cool stuff about GLSL Lots of handy math functions They know it s for graphics! Limited control structures parallel execution means all the same Conditional functions step, softstep,