Mining the Rendering Power in Web Browsers. Jianxia Xue Jan. 28, 2014

Size: px

Start display at page:

Download "Mining the Rendering Power in Web Browsers. Jianxia Xue Jan. 28, 2014"

Angelica Fitzgerald
6 years ago
Views:

1 Mining the Rendering Power in Web Browsers Jianxia Xue Jan. 28, 2014

2 Outline Web application as software deployment platform WebGL: Graphics API inside browsers Explore browser rendering capability through three case studies Looking into the future

online latex editor Autodesk AutoCAD 360

3 Web Applications for Current Web App WebGL Case Studies Future web app Content Creation Online editing of document, presentation, spreadsheet online latex editor Autodesk AutoCAD 360 Diagram creator No download, direct manipulation through browser Easy collaborative content creation Easy sharing, publishing, cloud backing up

4 Current Web App WebGL Case Studies Future web app The 5th revision of the markup language for web content since HTML5 + CSS3 + Javascript as one static content, and wrapper of all styling even with variable support functions that fetch, store, upload, process dynamic content - The workhorse of web app logics

Current Web App WebGL Case Studies Future web app Javascript

dynamic typing, multiparadigm (procedural, object oriented,

high-traffic apps Easy UI and AJAX tools three.

5 Current Web App WebGL Case Studies Future web app Javascript web browser client-side scripting language prototype-based, dynamic typing, multiparadigm (procedural, object oriented, functional) Rich libraries Server-side engine for real-time high-traffic apps Easy UI and AJAX tools three.js r65 3D graphics lib Yahoo User Interface (YUI) Google Web Toolkit (GWT) 2D data visualizer

6 Graphics API Inside Current Web App WebGL Case Studies Future Web App Browsers A Javascript API for Web Graphics Library Web browser can use rendering capabilities from OpenGL ES 2.0 and GLSL directly Designed and maintained by Khronos Group who also manages OpenGL graphics API series This means that one can use GPU shaders directly inside a web app!

7 Current Web App WebGL Case Studies Future Web App

0 57% Chrome Firefox Internet Explorer Safari Opera v9 Data from 2013

8 Current Web App WebGL Case Studies Future Web App WebGL Browser Support 2% 4% v11 9% v6.0 v11 28% v4.0 57% Chrome Firefox Internet Explorer Safari Opera v9 Data from 2013 December Rendering Power in Browsers J. Xue physics.olemiss 1/28/14

9 The OpenGL Graphics Current Web App WebGL Case Studies Future Web App OpenGL with GLSL since 2.0 OpenGL ES 2.0 Slimmed, and optimized for embedded system API WebGL a close binding to Javascript Vender 3D libraries such as three.js web graphics application OpenGL ES 2.0 made for GPU based rendering, enforcing the use of vertex and fragment shaders

10 Modern Graphics Current Web App WebGL Case Studies Future Web App Vertex Processor Fragment Processor Vertex Processor Fragment Processor Pipeline Application Data... Rasterizer Vertex Processor Fragment Processor Vertex Processor Fragment Processor RGBAZ RGBAZ RGBAZ RGBAZ Frame Buffer Blender Rendered Frame

11 Current Web App WebGL Case Studies Future Web App Reaching to the Coding Spot <html lang="en"><head><script> var vertexshader = "..."; var fragmentshader = "..."; function initwebgl(canvas) { GLSL strings if (!canvas) { canvas = document.createelement('canvas'); document.body.appendchild(canvas); } var gl = null; try { gl = canvas.getcontext("experimental-webgl") canvas.getcontext("webgl"); } catch(e) {} if (!gl) { alert("your browser does not support WebGL!"); return gl; } // creates, compile, attach, link, and activate a GLSL program object // setup application data return gl; } function drawscene(gl) {... } </script></head> <body><script> var gl = initwebgl(); drawscene(gl); </script></body></html> shader programs in Application data data and and draw routine draw routine canvas dom element as the frame container for webgl to draw upon canvas.getcontext to reach WebGL API shader programs are dynamically activated using WebGL API

12 Current Web App WebGL Case Studies Future Web App Example Apps Rendering Power in Browsers J. Xue physics.olemiss 1/28/14

13 Case Studies to Explore the Current Web App WebGL Case Studies Future Web App Rendering Power in Browsers Fragment shader capability: Mandelbrot Set Navigator Vertex shader capability: Color space morphing Realtime Image Processing and Computer Vision using WebRTC (Chrome and Firefox only)

Mandelbrot Set Current Web App WebGL Case Studies Future Web App Navigator Vertex data are simply 4 corner points of a 2D rectangle, the graphics pipeline creates the fragments according to the

14 Mandelbrot Set Current Web App WebGL Case Studies Future Web App Navigator Vertex data are simply 4 corner points of a 2D rectangle, the graphics pipeline creates the fragments according to the viewport size Fragment shader carries the main computation of the escape method Application data manages the viewport pixel to complex value mapping according to user controlled zooming and translation factors

15 Current Web App WebGL Case Studies Future Web App Inside the Fragment Shader Per Pixel Computation uniform float ccx; uniform float ccy; uniform vec2 fviewportsize; uniform float zoom; uniform int nmaxiter; void main(void) { const int MaxIter = 512; // Rescale the coordinate float Real0 = (gl_fragcoord.x - fviewportsize.x / 2.0); // GLSL use lower left as the origin by default float Imag0 = -(gl_fragcoord.y - fviewportsize.y / 2.0); float x0 = ccx + zoom * Real0; float y0 = ccy + zoom * Imag0; float R = 0.0; float I = 0.0; float R2 = R*R; float I2 = I*I; } int LastIter=-1; for (int iter=0; iter < MaxIter; iter++) { I=(R+R)*I + y0; R=R2-I2 + x0; R2=R*R; I2=I*I; if (R2+I2 >= 4.0 ) { LastIter = iter; break; } if (iter >= nmaxiter) { LastIter = 0; break; } } vec4 color; color = vec4(1.0 - ( *cos(angle*2.0)), ( *cos(angle*3.0)), ( *cos(angle*5.0)), 1.); } gl_fragcolor = color;

16 Profiling Method and Current Web App WebGL Case Studies Future Web App Result Use console.time and console.timeend Real-time navigation speed on full HD resolution rendering < 1 millisecond rendering time for 1M pixel The pixelation at zoom-in scale of 2-20 real values are due to the GLSL support of float type only.

update of every vertex s position per frame Fragment

17 Current Web App WebGL Case Studies Future Web App Color Space Morphing RGB HSV Vertex shader carries the update of every vertex s position per frame Fragment shader simply copies the vertex color since only points exist

$attribute vec3 argbpos; attribute vec3 ahsvpos; varying vec4 vcolor; void main(void) { } gl_pointsize = upointsize; float s$ = 0.5*(cos(uTime)+1.0); vec3 pos = argbpos * s + ahsvpos * (1.0-s); vcolor = vec4(argbpos, 1.

= 0.5*(cos(uTime)+1.0); vec3 pos = argbpos * s + ahsvpos * (1.0-s); vcolor = vec4(argbpos, 1.

18 Current Web App WebGL Case Studies Future Web App Inside Vertex Shader uniform float upointsize; uniform float utime; attribute vec3 argbpos; attribute vec3 ahsvpos; varying vec4 vcolor; void main(void) { } gl_pointsize = upointsize; float s = 0.5*(cos(uTime)+1.0); vec3 pos = argbpos * s + ahsvpos * (1.0-s); vcolor = vec4(argbpos, 1.); per vertex color, will be relayed to fragment gl_position = projectionmatrix * modelviewmatrix * vec4(pos, 1.); Per Vertex Computation of position and color

19 Current Web App WebGL Case Studies Future Web App Profiling Result use~0.25 ms per frame use more advance profiling tool provided by Chrome about:tracing Vertex processing is more expensive than fragment processing

20 Current Web App WebGL Case Studies Future Web App ToyCam Use WebRTC getusermedia, streaming user webcam or mic data directly Video streaming can be used as a texture for GPU shaders to further process

21 Current Web App WebGL Case Studies Future Web App WebRTC Open project that enables browsers with Real-Time Communication (RTC) via javascript p2p chat can be done directly in an webapp Supported by Chrome, Firefox, and Opera as of now

22 Current Web App WebGL Case Studies Future Web App Web-RTC Architect

23 getusermedia() Current Web App WebGL Case Studies Future Web App <!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <title>html5 getusermedia Demo By Arunkumar Gudelli</title> <script src=" type="text/javascript"></script> <script> function onfailure(err) { alert("the following error occured: " + err.name); } jquery(document).ready(function () { var video = document.queryselector('#webcam'); navigator.getusermedia = (navigator.getusermedia navigator.webkitgetusermedia navigator.mozgetusermedia navigator.msgetusermedia); if (navigator.getusermedia) { navigator.getusermedia ( { video: true }, function (localmediastream) { video.src = window.url.createobjecturl(localmediastream); }, onfailure); } else { alert('oops No browser Support'); } }); </script> </head> <body> <div> <video id="webcam" width="500" autoplay></video> </div> </body> </html>

24 Current Web App WebGL Case Studies Future Web App ToyCam Data Flow webrtc getusermedia Texture mapping used in the application to apply video frames to arbitrary shapes dom element video GLSL sample2d texture fragment shader process

25 Current Web App WebGL Case Studies Future Web App Future Web App More UCIs without keyboard/mouse More 3D games with audio/visual interactions Strong client-side visualization and basic signal processing + server-side clouding computing using aggregated user data

26 More Apps Using Realtime Current Web App WebGL Case Studies Future Web App Client Audio-Visual Signal

27 Current Web App WebGL Case Studies Future Web App WebCL on its Way Javascript binding to OpenCL Firefox (by Nokia) and Webkit (by Samsung) open sourced prototype released in mid 2011 This makes computations that are not necessarily lined up with graphics pipeline possible in GPU, e.g. physics engines in 3D games, more advanced computer vision algorithms Web deployment platform will become more common for computation intensive applications

28 Conclude Browser rendering power strongly supports heavily graphics applications such as 3D games, scientific visualizations, etc Web app development is easy given the mutual and continuing web UI and browser engine optimization Javascript Client Computing + Cloud Server Computing will boom big data artificial intelligence inferred applications

29 Thank You! Questions?

BCA611 Video Oyunları için 3B Grafik

BCA611 Video Oyunları için 3B Grafik WebGL - Shader Programming Zümra Kavafoğlu Canvas element The HTML element is used to draw graphics on a web page. Create canvas with id, width and height