1 EEL 5771 Introduction to Computer Graphics Team Essay: Fall 2018 Active learning project: non-photorealistic rendering Submission deadlines: Questions: September 11, 2018 Essay: October 9, 2018 Team name: Team Antonia USFIDs: - U 1678-4438 U 9857-8275 U 7016-5520 E-mails: parush@mail.usf.edu, livyaharika@mail.usf.edu,amrutha1@mail.usf.edu TEAM MEMBERS CONTRIBUTIONS Member [Name]: Parush Gera Abstract, Section 1, Section 3 and Section 6 and review of whole essay. Member [Name]: Livya Harika Choday: Section 2 and review of whole essay. Member [Name]: Amrutha Mudragada : Section 4 and 5 and review of whole essay.
2 Abstract: Graphics is a major area of art and representation. But, It is not only limited to it. With the increase in the technology, graphics have been integrated with the computation capabilities which gave birth to a new era of computer graphics. We can see the use of computer graphics in our daily life in our Smartphones, Videos, Films etc. However, it is essential to understand in depth of various concepts used in it. One of those concepts is the concept of creating an image from a virtual scene. This is called rendering. Further the paper explores the area of non-photo realistic rendering and its techniques. Also, the difference between photorealism and nonphotorealism is a crucial topic to understand as a researcher. This topic has been covered in depth in this paper. There has been tremendous advancement in computer graphics which has made possible to represent various imaginary ideas into an understandable form. A brief history of computer graphics from 1970 1990 has been discussed. It is very exciting to understand how the advancement in technology has given such an influential implementation on human kind. Keywords: Non-photo realistic rendering, Photorealism, Non-Photorealism, Shading, Techniques of NPR, Augmented Reality 1.Introduction Since the inception of mankind Graphics has played vital role. With tremendous and continuous research there have been advancement in the concepts of computer graphics. Today, in 21 st century, we have such a advance technology which has enable us to implement many theoretical concepts which were founded by many researches in the earlier 90 s. The utility of computer graphics has a very huge effect on academia and industry which has triggered the interest of many developers and researchers to keep continuing the research. This is the main motivation being writing this paper to understand the basic concepts on Computer Graphics. One of the most important active learning topic is Non- Photorealistic rendering which has been explored in a advanced manner in this paper. There are many input and output devices now days which are made with the help of advancement in the hardware technology, and they are performing graphic computations very efficiently. This is why we have discussed the difference between photorealism and non-photorealism in Augmented reality. Various important techniques of Non-photorealistic rendering have been discussed broadly to understand the working of it more efficiently. It is equally important to know about the history of computer graphics to understand the advancement in the technology and the problems that researchers have faced while bringing this advancement to the particular area. Section 2 has described the history of computer graphics from 1970 till 1990. Section 3 is focusing on understanding the concept on Non-photorealistic rendering. Section 4 covers the basic principles to follow while creating a graphic illustration. Section 5 covers the description of basic NPR techniques and in section 6 a comparison of photorealism and non-photorealism in Augmented reality has been discussed broadly
3 2. History of computer graphics: from the 1970s to 1990s There have been huge amount of discoveries in 1970 s. Shading algorithms were introduced first and they were used to represent graphics with a new look having much deeper knowledge comparative to earlier methods which gave a flat look Henri Gouraud, Bui Toung Phong and Jim Blinn provided a major contribution towards the improvement in computer graphics as they founded shaded models for representation of graphics. When an object is represented using flat shading, they look like a solid which does not look real. To overcome this problem, the objects were created using small polygons. The usage of polygons resulted in enhancement of details and complexity of objects. This method was called as and was proposed by Henri Gouraud, which made rendering look more real by interpolating the colors across the edges of a polygon while representing a curved surface. [3] Figure 1: Interpolation of colors across the edges. Today with the availability of advanced GPU, it is possible to implement complex algorithms making photo rendering looks more realistic which were earlier used rarely due to non-availability of such machines. The method focuses on coloring of each pixel and pixels adjacent to them averagely. 1970 was a very important year in the history of computer graphics because of efficient and important upgradations brought up in the graphic computing. ACM (Association for computing machinery) held their very first major conference on computer graphics after recognizing computer graphics as special interest group. In 1974, application of texture mapping to 3d was introduced by Edwin Catmull. Texture mapping is a technique which defines high level of frequencies on a 3D graphic. The application of this algorithm is on polygon surfaces where the idea is to extend the 2D image on 3D object. In 1975 Fractal geometry was introduced by Mandelbrot. He described it as a method or stimulating natural phenomenon and creating realistic representation out if it. Later in 1980 the method of ray tracing was introduced by Turner Whitted. This method was used on surfaces with high reflections to simulate them. The idea behind this algorithm was that it traces the light reflection beginning from every ray from the viewer s point of view till the 3D till the time the ray hits something. This algorithm provides high level of realism with a complex and intense computation. In 80 s the computer graphics were adopted by a lot of audience as 16-bit microprocessor transformed computer graphics extensively, although they have are highly intelligent workstations. The graphics and its application were now considered as highly intelligent workstations. Due to increase in demand and adaptability of computer graphics, number of developers in this area were increased significantly. Apple adopted computer graphics and integrated into their first Macintosh computer in 1984. VGA standard of graphics were introduced in 1987 and SVGA were introduced in 1989. Widely used software AutoCAD was introduced in 1982. Vedio games were improved by the help of advancement in computer graphics in 1980s and it gave many developers an ability to code and build video games.
4 1990s was the golden era for the advancement and application of 3D modelling. This was integrated to home computers and gave a lot of convenience to the developers for computing. Earlier they were dependent on workstations which was a very costly mode for computing. With the help of advancement in hardware, many theoretical computer graphics concepts were now possible to be implemented in real time which leads to the development of many hitech software by Microsoft and Apple. In 1999, Nvedia Graphics released their first video card for home computers named GeForce 256 which has the ability to perform high graphics operations for smooth experience. Closer to the end of this decade many frameworks like DirectX were adapted for graphics processing. By the use of computer graphics technology, many film were produced, for ex Terminator, Star War, Jurassic Park, and toy story. industry and academia. Use of 3D throughout the 90s was highly adapted. Even till now it is used widely in most of the areas. [9,10,11] 3.Non-photorealistic rendering In the past five decades Computer Graphics have focused on photorealism. The basic factor of judging a picture generated by a computer is that how meticulously it represents the original photograph. NPR (Non-Photorealistic Rendering) is a perfect blend of technology and imagination. It is an area in computer graphics which focuses on communicating via art inspired by paintings, technical illustration, cartoons. The useful examples of non-photorealistic rendering are animated cartoons videos, animated GIF(Graphics Interchange Format) images and many more. We can define Nonphotorealistic rendering as a process of generating an image from a model, by means of computer programs-wikipedia. NPR is used all over the world to exchange ideas. For example, consider an X-ray of an injured knee and an elaborated picture generated by computer. Out of the two illustrations, for a layman which one is easy to get the information about the injury from? Obviously the second one, as it provides a embellished view as comparative to the X-ray image. There are three main categories in NPR which are important for research. [1,2,3] Figure 2: Tremendous use of graphics technology in Jurassic Park in 1993. Till today there has been continuous advancement in computer graphics because of the tremendous advancement in the hardware technology which has made many implementations possible which were just on papers earlier. One more important platform OpenGL was introduced in 1992. It is a API for 3D graphics cross language programing from different type of OS. This platform was highly appreciated and adapted by many User assisted image creation The idea is to guide a user operating software to assist in creating an image. Automatic Image Creation- The goal of this is to create an image automatically with previously defined goals. Artistic Media Simulation- The idea of this research is to work on physical properties of an artistic medium. For example, watercolor, pen and ink.
5 applied on the surface of the input. [1,2,3,8,16,17] 3.3 Basic techniques of nonphotorealistic rendering Stylization Abstraction Outlines and Silhouettes extraction 3.4 Emergence of Non-photorealistic rendering Figure 3: Results after applying NPR 3.1 Some basic NPR categories Pen and Ink Illustration. Techniques-Cross-Hatching, outlines, line art, etc. Painterly rendering Styles- Styles: impressionist, expressionist, pointilist, etc. Cartoons Effects: cartoon, shading, distortion, etc. Technical illustrations Characteristics: Matte shading, edge lines, etc. Scientific visualization Methods: splatting, hedgehogs, etc.[6] 3.2 2D and 3D Rendering. Non-photorealistic rendering is a result of overcoming the shortcoming of photo rendering methods. Considering to the practicality of photorealistic rendering and non-photorealistic rendering, photorealistic rendering comprises of only one style and non-photo realistic rendering comprises of many style which may be more suitable for art representation. [6] Figure 4: Flow chart depicting the emergence of NPR [6] 3.5 Basic Rendering Flow chart The 2D NPR systems can be referred to an image representing an art communicating ideas. For example, image of furniture, kitchen etc. There are some 2D NPR systems which takes 3D geometry information as input and gives a 2D image. Whereas, in 3D NPR systems two techniques are common i.e. cel shading and Gooch Shading. They are used for graphic representation in videos and video games. The input to a 3D NPR system can be a 3D geometry information which produces a 3D output having the same geometry information but modification in the material This approach is followed in virtual reality now days(vr). The approach is to sequentially perform two drawing operations in order to present the image to VR equipment. The requirement of a VR is 90 Frames/Sec, this means it requires a huge amount of geometry computation for the two viewpoints. The below flow chart demonstrates the process of rendering and presenting in both the viewpoints. It explains how multiple views are rendered in order to provide a realistic experience in a virtual way.[12]
6 lines with thinner interior lines are used which aid perception of spaces. Shading There are three modes of shading. Diffuse shading, metallic shading, adaptation of cool to warm shading. Any of these modes are used when shading is rendered. Shadowing Here shadowing includes drop shadow on a ground. It provides visual clues without missing any important details on the object. Dynamic illustration principles: When viewer s position changes as the object moves, the presentation won t be same as balanced line weight design. The light effect changes relative to the object. Figure 5: A traditional Stereo rendering process in VR [12] 4. Some basic principles that need to be followed when creating the look of a technical illustration Viewer Versus Object motion When an object is moved keeping the camera and light positions constant provides a detailed information about the object. Material Properties When the view direction or lighting changes, most of the material properties are similar. When metal texture is used, the environment looks interactive and convey the proper material property. To produce a high-quality design model, there is need to develop graphics keeping in mind what visually works and what distracts from the original and important content. The technical illustrations are used to make the design look interactive. This can be made by Static and Dynamic illustrations. Static illustration principles: The technical illustrations which are human drawn are mostly standalone pictures from a specific viewpoint. Lines in Technical illustrations - Here, illustrators usually use edge lines, surface boundaries, silhouettes, discontinuities. The edge lines are represented with black lines of uniform weight. The bold external 4.1 Other basic principles are: Alignment it provides a shape, ordered appearance by giving proper connection with each other. It cleans the design and removes the messiness that occur when some components are randomly placed in the design. Hierarchy When different elements are in a design, priority must be given to the important message. It is done by using different font sizes, using shapes. Contrast It attracts the viewer s eye and let them know what the important message in the design is, it also helps to organize the information in a better way.
7 Proximity It helps to group certain elements which are related to each other and cluster them. They are connected visually by using same font, size, color, type. Balance It helps to distribute the elements all over the design by evenly spacing which gives a balanced and stable look. It removes the messiness and make the appearance professional and attractive. Color The mood of the design an be depicted from the color. It plays a significant role, the color combinations should be attractive and pleasant. Using different textures will also add great effect to the words written over it. Space The part that is well designed with colors, images, text is important as leaving some place between the contents of the data in the design. The topics will be highlighted by adding required spaces, but spaces in the odd places create negative impact. [13,14,15] shades to create a flat look. Here the shadows and highlights appear pleasant instead of looking mixed. Painterly rendering: It is used for animations. It is used for creating static images. Computer rendering gives easy procedure to render with proper details. The animating geometry and characteristics of oil painting are taken as advantage in computer rendering which gives us a fine picture with great results. Line integral convolution: This method is used for developed for imaging vector fields. It has the potential to get images with directional characteristics. This is majorly used in scientific environments i.e., aesthetic virtual environments. 5. An overview of the NPR techniques in use today. In the initial days of developing computer graphics, replacing a person was never the goal, but through these techniques the things that don t physically exist are created. These days NPR techniques are often used to create photorealistic images that there are indistinguishable from the real ones. The best part is, it is not realized that computer graphics are used in certain times though they were used with great effects. Figure 6: Figure showing implementation of line integral convolution.[ 24] Cross hatching: This method is still used to create some comic images and cartoons. It is drawing to create a mesh pattern by hatching at right angles. For a pattern, if object has different texture, multiple layers are used. The modern NPR technique is Cel shading or toon shading used to make 3D computer graphics. This is different from conventional rendering, illumination model. The term cel shading is used to refer to the outlining process of the ink in animations and games. In the conventional model, the values of each lighting are calculated for each pixel. They are quantized to a smaller number of discrete. Figure 7: Figure showing implementation of cross hatching.[23]
8 Silhouettes and Outlines This is a 3D NPR technique, here only the edges are detected. The edges of the images are outlined and given as rough strokes. This type is used to image space methods. [3,19,20,21] Figure 8: Figure showing implementation of outlines and silhouettes. [22] 6. Photorealism Vs Non-Photorealism To better understand the difference between these two terms, it is important to understand how they work. To explore this, lets focus on the platform of Augmented Reality and how it uses photorealism and non-photorealism. Photorealism means making a scene more realistic in AR. So to achieve this step, the primary step to follow is Shading. Shading: To create a realistic scene, the primary effect which need to be implemented is shading. Most of the time when flat shading is applied it does not gives a realistic look but it can be used to enhance the view of augmented objects. The main thing that concerns us is that we should have a light source and shading implemented. Figure 9: Figure showing implementation of shadow technique and the result are more realistic. Shadow of an objectives is achieved by shading the area casted by the object on the other object with a black-transparent color. Bump Mapping This method also brings ups the reality on the rendered output my stimulating the wrinkles on the surface. The interesting part is that in achieving this we do not have to modify the geometric specifications of the model. Techniques like vertex shading or fragment shading is used. Non-Photorealism in AR In AR, NPR is used whenever we present a detail which is difficult to present otherwise. For example, If we have to show a all the details of a car engine, we use NPR to focus and represent all the details, and the optimal solution to show most of the details is to use abstraction. In this paper we will also cover some more techniques of NPR. Toon Shading Shadows: The idea behind this technique is to present The main motive of implementing shadows augmented data in a way that it does not to add realism to a rendered output and completely looks realistic. We detect the determine the special relationships. Its is outline of the object in the consideration and very difficult to achieve a perfect merge of fill the area enclosed by those lines with shading and shadows, but when done a sharp colors. Generally, to enhance the smooth merging of real and virtual world is performance pixel and vertex shaders are achieved. used. By implementing toon shading, a
9 rendered image can be represented in 3D and it can also change the rendering slide. Figure 11: Changes brought to a painterly rendering algorithm.[25] Painterly Rendering As mentioned above that change in the style of rendering is possible to achieve. As simple running painterly rendering algorithm when changed along with the usage or modern 3D hardware can give some astonishing results. After bring the changes the algorithm works as follow. 1. A preprocessing step is required before we start the rendering. Conversion of polygon meshes into 3D particles is a major step, and simultaneously computing the constant attributes of the brush strokes. 2. The whole process of rendering is done in two passes. The first pass is, minimum of two images that contains the color and depth information is rendered by using a arbitrary algorithm. 3. The second and last pass is when the brush strokes are rendered with the help of billboards. The output of the first pass is used to get the information about color depth and visibility. This pass needs a high computation and cost. [18] Figure 11: Changes brought to a painterly rendering algorithm.[18]
10 Refrences 1.https://en.wikipedia.org/wiki/Non-photorealistic_rendering 2.https://www.techopedia.com/definition/31617/non-photorealistic-rendering-npr 3.https://www.cl.cam.ac.uk/teaching/1011/RSL/Richardt.pdf 4.https://www2.cs.duke.edu/courses/spring04/cps124/notes/13_npr/index.html 5.https://www.researchgate.net/publication/236973460_Non-Photorealistic_Rendering 6.https://www2.cs.duke.edu/courses/compsci344/spring15/classwork/16_npr/cmu.pdf 7.http://dl.lcg.ufrj.br/cg2/downloads/nonphotorealistic%20rendering/Intro%20to%203D%20NPR %20Silhouettes%20and%20Outlines.pdf 8.http://eprints.mdx.ac.uk/6723/1/Schofield.phd.pdf 9.https://en.wikipedia.org/wiki/Computer_graphics#1970s 10.https://pdfs.semanticscholar.org/presentation/c723/94f0ca1cff9d5526f490813ce2bc4869641 6.pdf 11.https://deseng.ryerson.ca/dokuwiki/mec222:brief_history_of_computer_graphics 12.https://devblogs.nvidia.com/turing-multi-view-rendering-vrworks/ 13. Gooch, Bruce, et al. "Interactive technical illustration." Proceedings of the 1999 symposium on Interactive 3D graphics. ACM, 1999. 14. https://maddisondesigns.com/2009/03/the-5-basic-principles-of-design/ 16. Gooch, Bruce, and Amy Gooch. Non-photorealistic rendering. AK Peters/CRC Press, 2001. 17. Hearn, Donald D., M. Pauline Baker, and Warren Carithers. Computer graphics with open GL. Prentice Hall Press, 2010. 18. Haller, Michael. "Photorealism or/and non-photorealism in augmented reality." Proceedings of the 2004 ACM SIGGRAPH international conference on Virtual Reality continuum and its applications in industry. ACM, 2004. 19. Sayeed, Rezwan, and Toby Howard. "State of the Art Non-Photorealistic Rendering (NPR) Techniques." TPCG. 2006. 20. Meier, Barbara J. "Painterly rendering for animation." Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. ACM, 1996. 21. Interrante, Victoria. "NPR Techniques for Scientific Visualization." 15.https://blog.adobespark.com/2016/07/27/8-basic-design-principles-to-help-you-create-bettergraphics/ 22.http://www.vitaminjdesign.com/12429/palm-tree-outlines-25-05-2018/sturdy-palm-treeoutlines-tropical-trees-black-silhouettes-outline-stock-vector-373790335/
11 23.https://www.google.com/search?biw=1242&bih=587&tbm=isch&sa=1&ei=QPS8W9CUF4izjw SDsYXgBA&q=Cross+hatching+&oq=Cross+hatching+&gs_l=img.3..0l10.30218.30218..30662...0.0..0.79.79.1...1...1j2..gws-wiz-img.M3f89RoU5Ic#imgrc=uD-vz8xTwm0YzM: 24. https://www.researchgate.net/figure/line-integral-convolution-lic_fig1_254005603 25. https://en.wikipedia.org/wiki/cel_shading
12 BACKGROUND Early graphics systems produced images on such devices as printers using only dots or other special characters. As time went on, emphasis was placed on realism, i.e. the image generated on a graphics device should not be different from that of a photograph. Since research goes around in circles, current efforts focus on generating pictures that are not realistic, i.e. developing tools for the artists to express themselves using graphics technology. Hence non-photorealistic rendering was born. OBJECTIVES In this project you will learn about techniques that are used to create pictures that are the result of imagination and not that of copying the real world. These tools allow the user to render images with added characteristics, such as graphic pen, blurring or sharpening. QUESTIONS One of the most important parts of learning is to ask questions. In the first part of the team essay project your team will assemble a set of five (5) questions that broadly cover the topic of nonphotorealistic rendering. The questions should be designed so that if properly answered, the team will learn everything about non-photorealistic rendering that is relevant in the graphics practice. Your five questions will be given to another team to be answered, and your team will receive five questions of another team. Please list your questions here: Q1: How do you define non-photorealistic rendering? Q2: What is the difference between photorealism and non-photorealism? Q3: What is the history of computer graphics: from the 1970s to 1990s? Q4: Provide an overview of the NPR techniques in use today?
13 Q5: What are some of the principles that need to be followed when creating the look of a technical illustration? ANSWERS You have to answer the five questions in the form of a complete essay. That is, the essay cannot be in a Q&A format! It has to be a standalone, full paper in which the answers are woven seamlessly into the text. PROJECT DESCRIPTION During this project your team will perform the following simple tasks: Understand the concept of texture, bump mapping, etc, as presented in the book. Do a search on topics like texturing, surface material, simulating the brush, graphics pen, etc. Do a search on non-photorealistic rendering and dig out all useful and understandable information. Consider a few important applications: computer art, document processing, printing, promotional materials, presentation materials, etc. Do a search on existing systems using non-photorealistic techniques. Find methods for creating special effects during rendering. Find publicly available code and/or demo programs related to non-photorealistic rendering. Once the research is done, digest all the information you gathered and write-up your experience with your independent research as described in the next section. DELIVERABLES There are two deliverables: (1) five questions and (2) the essay. Please use this template to enter the five questions above and return the template by September 11, 2018, 5:00 PM. Using this file as a template, write an essay that contains, as a minimum, the following components: A short abstract of no more than 200 words. A list of keywords of no more than 5. An introduction section where you describe the problem domain and give a short description of prior approaches as far as you know. A section on general non-photorealistic methods. A section on special techniques, used in available systems, which describes the result of your independent research.
14 A section on algorithm design where you show a code architecture and/or flow chart to design an algorithm for various forms of rendering. Proper illustrations throughout the text. A section on examples and demos, if any. A section on experiments, if available or if you have done one or more yourself. FORMAT Use this MS Word file as the template and enter your text directly after the team members contributions. Please name the file as in the following example: Team_Amanda_Essay.doc When submitting the file, the subject line should contain the course prefix as well as the file name above. Example: EEL_5771_ Team_Amanda_Essay.doc SUBMISSION Your essay should be submitted electronically via e-mail attachment. Please send the file to lpiegl@gmail.com no later than: Questions: September 11, 2018, 5:00 PM Essay: October 9, 2018, 5:00 PM