What is Computer Graphics? (2/2) What is Computer Graphics? (1/2) What is Interactive Computer Graphics? (1/3)

Transcription

1 What is Computer Graphics? (1/2) Computer graphics is commonly understood to mean the creation, storage and manipulation of models and images. Such models come from a diverse and expanding set of fields including physical, mathematical, artistic, biological, and even conceptual (abstract) structures. What is Computer Graphics? (2/2) The term computer graphics was coined in 1960 by William Fetter to describe new design methods he was pursuing at Boeing. He created a series of widely reproduced images on a plotter exploring cockpit design using a 3D model of a human body. Frame from an animation by William Latham, shown at SIGGRAPH Latham uses rules that govern growth patterns of natural forms to create his artwork. Perhaps the best way to define computer graphics is to find out what it is not. It is not a machine. It is not a computer, nor a group of computer programs. It is not the know-how of a graphic designer, a programmer, a writer, a motion picture specialist, or a reproduction specialist. Computer graphics is all these -- a consciously managed and documented technology directed toward communicating information accurately and descriptively. Computer Graphics, by William A. Fetter, 1966 Andries van Dam September 10,1998 Introduction 1/57 Andries van Dam September 10,1998 Introduction 2/57 What is Interactive Computer Graphics? (1/3) User controls contents, structure, and appearance of objects and their displayed images via rapid visual feedback Basic components of an interactive graphics system input (e.g., mouse, tablet and stylus, force feedback device, scanner...) processing (and storage) display/output (e.g., screen, paper-based printer, video recorder...) First truly interactive graphics system, Sketchpad, pioneered at MIT by Ivan Sutherland for his 1963 Ph.D. thesis. What is Interactive Computer Graphics? (2/3) Before Sketchpad, output via plotters/ printers, input via keypunch, both in batch Batch (1950s - now) Card punching (left). The IBM 704 (right) took up a whole room and had less computing power than a Mac Plus. Cool facts: Whirlwind, built in the early 50 s at MIT, cost $4.5 million and could perform 40,000 additions/second. Mac 512K, list price $3,195 in 1984, could do 500,000. Today, high-end processors approaching one billion operations/second (peak). Sketchpad in Note the use of a CRT monitor, light pen and function-key panel. Andries van Dam September 10,1998 Introduction 3/57 Andries van Dam September 10,1998 Introduction 4/57

2 What is Interactive Computer Graphics? (3/3) Almost all the key elements of an interactive graphics system are expressed in the first paragraph of Sutherland s thesis, entitled, Sketchpad, A Man-Machine Graphical Communication System: The Sketchpad system uses drawing as a novel communication medium for a computer. The system contains input, output, and computation programs which enable it to interpret information drawn directly on a computer display. Sketchpad has shown the most usefulness as an aid to the understanding of processes, such as the motion of linkages, which can be described with pictures. Sketchpad also makes it easy to draw highly repetitive or highly accurate drawings and to change drawings previously drawn with it... (1963) Today, still use batch mode for final production-quality video and film Environmental (R)evolution (1/6) Graphics in Context Graphics has been a key enabling technology in the evolution of computing environments: graphical user interfaces visual computing, e.g., desktop publishing, scientific visualization, information visualization Hardware revolution drives everything every 18 months, computer power improves by factor of 2 in price/performance - Moore s Law -wristwatch computer -Hallmark singing card Andries van Dam September 10,1998 Introduction 5/57 Andries van Dam September 10,1998 Introduction 6/57 Environment (R)evolution (2/6) Character Displays (1960s - now) Display: text plus alphamosaic pseudographics Object and command specification: command line typing Control over appearance: coding for text formatting (.p = paragraph,.i 5 = indent 5) Application control: single task Environment (R)evolution (3/6) Vector Displays ( s) Display: line drawings and stroke text; 2D and 3D transformation hardware Object and command specification: command-line typing, function keys, menus Control over appearance: pseudo WSIWG Application control: single or multitasked, host-satellite distributed computing Andries van Dam September 10,1998 Introduction 7/57 Andries van Dam September 10,1998 Introduction 8/57

3 Environment (R)evolution (4/6) 2D bitmap raster displays for PCs and workstations (1972 at erox PARC- now) Display: windows, icons, legible text and flat earth graphics. Note: late 60 s saw first use of raster graphics, especially for flight simulators. Object and command specification: minimal typing via WIMP (Windows, Icons, Menus, Pointing) GUI (Graphical User Interface): point-and-click selection of menu items and objects, widgets and direct manipulation (e.g., drag and drop), the messy desktop metaphor Control over appearance: WSIWG (and WSIAG) Application control: multitasking, networked client-server computation and window management (even terminals) Environment (R)evolution (5/6) 3D graphics workstations (1984 at SGI - now) Display: real-time, pseudo-realistic images of 3D scenes Object and command specification: 2D, 3D and nd input devices (controlling 3+ degrees of freedom) and force feedback haptic devices for point-and-click, widgets, and direct manipulation Control over appearance: WSIWG (and WSIAG) Application control: multi-tasking, networked (client/server) computation and window management Andries van Dam September 10,1998 Introduction 9/57 Andries van Dam September 10,1998 Introduction 10/57 Environment (R)evolution (6/6) Classical time-sharing is dead 1:n n:1 PCs and Workstations merging in distributed heterogeneous computer networks (e.g., LANs, WANs, Internet) But file-, print- and compute-servers and network are shared Client/server computing, component software technologies are dominant paradigms NC s (Network Computers), thin clients attached to powerful servers reprise dumb terminals and provide central control New Forms of Computing: 1990s - (1/2) Multimedia: text and graphics synchronized with sound and video Hypermedia: multimedia with links (also called Interactive Multimedia) Digital Convergence : merging of digital television and distributed computing, consumer electronics: set-top computers (e.g., for Interactive TV, Video-On-Demand), telecomputers - flash in the pan The Internet and Internet appliances Embedded computing (information appliances, Personal Digital Assistants) Ubiquitous computing, active badges à la erox PARC Virtual Reality: desktop ( fishtank ) VR, immersive VR (via Head-mounted Displays, Cave, Responsive Workbench), augmented VR (via video see-through optics) Andries van Dam September 10,1998 Introduction 11/57 Andries van Dam September 10,1998 Introduction 12/57

4 New Forms of Computing: 1990s - (2/2) New Interaction technology Inexpensive interaction devices from research lab into marketplace makes 2D and 3D graphics no longer special 3D (even time-varying, 4D ) interactive illustrations as clip art/clip models coming soon Kids using computer graphics with joysticks, game controller; VR games and rides with HMD and force-feedback input devices New forms of user-interface 3D Widgets, gesture-based ( Sketch ); VR demands new interaction technology Social interfaces (Microsoft s Bob bombed) Powerful, Inexpensive Processing Chips are key in graphics subsystems Advances driven by Moore s Law price/performance improves 2x every 18 months due to doubling of number of devices only exponential in technology except for growth of WWW (and this one will die out first) CPU MIPS Technologies MIPS Sun UltraSPARC IBM/Apple/Motorola Power PC Intel Pentium Intel/Compaq Alpha Graphics subsystems SGI, Sun, HP, Evans & Sutherland chip vendors for games and set-top boxes and for PC boards Sun s VIS and Intel s MM/KNI instructions add multimedia support to main processor; in the future 3D graphics in every chip Agents/knowbots for indirect control Andries van Dam September 10,1998 Introduction 13/57 Andries van Dam September 10,1998 Introduction 14/57 Use of Chip Technology Advances in Games & Commodity Graphics Cards New game platforms and set-top boxes use high-end processors (64-bit architectures, great graphics capabilities) Sony Playstation SGI/Nintendo 64 3dfx Voodoo2 nvidia TNT processor Significant advances in commodity graphics chips every 6 months, outrunning CPU chip advances Seriously threatening high-end workstations Application Distinctions Two basic paradigms Sampled-based graphics: discrete samples are used to describe visual information pixels can be created by digitizing images, using a sample-based painting program, etc. often some aspect of the physical world is sampled for visualization, e.g., temperature across the US example programs: Adobe Photoshop, The GIMP (on UNI) Geometry-based graphics: a geometrical model is created, along with various attributes, and is then sampled for visualization (this process is called rendering) often some aspect of the physical world is visually simulated, or synthesized example programs: Adobe Illustrator, Claris s MacDraw, Autodesk s AutoCAD and Studio 3D, Alias Wavefront Studio and Maya, SoftImage 3D Andries van Dam September 10,1998 Introduction 15/57 Andries van Dam September 10,1998 Introduction 16/57

5 Sampled-based Graphics (1/2) Images are made up of a grid of discrete pixels for 2D picture elements Mathematical pixel grid CRT beam illumination pattern LCD display light intensity 1 pixel NB: Can t resolve adjacent pixels on CRT Pixels are point locations with associated sample values, usually of light intensities/ colors. Sample-based Graphics (2/2) Samples can be created directly in a paint-type program, or as result of sampling continuous (analog) visual materials. E.g, a continuous image, such as a photograph, can be sampled (light intensity/color measured at regular intervals) with many devices including: flatbed and drum scanners digital still and motion (video) cameras add-on boards such as frame grabbers Sample values can also be input numerically (e.g., with numbers from a computed dataset) Once an image is defined as a pixel-array, it can be manipulated image editing describes changes made by a user to an image, such as cutting and pasting sections, using brush-type tools, and processing selected areas. image processing describes algorithmic operations that are performed on an image (or preselected portion of an image) without user intervention. These include such operations as blurring, sharpening, edge-detection, color balancing, rotating and warping. Andries van Dam September 10,1998 Introduction 17/57 Andries van Dam September 10,1998 Introduction 18/57 Sampling an Image Lets do some sampling of the CIT building The 3D scene What s the Advantage? Once the image is defined in terms of colors at (x, y) locations on the grid, changes to the image can easily be made by altering location or color values E.g., if we reverse our mapping above and make 10 = white and 0 = black, the image would look like this: A color value is measured at every grid point and used to color a corresponding grid square 0 = white 5 = gray 10 = black Pixel information from one image can be copied and pasted into another. Note: this poor sampling and image reconstruction method creates a blocky image Andries van Dam September 10,1998 Introduction 19/57 Andries van Dam September 10,1998 Introduction 20/57

6 What s the Disadvantage WSIWG (What ou See Is What ou Get): There is no additional information no depth information can t examine scene from a different point of view at most can play with the individual pixels or groups of pixels to change colors, enchance contrast, find edges, etc. Examples of 2D Image Manipulation (1/3) There are many things one can do with sampled images: uses of computer imaging range from military applications to entertainment, medicine, art, and design. The news: digitally enhanced images are more and more common. Usually just sharpening, color balancing, but sometimes a lot more National Geographic s Pyramid cover TV Guide s Oprah cover Time Magazine s O.J. Simpson cover Andries van Dam September 10,1998 Introduction 21/57 Andries van Dam September 10,1998 Introduction 22/57 Examples (2/3) Movies: People can now mutate into other people or objects through morphing, and can carry on conversations in different times and places Michael Jackson s Black or White music video Forrest Gump, Jurassic Park, Twister, etc. [video clip] Examples (3/3) In artwork the processes and techniques of photography and painting are merging in the art of digital imaging Michele Turre: the artist, her daughter, and her mother, all at 3 years of age The belief in a very strong connection between photorealistic images, still or moving, and reality is being severed no way to tell if news photos are real photographs photographic evidence no longer considered proof in a court of law without clear provenance of the image future of the family photo album? This course emphasizes geometry-based graphics but we do one assignment on samplebased graphics to learn simple image processing. image processing in general includes image transformation, feature detection, pattern recognition, machine vision Andries van Dam September 10,1998 Introduction 23/57 Andries van Dam September 10,1998 Introduction 24/57

7 Geometry-Based Graphics Geometry-based graphics applications store mathematical descriptions, or models, of geometric elements (lines, polygons, polyhedrons...) and their associated attributes (e.g., color, material properties). These elements are primitive geometric shapes, primitives for short. Images are created as pixel arrays (via sampling of the geometry) for viewing, but are not stored as part of the model. Images of many different views can be generated from the same model. One cannot usually work directly with the individual pixels in a geometry-based program; one manipulates the geometric elements, then they are resampled and redisplayed. What is Geometric Modeling? What is a model? Captures the salient features (data, behavior) of the thing/phenomenon being modeled -data includes geometry, appearance attributes... -note similarity to OOP notions Abstract: no inherent geometry, but for visualization organizational (e.g., company org. chart) quantitative (e.g., graph of stock market data) Real: some geometry inherent physical (e.g., actual object such as a pump) non-physical (e.g., mathematical function, weather data) Modeling is coping with complexity Our focus: modeling and viewing simple everyday objects Consider this: Through 3D computer graphics, for the first time in human history we have abstract, easily changeable 3D forms. This has revolutionized the working process of many fields -- science, engineering, industrial design, architecture, commerce, entertainment, etc. Andries van Dam September 10,1998 Introduction 25/57 Andries van Dam September 10,1998 Introduction 26/57 Lecture Topics Primitive shapes are manipulated with geometric transformations (translation, rotation, scale). These transformations are essential for model organization, the process of composing complex objects from simpler components. Hierarchical models and the same geometric transformations are also essential for animation. Once the objects geometry is established, they must be viewed on a screen. But viewing 3D object descriptions on a 2D screen presents a challenge: we must map from 3D to 2D for viewing and from 2D to 3D for 2D input devices (e.g., the mouse). In the process of mapping from 3D to 2D, object (surface) material properties and lighting effects are used in rendering one s constructions. The rendering process is also called image synthesis. Model Layout and Component Relationships (1/2) Spatial Layout of Sparc10 (Ultra is similar) VRAM Sun G Display processor LSI Logic S-bus monitor P-bus LSI Logic Super Sparc Microprocessor TM5390 Cache Controller Andries van Dam September 10,1998 Introduction 27/57 Andries van Dam September 10,1998 Introduction 28/57

8 Model Layout and Component Relationships (2/2) Connectivity and composition of components of a Sparc10 (DAG) floppy drive system I/O bus SS10 CPU hard memory bus drive RAM video subsystem keyboard serial port mouse Application-specific data, also called manufacturer s data (from parts catalogue) -capacities (feeds and speeds) and performance data thermal characteristics... manufacturing process information Information for Image Synthesis lighting surface properties: reflectance and luminosity of surfaces Decomposition of a Geometric Model Divide and Conquer Hierarchy of geometrical components Reduction to primitives (e.g., spheres, cubes, etc.) Simple vs. not-so-simple elements (nail vs. screw) Head Shaft Point composition decomposition Andries van Dam September 10,1998 Introduction 29/57 Andries van Dam September 10,1998 Introduction 30/57 Hierarchical (Tree) Diagram of Nail The object to be modeled is (visually) analyzed, and then decomposed into collections of primitive shapes. The tree diagram provides a visual method of expressing the composed of relationships of the model. Composition of a Geometric Model Translate Translate and Scale Nail root node Head (cylinder) Body Translate and Rotate Shaft (cylinder) Point (cone) leaf nodes Primitives Composition tree diagram Such diagrams are sometimes part of 3D program interfaces (high-end, e.g., Alias Wavefront Studio). Andries van Dam September 10,1998 Introduction 31/57 The primitives created in the decomposition process must be assembled to create the final object. This is done with affine transformations, T, R, S (as in the example above). Other composition operators will be discussed later (e.g., Constructive Solid Geometry (CSG) with Boolean operators). Andries van Dam September 10,1998 Introduction 32/57

9 What Kind of Math do We Need? Cartesian Coordinates Typically modeling space is floating point, screen space is integer Primitives in 2D and 3D (1/4) Lines (x 1, y 1 ) (x 1, y 1, z 1 ) (x o, y o ) (x o, y o, z o ) R point at (2, 3) pixel at (2.1, 3.1) Polylines (x, y) (x, y, z) x, y Cartesian grid Integer grid NB: Often, screen coordinates are measured top to bottom (0, 0) Andries van Dam September 10,1998 Introduction 33/57 Andries van Dam September 10,1998 Introduction 34/57 Primitives in 2D and 3D (2/4) Rectangles (x, y, z) (x, y) Primitives in 2D and 3D (3/4) Circles Spheres (x, y) (x, y, z) (x, y, z) Cubes (cuboids) Polygons (x, y) Polyhedra (x, y, z) Cylinders (x, y, z) Cones (x, y, z) Andries van Dam September 10,1998 Introduction 35/57 Andries van Dam September 10,1998 Introduction 36/57

10 Primitives in 2D and 3D (4/4) Curves Patches (x, y) (x, y, z) Conceptual Framework for Interactive Graphics Graphics library/package (e.g., GP, OpenGl) is intermediary between application and the display hardware Application program maps application objects to views (images) of those objects by calling on graphics library User interaction results in modification of model and/or image Splines are parametric polynomials, not functions e.g., Bezier curves used for objects, character outlines allows description of intersecting curves and surfaces, infinite slopes Images are usually means to an end: synthesis, design, manufacturing, visualization,... This framework is 3 decades old but is still useful, indeed dominant xt () = a x t 3 + b x t 2 + c x t + d x Application model Application program Graphics system yt () = a y t 3 + b y t 2 + c y t + d y zt () = a z t 3 + b z t 2 + c z t + d z Andries van Dam September 10,1998 Introduction 37/57 Andries van Dam September 10,1998 Introduction 38/57 Graphics Library Primitives Attributes color line style... material properties for 3D Lights Transformations Immediate mode vs. retained mode immediate mode: no stored representation, package holds only attribute state, and application must completely draw each frame retained mode: library compiles and displays from a scene graph Sample- vs. Geometry-Based (1/2) Flat Earth bitmap painting - sample-based diddle pixels in integer device coordinates text, paint, 2D shapes,... RasterOp/BitBlt QuickDraw, lib, Motif, AWT, Win32 GDI 2D and 3D drawing and design (e.g. CAD) - geometric model-based editable objects in floating point World Coordinates no pixels to diddle hi-level logical input devices; event queues, time stamps Vector graphics libraries (obsolete) -Plot 10, Core, GKS Raster graphics libraries -Display PostScript (2D, Adobe) -OpenGL (SGI, OpenGL consortium) -DirectDraw, Direct3D (MS) -Fahrenheit (MS/SGI/HP) -Java 3D (Sun) Andries van Dam September 10,1998 Introduction 39/57 Andries van Dam September 10,1998 Introduction 40/57

11 Sample- vs. Geometry-Based (2/2) Higher level libraries OpenInventor (SGI) Performer (SGI) Fahrenheit Scene Graph (MS/SGI/HP) RenderMan (Pixar) Alias/Wavefront, SoftImage APIs (NB: APP API!) Window Systems reconcile objects and pixels (graphics library + tools/widgets) Mac OS, Windows 98, Windows System Interchange standards CGM, IGES VRML 2D Hardware/Algorithms Outline Display Hardware: Raster scan vs. random scan Drawing primitives by scan conversion lines polygons circles and ellipses characters attributes (color, line style, fill pattern...) Clipping to clip rectangle: three methods analytically compute intersections and draw clipped primitives test each pixel and write only if inside compute spans inside the primitive and fill entire span without testing BitBlt/RasterOp for operating on blocks of pixels Color table indirect specification of (pseudo) color color correction, simple types of animation Andries van Dam September 10,1998 Introduction 41/57 Andries van Dam September 10,1998 Introduction 42/57 Graphics Display Hardware Vector (calligraphic, stroke, random-scan) still used in plotters Vector Architecture Interface to Host Computer Display Commands Interaction Data Ideal Drawing Vector Drawing Raster (TV, bitmap, pixmap), used in displays and laser printers Raster MOVE LINE CHAR Lu cy LINE JMP Display Controller (DC) Lucy outline primitives filled primitives Refresh Buffer Vector display works with a display list/file stored in refresh buffer Display controller draws all vectors at 60 Hz (often at variable rate); flicker is a problem Andries van Dam September 10,1998 Introduction 43/57 Andries van Dam September 10,1998 Introduction 44/57

12 2D Raster Architecture Interface to Host Computer Display Commands Interaction Data Drawing Lines Ideal line Display Controller (DC) serial port, keyboard, mouse Refresh Buffer Video Controller For horizontal, vertical and diagonal lines all pixels lie on the ideal line: special case For lines at arbitrary angle, pick pixels closest to the ideal line (Bresenham s = midpoint scan conversion algorithm) For thick lines, use multiple pixels in each column or fill a rotated rectangle Raster display stores bitmap/pixmap in refresh buffer, also known as bitmap, frame buffer; can be in separate hardware (VRAM) or in CPU s main memory (DRAM) Video controller draws all scan-lines at consistent 60 Hz; separates update rate and refresh rate Andries van Dam September 10,1998 Introduction 45/57 Sampling a continuous line on a discrete grid introduces sampling errors: the jaggies Andries van Dam September 10,1998 Introduction 46/57 Drawing Filled Polygons Drawing Circles and Ellipses 1. Find intersection of scanline with polygon edges 2. Sort intersections by increasing x 3. Fill the polygon between pairs of intersections(spans) Scan line circle outline filled ellipse Andries van Dam September 10,1998 Introduction 47/57 Andries van Dam September 10,1998 Introduction 48/57

13 Thickness Attribute Thick circle drawn by tracing a rectangular pen Drawing Characters For characters defined by small bitmap selectively write pixels of refresh buffer corresponding to true bits of character bitmap transparent mode: don t write 0 s BitBlt with OR opaque mode: write background color for 0 s Descenders & proportional spacing easily accommodated Base line Treat such primitives as regions and fill them, e.g., approximate them as sequences of connected quadrilaterals Andries van Dam September 10,1998 Introduction 49/57 Andries van Dam September 10,1998 Introduction 50/57 Drawing Characters (cont.) Outline fonts: defined in terms of (mathematical) drawing primitives (lines, arcs, splines) and thus scalable, but more CPU intensive (e.g. PostScript, TrueType) 1-bit Bilevel Display N 1 video controller CRT M frame buffer Digital intensity value Digital to Analog Conversion (DAC) analog signal to drive electron beam Black & white (or any 2 colors, depending on the monitor phosphor color) Original Mac resolution was 512 x 384 pixels, now from 640x480 up to 1280x1024, even 3Kx3K. Our Suns are 1280x x1024 memory - 15 nanosec/pixel => 32 pixel chunks Frame buffer Address Address Data address address Set or increment Set or decrement Raster scan generator Pixel value DAC Video Controller Deflection Intensity Andries van Dam September 10,1998 Introduction 51/57 Andries van Dam September 10,1998 Introduction 52/57

14 n-bit Display Image Display System Look-up Table n N M frame buffer video controller CRT N n M 2 n LOOK- UP TABLE 24 CRT 2 n intensities or colors: 1 (grayscale) or 3 (color) DACs & guns 1000 line color monitors cost < $1,000 video controller Any specific 2 n colors may be inadequate (n typically 8-12 in low-end systems) Look-up table allows 2 n colors out of 2 24 colors to be used in one picture, some other 2 n in another picture 2 24 = apr million, exceeds eye s ability to discriminate (somewhere between 7-10 million) Andries van Dam September 10,1998 Introduction 53/57 Andries van Dam September 10,1998 Introduction 54/57 Color Look-Up Table Operation 7 N-1 0 R G B Pixel in memory 255 y x M-1 8-bit Frame Buffer } Color Table RED GREEN BLUE R G B 3 colors x 4 bits/col = 12 bits y pixel at (x, y) x CRT display BitBlt / RasterOp (1/2) Logically operate on rectangular regions in same or different pixmaps to achieve dynamics, e.g., to move/scroll windows on screen RasterOp (Source, Destination) Destination In some implementations either S or D may be masked, and need not be same size Pixel value is indexed to color look up table (CLUT) where color is stored. Here we use only 12 bits (4bits per color) for clarity - typically, 24bits are used CLUT look up done at video rates, overlapped with fetch and DAC! In 24-bit true color systems, 3 x 8 bits for R, G, B; each color has its own 8-bit CLUT (0-255) CLUT allows variety of effects pseudo coloring (LandSat images, stress diagrams, thermograms...) fast image changes: change table rather than stored image multiple images: select or composite/blend animation hack S D pixmap Andries van Dam September 10,1998 Introduction 55/57 Andries van Dam September 10,1998 Introduction 56/57

15 BitBlt / RasterOp (2/2) Replace (S,D): S destructively replaces D, i.e., is deleted and copied on top of D (also called Move); used for making opaque characters, icons, scroll Copy (S,D) as above, but S is not deleted OR (S,D): S is non-destructively added to D; used for painting, transparent and kerned characters (where characters extend beyond their boxes); not as useful in n-bit systems AND (S,D): S can mask out pixels in D OR (S,D): S selectively inverts D; used in 1-bit systems for rubberbanding/dragging, cheap cursors: S OR (S OR D) = D 0 = W, 1 = B OR = ; S D D' OR = S D' D Andries van Dam September 10,1998 Introduction 57/57