Digital Images Kyungim Baek Department of Information and Computer Sciences ICS 101 (November 1, 2016) Digital Images 1
iclicker Question I know a lot about how digital images are represented, stored, and manipulated in computing devices. A. Strongly agree B. Agree C. Don t agree or disagree D. Disagree E. Strongly disagree ICS 101 (November 1, 2016) Digital Images 2
Outline What is a digital image? Early digital images Digital image representation (Spatial) Resolution Grayscale image representation Color image representation Computer Vision vs. Computer Graphics CV Applications ICS 101 (November 1, 2016) Digital Images 3
What is a digital image? A digital image is a representation of a twodimensional images as a finite set of digital values Figure from B. Mac Namee ICS 101 (November 1, 2016) Digital Images 4
Early Digital Images One of the first applications of digital images was in the newspaper industry in early 1920s The Bartlane cable picture transmission system Pictures were sent by submarine cable between London and New York More than a week less than three hours Pictures were coded for cable transfer and then reproduced at the receiving end on a telegraph printer A digital picture produced in 1921 (McFarlane, 1972) ICS 101 (November 1, 2016) Digital Images 5
Early Digital Images Improvements to the Bartlane system resulted in higher quality images A digital picture produced in 1922 showing improvement both in tonal quality and in resolution A digital picture transmitted in 1929 from London to New York with 15 tones Images from McFarlane, 1972 ICS 101 (November 1, 2016) Digital Images 6
Digital Images The Problem: How to represent a two-dimensional image in digital form ICS 101 (November 1, 2016) Digital Images 7
How is this figure stored? Lines and Arcs Dots ICS 101 (November 1, 2016) Digital Images 8
How is this figure stored? Lines and Arcs Dots ICS 101 (November 1, 2016) Digital Images 9
A Digital View ICS 101 (November 1, 2016) Digital Images 10
Image vs. Data 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 1 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 ICS 101 (November 1, 2016) Digital Images 11
Digital Image Representation Pixel: Each cell or dot (picture element) A digital image is a matrix of numbers called pixel values an approximation of a real scene (digitization) 5 1 pixel 20 80 98 Figures from B. Mac Namee ICS 101 (November 1, 2016) Digital Images 12
Spatial Resolution The number of pixels in width and height is the image s spatial resolution E.g. 1920 1080 = 2,073,600 pixels ( 2.1 Megapixels) Can you claim that an image with a particular spatial resolution has a particular size in the real world? ICS 101 (November 1, 2016) Digital Images 13
Pixel Density Pixel density: Number of pixels per inch (ppi) Connect size (in inches) to spatial resolution Sometimes called dpi (dots per inch) Given an image with a particular size, high pixel density implies high spatial resolution Example: Consider a 12 (width) by 9 (height) picture 1024 768 image: 0.8 Megapixels (spatial resolution) 1024 pixels/12 = 85.3 ppi (width) 768pixels/9 = 85.3 ppi (height) 1920 1080 image: 2.1 Megapixels (spatial resolution) 1920 pixels/12 = 160 ppi (width) 1080 pixels/9 = 120 ppi (height) ICS 101 (November 1, 2016) Digital Images 14
Digital Image Quality The impact of spatial resolution (or pixel density) on the image quality Smaller pixel size (given an image with a particular size), i.e. higher spatial resolution and higher pixel density Sharper lines & more detail High resolution more data to store ICS 101 (November 1, 2016) Digital Images 15
Comparing Pixel Density ICS 101 (November 1, 2016) Digital Images 16
Important Things to Learn The resulting image will never be a complete or exact reproduction of the original The quality of the image depends upon the dot density As the pixel density increases we must store more data ICS 101 (November 1, 2016) Digital Images 17
iclicker Question Consider an analog image with 5 (w) by 4 (h) in actual size and a digitized version of the image with 250 (w) 200 (h) pixels in spatial resolution. What happens when the resolution changes to 500 (w) 400 (h) pixels? A. The image is stretched horizontally B. The image is stretched vertically C. The image becomes blurry D. The image becomes sharper E. No change in the appearance and the quality of the image ICS 101 (November 1, 2016) Digital Images 18
Workspace 1 1. Describe how a digital image is represented. What does spatial resolution of an image mean? 2. Describe the relationship between the pixel density and the quality of digital images. ICS 101 (November 1, 2016) Digital Images 19
Building a Digital Image The world is not black & white but many shades of gray! How many? 20 ICS 101 (November 1, 2016) Digital Images
How do we digitize whiteness? What kind of measurement shall we use? How accurate is it? Should we use a scale of 0 to 100? How many different shades is reasonable? ICS 101 (November 1, 2016) Digital Images 21
How much white is there? 0 50 100 light meter ICS 101 (November 1, 2016) Digital Images 22
How do we store the result? We think in base 10 We would accept a concept of 0% to 100% white (black to white) But computers don t store numbers in base 10 Computers use only binary (on-off) ICS 101 (November 1, 2016) Digital Images 23
How computers store numbers The most elementary on-off unit is a bit (stands for binary digit) We use 1 for on, 0 for off Computers organize single bits into groups of 8 bits called a byte. ICS 101 (November 1, 2016) Digital Images 24
Bits & Bytes bit 1 0 0 1 1 0 1 0 byte ICS 101 (November 1, 2016) Digital Images 25
The Capacity of a Byte How many different values can be represented in 1 byte? 2 8 = 256 values ICS 101 (November 1, 2016) Digital Images 26
The Capacity of a Byte 2 8 = 256 Range: 0 to 255 1 0 0 1 1 0 1 0 2 7 + 2 4 + 2 3 + 2 1 = 128 + 16 + 8 + 2 = 154 ICS 101 (November 1, 2016) Digital Images 27
How much white is there? 0 128 255 light meter ICS 101 (November 1, 2016) Digital Images 28
Bit Depth The bit depth (or radiometric resolution) of an image is the number of bits used to represent the pixel value Determine the number of colors/grayscales each pixel can possibl take on Small bit depth can cause quantization artifacts 4 bits per pixel = 2 4 = 16 unique values 8 bits per pixel = 2 8 = 256 unique values Images from C. Perry ICS 101 (November 1, 2016) Digital Images 29
Bit Depth bits range 1 0 1 Binary image 8 0 255 Typical grayscale image 12 0 4095 High quality grayscale image 16 0 65535 Very high quality grayscale image 32 0.0 1.0 Floating point format 24 0 255/color channel 24 (= 8 + 8 + 8) bit True Color ICS 101 (November 1, 2016) Digital Images 30
Wasteful Storage? Need 1 byte but only 1 bit of it! 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 1 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 ICS 101 (November 1, 2016) Digital Images 31
The File Contents Not very compact 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 1 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 ICS 101 (November 1, 2016) Digital Images 32
File Compression Run-length encoded e.g. 5 0 4 1 5 0 1 1 4 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 1 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 ICS 101 (November 1, 2016) Digital Images 33
iclicker Question Which one represents the run-length encoding of the following bit string? 1 1 1 1 0 0 0 0 1 1 1 0 0 0 1 1 A. 7 0 9 1 B. 4 1 0 3 1 0 2 1 C. 4 1 4 0 3 1 3 0 2 1 D. 2 1 2 1 2 0 2 0 3 1 3 0 2 1 E. 4 1 3 1 2 1 4 0 3 0 ICS 101 (November 1, 2016) Digital Images 34
Digitizing Color ICS 101 (November 1, 2016) Digital Images 35
Trichromatic Theory Trichromatic ( tri = 3; chroma = color) The colors of the visible spectrum can be made by combining varying amounts of the three primary colors Primary colors of light: Red, Green, and Blue Monitors use the additive color System Printing devices use the subtractive color system ICS 101 (November 1, 2016) Digital Images 36
RGB Color Space Primary colors of light are monochromatic energies with 645.2nm (Red), 526.3nm (Green), 444.4nm (Blue) Used mainly in color monitor and video camera Grayscale is represented by the diagonal joining black to white Additive color system: Colors are created by adding components to black (0, 0, 0) http://www.dig.cs.gc.cuny.edu/manuals/ Gimp2/Grokking-the-GIMP-v1.0/ node50.html RGB 24-bits color cube. Courtesy of R. C. Gonzalez & R. E. Woods ICS 101 (November 1, 2016) Digital Images 37
CMY Color Space Each color is represented by the three colors of light: Cyan (C), Magenta (M), and Yellow (Y) (C, M, Y secondary colors of light or the primary colors of pigments) Subtractive color system: It models printing on white paper and subtracts from white rather than adds to black as the RGB system does Mainly used in color printing devices that deposit color pigments It is related to the RGB color space by: ICS 101 (November 1, 2016) Digital Images 38 C M Y = 255 255 255 R G B
iclicker Question Which one of the following represents pure Blue color in CMY color space? (Assume that each color component is represented by 1 byte.) A. (255, 255, 0) B. (255, 0, 255) C. (0, 255, 255) D. (0, 0, 255) E. (255, 0, 0) ICS 101 (November 1, 2016) Digital Images 39
Workspace 2 1. How many different values can be represented with 5 bits? 2. Describe the additive and subtractive color models. How do they work? What colors do they use? Primary colors of light Primary colors of pigments What output devices are they used for? ICS 101 (November 1, 2016) Digital Images 40
Digitizing Color How red is a rose? How many shades of red? ICS 101 (November 1, 2016) Digital Images 41
How much Red is there? 0 50 100 red meter ICS 101 (November 1, 2016) Digital Images 42
How much Red is there? 0 50 100 red meter ICS 101 (November 1, 2016) Digital Images 43
How much Red is there? 0 128 255 red meter ICS 101 (November 1, 2016) Digital Images 44
Mixing the Three Components 0 128 255 red meter 0 128 255 0 128 255 blue meter green meter ICS 101 (November 1, 2016) Digital Images 45
Back to the Byte! Each byte has 256 possible combinations 3 bytes used per color, so 256 256 256 = 16 million colors!!! This is called true color or 24 bit 01001011 11010110 11101001 ICS 101 (November 1, 2016) Digital Images 46
8-Bit Color 8-bit = indexed color 256 color palette Each pixel has a 1 byte address to the palette ICS 101 (November 1, 2016) Digital Images 47
Workspace 3 1. Describe how a color image is digitized. 2. What is true color? How many bits (or bytes) are used to represent colors in true color? ICS 101 (November 1, 2016) Digital Images 48
iclicker Question I gained better understanding of representation and quality of digital images. A. Strongly agree B. Agree C. Don t agree or disagree D. Disagree E. Strongly disagree ICS 101 (November 1, 2016) Digital Images 49
Digital Image Processing The continuum from image processing to computer vision can be broken up into low-, mid- and highlevel processes Low Level Process Input: Image Output: Image Examples: Noise removal, image sharpening Mid Level Process Input: Image Output: Attributes Examples: Object recognition, segmentation High Level Process Input: Attributes Output: Understanding Examples: Scene understanding, autonomous navigation Source: B. Mac Namee ICS 101 (November 1, 2016) Digital Images 50
What is (Computer) Vision? When we see something, what does it involve? Take a picture with a camera, it is just a bunch of colored dots (pixels) Want to make computers understand images Inferring the properties of the world from one or more images Looks easy, but not really Image (or video) Sensing device Interpreting device Interpretations Corn / mature corn in a cornfield Plant/blue sky in the background Etc. Slide adapted from F-F, Li ICS 101 (November 1, 2016) Digital Images 51
TED Talk by Fei-Fei Li: How we re teaching computers to understand pictures http://www.ted.com/talks/fei_fei_li_how_we_re_teaching_computers_to_understand_pictures ICS 101 (November 1, 2016) Digital Images 52
CG versus CV Computer vision is the inverse of computer graphics (sometimes called inverse graphics ) 3D models of objects, locations Lighting information Camera parameters Computer Graphics Computer Vision Images The forward process is unique, the inverse process is not! ICS 101 (November 1, 2016) Digital Images 53
CV Applications: Security Biometrics: verify identity from images of fingerprints, retina, faces, etc. www.research.ibm.com Visual surveillance and activity recognition www.visionsystems.co.nz ICS 101 (November 1, 2016) Digital Images 54
CV Applications: Face Detection Most digital cameras detect faces (and more) Source: D. Hoiem ICS 101 (November 1, 2016) Digital Images 55
CV Applications: Object Recognition Nokia Point & Find (Object recognition in mobile phones) Google goggles ICS 101 (November 1, 2016) Digital Images 56
CV Applications: Sports Improve viewer experience and/or broadcasting FoxTrax www.pinterest.com ICS 101 (November 1, 2016) Digital Images 57
CV Applications: Augmented Reality Google glasses (Augmented Reality) ICS 101 (November 1, 2016) Digital Images 58
CV Applications: Smart Cars Mobileye: vision systems currently in many cars Pedestrian collision warning, Lane departure, Forward collision warning, Headway monitoring, Intelligent high-beam, Speed limit indication ICS 101 (November 1, 2016) Digital Images 59
CV Applications: Autonomous Navigation Autonomous vehicle navigation Estimate motion and position of vehicle Detect and model obstacles Find safe path through environment Mars Rover Curiosity (http://www.nasa.gov) Google s self-driving car ICS 101 (November 1, 2016) Digital Images 60
CV Applications: Medical Operation Surgical robot da Vinci surgical robot (Intuitive Surgical) Image from https://mocana.com Tele-operations Control remote by gesture input TV control by hand motion Photo credit: Ben-Gurion University of the Negev, Israel ICS 101 (November 1, 2016) Digital Images 61
CV Applications: Interactive Games Xbox Kinect ICS 101 (November 1, 2016) Digital Images 62
CV Applications: Automated Image Retrieval CBIR http://www.iosb.fraunhofer.de http://www.fnal.gov ICS 101 (November 1, 2016) Digital Images 63
More CV Applications Medicine Automated blood vessel counting Computer aided diagnosis Optical character recognition Processing text from scanned image, mobile note taker, etc. Detect and identify characters Industrial quality control Visual inspection of assembled products Etc http://www.plate-recognition.info http://www.designworldonline.com ICS 101 (November 1, 2016) Digital Images 64