ITNP80: Course Organization

Transcription

1 1 ITNP80: Course Organization Lecturers: Prof. Amir Hussain (coordinator) (Room 4B64: Dr. Andrew Abel (Teaching Assistant) (Room 4B59; Teaching (see Schedule on Course Website for what happens when, updated regularly!) two lectures per week (most weeks) one practical in most weeks (first one on 25 Feb) one tutorial in certain weeks Assessment: assignment 50% examination 50% Course Website: 2 What is Multimedia? Multi = many, Media = plural of medium A medium is a means of communicating information 3 1

2 What is Multimedia? (2) Multimedia on computer is combining text, graphics, sound, animation, video... to enhance the user experience. Spring Multimedia is the seamless integration of text, sound, images of all kinds and control software within a single digital information environment. Tony Feldman, multimedia consultant (quoted in England and Finney, Managing Multimedia ) 5 Overview of IT80 Aim: To give you a guided tour of Multimedia, and to provide a foundation for whatever further work you may do in this area Course contents span a large range of topics, so the course has more breadth than depth Tour more of the form of a rollercoaster than a sedate walking tour - blink and you will miss something! 6 2

3 Topics in ITNP80 We will consider only multimedia on computers. Multimedia: Multimedia design Multimedia authoring Graphics: Still images Animation and video Sound Digitization Use Software: Adobe Photoshop, Flash, Audacity 7 Multimedia Books Stephen McGloughlin: Multimedia: Concepts and Practice, Prentice Hall, 2001 Nigel Chapman and Jenny Chapman: Digital Multimedia, John Wiley, (2 nd Ed or 3 rd Ed. 2009) and Digital Media Tools, 3 rd Ed Cunliffe & Elliott, Multimedia Computing. Lexden, 2005 Y Rogers, H Sharp and J Preece, Interaction Design: Beyond Human-Computer Interaction, Wiley, 2011, ISBN: You are not expected to buy every book. Recommendation: buy whichever will be most helpful to you in the future & use library & on-line resources for the rest. On-line resources Many of them, available from the course website: 8 Graphics ITNP80 (Multimedia), Spring 2014 A. Hussain, B. P. Graham, D. Cairns University of Stirling 9 3

4 Overview Chapman & Chapman: Chapters 3, 4, 5 Richardson: Chapter 6 Representing Graphical Data (today s lecture) Digitising Pictures and Text (next lecture) Graphical File Formats (To be covered after the Colour Topic) Compressing Graphical Data (To be covered later) 10 Representing Graphical Data Chapman & Chapman, chapters 3,4,5 Richardson 11 Overview ITNP80: input, output and representation of graphical data Java for graphics programming Wider coverage of computer graphics: 2-D images 3-D scenes virtual worlds general graphics programming (e.g. OpenGL) 12 4

5 Graphics in ITNP80 ITNP80 has a practically-oriented style: Richardson s Practical Computer Graphics is one of the few books to take this approach The aim is not to make you a graphic designer, but instead: give you an understanding of issues concerning graphics input, output and representation equip you for practical situations where you might need to use graphics allow you to use graphics in web pages, icon design, reports, talks 13 Representing Graphical Data Logical and physical aspects need defined Use of colour: pixels colours transparency palettes Types of representation: bitmaps vector data other formats 14 Physical / Logical Representation Differing representations of graphical data: physical representation of graphical data is how it actually appears on devices Virtual representation of graphical data may be in a graphics file, or internally in a program We may also have a logical representation of the data s structure in our minds These are often not the same: the differences vary from slight to very large converting from a virtual representation to an actual display on a physical device is called rendering 15 5

6 Pixels Computer graphics deals with screen or display device properties Displays have many small dots called pixels (smallest screen display element) a pixel is the smallest logical unit of display on the screen can be monochrome (black and one colour) or (multi-)coloured arranged logically in a 2D grid e.g. a 3 x 3 array of coloured pixels: 16 Pixels Physical display may not be a perfect 2D grid: Colours specified by Red (R), Green (G), Blue (B) values To display an image on the monitor, a graphics display program renders coloured areas using appropriate pixel settings i.e. set each pixel to an appropriate colour or shade of grey, so that pattern of pixels on the screen produces the desired image. 17 Colours of Pixels Black or white pixels need a single bit Colours are specified (or converted) using colour code values (e.g. RGB values) in some way A typical format (RGB) uses 24 bits format: 8 bits: (R,G,B) takes up 1 byte for each colour A common feature these days is to also have an alpha channel, specifying a level of transparency. e.g. in Java 2 (R,G,B,α) takes up 4 bytes see Chapman & Chapman p

7 Transparency 19 Colour Terminology Lots of confusion! black and white is often not a good terminology to use black and white photographs are not black or white, but grayscale grayscale refers to shades of gray, where the RGB values are all the same monochrome refers not to one single colour, but historically to one colour with black, so monochrome really means two colours, usually black and white (but may mean grayscale) monochromatic in colour blindness refers to grayscale! 20 Palettes A palette is a mapping from a small set of numbers to specifically chosen colours from a wide range (2 24 typically): Many images need only few specific colours: a palette is a subset of colours from a larger range an image file can have its own palette (saved as part of image file) indexed images use palettes (as lookup tables) Used in various file formats, monitor displays Example: A web-safe palette 216 colours reproduced by all web-browsers on any system (using 8-bit colour display) not very good for photo quality images Chapman & Chapman: Pages

8 Representing Image Data: Bitmap Images Bitmapped formats: model an image as an array of pixel values Bitmap data is (logically) a 2-D array of pixels A bitmap gives the colours of the picture, pixel-bypixel (bit-by-bit), in this example: each pixel is represented by one bit (on or off) 22 Bitmaps A Bitmap was also known as a raster maps (the term is still in use in some circumstances- define lines of dots (pixels): when there were just monochrome monitors, bitmaps did code with single bits per pixel when colour was introduced, the term pixelmap was used for coloured images Nowadays, bitmaps refer to 2-D arrays of bits or colours Each pixel is mapped to graphics/video memory one or more bits per pixel Logically bitmaps are 2-D arrays, but may be stored in other ways (e.g. Java 2 used a 1-D int array) 23 Graphical Data Representation Bitmaps have a fixed resolution (level of detail) in an image: limited by number of pixels on display screen There are other ways of representing image data which do not have a fixed resolution the resolution varies dependent upon the content of the image: Some are general purpose Some are program-specific Some are application-specific State-of-the-art graphics programs have proprietary formats (images are represented internally in an application-specific way), but then exported to bitmap formats for display Richardson: Section

9 Another way to represent image data Vector-based formats contain descriptions of one or more objects, rather than pixels Uses a draw-then-edit method of image creation Often the objects are mathematically based/specified eg line segments, polygons, circles, splines Image is stored as a mathematical description of objects e.g. collection of lines, curves & shapes making up the image 25 Vector Images Vector formats are well suited to 2-D images: line drawings, graphs, architectural/engineering drawings, 26 Bitmap vs. Vector Format Bitmap format: fixed resolution every logical pixel is stored scaling leads to loss of quality Vector format: description of component shapes can be displayed at any scale 27 9

10 Scaling Bitmaps Resampling Increase in size: add new pixels upsampling Decrease in size: throw pixels away downsampling (Images MacAvon Media Productions) 28 Scaling Bitmaps (2) How best to calculate new pixel colours? Pixel interpolation Nearest neighbour Choose colour of pixel with largest overlap Bilinear interpolation Average colours of surrounding pixels Weight by their level of overlap More complex mappings Bicubic interpolation (Image MacAvon Media Productions) 29 Scaling Bitmaps (3) Nearest neighbour Bilinear interpolation Bicubic interpolation (Image MacAvon Media Productions) 30 10

11 Editing a vector file (objects can be selected & transformed using mathematical operations) Bitmaps vs Vector Files can t in a bitmap? (as these are stored as collection of bits, editing can only take place at level of pixels!) 31 Bitmap vs. Vector Format Advantages of vector format: good for storing images composed of line-based or 3-D objects (e.g. wire-frame models) easy to convert to bitmap format Can be very compact Disadvantages of vector format: not good for storing complex images (e.g. photographs) Appearance of image can vary widely, depending upon the Application rendering the image may take significantly longer than for bitmaps 32 From Vectors to Bitmaps Historically, vector data was widely used: Pen plotters used pens to draw on paper (an early form of graphics printer) these were cheap and produced line-based drawings storage of high-volume bitmap files was expensive With advent of cheap storage and highresolution output, many images are now bitmaps. Bitmaps are everywhere! digital cameras Just look at the web use of images and graphics (GIFs, PNGs, JPEGs everywhere!) 33 11

12 and back again Trends are shifting towards a greater use of vector data the bitmap trend may not last. Size is again an issue... big bitmaps take more memory, storage and transmission time over the internet vector formats are better for 3-D imaging, and 3D imaging is growing more important Fuelled by such concerns as film-making, entertainments industry Other Graphics Representations: hybrid, e.g. metafile formats and fractal representation techniques Animation formats including Special purpose 3-D formats 34 Metafile Formats A metafile can store both vector and bitmap data: usually for vector files that might have some bitmapped data added a bitmap is typically regarded as one type of vector object typically most elements in the file are vectors, with occasional bitmaps (e.g. file contains vector instruction to draw square then fill with bitmap pattern) Richardson: Section Fractal Image Representation: Sierpinski Triangle Same structure at all levels of detail, i.e. selfsimilar at all scales Can use any repeated shape 36 12

13 Fractal Image Compression A recent development in image compression: An image (or part of an image) is represented by a mathematical formula which represents a pattern that repeats at different scale (aim is to describe underlying patterns/self-similarity in an image) to display an image on a device, the formula is repeatedly applied to (perhaps) a blank seed image of required size a resolution-independent way of storing images (takes less space than a bitmap) Really a form of image encoding than compression for its own sake It is compression because the formula takes up less space than a bitmap would. Many real-world scenes (e.g. mountains, clouds, landscapes) can be described fractally 37 Fractal Image Compression Typically an image would start in bitmap form The compressed image has an optimal resolution close to that of the original image result is softer, not pixellated sometimes used for image enhancement Self-similar images compress well: works for many photos, e.g. of forest, sea or sky Compression/encoding takes time, but decompression/decoding is very fast Chapman & Chapman: Pages A Fractal Mountain Spring 2014 University of Stirling 39 13

14 Fractal Clouds Spring 2014 University of Stirling 40 Bitmap versus Fractals x2 Original bitmap x2 Fractal version Spring 2014 University of Stirling 41 Practical Graphics Issues Spring 2014 University of Stirling 42 14

15 Digitising Pictures and Text 43 Overview Chapman & Chapman: Chapters 4 & 7 Digitizing and Rendering From image to bitmap Anti-aliasing What is it and how to do it? Lines Shapes Fonts Anatomy and types of a fonts How to make fonts look nice Font sources 44 Digitising Pictures How do we go from a continuous image to a bitmap? A picture is scanned/sampled and the bits of the bitmap are set according to what colour wins for that pixel - resulting in a bitmap for the best colour in each pixel: results in jagged edges 45 15

16 Digitising Pictures The same principle applies to rendering text (text reproduction), and to converting vector to bitmap formats (which is first step in rendeing an image!) 46 Digitising Pictures Each bit/pixel set to one of two colours because screens were originally monochrome: results are often not good how to achieve better results with colour monitors that can also display grayscale? Monochrome bitmaps are still useful for highcontrast images: e.g. dark text on a light background Non-colour printing involves just one colour (black) on white paper 47 Anti-Aliasing Previous examples use only two colours The results can be more visually appealing if a range of colours is used at the edges of an object. Anti-aliasing effectively blurs the edges to create a smoother edge. - use different shades of colour or gray to reduce jaggedness Richardson: Sections 1.5,

17 Anti-Aliasing (a close-up) 49 Anti-Aliasing Shapes Specify gray level or opaqueness in proportion to coverage of shape 50 Anti-Aliasing Lines Opaqueness in proportion to how many subpixels the line passes through Sub-divide each pixel into sub-grids Count how many sub-pixels the line passes through to determine gray level 51 17

18 Fonts Historically, computer did not have fonts (styles of text) as such Characters were just characters, displayed as dots: on screen (fixed number of columns and rows) on printer (dot matrix, daisywheel) Chapman & Chapman: Chapter 7 Richardson: Chapter 5 52 Anatomy of A Font One point is 1/72 of an inch sans serif baseline point size ascent descent! descent leading serif Point size is a measure of the body size (the distance from the top of the highest character to the bottom of the lowest character) in 72nds of an inch 53 Fonts Not to be confused with character sets! Fonts are different ways (styles) of representing characters e.g. to get the Greek letter alpha, a in Symbol font is displayed as α α can also be obtained as character 03B1x of the Unicode character set Not a 1-1 correspondence between characters and font shapes (glyphs) some characters consist of more than one glyph some glyphs represent a combination of characters (ligatures): N.B. A glyph is *not* a character. It is a shape for one (or more) characters. Spring 2014 University of Stirling 54 18

19 Some Common Fonts For: normal writing, display, script, symbol, fixed-width, fractional widths early development typewriter Arial (sans serif, TrueType) Times New Roman (serif, TrueType) Lucida Console (TrueType, fixedwidth) Courier New (serif, TrueType, fixedwidth) Courier (serif) Sc!pt (Sc!pt, a sc!pt font) Σψνβολ (Symbol, a symbol font) W i n g d i n g s (Wingdings, a symbol font) 55 Bitmapped Fonts Each font shape/glyph is represented as a bitmap so scaling doesn t work well Characters do not scale Entire font families are needed at different resolutions (how fonts were used in early word processors), e.g. 8 point 10 point 12 point 14 point 16 point 18 point 20 point 24 point 36 point 72 point 56 Vector-Based Fonts Nowadays, vector-based fonts have taken over referred to as scalable or outline fonts These are defined in terms of: a set of component shapes (straight lines and curves) hints (see later) Scalable fonts are converted to bitmapped glyphs as needed for rendering on a screen or a printer Aside: Java s fonts in Java2D are scalable fonts These use Unicode as the character set 57 19

20 Displaying Fonts Nicely Solutions from printing industry Solutions specific to computer-based fonts Ways to do this include: Non-fixed-width Anti-aliasing Hinting Kerning (spacing) 58 Anti-Aliasing Example 59 Anti-Aliasing Text The result may not always be better: e.g. here s a 8pt plain A in Times New Roman normal anti-aliased 60 20

21 Problems with Small Text Digitisation may result in uneven display of small text: consider the results for h and m 61 Hinting Hinting is the provision of hints as to how to display fonts in certain circumstances A font may be defined without hints, but having hinting results in a better quality font A Microsoft hint to leave vertical and horizontal strokes solid! Spring 2014 University of Stirling 62 Kerning Improves text appearance by changing the spacing of adjacent text (even to the point of overlapping) Done by use of kern pairs A high quality font (such as TrueType fonts) have suitable kern pair settings for different combinations of characters TrueType fonts may have anything from 100 to 500 kern pairs (150 is typical) depending upon potential character overlaps

22 Without kerning: With kerning: Spring 2014 University of Stirling 64 Font Foundries/Types The rivals! Adobe Postscript (industry standard): a page description language (PDL) for printers often directly hardware-implemented in printer provides many fonts (more than 20,000) expensive licence Apple TrueType (licensed by Microsoft): A collection of software-implemented, high quality vectorbased fonts Suitable for non-printing uses Converted to bitmaps which are then sent to a printer collection of high quality fonts Both are scalable and use hinting Font foundries Other fonts required can be purchased! NEXT LECTURE ON COLOUR 65 Graphical File Formats To be Covered (AFTER next 2 Lectures on Colour ) 66 22

23 Overview Why have graphics file formats? What to look for when choosing a file format A sample tour of different file formats, including bitmap-based formats vector-based formats metafiles proprietary formats fractal compression formats The Graphics File Format Page The Graphics File Formats FAQ (see online references) 67 Requirements for Formats Need to store and retrieve graphical data in an efficient and logical way Data stored according to specific format conventions Formats are immortal - technology evolves, new formats appear, but the old ones will still be there! No one universal format - different formats for different purposes (tradeoffs!) You (probably) won t need to access the formats in detail there is (usually) library code to input/output/convert images for you useful to understand what is going on behind the scenes, for making the best image format choices 68 Terminology - Format Characteristics (or what to look out for in file formats) When choosing which is more appropriate for your purpose, some common factors to consider: Lossy or lossless compression? What is the compression ratio? 8-bit or 7-bit (binary or text)? Is it stored at a fixed resolution? How many images per file (static or animated)? progressive download (for the Web).amongst other things 69 23

24 Bitmaps Generalised structure of a bitmap file: In practice there are many format variations Palette -indexed mode why a palette? only a few colours used - takes up too much space to use 3 bytes for each pixel, instead label each colour with a number, it s the number what is stored in the palette 70 PBM, PGM, PPM Family of portable bitmaps: Portable Bit/Gray/Pix Map Designed to be as simple as possible Written in ASCII 7 bit code No compression 1 image per file 71 PBM Format Portable Bit Map for monochrome images feep.pbm:

25 PGM Format Portable Gray Map for grayscale images feep.pgm: PPM Format Portable Pix Map for coloured images square.ppm: BMP Standard bitmap storage format for Microsoft Windows (also supported by some other non-microsoft applications) ) not especially designed for portability not especially designed for data interchange format across different operating systems 1 image per file with RLE (run-length encoding) therefore uses lossless compression original bitmap did not support Compression huge file sizes, e.g. this photo is 40.3Kb as JPEG, 963Kb as BMP: 75 25

26 BMP Format 76 GIF Graphics Interchange Format (1987 & 1989 versions): pronounced jiff Uses a palette of up to 256 colours: good for text/diagrams, not ideal for photos Uses LZW (Lempel, Ziv, Welch) compression: possible copyright problems (Unisys patent) reasonably easy to read and decompress Supports multiple images (for animation) GIF 1989 supports transparency & interlacing At most one colour in the palette may be declared transparent Interlacing for progressive download 77 GIF Format 78 26

27 Interlacing Pixel Data Row Number Interlace Pass Interlacing 80 PNG Portable Network Graphics: informally stands for PNG Not GIF pronounced ping Successor to GIF as a lossless but patent-free solution! Unrestricted palette - up to 48-bit true colour Transparency, interlacing, but not animation: animation available in a related format, MNG Transparency through an alpha channel also supports anti-aliasing Compression marginally better than GIF (based on LZ77 earlier than LZW) 81 27

28 JFIF JPEG File Interchange Format There is no (single) JPEG format Basically a data stream with a few restrictions and identifying markers JPEG (Joint Photographic Experts Group) properly refers to the compression algorithms used typically between 10:1-20:1 for photographs 82 TIFF Tagged Image File Format Designed mainly for desktop publishing and related applications Contains lots of tags in the file, identifying different types of content makes it easier to add new features Rationale behind using tags is that it is easier to incorporate new features in future versions of the format without confusing old software, or making old TIFF images useless Very complex since may contain multiple internal image formats 83 FIG Facility for Interactive Generation (of figures): proprietary vector format used with jfig and xfig drawing programs text file 84 28

29 SVG Scalable Vector Graphics XML language for vector graphics in web pages: plug-ins available for common browsers separate text file or embedded in HTML <?xml version="1.0"?> <svg xmlns=" <g style="fill-opacity:0.7; stroke:black; stroke-width:0.1cm;"> <circle cx="6cm" cy="2cm" r="100" style="fill:red;" transform="translate(0,50)"/> </g> </svg> 85 SVG Example 86 CGM Computer Graphics Metafile An international standard Three different encodings: One for minimal size/ease of transmission binary encoded for speed of access clear text for human readability/editing Primarily vector graphics, but also bitmap data An ISO standard 87 29

30 POVRay POVRay is a 3-D computer graphics ray-tracing rendering program: it has its own proprietary 3D scene-based format, looks like program code example 88 VRML / X3D Virtual Reality Modelling Language: pronounced virmal Interactive 3-D worlds with browser plug-ins 89 FIF Fractal Image Format Barnsley (and later, Jacquin) developed the compression technique Commercial format by Iterated System Inc Somewhat over-hyped: e.g. claims of 1000s:1 compression compression ratios more typically 4:1-100:1 Good for most pictures, particularly photographs and scenes with some selfsimilarity (which is common) 90 30

31 Comparison Table Raw Data 100% 100% 100% PPM 407% 406% 406% BMP 100% 33% 33% GIF 19% 18% 1% PNG 32% 16% 1% TIFF (LZW) 111% 18% 1% JPEG 5% 5% 10% FIF 5% 6% 3% 91 Data Compression (for Images) -Compressing Graphical Data 92 The Need for Data Compression Graphical images in bitmap format take a lot of memory e.g x 768 pixels x 24 bits-per-pixel = 18,874,368 bits or 2.4Mbytes 16 Mega pixel RAW format from a camera = 20Mbytes But 100s of Gbyte /Terabytes hard drives are now readily available!! So why do we need to compress? How long did that graphics-intensive web page take to download? Even over a broadband connection it could be slow How many images can your digital camera hold? Picture messaging over mobile phones? CD (650Mb) can only hold less than 10 seconds of uncompressed video (and DVD only a few minutes) We need to make graphical image data as small as possible for many applications 93 31

32 Types of Compression Pixel packing RLE (run-length encoding) Dictionary-based methods JPEG compression Factors to look out for: Lossy or lossless compression? What sort of data is a method good at compressing? What is its compression ratio? Richardson, Chapter 6; Chapman & Chapman, Chapter 5 Spring 2014 University of Stirling 94 Lossy vs Lossless compression Lossless compression implies that one can recreate the original data precisely. Usable anywhere, Critical for e.g. documents, binary programs, etc. Usually requires significant processing to re-create original Lossy compression implies that one cannot recreate the precise original But one can re-create something almost as good Useful for images, sounds, where what matters is the percept, not precise bit-for-bit re-creation Some processing required but less than lossless approach Spring 2014 University of Stirling 95 Pixel Packing Not a standard data compression technique but nevertheless a way of not wasting space in pixel data e.g. suppose pixels can take grey values from 0-15 each pixel requires half a byte but computers prefer to deal with bytes two pixels per byte doesn t waste space Pixel packing is simply ensuring no bits are wasted in the pixel data (Lossless if assumption true) Spring 2014 University of Stirling 96 32

33 Run-Length Encoding A sequence of the same values is stored as a repetition count and the value: AAAAAAAAAAAAAAA would encode as 15A AAAAAACCCBBBBBD = 6A 3C 5B 1D This works best for large expanses of the same colour, or is it? consider the image below: the small 10 x 11 block has 59 different colours the full (371x247 bitmap) image is 275 Kb raw data the RLE-compressed image is 91 Kbytes (3:1 ratio) RLE compression ratios not good in general, because there are rarely repeat runs of pixels 97 RLE compression ratio Another example, with a diagram this time Full image: bitmap 277Kb raw data ( = ) bytes 46.5K RLE encoded Compression ratio approx 6:1 in this case Spring 2014 University of Stirling 98 Dictionary Methods A common way to compress data (pixels, characters, whatever!) is to use a dictionary The dictionary contains strings of bytes e.g. particular pixel patterns not limited to patterns of one colour, as with RLE Data is encoded by replacing each data string that has an entry in the dictionary with its index number in the dictionary Shorter to write an index number than a whole string! (what are the implicit assumptions here?) Dictionary may be particular to the image, or may be standard for particular image types Spring 2014 University of Stirling 99 33

34 Patterns of Pixels Poor results with RLE as runs of pixels with same colour are very short But there are repeating patterns with two colours that could be included in a dictionary Trade-off between pattern size and likelihood of being in the dictionary Spring 2014 University of Stirling 100 Huffman and CCITT Compression Developed for fax machines and document scanners Uses a predefined dictionary of commonly occurring byte patterns from B&W documents containing large amounts of text in a variety of languages and typical examples of line art Commonly occurring patterns are given low indices (coded using short codes) in the dictionary Data is encoded by replacing each image string that occurs in the dictionary with its index number Dictionary is not part of the compressed file. Spring 2014 University of Stirling 101 The Lempel-Ziv-Welch Algorithm The Lempel-Ziv-Welch method is another such dictionary algorithm, in which the dictionary is constructed as the encoding (compression) progresses (actually Ziv was the first author on the original papers!) LZW starts with a dictionary of byte strings: Entries refer to those individual bytes Entries 256 onwards will be defined as the algorithm progresses Each time a new code is generated it means a new string of bytes has been found. New strings are generated by appending a byte c to the end of an existing string w Single pass algorithm Spring 2014 University of Stirling

35 When Is LZW Useful? Good for encoding pixel data with a limited palette, and/or repetitive data line drawings diagrams plain text on a plain background Not good for photographic images large colour range and complex features results in few repeating patterns to include in a dictionary Lossless and fast Spring 2014 University of Stirling 103 JPEG Joint Photographic Experts Group Designed to compress photographs colour or greyscale good at compressing real scenes not good for line drawings, diagrams, lettering, cartoons Designed for human viewing, exploits our inability to see a full range of colours Lossy algorithm Not good for computer analysis of data e.g. medical imaging May throw away vital data! Spring 2014 University of Stirling 104 JPEG Compression Compression steps: transform to brightness/colour model (e.g. HSB) down-sample the colour values (reduce their scale) split brightness/colour maps into 8 x 8 pixel blocks identify spatial frequencies in these blocks quantise frequency values using a Q factor store discrete values efficiently (e.g. RLE) Different Q (Quantisation) factors give different compression ratios: 10:1 to 20:1 reasonable, 100:1 for low quality Trade-off of quality versus size of compressed data Data is lost on (de/re)compression (from quantization processes)

36 JPEG Examples 15:1 photo compression: JPEG tries to smooth sharp boundaries, e.g. the edges of letters (Text / background boundary is a high spatial frequency - JPEG attempts to smooth this with poor results!!) 106 JPEG Compression How good is it? For full-colour images, the best-known lossless compression gives about 2:1 compression For reasonable quality, compression ratios of 10:1 or 20:1 quite feasible for JPEGs Clive s picture compressed with a ratio of 15:1 For poor quality images (thumbnails?), 100:1 possible Repeating the process with subsequent re-encoding loses more information Spring 2014 University of Stirling 107 Compressing Images LZW: good for a limited palette and repeated patterns e.g. plain text on a plain background, line drawings and diagrams not good for photos due to many colours and complex patterns JPEG: designed for colour or grayscale photos compression is lossy good for natural scenes not good for line drawings, diagrams or cartoons

37 Video Compression/How Do We Compress Movies? Compress individual frames using any of the techniques mentioned already spatial compression High, lossy compression is OK as the quality of individual frames can be lower than for still images as our perception is dominated by motion Make use of limited changes between frames key frames difference frames temporal compression More on this in a later lecture