3 Image Compression. Multimedia Data Size/Duration Kbits Telephone quality speech. A Page of text 11 x 8.5

Transcription

1 3 Image Compresson Multmeda - Department of Computng, Imperal College Professor GZ Yang! Why Image Compresson Research n compresson technques has stemmed from the ever-ncreasng need for effcent data transmsson, storage and utlsaton of hardware resources. Uncompressed graphcs, audo and vdeo data requre consderable storage capacty and transmsson bandwdth. Despte rapd progresses n mass-storage densty, processor speeds, and dgtal communcaton system performance, demand for data storage capacty and data-transmsson bandwdth contnues to outstrp the capabltes of aval-able technologes. The recent growth of data ntensve dgtal audo, mage, and vdeo (multmeda) based applcatons, have not only sustaned the need for more effcent ways to encode sgnals and mages but have made compresson of such sgnals central to sgnal-storage and dgtal communcaton technology. Here are some examples of the space requred for storng dfferent knds of multmeda nformaton: Multmeda Data Sze/Duraton Bts/Pxel or Bts/Sample Uncompressed Sze A Page of text 11 x 8.5 Varyng resoluton Kbts Telephone qualty speech 1 sec 8 bps 64 Kbts Grayscale mage 512 x bpp 2.1 Mbts Color mage 512 x bpp 6.29 Mbts Medcal mage 2048 x bpp 100 Mbts Full-moton vdeo 640 x 480, 10 sec 24 bpp 2.21 Gbts Image compresson methods are typcally dvded nto two categores, lossless and lossy. Compresson technques belongng to the frst category have the man characterstc that the mage nvolved can be perfectly reconstructed from the compressed fle, thus havng no nformaton loss. On the other hand lossy compresson methods rely on the fact that some of the data can be dscarded wth almost no detectable loss n mage qualty by the human vsual system. When research nto mage compresson began n the late 1970s, most compresson concentrated on usng conventonal lossless technques, meanng that the reconstructed mage after compresson s numercally dentcal to the orgnal mage on a pxel-by-pxel bass. However, such types of compresson technques, whch ncluded statstcal and dctonary methods of compresson, dd not tend to perform well on photographc, or contnuous tone mages. The prmary problem wth statstcal technques s due to the fact that pxels n photographc mages tend to be well spread out over ther entre range. Hence, f the colours n an mage are plotted as a hstogram based on frequency, the hstogram s not as spky as t would be requred for statstcal compresson to be effectve. Each pxel code has approxmately the same chance of appearng as any other, negatng any opportunty for explotng entropy dfferences. By the late 1980s, extensve research pushed the development of lossy compresson algorthms that take advantage of known lmtatons of the human eye. Such algorthms play on the dea that slght modfcatons and loss of nformaton durng the compresson/decompresson process often do not affect the qualty of the mage as perceved by the human user. One such technque s to explot the self smlarty nature of mage patterns based on fractal geometry for mage compresson. -1-

2 Self smlarty n mages Fractal and Iterated Functon Systems The brth of fractal geometry s usually traced to IBM mathematcan Benot B. Mandelbrot and the 1977 publcaton of hs semnal book The Fractal Geometry of Nature [35]. It stresses the fact that tradtonal geometry wth ts straght lnes and smooth surfaces does not resemble the geometry of trees and clouds and mountans. Fractal geometry, wth ts convoluted coastlnes and detal ad nfntum, does. Ths nsght opened vast possbltes, allowng computer scentsts to generate artfcal yet realstc lookng forms. Shortly after Mandelbrot s work, mathematcans searched for a framework underlyng fractal geometry. As John Hutchnson demonstrated n 1981, t s the branch of mathematcs known as Iterated Functon Theory. Later n the decade Mchael Barnsley authored Fractals Everywhere, another mlestone work. The book presents the mathematcs of Iterated Functons Systems (IFSs), and develops a result known as the Collage Theorem. The Collage Theorem states what condtons an Iterated Functon System must satsfy n order to represent an mage. Ths presented an ntrgung possblty. If, n the forward drecton, fractal mathematcs s good for generatng natural lookng mages, then, n the reverse drecton, could t not serve to compress mages? Gong from a gven mage to an Iterated Functon System that can generate the orgnal (or at least closely resemble t), s known as the nverse problem. In ts general form, the nverse problem remans unsolved. In search of somethng practcal, Arnaud Jacqun, one of Barnsley s students, arrved at a modfed scheme for representng mages usng Parttoned Iterated Functon Systems (PIFSs). In hs PhD thess, Jacqun developed the necessary mathematcal foundatons and mplemented the new approach n software, a descrpton of whch appears n hs landmark 1992 paper Image Codng Based on a Fractal Theory of Iterated Contractve Image Transformatons. The algorthm was not sophstcated, and was computatonally expensve, but t was fully automatc. All contemporary fractal mage compresson programs are based upon Jacqun s approach. -2-

3 An elegant way of ntroducng the noton of Iterated Functons Systems s by the metaphor of a Multple Reducton Copyng Machne (MRCM). An MRCM s magned to be a regular copyng machne except that: There are multple lens arrangements to create multple overlappng copes of the orgnal. Each lens arrangement reduces the sze of the orgnal. The coper operates n a feedback loop, wth the output of one stage the nput to the next. The ntal nput may be any mage. The above fgure depcts ths process for Serpnsk s Trangle, one of the smplest (and most well known) IFS. It s comprsed of three component functons ( lenses ), each of whch shrnks the nput mage by one half and translates t to a new poston. Ths contractve property s crucal, for t guarantees convergence of the teratve process. Because all ntal mages are drawn towards the same fnal result, t s varously referred to as the attractor of the IFS, or the fxed pont mage. Mathematcally, one can represent each reproducton lens as a contractve affne transformaton whch rotates, scales, shears, and translates the orgnal copy to a target locaton,.e., a b e w = c d + f whch maps a gven pont n the orgnal mage (x, y) to a new coordnates (x,y ), where x' a y' = c b x e d y + f For the Serpnsk Trangle shown above, the three transformatons used can therefore be represented as w1 =. w2 w = = It can be proven that f the determnant of each transform s strctly less than one,.e., ad-cb <1, then the IFS as a whole wll converge to the attractor mage from any ntal mage. Fractal Image Compresson Fractal mage compresson s based on the observaton that many natural scenes possess a detal-wthn-detal structure and IFS can generate fractal mages that resemble natural scenes. The IFS can be reverse-engneered from the orgnal mage such that the correspondng IFS can be represented compactly for the orgnal mage. One unque feature of IFS based mage compresson s that we only need to store the -3-

4 transformaton found wthn the mage, and the decompresson process s to apply these transformatons to any gven pattern so as to restore the orgnal data. The nature of how the Parttoned IFS s used for mage compresson s llustrated n the followng fgure. The basc dea s ths: f fndng self-smlarty between an mage n the whole and ts parts s unrealstc, then seek self-smlarty between larger parts and smaller parts. Ths s accomplshed, as the name suggests, by parttonng the orgnal mage at dfferent scales. Snce mages usually take the form of a rectangular Doman blocks Range blocks array of pxels, parttonng the orgnal mage nto blocks s a natural choce. Usng Jacqun s notaton, the large parttons are called doman blocks, and the small parttons are range blocks. The range blocks evenly partton the mage so that every pxel s ncluded. The larger doman blocks may overlap, and need not contan every pxel. The goal of the compresson process s to fnd a closely matchng doman block for every range block. The set of doman blocks consdered n ths operaton s called the doman pool. When appled to gray scale mages, the ntensty value of a pxel, z, s treated as a thrd spatal dmenson. That s, the blocks n the above fgure are actually cubods, although the orgnal termnology remans. To acheve convergence the ntensty value of a pxel must also be scaled and offset,.e. z' = s z + o so that the affne transformaton mentoned earler becomes x' x a b y' = w y = c d z' z x e 0 y + f s z o The parameter s scales the pxel lumnance and ts effect s lke the contrast knob on a televson. When s s 0 the doman block maps to black, when equal to 1 t remans unchanged; between 0 and 1 the block loses contrast, and above 1 t gans contrast. The parameter o ntroduces an offset to the pxel lumnance and s lke the brghtness knob on a televson. Postve values of o brghten the block and negatve values darken t. Wth contract and brghtness control avalable, the extended affne transformaton can accurately map grayscale doman blocks to grayscale range blocks. The above expresson ndcates that n order to compress a gray scale mage, we have to fnd 8 varables for the transformaton equaton of each range block. If -4-

5 each coeffcent s parttoned as 100 steps, the search space s of the sze of 10 16, whch s computatonally very expensve. To make the compresson process tractable, Jacqun restrcted equaton so that doman blocks are always square (not rectangles or parallelograms), and always twce the sze of range blocks. If the range blocks are, say, 8x8 pxels n sze, then the doman blocks are always 16x16. By dong so, t greatly reduces the sze of the doman pool. Ths s favourable snce t shortens the search tme, but reconstructon qualty suffers as optmal parngs may be excluded from consderaton. One smple and effectve way of mprovng codng qualty s by allowng doman blocks to undergo an sometrc symmetry operaton pror to beng transformed. The beneft of such an operaton s llustrated n the followng fgure, where block 1 s frst rotated clockwse by 270 to mprove the smlarty between t and the range block. If the eght symmetry operatons are not allowed, then a less optmal parng, transformaton w2, must be used nstead. Wth the above smplfcatons, the new transformaton for the gray scale extenson of the PIFS becomes x' y' = M z' 0 0 s x e y + f z o where M s a 3x3 matrx representng one of the eght symmetry operatons. So the mage compresson process nvolves the dentfcaton of (e,f,m, o,s) for each range block, where the frst two coeffcents locate the doman block, the thrd apples a symmetry operaton, and the last two ntroduce an offset and scalng factor. Because the numbers assocated wth these coeffcents mplctly defne a set of affne transformatons, a fractal encoded mage s sometmes descrbed as beng composed of mathematcal equatons. For a gven mage, f we use the followng quantzaton for each of the fve parameters, e 8 bts horzontal postons f 8 bts horzontal postons M 3 bts - 8 horzontal postons s 5 bts - suffcent from emprcal tests o 6 bts - suffcent from emprcal tests we only need 4 bytes for representng each w. If the 256x256 mage s dvded nto 8x8 range blocks, there are 1024 altogether, therefore the compressed mage only requres 1024 x 32 bts = 4096 bytes (f uncompressed, t requres 64 Kbytes). -5-

6 Matchng Doman and Range Blocks Before the actual compresson can take place, we need to determne the scalng parameters for the best range-doman parng. In the orgnal algorthm of Jacqun, the goal s to mnmze the Haussdorff dstance (.e. greatest pxel-to-pxel dfference) between a specfc range block and a canddate doman block. To do so, a small set of scale values {0.45, 0.60, 0.80, 0.97} are tested n sequence, and the one that produces the smallest Haussdorff dstance s retaned. If, nstead, the mean square error measure s used, the optmal scalng parameter can be determned algebracally. Frst, assume that the doman block Dxy has been reduced to the sze of the range block Rxy (by averagng 2x2 pxel cells), and that they have been adjusted to a zero-mean ntensty level. Then, the mean square error between the blocks s 1 2 e = 2 ( s Dxy Rxy ) n x, y Ω By mnmsng e, one can solve for s. Ths can be acheved by takng the dervatve of e wth respect to s as zero,.e., de 2 = 2 ( s Dxy Rxy ) Dxy = 0 ds n therefore, x, y Ω s R xy x, y Ω = D x, y Ω D 2 xy xy In other words, the optmal scalng factor between a range block and a doman block s ther nner product dvded by the doman block sum-of-squares. Ths value s calculated for all canddate doman blocks, under all eght symmetry operatons, n search of the smallest error. To ensure convergence of the decompresson process t s common practce to force all component transforms to be contractve, that s, to restrct s < 1.0. Ths s not strctly necessary. In fact, releasng ths constrant has been shown to mprove mage qualty n some cases. Search Strategy and Image Parttonng Wth fractal mage compresson, two other mportant ssues needs to be consdered. One s the search strategy used for fndng range-doman pars, and the other s related to how to effectvely partton the range blocks. The development of an effectve search strategy s mportant n that by usng a 8x8 range block parttonng for a 256x256 mage, 1024 parngs need to be establshed. -6-

7 Even wth Jacqun s smplfcaton, the doman pool contans 8x( ) 2 = 464,648 elements (recall that eght symmetry operatons are allowed). In total, 464,648 x 1024 = 475,799,552 possble parngs are requred for testng, whch requres 128 Gflops (floatng pont operatons), and t takes 10 seconds on a Cray YMP-16 supercomputer! (You can workout how long t wll take for a 1024x1024 mage). To address ths problem, the followng search strateges have been developed over the years. Heavy brute force - an exhaustve search method, don t expect the compresson algorthm wll work on a desktop computer Lght brute force - look at x2, or x4 pxel locatons durng matchng Restrcted area search - restrct the search only to nearby areas Local Spral Search - dramatcally recudes the search tme The development of an effcent search strategy s only part of the story. In fact, the performance of the system s closely related to how one parttons the mage. In general, the smaller the number of range blocks, the fewer the number of parngs needs to be dentfed. The above fgure shows three dfferent parttonng schemes where the number of range blocks vares from 5008, 2910, to Decompresson The decompresson process usually begns by settng the computer s mage buffer to a unform md-gray value. Ths s used as the seed mage. Durng one teraton, the pxels of each range block n the transform lst are evaluated. The result s used as the nput for the second stage of teraton. The followng fgure shows that after just two teratons, the orgnal mage s recognzable, and after four the process wll usually have converged (when eght bt precson s used per pxel). Due to the very nature of the IFS, the choce of the seed mage s not mportant, they wll all ultmately converge to the attractor mage. Although the choce of seed mage does not affect the outcome, t can affect how quckly the decompresson process converges. One could nstead begn wth an all-black seed mage, or an all-whte one, but usually mdgray s preferable. A successful way of ncreasng decompresson speed, as frst descrbed by Beaumont s to begn wth a low resoluton verson of the orgnal. Ths s accomplshed by modfyng the PIFS equaton so that o descrbes the mean value of a range block, rather than the relatve offset from the correspondng doman block. -7-

8 Lmtatons and Conclusons In summary, fractal mage compresson s a promsng technology, although t s stll relatvely mmature. The technque s block based, lossy compresson method. In general, decompresson s very fast, but the compresson process can be very slow. Compared to other parallel technques, the followng table summarsed the pros and cons of each approach: Image Category Low Compresson Hgh Compresson text and lne art poor poor computer graphcs poor - good poor - far photo-realstc mages good very good Source J Komnek, Advances n Fractal Compresson for Multmeda Applcatons. -8-