L. Yroslvsky. Fundmentls of Digitl Imge Processing. Course 0555.330 Lecture. Imge enhncement.. Imge enhncement s n imge processing tsk. Clssifiction of imge enhncement methods Imge enhncement is processing imed t ssisting imge visul nlysis. In visul imge nlysis, the user tht observes imges plys role of decision mking mchine while imges my be regrded s sets of fetures tht re needed for the decision mking nd tht re represented in specil wy perceivble to the mchine. Humn visul system hs certin limittions in its cpbility to perceive informtion crried by imges. Visul system cn not detect brightness contrsts tht re lower then certin contrst sensitivity threshold. It hs lso limited resolving power. It fils to nlyze imges corrupted by noise, especilly by impulse noise or highly correlted noise ptterns. From the other side, visul system perceives colors, it hs 3-D cpbility through stereo vision, it is cpble of efficient detecting chnging imges in time. One cn sy tht visul system hs five chnnels per pixel to perceive informtion: three for RGB colors nd dditionl two for stereo nd for dynmicl vision. Imge enhncement is intended to convert imges into form tht mkes use of cpbilities of humn visul system to perceive informtion to their highest degree. Theoreticlly, imge enhncement methods my be regrded s n extension of imge restortion methods. However, in contrst to imge restortion, imge enhncement frequently requires intentionl distorting imge signl such s exggerting brightness nd color contrsts, deliberte removing certin detils tht my hide importnt objects, converting gry scle imges into color, stereo nd movie ones to disply three-four-five component fetures, etc. In this sense imge enhncement is imge preprtion or enrichment in the sme mening these words hve in ing. An importnt peculirity of imge enhncement s imge processing is its interctive nture. The best results in visul imge nlysis cn be chieved if it is supported by feedbck from the user to the imge processing system. Imge enhncement methods dte bck to the beginning of the lst century photogrphy nd microscopy methods such s unshrp msking, solriztion, phse contrst. Electronic television enbled using more sophisticted methods for mnipulting imge contrsts, brightness, color nd geometry. However only digitl computers equipped with pproprite disply devices enbled implementtion of truly interctive imge processing. Flexibility of progrmmble digitl computers mke them the idel vehicle for imge enhncement. From the point of view of implementtions, imge enhncement methods cn be clssified into four ctegories: gry scle, or histogrm mnipultion methods, imge spectr mnipultion methods, geometricl trnsformtion methods nd disply methods. Gry scle, or histogrm mnipultion methods implement different modifictions of imge disply trnsfer function. We will review them in Sect... Imge spectr mnipultion methods implement modifiction of imge spectr in different bses, most frequently DFT nd DCT ones. They re reviewed in Sect..3. Specil informtion disply methods for imge enhncement re briefly outlined in Sect..4.. Gry level histogrm modifiction methods One of the fetures tht detere the cpbility of vision to detect nd nlyze objects in imges is contrst of objects in their brightness with respect to their bckground. While globlly imge my utilize the entire dynmic rnge of the disply device, loclly, within smll windows imge signl hs very frequently much lower dynmic rnge. Smll locl contrsts my mke it difficult to detect nd nlyze objects. The simplest nd the most strightforwrd wy for mplifiction of locl contrsts is stretching locl dynmic rnge of imge signl in sliding window to the entire dynmic rnge of the disply device. The two computtionlly inexpensive methods for stretching locl contrsts re mx- stretching nd locl dynmic rnge normliztion by locl men nd vrince. They re described correspondingly s ˆ =, (.) mx
nd where nd mx, if ˆ~ mx ˆ = ˆ~,, ˆ~ k l if mx, (.) ˆ~, if ( W, W ) ˆ~ men mx = g +, stdev (.3) ˆ re input nd output imge smples in the window position ( l ) mximl nd iml signl vlues within the window, device dynmic rnge, W ( W, W ) men = W + W + m is locl men vlue nd stdev ( )( ) = W n= k,, mx,, re mx nd re boundries of the disply W W k m, l n, (.4) ( W, W ) ( ), (.5) W W = ( )( ) + + men k m, l n W W m= W n= W is locl stndrd devition within the window.locl dynmic rnge normliztion by locl men nd vrince is illustrted in Fig.., upper row. Alterntive methods of mplifiction of locl contrsts re methods of the direct modifiction of imge histogrms: globl nd locl histogrm equliztion nd P-histogrm equliztion. q ˆ with Histogrm equliztion converts n imge {, } with histogrm h ( ) into n imge { } uniform histogrm by mens of the following gry level trnsformtion: ˆ = k, l ( qmx q ) h q= q mx h q= q k l ( q) ( q) + q, (.6) where q nd qmx re iml nd mximl quntized vlues of imge signl (for 56 quntiztion levels these re 0 nd 55). It follows from Eq..6 tht the slope of the histogrm equliztion trnsfer function ˆ is proportionl to the histogrm vlue: ( ) ˆ h ( q) q=. (.7) This mens tht histogrm equliztion results in mplifiction of contrsts tht, for ech gry level, is proportionl to its histogrm vlue, or to the gry level crdinlity. P-histogrm equliztion defined by the eqution ˆ = k, l ( qmx q ) q= q qmx q= q [ h( q) ] P [ h( q) ] P + q, (.8) is nturl generliztion of the histogrm equliztion tht mkes it more flexible through the use of user-defined prmeter P. If P = 0, P-histogrm equliztion is equivlent to the utomtic stretching imge dynmic rnge from (, mx ) to ( q, q mx ). P = corresponds to histogrm equliztion. Intermedite vlues 0 P result in more soft contrst enhncement tht is frequently better perceived visully. The described histogrm modifiction techniques cn be pplied both globlly nd loclly. In the ltter cse, the modifiction is locl dptive. Imge unshrp msking, histogrm equliztion nd P-histogrm equliztion re illustrted in Fig.. nd Fig. compres globl histogrm equliztion nd P-histogrm equliztion nd
their corresponding trnsfer functions (compre them with the imge histogrm). Note tht the P- histogrm equliztion my be regrded s n implementtion of the nonliner pre-distortion for optiml compnder quntiztion (Lect. 5). Initil imge (56x56) Normliztion of locl men nd stndrd devition in the window 33x33 Locl histogrm equliztion, SzW=33x33 Locl histogrm P-equliztion, SzW=33x33; P=0.5 Initil imge (56x56) Locl histogrm equliztion, SzW=5x5 Figure.. Locl histogrm modifiction by mens of stndrdiztion of locl men nd vrince (upper row) nd by P-histogrm equliztion (middle nd bottom rows)
000 500 000 500 0 50 00 50 00 50 Imge histogrm Initil imge 50 00 50 00 50 0 50 00 50 00 50 Histogrm equliztion trnsfer function The imge fter histogrm equliztion 50 00 50 00 50 0 50 00 50 00 50 Histogrm P-equliztion trnsfer function (P=0.) The imge fter P-histogrm equliztion (P=0.) Figure. Globl histogrm modifiction by mens of histogrm equliztion nd P-histogrm equliztion. Left column (from top to bottom): initil imge, result of histogrm equliztion nd P- histogrm equliztion (P=0.). Right column, from top to bottom: histogrm of the initil imge nd corresponding gry level trnsformtion trnsfer functions
.3 Imge spectr modifiction methods Two imge spectr mnipultion methods for locl contrst enhncement re the most simple in the implementtion: unshrp msking nd nonliner spectr coefficients trnsformtions. They re implemented, respectively, in signl domin nd trnsform domin filtering. Unshrp msking is defined by the eqution: ( W ) ( ), W ˆ = k, l + g k, l men, (.9) ( W ) where men, W W + W pixels defined by is the imge locl men in the window of ( )( + ) Eq..3 nd g is user defined locl contrst mplifiction prmeter. Window size prmeters W nd W re lso user defined prmeters tht re commensurte with the size of the objects to be enhnced. Modifiction of imge spectr tht results from pplying unshrp msking is detered by the unshrp msking frequency response. As it follows from Eq..9, it is, for imge of N smples, equl to: N [ sincd( W ; N ; r) sincd( W ; N r) ] η r, s = + g ; (.0) where ( ; ; ) sincd is the discrete sinc-function. Fig..3 illustrtes -D sections of unshrp msking frequency responses for windows of 3 nd 5 pixels...8 W=5 W=3.4 Normlized frequency Figure.3. -D sections of unshrp msking frequency responses for window sizes 3 nd 5 pixels with mplifiction coefficient g= Unshrp msking is non-dptive locl contrst enhncement procedure. With unshrp msking, the degree of mplifiction of imge high frequencies is the sme for ny imge. An lterntive nd dptive method for modifiction of imge spectr tht results in locl contrst enhncement is P-th lw nonliner modifiction of bsolute vlues of imge spectr coefficients. It is described by the eqution α r, s P ˆ α = α, (.9) where { α r,s } nd { r,s } 0. 0.4 0.6 0.8 r, s r, s α r, s ˆα re initil nd trnsformed imge spectrl coefficients in selected bsis, respectively, nd P is user defined prmeter. As trnsform, usully DFT or DCT re used. When 0 P <, this modifiction redistributes energy of spectrl coefficients in fvor of low energy coefficients. The degree to which individul coefficients re mplified depends now on the imge spectrum. Such spectrum modifiction my be pplied globlly to the spectrum of the entire imge nd loclly in user specified sliding window to spectr of the window smples in ech position of the window. In the ltter cse, the modifiction is locl dptive. In Fig..4 one cn compre imge locl contrst enhncement by mens of unshrp msking nd globl nd locl nonliner modifiction of imge DCT spectrum.
Initil imge (56x56) Unshrp msking in the window 5x5 DCT globl spectrum enhncement, P=0.5 DCT locl spectrum enhncement, in the window 5x5; P=0.5 Figure.4 Unshrp msking nd nonliner spectrum modifiction for imge enhncement 8.4 Using color, stereo nd dynmicl vision for imge enhncement The bsic principle of using color, stereo nd dynmic cpbilities of vision is strightforwrd. Imge is processed to produce severl output imges tht represent certin severl fetures of the input imge. For instnce, this processing my be imge sub-bnd decomposition, imge locl histogrm P- equliztion using severl different window sizes, edge enhncement nd extrction with different lgorithm prmeters, object detection, to nme few. The obtined set of imges my then be used to generte nimted rtificil movie by using generted imges s movie frmes, or to generte rtificil color imges by representing combintion of three generted imges s red, green nd blue components of the color imge tht is displyed for visul nlysis (Fig..5), or to generte rtificil stereoscopic imges for left nd right eyes from couples of the obtined imges. In the ltter cse, one of the processed imges or the initil imge is treted s reflectnce mp of n rtificil 3-D surfce nd nother s its depth mp. Using the reflectnce mp nd the depth mp, one cn generte pir of imges for right nd left eyes by introducing to every pixel of the reflectnce mp horizontl shift proportionl the vlue of the depth mp for this pixel. Fig..6 illustrtes n exmple of such n rtificil stereo imge generted from ir photogrph of 56x56 pixels treted s reflectnce mp nd n imge of its locl mens in the window of 45x45 pixels treted s depth mp.
Originl imge Unshrp msked imge (medn35x35) Unshrp msked imge (medn7x7) Colored imge of the bove 3 components Fig..5 Imge coloriztion
Input imge (56x56) Input imge locl men in the window 45x45 pixels (imge size is 56x56) Artificil stereoscopic imges for left nd right eye generted using the imge locl men s depth mp. Figure.5. Stereo visuliztion of n imge nd its locl men. One cn observe 3_d imge using stereoscope or by looking t the imges with squinted eyes. In the ltter cse, one should try to squint eyes until two pirs of imges seen with ech eye overlp into 3 imges. Then the centrl of those three imges with be seen s 3-D imge.
Summry Imge enhncement in imge processing imed t ssistnce to humn opertor in visul imge nlysis. Imge enhncement methods cn be clssified into four groups: histogrm modifiction methods imge spectr modifiction methods unshrp msking methods visuliztion methods Histogrm modifiction methods, imge spectr modifiction nd unshrp msking methods llow enhncing imge locl contrsts. Visuliztion methods imed t exploiting ll cpbilities of visul system to perceive visul informtion. Questions for self-testing. Explin wht is imge dynmic rnge stretching.. Explin wht is histogrm equliztion nd in wht sense it improves imge visul qulity. 3. Explin wht is P-histogrm equliztion nd in wht sense it improves imge visul qulity 4. Explin why locl histogrm modifiction is dvntgeous to the globl one. 5. Explin bsic principle of imge unshrp msking. How unshrp msking modifies imge spectrum? 6. Describe methods of imge spectrum nonliner modifiction nd explin when nd why they improve imge visul qulity. Home work: Demonstrte, using Mtlb functions conv nd fft, imge enhncement by unshrp msking nd by nonliner spectrum modifiction.