Lecture 13: High-dimensional Images

Transcription

1 Lec : Hgh-dmensonal Images Grayscale Images Lecture : Hgh-dmensonal Images Math 90 Prof. Todd Wttman The Ctadel A grayscale mage s an nteger-valued D matrx. An 8-bt mage takes on values between 0 and Commonly, satelltes have a grayscale camera wth a wde frequency response. These mages are called panchromatc mages. Color Images Color mages consst of three separate D matrces representng Red, Green, & Blue lght (RGB). So each pxel s a -tuple. e.g. (55,0,55) = PURPLE Alternate "Color" Sensors We can buld camera sensors to pck up any wavelength on the EM spectrum. Source: Multspectral Images A multspectral mage s typcally a -6 band mage: RGB + one or more nfrared bands. False Color Images If we pck out the RGB bands, we can dsplay the "true" color composte mage. Ths corresponds to what we would see wth our eyes. mshow( A(:,:, :) ); We can pck any bands to make a "false" color composte mage. Ths sometmes reveals nterestng nformaton. mshow( A(:,:, [,,]) );

2 Lec : Hgh-dmensonal Images False Color Images For example, vewng the false color mages can locate wldfres that were obscured by smoke n the vsble bands. Hyperspectral Images A hyperspectral mage typcally has ~00 bands, each band representng the response to a precse wavelength of lght. So each pxel s a 00- dmensonal sgnal. The sgnal can potentally dentfy the materal present. Tree Asphalt Source: 0-Band Hyperspectral Image of a Walmart n Texas Hyperspectral Sgnatures A representatve sgnal for a materal s called a hyperspectral sgnature. We can form these sgnatures nto a spectral lbrary. Source: Remote Sensng Source: Applcatons Geoscence & Remote Sensng Agrcultural & geologcal surveys Envronmental montorng Mltary applcatons, e.g. target detecton Forenscs Analyss of artwork Document verfcaton Medcal Imagng Detecton of certan types of cells

3 Lec : Hgh-dmensonal Images Dffculty wth spectral mages It s dffcult to buld a camera wth both hgh spectral and spatal resoluton. As the camera sensors are fne-tuned to a specfc wavelength of lght, the sensor loses spatal accuracy. So obtanng the extra mage bands comes at the prce of "bgger pxels". Spatal vs. Spectral Resoluton Suppose we have a tny camera that can hold sensors. Panchromatc Hyperspectral (false color) Typcal Resolutons Panchromatc ( band): Multspectral (-0 bands): Hyperspectral (00-00 bands): Spectral Response To get the hgher spatal accuracy the panchromatc sensor has a very wde spectral response. Sensor Response B G R IR Pan Target Detecton We want to locate a specfc materal or object n an mage A. Suppose we have a target sgnature T from our spectral lbrary. We examne every pxel (x,y) n our mage A and see whch pxels best match the target T. mn,, λ Dealng wth background clutter s problematc (matched flter). Target Detecton The choce of dstance metrc makes a bg dfference. Eucldean dstance: Cosne dstance:, =, = Whch would be better for hyperspectral mages? Classfcaton Snce each pxel has a vector of length ~00, K-Means actually works very well on hyperspectral mages. We typcally get better results usng the Cosne dstance metrc, rather than Eucldean. K-Means K-Means + Mode Flter

4 Lec : Hgh-dmensonal Images Agrcultural Survey Indan Pnes, Iowa Geologcal Survey Cuprte, Nevada Anomaly Detecton Another task s to see whch pxels "do not belong" n the mage. For example, we'd want to pck out a vehcle n the mddle of a desert. In ths case, we do not know the target sgnature T. The RX Detector We could look at a small neghborhood around each pxel. If the sgnal at the center of the pxel s dfferent than the average sgnal of the neghborhood, then we would declare that pxel an anomaly.,, > Pxel x,y s anomaly Ths s called the RX detector and typcally uses the Mahalonobs dstance metrc (Reed-Xu, 990). The neghborhood sze and threshold T of the RX detector need to be set carefully. Anomaly Detecton RIT put a aeral hyperspectral mage onlne and challenged researchers to fnd the peces of cloth n the felds. Spectral Unmxng Snce each pxel s a large patch on the ground, t s possble that each pxel contans multple materals. e.g. some grass, drt, and car Spectral unmxng (demxng) s the process of tryng to determne what materals and how much of them are n each pxel.

5 Lec : Hgh-dmensonal Images Spectral Unmxng Gven a spectral lbrary matrx L, where each column of L s a spectral sgnature. We call the sgnatures n the lbrary L the endmembers for the mage. Typcally, these endmembers are chosen manually. Spectral Unmxng For a gven sgnal f, we want to fnd the abundances of each materal specfed by L. mn Ths s called non-negatve least squares (NNLS) mnmzaton. Pan-sharpenng (Image Fuson) Pan-sharpenng s the process of fusng these two mages nto one mage wth hgh spatal and spectral qualty. + = + IHS Pan-sharpenng The standard pan-sharpenng technque s the IHS (Intensty-Hue-Saturaton) transform. For a multspectral mage M and a panchromatc mage P, compute = M + P I F I = M + M + M + M M F Multspectral Panchromatc Pan-sharpened Image IHS Pan-sharpenng We can generalze the IHS model to arbtrary coeffcents. F = M + P I I = α + M+ α M + αm α M Ideally, these coeffcents would be derved from nformaton about the sensor. (Cho-Cho-Km, 008) suggested expermentally determned values for the IKONOS satellte I = 0.M + M + 0.5M M Adaptve IHS Pan-sharpenng Wthout knowng the sensor detals, can we reverse engneer the coeffcents from the mage? We want to approxmate the panchromatc mage as a lnear combnaton of the multspectral bands: I = αm+ α M + αm + αm Pan We calculate the coeffcents whch mnmze the followng functon: E( α) = ( ( αm ( x)) P( x)) + γ (max(0, α )) x Furthermore, we note that the mage colors should match away from edges. If e(x) s an edge detector wth e=0 away from edges, then we want F=M where e=0 and use the standard IHS on the edges. ( ) λ F = M+ e( x) P I e( x) = exp P 5

6 Lec : Hgh-dmensonal Images Adaptve IHS Pan-sharpenng The adaptve IHS method gves the same spatal qualty, but the spectral nformaton (colors) match the orgnal mage better (Rahman-Strat-Merkurjev- Moeller-W, 00). Student-Wrtten Software The REU student team wrote Matlab software whch runs several pansharpenng methods and evaluates the performance under a sute of qualty metrcs. 6