Time in Hyperspectral Processing: a Temporal based Classification Approach

Transcription

1 The 6 th IEEE Intenational Confeence on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications Septembe 2011, Pague, Czech Republic Time in Hypespectal Pocessing: a Tempoal based Classification Appoach Blanca Piego, Daniel Souto, Fancisco Bellas, Richad J. Duo and Fenando López-Peña Gupo Integado de Ingenieía, Univesidade da Couña Mendizábal s.n., Feol, Spain {blanca.piego, dsouto, fan, ichad, flop}@udc.es Abstact This pape deals ith the poblem of classifying pocesses using the tempoal infomation in the sequence of hypespectal images that ae obtained as they tae place. That is, taing into account the tempoal evolution of the pocess in the discimination that must be made. To this end e have consideed a paticula type of atificial neual netos ith tainable delays in thei synapses. The classification scheme is studied and applied to the case of esin cuing pocesses. Seveal test cases involving diffeent popotions of esin components as ell as cetain envionmental conditions such as humidity ee ceated and the system as tested ove them poducing vey pomising esults. Keyods Tempoal Delay Based Atificial Neual Netos; Hypespectal imaging;real Time pocessing. I. INTRODUCTION Hypespectal imaging has taditionally been used in emote sensing instuments. The fist hypespectometes ee developed fo imaging fom high flying aibone platfoms o satellites. They ee usually lage instuments ith complex deployment and handling chaacteistics typically un by space agencies o othe lage povides of imaging esouces. Hoeve, due to the populaization of imaging sensos and the advances in digital photogaphy and video captue technology, in the last ten yeas e have seen the implementation of many smalle designs and platfoms that have opened up ne application domains, especially in close up inspection tass such as medical imaging o quality contol in pocessing plants, leading to a fluy of activity in eseach into ne algoithms and stategies that ould adapt to these ne application aeas. In hypespectal imaging systems, fo each spatial esolution element (pixel), data is collected ith high spectal esolution ove the electomagnetic spectum in the nm band (visible to nea infaed). It is commonplace to use 50 to 250 spectal bands ith bandidths in the 5 to 20 nm ange. The lage amount of infomation that any hypespectal image povides pemits a detailed desciption of the spectal signatue fo each pixel in the image, thus geatly impoving the ability to detect and identify individual mateials o classes ith espect to othe emote sensing techniques. Unfotunately, on the othe hand, these lage amounts of data ae the souce of some of the main challenges cuently associated ith hypespectal imaging: efficient handling of high data ates and accuate and fast segmentation of images. Hypespectometic systems epesent today a matue technology, and have been applied to many diffeent instances. Typical applications of emote hypespectal sensing ae elated to vegetation monitoing [1]-[3], taget detection [4], [5], and many othes. In addition to emote sensing applications, hypespectal imaging is also being used in close up inspection tass such as medical imaging o pocessing plant visual inspection [6], [7]. Thee is pesently a ide open field fo ne applications using hypespectal imaging at close and mid-ange that ould mae these instuments much moe accessible and ubiquitous. Hoeve, it does equie an impulse in to main diections fo the technology to become commonplace and popula. On one hand, hypespectometes must be made moe affodable, small, light and ugged and, on the othe, they must be made as autonomous as possible and vey easy to use by nonexpets. This obviously equies developments in the hadae and contol stuctue and motivates the eseach into efficient and accuate methods to pocess this data. In paticula, as autonomy and data pocessing esilience ould be a vey impotant aspect of this ne geneation of sensos, the intoduction of moe intensive computational intelligence techniques that allo the systems to be used in less specialized applications than those cuently contemplated becomes a necessity. Accoding to Manolais et al [8], most algoithms used in hypespectal applications can be gouped into fou categoies: change detection, taget/anomaly detection, classification, and spectal unmixing. Up to no, almost all eseach effots have focused on the analysis of static hypespectal images in tems of the last thee categoies. Hoeve, in the last fe yeas, as a consequence of impovements in hypespectometes, it is becoming feasible to captue a continuous flux of images alloing one to loo at hypespectal infomation dynamically. Some algoithms have been designed using tempoal infomation [9] [10], mostly fo the detection of changes, but still ithout eally using the evolution seen /11/$ IEEE 341

2 in the fame sequence as a classification stategy. Hee, ou objective is to fomulate techniques that pemit addessing the poblem of using tempoal sequences in the classification pocess and not only to detect changes. In fact, thee ae many poblems hee infomation fom a single image povides ambiguous classification esults and it is the integation of the evolution of the subject in time that eally povides an unambiguous classification. This poblem has been tacled in the image pocessing field dealing ith one o thee dimensional (RGB) images by using video sequences to obtain bette classifications, to eliminate noise o to impove image esolution. Hee the objective is to extend this o to the ealm of high dimensional images, such as those obtained fom hypespectal sensos, though the use of ne techniques and algoithms that ae adapted to the high dimensionality involved. In paticula, in this pape e mae use of including specific types of neual neto achitectues and taining pocedues fo the tempoal pocessing of this type of images. Figue 1. Expeimental setup (top) and schematic vie of the hypespectomete developed II. SENSING SYSTEM The senso used in this o has been developed by ou goup as a small, light and easy to deploy hypespectal senso that as designed to exploe the 400 to 1000 nm band ith a spectal esolution of up to 1040 bands. Figue 1 displays the expeimental setup that as used and a schematic vie of the hypespectomete. This senso is capable of captuing up to 47.2 hypespectal 1040 pixel lines pe second ith the spectal depth indicated befoe. III. TEMPORAL BASED PIXEL CLASSIFICATION To addess the poblem of pefoming classifications taing into account the tempoal evolution of the specta of the pixels, e have chosen to use a neual neto achitectue developed in ou goup that is called Tempoal Delay Based ANN as ell as its taining algoithm (TDBP). The achitectue and taining algoithm of the atificial neual neto e conside ee intoduced in [11], [12]. The neto consists of seveal layes of neuons connected lie a Multiple Laye Pecepton (MLP), that is, evey neuon of one laye is connected though a synapsis to evey neuon of the next laye. Each neuon pefoms a sum of its inputs and passes these values though some non-linea function (in this case a sigmoid). It is obviously a feed-foad neto. The only diffeence ith espect to a taditional MLP is that the synapses ae epesented by to tainable paametes: the classical eight tem and a delay tem. Consequently, no the synaptic connections beteen neuons ae chaacteized by a pai of values, ( ij, ij ), hee ij is the eight descibing the ability of the synapsis to tansmit infomation fom neuon i to neuon j and ij is a delay, hich can be taen as an indication of the length of the synapsis beteen neuons i and j, the longe it is, it ill tae moe time fo infomation to tavese it and each the taget neuon. The taining algoithm is descibed in detail in [11] hee it as used on one-dimensional signals. Hee e ill just povide a summay of its main points and ho it is used on multidimensional signals. The main assumption duing taining in this algoithm is that each neuon in a given laye can choose the delay it ishes to impose on its inputs. Time is discetized into instants, each one of hich coesponds to the peiod of time beteen an input to the neto and the next input. Evey neuon of the neto computes an output each instant of time. In ode to choose fom the possible inputs to a neuon the ones e ae actually going to tae as input it in a given instant of time, e add a selection function to the pocessing of the neuon. This selection function can be something as simple as: 1 i j ij (1) 0 i j 342

3 Maing output of neuon in instant t: O t F N t i0 j0 j( t i h i ) ij (2) hee F is the activation function of the neuon, h ij is the output of neuon i of the pevious laye in instant j and i is the eight of the synapsis beteen neuon i and neuon. The fist sum is ove all the neuons that each neuon (those of the pevious laye) and the second one is ove all the instants of time consideed. The esult of this function is the sum of the outputs of the hidden neuons in times t- j (hee j is the delay in the synapsis) eighed by the coesponding eight values. This detemines the output of evey neuon as a function of the outputs of the neuons in the pevious laye and the eights and delays in the synapses. To tain these eights and delays e have esoted to a modification of the basic gadient descent algoithm employed in taditional bacpopagation, taing into account the delay tems hen computing the gadients of the eo ith espect to eights and delays. Thus, the gadient tems can be itten as: E j E j h (3) j h j( t j ) h j( t j ) j( t j1) in hich E is the squaed eo fo all the taining vectos and E O O ONet (4) 2( O T ) F( ONet ) (5) hee T is the taget output, O the one eally obtained and ONet is the combination of inputs to neuon, hen e conside output neuons and: E E E hnet hnet j( t j ) ( ) j hnet j j E hnet hnet j j I F (6) I j t ) I j( t 1) F( hnet ) ( j j (7) hee index epesents the neuon of the next laye, hethe output o hidden. We assume: and E F hnet ( ) hnet (8) hnet j N t i0 n0 n( t i, ) fo the case of neuons in a hidden laye. Assuming thee is cetain continuity in the tempoal vaiation of the outputs of the neuons, the deivative in hnet of equation (9) has been discetized in ode to obtain (7). Summaizing, by discetizing the time deivative simple expessions may be obtained fo the modification of the eights and delays of the synapses in an algoithm that is basically a bacpopagation algoithm hee the activation function of the neuon has been modified to pemit selecting delays o, in ode ods, choosing fom the list of pevious outputs of the neuon en the pevious laye. By adding input neuons to the neto, any dimensionality of the signals can be chosen. IV. j i EXPERIMENTAL RESULTS The objective of this pape is to demonstate ho by consideing the tempoal sequence of hypespectal images taen duing the evolution o a pocess and appopiately pocessing them e can glean infomation that is had o impossible to obtain using static images. To test this hypothesis e have pefomed a seies of expeiments elated to quality contol of the dying o cuing pocesses afte applying diffeent suface coatings. The basic idea is to detemine if the esulting suface coating meets the quality levels as a function of the tempoal dying sequence. To this end, e mae use of the sequence of hypespectal images of the poducts as time pogesses. To obtain adequately cued esins it is necessay to mix cetain components (in this case to) in the appopiate popotions and let the esin cue fo a peiod of time unde adequate envionmental conditions. In fact, this is eally a spatial-tempoal poblem as the classification may be diffeent fo diffeent aeas of the esin suface as shon in figue 2. In the end, it is the detemination of hethe the esin has cued appopiately in evey point that is impotant. This is hee the use of hypespectal imaging maes sense. Figue 2. Thee instants in the evolution of the dying pocess as obtained by composing bands fom the visible specta. Seveal diffeent mixtues ee made unde diffeent humidity conditions and ith diffeent popotions of components. One of them, hich ill be called coect cuing pocess in the gaphics and comments that ill follo, has been taen as the efeence mixtue and conditions fo hich the cuing pocess is optimal. The I in (9) 343

4 emaining cases ee consideed suboptimal. Table 1 povides a bief desciption of the diffeent expeiments. Using the hypespectometic setup descibed befoe, the mixtues ee made and left to cue fo 500 seconds. Images ee taen by the hypespectomete of a line of 1000 points tavesing the mixtues at a ate of close to one image pe second (485 images in 500 seconds). The specta coesponding to the points obtained ee nomalized and coected using as a efeence the bacgound aea hee thee as no mixtue. The objective as fo the pocessing system to disciminate the optimal cuing pocesses fom suboptimal ones as soon as possible. Figue 2 pesents the visual appeaance of the cuing pocess as it pogesses. As a fist test in ode to validate the need to pefom a time based pocessing, e evaluated the possibility of disciminating the coectly cued esin fom the est once the hole cuing pocess as finished. Figue 3 displays the specta coesponding to points on thee diffeent expeiments once all of them had cued. As shon, the specta ae pactically identical and it as impossible to obtain any ind of coect classification using only this infomation. Table I. CURING PROCESSES Figue 3. Specta of the final esins afte cuing has ended fo diffeent cases (a coect and thee incoect cuing pocesses). As a consequence, e consideed the tempoal evolution and intoduced the delay based neual netos as classifies. Figue 4 displays a sequence of specta fo to cases, one the coect cuing pocess and anothe one that is incoect. To tain these netos e used 80% of the samples obtained as descibed above. The emaining 20% ee used as test samples. Fo each expeiment, the tempoal sequence as discetized into ten time intevals by aveaging the specta ithin each inteval and the specta themselves ee binned into 16 bands. A 16 input TDANN as used and diffeent popotions of the tempoal sequence ee used fo taining and testing in ode to detemine ho much tempoal infomation fom the cuing pocess as necessay in ode to achieve 100% discimination (in this case positive classification ould etun a value of 0.5 and a negative of -0.5). This pocess as stated using the fist 36 seconds of cuing and, as shon in figue 5, even though ith this infomation a cetain ode in tems of quality seems to be pesent, the discimination is still not good enough and thee is a cetain level of confusion. This level deceases if moe time is consideed as shon in the gaphs of figue 6, coesponding to 42, 47, 53 and 480 seconds of sampling. In fact, in cases ith moe than 47 seconds of sampling, all the coectly cued samples (blue dots) ae appopiately classified ith a value above zeo and all the est of incoect cuing pocesses ae assigned values 344

5 belo zeo by the neto. Consequently, by monitoing the pocesses fo fifty seconds e ae able to disciminate pefectly if it is going to cue coectly o not. Obviously, if moe time is consideed, the esults become even bette as shon in the bottom gaph coesponding to the esults poduced by the neto consideing 48 seconds, that is, almost the hole 500 second time inteval. Figue 4. Evolution of the specta duing the cuing pocess fo the coect one (top) and that of expeiment 4.3 (bottom). Figue 5. Results poduced by the TDANN hen using a sequence of 10 aveaged specta fom the fist 36 seconds of the cuing pocess. Figue 6. Same as figue 5 (including pocess labels) but taing into account, fom top to bottom, 42, 47, 53 and 480 seconds of cuing pocess. 345

6 Finally, and to demonstate ho fast this algoithm can lean, in Figue 7 e display the evolution of the Mean Squaed Eo as taining taes place fo some of the pevious cases. The algoithm only equies aound 150 epochs of taining to achieve vey lo eos in those cases hee the tempoal infomation is enough to establish a eliable discimination. In fact, this numbe deceases to aound 70 epochs in the case of the 480 second sampling. V. CONCLUSIONS In this pape e have shon ho the tempoal evolution of the specta ithin hypespectal images can be used in ode to classify the quality of pocesses even hen the final specta ae almost the same fo coectly pefomed pocesses o pocesses ith deficiencies. To this end e have consideed the application of synaptic delay based atificial neual netos to the tas. It has been shon that this type of appoach can disciminate vey clealy beteen coectly pefomed cuing pocesses and pocesses that pesent poblems due to ong component mixtues o humidity. In fact, this discimination can be pefomed using just the fist fe seconds of the cuing pocess. We ae no in the pocess of extending these esults to othe pocesses in hat e thin is a vey pomising appoach to expand the use of hypespectal imaging. ACKNOWLEDGMENT This o as patially funded by the Xunta de Galicia and Euopean Regional Development Funds though pojects 09DPI012166PR and 10DPI005CT. Figue 7. Taining Eo fo the taining pocesses using 47, 53 and 480 seconds of sampling REFERENCES [1] Jens Oldeland, et al. Combining vegetation indices, constained odination and fuzzy classification fo mapping semi-natual vegetation units fom hypespectal imagey, Remote Sensing of Envionment, 114(6): , June [2] V. Heiinen, T. Toola, J. Painen, I. Kopela, and T. Jaaselainen. Simulated multispectal imagey fo tee species classification using suppot vecto machines, Geosci. Rem. Sens., IEEE Tans., 48(3): , [3] G.H. Miti and I.Z. Gitas. Mapping postfie vegetation ecovey using EO- 1 hypeion imagey, Geosci. Rem. Sens., IEEE Tans., 48(3): , [4] D.E. Ba, K. Woloelsy, Y. Sisi, Z. Figov, A. Michaeli, Y. Vaynzof, Y. Abamovitz, A. Ben-Dov, O. Yaon, L. Weizman, and R. Ada. Taget detection and veification via aibone hypespectal and High-Resolution imagey pocessing and fusion, Sensos Jounal, IEEE, 10(3): , [5] J. Boadate and R. Chellappa. Hybid detectos fo subpixel tagets, Patten Analysis and Machine Intelligence, IEEE Tans., 29(11): , [6] Hioshi Oamoto and Won Su Lee. Geen citus detection using hypespectal imaging, Computes and Electonics in Agicultue, 66(2): , May [7] L. Liu, M.O. Ngadi, et al. Categoization of po quality using gabo filte-based hypespectal imaging technology, Jounal of Food Engineeing, 99(3): , August [8] D. Manolais, C. Siacusa and G. Sha, Hypespectal subpixel taget detection using the linea mixing model, IEEE Tans. Geoscience and Remote Sensing, vol. 39, pp , [9] Muzina, Maina V.A. and Faell, J. Paul. Dynamic Hypespectal Imaging. Nondestuctive Detection, vol. 5679, pp [10] Blacbun, Joshua, Featue Aided Tacing ith Hypespectal Imagey, Signal and Data Pocessing of Small Tagets, vol. 6699S, [11] Duo, R.J., Santos J. Discete Time Bacpopagation fo taining Synaptic Delay Based Atificial Neual Netos, IEEE Tansactions on Neual Netos. vol.10, pp [12] Santos, J. and Duo, R.J. Influence of Noise on Discete Time Bacpopagation Tained Netos, Neuocomputing. vol. 41, pp (2001). 346