Multiview hyperspectral topography of tissue structural and functional characteristics

Transcription

1 Multiview hyperspectrl topogrphy of tissue structurl nd functionl chrcteristics Shiwu Zhng,b, Peng Liu, Jiwei Hung b, Ronld Xu,b Deprtment of Precision Mchinery nd Precision Instrumenttion, University of Science nd Technology of Chin, Hefei, Chin, b Deprtment of Biomedicl Engineering, The Ohio Stte University, Columbus, OH ABSTRACT Accurte nd in vivo chrcteriztion of structurl, functionl, nd moleculr chrcteristics of biologicl tissue will fcilitte quntittive dignosis, therpeutic guidnce, nd outcome ssessment in mny clinicl pplictions, such s wound heling, cncer surgery, nd orgn trnsplnttion. However, mny clinicl imging systems hve limittions nd fil to provide noninvsive, rel time, nd quntittive ssessment of biologicl tissue in n opertion room. To overcome these limittions, we developed nd tested multiview hyperspectrl imging system. The multiview hyperspectrl imging system integrted the multiview nd the hyperspectrl imging techniques in single portble unit. Four plne mirrors re cohered together s multiview reflective mirror set with rectngulr cross section. The multiview reflective mirror set ws plced between hyperspectrl cmer nd the mesured biologicl tissue. For single imge cquisition tsk, hyperspectrl dt cube with five views ws obtined. The five-view hyperspectrl imge consisted of min objective imge nd four reflective imges. Three-dimensionl topogrphy of the scene ws chieved by correlting the mtching pixels between the objective imge nd the reflective imges. Three-dimensionl mpping of tissue oxygention ws chieved using hyperspectrl oxygention lgorithm. The multiview hyperspectrl imging technique is currently under quntittive vlidtion in wound model, tissue-simulting blood phntom, nd n in vivo biologicl tissue model. The preliminry results hve demonstrted the technicl fesibility of using multiview hyperspectrl imging for three-dimensionl topogrphy of tissue functionl properties. Keywords: multi-view, hyperspectrl, oxygention, topogrphy, reconstruction. 1. INTRODUCTION Accurte nd in vivo chrcteriztion of structurl, functionl, nd moleculr chrcteristics of biologicl tissue will fcilitte quntittive dignosis, therpeutic guidnce, nd outcome ssessment in mny clinicl pplictions, such s wound heling, cncer surgery, nd orgn trnsplnttion. Trditionlly, structurl chrcteristics of wound tissue cn be mesured with stndrd phntom or trnsprent pper, which re invsive nd inconvenient[1]. Opticl techniques re non-invsive, rpid, nd quntittive for chrcterizing tissue topogrphy, which include multi-view imging nd opticl coherence tomogrphy (OCT) [2], etc. Multi-view imges utilize multiple viewpoints nd cptured multiple pictures of objective tissue. With three-dimensionl reconstruction lgorithm, the topogrphy of the tissue cn be obtined. OCT technique cn be used to obtin depth informtion of the tissue. With penetrtion depth up to 3 mm, OCT hs been used in the skin disese dignosis [3-5]. Functionl chrcteristics re very importnt for ccurte nd quntittive ssessment of tissue disorder, such s wound, cncer, nd trnsplnttion orgn. Tissue oxygention, perfusion, vsculr, nd inflmmtion re ll importnt functionl fctors to ssess tissue sttus, which is useful for further therpy. Hyperspectrl imging, lser speckle imging, lser Doppler imging, nd thermogrpic imging re effective techniques to cquire tissue oxygention mp, tissue vsculr, blood perfusion, nd het emission chrcteristics of tissue[6-9], some of the techniques hve been pplied in clinicl tretment. However, mny clinicl imging systems hve limittions nd fil to provide noninvsive, rel time, nd quntittive ssessment on structurl nd functionl chrcteristics of biologicl tissue simultneously in n opertion room. To overcome these limittions, we developed nd tested multiview hyperspectrl imging system, which integrted the multiview nd the hyperspectrl imging techniques in single portble unit. Four plne mirrors were cohered together s multiview reflective mirror set with rectngulr cross section. The multiview reflective mirror set ws plced between hyperspectrl cmer nd the mesured biologicl tissue. For single imge cquisition tsk, hyperspectrl dt cube with five views ws obtined. The five-view hyperspectrl imge consisted of min objective imge nd four reflective Optics in Helth Cre nd Biomedicl Optics V, edited by Qingming Luo, Ying Gu, Xingde D. Li, Proc. of SPIE Vol. 8553, 85530P 2012 SPIE CCC code: /12/$18 doi: / Proc. of SPIE Vol P-1

2 imges. Three-dimensionl topogrphy of the scene ws chieved by correlting the mtching pixels between the objective imge nd the reflective imges. Three-dimensionl mpping of tissue oxygention ws chieved using simple functionl diffuse reflectnce spectroscopy (fdrs) lgorithm. In the lgorithm, only four wve length spectroscopy, 530, 545, 570, nd 584 nm were pplied to obtin tissue oxygention point by point, which cn reduce scttering effects nd melnin bsorption [10]. Our multiview hyperspectrl imging technique is currently under quntittive vlidtion in wound model, tissue-simulting blood phntom, nd n in vivo biologicl tissue model. Our preliminry results hve demonstrted the technicl fesibility of using multiview hyperspectrl imging for threedimensionl topogrphy of tissue functionl properties. 2. MULTIVIEW HYPERSPECTRAL IMAGING SYSTEM A multiview imging system nd multispectrl imging system were integrted together to obtin the topogrphic nd oxygention chrcteristics of tissue. The integrted system is simple, cost-effective, nd portble tht cn be esily used in clinicl tretment. 2.1 Multi-viewpoint imging A multiview imging system consists of cmer nd squire mirror set. The squre mirror set is composed of four plne mirrors tht re cohered together with squire cross section. This configurtion llows us to obtin five different views of the trget within single imge t the exctly sme time. With 3-dimension reconstruction lgorithm, the three dimensionl superficil chrcteristics of the tissue cn be obtined. The bsic principle underlying the multi-viewpoint imging system is shown in Figure 1. The cmer is plced on one side of the Squre Mirror, nd its opticl xis is coincident with the xis of the squre mirror. Ech of the four mirrors round produces virtul cmer with the sme cmer prmeters on the other side. Thus the virtul cmers re symmetric with respect to the opticl xis, nd the distnce between rel cmer nd virtul cmer is equl to tht between the opposite mirrors. The five cmers will tke pictures of Point P from different viewpoints. As result, three-dimensionl topogrphy of the scene cn be chieved by correlting the mtching pixels between the objective imge nd the reflective imges. +. Squre Mirror 1 Opticl Axis Fig. 1 Principle of the multi-view imging system After obtining multiview photogrphy of the trget, we need to cut nd mirror the four imges tken by the virtul cmers in order to crry out the correltion mtching. Figure 2 gives n exmple of how to cut nd mirror the right imge. Assume tht Point A nd B re mtching points in the middle nd right imges respectively. After mirroring the photo horizontlly, we trnsform Point B in the originl right imge into Point B in the mirrored imge. Proc. of SPIE Vol P-2

3 B. u us Vis B(u,,vz) 'O(uo,vo) Middle Imge Right Imge Mirrored Right Imge Originl Imge Mirrored Imge Fig. 2 Imge cutting nd mirroring Assuming tht the coordinte of A in the originl imge is (u 1,v 1 ), the coordinte of B in the mirrored imge is (u 2,v 2 ), the coordinte of A in the middle imge is (u 1S,v 1S ), the coordinte of B in the mirrored right imge is (u 2S,v 2S ), the coordinte of the top left corner of the middle imge is (u 1C,v 1C ), the coordinte of A the top left corner of the mirrored right imge is (u 2C,v 2C ), the width of the originl imge nd middle imge is U 0 nd Column, then we get the trnsformtion of coordintes u1 u1 C + u1s u2 u2c + u2s (1) v1 v1 C + v1s v2 v2c + v2s U u + u + Column (2) 0 1C 2C D reconstruction As shown in Figure 1, Point P(x,y,z) is on the trget object. It cn be projected on the imges tken by the rel nd virtul cmers s Point A(u 1,v 1 ) nd B(u 2,v 2 ). As the prmeters of the rel nd virtul cmers re exctly the sme nd their opticl xis re prllel, x, y nd z cn be described like bu ( 1 u0) x u1 u2 bα x( v1 v0) y (3) α y ( u1 u2) bα x z u1 u2 where b is the distnce between rel nd virtul cmers, (u 0 v 0 ) denotes the center of the originl imge, shown in Figure 1. α x, α y, u 0 nd v 0 re the prmeters of the cmer, which cn be obtined through cmer clibrtion. Especilly, u 1 -u 2 is often clled disprity, nd in this cse, disprity u1 u2 2u1 C U0 + 2 Column + ( u1s u2s) (4) Therefore, (x,y,z) cn be written s bu ( 1C + u1s u0) x 2u1 C U0 + 2 Column+ ( u1s u2s) bα x( v1 C + v1s v0) y (5) α y[2u1c U0 + 2 Column+ ( u1s u2s)] bα x z 2u1 C U0 + 2 Column+ ( u1s u2s) For convenience, we define new prmeter, the disprity, in the following eqution, disprity u u (6) ' 1S 2S Proc. of SPIE Vol P-3

4 where disprity is the result of the correltion mtching between the middle nd mirrored right imges. As we cn see from Equtions 5 nd 6, the 3D coordintes (x,y,z) of point on the object is relevnt to its disprity tht is obtined from the correltion mtching. In the mtching process, when obtining disprity, we used locl nd window-bsed method for stereo mtching tht is bsed on normlized cross-correltion (NCC)[11]. Incorporting sub-pixel[12] computtion into the mtching mtrix to void the contouring effects cused by integer-vlued disprity estimtes, nd detecting the occluded res using cross-checking[13], we hve gretly refined the ccurcy of the disprity estimtes. 2.3 Multiview imging system With multiview reflective mirror set between the cmer nd objective tissue, multiview imging system cn be set up to obtin the three dimensionl topogrphy of the objective tissue. Furthermore, if the imging system cptures the hyperspectrl imges of the objective tissue, the functionl chrcteristics such s oxygen sturtion of the objective tissue cn be clculted. Finlly, the three dimensionl topogrphy with the functionl chrcteristics of the tissue cn be obtined immeditely nd conveniently. In order to obtin the hyperspectrl imges of the objective tissue, OL 490-VIS (Optronic Lbortories Llc) ws used s the light source tht provide wide wvelength rnge [500~900] nm t spectrl resolution of 2 nm. A light ring is connected to the tunble light source to provide uniform illumintion. After obtining the hyperspectrl imges of the objective tissue, simple functionl diffuse reflectnce spectroscopy (fdrs) lgorithm ws used to remove the effect of scttering nd melnin bsorption, nd reconstruct tissue oxygention mps bsed on hyperspectrl imges [10]. 3. EXPERIMENTAL VALIDATION OF THE SYSTEM Our multiview hyperspectrl imging techniques re currently under quntittive vlidtion in wound model, tissuesimulting blood phntom, nd n in vivo biologicl tissue model. Our preliminry results hve demonstrted the technicl fesibility of using multiview hyperspectrl imging for three-dimensionl topogrphy of tissue functionl properties. 3.1 Multiview imging of wound model A wound model (s shown in Figures 3 (c) (d)) with four chronic wounds tht represent different phses of the pressureinduced chronic wound ws used in the experiment to vlidte the multiview imging system. A cnon 5D mrk II cmer with EF mm lens ws used to cpture the multiview photogrphy of the wound model. One of the wounds ws mesured by the multiview imging system (s shown in Figure 3 () (e)) nd coordinte mesuring mchine (Sheffield Mesurement, CORDAX RS-30, s shown in Figure 3 (b)) respectively. With the lgorithm described in the previous section, we could obtin the 3D morphology of the wound. c e \ d b Fig.3 Multiview imging for 3D reconstruction of the chronic wound model: () multiview imging system for 3D reconstruction of the wound model; (b) CMM system for mesuring the topogrphy of the wound model; (c) the wound model; (d) four representtive wounds on the wound model; (f) photogrphy of the wound tken by the multiview imging system. Proc. of SPIE Vol P-4

5 Figure 4 displys the results of the multiview imging system nd CMM respectively. From the figures, we could conclude tht 3D topogrphy of the wound model tht is obtined from the multi-view imging system is very close to tht of the CMM system, which lso vlidte the multiview imging system preliminrily. b Io_ c M.570 o, MM Fig. 4 () Topogrphy of the wound model obtined from the multi-view system; (b) topogrphy of the wound model obtined from the multi-view system; (c) topogrphy of the wound model obtined from CMM. 3.2 Multi-view hyperspectrl imging of tissue-simulting blood phntom Another experiment ws conducted to combine the multi-view imging nd the hyperspectrl imging together. The experimentl setup is displyed in Figure 5 (). The multi-view hyperspectrl system consists of NIR high resolution cmer, multiview reflective mirror set, tunble light source, light ring, nd tissue-simulting blood phntom. A tissue-simulting blood phntom ws put in tube tht ws plced on tilt in the view field of the multiview hyperspectrl system to verify the performnce of the system. With the multiview system, the topogrphy of the tube ws reconstructed s shown in Figure 5 (b), nd the tissue oxygention mps were clculted bsed on hyperspectrl imges. The followings re the steps to mke tissue-simulting blood phntom: (1) dd 20 g milk powder into wter nd get 200 ml milk; (2) get mixture of blood by mixing 200 ml milk, 70 ml wter, nd 12 ml blood of helthy subject; (3) pour the mixture of blood into tube with 60 ml volume; (4) dd 1 g hydroxisile into 40 ml wter to get the sodium hydroxisile. After one hyperspectrl mesurement, we cn obtin the oxygention of the simulting blood phntom. Then we use syringe to get sodium hydroxisile nd slowly inject it into the tube. Thus new recipe of the blood phntom with decresed oxygention ws prepred for new mesurement by the multiview hyperspectrl imger. For ech recipe, the volume of the sodium hydroxisile tht be injected in the tube is 0, 0.1, 0.15, 0.15, 0.20, 0.25, nd 0.30 ml respectively. b c d e f g h i Fig. 5 Multiview hyperspectrl imging of tissue-simulting blood phntom: () experimentl setup; (b) topogrphy of the tube; (c)~(i) topogrphic oxygention mp of the tissue-simulting blood phntom from recipe 1 to 7, red color denotes higher oxygen sturtion, while blue color denotes lower oxygen sturtion. Figures 5 (c) ~ (i) disply oxygention mps of the tissue-simulting blood phntom with the topogrphy chrcteristics. From the figure, it cn be esily obtined tht the oxygen sturtion of the blood phntom decreses long with the volume of the sodium hydroxisile incresing. The result indictes tht the structurl nd functionl chrcteristics of the simulted blood phntom cn be esily chieved with the simple multiview hyperspectrl system. Proc. of SPIE Vol P-5

6 3.3 Multi-view hyperspectrl imging of n in vivo tissue model An in vivo experiment ws lso designed to verify the integrted multiview hyperspectrl system. A helthy subject s fingers were put in the view field of the integrted system, nd n occlusion experiment ws conducted, in which the fingers were continuously monitored from the beginning of the occlusion to the post occlusive rective hyperemi. Before the experiment, pressure cuff ws pplied round the upper rm of the subject. Then we begn to monitor the tissue structurl nd functionl chrcteristics with the multiview hyperspectrl imging system simultneously. After 1 minute bseline mesurement, n occlusion ws creted with the pressure cuff. We inflted the cuff quickly bove 180 mm/hg nd mintined the pressure for nother 2 minutes, which ws clled s occlusion stge. After tht, the cuff ws relesed nd rective hyperemi emerged t the beginning of the reperfusion stge. The dynmics of three dimensionl oxygen sturtion mps of the fingers could be obtined from the nlysis on the dt from the multiview hyperspectrl imging system. b c d e, 20 o Fig. 7 Multiview hyperspectrl imging of helthy subject s fingers in n occlusion experiment: () 3D topogrphy of the fingers; (b) 3D topogrphy of the fingers with texture; (c) 3D oxygention distribution of the fingers t one moment during the bseline stge; (d) 3D oxygention distribution of the fingers t one moment during the occlusion stge; (e) 3D oxygention distribution of the fingers t one moment during the reperfusion stge. Figure 7 displys the result of the occlusion experiment on the helthy subject. Figure 7 () is the topogrphy of the fingers, while Figure 7 (b) is the textured topogrphy of the fingers. Figures 7 (c), (d), (e) re topogrphy of the fingers with oxygention chrcteristics t one moment during the bseline, occlusion, nd reperfusion stge respectively. The result indictes the occlusion effect nd hyperemi effect clerly, which lso implies tht the system cn obtin topogrphic nd oxygention chrcteristics simultneously in live tissue. 4. CONCLUSION A multiview hyperspectrl imging system ws developed to mp the three-dimensionl oxygention distribution on biologicl tissue. A multiview reflective mirror is plced in front of the lens of the hyperspectrl imger, nd the topogrphy of tissue with oxygention mp could be obtined with three-dimension reconstruction lgorithm nd wide gp second derivtive spectroscopic lgorithm. The imging system is simple, portble, noninvsive, cost effective, nd cliniclly implementble for pplictions such s wound heling ssessment, intropertive surgicl nvigtion, nd orgn trnsplnttion. Three experiments were conducted to vlidte the system. A wound model ws imged by the integrted system nd CMM system respectively, the results from the multiview imging system nd the CMM system re in ccordnce which verified the model preliminrily. A tissue-simulting blood phntom nd n in vivo experiment were lso conducted nd the results lso indicte tht the integrted system cn chieve the structurl nd functionl chrcteristics of the tissue simultneously. Our preliminry results hve demonstrted the technicl fesibility of using multiview hyperspectrl imging for three-dimensionl topogrphy of tissue functionl properties, which is very useful for clinicl tretment. The future work will minly concern the clibrtion of the multiview imging nd hyperspectrl imging system. ACKNOWLEDGMENT This reserch is sponsored by Ntionl Institute of Stndrds nd Technology (60NANB10D184), US Army Medicl Reserch Acquisition Act (W81XWH ) nd Ntionl Science Foundtion of Chin ( ). Proc. of SPIE Vol P-6

7 REFERENCES [1] Ahn, C. nd Slcido, R. S., "Advnces in wound photogrphy nd ssessment methods," Adv Skin Wound Cre, 21 (2), 85-93, (2008). [2] Fercher, A. F., "Opticl coherence tomogrphy," Journl of Biomedicl Optics, 1 (2), , (1996). [3] Pierce, M. C., Strsswimmer, J., Prk, B. H., Cense, B., nd de Boer, J. F., "Advnces in opticl coherence tomogrphy imging for dermtology," Journl of Investigtive Dermtology, 123 (3), , (2004). [4] Gldkov, N. D., Petrov, G. A., Nikulin, N. K., Rdensk Lopovok, S. G., Snopov, L. B., Chumkov, Y. P., Nsonov, V. A., Gelikonov, V. M., Gelikonov, G. V., nd Kurnov, R. V., "In vivo opticl coherence tomogrphy imging of humn skin: norm nd pthology," Skin Reserch nd Technology, 6 (1), 6-16, (2000). [5] Welzel, J., Lnkenu, E., Birngruber, R., nd Engelhrdt, R., "Opticl coherence tomogrphy of the humn skin," J Am Acd Dermtol, 37 (6), , (1997). [6] Kellicut, D. C., Weiswsser, J. M., Aror, S., Freemn, J. E., Lew, R. A., Shumn, C., Mnsfield, J. R., nd Sidwy, A. N., "Emerging technology: hyperspectrl imging," Perspectives in vsculr surgery nd endovsculr therpy, 16 (1), 53-57, (2004). [7] Nilsson, G. E., Tenlnd, T., nd Oberg, P. A., "Evlution of lser Doppler flowmeter for mesurement of tissue blood flow," IEEE Trnsctions on Biomedicl Engineering, (10), , (1980). [8] Kim, K., Hung, S. W., Ashkenzi, S., Odonnell, M., Agrwl, A., Kotov, N. A., Denny, M. F., nd Kpln, M. J., "Photocoustic imging of erly inflmmtory response using gold nnorods," Applied Physics Letters, 90 (22), , (2007). [9] Kirkptrick, S. J. nd Cipoll, M. J., "Lser speckle microstrin mesurements in vsculr tissue," in Proc. SPIE, 1999, pp [10] InSeok Seo, P. R. B., nd Nikiforos Kollis, "Simultneous ssessment of pulsting nd totl blood in inflmmtory skin lesions using functionl diffuse reflectnce spectroscopy in the visible rnge," Journl of Biomedicl Optics, 15 (6), [11] Schrstein, D. nd Szeliski, R., "A txonomy nd evlution of dense two-frme stereo correspondence lgorithms," Int J Comput Vis, 47 (1), 7-42, (2002). [12] Ryn, T., Gry, R., nd Hunt, B., "Prediction of correltion errors in stereo-pir imges," Opticl Engineering, 19 (3), , (1980). [13] Fu, P., "A prllel stereo lgorithm tht produces dense depth mps nd preserves imge fetures," Mchine vision nd pplictions, 6 (1), 35-49, (1993). Proc. of SPIE Vol P-7