Voxel-based Registration Methods in in vivo Imaging

Transcription

1 Vladimír Ulman Laboratory of Optical Microscopy, FI MU 14th April, 2005 Informatics seminar, FI MU

2 Outline Introduction to bioinformatics Introduction to in vivo imaging Point-based registration Optical flow registration using MHI Expectations, future work

3 Introduction to bioinformatics Introduction to in vivo imaging Bioinformatics relatively new scientific field biologists and computer scientists work together

4 Introduction to bioinformatics Introduction to in vivo imaging Bioinformatics relatively new scientific field biologists and computer scientists work together typical situation in bioinformatics: biologists generate tasks/problems/challenges computer scientists solve them (if they re able to) computer scientists provide useful feedback to biologists

5 Introduction to bioinformatics Introduction to in vivo imaging Laboratory of Optical Microscopy bioinformatics group located at FI MU collaboration with Institute of Biophysics (Academy of Sciences) as well as with several other medical centers (clinical research) 15 members, including 4 PhD students

6 Introduction to bioinformatics Introduction to in vivo imaging Laboratory of Optical Microscopy bioinformatics group located at FI MU collaboration with Institute of Biophysics (Academy of Sciences) as well as with several other medical centers (clinical research) 15 members, including 4 PhD students chromatin organization in human cell nuclei study of mechanisms of induction, diagnostics and prevention of deleterious human diseases

7 Introduction to bioinformatics Introduction to in vivo imaging Laboratory of Optical Microscopy bioinformatics group located at FI MU collaboration with Institute of Biophysics (Academy of Sciences) as well as with several other medical centers (clinical research) 15 members, including 4 PhD students chromatin organization in human cell nuclei study of mechanisms of induction, diagnostics and prevention of deleterious human diseases microarrays and in vivo imaging, statistics and visualization

8 Introduction to bioinformatics Introduction to in vivo imaging in vivo imaging biologists point of view: observation of live cells and their content during time period cells evolve, move, divide, join intracellular objects change their structure, intensity, positions

9 Introduction to bioinformatics Introduction to in vivo imaging in vivo imaging computer scientists point of view: sequence of time-lapse 3-D images of region of interest 3-D image stack of 2-D images 2-D image size: 1300px 1030px, 8bits per pixel voxel size: 0.124µm 0.124µm 0.3µm number of samples in z dimension: number of samples in λ dimension: 2 3 number of time steps:

10 Introduction to bioinformatics Introduction to in vivo imaging Analyzing in vivo images to process two consequent images means to: register images and get alignment find correspondence of objects and get mapping

11 Introduction to bioinformatics Introduction to in vivo imaging Analyzing in vivo images to process two consequent images means to: register images and get alignment find correspondence of objects and get mapping point-based registration methods } optical flow method voxel-based registration methods methods based on MHI all are designed to solve registration and correspondence simultaneously

12 Introduction to bioinformatics Introduction to in vivo imaging Illustration of alignment and mapping in 2-D alignment: blue objects in time t black objects in t + rotated 20 o clockwise, translated 20px to the right

13 Introduction to bioinformatics Introduction to in vivo imaging Illustration of alignment and mapping in 2-D alignment: blue objects in time t black objects in t + mapping: time t time t + rotated 20 o clockwise, translated 20px to the right blue lines visualize part of mapping

14 Point-based registration Optical flow registration using MHI Point-based methods images contain small objects object can be easily represented by its center of mass (point) additional characteristics can be used (volume, roundness, axis orientation)

15 Point-based registration Optical flow registration using MHI Point-based methods images contain small objects object can be easily represented by its center of mass (point) additional characteristics can be used (volume, roundness, axis orientation) image preprocessing labeling, we get sets of points graph theory

16 Point-based registration Optical flow registration using MHI Example of preprocessing original thresholded filtered centers original image was first inverted from acquired image and then cropped prior to preprocessing text file describing objects is created text files representing time t and t + is processed in order to find mapping

17 Point-based registration Optical flow registration using MHI Optical flow method optical flow is a distribution of apparent movement velocities of brightness patterns in an image this distribution is described by the flow field

18 Point-based registration Optical flow registration using MHI Optical flow method optical flow is a distribution of apparent movement velocities of brightness patterns in an image this distribution is described by the flow field images contain small or reasonably large objects constant brightness of the same object during the time period

19 Point-based registration Optical flow registration using MHI Optical flow method optical flow is a distribution of apparent movement velocities of brightness patterns in an image this distribution is described by the flow field images contain small or reasonably large objects constant brightness of the same object during the time period less or no preprocessing required implementations are rather specialized

20 Point-based registration Optical flow registration using MHI Insight into optical flow method velocity (u, v) at pixel coordinates (x, y): moving object gives rise to a moving constant brightness pattern: I (x, y, t) = I (x + dx, y + dy, t + dt) equivalently: ɛ a = (I x, I y ) (u, v) + I t 0, u = dx dt, v = dy dt

21 Point-based registration Optical flow registration using MHI Insight into optical flow method velocity (u, v) at pixel coordinates (x, y): moving object gives rise to a moving constant brightness pattern: I (x, y, t) = I (x + dx, y + dy, t + dt) equivalently: ɛ a = (I x, I y ) (u, v) + I t 0, u = dx dt, v = dy dt limit the difference between the flow velocity at a point and the average velocity over local neighborhood which is equivalent to: 2 u + 2 v = 2 u + 2 u + 2 v + 2 v 0 x 2 y 2 x 2 y 2 2 u κ(ū u) and 2 v κ( v v), where κ 3 and ū, v are local averages minimize: ɛ 2 = ɛ 2 a + α 2 (ū u) 2 + α 2 ( v v) 2

22 Point-based registration Optical flow registration using MHI Example of flow field composed images: flow field, illustration: red objects moved (and changed) into blue objects during the time gray dots imply no detected movement while gray lines display velocity vectors, each vector heads from its red end

23 Point-based registration Optical flow registration using MHI MHI method, original idea MHI means Motion History Images computed from several consequent images for every time step

24 Point-based registration Optical flow registration using MHI MHI method, original idea MHI means Motion History Images computed from several consequent images for every time step images can contain small, large or complex objects first detect a type of a movement then its parameters

25 Point-based registration Optical flow registration using MHI MHI method, original idea MHI means Motion History Images computed from several consequent images for every time step images can contain small, large or complex objects first detect a type of a movement then its parameters still parts of image are removed, history image is created history image is classified with moment features moment features are compared to database of known movements for classification

26 Point-based registration Optical flow registration using MHI Example of MHI output MHI: MHI with mapping: MHI(x, y, t) = { fg(i (x, y, t)) 0 min(0, MHI(x, y, t 1) 1) otherwise fg(i (x, y, t)) 0 indicates part of object, size of history window

27 Point-based registration Optical flow registration using MHI Derived MHI method living cells don t fulfill some limitations of original MHI method cells are moving too slow or too fast (due to sparse sampling)

28 Point-based registration Optical flow registration using MHI Derived MHI method living cells don t fulfill some limitations of original MHI method cells are moving too slow or too fast (due to sparse sampling) compute MHI over entire time-lapse sequence with the difference that every pixel in MHI should encode number of frame in which it was present (part of some object) slowly moving and static cells can be detected and removed from MHI

29 Point-based registration Optical flow registration using MHI Derived MHI method living cells don t fulfill some limitations of original MHI method cells are moving too slow or too fast (due to sparse sampling) compute MHI over entire time-lapse sequence with the difference that every pixel in MHI should encode number of frame in which it was present (part of some object) slowly moving and static cells can be detected and removed from MHI graph representation is constructed from resulting MHI labeling of nodes according to special rules

30 Point-based registration Optical flow registration using MHI Example of derived MHI output reduced MHI, graph: green squares denotes initial graph representation, the rest of green edges has to be find out let A, B, C be graph nodes: (e(a, B) v(a) v(b)) e(b, C) = v(b) v(c), e(a, B) indicates edge between A and B, v(a) holds volume of object under A

31 Expectations, future work Expectations and future work point-based methods work well when establishing correspondence of point objects (or representable by point) optical flow should handle larger complex objects but it s constraints parameters should be adjusted before derived MHI is expected to be superior to both of them

32 Expectations, future work Expectations and future work point-based methods work well when establishing correspondence of point objects (or representable by point) optical flow should handle larger complex objects but it s constraints parameters should be adjusted before derived MHI is expected to be superior to both of them for point objects it has capabilities to behave similarly to some point-based technique mappings of larger slower objects should be handled too (domain of optical flow techniques) hopefully, constrained graph labeling will do the rest

33 Expectations, future work Expectations and future work point-based methods work well when establishing correspondence of point objects (or representable by point) optical flow should handle larger complex objects but it s constraints parameters should be adjusted before derived MHI is expected to be superior to both of them for point objects it has capabilities to behave similarly to some point-based technique mappings of larger slower objects should be handled too (domain of optical flow techniques) hopefully, constrained graph labeling will do the rest future work: implementation and testing

34 Expectations, future work a progress in bioinformatics field constantly announces new challenges as new technologies emerge new types of imaging and images result which require new methods to be developed or older to be improved

35 Expectations, future work a progress in bioinformatics field constantly announces new challenges as new technologies emerge new types of imaging and images result which require new methods to be developed or older to be improved principle of point-based registration methods was outlined as well as classical voxel-based technique an optical flow method from video surveillance sort of problems was presented and it s improvement was suggested

36 Expectations, future work Thank You for Your attention. See You next year when presenting encouraging results ;-)