SEVENTH FRAMEWORK PROGRAMME. "Ideas" Specific Programme. European Research Council

Transcription

1 SEVENTH FRAMEWORK PROGRAMME "Ideas" Specific Programme European Research Council Grant agreement for Starting Grant Annex I - Description of Work Project acronym: Lie Analysis Project full title: Lie Group Analysis for Medical Image Processing Grant agreement no.: Duration: 60 months Date of preparation of Annex I (latest version): 23th June 2013 Principal Investigator: Remco Duits Host Institution: Eindhoven University of Technology Department of Mathematics and Computer Science

2 European Research Council ERC Starting Grant 2013 Research proposal [Part B1] Lie Group Analysis for Medical Image Processing LIE ANALYSIS Name of the Principal Investigator (PI): Remco Duits Name of the PI s host institution for the project: TU/e Proposal full title: Lie Group Analysis for Medical Image Processing Proposal short name: Lie Analysis Proposal duration in months: 60 Proposal summary: The aim of this project is to substantially improve computer algorithms for image analysis in medical imaging. Currently available techniques often require significant applicationspecific tuning and have a limited application scope. This is mostly due to the involvement of feature spaces that involve many physical dimensions and lack a clear group structure. Therefore, we take inspiration by the superior pattern recognition capabilities of the human brain and recent insight on how this is accomplished, and formulate a novel operator design aiming at better results and a wider applicability. This novel operator design combines (partial) differential equations on non-compact Lie groups (induced by stochastic processes and sub-riemannian geometric control) with wavelet transforms. Many mathematical challenges arise in the analysis and (numerical) solutions of these operators. The research starts with previously developed insights of the PI on invertible orientation scores, which can be regarded as a single entity in a larger Lie group theoretical framework. Within this framework one obtains a comprehensive invertible score defined on a higher dimensional Lie group beyond position space. The key challenge is to appropriately exploit these scores, their survey of multiple features per position, their underlying group structures, and their invertibility. We will tackle this via left-invariant evolutions and left-invariant sub-riemannian optimal control within the score. The orientation score approach will be systematically extended towards multi-scale-and-orientation, multi-velocity and multi-frequency encoding and processing, widening the application scope. Moreover, improvements in (crossing preserving) contextual enhancement via invertible scores and improvements in optimal curve extractions in the Lie group domain of the score will be pursued. We will apply the resulting algorithms to a wide range of medical imaging challenges in neurology, retinal and cardiac applications. Our team, consisting of 2 PhD s, 1 PostDoc, 1 scientific programmer and the PI, will therefore cooperate closely with advanced clinical - as well as mathematical - partners from around the world. 1

3 Duits Part B1 Lie Analysis, no The Principal Investigator a. Extended Synopsis 1. Introduction: State of the art, Objectives, Approach and Lie Groups Today, the human visual system outperforms the computer in recognizing patterns in images, especially when these images are noisy and display overlapping structures. In such circumstances a human observer can often still identify object contours and recognize crossings and bifurcations, where computer algorithms fail. Worldwide, a substantial amount of research is ongoing to improve image analysis techniques. Many practical image processing problems are targeted both in medical applications and in e.g. industrial, robotic and aerial vision. The researched algorithms are often very successful in their particular applications but usually require significant application-specific tuning and have a limited application scope. This is mostly due to the involvement of feature spaces that involve many physical dimensions and lack a clear group structure. Our objective is to develop an improved, generally applicable, approach to computer vision, via advanced differential geometry on Lie groups. Our approach is inspired by recent insights in the anatomy and functioning of the first stages of the visual system in the human brain. These show that - in contrast with most current image analysis architectures - the brain does not strive for an early reduction of computational complexity. Instead, higher dimensional image representations (so-called scores) are created (multi-scale, multi-orientation, multifrequency...) which are subsequently analyzed in the massively parallel architecture of the visual cortex. Earlier work of the PI has explored the application of Lie group modeling in medical image processing ( orientation scores ) with significant succes [1, 2]. In the proposed research, we will improve this approach and apply it to a range of conventional (scalar) 2D and 3D images in medical imaging. In close cooperation with advanced clinical partners, we aim to demonstrate that substantial progress can be made in automated image analysis in e.g. detection and analysis of blood vessel structure in optical images of the retina and detection of catheters in X-ray images of the heart. Moreover, we will extend our approach to more complex Lie groups, allowing the application of our techniques to higher-dimensional complex images which are generated in advanced (neuro) imaging techniques such as DW-MRI. Again we will apply the new algorithms to several medical issues such as epileptic brain surgery and deep-brain stimulation and aim to bring clinical progress. In general, we expect that with our generalized computer vision approach, substantial progress can be made in automated medical image analysis, increasing both efficiency and accuracy in global healthcare. Mathematical Principles of our Novel Approach In analogy to the human brain, we will transform each image into a higher dimensional orientation score, which attributes a complete distribution of orientations to each position in the image. On such a score new classes of analysis techniques can be performed before the stable inverse transformation is applied. The orientation score can be regarded as just a single entity in a larger Lie group theoretical framework. We will use this embedding to include local multi-scale and multi-frequency encoding as well. This further improves the performance of the algorithms and widens the scope of applications. Our new image analysis method relies on 5 principles: A B C the image data are coherently transformed to a score, which is a complex-valued function on a higher dimensional space G beyond position space, the image data will be stably reconstructed from the higher dimensional space to ensure we do not spoil data-evidence before processing takes place in G, only the coherent features in G are amplified by means of contextual enhancement via left-invariant evolutions (PDE s) on the score, D optimal curves are extracted in G via geometrical control theory based on the enhanced scores, E processing of multiple features (crossing lines, crossing textures, occlusions) do not involve an ad-hoc classification of complex structures (such as crossings). The higher dimensional space G is typically a Lie group G = R d T obtained by taking the semi-direct product of R d with another Lie group T associated to the local features of interest. The combination of these five principles will generate a new and innovative computer vision theory beyond the state-of-the-art.

4 Duits Lie Analysis, no Part B1 2. Algorithm Development In this project we will exploit the group structure both in the reconstruction from scores and in the processing on the scores, applying the commutative diagram in Fig. 1. In particular the latter has been widely neglected Figure 1: A schematic view on processing images f : Rd R via invertible scores Wψ f : G C defined on Lie group G = Rd o T. Appropriate design of operator Φ is the main objective of this proposal. despite the vast literature on wavelets, edgelets, curvelets, and Gabor transforms. The processing on scores has our primary focus and consists of enhancement (diffusion and erosion) and subsequent detection. Subsequently, we address our 5 principles A E and then formulate our challenges. A. A score provides a comprehensive overview of how an image is decomposed out of local features of interest (e.g. orientations/velocities/frequencies). Such a score Wψ f : G C is obtained by probing an image f : Rd R by a family of group coherent wavelets {Ug ψ}g G obtained from a square-integrable mother wavelet ψ : Rd R using a unitary, irreducible representation U of an affine group G = Rd o T : (Wψ f )(g) = (Ug ψ, f ), for all g G, where (, ) denotes the L2 (Rd )-inner product. Example (Orientation Scores): if d = 2, T = SO(2) and G = SE(2), the group of planar rotations and translations, and if Ug ψ denotes the rotated and translated wavelet associated to g SE(2), the score is a so-called orientation score. A key advantage of an orientation score is that lines involved in a crossing are automatically disentangled, allowing crossing preserving flow and processing, cf. Fig. 2,Fig. 3. Figure 2: Left & middle: Lines crossing in the spatial domain no longer do so when lifted to an orientation score. Right: Two crossing blood vessels are disentangled in the score, allowing a separate segmentation. B. The stability of the transforms between image and score is of crucial importance in our commutative diagram depicted in 1. If the group representation U is irreducible and square integrable the transformation between image and score is perfectly stable, cf. [7]. This follows by the Plancherel theorem of inverse Fourier transform on the group G. If U is reducible, one applies a decomposition into irreducible representations to get stable reconstructions [2]. This puts serious conditions on the mother-wavelet [8]. We have derived classes of proper mother wavelets for orientation scores [8]. For better grip on the balance between detection quality and reconstruction stability we aim to expand the wavelet (transform) into a windowed Zernicke-polynomial basis instead of the polar eigenfunctions of the harmonic oscillator [8]. C. Regarding enhancements of scores, we avoid nongeometrical nonlinear processing such as (soft)-thresholding of wavelet coefficients. Instead we aim for geometrical flows induced by wellposed PDE s (parabolic evolutions) in the score domain. As operators ϒ in the image domain typically must commute with the groups representation used to construct the score, we must restrict ourselves to left-invariant evolutions Φ on the scores [9, 8, 2]. We focus on 2 types of left-invariant evolutions on the scores: 1. left-invariant (convection)-diffusions induced by stochastic processes for feature propagation. 2. left-invariant Hamilton-Jacobi-Bellman systems induced by cost processes for feature sharpening. D. Regarding salient curve extraction we rely on sub-riemannian geometrical control problems (curve optimization) in the Lie group domain of a score. The advantage of detecting the most salient curves within the score is the disentanglement of crossings and bifurcations, cf. Fig.2. Basic approaches already produce very

5 Duits Part B1 Lie Analysis, no good results compared to state of the art, but we expect further improvement, understanding and control via formal geometrical control theory [13] on sub-riemannian manifolds within G. E. The key advantage of applying such evolutions via the scores is that this automatically includes the geometric context of features in the score in a crossing preserving way without a need to classify the type of crossing, cf. Fig. 3. At the same time they lead to new insights, extensions, solutions (exact and numeric [10, 2, 9]) to well-known mathematical problems (e.g. [11, 12, 20]). +Noise Original CED-OS original Original +Noise image t = 10 CED: CED-OS standard +Noise t = approach t = CED-OS: CEDourt approach = t = CED t = 10 Figure 3: Left-invariant processing via invertible orientation scores is the best approach to generically deal with crossings. Left : original image.middle: result of coherence enhancing diffusion (CED), cf. [4]. Right: CED via invertible orientation score (CED-OS) cf. [2, part II]. CED-OS is capable of handling crossings and bifurcations, whereas CED produces spurious artifacts at such junctions. 3. Challenges: Challenge I. Improve the framework of image processing via orientation scores. 1. Analyze the trade-off of between detection and reconstruction properties of proper wavelets. pical Figure different 15: 2. Shows behavior Steer the the of typical nonlinear CED-OS Figure different 15: compared left-invariant Shows behavior the to CED. typical convection-diffusions of CED-OS Indifferent CED-OS compared behavior crossing on toscores CED. of CED-OS In viaced-os best compared exponential crossing to CED. curveinfits. CED-OS crossing reserved. structures are 3. Can better wepreserved. setstructures up a central are limit better theorem preserved. on R d S d 1, for d = 2,3? Can we employ this for fast approximation of linear left-invariant (hypo)-elliptic diffusions. +Noise Original CED-OS Original +Noise t = 30 CED-OS +Noise t = t 30 = 30 CED-OS t = t 30 = 30 CED t = 30 Challenge II. Development crossing preserving diffusion on DW-MRI which includes data-adaptive curvature, torsion in its flow. Challenge III. Application and analysis of erosions (HJB-PDE s) to DW-MRI for well-posed sharpening and detection of fibers in DW-MRI. This would avoid ill-posed (constrained) spherical de-convolutions. Challenge IV. Can we derive exact and approximate Green s functions for contour completion and contour enhancement on SE(3) by spectral decomposition of Fourier transform on SE(3)? D-OS Figure and 16: Challenge CED Result on microscopy of V. CED-OS Utilize Figure images the and 16: score CED Result of bone framework of microscopy tissue. CED-OS on Additional the and images Heisenberg CED of Gaussian bone microscopy groups tissue. H(2d Additional images + 1) ofin Gaussian bone medical tissue. imaging. Additional We aim Gaussian the noise behaviour is for added left-invariant on to noisy verify images. the noise evolution behaviour is added equations to noisy verify on velocity images. the behaviour scores and on noisy Gaborimages. transforms. This provides contextual motion extraction models that deal with interference of multiple moving objects and frequencies. Employ the duality between estimated frequencies from (enhanced) Gabor transforms and deformation fields to directly quantify effect Figure of CED-OS 15 shows compared the effect tofigure CED of CED-OS on 15 shows artificial compared theimages effect toof with CED CED-OS crossing on artificial compared line images to CED with crossing on artificial lineimages with crossing line structures. cardiac Thewall upper deformations image shows fromanmri-tagging additive superimposition images, cf. Fig. of6 image shows an additive superimposition structures. The of upper two imagesshows with concentric an additive two images superimposition with concentric of two images with concentric circles. Our method is able to preserve this structure, while CED can not. The same holds for able to preserve Challenge thisvi. structure, Extraction circles. while Our of curves CED method can by is optimal not. ablethe to control preserve sameonholds sub-riemannian thisfor structure, while manifolds CEDwithin can not. SE(d). TheInsame addition holds for the lower image with crossing straight lines, where it should be noted that our method leads to rossing straight to [15, lines, 13, 16, where 2] the we it lower should include image beenhanced noted withthat crossing scores our(via straight method Challenge leads lines, to where I,II,III) itasshould a costbe function. noted that Apply ourthis method to fiber leads to amplification of the crossings, which is because the lines in the original image are not superimposed ssings, which linearly. tracking is because In thisinexperiment, DW-MRI the amplification lines(d=3) the no deviation and original of e.g. the catheter image crossings, are from horizontality detection which not superimposed is in because noisy was taken X-ray the lines into (d=2). in the original image are not superimposed account, and the iment, no deviation from horizontality linearly. In this wasexperiment, taken into account, no deviation and the from horizontality was taken into account, and the numerical scheme of Section 7.2 is used. The non-linear diffusion parameters for CED-OS are: ection 7.2 is Challenge used. The VII. non-linear Find numerical the relation diffusion scheme between parameters of Section erosion, 7.2 forhypo-elliptic is CED-OS used. The are: diffusion, non-linear anddiffusion optimal curves parameters in R d S for d 1 CED-OS. Here are: n θ = 32, t s = 12, ρ s = 0, β 0.058, and c = The parameters that we used for CED are (see 0, β = 0.058, and c = The parameters that we used for CED are (see [40]): σ we 1, rely ρ = on 1, the C Fenchel n θ = 32, = 1, and transform t s = 12, α = onρ s The the = Lie-algebra 0, β = 0.058, images have ofand a left-invariant c = size of vector The parameters pixels. fields on SE(d), that we instead usedof forthe CED intangible 1 at Cramer the beginning transform [40]): σ are (see 1, and α = The images have a size of pixels. Figure of[17]. = 1, the paper Mumford s ρ = 1, C = shows the approach 1, and α results[11] = on an relates The image elastica images of collagen curves have fibres toa modes size of obtained of 56 a contour 56 pixels. completion nning of the paper shows the results Figureon1 an at the image beginning of collagen of the fibres paper obtained shows the results on an image of collagen fibres obtained using 2-photon process, microscopy. whereas Brownian These kind bridge of images theoryare relates acquired sub-riemannian in tissue engineering geodesicsresearch, to a contour where enhancement process opy. These kind of images are using acquired 2-photon in tissue microscopy. engineering These research, kind of where images are acquired in tissue engineering research, where the goal[19, is to2]. create If erosions artificial shrink heart towards valves. optimal All parameters curves (holding duringfor these approximations, experiments were cf. [10]) set the we can avoid boundary tificial heart valves. All parameters the goal is during to create theseartificial experiments heartwere valves. setall theparameters during these experiments were set the same asconditions. the artificial images mentioned above except for CED parameter ρ = 6. The image size mages mentioned above except samefor asced the artificial parameter images ρ = 6. mentioned The image above sizeexcept for CED parameter ρ = 6. The image size is pixels. is pixels. Figures Challenge 16 and VIII. 17 show Derive examples exact and of the approximative method on other reconstruction microscopy schemes data. from The same multiple parameters scale orientation scores, how examples are used of the as above method except on Figures other 16 for t microscopy and 17 show s = 25 in Figure data. examples 17. The Clearly, same of the paramexcept the method on other microscopy data. The same parameters where we combine the strengths of curvelet [6] and directional curve wavelet enhancement transforms andin noise our design. Construct suppression for t s = of 25 the in Figure crossing 17. are curves Clearly, used is good the as above curveexcept in our enhancement for t s = method, while and 25 in standard noise Figure 17. Clearly, the curve enhancement and noise numerical left-invariant coherence enhancing ssing diffusion curves tends is good to in destruct oursuppression evolutions method, crossings while of the on and standard crossing multiple create artificial coherence curves scale orientation is oriented enhancing good scores our method, where scale while layers standard interact, coherence to enhance, enhancing detect and complete structures. uct crossings Figure 18 and demonstrates create artificial diffusion contours the advantage oriented tends at multiple to structures. destruct scales. crossings Applyand thiscreate in retinal artificial imaging. oriented structures. of including curvature. Again, the same parameters and ates the advantage of includingfigure curvature. 18 demonstrates Again, the same the advantage parameters ofand including curvature. Again, the same parameters and

6 Duits Lie Analysis, no Part B1 4. Medical Imaging Applications To explore and demonstrate the applicability of our new approach, we cooperate with renowned clinical research institutes to apply our techniques to a wide range of practical medical imaging problems. From the nature and structure of the different imaging modes, we have identified the logical relation between each of these practical image processing challenges and the appropriate Lie group to be used in each case. a) neurological imaging (Lie group quotient R3 o S2 := SE(3)/({0} SO(2)) ) we will analyze DW-MRI images to automatically detect overlapping fibers in the human brain for supporting surgeons to stay away from (optical) nerve bundles in epileptic surgery, identify the location for needle placement in Deep Brain Stimulation, identify and localize brain damage in Multiple Sclerosis patients. a) b) c) d) e1) 30% 2% Figure 4: a: transversal anatomical cross-section of the brain, showing optical radiation fibers. b, c, d: MR images of longitudinal cross-sections of the brain, showing b: fmri derived beginning and end points; c: state of the art fiber tracking methods giving high uncertainty and d: preliminary results of our improved tracking. b) retinal imaging (Lie groups SIM(2) and SE(2)) we want to improve the automatic detection of retinal vascular structure from optical retinal images for the early detection of diabetes Figure 5: left: optical retina image; middle and right: automated image analysis identifies bloodvessel structure in normal (middle) and diabetic (right) case. Currently available algorithms need substantial improvement which we aim to achieve with enhanced multiple scale orientation scores on SIM(2) c) cardiac imaging (Lie groups SE(d) and H(5)) enhance and detect (crossing) heart fibers in noisy ultrasound images (Lie group SE(d)), automatically detect catheters in low-dose biplane X-ray images (SE(2)), analyze optical flow in dynamic MR-tagging images (H(5)), determine cardiac deformations via frequency fields in dynamic MR-tagging images (H(5)). Figure 6: MRI tagging images of the human left ventricle in diastole (left) and systole (right). New image processing technology is needed to perform such analysis automatically. 5. Project Plan and Partners Our team consists of 2 PhD s, 1 Scientific Programmer, 1 PostDoc and the PI. We will allocate to each PhD student a number of the mathematical challenges associated to a Lie group case, in combination with one or more matching clinical applications. To facilitate introduction, the hiring of the different PhD s will be spread out over the first year of the grant. The collaborating mathematical/clinical partners are presented in the following table. 30%

7 REFERENCES 6 Medical imaging Institute partner dr. P. Ossenblok Neuro Epilepsy Institute, Kempenhaeghe, NL Prof. M. Deppe Neuro Klinik für Neurologie, Munster, Germany dr.ir.a.vilanova Neuro IST/e, Netherlands Prof. B.M.ter Haar Romeny Retinal BMI/a TU/e Netherlands dr. T. Berendschot Retinal Maastricht U.H., NL Prof. W. He Retinal He Eye Care, China dr. J. Westenberg Cardiac LUMC, Leiden, NL Prof. M. Breeuwer Cardiac Philips Healthcare, NL dr. H.A.C. van Assen Cardiac IST/e, Netherlands Mathematical partner dr. U. Boscain dr. A. Sarti Prof. H. Führ Prof. J. Polzehl dr. K. Tabelow Prof. Y. Sachkov dr. C. Citti dr. F. Rossi Prof. M.Felsberg Prof. L. Florack dr. A.J.E.M. Janssen Institute CMAP, France Ec. Pol. Paris, France RWTH Aachen, Germany WIAS, Berlin, Germany Pereslavl, Russia UdB, Bologna, Italy LSIS, France LiU Linköping, Sweden IST/e, Netherlands TU/e Eindhoven, Netherlands References [1] R. Duits, M. Felsberg, G. Granlund, and B. M. ter Haar Romeny, Image analysis and reconstruction using a wavelet transform constructed from a reducible representation of the Euclidean motion group, IJCV, vol. 79(1) pp , [2] R. Duits and E. M. Franken, Left invariant parabolic evolution equations on SE(2) and contour enhancement via invertible orientation scores QAM(AMS), vol. 68, pp , [3] G.Medioni, M.-S. Lee, and C.-K. Tang, Tensor Voting: A Perceptual Organization Approach to Computer Vision and Machine Learning, [4] J. Weickert, Coherence-enhancing diffusion filtering., IJCV, vol. 31(2/3), pp , [5] M. Felsberg, Mathematical Methods for Signal and Image Analysis and Representation, ch. Adaptive Filtering Using Channel Representations, pp [6] D. Donoho and E. Candès, Continuous curvelet transform ACHA, vol. 19(2), pp , [7] S. Ali, J. Antoine, and J. Gazeau, Coherent States, Wavelets and Their Generalizations, [8] R. Duits, Perceptual Organization in Image Analysis. PhD thesis, TU/e, The Netherlands, [9] R. Duits, H. Fuehr, B. Janssen, L. Bruurmijn, L. Florack, and H. van Assen, Evolution equations on Gabor transforms and their applications, ACHA, accepted for publication to appear. [10] R. Duits and M. van Almsick, The explicit solutions of linear left-invariant second order stochastic evolution equations on the 2d-Euclidean motion group, QAM, AMS, vol. 66, pp , [11] D. Mumford, Elastica and computer vision, Algebraic Geometry and Its Applications. Springer-Verlag, pp , [12] A. Agrachev, U. Boscain, J.-P. Gauthier, and F. Rossi, The intrinsic hypoelliptic laplacian and its heat kernel on unimodular Lie groups, Journ. of Functional Analysis, vol. 256, pp , [13] A. A. Agrachev and Y. L. Sachkov, Control Theory from the Geometrical Viewpoint, Springer, [14] N. F. Osman, W. S. Kerwin, E. R. McVeigh, and J. L. Prince, Cardiac motion tracking using cine harmonic phase (harp) MRI, MR in Medicine, vol. 42(6), pp , [15] U. Boscain, G. Charlot, and F. Rossi, Existence of planar curves minimizing length and curvature, Differential equations and dynamical systems, Tr. Mat. Inst. Steklova, vol. 270, pp , [16] Y. Sachkov, Maxwell strata in the euler elastic problem, Dynamical & Control Systems, vol. 14(2) pp , [17] B. Burgeth and J. Weickert, An explanation for the logarithmic connection between linear and morphological systems, LNCS, Proc. 4th int. Conference Scale Space, pp , [18] R. Duits and E. M. Franken, Left-invariant diffusions on R 3 S 2 and their application to crossingpreserving smoothing of HARDI images., IJCV, vol. 92, pp , [19] G. Citti and A. Sarti, A cortical based model of perceptional completion in the roto-translation space, JMIV, vol. 24(3), pp , [20] K. Gröchenic, Foundations of Time-Frequncy Analysis, Birkhäuser, 2001.

8 Duits Part B1 Lie Analysis, no b. Curriculum Vitae Remco Duits Personal Data Name: R.Duits (Remco) Birth Date & Place: , Weert, The Netherlands Nationality: Dutch Partner: Tamara Nijsen, Child: Kyra Duits (daughter), Affiliation Dr.ir. R.Duits Eindhoven University of Technology (TU/e) Dep. of Mathematics and Computer Science (0.6 fte) & Dep. of Biomedical Engineering (0.4 fte) MF 5.107a & Gemini Den Dolech 2 / PO Box 513 NL-5600 MB Eindhoven The Netherlands Educational Background T: T: F: E: R.Duits@tue.nl W: : Eindhoven University of Technology, Department of Mathematics and Computer Science. Master degree in Applied Mathematics (5 year study) August Industrial master project on the application of matrix groups in geodesy, rewarded with maximum grade of 10. Master thesis (Honors: cum laude) on a new functional analytical approach to the Stokes boundary value problem, Research Experience and Former Affiliations : PhD degree in biomedical engineering at Eindhoven University of Technology. PhD thesis: Perceptual Organization in Image Analysisa Mathematical Approach based on Scale, Orientation and Curvature, Honors: cum laude (among 2% best) (best thesis of the biomedical engineering dep. in 2005, nominated for the ASML promotion prize of best PhD thesis at TU/e in 2005) : Postdoc at biomedical engineering within the biomedical image analysis (BMI/a, group of Prof. Bart ter Haar Romeny at TU/e now: Assistant Professor (UD), fixed-term position, within the Center for Analysis, Scientific Computing and Applications (CASA, headed by Prof. Mark Peletier at the Department of Mathematics and Computer Science at the TU/e. Within this position R.Duits is affiliated part-time (0.4 fte) to the biomedical engineering department for applying his theory to medical imaging applications and supervision of PhD and master students. Furthermore, long-term visits (each varying from 1 month to 4 months) to excellent international research institutes abroad include: October, 2002: Collaborative visit to Prof. Stephen Pizer, Dep. of Computer Science, and Prof. James Damon, Dep. of Mathematics, University of North Carolina Chapel Hill, USA. November, 2004: Collaborative visit to Prof. Michael Felsberg, Computer Vision Laboratory, Linköping University, Linköping, Sweden. March & April, 2006: Collaborative visit to Prof. David Mumford (fields medal winner), Brown University, Div. Applied Mathematics, USA, to discuss my solutions to his longstanding problem [11]. Relevant for challenges I,IV,VI,VIII in this proposal. June 2007: Collaborative visit to Prof. Joachim Weickert,Mathematical Image Analysis group of Saarland University, Germany. Relevant for challenges I, VII, VIII in this proposal. March 2008: Collaborative visit to Prof. Michael Felsberg, Computer Vision Laboratory, Linköping University, Linköping, Sweden. Relevant for challenges I,IV, VI in this proposal. April, May, June 2008, March 2009: Collaborative visit to Prof. Harmut Führ, Lehrstuhl für die Mathematik A, RWTH Aachen, Germany, for collaborations related to challenge V in this proposal.

9 Duits Part B1 Lie Analysis, no Teaching Experience and Supervision Coarses taught at Eindhoven University of Technology: responsible teacher of Mathematical Modeling in Physiology (2DX00) for master students BME and Chinese BME students from Shenyang university, yearly responsible teacher of mathematics course Applied Functional Analysis (2WA08) yearly teacher of calculus and (image) analysis bachelor coarses (2DM00,2WBB0,8G116,2Y060), tutor at many design-based learning projects at BME, My research attracts top students from various TU/e departments. I have successfully supervised 3 PhD projects of which 1 was rewarded cum laude, and 2 (Erik Franken & Bart Janssen) were nominated for the ASML promotion prize for the yearly best TU/e PhD thesis, cf. subsection c2. 11 master projects, 7 of which were rewarded cum laude. 5 trainingship image analysis projects. Current supervision 1 PhD mathematics student, 1 PhD student at biomedical engineering, 1 EU-Marie Curie PhD mathematics student and 2 master students at the biomedical engineering department. Language Skills Dutch: mother tongue. English: CEF (Common European Framework of Reference) level C2 (proficient). Good knowledge of German, cursory knowledge of French. Other Professional Activities organizer and chair on three international workshops held at Eurandom TU/e ( These workshops are intended for both mathematicians (probability theory, harmonic analysis and statistics) and mathematically inclined engineers (statistics and imaging): Dec.11-13,2006 : Image Analysis and Inverse Problems organized by Laurie Davies, Remco Duits and Marie-Colette van Lieshout. Sept.24-26,2007 : Algorithms in Complex Systems organized by Laurie Davies, Remco Duits, Geurt Jongbloed and Marie-Colette van Lieshout. Nov.24-26,2008 : Locally Adaptive Filters in Signal and Image Processing organized by Laurie Davies, Remco Duits, Luc Florack, Geurt Jongbloed and Marie-Colette van Lieshout. The last workshop has led to a book publication [RD 52], to which the PI contributed as editor. member of several programme committees, e.g. of the international conferences on scale space and variational methods 2007, 2009, 2011, 2013 and Dagstuhl seminar on Visualization and Processing of Tensors and Higher Order Descriptors for Multi-Valued Data in organizer of workshop Image Analysis and Control Theory in Aug. 28 Sept. 2, 2011 at the TU/e to bridge the fields of geometric control theory and image analysis. It has led to EU-collaboration [RD 48] and inter-disciplinary journal articles we are currently finishing. member of the EMaCs organizing committee.emacs (Eindhoven Mathematics Colloquiums, cf. www. win.tue.nl/emacs/) is a series of mathematics lectures at the TU/e aimed at attracting a broad audience of mathematicians and students. Regular scientific reviewing for journals such as JMIV (Journal of Mathematical Imaging and Vision), IJCV (International Journal of Computer Vision), Journal of Physiology, and IEEE-journal Transactions on Medical Imaging. Scientific reviewer of several conference proceedings and related books, e.g. the proceedings of the conference on scale space and variational methods, Eurandom workshops, Vis2010, Dagstuhl seminars etc. For a complete overview of invited short-term international visits and invited presentations see bmt.tue.nl/people/rduits/presentations.pdf Funding ID The PI currently supervises 2nd year PhD Eric Creusen (contextual processing of DW-MRI) within IST/e, granted by a TU/e high potential programme, and EU-Marie Curie student Arpan Ghosh (directional processing in imaging) within Initial Training Network-FIRST, Agr.No. PITN-GA The PI is advisor in a PhD project on Finsler geometry in DW-MRI (Tom Dela Haije,IST/e). There is and there will be no funding overlap with the ERC grant requested and any other source of funding for the same activities and costs that are foreseen in this project.

10 Duits Part B1 Lie Analysis, no c. Early Achievements track-record c1 Publications Type of Publication # # without the PhD supervisor peer-reviewed international journals peer-reviewed conferences proceedings book chapters 2 2 books 2 All publications contain multiple novelties both on the theoretical as on the application side. In all publications I have acted independently of my supervisors and brought up the original theoretical idea myself. Close multidisciplinary collaborations helped me to bring novel mathematical theory (e.g. [RD2 4]) to medical imaging applications. Key publications are available on my homepage: For the complete list of 52 publications see: Number of citations (excl. self-citations): 84. Journal publications take place in two different types of journals: International Mathematical Journals, such as Quarterly of Applied Mathematics, American Mathematical Society (4 times) and (ACHA), Applied Computational Harmonic Analysis, (NM TMA), International Image analysis journals such as (IJCV), International Journal of Computer Vision (6 times), which is the top journal in image analysis (regarding impact factors over the last 5 years), and JMIV Journal of Mathematical Imaging and Vision (2 times) and are either induced by personal invitation via best paper selections at conferences (e.g. Early Cognitive Vision Workshop 2005, PRIA 2004, Scale Space and Variational Methods, 2003, 2005, 2007, 2009, 2011) or via international visits (e.g. to Prof. David Mumford, USA, to Prof. Hartmut Führ, Germany). The 31 conference articles have been reviewed (double-blindly) by at least 3 external reviewers, with low passing rates. They include 7 selected best papers [RD18,RD31,RD32,RD34,RD37,RD38,RD43] and 3 best paper awards [RD20,RD26,RD33]. Book [RD49] and book chapters [RD50,RD51] relate to international workshops on mathematical imaging. All publications 1 are directly relevant to this proposal. Selection Peer-reviewed Journal Publications: [RD 1] R. Duits, L. M. J. Florack, J. de Graaf, and B. M. ter Haar Romeny, On the axioms of scale space theory, Journal of Mathematical Imaging and Vision, vol. 20, pp , (citations: 25) [RD 2*] R. Duits and M. van Almsick, The explicit solutions of linear left-invariant second order stochastic evolution equations on the 2d-Euclidean motion group, Quarterly of Applied Mathematics, American Mathetical Society, vol. 66, pp , April (citations: 5) [RD 3*] R. Duits and E. M. Franken, Left invariant parabolic evolution equations on SE(2) and contour enhancement via invertible orientation scores, part I: Linear left-invariant diffusion equations on SE(2), Quarterly of Applied mathematics, AMS, vol. 68, pp , June (citations: 3) [RD 4*] R. Duits and E. Franken, Left invariant parabolic evolution equations on SE(2) and contour enhancement via invertible orientation scores, part II: Nonlinear left-invariant diffusion equations on invertible orientation scores, Quarterly of Applied mathematics, AMS, vol. 68, pp , June (citations: 3) [RD 5*] R. Duits, M. Felsberg, G. Granlund, and B. M. ter Haar Romeny, Image analysis and reconstruction using a wavelet transform constructed from a reducible representation of the Euclidean motion group, International Journal of Computer Vision, vol. 79, no. 1, pp , (citations: 3) [RD 6] B. J. Janssen & F. M. W. Kanters & R. Duits (joint first authors), L. M. J. Florack, and B. M. ter Haar Romeny, A linear image reconstruction framework based on Sobolev type inner products., International Journal of Computer Vision, vol. 70, no. 3, pp , (citations: 3) [RD 8*] R. Duits and E. M. Franken, Left-invariant diffusions on the space of positions and orientations and their application to crossing-preserving smoothing of HARDI images., International Journal of Computer Vision vol. 92, pp , March 2011.(citations: 6) [RD 11*] E. M. Franken and R. Duits, Crossing preserving coherence-enhancing diffusion on invertible orientation scores, International Journal of Computer Vision, vol. 85, no. 3, pp , (citations: 3) 1 Publications without the PhD supervisor Luc Florack are marked with a star.

11 Duits Part B1 Lie Analysis, no [RD 13] R. Duits, H. Führ, B.J. Janssen, M. Bruurmijn, L.M.J. Florack and H.A.C.van Assen, Evolution Equations on Gabor Transforms and their Applications, ACHA, accepted to appear in bmt.tue.nl/people/rduits/article_dfjbfva.pdf [RD 17*] R.Duits, T.C.J.Dela Haije, E.J. Creusen and A.Ghosh, Morphological and Linear Scale Spaces for Fiber Enhancement in DW-MRI, JMIV, accepted for publication, to appear in tue.nl/people/rduits/jmivduits2011final.pdf, Selection Peer-reviewed proceedings/conference publications: [RD 18*] R. Duits, T. C. J. Dela Haije, A. Ghosh, E. J. Creusen, A. Vilanova, and B. ter Haar Romeny, Enhancement of DW-MRI, in Scale Space and Variational Methods in Computer Vision (Lecture Notes in Computer Science), vol. 6667, pp. 1 13, September [RD 22*] R. Duits and B. Burgeth, Scale spaces on Lie groups, in Scale Space and Variational methods (M. Sgallari and Paragios, eds.), (Ischia, Italy), pp , (citations: 5) [RD 38*] E. J. Creusen, R. Duits, and T. C. J. Dela Haije, Numerical schemes for linear and non-linear enhancement of DW-MRI, in Scale Space and Variational Methods in Computer Vision (Lecture Notes in Computer Science), vol. 6667, (Heidelberg), pp , Springer-Verlag, September [RD 48*] U. Boscain, R. Duits, F. Rossi and Y. Sachkov Optimal control for reconstruction of curves without cusps, Accepted for publication, to appear in Proc. IEEE Conference on Decision and Control, Mauii, USA, December Selection Books and book chapters: [RD 49] L. M. J. Florack, R. Duits, G. Jongbloed, M.-C. van Lieshout, and L. Davies, Mathematical Methods for Signal and Image Analysis and Representation. Springer-Verlag, Berlin, [RD 50*] R. Duits, H. Führ, and B. Janssen, Left Invariant Evolution Equations on Gabor Transforms, chapter 8 in book [RD 49], pp Springer-Verlag, c2. Invited Presentations and Awards The list of invited international presentations can be found on my website RDuits/presentations.pdf. They include 34 invited talks/visits at prestiguous institutes (e.g. Ecole Polytechnique & Henri Poincare Inst. Paris, WIAS Berlin, Brown University USA, Helmholtz Zentrum München) and international conferences (such as Scale Space and Variational Methods). Awards: Cum laude (i.e. among the 5% best at TU/e) reward for PhD-thesis in 2005 [RD52]. Cum laude reward for Master thesis (supervised by Prof. J. de Graaf) on a new functional analytic approach to Stokes problems in Industrial master project in company Geodelta in Delft in 2001 on Application of Orthogonal Matrix Groups in Geodesy, was rewarded with maximum grade of 10. Best Paper Award on MMBIA workshop ICCV 2007 (Brasil) yielding invited submission [RD11] to International Journal of Computer Vision (IJCV). Best Paper Award on Scale Space conference (Scotland) 2003 yielding invited submission [RD12] to IJCV. Best Paper Award on PRIA Conference 2006 (Russia) yielding invited submission to Image Processing, Analysis, Recognition and Understanding [RD7]. Best PhD-thesis award of the department Biomedical Engineering, TU/e, Nominated for best PhDthesis award of all departments of TU/e (ASML-promotion prize), Selected paper on Scale Space and Variational Methods (SSVM) conference 2007 (Isschia, Italy) for invited submission [RD10] to IJCV. Selected paper on Cognitive Vision Workshop (Scotland) 2005 for invited submission [RD5] to IJCV. Selected paper on SSVM conference 2009 (Norway) for invited submission, [RD8], to IJCV. Selected paper on SSVM conference 2011 (Israel) for invited submission [RD17] to JMIV. Copromotor and supervisor of cum laude (i.e. among the 5% best at TU/e) thesis by E. Franken on Enhancement of Crossing Elongated Structures in Images, Copromotor and supervisor of the PhD thesis by B.J.Janssen on Representation and Manipulation of Images Based on Linear Functionals, html, nominated (not rewarded) for the ASML-promotion prize, Keynote speaker at SSVM 2011 held in Israel, see

12 Duits Part B2 Lie Analysis, no Part B2: The Project Proposal a. State-of-the-art and Objectives: ERC Starting Grant 2013 Research proposal [Part B2)] Today, the human visual system outperforms the computer in recognizing patterns in images, especially when these images are noisy and display overlapping structures. In such circumstances a human observer can often still identify object contours and recognize crossings and bifurcations, where computer algorithms fail. Worldwide, a substantial amount of research is ongoing to improve image analysis techniques. Many practical image processing problems are targeted both in medical imaging and in e.g. industrial, robotic and aerial vision. The researched algorithms are often very successful in their particular applications but often require significant application-specific tuning and have a limited application scope. This is mostly due to the involvement of feature spaces that involve many physical dimensions and lack a clear group structure. Our objective is to develop an improved, generally applicable, approach to computer vision, via advanced differential geometry on Lie groups. Our approach is inspired by recent insights in the anatomy and functioning of the first stages of the visual system in the human brain. These show that - in contrast with most current image analysis architectures - the brain does not strive for an early reduction of computational complexity. Instead, higher dimensional image representations (so-called scores) are created (multi-scale, multi-orientation, multifrequency...) which are subsequently analyzed in the massively parallel architecture of the visual cortex. Earlier work of the PI has explored the application of Lie group modeling in medical image processing ( orientation scores ) with significant succes [1, 2, 3, 4]. In the proposed research, we will improve this approach and apply it to a range of conventional (scalar) 2D and 3D images in medical imaging. In close cooperation with advanced clinical partners, we aim to demonstrate that substantial progress can be made in quantitative image analysis in e.g. detection and analysis of blood vessel structure in optical images of the retina, detection of catheters in X-ray images of the heart. Moreover, we will extend our approach to more complex Lie groups, allowing the application of our techniques to higher-dimensional complex images which are generated in advanced (neuro) imaging techniques such as Diffusion Weighted Magnetic Resonance Imaging (DW-MRI). Again we will apply the new algorithms to several medical issues such as epileptic brain surgery and deep-brain stimulation. In general, we expect that with our innovative, generic computer vision approach, substantial progress can be made in automated medical image analysis, increasing both efficiency and accuracy in global healthcare. a1. Mathematical Principles of our Approach The human visual brain has a very structured way of observing incoming data and is incredibly good in judging whether a local feature (e.g. elongated structure, blob, velocity, texture) is coherent with its surroundings (context). Therefore, in analogy to the functional architecture of the primary visual cortex in the human brain, we will transform each image into a higher dimensional orientation score. Such an orientation score attributes a complete distribution of orientations to each position in the image. Furthermore, such a score allows us to perform a new class of contextual image analysis techniques, before the stable inverse transformation is applied. The orientation score can be regarded as just a single entity in a larger Lie group theoretical framework. We will use this relationship to generalize the orientation approach to also include multi-scale, multi-velocity and multi-frequency analysis. This further improves the performance of the algorithms and widens the scope of applications. For instance, optical retinal images require both a multi-scale and multi-orientation approach, whereas cardiac MRI-tagging images require a multi-frequency approach. In mathematical terms, our new image analysis method uses the following set of design-principles: A the image data are coherently transformed to a score, which is a complex-valued function on a higher dimensional space G beyond position space, B the image data will be stably reconstructed from the higher dimensional space to ensure we do not spoil data-evidence before processing takes place in G, C only the coherent features in G are amplified by means of contextual enhancement via left-invariant evolutions (PDE s) on the score,

13 Duits Part B2 Lie Analysis, no D optimal curves are extracted in G via geometrical control theory based on the enhanced scores, E processing of multiple features (crossing lines, crossing textures, occlusions) do not involve an ad-hoc classification of complex structures (such as crossings). The higher dimensional space G is typically a Lie group G = R d T obtained by taking the semi-direct product of R d with another Lie group T associated to the local features of interest. The combination of all five points is precisely where state-of-the-art imaging processing needs to be improved. E.g. such improvement is needed in image processing based on spherical harmonics [5], orientation histograms, auto-encoders, frequency histograms, tensor voting [6, 7, 8], coherence enhancing diffusion [9, 10], thinning [7], specific crossing preserving diffusions on position space [11], channel representations [12], ridgelets, curvelets [13], directional wavelet transforms [14] and Gabor transforms [15]. Our approach is highly promising for difficult problems in medical image analysis, such as complex brain connectivity from diffusion weighted MRI (DW-MRI), crossing fiber denoising, local deformation analysis in heart infarcts, and computeraided diagnosis of retinal images for early detection of diabetes. a2. Biological Motivation The Nobel laureates Hubel and Wiesel [16] discovered that certain visual cells in the striate cortex of cats have a directional preference. It has turned out that the majority of neurons in the primary visual cortex exhibits such an orientation preference [17]. Moreover, there exists an intriguing spatial and directional organization into so-called cortical hypercolumns, see Fig. 1. Moreover, correlated horizontal connections between hypercolumns have been identified. Synaptic physiological studies of the horizontal pathway in cat striate cortex show that neurons with aligned receptive field sites excite each other [17]. Apparently the visual system not only constructs a score of local orientations, it also accounts for context and alignment by excitation and inhibition. 500µm 0 o 45 o 90 o 135 o 180 o Figure 1: Receptive fields in the visual cortex of many mammalians are tuned to various locations and orientations. Assemblies of oriented receptive fields are grouped together on the surface of the primary visual cortex in a pinwheel like structure. Left: Schematic drawing of a hypercolumn in the primary visual cortex, where orientation selective parts are color-coded. Right: Orientation sensitivity in the primary visual cortex of a tree shrew, replicated from [17], c 1997 Society of Neuroscience. Black dots indicate horizontal connections to aligned neurons with an 80 orientation preference shown by the white dots. The figure on the right indicates horizontal connections at 160. The intriguing analogy with our approach in computer vision will lead to spin-off via mathematical explanations that help understanding biological visual systems. For example, Recent work [18] motivates the functional architecture in the cortical columns via a simplification of our invertible orientation score approach, where restriction to sharp rings in the Fourier domain are applied. However, a soft and smooth localization around specific frequency radii seems more reasonable, cf. [18, 13, 19, 14]. Together with log-polar cortical magnification in the periphery, cf. [20] this motivates an extension to multiple scale orientation scores (See Challenge VIII in this proposal and Fig.7) for modeling and processing of the cortical columns. This extension is also needed in computer vision applications containing elongated structures at multiple scales, See Fig.9.

14 Duits Part B2 Lie Analysis, no Sub-Riemannian geometry plays a major role in the functional architecture of the primary visual cortex (V1), cf. [21]. Petitot showed that the horizontal connections of V1 implement the contact structure of a continuous fibration with base space the retinal area and a projective line of orientations [21]. Association fields [22] and visual hallucinations are modeled by variational curve models [23]. The circle bundle model by Petitot is actually solved by sub-riemannian geodesics within the group of planar rotation and translations and there appears to be a striking similarity with association field lines. Moreover, the location of their cusps [24] (where stationary curves loose optimality) provide an explanation for the sudden endings (grouping criterium) of the association field lines. Therefore, sub-riemmannian geodesics (and their dataadaptive extensions in Challenge VI) seem to be more suited to model association fields than horizontal exponential curves (i.e. co-circularity) in cortical cf. [18] and computer vision, cf. [6]. Besides orientations, the visual system also scores multiple scales, frequencies, velocities (via so-called Reichardt detectors [25]), which all falls within the scope of this proposal. b. Methodology Our proposal consists on the one hand of the development of a new mathematical operator design (with novel algorithms), and on the other hand of the application of these algorithms to a range of clinically relevant image analysis problems. We feel that this combination, and the multi-disciplinary teamwork that goes with this, is important to ensure that our mathematical efforts remain focused on practical progress in society. In this section we first illustrate the theoretical setup (Subsection b1,b2,b3), consider extensions to other image modalities (Subsection b4), formulate the goals (Subsection b5) and the corresponding research challenges (Subsection b6), and then we consider biomedical imaging applications within this project in Subsection b7. In Subsection b8 we conclude with the research plan. Our generic approach typically consists of three stages: 1. Construction of an invertible score which is a complex-valued function on a Lie group G = R d T beyond position space, where T is the Lie-group underlying the local features of interest. 2. Enhancement/completion within the score via concatenations of convection-diffusions (for coherent feature propagation) and erosions (for sharpening towards the most salient features). The convection-diffusions are forward Kolmogorov equations of stochastic processes on G, whereas the erosions are Hamilton- Jacobi-Bellman (HJB) systems of cost processes on G. 3. Extraction of optimal pathways within the score via sub-riemannian geometric control within G. In Subsections b1,b2,b3 we will illustrate these three steps for the specific case G = SE(2), where the score is an invertible orientation score. b1. Invertible Orientation Scores of 2D-images (the case G = SE(2)) In many biomedical applications elongated structures play a prominent role. Due to low contrast and occlusions parts of such structures may be invisible or even absent altogether, thus requiring contour completion. Furthermore, reduction of radiation dose/ acquisition time can result in noisy images. Such images typically require contour enhancement. On top of that, elongated structures may exhibit crossings or bifurcations, which causes notorious problems in conventional algorithms. To address these problems generically I have recently introduced the concept of an invertible orientation score, which attributes a complete distribution of orientations to each position in the image. Such an orientation score is given by a generalized wavelet transform: W ψ f (g) = ψ(r 1 R 2 θ (y x)) f (y) dy, with g = (x,eiθ ) SE(2) =. R 2 S 1,R θ SO(2). (1) So the orientation score W ψ f : SE(2) R is obtained from image f : R 2 R by convolution with a directed anisotropic kernel ψ : R 2 R rotated over all angles, akin to the hypercolumns of Fig.1. I have derived classes of directional kernels ψ that allow a stable reconstruction by the adjoint wavelet transform W ψ, cf. [26, 3, 1]. This result generalizes wavelet theory [27] and coherent state theory [19, 14], see [3, 1, 26] and fits within the framework by Führ [28]. Well-posed invertibility is a crucial property, as it allows us

15 Duits Part B2 Lie Analysis, no Image f Υ ψ W ψ Orientation Score U f Original +Noise CED-OS Original t = 10 CED +Noise t Processed WOriginal = 10 Processed CED-OS t +Noise = 10 CEDCED-OS t = 10 t = 10 CED t = 10 ψ Image Orientation Score Φ Figure 2: A schematic view on image processing via invertible orientation scores. to relate operators on images to operators on orientation scores, see Fig.2. The advantage of making a detour via the orientation score is that it allows us to resolve orientation ambiguities at any position, see the results in Fig.3. An invertible orientation score carries per position a complete distribution of orientations and thus manifestly original image CED: standard approach CED-OS: our approach g. 5. Shows the typical different behavior Fig. of5. CED-OS Shows the compared typical to different Fig. CED. 5. Shows In behavior CED-OS theof typical CED-OS different compared behavior to CED. of CED-OS In CED-OS compared to CED. In CED-O ssing structures are better preserved. crossing structures are better crossing preserved. structures are better preserved. Original CED-OS t = 2 CED-OS Originalt = 30 CED-OS Original t = 30 2 CED-OS CED-OS t = 30t = 2 CED CED-OS t = 30t = 30 CED t = 30 Original Original +Noise CED-OS +Noise Original t = 10 CED-OS +Noise t t = CED-OS t = 10 t = 10 CED t = 10 g. 6. Shows results on an image constructed Fig. 6. from Showstwo results rotated on an Fig. 2-photon image 6. Shows constructed images results of on fromantwo image rotated constructed 2-photon from images two of rotated 2-photon images lagen tissue in the heart. At t = 2 we collagen obtaintissue a nice inenhancement the heart. collagen Atoft the tissue = 2image. we in obtain the heart. a nice Atenhancement t = 2 we obtain of the a nice image. enhancement of the imag mparing with t = 30 a nonlinear scale-space Comparing behavior withcan t = be 30 seen. a nonlinear Comparing For comparison, scale-space with t = 30 behavior a nonlinear can be scale-space seen. Forbehavior comparison, can be seen. For compariso right column shows the behavior of CED. the right column shows the the behavior right column of CED. shows the behavior of CED. Conclusions 9 Conclusions 9 Conclusions this paper Figure we introduced 3: Left-invariant nonlinearindiffusion processing this paper on the via we introduced invertible Euclidean In this orientation motion nonlinear paper group. we diffusion scores introduced isonthe the nonlinear right Euclidean approach diffusion motion to on generically group. the Euclidean deal motion grou arting fromwith a 2Dcrossings image, weand constructed bifurcations. Starting a three-dimensional from Left acolumn: 2D image, original Starting orientation we constructed images. fromscore a 2DMiddle aimage, three-dimensional column: we constructed result orientation of a standard three-dimensional score coherence orientation sco ing rotatedenhancing versions ofdiffusion a directional applied using quadrature directly rotatedon filter. versions the We image of considered using a(ced), directional rotated cf. the[9]. versions oritation scoreviasinvertible a functionorientation the entation Euclidean score cf. score motion [4, 2]. as agroup Top function entation row: and 2-photon defined score the Euclidean the as microscopy a function motion image ongroup the of Euclidean bone and tissue. defined motion Middle the group row: and defined t quadrature Right ofcolumn: afilter. directional We coherence considered quadrature enhancing thefilter. ori-diffusion We considered the o t-invariant collagen diffusion fibers equation. of the Weheart. showed left-invariant Bottom how one row: diffusion can artificial use equation. left-invariant normal noisygaussian We interference showed diffusionhow pattern. equation. one can CED-OS We useshowed normal is capable how Gaussian oneofcan handling use normal Gaussia rivatives tocrossings calculate regularized and bifurcations, derivatives whereas in to the CED calculate orientation produces regularized derivatives score. spurious The derivatives to artifacts nonear diffusion is steered by estimates linear for oriented diffusion feature is steered strength linear by estimates and diffusion curvature for isoriented steered by feature estimates strength for oriented and curvature feature strength and curvatu calculate at inregularized such the orientation junctions. derivatives score. in The thenon- orientation score. The no 5: Shows the typical Figure different 15: Shows behavior the typical of CED-OS Figure different 15: compared behavior Shows to the of CED. typical CED-OS In different CED-OS compared behavior crossing to CED. of CED-OS In CED-OS compared crossing to CED. In CED-OS crossing at es are arebetter obtained preserved. structures from Gaussian are better derivatives. preserved. that are Furthermore, obtained fromwe Gaussian that proposed are obtained derivatives. to usefromfurthermore, Gaussian derivatives. we proposed Furthermore, to use we proposed to u unwraps crossing lines [4, structures 7]. cf. are Fig. better 4. The preserved. key challenge in operator design for the commuting diagram, cf. ite differences that approximate thefinite left-invariance differencesofthat the approximate derivative finite differences operators. the left-invariance that approximate of the the derivative left-invariance operators. of the derivative operator Fig. 2, is to Original +Noise Original effectively exploit +Noise an CED-OS t = 30 CED t = 30 The experiments show that we are indeed The CED-OS able experiments Original orientation score, t = 30 to enhance show elongated that The its t +Noise underlying = we 30 experiments group patterns are indeedshow able CED-OS structure, to that enhance t = we 30 are and elongated indeed its intrinsic CED able patterns t = to30 invertibility. enhance elongated patter In order to achieve this, images and that including curvature inone images must helps toand appreciate enhance that including that lines an images orientation withcurvature large andcur- that helps score including to is aenhance complex-valued curvature lines with helps function large to enhance cur- on thelines Eu-witclidean motion group SE(2) = R 2 S 1, the group of rotations and translations in the plane. The construction of large cu an orientation score, cf. (1), can be written in inner-product form, W ψ [ f ](g) = (U g ψ, f ) L2 (R 2 ), with g SE(2), (2) where g U g is a reducible representation of the Euclidean motion group given by U g ψ(y)=ψ(r 1 (y x)), for all ψ L 2 (R 2 ) and g =. (x,e iθ ) SE(2). I will exploit the group structure both in the reconstruction from orientation scores and in the processing on the scores. In particular the latter has been widely neglected despite the vast literature on wavelets, edgelets, curvelets, and Gabor transforms. Instead of soft-thresholding of wavelet θ 6: Result of CED-OS Figure and 16: CED Result on of microscopy CED-OS Figure and images 16: CED of Result on bone microscopy of tissue. CED-OS Additional images and CED of Gaussian bone tissue. microscopy Additional imagesgaussian of bone tissue. Additional Gaussian added to verify the noise behaviour is added on to noisy verify images. thenoise behaviour is added on to noisy verify images. the behaviour on noisy images. re 15 shows the effect Figure of CED-OS 15 shows compared the effect to of Figure CED CED-OS on15artificial shows compared the images effect to CED with of CED-OS on crossing artificial compared line imagesto with CED crossing on artificial line images with crossing line coefficients structures. we Theaim upper forimage PDE s es. The upper image shows an additive superimposition structures. shows in the anthe wavelet/gabor additive ofupper two images superimposition with shows domain. concentric anofadditive two images superimposition with concentric of two images with concentric circles. Our method is able to preserve this structure, while CED can not. The same holds for Our method is able to preserve this structure, circles. while Our CED method can not. is able The to same preserve holds this forstructure, while CED can not. The same holds for the lower image with crossing straight lines, where it should be noted that our method leads to r image with crossing straight lines, wherethe it should lower image be noted withthat crossing our method straightleads lines, towhere it should be noted that our method leads to amplification of the crossings, which is because the lines in the original image are not superimposed tion of the crossings, which is because the lines amplification in the original of theimage crossings, are not which superimposed is because the lines in the original image are not superimposed linearly. In this experiment, no deviation from horizontality was taken into account, and the In this experiment, no deviation from horizontality linearly. In was this taken experiment, into account, no deviation and the from horizontality was taken into account, and the numerical scheme of Section 7.2 is used. The non-linear diffusion parameters for CED-OS are: al scheme of Section 7.2 is used. The non-linear numerical diffusion schemeparameters of Sectionfor 7.2CED-OS is used. are: The non-linear diffusion parameters for CED-OS are:

16 Duits Part B2 Lie Analysis, no Figure 4: Left: lines crossing in the spatial domain no longer do so when lifted to an invertible orientation score. Right: crossing blood vessels are disentangled in the score, allowing a separate segmentation. Operators on images are usually required to be Euclidean invariant, i.e. they must commute with rotations and translations. The effective operator ϒψ on the image, cf. Fig. 2, is Euclidean invariant iff the operator Φ on an orientation score is left-invariant, cf.[26, p.153]. An operator on a score is called left-invariant if it commutes with the left-regular representation. Also in the general group theoretical setting, only left-invariant operators on scores are of interest [26, 29, 30, 31]. b2. Left-Invariant Enhancement Φ on Orientation Scores In the development of suitable left-invariant operators we will establish explicit connections to stochastic and decision processes for contour completion and enhancement on orientation scores. The stochastic process model excitations (sketching) of lines in the score, whereas the direction processes model inhibitions (erasing) towards lines in the scores. Ito s formula applied to conditional transition probabilities on a stochastic process on the coupled space of positions and orientations Rd o Sd 1 := SE(d)/({0} SO(d 1)) embedded in the group SE(d) of rigid body motions on Rd, provides an evolution (convection-diffusion) that describes the dynamics of the stochastic process for contour-completion and enhancement cf. [3, 8, 32]. In case d = 2 we get the following left-invariant evolution system on the group G = SE(2):! t W (g,t) = ai Ai + Ai Di j A j W (g,t) (g G,t > 0), (3) Φ: j=1 i=1 i=1 W (g, 0) = Wψ f (g) (g G). It is generated by a left-invariant quadratic form with symmetric positive semi-definite diffusion matrix [Di j ] R3 3 and convection parameters (a1, a2, a3 ) R3, on the left-invariant vector fields A1 = θ, A2 = cos θ x + sin θ y, A3 = sin θ x + cos θ y. (4) Summarized, the orientation score of a raw grayscale image on R2 initializes a left-invariant convection-diffusion process, generating a time-dependent distribution on G = SE(2). Convection models deterministic transport, diffusion the random behavior of the stochastic process. In the linear case, with constant diffusion and convection parameters, the product of resolvents for forward and backward transport yields a so-called completion field, cf.[33]. Under the Ho rmander condition the solutions of (3) are obtained by SE(2)-convolution [34, 7, 26] with a smooth Green s function [3, 35]. Instead of manually selecting sources and sinks [33], or unstable lifts of iso-contours we use the well-posed orientation score as an initial condition [3, 4]. In the non-linear case, convection and diffusion parameters are locally adapted to the differential structure in the orientation score, cf. [4, 2, 7]. Promising feasibility studies have been conducted on non-linear contour enhancement, cf. [4, 2], clearly improving state of the art algorithms [9], recall Fig.3. As for the decision processes for contour enhancement we propose the following Hamilton-Jacobi-Bellman equations on G = SE(2): ( 1 t W (g,t) = 21 (c1 (A1W (g,t))2 + c3 (A3W (g,t))2 ) p, (g G,t > 0, 1 p 2) (5) W (g, 0) = U(g), (g G), recall (4). Here the initial condition U is the outcome of (3) for fixed t = T, or of the corresponding resolvent system. This erosion evolves towards the most salient curves in SE(2) and can be solved by a so-called

17 Duits Part B2 Lie Analysis, no morphological convolution on SE(2) with the corresponding Green s function. This generalizes morphology on grayscale images defined on R d, cf. [36], to erosion and dilation of functions on SE(d). Fig.5 shows the effect of left-invariant diffusion and erosion via the orientation score. Figure 5: From left to right: raw image f, processed image ϒ (1) ψ f = Wψ Φ W ψ f where Φ denotes a concatenation of a left-invariant convection-diffusion [3, 35] and an erosion, recall Fig. 2. Such concatenations might explain the horizontal connections in our visual cortex, recall Fig.1. b3. Optimal Pathways Within the Orientation Score Curve extraction via well-posed invertible scores is very promising in various applications (detection vascular tree in the retina [37], catheter detection for treatment of cardiac arrythmia [7], see Fig. 8) and has as main advantage over curve-extraction in the image domain that it works regardless the type of crossing (recall Fig. 4), cf. [37]. So far we have applied detection of the most salient curves in the scores by means of reasonable applied engineering techniques [37, 7, 8], that are not yet cast into a coherent geometrical control theory ([38, 39]).Within this project (Cf. Challenges VI, VII & VIII in Section b6) we will achieve this via formal curve optimization on sub-riemannian manifolds within SE(d). Furthermore, we investigate an intriguing relation between erosions (HJB-equations) on sub-riemannian manifolds within SE(d) which would allow us to extend wavefront propagation methods [40, 41] from images to scores (to avoid colliding wavefronts). For t the erosions concentrate towards curves that coincide with zero-crossings of the left-invariant vector fields involved in the Hamiltonian. Interestingly, these curves relate to sub-riemannian geodesics and elastica [35, 4, 23]. The minimizing curves can be found by Euler-Lagrange techniques on contact manifolds [39] or the Pontryagin maximum principle [38]. This provides a profoundly novel mathematical method for salient curve extraction within scores. b4. Extension to other Image Modalities: Diffusion Weighted MRI (case SE(3)) DW-MRI techniques provide for each 3D-position and for each orientation an MRI signal attenuation profile, which can be related to the local diffusivity of water molecules in the corresponding direction. Such profiles provide rich information in tissues with a fibrous architecture, such as brain white matter and muscle. The resolution of spatial and angular sampling is highly dependent on the DW-MRI protocol and subsequent modeling. For example, in case of DTI (Diffusion tensor imaging) the angular resolution is low compared to HARDI (High Angular Resolution Diffusion Imaging), cf. Fig. 6. For the purpose of tractography (detection of biological fibers) DTI HARDI fibertracking Figure 6: DTI versus HARDI: glyphs reflect the local diffusivity of water in all directions. The rank-2 limitation of a DTI tensor constrains the corresponding glyph to be ellipsoidal, whereas no such constraint applies to HARDI. However, DTI requires much less scanning directions and Challenges II, III allow us to extrapolate high angular resolution data from DTI [42].

18 Duits Part B2 Lie Analysis, no and visualization, DTI and HARDI data should be enhanced such that fiber junctions are maintained, while reducing high frequency noise and non-aligned parts in the joined domain of positions and orientations. Promising research has been done on constructing regularization of DTI and HARDI (via spherical harmonics) [5] as an essential pre-processing step for robust fiber tracking. In these approaches position and orientation space are decoupled, and diffusion is only performed over the angular part, disregarding spatial context. Consequently, these methods are inadequate for denoising/enhancement, and tend to fail precisely at the interesting locations where fibres cross. I propose to extend recent work on enhancement of elongated structures in 2D grayscale images [8, 2, 7, 35, 1, 30, 26, 4, 3] to the 3D case of HARDI/DTI, since this 2D-approach has proven to be capable of handling all aforementioned problems in various feasibility studies. b5. Goals The entire approach generalizes to the general score setting where g U g is any unitary Lie group representation of a Lie group G = R d T into the space of images. For example: modulations and translations, velocity shifts and translations, scaling and translation, scaling translations and rotations, where the score (Eq. (2)) is respectively a Gabor transform, velocity score, scale space, wavelet transform defined on the similitude group. Our goals are: (G1) Better adaptation of proper wavelets ψ in the orientation score transform. (G2) Improve the adaptation of the non-linear diffusions via the orientation scores. (G3) Detection of elongated structures. Extraction of curves inside orientation scores. (G4) Improve orientation estimations via left-invariant diffusion and erosion on SE(2). (G5) Combine the strengths of curvelet [13] and directional wavelet transforms [14] in our design to improve enhancement, completion and detection techniques using multi-scale orientation scores, cf. Fig.7. Employ this in retinal imaging where standard vessel detection fails at bifurcations. (G6) Enhancement and detection of multiple local frequency patterns in noisy medical images. (G7) Motion extraction models that deal with interference of multiple moving objects. (G8) Crossing preserving enhancement of DW-MRI, Fig. 10 and Fig. 11. Employ this to improve existing fiber-tracking algorithms and to reduce the amount of scanning directions. (G9) Develop novel geometric fiber-tracking in DW-MRI via optimal control problems within sub-riemannian manifolds within SE(3) using enhanced DW-MRI. (G10) Employ the duality between estimated frequencies from (enhanced) Gabor transforms and deformation fields to quantify cardiac wall deformations from MRI-tagging images during a heart beat cycle, cf. Fig.8. Can we avoid HARP preprocessing, cf. [43]? b6. Challenges Next we provide a list of novel, technical challenges that serve the indicated goals of the previous section. Challenge I. Development of non-linear left-invariant convection-diffusion equations and energy functionals on invertible orientation scores based on Cartan differential geometry on SE(d). As linear systems lack adaptivity, locally adaptive frames are needed to steer the non-linear process. Successful preliminary experiments have been conducted, in case d = 2, [2, 4], but important open issues remain: (G1) Can we improve the design of disk-limited proper wavelets ψ L 2 (R d ) by expansion in a windowed Zernike polynomial basis [44, 45] instead of the eigenfunctions of the Harmonic oscillator [26, 8], to get a better grip on the tradeoff between invertibility and detection quality of W ψ? (G2,G8) What are locally optimal adaptive frames? What energies to use in a variational approach? (G4) Fast contextual orientation processing, such as tensor voting [6] and channel-representations [12] rely on shift-twist convolutions on R d S d 1 that are singular in S d 1. When concatenating such convolutions with angular convolutions one does obtain some interaction between orientation layers. Can we set up a central limit theorem on R d S d 1 to relate such concatenations to our diffusions?

19 Duits Part B2 Lie Analysis, no Figure 7: Top row: original image, the real-part of the orientation score reflects the centerlines, the imaginary part of the orientation reflects the edges of the bloodvessels, the orientation score (color represents phasedirection and intensity represents the absolute value). Bottom row: visualizations of multiple scale orientation scores that allow us to include scale adaptation in our enhancement and detection. Challenge II. (G8) Development of crossing preserving diffusion on DW-MRI. Preliminary results are encouraging, [31, 42], and first extensions of adaptive Perona & Malik-type diffusions to SE(3) have been developed [46]. Can we adapt the diffusions locally to the data via crossing preserving CED on DW-MRI (and include adaptive curvature, deviation from horizontality [4], and torsion in our flows)? Challenge III. (G8) Application of erosions (HJB-PDE s) to DW-MRI for fiber tractography and connectivity analysis. The viscosity solutions of HJB equations on R 3 S 2 solved by left-invariant erosions with Green s functions or finite difference schemes. Do erosions provide a well-posed alternative to current (constrained) spherical de-convolutions? Challenge IV. (G8) Derivation of exact and approximate Green s functions for contour completion and contour enhancement on SE(3) by spectral decomposition of Fourier transform on SE(3). Challenge V. Utilize the score framework on the Heisenberg groups H(2d + 1) in medical imaging. We aim for left-invariant evolution equations on velocity scores and Gabor transforms. This provides contextual motion extraction models that deal with interference of multiple moving objects and frequencies. Employ the duality between estimated frequencies from (enhanced) Gabor transforms and deformation fields to directly quantify cardiac wall deformations from MRI-tagging images, cf. Fig.8. Challenge VI. Extraction of curves by optimal control on sub-riemannian manifolds within SE(d). In addition to [24, 38, 3] we include enhanced scores (via Challenge I,II,III) as a cost function. Apply this to fiber tracking in DW-MRI (d=3) and e.g. catheter detection in noisy X-ray (d=2). Determine the points in SE(d) that are connected via a global minimizing geodesic without cusps [24]. Challenge VII. (G3,G9) What is the right connection between erosion and diffusion, and does the erosion indeed concentrate towards optimal curves in R d S d 1? For the connection we will rely on the Fenchel transform on the Lie-algebra of left-invariant vector fields on SE(d), that maps the underlying Lagrangian to the underlying Hamiltonian. Regarding optimal curves in R d S d 1, and their connections to stochastic evolutions there are only two [31, 3, 4] reasonable options: 1. Mumford s approach [23, 47] relates elastica curves to a contour completion process, 2. Brownian bridge theory relates sub-riemannian geodesics to a contour enhancement process [48, 3]. I would like to investigate this discrepancy. Finally, if erosions converge to the optimal curves (holding for the Heisenberg approximations, cf. [35]) we can avoid boundary conditions. Challenge VIII. (G5) Derivation exact and fast approximative reconstruction schemes from multiple scale orientation scores, cf. Fig.7. Construction (numerical) data-adaptive evolutions on multiple scale orientation scores where multiple scale layers interact, to enhance, detect and complete contours at multiple scales.

20 Duits Part B2 Lie Analysis, no b7. Impact on Biomedical Imaging Applications Regarding biomedical image analysis applications, this project focuses on the analysis of complex cardiac, retinal and neuro images. Before we explain these application tracts and the role of the challenges listed in b6, we note that our generic methodology (e.g. crossing preserving diffusions and erosions on R d S d 1 ) also has serious potential impact on other fields such as robotics/mechanics [34, 12] and many other industrial applications. b7-i Cardiac Imaging Applications Regarding cardiac images we need to enhance and detect (crossing) heart fibers in medical images. Our approach via invertible orientation scores (Challenge I) is well-suited for this problem. We apply similar techniques for bi-plane navigation of catheters in low-contrast X-ray images (Challenge VI), needed in treatments of cardiac arrhythmias [7]. Furthermore, we aim for promising optical flow approaches for MRI-tagging cardiac images (cf. Challenge V). Finally, again via Challenge V, we relate changing frequency fields in dynamic MR-tagging images to deformations, cf. Fig.8. Evaluations on ground truth phantom data-sets are highly promising (compared to HARP [43]). However, application to cardiac tagging MR-datasets reveals instabilities. We will tackle this both by contextual processing on Gabor transforms (to better extract frequency fields) and by stabilizing the numerics induced by the tensorial relation between frequency field changes and deformation fields. Figure 8: Cardiac imaging applications. left column: catheter detection in noisy X-ray. Middle column: deducing deformations from MRI-tagging images: top: phantom input, bottom: deformation field net (in red) obtained by frequency changes in MRI-tagging images is close to ground truth (in black), Right column: the same technique applied to real MRI-tagging images of the heart (during a heart-beat cycle) via a polar deformation net. b7-ii Retinal Imaging Applications Regarding retinal imaging, we note that the vascular structure in the retina can be used to diagnose diabetes (and Alzheimer s disease). Standard approaches of enhancement and detection of elongated structures in medical imaging such as vesselness filtering [49] typically fail in the vicinity of crossings and bifurcations. Our approach [37] (ET-OS: edge tracking via orientation scores) depicted in Fig. 9 manifestly deals with crossing/bifurcating structures in noisy medical images and outperforms/ competes with the current state of the art vessel tracking algorithms e.g. [50, 51] in validations on ground truth datasets e.g. [52]. However, the method requires multiorientation and multi-scale adaptation in order to properly detect the smallest scale vessels and to deal with image acquisition artifacts e.g. reflections in the centerline of the vessels. Furthermore, ETOS only relies the imaginary part of an orientation score and thereby ignores the phase information. Finally, we also have to cope with challenging cases with incomplete data where human observes can only detect bloodvessel fragments by extrapolation from their context. To overcome these problems we aim (Challenge VIII) to construct multiple-scale orientation scores as depicted

21 Duits Part B2 Lie Analysis, no in Fig. 7, where we will exploit both the group-structure and the phase in the wavelet domain for subsequent enhancement, detection and segmentation of complex elongated structures. Figure 9: Top row: ET-OS [37] segmentation of the bloodvessels in a magnification retina fundus image of a healthy volunteer appropriately deals with the indicated crossings. Bottom row: fully automatic segmentation of the vascular tree and its topology works well for healthy volunteers but needs improvement in many cases of diabetic patients. ET-OS [37] segmentation results of a healthy fundus image (left), and of a diabetic fundus image (right) taken from the HRFI-database [52]. Compared to the provided golden standard segmentations, the true positives are shown in green, false positives in red, true negatives in white and false negatives in blue. b7-iii Neuro Imaging Applications Regarding neuro imaging we consider the enhancement and detection of biological fibers in DW-MRI (DTI and HARDI) images of the brain. That the PI s group theoretical approach (Challenge II) has great practical potential has been recognized witnessing an invited publication in IJCV [31] and subsequent publications [42, 53, 46] yielding crossing preserving diffusions on DTI/HARDI, Fig. 3. By including the context of all fiber fragments by probabilistic line-extension models we create crossings at locations where the original DTI inadequately yields isotropic angular diffusivity profiles. Therefore our techniques could provide an alternative to expensive high angular resolution scanning, cf. [42, 53]. The next three subsections address specific neuro-imaging applications that are test-cases for our generic approaches in Challenges II, III, VI and VII. b7-iii-1 Epilepsy Surgery A typical neuro imaging application is epilepsy surgery. Together with Kempenhaeghe (dr. P. Ossenblok) we (dr. A. Vilanova and the PI) have established a collaboration via common supervision of master projects, e.g. [54]. The challenge in epileptic surgery is to remove epileptogenic regions such that the patient is seizure free without causing neurological deficits. Anterior temporal lobe resection (ATLR) is the standard surgery for partial seizures.visual field deficits are a known complication of ATLR and occur because of disruption of the optic radiation which varies considerably between different patients. The generic techniques in Challenges II, III depicted in Fig. 10 will improve robustness and reliability of the fiber tracts obtained from DW-MRI data. Our methods can deal with the problem of crossing fibers, frequently occurring near the optic radiation, see Fig.11, Challenge II,III will improve the contextual enhancement and Challenge VI allows us to avoid the inefficient step at c) allowing robust OR extraction for all healthy controls and for patients with severe anatomical abnormalities. b7-iii-2 Deep Brain Stimulation Another neuro imaging application where this proposal has potential impact is deep brain stimulation (DBS). Stimulation of specific deep brain regions (such as the STN, subthalamic nucleus) has provided remarkable therapeutic benefits for otherwise treatment-resistant neurological disorders. DBS-therapy is a common and crucial therapy for essential tremor and advanced Parkinsons disease, chronic pain and dystonia. DBS often

22 Duits Part B2 Lie Analysis, no x Figure 10: Contextual enhancement of DW-MRI. Top: DTI data consisting of angular diffusivity profiles (glyphs) of water depicted in a slice within a region of interest in the brain where fibers of corpus callosum and corona radiata cross. Bottom: output concatenation of left-invariant diffusion and erosion applied to top DTI dataset. In the output glyphs are much better aligned with their context (via well-understood stochastic processes 1 for fiber propagation and cost processes for fiber sharpening), which is very useful for visualization and fiber tracking. a) b) c) d) e1) e2) 30% 2% 30% 2% Figure 11: Application of contextual processing via Lie group SE(3) to epilepsy surgery planning. a) In treating epilepsy surgeons should not damage the optic radiation fibers (OR). b) via fmri we know where the OR starts (V1, in red) and ends (LGN, in blue). c) current probabilistic fiber trackings generate an incredible amount of tracts selecting only those tracts that start in V1 and end in LGN. This produces a wild cluther of tracts (due to nearby crossing fibers). d) We score these tracts (showing 2% and 30% best fibers) using the enhanced DTI data and select the OR. e1) Standard scoring (e.g. [55]) of tracts produces many anatomically incorrect tracts, e2) scoring of tracts based on our enhanced DTI selects the OR much better in a healthy control case. results in a long-term improvement in motor function, however, a significant number of patients (> 55%) suffers from severe side effects after DBS. We aim for exact localization of stimulation targets for DBS based on DTI and HARDI. The enhanced DTI/HARDI images and the inherit network of fiber-tracts provides a roadmap towards target locations. Only recently, hyper-direct pathways between the motor cortex and STN have been identified [56]. Interestingly, these tracts may allow less risky stimulation via the motor cortex, but their tracking is difficult due to gray matter and nearby crossing fibers. Our techniques would, for the first time, include context of (crossing) fibers for precise, robust localization and navigation. b7-iii-3 Lesions in DW-MRI of MS-patients Multiple Sclerosis (MS) is a neurodegenerative disease that affects the brain white matter (WM). MS leads to demyelination and scarring of WM and thereby DTI fiber tracking reconstruction is greatly impaired [57] providing a potential biomarker for MS. Often in DW-MRI group analysis multiple subjects need to be aligned in the same space requiring efficient registrations. Such cumbersome procedures, are misleadingly distorted by brain atrophy effects and require very high resolution in mapping the lesions from the T 1-weighted image into the DW-MRI resulting in inaccurate automated lesion inpainting. Therefore we want to apply contextual evolutions on R 3 S 2 using DW-MRI as initial condition for fiber-completion allowing for better lesion inpainting and streamline tractography, and ultimately proposing fiber-based biomarkers for MS based on both original and evolved data per subject. We will collaborate with dr.p.rodrigues and dr.v.prčkovska (Hospital Clinic, center for Neuroimmunology, Dep. of Neurosciences, Barcelona) on this topic.