Image Registration Lecture 1: Introduction

Transcription

1 Image Registration Lecture 1: Introduction Prof. Charlene Tsai Outline Syllabus Registration problem Applications of registration Components of a solution Thematic questions underlying registration Software toolkits 2 1

2 Syllabus - Topic Image registration: Determining the mapping between two images of the same object, similar objects, the same region or similar regions Image registration is the process of determining the spatial transform that maps points from one image to homologous points on an object in the second image. All aspects of the problem will be covered: Underlying mathematics Images Algorithms Implementations Applications Special emphasize on software toolkits 3 Syllabus - Hours Lecture hours: Monday & Wednesday: 9:10~12noon 4 2

3 Syllabus - Prerequisites Data structures Calculus Linear algebra: Vectors and matrices Experience working with images C++ programming experience Templates! 5 Syllabus - Requirements 6% - attendance 48% - 6 weekly homework assignments (8% each) 22% - 10-page research report (due before 23:59 of 8/16, Sunday) 25% - final exam (8/19, Monday). Late assignments will not be accepted without prior arrangement or a verified personal emergency For all homework and research assignments, students can work in pairs, and each team only needs to hand in one copy. 6 3

4 Syllabus - Course Materials Powerpoint lectures will be placed on the course website Software toolkits will include tutorials Reading materials, mostly journal papers, will also be placed on the website Most lecture slides by courtesy of Prof Chuck Stewart from RPI and Dr. Luis Ibanez from Kitware. 7 Syllabus - Topics Introduction Mathematical background First examples Intensity-based registration and ITK Feature-based registration and the RPI toolkit Initialization techniques 8 4

5 (cont) Multiresolution techniques Mutual information Video registration and image mosaics Deformable registration Project presentation 9 Syllabus - Academic Integrity PLAGIARISM IS PROHIBITED. Solutions must be written in students own words and, copying tutorials is also considered as plagiarism. The research report must include appropriate citations A serious incident will result in failing the course 10 5

6 Registration Problem Definition p = (825,856) Pixel location in first image q = T(p;a) q = (912,632) Homologous pixel location in second image Pixel location mapping function 11 Example Mapping Function p = (825,856) q = (912,632) Pixel scaling and translation 12 6

7 Registration Problem Definition p = (825,856) q = T(p;θ) Problems: Form of mapping function T Unknown mapping parameters θ Unknown correspondences, p,q q = (912,632) Chicken-and-egg problem 13 Applications: Multimodal Integration Two or more different sensors view same region or volume Different viewpoints (Some specialized sensors have two or more coincident modalities, so registration is not needed.) Different information is prominent in each image The images may even have different dimensions! Range images vs. intensity images CT volumes vs. fluoro images 14 7

8 Example: MR-CT Brain Registration MR (magnetic resonance) measures water content CT measures x-ray absorption Bone is brightest in CT and darkest in MR Both images are 3d volumes MR CT Source: 15 MR-CT Registration Results Aligned images Superimposed images, with bone structures from CT in green 16 8

9 Retinal Angiogram and Color Image 17 Applications: Image Mosaics Many, partially overlapping images No one gives a complete view Goal: stitch images together Requires: Limited camera viewpoint such as rotation about optical center Simple surface geometry such as plane or quadratic 18 9

10 Retinal Image Mosaics 19 Sea-Floor Mosaics Courtesy Woods Hole Oceanographic Institution 20 10

11 Spherical Mosaics Images from Sarnoff Corporation 21 Applications: Building 3d Models Range scanners store an (x,y,z) measurement at each pixel location Each range image gives a partial view Must register range images and texture map them Applications: Reverse engineering Digital architecture and archaeology 22 11

12 Examples 23 Applications: Change Detection Images taken at different times Following registration, the differences between the images may be indicative of change Deciding if the change is really there may be quite difficult 24 12

13 Retinal Change Example 25 Regions Showing Change 26 13

14 Applications: Video Super-Imposed on 3d Model Taken from Sarnoff Corporation research 27 Other Applications Multi-subject registration to develop organ variation atlases. Used as the basis for detecting abnormal variations Object recognition - alignment of object model instance and image of unknown object Industrial inspection Compare CAD model to instance of part to determine errors in manufacturing process 28 14

15 Steps Toward a Solution Analyze the images Determine the appropriate image primitives Determine the transformation model Geometric and intensity Design an initialization technique Develop constraints and an error metric on the transformation estimate Design a minimization algorithm Develop a convergence criteria 29 Software Toolkits ITK Medical image processing, segmentation, and registration toolkit C++, heavily templated, data flow architecture Registration stresses intensity-based approaches VXL Computer vision applications C++, moderate templating Registration stresses feature-based approaches 30 15

16 Summary: Pervasive Questions Three questions to consider in approaching any registration problem: What intensity information or image structures is/are consistent between the images to be registered? What is the geometric relationship between the image coordinate systems? What prior information can be used to constrain the domain of possible transformations? 31 Homework Problem Due Wednesday, July 8 th before class (via to me) Problem: Find an application of registration, preferably in a research area of interest to you. In a short writeup (less than a full page), describe the problem and attempt to sketch answers to the three Pervasive Questions posed