25 th SSIP Summer School on Image Processing 17 July 2017, Novi Sad, Serbia Continuous and Discrete Image Reconstruction Péter Balázs Department of Image Processing and Computer Graphics University of Szeged, HUNGARY
Steps of Machine Vision Image acquisition Preprocessing Segmentation Feature extraction Classification, interpretation Actuation 2
Image Acquisition by visible light by X-rays 3
1895 - Wilhelm Conrad Röntgen describes the properties of X-rays X-rays Kind of electromagnetic radiation (similar to light but having more energy) Attenuation of X-rays depends on tissue Shadow of the object from one direction
X-rays are Useful in Radiology 5
X-rays are Useful in Radiology (in some cases) 6
and in Security 7
and in Industrial Quality Control 8
and in Food Industry and Metals, Stone, Glass, Rubber, Bone, Void http://techvalley.co.kr/eng/mlb-inspection.html?ckattempt=1 9
1000 year old Buddha Statue http://blogs.discovermagazine.com/d-brief/2015/02/23/xrays-buddhist-statue-mummified-monk/#.vw69fvmltiu 10
and Its Inhabitant http://blogs.discovermagazine.com/d-brief/2015/02/23/xrays-buddhist-statue-mummified-monk/#.vw69fvmltiu 11
Tomography Tomos = part, section Grapho = to write Tomos + Grapho imaging by cross-sections (slices) 12
Computerized Tomography A technique for imaging the 2D cross-sections of 3D objects (human organs) without seriously damaging them Take X-ray images from many angles and combine them in a clever way X-ray tube beams detectors 13
Slide: Attila Kuba
A Modern CT Scanner Scanner CT image 15
Image Quality: Then and Now first CT scanners modern CT scanners 16
or Even on-line scanning of how a fly tries to fly 17
The Mathematics of CT y X-rays Reconstruct f(x,y) from its f (x,y) projections where a projection in direction u (defined by angle ϴ) can be obtained by calculating the line integrals along each line parallel to u. ϴ x g ( l, ) f ( l co s u sin, l sin u co s ) du 18
Sinogram Object Sinogram of the Object Sinogram: image of g(l, ϴ) with l and ϴ as rectilinear coordinates Reconstruction: sinogram image 19
Backprojection Summation Image (laminogram) One backprojection image Backprojection summation image (blur!) 20
Filtered Backprojection with 240 Projections 21
Filtered Backprojection The FBP reconstruction process 2D sinogram (projections) high pass filtered for all angles sinogram is backprojected into the image domain Works well only when 180 is equiangularly and densly covered MATLAB functions: radon, iradon Movie: http://hendrix.ei.dtu.dk/movies/moviehome.html 22
ART Algebraic Reconstruction Technique The interaction of the projection rays and the image pixels can be written as a system of equations Direct inverse methods are not applicable: big system underdetermined (#equations << #unknowns) possibly no solution (if there is noise) Solve it iteratively satisfying just one projection in each step 23
Digital Images position along the line 24
Projection = Line Sums 1289 25
Projection = Line Sums 1289 1248 26
Projection = Line Sums 1289 1248 1258 1146 27
Reconstruction 1289 1248 1258 1146 28
An Example 29
An Example a+b=12 c+d=8 30
An Example a+c=11 b+d=9 31
An Example a+d=5 b+c=15 32
Discrete/Binary Tomography FBP and ART need several hundreds of projections time consuming expensive may damage the object not possible In certain applications the range of the function to be reconstructed is discrete and known DT (only few (2-10) projections are needed) Binary Tomography: the range of the function is {0,1} (absence or presence of material) 33
Tomography from a Few Projections projections unknown image 34
Tomography from a Few Projections projections continuous reconstruction 35
Tomography from a Few Projections projections binary tomography 36
Binary Reconstruction from 2 Projections?? 37
Binary Reconstruction from 2 Projections? 38
Nonograms 39
Example for Uniqueness 40
Example for Inconsistency 41
Classification 42
Two Main Problems Reconstruction: Construct a binary image from its projections. Uniqueness: Is a binary image uniquely determined by a given set of projections? 43
Reconstruction Ryser, 1957 from row sums R and column sums S Order the elements of S in a non-increasing way by π S Fill the rows from left to right B (canonical matrix) Shift elements from the rightmost columns of B to the columns where S(B) < S Reorder the colums by applying the inverse of π 44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Uniqueness and Switching Components The presence of a switching component is necessary and sufficient for non-uniqueness 61
62 Ambiguity Due to the presence of switching components there can be many solutions with the same two (or even more) projections 2 3 3 3 3 2 3 5 2 2 2 3 3 3 3 2 3 5 2 2 2 3 3 3 3 2 3 5 2 2 Use prior information (convexity, smoothness, etc.) of the binary image to be reconstructed
Reconstruction as Optimization x 1 x 2 x 3 x 4 x 5 x 6 63
Optimization Problems: P x binary variables big system b x { 0,1 } underdetermined (#equations << #unknowns) possibly no solution (if there is noise) C ( x) x {0,1} Px b 2 m n g ( x) m min n Term for prior information: smoothness, similarity to a model image, etc. 64
Solving the Optimzation Task Problem: Classical hill-climbing algorithms can become trapped in local minima. Idea: Allow some changes that increase the objective function. p = 1 0 < p < 1 65
Simulated Annealing Annealing: a thermodinamical process in which a metal cools and freezes. Due to the thermical noise the energy of the liquid in some cases grows during the annealing. By carefully controlling the cooling temperature the fluid freezes into a minimum energy crystalline. Simulated annealing: a random-search technique based on the above observation. 66
Outline of SA Set inital solution x and temperature T 0 Modify x act x Calculate C(x ) C(x ) < C(x act )? x act = x Y x act = x with probability p=e - C/T N Lower temperature N Termination? Modify x act x Y Stop Greater uphill steps are less probable to be accepted in later phases of the process 67
Finding the Optimum Tuning the parameters appropriately SA finds the global optimum Fine-tuning of the parameters for a given optimization problem can be rather delicate Sourse: https://www.slideshare.net/idforjoydutta/simulated-annealing-24528483 68
69 SA in Pixel Based Reconstruction A binary matrix describes the binary image Initial suggestion: random binary matrix Randomly invert matrix value(s) 0 0 1 1 0 0 0 0 0 1 1 1 1 1 1 0 1 0 1 1 0 0 0 0 0 1 1 1 0 1 1 0
Nondestructive Testing: Pipe Corrosion, Deposit, Crack, etc. Study 32 fan beam X-ray projections 70
Results with and without Noise no noise 10 % Gaussian noise Source: A. Nagy 71
Neutron Tomography Gas pressure controller 18 projections, also multilevel FBP DT Source: A. Nagy 72
Further Applications pebble beds metal-, plastic foams cracks air bubbles, metal alloy defects 73
Electron Microscopy QUANTITEM: a method which provides quantitative information for the number of atoms lying in a single atomic column from HRTEM images Possible to detect crystal defects (e.g. missing atoms) Source: Batenburg, Palenstijn 74
and Many More http://www.nottingham.ac.uk /microct/gallery/index.aspx 75