Machine Vision: Theory, Algorithms, Practicalities

Machine Vision: Theory, Algorithms, Practicalities 2nd Edition E.R. DAVIES Department of Physics Royal Holloway University of London Egham, Surrey, UK ACADEMIC PRESS San Diego London Boston New York Sydney Tokyo Toronto

Preface to First Edition xxi Preface to Second Edition xxiv About the Author xxvi Glossary of Acronyms and Abbreviations Acknowledgements xxix xxvii 1 Vision, the Challenge 1 1.1 Introduction man and his senses 1 1.2 The nature of vision 3 1.2.1 The process of recognition 3 1.2.2 Tackling the recognition problem 4 1.2.3 Object location 7 1.2.4 Scene analysis 10 1.2.5 Vision as inverse graphics 10 1.3 Automated visual inspection 11 1.4 What this book is about 13 1.5 The following chapters 14 1.6 Bibliographical notes 15 Part 1 Low-Level Processing 2 Images and Imaging Operations 19 2.1 Introduction 19 2.1.1 Grey scale versus colour 21 VII

VIM 2.2 Image processing operations 22 2.2.1 Some basic operations on grey-scale images 23 2.2.2 Basic operations on binary images 27 2.2.3 Noise suppression by image accumulation 31 2.3 Convolutions and point spread functions 33 2.4 Sequential versus parallel operations 36 2.5 Concluding remarks 38 2.6 Bibliographical and historical notes 38 2.7 Problems 39 3 Basic Image Filtering Operations 41 3.1 Introduction 41 3.2 Noise suppression by Gaussian smoothing 43 3.3 Median filtering 46 3.4 Mode filtering 48 3.5 Bias generated by noise suppression filters 55 3.5.1 Theory of edge shifts caused by median filters in binary images 56 3.5.2 Edge shifts caused by median filters in grey-scale images 61 3.5.3 Edge shifts arising with hybrid median filters 64 3.5.4 Problems with statistics 66 3.6 Reducing computational load 66 3.6.1 A bit-based method for fast median filtering 69 3.6.2 VLSI implementation of the median filter 71 3.7 The role of filters in industrial applications of vision 71 3.8 Sharp-unsharp masking 72 3.9 Concluding remarks 73 3.10 Bibliographical and historical notes 75 3.11 Problems 76 4 Thresholding Techniques 79 4.1 Introduction 79 4.2 Region-growing methods 80 4.3 Thresholding 81 4.3.1 Finding a suitable threshold 81 4.3.2 Tackling the problem of bias in threshold selection 83

4.3.2.1 Methods based on finding a valley in the intensity distribution 85 4.3.2.2 Methods which concentrate on the peaked intensity distribution at high gradient 85 4.3.3 A convenient mathematical model 87 4.3.4 Summary 93 4.4 Adaptive thresholding 93 4.4.1 The Chow and Kaneko approach 94 4.4.2 Local thresholding methods 95 4.5 Concluding remarks 98 4.6 Bibliographical and historical notes 100 4.7 Problems 101 Locating Objects via Their Edges 103 5.1 Introduction 103 5.2 Basic theory of edge detection 104 5.3 The template matching approach 106 5.4 Theory of 3 x 3 template operators 107 5.5 Summary design constraints and conclusions 113 5.6 The design of differential gradient operators 114 5.7 The concept of a circular operator 116 5.8 Detailed implementation of circular operators 117 5.9 Structured bands of pixels in neighbourhoods of various sizes 119 5.10 The systematic design of differential edge operators 123 5.11 Problems with the above approach some alternative schemes 124 5.12 Concluding remarks 128 5.13 Bibliographical and historical notes 129 5.14 Problems 130 Binary Shape Analysis 131 6.1 Introduction 131 6.2 Connectedness in binary images 132 6.3 Object labelling and counting 133 6.4 Metric properties in digital images 137 6.5 Size filtering 140

X 6.6 The convex hull and its computation 142 6.7 Distance functions and their uses 146 6.8 Skeletons and thinning 151 6.8.1 Crossing number 153 6.8.2 Parallel and sequential implementations of thinning 156 6.8.3 Guided thinning 159 6.8.4 A comment on the nature of the skeleton 161 6.8.5 Skeleton node analysis 161 6.8.6 Application of skeletons for shape recognition 163 6.9 Some simple measures for shape recognition 163 6.10 Shape description by moments 164 6.11 Boundary tracking procedures 165 6.12 Concluding remarks 167 6.13 Bibliographical and historical notes 168 6.14 Problems 169 7 Boundary Pattern Analysis 171 7.1 Introduction 171 7.2 Boundary tracking procedures 173 7.3 Template matching a reminder 174 7.4 Centroidal profiles 174 7.5 Problems with the centroidal profile approach 176 7.5.1 Some solutions 178 7.6 The 0, V) plot 181 7.7 Tackling the problems of occlusion 183 7.8 Chain code 186 7.9 The (r, s) plot 187 7.10 Accuracy of boundary length measures 187 7.11 Concluding remarks 189 7.12 Bibliographical and historical notes 190 7.13 Problems 191 Part 2 Intermediate-Level Processing 8 Line Detection 195 8.1 Introduction 195

XI 8.2 Application of the Hough transform to line detection 196 8.3 The foot-of-normal method 200 8.3.1 Error analysis 202 8.3.2 Quality of the resulting data 204 8.3.3 Application of the foot-of-normal method 205 8.4 Longitudinal line localization 207 8.5 Final line fitting 208 8.6 Concluding remarks 208 8.7 Bibliographical and historical notes 209 8.8 Problem 210 9 Circle Detection 211 9.1 Introduction 211 9.2 Hough-based schemes for circular object detection 212 9.3 The problem of unknown circle radius 217 9.3.1 Experimental results 222 9.4 The problem of accurate centre location 224 9.4.1 Obtaining a method for reducing computational load 226 9.4.2 Improvements on the basic scheme 228 9.4.3 Discussion 230 9.4.4 Practical details 230 9.5 Overcoming the speed problem 232 9.5.1 More detailed estimates of speed 233 9.5.2 Robustness 235 9.5.3 Experimental results 237 9.5.4 Summary 240 9.6 Concluding remarks 242 9.7 Bibliographical and historical notes 242 9.8 Problem 243 10 The Hough Transform and Its Nature 245 10.1 Introduction 245 10.2 The generalized Hough transform 246 10.3 Setting up the generalized Hough transform some relevant questions 248 10.4 Spatial matched filtering in images 248

XII 10.5 From spatial matched filters to generalized Hough transforms 249 10.6 Gradient weighting versus uniform weighting 251 10.6.1 Calculation of sensitivity and computational load 252 10.7 Summary 255 10.8 Applying the generalized Hough transform to line detection 256 10.9 An instructive example 257 10.10 Tradeoffs to reduce computational load 258 10.11 The effects of occlusions for objects with straight edges 260 10.12 Fast implementations of the Hough transform 263 10.13 The approach of Gerig and Klein 266 10.14 Concluding remarks 267 10.15 Bibliographical and historical notes 268 11 Ellipse Detection 271 11.1 Introduction 271 11.2 The diameter bisection method 271 11.3 The chord-tangent method 274 11.4 Finding the remaining ellipse parameters 275 11.5 Reducing computational load for the generalized Hough transform method 277 11.5.1 Practical details 282 11.6 Comparing the various methods 286 11.7 Concluding remarks 288 11.8 Bibliographical and historical notes 289 11.9 Problems 290 12 Hole Detection 291 12.1 Introduction 291 12.2 The template matching approach 291 12.3 The lateral histogram technique 293 12.4 The removal of ambiguities in the lateral histogram technique 294 12.4.1 Computational implications of the need to check for ambiguities 294 12.4.2 Further detail of the subimage method 296

XIII 12.5 Application of the lateral histogram technique for object location 298 12.5.1 Limitations of the approach 302 12.6 A strategy based on applying the histograms in turn 303 12.7 Appraisal of the hole detection problem 305 12.8 Concluding remarks 307 12.9 Bibliographical and historical notes 308 12.10 Problems 308 13 Polygon and Corner Detection 309 13.1 Introduction 309 13.2 The generalized Hough transform 310 13.2.1 Straight edge detection 310 13.3 Application to the detection of regular polygons 311 13.4 The case of an arbitrary triangle 312 13.5 The case of an arbitrary rectangle 313 13.6 Lower bounds on the numbers of parameter planes 315 13.7 An extension of the triangle result 319 13.8 Discussion 320 13.9 Determining orientation 324 13.10 Why corner detection? 324 13.11 Template matching 325 13.12 Second-order derivative schemes 326 13.13 A median-based corner detector 329 13.13.1 Analysing the operation of the median detector 329 13.13.2 Practical results 332 13.14 The Hough transform approach to corner detection 334 13.15 The lateral histogram approach to corner detection 338 13.16 Corner orientation 340 13.17 Concluding remarks 340 13.18 Bibliographical and historical notes 342 13.19 Problems 343 Part 3 Application-Level Processing 14 Abstract Pattern Matching Techniques 347 14.1 Introduction 347

xiv 14.2 A graph-theoretic approach to object location 348 14.2.1 A practical example locating cream biscuits 353 14.3 Possibilities for saving computation 356 14.4 Using the generalized Hough transform for feature collation 359 14.4.1 Computational load 360 14.5 Generalizing the maximal clique and other approaches 362 14.6 Relational descriptors 363 14.7 Search 367 14.8 Concluding remarks 368 14.9 Bibliographical and historical notes 369 14.10 Problems 370 15 The Three-Dimensional World 373 15.1 Introduction 373 15.2 Three-dimensional vision the variety of methods 374 15.3 Projection schemes for three-dimensional vision 376 15.3.1 Binocular images 378 15.3.2 The correspondence problem 380 15.4 Shape from shading 384 15.5 Photometric stereo 389 15.6 The assumption of surface smoothness 391 15.7 Shape from texture 393 15.8 Use of structured lighting 394 15.9 Three-dimensional object recognition schemes 396 15.10 The method of Ballard and Sabbah 398 15.11 The method of Silberberg et al. 401 15.12 Horaud 's junction orientation technique 402 15.13 The 3DPO system of Bolles and Horaud 407 15.14 The IVISM system 409 15.15 Lowe's approach 411 15.16 Concluding remarks 413 15.17 Bibliographical and historical notes 414 15.18 Problems 416 16 Tackling the Perspective N-Point Problem 417 16.1 Introduction 417 16.2 The phenomenon of perspective inversion 417

xv 16.3 Ambiguity of pose under weak perspective projection 419 16.4 Obtaining unique solutions to the pose problem 422 16.4.1 Solution of the three-point problem 426 16.4.2 Using symmetric trapezia for estimating pose 427 16.5 Concluding remarks 428 16.6 Bibliographical and historical notes 429 17 Motion 431 17.1 Introduction 431 17.2 Optical flow 431 17.3 Interpretation of optical flow fields 435 17.4 Using focus of expansion to avoid collision 437 17.5 Time-to-adjacency analysis 439 17.6 Basic difficulties with the optical flow model 441 17.7 Stereo from motion 441 17.8 Applications to the monitoring of traffic flow 444 17.8.1 The system of Bascle et al. 444 17.8.2 The system of Koller et al. AA1 17.9 Concluding remarks 451 17.10 Bibliographical and historical notes 452 18 Invariants and Their Applications 455 18.1 Introduction 455 18.2 Cross ratios: the "ratio of ratios" concept 457 18.3 Invariants for noncollinear points 461 18.4 Invariants for points on conies 465 18.5 Concluding remarks 467 18.6 Bibliographical and historical notes 468 19 Automated Visual Inspection 471 19.1 Introduction 471 19.2 The process of inspection 472 19.3 Review of the types of object to be inspected 473 19.3.1 Food products 473 19.3.2 Precision components 474 19.3.3 Differing requirements for size measurement 475

XVI 19.3.4 Three-dimensional objects 475 19.3.5 Other products and materials for inspection 476 19.4 Summary the main categories of inspection 476 19.5 Shape deviations relative to a standard template 478 19.6 Inspection of circular products 479 19.6.1 Computation of the radial histogram: statistical problems 482 19.6.2 Application of radial histograms 485 19.7 Inspection of printed circuits 486 19.8 Steel strip and wood inspection 488 19.9 Inspection of products with high levels of variability 488 19.10 X-ray inspection 493 19.11 Bringing inspection to the factory 497 19.12 Concluding remarks 499 19.13 Bibliographical and historical notes 500 20 Statistical Pattern Recognition 503 20.1 Introduction 503 20.2 The nearest neighbour algorithm 504 20.3 Bayes'decision theory 507 20.4 Relation of the nearest neighbour and Bayes' approaches 510 20.4.1 Mathematical statement of the problem 510 20.4.2 The importance of the nearest neighbour classifier 513 20.5 The optimum number of features 513 20.6 Cost functions and error-reject tradeoff 514 20.7 Cluster analysis 516 20.7.1 Supervised and unsupervised learning 516 20.7.2 Clustering procedures 517 20.8 Principal components analysis 522 20.9 The relevance of probability in image analysis 525 20.10 Concluding remarks 526 20.11 Bibliographical and historical notes 527 20.12 Problems 528 21 Biologically Inspired Recognition Schemes 529 21.1 Introduction 529

XVII 21.2 Artificial neural networks 530 21.3 The back-propagation algorithm 536 21.4 MLP architectures 539 21.5 Overfitting to the training data 540 21.6 Optimizing the network architecture 543 21.7 Hebbian learning 544 21.8 Case-study: noise suppression using ANNs 549 21.9 Genetic algorithms 554 21.10 Concluding remarks 557 21.11 Bibliographical and historical notes 558 22 Texture 561 22.1 Introduction 561 22.2 Some basic approaches to texture analysis 564 22.3 Grey-level co-occurrence matrices 566 22.4 Laws' texture energy approach 569 22.5 Ade's eigenfilter approach 572 22.6 Appraisal of the Laws and Ade approaches 574 22.7 Fractal-based measures of texture 576 22.8 Shape from texture 577 22.9 Markov random field models of texture 578 22.10 Structural approaches to texture analysis 579 22.11 Concluding remarks 579 22.12 Bibliographical and historical notes 580 23 Image Acquisition 583 23.1 Introduction 583 23.2 Illumination schemes 584 23.2.1 Eliminating shadows 586 23.2.2 Arranging a region of uniform illumination 589 23.2.3 Use of linescan cameras 590 23.3 Cameras and digitization 592 23.3.1 Digitization 595 23.4 The sampling theorem 596 23.5 Concluding remarks 600 23.6 Bibliographical and historical notes 600

XVIII 24 The Need for Speed: Real-Time Electronic Hardware Systems 603 24.1 Introduction 603 24.2 Parallel processing 604 24.3 SIMD systems 606 24.4 The gain in speed attainable with N processors 608 24.5 Flynn's classification 609 24.6 Optimal implementation of an image analysis algorithm 24.6.1 Hardware specification and design 613 24.6.2 Basic ideas on optimal hardware implementation 613 24.7 Board-level processing systems 615 24.8 VLSI 616 24.9 Concluding remarks 617 24.10 Bibliographical and historical notes 618 Part 4 Perspectives on Vision 25 Machine Vision, Art or Science? 623 25.1 Introduction 623 25.2 Parameters of importance in machine vision 624 25.3 Tradeoffs 626 25.3.1 Some important tradeoffs 627 25.3.2 Tradeoffs for two-stage template matching 628 25.4 Future directions 629 25.5 Hardware, algorithms and processes 631 25.6 A retrospective view 631 25.7 Just a glimpse of vision? 633 25.8 Bibliographical and historical notes 633 Appendices Appendix A: Programming Notation 635 A. 1 Introduction 635 A.2 The Pascal language 636 A.2.1 Control structures 636 A.2.2 Procedures and functions 638

XIX A.2.3 Other details of Pascal syntax 639 A.2.4 The need for special syntax 642 A.3 Special syntax embedded in Pascal 642 A.3.1 Image handling notation 642 A.3.2 Other succinct notation 643 A.4 On the validity of the "repeat until finished" construct 646 Appendix B: Mathematical Morphology 647 B.l Introduction 647 B.2 Dilation and erosion in binary images 648 B.2.1 Dilation and erosion 648 B.2.2 Cancellation effects 648 B.2.3 Modified dilation and erosion operators 649 B.3 Mathematical morphology 649 B.3.1 Generalized morphological dilation 649 B.3.2 Generalized morphological erosion 651 B.3.3 Duality between dilation and erosion 652 B.3.4 Closing and opening 655 B.3.5 Hit-and-miss transform 656 B.3.6 Template matching 657 B.4 Connectedness-based analysis of images 658 B.4.1 Skeletons and thinning 658 B.5 Concluding remarks 660 B.6 Bibliographical and historical notes 660 Appendix C: Image Transformations and Camera Calibration 663 C. 1 Introduction 663 C.2 Image transformations 663 C.3 Camera calibration 668 C.4 Intrinsic and extrinsic parameters 671 C.5 Concluding remarks 675 C.6 Bibliographical and historical notes 675 Appendix D: Robust statistics 677 D. 1 Introduction 677 D.2 Preliminary definitions and analysis 680

XX D.3 The M-estimator (influence function) approach 683 D.4 The least median of squares approach to regression 688 D.5 Overview of the robustness problem 691 D.6 Concluding remarks 693 D.7 Bibliographical and historical notes 694 References 695 Subject Index 725 Author Index 741