Using MATLAB and Simulink for Image and Video Processing

Transcription

1 Using MATLAB and Simulink for Image and Video Processing Sofia Mosesson Application Engineer 2007 The MathWorks, Inc.

2 Agenda 9:00 a.m. Using MATLAB for Image Processing 10:30 a.m. Break 10:45 a.m. Using Simulink for Video Processing 11:45 a.m. System Design and Implementation 12:00 a.m. MathWorks Overview and Wrap-up 2

3 Some Image Processing Applications Automotive Defense Biometrics Medical diagnosis Electronics Microscopy Semiconductors Drug discovery 3

4 Common Challenges Gather data from multiple sources Closely inspect images to gain insight Automate standard image analysis tasks Create new algorithms or modify existing ones Share results and generate reports Create applications and deliver tools 4

5 Using MATLAB for Image Processing Image processing Image analysis Visualization Algorithm development Image acquisition Idea sharing & reporting Data access Application deployment 5

6 Demo: Lane Marking Detection Identify lane markings in a sample image using two techniques 6

7 The language of technical computing Data access Visualization Data analysis Algorithm development Sharing results 7

8 Go Farther with MATLAB and Toolboxes Statistics Toolbox Signal Processing Toolbox Mapping Toolbox Database Toolbox Image Processing Toolbox Image Acquisition Toolbox MATLAB Compiler 13

9 Image Processing Toolbox Perform image processing, analysis, visualization, and algorithm development Image visualization Image pre- and postprocessing Image analysis Spatial transformations Image registration Color image processing 14

10 Image Visualization Graphical tools Overview Pixel region detail Image information Contrast adjustment Pan and zoom Open and modular Easy to customize Easy to extend 15

11 Image Pre- and Post-Processing Enhancement Noise removal Deblurring Region-based processing Transforms (FFT, DCT, ) 16

12 Image Analysis Edge detection Segmentation Morphological operators Image statistics Boundary tracing Region properties Texture analysis Hough transform 17

13 Spatial Transformations and Image Registration Supports many types of spatial transformations Affine, polynomial, user-defined, projective, piecewise-linear Apply multi-dimensional spatial transformations with a flexible engine Perform control point-based image registration 18

14 Color Image Processing with MATLAB Convert between color spaces Create device-independent image processing algorithms Manage ICC color profiles Quantitatively measure color accuracy Perform soft-proofing Supported Color Spaces: srgb CIELAB, CIEXYZ uvl, HSV, YCrCb, L*ch 19

15 Return to Lane Marking Demo Directory browser Workspace browser Import wizard imtool imcrop Cell mode execution Thresholding Morphological operators Region properties Hough transform 20

16 Image Acquisition Toolbox Stream images and video directly into MATLAB Configure device properties Background image acquisition Live video previewing Support for multiple hardware vendors 21

17 Image Acquisition Toolbox: Supported Hardware Frame grabbers DALSA Coreco Data Translation Matrox Imaging Digital cameras Hamamatsu QImaging DCAM-compatible cameras Windows video devices (WDM or VFW) Third-party adaptors 22

18 Capturing Video from a Webcam >> vidobj = videoinput('winvideo');... >> inspect(vidobj)... >> start(vidobj)... >> data = getdata(vidobj);... >> delete(vidobj); >> clear(vidobj); } Create Configure Connect Access Clean-up Demonstration next 23

19 Designing a Graphical User Interface with MATLAB GUIDE (>> guide) Design and edit GUI Add buttons, pull-down menus, list boxes, MATLAB plots, and ActiveX components Add callbacks Generate MATLAB code Design it yourself in code Demonstration next 24

20 Application Deployment with MATLAB MATLAB Compiler MATLAB Builder for.net MATLAB Builder for Java MATLAB Builder for Excel.EXE.DLL.LIB COM.NET www Java Excel Add-in 25

21 Application Deployment Workflow MATLAB Desktop End-User Machine 27

22 Case Study: MetsGUI Microscopy application in MATLAB to find metastasized tissue (mets) in cancerous specimens Use GUIDE to build or modify GUI 28

23 Why use MATLAB for Image Processing? Complete environment Verified and trusted algorithms Faster than hand-coding algorithms in C/C++ Open and flexible architecture enables a customized workflow 29

24 Customer Profile: Given Imaging The Challenge To find an alternative to the endoscope and other invasive gastrointestinal imaging procedures The Solution Use MATLAB and MATLAB toolboxes to implement and develop an alternative diagnostic tool The M2A capsule, shown next to a US quarter The Results Fast, efficient development Easy access to precise diagnostic information As of 2007, shipped 500,000 PillCams MATLAB and the MATLAB Image Processing Toolbox enabled us to cut down development time by several orders of magnitude." Dr. Arkady Glukhovsky Given Imaging 30

25 From a Leading Textbook Author... "I have used a number of commercial image processing packages over the years, and prefer The MathWorks Image Processing Toolbox for several reasons: the wide variety of functions it provides, the user s ability to write additional functions with minimal effort, the quality of the software, and the high level of support." Rafael C. Gonzalez Professor Emeritus University of Tennessee 31

26 Agenda 9:00 a.m. Using MATLAB for Image Processing 10:30 a.m. Break 10:45 a.m. Using Simulink for Video Processing 11:45 a.m. System Design and Implementation 12:00 a.m. MathWorks Overview and Wrap-up 32

27 Agenda 9:00 a.m. Using MATLAB for Image Processing 10:30 a.m. Break 10:45 a.m. Using Simulink for Video Processing 11:45 a.m. System Design and Implementation 12:00 a.m. MathWorks Overview and Wrap-up 33

28 Tools for a Complete Workflow Research and development System design Implementation and test Data Access Data Data Analysis & Visualization Mathematical Modeling Algorithm Design Environment effects Embedded algorithms System components System Modeling, Simulation and and Partitioning Rapid Prototyping Code Code Generation Embedded Software Verification, HIL HIL Test Test Embedded Hardware 34

29 Some Video Processing Applications Unmanned vehicle surveillance Automotive active safety Video conferencing Security cameras Office equipment Electronics 35

30 Challenges of Embedded Design Extreme computational demands of video Real-time and concurrency requirements Manage trade-offs for embedded implementation Power, performance, size, and cost Target hardware options: RISC processors, DSPs, FPGAs, etc Testing and validation without hardware 36

31 Model Based Design 37

32 Model-Based Design Requirements and specifications Design Implementation Test and verification Model elaboration Executable specifications Design with simulation Automatic code generation Continuous verification 38

33 Using Simulink for Video Processing Analyze system trade-offs Automatically generate C-code Target embedded devices Implicit timing and concurrency Continuous verification Rapidly prototype ideas 39

34 What is Simulink? Simulation, modeling, and design tool GUI-based block diagram environment Platform for Model-Based Design 40

35 Simulink Key Features Hierarchical, component-based modeling MATLAB integration Extensive library of predefined blocks Application-specific libraries available Open Application Program Interface (API) 41

36 Video and Image Processing Blockset Model, simulate, implement, and verify real-time video and imaging systems Includes over 60 components and 100 s of algorithms Ideal for implementation of embedded systems Streaming video in/out Detection, thresholding Tracking, counting Background estimation 42

37 Video and Image Processing Blockset Analysis and enhancement Conversions Filtering Geometric transforms Morphological operations Sinks Sources Statistics Text and graphics Transforms Utilities 43

38 Demo: Edge Detection Build a basic model in Simulink with the Video and Image Processing Blockset 45

39 Demo: Edge Detection Easy to import streaming video into the simulation Handy viewers for inspecting video at any point in the algorithm Flexible block implementing several edge detection methods 46

40 Demo: Edge Detection Live Video Build a basic model in Simulink with the Video and Image Processing Blockset 47

41 Case Study: Counting Cells Use a combination of basic morphological operators and blob analysis to count cells in a video 49

42 Case Study: Video Stabilization Stabilize video from a moving camera 50

43 Case Study: Lane Departure Warning System Detect lane markings in real time Calculate distance from lane markings to determine lane departure 51

44 Why use Simulink for Video Processing? Modeling and simulation Easily process streaming video Visualize results at each step in your model Move quickly from algorithm ideas to accurate system behavior Extensive block library Hundreds of blocks supporting video and image processing No need to recreate standard algorithms Analyze system tradeoffs Perform what-if analyses Compare algorithms against specifications for numerical accuracy, computational complexity, and more 52

45 Agenda 9:00 a.m. Using MATLAB for Image Processing 10:30 a.m. Break 10:45 a.m. Using Simulink for Video Processing 11:45 a.m. System Design and Implementation 12:00 a.m. MathWorks Overview and Wrap-up 53

46 Model-Based Design with Simulink How is this useful for video? Implicit timing and concurrency Extensive algorithm library Fixed-point modeling Automatic code generation Rapid prototyping 54

47 Fixed-Point Modeling* Avoid inaccurate results due to finite word effects Access tools built directly into block interfaces Easily change parameters to model the impact of rounding, overflow, and scaling * Requires Simulink Fixed Point for integer and fixed-point data types 55

48 Targeting Embedded Microprocessors and DSPs Real-Time Workshop Real-Time Workshop Embedded Coder Specific Peripheral S/W Drivers C-code C-compiler IDE (Integrated Design Environment) Compiler, Linker, and Loader Specific Board 57

49 Link for Code Composer Studio and Target for TI C6000 Link for Code Composer Studio Co-simulation, project automation, debugging, and verification for C2000, C5000, and C6000 DSPs Target for TI C6000 Includes target-optimized signal processing blocks Supports on-chip peripherals and evaluation boards DSPs: TI DM64x, C64x, C62x, and C67x families Key benefit: rapid prototyping and evaluation with hardware in the loop 58

50 Generating C code from a Simulink Model Using Embedded Target for TI C6000 to prototype a Simulink-based design on the DM642 platform 59

51 Co-Simulation and HDL Generation MathWorks and Third-party Products Synthesizable VHDL and Verilog from Simulink NEW Simulink HDL Coder (MathWorks) Filter Design HDL Coder (MathWorks) System Generator for DSP (Xilinx) DSP Builder (Altera) Synplify DSP (Synplicity) Co-simulation of VHDL and Verilog in Simulink Link for ModelSim (MathWorks) with ModelSim (Mentor) NEW Link for Incisive (MathWorks) with Incisive (Cadence) ModelSim Xilinx Edition (Xilinx) ModelSim Altera Edition (Altera) 66

52 The Value of Model-Based Design Model-Based Design Executable specification Design with simulation Implementation through code generation Continuous test and verification Innovation Rapid design iterations What-if studies Unique features and differentiators Quality Reduce design errors Minimize hand coding errors Unambiguous communication internally and externally Cost Reduce expensive physical prototypes Reduce re-work Automate testing Time-to-market Get the product right the first time 67

53 Key Benefit of Continuous Test and Verification 70% 60% 50% 40% 30% 20% 10% Where Errors Are Introduced... and Detected 60% 55% 22% 21% 15% 8% 12% Each delay in the detection and correction of a design error makes it an order of magnitude more expensive to fix. Clive Maxfield and Kuhoo Goyal EDA: Where Electronics Begins TechBites Interactive, October 1, 2001 ISBN: % 7% Detected Spec Design Implement Test Introduced Source: Migration from Simulation to Verification with ModelSim by Paul Yanik. EDA Tech Forum, 2004 Mar 11, Newton MA 68

54 Doheny Eye Institute Enables The Blind To See With MathWorks Tools The Challenge To develop a retinal prosthetics proof-ofconcept prototype that interfaces with a permanent micro-electronic retinal implant The Solution Used Simulink, Blocksets, Real-Time Workshop, and Target for TI C6000 to build and validate a rapid prototype on a DM642 The Results Completed phase II of research project ahead of schedule Currently in trials with patients though Doheny Eye Institute at the University of Southern California Example of the Prototype We are working on real-time image processing with the TI DM642 processor as the target. The Video and Image Processing Blockset makes the task of creating our design and working prototypes much simpler. Dr. James Weiland Director, Intraocular Retinal Prosthesis Lab 69

55 Return on Investment Study 2005 study by International Business Strategies Analysis of five cases of using Simulink for electronics system design Case Study #4: Video processor design Highlights: Increased productivity of design engineers Direct savings in design costs Improved time-to-market Increased IP reuse across design iterations and derivative projects White paper available at 70

56 Video and Imaging Design Products Base (required): MATLAB, Signal Processing Toolbox, Image Processing Toolbox Simulink, Signal Processing Blockset, Video and Image Processing Blockset Fixed Point Modeling: Fixed-Point Toolbox, Simulink Fixed Point, Simulink Accelerator Implementation: Real-Time Workshop Real-Time Workshop Embedded Coder Link for Code Composer Studio Target for TI C6000 Link for ModelSim Partner products (Altera, Xilinx, Synplify, TI, Mentor Graphics, etc) 71

57 Agenda 9:00 a.m. Using MATLAB for Image Processing 10:30 a.m. Break 10:45 a.m. Using Simulink for Video Processing 11:45 a.m. System Design and Implementation 12:00 a.m. MathWorks Overview and Wrap-up 72

58 The MathWorks at a Glance Headquarters: Natick, Massachusetts USA USA: California, Michigan, Washington DC, Texas Europe: UK, France, Germany, Switzerland, Italy, Spain, the Netherlands, Sweden Asia-Pacific: Korea, Australia Worldwide training and consulting Distributors in 25 countries Earth s topography on an equidistant cylindrical projection, created with the MATLAB Mapping Toolbox 73

59 The MathWorks Today Revenues exceeding $400M in 2006 Privately held Over 1,600 employees worldwide Worldwide revenue balance: 50% North America, 50% International More than 1,000,000 users in 175+ countries

60 Key Industries Aerospace and Defense Automotive Biotech and Pharmaceutical Chemical/Petrochemical Communications Education Electronics Financial Services Industrial Automation and Machinery Power and Energy Semiconductor 76

61 Core MathWorks Products The leading environment for technical computing Numeric computation Data analysis and visualization The de facto industry-standard, high-level programming language for algorithm development Toolboxes for signal and image processing, statistics, optimization, symbolic math, and other areas Foundation of The MathWorks product family 77

62 Core MathWorks Products The leading environment for modeling, simulating, and implementing dynamic and embedded systems Linear, nonlinear, discrete-time, continuoustime, hybrid, and multirate systems Foundation for Model-Based Design, including physical-domain modeling, automatic code generation, and verification and validation Open architecture for integrating models from other tools Applications in controls, signal processing, communications, and other system engineering areas 78

63 Advantages of The MathWorks Complete solutions for the entire development process Customer advisory boards provide strong ties to our developers Strong imaging expertise Broad network of third-party developers Technical support, training, and consulting services 79

64 Support and Community 80

65 Engineering expertise and deep product knowledge, specializing in: Application development using MATLAB Model-Based Design using Simulink and Stateflow Embedded-system development Enterprise-wide integration of MathWorks products into engineering process and systems Jumpstart services Project-based services for a growing number of industries, including aerospace and defense, automotive, communications, power and marine, and financial services 81

66 Three ways to get training Public training Offered throughout the world Schedule and course information at Onsite training Bring training to your site, with course customization available Web-based training Instructor-led e-learning Train at work or at home, with flexible dates and times Example course topics Introductory and intermediate training on MATLAB, Simulink, and Stateflow Specialized courses in control design, signal processing, test and measurement, communications, financial analysis, and other areas 82

67 MATLAB Central File exchange and newsgroup access for MATLAB and Simulink user community 184,000 visitors per month More than 4,000 contributed files in the exchange General-purpose functions, industry- and application-specific tools and examples 100 new submissions per month 5,000 downloads per day Web interface to the MATLAB Usenet forum comp.soft.sys.matlab (CSSM): Over 6,000 posts a day MATLAB Web Resources, Blogs, Programming Contests, and RSS feeds 83

68 The MathWorks Connections Program More than 300 add-on products and services from partners that complement and extend MathWorks products Specialized third-party toolboxes for MATLAB Interfaces to partners software and hardware products Specialized training courses and consulting services System integrators and suppliers that incorporate MathWorks products 84

69 990+ textbooks for educational and professional use, in 26 languages Controls Signal Processing Image Processing Biosciences Communications Mechanical Engineering Mathematics Aerospace Engineering Environmental Sciences Chemistry Finance Electronics, and more 85

70 Technical Support Resources Over 100 support engineers All with MS degrees (EE, ME, CS) Local support in North America, Europe, and Asia Comprehensive product-specific web support resources High Customer Satisfaction 95% of calls answered within three minutes 70% of issues resolved within 24 hours 80% of customers surveyed rate satisfaction at % 86

71 Next Steps, Questions and Answers Visit Product literature User Stories Visit MATLAB Central Click on File Exchange Upcoming seminars in your area Upcoming webinars Feel free to ask questions afterwards 87