OFF-LINE ROBOT VISION SYSTEM PROGRAMMING USING A COMPUTER AIDED DESIGN SYSTEM S. SRIDARAN. Thesis submitted to the Faculty of the

Transcription

1 OFF-LINE ROBOT VISION SYSTEM PROGRAMMING USING A COMPUTER AIDED DESIGN SYSTEM by S. SRIDARAN Thesis submitted to the Faulty of the Virginia Polytehni Institute and State University in partial fulfillment of the requirements for the degree of Master of Siene in Industrial Engineering and Operations Researh APPROVED: Dr. Mihael Deisenroth Dr. K. B. Yu Otober 16, 1985 Blaksburg, Virginia

2 OFF-LINE ROBOT VISION SYSTEM PROGRAMMING USING A COMPUTER AIDED DESIGN SYSTEM by S. SRIDARAN Dr. Mihael Deisenroth Industrial Engineering and Operations Researh (ABSTRACT) Robots with vision apability have been taught to reognize unknown objets by omparing their shape features with those of known objets, whih are stored in the vision system as a knowledge base. Traditionally, this knowledge base is reated by showing the robot the set of objets that it is likely to ome aross. This is done with the vision system to be used and must be done in an on-line mode. An approah to teah the robot in an off-line mode by integrating the robot vision system and an off-line graphi system, has been developed in this researh. Instead of showing the objets that the robot is likely to ome aross, graphi models of the objets were reated in an off-line graphi system and a FORTRAN program that proesses the models to extrat their shape parameters was developed. parameters were passed to the vision system. These shape A program to proess an unknown objet plaed in front of the vision sys-

3 tern was developed to extrat its shape parameters. A program that ompares the parameters of the unknown objet with those of the known models was also developed. The vision system was alibrated to measure the the pixel dimensions in inhes. In the vision system, shape parameters of the objets were found to vary with different orientations. The range of variation for eah parameter was established and this was taken into onsideration in the parameter omparison program.

4 ACKNOWLEDGEMENTS I take this opportunity to express my sinere thanks to for his wise and patient steermanship, enouragement, involvement, and inspiring guidane throughout the ourse of this researh. He aepted the awesome task of editing the manusript of this thesis, and redued the literary ineptness of the author to a minimum without altering the meaning of the written word. The author also wishes to express his thanks to all the faulty and staff of the Department of Industrial Engineering and Operations Researh for their support and enouragement. Aknowledgements iv

5 TABLE OF CONTENTS 1.0 INTRODUCTION 1.1 Bakground 1.2 Problem Definition 1.3 Sope of Researh LITERATURE REVIEW CAD System and Objet Desriptions Vision System Tehnology CAD GEOMETRIC INTERFACE AND PROCESSING CADAM System CADAM Geometry Interfae Exeution and Linkage Commands 3.2 Geometri Parameter Extration Algorithm Conatenation Phase Parameter Extration Phase 3.3 Shape Parameter Extration Program 3.4 Desription of Subroutines VISION SYSTEM 4.1 Imaging Tehnology System Desription 4.2 Programming the PCVISION frame grabber 4.3 Vision Algorithm Table of Contents v

6 4.3.1 Binary Proessing Connetivity analysis algorithm Parameter Extration Proedures 4.4 Vision system Program 4.5 Desription of Subroutines INTEGRATION OF CADAM AND PCVISION SYSTEMS 5.1 File Transfer 5.2 Calibration of the mahine vision system 5.3 Vision system parameter ranges RESULTS AND CONCLUSIONS 80 LIST OF REFERENCES 90 APPENDIX A. CAD SYSTEM PROGRAM 92 APPENDIX B. VISION SYSTEM PROGRAM 121 VITA 130 Table of Contents vi

7 LIST OF ILLUSTRATIONS Figure 1. An eight onneted pixel 16 Figure 2. Chain ode of an outline. 17 Figure 3. CADAM system 21 Figure 4. Parametri representation of a point on a irle 26 Figure 5. Determination of area of a polygon 29 Figure 6. Determination of inward pointing normal 31 Figure 7. Possible ases of ars 33 Figure 8. Determination of maximum radius 35 Figure 9. Determination of minimum radius 36 Figure 10. Mahine Vision System 48 Figure 11. Frame memory organization 49 Figure 12. Display logi 51 Figure 13. Histogram 56 Figure 14. Run-length ode 59 Figure 15. Multilabelled blob 60 Figure 16. The 16 possible states of a pixel 62 Figure 17. Variation of area 76 Figure 18. Variation of perimeter 77 Figure 19. Variation of maximum radius 78 Figure 20. Variation of minimum radius 79 Figure 21. Test Model 1 87 Figure 22. Test Model 2 88 Figure 23. Test Model 3 89 List of Illustrations vii

8 LIST OF TABLES Table 1. Variation of parameters Table 2. CAD system parameters Table 3. Parameter ranges Table 4. Vision system parameters List of Tables viii

9 1.0 INTRODUCTION 1.1 BACKGROUND The task of enabling robots to visualize and reognize objets has been the subjet of a signifiant amount of researh. This is beause of the potential advantages of a robot whih an see, over a sightless robot. A robot whih an see may be more effetive in arrying out tasks that an also be exeuted by sightless robots. This may be illustrated by an example (4). Consider that a sightless robot is used to move objets arriving via an input onveyor to either of two output onveyors where the objets will be arefully hannelled and marked by a paint brush plaed in a fixed position. These objets might be bonnets and tailgates of toy ars whih eventually would be painted blue and green respetively. Consider the tehnial problems. First, the input onveyor belt would have to ontain jigs that present the parts to the manipulator in a predetermined manner. The manipulator ontroller would have to be told exatly how many parts of eah kind are ontained in eah bath, so that the ontrol program will know when to branh between one part of the program and the other. INTRODUCTION 1

10 Now imagine the presene of a vision amera and a reognition system that an be pointed at the input onveyor belt. Clearly there would be no need for speial jigs as the vision system would not only identify the parts but also inform the manipulator as to where they were plaed. Then armed with a paint brush, the robot ould mark the parts appropriately without having to pik them up. This illustrates the way in whih the use of vision transfers the development overheads from mehanial omplexity and preision to visual proessing whih is ost-benefit effetive beause of the availability of heaper miroproessors used for vision proessing. A mahine vision system uses the information from known objets to reognize the unknown objets presented to it. There are two approahes to building a set of known objet patterns. One approah is to treat a pattern generated from a previous enounter as known and ompare this with other images. This is alled 'teah by showing' and is used in SRI system (1). Another approah is to generate patterns in an off-line mode with respet to the vision system and pass the data as known patterns to the vision system. This method is being researhed by many industries. The inherent disadvantage with 'teah by showing' method is that it is time onsuming when the user must physially present to the amera a large number INTRODUCTION 2

11 of objets or a large number of expeted onfigurations (stable states). Another disadvantage is that samples of the finished produt must be available. Additionally parameters obtained by 'teah by showing' depend upon the speifi onfiguration of the system and are not transferable to other setups. Primarily there are two ways of omparing the unknown model with the known dataset. The known objet model an be onsidered as a template and reognition is ahieved by moving the template over the unknown unti 1 ompletely aligned with the objet. the template beomes In digital system, the template is stored in memory as a two dimensional matrix. This matrix is referred to as the referene. When a frame of video data is loaded into system memory, the system overlays the referene on the upper left hand orner of the video frame and alulates the number of mathes. This proess, referred to as orrelation, ontinues until the referene has been ompared with the video frame throughout the field of view. If there is no aeptable math anywhere in the field of view another referene may be used and the proess repeated. This ontinues until the objet is identified. An advantage of this type of system is that it is very tolerant of noise and bakground information. INTRODUCTION 3

12 The disadvantage with template system is that they annot easily reognize objets with orientation different from that of the known pattern. Another approah is alled 'mathing based on feature extration'. This is an alternative to template mathing, whih proeeds at the image level. It abstrats some measurements or features (suh as area, perimeter for a two dimensional sene) of the image and then mathes these data against the same features of different known patterns. The hoie of features is problem dependent. The features an be desribed in terms of shape, position and orientation ( 7). Pot et.al (19) argue that an interesting feature should have the following parameters: 1. It must be stable with respet to the variations of the images of the same objet. 2. It must be distinguishing. 3. It must be fast to ompute. They also reommend a number of basi parameters that may be derived from an arbitrary shape to provide valuable lassifiation information. These inlude: 1. Area INTRODUCTION 4

13 2. Perimeter 3. Minimum enlosing retangle 4. Center of area 5. Minimum and maximum radius vetors 6. Holes (number, size, position) 7. Number of orners 1.2 PROBLEM DEFINITION The problem addressed in this researh was to integrate a mahine vision system with a omputer-aided design (CAD) system operated in an off-line mode with respet to the vision system. The idea of generating patterns in an off-line mode was onsidered owing to the disadvantages of the 'teah by showing' system, and to the fat that the data in a CAD system, already available in a omputer integrated manufaturing faility, ould be onurrently used by the vision system. This enhaned the integration of the CAD and CAM systems and improved the utilization of the system by putting the CAD data to an additional use. The method of feature extration was onsidered for reognition here, beause this method an help reognize objets differing in orientation with respet to the known pattern. Moreover this was more suitable in the ontext of the objetive mentioned above, sine the information required by the INTRODUCTION 5

14 mahine vision system is only the features of the known model. This data an be extrated from the models residing in the CAD system and supplied to the mahine vision system. The integration of the two systems was effeted by passing the feature information. The CAD system used in this researh was the CADAM (R) (Computer Augmented Design And Manufaturing system) system, and the mahine vision system used was the Imaging Tehnology vision system. The different proedures involved in the integration of the mahine vision system with the CAD system were: 1. The reation of parametri desriptions of known objet models in an off-line mode by using the data from the CADAM graphi system. To aomplish this: Wire-frame desriptions of known objets were generated in the CADAM graphi system and added to the CAD database. An algorithm was developed and implemented on the graphi system to retrieve the wire-frame model of INTRODUCTION 6

15 the objet from the CAD database and extrat features from the wire-frame representation. 2. In order for objet reognition to our the features extrated from the known objet were then ompared with the features of the unknown objet presented to the mahine vision system. To aomplish this: The digitized image of the unknown objet, available in the memory of the omputer onneted to the mahine vision system, was segmented from the bakground. Then the segmented image was proessed to extrat parameters. An algorithm was developed and implemented on the omputer onneted to the mahine-vision system to aomplish the above mentioned tasks. A deision rule to ompare the parameters extrated from the CAD system with those extrated from the mahine-vision system was formulated and implemented on the vision system omputer. The units of measurement for the CAD system and those for the mahine-vision system were not ompatible sine the units of INTRODUCTION 7

16 measurement in the CAD system is in inhes while those of the mahine-vision system are in pixels. Gruver et.al (13) suggested that the vision system be alibrated to measure the mahine vision system parameters in terms of CAD system units. The vision system parameters varied a range of values, sine the number of pixels overed by the edges varies depending on the position of the part. Therefore, the range of variation of eah parameter was determined experimentally. 1.3 SCOPE OF RESEARCH In this researh only a two dimensional view of an objet was onsidered for reognition. and irular holes in them. The objets ould have polygonal The boundary of the objet had to have at least one straight edge. approximated as ars of irles. All urved edges were The view of the objet presented to the mahine vision system was the same as that stored in the CADAM system, but the former may be different in orientation with respet to the latter. In this researh only two levels of intensity were onsidered for eah pixel. That is, the image was onsidered to be binary. INTRODUCTION 8

17 The reasons for onsidering a binary image were: 1. It is evident from previous researh ( 8) that a binary image is adequate for reognition of objets for a variety of robot vision appliations. 2. The image an be represented in a ompat form. 3. The image analysis algorithms are simpler for binary images beause of the struture. 4. Sine only one bit of intensity information is required to be stored, an image of size 480 x 512 piture elements an be stored in 30, 720 bytes of memory. Thus even a modest miroomputer an store a omplete binary image of average spatial resolution and still have memory left over for image analysis software. 5. The algorithms that analyze a binary image are simpler than those whih analyze gray sale images. With a binary image, the omputations used in suh funtions as loating the edge of an objet are redued to logial operations involving only a few piture elements (pixels). As a result, binary image analysis programs are generally muh faster than omparable programs that proess gray sale images. INTRODUCTION 9

18 2.0 LITERATURE REVIEW 2.1 CAD SYSTEM AND OBJECT DESCRIPTIONS A CAD system is used to model both geometri and nongeometri parts that the user designs. In addition, a CAD database provides the means to model relationships between the omponents of an assembly in a generi fashion. The designer an desribe an objet to the graphi system in geometri terms; that is in terms of graphi output primitives suh as points, lines, polygons or harater strings geometrially oriented in a two or three dimensional world. The data struture in the graphi system stores the objet desription. It must ontain the geometri oordinate data of the graphi primitives, and onnetivity relationships and positioning data that defines how the omponents fit together. The user reates the geometri desription of objet and stores it in the data struture through a CAD appliation program. The appliation program, by issuing ommands to the graphi display system, an make the piture appear on a sreen. The graphi system software whih onsists of a group of plotting subroutines drives the speifi devies and auses the devie to display the piture. Literature Review 10

19 Three different tehniques are used in representing graphial data. The objet desription method in whih two dimensional and three dimensional objets are desribed in terms of line drawings is alled WIRE-FRAME desription. Vandam (10) says that wire-frame desriptions are omputationally effiient, but they do not give a realisti view of the objet. Additionally, they do not allow removal of hidden surfaes from the display, or alulate the weight or volume of an objet. In situations where a realisti representation of an objet is required, an objet an be defined by ombinations of mathematial surf aes suh as planes, spheres et. In this desription it is possible to remove hidden surfaes. The objets an be desribed in terms of polygonally bound surfaes alled polygonal mesh whih are easy to manipulate. These provide only an approximate desription of urved surf aes. This tehnique is alled SURFACE MODELLING. Another tehnique of representing an objet, alled SOLIDS MODELLING, deals diretly with solid objets, and uses solids suh as ubes, ones, spheres, and ylinders as primitives, whih are added and subtrated to form shapes. Though surfae and solids modelling tehniques give a more realisti desription, they are omputationally very expensive and are yet to be brought out of the laboratory. Literature Review 11

20 Vandam ( 10) reommended the use of wire-frame models in an engineering environment, sine all engineering drawings are two dimensional line representations of the objet. Moreover wire-frame models give an aurate representation of the orthographi projetion of a partiular view of an objet. This is suffiient for many reognition purposes, though protrusions in an orthogonal diretion with respet the projeted view are not represented well. In many ommerially available graphi systems suh as CADAM, wire-frame desriptions of the objets are used for representation. Wireframe drafting gives an inexpensive, reasonably good representation of objets and sti 11 provides the rapid responses neessary for interative system use. 2.2 VISION SYSTEM TECHNOLOGY A vision system provides part identifiation by a omparison of the digitized image produed by the video-amera and digitizing logi omponents of the system, with a stored pattern of parameters. Before omparing the patterns, the digitized image has to be proessed to extrat the parameters. The image array an either be proessed as a gray level image with several levels of intensity, or they an be proessed as binary images with only two levels of intensity. Cunningham (8) reommended the use of binary images for robot vision appliations, sine they are more ompat for repre- Literature Review 12

21 sentation, and algorithms an be easily developed and moreover binary images are suffiient for reognition of the objet boundaries in a typial manufaturing environment. Cunningham also reommended the use of edge detetion tehniques using onvolution alulations suh as onnetivity analysis and boundary traking, implementable for a binary images. sine they are easily Gruver et.al (13) have developed an off-line programmed robot vision system using CAD. Their work is one of the major researh efforts in building off-line programmed robot vision system. In their work they have presented a oneptual idea of the operational aspets of a CAD based robot vision system. Their system uses the feature mathing method to reognize the unknown objets. A speial feature about the CAD system used in their work is that the objet models need not be drafted manually. The vision programming module in the CAD system has the provision to generate the objet model and to extrat features from it automatially. To take a piture of the objet, the user has to invoke the PERSPECTIVE funtion of the CAD graphis system. This produes a 2-D image of the objet. This 2-D image is then edited graphially to remove hidden lines, so that profiles of visible features on the objet result. Proedures are then invoked from the CAD system to alulate the required objet features. Literature Review 13

22 Curtis (9) has developed an off-line programmed roboti inspetion system using CAD. The vision system emulates the human inspetor. After a part is designed on the CAD system, a translation program translates the CAD information into a database suitable for inspetion purposes. Two artifiial intelligene modules AI-1, AI-2 at as interfae between this database and the vision system. AI-1 module determines and simulates the inspetion proedure and outputs general instrutions to the AI-2 module. AI-2 module performs the atual atual ommands to operate the robot and vision system. There are many reports found in the literature about extration of shape parameters from a binary image. Two appreahes to extrat shape parameters from a binary image have evolved. The first approah alled 'onnetivity analysis' involves a sequential sanning of eah line of the image and the onnetivity of eah pixel around its neighbors is used to build blob desriptions. The advantage with this method is that the proessing is done in one raster san of the image. The other approah alled 'boundary traking' follows the boundary of eah blob to extrat the shape parameters. Prior to applying the algorithm a 'hain ode' represenation of the image is required. This is a representation of an image in terms of its boundary pixels. Literature Review 14

23 Freeman (11) suggested the idea of representing an image as a hain ode. The hain ode is a list data struture. Its first entries are the x,y oordinates of the point from whih to begin traing the outline. Subsequent entries are numbers giving the diretion between eah point in the outline and its neighbor. There are eight possible diretions between a point and its neighbour (Figure 1). These eight diretions are numbered "O" through "7", ounter-lokwise. Figure 2 shows an outline and its hain ode. A.K.Agrawala and A.V.Kularni (2) desribed a sequential approah to extrat shape features from an image. Here only the edge points of eah line are used. At any instant only the edge points of the previous san line and urrent line are required for the alulations. All of the information about the urrent san line is proessed in a single pass through the image matrix. Literature Review 15

24 p 0 s 5-1 ( - Figure 1. An eight onneted pixel Literature Review 16

25 ''] 2, ) ''] 4 ;, ), s 4 Figure 2. Chain ode of an outline. Literature Review 17

26 3.0 CAD GEOMETRIC INTERFACE AND PROCESSING The first step involved in the proessing of CADAM geometri data was to reate the geometri models (of objets) in the CADAM database. The CADAM graphi system already available to the user was used for this purpose. The next step was to aess the. geometri model for the alulation of the desired shape parameters. CADAM geometry interfae routines available to the user were used to aess the geometri models. Then, an algorithm to extrat shape parameters of these graphi models was developed. An appliation program to implement this algorithm was developed and implemented. This program inludes all statements to CADAM geometry interfae routines. The shape parameters extrated were passed to the vision system program for omparison with the shape parameters of the unknown objet extrated by the vision system program. The details of the available CADAM geometry interfae routines are disussed below but the graphi routines used for the onstrution of graphi models in the CADAM system are not disussed. Also a detailed desription of the shape parameter extration algorithm developed for the CADAM system is given below. CAD Geometri Interfae and Proessing 18

27 3.1 CADAM SYSTEM The CADAM system is a set of omputer programs that an be used to prepare mehanial drawings on a omputer terminal. The CADAM system an be used to onstrut a wire frame model of an objet by providing data about primitive elements suh as lines, points, irles et. Sine eah graphi entity is assoiated with a speifi view (that has a geometri relationship with other world views), any desired view of an This inobjet an be reated by graphial onstrution. eludes isometri view as well as other auxiliary views. Graphi model data are stored in a file for reall and modifiation. This data is also aessible for manipulation by appliation programs for external digestion and analysis CADAM GEOMETRY INTERFACE The geometry interfae module available in the system provides the ability to aess a partiular CADAM file in the CADAM database system from an appliation program instead of from a graphis terminal. The interfae modules at as interfae between an appliation program written in a higher level language and the plotting routines and routines available in the graphi system. other software There are four geometry interfae funtions available in CADAM. CAD Geometri Interfae and Proessing 19

28 LOFT Takes urve and surfae shape definitions and assembles them into a CADAM model ontaining parametri ubi splines. The model an then be added (as a CADAM drawing) to the CADAM database. CAD CD Provides a olletion of subroutines whih are driven by FORTRAN CALL statements to produe CADAM elements. The generated elements an then be added as a CADAM model to a drawing file. CADET Provides a olletion of CALL statements whih drive passive subroutines whih are designed to reeive disassembled CADAM elements. The CADAM model is retrieved from a drawing file by the module. CADMACGM Provides a olletion of subroutines whih are driven by CALL statements to produe CADAM elements. The generated elements are added to a CADAM model in a foreground mode. The CADET module was used in this researh sine the desire was to aess the part geometri data. The CADET (CADAM Element Transfer) program ombines user written software within an existing program struture. It transfers CADAM model data from a CADAM drawing to user-written routines. An existing CAD Geometri Interfae and Proessing 20

29 USER PROGRAM l INTERFACE r ~- - - ~/13ETWEEN I CADE! i CRDFlM RND I CALL LJSER I ROUTINE ; --. ;_ _J MODEL CREATION D CRDAM DATABASE RRPI -r G -.-ill I ~ERM-NRL Figure 3. CADAM system CAD Geometri Interfae and Proessing 21

30 CADAM model is disassembled into elemental piees (points, lines, text, irles et) and then eah piee is proessed by the CADET main program. Eah element data is then passed to the appropriate user-written routine. These routines proess or store the data as needed and then return to the CADET ontrol program. The proessing ontinues until all CADAM model elements have been transferred. An end of view routine is then alled to allow the user additional proessing for that view before moving to geometri elements in the next view. A skeleton FORTRAN routine alled 'subroutine RESOLV', ontaining entry points and parameter lists for eah type of element, was provided with the CADAM software pakage. Those routines that deal speifially with the geometri elements of interest were expanded to extrat the information needed in an ordered struture and to provide for post view proessing for vision parameter omputation and analysis. Sine the CADET main program proessed eah geometri element separately as it is enountered, it was neessary to onatenate the elements to reonstrut the desired ontour. The elements of internal holes or pokets were distinguished from those of the outer ontours by using the CADAM ATTRIBUTE funtion. While the drawings were being reated, elements lying inside the ontour were assigned an attribute value CAD Geometri Interfae and Proessing 22

31 of 2 and those on the outer ontour were assigned a value of 1. When an element is retrieved the attribute value an be heked. A subroutine to onatenate and proess the ontour was inluded in the expanded RESOLV subroutine as part of the post view proessing routine EXECUTION AND LINKAGE COMMANDS A main program alled CADETMN inluded in the CADAM system. alls the CADET module and was The main program and the modified RESOLV suboutine were ompiled and linked using the ommand 'CADETLNK CADETMN'. To exeute the program module reated by CADETLNK, the CADET exe file must be used. The ommand CADET f n ft group was used where 'fn' and 'ft' were respetively the name and type of the input file for the appliation program. The 'group' was the name of the CADAM group to whih the drawing belongs. The input file ontained the user id in olumns 2-5, the drawing file name in olumns 10-25, and find name starting from olumn 26. All data was left justified in the appropriate field. CAD Geometri Interfae and Proessing 23

32 3.2 GEOMETRIC PARAMETER EXTRACTION ALGORITHM The shape parameter extration algorithm was divided into two main phases, namely, the onatenation phase and the parameter extration phase CONCATENATION PHASE In the foregoing disussion it was pointed out that it was essential to distinguish the elements of the outer ontour from those of the holes. This was ahieved by assigning different attribute values to the elements. The onatenation proedure although the same for both, was arried out separately for the internal and external ontours. If there are multiple holes, they are distinguished from one another by assigning a hole number to eah hole. The onatenation proedure started by omparing the first endpoint oordinates of the first element with those of all other elements. If there was a math, then the mathing oordinate values were not onsidered for further omparison, and the other end point of the mathing element was ompared with all remaining elements. This proedure ontinued until the ontour loses. This ourred when a mathing end point CAD Geometri Interfae and Proessing 24

33 was found for the seond end point of the initial geometri element PARAMETER EXTRACTION PHASE Algorithms and appropriate software were developed for the following parameters: perimeter, area, and minimum and maximum radius. PERIMETER: The perimeter was omputed by summing the distane between suessive end points of the ontour elements. An exeption ourred when the element was an ar. In this ase, the ar length was omputed and added. The oordinates of the ar enter and of the ar end points were known. To determine the ar length, a parametri representation of a point on a irle as given in (20) was used. Points on the irumferene of the irle were represented by values between 0 and 4.0 as a funtion of the slope. one quadrant of the irle was onsidered. At any time only The parameter value of any point on this quadrant was given by T=(SQRT(l.O+TANT*TANT)-1.0)/TANT where TANT = Y/X if the point was in first or third quadrant TANT -Y/X if the point was in seond or fourth quadrant CAD Geometri Interfae and Proessing 25

34 T=l T=2 T=0 J >< T=4 T=3 Figure 4. Parametri representation of a point on a irle CAD Geometri Interfae and Proessing 26

35 X, Y were the differenes between the x, y-oordinate values of the enter of the irle and that of the onerned point respetively. The signs of X, and Y were used to determine the quadrant of the irle assoiated with the point. For example, if X was negative and Y was positive then the onerned point was in the seond quadrant. If a point was found to be in seond, third or fourth quadrant, the parameter value was inremented by 1, 2, or 3 respetively. The diretion onsidered for ar definitions was ounterlokwise proeeding from ar point-1 to ar point-2. The parameter differene assoiated with two ar points was PD = T2-Tl if parameter value of ar point-2 (T2) was greater than parameter value of ar point-1 (Tl). If Tl was greater than T2, then the the parameter differene was PD = 4-(Tl-T2) The length of the ar was given by Ar length= PD/4.0 *radius of the ar *2* TI AREA: Initially the ars were ignored and the ends of ars were onsidered to be onneted by straight edges. This redued the problem to that of determining the area of a polygon. A single vertex was onneted to all the verties of the polygon to form triangles as shown in Figure 5. The CAD Geometri Interfae and Proessing 27

36 area was then found by summing the areas of the triangles. With this approah the enter of area of the polygon an also be determined. The area of a triangle with verties (XL,YL), (XK,YK), (XM,YM), was given by the vetor produt (XL - XK) (XM - XK) (YL - YK) (YM - YK) and the enter of area was given by XCG = (XK+XL+XM)/3 YCG = (YK+YL+YM)/3 The area of the polygon was the sum of the vetor produts of all triangles. If the vetor produt was negative then that triangular area has to be subtrated. Again the onvention adapted here was ounter-lokwise. The areas ontributed by the ars had to be determined separately. A problem with ars was that it was not known whether the ar was onvex or onave with respet to the ontour of the objet. The area ontributed by the ar had to be added or subtrated depending on whether the ar was onvex or onave, and whether the ar enter lay in the positive diretion of inward pointing normal at the midpoint of the side or in its negative diretion. To determine this, the normal CAD Geometri Interfae and Proessing 28

37 Figure 5. Determination of area of a polygon CAD Geometri Interfae and Proessing 29

38 pointing into the area from eah side of the polygon was to determined. To determine the diretion of an inward pointing normal to a given side, point A, at unit distane from the midpoint of the side is onsidered (Figure 6). To determine if this point was inside the area, a line was drawn whih onnets this point to a point B outside the objet. If the number of valid intersetions of the line segment was odd, the point was inside the objet and MA was the normal pointing into the area. The normal for all the other sides was obtained by performing two dimensional transformation on the first normal. There were three possible loations of the ar enter. The ar enter ould be in the same diretion as that of the normal or it may have oinided with the midpoint of the side or it may lie in the diretion opposite to that of the normal. For eah loation of the ar two ases were be onsidered ( Figure 7 ). These ases depend on whether the ar diretion was lokwise or ounter lokwise. The areas were then appropriately added or subtrated from the polygonal area. The ar areas for ases 7a, and 7b were alulated by determining the area of the segment of the irle plus the area of the triangle ABC, while the areas for ases 7, and 7d were alulated by subtrating the area of the triangle ABC from the area of the segment of the irle. CAD Geometri Interfae and Proessing 30

39 Fl M Figure 6. Determination of inward pointing normal CAD Geometri Interfae and Proessing 31

40 Case l. Case b Case..., C,;,1se l Figure 7. Possible ases of ars CAD Geometri Interfae and Proessing 32

41 CENTROID: The entroid was determined as it was needed as a referene point to define the maximum and minimum radius. The entroid of the objet was omputed by summing the produt of the area of a segment of the objet and the di stane of the enter of that segment from the X and Y axes. XCG = I(XCG(i) * AREA(i))/TOTAL AREA YCG I(YCG(i) * AREA(i))/TOTAL AREA Where XCG(i) and YCG(i) were the oordinates of the enter of area of segment i and AREA(i) was the area of segment i. If there were holes with the objet, the area and entroid of eah hole was first determined by onsidering the ontour of the hole as if it were the ontour of an objet. The hole area and entroid were then multiplied together and subtrated in the alulations of the total objet entroid. MAXIMUM RADIUS: It is known from the theory of linear programming that the maximum point of a onvex polygon always lies on one of the verties. This priniple was used in determining the maximum radius of the ontour. When there was an ar element, the above priniple did not hold. If the ar was onvex, then a point on the ar may be farther away from the entroid than the farthest displaed vertex from the entroid. Hene, if there was an ar element, the distane of the farthest point on the ar from the entroid was al- CAD Geometri Interfae and Proessing 33

42 ulated, and this was ompared against the displaements of all verties and other ars from the entroid. The maximum of all the displaements is defined as the maximum radius as shown in Figure 8. MINIMUM RADIUS: The minimum radius was defined as the distane of the nearest point on the outer ontour from the entroid. ( Figure 9). The minimum distane may have been the distane of a vertex from the entroid or the shortest distane from an edge of the outer ontour. The above mentioned ases had to be onsidered and the minimum of all the distanes was defined as the minimum radius. 3.3 SHAPE PARAMETER EXTRACTION PROGRAM The parameter extration routines were added to the subroutine RESOLV available in the CADAM system. There were seven main modules in the program whih are desribed below. A listing of the program is inluded in Appendix-A. MODULE-1: The module ontained the proedure to retrieve the CADAM drawing elements from a drawing. Additionally, entry points CDTST (CADET start proessing), CDTBVU (CADET begin view information), and CDTEVU (CADET end view information) were inluded. Entry points whih retrieve elements and in- CAD Geometri Interfae and Proessing 34

43 E R Figure 8. Determination of maximum radius CAD Geometri Interfae and Proessing 35

44 D E 5 l 2 R 1 B Figure 9. Determination of minimum radius CAD Geometri Interfae and Proessing 36

45 formation whih were not required were inluded after the parameter extration module. For eah geometri element of interest, X and Y oordinate values were passed to the proedure developed in the researh. These routines saved data for post view proessing. The CDTLN routine reeived data assoiated with eah line element in the C and D arrays given in the argument list: CDTLN(C,D,LT). The CDTARC routine proessed ar data as the oordinate enter (C) and the end points D and F. This was passed in the argument list CDTARC(C,D,E,LT). CDTCIR routine proessed the irular element Finally, the data as the oordinates of the enter C, and the radius, RAD. This data was passed via CDTCIR(C,RAD,LT). The ATTRIB subroutine was alled by the CDTLN, CDTARC, and CDTCIR proedures to determine the attribute value of an element. If the attribute value of an element was l, it formed part of the outer ontour and if it was 2, it formed part of a hole. MODULE-2: This module ontained the proedure for the onatenation of the elements of the objet. The proedure first linked all of the elements of the outer ontour of the objet as desribed earlier. Next the elements of interior holes were hained to form ontours appropriately. This CAD Geometri Interfae and Proessing 37

46 module ontinued until all elements were aounted for the objet ontour. MODULE-3: This module ontained the the proedure that omputed the perimeter of the objet distane by alulating the length of eah outer ontour element. The proedure ompared the value of the variable POLARC(i,l) and POLARC(i,2) for eah vertex. If POLARC(i,l) and POLARC(i+l,2) were equal to 1, the element was an ar. Subroutine PARMET was alled to find the parameter values of the two ar end points. Then the ar length was omputed as desribed in the algorithm. If an element is a line, the length was omputed as the distane between the endpoints. All the element lengths were added to alulate the objet perimeter. MODULE-4: The proedure to ompute the area of the outer ontour was ontained in this module. The subroutines NORMl and NORM2 were alled to determine the normals on eah side, that point towards the ontour. Then the subroutine PLAREA was alled to alulate the area of the ontour onsidering only straight edges, that is irular edges were ignored and onsidered as straight. Next the subroutine CRAREA was alled to determine the area ontributed by the irular edges. The algebrai sum of the above two areas gives the effetive area of the outer ontour. This module also on- CAD Geometri Interfae and Proessing 38

47 tained the steps to determine the oordinates of the enter of area of the outer ontour. MODULE-5: This module ontained the proedure to proess the holes. The area, and entroid parameters for eah hole were alulated in this module. MODULE-6: This module ontained the proedure to determine the enter of area of the ontour. The areas of holes were taken into onsideration in the omputation of the effetive entroid. MODULE-7: This module determined the maximum and minimum radii of the outer ontour. The effetive entroid of the outer ontour, that is the entroid alulated after taking into onsideration the area of the holes was onsidered. The subroutines MAXIR and MINIR were alled to determine the maximum and minimum radii. 3.4 DESCRIPTION OF SUBROUTINES SUBROUTINE NORMl: This subroutine determined the normal that points towards the ontour for an edge of the ontour. The endpoints of the edge were passed as arguments to the subroutine. Initially the edge with end points CAD Geometri Interfae and Proessing 39

48 (POLYLN{l,l),POLYLN{l,2)) and (POLYLN{2,l),POLYLN(2,2)) was onsidered. A line segment formed by a point (PINMID(l,l},PINMID(l,2)) loated at 0.25" from the midpoint of the edge and the point (PINC3, PINC4) whih lies outside the ontour was onsidered. The subroutine INTLN was alled to find the intersetion point of this line with all the edges. An intersetion point with an edge was valid if it fell within the edge. If the number of intersetion points was odd then the point was within the ontour and the line formed by this point and the midpoint of the edge is the normal pointing towards the objet. SUBROUTINE NORM2: This subroutine determined the inward pointing normals for all the edges other than first one (disussed in Subroutine NORMl). To determine the inward pointing normal segments for all the other edges, a rotational transformation of the point ((PINMID(l,l},PINMID(l,2)) was performed through an angle 'THETA' (where 'THETA' was the angle between suessive edges. The subroutine ACANG2 was alled to determine the angle 'THETA'. SUBROUTINE PLAREA: This subroutine alulated the area of the ontour onsidering eah element as a straight line. The oordinates of the verties of the polygon were passed CAD Geometri Interfae and Proessing 40

49 through the array POLYLN. The area and entroid of the polygon were omputed as desribed in the algorithm. SUBROUTINE CRAREA: This subroutine alulated the area ontributed by the irular portions. The oordinate values of the verties, the midpoints of the edges, and a point on the normal to eah edge whih was at a unit distane from the midpoint of the edge were passed to the subroutine. Additionally, the POLARC whih indiated if a vertex formed an end point of an ar, and FLPLN, whih indiated if the edge was flipped or not, were passed. If the value POLARC(i,l) and POLARC(i+l,2) were equal to 1, then the verties i, and i+l formed the endpoints of an ar element. The ar enter loation on the normal was determined by treating the normal as a parameterized line represented by XC=XO+T(XN-XO) where XO was the beginning of the line segment XN was the end point of the line segment x was any point on the line segment T was the parameter value for the point x. CAD Geometri Interfae and Proessing 41

50 The positive diretion was from XO to XN. The loation of the ar enter on the normal was determined by omputing the T value of the ar enter. The subroutine FINDT was alled to ompute T. If T was less than 0.02, then the ar was a semi-irle. The parameter values of the ar end points were omputed using the subroutine PARMET, and the ar element was resolved to one of the ases desribed in the algorithm, and its area was omputed. The entroid of the ar segment and the entroid of the omposite area were determined as desribed in the algorithm. SUBROUTINE MAXIR: This subroutine determined the maximum radius of the ontour. The distane between eah vertex and the entroid was omputed. If there was an ar element, the distane of the farthest point on the ar from the entroid was also onsidered. The maximum of the omputed displaements was found by pairwise omparison of the displaements. The oordinates of the verties, entroid were passed. SUBROUTINE MINIR: This subroutine determined the minimum radius point on the outer ontour. The distane of eah vertex from the entroid was omputed, and the shortest distane of the entroid from eah side was omputed. If a side was omposed of an ar element, then the distane of the nearest point on the ar from the entroid was also omputed. The CAD Geometri Interfae and Proessing 42

51 minimum value of all the omputed displaements gave the minimum radius of the outer ontour. SUBROUTINE PARMET: This subroutine alulated the value of the parameter for the end point of an ar element. The proedure was a translation of the method to alulate the parameter value that was disussed in the algorithmi proedure that desribes the alulation of perimeter. The values of the oordinates of the end point of the ar element, and those of the enter of the ar element were passed to the subroutine. SUBROUTINE INTLIN: This subroutine determined the intersetion point between two line segments. The values of the oordinates of the end points of both the lines were passed to the subroutine. The proedure heked if the intersetion point were within the line segments or not and passed this information out. SUBROUTINE ACANG2: This subroutine determined the angle subtended by an ar at its enter. The values of the oordinates of the ar enter, and those of the end points of the ar were passed to the subroutine and the value of the ar angle was passed out. CAD Geometri Interfae and Proessing 43

52 SUBROUTINE INTCIR: This subroutine determined the intersetion points of a line segment and an ar element. The values of the oordinates of the end points of the line segment, ar end points and the ar enter were passed to the subroutine. The oordinates of the points of intersetion were passed out. SUBROUTINE ARCENG: This subroutine determined the oordinates of the enter of area of a hord. The values of the oordinates of the end points of the hord, enter of the ar that forms the hord, and the radius of the ar were passed to the subroutine. The oordinates of the entroid of the hord were passed out. SUBROUTINE SLPLN: This subroutine alulated the slope of a line segment. The oordinates of the end points of the line segment were passed to the subroutine, and the slope of the line segment was passed out. SUBROUTINE FINDT: This subroutine took the values of the oordinates of end points of a line and onverted it to a parametri representation of the form y Where X and Y were the oordinates of any point on the CAD Geometri Interfae and Proessing 44

53 infinite line x 0, Y 0 were the oordinates of the first point or the origin of the line segment x 1, Y 1 were the oordinates of the seond or end of the line segment t was the parameter value of a point lying on the infinite line with oordinates X, Y. If the t was between 0 and l, the point with oordinates X, Y was within the line segment formed by (X 0,Y 0 ) and (X 1,Y 1 ), otherwise the point with oordinates X, Y was outside the line segment formed by (X 0,Y 0 ) and (X 1,Y 1 ) In this subroutine the oordinates of the endpoints of the line segment and the point for whih the parameter value was sought were passed to the subroutine. The value of the parameter was passed out. CAD Geometri Interfae and Proessing 45

54 4.0 VISION SYSTEM In this hapter, a brief desription of the vision system used in this researh is disussed. The ASSEMBLY language subroutines that were written to aess and modify the the piture elements are desribed. A detailed disussion of the proessing of the binary image of an objet plaed in front of the vision system is also presented. 4.1 IMAGING TECHNOLOGY SYSTEM DESCRIPTION The mahine vision system used in this researh was omprised of the omponents as shown in figure 13. The PCVISION system was hooked to the IBM-PC/AT to proess the image of an objet grabbed by the frame grabber of the PCVISION system. The major omponents of the PCVISION system were the frame grabber, the video amera, and the external monitor. were three major omponents in the frame grabber. There They were: the digitizer, the display logi, and the frame memory. The image of the objet was aptured in the form of an analog signal by the video amera. The analog signal was RS-1 70 ompatible and needed to be transformed to a digital form to be able to be stored in the frame memory of frame grabber. Vision System 46

55 The inoming video signal was digitized to 8 bits of resolution at a rate of 30 frames per seond. The frame memory had 512 rows and 512 olumns, but only 480 rows and 512 olumns were atually used. So, the total number of pixels or piture elements in the stored image was 480 x 512. Sine eah pixel was represented by 8 data bi ts, 256 possible shades of gray were possible. The stored image was be displayed on the video monitor by the display logi, whih onverted the stored digital image bak to analog form. The PCVISION frame grabber was diretly ompatible with the bus struture found in the IBM personal omputer. This interfae was the ommuniation path between the IBM PC and the PCVISION frame grabber. The frame memory interfae enabled the IBM-PC to aess the frame memory. The PCVISION frame grabber mapped 64 kbytes of its frame memory into the memory spae of the IBM PC. Sine there was a total of 256 kbytes of frame memory on the PCVISION frame grabber, only one fourth of the frame memory was aessible from the IBM PC Bus at any time. But FORTRAN allable routines written in ASSEMBLY language provided aess to the frame memory and mask this hardware memory segmentation. The frame memory subsystem was apable of simultaneously aquiring and displaying an image. This was aomplished with a read/modify/write yle on eah frame memory aess. The Vision System 47

56 ~. IBM PC/RT ' ' I PCvISION l="rrme GRABBER I i i i... CRMERR _,..- DIGITIZER - ' / i... ~! r="rrme I I i I i I MEMORY I I [...--'" D<TERNRL i I I~ DISPLAY LOGIC ~ MONITOR i I i I i I L J I I I Figure 10. Mahine Vision System Vision System 48

57 i,j Fl/D CON\/E.RTER 4 IMAGE. PLANES r--,""'' ~ BIT PLANES Figure 11. Frame memory organization Vision System 49

58 pixel whih was read was transmitted to the display logi for digital-to-analog onversion, and the new pixel from the digitizing logi was written into the memory. Therefore, a one frame lag existed (1/30th seond) when the PCVISION frame grabber was simultaneously aquiring and displaying an image. The PCVISION frame grabber display logi ontained 4 output lookup tables (LUT). Eah look-up table was 256 bytes deep and is programmable through the IBM PC bus interfae. Look-up tables were used to transform the pixel intensities prior to display. This permitted threshold viewing of grey sale images that were onverted to binary. Different LUT's ould be used to visualize effets of possible threshold values. 4.2 PROGRAMMING THE PCVISION FRAME GRABBER FORTRAN allable ASSEMBLY language routines were written to assist the proessing of the the image stored in the frame memory. Eah subroutine is disussed briefly below. SUBROUTINE VINIT: This subroutine initialized the PCVISION frame grabber, look up tables et. It was alled prior to any other vision system subroutine all. Vision System 50