A Scanning Method for Industrial Data Matrix Codes marked on Spherical Surfaces ION-COSMIN DITA, MARIUS OTESTEANU Politehnica University of Timisoara Faculty of Electronics and Telecommunications Bd. Vasile Parvan 2, 300223 - Timisoara ROMANIA cosmin.dita@etc.upt.ro, marius.otesteanu@etc.upt.ro FRANZ QUINT, NAGARAJU ANNAMDEVULA University of Applied Sciences, Karlsruhe Faculty of Electrical Engineering and Information Technology Moltkestrasse 30, 733 - Karlsruhe GERMANY franz.quint@hs-karlsruhe.de, raajz43@gmail.com Abstract: In this paper we provide a method to localize the modules of the industrial Data Matrix Code (DMC) marked on curved surfaces. An imaginary grid of points is constructed which has the same orientation as the surface. The imaginary grid is projected on to the image and according to this grid the modules of the code are scanned. If the surface considered is not a perfect sphere then the results are not accurate. To overcome this using an error correction based on the assumed centers and the computed centers a better approximation is obtained. Using this method we can localize the modules irrespective of the camera view or orientation of the DMC. Key Words: DMC Scanning Method, Industrial Data Matrix Codes, Data Matrix Code Reader Introduction Data Matrix Code (DMC) is a matrix (two dimensional) bar code which may have a square or rectangular shape made up of individual dots or squares called []. DMCs are composed of two separate parts, the finder pattern which is used to locate the symbol, and the data region. The finder pattern defines the shape (square or rectangle) and the number of rows and columns of the DMC. The finder pattern itself has two separate parts: the L finder pattern and the Clock track or timing pattern. In Fig., we can see that there are two edges with continuous modules and the other two with modules marked alternately. The L shaped edges that have the continuous modules is the L finder pattern. It is primarily used to determine the size, orientation and distortion of the code. Initially, DMC s have been developed to be printed with dark color on bright and plane surfaces (usually printed on paper). However, more and more DMC s are used in industrial environment to identify parts in a manufacturing process. Then, they are not only printed, but also milled, etched or laser marked directly on different kind of surfaces like metal, plastic etc. The surface on which they are marked are no longer plane, so their appearance in the image will no longer be that of a rectangular grid. This, combined with small contrast due to the various materials they are marked on, and with the typical disturbances in industrial environment like scratches, rust, oil drops etc. makes localization and decoding a challenging task. In this paper we present a method for localizing the modules of a DMC on curved surfaces. 2 Overview of the localization method Figure : Finder pattern and encoded data The other two edges of the finder pattern with alternating light and dark elements are known as timing pattern. This defines the basic structure of the code and can also help to determine its size and the distortion []. The number of rows and columns varies between 8x8 up to 44x44. The DMC is capable of encoding up to 2KB data. Fig. 2 gives an overview of the localization process of the Data Matrix Code dotted on curved surfaces. After image acquisition, the process starts with the coarse localization of the Data Matrix Code. This step is used to determine a Region of Interest (ROI) inside which the Data Matrix Code lies [2]. For this it considers the real world size of Data Matrix Code, the surface radius and as well as the focal length of the camera s lens to be approximately known. The ISBN: 978--804-08-7 38
image is thresholded using an adaptive threshold level [3] []. To reliable detect the Data Matrix Code it is convenient to have a solid shape rather than separated modules. For that, using morphological operations, the image is closed in order to fill the empty spaces between modules [5]. The maximum and the minimum coordinates gives us the four corners of each object. After the image is thresholded, in the image are objects. Heaving now established a coarse localization of the DMC, the process continues with scanning the DMC to find the positions of the individual modules. First we search for the positions of the modules along the edges of the DMC. Out of the modules on the edges, we choose control points. These will be used to calculate the transformation matrix between the module positions in a 3D-coordinate system locally attached to the code ( imaginary grid ) and the module positions in the image. Due to the fact, that the curved surfaces will not necessarily be spherical or cylindrical as the model assumes, we will need to correct, i.e. relocate the module positions during the scanning process. We will subsequently explain this in more detail. Video Camera Image acquisition Data Matrix Localization Data Matrix Corners Detection Labeling Coarse DMC Localization Scanning process for DMC marked s Input parameters ~ The radius of curvature ~ The length of the DMC Scanning along the edges Identifying modules and computing Centers Control Points and Surface Classification Matrix Construction Relocating the module position 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 DMC 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Matrix 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Figure 2: The flow process of the Data Matrix Codes for spherical surfaces Figure 3: Scanning for modules along the edges 3 Scanning along the edges All edges are scanned to find the locations of the modules along the timing and the finder pattern of the DMC. In Fig. 3 a labeled image of DMC is shown with its corners named. Corners A and B are used to find the orientation of the first edge with the x-axis (eq. ). ( ) yb y A θ = atan, () x B x A The point that stands at distance d at an angle θ with point A is computed (eq.2,3). d being a first estimate of the distances between modules. x = x + d cos(θ ), (2) y = y + d sin(θ ), (3) In the case of the spherical surface the distance between the modules will not be constant in the image. But the distance between adjacent modules will be almost the same. So for that reason every time the distance value has to be updated with the value computed from the latest points that is known. Every time a new point is computed, it is taken as a current point. First, the distance and angle values are computed using the current point and the point B, being updated. Starting with point A as a current point we repeatedly estimate the module position of the next point cf. eq. 2 and 3. A frame is constructed around that approximate position, searching for the desired object inside of it, computing the centers of this object. The currentpoint is updated with this value and the process continues. For the angle computation, A and B are used at beginning. After then the angle is computed using A and the current point, the values being updated after each iteration. For the distance computation the last two points from the previous edge are used at beginning. Then the corner A and the current point are used to update the value after every iteration. ISBN: 978--804-08-7 39
4 Selection of control points Selection of the control points is important because the transformation process dependents on the matrices that are computed from these values. The control points are picked from the modules along the edges of the DMC. We have chosen to select fifteen control points from different parts of the code. Five fixed points are used as control points, which represent the corners of the DMC. The modules that are adjacent to the corner points are also considered as the control points. The rest of the points are chosen to be distributed evenly across the finder pattern. Having this control points and other information about the code like real world size and radius curvature, we can construct the estimated grid for the DMC modules. This estimated grid is a connection between the real world modules position and the position of them in the image. 5 Cylindrical and Spherical Surfaces We use cylindrical and spherical models for our surfaces. To differentiate between the cylindrical and spherical surface we use the orientation of the edges of the DMC. If the surface is a cylindrical then irrespective of the camera view every time two of the DMC edges will be linear and if the camera view is orthogonal to the axis of cylinder then all the four sides will be linear. When it comes to the spherical surfaces all the edges are curved. Furthermore opposite sides are curved in opposite directions. Consider a straight line L with slope m and a y-intercept c, Fig. 4. The points P (x, y ) and P 2 (x 2, y 2 ) are two points in the xy-plane. If the point P lies on the line, then L(P ) will be zero i.e., y = m x + c, where as if P and P 2 lie on either side of the straight line then the value will be of opposite sign. Using this it can be determined if the sides are curved or linear. Y-axis 50 45 40 35 30 25 20 5 0 5 0-8 -ve egion 2 2 0 5 8 3 2 3 4 5 7 8 9 0 - +ve region X- axis y=4x+5 Random points Figure 4: Demonstrating the classification of surfaces Table : The possible values for the edges of the DMC on a cylinder or on a sphere Surface lin lin2 lin3 lin4 lin Planar +4 Cylindrical Spherical +2 +2 + -2-2 -2-2 -2-2 +2 +2 + Every time opposite sides are oriented in opposite directions hence giving the sum as zero We observe that this way there are two regions defined in the image. The region above the line is the negative region and the region below the line is the positive region. The line L shall be the line joining the two corners of the DMC code and the other points represent the centers of the modules along the edge. By substituting these points in the line equation, we can find whether they lie on the line or in the negative region or in the positive region. If a point lies approximately on the line, it is given a value of unity. If the number of points lie on either side of the edge then a value of 2 or +2 is given depending on the sign of the residuals. Assume that the residuals of the second edge are positive and so it has a value +2 and it is the same with the fourth edge. So it also have the same value +2. Whereas the first and third edges are linear so they take a value of. The sum of the values given to all the edges are used to classify the surfaces. For a spherical surface it may have all the edges curved and the opposite edges are curved in a direction opposite to each other. That indicates the overall sum will be zero for a spherical surface. Matrix Construction Due to the imprinting of the modules on non-planar surfaces, the modules in the image will not appear on a rectilinear and equidistant grid. The relation between the local real world coordinates and the image coordinates is given by the projection equation: ku kv k = H H 2 H 3 H 4 H 5 H H 7 H 8 H 9 H 0 H H 2 x y z (4) ISBN: 978--804-08-7 40
where, (x, y, z) are the coordinates of the objects in the real world and the (u, v) represents the projected points in the image. Since the real world dimension of the DMC is approximatively known and also the radius of the curvature is known, the real world coordinates of the corner points of the DMC can be calculated. Having the real world coordinates of the control points and also their coordinates in the image, one can solve eq. 5 for the unknown projection parameters H. x y z 0 0 0 0 u x 0 0 0 0 x y z v x x 2 y 2 z 2 0 0 0 0 u 2 x 2 0 0 0 0 x 2 y 2 z 2 v 2 x........ x n y n z n 0 0 0 0 u n x n 0 0 0 0 x n y n z n v n x n u y u z u H v y v z v H 2 u 2 y 2 u 2 z 2 u 2 H 3 v 2 y v 2 z v 2 H 4 = [0]... u n y n u n z n u n v n y n v n z n v n. H H 2 (5) 8 Conclusion To localize the modules in DMC, scanning inside the image of DMC is necessary. The programmer does not know where to start the scanning process. In that case this implementation will provide the scanner with some approximate locations where the modules are located. It can scan around that location to find the exact position of the module. This algorithm works in many different situations like the DMC is rotated in its plane, the view of the camera is not orthogonal with the surface of the object and the DMC is not a square. One only needs an approximately value for the size of the DMC whereas the current value is estimated during the localization process. (a) Data Matrix Code (b) Predicted grid (c) Detected modules Figure 5: Ex. Industrial Data Matrix Code marked where, (x i, y i, z i ) are the real world coordinates of the control points and the (u i, v i ) represents the coordinates of the control points in the image. This is solved using the singular value decomposition method to obtain the values of H-matrix [4]. Having these parameters one can predict the location of any real world coordinates in the image. (a) Data Matrix Code (b) Predicted grid (c) Detected modules Figure : Ex.2 Industrial Data Matrix Code marked 7 Relocating the module position Projecting the grid on to the image, all the modules can be located approximately. There is some error because of the inaccurate points of correspondence that are used. Due to this reason, using a frame, one searches in rows and columns for each dot creating a matrix of coordinates of the dotted and un-dotted modules. The size of the frame is equal with the distance between the modules. The center of the frame follows the predicted grid(transformation matrix) of Data Matrix Code. If in the frame exists a module then the matrix element is an one. If the frame is empty then the matrix element is a zero [2]. (a) Data Matrix Code DMC detected: Angle=29, Distance= (b) Detected modules Figure 7: Ex.3 Industrial Data Matrix Code marked ISBN: 978--804-08-7 4
In Fig. 5-7, are presented two examples of situation with Data Matrix Code marked s. The proposed method takes the extreme points of the Data Matrix pattern computing the predicted grid where the Data Matrix modules should be distributed. The information is extracted based on this predicted grid, only the modules which intersect with the grid being analyzed. This method works well for all images where the extremes modules of the code can be seen, doesn t depend by the orientation angle or the size of the pattern or the type of the materiel where the pattern is marked. Using this, not the image is geometrical transformed, only a grid is created based on the image. The goal is that the image is not processed and no information from the image is loosed, also the processing system doesn t need to much power for Data Matrix scanning. Acknowledgements: This work was partially supported by the strategic grant POSDRU /.5/S/3, (2008) of the Ministry of Labour, Family and Social Protection, Romania, co-financed by the European Social Fund Investing in People. References: [] INTERNATIONAL STANDARD: Information technology International symbology specification Data matrix (2000-05-0) [2] Ion-Cosmin Dita, Marius Otesteanu, Frant Quint: Data Matrix Code - A Reliable Optical Identification of Microelectronic Components. 20 IEEE 7th International Symposium for Design and Technology in Electronic Packaging (SIITME) (20) [3] N. Otsu: A Threshold Selection Method from Gray-Level Histograms. IEEE Transactions on Systems, Man and Cybernetics, DOI - 0.09/TSMC.979.43007 (Systems, Man and Cybernetics, IEEE Transactions on) 9(), 2 (979) [4] Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes: The Art of Scientific Computing, (3rd ed.) edn. New York: Cambridge University Press (2007) [5] V Gui, D Lacrama, D Pescaru: Prelucrarea imaginilor, vol. ISBN-973-9389-2-0. Editura Politehnica Timisoara (999) [] Ye Zhang, Hongsong Qu, Yanjie Wang: Adaptive Image Segmentation Based on Fast Thresholding and Image Merging. Artificial Reality and Telexistence Workshops, 200. ICAT 0. th International Conference on pp. 308 3 (200) ISBN: 978--804-08-7 42