Modelling imprints of pharmaceutical tablets for imprint quality visual inspection

Transcription

1 Modelling imprints of pharmaceutical tablets for imprint quality visual inspection Miha Možina 1, Dejan Tomaževič 1,2, Franjo Pernuš 1,2 and Boštjan Likar 1,2 1 Sensum, Computer Vision Systems Tehnološki park 21, 1000 Ljubljana, Slovenia 2 University of Ljubljana, Faculty of Electrical Engineering, Laboratory of Imaging Technologies, Tržaška 25, 1000 Ljubljana, Slovenia Abstract. Identification of pharmaceutical tablets plays a key role in preventing mix-ups among various types of tablets. Since identification of tablets is most frequently done by imprints, good imprint quality, a property that makes the imprint readable, is of utmost importance. In this paper, we propose a novel method for automated visual inspection of tablets for defect detection and imprint quality inspection. Performance of the method was evaluated on a real tablet image database of imprinted tablets. A "gold standard" was established by manually classifying the tablets into good and defective class. The ROC (receiver operating characteristics) analysis indicated that the proposed method yields better sensitivity and specificity than the previous defect detection method. Keywords: Modelling, automatic visual inspection, imprint quality 1. Introduction Minimal standards for manufacturing and distributing pharmaceutical products are enforced by national regulators in order to protect the customer. One of such security issues is reliable drug identification [1]. In order to avoid mix-ups among various types of tablets, a regulation code 21CFR206 [2], issued by the Food and Drug Administration, enforces the pharmaceutical companies to produce tablets with unambiguous identification of tablet's active substance and dosage by tablet's size, shape, texture, imprint and/or other visual and physical characteristics. Besides the identification, visual appearance plays an important role in marketing, since imperfect appearance of a single tablet in a package can raise serious doubts about integrity and quality of the product. Therefore, the pharmaceutical companies try to assure highquality of every single product. Since manual visual inspection of large tablet batches is subjective, unreliable, slow, tedious and costly, automated visual tablet inspection systems are more and more commonly used. A vast amount of research was done on automated visual inspection of various objects [3][4], however, automated visual tablet inspection is especially challenging because tablets come in different sizes, shapes, colours and imprints and may have different visual defects. Companies, such as Eisai, Ikegami, Proditec, Viswill and Sensum, produce and market various automated visual tablet inspection systems, that have to be able to inspect a large variety of tablets and, therefore, in general detect well large defects, such as cracks, broken products, contrast stains, dots, and size variations, on smooth tablets without imprints or texture, but have suboptimal performance for some specific tablets, such as tablets with texture and/or imprints. In case of tablets with imprints, tablet inspection is more challenging. Firstly, defects in the vicinity of imprints can have similar contrast to imprints and therefore, before the classification, the system should distinguish defects from imprints. Secondly, detection of low contrast defects on imprints presents an additional challenge. Last, but not least, imprint quality has to be inspected. Good imprint quality is an imprint property, which makes the imprint readable and, therefore, it is crucial for the identification. The imprint quality is reduced by imprint degradations, such as blurred print, shallow debossing, colour and size variations of imprint, etc.

2 Fig. 1 - Images of the same tablet at different rotation position in the scene (top row) and visualization of appearance variations with debossing close-up (bottom row) In the most recent work [5], two methods were investigated for defect detection of imprinted tablets. The first method was a geometrical method which incorporated geometrical a priori knowledge of the imprint. The imprint was represented by a imprint skeleton, which was rigidly registered to the image of the analyzed tablet to divide the tablet to imprint and non-imprint tablet region. Each region was then separately analyzed, which solved the problem of distinguishing the imprints from defects around the imprint. However, the method was less sensitive to low contrast defects on imprints and to imprint quality inspection. The second method was a statistical method, which used the a priori knowledge in the form of a statistical appearance model. The model was derived from a set of rigidly registered training images. With this method, not only the imprint and non-imprint tablet regions but also each image element (pixel) were inherently analyzed. The method showed improvement in detection of defects on imprints, it was observed, however, that the performance of the method is still suboptimal for imprint quality inspection, especially for debossed imprints. The key reason for this are the imprint appearance variances, which occur because imprint appearance is a function of illumination, 3-D geometry and pose in the scene (see Fig. 1). Therefore, tablets with good imprint quality can have greater imprint appearance variations than the imprint appearance difference due to low imprint quality. The aim of this paper is to improve the sensitivity of the automated visual tablet inspection system considered by Bukovec et al [5] by improving the sensitivity and specificity for imprint quality inspection of debossed tablets. The main idea of the paper is to generate an appearance model invariant to defects to avoid suboptimal appearance models in the case of tablets with degraded imprint quality. 2. Method In this section, two methods are presented. In the first subsection, we present the general statistical appearance method [5]. In the second subsection, we propose the statistical appearance method incorporating rotation information. 2.1 General statistical appearance method The general statistical appearance method is based on comparison of an inspected tablet to an appearance model of a non-defective tablet. Before comparison, segmentation, i.e. partitioning the image into tablet and background regions, has to be performed. Then, registration is needed to set tablet imprints into spatial correspondence with the model.

3 The tablet appearance x can be modelled by a linear model: xxap (1) consisting of an average appearance x, a matrix of variations A and corresponding appearance parameters p. The appearance model is obtained statistically in training phase from a set of n segmented and registered training images, x 1,..., x n, i.e. images without defects by principal component analysis method (PCA) [6]. The average appearance x is the average value of training images x 1,..., x n and the matrix of variations A is a set of linear independent eigenvectors called eigenspace A. Let x t be an inspected tablet after segmentation and registration. The tablet appearance x of the x t is defined by appearance parameters p(x t ), which best describe x t, i.e. the root mean square difference between the x and the x t is minimum: t t x x x Ap x (2) The appearance parameters p(x t ) of inspected tablet x t are calculated by projecting the x t onto the eigenspace A: T pxt Ax t (3) The defect detection feature was chosen as the maximum difference between the tablet appearance model x x t and the inspected tablet x t and was calculated in equations (4) and (5). Firstly, image of error e t (x t ) is defined as absolute difference of x t and xx : t e x x x x (4) t t t t The defect classification feature S(x t ) is then calculated by finding maximum value e t (x t ) over the entire tablet surface x t : max t t t S x we x (5) xt where e t (x t ) is regularized by filter w for controlling the scale of defect detection. Classification of inspected tablet x t to defective or non-defective is then done by thresholding the feature S(x t ). Fig. 2 - Framework of the general statistical appearance method for visual quality inspection of tablet appearance The complete general statistical appearance method is outlined in Fig. 2. The left side of Fig. 2 represents the training (off-line) phase. The input is a set of training images. The output is the eigenspace obtained by the PCA. The right side of Fig. 2 illustrates the inspection (on-line) phase. As input, it receives the output of the training phase (eigenspace) and an inspected image. The output is the classification of the inspected tablet to defective or non-defective. 2.2 Statistical appearance method incorporating pose information As the general statistical appearance method, the proposed method is also based on principal component analysis. The difference is how the appearance is modelled. While the tablet appearance x of the x t is still calculated by equation (2), the appearance parameters p are not obtained from the x t, i.e. by equation (3), but from the rotation information and are denoted by p(φ t ), where φ t is rotation parameter of the x t obtained at the registration. The main idea of the proposed method is to describe the appearance parameters p(φ) with parametric functions: 0 K 1k 2k k1 cos( k) sin( k) p b b b (6) where k=1,...,k are angular frequencies and [b 0 b 1k b 2k ] are the corresponding parameters, which are obtained by the least squares regression between

4 p(φ i ) and p(x i ); i=1,...,n, for a set of training images. The appearance parameters p(φ t ) can be obtained for every single angle φ t and are used to reconstruct the x : appearance t t t x x Ap (7) The defect classification feature S(x t ) is then calculated as described in Section 2.1 by equation (4) and (5), where x t is used instead of the x x t. The proposed method is outlined in Fig. 3. Similar to Fig. 2, the left side of Fig. 3 represents the training (off-line) phase. The input is a set of training images. The output is the eigenspace and the parametric functions, describing the correlation of tablet appearance and rotation parameter φ, derived from the training images. The right side of Fig. 3 illustrates the inspection (on-line) phase, where input is the output of the training phase and an inspected image. The output is a classified inspected tablet. Training (off-line) Set of training images after preprocessing Inspection (on-line) Input image after preprocessing on a set of real tablet images of tablets with debossing. Details about "gold standard" database, implementation details and performance evaluation are given in the following subsections. 3.1 Image database with "gold standard" The image database was acquired with a Sensum SPINE (Fig. 4) automatic tablet inspection machine (Sensum, Computer Vision Systems, The machine has a reliable tablet manipulation mechanism, providing accurate positioning by filling the tablets into the special inspection pockets where they are held in the reproducible positions by a vacuum system. The optical part of the machine consists of line-scan camera and white LED illumination directed from a low angle to the tablet surface (Fig. 4). The experimental image database consisted of 40 training images and 130 inspected images, i.e. 71 non-defective and 59 defective images. The "gold standard" was set by manually classifying the inspected tablets by labelling tablets as nondefective and defective (see examples in Fig. 5). Rotation parameters of corresponding set of training images Rotation parameter of input image Statistical appearance model Modelling rotation Parametric functions defining appearance parameters Eigenspace Appearance model extraction Feature extraction and classification Fig. 4 - Sensum SPINE visual quality inspection machine (left), which automatically inspects and sorts pharmaceutical tablets at the speeds of up to 360,000 units per hour. The machine uses illumination from the sides (illustrated on the right) for enhancing the 3-D geometry of products. ( Fig. 3 - Framework of the proposed method - statistical appearance method incorporating rotation information 3. Experiment and results The proposed method was evaluated by comparison to general statistical appearance method 3.2 Implementation details The segmentation of images, i.e. partitioning the images into tablet region and the background, was done using the border tracking algorithm [7]. The tablet imprints were set into spatial correspondence by rigid registration, where the translation

5 Fig. 5 - A raw image of a non-defective tablet (left), a preprocessed image with enhanced contrast of a non-defective tablet (middle) and a preprocessed image with enhanced contrast of a tablet with shallow debossing (right). parameters were obtained by matching the centre of masses of the segmented tablet surfaces, while rotation parameter was obtained with circular profile matching algorithm [8]. The eigenspace A consisted of p, p < n, eigenvectors, where first p eigenvectors with highest corresponding eigenvalues were used to represent the tablet appearance x to a sufficient degree of accuracy. The parameter p was set to 2 (p = 2), which offered sufficient degree of accuracy of the appearance x (see Fig. 6) and kept the method applicable in real-time. To model the symmetrical illumination used in the image formation process, the parametric function, that is used to calculate appearance parameters p(φ), was harmonic function with angular frequency of 2: b12 b22 p b 0 cos(2 ) sin(2 ) (8) The filter w was an uniform filter of size 7x7 pixels. 3.3 Performance evaluation The specificity and sensitivity of each method was obtained by the receiver operating characteristics (ROC) analysis [9]. The ROC curve relates the tradeoffs between the true positive (TPR) and the corresponding false positive (FPR) defect detection rate of the classificator S(x t ). The TPR = TP/P is the ratio between the number of correctly detected tablets with defects (TP) and all defective (P) tablets, while the FPR= FP/N represents the ratio between the number of incorrectly detected nondefective tablets (FP) and all non-defective (N) tablets. TPR is a measure of sensitivity, while 1 FPR is a measure of defect detection specificity. A ROC analysis is insensitive to the ratio between the number of defective (P) and non-defective (N) samples used for evaluation [9] and is insensitive to threshold value used at classification, which makes different methods easily comparable. The obtained results for the given dataset for both methods are given with ROC curve in Fig. 7. In Table 1 TPR values at FPR = 0.2 and FPR values at TPR = 1.0 are given for both methods. Table 1 - TPR values at FPR = 0.2 and FPR values at TPR = 1.0 for the proposed method and the general statistical appearance method (GSAM) for the given dataset. Proposed method GSAM TPR at FPR= FPR at TPR= Fig. 6 - An image of a non-defective tablet (left), a corresponding appearance model obtained by the general statistical appearance method (middle) and a corresponding appearance model obtained by the proposed method (right)

6 TPR GSAM Proposed method FPR Fig. 7 - ROC curves for the proposed method and the general statistical appearance method (GSAM) for the given dataset. 4. Discussion and conclusion In this paper, an alternative modelling strategy of the object-dependent imprint appearance variations, i.e. appearance variations caused by illumination, 3- D geometry and pose of the tablet in the scene at the image formation process, was proposed in a novel image analysis method. The method was designed for automated visual tablet inspection systems for defect detection and imprint quality inspection of tablets. The proposed method was compared to the general statistical appearance method [5], the most recent method for defect detection of imprinted tablets. Both methods are appearance based methods and, therefore, are based on comparison of an inspected tablet to an appearance model of nondefective tablet. The appearance model of nondefective tablet is obtained from principal components and appearance parameters. The principal components are obtained in the training phase from the training images, i.e. set of nondefective images, while the appearance parameters are obtain in the inspection phase. The general statistical appearance method obtains the appearance parameters by projecting an inspected image onto the principal components. Although the method accurately models non-defective imprint appearance variations, the estimation of appearance parameters is unable to reliably cope with outliers such as defects [10]. On the other hand, the proposed method uses solely the rotation parameter to obtain the appearance parameters, which makes the appearance parameters invariant to any defects of inspected images. The proposed method was evaluated by comparison to the general statistical appearance method on a dataset of real images acquired on a real industrial machine vision system Sensum SPINE. The dataset had low contrast defects, where appearance differences due to the defects were no greater than appearance variations of non-defective tablets. The specificity and sensitivity of the method was assessed with ROC curve (Fig. 7), where the proposed method showed better detection of defects, i.e. at FPR = 0.2 the proposed method yielded 0.96 TPR (sensitivity), while to general statistical appearance method yielded 0.88 TPR. In addition, the proposed method had significantly higher specificity (1 - FPR), i.e. lower FPR, at high sensitivity. At TPR = 1, w here all defective tablets are found as defective, FPR values for the proposed method and the general statistical appearance method were 0.33 and 0.55, respectively. In terms of speed and computational load, the proposed method has an advantage over the general statistical appearance method in the time critical phase, i.e. inspection phase. The calculation of appearance parameters by projection of a inspected image onto the principal components (3) is computationally more demanding than the extraction of the appearance parameters from the parametric functions (8). For the implementation details given in this paper, number of operations (multiplications) needed to obtain the appearance model is twice as much for the general statistical appearance method, i.e. 2mp and mp operations for the general statistical appearance method and the proposed method, respectively, where m is the number of elements in the tablet appearance x and p is the number of used principal components. According to the results of this study, the proposed analysis method has outperformed the general statistical appearance method for visual tablet inspection in terms of sensitivity, specificity and speed for defect detection of debossed tablets with low imprint quality. Acknowledgements This work was supported by the Ministry of Higher Education, Science and Technology, Republic of Slovenia under grants P2-0232, L2-7381, L2-9758, Z2-9366, by Sensum, Computer

7 Vision Systems, and by the European Union, European Social Fund. References [1] A. Berman, Reducing medication errors through naming, labeling, and packaging, Journal of medical systems, vol. 28, no. 1, pp. 9 29, [2] FDA, FDA 21CFR206, Imprinting of solid oral dosage form drug products for human use, Revised [3] T. S. Newman and A. K. Jain, A Survey of Automated Visual Inspection., Computer Vision and Image Understanding, vol. 61, no. 2, pp , [4] E. N. Malamas, E. G. Petrakis, M. Zervakis, L. Petit, and J. D. Legat, A survey on industrial vision systems, applications and tools, Image and Vision Computing, vol. 21, no. 2, pp , [5] M. Bukovec, Ž. Špiclin, F. Pernuš, and B. Likar, Automated visual inspection of imprinted pharmaceutical tablets, Measurement Science and Technology, vol. 18, no. 9, pp , [6] I. Jolliffe, Principal component analysis. New York: Springer-Verlag, [7] M. Možina, D. Tomaževič, F. Pernuš, and B. Likar, Real-time image segmentation for visual inspection of pharmaceutical tablets, Machine Vision and Applications, [8] Ž. Špiclin, M. Bukovec, F. Pernuš, and B. Likar, Image registration for visual inspection of imprinted pharmaceutical tablets, Machine Vision and Applications, [9] T. Fawcett, An introduction to ROC analysis, Pattern recognition letters, vol. 27, no. 8, pp , [10] A. Leonardis and H. Bischof, Robust Recognition Using Eigenimages, Computer Vision and Image Understanding, vol. 78, no. 1, pp , 2000.