Fuzzy recognition of the defect of TFT-LCD

Transcription

1 Fuzzy recognition of the defect of TFT-LCD Zhang Yu, Zhang Jian Institute of Ultra Precision Optical & Electronic Instrument Engineering, Harbin Institute of Technology, Harbin, , China ABSTRACT On-line real-time detection method for the defect of TFT-LCD is becoming increasingly important as TFT-LCD has replaced CRT displays and become the first choice in many applications. Traditional defect inspection methods of TFT- LCD are based on clear features and exact mathematic models. However, the defects of TFT-LCD are of strong complexity and vagueness. Moreover, determining the defects is a complicated process, which is influenced by the objective characteristics of the defects as well as the subjective factors of the observer. Therefore, it is very difficult to establish the accurate mathematical models for the defects. A fuzzy expert system approach is proposed for the defect inspection of TFT-LCD. Tests indicate that this system could emulate the experts or experienced operators to realize the automatization of the defect inspectin of TFT-LCD. Keywords: TFT-LCD, defect inspection, fuzzy expert system 1. INTRODUCTION Thin-film transistor liquid crystal displays (TFT-LCDs) gradually become mainstream as the displays of personal computers because the prices are going down and the visual characteristics, that is to say, the resolution, the brightness and the viewing-angle are dramatically being improved. According to Displaysearch, the world market for displays in 2003 are million units in which the share of LCD is million occupying 43.4% of the market, compared with 29% in Moreover, according to Displaysearch's forecast, the world market for LCD will reach million in 2004, overtaking CRT's million. And in 2006, the world market for LCD will reach 100 million, compared with CRT's million, which means LCD will thoroughly replace CRT. However, the market share of TFT-LCD is stilly under the influence of the price. So, it is very important to develop the detection method for the defect of TFT-LCD to decrease the production cost of LCD. Till now, several detection methods have been developed. However, owing to the complexity and vagueness of the defect of TFT-LCD, these methods are not able to fulfill the task of defect detection completely. Therefore, most defect detections are still performed by experienced inspectors. 2. THE COMPLEXITY AND VAGUENESS OF THE DEFECT OF TFT-LCD The manufacturing process of TFT-LCD is very complicated, which comprises of about 100 working procedures. Though most of them are performed in the net room, the TFT-LCD panel would exhibit some visual defects inescapability. In the TFT-LCD productions, there are plenty of defects that could be classified in different ways. For example, they are classified as electrical or non-electrical, depending on whether or not the reason of defect is electrical. They are classified as point defect (or pixel defect), line defect and region defect, depending on whether the shape and size of the defect is like a point, a piece of line or a region. Moreover, point defects could be grouped into bright point and dark point, depending on the defective pixel is brighter or darker than the background. Point defects could also be classified as single point or multiple points, depending on the number of defective pixels neighboring. Similarly, line defects could be grouped into bright line and dark line depending on the defective line is brighter or darker than the background. Meanwhile, line defects could be grouped into horizontal line defect and vertical line defect depending on the direction of defective line is horizontal or vertical. In the same way, region defects could also be divided into bright region and dark region, depending on whether the region is brighter or darker than the background. Mura, a Japanese word means blemish that has been adopted in English, is one of the most familiar region defects. Muras could also be classified in various ways. For example, they are grouped into line mura, block mura and stripe mura, depending on the shape of mura is like a line, a block or stripes. Electronic Imaging and Multimedia Technology IV, edited by Chung-Sheng Li, Minerva M. Yeung, Proc. of SPIE Vol (SPIE, Bellingham, WA, 2005) X/05/$15 doi: /

2 In all of them, the most familiar defects are point defects and line defects, which are mainly caused by open and shorted signal and scanning lines and defective transistors. Point defect always appears as a bright dot or a dark dot in an area of about one pixel. Manufacturers usually tolerate a limited number of point defects if they are not clustered together. Line defect appears as a whole line across the LCD panel or just a portion of it, which is bright or dark. Line defect must be detected because it is generally considered fatal. The most difficult testing challenge is the detection and quantification of mura caused by a variety of non-electrical factors, such as non-uniform distribution of the liquid crystal material or foreign particles within the liquid crystal. It appears as low contrast region of a variety of sizes and shapes that is brighter or darker than the average background luminance, typically larger than single pixel. Traditional defect inspection methods of TFT-LCD are all based on clear features and exact mathematic models. However, it is a complicated process to determine the defects of TFT-LCD, which is not only related to the objective characteristic of the defects but also related to the subjective factors of observers. Therefore, the defects of TFT-LCD are of strong complexity and vagueness and it is very difficult to establish the accurate mathematical models. 3. FEATURES SELECTION FOR THE DEFECT INSPECTION OF TFT-LCD Actually, defect detection is a process of classification in terms of various features extracted from the object of interest. The selection of feature will directly influence the difficulty, reliability and accuracy of the defect inspection. As mentioned above, there are different categories of defects of TFT-LCD, which behave variously. Accordingly, different features should be selected in terms of different types and characteristics of defects. For example, since a point defect always appear as a constant bright or dark point whose area is about one pixel and the adjacent or neighboring two or more points seem to be more serious. So, three features, the contrast, area and location of the area of interest should be considered while judging point defects. In the same way, a line defect always appear as a constant bright or dark line whose width is about one pixel. Four features, the contrast, area, location and direction of the area of interest should be considered while judging line defects. Mura defects appear as low contrast, non-uniform brightness regions, typically larger than a single pixel. By experience, mura defects with high contrast and small area are easy to be detected by observers. On the contrary, the mura defects with low contrast and big area are not easy to be detected. In addition, the mura defects with clear outline, regular shape, single direction, uniform gray value and close to center of panel, are easy to be detected. On the contrary, the mura defects with vague outline, irregular shape, unfixed direction, uneven gray value and far from the center of panel, are not easy to be detected. Generally, traditional detection algorithms just take several main features such as contrast and area into account, which are not consistent to the vision characteristic and mentality of human beings. Therefore, while judging mura, the features extracted by the algorithm presented in this paper should include the area, contrast, shape, outline, location, direction and uniformity. 4. THE ARCHITECTURE AND CHARACTERISTIC OF A FUZZY EXPERT SYSTEM Expert systems are computer programs that emulate the reasoning process of human experts. And a fuzzy expert system is an expert system that uses a collection of fuzzy rules, instead of Boolean logic, to reason about data. Generally, a classical fuzzy expert system consists of several components as follows: fuzzy knowledge base; fuzzy inference engine; fuzzy data base; user interface; interpreting module and knowledge acquisition module, as shown in Fig.1. User interface Knowledge acquisition module Fuzzy knowledge base Fuzzy data base Fuzzy inference engine Interpreting module Figure 1: Structure of a classical fuzzy expert system 234 Proc. of SPIE Vol. 5637

3 1) Fuzzy knowledge base. A fuzzy knowledge base consists of facts base and rule data base, which are used to store the domain specific knowledge and experience. 2) Fuzzy inference engine. The function of fuzzy inference engine is to resolve the problem brought forward by the system and provide rational suggestion and conclusion according to the information input to the system using fuzzy reasoning (uncertainty reasoning) strategy. 3) Fuzzy data base. Fuzzy data base is mainly used to store various static and dynamic date needed and produced by running the system, which provide necessary data for the inference and interpreting of expert system. 4) User interface. User interface provides the link from system to user. It is designed using graphics and hypertext. 5) Interpreting module. The function of interpreting module is to answer the problems asked by user emulating domain experts. Meanwhile, experts could know the state of system with interpreting module. 6) Knowledge acquisition module. The function of knowledge acquisition module is to comprehend and transform the knowledge of domain experts into symbolic format required for system, which are appended to knowledge base as new knowledge. Conventional expert system is based on probability theory, whose information in their knowledge base is generally inexact, incomplete. Accordingly, traditional approximate reasoning method, such as Bayesian formula, doesn't work very well facing such information. That is mainly because the knowledge of human beings reflects fuzziness and vagueness rather than randomicity. Fuzzy logical provide a foundation for building a more systemic and reliable reasoning method based on the knowledge base. Compared with conventional expert system based on probability theory, fuzzy expert system has some characteristics as follows: 1) The premise portion and consequence portion of a rule could be vague. 2) The premise portion of a rule could be partially matched with the fact. 3) The fact in knowledge base could be expressed with fuzzy set. With the characteristics above, fuzzy expert system could obtain exact results from uncertainty problem, without any "frangibility" existed in common expert system. Image acquisition Image preprocessing Image segmentation Expert Feature extraction User User interface Knowledge acquisition Fuzzy data base Interpreting module Fuzzy knowledge base Fuzzy inference engine Figure 2: Structure of the proposed system 5. A FUZZY EXPERT SYSTEM FOR THE DEFECT INSPECTION OF TFT-LCD Proc. of SPIE Vol

4 5.1 Structure of the system Fig.2 shows the overall structure of a fuzzy expert system for defect inspection of TFT-LCD. A Basler area scan and industrial camera is adopted as the component of image acquisition, which has a fastest full resolution of 1300 x 1030 pixels. To reduce the influence of noises in the picture captured by CCD, multiple sampling is performed and afterward image averaging is done on the pictures sampled. By experience, the influence of noise will decrease along with the increase in the number of pictures. Meanwhile, median filtering is adopted to get rid of the superfine details and join the disconnected points within the object. Image segmentation is a process of grouping an image into units that are homogeneous with respect to one or more characteristics and obtaining the object of interest. In the component of image segmentation, owing to the fact that point defects and line defects generally have clear outline and regular shape, threshold method is adopted. And because the area, shape and edge of mura are variable and ambiguous, effective segmentation is hardly possible with traditional segmentation techniques such as threshold method and edge detection method. Consequently, a region growing approach based on background knowledge is adopted in the mura detection algorithm. In the component of feature extraction, proper features are extracted from the image being tested in terms of the type of defects, which are provided for the fuzzy expert system to recognize the defects. 5.2 Determine the membership functions Membership function is the core of fuzzy pattern recognition. The forming of the membership function has an effect on the setting up of fuzzy knowledge base as well as the result of pattern recognition. However, it is a hard task to set up the membership function, especially for the objects of complexity and vagueness. As far as the forming method employed is concerned, statistical histogram approach combining analog approach are fit for the discrete objects to determine the membership function of each fuzzy term. 1) Fuzzify the input data. Due to the complexity and fuzziness of the defects of TFT-LCD, the actual value of each feature is scattered and the characteristic information is vague and no concentrated. Therefore, original data are orthogonalized and then normalized using maxi-minimum rule. 2) Construct the statistical histogram. Statistics are used to make membership functions for a fuzzy rule-based algorithm. Membership functions of each fuzzy item, such as area, contrast, shape, outline, location, direction and uniformity, are estimated in terms of each type of defect. Hundreds of samples with defects are selected to generate membership functions. Since the samples are insufficient to produce smooth histogram distribution, a smoothing operator (averaging operator with size eleven) is used to make the histograms smooth, and the smoothed histograms are normalized so that the maximum value is Set up fuzzy knowledge base Real-time is required and maintainability of the knowledge is needed for the defect detections of TFT-LCD. Among the common knowledge representation methods, rule representation has advantage over others as follows: first, it is consistent with the knowledge represented in human mind so it is easy to understand and set up; second, it is modularized and coherent so make the knowledge in knowledge base has the same format and the data base could be accessed by all rules. Each rule is independent thus it is easy to add, delete and change, which makes knowledge base is easy to be maintained and extended, too. Accordingly, in terms of the characteristics of the defect of TFT-LCD, the general form of fuzzy production rule is adopted by this system as follow: IF w * P1Λ w2* P2Λ... Λw THEN Q(CF,τ ), 1 n * Pn where Pi (i = 1,2,..., n), Q are all fuzzy terms, which take any value from the interval [ 0, 1 ]; w i ( i = 1,2,...n) is the weight coefficient of premise Pi and wi 0, wi = 1; CF is the degree of confidence of rule, and 0 CF 1;τ is the practicable threshold of the rule, 0 τ 1. The essence of the rules above is: if the degree of truth of the premise T(P) = w i * T(P) 236 Proc. of SPIE Vol. 5637

5 is greater than or equal to the threshold τ, the rule could be adopt, which means corresponding defect is likely to appear in the image being inspected. Where T(Pi ) is the degree of truth of Pi (i = 1,2,..., n).the result of applying the rule is the derivation of conclusion Q, whose degree of truth is T(Q) = T(P) ΛCF. Where Λ is multiplication operator. In many practical applications, each sub-premise of a rule premise has different importance and meaning. The same problem exist in the system proposed in this paper, such as a piece of knowledge: IF the contrast of the region-of-interest is high AND the area of the region-of-interest is big AND the outline of the region-of-interest is vague AND the location of the region-of-interest is central AND the shape of the region-of-interest is irregular AND the direction of the region-of-interest is vague AND the uniformity of the region-of-interest is low THEN the panel has serious mura defect In this rule, the importance of each sub-premise is invisible, which means that each sub-premise has the same influence on the judgment of defect. However, in practice, each condition plays different role in the process of defect detection for expert. To make the rule representation is more consistent with the facts and reflect the importance of each fact, different weight coefficient is assigned to each sub-premise of the rules. For example, the degree of importance is classified into 5 grades: most important; very important; important; common; unimportant. Each of the 5 grades is corresponding to a certain numerical value respectively. Therefore, the weight coefficient of each sub-premise is the numerical value corresponding to this sub-premise divided by the sum of the numerical values corresponding to all the sub-premises. An example illustrating this approach is as follow: IF the contrast of the region-of-interest is high (most important) AND the area of the region-of-interest is big (most important) AND the outline of the region-of-interest is vague (very important) AND the location of the region-of-interest is central (important) AND he shape of the region-of-interest is irregular (common) AND the direction of the region-of-interest is vague (unimportant) AND the uniformity of the region-of-interest is low (unimportant) THEN the panel has serious mura defect. Supposed that each of the 5 grades is corresponding to a numerical value 5, 4, 3, 2 and 1 respectively. Then the weight coefficients of each fact can be calculated. For example, the weight coefficient of contrast is high is w = 5 /( ) = Similarly, each fact of a rule is corresponding to a weight coefficient. After the knowledge representation is determined, it is easy to work out the structure of knowledge base. 5.4 Fuzzy inference To achieve the purpose of inspect the defects of TFT-LCD in real-time, a forward chaining inference strategy is used in this system. The process of inference is as follows: 1) Receive data from fuzzy data base, then match them with the data in the premise partial of each rule in terms of Optimum Matching and give the value of matching result to the premise of rule. 2) Calculate the degree of matching between the data in data base and fuzzy term in the premise of each rule δ match(ri, E i), where R i denotes a certain rule, E i denotes the fact in data base. δ match (Pi(R), E(P i)), E(Pi ) = µ Pi E(Pi ) If the premise portion of the rule comprises several sub-premises, apply Proc. of SPIE Vol

6 a-1 a-2 b-1 b-2 c-1 Figure 3:Examples of defect inspection c Proc. of SPIE Vol. 5637

7 δ match(r, E) = match (P1(R), E(P1)) δ Λ δmatch (P1(R), E(P2)) Λ Λ δ match (Pn(R), E(Pn)) and δmatch(r, E) δ 0 (activating threshold) to estimate how many rules being activated. Where Λ denote fuzzy AND operator. 3) No rule being activated shows there is no defect in the panel. If there are one more rules are activated, they will constitute an conflict set (R1, R 2,...,Rn). Then Optimum Matching is applied, that is to say, selecting the rule of maximum degree of matching as the conflict resolution strategy. 4) Calculate the degree of confidence for conclusion δ opr(q(r)) = δmatch(p1(r), E(P 1)) Λδmatch(P2(R),E(P 2)) Λ... Λδmatch(Pn(R),E(P n)) Λδ opd (q(r) ΛCF(R)) where P1(R), P2(R),..., Pn(R) is the premise of each rule respectively; E(P1 ), E(P2),...,E(Pn) is the data best matching to P 1, P2,..., Pn respectively; q(r) is the action of rule; δ opd is the degree of confidence for action; δ opr(q(r) is the degree of confidence for implementing q(r). 6. EXPERIMENT RESULTS The developed system has been tested in plenty of applications. Here we provide several examples of how this system has been effectively used to recognize the defects as shown in Fig. 3. Fig. 3 (a-1), Fig. 3(b-1) and 3 (b-3) show the captured images. Fig. 3 (a-2), 3 (b-2) and 3 (c-2) are the processed images by image averaging, median filtering and image segmenting. According to the features extracted from objects of interest, the fuzzy expert system determines the three objects as serious dark point, vertical bright line and mura defect respectively. Tab. 1 is the defect reports of the three examples created by the system proposed in this paper. Table 1: Defect reports created by the system Sample X Y Width Height Area Contrast Result (pixels) (pixels) (mm) (mm) (mm 2 ) (%) Dark point Dark point Vertical bright line Mura 7. CONCLUSIONS This paper has presented a fuzzy expert system approach for the development of an inspection system of the defects of LCDs. An expert system is the most popular technique because of its capability to deal with fuzziness, handle uncertainty, and process ill-structured knowledge. Owing to the fact that the defects of TFT-LCD are of strong complexity and vagueness, different features are selected in terms of different types and characteristics of defects to be provided for the fuzzy expert system. Tests indicate that the system could emulate expert or experienced operator to realize the automatization of defect inspectin of TFT-LCD. ACKNOWLEDGEMENTS The authors wish to thank AU Optronics (Suzhou) Corporation for its support to this project. Proc. of SPIE Vol

8 REFERENCES 1. Y. Mori, K. Tanahashhi, Extraction and evaluation of mura in liquid crystal displays, SPIE Vol. 4471(2001): William K. Pratt, Sunil S. Sawkar, Automatic blemish detection in liquid crystal flat panel displays, Proceeding of SPIE, 2000, Vol. 3306: Y. Mori, K. Tanahashhi, S. Tsuji, Quantitative evaluation of visual performance of liquid crystal displays, proceeding of SPIE Vol. 4113(2000): YE Qi-xiang, GAO Wen, WANG Wei-Qiang, A Color Image Segmentation Algorithm by Using Color and Spatial Information, Journal software, 2004, 15(4): Li Deng-feng, Cheng Chun-tian, New similarity measures of intuitionistic fuzzy sets and application to pattern recognitions, Pattern Recognition Letters, 2002, 23: Kido, Takashi, In-process functional inspection technique for TFT-LCD arrays, Journal of the Society for Information Display, 1993, 1(4): Jeff H, Electro-optics technology tests flat-panel displays, Laser Focus World, 2000, 36: Qu Hui-ming, Study of defect test for TFT-LCD panel, Optoelectronic Technology, 1997,17(2): William K. Pratt, Machine vision methods for automatic defect detection in liquid crystal displays, Advanced Imaging, 1998, 13(4): Christian D. Klose, Fuzzy rule-based expert system for short-range seismic prediction, Computers & Geosciences 28 (2002) Enbo Feng, Haibin Yang, Ming Rao, Fuzzy expert system for real-time process condition monitoring and incident prevention, Expert Systems with Applications 15 (1998) S. Hirano, Y. Hata, Fuzzy expert system for foot CT image segmentation, Image and Vision Computing 19 (2001) Ha-jin Yu, Yung Hwan Oh, Fuzzy expert system for continuous speech recognition, Expert system with Application, Vol 9, No. 1, pp.81-89, T. W. Liao, Classification of welding flaw types with fuzzy expert systems, Expert system with Application 25 (2003) Steven Mackinson, An adaptive fuzzy expert system for prediction structure, dynamics and distribution of herring shoals, Ecological Modeling 126 (2000) L. A. Zadeh, Fuzzy sets, Information and Control 8 (1965) L. A. Zadeh, Fuzzy logic, Computer 1 (1988) zhangyu111_72@sina.com; phone: ; fax: ; Address: Institute of Ultra Precision Optical & Electronic Instrument Engineering, Harbin Institute of Technology, Harbin , China 240 Proc. of SPIE Vol. 5637