RESOLVING AMBIGUITIES IN PREPOSITION PHRASE USING GENETIC ALGORITHM

Size: px

Start display at page:

Download "RESOLVING AMBIGUITIES IN PREPOSITION PHRASE USING GENETIC ALGORITHM"

Caitlin Underwood
5 years ago
Views:

International Journal of Computer Engineering and Applications, Volume VIII, Issue III, December 14 RESOLVING AMBIGUITIES IN PREPOSITION PHRASE USING GENETIC ALGORITHM Department of Computer

1 International Journal of Computer Engineering and Applications, Volume VIII, Issue III, December 14 RESOLVING AMBIGUITIES IN PREPOSITION PHRASE USING GENETIC ALGORITHM Department of Computer Engineering, Sri Guru Granth Sahib World University, Fatehgarh Sahib, Punjab, India ABSTRACT: Text mining generally refers to the method of extracting some relevant, useful, novel and interesting knowledge from unstructured text. While performing this step, the major problem that is often faced, is that some words or statements are capable of being interpreted in two or more meanings especially in case of natural languages. This problem is known as ambiguity. As ambiguity can lead to incorrect conclusions, detecting ambiguous words and sentences and resolving them becomes a fundamental aspect in text mining. A new technique is represented in this paper for resolving ambiguous preposition phrase. The approach is based on the integration of possibility theory, fuzzy sets and genetic algorithm. Fuzzy sets are most widely used to overcome the preposition phrase ambiguity. Genetic algorithm is used to solve optimization and search problems using various operators based on ideas of natural evolution. The overall obtained results indicate the approach is viable. Keywords: Text mining, Ambiguity, Genetic Algorithm, NLP, Fuzzy set, Phrase [1] INTRODUCTION [1.1] TEXT MINING Text mining, which is also known as knowledge discovery from textual databases can be viewed as the process of deriving valuable and non-trivial patterns or information from text documents embedded in unstructured form from a fixed domain. Text mining helps users to find useful information from a large amount of digital text data. It is therefore important that a good text mining model should retrieve the information that meets the requirements of user with relevant efficiency. Traditional Information Retrieval (IR) has the same objective of automatically retrieving as many relevant documents as possible whilst filtering out irrelevant documents at the same time. Many text mining methods have been developed in order to achieve the goal of retrieving useful information for users. Most text mining methods use the keyword based approaches, while others choose the phrase technique to create a text representation for a number of documents. As more information is carried by a phrase than by a single term, it is understood that the phrase-based approaches should perform better than the keyword-based ones. The advance in database technology and the extensive use of computers have provided vast amounts of data. Due to which, knowledge discovery and data mining have attracted great attention with an urgent need for converting such data into useful 61

2 Resolving Ambiguities In Preposition Phrase Using Genetic Algorithm information and knowledge. The explosive growth of data day by day in databases has generated requirement for effective and efficient data mining techniques to find out important information and knowledge from large quantity of data. Applications like market analysis and business management can benefit by the use of such techniques. In the past years, a large number of data mining techniques have been developed in order to carry out different knowledge tasks. These techniques consist of text mining, sequential pattern mining, frequent item set mining, association rule mining, etc. [1.2] GENETIC ALGORITHM Genetic algorithms (GAs) are adaptive heuristic search methods based on ideas of natural evolution and genetics which is used to solve optimization and search problems. GAs is designed to imitate processes in natural systems required for evolution, which follow the theory of "survival of the fittest". In GA, a population of candidate solutions is maintained within search space, each solution signify a possible solution to a given optimization problem. GAs encodes each candidate solution or individual of a search problem into finite length strings of variables, or alphabets, generally the binary alphabet (0, 1). These individuals or strings are called as chromosomes, the variables are likened to genes and the values of genes are referred to as alleles. For example, the traveling salesperson problem is the problem of finding the best possible path to cross, say, 10 cities. So, in this problem, the salesperson can start from any city and its solution will be a permutation of the 10 cities: In this problem, a city represents a gene and a route represents a chromosome. Fitness score means a measure for distinguishing good solutions from bad solutions or it is a measure of how good that chromosome is at solving the problem at hand. This fitness score is assigned to every possible solution which represents the abilities of each solution to compete'. The solution with the best fitness score is sought. The population size, usually a user specified parameter, is one of the significant factor which can affect the scalability and performance of GAs. Small population sizes may lead to premature convergence and can give substandard solutions. Large population sizes may lead to unnecessary loss of valuable computational time. Once each individual is coded as a chromosomal method and a fitness measure for distinguishing best solutions over worst ones has been selected, solutions to the search problem can be evolved by the following steps [Figure 1]: 1. Initialization: The initial population of candidate solutions is randomly generated for the problem at hand within the search space. The size of population may include hundreds to thousands of candidate solutions. However, domain-specific knowledge or other information can be easily integrated. 2. Evaluation: After initialization of the population the fitness measure of each candidate solutions are evaluated. 3. Selection: In this step, two or more members or individual solutions with highest fitness values are selected and forms a mating pool. The key idea of selection is to select good solutions over worse ones, allowing them to pass on their genes to the next generation and the goodness of every individual solution depends on its fitness measure. Many selection 62

International Journal of Computer Engineering and Applications, Volume VIII, Issue III, December 14 procedures have been developed to carry out this idea, like roulette-wheel selection, ranking

3 International Journal of Computer Engineering and Applications, Volume VIII, Issue III, December 14 procedures have been developed to carry out this idea, like roulette-wheel selection, ranking selection, and tournament selection. 4. Crossover: Crossover (also known as recombination) recombines parts of two or more individual solutions to form better and new solutions. The two individual solutions or strings participating in crossover operation are called parental solutions and the resulting strings or solutions are called children or offspring solutions. In this, new solutions are created by exchanging information among parental solutions of the mating pool. 5. Mutation: In this step, new information is added in a random way to the genetic search process. While recombination works on two or more parental solutions or chromosomes, mutation modifies a solution locally and randomly. It usually involves one or more modification being made to an individual solution's one or more characteristics whenever the population tends to become homogeneous due to repeated use of selection and recombination operators. So, mutation executes a random walk in the surrounding area of a candidate solution. Mutation may cause the chromosomes of resulting solution to be different from those of their parent solutions. 6. Replacement: At last, the original parental population is replaced by the resulting population formed by selection, crossover and mutation operators. Replacement techniques like generation-wise replacement, steady-state replacement and elitist replacement methods are used in GAs. 7. Steps 2-6 are repeated until a terminating condition is met. Figure: 1. Steps in GA [2] RELATED WORK Alfawareh et al. [2] [2013] in the paper "Resolving Ambiguous Preposition Phrase for Text Mining Applications" tried to resolve the problem of ambiguities occurring in preposition phrase. This paper includes background and detail view of text mining and natural language processing (NLP) and discusses about ambiguity problem. The work presented in this paper describes a new approach that is based on the integration of NLP techniques, possibility theory, fuzzy set and context knowledge. 63

4 Resolving Ambiguities In Preposition Phrase Using Genetic Algorithm Shaidah et al. [6] [2012] in the paper "Techniques, Applications and Challenging Issue in Text Mining" provide an general idea of text mining in the perspective of its techniques, application domains and the most challenging issues. Fundamentals methods of text mining which consist of natural language possessing (NLP) and information extraction (IE) are described briefly. Also a short review on application domains which have employed text mining is given. This paper also addresses the challenging issue in text mining caused by the complexity in a natural language. Shivani et al. [8] [2013] in the paper "A Detailed Study on Text Mining using Genetic Algorithm" discusses the concept of text mining and genetic algorithm in detail. This paper represents a brief view on text mining process and methods along with fundamental objectives and importance of text mining studies. Also Genetic Algorithm and its usage in text mining are described with the help of flow chart. This researches show an improved performance in the text mining field using Genetic Algorithm. Genetic Algorithm has been used to provide useful solutions to optimization problems. [3] PROPOSED APPROACH AND RESULTS The framework of proposed approach consists of four components [Figure 2]. These components are subject knowledge store, possibility theory, fuzzy grammar and NLP techniques. The illustration of components is shown in flowchart. These components are utilized as unambiguous fact extraction processor. In this, input is sentence and output is unambiguous fact. Figure: 2. Flow Chart of Work 64

5 International Journal of Computer Engineering and Applications, Volume VIII, Issue III, December 14 Steps in Flow Chart: Step 1: Sentence It is treated as input to the fact extraction processor. Before producing the unambiguous fact each sentence is processed syntactically and semantically. Parsing with set Optimized GA Genetic algorithm maintains a population of randomly generated candidate solutions (individuals). Though GA is an iterative process the generated population is called a generation for each iteration. Now, the fitness of each candidate solution (individual) in the population is evaluated in each generation. The fitness is generally a measure which can be an objective function of how good that individual is at the optimization problem being solved. Then the new generation of candidate solution is used in next iteration of GA. Usually, when a suitable fitness level has been achieved for the population or a large number of generations have been generated, this algorithm terminates. A genetic algorithm requires a genetic representation of the solution domain and a fitness measure to evaluate that solution domain. For the purpose of this work, the template given in Algorithm 1is used [Figure 3]. Begin Initialization: The initial population P (t = 0) is generated of n individuals Fitness: The fitness of every individual of the population is evaluated. Evaluate (P(t)) While (not termination condition) do Selection: A subset of m pairs from P(t) is selected. Let P1(t) = Select(P(t)). Crossover: Cross each of the m selected pairs with probability pc. Let P2(t) = Cross(P1(t)) be the set of off springs. Mutation: Mutate each offspring in P2(t) with probability pm. Let P3(t) = Mutate(P2(t)). Fitness: Then the fitness of every offspring is evaluated. Evaluate (P3(t)). Replacement: A new generation from individuals in P(t) and P3(t) is created. Let P(t + 1) = Replace(P(t); P3(t));t = t + 1. While Return Best found solution; End Figure :3. Represent P-best value 65

6 Resolving Ambiguities In Preposition Phrase Using Genetic Algorithm When GA is applied to dataset crossover and mutation are calculated. On the basis of these results plotation of P-best values are done here [Figure 4]. Figure:4. Represent Gross value G-best is calculated after calculating P-best in previous section. Here line start from 0 shows gross result. Step 2: Handling Multiple Parsing In this we improve the early algorithms to handle the multiple parsing. If a sentence has more than one syntactic structure, it produces more than one parsing tree. This step is combining into the step of parsing sentence with fuzzy grammar. Step 3: Resolving Ambiguous Fact during Parsing This step represents the heart of unambiguous facts extraction technique. To resolve the ambiguous fact we combine the knowledge about the subject context and possibility theory. Step 4: Semantic Attachment The output of previous three steps is taken and after than semantic attachment is conducted from the previous results to assign the correct semantic of its constituent. Step 5: Converting a parse tree into a graph Parse tree with semantic is converted into graphs. This step play a very important role in pattern matching process with graph and that graph is stored in the knowledge base. Implementation: Step 6: Matching Graphs The graph which is created with parse tree is matched with the existing graph. In the subject context knowledge-base, it is achieved by searching a same pattern of graph in the subject context knowledge-base. When the same pattern exists, thus the graph is match [Figure 5]. 66

International Journal of Computer Engineering and Applications, Volume VIII, Issue III, December 14 Figure:5. Represent clustering on number of data Here potation of dataset and clustering is shown.

Step 7: Graph Selection When the process of pattern search is succeed and matching graph is identified. So we take the graph as a input.

7 International Journal of Computer Engineering and Applications, Volume VIII, Issue III, December 14 Figure:5. Represent clustering on number of data Here potation of dataset and clustering is shown. The vertical lines represent clustering and bubbles represent number of data. Superposed clusters are ambiguous clusters. Step 7: Graph Selection When the process of pattern search is succeed and matching graph is identified. So we take the graph as a input. If there is more than one graph then there will be more graph matched with the graph in the subject Context knowledge-base. So after that we find the most effective graph. We select the graph on the basis of graph selection is involved with calculating the plausibility value of each graph and selecting the most possible graph by taking the graph that has the highest plausibility value. Step 8: Fact Representation After selection, the graph is converted into formal knowledge representation. We used a predicate calculus [Figure 6]. Then this is stored in to knowledge base. Than we select the most possible graph. To convert the graph into predicate calculus, two things are important, the nodes and the relationship name that is associated with the edge. The node will be converted into atom of predicate calculus while the relationship name will be converted into relation of predicate calculus. A result of the conversion process from a graph into a predicate calculus can be represented. Figure : 6. Represent detection of ambiguous data 67

Resolving Ambiguities In Preposition Phrase Using Genetic Algorithm Data is distinguished according to clusters. Here four sections are divided and these are for seven iterations.

8 Resolving Ambiguities In Preposition Phrase Using Genetic Algorithm Data is distinguished according to clusters. Here four sections are divided and these are for seven iterations. In first section one ambiguity is found, in second other ambiguities is found, in third classification is done and in last ambiguous data is shown. Fact Knowledge-base It is a conceptually knowledge-base where all unambiguous extracted facts will be stored. The knowledge-base will be used in the next step of a text mining system. After that convert the parse tree in graphical form then match these all graphs with each other. After matching the graphs with each other we do the selection of optimized graph and that optimized graph show the Fact representation and knowledge database [figure 7]. Figure :7 Represent ambiguous data Among four segments the segment which showing ambiguous data is shown. [4] ANALYSIS Sentence 1: The shopper heard a shot sound. Sentence 2: The shopper saw a robber with a gun. Sentence 3: He called a police station. Sentence 4: The police came to the place. Sentence 5: The police ran into the shop. Sentence 6: The police shot the robber in the shop. Sentence no. Type of ambiguity Graph tree (1) Graph tree (2) Selected Graph 1 none None None None 2 Structural and fact none None None None 4 none None None None 5 Structural Structural and fact This table represents the results of the test case and extracted unambiguous facts. 1. Type of Ambiguity presents the results for the steps of parsing with fuzzy grammar and handling multiple parsing. 68

9 International Journal of Computer Engineering and Applications, Volume VIII, Issue III, December Graph/Tree (1)/ (2) present the results of steps for resolving structural ambiguity, converting parse tree into graph and graph matching. 3. Selected Graph presents the results of the step graph selection. 4. Unambiguous fact presents the fact representation step. 5. The sentence sequence in the test case is indicated by Sentence No. 6. Type of Ambiguity represents either the sentence is ambiguous or unambiguous. In analyzing the obtained results, a preposition phrase have been classified into two groups first is a certain preposition 2 nd is uncertain preposition. Uncertain preposition phrase is as a preposition phrase that contains preposition words such as on', in', with', below', and behind'. Certain preposition phrase is that contains preposition words such as onto', into', of' and to', that are unlikely to cause ambiguity to the facts. By classifying PPs, we can differentiate between a sentence that has both structural ambiguity and fact ambiguity or only structural ambiguity. [5] CONCLUSION AND FUTURE WORK Prepositions are often among the most frequent words in a language. Ambiguity problem is the main difficult issue in preposition phrase. In resolving the ambiguities for preposition phrase various techniques are used like fuzzy set, possibility theory, classifier, nearest neighbour method, genetic algorithm etc. This research describes a new technique to resolve ambiguity problems in preposition phrase and also different techniques used in related work. The approach is based on the integration of possibility theory, NLP techniques, context knowledge-based approach, fuzzy sets and genetic algorithm. The knowledge-based approach was utilized for the implementation of context knowledge. The fuzzy sets, possibility theory, context knowledge have been utilized in selecting the most possible fact from many possible facts. The proposed approach has been implemented and obtained results has been presented and discussed in this paper. 69

10 Resolving Ambiguities In Preposition Phrase Using Genetic Algorithm REFERENCES [1] Aguilar N., Alemany L.A., Lloberes M., Castellon I., "Resolving prepositional phrase attachment ambiguities in Spanish with a classifier" Procesamiento Del Lenguaje Natural 46, pp , [2] Alfawareh H.M. and Jusoh S., "Resolving Ambiguous Preposition Phrase for Text Mining Applications" International Conference of Computer application technology (ICCAT), pp. 1-5, [3] Chodorow M., Tetreault J. R. and Han N.R., "Detection of grammatical errors involving prepositions," Proceedings of the 4th ACL-SIGSEM Workshop on Prepositions, pp , [4] Ghosh S., Roy S. and Bandyopadhyay S. K., "A tutorial review on Text Mining Algorithms" International Journal of Advanced Research in Computer and Communication Engineering, vol. 1, no. 4, pp , June [5] Jusoh S., Alfawareh H.M., "Agent-based Knowledge Mining Architecture" Proceedings of the 2009 International Conference on Computer Engineering and Applications, vol. 2, pp , [6] Jusoh S. and Alfawareh H. M., "Techniques, Applications and Challenging Issue in Text Mining" IJCSI International Journal of Computer Science Issues, vol. 9, no.6, pp. 431, November [7] Kiyavitskaya N., Zeni N., Mich L. and Berry D., "Requirements for Tools for Ambiguity Identification and Measurement in Natural Language Requirements Specifications", Requirements Engineering Journal, vol. 13, no. 3, pp , 2008, Toronto Canada. [8] Patel S., Gandhi P., "A Detailed Study on Text Mining using Genetic Algorithm" International Journal of Engineering Development and Research, vol. 1,no. 2, pp ,September [9] Redfearn J., "Text mining" Joint Information System Committee, pp. 1-2, [10] Stokoe C., Oakes M.P., and Tait J., "Word Sense Disambiguation in Information Retrieval Revisited", Proceedings of the 26 th annual international ACM SIGIR conference on research and development in information retrieval, pp , 2003, Toronto Canada. [11] Zhao S. and Lin D., "A Nearest-Neighbor Method for Resolving PP-Attachment Ambiguity", Proceedings of the First International Joint Conference on Natural Language Processing, pp , [12] Zhong N., Li Y. and Wu S.T., "Effective Pattern Discovery for Text Mining" IEEE Transactions on Knowledge and Data Engineering, vol. 24, no. 1, pp , January

The k-means Algorithm and Genetic Algorithm

The k-means Algorithm and Genetic Algorithm k-means algorithm Genetic algorithm Rough set approach Fuzzy set approaches Chapter 8 2 The K-Means Algorithm The K-Means algorithm is a simple yet effective