Domain Specific Search Engine for Students

Transcription

1 Domain Specific Search Engine for Students Domain Specific Search Engine for Students Wai Yuen Tang The Department of Computer Science City University of Hong Kong, Hong Kong Lam For Kwok The Department of Computer Science City University of Hong Kong, Hong Kong Abstract: There are many different types of search engines, but none are specifically designed to suit the needs of students, or to match their curriculums. In this paper we explain the difference between general purpose search engines and domain specific search engines. To illustrate components of a domain specific search engine, we discuss the requirements of domain specific search engines designed for students. We propose an architectural framework for such a domain specific search engine, and use the curriculum of secondary mathematics in Hong Kong as an example. 1. Introduction Nowadays, students no longer acquire knowledge through books or journals alone. Instead, with the advance of technology and convenience of the Internet, most students make use of search engines for a number of reasons: to locate useful information and web pages for their project work; to retrieve additional materials or references related to their studies; and to find web pages of their own interest. Search engines have already become one of the popular educational tools. However, can they really satisfy the needs of students and assist learning? Are the results of searching really what they want? There are many different types of search engines, but none of them are specifically designed to suit the needs of students and match their curriculums. In this paper, we compare and contrast general purpose search engines with domain specific search engines; discuss the needs of domain specific search engines for students; and propose an architectural framework of a domain specific search engine which aims to provide domain specific search services for students. Through discussion of the requirements of this search engine, we would like to examine how the architectural framework operates using senior secondary mathematics as an example. 2. General Purpose Search Engine vs Domain Specific Search Engine A general purpose search engine is not designed to acquire a specific piece of information on the web, but instead attempts to cover as much as possible on the web in order to answer and handle all possible queries. Some famous and popular examples of general purpose search engines are Yahoo ( Google ( and AltaVista ( 747

2 International Conference on Computers in Education 2004 A domain specific search engine, on the other hand, can be defined as a specialized search engine that is designed to cover only a certain part of the web, with the same or similar category in the domain of interests. Examples include: CiteSeer ( which is an index of computer science journals and conference papers; and AlltheWeb ( which only indexes news documents. A general purpose search engine has the advantage of broad coverage as it tries to answer everything just like an encyclopedia would but its relevancy is usually low. Simple queries with keyword matching often generate too many search results, which are then difficult and sometimes confusing for students to choose from. Information overload, which may distract students from the main subject of learning, is a major problem when having to handle too much information at one time. Moreover, it is difficult to maintain and update such a large collection of web documents in a general purpose search engine. In comparison, the coverage of a domain specific search engine is small and cannot answer queries outside of its domain of interest. For example, it is impossible to search a news document in CiteSeer, as it will not index any news documents at all. However, its relevancy is very high, as the entire search results are within the domain, and irrelevant materials have been eliminated beforehand. For example, what you search in CiteSeer must be conference and journal papers in computer science. It will not search anything else. Furthermore, it is much easier to maintain and update a domain specific search engine due to its relatively small size compared to a general purpose search engine. 3. Needs of Domain Specific Search Engines for Students Domain specific search engines are necessary for students because general purpose search engines like Yahoo and Google may not be able to generate relevant search results. With the tremendous growth of the Internet, the one-for-all concept in many general purpose search engines, which try to cover the whole Internet, may no longer be useful or effective enough for some users with special interests or targets. Too many results are generated for manual selection, and their relevance is too low. Most search results of general purpose search engines cannot satisfy the needs of students. For example, a student studying mathematics may want to get more information about angle. He then uses angle as the keyword to search in Yahoo and Google, resulting in mostly commercial sites with a name using the word angle, rather than the relevant mathematical information. In fact, our recent search using the word angle shows that there are about 15 million sites in Yahoo with only five from the top fifty links being mathematics-related; and 11.5 million sites in Google, with only three from the top fifty links. Traditionally, general purpose search engines suffer from the problem of information overload because they try to cover the whole web on the Internet. With the exponential growth of the web, too many web pages will be found with simple keywords matching, which creates a problem for retrieval and/or identification of information. The problems of information overloading and low relevancy in search results, can be overcome by domain specific search engines because they only cover the part of the web within a domain of interest, filtering irrelevant materials that do not belong to these domains, and thus restricting the size of the search results to those of very high relevancy. Therefore, students need less time to select and evaluate the appropriateness of the results, leaving them more time to learn and focus on the information they want. It is desirable to have a domain specific search engine that meets the needs of students, and that is especially related to their curriculum of studies. In this study, we are interested in designing an educational domain specific search engine that aims to provide search results relevant to the domain of studies, and that restricts the size of search results to those especially under a specific domain. Relevancy of results will thus be raised, and will meet the needs of students. In Section 5, we use senior secondary mathematics as an example to explain how a domain specific search engine works. 748

3 Domain Specific Search Engine for Students 4. Architectural Framework of a Domain Specific Search Engine The proposed architectural framework of a domain specific search engine contains four layers and twelve components. Their respective functions are summarized in Table 1. The preparation layer is responsible for preparing the domain knowledge structure and acquiring the relevant web documents. The presentation layer is used for the actual searching process, and for ranking and presentation of results to users. The personalization layer is responsible for providing personalized service so that users experiences can be recorded for analysis later. The analysis layer is used for performing analysis on users access patterns, and discovering new knowledge from the analysis to provide better service and improvement of the search engine. Table 1: Architectural framework of a domain specific search engine for students Layer Component Description Preparation Presentation Personalizatio n Analysis Domain Construction Structure Knowledge Exploration Crawling and indexing Mapping of Web Pages and Domain Structure Construction of key terms, commonly used glossary, and the domain curriculum structure Classification of crawled web pages Interface Interface for communication between users and the search engine Query Engine Responsible for searching and interaction with users Ranking Algorithm Ranking according to the level of documents difficulty Access Log Individual profile to keep records of users experiences Bookmark Platform for individual s bookmark of mathematical links Links Suggestion Allows users to suggest links Links Evaluation Allows users to evaluate links Statistical Analysis Analysis of the usage log and personal profiles Generation of statistical results about usage pattern Knowledge Discovery Discover knowledge from statistical analysis Find users interpretations through the analysis Predict users preferences and search according to users needs The interaction between the four layers of the domain specific search engine is shown in Figure 2. Domain experts are supposed to have the clearest concept and idea of what the domain structure is, therefore they are responsible for preparing the key terms, curriculum structure, and the mapping of key terms into different categories in the curriculum structure. The preparation layer of the search engine can collect web documents according to the terms provided by the domain experts, and can perform classification according to the curriculum structure and their respective key terms. This information can then be stored in the database for future queries in the presentation layer. Users need to communicate with the search engine to conduct searches; the presentation layer provides users with an interface to specify the query, and to help them search the database prepared 749

4 International Conference on Computers in Education 2004 in the preparation layer. Because users are sometimes vague about the topic that they are searching, the presentation layer helps users to refine results by asking questions designed to reduce the scope and radius of the search. Finally, it presents ranked results according to the user s preference. All the actions and experiences of users during the query, interaction, and searching phases within the presentation layer are recorded by the personalization layer. This also allows users to bookmark useful web pages of a specific topic in a domain, and suggests useful web pages to the domain specific search engine. Evaluation function is provided for users to evaluate the relevancy of resulting links. All this information is important and useful for generating a user profile for later analysis. The analysis layer makes use of the logs and records in the user profile database to analyze access patterns. It also tries to discover new knowledge through interpretation of users experiences in order to predict one s access pattern. This provides feedback to the query engine in the presentation layer for generating more relevant results according to user interests. User-suggested links, and the new key terms that are found in the analysis, can be used as feedback to the knowledge exploration in the preparation layer when the domain specific search engine updates. Figure 2: Interactions of the four layers in the domain specific search engine 5. Senior Secondary Mathematics: A Case Study In order to prove that the concept of our domain specific search engine is viable, we use the curriculum of senior secondary mathematics in Hong Kong in our case study and examine how it can fit into our conceptual framework of a domain specific search engine. Our aim is to find useful web pages that are relevant to the subject. The mathematics curriculum chosen is at Key Stage 4 for Form 4 and Form 5 students aged about 15 to 18-years-old. Details of how the individual components work in the architectural framework of the domain specific search engine will be discussed in the following section. 750

5 Domain Specific Search Engine for Students 5.1 Key Terms and Domain Structure Construction When preparing the key terms and domain structures for Key Stage 4 secondary mathematics, we looked at the Syllabuses For Secondary School Mathematics Secondary (The Curriculum Development Council, 1999) and extracted key terms, commonly used glossary, and the curriculum structure, using the help of experienced teachers. A curriculum tree for Key Stage 4 mathematics is then constructed and acts as the index and domain structure of our search engine (Figure 3). The curriculum is divided into three main categories, namely Number and Algebra Dimension, Measure, Shape and Space Dimension and Data Handling Dimension. Each category contains several sub-categories. We map the commonly used glossary to their respective categories in the curriculum tree for classification of web documents. For example, keywords such as sine, cosine and tangent are mapped to the category of Trigonometry in the curriculum tree. Web pages with these keywords are then stored in the category of Trigonometry in the database. Figure 3: Curriculum tree and domain structure of the domain specific search engine 5.2 Knowledge Exploration Web documents have to be collected from the Internet and then indexed before the search engine can use them. The process of collecting web pages from the Internet is known as crawling in which we supply the URL links for the computers acquiring the respective documents. There are many approaches for efficient crawling in order to build a domain specific search engine, such as using machine learning (McCallum et al, 1999) and clustering (Zamir & Etzioni, 1998). Our focus is to collect and classify web documents related to the curriculum. We do not simply want web pages about mathematics, but to determine if they are part of the syllabus. In this case, we use a simple approach to restrict the knowledge exploration and make use of the links provided in the Hong Kong Curriculum Development Council, Yahoo and Google mathematics directory. These links serve as a quick starting point to ensure that all the web pages that we are going to collect will be within the mathematics domain. 751

6 International Conference on Computers in Education Mapping of Web Pages and Domain Structure After the web pages have been crawled, they have to be indexed, classified and stored inthe correct category of the domain structure. This is achieved by mapping key terms that exist in the web page to the appropriate category in the domain structure (defined in Section 5.1). For example, a web page containing the key term sine will be assigned to the Trigonometry category in the domain structure. Those that do not fall into any category will be marked as being outside of the syllabus. 5.4 Interface and Query Engine Interface is responsible for communication and interaction between users and the search engine, acquiring input and presenting results to users. Query engine is responsible for the searching process. Many users are either vague about the topic that they are going to search, or cannot produce a useful term for the query, thus lowering the results relevancy. Interactions with users are therefore necessary in the query engine, which helps users to clarify or supply additional information about the query, and helps the search engine to reduce both the number of documents and the domain radius for searching. Thereby, the relevance of search results is increased so that they are much closer to users expectations. For example, a student inputs area as a query term in the search engine. The search engine finds documents that contain area in three categories such as 3D Figures, Circle and Rectangular. It interacts with the student by prompting whether they are searching area in the category of 3D Figures or Circle or Rectangular. It allows them to select in order to reduce the searching size, and to try to match the student s concept of area with the one that exists in the curriculum tree of the search engine. 5.5 Ranking Algorithm Most search engines rank web pages only according to their popularity based on link analysis. A web page will have a higher rank if many other web documents have a reference link pointing to it. This can be an effective way for ranking in terms of general interests, but it may not really reflect the usefulness or relevancy of the web page to users. Students may be interested to know whether the content of a web page is suitable to be browsed according to the curriculum, their level of understanding and ability, etc. Comparing the level of difficulty between documents is meaningful only when documents are in the same category. Therefore, we give a ranking of difficulty only to documents belonging to the same category. However, defining the level of difficulty of a document, teaching a computer to identify the degree of difficulty of documents, and comparing the level of difficulty between contents of web pages are all challenging activities. We are still at the experimental stage in addressing these issues, but preliminary findings indicate that for documents in the same class and category, those documents that contain more distinct keywords are more difficult. For example, a web page containing both keywords sine and cosine will be more difficult than the web page that contains only the keyword tangent in the category of Trigonometry. This is logical because the former covers two concepts while the latter covers only one in Trigonometry. 752

7 Domain Specific Search Engine for Students Access Log and Bookmark Access logs in the profiles of different users allow the search engine to keep track of individual user s activities, and keep logs on the query terms, choices and selection of result links for further analysis. This is in order to provide better and tailor-made services for individuals. Users are classified into two main categories: teachers and students. The teacher class is considered to be a class of domain experts who have a better knowledge and interpretation of mathematics than students. Therefore, their actions and evaluations have higher weighting than those of the student class. The search engine also provides a personalization function that allows users to bookmark the mathematics web pages that they are interested in, just like the Favourite function in the popular browsers. This allows the search engine to provide a platform for users to use their bookmarks anywhere, and does not restrict a user to use the bookmarks on designated computers only. Moreover, this function allows us to gain a clearer idea of how the user organizes, classifies and interprets certain web pages for further analysis. 5.6 Links Suggestion and Links Evaluation One of the methods used to increase document collections and yet maintain a high degree of document relevancy for a search engine, is to allow users to suggest useful links. However, it is difficult to examine every suggested link manually, and it is necessary to make the evaluation process automatic. It is straightforward in that all we need to do is repeat Section 5.2 in which the link evaluation component will filter the web page if none of the terms inside match the commonly used glossary. Otherwise we assign it to one of the respective categories in the curriculum tree where the terms match. A user can also manually assign a suggested link to a category which they consider relevant, so that we can gain further understanding about the user s interpretation of the suggested links. Evaluation of searching results is also important to the success of the search engine. Evaluation can be based on the following: 1. Relevancy to the mathematics domain 2. Relevancy to the assigned category in the curriculum tree 3. Relevancy to the syllabus of studies 4. Ranking of level of difficulty in relation to content of the documents in the same category In this regard, the profile discussed in Section 5.6 plays a very important role as it helps to identify whether the evaluator is a student or a teacher. Different users may have different interpretations of the subject matter, and evaluations are subjective. However, teachers are assumed to be the domain experts and they should have better knowledge, understanding and interpretation of the subject than students. Thus, teachers will have a higher weighting during the evaluation. Also, in order to obey the majority rule, those who always have different evaluation results from their own group will have a lower weighting (Hwang et al, 2004). 5.7 Statistical Analysis Statistical analysis will be applied to the logs and the personal profiles of the search engine in order to provide both public statistics and personal statistics respectively. Public statistics can include the statistics of the most popular query, the category that accesses most, the most common query term, and results of link selection. These can be used to act as feedback to the query search engine, in order to suggest user alternatives during a query based on public statistics. Personal statistics can be 753

8 International Conference on Computers in Education 2004 used to show individuals about their statistics in relation to access patterns in the search engine, using query patterns within particular categories of the curriculum tree, etc. 5.8 Knowledge Discovery Using the analysis discussed in Section 5.7, we may even discover some new terms that are important, or are in the majority of favourites, but that have been previously ignored. This is done by checking the most common query terms in the search engine logs that do not exist in the commonly used glossary. We may even discover individual term-to-term relationships and concepts with respect to a particular user. These are identified through interactions between the user and the query engine (see Section 5.4). With all this data, we can then try to predict user behaviour and access patterns, returning results according to his preferences or habits. Conclusion We described a framework of a domain specific search engine. It seems that a domain specific search engine will be more desirable and effective in assisting students learning than a general purpose search engine. We are currently at the stage of working on the preparation layer and presentation layer; and conducting experiments on some of the advanced features such as document understanding, and classifying the level of the documents difficulty. The aim of this work is to better enhance the search engine s effectiveness in helping students to learn. Evaluation will be done by asking students to rank documents in certain categories like trigonometry by their own perception of a document s difficulty. We then take the students ranking of document difficulty levels and compare these to our own ranking algorithm. This tests whether our hypothesis in ranking by a document s level of difficulty is correct and useful. References The Curriculum Development Council (1999). Syllabuses for Secondary School Mathematics Secondary 1 5. HK: The Education Department. The Curriculum Development Council (1991). An English-Chinese Glossary of Terms Commonly Used in the Teaching of Mathematics in Secondary Schools. HK: The Education Department. Hwang G.-Jen, Huang T.C.K., Tseng J.C.R. (2004). A group-decision approach for evaluating educational web sites, Computers & Education 42 (1). Elsevier Science McCallum, A., Nigam, K., Rennie, J. and Seymore, K. (1999). Building domain-specific search engines with machine learning techniques, Proc. AAAI-99 Spring Symposium on Intelligent Agents in Cyberspace. USA: The AAAI Press. Zamir, O. and Etzioni, O. Web document clustering: feasibility demonstration Annual ACM Conference on Research and Development in Information Retrieval: Proceedings of the 21 st Annual International Conference ACM SIGIR Conference on Research and Development in Information Retrieval