Automatic Hidden Web Database Classification

Transcription

1 Automatic idden Web atabase Cassification Zhiguo Gong, Jingbai Zhang, and Qian Liu Facuty of Science and Technoogy niversity of Macau Macao, PRC Abstract. In this paper, a method for automatic cassification of idden-web databases is addressed. In our approach, the cassification tree for idden Web databases is constructed by taioring the we accepted cassification tree of MOZ irectory. Then the feature for each cass is extracted from randomy seected Web documents in the corresponding category. For each Web database, query terms are seected from the cass features based on their weights. A hidden-web database is then probed by anayzing the resuts of the cass-specific query. To raise the performance further, we aso use Web pages which have inks pointing to the hidden-web database (W-B) as another important source to represent the database. We combine ink-based evauation and query-based probing as our fina cassification soution. The experiment shows that the combined method can produce much better performance for cassification of hidden Web atabases. 1 Introduction With the exposive growth of the Word Wide Web, the traditiona Crawers fai to satisfy the users demand for information searching yet. Many recent studies [1, 2] have observed that a significant fraction of Web content known as the idden-web (W) [3], the Invisibe Web [4], or the eep Web [2], ies outside the PIW. In fact, these pages can ony be dynamicay generated in response to users queries, which the traditiona Crawers cannot hande. owever, we cannot simpy ignore them, because some recent studies caim that the size of the idden-web pages are as many as 500 biion pages, comparing to ony two biion pages of the ordinary web [5]. Furthermore, the information on the W is usuay generated from structured databases, which are referred to as idden-web atabases (W-B) [6]. In [7], the study has estimated that there are 250,000 private databases, and the access of 95% of them is free. These databases represent 54% of the idden Web. In this paper, in order to effectivey guide users to find the reevant information from such databases, we present a prototype system for cassifying the W-B into a predefined category hierarchy which is taiored from some existing cassification tree for Web documents. The feature for each cass is extracted from randomy seected Web documents in corresponding Web cass. For each Web database, query terms are seected from such cass features based on their weights. A hidden-web database is J.N. Kok et a. (Eds.): PK 2007, LNAI 4702, pp , Springer-Verag Berin eideberg 2007

2 Automatic idden Web atabase Cassification 455 then probed by anayzing the resuts of the cass-specific query to the hidden database. To raise the performance further, we aso use Web pages which have inks pointing to the hidden database as another important source to represent the database. We combine ink-based evauation and query-based probing as our fina cassification soution for hidden database cassification. In addition, our focus is on text databases, since 84% of a searchabe databases on the web are estimated to provide access to text documents [2], and other kinds of databases ike image or video databases are out of the scope of this paper. The contributions presented in this artice are organized as foows. We present the detais of our W-B cassification system based on query probing and based on ink evauation in Section 2. A system evauation is conducted and important experimenta resuts are discussed in Section 3. And finay section 4 provides concusions. 2 idden atabases Cassification Our system aims to automaticay assign each idden-web atabase to the best category or categories of the cassification scheme. Instead of constructing a new cassification scheme manuay, simiar to the approaches proposed by [5, 7], we expoit a category hierarchy for W-B cassification from the popuar MOZ irectory. Fig.1. shows a fraction of the category hierarchy used in our system. Fig. 1. A fraction of the category hierarchy for W-B used in our system 2.1 Cassification Modes In order to assign a Web document to corresponding categories, a cassifier agorithm is needed. Severa cassifier modes exist in iterature, such as SVM (Support Vector Machine) [8], knn (key nearest Neighbor) [9], LLSF (Linear Least Square Fit) [10], NNet (Neura Network) [11], NB (Naïve Bayes) [11], and RIPPER [12]. Though [6] shows that RIPPER can provide good overa performance for W-B cassification, the correctness of the rues for each category are critica for the precision of the cassifications. owever, it is a hard work for correct rue extractions. Take the rue ( ibm AN computer Computer ) as an exampe, even though some document

3 456 Z. Gong, J. Zhang, and Q. Liu may contain both ibm and computer, it may not beong to the category of Computer in many cases. Furthermore, the cassification needs to extensivey interact with a W-B. That means, for each rue the system needs to interact at east one time with the database. Y. Yang and X. Liu compared other cassifiers and pointed out that mode SVM, knn and NB can aways produce better performance for the document cassifications over LLSF and NNet. Considering both effectiveness and efficiency as the important factors, in our work, we empoy knn as the cassifier for W-B cassifications. To use knn, training documents for each category are needed. In our impementation, for any category c i, we seect N Web documents (d j,1, d j,2,, d j,n) from the corresponding MOZ directory. Then, for any Web document x, the cassification rue in knn can be written as: Simiarity( x, c j ) = 1 sim x d i (, N j, i ) b j (1) where sim(x,d j,i ) is the metric of simiarity between x and d j,i, b j is the category specific threshod for the binary cassification. In order to cacuate sim(x,d j,i ), we represent each Web document d as a vector (w 1,w 2, w M ), where w is the weight of term t in d. And w is defined as: w = tf ( t, d ) Max { tf ( t, { n } n d where tf(t,d) is the frequency of term t in d. With this definition, for any two web documents d=( w 1,w 2, w M ) and d =( w 1,w 2, w M ), sim(d,d ) is defined as the cosine vaue between them: w w' sim ( d, d ') =. 2 2 ( w ) ( w' ) The origina knn cassifier is designed for document cassification. For a hidden Web database, et {hd 1, hd 2,,hd R } be a the documents contained in. We concatenate a the documents of into one document, sti denoted as. That is, =mhd j. Then, is assigned to category c if and ony if Simiarity(,c) Â0. owever, we do not have the knowedge about the documents within. To sove the probem, we use two techniques to approximate it in this study. Firsty, we detect the W-B through probing; secondy, ink structure of the Web is used. 2.2 idden atabase Probing In each category, some queries are needed for probing hidden databases. [6] uses extensive number of rues or queries for probing. As mentioned before, mutipe query probing is expensive for both rue extracting and database probing. For such reasons, in our approach, we ony use one query for probing in each category. Our one-query probing is based on the assumption that it does not affect the cassification too much because every category uses the same number of queries (one query in our paper). We extract candidate query terms for each category from the concatenation of its a training documents seected from the corresponding MOZ irectory. Those )} (2) (3)

4 Automatic idden Web atabase Cassification 457 terms, caed category feature, are ordered with their weights. We chose severa terms according to their weights as a query to probe hidden databases. After sending the request message incuding form fied-out information to the server, our proposed system wi receive the resut pages. Perhaps the most common case is that a web server returns resuts page by page consecutivey, with a fixed number, say ten or twenty, resut matches per page. To cassify the W databases effectivey, we need to anayze the content of each resut document. owever, fu-text of resuts from some W-B cannot be obtained for some reasons ike copyright. So the system handes differenty for these two situations Resut ocuments Without Fu-Text In this situation, ony the number of returned documents for the query can be used for anaysis. Let c 1, c 2,, c K be K categories under the same parent node in the hierarchy, the returned number for each category by a hidden database are L 1, L 2,,L K respectivey. Then, we approximate as: j f K j L j 1 (4) where f j is the category feature of cass c j. In fact, f j is the centroid of the training documents in c j Resut ocuments with Fu-Text For the hidden databases whose fu-texts can be accessed, our system can anayze the document content further to get more accurate approximation for. In such case, not ony the number of the resuts for a category can be got, but aso the reevance of the documents can be used. To save the cost, we ony access documents in severa positions aong the resut ist. For exampe, the positions can be set to the first resut and the ast resut, or more compex to 0%, 25%, 50%, 75%, and 100% of the resut ist. Suppose we ony access the first document hd j,1 and the ast document hd j,lj for category c j. Then, the hidden database can be approximated as: 1 2 ( hd j K j + hd j L ) L 1 1, j j, (5) where L j is defined as before. It can be easiy extended to support more document accesses. With the probed resut for a hidden database (equation 4, or 5), can aways be cassified using equation (1). 2.3 W-B Cassification Based on Link Structure In ast subsection, we introduced the methods for the idden-web databases cassification based on probing, which produces good experimenta resuts. owever, it does not make use of the properties of Web structure, especiay the inks among the Web documents. Actuay, ink structure of the Web provides another important cue

5 458 Z. Gong, J. Zhang, and Q. Liu for W-B cassifications. In fact, as in fig.2, a hidden database may be referenced by many Web pages. Those pages can aso be used to derive the semantics of the hidden database. Fig. 2. A idden database inked by other pages (neighbor pages) Web pages, which have inks to the hidden database, are caed neighbor pages for this database. To use them for the cassification of, we concatenate a of the neighbor pages into one document caed NP. Then, the semantics of is represented with a vector of terms extracted from NP. Therefore, is aso can be cassified by the vaues of Simiarity(NP,c). 2.4 Combined Cassifier for idden atabases To raise the performance of the cassification, we try to combine probing mode with ink-based mode. In fact, new hidden databases often have ess neighbor pages to be referenced. Therefore, probing method is the ony way for the cassification in such situation. To avoid outier, we use ink-based cassifiers ony for hidden databases which have at east 20 neighbor pages. The combined cassifier is defined as: C-Simiarity(, c)=w*simiarity(,c)+(1-w)*simiarity(np,c) (6) where W is used to baance this two cassifiers. 3 Experiment Our objective functions for system performance are based on two basic metrics precision and reca [5]. When evauating the resut of cassification, there are three important vaues for each category: A ---- Number of documents which are cassified into the category correcty; B ---- Number of documents which are cassified into the category wrongy;

6 Automatic idden Web atabase Cassification 459 C ---- Number of documents which are cassified into other category wrongy; Reca is the ratio of the number of documents cassified into a category correcty to the tota number of reevant documents in the same category. Precision is the ratio of the number of documents cassified into a category correcty to the tota number of irreevant and reevant documents cassified into the same category. Both of them can be represented with the above vaues, A, B, and C. A precision = 100% A + B, reca = A 100% A + C To condense precision and reca into one number, we use the F 1 -measure metric [5]: 2 precision reca F 1 = precision + reca which is ony high when both precision and reca are high, and is ow for design options that triviay obtain high precision by sacrificing reca or vice versa. Reca and precision are eveny weighted. 3.1 etermining the Number of the Feature Terms for Form Fiing-Out There are two types of form-eements in genera, A-Eement (support Booean AN ) and O-Eement (support Booean OR ). We must choose a proper number of query terms for fiing out form eements for these two types. A-Eement and O- Eement modes occupy 41% and 59% respectivey in our testing hidden databases. We fi-out those hidden databases with changing number of query terms. K F L K Z % ' : + I R R L W 5 G L I L V V O F G W R & &RFW5WLRRI +:'% &OVVLILFWLRQ $(OPQW FRFW5WLRRI +:'% &OVVLILFWLRQ 2(OPQW 1XPERI7PVIR)LOOLQJRXW)RPV Fig. 3. The ratio of W-B which are cassified correcty using different quantities of terms Fig.3. shows the correct ratio of W-B cassification using different numbers of feature terms. The horizonta axis shows the number of terms to fi-out the forms and the vertica axis shows the ratio of W-B which are cassified correcty. It can be seen from the figure, for A-Eement, the correct radio reaches its summit 63% when we choose 3 terms to fi-out the forms. For O-Eement, the optima number of terms is 6, which eads to 61% correct radio. That is, we shoud choose 6 terms to fi-out the forms for O-Eement, in order to receive the maxima correct ratio.

7 460 Z. Gong, J. Zhang, and Q. Liu 3.2 Evauating Resuts over ifferent Cassification Approaches In our system, three basic modes for cassification of hidden databases are addressed, incuding fu-text probing M 1, resut-number ony probing M 2, and ink-based cassifying M 3. By combining M 1 with M 3, M 2 and M 3, we get two combined cassification modes. Fig.4. shows the cassification performances for basic mode M 1 and M 2, as we as the two combined modes. It is cear from the figure, by far the combined method (M 1 + M 3 ) receives the best performance when baance perimeter W=0.4. And the second combined mode (M 2 + M 3 ) reaches its optima performance when W=0.3. X V P ) J Y $ 0QG0 &RPELQWLRQ 0QG0 &RPELQWLRQ 9OXRI: Fig. 4. The Cassification Performance of Combine Methods by ifferent Weight The average F 1 -measures of those methods are shown in Tabe 1. By far, the combined method (M 1 +M 3 ) is the best approach for W-B cassification. owever, other methods shoud not be abandoned since each method has its own merit. Method M 1 and M 3 are the basic ones for the combine method (M 1 +M 3 ). Athough method M 2 shows the worst performance among them, it is a good aternative if a W-B cannot returns fu-text of resut documents. In addition, M 2 is with the ow cost comparing with M 1. Tabe 1. Average F 1 -measure of M 1, M 2, M 3 and the Combine Methods Methods M 1 M 2 M 2 +M 3 M 1 +M 3 Average F measure 4 Concusions In this paper, we have proposed a nove and efficient approach for cassification of idden-web atabases. We have introduced a category hierarchy for W-B and described the process to extract the feature for each category. With terms of the features, we probe the hidden databases and anayze the resuts documents in order to

8 Automatic idden Web atabase Cassification 461 cassify the W-B. To raise the performance further, we aso use Web pages which have inks pointing to the hidden-web database as another important source to represent the databases. We combine ink-based evauation and query-based probing as our fina cassification soution. Our experiment shows the combined approach can generate a much better performance for the W-B cassification. Acknowedgement. This Work was supported in part by the niversity Research Committee under Grant No. RG069/05-06S/07R/GZG/FST and by the Science and Technoogy eveopment Found of Macao Government under Grant No. 044/2006/A. References [1] Lawrence, S., Gies, C.L.: Accessibiity of Information on the Web. Nature 400, (1999) [2] Bergman, M.K.: The eep Web: Surfacing idden Vaue Latest Access: 11/1/2007 (September 2001), [3] Raghavan, S., Garcia-Moina,.: Crawing the idden Web. In: Proceedings of the 27th Internationa Conference on Very Large ata Bases (VLB) (2001) [4] Lin, K.I., Cheng,.: Automatic Information iscovery form the Invisibe Web. In: Proceedings of the Internationa Conference on Information Technoogy: Coding and Computing (ITCC) (2002) [5] Ipeirotis, P.G., Gravano, L., Sahami, M.: Probe, Count, and Cassify: Categorizing idden-web atabases. In: Proceedings of the 20th ACM SIGMO Internationa Conference on Management of ata, ACM Press, New York (2001) [6] Gravano, L., Ipeirotis, P.G., Sahami, M.: QProber: A System for Automatic Cassification of idden-web atabases. ACM Transactions on Information Systems (TOIS) 21(1), 1 41 (2003) [7] Berghoz, A., Chidovskii, B.: Crawing for omain-specific idden Web Resources. In: Proceedings of the 4th Internationa Conference on Web Information Systems Engineering (WISE 03) (2003) [8] Vapnik, V.: The Nature of Statistic Learning Theory. Springer, New York (1995) [9] asarathy, B.V.: Nearest Neighbor (NN) Norms: NN Pattern Cassification Techniques. In: McGraw-i Computer Science Series, IEEE Computer Society Press, Las Aamitos, Caifornia (1991) [10] Yang, Y., Chute, C.G.: An exampe-based mapping method for text categorization and retrieva. ACM Transaction on Information Systems (TOIS) 12(3), (1994) [11] Mitche, T.: Machine Learning. McGraw i, New York (1996) [12] Cohen, W.W.: Learning trees and rues with set-vaued features. In: Proceedings of the Thirteenth Nationa Conference on Artificia Inteigence, pp (1996)