Special Issue of IJCIM Proceedings of the

Transcription

1 Special Issue of IJCIM Proceedings of the Eigh th-.. '' '. Jnte ' '... : matio'....' ' nal'.. ' ;p ~--.' :'.... :... ej.lci! -1'--: "'..f(~aa, D-.,.,a...l~ OR elattmng tot.~av-~e-ijajil:u. ~~ Pta~.,

2 Strategies to Improve Query Processing Time in Searching Membership Queries of Virtual Classroom by Using DBIC Demonstration School Faculty of Education, Prince of Songkhla University Songkhla, Thailand Abstract The Virtual Classroom System of Faculty of Education, Prince of Songkla University Pattani Campus has been used for many years. Therefore, the database size is increasing and the search query speed is slow. The overall system performance becomes consequently delayed. Creating an index is one of the ways to optimize the effectiveness of searching membership queries to reduce expenses on hardware installation. In this paper, we represented ways to create and apply the Data Clustering with Dual Bitmap Index (DBIC), leading to less query time in searching membership queries of the Virtual Classroom System. Thus, it was found that using DBIC index could properly reduce time in searching membership queries. Keywords- Dual Bitmap Index, Data Clustering, Virtual Classroom 1. Introduction Data Warehousing (DW) and On-Line Analytical Processing (OLAP) systems are large repositories of data that contain historical and integrated data collected from a number of heterogeneous sources [4] Such sources might include Online Transaction Processing (OLTP) or legacy operational systems over a long period of time. Requests for information from a DW are usually complex and iterative queries of what has happened in the past. For example, a business DW containing information on specific customers might issue a query "Find the number of customers who purchased a specific product during a marketing promotion, and identify other purchases made by more than 75% of those customers during the past year." Most of the queries contain aggregate functions such as group-by and a lot of join operations involving large numbers of records. Such complex queries could take several hours or days to process, depending on the amount of data searched and how the data is organized. A majority of requests for information from a DW involve dynamic ad hoc queries [ 6] users can ask any question at any time for any reason against the base table in a DW. The ability to answer complex queries quickly is a critical issue in a DW environment. The Virtual Classroom system of Faculty of Education, Prince of Songkla University Pattani Campus has been used for many years. Therefore, the database size is increasing and the search query speed is slow. The overall system performance becomes consequently delayed [12]. There are many solutions to improve query processing time, including using parallel machines, summary tables, and indexing techniques [7,8,9] Summary tables can significantly improve efficiency for predetermined queries. However, the system must scan, fetch, and sort the actual data for all ad hoc queries, resulting in much slower retrieval. Additionally, whenever the base table changes, the summary tables must be recomputed. Summary tables only support known, frequent queries, and they often require more time and more space than the

3 original data. It is impossible to build every possible summary table, and the ones chosen may not be optimal for every user [2]. Indexing techniques [1] using bitmap representations can answer complex, ad hoc queries efficiently without adding additional hardware. They significantly improve query processing time by utilizing fast, efficient Boolean operations and multiple index scans, performing simple predicate conditions on the index level before going to the primary data source. Of the various bitmap indexing techniques, Scatter Bitmap Index is well suited to process complex DW queries. It is simple to represent, uses less space and is more CPU -efficient than variant bitmap indices. Scatter Bitmap Index uses individual bitmap vectors to represent many attribute values. A single unique attribute value is represented by crossing two vectors. Ind~xing data with a Scatter Bitmap Index is more efficient than other indexing techniques for equality and membership queries, in terms of space-time trade-off [3]. In this paper, we combine Dual Bitmap Index with a data mining technique called Data Clustering to improve membership query efficiency. We fmd the relationships among attribute values in queries, resulting in improved query processing time. The rest of the paper is organized as follows: In Section 2 we give an overview of Dual Bitmap Index. In Section 3 we present our Using Data Clustering to Optimize Dual Bitmap Index (DBIC). In section 4 we discuss our comparative study for improving Virtual Classroom system. Finally, m Section 5 we give conclusions. measure, which determines the similarity or proximity of two data elements. Presently, several Data Clustering algorithms have been proposed, including Partitioning Clustering, Hierarchical Clustering, Density-based Clustering, and Grid-based Clustering. We choose Partitioning Clustering for our work. Partitioning Clustering [10] is simple and speedy in a large data set. The algorithm steps are 1. Select the number of clusters, k. 2. Generate k random points as cluster centers. 3. Assign each data element to the nearest cluster center. 4. Recompute the new cluster centers. 5. Repeat step 3 and 4 until the assignment has not changed or the distance of two data elements is acceptable. Note that all k resulting clusters are determined simultaneously and each data element is assigned into different clusters. However, the k clusters are different in each run because the clusters depend on the method used in generating cluster centers. K-mode algorithm uses the value that occurs the most frequently in the data set as the center. It is suitable for categorical data such as gender, color, etc B. Dual Bitmap Index Dual Bitmap Index (DBIC) [11] overcomes a disadvantage of this problem by reducing space requirements as well as improving The idea is to fmd a total number of bitmap vectors used to represent all indexed attribute values under condition of using only two bitmap vectors to represent. 2. Related Work A. Data Clustering Data Clustering [8] is one data mining technique, which is used in many fields, including pattern recognition, bioinformatics and image analysis. It divides a data set into clusters by using some defined distance Fig. 1 Diagram of Dual Bitmap Index with 6 bitmap vectors

4 Therefore, for cardinality C we have G)~c Solving the above equation, we get n = lv'2c j Two bitmap vectors represent each attribute { (n-rxn-r-1)) d value are D' and D,._ 1 v 2 mo,. Where r=l~2(hic-v) j TABLE I NOTATION USED IN DUAL BITMAP INDEX'S ALGORITHM where r = l~2(hic -v) j, s=r-1-(v (n-rx;-r- 1 ) )modr and v is the value of an indexed attribute for any record. 2. Equality and Membership Queries The algorithm for answering equality queries with Dual Bitmap Index is described below. 2.1 Find the sequence number ofthe searching value. "A = v" = D,. 1\ D' Where r = l~2(hic -v) o.sj, s=r-1-(v (n-rx;-r- 1 ))modr 1. hic Bitmap vectors created for our Dual Bitmap Index, wh~ j.. O,l,...,n-1 The highest valu of C that oon be re~ted by n bitmap vectors. The Creation of Dual Bitmap Index Dual Bitmap Index on an attribute A is a set of the bitmap vectors D = {D 0, D\..., D"- 1 }.Each bitmap vector represents a subset of values of indexed attribute A as shown in Fig. 1. A bit at position i in the bitmap vector is set to 1 if the record at position i in the indexed table is satisfied equation. We summarize in Table I the notation used in the algorithm. The algorithm of building Dual Bitmap Index is described below (for details refer to [11]). 1.1 Assign an increasing sequence of numbers to each of the distinct values of A (i.e., 0,1..., C -1 ). 1.2 Calculate n=bl2c j 1.3 Calculate hic = (;) 1.4 For each value von record at position iina if j =r or j = s otherwise. and v is the value of an indexed attribute for any record. 3. Dual Bitmap Index Optimization Using Data Clustering In this section, we present our Dual Bitmap Index Optimization using Data Clustering (DBIC).The DBIC consists of three phases (see Fig. 1): Data Preparation, Attribute Clustering and Optimizing Dual Bitmap Index. A. Phase 1: Data Preparation There are four steps in the Data Preparation phases. Step 1: Attributes values are extracted from the workload of SQL queries to create an attribute value table. Fig. 1 shows an example of the workload which contains values of attribute Y extracted from the workload, where the number of distinct attribute values of attribute C is 15 Step2: The value m is calculated as m =.JC + 1.In our example, m is equal to 5.(m= Value of group)

5 Step3: The weighted bitmatrix table is built from scanning the attribute value table at once. The weighted bitmatrix table is a R x N table, where R is the number of all queries in the workload and N is the number of distinct attribute values. Each column represents the value of the indexed attribute in query Qi and each row represents a query Qi having the attribute value. The bitmatrix[ij] is assigned using the following rule: Let Qi be a membership query having selection condition "Yin {v1, v2,..., Vn}", and let n be the total number of attribute values appearing in Qi. If n is equal to 1 then each attribute value in Qi (i.e., Vj) is weighted 0. If n is equal to rxm, where r 2: 1 then Vj is weighted 4. Otherwise, Vj is weighted 2. Step4: The total weight table is created by summing the weight of each attribute value. B. Phase 2: Attribute Clustering In this phase, the attribute weight values are sorted in ascending order. According to the weight distance, attribute values having the proximity weight are clustered together. Then, k (i.e., m-1) clusters are generated. Each cluster has at most m elements. C. Phase 3: Optimizing Dual Bitmap Index Then the Boolean operation OR is used to get the final result. DBI and DBIC may take only one bitmap vector if the selection condition is composed of attribute values in the same group. We choose query Ql in Fig. 1 to show the performance difference between DBI and DBIC. Q 1 has the selection condition Y in (A, B, C, E, J, K, M, N). By using the grouping scheme in Fig. 1 for (A, B, E, K, M). the retrieval function to answer is ( D 5 "D 4 ) V ( D 5 "D 3 ) V ( D 5 "D 2 ) V ( D 5 "D 1 ) v (n 5 "n ) but DBIC can retrieval D 5 only. To confirm our analysis, experiments on data sets from TPC-H [5] (Transaction Processing Performance Council, 2010) and Ql: select* from Twhere Yin(A, B, C, E, J, K, M, N) Q2: select * from T where Yin (D, F, H, L, N) Q3: select* from Twhere Yin (G, J, L) Q4: select* from T where Yin (A, B, D, E, F, H, K, L, M, N) Query Fig.l Workload ofsql queries Attribute Y A B c D E F G H I J K L M Ql QZ Q Q Total Fig. 2 Weighted bitmatrix table and Total N In the last phase, we group each of distinct attribute values using the resulting clusters from the previous phase. The result in this phase is a well-defined grouping scheme that leads to improve performance of response time in answering membership queries (see Fig. 4). 4. Performance Study This section presents the results of comparing the time performance of two bitmap indexing techniques (DBI and DBIC). For answering membership queries, DBI and DBIC create a bitmap vector representing each distinct attribute value Fig. 3 Resulting clusters Fig.4 Grouping scheme ofdbic on attribute Y.

6 OBI DBIC membership queries, in terms of space-time trade-of References Equality query Membership query Fig. 5 Time performance of DBI vs. DBIC data sets from Virtual Classroom system of Faculty of Education, Prince of Songkla University Pattani Campus (C=150) Virtual Classroom system of Faculty of Education, Prince of Songkla University Pattani Campus were conducted [12]. The experiments were run on a 1.69 GHz Intel Celeron with 504 MB main memory. Fig. 5 illustrates the results of comparing the time performance of DBI and DBIC for evaluating membership queries on the indexed attribute. As seen, using DBIC outperforms using DBI. V. Conclusion A critical issue for data warehouse applications or Virtual Classroom system of Faculty of Education, Prince of Songkla University Pattani Campus are the ability to answer complex, iterative, and ad-hoc queries quickly. Both DBI and DBIC apparently outperform other techniques in term of space-time efficiency improvement. It is imperative that the grouping scheme is well-defined when using Dual Bitmap Index. In this paper, we represented ways to create and apply the Data Clustering with Dual Bitmap Index (DBIC), leading to less query time in searching membership queries of the Virtual Classroom System. Thus, it was found that using DBIC index could properly reduce time in searching membership queries. Our comparative study and experimental results show that, in the best case, DBIC performs better than existing techniques for [1] A. Silberschatx, H.F.Korth and S.Sudarshan (2001), "Database System Concept", Me Graw Hill. [2] C.Y. Chan and Y. E. Joannidis (1999), "An Efficient Bitmap Encoding Scheme for Selection Queries", Proceeding of ACMSIGMOD. [3] S. Vanichayobon, J. Manfuekphan and L. Gruenwald (2006), "Scatter Bitmap: Space-Time Efficient Bitmap Indexing for Equality and Membership Queries", Proceedings of IEEE International Conferences on Cybernetics and Intelligent Systems (CIS 2006) [4] A. Berson, S. J. Smith (1977), "Data Warehousing and Data Mining", McGraw -Hill. [5] The TPC Benchmark?H (TPC-H), Transaction Processing Performance Council. [6] J. Han and M. Kamber (2001), "Data Mining: Concepts and Techniques", Morgan-Kaufmann, San Francisco. [7] R. Kimball (2008), "Data Warehouse Designer", Intelligent Enterprise Magazine. [8] M. Zaman, J. Surabattula and L. Gruenwald (2004), "An Auto-Indexing Technique for Databases Based on Clustering", Proceedings of the Database and Expert Systems Applications, 15th International Workshop on (DEXA'04). [9] J. Manfuekphan S. Vanichayobon and S. dararat (2006), "Scatter Bitmap Index Optimization by using Association Rules", The Joint Conference Computer Science and Software Engineering (JCSSE 2006), Bangkok, Thailand. [10] W. Weahama, S. Vanichayobon, and J. Manfuekphan (2009), "Using Data Clustering to Optimize Scatter Bitmap Index for Membership Queries", International Conference on Computer

7 and Automation Engineering (ICCAE 2009), Bangkok, Thailand. [11] N. Wattanakitrungroj, S. Vanichayabon (2006), "Dual Bitmap Index: Space Time Efficient Bitmap Index for Equality and Membership Queries", International Symposium on Communications and Information Technologies (ISCIT'06), pp , [12] Weahama, W. (2011). Strategies to Improve Query Processing Time in Searching Membership Queries of Virtual Classroom by Using DBIC Technical Report, Faculty of Education, Prince of Songkla University, Thailand.