Density Estimation Over Data Stream

Transcription

1 Density Estimation Over Data Stream Aoying Zou Dept. of Computer Science, Fudan University 22 Handan Rd. Sangai, 2433, P.R. Cina Ziyuan Cai Dept. of Computer Science, Fudan University 22 Handan Rd. Sangai, 2433, P.R. Cina Li Wei Dept. of Computer Science, Fudan University 22 Handan Rd. Sangai, 2433, P.R. Cina ABSTRACT Density estimation is an important but costly operation for applications tat need to know te distribution of a data set. Moreover, wen te data comes as a stream, traditional density estimation metods cannot cope wit it efficiently. In tis paper, we examined te problem of computing density function over data streams and developed a novel metod to solve it. A new concept M-Kernel is used in our algoritm, and it is of te following caracteristics: () te running time is in linear wit te data size, (2) it can keep te wole computing in limited size of memory, (3) its accuracy is comparable to te traditional metods, (4) a useable density model could be available at any time during te processing, (5) it is flexible and can suit wit different stream models. Analytical and experimental results sowed te efficiency of te proposed algoritm. Keywords Data Stream, Kernel density estimation. INTRODUCTION Recently, it as been found tat te tecnique of processing data streams is very important in a wide range of scientific and commercial applications. A data stream is suc a model tat a large volume of data is arriving continuously and it is eiter unnecessary or impractical to store te data in some forms. For example, transaction of banks, call records of telecommunications company, it logs of web server are all tis kinds of data. In tese applications, decisions sould be made as soon as various events (data) being received. It is not likely tat processing accumulated data periodically by batces is allowed. Moreover, data streams are also regarded as a model to access large data sets stored in secondary memory were performance requirements necessitate access wit linear scans. In most cases, comparing (Produces te permission block, copyrigt information and page numbering). For use wit ACM PROC ARTICLE- SP.CLS V2.. Supported by ACM. to te size of total data in auxiliary storage, te size of main memory is suc small tat every time only a small portion of data can be load in memory to process, and under te time restrict only one scan is allowed to te data. Terefore, despite all our efforts in scaling up data mining algoritms, tey still cannot andle suc data efficiently. How to apply an efficient way to organize and extract useful information from te data stream is a problem met by researcers from almost all fields. Toug tere are many algoritms for data mining, tey are not designed for data stream. To process te ig-volume, open-ended data streams, a metod sould meet some stringent criteria. In [6], Domingos presents a series of designed criteria, wic are summarized as following:. Te time needed by te algoritm to process eac data record in te stream must be small and constant; oterwise, it is impossible for te algoritm to catc up te pace of te data. 2. Regardless of te number of records te algoritm as seen, te amount of main memory used must be fixed. 3. It must be a one-pass algoritm, since in most applications, eiter te data is still not available, or tere is no time to revisit old data. 4. It must ave te ability to make a usable model available at any time, since we may never meet te end of te stream. 5. Te model must be up-to-date at any point in time, tat is to say, it must keep up wit te canges of te data. Te first two criteria are most important and ard to acieve. Altoug many works ave been done on scalable data mining algoritms, most algoritms still require an increasing main memory in proportion to te data size, and teir computation complexity is muc iger tan linear wit te data size. So tey are not equipped to cope wit data stream, because tey will exaust all available main memory or fall beind te data, some time or oter. Recent proposed tecniques include clustering algoritms wen te objects arrive as a stream [8, 4], computing de-

2 cision tree classifiers wen te classification examples arrive as a stream [5,, 7], as well as approximate computing medians and quantiles in one pass [2, 3, 2]. Anoter common but useful tecnique used on data stream is Density estimation. Given a sequence of independent random variables identically drawn from a specific distribution, te density estimation problem is to construct a density function of te distribution based on te data drawn from it. Density estimation is a very important problem in numerical analysis, data mining and many scientific researc fields [9, 5]. By knowing te density distribution of a data set, we can ave an idea of te distribution in te data set. Moreover, based on te knowledge of density distribution, we can find te dense or sparse area in te data set quickly; and medians and oter quantiles can be easily calculated. So in many data mining applications, suc as density-biased sampling, density-based clustering, density estimation is an inevitable step to tem [3, ]. Kernel density estimation is a widely studied nonparametric density estimation metod [4]. But it becomes computationally expensive wen involving large data sets. Zang et.al. provide a metod to obtain a fast kernel estimation of te density in very large data sets by construct a CF-tree on te data [6]. Calculating density function over data stream as many practical applications. A straigtforward one is tat it can be used to verify weter two or more data streams are drawn from te same distribution.. Our Contribution Te contributions of te paper can be summarized as:. To te best of our knowledge, it is te earliest work of density estimate over data stream. 2. We bring forward a new concept of M-Kernel, wic solves te incompatibility of limited main memory and continuously coming data. And an algoritm is proposed based on it to solve density estimation problem over data stream. 3. Analytical and experimental results prove te effectiveness and efficiency of our algoritm. It is a onepass algoritm and needs only a fixed-size main memory. Te running time is in linear wit te size of te data stream. Meanwile, te algoritm as an acceptable error rate wen compared wit traditional kernel density estimation. Anoter advantage is tat it can maintain a useable model at any point in time. 4. Wat we provide is a basic notion to andle data stream, wic can be extended and integrated to many data mining applications suc as density-biased sampling..2 Paper Organization Te organization of te rest of te paper is as follows: in te next section (Section 2) we formally define te problem. A new density estimation metod tat can andle data stream is presented in section 3. Section 4 includes te experimental results. And we conclude in section x Figure : Construction of a fixed weigt kernel density estimate(solid curve). Te normal kernels are sown as te dotted lines. 2. PROBLEM DESCRIPTION In tis section, we briefly review te kernel density estimation metod and sow te problems wen using it to deal wit stream data. Given a sequence of independent random variables x, x 2,... identically distributed wit density f, te density estimation problem is to construct a sequence of estimators ˆf n (x n ; x) of f(x) based on te sample (x,..., x n ) x n. Te kernel metod is a widely studied nonparametric density estimation metod. Te equation of it based on n data points is defined as: ˆf n(x) = n nx i= K( x X i ) = n nx i= K (x X i ) () Note tat K (t) = K( t) for >. Clearly, ˆf(x) is nonnegative, integrates to one, and is a density function. In Figure, te construction of suc estimates is demonstrated. Te dased lines denote te individual standard normal saped kernels, and te solid line te resulting estimate. Kernel density estimation can be tougt of as being obtained by placing a bump at eac point and ten summing te eigt of eac bump at eac point on te X- axis. Te sape of te bump is defined by a kernel function, K(x), wic is taken to be a unimodal, symmetric, nonnegative function tat centers at zero and integrates to one. Te spread of te bump is determined by a window or bandwidt,, wic controls te degree of smooting of te estimation. Kernel density estimation as many desirable properties [6]. It is simple, no curse of dimension, no need to know te data range in advance, and te estimate is asymptotically unbiased, consistent in a mean-square sense, and uniformly consistent in probability. For low to medium data sizes, kernel estimation is a good practical coice. However if te kernel density estimation is applied to very large data sets, it becomes computationally expensive and space intensive. Because tere are n distinct bumps, or kernel functions in ˆf(x), generally speaking, it needs O(n) space to store te n kernel functions. If we want to calculate te density value at a specific point α, it needs

3 to scan all n kernel functions to compute ˆf(α). And te most great drawback of te metod is tat it must get te n data before begin to estimate te density function. Tat is to say, te kernel density estimation can only deal wit static data sets, it cannot andle data stream. 3. DEAL WITH DATA STREAM In tis section, we first give two observations on classical kernel density estimation metod, and ten present a new density estimation metod, wic can calculate density function over data stream efficiently. Te algoritm seems like classical kernel density estimation, but tere is one great difference between tem, tat is, te efficiency. It only uses a fix-sized main memory, wic is irrespective of te total number of records in te stream, all troug te processing. And te time complexity is in linear wit te size of te data stream. 3. Incremental Density Estimation Traditionally, ˆf n(x)(n =, 2,...) is calculated independently from random variables X, X 2,.... Here, we want to calculate ˆf t+ (x) based on ˆf t(x) and te random variable X t+. Observation. Te classical kernel density estimation can be calculated in an incremental manner. If te main memory is unlimited, ten te density function can be calculated in an incremental way. Tat is to say, te data stream can be scanned only once. Naively, we can estimate te density function in suc an incremental manner. ˆf t+ (x) = = = = X t+ t + t + i= K ( x Xi ) tx i= K ( x X i ) + K ( x X t+ ) t t + ˆf t(x) + K ( x X t+ ) t t + ˆf t(x) + t + K ( x Xt+ ) (2) Obviously, in equation (2), ˆft+ (x) can be calculated from ˆf t(x) and X t+. So wen a new data record comes, only some predetermined calculations are needed, wile we need to keep t kernels in main memory wen processing te t + data record. 3.2 Kernel Merging Wen te amount of data increases, wat we need to record expands accordingly and it will be difficult to keep tem in te limited main memory wen te data is tremendous. Furtermore, it is also computationally expensive since totally tere are still n terms in te ˆf n(x), and for te density on a specific point α, it still needs to scan all n kernel functions to compute ˆf(α).! How to solve te conflict of increasing data and limited memory? We observe tat wit some acceptable information lost we can summary te kernel functions, and keep tem all in te main memory. Traditionally, eac data is represented by a kernel function. If te sample size from te distribution is n ten tere are n kernels to be calculated and stored in ˆf n (x). Now we try to reduce it and fix te amount of kernels wen calculating ˆf n (x). A weigt value is given to te kernel function, so it can represent more tan one data. Ten te amount of kernels can be smaller tan te amount of data points. We call kernel represent only one data simple kernel and kernel represent more tan one data M-Kernel. Observation 2. Te sum of two kernel functions can be approximated by a large kernel function (M-Kernel) wose weigt equates to te amount of data it will represented. Based on te observation 2, two simple kernels (i and j) can be represented by one M-Kernel (m) wit weigt two. Tat is, K( x X i i i ) + j K( x X j j ) = 2 m K( x X m ) (3) m Te left of te equation stands for te sum of two kernels placed at point X i and X j wit bandwidt i and j. At most time, we cannot find a kernel wit parameter X m and m to make te two side of Equation 3 absolutely equivalent. But analysis and experiments will sow te difference between tem is very small. More generally, two M-Kernels (s and t) can also be represented by a single M-Kernel (u). ps K( x X s )+ p t K( x X t s s t ) = p s + p t t u K( x X u ) (4) u Wit Equations 3 and 4, after (n m) times of merging, we get m M-kernel wit parameter X i, i, p i i = (... m) from X,..., X n. Ten te estimator can be written as were ˆf n(x) = n mx p j j= j mx j= K( x X j j ) (5) p j = n (6) Figure 2 illustrates te kernel merging operation. Kernel and kernel 2 are two different kernels (simple kernel or M- kernel), and we want to find a kernel (M-kernel) to represent te two kernel functions. Intuitively, it must be very close to te dased-line in te figure, wic is te sum of te two kernel functions. In Figure 3, we merge some kernel functions into M-kernel (te dotted line). Compared to Figure, tere are only four M-Kernel left after merging, so te merging operation

4 can significantly reduce te memory used by te algoritm. However, te final density curve (te solid line) is very close to te one syntesized by ten kernel functions. Te difference between te two curves is te accuracy lost wen we do te merging, wic we call it merging cost. Our experiments will sow tat te difference between te two density curves is very small, and can be controlled by some parameters. sum of kernel and kernel 2 kernel after merge kernel kernel 2 Figure 2: Kernel merging operation 3.3 Applied to Data Stream Based on te two observations we proposed above, by using te incremental and merging tecniques, now we can extend te classical kernel density estimation to cope wit data stream. We keep a fixed-size kernel function list in te main memory. Every time a new data record comes, te density function, wic we put on te data point, is added to te kernel function list. Wen te list is full, we merge some kernel functions aving te lowest merging cost in te list to empty some entries. We can see tat te memory used by te algoritm is fixed, and it only requires sort constant time to process eac record in te stream. At any time of te processing, we can syntesize te kernel function by summing all te kernel functions in te list. Because te amount of kernel functions in te list is fixed, te syntesization can also be done in constant time. 2 x Algoritm Te detailed algoritm is sown in Figure 4. In te course of te processing, an m-entry buffer is maintained in main memory. It contains an array of elements wit te format µ, σ, ρ, m cost. Te element denotes tat kernel K(x) ave parameters µ, bandwidt σ, and weigt ρ, wic means te kernel function is like ρ x µ K( ), and m cost is te merging σ σ cost wit next kernel in te buffer. A euristic metod is applied ere, tat is, kernels wit small distance in value µ will ave smaller merging cost tan kernels wit large distance in value µ. So te buffer is sorted by µ, and we just need to calculate te merging cost of neigboring kernels in te buffer since it will smaller tan tat of te kernels wic are not neigboring in te buffer. Wen a new data comes, an element will be generated and inserted into te buffer. If te buffer is full, an element wit te smallest m cost will be picked out. Suppose tat it is te it element in te buffer, ten te it element and (i + )t element will be merged into a new element. Ten a buffer entry will be freed. And te procedure will go on to deal wit next data in te stream Figure 3: Construction of a distinct weigt kernel density estimate (solid curve). Te M-kernel are sown as te dotted lines Line 2 to Line 7 is te main loop of te algoritm. Every time a new data comes, we generate an element for it and insert it into te buffer (Line4 5). Ten we sort te buffer and count te merging value of te new data (Line6 7). If te buffer is full, we need to find a pair of kernels wit te lowest merging cost and merge tem (Line9 4). Line 7 and line 4 are very important in te algoritm, for tey take carge of maintaining te list of merging cost. At most time, only one element in te buffer canged, so at most two merging costs (in -dimensional) need to be updated. 3.5 Algoritm Analysis It is easy to see tat during te procedure, te buffer size is kept constant. So te space complexity of te algoritm is O(m), were m is irrespective wit te size of te data

5 stream. And for every record processed, we only ave to calculate te merging cost wit its neigboring kernel functions, and do te merging if te list is full. So, we can see tat te computational cost of eac record is witin a constant value. Tat is to say, te total calculating cost is in linear wit te size of te data stream. We ave detail analysis about it in our experiments. Algoritm DensityOverStream () input: Data stream (x, x 2,...) output: Density function ˆf(x) procedure. Initiate a buffer wit m entries; 2. Wile te stream is not end 3. Read a new data point x i from te stream; 4. Fill te element µ, σ, ρ, m cost wit x i,,, max cost 5. Insert te element into te buffer 6. Sort te buffer by µ 7. Update te column m cost. 8. If te buffer is full 9. Find te entry wit te smallest m cost value in te buffer;. Merge te two entries into one. Insert te merged entry into te buffer 2. Free an entry 3. Sort te buffer by µ 4. Update te column m cost. 5. End If 6. Output te buffer and syntesize te density function ˆf(x) if needed 7. End Wile endprocedure Figure 4: Stream algoritm We use some merging metods to reduce te memory requirement of te algoritm, but te cost is te calculation results would definitely ave variances from te previous results. Te cost of a single merge operation, wic is sown in Equation 4 is Z ( p s K( x X s )+ p t s s K( x X t t ) p s + pt t u and te total cost of te estimator (Equation 5) is Z K( x X u )) 2 dx u (7) ( ˆf n (x) ˆf n(x)) 2 dx (8) were ˆf n(x) is te traditional estimation result and ˆf n(x) is te result. 3.6 Discussion Te algoritm can run in two models: te complete model and te window model. In complete model, all data in te stream are of equal interest. Wile in window model, we take more empasis on te recent data tan te old data. Wit a fadeout function to decrease te contribution of te old data to te result, te complete model can be easily extended to window model. Te function can be implemented differently according to users requirement. For example, one fadeout function, wic decreases te weigt of old data gradually, may work in following way. Suppose at one time, tere are m kernels in te buffer. Ten a new data comes, and we decrease te weigt of te original m kernels by a small proportion, say.%, and assign corresponding weigt to te new coming data to keep te sum of te weigt of all kernels equal to te amount of te data. In tis way, new data will be drawn more empasis tan te old data. Since we are discussing ow to andle data stream efficiently in tis paper, we focus on complete model and omit te details of fadeout functions ere. Te algoritm can cope wit multi-dimensional data as well as -dimensional data. In traditional kernel density estimation, multivariate estimator as te same form wit univariate estimator. Because our algoritm is based on te result of traditional kernel estimate, so it can be easily generalized to any dimension. 4. EXPERIMENTS Te algoritm is implemented in C++. Te program runs on a PC workstation wit.4ghz Pentium IV processor and 256Mb RAM using Windows 2 Server.

6 Table : Test Data sets Distribution Density function Distribution I f (x) = N(2, 2) Distribution II f 2 (x) =.4N(2, ) +.3N(8, 2) +.2N(9, 5) +.N(, 5) P Distribution III f 3(x) =.N(i 3, 5) i= 2 x Table 2: Test Data Streams Data Streams Size Distribution Type Data Stream, Distribution I Random Data Stream 2, Distribution II Random Data Stream 3, Distribution III Random Data Stream 4, Distribution II Ordered We construct four large syntetic data sets and carry out a series of experiments on tem to test te performance of our algoritm. Te goal of te experiments is to evaluate te accuracy of te proposed algoritm, and to prove tat te time cost by per record is witin a small constant value, wic means te algoritm can output te results in linear time. We also do experiments to sow te relationsip between te buffer size and te precision of te result. Additionally, experiments sow te ability of te algoritm to output a usable model at any point of te processing..6 x Figure 5: Distribution I 4. Experimental Setup Some syntesized data streams are generated to test our algoritm. First, we coose tree distribution functions, sown in Table. Te distribution of tese tree density functions are illustrated in Figures 5, 6 and 7, respectively. Next, data streams are drawn randomly and independently from tese distributions. Table 2 sows tat. Data Stream -3 are drawn randomly from distribution I-III, accordingly. Data Stream 4 is also drawn from distribution II, but it is ordered by te value. Te classical kernel density estimation can be regarded as te best nonparametric density estimation algoritm. So we take te result produced by kernel density estimation as te base line to measure te accuracy of our algoritm. Te difference between te outputs of te two algoritms can be regarded as te error of our algoritm. Te results of kernel density estimation (KDE) tat will be used as te criteria are calculated by Bearda s program [] implemented in MATLAB. Te accuracy of our algoritm can be evaluated by absolute error= R ˆf(x) f(x) dx, were ˆf(x) is te kernel density estimation result on static data set and f(x) is te result of our algoritm over te corresponding data stream. 4.2 Experimental Results Te experiments are divided into tree parts: Firstly, test of te running time on data streams of different sizes reveals tat te running time of our algoritm is in linear wit te data size x Figure 6: Distribution II Figure 7: Distribution III

7 Table 3: Running Time at different Stream Size Stream Size Running Time (sec) Stream Stream 2 Stream 3 Stream 4, , , , , , , , , , Running Time (sec) dataset dataset2 dataset3 dataset4 Table 4: Absolute error & Running Time at different Buffer Size Buffer Size Absolute error Running Time(sec) e e e e e , e , e , e , e , e , e , e , e , e , e , e Number of Points x 4 Figure 8: Running Time vs Stream Size Accuracy Running Time (sec) Secondly, studying te effect of te buffer size on te precision of te result, tis sows tat te error rates decrease wit te increasing of te buffer size. And we can make te conclusion tat, given a large enoug buffer size, te accuracy of our algoritm will be rater ig. Tirdly, we demonstrate te ability of our algoritm to output te result at any point of te processing. Running Time: Te crucial caracter of stream algoritm is tat it must keep up wit te continuously coming data stream, tat is to say, te running time must be in linear wit te data size. We test data streams of different sizes and record corresponding running time of te algoritm, wic is sown in Table 3. Figure 8 sows ow te running times scale up on te four data sets. We can see tat as te size of te data sets increases, te running time increases linearly. Tis figure also exibits tat one curve as a distinctly low slope compared wit te oter tree curves. We notice tat tis curve represents te data stream in wic data increases strictly. Apparently, a sorted data set will consume less time wile being processed, because te sorting procedure in te algoritm will run faster on a sorted data. Buffer Size:Te buffer size is an important parameter in tis algoritm; it as large infection on te accuracy of estimation result. In tis experiment, on data stream 2, te buffer size used by te algoritm is ranged from 5 to 2, Buffer Size Figure 9: Buffer size vs accuracy and running time to ceck te cange in accuracy rate. Table 4 sows te result and in Figure 9 te solid line denotes te error rate wile dotted line te running time. It can know from te figure tat wen te buffer size increases, te error rate decreases, wile te running time keeps almost consistent. And obviously, if te buffer size equals to te data size, our algoritm is generalized to kernel density estimation, te error rate drop to zero. Incremental: One of te main caracteristics of stream data is tat it may be infinitive. So te ability to output result at any point of te processing is of great importance to data stream algoritms. To test tis capability, we run our program on data stream 2 and 4. Te two data stream are drawn from te same distribution, wile stream 4 as been sorted, stream 2 as not. Te two data streams bot ave k points of data and we output te results after every k points of data are processed. For eac data steam, we get small figures, sown in Figures and respectively. Te dotted curves represent te density estimation made over te wole data set and te solid curves represent te density estimation got in te midst of processing. In bot figures, we can see tat te medial results accord wit te final result, but tey approac it in different ways. Since 2

8 data stream 2 is not sorted, te data points are distributed randomly all troug te data set. We can get a roug outline of te final density curve at any output point. Te more data we get, te closer te medial result is to te final result, wile it is not te case to te sorted data stream 4. In te middle of te processing, only part of te data is available. And because it as been sorted, it concentrates on one area of te data set. So te density curve we get at eac interval is only part of te final curve. But te part is very precise compared wit te final result, since nearly all data needed to generate te part as been obtained. 5. CONCLUSIONS AND FUTURE WORK Density estimation over data stream as become more and more interesting in database community, wile classical kernel metod cannot andle data stream effectively. In tis paper, we bring forward a new concept of M-Kernel and propose a novel density estimation algoritm based on it. Te former guarantees tat te latter only needs a fixedsize memory, regardless te amount of data in te stream. Analysis and experiments prove tat our algoritm is capable to process data stream and build up its density function at te speed data arrival. Te result is comparable to tat of te kernel density estimation. And tis metod is superior tan te existent metods for it can output a useable model at any time of te processing. In te future, we will integrate tis algoritm to more data mining applications to testify its efficiency. Anoter improvement we are going to make is to incorporate te bandwidtdecision tecnique into our algoritm so tat te bandwidt can be self-adapted to data streams. 6. ADDITIONAL AUTHORS Additional autors: Weining Qian (Dept. of Computer Science, Fudan University, wnqian@fudan.edu.cn). 7. REFERENCES [] C. C. Bearda and M. J. Baxter. Matlab routines for kernel density estimation and te grapical representation of arcaeological data. Tecnical report, Department of Matematics, Statistics and Operational Researc, Te Nottingam Trent University, ttp://science.ntu.ac.uk/msor/ccb/densest.tml, 995. [2] F. Cen, D. Lambert, and J. C. Pineiro. Incremental quantile estimation for massive tracking. In Proceedings of International Conference on Knowledge Discovery and Data Mining (KDD), pages , 2. [3] Y. Ceng. Mean sift, mide seeking, and clustering. IEEE Transactions on Pattern Analysis and Macine Intelligence, 7(8):79 799, August 995. [4] D. Comaniciu and P. Meer. Distribution free decomposition of multivariate data. In Int l Worksop on Statistical Tecniques in Pattern Recognition, 998. [5] P. Domingos and G. Hulten. Mining ig-speed data streams. In Proceedings of International Conference on Knowledge Discovery and Data Mining (KDD), pages 7 8, 2. [6] P. Domingos and G. Hulten. Catcing up wit te data: Researc issues in mining data streams. In Worksop on Researc Issues in Data Mining and Knowledge Discovery, 2. [7] P. Domingos and G. Hulten. Learning from infinite data in finite time. In Advances in Neural Information Processing Systems, 22. [8] S. Gua, N. Misra, R. Motwani, and L. O Callagan. Clustering data streams. In Proceedings of Symposium on Foundations of Computer Science (FOCS), pages , 2. [9] D. Gunopulos, G. Kollios, V. J. Tsotras, and C. Domeniconi. Approximating multi-dimensional aggregate range queries over real attributes. In Proceedings of ACM SIGMOD International Conference on Management of Data, pages , 2. [] G. Hulten, L. Spencer, and P. Domingos. Mining time-canging data streams. In Proceedings of International Conference on Knowledge Discovery and Data Mining (KDD), pages 97 6, 2. [] G. Kollios, D. Gunopoulos, N. Koudas, and S. Berctold. An efficient approximation sceme for data mining tasks. In Proceedings of International Conference on Data Engineering (ICDE), pages , 2. [2] G. S. Manku, S. Rajagopalan, and B. G. Lindsay. Approximate medians and oter quantiles in one pass wit limited memory. In Proceedings of ACM SIGMOD International Conference on Management of Data, pages , 998. [3] G. S. Manku, S. Rajagopalan, and B. G. Lindsay. Random sampling tecniques for space efficient online computation of order statistics of large datasets. In Proceedings of ACM SIGMOD International Conference on Management of Data, pages , 999. [4] L. O Callagan, N. Misra, A. Meyerson, and S. Gua. Streaming-data algoritms for ig-quality clustering. In Proceedings of International Conference on Data Engineering (ICDE), 22. [5] S. R. Sain. Adaptive Kernel Density Estimation. PD tesis, Rice University, August 994. [6] T. Zang, R. Ramakrisnan, and M. Livny. Fast density estimation using cf-kernel for very large databases. In Proceedings of International Conference on Knowledge Discovery and Data Mining (KDD), pages 32 36, 999.

9 2 x 3 2 x 3 2 x 3 2 x 3 2 x x 3 2 x 3 2 x 3 2 x 3 2 x Figure : Te medial outputs of data stream 2 2 x 3 2 x 3 2 x 3 2 x 3 2 x x 3 2 x 3 2 x 3 2 x 3 2 x Figure : Te medial outputs of data stream 4