Fractal Based Approach for Indexing and Querying Heterogeneous Data Streams
|
|
- Ellen Hoover
- 5 years ago
- Views:
Transcription
1 Fractal Based Approach for Indexing and Querying Heterogeneous Data Streams G. Netaji & MHM. Krishna Prasad Dept. of IT, JNTUK UCEV, Vizianagaram, A.P., , India Abstract Efficient handling of data stream is become mandatory in various domains such as business, finance, sensor networks etc. Managing these data streams poses challenges that are distinct from those addressed by traditional database management systems. The enormous increase of cellular phones, GPS-like devices, and RFIDs results in highly dynamic environments where objects as well as queries are continuously moving. This paper presents a continuous query processor designed specifically for highly dynamic environments. This paper presents a data stream indexing system that satisfies the requirements in evolving data streams. Finally, applying the fractal based indexing scheme that enable the system for fast indexing and querying. This paper also address the issue of memory based and disk based fractal management to support skew distributions of data stream. Keywords Data stream, Indexing, Querying, Self similarity, Fractal Dimension, Fractal Tree. The existing query processing techniques focus on solving special cases of continuous spatio-temporal queries are valid only for moving queries on stationary objects, are valid only for stationary range queries. Thus, a system that manages infinite heterogeneous data streams must satisfy the following requirements 1. Mechanism for rate control must be in place to deal with data streams that are nearly always too fast for any indexing system. 2. There must be capacity control because the data streams are infinite while the storage space is finite. I. INTRODUCTION Most of spatio-temporal queries are continuous in nature. Unlike snapshot queries that are evaluated only once, continuous queries require continuous evaluation as the query result becomes invalid with the change of information. The Existing algorithms for continuous spatiotemporal queries aim to optimize the time interval between each two instances of the snapshot queries with the large number of continuous queries, reevaluating a continuous spatio-temporal query, even with large time intervals, poses a redundant processing for the locationaware servers. Those focus on evaluating only one spatio-temporal query In a typical location-aware server, there is a huge number of concurrently outstanding continuous spatiotemporal queries. Handling each query as an individual entity dramatically degrades the performance of the location-aware server. Fig1: B-Tree Indexing 3. The structure of the data varies in time (i.e., heterogeneity of data streams), therefore adaptive indexing methods must be developed. 4. Data should be efficiently stored. 5. Volume can be huge, therefore everything must be append-only; both indexing and query processing must only use forward-only file access. 6. Users can query and filter the index streams efficiently. 84
2 II. RELATED WORK Shifted-Wavelet Tree (SWT) is a wavelet-based summary structure proposed for detecting bursts of events on a stream of data [1]. For a given set of query windows SWT maintains moving aggregates using a wavelet tree for incremental computation. A window is monitored by the lowest level that satisfies windows condition. Therefore, associated with each level, there is a threshold equal to the smallest of the thresholds of windows monitored by that level. Whenever the moving sum at some level exceeds the level threshold all query windows associated with this level are checked using a brute force approach. MR-Index addresses variable length queries over time series data [2]. Wavelets are used to extract features from a time series at multiple resolutions. At each resolution, a set of feature vectors are combined into an MBR and stored sequentially in the order they are computed. A given query is decomposed into multiple sub-queries such that each sub-query has resolution corresponding to a resolution at the index. A given set of candidate MBRs are refined using each query as a filter to prune out non-potential candidates. Versions of piecewise constant approximation are proposed for time series similarity matching. Specifically, adaptive piecewise constant approximation (APCA) represents data regions of great fluctuations with several short segments, while data regions of less fluctuations are represented with fewer, long segments [3]. An extension of this approximation allows error specification for each point in time [4]. The resulting approach can approximate data with fidelity proportional to its age. GeneralMatch, a refreshingly new idea in similarity matching, divides the data sequences into disjoint windows, and the query sequence into sliding windows [5]. This approach is the dual of the conventional approaches, i.e., dividing the data sequence into sliding windows, and the query sequence into disjoint windows. The overall framework is based on answering pattern queries using a singleresolution index built on a specific choice of window size. The allowed window size depends on the minimum query length, which has to be provided a-priori before the index construction. Methods based on multi-variate linear regression are considered for analyzing co-evolving time sequences [6]. For a given stream S, its current value (dependent variable) is expressed as a linear combination of values of the same and other streams (independent variables) under sliding window model. Given v independent variables and a dependent variable y with N samples each, the model identifies the best b independent variables in order to compute the current value of the dependent variable in O(Nbv 2 ) time. Stat stream is a state of the art system proposed for monitoring a large number of streams in real time [7]. It subdivides the history of a stream into a fixed number of basic windows and maintains DFT coefficients for each basic window. This allows a batch update of DFT coefficients over the entire history. It superimposes an orthogonal regular grid on the feature space, and partitions the space into cells of diameter r, the correlation threshold. Each stream is mapped to a number of cells (exactly how many depends on the "lag time") in the feature space based on a subset of its DFT coefficients. It uses proximity in this feature space to report correlations [8]. III. FRACTALS Fractals are of rough or fragmented geometric shape that can be subdivided in parts, each of which is (at least approximately) a reduced copy of the whole. They are crinkly objects that defy conventional measures, such as length and are most often characterized by their fractal dimension They are mathematical sets with a high degree of geometrical complexity that can model many natural phenomena. Almost all natural objects can be observed as fractals (coastlines, trees, mountains, and clouds). Their fractal dimension strictly exceeds topological dimension Fractal dimension The number, very often non-integer, often the only one measure of fractals. It measures the degree of fractal boundary fragmentation or irregularity over multiple scales. It determines how fractal differs from Euclidean objects (point, line, plane, circle etc) Self-similarity/ Semi-self similarity Fractal is strictly self-similar if it can be expressed as a union of sets, each of which is an exactly reduced copy (is geometrically similar to) of the full set. The most fractal looking in nature do not display this precise form Natural objects are not union of exact reduced copies of whole. A magnified view of one part will not precisely reproduce the whole object, but it will have the same qualitative appearance. This property is called statistical self-similarity or semi-self-similarity How Fractal Tree Databases Work Insertion bottlenecks lie at the heart of database and file-system innovations, best practices, and system workarounds. Most databases and file systems are based on B-tree data structures, and suffer from the performance cliffs and unpredictable run times of B- trees. In this talk, we introduce the Fractal Tree data structure and explain how it works and how it provides 85
3 dramatically improved performance in both theory and in practice. From a theoretical perspective, if B is the blocktransfer size, the B-tree performs O(log N/log B) block transfers per insert in the worst case. In contrast, the Fractal Tree structure performs O((log N)/B) memory transfers per insert, which translates to run-time improvements of two orders of magnitude. To relate that theory to practice, we present an algorithmic model for B-tree performance bottlenecks. We explain how the bottlenecks affect best practice and how database designers typically modify B-trees to try to mitigate the bottlenecks. Then we show how Fractal Tree structures can attain faster insertion rates, intuitively by transforming disk-seek bottlenecks into disk-bandwidth bottlenecks. Fractal Tree data structure Advantages A drop-in B-tree replacement supporting fast insertions for high entropy data. 100x faster inserts good data locality with no memory-specific parameterization Data can be indexed quickly on disk, even if the data arrival order is independent of data-query order. Thus, insertion performance is a currency that we can use to achieve faster query performance through indexing Fractal Trees are generally faster, easier to implement, and platform independent Fractal Trees scale with disk bandwidth not seek time Properties of a Simplified Fractal Tree log N arrays, one array for each power of two Each array is completely full or empty Each array is sorted Searching in a Simplified Fractal Tree Fractal Tree indexes can use 1/100th the power of B-trees Fractal Tree Indexes There are same operations in B-trees and fractal trees. The main difference is at performance. The Random input output is replaced with sequential Input /output so faster. So, Fractal Trees index data at closer disk bandwidth rates, not on the structure of the primary and secondary keys, and have some range of queries that stream data and close disk bandwidth rates, even as the database increases. Hence, more indexes shall be maintained without a loss of performance. Adding data to the indexes makes the B-trees stress on performance of B-trees, but performs well in Fractal Tree indexes. Scope of Fractal Tree Indexes Fractal Tree indexes will make good use of cores. Fractal Tree indexes ride the right technology trends In the future, all storage systems will use Fractal Tree indexes Index maintenance As new values stream in, new features are computed and inserted into the corresponding index structures while features that are out of history of interest are deleted to save space. Coefficients are computed at multiple resolutions starting from level 0 up to a configurable level J: at each level a sliding window is used to extract the appropriate features. Computation of features at higher levels is accelerated using the MBRs at lower levels. The MBRs belonging to a specific stream are threaded together in order to provide a sequential access to the summary information about the stream. This approach results in a constant retrieval time of the MBRs. The complete algorithm is shown in Algorithm 1. Require: B MBR at level j for stream S. w := W (the window size at the lowest resolution); tn denotes the current discrete time; for j:=0 to J do B := the current MBR at level j for stream S; If j = 0 then y := S[tn - w + 1 : tn]; normalize y if F = DWT, F := F(y); else find MBR that contains the feature for the subsequence S[tn - w + 1 : tn - y]; find MBR B, S, tn at contains the feature for the subsequence S[tn - y + 1 : tn]; if number of features in B < c (box capacity) then insert F in to B ; else insert BSi into index at level j; start a new MBR B insert Fj into B adjust the sliding window size to w := w * 2; end for Algorithm 1: Algorithm for Indexing 86
4 Features at a given level are maintained in a high dimensional index structure. The index combines information from all the streams, and provides a scalable the access medium for answering queries over multiple data streams. However, each MBR inserted into the index is specific to a single stream. The R*-Tree family of index structures are used for indexing MBRs at each level [9]. An R*-Tree, a variant of R-Tree [10], is a spatial access method, which splits feature space in hierarchically nested, possibly overlapping MBRs. In order to support frequent updates, the techniques for predicting MBR boundaries outlined in can be used to decrease the cost of index maintenance. Querying Streams Assume that the query window size is a multiple of W. An aggregate query with window size w and threshold T is answered by first partitioning the window into multiple sub-windows. For a given window of length bw, the partitioning corresponds to the ones in the binary representation of b. The current aggregate over a window of size w is computed using the subaggregates for sub-windows in the partitioning. Assume that W = 2 and c = 2. Consider a query window w = 26. The binary representation of b = 13 is 1101, and therefore the query is partitioned into three sub-windows wo = 2, w2 = 8, and w3 = 16. The current aggregate over a window of size w = 26 is approximated using the extents of MBRs that contain the corresponding subaggregates. The computation is approximate in the sense that the algorithm returns an interval F such that the upper coordinate F[2]is always greater than or equal to the true aggregate. If F[2]is larger than the threshold T, the most recent subsequence of length w is retrieved, and the true aggregate is computed. If this value exceeds the query threshold, an alarm is raised. The complete algorithm is shown in Algorithm 2. Given Stream S, Window w, Threshold T initialize t to tn, the current discrete time; partition w into n parts as wl, w2,..., wn; initialize aggregate T; for i := 1 to i := n do find the resolution level j such that Wi = W2 j ; MBR B contains the feature on S[t - wi + 1 : t]; merge sub-aggregate B to T := F(B, F); adjust offset to t := t - Wi for next sub-window; end for if T < F[2] then retrieve S[tn - w + 1 : t,n]; raise an alarm; Algorithm 2: Algorithm for Querying IV. EXPERIMENTAL WORK We used both real life and synthetic data streams for conducting experiments. Our intention is to reduce the time taken for both indexing and querying on data streams, to find the effect of the frequency and magnitude of the records on Accuracy and to analyze the performance of our approach. The tests are done on a unix machine with a 2400 MHz CPU and 2GB main memory. The both Real life data and synthetic data have been used in our experiments. Adult Data. We use real life Adult data.we sampled the data from the records with in one period with five million transactional records. The attributes include age, Age, Fnl-wgt, Education-num, Capital_Gain, Capital Loss, Workclass, Education, Martial_status, Occupation, Relationship, Race, Sex, Native_Country etc. Synthetic Data. We create synthetic data that have been used to simulate time-changing concepts to analyise the performance of indexing and Querying on data streams. Time analysis, we analyze the time taken for insertion into our Fractal tree and B-tree. And we found that our tree is performing good work than the B-tree. The below figure(i.e.fig2). Fig. 2: Random Inserts into Fractal Tree and B-tree Querying Performance, We analyzed the performance of our Tree with B.tree in Querying the data. The Performance of Fractal tree with B-Tree is shown in the below figure(i.e. Fig 3). 87
5 Fig. 3: Searches in Fractal Tree B- Tree V. CONCLUSION AND FUTURE SCOPE We conclude with performance results showing how a Fractal Tree storage engine can maintain rich indexes more efficiently than B-trees. Surprisingly, Fractal Tree structures seem to maintain their order-ofmagnitude competitive advantage over B-trees on SSDs as well as traditional rotating media. Here we used centralized monitoring so, there is scope in distributing the monitoring system and content distribution over networks. VI. REFERENCES [1] Y. Zhu and D. Shasha. Efficient elastic burst detection in data streams. In SIGKDD, pages ,2003. [2] T. Kahveci and A. Singh. Variable length queries for time series data. In ICDE, pages ,2001. [3] E. Keogh, K. Chakrabarti, S. Mehrotra, and M. Pazzani. Locally adaptive dimensionality reduction for indexing large time series databases. In SIGMOD, pages ,2001. [4] T. Palpanas, M. Vlachos, E. Keogh, D. Gunopulos, and W. Truppel. Online amnesic approximation of streaming time series. In ICDE, pages ,2004. [5] Y. Moon, K. Whang, and W. Han. General match: a subsequence matching method in timeseries databases based on generalized windows. In SIGMOD, pages ,2002. [6] B. Yi, N. Sidiropoulos, T. Johnson, H. Jagadish, C. Faloutsos, and A. Biliris. Online data mining for co-evolving time sequences. In ICDE,2000. [7] Y Zhu and D. Shasha. Statstream: Statistical monitoring of thousands of data streams in real time. In VZDB, pages ,2002. [8] J. Bentley, B. Weide, and A. Yao. Optimal expected time algorithms for closest point problems. In ACM Trans. on Math. Software, volume 6, pages ,1980. [9] N. Beckmann, H. Kriegel, R. Schneider, and B. Seeger. The R*-tree: An efficient and robust access method for points and rectangles. In SIGMOD, pages ,1990. [10] Guttman. R-trees: A dynamic index structure for spatial searching. In SIGMOD, pages 47-57,
A Unified Framework for Monitoring Data Streams in Real Time
A Unified Framework for Monitoring Data Streams in Real Time Ahmet Bulut and Ambuj K. Singh Department of Computer Science UC Santa Barbara Santa Barbara, CA 93106-5110 {bulut,ambuj}@cs.ucsb.edu Abstract
More informationThe Grid File: An Adaptable, Symmetric Multikey File Structure
The Grid File: An Adaptable, Symmetric Multikey File Structure Presentation: Saskia Nieckau Moderation: Hedi Buchner The Grid File: An Adaptable, Symmetric Multikey File Structure 1. Multikey Structures
More informationHOT asax: A Novel Adaptive Symbolic Representation for Time Series Discords Discovery
HOT asax: A Novel Adaptive Symbolic Representation for Time Series Discords Discovery Ninh D. Pham, Quang Loc Le, Tran Khanh Dang Faculty of Computer Science and Engineering, HCM University of Technology,
More informationA Study on Reverse Top-K Queries Using Monochromatic and Bichromatic Methods
A Study on Reverse Top-K Queries Using Monochromatic and Bichromatic Methods S.Anusuya 1, M.Balaganesh 2 P.G. Student, Department of Computer Science and Engineering, Sembodai Rukmani Varatharajan Engineering
More informationExperimental Evaluation of Spatial Indices with FESTIval
Experimental Evaluation of Spatial Indices with FESTIval Anderson Chaves Carniel 1, Ricardo Rodrigues Ciferri 2, Cristina Dutra de Aguiar Ciferri 1 1 Department of Computer Science University of São Paulo
More informationIndexing by Shape of Image Databases Based on Extended Grid Files
Indexing by Shape of Image Databases Based on Extended Grid Files Carlo Combi, Gian Luca Foresti, Massimo Franceschet, Angelo Montanari Department of Mathematics and ComputerScience, University of Udine
More informationDetermining Differences between Two Sets of Polygons
Determining Differences between Two Sets of Polygons MATEJ GOMBOŠI, BORUT ŽALIK Institute for Computer Science Faculty of Electrical Engineering and Computer Science, University of Maribor Smetanova 7,
More informationIntroduction to Indexing R-trees. Hong Kong University of Science and Technology
Introduction to Indexing R-trees Dimitris Papadias Hong Kong University of Science and Technology 1 Introduction to Indexing 1. Assume that you work in a government office, and you maintain the records
More informationSimilarity Search in Time Series Databases
Similarity Search in Time Series Databases Maria Kontaki Apostolos N. Papadopoulos Yannis Manolopoulos Data Enginering Lab, Department of Informatics, Aristotle University, 54124 Thessaloniki, Greece INTRODUCTION
More informationIntroduction to Spatial Database Systems
Introduction to Spatial Database Systems by Cyrus Shahabi from Ralf Hart Hartmut Guting s VLDB Journal v3, n4, October 1994 Data Structures & Algorithms 1. Implementation of spatial algebra in an integrated
More informationData Organization and Processing
Data Organization and Processing Spatial Join (NDBI007) David Hoksza http://siret.ms.mff.cuni.cz/hoksza Outline Spatial join basics Relational join Spatial join Spatial join definition (1) Given two sets
More informationX-tree. Daniel Keim a, Benjamin Bustos b, Stefan Berchtold c, and Hans-Peter Kriegel d. SYNONYMS Extended node tree
X-tree Daniel Keim a, Benjamin Bustos b, Stefan Berchtold c, and Hans-Peter Kriegel d a Department of Computer and Information Science, University of Konstanz b Department of Computer Science, University
More informationCOMP 465: Data Mining Still More on Clustering
3/4/015 Exercise COMP 465: Data Mining Still More on Clustering Slides Adapted From : Jiawei Han, Micheline Kamber & Jian Pei Data Mining: Concepts and Techniques, 3 rd ed. Describe each of the following
More informationA Real Time GIS Approximation Approach for Multiphase Spatial Query Processing Using Hierarchical-Partitioned-Indexing Technique
International Journal of Scientific Research in Computer Science, Engineering and Information Technology 2017 IJSRCSEIT Volume 2 Issue 6 ISSN : 2456-3307 A Real Time GIS Approximation Approach for Multiphase
More informationTemporal Range Exploration of Large Scale Multidimensional Time Series Data
Temporal Range Exploration of Large Scale Multidimensional Time Series Data Joseph JaJa Jusub Kim Institute for Advanced Computer Studies Department of Electrical and Computer Engineering University of
More informationOnline Pattern Recognition in Multivariate Data Streams using Unsupervised Learning
Online Pattern Recognition in Multivariate Data Streams using Unsupervised Learning Devina Desai ddevina1@csee.umbc.edu Tim Oates oates@csee.umbc.edu Vishal Shanbhag vshan1@csee.umbc.edu Machine Learning
More informationUsing Natural Clusters Information to Build Fuzzy Indexing Structure
Using Natural Clusters Information to Build Fuzzy Indexing Structure H.Y. Yue, I. King and K.S. Leung Department of Computer Science and Engineering The Chinese University of Hong Kong Shatin, New Territories,
More informationMultidimensional Data and Modelling - DBMS
Multidimensional Data and Modelling - DBMS 1 DBMS-centric approach Summary: l Spatial data is considered as another type of data beside conventional data in a DBMS. l Enabling advantages of DBMS (data
More informationSPATIOTEMPORAL INDEXING MECHANISM BASED ON SNAPSHOT-INCREMENT
SPATIOTEMPORAL INDEXING MECHANISM BASED ON SNAPSHOT-INCREMENT L. Lin a, Y. Z. Cai a, b, Z. Xu a a School of Resource and Environment Science,Wuhan university, Wuhan China 430079, lilin@telecarto.com b
More informationClustering Billions of Images with Large Scale Nearest Neighbor Search
Clustering Billions of Images with Large Scale Nearest Neighbor Search Ting Liu, Charles Rosenberg, Henry A. Rowley IEEE Workshop on Applications of Computer Vision February 2007 Presented by Dafna Bitton
More informationStriped Grid Files: An Alternative for Highdimensional
Striped Grid Files: An Alternative for Highdimensional Indexing Thanet Praneenararat 1, Vorapong Suppakitpaisarn 2, Sunchai Pitakchonlasap 1, and Jaruloj Chongstitvatana 1 Department of Mathematics 1,
More informationEffective Pattern Similarity Match for Multidimensional Sequence Data Sets
Effective Pattern Similarity Match for Multidimensional Sequence Data Sets Seo-Lyong Lee, * and Deo-Hwan Kim 2, ** School of Industrial and Information Engineering, Hanu University of Foreign Studies,
More informationApproximation Algorithms for Geometric Intersection Graphs
Approximation Algorithms for Geometric Intersection Graphs Subhas C. Nandy (nandysc@isical.ac.in) Advanced Computing and Microelectronics Unit Indian Statistical Institute Kolkata 700108, India. Outline
More informationSpatiotemporal Access to Moving Objects. Hao LIU, Xu GENG 17/04/2018
Spatiotemporal Access to Moving Objects Hao LIU, Xu GENG 17/04/2018 Contents Overview & applications Spatiotemporal queries Movingobjects modeling Sampled locations Linear function of time Indexing structure
More informationCMSC 754 Computational Geometry 1
CMSC 754 Computational Geometry 1 David M. Mount Department of Computer Science University of Maryland Fall 2005 1 Copyright, David M. Mount, 2005, Dept. of Computer Science, University of Maryland, College
More informationQuadrant-Based MBR-Tree Indexing Technique for Range Query Over HBase
Quadrant-Based MBR-Tree Indexing Technique for Range Query Over HBase Bumjoon Jo and Sungwon Jung (&) Department of Computer Science and Engineering, Sogang University, 35 Baekbeom-ro, Mapo-gu, Seoul 04107,
More informationCollision and Proximity Queries
Collision and Proximity Queries Dinesh Manocha (based on slides from Ming Lin) COMP790-058 Fall 2013 Geometric Proximity Queries l Given two object, how would you check: If they intersect with each other
More informationModule 8: Video Coding Basics Lecture 42: Sub-band coding, Second generation coding, 3D coding. The Lecture Contains: Performance Measures
The Lecture Contains: Performance Measures file:///d /...Ganesh%20Rana)/MY%20COURSE_Ganesh%20Rana/Prof.%20Sumana%20Gupta/FINAL%20DVSP/lecture%2042/42_1.htm[12/31/2015 11:57:52 AM] 3) Subband Coding It
More informationExtending Rectangle Join Algorithms for Rectilinear Polygons
Extending Rectangle Join Algorithms for Rectilinear Polygons Hongjun Zhu, Jianwen Su, and Oscar H. Ibarra University of California at Santa Barbara Abstract. Spatial joins are very important but costly
More informationCHAPTER 3 A FAST K-MODES CLUSTERING ALGORITHM TO WAREHOUSE VERY LARGE HETEROGENEOUS MEDICAL DATABASES
70 CHAPTER 3 A FAST K-MODES CLUSTERING ALGORITHM TO WAREHOUSE VERY LARGE HETEROGENEOUS MEDICAL DATABASES 3.1 INTRODUCTION In medical science, effective tools are essential to categorize and systematically
More informationMax-Count Aggregation Estimation for Moving Points
Max-Count Aggregation Estimation for Moving Points Yi Chen Peter Revesz Dept. of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln, NE 68588, USA Abstract Many interesting problems
More informationADS: The Adaptive Data Series Index
Noname manuscript No. (will be inserted by the editor) ADS: The Adaptive Data Series Index Kostas Zoumpatianos Stratos Idreos Themis Palpanas the date of receipt and acceptance should be inserted later
More informationConflict Serializable Scheduling Protocol for Clipping Indexing in Multidimensional Database
Conflict Serializable Scheduling Protocol for Clipping Indexing in Multidimensional Database S.Vidya Sagar Appaji, S.Vara Kishore Abstract: The project conflict serializable scheduling protocol for clipping
More informationAn Introduction to Spatial Databases
An Introduction to Spatial Databases R. H. Guting VLDB Journal v3, n4, October 1994 Speaker: Giovanni Conforti Outline: a rather old (but quite complete) survey on Spatial DBMS Introduction & definition
More informationIteration Reduction K Means Clustering Algorithm
Iteration Reduction K Means Clustering Algorithm Kedar Sawant 1 and Snehal Bhogan 2 1 Department of Computer Engineering, Agnel Institute of Technology and Design, Assagao, Goa 403507, India 2 Department
More informationMining Data Streams. Outline [Garofalakis, Gehrke & Rastogi 2002] Introduction. Summarization Methods. Clustering Data Streams
Mining Data Streams Outline [Garofalakis, Gehrke & Rastogi 2002] Introduction Summarization Methods Clustering Data Streams Data Stream Classification Temporal Models CMPT 843, SFU, Martin Ester, 1-06
More informationHierarchical Intelligent Cuttings: A Dynamic Multi-dimensional Packet Classification Algorithm
161 CHAPTER 5 Hierarchical Intelligent Cuttings: A Dynamic Multi-dimensional Packet Classification Algorithm 1 Introduction We saw in the previous chapter that real-life classifiers exhibit structure and
More informationDistributed k-nn Query Processing for Location Services
Distributed k-nn Query Processing for Location Services Jonghyeong Han 1, Joonwoo Lee 1, Seungyong Park 1, Jaeil Hwang 1, and Yunmook Nah 1 1 Department of Electronics and Computer Engineering, Dankook
More informationFoundations of Computing
Foundations of Computing Darmstadt University of Technology Dept. Computer Science Winter Term 2005 / 2006 Copyright c 2004 by Matthias Müller-Hannemann and Karsten Weihe All rights reserved http://www.algo.informatik.tu-darmstadt.de/
More informationJoint Entity Resolution
Joint Entity Resolution Steven Euijong Whang, Hector Garcia-Molina Computer Science Department, Stanford University 353 Serra Mall, Stanford, CA 94305, USA {swhang, hector}@cs.stanford.edu No Institute
More informationClustering and Visualisation of Data
Clustering and Visualisation of Data Hiroshi Shimodaira January-March 28 Cluster analysis aims to partition a data set into meaningful or useful groups, based on distances between data points. In some
More informationInternational Journal of Modern Trends in Engineering and Research e-issn No.: , Date: 2-4 July, 2015
International Journal of Modern Trends in Engineering and Research www.ijmter.com e-issn No.:2349-9745, Date: 2-4 July, 2015 Privacy Preservation Data Mining Using GSlicing Approach Mr. Ghanshyam P. Dhomse
More informationLeveraging Set Relations in Exact Set Similarity Join
Leveraging Set Relations in Exact Set Similarity Join Xubo Wang, Lu Qin, Xuemin Lin, Ying Zhang, and Lijun Chang University of New South Wales, Australia University of Technology Sydney, Australia {xwang,lxue,ljchang}@cse.unsw.edu.au,
More informationKeywords: clustering algorithms, unsupervised learning, cluster validity
Volume 6, Issue 1, January 2016 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Clustering Based
More information26 The closest pair problem
The closest pair problem 1 26 The closest pair problem Sweep algorithms solve many kinds of proximity problems efficiently. We present a simple sweep that solves the two-dimensional closest pair problem
More informationTrajStore: an Adaptive Storage System for Very Large Trajectory Data Sets
TrajStore: an Adaptive Storage System for Very Large Trajectory Data Sets Philippe Cudré-Mauroux Eugene Wu Samuel Madden Computer Science and Artificial Intelligence Laboratory Massachusetts Institute
More informationA Review on Cluster Based Approach in Data Mining
A Review on Cluster Based Approach in Data Mining M. Vijaya Maheswari PhD Research Scholar, Department of Computer Science Karpagam University Coimbatore, Tamilnadu,India Dr T. Christopher Assistant professor,
More informationBig Data Analytics CSCI 4030
High dim. data Graph data Infinite data Machine learning Apps Locality sensitive hashing PageRank, SimRank Filtering data streams SVM Recommen der systems Clustering Community Detection Queries on streams
More informationContinuous Query Processing in Spatio-temporal Databases
Continuous uery rocessing in Spatio-temporal Databases Mohamed F. Mokbel Department of Computer Sciences, urdue University mokbel@cs.purdue.edu Abstract. In this paper, we aim to develop a framework for
More informationGeneration of 3D Fractal Images for Mandelbrot and Julia Sets
178 Generation of 3D Fractal Images for Mandelbrot and Julia Sets Bulusu Rama #, Jibitesh Mishra * # Department of Computer Science and Engineering, MLR Institute of Technology Hyderabad, India 1 rama_bulusu@yahoo.com
More informationEvent Detection using Archived Smart House Sensor Data obtained using Symbolic Aggregate Approximation
Event Detection using Archived Smart House Sensor Data obtained using Symbolic Aggregate Approximation Ayaka ONISHI 1, and Chiemi WATANABE 2 1,2 Graduate School of Humanities and Sciences, Ochanomizu University,
More informationChallenges in Ubiquitous Data Mining
LIAAD-INESC Porto, University of Porto, Portugal jgama@fep.up.pt 1 2 Very-short-term Forecasting in Photovoltaic Systems 3 4 Problem Formulation: Network Data Model Querying Model Query = Q( n i=0 S i)
More informationCHAPTER 3 DIFFERENT DOMAINS OF WATERMARKING. domain. In spatial domain the watermark bits directly added to the pixels of the cover
38 CHAPTER 3 DIFFERENT DOMAINS OF WATERMARKING Digital image watermarking can be done in both spatial domain and transform domain. In spatial domain the watermark bits directly added to the pixels of the
More informationConstructing a G(N, p) Network
Random Graph Theory Dr. Natarajan Meghanathan Professor Department of Computer Science Jackson State University, Jackson, MS E-mail: natarajan.meghanathan@jsums.edu Introduction At first inspection, most
More informationCollaborative Rough Clustering
Collaborative Rough Clustering Sushmita Mitra, Haider Banka, and Witold Pedrycz Machine Intelligence Unit, Indian Statistical Institute, Kolkata, India {sushmita, hbanka r}@isical.ac.in Dept. of Electrical
More informationMining Frequent Itemsets from Data Streams with a Time- Sensitive Sliding Window
Mining Frequent Itemsets from Data Streams with a Time- Sensitive Sliding Window Chih-Hsiang Lin, Ding-Ying Chiu, Yi-Hung Wu Department of Computer Science National Tsing Hua University Arbee L.P. Chen
More informationLecturer 2: Spatial Concepts and Data Models
Lecturer 2: Spatial Concepts and Data Models 2.1 Introduction 2.2 Models of Spatial Information 2.3 Three-Step Database Design 2.4 Extending ER with Spatial Concepts 2.5 Summary Learning Objectives Learning
More informationReferences. Additional lecture notes for 2/18/02.
References Additional lecture notes for 2/18/02. I-COLLIDE: Interactive and Exact Collision Detection for Large-Scale Environments, by Cohen, Lin, Manocha & Ponamgi, Proc. of ACM Symposium on Interactive
More informationA Framework for Clustering Massive Text and Categorical Data Streams
A Framework for Clustering Massive Text and Categorical Data Streams Charu C. Aggarwal IBM T. J. Watson Research Center charu@us.ibm.com Philip S. Yu IBM T. J.Watson Research Center psyu@us.ibm.com Abstract
More informationImplementation Techniques
V Implementation Techniques 34 Efficient Evaluation of the Valid-Time Natural Join 35 Efficient Differential Timeslice Computation 36 R-Tree Based Indexing of Now-Relative Bitemporal Data 37 Light-Weight
More informationImplementation of Lifting-Based Two Dimensional Discrete Wavelet Transform on FPGA Using Pipeline Architecture
International Journal of Computer Trends and Technology (IJCTT) volume 5 number 5 Nov 2013 Implementation of Lifting-Based Two Dimensional Discrete Wavelet Transform on FPGA Using Pipeline Architecture
More informationModule 7 VIDEO CODING AND MOTION ESTIMATION
Module 7 VIDEO CODING AND MOTION ESTIMATION Lesson 20 Basic Building Blocks & Temporal Redundancy Instructional Objectives At the end of this lesson, the students should be able to: 1. Name at least five
More informationCOMP3121/3821/9101/ s1 Assignment 1
Sample solutions to assignment 1 1. (a) Describe an O(n log n) algorithm (in the sense of the worst case performance) that, given an array S of n integers and another integer x, determines whether or not
More informationComputational Geometry
Lecture 1: Introduction and convex hulls Geometry: points, lines,... Geometric objects Geometric relations Combinatorial complexity Computational geometry Plane (two-dimensional), R 2 Space (three-dimensional),
More informationMultiresolution Motif Discovery in Time Series
Tenth SIAM International Conference on Data Mining Columbus, Ohio, USA Multiresolution Motif Discovery in Time Series NUNO CASTRO PAULO AZEVEDO Department of Informatics University of Minho Portugal April
More informationDensity Based Clustering using Modified PSO based Neighbor Selection
Density Based Clustering using Modified PSO based Neighbor Selection K. Nafees Ahmed Research Scholar, Dept of Computer Science Jamal Mohamed College (Autonomous), Tiruchirappalli, India nafeesjmc@gmail.com
More informationData Preprocessing. Slides by: Shree Jaswal
Data Preprocessing Slides by: Shree Jaswal Topics to be covered Why Preprocessing? Data Cleaning; Data Integration; Data Reduction: Attribute subset selection, Histograms, Clustering and Sampling; Data
More informationR-Trees. Accessing Spatial Data
R-Trees Accessing Spatial Data In the beginning The B-Tree provided a foundation for R- Trees. But what s a B-Tree? A data structure for storing sorted data with amortized run times for insertion and deletion
More informationMotion Planning. O Rourke, Chapter 8
O Rourke, Chapter 8 Outline Translating a polygon Moving a ladder Shortest Path (Point-to-Point) Goal: Given disjoint polygons in the plane, and given positions s and t, find the shortest path from s to
More information9. Three Dimensional Object Representations
9. Three Dimensional Object Representations Methods: Polygon and Quadric surfaces: For simple Euclidean objects Spline surfaces and construction: For curved surfaces Procedural methods: Eg. Fractals, Particle
More informationA New Online Clustering Approach for Data in Arbitrary Shaped Clusters
A New Online Clustering Approach for Data in Arbitrary Shaped Clusters Richard Hyde, Plamen Angelov Data Science Group, School of Computing and Communications Lancaster University Lancaster, LA1 4WA, UK
More informationTime Series Analysis DM 2 / A.A
DM 2 / A.A. 2010-2011 Time Series Analysis Several slides are borrowed from: Han and Kamber, Data Mining: Concepts and Techniques Mining time-series data Lei Chen, Similarity Search Over Time-Series Data
More informationHow TokuDB Fractal TreeTM. Indexes Work. Bradley C. Kuszmaul. Guest Lecture in MIT Performance Engineering, 18 November 2010.
6.172 How Fractal Trees Work 1 How TokuDB Fractal TreeTM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November 2010. 6.172 How Fractal Trees Work 2 I m an MIT
More informationG 6i try. On the Number of Minimal 1-Steiner Trees* Discrete Comput Geom 12:29-34 (1994)
Discrete Comput Geom 12:29-34 (1994) G 6i try 9 1994 Springer-Verlag New York Inc. On the Number of Minimal 1-Steiner Trees* B. Aronov, 1 M. Bern, 2 and D. Eppstein 3 Computer Science Department, Polytechnic
More informationComparative Study of Subspace Clustering Algorithms
Comparative Study of Subspace Clustering Algorithms S.Chitra Nayagam, Asst Prof., Dept of Computer Applications, Don Bosco College, Panjim, Goa. Abstract-A cluster is a collection of data objects that
More informationMotion Estimation for Video Coding Standards
Motion Estimation for Video Coding Standards Prof. Ja-Ling Wu Department of Computer Science and Information Engineering National Taiwan University Introduction of Motion Estimation The goal of video compression
More informationPoints covered an odd number of times by translates
Points covered an odd number of times by translates Rom Pinchasi August 5, 0 Abstract Let T be a fixed triangle and consider an odd number of translated copies of T in the plane. We show that the set of
More informationM. Andrea Rodríguez-Tastets. I Semester 2008
M. -Tastets Universidad de Concepción,Chile andrea@udec.cl I Semester 2008 Outline refers to data with a location on the Earth s surface. Examples Census data Administrative boundaries of a country, state
More informationEvaluating Continuous Nearest Neighbor Queries for Streaming Time Series via Pre-fetching
Evaluating Continuous Nearest Neighbor Queries for Streaming Time Series via Pre-fetching Like Gao Zhengrong Yao X. Sean Wang Department of Information and Software Engineering, George Mason University
More informationHow TokuDB Fractal TreeTM. Indexes Work. Bradley C. Kuszmaul. MySQL UC 2010 How Fractal Trees Work 1
MySQL UC 2010 How Fractal Trees Work 1 How TokuDB Fractal TreeTM Indexes Work Bradley C. Kuszmaul MySQL UC 2010 How Fractal Trees Work 2 More Information You can download this talk and others at http://tokutek.com/technology
More informationTrajStore: an Adaptive Storage System for Very Large Trajectory Data Sets
TrajStore: an Adaptive Storage System for Very Large Trajectory Data Sets Philippe Cudré-Mauroux Eugene Wu Samuel Madden Computer Science and Artificial Intelligence Laboratory Massachusetts Institute
More informationGPU ACCELERATED SELF-JOIN FOR THE DISTANCE SIMILARITY METRIC
GPU ACCELERATED SELF-JOIN FOR THE DISTANCE SIMILARITY METRIC MIKE GOWANLOCK NORTHERN ARIZONA UNIVERSITY SCHOOL OF INFORMATICS, COMPUTING & CYBER SYSTEMS BEN KARSIN UNIVERSITY OF HAWAII AT MANOA DEPARTMENT
More informationMultidimensional Data and Modelling
Multidimensional Data and Modelling 1 Problems of multidimensional data structures l multidimensional (md-data or spatial) data and their implementation of operations between objects (spatial data practically
More informationClustering from Data Streams
Clustering from Data Streams João Gama LIAAD-INESC Porto, University of Porto, Portugal jgama@fep.up.pt 1 Introduction 2 Clustering Micro Clustering 3 Clustering Time Series Growing the Structure Adapting
More informationA Parallel Access Method for Spatial Data Using GPU
A Parallel Access Method for Spatial Data Using GPU Byoung-Woo Oh Department of Computer Engineering Kumoh National Institute of Technology Gumi, Korea bwoh@kumoh.ac.kr Abstract Spatial access methods
More informationA Image Comparative Study using DCT, Fast Fourier, Wavelet Transforms and Huffman Algorithm
International Journal of Engineering Research and General Science Volume 3, Issue 4, July-August, 15 ISSN 91-2730 A Image Comparative Study using DCT, Fast Fourier, Wavelet Transforms and Huffman Algorithm
More informationChap4: Spatial Storage and Indexing. 4.1 Storage:Disk and Files 4.2 Spatial Indexing 4.3 Trends 4.4 Summary
Chap4: Spatial Storage and Indexing 4.1 Storage:Disk and Files 4.2 Spatial Indexing 4.3 Trends 4.4 Summary Learning Objectives Learning Objectives (LO) LO1: Understand concept of a physical data model
More informationCARNEGIE MELLON UNIVERSITY DEPT. OF COMPUTER SCIENCE DATABASE APPLICATIONS
CARNEGIE MELLON UNIVERSITY DEPT. OF COMPUTER SCIENCE 15-415 DATABASE APPLICATIONS C. Faloutsos Indexing and Hashing 15-415 Database Applications http://www.cs.cmu.edu/~christos/courses/dbms.s00/ general
More information66 III Complexes. R p (r) }.
66 III Complexes III.4 Alpha Complexes In this section, we use a radius constraint to introduce a family of subcomplexes of the Delaunay complex. These complexes are similar to the Čech complexes but differ
More informationUNIT 2 Data Preprocessing
UNIT 2 Data Preprocessing Lecture Topic ********************************************** Lecture 13 Why preprocess the data? Lecture 14 Lecture 15 Lecture 16 Lecture 17 Data cleaning Data integration and
More informationLocalized and Incremental Monitoring of Reverse Nearest Neighbor Queries in Wireless Sensor Networks 1
Localized and Incremental Monitoring of Reverse Nearest Neighbor Queries in Wireless Sensor Networks 1 HAI THANH MAI AND MYOUNG HO KIM Department of Computer Science Korea Advanced Institute of Science
More information[10] Industrial DataMatrix barcodes recognition with a random tilt and rotating the camera
[10] Industrial DataMatrix barcodes recognition with a random tilt and rotating the camera Image processing, pattern recognition 865 Kruchinin A.Yu. Orenburg State University IntBuSoft Ltd Abstract The
More informationParallel Patterns for Window-based Stateful Operators on Data Streams: an Algorithmic Skeleton Approach
Parallel Patterns for Window-based Stateful Operators on Data Streams: an Algorithmic Skeleton Approach Tiziano De Matteis, Gabriele Mencagli University of Pisa Italy INTRODUCTION The recent years have
More informationDesign and Analysis of an Euler Transformation Algorithm Applied to Full-Polarimetric ISAR Imagery
Design and Analysis of an Euler Transformation Algorithm Applied to Full-Polarimetric ISAR Imagery Christopher S. Baird Advisor: Robert Giles Submillimeter-Wave Technology Laboratory (STL) Presented in
More informationIterated Functions Systems and Fractal Coding
Qing Jun He 90121047 Math 308 Essay Iterated Functions Systems and Fractal Coding 1. Introduction Fractal coding techniques are based on the theory of Iterated Function Systems (IFS) founded by Hutchinson
More informationClustering Of Ecg Using D-Stream Algorithm
Clustering Of Ecg Using D-Stream Algorithm Vaishali Yeole Jyoti Kadam Department of computer Engg. Department of computer Engg. K.C college of Engg, K.C college of Engg Thane (E). Thane (E). Abstract The
More informationWavelet Based Image Compression Using ROI SPIHT Coding
International Journal of Information & Computation Technology. ISSN 0974-2255 Volume 1, Number 2 (2011), pp. 69-76 International Research Publications House http://www.irphouse.com Wavelet Based Image
More informationA Survey on Disk-based Genome. Sequence Indexing
Contemporary Engineering Sciences, Vol. 7, 2014, no. 15, 743-748 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/10.12988/ces.2014.4684 A Survey on Disk-based Genome Sequence Indexing Woong-Kee Loh Department
More informationTime series representations: a state-of-the-art
Laboratoire LIAS cyrille.ponchateau@ensma.fr www.lias-lab.fr ISAE - ENSMA 11 Juillet 2016 1 / 33 Content 1 2 3 4 2 / 33 Content 1 2 3 4 3 / 33 What is a time series? Time Series Time series are a temporal
More informationAnalyzing Data Streams by Online DFT
Analyzing Data Streams by Online DFT Alexander Hinneburg 1, Dirk Habich 2, and Marcel Karnstedt 3 1 Martin-Luther University of Halle-Wittenberg, Germany hinneburg@informatik.uni-halle.de 2 Dresden University
More information