Data Analytics: From Conceptual Modelling to Logical Representation

Size: px
Start display at page:

Download "Data Analytics: From Conceptual Modelling to Logical Representation"

Transcription

1 Data Analytics: From Conceptual Modelling to Logical Representation Qing Wang (B) and Minjian Liu Research School of Computer Science, Australian National University, Canberra, Australia Abstract. In recent years, data analytics has been studied in a broad range of areas, such as health-care, social sciences, and commerce. In order to accurately capture user requirements for enhancing communication between analysts, domain experts and users, conceptualising data analytics tasks to provide a high level of modelling abstraction becomes increasingly important. In this paper, we discuss the modelling of data analytics and how a conceptual framework for data analytics applications can be transformed into a logical framework that supports a simple yet expressive query language for specifying data analytics tasks. We have also implemented our modelling method into a unified data analytics platform, which allows to incorporate analytics algorithms as plug-ins in a flexible and open manner, We present case studies on three real-world data analytics applications and our experimental results on an unified data analytics platform. Keywords: Data analytics Conceptual modelling Logical model Query language 1 Introduction Data analytics is rapidly growing in popularity, with a variety of applications in many areas, e.g., health-care, social sciences, commerce, etc. This has led to the recent development of a large number of data analytics tools and systems, most of which are built upon graph models, such as GraphLab [11] and Pregel [12]. Nonetheless, in practice, many data analytics applications are still conducted in an ad-hoc way, due to the lack of general principles to design, develop and implement data analytics applications. For example, the decision on choosing data models for data analytics applications often relies on individuals own expertise, rather than a systematic consideration of requirements. This calls for a formal design paradigm that can provide a high level of modelling abstraction to support users in understanding their data analytics requirements. In particular, with the increasing complexity of data analytics applications, the Q. Wang and M. Liu Contributed equally to this work. c Springer International Publishing AG 2016 I. Comyn-Wattiau et al. (Eds.): ER 2016, LNCS 9974, pp , DOI: /

2 416 Q. Wang and M. Liu need to explicitly represent data analytics requirements into a conceptual model is pressingly required [6]. Recently, several methods for conceptually modelling data analytics applications have been reported [2, 15, 16]. A conceptual modelling paradigm for network analytics applications, called the Network Analytics ER model (NAER), wasproposedin[15]. In a nutshell, the NAER model extends the concepts of the traditional ER models [4] in three aspects: (a) the structural aspect - analytical entity and relationship types are added to represent first-class entities and relationships from the data analytics perspective; (b) the manipulation aspect - topological constructs are added to explicitly represent different topological structures of interest; and (c) the integrity aspect - constraints are added for governing integrity among different data analytics tasks. Based on this conceptual modelling paradigm, a set of design guidelines has further been provided in [15], through which users can benefit from establishing a conceptual framework that provides a coherent and comprehensive view on data analytics applications. As depicted in Fig. 1, such a conceptual framework may consist of a core schema, which has basic entity and relationship types to capture data requirements as in the traditional ER modelling, topology schemas, which have analytical entity and relationship types to capture query requirements of data analytics applications, and query topics, which describe the structure of queries in query requirements and are associated with both the core schema and the topology schemas. Fig. 1. A general process of modelling data analytics applications Nonetheless, how can we transform such a conceptual framework into a logical framework which is well suited to model the logical structure of data analytics applications without ambiguities? Although basic entity and relationship types in the core schema can be easily transformed into relation schemas following the existing rules [14], it is not yet clear: (1) How can analytical entity and relationship types be accurately defined at the logical level? (2) What logical structure can topology schemas be translated into? (3) How can topological constructs be specified using a query language, ideally in a declarative way? These questions

3 Data Analytics: From Conceptual Modelling to Logical Representation 417 are left unanswered in the previous works [15,16]. This paper aims to answer these questions by exploring the connections between such a conceptual framework for data analytics and its corresponding logical representation. Contributions. We have the following contributions in this paper: We discuss how a conceptual framework for data analytics as introduced in [15] can be effectively transformed into a logical framework. We introduce a novel query language for data analytics, which extends SQL with the ability to query topological properties of interest. We have implemented our modelling method into a unified data analytics platform, which allows to incorporate analytics algorithms as plug-ins in a flexible and open manner. We present three real-world data analytics applications to illustrate the expressive power and simplicity of our modelling method, and the experimental results of evaluating the performance of our data analytics platform. Outline. In the following, Sect. 2 discusses the modelling of data analytics and Sect. 3 introduces our query language for data analytics. We discuss three data analytics applications in Sect. 4, and present our experimental results in Sect. 5. The paper is concluded in Sect Modelling Data Analytics In this section, we discuss data analytics from a modelling perspective. This is because, in practice, many organisations are facing the challenges of managing data analytics tasks in a complex environment, and using modelling techniques can bring in several advantages to addressing these challenges, including: enhancing communications among multiple stakeholders, understanding connections among complex analysis requirements, and detecting design flaws earlier and right from the start before implementing any code. We first recall the Network Analytics ER (NAER) modelling method [15], then elaborate on the transformation from a conceptual model into the logical representation for data analytics applications. Generally, the NAER modelling method supports two kinds of entities and relationships [15]: (1) base entities and relationships which specify first-class entities and relationships that should be stored in a database system from a data management perspective, as in the traditional ER modelling; (2) analytical entities and relationships which specify first-class entities and relationships used for the data analytics purpose. In the NAER model, base types are the ground from which analytical types can be derived, and the base types that define an analytical type are called the support of the analytical type. To conceptualise data analytics tasks, not only data requirements (i.e., what kind of data is needed) but also query requirements (i.e., what kinds of queries are used) are considered in the conceptual modelling process. Base entity and relationship types are used to capture data requirements, leading to a core schema,

4 418 Q. Wang and M. Liu while analytical entity and relationship types are used to capture query requirements, which yields a number of topology schemas. That is, a core schema contains a set of base types, and each topology schema contains a set of analytical types, and the support of each analytical type in a topology schema is a subset of base types in the core schema. In general, each conceptual framework for data analytics applications contains a core schema which is relatively large, and a number of topology schemas which are often small. Although being small, topology schemas can be flexibly composed into larger schemas if needed [16]. After a conceptual framework has been established as previously discussed, the question arising is: how can such a conceptual framework be transformed into a logical framework? Although, in principle, it is possible to choose any logical data model, e.g., the relational data model, a graph model or a combination of several data models, data in many real-world applications is stored in relational databases. Moreover, data analytics tasks often require sophisticated analysis on both relational and topological properties of data. For these reasons, we develop the following data model at the logical level: Transform basic entities and relationships in the core schema into a set of relations for storage, as in the traditional ER modelling approach [14]; Transform analytical entities and relationships in the topology schemas into a set of entity-relationship (ER) graphs for analytics [9]. That is, one topology schema corresponds to one type of ER graphs in which each vertex represents an analytical entity and each edge between two vertices represents an analytical relationship between two analytical entities. Topology schemas Graph schemas ER-graph Graph mapper ER-graph Graph mapper Core schema Graph mapper Relation Graph mapper Relation schemas ER-graph Relation Relation ER-graph Fig. 2. A hybrid data model at the logical level Figure 2 illustrates a hybrid data model, in which a collection of ER graphs are constructed on top of relations through graph mappers either on the fly or in a materialized manner, as will be formally defined in Sect. 3. Accordingly, the core schema and topology schemas in a conceptual model are transformed into a set of relation schemas and graph schemas in a logical model. In practice, such a hybrid data model can be easily built by applying the above transformation rules to a conceptual model that describes data analytics tasks. Since data analytics tasks

5 Data Analytics: From Conceptual Modelling to Logical Representation 419 often require additional querying capability over graphs, for example, finding paths, detecting communities, clustering, ranking, etc., in order to implement such a hybrid data model at the logical level, we would need a query language that can support joint analytics of relations and graphs. 3 A Query Language for Data Analytics We present a SQL-like query language for data analytics, called RG-SQL, which extends the standard SQL with new features to facilitate joint analytics of relations and graphs. More specifically, RG-SQL provides data definition statements that can create graphs from relations in a flexible way, and data manipulation statements to conduct various data analytics operations over relations and graphs. In the following, we explain these new features of RG-SQL in detail. Creating graphs. RG-SQL can create two types of graphs: undirected graphs and directed graphs, through the specification on graph types using UNGRAPH and DIGRAPH, respectively. Graphs can be created either on the fly or in a materialized manner with the following syntax: Graphs on the fly SELECT <attribute list> FROM <relations graphs> WHERE <graph name> IS <graph type> AS (graph mapper); Materialized graphs CREATE <graph type> <graph name> AS (graph mapper); where <graph type> := UNGRAPH DIGRAPH, and a graph mapper is a SQL query that extracts an edge list (i.e., a list of edges of a graph, which is a common data structure for representing a graph) from relations in the underlying databases for graph construction. Ranking. To assess the importance of vertices within a graph, RQ-SQL provides a RANK operator with the following syntax: RANK( <graph name>, <measure>) <measure> := degree indegree outdegree betweenness closeness pagerank A number of measures are available for determining the importance of vertices [3]. One may choose the most suitable measure for a specific query based on the type of the graph and desired properties. Each RANK( <graph name>, <measure>) yields a relation with two attributes: vertexid, andvalue. Clustering. To explore the clustering structure of vertices over a graph, RG- SQL provides a CLUSTER operator with the following syntax: CLUSTER( <graph name>, <algorithm>) <algorithm> := CC SCC GN CNM MC

6 420 Q. Wang and M. Liu where CC refers to an algorithm of finding connected components, SCC an algorithm of finding strongly connected components, and GN, CNM and MC three algorithms for community detection, which respectively correspond to Girvan-Newman algorithm [7], Clauset-Newman-Moore Algorithm [5] and Peixoto s modified Monte Carlo Algorithm [13]. Each CLUSTER( <graph name>, <algorithm>) yields a relation with three attributes: clusterid, size and members. Path finding. To find paths among two or more vertices, RG-SQL provides a PATH operator with the following syntax: PATH( <graph name>, <path expression>) <path expression> :=. V <path expression>/ <path expression> <path expression>// <path expression> where V is a vertex expression that imposes certain condition on the vertices of a path,. is a do-not-care symbol indicating that any vertex is allowed in its position, / represents one edge, and // represents any number of edges. A path expression is valid if it contains a vertex expression in the first and last positions. For example, an expression V1//V2 specifies a path between two vertices V1 and V2, regardless of the length of the path. Each PATH( <graph name>, <path expression>) yields a relation with three attributes: pathid, length and path. 3.1 Discussion We now briefly discuss the expressive power of RG-SQL in comparison with the relational query language SQL and the graph query language Cypher used in Neo4j ( Since RG-SQL extends the standard SQL with the additional operations, such as ranking, clustering and path finding, RQ-SQL is strictly more expressive than SQL and has the expressive power beyond the first order logic [1], for example, recursion in a path finding expression V1//V2 cannot be expressed by SQL but can be expressed by RG-SQL. For Cypher, it is a query language designed to express graph patterns, which can nonetheless be expressed by RG-SQL or its variations through a combination of path finding operations. However, not all operations of RG-SQL can be expressed by Cypher, e.g., ranking operations using betweenness and clustering operations using GN. 4 Data Analytics Applications In this section we study data analytics tasks in three real-world applications and explain how data analytics requirements can be conceptualized in our work. 4.1 Digital Library Digital Library ( is a bibliographical network containing a collection of articles, authors, and publishers. Each article is written by one

7 Data Analytics: From Conceptual Modelling to Logical Representation 421 or more authors, one article may cite a number of other articles, and articles are included in conference proceedings or journals published by publishers. Figure 3 depicts a conceptual schema for this data analytics application, which includes the topology schemas S a1 and S a2 required by the following queries: Q1: [Collaborative communities] Find the communities that consist of authors who collaborate with each other to publish articles together. Q2: [Influential articles] Find the top 3 most influential articles. For Q1, we may use RQ-SQL to create a materialized coauthorship graph for coauthorship over S a1, then find the collaborative communities in the coauthorship graph by applying the MC algorithm in CLUSTER. CREATE UNGRAPH coauthorship AS (SELECT w1.aid, w2.aid AS coaid FROM WRITE AS w1, WRITE AS w2 WHERE w1.aid!=w2.aid AND w1.pid=w2.pid); SELECT clusterid, size, members FROM CLUSTER(coauthorship, MC); For Q2, we may create a citation graph over S a2 on the fly and then to find influential articles in the citation graph using the measure betweenness. SELECT vertexid, value FROM RANK(citation, betweenness) WHERE citation IS DIGRAPH AS (SELECT aid, citedaid FROM CITE) LIMIT 3; Topology Schemas S a1 Sa2 S a3 from AUTHOR* COAUTHOR SHIP ARTICLE* CITATION JOURNAL* COCITATION to Core Schema AUTHOR WRITE ARTICLE PUBLISH PUBLISHED _BY PUBLISHER + CITE JOURNAL PROCEEDINGS Fig. 3. A conceptual schema for Digital Library

8 422 Q. Wang and M. Liu 4.2 Twitter Twitter ( is a social network which enables users to post tweets. Users may follow one another. A tweet can mention one or more users and be labelled by one or more tags. Figure 4 depicts a conceptual schema for data analytics in Twitter. Typical data analytics tasks in Twitter include to analyse how users follow each other and to find the most followed people as described by the following queries: Q3: [Shortest path] Find the shortest path between Jack and Max. Q4: [Most followed people] Find the most followed people who have posted at least one tweet about ANU. Firstly, the following graph over the topology schema S t1 is created based on entities of user and their relationships in following. Then for Q3 we may find the shortest path between Jack and Max using the following RG-SQL query: SELECT * FROM PATH(following, v1//v2) WHERE v1 AS (SELECT uid FROM USER WHERE name = Jack ) AND v2 AS (SELECT uid FROM USER WHERE name = Max ) ORDER BY length ASC LIMIT 1; For Q4, we need to not only find the most followed people in the following graph but also people who have posted a tweet tagged from the relations over the core schema, as illustrated by the following RG-SQL query. SELECT uid, value FROM RANK(following, pagerank) AS p1, POST AS p2, LABELLED_BY AS l WHERE p1.vertexid=p2.uid AND p2.twid=l.tid AND l.label= ANU ORDER BY value DESC; 4.3 Stack Overflow Stack Overflow ( is a collaboratively edited question and answer site for programmers. Users may ask questions or post answers. A question may have zero or more answers and be labelled by tags. For each question, one answer can be accepted as the accepted answer. A conceptual schema for data analytics in Stack Overflow is presented in Fig. 5. Q5: [Python experts] Find top 10 Python experts in Stack Overflow (i.e. users who often reply Python questions and their answers are often accepted). Q6: [Most influential expert] Find the influential expert in Stack Overflow who is involved in one of the top 3 largest question-answer communities. Similarly, we first create the getting answers graph over the topology schema S s1. Then the RQ-SQL query for Q5 is follows:

9 Data Analytics: From Conceptual Modelling to Logical Representation 423 Fig. 4. A conceptual schema for Twitter SELECT * FROM RANK(getting_answers, pagerank) WHERE vertexid IN (SELECT owner_id FROM ANSWER AS a, LABELLED_BY AS l, TAG AS t WHERE a.parent_qid=l.qid AND l.tid=t.tid AND tag_label = python ) LIMIT 10; For Q6, we have the following RQ-SQL query, in which both RANK and CLUSTER operators are applied over two different graphs and their results can be flexibly combined to support further analytics. SELECT r.vertexid FROM RANK(getting_answers, pagerank) AS r, (SELECT members FROM CLUSTER(co-answering, MC) ORDER BY size DESC LIMIT 3) AS c WHERE r.vertexid=any(c.members); 5 Experiments We have implemented our modelling method into a unified data analytics platform, called Rogas, which allows to incorporate analytics algorithms as plug-ins in a flexible and open manner [10]. To understand how well Rogas can perform in comparison with other database systems, we have conducted experiments to compare the expressive power of query languages and the time efficiency of query execution in three different systems: PostgreSQL ( Neo4j ( and Rogas. These experiments were performed on a Dell Optiplex 9020 desktop computer with the Intel(R) Core(TM) i CPU 3.6 GHz 8 cores processor, 16 GB of memory and 256 GB disk. Rogas extends the query engine of PostgreSQL 9.4.4, with additional functionalities implemented using Python The version of Neo4j we used is community

10 424 Q. Wang and M. Liu Fig. 5. A conceptual schema for Stack Overflow In our experiments, we used the data sets from the data analytics applications discussed in Sect. 4: (1) Digital Library ( ) data set provided by the Digital Library ( (2) Stack Overflow data set from the Stanford Network Analytics Platform ( snap-icwsm/), and (3) Twitter data set provided by Haewoon Kwak ( kaist.ac.kr/traces/www2010.html). Table 1 presents more details about these three data sets. Table 2 depicts the queries used in our experiments, which can be generally divided into three categories: (1) Q1 Q3 are relational queries including join, sorting, and aggregate operations; (2) Q4 Q10 are queries about graph properties, including: triangle counting, pagerank centrality, path finding and community detection; (3) Q11 Q12 are sophisticated queries that may combine several graph properties, e.g., Q11 combines pagerank centrality with finding connected components and Q12 combines pagerank centrality with path finding. Our first experiment is to illustrate the expressive power of the three query languages: PostgreSQL, RG-SQL and Cypher in terms of the queries Q1 Q12. As shown in Table 3, PostgreSQL, RG-SQL and Cypher do have different expressive powers. SQL cannot be used to specify Q6 Q12 and Cypher cannot be used to specify Q10 Q12. Nonetheless, RG-SQL is expressive enough to specify all these queries. Our second experiment is to evaluate the time efficiency of query execution in Rogas, PostgreSQL and Neo4j. As not all queries can be expressed by PostgreSQL and Neo4j, we have thus compared Q1-Q5 over the three systems, and Q6-Q9 only over Rogas and Neo4j. Note that, for Q6 and Q7, Neo4j needs to use an extension, called Neo4j Mazerunner, to run graph analytics algorithms at scale with Hadoop HDFS and Apache Spark ( developer/apache-spark/#mazerunner), and is thus required to send an HTTP

11 Data Analytics: From Conceptual Modelling to Logical Representation 425 Table 1. Three data sets used in our experiments Data set Raw data size No of vertices in graphs (Neo4j) No of edges in graphs (Neo4j) No of records in relations (PostgreSQL) 14.9 GB (XML) 1,128,243 2,488,849 publisher 50 journal 128 proceedings 6,421 article 337,006 author 784,638 write 932,400 cite 1,212,894 Stack Overflow 30.6 GB (XML) 21,713,109 31,747,662 question 7,990,787 answer 13,684,117 tag 38,205 labelled by 13,466,686 Twitter 29.7 GB (TXT) 13,250, ,368,797 tweet 10,762,104 tag 210,121 user 2,277,971 follow 259,602,970 mentioned in 3,108,776 labelled by 1,657,051 GET request to Neo4j Mazerunner. In such cases, the time of executing queries in Neo4j includes the time for sending and receiving the requests. For each query, we ran it 5 times in each system and took the average time for plotting. Figure 6 presents our experimental result. The key observations are as follows: For Q1 Q5, Rogas performed equally well with PostgreSQL, and better than Neo4j in most queries except for Q4. This is because Q4 is about pattern matching which requires to navigate hyper-connectivity on graphs, and Neo4j has been particularly optimised for such queries whereas we have not yet implemented any query optimisation techniques. For Q5, it is not surprising that Rogas performed better than Neo4j since it handles the problem of triangle counting, for which the study in [8] has also experimentally verified that relational databases can perform the triangle counting task very efficiently through expressing a three-way self-join. For Q6 Q7, as Neo4j needs to use Neo4j Mazerunner, it requires time on sending and receiving the requests. Thus, Rogas performed better than Neo4j. However, for Q8 Q9, similar to Q4, these queries need to navigate hyperconnectivity on graphs and Neo4j performed better than Rogas. In addition to Q1 Q12, we have also run several queries about closeness centrality over Twitter using Rogas and Neo4j. Rogas can successfully complete the queries and return the query results, while Neo4j failed and the system reported the OutOfMemory error. The reason for this is that the graphs created in Twitter are large so that processing these queries exceeded the memory limitation of Neo4j.

12 426 Q. Wang and M. Liu Table 2. Queries used in our experiments Query Data set Query description Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Join Operation + Sorting Operation Stack Show the question id, the owner id and the tag label of top 10 Overflow questions that have the most view count. Join Operation + Sorting Operation + Aggregate Operation Show the top 5 answerers and their latest reputation score in an Stack descending order based on the number of their answers that accepted Overflow by questions. Join Operation + Sorting Operation + Aggregate Operation Twitter Stack Overflow Show the number of articles of each journal and proceeding along with the journal name and the proceeding title in a descending order. Pattern Matching Recommend 10 twitter users for Jack who currently does not follow these users but Jack follows somebody who are following them. Triangle Counting Count the number of triangles of the co-authorship network. PageRank Centrality Find the top 10 influential authors according to the pagerank centrality in the co-authorship network. Connected Component Count the number of connected components of the co-authorship network. Path Finding Find paths with length less than 2, which connect two author V1 and V2 in the co-authorship network where author V1 is affiliated at ANU and author V2 is affiliated at UNSW. Shortest Path Find a shortest paths between two authors Michael Norrish and Kevin Elphinstone in the co-author network. Community Detection Find a group of tags that they are often used together to label a question. PageRank Centrality + Connected Component According to the pagerank centrality, find the top 3 authors of the biggest collaborative community in the co-authorship network. PageRank Centrality + Path Finding According to the pagerank centrality, show how the top 2 authors connect with each other in the co-authorship network.

13 Data Analytics: From Conceptual Modelling to Logical Representation 427 Table 3. Comparison on the expressive power of the query languages PostgreSQL, RQ-SQL and Cypher over the queries Q1 Q12, where and indicate expressible and not expressible, respectively Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 PostgreSQL RG-SQL Cypher Fig. 6. Comparison on the time efficiency of query execution in Rogas, PostgreSQL and Neo4j over the queries Q1 Q9

14 428 Q. Wang and M. Liu 6 Conclusions In this paper, we have discussed how data analytics tasks can be conceptualised by a conceptual model and then transformed into a logical model. We have also proposed a query language for data analytics, and implemented the proposed methods into a data analytics platform that can unify various data analytics tasks and algorithms. This work was based on our case studies on several realworld data analytics applications. In the future, we plan to add query topics into our data analytics platform and investigate the development of a query language at a higher level through query topics. We will also study network dynamics and develop techniques to analyse and visualise networks that dynamically change over time. Acknowledgement. We thank the Digital Library for providing the data set of the bibliographical network. References 1. Abiteboul, S., Hull, R., Vianu, V.: Foundations of Databases. Addison-Wesley, Reading (1995) 2. Bao, Z., Tay, Y., Zhou, J.: sonschema: a conceptual schema for social networks. In: Conceptual Modeling, pp (2013) 3. Brandes, U., Erlebach, T.: Network Analysis: Methodological Foundations. Springer Science & Business Media, New York (2005) 4. Chen, P.: The entity-relationship model - toward a unified view of data. TODS 1(1), 9 36 (1976) 5. Clauset, A., Newman, M.E., Moore, C.: Finding community structure in very large networks. Phys. Rev. E 70(6), (2004) 6. Embley, D.W., Liddle, S.W.: Big data conceptual modeling to the rescue. In: Ng, W., Storey, V.C., Trujillo, J.C. (eds.) ER LNCS, vol. 8217, pp Springer, Heidelberg (2013). doi: / Girvan, M., Newman, M.E.: Community structure in social and biological networks. PNAS 99(12), (2002) 8. Jindal, A., Madden, S.: GRAPHiQL: a graph intuitive query language for relationaldatabases. In: IEEE International Conference on Big Data, pp (2014) 9. Kasneci, G., Ramanath, M., Sozio, M., Suchanek, F.M., Weikum, G.: Star: steinertree approximation in relationship graphs. In: ICDE, pp (2009) 10. Liu, M., Wang, Q.: Rogas: a declaratice framework for network analysis. In: VLDB (2016) 11. Low, Y., Gonzalez, J.E., Kyrola, A., Bickson, D., Guestrin, C.E., Hellerstein, J.: Graphlab: a new framework for parallel machine learning. arxiv preprint arxiv: (2014) 12. Malewicz, G., Austern, M.H., Bik, A.J., Dehnert, J.C., Horn, I., Leiser, N., Czajkowski, G.: Pregel: a system for large-scale graph processing. In: SIGMOD, pp (2010) 13. Peixoto, T.P.: Efficient Monte Carlo and greedy heuristic for the inference of stochastic block models. Phys. Rev. E 89(1), (2014)

15 Data Analytics: From Conceptual Modelling to Logical Representation Thalheim, B.: Entity-relationship Modeling: Foundations of Database Technology. Springer Science & Business Media, New York (2013) 15. Wang, Q.: Network analytics ER model-towards a conceptual view of network analytics. In: ER, pp (2014) 16. Wang, Q.: A conceptual modeling framework for network analytics. Data Knowl. Eng. 99, (2015)

Towards a Unified Framework for Network Analytics

Towards a Unified Framework for Network Analytics Towards a Unified Framework for Network Analytics Minjian Liu A thesis submitted in partial fulfillment of the degree of Master of Computing at The Department of Computer Science Australian National University

More information

I ++ Mapreduce: Incremental Mapreduce for Mining the Big Data

I ++ Mapreduce: Incremental Mapreduce for Mining the Big Data IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 18, Issue 3, Ver. IV (May-Jun. 2016), PP 125-129 www.iosrjournals.org I ++ Mapreduce: Incremental Mapreduce for

More information

A Fast and High Throughput SQL Query System for Big Data

A Fast and High Throughput SQL Query System for Big Data A Fast and High Throughput SQL Query System for Big Data Feng Zhu, Jie Liu, and Lijie Xu Technology Center of Software Engineering, Institute of Software, Chinese Academy of Sciences, Beijing, China 100190

More information

Memory-Optimized Distributed Graph Processing. through Novel Compression Techniques

Memory-Optimized Distributed Graph Processing. through Novel Compression Techniques Memory-Optimized Distributed Graph Processing through Novel Compression Techniques Katia Papakonstantinopoulou Joint work with Panagiotis Liakos and Alex Delis University of Athens Athens Colloquium in

More information

MPGM: A Mixed Parallel Big Graph Mining Tool

MPGM: A Mixed Parallel Big Graph Mining Tool MPGM: A Mixed Parallel Big Graph Mining Tool Ma Pengjiang 1 mpjr_2008@163.com Liu Yang 1 liuyang1984@bupt.edu.cn Wu Bin 1 wubin@bupt.edu.cn Wang Hongxu 1 513196584@qq.com 1 School of Computer Science,

More information

Big Graph Processing. Fenggang Wu Nov. 6, 2016

Big Graph Processing. Fenggang Wu Nov. 6, 2016 Big Graph Processing Fenggang Wu Nov. 6, 2016 Agenda Project Publication Organization Pregel SIGMOD 10 Google PowerGraph OSDI 12 CMU GraphX OSDI 14 UC Berkeley AMPLab PowerLyra EuroSys 15 Shanghai Jiao

More information

Searching SNT in XML Documents Using Reduction Factor

Searching SNT in XML Documents Using Reduction Factor Searching SNT in XML Documents Using Reduction Factor Mary Posonia A Department of computer science, Sathyabama University, Tamilnadu, Chennai, India maryposonia@sathyabamauniversity.ac.in http://www.sathyabamauniversity.ac.in

More information

Parallel HITS Algorithm Implemented Using HADOOP GIRAPH Framework to resolve Big Data Problem

Parallel HITS Algorithm Implemented Using HADOOP GIRAPH Framework to resolve Big Data Problem I J C T A, 9(41) 2016, pp. 1235-1239 International Science Press Parallel HITS Algorithm Implemented Using HADOOP GIRAPH Framework to resolve Big Data Problem Hema Dubey *, Nilay Khare *, Alind Khare **

More information

Community Mining Tool using Bibliography Data

Community Mining Tool using Bibliography Data Community Mining Tool using Bibliography Data Ryutaro Ichise, Hideaki Takeda National Institute of Informatics 2-1-2 Hitotsubashi Chiyoda-ku Tokyo, 101-8430, Japan {ichise,takeda}@nii.ac.jp Kosuke Ueyama

More information

Semantic Web in a Constrained Environment

Semantic Web in a Constrained Environment Semantic Web in a Constrained Environment Laurens Rietveld and Stefan Schlobach Department of Computer Science, VU University Amsterdam, The Netherlands {laurens.rietveld,k.s.schlobach}@vu.nl Abstract.

More information

FPGP: Graph Processing Framework on FPGA

FPGP: Graph Processing Framework on FPGA FPGP: Graph Processing Framework on FPGA Guohao DAI, Yuze CHI, Yu WANG, Huazhong YANG E.E. Dept., TNLIST, Tsinghua University dgh14@mails.tsinghua.edu.cn 1 Big graph is widely used Big graph is widely

More information

Jure Leskovec Including joint work with Y. Perez, R. Sosič, A. Banarjee, M. Raison, R. Puttagunta, P. Shah

Jure Leskovec Including joint work with Y. Perez, R. Sosič, A. Banarjee, M. Raison, R. Puttagunta, P. Shah Jure Leskovec (@jure) Including joint work with Y. Perez, R. Sosič, A. Banarjee, M. Raison, R. Puttagunta, P. Shah 2 My research group at Stanford: Mining and modeling large social and information networks

More information

How Primo Works VE. 1.1 Welcome. Notes: Published by Articulate Storyline Welcome to how Primo works.

How Primo Works VE. 1.1 Welcome. Notes: Published by Articulate Storyline   Welcome to how Primo works. How Primo Works VE 1.1 Welcome Welcome to how Primo works. 1.2 Objectives By the end of this session, you will know - What discovery, delivery, and optimization are - How the library s collections and

More information

Open Access The Three-dimensional Coding Based on the Cone for XML Under Weaving Multi-documents

Open Access The Three-dimensional Coding Based on the Cone for XML Under Weaving Multi-documents Send Orders for Reprints to reprints@benthamscience.ae 676 The Open Automation and Control Systems Journal, 2014, 6, 676-683 Open Access The Three-dimensional Coding Based on the Cone for XML Under Weaving

More information

Clash of the Titans: MapReduce vs. Spark for Large Scale Data Analytics

Clash of the Titans: MapReduce vs. Spark for Large Scale Data Analytics Clash of the Titans: MapReduce vs. Spark for Large Scale Data Analytics Presented by: Dishant Mittal Authors: Juwei Shi, Yunjie Qiu, Umar Firooq Minhas, Lemei Jiao, Chen Wang, Berthold Reinwald and Fatma

More information

Digree: Building A Distributed Graph Processing Engine out of Single-node Graph Database Installations

Digree: Building A Distributed Graph Processing Engine out of Single-node Graph Database Installations Digree: Building A Distributed Graph ing Engine out of Single-node Graph Database Installations Vasilis Spyropoulos Athens University of Economics and Business Athens, Greece vasspyrop@aueb.gr ABSTRACT

More information

PROJECT PERIODIC REPORT

PROJECT PERIODIC REPORT PROJECT PERIODIC REPORT Grant Agreement number: 257403 Project acronym: CUBIST Project title: Combining and Uniting Business Intelligence and Semantic Technologies Funding Scheme: STREP Date of latest

More information

Structural Analysis of Paper Citation and Co-Authorship Networks using Network Analysis Techniques

Structural Analysis of Paper Citation and Co-Authorship Networks using Network Analysis Techniques Structural Analysis of Paper Citation and Co-Authorship Networks using Network Analysis Techniques Kouhei Sugiyama, Hiroyuki Ohsaki and Makoto Imase Graduate School of Information Science and Technology,

More information

CIM/E Oriented Graph Database Model Architecture and Parallel Network Topology Processing

CIM/E Oriented Graph Database Model Architecture and Parallel Network Topology Processing CIM/E Oriented Graph Model Architecture and Parallel Network Topology Processing Zhangxin Zhou a, b, Chen Yuan a, Ziyan Yao a, Jiangpeng Dai a, Guangyi Liu a, Renchang Dai a, Zhiwei Wang a, and Garng M.

More information

Analyzing Flight Data

Analyzing Flight Data IBM Analytics Analyzing Flight Data Jeff Carlson Rich Tarro July 21, 2016 2016 IBM Corporation Agenda Spark Overview a quick review Introduction to Graph Processing and Spark GraphX GraphX Overview Demo

More information

Automated Tagging for Online Q&A Forums

Automated Tagging for Online Q&A Forums 1 Automated Tagging for Online Q&A Forums Rajat Sharma, Nitin Kalra, Gautam Nagpal University of California, San Diego, La Jolla, CA 92093, USA {ras043, nikalra, gnagpal}@ucsd.edu Abstract Hashtags created

More information

Research Works to Cope with Big Data Volume and Variety. Jiaheng Lu University of Helsinki, Finland

Research Works to Cope with Big Data Volume and Variety. Jiaheng Lu University of Helsinki, Finland Research Works to Cope with Big Data Volume and Variety Jiaheng Lu University of Helsinki, Finland Big Data: 4Vs Photo downloaded from: https://blog.infodiagram.com/2014/04/visualizing-big-data-concepts-strong.html

More information

4th National Conference on Electrical, Electronics and Computer Engineering (NCEECE 2015)

4th National Conference on Electrical, Electronics and Computer Engineering (NCEECE 2015) 4th National Conference on Electrical, Electronics and Computer Engineering (NCEECE 2015) Benchmark Testing for Transwarp Inceptor A big data analysis system based on in-memory computing Mingang Chen1,2,a,

More information

UML-Based Conceptual Modeling of Pattern-Bases

UML-Based Conceptual Modeling of Pattern-Bases UML-Based Conceptual Modeling of Pattern-Bases Stefano Rizzi DEIS - University of Bologna Viale Risorgimento, 2 40136 Bologna - Italy srizzi@deis.unibo.it Abstract. The concept of pattern, meant as an

More information

Acolyte: An In-Memory Social Network Query System

Acolyte: An In-Memory Social Network Query System Acolyte: An In-Memory Social Network Query System Ze Tang, Heng Lin, Kaiwei Li, Wentao Han, and Wenguang Chen Department of Computer Science and Technology, Tsinghua University Beijing 100084, China {tangz10,linheng11,lkw10,hwt04}@mails.tsinghua.edu.cn

More information

FOUNDATIONS OF A CROSS-DISCIPLINARY PEDAGOGY FOR BIG DATA *

FOUNDATIONS OF A CROSS-DISCIPLINARY PEDAGOGY FOR BIG DATA * FOUNDATIONS OF A CROSS-DISCIPLINARY PEDAGOGY FOR BIG DATA * Joshua Eckroth Stetson University DeLand, Florida 386-740-2519 jeckroth@stetson.edu ABSTRACT The increasing awareness of big data is transforming

More information

Centralities (4) By: Ralucca Gera, NPS. Excellence Through Knowledge

Centralities (4) By: Ralucca Gera, NPS. Excellence Through Knowledge Centralities (4) By: Ralucca Gera, NPS Excellence Through Knowledge Some slide from last week that we didn t talk about in class: 2 PageRank algorithm Eigenvector centrality: i s Rank score is the sum

More information

A FRAMEWORK FOR EFFICIENT DATA SEARCH THROUGH XML TREE PATTERNS

A FRAMEWORK FOR EFFICIENT DATA SEARCH THROUGH XML TREE PATTERNS A FRAMEWORK FOR EFFICIENT DATA SEARCH THROUGH XML TREE PATTERNS SRIVANI SARIKONDA 1 PG Scholar Department of CSE P.SANDEEP REDDY 2 Associate professor Department of CSE DR.M.V.SIVA PRASAD 3 Principal Abstract:

More information

3.4 Data-Centric workflow

3.4 Data-Centric workflow 3.4 Data-Centric workflow One of the most important activities in a S-DWH environment is represented by data integration of different and heterogeneous sources. The process of extract, transform, and load

More information

Graph Data Management

Graph Data Management Graph Data Management Analysis and Optimization of Graph Data Frameworks presented by Fynn Leitow Overview 1) Introduction a) Motivation b) Application for big data 2) Choice of algorithms 3) Choice of

More information

Distributed Graph Storage. Veronika Molnár, UZH

Distributed Graph Storage. Veronika Molnár, UZH Distributed Graph Storage Veronika Molnár, UZH Overview Graphs and Social Networks Criteria for Graph Processing Systems Current Systems Storage Computation Large scale systems Comparison / Best systems

More information

Optimizing CPU Cache Performance for Pregel-Like Graph Computation

Optimizing CPU Cache Performance for Pregel-Like Graph Computation Optimizing CPU Cache Performance for Pregel-Like Graph Computation Songjie Niu, Shimin Chen* State Key Laboratory of Computer Architecture Institute of Computing Technology Chinese Academy of Sciences

More information

Iterative Graph Summarization based on Grouping

Iterative Graph Summarization based on Grouping Iterative Graph Summarization based on Grouping Sirui Li Supervisor: Dr. Qing Wang COMP4560: Advanced Computing Project Australian National University Semester 1, 2017 May 26, 2017 Acknowledgements First

More information

Topics. Big Data Analytics What is and Why Hadoop? Comparison to other technologies Hadoop architecture Hadoop ecosystem Hadoop usage examples

Topics. Big Data Analytics What is and Why Hadoop? Comparison to other technologies Hadoop architecture Hadoop ecosystem Hadoop usage examples Hadoop Introduction 1 Topics Big Data Analytics What is and Why Hadoop? Comparison to other technologies Hadoop architecture Hadoop ecosystem Hadoop usage examples 2 Big Data Analytics What is Big Data?

More information

A Novel Parallel Hierarchical Community Detection Method for Large Networks

A Novel Parallel Hierarchical Community Detection Method for Large Networks A Novel Parallel Hierarchical Community Detection Method for Large Networks Ping Lu Shengmei Luo Lei Hu Yunlong Lin Junyang Zou Qiwei Zhong Kuangyan Zhu Jian Lu Qiao Wang Southeast University, School of

More information

Implementation of Network Community Profile using Local Spectral algorithm and its application in Community Networking

Implementation of Network Community Profile using Local Spectral algorithm and its application in Community Networking Implementation of Network Community Profile using Local Spectral algorithm and its application in Community Networking Vaibhav VPrakash Department of Computer Science and Engineering, Sri Jayachamarajendra

More information

Structure Mining for Intellectual Networks

Structure Mining for Intellectual Networks Structure Mining for Intellectual Networks Ryutaro Ichise 1, Hideaki Takeda 1, and Kosuke Ueyama 2 1 National Institute of Informatics, 2-1-2 Chiyoda-ku Tokyo 101-8430, Japan, {ichise,takeda}@nii.ac.jp

More information

Session 7: Oracle R Enterprise OAAgraph Package

Session 7: Oracle R Enterprise OAAgraph Package Session 7: Oracle R Enterprise 1.5.1 OAAgraph Package Oracle Spatial and Graph PGX Graph Algorithms Oracle R Technologies Mark Hornick Director, Oracle Advanced Analytics and Machine Learning July 2017

More information

Ontology based Model and Procedure Creation for Topic Analysis in Chinese Language

Ontology based Model and Procedure Creation for Topic Analysis in Chinese Language Ontology based Model and Procedure Creation for Topic Analysis in Chinese Language Dong Han and Kilian Stoffel Information Management Institute, University of Neuchâtel Pierre-à-Mazel 7, CH-2000 Neuchâtel,

More information

Cluster Computing Architecture. Intel Labs

Cluster Computing Architecture. Intel Labs Intel Labs Legal Notices INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED

More information

Linking Entities in Chinese Queries to Knowledge Graph

Linking Entities in Chinese Queries to Knowledge Graph Linking Entities in Chinese Queries to Knowledge Graph Jun Li 1, Jinxian Pan 2, Chen Ye 1, Yong Huang 1, Danlu Wen 1, and Zhichun Wang 1(B) 1 Beijing Normal University, Beijing, China zcwang@bnu.edu.cn

More information

A User Study on Features Supporting Subjective Relevance for Information Retrieval Interfaces

A User Study on Features Supporting Subjective Relevance for Information Retrieval Interfaces A user study on features supporting subjective relevance for information retrieval interfaces Lee, S.S., Theng, Y.L, Goh, H.L.D., & Foo, S. (2006). Proc. 9th International Conference of Asian Digital Libraries

More information

FedX: A Federation Layer for Distributed Query Processing on Linked Open Data

FedX: A Federation Layer for Distributed Query Processing on Linked Open Data FedX: A Federation Layer for Distributed Query Processing on Linked Open Data Andreas Schwarte 1, Peter Haase 1,KatjaHose 2, Ralf Schenkel 2, and Michael Schmidt 1 1 fluid Operations AG, Walldorf, Germany

More information

Formulating XML-IR Queries

Formulating XML-IR Queries Alan Woodley Faculty of Information Technology, Queensland University of Technology PO Box 2434. Brisbane Q 4001, Australia ap.woodley@student.qut.edu.au Abstract: XML information retrieval systems differ

More information

INDEX-BASED JOIN IN MAPREDUCE USING HADOOP MAPFILES

INDEX-BASED JOIN IN MAPREDUCE USING HADOOP MAPFILES Al-Badarneh et al. Special Issue Volume 2 Issue 1, pp. 200-213 Date of Publication: 19 th December, 2016 DOI-https://dx.doi.org/10.20319/mijst.2016.s21.200213 INDEX-BASED JOIN IN MAPREDUCE USING HADOOP

More information

HBase vs Neo4j. Technical overview. Name: Vladan Jovičić CR09 Advanced Scalable Data (Fall, 2017) Ecolé Normale Superiuere de Lyon

HBase vs Neo4j. Technical overview. Name: Vladan Jovičić CR09 Advanced Scalable Data (Fall, 2017) Ecolé Normale Superiuere de Lyon HBase vs Neo4j Technical overview Name: Vladan Jovičić CR09 Advanced Scalable Data (Fall, 2017) Ecolé Normale Superiuere de Lyon 12th October 2017 1 Contents 1 Introduction 3 2 Overview of HBase and Neo4j

More information

Drakkar: a graph based All-Nearest Neighbour search algorithm for bibliographic coupling

Drakkar: a graph based All-Nearest Neighbour search algorithm for bibliographic coupling Drakkar: a graph based All-Nearest Neighbour search algorithm for bibliographic coupling Bart Thijs KU Leuven, FEB, ECOOM; Leuven; Belgium Bart.thijs@kuleuven.be Abstract Drakkar is a novel algorithm for

More information

A Parallel Community Detection Algorithm for Big Social Networks

A Parallel Community Detection Algorithm for Big Social Networks A Parallel Community Detection Algorithm for Big Social Networks Yathrib AlQahtani College of Computer and Information Sciences King Saud University Collage of Computing and Informatics Saudi Electronic

More information

QUERY RECOMMENDATION SYSTEM USING USERS QUERYING BEHAVIOR

QUERY RECOMMENDATION SYSTEM USING USERS QUERYING BEHAVIOR International Journal of Emerging Technology and Innovative Engineering QUERY RECOMMENDATION SYSTEM USING USERS QUERYING BEHAVIOR V.Megha Dept of Computer science and Engineering College Of Engineering

More information

A Benchmarking Criteria for the Evaluation of OLAP Tools

A Benchmarking Criteria for the Evaluation of OLAP Tools A Benchmarking Criteria for the Evaluation of OLAP Tools Fiaz Majeed Department of Information Technology, University of Gujrat, Gujrat, Pakistan. Email: fiaz.majeed@uog.edu.pk Abstract Generating queries

More information

CERTIFICATE IN SOFTWARE DEVELOPMENT LIFE CYCLE IN BIG DATA AND BUSINESS INTELLIGENCE (SDLC-BD & BI)

CERTIFICATE IN SOFTWARE DEVELOPMENT LIFE CYCLE IN BIG DATA AND BUSINESS INTELLIGENCE (SDLC-BD & BI) CERTIFICATE IN SOFTWARE DEVELOPMENT LIFE CYCLE IN BIG DATA AND BUSINESS INTELLIGENCE (SDLC-BD & BI) The Certificate in Software Development Life Cycle in BIGDATA, Business Intelligence and Tableau program

More information

WEB STRUCTURE MINING USING PAGERANK, IMPROVED PAGERANK AN OVERVIEW

WEB STRUCTURE MINING USING PAGERANK, IMPROVED PAGERANK AN OVERVIEW ISSN: 9 694 (ONLINE) ICTACT JOURNAL ON COMMUNICATION TECHNOLOGY, MARCH, VOL:, ISSUE: WEB STRUCTURE MINING USING PAGERANK, IMPROVED PAGERANK AN OVERVIEW V Lakshmi Praba and T Vasantha Department of Computer

More information

Apache Spark: A Literature Review. Presenter: Aaron Sarson

Apache Spark: A Literature Review. Presenter: Aaron Sarson Apache Spark: A Literature Review Presenter: Aaron Sarson Outline Introduction to Spark Problem to be addressed Proposed Approach Ø Research Questions Contributions Results Ø RQ1, RQ2, RQ3 Conclusion &

More information

Data Processing at Scale (CSE 511)

Data Processing at Scale (CSE 511) Data Processing at Scale (CSE 511) Note: Below outline is subject to modifications and updates. About this Course Database systems are used to provide convenient access to disk-resident data through efficient

More information

GRAPH DB S & APPLICATIONS

GRAPH DB S & APPLICATIONS GRAPH DB S & APPLICATIONS DENIS VRDOLJAK GUNNAR KLEEMANN UC BERKELEY SCHOOL OF INFORMATION BERKELEY DATA SCIENCE GROUP, LLC PRESENTATION ROAD MAP Intro Background Examples Our Work Graph Databases Intro

More information

Subjective Relevance: Implications on Interface Design for Information Retrieval Systems

Subjective Relevance: Implications on Interface Design for Information Retrieval Systems Subjective : Implications on interface design for information retrieval systems Lee, S.S., Theng, Y.L, Goh, H.L.D., & Foo, S (2005). Proc. 8th International Conference of Asian Digital Libraries (ICADL2005),

More information

DESIGN AND IMPLEMENTATION OF TOOL FOR CONVERTING A RELATIONAL DATABASE INTO AN XML DOCUMENT: A REVIEW

DESIGN AND IMPLEMENTATION OF TOOL FOR CONVERTING A RELATIONAL DATABASE INTO AN XML DOCUMENT: A REVIEW DESIGN AND IMPLEMENTATION OF TOOL FOR CONVERTING A RELATIONAL DATABASE INTO AN XML DOCUMENT: A REVIEW Sunayana Kohli Masters of Technology, Department of Computer Science, Manav Rachna College of Engineering,

More information

SEMANTIC WEB POWERED PORTAL INFRASTRUCTURE

SEMANTIC WEB POWERED PORTAL INFRASTRUCTURE SEMANTIC WEB POWERED PORTAL INFRASTRUCTURE YING DING 1 Digital Enterprise Research Institute Leopold-Franzens Universität Innsbruck Austria DIETER FENSEL Digital Enterprise Research Institute National

More information

Fine-grained Data Partitioning Framework for Distributed Database Systems

Fine-grained Data Partitioning Framework for Distributed Database Systems Fine-grained Partitioning Framework for Distributed base Systems Ning Xu «Supervised by Bin Cui» Key Lab of High Confidence Software Technologies (Ministry of Education) & School of EECS, Peking University

More information

Semi-Structured Data Management (CSE 511)

Semi-Structured Data Management (CSE 511) Semi-Structured Data Management (CSE 511) Note: Below outline is subject to modifications and updates. About this Course Database systems are used to provide convenient access to disk-resident data through

More information

An Industrial Employee Development Application Protocol Using Wireless Sensor Networks

An Industrial Employee Development Application Protocol Using Wireless Sensor Networks RESEARCH ARTICLE An Industrial Employee Development Application Protocol Using Wireless Sensor Networks 1 N.Roja Ramani, 2 A.Stenila 1,2 Asst.professor, Dept.of.Computer Application, Annai Vailankanni

More information

Design and Implementation of Agricultural Information Resources Vertical Search Engine Based on Nutch

Design and Implementation of Agricultural Information Resources Vertical Search Engine Based on Nutch 619 A publication of CHEMICAL ENGINEERING TRANSACTIONS VOL. 51, 2016 Guest Editors: Tichun Wang, Hongyang Zhang, Lei Tian Copyright 2016, AIDIC Servizi S.r.l., ISBN 978-88-95608-43-3; ISSN 2283-9216 The

More information

Deep Web Crawling and Mining for Building Advanced Search Application

Deep Web Crawling and Mining for Building Advanced Search Application Deep Web Crawling and Mining for Building Advanced Search Application Zhigang Hua, Dan Hou, Yu Liu, Xin Sun, Yanbing Yu {hua, houdan, yuliu, xinsun, yyu}@cc.gatech.edu College of computing, Georgia Tech

More information

Popularity of Twitter Accounts: PageRank on a Social Network

Popularity of Twitter Accounts: PageRank on a Social Network Popularity of Twitter Accounts: PageRank on a Social Network A.D-A December 8, 2017 1 Problem Statement Twitter is a social networking service, where users can create and interact with 140 character messages,

More information

Why do we need graph processing?

Why do we need graph processing? Why do we need graph processing? Community detection: suggest followers? Determine what products people will like Count how many people are in different communities (polling?) Graphs are Everywhere Group

More information

AN ONTOLOGY-BASED KNOWLEDGE AS A SERVICE FRAMEWORK: A CASE STUDY OF DEVELOPING A USER-CENTERED PORTAL FOR HOME RECOVERY

AN ONTOLOGY-BASED KNOWLEDGE AS A SERVICE FRAMEWORK: A CASE STUDY OF DEVELOPING A USER-CENTERED PORTAL FOR HOME RECOVERY AN ONTOLOGY-BASED KNOWLEDGE AS A SERVICE FRAMEWORK: A CASE STUDY OF DEVELOPING A USER-CENTERED PORTAL FOR HOME RECOVERY Marut Buranarach, Thepchai Supnithi and Passakon Prathombutr (NECTEC, Thailand) Abstract

More information

Parallelizing Structural Joins to Process Queries over Big XML Data Using MapReduce

Parallelizing Structural Joins to Process Queries over Big XML Data Using MapReduce Parallelizing Structural Joins to Process Queries over Big XML Data Using MapReduce Huayu Wu Institute for Infocomm Research, A*STAR, Singapore huwu@i2r.a-star.edu.sg Abstract. Processing XML queries over

More information

A Semi-automatic Support to Adapt E-Documents in an Accessible and Usable Format for Vision Impaired Users

A Semi-automatic Support to Adapt E-Documents in an Accessible and Usable Format for Vision Impaired Users A Semi-automatic Support to Adapt E-Documents in an Accessible and Usable Format for Vision Impaired Users Elia Contini, Barbara Leporini, and Fabio Paternò ISTI-CNR, Pisa, Italy {elia.contini,barbara.leporini,fabio.paterno}@isti.cnr.it

More information

Introducing Oracle R Enterprise 1.4 -

Introducing Oracle R Enterprise 1.4 - Hello, and welcome to this online, self-paced lesson entitled Introducing Oracle R Enterprise. This session is part of an eight-lesson tutorial series on Oracle R Enterprise. My name is Brian Pottle. I

More information

Performance Evaluation of Semantic Registries: OWLJessKB and instancestore

Performance Evaluation of Semantic Registries: OWLJessKB and instancestore Service Oriented Computing and Applications manuscript No. (will be inserted by the editor) Performance Evaluation of Semantic Registries: OWLJessKB and instancestore Simone A. Ludwig 1, Omer F. Rana 2

More information

Enhancing Wrapper Usability through Ontology Sharing and Large Scale Cooperation

Enhancing Wrapper Usability through Ontology Sharing and Large Scale Cooperation Enhancing Wrapper Usability through Ontology Enhancing Sharing Wrapper and Large Usability Scale Cooperation through Ontology Sharing and Large Scale Cooperation Christian Schindler, Pranjal Arya, Andreas

More information

INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY

INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY A PATH FOR HORIZING YOUR INNOVATIVE WORK SURVEY ON BIG DATA USING DATA MINING AYUSHI V. RATHOD, PROF. S. S. ASOLE BNCOE,

More information

Top-k Keyword Search Over Graphs Based On Backward Search

Top-k Keyword Search Over Graphs Based On Backward Search Top-k Keyword Search Over Graphs Based On Backward Search Jia-Hui Zeng, Jiu-Ming Huang, Shu-Qiang Yang 1College of Computer National University of Defense Technology, Changsha, China 2College of Computer

More information

A Proposed Method in Agile Practices to Create Requirements Documentation and Test Cases

A Proposed Method in Agile Practices to Create Requirements Documentation and Test Cases A Proposed Method in Agile Practices to Create Requirements Documentation and Cases Palash Bera 1 and Abhimanyu Gupta 2 1 Saint Louis University pbera@slu.edu 2 Saint Louis University abhimanyugupta@slu.edu

More information

Authoring and Maintaining of Educational Applications on the Web

Authoring and Maintaining of Educational Applications on the Web Authoring and Maintaining of Educational Applications on the Web Denis Helic Institute for Information Processing and Computer Supported New Media ( IICM ), Graz University of Technology Graz, Austria

More information

LOGICAL OPERATOR USAGE IN STRUCTURAL MODELLING

LOGICAL OPERATOR USAGE IN STRUCTURAL MODELLING LOGICAL OPERATOR USAGE IN STRUCTURAL MODELLING Ieva Zeltmate (a) (a) Riga Technical University, Faculty of Computer Science and Information Technology Department of System Theory and Design ieva.zeltmate@gmail.com

More information

FREQUENT PATTERN MINING IN BIG DATA USING MAVEN PLUGIN. School of Computing, SASTRA University, Thanjavur , India

FREQUENT PATTERN MINING IN BIG DATA USING MAVEN PLUGIN. School of Computing, SASTRA University, Thanjavur , India Volume 115 No. 7 2017, 105-110 ISSN: 1311-8080 (printed version); ISSN: 1314-3395 (on-line version) url: http://www.ijpam.eu ijpam.eu FREQUENT PATTERN MINING IN BIG DATA USING MAVEN PLUGIN Balaji.N 1,

More information

CPSC 532L Project Development and Axiomatization of a Ranking System

CPSC 532L Project Development and Axiomatization of a Ranking System CPSC 532L Project Development and Axiomatization of a Ranking System Catherine Gamroth cgamroth@cs.ubc.ca Hammad Ali hammada@cs.ubc.ca April 22, 2009 Abstract Ranking systems are central to many internet

More information

Chapter 8 Visualization and Optimization

Chapter 8 Visualization and Optimization Chapter 8 Visualization and Optimization Recommended reference books: [1] Edited by R. S. Gallagher: Computer Visualization, Graphics Techniques for Scientific and Engineering Analysis by CRC, 1994 [2]

More information

Community Detection: Comparison of State of the Art Algorithms

Community Detection: Comparison of State of the Art Algorithms Community Detection: Comparison of State of the Art Algorithms Josiane Mothe IRIT, UMR5505 CNRS & ESPE, Univ. de Toulouse Toulouse, France e-mail: josiane.mothe@irit.fr Karen Mkhitaryan Institute for Informatics

More information

Apache Spark Graph Performance with Memory1. February Page 1 of 13

Apache Spark Graph Performance with Memory1. February Page 1 of 13 Apache Spark Graph Performance with Memory1 February 2017 Page 1 of 13 Abstract Apache Spark is a powerful open source distributed computing platform focused on high speed, large scale data processing

More information

Graph Sampling Approach for Reducing. Computational Complexity of. Large-Scale Social Network

Graph Sampling Approach for Reducing. Computational Complexity of. Large-Scale Social Network Journal of Innovative Technology and Education, Vol. 3, 216, no. 1, 131-137 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/1.12988/jite.216.6828 Graph Sampling Approach for Reducing Computational Complexity

More information

Blended Learning Outline: Cloudera Data Analyst Training (171219a)

Blended Learning Outline: Cloudera Data Analyst Training (171219a) Blended Learning Outline: Cloudera Data Analyst Training (171219a) Cloudera Univeristy s data analyst training course will teach you to apply traditional data analytics and business intelligence skills

More information

Evaluating Use of Data Flow Systems for Large Graph Analysis

Evaluating Use of Data Flow Systems for Large Graph Analysis Evaluating Use of Data Flow Systems for Large Graph Analysis Andy Yoo and Ian Kaplan, P. O. Box 808, Livermore, CA 94551 This work performed under the auspices of the U.S. Department of Energy by under

More information

PREGEL AND GIRAPH. Why Pregel? Processing large graph problems is challenging Options

PREGEL AND GIRAPH. Why Pregel? Processing large graph problems is challenging Options Data Management in the Cloud PREGEL AND GIRAPH Thanks to Kristin Tufte 1 Why Pregel? Processing large graph problems is challenging Options Custom distributed infrastructure Existing distributed computing

More information

USING DYNAMOGRAPH: APPLICATION SCENARIOS FOR LARGE-SCALE TEMPORAL GRAPH PROCESSING

USING DYNAMOGRAPH: APPLICATION SCENARIOS FOR LARGE-SCALE TEMPORAL GRAPH PROCESSING USING DYNAMOGRAPH: APPLICATION SCENARIOS FOR LARGE-SCALE TEMPORAL GRAPH PROCESSING Matthias Steinbauer, Gabriele Anderst-Kotsis Institute of Telecooperation TALK OUTLINE Introduction and Motivation Preliminaries

More information

Development of DKB ETL module in case of data conversion

Development of DKB ETL module in case of data conversion Journal of Physics: Conference Series PAPER OPEN ACCESS Development of DKB ETL module in case of data conversion To cite this article: A Y Kaida et al 2018 J. Phys.: Conf. Ser. 1015 032055 View the article

More information

INTERNATIONAL JOURNAL OF RESEARCH IN COMPUTER APPLICATIONS AND ROBOTICS ISSN EFFECTIVE KEYWORD SEARCH OF FUZZY TYPE IN XML

INTERNATIONAL JOURNAL OF RESEARCH IN COMPUTER APPLICATIONS AND ROBOTICS ISSN EFFECTIVE KEYWORD SEARCH OF FUZZY TYPE IN XML INTERNATIONAL JOURNAL OF RESEARCH IN COMPUTER APPLICATIONS AND ROBOTICS ISSN 2320-7345 EFFECTIVE KEYWORD SEARCH OF FUZZY TYPE IN XML Mr. Mohammed Tariq Alam 1,Mrs.Shanila Mahreen 2 Assistant Professor

More information

CSCI 320 Group Project

CSCI 320 Group Project CSCI 320 Group Project Project Description This is a semester long group project. Project Goals Group project of 3-4 students. Groups will not change after assigned. Select a project domain from the list

More information

The Unified Modelling Language. Example Diagrams. Notation vs. Methodology. UML and Meta Modelling

The Unified Modelling Language. Example Diagrams. Notation vs. Methodology. UML and Meta Modelling UML and Meta ling Topics: UML as an example visual notation The UML meta model and the concept of meta modelling Driven Architecture and model engineering The AndroMDA open source project Applying cognitive

More information

Community Detection in Bipartite Networks:

Community Detection in Bipartite Networks: Community Detection in Bipartite Networks: Algorithms and Case Studies Kathy Horadam and Taher Alzahrani Mathematical and Geospatial Sciences, RMIT Melbourne, Australia IWCNA 2014 Community Detection,

More information

AN EFFECTIVE SEARCH ON WEB LOG FROM MOST POPULAR DOWNLOADED CONTENT

AN EFFECTIVE SEARCH ON WEB LOG FROM MOST POPULAR DOWNLOADED CONTENT AN EFFECTIVE SEARCH ON WEB LOG FROM MOST POPULAR DOWNLOADED CONTENT Brindha.S 1 and Sabarinathan.P 2 1 PG Scholar, Department of Computer Science and Engineering, PABCET, Trichy 2 Assistant Professor,

More information

Finding Topic-centric Identified Experts based on Full Text Analysis

Finding Topic-centric Identified Experts based on Full Text Analysis Finding Topic-centric Identified Experts based on Full Text Analysis Hanmin Jung, Mikyoung Lee, In-Su Kang, Seung-Woo Lee, Won-Kyung Sung Information Service Research Lab., KISTI, Korea jhm@kisti.re.kr

More information

PREGEL: A SYSTEM FOR LARGE- SCALE GRAPH PROCESSING

PREGEL: A SYSTEM FOR LARGE- SCALE GRAPH PROCESSING PREGEL: A SYSTEM FOR LARGE- SCALE GRAPH PROCESSING G. Malewicz, M. Austern, A. Bik, J. Dehnert, I. Horn, N. Leiser, G. Czajkowski Google, Inc. SIGMOD 2010 Presented by Ke Hong (some figures borrowed from

More information

PREGEL: A SYSTEM FOR LARGE-SCALE GRAPH PROCESSING

PREGEL: A SYSTEM FOR LARGE-SCALE GRAPH PROCESSING PREGEL: A SYSTEM FOR LARGE-SCALE GRAPH PROCESSING Grzegorz Malewicz, Matthew Austern, Aart Bik, James Dehnert, Ilan Horn, Naty Leiser, Grzegorz Czajkowski (Google, Inc.) SIGMOD 2010 Presented by : Xiu

More information

Survey on Community Question Answering Systems

Survey on Community Question Answering Systems World Journal of Technology, Engineering and Research, Volume 3, Issue 1 (2018) 114-119 Contents available at WJTER World Journal of Technology, Engineering and Research Journal Homepage: www.wjter.com

More information

Change Detection System for the Maintenance of Automated Testing

Change Detection System for the Maintenance of Automated Testing Change Detection System for the Maintenance of Automated Testing Miroslav Bures To cite this version: Miroslav Bures. Change Detection System for the Maintenance of Automated Testing. Mercedes G. Merayo;

More information

Report. X-Stream Edge-centric Graph processing

Report. X-Stream Edge-centric Graph processing Report X-Stream Edge-centric Graph processing Yassin Hassan hassany@student.ethz.ch Abstract. X-Stream is an edge-centric graph processing system, which provides an API for scatter gather algorithms. The

More information

International Journal of Data Mining & Knowledge Management Process (IJDKP) Vol.7, No.3, May Dr.Zakea Il-Agure and Mr.Hicham Noureddine Itani

International Journal of Data Mining & Knowledge Management Process (IJDKP) Vol.7, No.3, May Dr.Zakea Il-Agure and Mr.Hicham Noureddine Itani LINK MINING PROCESS Dr.Zakea Il-Agure and Mr.Hicham Noureddine Itani Higher Colleges of Technology, United Arab Emirates ABSTRACT Many data mining and knowledge discovery methodologies and process models

More information

Constructing Object Oriented Class for extracting and using data from data cube

Constructing Object Oriented Class for extracting and using data from data cube Constructing Object Oriented Class for extracting and using data from data cube Antoaneta Ivanova Abstract: The goal of this article is to depict Object Oriented Conceptual Model Data Cube using it as

More information