The Many Faces Of Apache Ignite. David Robinson, Software Engineer May 13, 2016
|
|
- Molly Holland
- 6 years ago
- Views:
Transcription
1 The Many Faces Of Apache Ignite David Robinson, Software Engineer May 13, 2016
2 A Face In elementary geometry, a face is a two-dimensional polygon on the boundary of a polyhedron. 2 Attribution:Robert Webb's Stella software
3 Some Faces of Apache Ignite Data Streaming SQL Transactions Services File System Data Grid 3 Persistence Clusters Spark Integration Attribution:Robert Webb's Stella software
4 Background The Market, Apache Ignite, A Use Case 4
5 Understanding the In Memory Eco-System Fabrics In-Memory Database Apache Ignite Distributed Caches Redis Memcached Data Grid Hazelcast Alluxio(?) Dashdb SAP Hana The distinctions may be blurring coming down to performance and scale 5
6 Apache Ignite Forms a Cluster All those faces potentially running on each node 6 Source:
7 What Is Genesis Graph? Running Today (short demo) A Property Graph Database built on Apache Ignite Basic Pieces Vertex Edge Vertex Properties Edge Properties 7
8 Leveraging Capabilities in the Grid less usage Genesis Graph DB Today planned more usage 8
9 How the Apache Ignite Grid Is Used For Genesis Graph towards a market leading (open governance/source) graph database store 9
10 Apache Ignite And Building A Big Data Graph Database Capabilities to construct a graph database. ID Generation Data representation and storage Multi-model + Analytics Integration Data Streaming and Eventing Transactions Partition awareness Fringe Benefits Keeping all, or large parts, of the graph in memory Notebook Integration Available for Data Scientists Real Time Graphs with the streaming 10
11 Future ID Generation On The Ignite Grid genesis graph Apache genesis Ignite Grid genesis Computing genesis Framework graph graph Custom AtomicID Service graph genesis graph get an id write a vertex Genesis Graph Client Atomics in Ignite are distributed across the cluster, essentially enabling performing atomic operations (such as increment-and-get or compare-and-set) with the same globally-visible value 11 Slide contains animation
12 Graph Storage On The Ignite Grid Ignite indexes index index partitioned cache index partitioned cache index partitioned cache index index partitioned cache partition aware index index Apache Ignite Grid Computing Framework read / write through to disk partitioned cache Genesis Graph Client H2 Cassandra HBase 12 Partitioned (with back up?) cache Off heap memory Write and read through persistence Slide contains animation
13 The Challenges Of Data Locality vertex Ex: hotel Key, Value Ex: name, hyatt network 13 This slide has automation
14 Forcing Data Locality through Affinity Keys vertex Key, Value Affinity Interface mapkeytonode(k key) int[] allpartitions(clusternode n) network 14 Co-location is required to use the Ignite SQL Join capability This slide has automation
15 Data Representation And Storage Challenges The Graph will need to implement its own, graph level indexes Ignite Hash Map data structure is inefficient at large scales public class InternalVertex implements Serializable { /** vertex id (indexed). = true) public Long id; /** ability to query via Ignite public String label;... Most efficient for query would be to inject new fields into this as user defines schema 15 This slide has automation
16 Data Representation And Storage Challenges The Graph will need to implement its own, graph level indexes Ignite Hash Map data structure is inefficient at large scales public class InternalVertex implements Serializable { /** vertex id (indexed). = true) public Long id; /** ability to query via Ignite public String label;... public class UserVertexIndex implements Serializable { /** vertex id (indexed). = true) public String public Object value; Next idea is to auto generate beans that represent? indexes and let Ignite efficiently handle the indexing
17 Data Representation And Storage Challenges Tuning TinkerPop 3.x Strategies To Match the Storage Model Custom steps and strategies? Gremlin: g.e().has("since", "2005").fill(m); select * from edgestorecache where since=
18 Creating A Cache For the Graph public void opengraphvertexcache() { String namespacedcachename = getnamespacedcachename(ggdefinitions.genesisgraph_vertexcache_prefix); CacheConfiguration<Long, InternalVertex> cfg = new CacheConfiguration<>(namespacedCacheName); // we want to support transactions on all of our caches // this does not rule out atomic updates outside of a transaction cfg.setatomicitymode(cacheatomicitymode.transactional); cfg.setcachemode(cachemode.partitioned); // NOTE: the index here must be key/value pairs (in twos) // cfg.setindexedtypes(affinitykey.class, InternalVertex.class); cfg.setindexedtypes(long.class, InternalVertex.class); // must force close transactions because we cannot stop caches with open transactions IgniteTransactions txcontainer = this.igniteclientconnection.gethandletotxinterface(); if (txcontainer!= null) { Transaction atx = txcontainer.tx(); if (atx!= null) { if (atx.state().ordinal() == TransactionState.ACTIVE.ordinal()) { atx.commit(); } } } IgniteCache<Long, InternalVertex> internalvertexcache = this.igniteclientconnection.ignite.getorcreatecache(cfg); } // add the new cache into the list of caches to be closed this.cachesallocated.put(namespacedcachename, internalvertexcache); 18
19 Multi-Model + Analytic Processing Integration Spark RDDs Gremlin Graph Traversals SQL Property Queries data streaming 19
20 Analytic Processing: Spark Example scala> import org.apache.tinkerpop.gremlin.ignitegraph.structure.internal._ import org.apache.tinkerpop.gremlin.ignitegraph.structure.internal._ scala> val ic = new IgniteContext[Integer, InternalVertex](sc, () => new IgniteConfiguration()) ic: org.apache.ignite.spark.ignitecontext[integer,org.apache.tinkerpop.gremlin.ignitegraph.structure.internal.internalvertex] = org.apache.ignite.spark.ignitecontext@713935c8 scala> val vertices = sharedrdd.collect() vertices: Array[(Integer, org.apache.tinkerpop.gremlin.ignitegraph.structure.internal.internalvertex)] = Array((1,InternalVertex [id=1, collocateid=1, label=person, ]), (2,InternalVertex [id=2, collocateid=1, label=person, ]), (3,InternalVertex [id=3, collocateid=1, label=person, ]), (4,InternalVertex [id=4, collocateid=1, label=address, ]), (5,InternalVertex [id=5, collocateid=1, label=phonenumber, ])) scala> sharedrdd.foreach(println) scala> vertices.foreach(println) (1,InternalVertex [id=1, collocateid=1, label=person, ]) (2,InternalVertex [id=2, collocateid=1, label=person, ]) (3,InternalVertex [id=3, collocateid=1, label=person, ]) (4,InternalVertex [id=4, collocateid=1, label=address, ]) (5,InternalVertex [id=5, collocateid=1, label=phonenumber, ]) 20
21 Analytic Processing: SQL Example private void dowork() { String JDBCSTRING = "jdbc:ignite:cfg://cache=ignitegraph1graphvertexcache@file:/users/graphie/downloads/apacheignite/ignite-fabric final/david/ david-ignite.xml"; try { // Register JDBC driver. Class.forName("org.apache.ignite.IgniteJdbcDriver"); // Open JDBC connection (cache name is not specified, which means that we use default cache). Connection conn = DriverManager.getConnection(JDBCSTRING); Statement stmt1 = conn.createstatement(); ResultSet rs = stmt1.executequery("select * from internalvertex"); while (rs.next()) { System.out.println("Id "+rs.getlong("id")+" Label "+rs.getstring("label")); } stmt1.close(); conn.close(); } catch (Exception e) { e.printstacktrace(); } Id 3 Label person Id 1 Label person Id 2 Label person Id 4 Label address Id 5 Label phonenumber 21
22 Apache Ignite And Building A Big Data Graph Database Capabilities to construct a graph database. ID Generation Data representation and storage Multi-model Data Streaming and Eventing Transactions Partition awareness Fringe Benefits Keeping all, or large parts, of the graph in memory Notebook Integration Available for Data Scientists Real Time Graphs with the streaming 22
23 Partition Awareness On The Ignite Grid vertex vertex cache property vertex property cache metaprop cache Ignite internals Can also be off heap rather than same JVM Apache Ignite JVM Data location can be controlled via Affinity Keys in Ignite Compute can also be co-located 23
24 Genesis Graph Visualization Visualization becomes much easier with all of the possible ways to access the graph data Gremlin Server Integration or Other Data Integration 24 UK to France International Air Routes Attribution: Graham Wallis, IBM
25 Genesis Graph Visualization Airports Sized By Number Of Routes Via Gremlin Server Interface 25 Attribution: Graham Wallis, IBM
Accelerate MySQL for Demanding OLAP and OLTP Use Cases with Apache Ignite. Peter Zaitsev, Denis Magda Santa Clara, California April 25th, 2017
Accelerate MySQL for Demanding OLAP and OLTP Use Cases with Apache Ignite Peter Zaitsev, Denis Magda Santa Clara, California April 25th, 2017 About the Presentation Problems Existing Solutions Denis Magda
More informationOverview. Prerequisites. Course Outline. Course Outline :: Apache Spark Development::
Title Duration : Apache Spark Development : 4 days Overview Spark is a fast and general cluster computing system for Big Data. It provides high-level APIs in Scala, Java, Python, and R, and an optimized
More informationProcessing of big data with Apache Spark
Processing of big data with Apache Spark JavaSkop 18 Aleksandar Donevski AGENDA What is Apache Spark? Spark vs Hadoop MapReduce Application Requirements Example Architecture Application Challenges 2 WHAT
More informationCSE 444: Database Internals. Lecture 23 Spark
CSE 444: Database Internals Lecture 23 Spark References Spark is an open source system from Berkeley Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing. Matei
More informationAnalyzing Flight Data
IBM Analytics Analyzing Flight Data Jeff Carlson Rich Tarro July 21, 2016 2016 IBM Corporation Agenda Spark Overview a quick review Introduction to Graph Processing and Spark GraphX GraphX Overview Demo
More informationAsanka Padmakumara. ETL 2.0: Data Engineering with Azure Databricks
Asanka Padmakumara ETL 2.0: Data Engineering with Azure Databricks Who am I? Asanka Padmakumara Business Intelligence Consultant, More than 8 years in BI and Data Warehousing A regular speaker in data
More informationDistributed Systems. 22. Spark. Paul Krzyzanowski. Rutgers University. Fall 2016
Distributed Systems 22. Spark Paul Krzyzanowski Rutgers University Fall 2016 November 26, 2016 2015-2016 Paul Krzyzanowski 1 Apache Spark Goal: generalize MapReduce Similar shard-and-gather approach to
More informationIntroduction to NoSQL Databases
Introduction to NoSQL Databases Roman Kern KTI, TU Graz 2017-10-16 Roman Kern (KTI, TU Graz) Dbase2 2017-10-16 1 / 31 Introduction Intro Why NoSQL? Roman Kern (KTI, TU Graz) Dbase2 2017-10-16 2 / 31 Introduction
More informationDATA SCIENCE USING SPARK: AN INTRODUCTION
DATA SCIENCE USING SPARK: AN INTRODUCTION TOPICS COVERED Introduction to Spark Getting Started with Spark Programming in Spark Data Science with Spark What next? 2 DATA SCIENCE PROCESS Exploratory Data
More informationCSC System Development with Java. Database Connection. Department of Statistics and Computer Science. Budditha Hettige
CSC 308 2.0 System Development with Java Database Connection Budditha Hettige Department of Statistics and Computer Science Budditha Hettige 1 From database to Java There are many brands of database: Microsoft
More informationBlended Learning Outline: Developer Training for Apache Spark and Hadoop (180404a)
Blended Learning Outline: Developer Training for Apache Spark and Hadoop (180404a) Cloudera s Developer Training for Apache Spark and Hadoop delivers the key concepts and expertise need to develop high-performance
More informationMigrate from Netezza Workload Migration
Migrate from Netezza Automated Big Data Open Netezza Source Workload Migration CASE SOLUTION STUDY BRIEF Automated Netezza Workload Migration To achieve greater scalability and tighter integration with
More informationDatabases and Big Data Today. CS634 Class 22
Databases and Big Data Today CS634 Class 22 Current types of Databases SQL using relational tables: still very important! NoSQL, i.e., not using relational tables: term NoSQL popular since about 2007.
More informationE6895 Advanced Big Data Analytics Lecture 4:
E6895 Advanced Big Data Analytics Lecture 4: Data Store Ching-Yung Lin, Ph.D. Adjunct Professor, Dept. of Electrical Engineering and Computer Science Chief Scientist, Graph Computing, IBM Watson Research
More informationIn-memory data pipeline and warehouse at scale using Spark, Spark SQL, Tachyon and Parquet
In-memory data pipeline and warehouse at scale using Spark, Spark SQL, Tachyon and Parquet Ema Iancuta iorhian@gmail.com Radu Chilom radu.chilom@gmail.com Big data analytics / machine learning 6+ years
More informationAn Introduction to Apache Spark
An Introduction to Apache Spark 1 History Developed in 2009 at UC Berkeley AMPLab. Open sourced in 2010. Spark becomes one of the largest big-data projects with more 400 contributors in 50+ organizations
More informationERwin and JDBC. Mar. 6, 2007 Myoung Ho Kim
ERwin and JDBC Mar. 6, 2007 Myoung Ho Kim ERwin ERwin a popular commercial ER modeling tool» other tools: Dia (open source), Visio, ConceptDraw, etc. supports database schema generation 2 ERwin UI 3 Data
More informationCERTIFICATE IN SOFTWARE DEVELOPMENT LIFE CYCLE IN BIG DATA AND BUSINESS INTELLIGENCE (SDLC-BD & BI)
CERTIFICATE IN SOFTWARE DEVELOPMENT LIFE CYCLE IN BIG DATA AND BUSINESS INTELLIGENCE (SDLC-BD & BI) The Certificate in Software Development Life Cycle in BIGDATA, Business Intelligence and Tableau program
More informationMicrosoft. Exam Questions Perform Data Engineering on Microsoft Azure HDInsight (beta) Version:Demo
Microsoft Exam Questions 70-775 Perform Data Engineering on Microsoft Azure HDInsight (beta) Version:Demo NEW QUESTION 1 You have an Azure HDInsight cluster. You need to store data in a file format that
More informationApache Ignite and Apache Spark Where Fast Data Meets the IoT
Apache Ignite and Apache Spark Where Fast Data Meets the IoT Denis Magda GridGain Product Manager Apache Ignite PMC http://ignite.apache.org #apacheignite #denismagda Agenda IoT Demands to Software IoT
More informationSciSpark 201. Searching for MCCs
SciSpark 201 Searching for MCCs Agenda for 201: Access your SciSpark & Notebook VM (personal sandbox) Quick recap. of SciSpark Project What is Spark? SciSpark Extensions scitensor: N-dimensional arrays
More information2017 GridGain Systems, Inc. In-Memory Performance Durability of Disk
In-Memory Performance Durability of Disk Ignite the Fire in your SQL App Akmal B. Chaudhri Technology Evangelist GridGain Systems Agenda SQL Capabilities Connectivity Data Definition Language Data Manipulation
More informationIn-Memory Computing Essentials
In-Memory Computing Essentials for Architects and Developers: Part 1 Denis Magda Ignite PMC Chair GridGain Director of Product Management Agenda Apache Ignite Overview Clustering and Deployment Distributed
More informationThe Evolution of Big Data Platforms and Data Science
IBM Analytics The Evolution of Big Data Platforms and Data Science ECC Conference 2016 Brandon MacKenzie June 13, 2016 2016 IBM Corporation Hello, I m Brandon MacKenzie. I work at IBM. Data Science - Offering
More informationApache Spark 2.0. Matei
Apache Spark 2.0 Matei Zaharia @matei_zaharia What is Apache Spark? Open source data processing engine for clusters Generalizes MapReduce model Rich set of APIs and libraries In Scala, Java, Python and
More informationGetting Started with Apache Ignite as a Distributed Database
Getting Started with Apache Ignite as a Distributed Database VALENTIN KULICHENKO Lead Architect GridGain Systems, Inc. 2018 GridGain Systems, Inc. Agenda Apache Ignite as a Distributed Database Connectivity
More informationAccelerate MySQL for Demanding OLAP and OLTP Use Case with Apache Ignite December 7, 2016
Accelerate MySQL for Demanding OLAP and OLTP Use Case with Apache Ignite December 7, 2016 Nikita Ivanov CTO and Co-Founder GridGain Systems Peter Zaitsev CEO and Co-Founder Percona About the Presentation
More informationBig Data Architect.
Big Data Architect www.austech.edu.au WHAT IS BIG DATA ARCHITECT? A big data architecture is designed to handle the ingestion, processing, and analysis of data that is too large or complex for traditional
More informationUnifying Big Data Workloads in Apache Spark
Unifying Big Data Workloads in Apache Spark Hossein Falaki @mhfalaki Outline What s Apache Spark Why Unification Evolution of Unification Apache Spark + Databricks Q & A What s Apache Spark What is Apache
More informationBig Data. Big Data Analyst. Big Data Engineer. Big Data Architect
Big Data Big Data Analyst INTRODUCTION TO BIG DATA ANALYTICS ANALYTICS PROCESSING TECHNIQUES DATA TRANSFORMATION & BATCH PROCESSING REAL TIME (STREAM) DATA PROCESSING Big Data Engineer BIG DATA FOUNDATION
More informationCloud Computing & Visualization
Cloud Computing & Visualization Workflows Distributed Computation with Spark Data Warehousing with Redshift Visualization with Tableau #FIUSCIS School of Computing & Information Sciences, Florida International
More informationBig Data Analytics using Apache Hadoop and Spark with Scala
Big Data Analytics using Apache Hadoop and Spark with Scala Training Highlights : 80% of the training is with Practical Demo (On Custom Cloudera and Ubuntu Machines) 20% Theory Portion will be important
More informationApache Ignite TM - In- Memory Data Fabric Fast Data Meets Open Source
Apache Ignite TM - In- Memory Data Fabric Fast Data Meets Open Source DMITRIY SETRAKYAN Founder, PPMC https://ignite.apache.org @apacheignite @dsetrakyan Agenda About In- Memory Computing Apache Ignite
More informationAbout Codefrux While the current trends around the world are based on the internet, mobile and its applications, we try to make the most out of it. As for us, we are a well established IT professionals
More informationBig Data Hadoop Developer Course Content. Big Data Hadoop Developer - The Complete Course Course Duration: 45 Hours
Big Data Hadoop Developer Course Content Who is the target audience? Big Data Hadoop Developer - The Complete Course Course Duration: 45 Hours Complete beginners who want to learn Big Data Hadoop Professionals
More informationHadoop 2.x Core: YARN, Tez, and Spark. Hortonworks Inc All Rights Reserved
Hadoop 2.x Core: YARN, Tez, and Spark YARN Hadoop Machine Types top-of-rack switches core switch client machines have client-side software used to access a cluster to process data master nodes run Hadoop
More informationIn-Memory Performance Durability of Disk GridGain Systems, Inc.
In-Memory Performance Durability of Disk Apache Ignite In-Memory Hammer for Your Data Science Toolkit Denis Magda Ignite PMC Chair GridGain Director of Product Management Agenda Apache Ignite Overview
More informationIntroduction to Big Data. NoSQL Databases. Instituto Politécnico de Tomar. Ricardo Campos
Instituto Politécnico de Tomar Introduction to Big Data NoSQL Databases Ricardo Campos Mestrado EI-IC Análise e Processamento de Grandes Volumes de Dados Tomar, Portugal, 2016 Part of the slides used in
More informationC:/Users/zzaier/Documents/NetBeansProjects/WebApplication4/src/java/mainpackage/MainClass.java
package mainpackage; import java.sql.connection; import java.sql.drivermanager; import java.sql.resultset; import java.sql.sqlexception; import java.sql.statement; import javax.ws.rs.core.context; import
More informationSpark Overview. Professor Sasu Tarkoma.
Spark Overview 2015 Professor Sasu Tarkoma www.cs.helsinki.fi Apache Spark Spark is a general-purpose computing framework for iterative tasks API is provided for Java, Scala and Python The model is based
More informationPrincipal Software Engineer Red Hat Emerging Technology June 24, 2015
USING APACHE SPARK FOR ANALYTICS IN THE CLOUD William C. Benton Principal Software Engineer Red Hat Emerging Technology June 24, 2015 ABOUT ME Distributed systems and data science in Red Hat's Emerging
More informationStudy of NoSQL Database Along With Security Comparison
Study of NoSQL Database Along With Security Comparison Ankita A. Mall [1], Jwalant B. Baria [2] [1] Student, Computer Engineering Department, Government Engineering College, Modasa, Gujarat, India ank.fetr@gmail.com
More informationChapter 4: Apache Spark
Chapter 4: Apache Spark Lecture Notes Winter semester 2016 / 2017 Ludwig-Maximilians-University Munich PD Dr. Matthias Renz 2015, Based on lectures by Donald Kossmann (ETH Zürich), as well as Jure Leskovec,
More informationAnalytics in Spark. Yanlei Diao Tim Hunter. Slides Courtesy of Ion Stoica, Matei Zaharia and Brooke Wenig
Analytics in Spark Yanlei Diao Tim Hunter Slides Courtesy of Ion Stoica, Matei Zaharia and Brooke Wenig Outline 1. A brief history of Big Data and Spark 2. Technical summary of Spark 3. Unified analytics
More informationTechno Expert Solutions An institute for specialized studies!
Course Content of Big Data Hadoop( Intermediate+ Advance) Pre-requistes: knowledge of Core Java/ Oracle: Basic of Unix S.no Topics Date Status Introduction to Big Data & Hadoop Importance of Data& Data
More informationTurning Relational Database Tables into Spark Data Sources
Turning Relational Database Tables into Spark Data Sources Kuassi Mensah Jean de Lavarene Director Product Mgmt Director Development Server Technologies October 04, 2017 3 Safe Harbor Statement The following
More informationBig Data Syllabus. Understanding big data and Hadoop. Limitations and Solutions of existing Data Analytics Architecture
Big Data Syllabus Hadoop YARN Setup Programming in YARN framework j Understanding big data and Hadoop Big Data Limitations and Solutions of existing Data Analytics Architecture Hadoop Features Hadoop Ecosystem
More informationIBM Data Science Experience White paper. SparkR. Transforming R into a tool for big data analytics
IBM Data Science Experience White paper R Transforming R into a tool for big data analytics 2 R Executive summary This white paper introduces R, a package for the R statistical programming language that
More informationDatabase Applications (15-415)
Database Applications (15-415) SQL-Part III & Storing Data: Disks and Files- Part I Lecture 8, February 5, 2014 Mohammad Hammoud Today Last Session: Standard Query Language (SQL)- Part II Today s Session:
More informationDatabricks, an Introduction
Databricks, an Introduction Chuck Connell, Insight Digital Innovation Insight Presentation Speaker Bio Senior Data Architect at Insight Digital Innovation Focus on Azure big data services HDInsight/Hadoop,
More informationaccess to a JCA connection in WebSphere Application Server
Understanding connection transitions: Avoiding multithreaded access to a JCA connection in WebSphere Application Server Anoop Ramachandra (anramach@in.ibm.com) Senior Staff Software Engineer IBM 09 May
More informationBig data systems 12/8/17
Big data systems 12/8/17 Today Basic architecture Two levels of scheduling Spark overview Basic architecture Cluster Manager Cluster Cluster Manager 64GB RAM 32 cores 64GB RAM 32 cores 64GB RAM 32 cores
More informationVanillaCore Walkthrough Part 1. Introduction to Database Systems DataLab CS, NTHU
VanillaCore Walkthrough Part 1 Introduction to Database Systems DataLab CS, NTHU 1 The Architecture VanillaDB JDBC/SP Interface (at Client Side) Remote.JDBC (Client/Server) Query Interface Remote.SP (Client/Server)
More information1 Big Data Hadoop. 1. Introduction About this Course About Big Data Course Logistics Introductions
Big Data Hadoop Architect Online Training (Big Data Hadoop + Apache Spark & Scala+ MongoDB Developer And Administrator + Apache Cassandra + Impala Training + Apache Kafka + Apache Storm) 1 Big Data Hadoop
More informationSQream Connector JDBC SQream Technologies Version 2.9.3
SQream Connector JDBC 2.9.3 SQream Technologies 2019-03-27 Version 2.9.3 Table of Contents The SQream JDBC Connector - Overview...................................................... 1 1. API Reference............................................................................
More informationBig Data Infrastructure CS 489/698 Big Data Infrastructure (Winter 2017)
Big Data Infrastructure CS 489/698 Big Data Infrastructure (Winter 2017) Week 10: Mutable State (1/2) March 14, 2017 Jimmy Lin David R. Cheriton School of Computer Science University of Waterloo These
More informationInnovatus Technologies
HADOOP 2.X BIGDATA ANALYTICS 1. Java Overview of Java Classes and Objects Garbage Collection and Modifiers Inheritance, Aggregation, Polymorphism Command line argument Abstract class and Interfaces String
More informationmicrosoft
70-775.microsoft Number: 70-775 Passing Score: 800 Time Limit: 120 min Exam A QUESTION 1 Note: This question is part of a series of questions that present the same scenario. Each question in the series
More informationManaging IoT and Time Series Data with Amazon ElastiCache for Redis
Managing IoT and Time Series Data with ElastiCache for Redis Darin Briskman, ElastiCache Developer Outreach Michael Labib, Specialist Solutions Architect 2016, Web Services, Inc. or its Affiliates. All
More informationCOP4540 TUTORIAL PROFESSOR: DR SHU-CHING CHEN TA: H S IN-YU HA
COP4540 TUTORIAL PROFESSOR: DR SHU-CHING CHEN TA: H S IN-YU HA OUTLINE Postgresql installation Introduction of JDBC Stored Procedure POSTGRES INSTALLATION (1) Extract the source file Start the configuration
More informationShark: Hive (SQL) on Spark
Shark: Hive (SQL) on Spark Reynold Xin UC Berkeley AMP Camp Aug 21, 2012 UC BERKELEY SELECT page_name, SUM(page_views) views FROM wikistats GROUP BY page_name ORDER BY views DESC LIMIT 10; Stage 0: Map-Shuffle-Reduce
More informationGridGain and Apache Ignite In-Memory Performance with Durability of Disk
GridGain and Apache Ignite In-Memory Performance with Durability of Disk Dmitriy Setrakyan Apache Ignite PMC GridGain Founder & CPO http://ignite.apache.org #apacheignite Agenda What is GridGain and Ignite
More informationSpark, Shark and Spark Streaming Introduction
Spark, Shark and Spark Streaming Introduction Tushar Kale tusharkale@in.ibm.com June 2015 This Talk Introduction to Shark, Spark and Spark Streaming Architecture Deployment Methodology Performance References
More informationCSE 135. Three-Tier Architecture. Applications Utilizing Databases. Browser. App. Server. Database. Server
CSE 135 Applications Utilizing Databases Three-Tier Architecture Located @ Any PC HTTP Requests Browser HTML Located @ Server 2 App Server JDBC Requests JSPs Tuples Located @ Server 1 Database Server 2
More informationShen PingCAP 2017
Shen Li @ PingCAP About me Shen Li ( 申砾 ) Tech Lead of TiDB, VP of Engineering Netease / 360 / PingCAP Infrastructure software engineer WHY DO WE NEED A NEW DATABASE? Brief History Standalone RDBMS NoSQL
More informationBig Data com Hadoop. VIII Sessão - SQL Bahia. Impala, Hive e Spark. Diógenes Pires 03/03/2018
Big Data com Hadoop Impala, Hive e Spark VIII Sessão - SQL Bahia 03/03/2018 Diógenes Pires Connect with PASS Sign up for a free membership today at: pass.org #sqlpass Internet Live http://www.internetlivestats.com/
More informationApache Ignite - Using a Memory Grid for Heterogeneous Computation Frameworks A Use Case Guided Explanation. Chris Herrera Hashmap
Apache Ignite - Using a Memory Grid for Heterogeneous Computation Frameworks A Use Case Guided Explanation Chris Herrera Hashmap Topics Who - Key Hashmap Team Members The Use Case - Our Need for a Memory
More informationMigrate from Netezza Workload Migration
Migrate from Netezza Automated Big Data Open Netezza Source Workload Migration CASE SOLUTION STUDY BRIEF Automated Netezza Workload Migration To achieve greater scalability and tighter integration with
More informationCertified Big Data Hadoop and Spark Scala Course Curriculum
Certified Big Data Hadoop and Spark Scala Course Curriculum The Certified Big Data Hadoop and Spark Scala course by DataFlair is a perfect blend of indepth theoretical knowledge and strong practical skills
More informationKhadija Souissi. Auf z Systems November IBM z Systems Mainframe Event 2016
Khadija Souissi Auf z Systems 07. 08. November 2016 @ IBM z Systems Mainframe Event 2016 Acknowledgements Apache Spark, Spark, Apache, and the Spark logo are trademarks of The Apache Software Foundation.
More informationA Tutorial on Apache Spark
A Tutorial on Apache Spark A Practical Perspective By Harold Mitchell The Goal Learning Outcomes The Goal Learning Outcomes NOTE: The setup, installation, and examples assume Windows user Learn the following:
More informationAccelerating Spark Workloads using GPUs
Accelerating Spark Workloads using GPUs Rajesh Bordawekar, Minsik Cho, Wei Tan, Benjamin Herta, Vladimir Zolotov, Alexei Lvov, Liana Fong, and David Kung IBM T. J. Watson Research Center 1 Outline Spark
More informationShark. Hive on Spark. Cliff Engle, Antonio Lupher, Reynold Xin, Matei Zaharia, Michael Franklin, Ion Stoica, Scott Shenker
Shark Hive on Spark Cliff Engle, Antonio Lupher, Reynold Xin, Matei Zaharia, Michael Franklin, Ion Stoica, Scott Shenker Agenda Intro to Spark Apache Hive Shark Shark s Improvements over Hive Demo Alpha
More informationA GridGain Systems In-Memory Computing White Paper
A GridGain Systems In-Memory Computing White Paper February 2017 Contents Five Limitations of MySQL... 2 Delivering Hot Data... 2 Dealing with Highly Volatile Data... 3 Handling Large Data Volumes... 3
More information10/18/2017. Announcements. NoSQL Motivation. NoSQL. Serverless Architecture. What is the Problem? Database Systems CSE 414
Announcements Database Systems CSE 414 Lecture 11: NoSQL & JSON (mostly not in textbook only Ch 11.1) HW5 will be posted on Friday and due on Nov. 14, 11pm [No Web Quiz 5] Today s lecture: NoSQL & JSON
More informationPROFESSIONAL. NoSQL. Shashank Tiwari WILEY. John Wiley & Sons, Inc.
PROFESSIONAL NoSQL Shashank Tiwari WILEY John Wiley & Sons, Inc. Examining CONTENTS INTRODUCTION xvil CHAPTER 1: NOSQL: WHAT IT IS AND WHY YOU NEED IT 3 Definition and Introduction 4 Context and a Bit
More informationPerform Database Actions Using Java 8 Stream Syntax Instead of SQL. Emil Forslund Java Developer Speedment, Inc.
Perform Database Actions Using Java 8 Stream Syntax Instead of SQL Emil Forslund Java Developer Speedment, Inc. About Me Emil Forslund Java Developer Speedment Palo Alto Age of Java Why Should You Need
More informationApache Spark is a fast and general-purpose engine for large-scale data processing Spark aims at achieving the following goals in the Big data context
1 Apache Spark is a fast and general-purpose engine for large-scale data processing Spark aims at achieving the following goals in the Big data context Generality: diverse workloads, operators, job sizes
More informationWelcome to the topic of SAP HANA modeling views.
Welcome to the topic of SAP HANA modeling views. 1 At the end of this topic, you will be able to describe the three types of SAP HANA modeling views and use the SAP HANA Studio to work with views in the
More informationSpark. Cluster Computing with Working Sets. Matei Zaharia, Mosharaf Chowdhury, Michael Franklin, Scott Shenker, Ion Stoica.
Spark Cluster Computing with Working Sets Matei Zaharia, Mosharaf Chowdhury, Michael Franklin, Scott Shenker, Ion Stoica UC Berkeley Background MapReduce and Dryad raised level of abstraction in cluster
More informationLecture Notes to Big Data Management and Analytics Winter Term 2017/2018 NoSQL Databases
Lecture Notes to Big Data Management and Analytics Winter Term 2017/2018 NoSQL Databases Matthias Schubert, Matthias Renz, Felix Borutta, Evgeniy Faerman, Christian Frey, Klaus Arthur Schmid, Daniyal Kazempour,
More informationSpark 2. Alexey Zinovyev, Java/BigData Trainer in EPAM
Spark 2 Alexey Zinovyev, Java/BigData Trainer in EPAM With IT since 2007 With Java since 2009 With Hadoop since 2012 With EPAM since 2015 About Secret Word from EPAM itsubbotnik Big Data Training 3 Contacts
More informationCloud Programming on Java EE Platforms. mgr inż. Piotr Nowak
Cloud Programming on Java EE Platforms mgr inż. Piotr Nowak Distributed data caching environment Hadoop Apache Ignite "2 Cache what is cache? how it is used? "3 Cache - hardware buffer temporary storage
More informationAnalytic Cloud with. Shelly Garion. IBM Research -- Haifa IBM Corporation
Analytic Cloud with Shelly Garion IBM Research -- Haifa 2014 IBM Corporation Why Spark? Apache Spark is a fast and general open-source cluster computing engine for big data processing Speed: Spark is capable
More informationOral Questions and Answers (DBMS LAB) Questions & Answers- DBMS
Questions & Answers- DBMS https://career.guru99.com/top-50-database-interview-questions/ 1) Define Database. A prearranged collection of figures known as data is called database. 2) What is DBMS? Database
More informationCSE 344 Final Review. August 16 th
CSE 344 Final Review August 16 th Final In class on Friday One sheet of notes, front and back cost formulas also provided Practice exam on web site Good luck! Primary Topics Parallel DBs parallel join
More informationAccelerate Big Data Insights
Accelerate Big Data Insights Executive Summary An abundance of information isn t always helpful when time is of the essence. In the world of big data, the ability to accelerate time-to-insight can not
More informationJDBC, Transactions. Niklas Fors JDBC 1 / 38
JDBC, Transactions SQL in Programs Embedded SQL and Dynamic SQL JDBC Drivers, Connections, Statements, Prepared Statements Updates, Queries, Result Sets Transactions Niklas Fors (niklas.fors@cs.lth.se)
More informationDistributed ACID Transac2ons in Apache Ignite
Distributed ACID Transac2ons in Apache Ignite Akmal Chaudhri GridGain hbp://ignite.apache.org #apacheignite My Background Pre-2000 Developer Academic (City University) Consultant Technical Architect Post-2000
More informationWhy use a database? You can query the data (run searches) You can integrate with other business systems that use the same database You can store huge
175 Why use a database? You can query the data (run searches) You can integrate with other business systems that use the same database You can store huge numbers of records without the risk of corruption
More informationChair of Software Engineering. Java and C# in Depth. Prof. Dr. Bertrand Meyer. Exercise Session 9. Nadia Polikarpova
Chair of Software Engineering Java and C# in Depth Prof. Dr. Bertrand Meyer Exercise Session 9 Nadia Polikarpova Quiz 1: scrolling a ResultSet (JDBC) How do you assess the following code snippet that iterates
More informationBUSINESS INTELLIGENCE LABORATORY. Data Access: Relational Data Bases. Business Informatics Degree
BUSINESS INTELLIGENCE LABORATORY Data Access: Relational Data Bases Business Informatics Degree RDBMS data access 2 Protocols and API ODBC, OLE DB, ADO, ADO.NET, JDBC JDBC Programming Java classes java.sql
More informationAgenda. Apache Ignite Project Apache Ignite Data Fabric: Data Grid HPC & Compute Streaming & CEP Hadoop & Spark Integration Use Cases Demo Q & A
Introduction 2015 The Apache Software Foundation. Apache, Apache Ignite, the Apache feather and the Apache Ignite logo are trademarks of The Apache Software Foundation. Agenda Apache Ignite Project Apache
More informationUsing the SDACK Architecture to Build a Big Data Product. Yu-hsin Yeh (Evans Ye) Apache Big Data NA 2016 Vancouver
Using the SDACK Architecture to Build a Big Data Product Yu-hsin Yeh (Evans Ye) Apache Big Data NA 2016 Vancouver Outline A Threat Analytic Big Data product The SDACK Architecture Akka Streams and data
More informationLecture 4, 04/08/2015. Scribed by Eric Lax, Andreas Santucci, Charles Zheng.
CME 323: Distributed Algorithms and Optimization, Spring 2015 http://stanford.edu/~rezab/dao. Instructor: Reza Zadeh, Databricks and Stanford. Lecture 4, 04/08/2015. Scribed by Eric Lax, Andreas Santucci,
More informationPouya Kousha Fall 2018 CSE 5194 Prof. DK Panda
Pouya Kousha Fall 2018 CSE 5194 Prof. DK Panda 1 Motivation And Intro Programming Model Spark Data Transformation Model Construction Model Training Model Inference Execution Model Data Parallel Training
More informationGoing Big Data on Apache Spark. KNIME Italy Meetup
Going Big Data on Apache Spark KNIME Italy Meetup Agenda Introduction Why Apache Spark? Section 1 Gathering Requirements Section 2 Tool Choice Section 3 Architecture Section 4 Devising New Nodes Section
More information1
1 2 3 6 7 8 9 10 Storage & IO Benchmarking Primer Running sysbench and preparing data Use the prepare option to generate the data. Experiments Run sysbench with different storage systems and instance
More informationHadoop. Introduction / Overview
Hadoop Introduction / Overview Preface We will use these PowerPoint slides to guide us through our topic. Expect 15 minute segments of lecture Expect 1-4 hour lab segments Expect minimal pretty pictures
More informationWe are ready to serve Latest Testing Trends, Are you ready to learn?? New Batches Info
We are ready to serve Latest Testing Trends, Are you ready to learn?? New Batches Info START DATE : TIMINGS : DURATION : TYPE OF BATCH : FEE : FACULTY NAME : LAB TIMINGS : PH NO: 9963799240, 040-40025423
More information