Introduction to NoSQL Databases

Size: px
Start display at page:

Download "Introduction to NoSQL Databases"

Transcription

1 Introduction to NoSQL Databases Roman Kern KTI, TU Graz Roman Kern (KTI, TU Graz) Dbase / 31

2 Introduction Intro Why NoSQL? Roman Kern (KTI, TU Graz) Dbase / 31

3 Introduction Introduction The birth of NoSQL Term appeared in 2009 Not only SQL Common properties (pros) Non relational Schema-less (schema free) Good scalability Potential down-sides (cons) Limited query abilities Not standardised (evolving technology) Roman Kern (KTI, TU Graz) Dbase / 31

4 Introduction Introduction Motivations for starting NoSQL 1 Growth of data User-generated Machine-generated, eg log-files, sensors Higher degree of connectedness 2 Need for flexibility instead of a rigid schema For semi-structured data (schema-free / schema-less) 3 No separation of data management and data processing Roman Kern (KTI, TU Graz) Dbase / 31

5 Introduction Introduction Data Management vs Data Processing Classic CRUD operations no longer sufficient for advanced data analytics need to combine both functionalities Paradigm shift: Bring the code to the data ie the locality of data is taken into considerations for the data processing Example applications: Online transaction processing (OLTP) relational databases Online analytical processing (OLAP) data warehousing High performance, scalability NoSQL Roman Kern (KTI, TU Graz) Dbase / 31

6 Introduction Introduction Scalability Scale up (scale vertically) vs scale out (scale horizontally) Scale up: Add more hardware to a single machine Scale out: Add more machines Degree of sharing Shared memory (single machine, single storage) Shared disk (multiple machines, single storage) Shared nothing (multiple machines, multiple storage) Roman Kern (KTI, TU Graz) Dbase / 31

7 Introduction Introduction Replication In an distributed system, data is replicated between nodes thus data is stored multiple times Types of replication 1 Synchronous (eager) All data is replicated to all nodes before ending the operation complex, even impossible in some configurations 2 Asynchronous (lazy) Operation is finished before all data has been written by all nodes potentially inconsistent Access for writing options 1 Single node accepts writing of data (master/slave, primary copy) 2 All nodes accept write operations (update anywhere) Roman Kern (KTI, TU Graz) Dbase / 31

8 Introduction Introduction Sharding In an distributed system, each node may be responsible for different parts of the full data still data is replicated for redundancy Also known as: partitioning, fragmentation Advantage: improved efficiency (fewer resources) Types of sharding: 1 Hash-based Hash-key determines partition no data locality 2 Range-based Assigns range (binning) rebalancing needed 3 Entity-group All data from single transactions assigned to a single partition partitions cannot easily change Roman Kern (KTI, TU Graz) Dbase / 31

9 Introduction Introduction ACID vs BASE ACID Atomicity Consistency Isolation Durability BASE Basically Available Soft state Eventually consistent Trade-offs for improved performance Some database systems prefer performance over durability Redundancy for improved performance (no normalisation) Roman Kern (KTI, TU Graz) Dbase / 31

10 Introduction Introduction CAP theorem Not possible to achieve all three properties: Consistent Reads are guaranteed to incorporate all previous writes (all nodes see the same data at the same time) Availability Every query returns an answer, instead of an error (failures do not prevent the remaining system to be operational) Partitioned The systems runs, even if a part of the system is not reachable (eg due to network failure, message loss) Implications of CAP One needs to find a trade-off between the properties, eg choose availability over consistency (as consistency is a major bottleneck for scalability) Roman Kern (KTI, TU Graz) Dbase / 31

11 Introduction Introduction Classification scheme of NoSQL systems 1 According to the data model Key-Values Tabular (wide column) Document Graph Specialised, eg time-series, triples, objects, XML, files, 2 According to the CAP trade-off Available & partition tolerant Consistent & partition tolerant Not partition tolerant 3 According to the replication & sharding types lazy vs eager hash based vs range based vs entity-group Roman Kern (KTI, TU Graz) Dbase / 31

12 Systems What types of NoSQL systems are out there? Roman Kern (KTI, TU Graz) Dbase / 31

13 Systems Distributed File System Data model Folders & files (plus metadata, eg time of creation, ) Interface File system operations Variations Examples NFS, GPFS, HDFS Network File System: (often) single storage Cluster File Systems: (multiple) storage Distributed File Systems: multiple, independent storage Roman Kern (KTI, TU Graz) Dbase / 31

14 Systems Key/Value Store Data model Key Value where the value is a (binary) opaque blob similar to hash-tables Interface CRUD operations Properties Excellent scalability May support redundant storage Examples Amazon Dynamo (AP, lazy, hash-based), Redis (CP, lazy, hash-based), Riak (AP, lazy, hash-based), Memcached (CP), Roman Kern (KTI, TU Graz) Dbase / 31

15 Systems Tabular / Wide Column Data model (Rowkey, Column, Timestamp) Value where the value is a (binary) opaque blob Interface CRUD operations, scan operations Properties Allow vertical and horizontal partitioning adjacent rows are stored closed to other certain columns are stored close to each other, eg via column families Each cell might have multiple versions (timestamps) Examples Cassandra (AP, lazy, hash-based), Google BigTable (CP, eager, range-based), HBase (CP, eager, range-based), Parquet, Roman Kern (KTI, TU Graz) Dbase / 31

16 Systems Example of Cassandra Query Language Roman Kern (KTI, TU Graz) Dbase / 31

17 Systems Document Storage Data model (Collection, Key) Value where the value is understood by the system Interface CRUD operations, specialised queries (eg JavaScript) Properties Documents are schema free, ie no need for schema migrations Documents may also be versioned Documents are often JSON Examples CouchDB (AP, lazy), MongoDB (CP, lazy eager, range-based), Amazon SimpleDB (AP), Cloudant, Rethink (lazy eager, range-based), Roman Kern (KTI, TU Graz) Dbase / 31

18 Systems Key/Value Store vs Document Storage vs Tabular Storage Key/Value store, if requirements are simple Document store, if need to access parts of the value Document store, if documents are independent units Tabular store, if multiple entries (eg rows) are updated at the same time Tabular store, if only certain columns need to be retrieved Things to watch out for Maximum size of value depends on actual implementation Avoid joins for optimal performance Roman Kern (KTI, TU Graz) Dbase / 31

19 Systems Consistency vs Availability vs Partitioning See also: Roman Kern (KTI, TU Graz) Dbase / 31

20 Systems Graph Storage Data model G = (V, E) where each vertex or edge may have additional properties Interface Graph traversals, specialised queries & insert/update methods Properties Optimised for graph traversal, ie no joins needed Types of edges can be specified by the user Examples Neo4J (CA), OrientDB (CA), TitanDB, Giraph, InfiniteGraph (CA), Roman Kern (KTI, TU Graz) Dbase / 31

21 Systems Search Storage Data model documents, metadata often stored as Vector Space Model Interface specialised query languages Properties Documents may consist of multiple fields (facets) field may be structured as well, eg date, integer, strings Fine control over indexing process, ie how each field is indexed Examples Solr, ElasticSearch, Roman Kern (KTI, TU Graz) Dbase / 31

22 Systems Object Oriented Storage Data model classes, objects, relations Interface CRUD, traversal methods Properties Known model from OO programming Often strong coupling between DB system and programming language Examples db4o (Ca), Versant (CA), Objectivity (CA), Roman Kern (KTI, TU Graz) Dbase / 31

23 Systems XML Databases Data model XML, RDF (triples) Interface CRUD, query languages (XQuery, SPARQL, ) Properties RDF based systems often called TripleStore Often used in combination with semantic technologies Examples BaseX, MarkLogic (CA), AllegroGraph (CA), BigData, Roman Kern (KTI, TU Graz) Dbase / 31

24 Systems Timeseries Databases Data model (timestamp) > value Interface CRUD, specialised query languages Variations Properties Type of value is the sample for all entries, typically simple, eg floating point number Complex value type, eg JSON Optimised for time series data, ie small storage requirements Query for time ranges Operations on time series Examples InfluxDB, KairoDB, Roman Kern (KTI, TU Graz) Dbase / 31

25 Systems In-Memory Databases Data model (key) > value but not limited to this model Interface CRUD, specialised query languages Properties Data is stored in RAM Often distributed over multiple machine (RAM is the new Disk) In its purest form does not satisfy durability criteria Examples Hazelcast, Redis, SAP HANA, Roman Kern (KTI, TU Graz) Dbase / 31

26 Systems API & Data Formats NoSQL system often use RESTful APIs Direct match with data model and CRUD operations Serialisation of objects Many techniques used eg Apache Avro, Protocol Buffers, Roman Kern (KTI, TU Graz) Dbase / 31

27 Systems Features Not all NoSQL systems support transactions Instead they support atomic single transactions Therefore not all operations are supported Not all NoSQL systems support security features eg access control Roman Kern (KTI, TU Graz) Dbase / 31

28 Systems Cloud Database Solutions Storage in the internet (cloud) DBaaS - Database as a Service Not limited to NoSQL, traditional SQL are available as well Multi-tenancy as important feature (separation of multiple clients) Private OS - all separate (eg Amazon RDS) Private process - same machine (eg Compose) Private schema - same database (eg Google DataStore) Shared schema - same tables (most SaaS apps) Roman Kern (KTI, TU Graz) Dbase / 31

29 Systems Current State Current state of data storage systems Depending on the actual requirements select a suitable storage solution Or select multiple solutions for each sub-system polyglot persistence Roman Kern (KTI, TU Graz) Dbase / 31

30 Systems Future of Outlook - NewSQL Attempt to achieve consistency and availability for distributed systems Eg Google Spanner, CockroachDB build on the Raft Consensus algorithm relies on specialised hardware Roman Kern (KTI, TU Graz) Dbase / 31

31 Systems The End Next: Graph Databases Credits Scalable Data Management: NoSQL Data Stores in Research and Practice Roman Kern (KTI, TU Graz) Dbase / 31

NOSQL EGCO321 DATABASE SYSTEMS KANAT POOLSAWASD DEPARTMENT OF COMPUTER ENGINEERING MAHIDOL UNIVERSITY

NOSQL EGCO321 DATABASE SYSTEMS KANAT POOLSAWASD DEPARTMENT OF COMPUTER ENGINEERING MAHIDOL UNIVERSITY NOSQL EGCO321 DATABASE SYSTEMS KANAT POOLSAWASD DEPARTMENT OF COMPUTER ENGINEERING MAHIDOL UNIVERSITY WHAT IS NOSQL? Stands for No-SQL or Not Only SQL. Class of non-relational data storage systems E.g.

More information

Overview. * Some History. * What is NoSQL? * Why NoSQL? * RDBMS vs NoSQL. * NoSQL Taxonomy. *TowardsNewSQL

Overview. * Some History. * What is NoSQL? * Why NoSQL? * RDBMS vs NoSQL. * NoSQL Taxonomy. *TowardsNewSQL * Some History * What is NoSQL? * Why NoSQL? * RDBMS vs NoSQL * NoSQL Taxonomy * Towards NewSQL Overview * Some History * What is NoSQL? * Why NoSQL? * RDBMS vs NoSQL * NoSQL Taxonomy *TowardsNewSQL NoSQL

More information

Presented by Sunnie S Chung CIS 612

Presented by Sunnie S Chung CIS 612 By Yasin N. Silva, Arizona State University Presented by Sunnie S Chung CIS 612 This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. See http://creativecommons.org/licenses/by-nc-sa/4.0/

More information

Introduction to Big Data. NoSQL Databases. Instituto Politécnico de Tomar. Ricardo Campos

Introduction to Big Data. NoSQL Databases. Instituto Politécnico de Tomar. Ricardo Campos Instituto Politécnico de Tomar Introduction to Big Data NoSQL Databases Ricardo Campos Mestrado EI-IC Análise e Processamento de Grandes Volumes de Dados Tomar, Portugal, 2016 Part of the slides used in

More information

Chapter 24 NOSQL Databases and Big Data Storage Systems

Chapter 24 NOSQL Databases and Big Data Storage Systems Chapter 24 NOSQL Databases and Big Data Storage Systems - Large amounts of data such as social media, Web links, user profiles, marketing and sales, posts and tweets, road maps, spatial data, email - NOSQL

More information

CompSci 516 Database Systems

CompSci 516 Database Systems CompSci 516 Database Systems Lecture 20 NoSQL and Column Store Instructor: Sudeepa Roy Duke CS, Fall 2018 CompSci 516: Database Systems 1 Reading Material NOSQL: Scalable SQL and NoSQL Data Stores Rick

More information

CISC 7610 Lecture 5 Distributed multimedia databases. Topics: Scaling up vs out Replication Partitioning CAP Theorem NoSQL NewSQL

CISC 7610 Lecture 5 Distributed multimedia databases. Topics: Scaling up vs out Replication Partitioning CAP Theorem NoSQL NewSQL CISC 7610 Lecture 5 Distributed multimedia databases Topics: Scaling up vs out Replication Partitioning CAP Theorem NoSQL NewSQL Motivation YouTube receives 400 hours of video per minute That is 200M hours

More information

Introduction to Computer Science. William Hsu Department of Computer Science and Engineering National Taiwan Ocean University

Introduction to Computer Science. William Hsu Department of Computer Science and Engineering National Taiwan Ocean University Introduction to Computer Science William Hsu Department of Computer Science and Engineering National Taiwan Ocean University Chapter 9: Database Systems supplementary - nosql You can have data without

More information

Non-Relational Databases. Pelle Jakovits

Non-Relational Databases. Pelle Jakovits Non-Relational Databases Pelle Jakovits 25 October 2017 Outline Background Relational model Database scaling The NoSQL Movement CAP Theorem Non-relational data models Key-value Document-oriented Column

More information

Distributed Non-Relational Databases. Pelle Jakovits

Distributed Non-Relational Databases. Pelle Jakovits Distributed Non-Relational Databases Pelle Jakovits Tartu, 7 December 2018 Outline Relational model NoSQL Movement Non-relational data models Key-value Document-oriented Column family Graph Non-relational

More information

CSE 544 Principles of Database Management Systems. Magdalena Balazinska Winter 2015 Lecture 14 NoSQL

CSE 544 Principles of Database Management Systems. Magdalena Balazinska Winter 2015 Lecture 14 NoSQL CSE 544 Principles of Database Management Systems Magdalena Balazinska Winter 2015 Lecture 14 NoSQL References Scalable SQL and NoSQL Data Stores, Rick Cattell, SIGMOD Record, December 2010 (Vol. 39, No.

More information

Goal of the presentation is to give an introduction of NoSQL databases, why they are there.

Goal of the presentation is to give an introduction of NoSQL databases, why they are there. 1 Goal of the presentation is to give an introduction of NoSQL databases, why they are there. We want to present "Why?" first to explain the need of something like "NoSQL" and then in "What?" we go in

More information

Graph and Timeseries Databases

Graph and Timeseries Databases Graph and Timeseries Databases Roman Kern ISDS, TU Graz 2017-10-23 Roman Kern (ISDS, TU Graz) Dbase2 2017-10-23 1 / 31 Graph Databases Graph Databases Motivation and Basics of Graph Databases? Roman Kern

More information

Jargons, Concepts, Scope and Systems. Key Value Stores, Document Stores, Extensible Record Stores. Overview of different scalable relational systems

Jargons, Concepts, Scope and Systems. Key Value Stores, Document Stores, Extensible Record Stores. Overview of different scalable relational systems Jargons, Concepts, Scope and Systems Key Value Stores, Document Stores, Extensible Record Stores Overview of different scalable relational systems Examples of different Data stores Predictions, Comparisons

More information

NoSQL Databases. Amir H. Payberah. Swedish Institute of Computer Science. April 10, 2014

NoSQL Databases. Amir H. Payberah. Swedish Institute of Computer Science. April 10, 2014 NoSQL Databases Amir H. Payberah Swedish Institute of Computer Science amir@sics.se April 10, 2014 Amir H. Payberah (SICS) NoSQL Databases April 10, 2014 1 / 67 Database and Database Management System

More information

Cassandra, MongoDB, and HBase. Cassandra, MongoDB, and HBase. I have chosen these three due to their recent

Cassandra, MongoDB, and HBase. Cassandra, MongoDB, and HBase. I have chosen these three due to their recent Tanton Jeppson CS 401R Lab 3 Cassandra, MongoDB, and HBase Introduction For my report I have chosen to take a deeper look at 3 NoSQL database systems: Cassandra, MongoDB, and HBase. I have chosen these

More information

Database Availability and Integrity in NoSQL. Fahri Firdausillah [M ]

Database Availability and Integrity in NoSQL. Fahri Firdausillah [M ] Database Availability and Integrity in NoSQL Fahri Firdausillah [M031010012] What is NoSQL Stands for Not Only SQL Mostly addressing some of the points: nonrelational, distributed, horizontal scalable,

More information

CIB Session 12th NoSQL Databases Structures

CIB Session 12th NoSQL Databases Structures CIB Session 12th NoSQL Databases Structures By: Shahab Safaee & Morteza Zahedi Software Engineering PhD Email: safaee.shx@gmail.com, morteza.zahedi.a@gmail.com cibtrc.ir cibtrc cibtrc 2 Agenda What is

More information

Introduction Aggregate data model Distribution Models Consistency Map-Reduce Types of NoSQL Databases

Introduction Aggregate data model Distribution Models Consistency Map-Reduce Types of NoSQL Databases Introduction Aggregate data model Distribution Models Consistency Map-Reduce Types of NoSQL Databases Key-Value Document Column Family Graph John Edgar 2 Relational databases are the prevalent solution

More information

Introduction to Graph Databases

Introduction to Graph Databases Introduction to Graph Databases David Montag @dmontag #neo4j 1 Agenda NOSQL overview Graph Database 101 A look at Neo4j The red pill 2 Why you should listen Forrester says: The market for graph databases

More information

Advances in Data Management - NoSQL, NewSQL and Big Data A.Poulovassilis

Advances in Data Management - NoSQL, NewSQL and Big Data A.Poulovassilis Advances in Data Management - NoSQL, NewSQL and Big Data A.Poulovassilis 1 NoSQL So-called NoSQL systems offer reduced functionalities compared to traditional Relational DBMSs, with the aim of achieving

More information

Challenges for Data Driven Systems

Challenges for Data Driven Systems Challenges for Data Driven Systems Eiko Yoneki University of Cambridge Computer Laboratory Data Centric Systems and Networking Emergence of Big Data Shift of Communication Paradigm From end-to-end to data

More information

CSE 344 JULY 9 TH NOSQL

CSE 344 JULY 9 TH NOSQL CSE 344 JULY 9 TH NOSQL ADMINISTRATIVE MINUTIAE HW3 due Wednesday tests released actual_time should have 0s not NULLs upload new data file or use UPDATE to change 0 ~> NULL Extra OOs on Mondays 5-7pm in

More information

CISC 7610 Lecture 2b The beginnings of NoSQL

CISC 7610 Lecture 2b The beginnings of NoSQL CISC 7610 Lecture 2b The beginnings of NoSQL Topics: Big Data Google s infrastructure Hadoop: open google infrastructure Scaling through sharding CAP theorem Amazon s Dynamo 5 V s of big data Everyone

More information

10. Replication. Motivation

10. Replication. Motivation 10. Replication Page 1 10. Replication Motivation Reliable and high-performance computation on a single instance of a data object is prone to failure. Replicate data to overcome single points of failure

More information

NoSQL systems. Lecture 21 (optional) Instructor: Sudeepa Roy. CompSci 516 Data Intensive Computing Systems

NoSQL systems. Lecture 21 (optional) Instructor: Sudeepa Roy. CompSci 516 Data Intensive Computing Systems CompSci 516 Data Intensive Computing Systems Lecture 21 (optional) NoSQL systems Instructor: Sudeepa Roy Duke CS, Spring 2016 CompSci 516: Data Intensive Computing Systems 1 Key- Value Stores Duke CS,

More information

COSC 416 NoSQL Databases. NoSQL Databases Overview. Dr. Ramon Lawrence University of British Columbia Okanagan

COSC 416 NoSQL Databases. NoSQL Databases Overview. Dr. Ramon Lawrence University of British Columbia Okanagan COSC 416 NoSQL Databases NoSQL Databases Overview Dr. Ramon Lawrence University of British Columbia Okanagan ramon.lawrence@ubc.ca Databases Brought Back to Life!!! Image copyright: www.dragoart.com Image

More information

A Study of NoSQL Database

A Study of NoSQL Database A Study of NoSQL Database International Journal of Engineering Research & Technology (IJERT) Biswajeet Sethi 1, Samaresh Mishra 2, Prasant ku. Patnaik 3 1,2,3 School of Computer Engineering, KIIT University

More information

PROFESSIONAL. NoSQL. Shashank Tiwari WILEY. John Wiley & Sons, Inc.

PROFESSIONAL. NoSQL. Shashank Tiwari WILEY. John Wiley & Sons, Inc. PROFESSIONAL NoSQL Shashank Tiwari WILEY John Wiley & Sons, Inc. Examining CONTENTS INTRODUCTION xvil CHAPTER 1: NOSQL: WHAT IT IS AND WHY YOU NEED IT 3 Definition and Introduction 4 Context and a Bit

More information

DATABASE DESIGN II - 1DL400

DATABASE DESIGN II - 1DL400 DATABASE DESIGN II - 1DL400 Fall 2016 A second course in database systems http://www.it.uu.se/research/group/udbl/kurser/dbii_ht16 Kjell Orsborn Uppsala Database Laboratory Department of Information Technology,

More information

Hands-on immersion on Big Data tools

Hands-on immersion on Big Data tools Hands-on immersion on Big Data tools NoSQL Databases Donato Summa THE CONTRACTOR IS ACTING UNDER A FRAMEWORK CONTRACT CONCLUDED WITH THE COMMISSION Summary : Definition Main features NoSQL DBs classification

More information

Sources. P. J. Sadalage, M Fowler, NoSQL Distilled, Addison Wesley

Sources. P. J. Sadalage, M Fowler, NoSQL Distilled, Addison Wesley Big Data and NoSQL Sources P. J. Sadalage, M Fowler, NoSQL Distilled, Addison Wesley Very short history of DBMSs The seventies: IMS end of the sixties, built for the Apollo program (today: Version 15)

More information

Distributed Databases: SQL vs NoSQL

Distributed Databases: SQL vs NoSQL Distributed Databases: SQL vs NoSQL Seda Unal, Yuchen Zheng April 23, 2017 1 Introduction Distributed databases have become increasingly popular in the era of big data because of their advantages over

More information

Distributed Data Store

Distributed Data Store Distributed Data Store Large-Scale Distributed le system Q: What if we have too much data to store in a single machine? Q: How can we create one big filesystem over a cluster of machines, whose data is

More information

10/18/2017. Announcements. NoSQL Motivation. NoSQL. Serverless Architecture. What is the Problem? Database Systems CSE 414

10/18/2017. Announcements. NoSQL Motivation. NoSQL. Serverless Architecture. What is the Problem? Database Systems CSE 414 Announcements Database Systems CSE 414 Lecture 11: NoSQL & JSON (mostly not in textbook only Ch 11.1) HW5 will be posted on Friday and due on Nov. 14, 11pm [No Web Quiz 5] Today s lecture: NoSQL & JSON

More information

A NoSQL Introduction for Relational Database Developers. Andrew Karcher Las Vegas SQL Saturday September 12th, 2015

A NoSQL Introduction for Relational Database Developers. Andrew Karcher Las Vegas SQL Saturday September 12th, 2015 A NoSQL Introduction for Relational Database Developers Andrew Karcher Las Vegas SQL Saturday September 12th, 2015 About Me http://www.andrewkarcher.com Twitter: @akarcher LinkedIn, Twitter Email: akarcher@gmail.com

More information

Introduction to NoSQL

Introduction to NoSQL Introduction to NoSQL Agenda History What is NoSQL Types of NoSQL The CAP theorem History - RDBMS Relational DataBase Management Systems were invented in the 1970s. E. F. Codd, "Relational Model of Data

More information

Database Evolution. DB NoSQL Linked Open Data. L. Vigliano

Database Evolution. DB NoSQL Linked Open Data. L. Vigliano Database Evolution DB NoSQL Linked Open Data Requirements and features Large volumes of data..increasing No regular data structure to manage Relatively homogeneous elements among them (no correlation between

More information

Advances in Data Management - NoSQL, NewSQL and Big Data A.Poulovassilis

Advances in Data Management - NoSQL, NewSQL and Big Data A.Poulovassilis Advances in Data Management - NoSQL, NewSQL and Big Data A.Poulovassilis 1 NoSQL So-called NoSQL systems offer reduced functionalities compared to traditional Relational DBMS, with the aim of achieving

More information

Data Informatics. Seon Ho Kim, Ph.D.

Data Informatics. Seon Ho Kim, Ph.D. Data Informatics Seon Ho Kim, Ph.D. seonkim@usc.edu NoSQL and Big Data Processing Database Relational Databases mainstay of business Web-based applications caused spikes Especially true for public-facing

More information

NoSQL Databases MongoDB vs Cassandra. Kenny Huynh, Andre Chik, Kevin Vu

NoSQL Databases MongoDB vs Cassandra. Kenny Huynh, Andre Chik, Kevin Vu NoSQL Databases MongoDB vs Cassandra Kenny Huynh, Andre Chik, Kevin Vu Introduction - Relational database model - Concept developed in 1970 - Inefficient - NoSQL - Concept introduced in 1980 - Related

More information

CSE 530A. Non-Relational Databases. Washington University Fall 2013

CSE 530A. Non-Relational Databases. Washington University Fall 2013 CSE 530A Non-Relational Databases Washington University Fall 2013 NoSQL "NoSQL" was originally the name of a specific RDBMS project that did not use a SQL interface Was co-opted years later to refer to

More information

NoSQL systems: introduction and data models. Riccardo Torlone Università Roma Tre

NoSQL systems: introduction and data models. Riccardo Torlone Università Roma Tre NoSQL systems: introduction and data models Riccardo Torlone Università Roma Tre Leveraging the NoSQL boom 2 Why NoSQL? In the last fourty years relational databases have been the default choice for serious

More information

Relational databases

Relational databases COSC 6397 Big Data Analytics NoSQL databases Edgar Gabriel Spring 2017 Relational databases Long lasting industry standard to store data persistently Key points concurrency control, transactions, standard

More information

5/2/16. Announcements. NoSQL Motivation. The New Hipster: NoSQL. Serverless. What is the Problem? Database Systems CSE 414

5/2/16. Announcements. NoSQL Motivation. The New Hipster: NoSQL. Serverless. What is the Problem? Database Systems CSE 414 Announcements Database Systems CSE 414 Lecture 16: NoSQL and JSon Current assignments: Homework 4 due tonight Web Quiz 6 due next Wednesday [There is no Web Quiz 5 Today s lecture: JSon The book covers

More information

Database Systems CSE 414

Database Systems CSE 414 Database Systems CSE 414 Lecture 16: NoSQL and JSon CSE 414 - Spring 2016 1 Announcements Current assignments: Homework 4 due tonight Web Quiz 6 due next Wednesday [There is no Web Quiz 5] Today s lecture:

More information

CMU SCS CMU SCS Who: What: When: Where: Why: CMU SCS

CMU SCS CMU SCS Who: What: When: Where: Why: CMU SCS Carnegie Mellon Univ. Dept. of Computer Science 15-415/615 - DB s C. Faloutsos A. Pavlo Lecture#23: Distributed Database Systems (R&G ch. 22) Administrivia Final Exam Who: You What: R&G Chapters 15-22

More information

Advanced Data Management Technologies

Advanced Data Management Technologies ADMT 2017/18 Unit 15 J. Gamper 1/44 Advanced Data Management Technologies Unit 15 Introduction to NoSQL J. Gamper Free University of Bozen-Bolzano Faculty of Computer Science IDSE ADMT 2017/18 Unit 15

More information

GridGain and Apache Ignite In-Memory Performance with Durability of Disk

GridGain and Apache Ignite In-Memory Performance with Durability of Disk GridGain and Apache Ignite In-Memory Performance with Durability of Disk Dmitriy Setrakyan Apache Ignite PMC GridGain Founder & CPO http://ignite.apache.org #apacheignite Agenda What is GridGain and Ignite

More information

CA485 Ray Walshe NoSQL

CA485 Ray Walshe NoSQL NoSQL BASE vs ACID Summary Traditional relational database management systems (RDBMS) do not scale because they adhere to ACID. A strong movement within cloud computing is to utilize non-traditional data

More information

Study of NoSQL Database Along With Security Comparison

Study of NoSQL Database Along With Security Comparison Study of NoSQL Database Along With Security Comparison Ankita A. Mall [1], Jwalant B. Baria [2] [1] Student, Computer Engineering Department, Government Engineering College, Modasa, Gujarat, India ank.fetr@gmail.com

More information

Data Management for Big Data Part 1

Data Management for Big Data Part 1 2018-04-09 2 Outline Today Part 1 Data Management for Big Data Part 1 Valentina Ivanova IDA, Linköping University RDBMS NoSQL NewSQL DBMS OLAP vs OLTP (ACID) NoSQL Concepts and Techniques Horizontal scalability

More information

A Review Of Non Relational Databases, Their Types, Advantages And Disadvantages

A Review Of Non Relational Databases, Their Types, Advantages And Disadvantages A Review Of Non Relational Databases, Their Types, Advantages And Disadvantages Harpreet kaur, Jaspreet kaur, Kamaljit kaur Student of M.Tech(CSE) Student of M.Tech(CSE) Assit.Prof.in CSE deptt. Sri Guru

More information

CS-580K/480K Advanced Topics in Cloud Computing. NoSQL Database

CS-580K/480K Advanced Topics in Cloud Computing. NoSQL Database CS-580K/480K dvanced Topics in Cloud Computing NoSQL Database 1 1 Where are we? Cloud latforms 2 VM1 VM2 VM3 3 Operating System 4 1 2 3 Operating System 4 1 2 Virtualization Layer 3 Operating System 4

More information

Facebook, 14 Fast projection index, 84 First database revolution data handling code, 6 DBMS, 6 network and hierarchical model, 6 7

Facebook, 14 Fast projection index, 84 First database revolution data handling code, 6 DBMS, 6 network and hierarchical model, 6 7 Index A Aerospike, 91, 217 Aerospike query language (AQL), 218 AJAX. See Asynchronous JavaScript and XML (AJAX) Alternative persistence model, 92 Amazon ACID RDBMS, 46 Dynamo, 14, 45 46 DynamoDB, 219 hashing,

More information

OPEN SOURCE DB SYSTEMS TYPES OF DBMS

OPEN SOURCE DB SYSTEMS TYPES OF DBMS OPEN SOURCE DB SYSTEMS Anna Topol 1 TYPES OF DBMS Relational Key-Value Document-oriented Graph 2 DBMS SELECTION Multi-platform or platform-agnostic Offers persistent storage Fairly well known Actively

More information

Why NoSQL? Why Riak?

Why NoSQL? Why Riak? Why NoSQL? Why Riak? Justin Sheehy justin@basho.com 1 What's all of this NoSQL nonsense? Riak Voldemort HBase MongoDB Neo4j Cassandra CouchDB Membase Redis (and the list goes on...) 2 What went wrong with

More information

NOSQL DATABASE SYSTEMS: DECISION GUIDANCE AND TRENDS. Big Data Technologies: NoSQL DBMS (Decision Guidance) - SoSe

NOSQL DATABASE SYSTEMS: DECISION GUIDANCE AND TRENDS. Big Data Technologies: NoSQL DBMS (Decision Guidance) - SoSe NOSQL DATABASE SYSTEMS: DECISION GUIDANCE AND TRENDS h_da Prof. Dr. Uta Störl Big Data Technologies: NoSQL DBMS (Decision Guidance) - SoSe 2017 163 Performance / Benchmarks Traditional database benchmarks

More information

CIT 668: System Architecture. Distributed Databases

CIT 668: System Architecture. Distributed Databases CIT 668: System Architecture Distributed Databases Topics 1. MySQL 2. Concurrency 3. Transactions and ACID 4. Database scaling 5. Replication 6. Partitioning 7. Brewer s CAP Theorem 8. ACID vs. BASE 9.

More information

An Brief Introduction to Data Storage

An Brief Introduction to Data Storage An Brief Introduction to Data Storage Jascha Schewtschenko Institute of Cosmology and Gravitation, University of Portsmouth May 10, 2018 JAS (ICG, Portsmouth) An Brief Introduction to Data Storage May

More information

SESSION TITLE GOES HERE Second Cosmos for Line the Goes Business Here Intelligence Professional

SESSION TITLE GOES HERE Second Cosmos for Line the Goes Business Here Intelligence Professional Azure Cosmos DB with Power BI SESSION TITLE GOES HERE Second Cosmos for Line the Goes Business Here Intelligence Professional Cosmos for the Business Intelligence Professional Speaker Name Speaker Title

More information

NewSQL Databases. The reference Big Data stack

NewSQL Databases. The reference Big Data stack Università degli Studi di Roma Tor Vergata Dipartimento di Ingegneria Civile e Ingegneria Informatica NewSQL Databases Corso di Sistemi e Architetture per Big Data A.A. 2017/18 Valeria Cardellini The reference

More information

The NoSQL Ecosystem. Adam Marcus MIT CSAIL

The NoSQL Ecosystem. Adam Marcus MIT CSAIL The NoSQL Ecosystem Adam Marcus MIT CSAIL marcua@csail.mit.edu / @marcua About Me Social Computing + Database Systems Easily Distracted: Wrote The NoSQL Ecosystem in The Architecture of Open Source Applications

More information

10 Million Smart Meter Data with Apache HBase

10 Million Smart Meter Data with Apache HBase 10 Million Smart Meter Data with Apache HBase 5/31/2017 OSS Solution Center Hitachi, Ltd. Masahiro Ito OSS Summit Japan 2017 Who am I? Masahiro Ito ( 伊藤雅博 ) Software Engineer at Hitachi, Ltd. Focus on

More information

Databases : Lecture 1 2: Beyond ACID/Relational databases Timothy G. Griffin Lent Term Apologies to Martin Fowler ( NoSQL Distilled )

Databases : Lecture 1 2: Beyond ACID/Relational databases Timothy G. Griffin Lent Term Apologies to Martin Fowler ( NoSQL Distilled ) Databases : Lecture 1 2: Beyond ACID/Relational databases Timothy G. Griffin Lent Term 2016 Rise of Web and cluster-based computing NoSQL Movement Relationships vs. Aggregates Key-value store XML or JSON

More information

NoSQL Databases An efficient way to store and query heterogeneous astronomical data in DACE. Nicolas Buchschacher - University of Geneva - ADASS 2018

NoSQL Databases An efficient way to store and query heterogeneous astronomical data in DACE. Nicolas Buchschacher - University of Geneva - ADASS 2018 NoSQL Databases An efficient way to store and query heterogeneous astronomical data in DACE DACE https://dace.unige.ch Data and Analysis Center for Exoplanets. Facility to store, exchange and analyse data

More information

SCALABLE CONSISTENCY AND TRANSACTION MODELS

SCALABLE CONSISTENCY AND TRANSACTION MODELS Data Management in the Cloud SCALABLE CONSISTENCY AND TRANSACTION MODELS 69 Brewer s Conjecture Three properties that are desirable and expected from realworld shared-data systems C: data consistency A:

More information

The Creation of Scalable Tools for Solving Big Data Analysis Problems Based on the MongoDB Database

The Creation of Scalable Tools for Solving Big Data Analysis Problems Based on the MongoDB Database The Creation of Scalable Tools for Solving Big Data Analysis Problems Based on the MongoDB Database O I Vasilchuk 1, A A Nechitaylo 2, D L Savenkov 3 and K S Vasilchuk 4 1 Volga Region State University

More information

Final Exam Logistics. CS 133: Databases. Goals for Today. Some References Used. Final exam take-home. Same resources as midterm

Final Exam Logistics. CS 133: Databases. Goals for Today. Some References Used. Final exam take-home. Same resources as midterm Final Exam Logistics CS 133: Databases Fall 2018 Lec 25 12/06 NoSQL Final exam take-home Available: Friday December 14 th, 4:00pm in Olin Due: Monday December 17 th, 5:15pm Same resources as midterm Except

More information

Lecture Notes to Big Data Management and Analytics Winter Term 2017/2018 NoSQL Databases

Lecture Notes to Big Data Management and Analytics Winter Term 2017/2018 NoSQL Databases Lecture Notes to Big Data Management and Analytics Winter Term 2017/2018 NoSQL Databases Matthias Schubert, Matthias Renz, Felix Borutta, Evgeniy Faerman, Christian Frey, Klaus Arthur Schmid, Daniyal Kazempour,

More information

Highly Scalable, Ultra-Fast and Lots of Choices

Highly Scalable, Ultra-Fast and Lots of Choices Highly Scalable, Ultra-Fast and Lots of Choices A Pattern Approach to NoSQL Tim Wellhausen kontakt@tim-wellhausen.de http://www.tim-wellhausen.de November 07, 2012 Introduction The main motivation to evaluate

More information

NoSQL Systems for Big Data Management

NoSQL Systems for Big Data Management 2014 IEEE 10th World Congress on Services NoSQL Systems for Big Data Management Venkat N Gudivada Weisburg Division of Computer Science Marshall University Huntington, WV, USA gudivada@marshall.edu Dhana

More information

5/1/17. Announcements. NoSQL Motivation. NoSQL. Serverless Architecture. What is the Problem? Database Systems CSE 414

5/1/17. Announcements. NoSQL Motivation. NoSQL. Serverless Architecture. What is the Problem? Database Systems CSE 414 Announcements Database Systems CSE 414 Lecture 15: NoSQL & JSON (mostly not in textbook only Ch 11.1) 1 Homework 4 due tomorrow night [No Web Quiz 5] Midterm grading hopefully finished tonight post online

More information

Big Data Technology Ecosystem. Mark Burnette Pentaho Director Sales Engineering, Hitachi Vantara

Big Data Technology Ecosystem. Mark Burnette Pentaho Director Sales Engineering, Hitachi Vantara Big Data Technology Ecosystem Mark Burnette Pentaho Director Sales Engineering, Hitachi Vantara Agenda End-to-End Data Delivery Platform Ecosystem of Data Technologies Mapping an End-to-End Solution Case

More information

Haridimos Kondylakis Computer Science Department, University of Crete

Haridimos Kondylakis Computer Science Department, University of Crete CS-562 Advanced Topics in Databases Haridimos Kondylakis Computer Science Department, University of Crete QSX (LN2) 2 NoSQL NoSQL: Not Only SQL. User case of NoSQL? Massive write performance. Fast key

More information

NoSQL Databases. Concept, Types & Use-cases.

NoSQL Databases. Concept, Types & Use-cases. NoSQL Databases Concept, Types & Use-cases 1of93 Hello World Alon Spiegel alon@brillix.co.il Mamram grad. Programmer since 1995 DBA since 1997 Co founder and CEO since 2007 Brillix VP products since 2014

More information

SQL, NoSQL, MongoDB. CSE-291 (Cloud Computing) Fall 2016 Gregory Kesden

SQL, NoSQL, MongoDB. CSE-291 (Cloud Computing) Fall 2016 Gregory Kesden SQL, NoSQL, MongoDB CSE-291 (Cloud Computing) Fall 2016 Gregory Kesden SQL Databases Really better called Relational Databases Key construct is the Relation, a.k.a. the table Rows represent records Columns

More information

Intro To Big Data. John Urbanic Parallel Computing Scientist Pittsburgh Supercomputing Center. Copyright 2017

Intro To Big Data. John Urbanic Parallel Computing Scientist Pittsburgh Supercomputing Center. Copyright 2017 Intro To Big Data John Urbanic Parallel Computing Scientist Pittsburgh Supercomputing Center Copyright 2017 Big data is a broad term for data sets so large or complex that traditional data processing applications

More information

Megastore: Providing Scalable, Highly Available Storage for Interactive Services & Spanner: Google s Globally- Distributed Database.

Megastore: Providing Scalable, Highly Available Storage for Interactive Services & Spanner: Google s Globally- Distributed Database. Megastore: Providing Scalable, Highly Available Storage for Interactive Services & Spanner: Google s Globally- Distributed Database. Presented by Kewei Li The Problem db nosql complex legacy tuning expensive

More information

Migrating Oracle Databases To Cassandra

Migrating Oracle Databases To Cassandra BY UMAIR MANSOOB Why Cassandra Lower Cost of ownership makes it #1 choice for Big Data OLTP Applications. Unlike Oracle, Cassandra can store structured, semi-structured, and unstructured data. Cassandra

More information

NoSQL Databases. an overview

NoSQL Databases. an overview NoSQL Databases an overview Who? Why? During studies: Excited by simplicity Crawler Project: 100 Million records Single server 100+ QPS Initially: Limited query options Now: Query them all Experimented

More information

NoSQL Databases. The reference Big Data stack

NoSQL Databases. The reference Big Data stack Università degli Studi di Roma Tor Vergata Dipartimento di Ingegneria Civile e Ingegneria Informatica NoSQL Databases Corso di Sistemi e Architetture per Big Data A.A. 2016/17 Valeria Cardellini The reference

More information

Shen PingCAP 2017

Shen PingCAP 2017 Shen Li @ PingCAP About me Shen Li ( 申砾 ) Tech Lead of TiDB, VP of Engineering Netease / 360 / PingCAP Infrastructure software engineer WHY DO WE NEED A NEW DATABASE? Brief History Standalone RDBMS NoSQL

More information

Making Sense of NoSQL Dan McCreary, Kelly-McCreary & Associates. Minnesota Web Design Community Meetup Monday, February 3rd, :00pm to 8:30pm

Making Sense of NoSQL Dan McCreary, Kelly-McCreary & Associates. Minnesota Web Design Community Meetup Monday, February 3rd, :00pm to 8:30pm Making Sense of NoSQL Dan McCreary, Kelly-McCreary & Associates Minnesota Web Design Community Meetup Monday, February 3rd, 2014 6:00pm to 8:30pm Food for thought What percentage of database transactions

More information

International Journal of Informative & Futuristic Research ISSN:

International Journal of Informative & Futuristic Research ISSN: www.ijifr.com Volume 5 Issue 8 April 2018 International Journal of Informative & Futuristic Research ISSN: 2347-1697 TRANSITION FROM TRADITIONAL DATABASES TO NOSQL DATABASES Paper ID IJIFR/V5/ E8/ 010

More information

NoSQL Databases. CPS352: Database Systems. Simon Miner Gordon College Last Revised: 4/22/15

NoSQL Databases. CPS352: Database Systems. Simon Miner Gordon College Last Revised: 4/22/15 NoSQL Databases CPS352: Database Systems Simon Miner Gordon College Last Revised: 4/22/15 Agenda Check-in NoSQL Databases Aggregate databases Key-value, document, and column family Graph databases Related

More information

Google big data techniques (2)

Google big data techniques (2) Google big data techniques (2) Lecturer: Jiaheng Lu Fall 2016 10.12.2016 1 Outline Google File System and HDFS Relational DB V.S. Big data system Google Bigtable and NoSQL databases 2016/12/10 3 The Google

More information

Motivation Overview of NoSQL space Comparing technologies used Getting hands dirty tutorial section

Motivation Overview of NoSQL space Comparing technologies used Getting hands dirty tutorial section NOSQL 1 GOAL Motivation Overview of NoSQL space Comparing technologies used Getting hands dirty tutorial section 2 MOTIVATION Assume we have a product that becomes popular 3. 1 TYPICAL WEBSERVER ARCHITECTURE

More information

NoSQL data stores and SOS: Uniform Access to Non-Relational Database Systems Paolo Atzeni Francesca Bugiotti Luca Rossi

NoSQL data stores and SOS: Uniform Access to Non-Relational Database Systems Paolo Atzeni Francesca Bugiotti Luca Rossi NoSQL data stores and SOS: Uniform Access to Non-Relational Database Systems Paolo Atzeni Francesca Bugiotti Luca Rossi Outline Context Rela&onal DBMS NoSQL Data Stores NoSQL Timeline NoSQL Data Stores

More information

Rule 14 Use Databases Appropriately

Rule 14 Use Databases Appropriately Rule 14 Use Databases Appropriately Rule 14: What, When, How, and Why What: Use relational databases when you need ACID properties to maintain relationships between your data. For other data storage needs

More information

Introduction to NoSQL by William McKnight

Introduction to NoSQL by William McKnight Introduction to NoSQL by William McKnight All rights reserved. Reproduction in whole or part prohibited except by written permission. Product and company names mentioned herein may be trademarks of their

More information

MySQL Cluster Web Scalability, % Availability. Andrew

MySQL Cluster Web Scalability, % Availability. Andrew MySQL Cluster Web Scalability, 99.999% Availability Andrew Morgan @andrewmorgan www.clusterdb.com Safe Harbour Statement The following is intended to outline our general product direction. It is intended

More information

Class Overview. Two Classes of Database Applications. NoSQL Motivation. RDBMS Review: Client-Server. RDBMS Review: Serverless

Class Overview. Two Classes of Database Applications. NoSQL Motivation. RDBMS Review: Client-Server. RDBMS Review: Serverless Introduction to Database Systems CSE 414 Lecture 12: NoSQL 1 Class Overview Unit 1: Intro Unit 2: Relational Data Models and Query Languages Unit 3: Non-relational data NoSQL Json SQL++ Unit 4: RDMBS internals

More information

SCALABLE DATABASES. Sergio Bossa. From Relational Databases To Polyglot Persistence.

SCALABLE DATABASES. Sergio Bossa. From Relational Databases To Polyglot Persistence. SCALABLE DATABASES From Relational Databases To Polyglot Persistence Sergio Bossa sergio.bossa@gmail.com http://twitter.com/sbtourist About Me Software architect and engineer Gioco Digitale (online gambling

More information

Modern Database Concepts

Modern Database Concepts Modern Database Concepts Basic Principles Doc. RNDr. Irena Holubova, Ph.D. holubova@ksi.mff.cuni.cz NoSQL Overview Main objective: to implement a distributed state Different objects stored on different

More information

CockroachDB on DC/OS. Ben Darnell, CTO, Cockroach Labs

CockroachDB on DC/OS. Ben Darnell, CTO, Cockroach Labs CockroachDB on DC/OS Ben Darnell, CTO, Cockroach Labs Agenda A cloud-native database CockroachDB on DC/OS Why CockroachDB Demo! Cloud-Native Database What is Cloud-Native? Horizontally scalable Individual

More information

Accelerate MySQL for Demanding OLAP and OLTP Use Cases with Apache Ignite. Peter Zaitsev, Denis Magda Santa Clara, California April 25th, 2017

Accelerate MySQL for Demanding OLAP and OLTP Use Cases with Apache Ignite. Peter Zaitsev, Denis Magda Santa Clara, California April 25th, 2017 Accelerate MySQL for Demanding OLAP and OLTP Use Cases with Apache Ignite Peter Zaitsev, Denis Magda Santa Clara, California April 25th, 2017 About the Presentation Problems Existing Solutions Denis Magda

More information

AN introduction to nosql databases

AN introduction to nosql databases AN introduction to nosql databases Terry McCann @SQLshark Purpose of this presentation? It is important for a data scientist / data engineer to have the right tool for the right job. We will look at an

More information

Large-Scale Key-Value Stores Eventual Consistency Marco Serafini

Large-Scale Key-Value Stores Eventual Consistency Marco Serafini Large-Scale Key-Value Stores Eventual Consistency Marco Serafini COMPSCI 590S Lecture 13 Goals of Key-Value Stores Export simple API put(key, value) get(key) Simpler and faster than a DBMS Less complexity,

More information

Comparing SQL and NOSQL databases

Comparing SQL and NOSQL databases COSC 6397 Big Data Analytics Data Formats (II) HBase Edgar Gabriel Spring 2014 Comparing SQL and NOSQL databases Types Development History Data Storage Model SQL One type (SQL database) with minor variations

More information