Redis Func+ons and Data Structures

Size: px
Start display at page:

Download "Redis Func+ons and Data Structures"

Transcription

1 Redis Func+ons and Data Structures

2 About This Talk Topic : Redis Func/ons and Data Structures Presenter: Redis Labs, the open source home and provider of enterprise Redis About Redis Labs: paying customers, 30k free customers 118k Redis databases managed Withstood several datacenter outages with no loss of data for customers who chose HA op/ons Two main products: Redis Cloud, Redis Labs Enterprise Cluster 2

3 About Redis # 1 # 1 # 1 # 12 NoSQL in User Sa/sfac/on and Market Presence [@G2CROWD] In growth among NoSQL databases [@DB- Engines] NoSQL database on containers [@DevOps.com & ClusterHQ] Out of 50 tools developers love to use [@Stackshare] Redis has the largest open source community among the NoSQL databases Created by Salvatore Sanfilippo (@an/rez) 3

4 Redis A Unique Database Concept Redis is an open source (BSD licensed), in- memory, data structure store, used as database, cache and message broker 4

5 Redis : A Data Structure Store Strings Hashes Lists Sets Sorted Sets Bit Arrays Hyper- LogLogs Geospa+al indexes Data structures are used like Lego building blocks, saving developers coding effort and +me 5

6 Mul+ple Use Cases & Ver+cal Markets USE CASES Business Applica/ons Loca/on- based Applica/ons Accelerator for Hadoop/Storm/Spark Any Business Applica/on Any Web or Mobile App High Speed Transac+ons Geo Search Analy+cs Job & Queue Caching VERTICALS High speed delivery of prices/ transac/ons Real- /me ad placements, personaliza/on Items Viewed, Similar Purchases, Top trends High speed data inges/on buffer for IoT Timeline, social graph, top followers, following Real /me analy/cs for leaderboards, dashboards, messaging Financial Services Adver+sing Retail/Ecommerce Manufacturing Social Gaming 6

7 What Can You Do With Redis? Use as in- memory database, cache or message broker Top Uses Leaderboards User session management Mutual Friends Message Broker/Queues Latest lis/ngs/posts Many more 7

8 Redis For User Sessions The Applica+on Problem Why Redis Rocks This Need to maintain state across mul/ple hdp requests Any number of session variables High speed and low latency requirements Hashes are wildly useful in this case HSET, HGET will let you save all session variables as key value pairs HINCRBY will let you increment any field within a hash structure 8

9 Redis Hashes Hashkey : User session 1 userid 8754 uservar1 xx userip 10:20:104:3 1.. HSET usersession:1 userid 8754 uservar1 xx.. HGET usersession:1 uservar1 HINCRBY 100 usersession1 userpoints HGET HDEL Hashes store a mapping of keys to values like a dic/onary or associa/ve array but of course, faster 9

10 Redis For Managing Queues of Work The Applica+on Problem Tasks need to be worked on asynchronously to reduce block/ wait /mes Lots of items to be worked on in a queue Assign items to worker process and remove from queue at the same /me Similar to buffering high speed data- inges/on Why Redis Rocks This Redis Lists are ordered collec/ons of strings in the order of inser/on or as specified RPOPLPUSH pops an item from one queue and pushes it to the same/different queue 10

11 Redis Lists LPUSH queue1 yellow LPUSH queue1 green LPUSH queue 1 blue RPUSH queue1 red RPUSH adds values at the tail of a list blue green yellow.... red LPUSH adds values at the head of a list 11

12 Redis Lists blue green yellow.... red RPOPLPUSH pops an item from one list and pushes it to the same/ another 12

13 Redis For Managing Tags Also can be used for friends of friends, mutual friends and exclusive friends The Applica+on Problem Loads of tags Find items with par/cular tags High speed and low latency requirements Why Redis Rocks This Redis SETs are unordered collec/ons of strings- SADD to add objects to each tag SISMEMBER to check if an object has a given tag SMEMBERS to get all the objects for a tag or all the tags for an object SINTER to find which items are in tag1, tag2 and also tag77 13

14 Redis Sets ar/cle 1 SADD tag:1 ar/cle:1 SADD ar/cle:1 tag:1 tag:2.. tag 1 tag 22 tag 24 tag 28 tag 57 tag 69 tag 1 ar/cle 1 ar/cle 3. SISMEMBER ar/cle:1 tag:x SMEMBERS ar/cle:1 tag 2 ar/cle 22 ar/cle 14 ar/cle 3.. SINTER tag1 tag2 ar/cle:3 14

15 Redis For Leaderboards The Applica+on Problem Why Redis Rocks This Many many users playing a game or collec/ng points Need to instantly show who s leading, in what order and by how many points Disk- based DBMS- es are too slow for this Sorted sets automa/cally keep list of users and scores updated and in order (ZADD) ZRANGE, ZREVRANGE will get your top users ZRANK will get any users rank instantaneously 15

16 Redis Sorted Sets id: ZADD game: id: id: 2 ZADD game: id: id 4 ZINCRBY game: id: id:4 ZREVRANGE game:1 0 0 id: id: id:1 16

17 So What? Data structures are en/rely in memory and blazingly fast Simplicity and speed make Redis easy to use Persistence, scale, high availability enable massive uses 17

18 Next Steps Enter to win a megadrone! Join our online group: Add your business cards to the fishbowl 18

19 Learn More.. redis.io à Open Source Redis redislabs.com à Redis Cloud & RLEC redislabs.com/ebook à Redis in Ac/on ebook campsite.org/redislabs à Join us online 19

20 Home of Redis Thank You!

Redis Functions and Data Structures at Percona Live. Dave Nielsen, Developer Redis

Redis Functions and Data Structures at Percona Live. Dave Nielsen, Developer Redis Redis Functions and Data Structures at Percona Live Dave Nielsen, Developer Advocate dave@redislabs.com @davenielsen Redis Labs @redislabs Redis = A Unique Database Redis is an open source (BSD licensed),

More information

LECTURE 27. Python and Redis

LECTURE 27. Python and Redis LECTURE 27 Python and Redis PYTHON AND REDIS Today, we ll be covering a useful but not entirely Python-centered topic: the inmemory datastore Redis. We ll start by introducing Redis itself and then discussing

More information

Revolutionizing the Datacenter Join the Conversation #OpenPOWERSummit

Revolutionizing the Datacenter Join the Conversation #OpenPOWERSummit Redis Labs on POWER8 Server: The Promise of OpenPOWER Value Jeffrey L. Leeds, Ph.D. Vice President, Alliances & Channels Revolutionizing the Datacenter Join the Conversation #OpenPOWERSummit Who We Are

More information

Home of Redis. April 24, 2017

Home of Redis. April 24, 2017 Home of Redis April 24, 2017 Introduction to Redis and Redis Labs Redis with MySQL Data Structures in Redis Benefits of Redis e 2 Redis and Redis Labs Open source. The leading in-memory database platform,

More information

Jason Brelloch and William Gimson

Jason Brelloch and William Gimson Jason Brelloch and William Gimson Overview 1. Introduction 2. History 3. Specifications a. Structure b. Communication c. Datatypes 4. Command Overview 5. Advanced Capabilities 6. Advantages 7. Disadvantages

More information

Buffering to Redis for Efficient Real-Time Processing. Percona Live, April 24, 2018

Buffering to Redis for Efficient Real-Time Processing. Percona Live, April 24, 2018 Buffering to Redis for Efficient Real-Time Processing Percona Live, April 24, 2018 Presenting Today Jon Hyman CTO & Co-Founder Braze (Formerly Appboy) @jon_hyman Mobile is at the vanguard of a new wave

More information

Amritansh Sharma

Amritansh Sharma 17.12.2018 Amritansh Sharma - 000473628 1 CONTENTS 1 Introduction and Background 3 1.1 Relational Databases 3 1.2 NoSQL Databases 4 1.3 Key Value Stores 5 2 Redis 7 2.1 Redis vs Other Key-Value Stores

More information

4 Myths about in-memory databases busted

4 Myths about in-memory databases busted 4 Myths about in-memory databases busted Yiftach Shoolman Co-Founder & CTO @ Redis Labs @yiftachsh, @redislabsinc Background - Redis Created by Salvatore Sanfilippo (@antirez) OSS, in-memory NoSQL k/v

More information

NoSQL: Redis and MongoDB A.A. 2016/17

NoSQL: Redis and MongoDB A.A. 2016/17 Università degli Studi di Roma Tor Vergata Dipartimento di Ingegneria Civile e Ingegneria Informatica NoSQL: Redis and MongoDB A.A. 2016/17 Matteo Nardelli Laurea Magistrale in Ingegneria Informatica -

More information

Database Solution in Cloud Computing

Database Solution in Cloud Computing Database Solution in Cloud Computing CERC liji@cnic.cn Outline Cloud Computing Database Solution Our Experiences in Database Cloud Computing SaaS Software as a Service PaaS Platform as a Service IaaS Infrastructure

More information

Harnessing the Full power of Redis. Daniel Magliola

Harnessing the Full power of Redis. Daniel Magliola Harnessing the Full power of Redis Daniel Magliola daniel@danielmagliola.com http://danielmagliola.com What is Redis? Redis is essentially like Memcached, but better I mean, it s an in-memory key-value

More information

Redis as a Time Series DB. Josiah Carlson

Redis as a Time Series DB. Josiah Carlson Redis as a Time Series DB Josiah Carlson - @dr_josiah Agenda Who are you? What is Redis? (3 minutes, optional) What is a time series database? Combining structures for success Analyzing/segmenting events

More information

Redis - a Flexible Key/Value Datastore An Introduction

Redis - a Flexible Key/Value Datastore An Introduction Redis - a Flexible Key/Value Datastore An Introduction Alexandre Dulaunoy AIMS 2011 MapReduce and Network Forensic MapReduce is an old concept in computer science The map stage to perform isolated computation

More information

INFO-H-415 Advanced Databases Key-value stores and Redis. Fatemeh Shafiee Raisa Uku

INFO-H-415 Advanced Databases Key-value stores and Redis. Fatemeh Shafiee Raisa Uku INFO-H-415 Advanced Databases Key-value stores and Redis Fatemeh Shafiee 000454718 Raisa Uku 000456485 December 2017 Contents 1 Introduction 5 2 NoSQL Databases 5 2.1 Introduction to NoSQL Databases...............................

More information

REDIS: NOSQL DATA STORAGE

REDIS: NOSQL DATA STORAGE REDIS: NOSQL DATA STORAGE 1 Presented By- Shalini Somani (13MCEC22) CATEGORIES OF NOSQL STORAGES Key-Value memcached Redis Column Family Cassandra HBase Document MongoDB Tabular BigTable Graph, XML, Object,

More information

Dr. Chuck Cartledge. 19 Nov. 2015

Dr. Chuck Cartledge. 19 Nov. 2015 CS-695 NoSQL Database Redis (part 1 of 2) Dr. Chuck Cartledge 19 Nov. 2015 1/21 Table of contents I 1 Miscellanea 2 DB comparisons 3 Assgn. #7 4 Historical origins 5 Data model 6 CRUDy stuff 7 Other operations

More information

Agenda. Introduction You Me JDriven The case : Westy Tracking Details Implementation Deployment

Agenda. Introduction You Me JDriven The case : Westy Tracking Details Implementation Deployment y t s e W g n i k c a r T e p o r u mmit E u S y r d un o F 6 d 1 u h t Clo 8 2 r e b m Septe Agenda Introduction You Me JDriven The case : Westy Tracking Details Implementation Deployment About you About

More information

How you can benefit from using. javier

How you can benefit from using. javier How you can benefit from using I was Lois Lane redis has super powers myth: the bottleneck redis-benchmark -r 1000000 -n 2000000 -t get,set,lpush,lpop,mset -P 16 -q On my laptop: SET: 513610 requests

More information

Agenda. About Me JDriven The case : Westy Tracking Background Architecture Implementation Demo

Agenda. About Me JDriven The case : Westy Tracking Background Architecture Implementation Demo y t s e W g n i k c a r T and l a v a j # 6 1 h c r 8 Ma Agenda About Me JDriven The case : Westy Tracking Background Architecture Implementation Demo About me Proud dad of two kids and a '82 VW Westy

More information

Using Redis As a Time Series Database

Using Redis As a Time Series Database WHITE PAPER Using Redis As a Time Series Database Dr.Josiah Carlson, Author of Redis in Action CONTENTS Executive Summary 2 Use Cases 2 Advanced Analysis Using a Sorted Set with Hashes 2 Event Analysis

More information

Big Data Management and NoSQL Databases

Big Data Management and NoSQL Databases NDBI040 Big Data Management and NoSQL Databases Lecture 5. Key-value stores Doc. RNDr. Irena Holubova, Ph.D. holubova@ksi.mff.cuni.cz http://www.ksi.mff.cuni.cz/~holubova/ndbi040/ Key-value store Basic

More information

whitepaper Using Redis As a Time Series Database: Why and How

whitepaper Using Redis As a Time Series Database: Why and How whitepaper Using Redis As a Time Series Database: Why and How Author: Dr.Josiah Carlson, Author of Redis in Action Table of Contents Executive Summary 2 A Note on Race Conditions and Transactions 2 Use

More information

My Other Car is a Redis. Etan Grundstein & Sasha Popov DYNAMIC YIELD

My Other Car is a Redis. Etan Grundstein & Sasha Popov DYNAMIC YIELD My Other Car is a Redis Etan Grundstein & Sasha Popov DYNAMIC YIELD About Dynamic Yield Dynamic Yield helps marketers increase revenue by personalizing customer interactions across web, mobile web, mobile

More information

COSC Redis. Paul Moore, Stephen Smithbower, William Lee. March 11, 2013

COSC Redis. Paul Moore, Stephen Smithbower, William Lee. March 11, 2013 March 11, 2013 What is Redis? - Redis is an in-memory key-value data store. - Can be a middle-ware solution between your expensive persistent data-store (Oracle), and your application. - Provides PubSub,

More information

IERG 4080 Building Scalable Internet-based Services

IERG 4080 Building Scalable Internet-based Services Department of Information Engineering, CUHK Term 1, 2016/17 IERG 4080 Building Scalable Internet-based Services Lecture 7 Asynchronous Tasks and Message Queues Lecturer: Albert C. M. Au Yeung 20 th & 21

More information

NoSQL Databases Analysis

NoSQL Databases Analysis NoSQL Databases Analysis Jeffrey Young Intro I chose to investigate Redis, MongoDB, and Neo4j. I chose Redis because I always read about Redis use and its extreme popularity yet I know little about it.

More information

Beating the Final Boss: Launch your game!

Beating the Final Boss: Launch your game! Beating the Final Boss: Launch your game! Ozkan Can Solutions Architect, AWS @_ozkancan ERROR The servers are busy at this time. Please try again later. (Error Code: 42 OOPS) Retry READY FOR LAUNCH?! WORST-CASE

More information

Redisco Documentation

Redisco Documentation Redisco Documentation Release rc3 Sebastien Requiem November 29, 206 Contents Object Relation Manager 3. Model................................................... 3.2 Attributes.................................................

More information

Managing IoT and Time Series Data with Amazon ElastiCache for Redis

Managing IoT and Time Series Data with Amazon ElastiCache for Redis Managing IoT and Time Series Data with ElastiCache for Redis Darin Briskman, ElastiCache Developer Outreach Michael Labib, Specialist Solutions Architect 2016, Web Services, Inc. or its Affiliates. All

More information

IEMS 5722 Mobile Network Programming and Distributed Server Architecture

IEMS 5722 Mobile Network Programming and Distributed Server Architecture Department of Information Engineering, CUHK MScIE 2 nd Semester, 2016/17 IEMS 5722 Mobile Network Programming and Distributed Server Architecture Lecture 9 Asynchronous Tasks & Message Queues Lecturer:

More information

Redis as a Reliable Work Queue. Percona University

Redis as a Reliable Work Queue. Percona University Redis as a Reliable Work Queue Percona University 2015-02-12 Introduction Tom DeWire Principal Software Engineer Bronto Software Chris Thunes Senior Software Engineer Bronto Software Introduction Introduction

More information

Redis Ma as, greitas, galingas. Specialiai VilniusPHP

Redis Ma as, greitas, galingas. Specialiai VilniusPHP Redis Ma as, greitas, galingas Specialiai VilniusPHP 2013.06.06 Sergej Kurakin Na, Jūs mane jau nekarta matėte, tai nieko nesakysiu apie save. Kaip aš susipa inau! Tai buvo prieš keletą metų! Projektas

More information

CS6200 Informa.on Retrieval. David Smith College of Computer and Informa.on Science Northeastern University

CS6200 Informa.on Retrieval. David Smith College of Computer and Informa.on Science Northeastern University CS6200 Informa.on Retrieval David Smith College of Computer and Informa.on Science Northeastern University Indexing Process Indexes Indexes are data structures designed to make search faster Text search

More information

Informa)on Retrieval and Map- Reduce Implementa)ons. Mohammad Amir Sharif PhD Student Center for Advanced Computer Studies

Informa)on Retrieval and Map- Reduce Implementa)ons. Mohammad Amir Sharif PhD Student Center for Advanced Computer Studies Informa)on Retrieval and Map- Reduce Implementa)ons Mohammad Amir Sharif PhD Student Center for Advanced Computer Studies mas4108@louisiana.edu Map-Reduce: Why? Need to process 100TB datasets On 1 node:

More information

Distributed Systems INF Michael Welzl

Distributed Systems INF Michael Welzl Distributed Systems INF 3190 Michael Welzl What is a distributed system (DS)? Many defini8ons [Coulouris & Emmerich] A distributed system consists of hardware and sodware components located in a network

More information

INFO-H-415 Advanced Databases

INFO-H-415 Advanced Databases INFO-H-415 Advanced Databases Prof. Esteban Zimanyi In-memory Databases and Redis Larissa Leite Alexandr Tretyak Brussels, December 2015 In-memory databases An in-memory database system is a database management

More information

pyredis Documentation

pyredis Documentation pyredis Documentation Release 0.0.5 Stephan Schultchen Jul 09, 2018 Contents 1 Introduction 3 1.1 Installing................................................. 3 1.2 Author..................................................

More information

Generalizing Map- Reduce

Generalizing Map- Reduce Generalizing Map- Reduce 1 Example: A Map- Reduce Graph map reduce map... reduce reduce map 2 Map- reduce is not a solu;on to every problem, not even every problem that profitably can use many compute

More information

h7ps://bit.ly/citustutorial

h7ps://bit.ly/citustutorial Before We Start Setup a Citus Cloud account for the exercises: h7ps://bit.ly/citustutorial Designing a Mul

More information

Microservice Layout in Netflix

Microservice Layout in Netflix Microservice Layout in Netflix Polyglot Persistence Powering Microservices Roopa Tangirala Engineering Manager Netflix Agenda 5 Use Cases Challenges Current Approach Takeaway AWS S3 CDE Search,

More information

Developing Enterprise Cloud Solutions with Azure

Developing Enterprise Cloud Solutions with Azure Developing Enterprise Cloud Solutions with Azure Java Focused 5 Day Course AUDIENCE FORMAT Developers and Software Architects Instructor-led with hands-on labs LEVEL 300 COURSE DESCRIPTION This course

More information

Opera&ng Systems ECE344

Opera&ng Systems ECE344 Opera&ng Systems ECE344 Lecture 8: Paging Ding Yuan Lecture Overview Today we ll cover more paging mechanisms: Op&miza&ons Managing page tables (space) Efficient transla&ons (TLBs) (&me) Demand paged virtual

More information

Document Databases: MongoDB

Document Databases: MongoDB NDBI040: Big Data Management and NoSQL Databases hp://www.ksi.mff.cuni.cz/~svoboda/courses/171-ndbi040/ Lecture 9 Document Databases: MongoDB Marn Svoboda svoboda@ksi.mff.cuni.cz 28. 11. 2017 Charles University

More information

THE FLEXIBLE DATA-STRUCTURE SERVER THAT COULD.

THE FLEXIBLE DATA-STRUCTURE SERVER THAT COULD. REDIS THE FLEXIBLE DATA-STRUCTURE SERVER THAT COULD. @_chriswhitten_ REDIS REDIS April 10, 2009; 6 years old Founding Author: Salvatore Sanfilippo Stable release: 3.0.3 / June 4, 2015; 3 months ago Fundamental

More information

Classifying malware using network traffic analysis. Or how to learn Redis, git, tshark and Python in 4 hours.

Classifying malware using network traffic analysis. Or how to learn Redis, git, tshark and Python in 4 hours. Classifying malware using network traffic analysis. Or how to learn Redis, git, tshark and Python in 4 hours. Alexandre Dulaunoy January 9, 2015 Problem Statement We have more 5000 pcap files generated

More information

Scalable Time Series in PCP. Lukas Berk

Scalable Time Series in PCP. Lukas Berk Scalable Time Series in PCP Lukas Berk Summary Problem Statement Proposed Solution Redis Basic Types Summary Current Work Future Work Items Problem Statement Scaling PCP s metrics querying to hundreds/thousands

More information

MapReduce, Apache Hadoop

MapReduce, Apache Hadoop NDBI040: Big Data Management and NoSQL Databases hp://www.ksi.mff.cuni.cz/ svoboda/courses/2016-1-ndbi040/ Lecture 2 MapReduce, Apache Hadoop Marn Svoboda svoboda@ksi.mff.cuni.cz 11. 10. 2016 Charles University

More information

Table of Contents GEEK GUIDE TAKE CONTROL OF GROWING REDIS NOSQL SERVER CLUSTERS

Table of Contents GEEK GUIDE TAKE CONTROL OF GROWING REDIS NOSQL SERVER CLUSTERS Table of Contents About the Sponsor... 4 Introduction... 5 Working with Redis... 10 Single-Server Redis... 15 Multi-Server Redis and Replication... 18 IBM s CAPI Technology... 21 Using Redis with CAPI...

More information

CSc 120. Introduction to Computer Programming II. 14: Stacks and Queues. Adapted from slides by Dr. Saumya Debray

CSc 120. Introduction to Computer Programming II. 14: Stacks and Queues. Adapted from slides by Dr. Saumya Debray CSc 120 Introduction to Computer Programming II Adapted from slides by Dr. Saumya Debray 14: Stacks and Queues linear data structures 2 Linear data structures A linear data structure is a collec6on of

More information

Evolving the Key/Value Programming Model to a Higher Level

Evolving the Key/Value Programming Model to a Higher Level Evolving the Key/Value Programming Model to a Higher Level Billy Newport (@billynewport) IBM Dis6nguished Engineer IBM WebSphere extreme Scale Chief Architect Redis Redis is a prefy cool KV (key/value)

More information

MapReduce, Apache Hadoop

MapReduce, Apache Hadoop Czech Technical University in Prague, Faculty of Informaon Technology MIE-PDB: Advanced Database Systems hp://www.ksi.mff.cuni.cz/~svoboda/courses/2016-2-mie-pdb/ Lecture 12 MapReduce, Apache Hadoop Marn

More information

@ COUCHBASE CONNECT. Using Couchbase. By: Carleton Miyamoto, Michael Kehoe Version: 1.1w LinkedIn Corpora3on

@ COUCHBASE CONNECT. Using Couchbase. By: Carleton Miyamoto, Michael Kehoe Version: 1.1w LinkedIn Corpora3on @ COUCHBASE CONNECT Using Couchbase By: Carleton Miyamoto, Michael Kehoe Version: 1.1w Overview The LinkedIn Story Enter Couchbase Development and Opera3ons Clusters and Numbers Opera3onal Tooling Carleton

More information

Raj Jain (Washington University in Saint Louis) Mohammed Samaka (Qatar University)

Raj Jain (Washington University in Saint Louis) Mohammed Samaka (Qatar University) APPLICATION DEPLOYMENT IN FUTURE GLOBAL MULTI-CLOUD ENVIRONMENT Raj Jain (Washington University in Saint Louis) Mohammed Samaka (Qatar University) GITMA 2015 Conference, St. Louis, June 23, 2015 These

More information

ICALEPS 2013 Exploring No-SQL Alternatives for ALMA Monitoring System ADC

ICALEPS 2013 Exploring No-SQL Alternatives for ALMA Monitoring System ADC ICALEPS 2013 Exploring No-SQL Alternatives for ALMA Monitoring System Overview The current paradigm (CCL and Relational DataBase) Propose of a new monitor data system using NoSQL Monitoring Storage Requirements

More information

Direc>ons in Distributed Compu>ng

Direc>ons in Distributed Compu>ng Direc>ons in Distributed Compu>ng Robert Shimp Group Vice President August 23, 2016 Copyright 2016 Oracle and/or its affiliates. All rights reserved. Safe Harbor Statement The following is intended to outline

More information

CIB Session 12th NoSQL Databases Structures

CIB Session 12th NoSQL Databases Structures CIB Session 12th NoSQL Databases Structures By: Shahab Safaee & Morteza Zahedi Software Engineering PhD Email: safaee.shx@gmail.com, morteza.zahedi.a@gmail.com cibtrc.ir cibtrc cibtrc 2 Agenda What is

More information

Database design and implementation CMPSCI 645. Lecture 08: Storage and Indexing

Database design and implementation CMPSCI 645. Lecture 08: Storage and Indexing Database design and implementation CMPSCI 645 Lecture 08: Storage and Indexing 1 Where is the data and how to get to it? DB 2 DBMS architecture Query Parser Query Rewriter Query Op=mizer Query Executor

More information

O Reilly RailsConf,

O Reilly RailsConf, O Reilly RailsConf, 2011-05- 18 Who is that guy? Jesper Richter- Reichhelm / @jrirei Berlin, Germany Head of Engineering @ wooga Wooga does social games Wooga has dedicated game teams Cooming soon PHP

More information

Building High Performance Apps using NoSQL. Swami Sivasubramanian General Manager, AWS NoSQL

Building High Performance Apps using NoSQL. Swami Sivasubramanian General Manager, AWS NoSQL Building High Performance Apps using NoSQL Swami Sivasubramanian General Manager, AWS NoSQL Building high performance apps There is a lot to building high performance apps Scalability Performance at high

More information

Search Engines. Informa1on Retrieval in Prac1ce. Annotations by Michael L. Nelson

Search Engines. Informa1on Retrieval in Prac1ce. Annotations by Michael L. Nelson Search Engines Informa1on Retrieval in Prac1ce Annotations by Michael L. Nelson All slides Addison Wesley, 2008 Indexes Indexes are data structures designed to make search faster Text search has unique

More information

Functionality, Challenges and Architecture of Social Networks

Functionality, Challenges and Architecture of Social Networks Functionality, Challenges and Architecture of Social Networks INF 5370 Outline Social Network Services Functionality Business Model Current Architecture and Scalability Challenges Conclusion 1 Social Network

More information

Home of Redis. Redis for Fast Data Ingest

Home of Redis. Redis for Fast Data Ingest Home of Redis Redis for Fast Data Ingest Agenda Fast Data Ingest and its challenges Redis for Fast Data Ingest Pub/Sub List Sorted Sets as a Time Series Database The Demo Scaling with Redis e Flash 2 Fast

More information

Outline. Spanner Mo/va/on. Tom Anderson

Outline. Spanner Mo/va/on. Tom Anderson Spanner Mo/va/on Tom Anderson Outline Last week: Chubby: coordina/on service BigTable: scalable storage of structured data GFS: large- scale storage for bulk data Today/Friday: Lessons from GFS/BigTable

More information

Index construc-on. Friday, 8 April 16 1

Index construc-on. Friday, 8 April 16 1 Index construc-on Informa)onal Retrieval By Dr. Qaiser Abbas Department of Computer Science & IT, University of Sargodha, Sargodha, 40100, Pakistan qaiser.abbas@uos.edu.pk Friday, 8 April 16 1 4.3 Single-pass

More information

CS 61C: Great Ideas in Computer Architecture Direct- Mapped Caches. Increasing distance from processor, decreasing speed.

CS 61C: Great Ideas in Computer Architecture Direct- Mapped Caches. Increasing distance from processor, decreasing speed. CS 6C: Great Ideas in Computer Architecture Direct- Mapped s 9/27/2 Instructors: Krste Asanovic, Randy H Katz hdp://insteecsberkeleyedu/~cs6c/fa2 Fall 2 - - Lecture #4 New- School Machine Structures (It

More information

ARCHITECTING WEB APPLICATIONS FOR THE CLOUD: DESIGN PRINCIPLES AND PRACTICAL GUIDANCE FOR AWS

ARCHITECTING WEB APPLICATIONS FOR THE CLOUD: DESIGN PRINCIPLES AND PRACTICAL GUIDANCE FOR AWS ARCHITECTING WEB APPLICATIONS FOR THE CLOUD: DESIGN PRINCIPLES AND PRACTICAL GUIDANCE FOR AWS Dr Adnene Guabtni, Senior Research Scientist, NICTA/Data61, CSIRO Adnene.Guabtni@csiro.au EC2 S3 ELB RDS AMI

More information

Making Session Stores More Intelligent KYLE J. DAVIS TECHNICAL MARKETING MANAGER REDIS LABS

Making Session Stores More Intelligent KYLE J. DAVIS TECHNICAL MARKETING MANAGER REDIS LABS Making Session Stores More Intelligent KYLE J. DAVIS TECHNICAL MARKETING MANAGER REDIS LABS What is a session store? A session store is An chunk of data that is connected to one user of a service user

More information

GETTING STARTED WITH NUODB

GETTING STARTED WITH NUODB February 15, 2017 GETTING STARTED WITH NUODB The elastic SQL database for hybrid cloud applications LOGISTICS AND INTRODUCTIONS 2 + All a&endees are muted + Submit ques3ons in the Q&A box on the right

More information

The Care and Feeding of a MySQL Database for Linux Adminstrators. Dave Stokes MySQL Community Manager

The Care and Feeding of a MySQL Database for Linux Adminstrators. Dave Stokes MySQL Community Manager The Care and Feeding of a MySQL Database for Linux Adminstrators Dave Stokes MySQL Community Manager David.Stokes@Oracle.com Simple Introduction This is a general introduction to running a MySQL database

More information

Redis for Real-Time Personalization A PSEUDO CODE APPROACH TO IMPLEMENTING PERSONALIZATION WITH REDIS

Redis for Real-Time Personalization A PSEUDO CODE APPROACH TO IMPLEMENTING PERSONALIZATION WITH REDIS Redis for Real-Time Personalization A PSEUDO CODE APPROACH TO IMPLEMENTING PERSONALIZATION WITH REDIS Contents Redis and the Enterprise 3 The Personalization App Implemented in Redis 3 Capabilities That

More information

Essential Features of an Integration Solution

Essential Features of an Integration Solution Essential Features of an Integration Solution September 2017 WHITE PAPER Essential Features of an Integration Solution When an enterprise uses multiple applications, it needs to connect them for a variety

More information

From Rack Scale to Network Scale: NVMe Over Fabrics Enables Exabyte Applica>ons. Zivan Ori, CEO & Co-founder, E8 Storage

From Rack Scale to Network Scale: NVMe Over Fabrics Enables Exabyte Applica>ons. Zivan Ori, CEO & Co-founder, E8 Storage From Rack Scale to Network Scale: NVMe Over Fabrics Enables Exabyte Applica>ons Zivan Ori, CEO & Co-founder, E8 Storage NVMe Over Fabrics Who? Why? How? Who is using NVMe over fabrics? Why do they need

More information

Scalability in a Real-Time Decision Platform

Scalability in a Real-Time Decision Platform Scalability in a Real-Time Decision Platform Kenny Shi Manager Software Development ebay Inc. A Typical Fraudulent Lis3ng fraud detec3on architecture sync vs. async applica3on publish messaging bus request

More information

IBM Cognitive Systems Cognitive Infrastructure for the digital business transformation

IBM Cognitive Systems Cognitive Infrastructure for the digital business transformation IBM Cognitive Systems Cognitive Infrastructure for the digital business transformation July 2017 Dilek Sezgün dilek@de.ibm.com 0160/90741619 Cognitive Solution Infrastructure Sales Leader Painpoints of

More information

Polyglot Persistence in Today s Data World

Polyglot Persistence in Today s Data World Polyglot Persistence in Today s Data World Kimberly Wilkins Principal Engineer Databases ObjectRocket by Rackspace www.linkedin.com/in/wilkinskimberly, kimberly.wilkins@rackspace.com, @dba_denizen 1 Background

More information

PNUTS: Yahoo! s Hosted Data Serving Platform. Reading Review by: Alex Degtiar (adegtiar) /30/2013

PNUTS: Yahoo! s Hosted Data Serving Platform. Reading Review by: Alex Degtiar (adegtiar) /30/2013 PNUTS: Yahoo! s Hosted Data Serving Platform Reading Review by: Alex Degtiar (adegtiar) 15-799 9/30/2013 What is PNUTS? Yahoo s NoSQL database Motivated by web applications Massively parallel Geographically

More information

Caching Memcached vs. Redis

Caching Memcached vs. Redis Caching Memcached vs. Redis San Francisco MySQL Meetup Ryan Lowe Erin O Neill 1 Databases WE LOVE THEM... Except when we don t 2 When Databases Rule Many access patterns on the same set of data Transactions

More information

CS261 Data Structures. Ordered Bag Dynamic Array Implementa;on

CS261 Data Structures. Ordered Bag Dynamic Array Implementa;on CS261 Data Structures Ordered Bag Dynamic Array Implementa;on Goals Understand the downside of unordered containers Binary Search Ordered Bag ADT Downside of unordered collec;ons What is the complexity

More information

Western Michigan University

Western Michigan University CS-6030 Cloud compu;ng Google App engine Sepideh Mohammadi Summer II 2017 Western Michigan University content Categories of cloud compu;ng Google cloud plaborm Google App Engine Storage technologies Datastore

More information

REdis: Implementing Redis in Erlang. A step-by-step walkthrough

REdis: Implementing Redis in Erlang. A step-by-step walkthrough R: Implementing Redis in Erlang A step-by-step walkthrough 1 2 : Implementing Redis in Erlang A step-by-step walkthrough 2 My Background Microsoft Visual Studio Visto Corporation Founded Inaka Moved to

More information

The Evolution of a Data Project

The Evolution of a Data Project The Evolution of a Data Project The Evolution of a Data Project Python script The Evolution of a Data Project Python script SQL on live DB The Evolution of a Data Project Python script SQL on live DB SQL

More information

Er XenApp død og begravet? Allan Bak, edgemo A/S

Er XenApp død og begravet? Allan Bak, edgemo A/S Er XenApp død og begravet? Allan Bak, edgemo A/S Agenda Architecture Install, Manage, Support New features Migration/upgrade Demo 7 Current Architectures Receiver Policy Console Delivery Services / WebInterface

More information

Today s Objec4ves. Data Center. Virtualiza4on Cloud Compu4ng Amazon Web Services. What did you think? 10/23/17. Oct 23, 2017 Sprenkle - CSCI325

Today s Objec4ves. Data Center. Virtualiza4on Cloud Compu4ng Amazon Web Services. What did you think? 10/23/17. Oct 23, 2017 Sprenkle - CSCI325 Today s Objec4ves Virtualiza4on Cloud Compu4ng Amazon Web Services Oct 23, 2017 Sprenkle - CSCI325 1 Data Center What did you think? Oct 23, 2017 Sprenkle - CSCI325 2 1 10/23/17 Oct 23, 2017 Sprenkle -

More information

Informa/on Retrieval. Text Search. CISC437/637, Lecture #23 Ben CartereAe. Consider a database consis/ng of long textual informa/on fields

Informa/on Retrieval. Text Search. CISC437/637, Lecture #23 Ben CartereAe. Consider a database consis/ng of long textual informa/on fields Informa/on Retrieval CISC437/637, Lecture #23 Ben CartereAe Copyright Ben CartereAe 1 Text Search Consider a database consis/ng of long textual informa/on fields News ar/cles, patents, web pages, books,

More information

The NoSQL Landscape. Frank Weigel VP, Field Technical Opera;ons

The NoSQL Landscape. Frank Weigel VP, Field Technical Opera;ons The NoSQL Landscape Frank Weigel VP, Field Technical Opera;ons What we ll talk about Why RDBMS are not enough? What are the different NoSQL taxonomies? Which NoSQL is right for me? Macro Trends Driving

More information

VOLTDB + HP VERTICA. page

VOLTDB + HP VERTICA. page VOLTDB + HP VERTICA ARCHITECTURE FOR FAST AND BIG DATA ARCHITECTURE FOR FAST + BIG DATA FAST DATA Fast Serve Analytics BIG DATA BI Reporting Fast Operational Database Streaming Analytics Columnar Analytics

More information

Agenda. AWS Database Services Traditional vs AWS Data services model Amazon RDS Redshift DynamoDB ElastiCache

Agenda. AWS Database Services Traditional vs AWS Data services model Amazon RDS Redshift DynamoDB ElastiCache Databases on AWS 2017 Amazon Web Services, Inc. and its affiliates. All rights served. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon Web Services,

More information

Streaming Integration and Intelligence For Automating Time Sensitive Events

Streaming Integration and Intelligence For Automating Time Sensitive Events Streaming Integration and Intelligence For Automating Time Sensitive Events Ted Fish Director Sales, Midwest ted@striim.com 312-330-4929 Striim Executive Summary Delivering Data for Time Sensitive Processes

More information

Creating a Recommender System. An Elasticsearch & Apache Spark approach

Creating a Recommender System. An Elasticsearch & Apache Spark approach Creating a Recommender System An Elasticsearch & Apache Spark approach My Profile SKILLS Álvaro Santos Andrés Big Data & Analytics Solution Architect in Ericsson with more than 12 years of experience focused

More information

NoSQL Databases. an overview

NoSQL Databases. an overview NoSQL Databases an overview Who? Why? During studies: Excited by simplicity Crawler Project: 100 Million records Single server 100+ QPS Initially: Limited query options Now: Query them all Experimented

More information

VMware Virtual SAN Technology

VMware Virtual SAN Technology VMware Virtual SAN Technology Today s Agenda 1 Hyper-Converged Infrastructure Architecture & Vmware Virtual SAN Overview 2 Why VMware Hyper-Converged Software? 3 VMware Virtual SAN Advantage Today s Agenda

More information

ITP 342 Mobile App Development. APIs

ITP 342 Mobile App Development. APIs ITP 342 Mobile App Development APIs API Application Programming Interface (API) A specification intended to be used as an interface by software components to communicate with each other An API is usually

More information

Carbonite Availability. Technical overview

Carbonite Availability. Technical overview Carbonite Availability Technical overview Table of contents Executive summary The availability imperative...3 True real-time replication More efficient and better protection... 4 Robust protection Reliably

More information

Sydney SQL Server Enterprise User Group. News: 2 February 2011

Sydney SQL Server Enterprise User Group. News: 2 February 2011 News: 2 February Agenda News Benefits News Patches Released Support PDW Denali Microsoft Certified Master Benefits Patches Released SQL Server 2005 SP4 RTM Cumulative Update #13 for SQL Server 2005 Service

More information

Introduc)on to Apache Ka1a. Jun Rao Co- founder of Confluent

Introduc)on to Apache Ka1a. Jun Rao Co- founder of Confluent Introduc)on to Apache Ka1a Jun Rao Co- founder of Confluent Agenda Why people use Ka1a Technical overview of Ka1a What s coming What s Apache Ka1a Distributed, high throughput pub/sub system Ka1a Usage

More information

Figure 1. GeneSiDer user personal homepage

Figure 1. GeneSiDer user personal homepage Sanger Sequencing and GeneMapper Services at The GenePool The GenePool will be introducing a new Laboratory Informa?on Management System (LIMS) for submission, processing and accessing of Sanger sequencing

More information

Market Data Publisher In a High Frequency Trading Set up

Market Data Publisher In a High Frequency Trading Set up Market Data Publisher In a High Frequency Trading Set up INTRODUCTION The main theme behind the design of Market Data Publisher is to make the latest trade & book data available to several integrating

More information

MapReduce. Tom Anderson

MapReduce. Tom Anderson MapReduce Tom Anderson Last Time Difference between local state and knowledge about other node s local state Failures are endemic Communica?on costs ma@er Why Is DS So Hard? System design Par??oning of

More information

Key metrics for effective storage performance and capacity reporting

Key metrics for effective storage performance and capacity reporting Key metrics for effective storage performance and capacity reporting Key Metrics for Effective Storage Performance and Capacity Reporting Objectives This white paper will cover the key metrics in storage

More information

Metadata state and history service for datasets

Metadata state and history service for datasets Faculty of Science and Technology Department of Computer Science Metadata state and history service for datasets Enable extracting, storing and access to metadata about a dataset over time. Roberth Hansen

More information