MATH is Hard: TTL Index Configuration and Considerations. Kimberly Wilkins Sr.
|
|
- Blaze Fitzgerald
- 6 years ago
- Views:
Transcription
1 MATH is Hard: TTL Index Configuration and Considerations Kimberly Wilkins Sr.
2 Drowning in Data?
3 TTL s are your lifeboat
4 Sources?
5 Amounts? 600 TB 115 million 58 million avg 1EB -2 yrs ^^ 5 B s IoT/IoE 1ZB
6 What do you KEEP? How do you REMOVE?
7 TTL s Time To Live Indexes by
8 What Collections Indexes Why 2.2 +, normal collection - special TTL index type Defines # seconds a document exists Control collection/database size Automatically remove based on BSON date type No MANUAL mass/batch deletes or crons
9 Event data Logs Time Series Session Info Use Cases User Activity Logs gaming, social Queuing Any time data needs to persist only for a short, specific period of time
10 What is Your Goal? Think Before Creating Save space? Improve Performance? Data retention policies? Business logic or requirements? System load? More Reads vs. Writes?
11 Use Expire AFTER vs. AT? Global expiration for all documents expireafterseconds (after 3 days) { "createdat": 1 }, { expireafterseconds: } Global policy determines Per document expiration for each document - Certain Clock Time Things to Think cont d {"expiresat": 1 }, { expireafterseconds: 0 } Application sets FUTURE time Combo - {"expiresat": 1 }, { expireafterseconds: } application and global
12 Numbers to Know Sample seconds minutes hours days math seconds / 60 seconds = 1440 minutes / 60 minutes = 24 hours = 1 day 1 hour = 3600 seconds 1 day = seconds 3 days = seconds * 3 = seconds 7 days = seconds * 7 = seconds -Offsets 10.5 days = seconds * 10.5 = seconds
13 How to Create TTL s Create a new index with ensureindex Specify expireafterseconds option -REQUIRED use "createdat or "expiresat leader 2 real Types : global & per document/application expireafterseconds -global type all docs expire X seconds after CREATED
14 How to Create TTL s cont d Expire AT Certain Clock Time -application set - a date set in future by the application code - expireafterseconds set to ZERO combo type - set an expires_at date in application code - but apply a global type expiration policy with "expireafterseconds Messages documents set to expire 24 hours after creation
15 Changing Your TTL How to Modify your TTL Time Index must be created as a TTL index - -from the beginning {"expiresat" : 1 } alone!= a TTL index Must use the collmod command to change seconds Example increase to a 3 day TTL db.runcommand({"collmod" : "messages", "index" : { "keypattern" : {"expiresat" : 1 }, "expireafterseconds" : } })
16 Considerations and Gotchas Aggressiv e deletes Space Usage TTL Instance Isolation* usepowerof2siz es limits fragmentation No _Id field No nulls Driver issues No compounds Stale timestamps downstream clients language s timezon e
17 More Considerations and Gotchas -Very heavy writes/inserts - turn off TTL -Busy app? Hard to catch up (^7-15M inserts per day) -Version Diffs - 2.6^ ttl monitor, index builds* -TTLMonitor thread runs ~60 seconds -Background or Foreground Build implications* Replica sets, Primary cascade, thread runs on all Check oplog size increase for busy systems *
18 More Considerations and Gotchas Warning: The TTL index does not guarantee that expired data will be deleted immediately. Duration of removal operation depends on workload hitting your mongod. Q^ ww-dww-d Expired data may exist beyond the 60 second period between runs of the TTL monitor. - <=N + 60 top mongod deletes
19 Is My TTL Working Pre-check gather info, prepare, know what to look for -expect
20 TTL Working? cont d -Confirm ttl index exists AND is correctly defined -Define a variable to check with ex. ttltime -Set new Date for checking
21 MATH and Associated Queries : TTL Working? cont d *ISODate is in MICROSECONDS; TTL is in SECONDS* db.messages.find({expires_at: {$lt:isodate(" t03:11:00.013z") }}).count()
22 Other Associated Queries Adjust to expire docs after 1 day : set ttltime and adjust ex. mongos> ttltime = new Date(ISODate()+3600*1000) ISODate(" :04:35:34.936Z") If older than exact ttl exists, wait & re-query get oldest documents in a collection using the _id as the determiner
23 Questions to Ask How Aggressive? How Busy? Time or Space Crunch? #docs to delete? Reads vs Writes? Hardware and Architecture? Busy system => Slower = Safer = Less Performance Impact Lots to remove => greater impact System architecture => more smaller shards = more write scopes, distributes impact TIP: Use gettime turns ISODates into numbers aka milliseconds
24 THE MATH- Determine Starting TTL seconds Good Rule +20% buffer in TTL seconds = > Math.round(1.2 * )
25 Formula After Known Start Time Formula : SS -ES = Secs2Reduce /StepinSecs =#ofhops(iterations) SS =Starting Seconds ES =Ending Seconds Step = How many docs/secs you want to delete during per run Hop = # of times to iterate through reduction Goal: Desire 60.5 day TTL Oldest doc time = >Starting Secs : / = 65 days Goal TTL Secs time = >Ending Secs : = 86400*60.5 days Step : 60 secs Hops : aka 60 secs worth of docs at a time
26 Explanation SS =Starting Seconds ES =Ending Seconds Step = How many docs you want to delete during per run Hop = # of times to iterate through reduction Formula: Starting Secs - Ending Secs = Seconds to Reduce / Your step increment in seconds = # of Hops (# of times you will have to iterate through the reducer) ex. Starting Seconds Ending Seconds = Total Seconds needed to Reduce to your Deired TTL Time / 60 sec step increment = 6480 hops or iterations
27 db = db.getsiblingdb("kw_events"); var step=60; var index_info = db.system.indexes.findone({ns:"kw_events.events","expireafterseconds": {$exists:1}}); print("starting index TTL spot") printjson(index_info); var doc_count = db.events.count(); print("starting doc count") printjson(doc_count); var endingsecs= ; ##// var endingsecs=index_info.expireafterseconds - (6480*step); while ( index_info.expireafterseconds > endingsecs ){ var collmodobj ={collmod: "events", index: {keypattern: index_info.key, expireafterseconds: (index_info.expireafterseconds-step)} }; printjson(db.runcommand(collmodobj)); index_info = db.system.indexes.findone({ns:"kw_events.events","expireafterseconds": {$exists:1}}); sleep(20000); } print("ending index TTL spot"); printjson(index_info);
28 Explained in layman s Terms See sublime english-ified More examples next.. Beg, mid, end
29 TTL Reducer Tips -This is a Process Run in a screen!!! - small batch deletes allows other work - Create new ttl with 20% padded seconds to start - Monitor impact with mongostat - Minimize dblock% on Primary to a few seconds - Beginning script - high padded ttl time until deletes start more steps and a low hops/interval ex. s-2000, h-30 - Mid js Adjust # of seconds in step expect docs, minimize impact, run this one longer ex. s-500, h End js Be very specific in iterations impact already known, remaining known ex. S-500, h Be exact to not pass desired TTL
30 ttl_reducer_begin.js beginning -Start with a larger step (interval 2000 secs) and fewer hops/iterations (30) -High initial threshold (+20%) -might take time to 1 st start hitting the deletes
31 ttl_reducer_mid.js mid-way - Midway - do fewer steps(500 secs) and more hops (2930) - Caution: deletes may negatively impact performance
32 ttl_reducer_end.js end - At end, optimize step (seconds to be deleted) and # hops to reach desired permanent TTL expireafterseconds
33 A Mass Pre-TTL Delete Loop Useful MATH, Queries, and Loop for non-ttl based deletes Run the below query and delete loop for a 30 minute period defined by x below
34 Enabling / Disabling TTL Instance Wide -Single global setting so no need to drop all TTL indexes -Version 2.4.x + : TTLMonitor thread - enabled by default -Via a command-line parameter and/or server command. To disable at startup: setparameter ttlmonitorenabled=false #config file of the MONGOD's only setparameter=ttlmonitorenabled=false ## To turn it back on (just remove that line) To disable at runtime: on ALL replica set members - primary and secondary on secondaries - -- rs.slaveok() db.admincommand({setparameter:1, ttlmonitorenabled:false}) To re-enable when ready: db.admincommand({setparameter:1, ttlmonitorenabled:true})
35 Links - slideshare of this deck: blog on this topic will be up eventually here: free ObjectRocket MongoDB 30 day Trial Rackspace Managed Data Services - Hadoop, MySQL Contacts:
36 Links Slideshare: Gist of the ttl_reducer.js scriptlets : Gist of additional queries and examples : Contacts:
37 Questions?
Reduce MongoDB Data Size. Steven Wang
Reduce MongoDB Data Size Tangome inc Steven Wang stwang@tango.me Outline MongoDB Cluster Architecture Advantages to Reduce Data Size Several Cases To Reduce MongoDB Data Size Case 1: Migrate To wiredtiger
More informationPercona Live Updated Sharding Guidelines in MongoDB 3.x with Storage Engine Considerations. Kimberly Wilkins
Percona Live 2016 Updated Sharding Guidelines in MongoDB 3.x with Storage Engine Considerations Kimberly Wilkins Principal Engineer - Databases, Rackspace/ ObjectRocket www.linkedin.com/in/wilkinskimberly,
More informationScaling MongoDB: Avoiding Common Pitfalls. Jon Tobin Senior Systems
Scaling MongoDB: Avoiding Common Pitfalls Jon Tobin Senior Systems Engineer Jon.Tobin@percona.com @jontobs www.linkedin.com/in/jonathanetobin Agenda Document Design Data Management Replica3on & Failover
More informationPercona Live September 21-23, 2015 Mövenpick Hotel Amsterdam
Percona Live 2015 September 21-23, 2015 Mövenpick Hotel Amsterdam MongoDB, Elastic, and Hadoop: The What, When, and How Kimberly Wilkins Principal Engineer/Database Denizen ObjectRocket/Rackspace kimberly@objectrocket.com
More informationCourse Content MongoDB
Course Content MongoDB 1. Course introduction and mongodb Essentials (basics) 2. Introduction to NoSQL databases What is NoSQL? Why NoSQL? Difference Between RDBMS and NoSQL Databases Benefits of NoSQL
More informationMongoDB: Comparing WiredTiger In-Memory Engine to Redis. Jason Terpko DBA, Rackspace/ObjectRocket 1
MongoDB: Comparing WiredTiger In-Memory Engine to Redis Jason Terpko DBA, Rackspace/ObjectRocket www.linkedin.com/in/jterpko 1 Background Started out in relational databases in public education then financial
More informationTime-Series Data in MongoDB on a Budget. Peter Schwaller Senior Director Server Engineering, Percona Santa Clara, California April 23th 25th, 2018
Time-Series Data in MongoDB on a Budget Peter Schwaller Senior Director Server Engineering, Percona Santa Clara, California April 23th 25th, 2018 TIME SERIES DATA in MongoDB on a Budget Click to add text
More informationMongoDB. David Murphy MongoDB Practice Manager, Percona
MongoDB Click Replication to edit Master and Sharding title style David Murphy MongoDB Practice Manager, Percona Who is this Person and What Does He Know? Former MongoDB Master Former Lead DBA for ObjectRocket,
More informationThe course modules of MongoDB developer and administrator online certification training:
The course modules of MongoDB developer and administrator online certification training: 1 An Overview of the Course Introduction to the course Table of Contents Course Objectives Course Overview Value
More informationMongoDB Chunks Distribution, Splitting, and Merging. Jason Terpko
Percona Live 2016 MongoDB Chunks Distribution, Splitting, and Merging Jason Terpko NoSQL DBA, Rackspace/ObjectRocket www.linkedin.com/in/jterpko, jason.terpko@rackspace.com My Story Started out in relational
More informationMongoDB 2.2 and Big Data
MongoDB 2.2 and Big Data Christian Kvalheim Team Lead Engineering, EMEA christkv@10gen.com @christkv christiankvalheim.com From here... http://bit.ly/ot71m4 ...to here... http://bit.ly/oxcsis ...without
More informationScaling for Humongous amounts of data with MongoDB
Scaling for Humongous amounts of data with MongoDB Alvin Richards Technical Director, EMEA alvin@10gen.com @jonnyeight alvinonmongodb.com From here... http://bit.ly/ot71m4 ...to here... http://bit.ly/oxcsis
More informationMongoDB Distributed Write and Read
VICTORIA UNIVERSITY OF WELLINGTON Te Whare Wananga o te Upoko o te Ika a Maui MongoDB Distributed Write and Read Lecturer : Dr. Pavle Mogin SWEN 432 Advanced Database Design and Implementation Advanced
More informationRun your own Open source. (MMS) to avoid vendor lock-in. David Murphy MongoDB Practice Manager, Percona
Run your own Open source Click alternative to edit to Master Ops-Manager title style (MMS) to avoid vendor lock-in David Murphy MongoDB Practice Manager, Percona Who is this Person and What Does He Know?
More informationMicrosoft SQL Server Fix Pack 15. Reference IBM
Microsoft SQL Server 6.3.1 Fix Pack 15 Reference IBM Microsoft SQL Server 6.3.1 Fix Pack 15 Reference IBM Note Before using this information and the product it supports, read the information in Notices
More informationHow to Scale MongoDB. Apr
How to Scale MongoDB Apr-24-2018 About me Location: Skopje, Republic of Macedonia Education: MSc, Software Engineering Experience: Lead Database Consultant (since 2016) Database Consultant (2012-2016)
More informationBreaking Barriers: MongoDB Design Patterns. Nikolaos Vyzas & Christos Soulios
Breaking Barriers: MongoDB Design Patterns Nikolaos Vyzas & Christos Soulios a bit about us and this talk Who we are what we do Christos Soulios Christos is a principal architect at Pythian Delivers Big
More informationNoSQL Databases Analysis
NoSQL Databases Analysis Jeffrey Young Intro I chose to investigate Redis, MongoDB, and Neo4j. I chose Redis because I always read about Redis use and its extreme popularity yet I know little about it.
More informationHigh Performance NoSQL with MongoDB
High Performance NoSQL with MongoDB History of NoSQL June 11th, 2009, San Francisco, USA Johan Oskarsson (from http://last.fm/) organized a meetup to discuss advances in data storage which were all using
More informationAsynchronous Method Calls White Paper VERSION Copyright 2014 Jade Software Corporation Limited. All rights reserved.
VERSION 7.0.10 Copyright 2014 Jade Software Corporation Limited. All rights reserved. Jade Software Corporation Limited cannot accept any financial or other responsibilities that may be the result of your
More informationData Model Design for MongoDB
Data Model Design for MongoDB Release 3.2.3 MongoDB, Inc. February 17, 2016 2 MongoDB, Inc. 2008-2016 This work is licensed under a Creative Commons Attribution-NonCommercial- ShareAlike 3.0 United States
More informationMongoDB An Overview. 21-Oct Socrates
MongoDB An Overview 21-Oct-2016 Socrates Agenda What is NoSQL DB? Types of NoSQL DBs DBMS and MongoDB Comparison Why MongoDB? MongoDB Architecture Storage Engines Data Model Query Language Security Data
More informationMike Kania Truss
Mike Kania Engineer @ Truss http://truss.works/ MongoDB on AWS With Minimal Suffering + Topics Provisioning MongoDB Replica Sets on AWS Choosing storage and a storage engine Backups Monitoring Capacity
More informationCIS 612 Advanced Topics in Database Big Data Project Lawrence Ni, Priya Patil, James Tench
CIS 612 Advanced Topics in Database Big Data Project Lawrence Ni, Priya Patil, James Tench Abstract Implementing a Hadoop-based system for processing big data and doing analytics is a topic which has been
More informationGroup13: Siddhant Deshmukh, Sudeep Rege, Sharmila Prakash, Dhanusha Varik
Group13: Siddhant Deshmukh, Sudeep Rege, Sharmila Prakash, Dhanusha Varik mongodb (humongous) Introduction What is MongoDB? Why MongoDB? MongoDB Terminology Why Not MongoDB? What is MongoDB? DOCUMENT STORE
More information~3333 write ops/s ms response
NoSQL Infrastructure ~3333 write ops/s 0.07-0.05 ms response Woop Japan! David Mytton MongoDB at Server Density MongoDB at Server Density 27 nodes MongoDB at Server Density 27 nodes June 2009-4yrs MongoDB
More informationPlug-in Configuration
Overview, page 1 Threading Configuration, page 2 Portal Configuration, page 3 Async Threading Configuration, page 3 Custom Reference Data Configuration, page 4 Balance Configuration, page 6 Diameter Configuration,
More informationMongoDB Backup & Recovery Field Guide
MongoDB Backup & Recovery Field Guide Tim Vaillancourt Percona Speaker Name `whoami` { name: tim, lastname: vaillancourt, employer: percona, techs: [ mongodb, mysql, cassandra, redis, rabbitmq, solr, mesos
More informationTracking CPS GUI and API Usage
Track Usage, page 1 Track Usage Use the Audit History to track usage of the various GUIs and APIs. If enabled, each request is submitted to the Audit History database for historical and security purposes.
More informationMMS Backup Manual Release 1.4
MMS Backup Manual Release 1.4 MongoDB, Inc. Jun 27, 2018 MongoDB, Inc. 2008-2016 2 Contents 1 Getting Started with MMS Backup 4 1.1 Backing up Clusters with Authentication.................................
More informationSQL, NoSQL, MongoDB. CSE-291 (Cloud Computing) Fall 2016 Gregory Kesden
SQL, NoSQL, MongoDB CSE-291 (Cloud Computing) Fall 2016 Gregory Kesden SQL Databases Really better called Relational Databases Key construct is the Relation, a.k.a. the table Rows represent records Columns
More information<Insert Picture Here> DBA Best Practices: A Primer on Managing Oracle Databases
DBA Best Practices: A Primer on Managing Oracle Databases Mughees A. Minhas Sr. Director of Product Management Database and Systems Management The following is intended to outline
More informationAzure Cosmos DB. Planet Earth Scale, for now. Mike Sr. Consultant, Microsoft
Azure Cosmos DB Planet Earth Scale, for now Mike Lawell, @sqldiver, Sr. Consultant, Microsoft Mission-critical applications for a global userbase need Building globally distributed applications Design
More informationDocument Object Storage with MongoDB
Document Object Storage with MongoDB Lecture BigData Analytics Julian M. Kunkel julian.kunkel@googlemail.com University of Hamburg / German Climate Computing Center (DKRZ) 2017-12-15 Disclaimer: Big Data
More information4/9/2018 Week 13-A Sangmi Lee Pallickara. CS435 Introduction to Big Data Spring 2018 Colorado State University. FAQs. Architecture of GFS
W13.A.0.0 CS435 Introduction to Big Data W13.A.1 FAQs Programming Assignment 3 has been posted PART 2. LARGE SCALE DATA STORAGE SYSTEMS DISTRIBUTED FILE SYSTEMS Recitations Apache Spark tutorial 1 and
More informationTopics. History. Architecture. MongoDB, Mongoose - RDBMS - SQL. - NoSQL
Databases Topics History - RDBMS - SQL Architecture - SQL - NoSQL MongoDB, Mongoose Persistent Data Storage What features do we want in a persistent data storage system? We have been using text files to
More informationWhat s new in Mongo 4.0. Vinicius Grippa Percona
What s new in Mongo 4.0 Vinicius Grippa Percona About me Support Engineer at Percona since 2017 Working with MySQL for over 5 years - Started with SQL Server Working with databases for 7 years 2 Agenda
More informationMarathon Documentation
Marathon Documentation Release 3.0.0 Top Free Games Feb 07, 2018 Contents 1 Overview 3 1.1 Features.................................................. 3 1.2 Architecture...............................................
More informationCSE 544 Principles of Database Management Systems. Magdalena Balazinska Winter 2015 Lecture 14 NoSQL
CSE 544 Principles of Database Management Systems Magdalena Balazinska Winter 2015 Lecture 14 NoSQL References Scalable SQL and NoSQL Data Stores, Rick Cattell, SIGMOD Record, December 2010 (Vol. 39, No.
More informationLesson 2 RMAN Architecture
RMAN Architecture 2.1 Lesson 2 RMAN Architecture An introduction to the architecture and components used by the RMAN utility. SKILLBUILDERS Author: Dave Anderson, SkillBuilders RMAN Architecture 2.2 2.2
More informationITG Software Engineering
Introduction to MongoDB Course ID: Page 1 Last Updated 12/15/2014 MongoDB for Developers Course Overview: In this 3 day class students will start by learning how to install and configure MongoDB on a Mac
More informationPrototyping Data Intensive Apps: TrendingTopics.org
Prototyping Data Intensive Apps: TrendingTopics.org Pete Skomoroch Research Scientist at LinkedIn Consultant at Data Wrangling @peteskomoroch 09/29/09 1 Talk Outline TrendingTopics Overview Wikipedia Page
More informationThe Right Read Optimization is Actually Write Optimization. Leif Walsh
The Right Read Optimization is Actually Write Optimization Leif Walsh leif@tokutek.com The Right Read Optimization is Write Optimization Situation: I have some data. I want to learn things about the world,
More informationWhy Choose Percona Server for MongoDB? Tyler Duzan
Why Choose Percona Server for MongoDB? Tyler Duzan Product Manager Who Am I? My name is Tyler Duzan Formerly an operations engineer for more than 12 years focused on security and automation Now a Product
More informationCLOUD-SCALE FILE SYSTEMS
Data Management in the Cloud CLOUD-SCALE FILE SYSTEMS 92 Google File System (GFS) Designing a file system for the Cloud design assumptions design choices Architecture GFS Master GFS Chunkservers GFS Clients
More informationMongoDB Architecture
VICTORIA UNIVERSITY OF WELLINGTON Te Whare Wananga o te Upoko o te Ika a Maui MongoDB Architecture Lecturer : Dr. Pavle Mogin SWEN 432 Advanced Database Design and Implementation Advanced Database Design
More informationReporting Best Practices
Note You can find troubleshooting information for Cisco Unified Customer Voice Portal (Unified CVP) Reporting on the Cisco Troubleshooting Doc Wiki site The chapter contains the following topics: Reporting
More informationMongoDB. copyright 2011 Trainologic LTD
MongoDB MongoDB MongoDB is a document-based open-source DB. Developed and supported by 10gen. MongoDB is written in C++. The name originated from the word: humongous. Is used in production at: Disney,
More informationAurora, RDS, or On-Prem, Which is right for you
Aurora, RDS, or On-Prem, Which is right for you Kathy Gibbs Database Specialist TAM Katgibbs@amazon.com Santa Clara, California April 23th 25th, 2018 Agenda RDS Aurora EC2 On-Premise Wrap-up/Recommendation
More informationMongoDB - a No SQL Database What you need to know as an Oracle DBA
MongoDB - a No SQL Database What you need to know as an Oracle DBA David Burnham Aims of this Presentation To introduce NoSQL database technology specifically using MongoDB as an example To enable the
More informationMySQL Schema Review 101
MySQL Schema Review 101 How and What you should be looking at... Mike Benshoof - Technical Account Manager, Percona Agenda Introduction Key things to consider and review Tools to isolate issues Common
More informationNoSQL BENCHMARKING AND TUNING. Nachiket Kate Santosh Kangane Ankit Lakhotia Persistent Systems Ltd. Pune, India
NoSQL BENCHMARKING AND TUNING Nachiket Kate Santosh Kangane Ankit Lakhotia Persistent Systems Ltd. Pune, India Today large variety of available NoSQL options has made it difficult for developers to choose
More informationFluentd + MongoDB + Spark = Awesome Sauce
Fluentd + MongoDB + Spark = Awesome Sauce Nishant Sahay, Sr. Architect, Wipro Limited Bhavani Ananth, Tech Manager, Wipro Limited Your company logo here Wipro Open Source Practice: Vision & Mission Vision
More informationScaling with mongodb
Scaling with mongodb Ross Lawley Python Engineer @ 10gen Web developer since 1999 Passionate about open source Agile methodology email: ross@10gen.com twitter: RossC0 Today's Talk Scaling Understanding
More informationArchiving and Purging Data
CHAPTER 10 These topics provide basic information about data purging and archiving and system stability: Prime Network Database Schemas, page 10-1, describes the schemas supported by the Prime Network
More informationMongoDB Schema Design for. David Murphy MongoDB Practice Manager - Percona
MongoDB Schema Design for the Click "Dynamic to edit Master Schema" title World style David Murphy MongoDB Practice Manager - Percona Who is this Person and What Does He Know? Former MongoDB Master Former
More informationMaking MongoDB Accessible to All. Brody Messmer Product Owner DataDirect On-Premise Drivers Progress Software
Making MongoDB Accessible to All Brody Messmer Product Owner DataDirect On-Premise Drivers Progress Software Agenda Intro to MongoDB What is MongoDB? Benefits Challenges and Common Criticisms Schema Design
More informationCS162 Operating Systems and Systems Programming Lecture 11 Page Allocation and Replacement"
CS162 Operating Systems and Systems Programming Lecture 11 Page Allocation and Replacement" October 3, 2012 Ion Stoica http://inst.eecs.berkeley.edu/~cs162 Lecture 9 Followup: Inverted Page Table" With
More informationCache and Forward Architecture
Cache and Forward Architecture Shweta Jain Research Associate Motivation Conversation between computers connected by wires Wired Network Large content retrieval using wireless and mobile devices Wireless
More informationMyRocks Engineering Features and Enhancements. Manuel Ung Facebook, Inc. Dublin, Ireland Sept th, 2017
MyRocks Engineering Features and Enhancements Manuel Ung Facebook, Inc. Dublin, Ireland Sept 25 27 th, 2017 Agenda Bulk load Time to live (TTL) Debugging deadlocks Persistent auto-increment values Improved
More informationProject: Relational Databases vs. NoSQL. Catherine Easdon
Project: Relational Databases vs. NoSQL Catherine Easdon Database Choice (Relational) PostgreSQL (using BigSQL tools) Open source Mature project 21 years old, release 10.1 Supports all major OSes Highly
More informationMigrating to Vitess at (Slack) Scale. Michael Demmer Percona Live Europe 2017
Migrating to Vitess at (Slack) Scale Michael Demmer Percona Live Europe 2017 This is a (brief) story of how Slack's databases work today, why we're migrating to Vitess, and some lessons we've learned
More informationProduct: DQ Order Manager Release Notes
Product: DQ Order Manager Release Notes Subject: DQ Order Manager v7.1.29 Version: 1.0 January 20, 2017 Distribution: ODT Customers DQ OrderManager v7.1.29 *** requires db update 20170120 or newer ***
More informationEMC Unity Family. Monitoring System Performance. Version 4.2 H14978 REV 03
EMC Unity Family Version 4.2 Monitoring System Performance H14978 REV 03 Copyright 2016-2017 Dell Inc. or its subsidiaries. All rights reserved. Published July 2017 Dell believes the information in this
More informationReducing Replication Bandwidth for Distributed Document Databases
Reducing Replication Bandwidth for Distributed Document Databases Lianghong Xu 1, Andy Pavlo 1, Sudipta Sengupta 2 Jin Li 2, Greg Ganger 1 Carnegie Mellon University 1, Microsoft Research 2 Document-oriented
More informationExtra Notes - Data Stores & APIs - using MongoDB and native driver
Extra Notes - Data Stores & APIs - using MongoDB and native driver Dr Nick Hayward Contents intro install MongoDB running MongoDB using MongoDB Robo 3T basic intro to NoSQL connect to MongoDB from Node.js
More information10. Replication. Motivation
10. Replication Page 1 10. Replication Motivation Reliable and high-performance computation on a single instance of a data object is prone to failure. Replicate data to overcome single points of failure
More information1 Big Data Hadoop. 1. Introduction About this Course About Big Data Course Logistics Introductions
Big Data Hadoop Architect Online Training (Big Data Hadoop + Apache Spark & Scala+ MongoDB Developer And Administrator + Apache Cassandra + Impala Training + Apache Kafka + Apache Storm) 1 Big Data Hadoop
More informationVMWARE VREALIZE OPERATIONS MANAGEMENT PACK FOR. MongoDB. User Guide
VMWARE VREALIZE OPERATIONS MANAGEMENT PACK FOR MongoDB User Guide TABLE OF CONTENTS 1. Purpose... 3 2. Introduction to the Management Pack... 3 2.1 How the Management Pack Collects Data... 3 2.2 Data the
More informationSQL Gone Wild: Taming Bad SQL the Easy Way (or the Hard Way) Sergey Koltakov Product Manager, Database Manageability
SQL Gone Wild: Taming Bad SQL the Easy Way (or the Hard Way) Sergey Koltakov Product Manager, Database Manageability Oracle Enterprise Manager Top-Down, Integrated Application Management Complete, Open,
More informationGenesys Info Mart. schedule Section
Genesys Info Mart schedule Section 11/26/2017 schedule Section aggregate-duration aggregate-schedule etl-end-time etl-frequency etl-start-time export-schedule maintain-start-time on-demand-migration run-aggregates
More informationLOADRUNNER INTERVIEW QUESTIONS
LOADRUNNER INTERVIEW QUESTIONS 1. Why should we automate the performance testing? It s a discipline that leverages products, people and processes to reduce the risk of application upgrade or patch deployment.
More informationSharding Introduction
search MongoDB Home Admin Zone Sharding Sharding Introduction Sharding Introduction MongoDB supports an automated sharding architecture, enabling horizontal scaling across multiple nodes. For applications
More informationFinal Exam Review 2. Kathleen Durant CS 3200 Northeastern University Lecture 23
Final Exam Review 2 Kathleen Durant CS 3200 Northeastern University Lecture 23 QUERY EVALUATION PLAN Representation of a SQL Command SELECT {DISTINCT} FROM {WHERE
More informationScaling. Marty Weiner Grayskull, Eternia. Yashh Nelapati Gotham City
Scaling Marty Weiner Grayskull, Eternia Yashh Nelapati Gotham City Pinterest is... An online pinboard to organize and share what inspires you. Relationships Marty Weiner Grayskull, Eternia Yashh Nelapati
More informationApplied NoSQL in.net
Applied NoSQL in.net Topics Survey data management world Install mongodb Map C# classes to mongo collections Explore schema design Write queries against mongodb Update records Breaking with tradition is
More informationMONGODB INTERVIEW QUESTIONS
MONGODB INTERVIEW QUESTIONS http://www.tutorialspoint.com/mongodb/mongodb_interview_questions.htm Copyright tutorialspoint.com Dear readers, these MongoDB Interview Questions have been designed specially
More informationScaling. Yashh Nelapati Gotham City. Marty Weiner Krypton. Friday, July 27, 12
Scaling Marty Weiner Krypton Yashh Nelapati Gotham City Pinterest is... An online pinboard to organize and share what inspires you. Relationships Marty Weiner Grayskull, Eternia Relationships Marty
More information<Insert Picture Here> New MySQL Enterprise Backup 4.1: Better Very Large Database Backup & Recovery and More!
New MySQL Enterprise Backup 4.1: Better Very Large Database Backup & Recovery and More! Mike Frank MySQL Product Management - Director The following is intended to outline our general
More informationTwo Success Stories - Optimised Real-Time Reporting with BI Apps
Oracle Business Intelligence 11g Two Success Stories - Optimised Real-Time Reporting with BI Apps Antony Heljula October 2013 Peak Indicators Limited 2 Two Success Stories - Optimised Real-Time Reporting
More informationConfiguring NTP. Information About NTP NTP. This chapter describes how to configure the Network Time Protocol (NTP) on Cisco MDS 9000 Series switches.
This chapter describes how to configure the Network Time Protocol (NTP) on Cisco MDS 9000 Series switches. Information About NTP Information About NTP, on page 1 Prerequisites for NTP, on page 2 Guidelines
More informationTokuDB vs RocksDB. What to choose between two write-optimized DB engines supported by Percona. George O. Lorch III Vlad Lesin
TokuDB vs RocksDB What to choose between two write-optimized DB engines supported by Percona George O. Lorch III Vlad Lesin What to compare? Amplification Write amplification Read amplification Space amplification
More informationEffective Testing for Live Applications. March, 29, 2018 Sveta Smirnova
Effective Testing for Live Applications March, 29, 2018 Sveta Smirnova Table of Contents Sometimes You Have to Test on Production Wrong Data SELECT Returns Nonsense Wrong Data in the Database Performance
More informationTurbocharge your MySQL analytics with ElasticSearch. Guillaume Lefranc Data & Infrastructure Architect, Productsup GmbH Percona Live Europe 2017
Turbocharge your MySQL analytics with ElasticSearch Guillaume Lefranc Data & Infrastructure Architect, Productsup GmbH Percona Live Europe 2017 About the Speaker Guillaume Lefranc Data Architect at Productsup
More informationArchiving and Purging Data from the Database
CHAPTER 10 These topics provide basic information about data purging and archiving and system stability: Prime Network Database Schemas, page 10-1, describes the schemas supported by the Prime Network
More informationMongoDB Monitoring and Performance for The Savvy DBA
MongoDB Monitoring and Performance for The Savvy DBA Key metrics to focus on for day-to-day MongoDB operations Bimal Kharel Senior Technical Services Engineer Percona Webinar 2017-05-23 1 What I ll cover
More informationMongoDB Web Architecture
MongoDB Web Architecture MongoDB MongoDB is an open-source, NoSQL database that uses a JSON-like (BSON) document-oriented model. Data is stored in collections (rather than tables). - Uses dynamic schemas
More informationC. Reporting Services Server Enter the Web Service URL for your Report Server site.
B I Z I N S I G H T 5. 0. 3 4 Release Notes The 5.0.34 release of BizInsight contains many updates and improvements. This document provides information on the changes and special instructions where needed
More informationCaching Memcached vs. Redis
Caching Memcached vs. Redis San Francisco MySQL Meetup Ryan Lowe Erin O Neill 1 Databases WE LOVE THEM... Except when we don t 2 When Databases Rule Many access patterns on the same set of data Transactions
More information11/5/2018 Week 12-A Sangmi Lee Pallickara. CS435 Introduction to Big Data FALL 2018 Colorado State University
11/5/2018 CS435 Introduction to Big Data - FALL 2018 W12.A.0.0 CS435 Introduction to Big Data 11/5/2018 CS435 Introduction to Big Data - FALL 2018 W12.A.1 Consider a Graduate Degree in Computer Science
More informationMEAN Stack. 1. Introduction. 2. Foundation a. The Node.js framework b. Installing Node.js c. Using Node.js to execute scripts
MEAN Stack 1. Introduction 2. Foundation a. The Node.js framework b. Installing Node.js c. Using Node.js to execute scripts 3. Node Projects a. The Node Package Manager b. Creating a project c. The package.json
More informationInfrastructures for Cloud Computing and Big Data
University of Bologna Dipartimento di Informatica Scienza e Ingegneria (DISI) Engineering Bologna Campus Class of Computer Networks M or Infrastructures for Cloud Computing and Big Data Global Data Storage
More informationScaling Without Sharding. Baron Schwartz Percona Inc Surge 2010
Scaling Without Sharding Baron Schwartz Percona Inc Surge 2010 Web Scale!!!! http://www.xtranormal.com/watch/6995033/ A Sharding Thought Experiment 64 shards per proxy [1] 1 TB of data storage per node
More informationBecome a MongoDB Replica Set Expert in Under 5 Minutes:
Become a MongoDB Replica Set Expert in Under 5 Minutes: USING PERCONA SERVER FOR MONGODB IN A FAILOVER ARCHITECTURE This solution brief outlines a way to run a MongoDB replica set for read scaling in production.
More informationIntra-cluster Replication for Apache Kafka. Jun Rao
Intra-cluster Replication for Apache Kafka Jun Rao About myself Engineer at LinkedIn since 2010 Worked on Apache Kafka and Cassandra Database researcher at IBM Outline Overview of Kafka Kafka architecture
More informationNoSQL Performance Test
bankmark UG (haftungsbeschränkt) Bahnhofstraße 1 9432 Passau Germany www.bankmark.de info@bankmark.de T +49 851 25 49 49 F +49 851 25 49 499 NoSQL Performance Test In-Memory Performance Comparison of SequoiaDB,
More informationKim Greene - Introduction
Kim Greene kim@kimgreene.com 507-216-5632 Skype/Twitter: iseriesdomino Copyright Kim Greene Consulting, Inc. All rights reserved worldwide. 1 Kim Greene - Introduction Owner of an IT consulting company
More informationCreating a Recommender System. An Elasticsearch & Apache Spark approach
Creating a Recommender System An Elasticsearch & Apache Spark approach My Profile SKILLS Álvaro Santos Andrés Big Data & Analytics Solution Architect in Ericsson with more than 12 years of experience focused
More informationIP SLAs QFP Time Stamping
This module describes how to configure the IP SLA QFP Time Stamping feature for IP Service Level Agreements (SLAs) UDP jitter operations. This new probe and responder structure enables more accurate network
More informationMongoDB for a High Volume Logistics Application. Santa Clara, California April 23th 25th, 2018
MongoDB for a High Volume Logistics Application Santa Clara, California April 23th 25th, 2018 about me... Eric Potvin Software Engineer in the performance team at Shipwire, an Ingram Micro company, in
More information