HOW INDEX TO STORE DATA DATA
|
|
|
- Caroline Gardner
- 5 years ago
- Views:
Transcription
1 Stratos Idreos
2 HOW INDEX DATA TO STORE DATA
3 ALGORITHMS data structure decisions define the algorithms that access data INDEX DATA
4 ALGORITHMS unordered [7,4,2,6,1,3,9,10,5,8] INDEX DATA
5 ALGORITHMS unordered [7,4,2,6,1,3,9,10,5,8] INDEX DATA
6 ALGORITHMS unordered [7,4,2,6,1,3,9,10,5,8] ordered [1,2,3,4,5,6,7,8,9,10] INDEX DATA
7 ALGORITHMS INDEX DATA
8 DATA SYSTEMS ALGORITHMS INDEX DATA
9 speed COMPUTE DATA MOVEMENT 2018 DATA STRUCTURES DEFINE PERFORMANCE
10 speed COMPUTE register = this room caches = this city DATA MOVEMENT memory = nearby city disk = Pluto Jim Gray, Turing Award
11 no perfect structure Update Read amplification Memory
12 no perfect structure Read Update Read amplification Memory Update Memory
13 no perfect structure Read point tree Read amplification Update Memory Update differential approximate Memory
14 no perfect structure Read point tree Hash-Table Array Read Update Memory B-tree amplification Skip-List Update Linked-List differential approximate Memory Sorted Array Trie
15 How do I make my data system run x times as fast? (sql,nosql,bigdata, )
16 How do I make my data system run x times as fast? (sql,nosql,bigdata, ) How do I minimize my bill in the cloud?
17 How do I make my data system run x times as fast? (sql,nosql,bigdata, ) How do I minimize my bill in the cloud? How do I extend the lifetime of my hardware?
18 How do I make my data system run x times as fast? (sql,nosql,bigdata, ) How do I minimize my bill in the cloud? How do I extend the lifetime of my hardware? How to accelerate statistics computation for data science/ml?
19 How do I make my data system run x times as fast? (sql,nosql,bigdata, ) How do I minimize my bill in the cloud? How do I extend the lifetime of my hardware? How to accelerate statistics computation for data science/ml? How do I train my neural network x times faster?
20 NEW APPLICATIONS
21 NEW APPLICATIONS existing systems need to change too
22 NEW APPLICATIONS existing systems need to change too WORKLOAD HARDWARE ADAPT
23 NEW APPLICATIONS existing systems need to change too IMPROVE WORKLOAD HARDWARE WITHIN A BUDGET WHAT WILL BREAK MY SYSTEM? ADAPT REASON
24 new applications more data continuous need for new storage solutions new h/w
25 learning outcome fundamental of storage software engineering data-driven startup research data structures, SQL, NoSQL, Big Data, Neural Networks, Statistics, Data Science
26
27 There is no such thing as a wrong question/answer!!!! interaction: in and out of class
28 Each student: 2 reviews per week/1 presentation review and slides should focus on what is the problem why is it important why is it hard why existing solutions do not work what is the core intuition for the solution solution step by step does the paper prove its claims exact setup of analysis/experiments are there any gaps in the logic/proof possible next steps Recent Research Papers * follow a few citations to gain more background
29 Each student: 2 reviews per week/1 presentation learn to judge constructively review and slides should focus on learn to present learn to prepare slides Recent Research Papers what is the problem why is it important why is it hard why existing solutions do not work what is the core intuition for the solution solution step by step does the paper prove its claims exact setup of analysis/experiments are there any gaps in the logic/proof possible next steps * follow a few citations to gain more background
30 semester project: due in the end of semester + a midway check in (early March,10%) systems project research project
31 semester project: due in the end of semester + a midway check in (early March,10%) systems project research project individual project NoSQL, in c/c++
32 semester project: due in the end of semester + a midway check in (early March,10%) systems project research project groups of three individual project NoSQL, in c/c++ NoSQL, Neural Networks Periodic Table of Data Structures only open to cs165 students (unless proven otherwise)
33 semester project: due in the end of semester + a midway check in (early March,10%) systems project research project groups of three individual project NoSQL, in c/c++ NoSQL, Neural Networks Periodic Table of Data Structures only open to cs165 students (unless proven otherwise)
34 ACM Special Interest Group In Data Management (SIGMOD) Undergrad Research Competition first prize in 2016, 2017, 2018 Adaptive Denormalization Evolving Trees Splaying LSM-Trees
35 ACM Special Interest Group In Data Management (SIGMOD) Undergrad Research Competition first prize in 2016, 2017, 2018 Adaptive Denormalization Evolving Trees Splaying LSM-Trees Design continuums at CIDR 2019, LSM/BTree hybrids in SIGMOD finals
36 piazza forum classes are recorded (links on class website) all announcements & discussions (link on class website - check out usage guidelines) Project: 40% Midway Check-in:10% Discussion: 20% Presentation: 15% Reviews: 15% NO LAPTOP/PHONE POLICY class is based on participation!
37 Get familiar with the very basics of traditional database architectures: Architecture of a Database System. By J. Hellerstein, M. Stonebraker and J. Hamilton. Foundations and Trends in Databases, 2007 Get familiar with very basics of modern database architectures: The Design and Implementation of Modern Column-store Database Systems. By D. Abadi, P. Boncz, S. Harizopoulos, S. Idreos, S. Madden. Foundations and Trends in Databases, 2013 Get familiar with the very basics of modern large scale systems: Massively Parallel Databases and MapReduce Systems. By Shivnath Babu and Herodotos Herodotou. Foundations and Trends in Databases, 2013 Check out: syllabus, preparation readings, project 0, systems project, online sections
38 Teaching Fellows: subarna wilson wasay Off class discussions are key! question on readings, ideas, help with code/analysis
39 questions on logistics?
40 Next few classes: BASICS of storage Intro to RESEACH topics Discussion phase/presentation as of week 3 ask/answer questions, keep notes/ideas
41 cheaper faster CPU registers on chip cache on board cache memory disk SRAM DRAM cache miss: looking for something which is not in the cache ~1ns ~10ns ~100ns speed memory wall memory miss: looking for something which is not in memory cpu mem time
42 cheaper faster CPU registers on chip cache on board cache memory disk SRAM DRAM cache miss: looking for something which is not in the cache ~1ns ~10ns ~100ns speed memory wall memory miss: looking for something which is not in memory cpu mem time
43 registers my head ~0 2x on chip cache this room 1 min 10x on board cache this building 10 min 100x memory New York 1.5 hours Jim Gray, IBM, Tandem, DEC, Microsoft ACM Turing award ACM SIGMOD Edgar F. Codd Innovations award 100Kx disk Pluto 2 years
44 CPU need to only read x but have to read all of page 1 registers data value x on chip cache page1 page2 page3 data move on board cache memory disk
45 query x<5 (size=120 bytes) memory level N memory level N page size: 5x8 bytes
46 query x<5 scan (size=120 bytes) memory level N memory level N page size: 5x8 bytes
47 query x<5 scan (size=120 bytes) memory level N memory level N page size: 5x8 bytes
48 40 bytes query x<5 scan (size=120 bytes) memory level N memory level N page size: 5x8 bytes
49 40 bytes query x<5 scan scan (size=120 bytes) memory level N memory level N page size: 5x8 bytes
50 40 bytes query x<5 scan scan (size=120 bytes) memory level N memory level N page size: 5x8 bytes
51 80 bytes query x<5 scan scan (size=120 bytes) memory level N memory level N page size: 5x8 bytes
52 80 bytes query x<5 (size=120 bytes) memory level N memory level N page size: 5x8 bytes
53 80 bytes query x<5 scan (size=120 bytes) memory level N memory level N page size: 5x8 bytes
54 80 bytes query x<5 scan (size=120 bytes) memory level N memory level N page size: 5x8 bytes
55 120 bytes query x<5 scan (size=120 bytes) memory level N memory level N page size: 5x8 bytes
56 an oracle gives us the positions query x<5 (size=120 bytes) memory level N memory level N page size: 5x8 bytes
57 an oracle gives us the positions query x<5 oracle (size=120 bytes) memory level N memory level N page size: 5x8 bytes
58 an oracle gives us the positions query x<5 oracle (size=120 bytes) memory level N memory level N page size: 5x8 bytes
59 an oracle gives us the positions 40 bytes query x<5 oracle (size=120 bytes) memory level N memory level N page size: 5x8 bytes
60 an oracle gives us the positions 40 bytes query x<5 oracle oracle (size=120 bytes) memory level N memory level N page size: 5x8 bytes
61 an oracle gives us the positions 40 bytes query x<5 oracle oracle (size=120 bytes) memory level N memory level N page size: 5x8 bytes
62 an oracle gives us the positions 80 bytes query x<5 oracle oracle (size=120 bytes) memory level N memory level N page size: 5x8 bytes
63 an oracle gives us the positions 80 bytes query x<5 (size=120 bytes) memory level N memory level N page size: 5x8 bytes
64 an oracle gives us the positions 80 bytes query x<5 oracle (size=120 bytes) memory level N memory level N page size: 5x8 bytes
65 an oracle gives us the positions 80 bytes query x<5 oracle (size=120 bytes) memory level N memory level N page size: 5x8 bytes
66 an oracle gives us the positions 120 bytes query x<5 oracle (size=120 bytes) memory level N memory level N page size: 5x8 bytes
67 when does it make sense to have an oracle how can we minimize the cost e.g., query x<
68 CPU DATA MOVEMENT (ENERGY) MEMORY REQUIREMENT ROBUSTNESS SPACE REQUIREMENT
69 SQL, NoSQL, Graph, Neural Nets, Statistics > Data to Processing CPU DATA MOVEMENT (ENERGY) MEMORY REQUIREMENT ROBUSTNESS SPACE REQUIREMENT
70 SQL, NoSQL, Graph, Neural Nets, Statistics > Data to Processing CPU DATA MOVEMENT (ENERGY) MEMORY REQUIREMENT ROBUSTNESS SPACE REQUIREMENT TIME CLOUD COSTS
71 Stratos Idreos
data systems 101 prof. Stratos Idreos class 2
class 2 data systems 101 prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS265/ 2 classes per week - OH/Labs every day 1 presentation/discussion lead - 2 reviews each week research (or systems)
column-stores basics
class 3 column-stores basics prof. HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS265/ Goetz Graefe Google Research guest lecture Justin Levandoski Microsoft Research projects option 1: systems project (now
data systems 101 prof. Stratos Idreos class 2
class 2 data systems 101 prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS265/ big data V s (it is not about size only) volume velocity variety veracity actually none of that is really new
basic db architectures & layouts
class 4 basic db architectures & layouts prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ videos for sections 3 & 4 are online check back every week (1-2 sections weekly) there is a schedule
column-stores basics
class 3 column-stores basics prof. HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS265/ project description is now online First background info will be given this Friday and detailed lecture on Feb 21 Basic Readings
class 5 column stores 2.0 prof. Stratos Idreos
class 5 column stores 2.0 prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ worth thinking about what just happened? where is my data? email, cloud, social media, can we design systems
Models & Intro to DB Architectures
class 3 Models & Intro to DB Architectures prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ welcome brave cs165 students! 42+44 Stratos Idreos 2 /49 NO LAPTOP/PHONE POLICY class is based
from bits to systems
class 2 from bits to systems prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ today logistics, goals, etc big data & systems (cont d) designing a data system algorithm: what can go wrong
SQL & intro to db architectures
class 3 SQL & intro to db architectures prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ welcome brave cs165 students! 35+62 Stratos Idreos 2 /55 guest lecture Laura Haas Data Systems
class 17 updates prof. Stratos Idreos
class 17 updates prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ UPDATE table_name SET column1=value1,column2=value2,... WHERE some_column=some_value INSERT INTO table_name VALUES (value1,value2,value3,...)
class 17 updates prof. Stratos Idreos
class 17 updates prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ early/late tuple reconstruction, tuple-at-a-time, vectorized or bulk processing, intermediates format, pushing selects
systems & research project
class 4 systems & research project prof. HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS265/ index index knows order about the data data filtering data: point/range queries index data A B C sorted A B C initial
CMPSCI 645 Database Design & Implementation
Welcome to CMPSCI 645 Database Design & Implementation Instructor: Gerome Miklau Overview of Databases Gerome Miklau CMPSCI 645 Database Design & Implementation UMass Amherst Jan 19, 2010 Some slide content
complex plans and hybrid layouts
class 7 complex plans and hybrid layouts prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ essential column-stores features virtual ids late tuple reconstruction (if ever) vectorized execution
Models & Intro to DB Architectures
class 3 Models & Intro to DB Architectures prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ welcome brave cs165 students! Stratos Idreos 2 /55 NO LAPTOP/PHONE POLICY class is based on
class 6 more about column-store plans and compression prof. Stratos Idreos
class 6 more about column-store plans and compression prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ query compilation an ancient yet new topic/research challenge query->sql->interpet
class 9 fast scans 1.0 prof. Stratos Idreos
class 9 fast scans 1.0 prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ 1 pass to merge into 8 sorted pages (2N pages) 1 pass to merge into 4 sorted pages (2N pages) 1 pass to merge into
class 11 b-trees prof. Stratos Idreos
class 11 b-trees prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ Midway check-in: Two design docs tmr (Canvas) & tests on Sunday Next weekend: Lab marathon for midway check-in & tests
Locality. CS429: Computer Organization and Architecture. Locality Example 2. Locality Example
Locality CS429: Computer Organization and Architecture Dr Bill Young Department of Computer Sciences University of Texas at Austin Principle of Locality: Programs tend to reuse data and instructions near
class 20 updates 2.0 prof. Stratos Idreos
class 20 updates 2.0 prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ UPDATE table_name SET column1=value1,column2=value2,... WHERE some_column=some_value INSERT INTO table_name VALUES
Database Architecture 2 & Storage. Instructor: Matei Zaharia cs245.stanford.edu
Database Architecture 2 & Storage Instructor: Matei Zaharia cs245.stanford.edu Summary from Last Time System R mostly matched the architecture of a modern RDBMS» SQL» Many storage & access methods» Cost-based
class 12 b-trees 2.0 prof. Stratos Idreos
class 12 b-trees 2.0 prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ A B C A B C clustered/primary index on A Stratos Idreos /26 2 A B C A B C clustered/primary index on A pos C pos
class 10 b-trees 2.0 prof. Stratos Idreos
class 10 b-trees 2.0 prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ CS Colloquium HV Jagadish Prof University of Michigan 10/6 Stratos Idreos /29 2 CS Colloquium Magdalena Balazinska
Course Introduction & History of Database Systems
Course Introduction & History of Database Systems CMPT 843, SPRING 2018 JIANNAN WANG https://sfu-db.github.io/dbsystems/ CMPT 843-2018 SPRING - SFU 1 Introduce Yourself What s your name? Where are you
CAS CS 460/660 Introduction to Database Systems. Fall
CAS CS 460/660 Introduction to Database Systems Fall 2017 1.1 About the course Administrivia Instructor: George Kollios, gkollios@cs.bu.edu MCS 283, Mon 2:30-4:00 PM and Tue 1:00-2:30 PM Teaching Fellows:
CS564: Database Management Systems. Lecture 1: Course Overview. Acks: Chris Ré 1
CS564: Database Management Systems Lecture 1: Course Overview Acks: Chris Ré 1 2 Big science is data driven. 3 Increasingly many companies see themselves as data driven. 4 Even more traditional companies
Internet Scale Storage
Internet Scale Storage SIGMOD 2011 James Hamilton, 2011/6/14 VP & Distinguished Engineer, Amazon Web Services email: James@amazon.com web: mvdirona.com/jrh/work blog: perspectives.mvdirona.com Agenda Cloud
CS 245: Principles of Data-Intensive Systems. Instructor: Matei Zaharia cs245.stanford.edu
CS 245: Principles of Data-Intensive Systems Instructor: Matei Zaharia cs245.stanford.edu Outline Why study data-intensive systems? Course logistics Key issues and themes A bit of history CS 245 2 My Background
CSE 544 Principles of Database Management Systems
CSE 544 Principles of Database Management Systems Alvin Cheung Fall 2015 Lecture 5 - DBMS Architecture and Indexing 1 Announcements HW1 is due next Thursday How is it going? Projects: Proposals are due
CS 61C: Great Ideas in Computer Architecture. Direct Mapped Caches
CS 61C: Great Ideas in Computer Architecture Direct Mapped Caches Instructor: Justin Hsia 7/05/2012 Summer 2012 Lecture #11 1 Review of Last Lecture Floating point (single and double precision) approximates
Computer Systems C S Cynthia Lee Today s materials adapted from Kevin Webb at Swarthmore College
Computer Systems C S 0 7 Cynthia Lee Today s materials adapted from Kevin Webb at Swarthmore College 2 Today s Topics TODAY S LECTURE: Caching ANNOUNCEMENTS: Assign6 & Assign7 due Friday! 6 & 7 NO late
CSE 344 JANUARY 3 RD - INTRODUCTION
CSE 344 JANUARY 3 RD - INTRODUCTION COURSE FORMAT Lectures Location: SIG 134 Please attend Sections: Content: exercises, tutorials, questions, new materials (occasionally) Locations: see web Please attend
Introduction to Data Management CSE 344. Lecture 2: Data Models
Introduction to Data Management CSE 344 Lecture 2: Data Models CSE 344 - Winter 2017 1 Announcements WQ1 and HW1 are out Use your CSE ids to access the HW docs Use Piazza to post questions OHs are up on
ENGIN 112 Intro to Electrical and Computer Engineering
ENGIN 112 Intro to Electrical and Computer Engineering Lecture 30 Random Access Memory (RAM) Overview Memory is a collection of storage cells with associated input and output circuitry Possible to read
Modern Database Systems CS-E4610
Modern Database Systems CS-E4610 Aristides Gionis Michael Mathioudakis Spring 2017 what is a database? a collection of data what is a database management system?... a.k.a. database system software to store,
CS61C : Machine Structures
inst.eecs.berkeley.edu/~cs61c/su05 CS61C : Machine Structures Lecture #21: Caches 3 2005-07-27 CS61C L22 Caches III (1) Andy Carle Review: Why We Use Caches 1000 Performance 100 10 1 1980 1981 1982 1983
CMPT 354: Database System I. Lecture 1. Course Introduction
CMPT 354: Database System I Lecture 1. Course Introduction 1 Outline Motivation for studying this course Course admin and set up Overview of course topics 2 Trend 1: Data grows exponentially 1 ZB = 1,
Fundamentals of Database Systems
Fundamentals of Database Systems Semester 1, 2017 Fundamentals of Database Systems COMPSCI/SOFTENG 351 COMPSCI 751 Instructors: Gill Dobbie, Miika Hannula, Sebastian Link, Gerald Weber Department of Computer
CSE 544 Principles of Database Management Systems. Alvin Cheung Fall 2015 Lecture 8 - Data Warehousing and Column Stores
CSE 544 Principles of Database Management Systems Alvin Cheung Fall 2015 Lecture 8 - Data Warehousing and Column Stores Announcements Shumo office hours change See website for details HW2 due next Thurs
Paging! 2/22! Anthony D. Joseph and Ion Stoica CS162 UCB Fall 2012! " (0xE0)" " " " (0x70)" " (0x50)"
CS162 Operating Systems and Systems Programming Lecture 10 Caches and TLBs" February 22, 2011! Anthony D. Joseph and Ion Stoica! http//inst.eecs.berkeley.edu/~cs162! Segmentation! Paging! Recap Segmentation
CompSci 516: Database Systems. Lecture 20. Parallel DBMS. Instructor: Sudeepa Roy
CompSci 516 Database Systems Lecture 20 Parallel DBMS Instructor: Sudeepa Roy Duke CS, Fall 2017 CompSci 516: Database Systems 1 Announcements HW3 due on Monday, Nov 20, 11:55 pm (in 2 weeks) See some
+ Random-Access Memory (RAM)
+ Memory Subsystem + Random-Access Memory (RAM) Key features RAM is traditionally packaged as a chip. Basic storage unit is normally a cell (one bit per cell). Multiple RAM chips form a memory. RAM comes
CS 61C: Great Ideas in Computer Architecture. The Memory Hierarchy, Fully Associative Caches
CS 61C: Great Ideas in Computer Architecture The Memory Hierarchy, Fully Associative Caches Instructor: Alan Christopher 7/09/2014 Summer 2014 -- Lecture #10 1 Review of Last Lecture Floating point (single
Announcements. Using Electronics in Class. Review. Staff Instructor: Alvin Cheung Office hour on Wednesdays, 1-2pm. Class Overview
Announcements Introduction to Databases CSE 414 Lecture 2: Data Models HW1 and WQ1 released Both due next Tuesday Office hours start this week Sections tomorrow Make sure you sign up on piazza Please ask
CS145: Intro to Databases. Lecture 1: Course Overview
CS145: Intro to Databases Lecture 1: Course Overview 1 The world is increasingly driven by data This class teaches the basics of how to use & manage data. 2 Key Questions We Will Answer How can we collect
Why are you here? Introduction. Course roadmap. Course goals. What do you want from a DBMS? What is a database system? Aren t databases just
Why are you here? 2 Introduction CPS 216 Advanced Database Systems Aren t databases just Trivial exercises in first-order logic (says AI)? Bunch of out-of-fashion I/O-efficient indexes and algorithms (says
Caches II CSE 351 Spring #include <yoda.h> int is_try(int do_flag) { return!(do_flag!do_flag); }
Caches II CSE 351 Spring 2018 #include int is_try(int do_flag) { return!(do_flag!do_flag); } Memory Hierarchies Some fundamental and enduring properties of hardware and software systems: Faster
CS639: Data Management for Data Science. Lecture 1: Intro to Data Science and Course Overview. Theodoros Rekatsinas
CS639: Data Management for Data Science Lecture 1: Intro to Data Science and Course Overview Theodoros Rekatsinas 1 2 Big science is data driven. 3 Increasingly many companies see themselves as data driven.
CMPSC 311- Introduction to Systems Programming Module: Caching
CMPSC 311- Introduction to Systems Programming Module: Caching Professor Patrick McDaniel Fall 2016 Reminder: Memory Hierarchy L0: Registers CPU registers hold words retrieved from L1 cache Smaller, faster,
Morsel- Drive Parallelism: A NUMA- Aware Query Evaluation Framework for the Many- Core Age. Presented by Dennis Grishin
Morsel- Drive Parallelism: A NUMA- Aware Query Evaluation Framework for the Many- Core Age Presented by Dennis Grishin What is the problem? Efficient computation requires distribution of processing between
CS162 Operating Systems and Systems Programming Lecture 13. Caches and TLBs. Page 1
CS162 Operating Systems and Systems Programming Lecture 13 Caches and TLBs March 12, 2008 Prof. Anthony D. Joseph http//inst.eecs.berkeley.edu/~cs162 Review Multi-level Translation What about a tree of
CMPT 354: Database System I. Lecture 7. Basics of Query Optimization
CMPT 354: Database System I Lecture 7. Basics of Query Optimization 1 Why should you care? https://databricks.com/glossary/catalyst-optimizer https://sigmod.org/sigmod-awards/people/goetz-graefe-2017-sigmod-edgar-f-codd-innovations-award/
Storing Data: Disks and Files
Storing Data: Disks and Files CS 186 Fall 2002, Lecture 15 (R&G Chapter 7) Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet Stuff Rest of this week My office
CS61C : Machine Structures
inst.eecs.berkeley.edu/~cs61c CS61C : Machine Structures CS61C L22 Caches II (1) CPS today! Lecture #22 Caches II 2005-11-16 There is one handout today at the front and back of the room! Lecturer PSOE,
Distributed Data Infrastructures, Fall 2017, Chapter 2. Jussi Kangasharju
Distributed Data Infrastructures, Fall 2017, Chapter 2 Jussi Kangasharju Chapter Outline Warehouse-scale computing overview Workloads and software infrastructure Failures and repairs Note: Term Warehouse-scale
CS 261 Fall Mike Lam, Professor. Virtual Memory
CS 261 Fall 2016 Mike Lam, Professor Virtual Memory Topics Operating systems Address spaces Virtual memory Address translation Memory allocation Lingering questions What happens when you call malloc()?
MEMORY HIERARCHY DESIGN
MEMORY HIERARCHY DESIGN Mahdi Nazm Bojnordi Assistant Professor School of Computing University of Utah CS/ECE 6810: Computer Architecture Overview Announcement Homework 3 will be released on Oct. 31 st
CSE 544 Principles of Database Management Systems
CSE 544 Principles of Database Management Systems Lecture 1 - Introduction and the Relational Model 1 Outline Introduction Class overview Why database management systems (DBMS)? The relational model 2
CS162 Operating Systems and Systems Programming Lecture 10 Caches and TLBs"
CS162 Operating Systems and Systems Programming Lecture 10 Caches and TLBs" October 1, 2012! Prashanth Mohan!! Slides from Anthony Joseph and Ion Stoica! http://inst.eecs.berkeley.edu/~cs162! Caching!
Professor Yashar Ganjali Department of Computer Science University of Toronto.
Professor Yashar Ganjali Department of Computer Science University of Toronto yganjali@cs.toronto.edu http://www.cs.toronto.edu/~yganjali Today Outline What this course is about Logistics Course structure,
Computing Curricula 2005
Computing Curricula Santiago, Chile 6 March John Impagliazzo Computing Curricula An Overview Report John Impagliazzo Computer Science Hofstra University New York John Impagliazzo Computing Curricula Santiago,
Big Data Processing Technologies. Chentao Wu Associate Professor Dept. of Computer Science and Engineering
Big Data Processing Technologies Chentao Wu Associate Professor Dept. of Computer Science and Engineering wuct@cs.sjtu.edu.cn Schedule (1) Storage system part (first eight weeks) lec1: Introduction on
Database Systems: Concepts, design, and implementation ISE 382 (3 Units)
Database Systems: Concepts, design, and implementation ISE 382 (3 Units) Spring 2013 Description Obectives Instructor Contact Information Office Hours Concepts in modeling data for industry applications.
Disks and Files. Jim Gray s Storage Latency Analogy: How Far Away is the Data? Components of a Disk. Disks
Review Storing : Disks and Files Lecture 3 (R&G Chapter 9) Aren t bases Great? Relational model SQL Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet A few
CS / Cloud Computing. Recitation 3 September 9 th & 11 th, 2014
CS15-319 / 15-619 Cloud Computing Recitation 3 September 9 th & 11 th, 2014 Overview Last Week s Reflection --Project 1.1, Quiz 1, Unit 1 This Week s Schedule --Unit2 (module 3 & 4), Project 1.2 Questions
Advanced Operating Systems (CS 202)
Advanced Operating Systems (CS 202) Presenter today: Khaled N. Khasawneh Instructor: Nael Abu-Ghazaleh Jan, 9, 2016 Today Course organization and mechanics Introduction to OS 2 What is this course about?
Module 1: Basics and Background Lecture 4: Memory and Disk Accesses. The Lecture Contains: Memory organisation. Memory hierarchy. Disks.
The Lecture Contains: Memory organisation Example of memory hierarchy Memory hierarchy Disks Disk access Disk capacity Disk access time Typical disk parameters Access times file:///c /Documents%20and%20Settings/iitkrana1/My%20Documents/Google%20Talk%20Received%20Files/ist_data/lecture4/4_1.htm[6/14/2012
Chapter 5. Memory Technology
Chapter 5 Large and Fast: Exploiting Memory Hierarchy Memory Technology Static RAM (SRAM) 0.5ns 2.5ns, $2000 $5000 per GB Dynamic RAM (DRAM) 50ns 70ns, $20 $75 per GB Magnetic disk 5ms 20ms, $0.20 $2 per
Lecture-14 (Memory Hierarchy) CS422-Spring
Lecture-14 (Memory Hierarchy) CS422-Spring 2018 Biswa@CSE-IITK The Ideal World Instruction Supply Pipeline (Instruction execution) Data Supply - Zero-cycle latency - Infinite capacity - Zero cost - Perfect
NoSQL database and its business applications
COSC 657 Db. Management Systems Professor: RAMESH K. Student: BUER JIANG Research paper NoSQL database and its business applications The original purpose has been contemporary web-expand dbs. The movement
Overview of Data Exploration Techniques. Stratos Idreos, Olga Papaemmanouil, Surajit Chaudhuri
Overview of Data Exploration Techniques Stratos Idreos, Olga Papaemmanouil, Surajit Chaudhuri data exploration not always sure what we are looking for (until we find it) data has always been big volume
CTL.SC4x Technology and Systems
in Supply Chain Management CTL.SC4x Technology and Systems Key Concepts Document This document contains the Key Concepts for the SC4x course, Weeks 1 and 2. These are meant to complement, not replace,
class 8 b-trees prof. Stratos Idreos
class 8 b-trees prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ I spend a lot of time debugging am I doing something wrong? maybe but probably not 1. learn to use gdb 2. after spending
CS 31: Intro to Systems Caching. Kevin Webb Swarthmore College March 24, 2015
CS 3: Intro to Systems Caching Kevin Webb Swarthmore College March 24, 205 Reading Quiz Abstraction Goal Reality: There is no one type of memory to rule them all! Abstraction: hide the complex/undesirable
SYLLABUS. 3. Total estimated time (hours/semester of didactic activities) 3.1 Hours per week 3 Of which: 3.2 course seminar/laboratory1 sem
SYLLABUS 1. Information regarding the programme 1.1 Higher education institution Babeş-Bolyai University of Cluj-Napoca 1.2 Faculty Faculty of Mathematics and Computer Science 1.3 Department Department
Modernizing Healthcare IT for the Data-driven Cognitive Era Storage and Software-Defined Infrastructure
Modernizing Healthcare IT for the Data-driven Cognitive Era Storage and Software-Defined Infrastructure An IDC InfoBrief, Sponsored by IBM April 2018 Executive Summary Today s healthcare organizations
Page 1. Goals for Today" TLB organization" CS162 Operating Systems and Systems Programming Lecture 11. Page Allocation and Replacement"
Goals for Today" CS162 Operating Systems and Systems Programming Lecture 11 Page Allocation and Replacement" Finish discussion on TLBs! Page Replacement Policies! FIFO, LRU! Clock Algorithm!! Working Set/Thrashing!
Introduction to Data Management. Lecture #2 (Big Picture, Cont.)
Introduction to Data Management Lecture #2 (Big Picture, Cont.) Instructor: Mike Carey mjcarey@ics.uci.edu Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Announcements v Still hanging
Evolution of Database Systems
Evolution of Database Systems Krzysztof Dembczyński Intelligent Decision Support Systems Laboratory (IDSS) Poznań University of Technology, Poland Intelligent Decision Support Systems Master studies, second
Modeling and evaluation on Ad hoc query processing with Adaptive Index in Map Reduce Environment
DEIM Forum 213 F2-1 Adaptive indexing 153 855 4-6-1 E-mail: {okudera,yokoyama,miyuki,kitsure}@tkl.iis.u-tokyo.ac.jp MapReduce MapReduce MapReduce Modeling and evaluation on Ad hoc query processing with
Principles of Data Management. Lecture #2 (Storing Data: Disks and Files)
Principles of Data Management Lecture #2 (Storing Data: Disks and Files) Instructor: Mike Carey mjcarey@ics.uci.edu Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Today s Topics v Today
Information Security CS526
Information CS 526 Topic 3 Ciphers and Cipher : Stream Ciphers, Block Ciphers, Perfect Secrecy, and IND-CPA 1 Announcements HW1 is out, due on Sept 10 Start early, late policy is 3 total late days for
CISC 360. Cache Memories Exercises Dec 3, 2009
Topics ν CISC 36 Cache Memories Exercises Dec 3, 29 Review of cache memory mapping Cache Memories Cache memories are small, fast SRAM-based memories managed automatically in hardware. ν Hold frequently
Introduction to Data Management. Lecture 14 (Storage and Indexing)
Introduction to Data Management Lecture 14 (Storage and Indexing) Instructor: Mike Carey mjcarey@ics.uci.edu Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Announcements v HW s and quizzes:
CompSci 516 Database Systems
CompSci 516 Database Systems Lecture 20 NoSQL and Column Store Instructor: Sudeepa Roy Duke CS, Fall 2018 CompSci 516: Database Systems 1 Reading Material NOSQL: Scalable SQL and NoSQL Data Stores Rick
CSC Memory System. A. A Hierarchy and Driving Forces
CSC1016 1. System A. A Hierarchy and Driving Forces 1A_1 The Big Picture: The Five Classic Components of a Computer Processor Input Control Datapath Output Topics: Motivation for Hierarchy View of Hierarchy
CSC 261/461 Database Systems. Fall 2017 MW 12:30 pm 1:45 pm CSB 601
CSC 261/461 Database Systems Fall 2017 MW 12:30 pm 1:45 pm CSB 601 Agenda Administrative aspects Brief overview of the course Introduction to databases and SQL ADMINISTRATIVE ASPECTS Teaching Staff Instructor:
61A LECTURE 1 FUNCTIONS, VALUES. Steven Tang and Eric Tzeng June 24, 2013
61A LECTURE 1 FUNCTIONS, VALUES Steven Tang and Eric Tzeng June 24, 2013 Welcome to CS61A! The Course Staff - Lecturers Steven Tang Graduated L&S CS from Cal Back for a PhD in Education Eric Tzeng Graduated
Recent Innovations in Data Storage Technologies Dr Roger MacNicol Software Architect
Recent Innovations in Data Storage Technologies Dr Roger MacNicol Software Architect Copyright 2017, Oracle and/or its affiliates. All rights reserved. Safe Harbor Statement The following is intended to
Topics. History. Architecture. MongoDB, Mongoose - RDBMS - SQL. - NoSQL
Databases Topics History - RDBMS - SQL Architecture - SQL - NoSQL MongoDB, Mongoose Persistent Data Storage What features do we want in a persistent data storage system? We have been using text files to
Department of Information Technology B.E/B.Tech : CSE/IT Regulation: 2013 Sub. Code / Sub. Name : CS6302 Database Management Systems
COURSE DELIVERY PLAN - THEORY Page 1 of 6 Department of Information Technology B.E/B.Tech : CSE/IT Regulation: 2013 Sub. Code / Sub. Name : CS6302 Database Management Systems Unit : I LP: CS6302 Rev. :
/ Cloud Computing. Recitation 3 Sep 13 & 15, 2016
15-319 / 15-619 Cloud Computing Recitation 3 Sep 13 & 15, 2016 1 Overview Administrative Issues Last Week s Reflection Project 1.1, OLI Unit 1, Quiz 1 This Week s Schedule Project1.2, OLI Unit 2, Module
Parallel DBMS. Chapter 22, Part A
Parallel DBMS Chapter 22, Part A Slides by Joe Hellerstein, UCB, with some material from Jim Gray, Microsoft Research. See also: http://www.research.microsoft.com/research/barc/gray/pdb95.ppt Database
Overview of Data Management
Overview of Data Management Grant Weddell Cheriton School of Computer Science University of Waterloo CS 348 Introduction to Database Management Spring 2016 CS 348 (Intro to DB Mgmt) Overview of Data Management
Cycle Time for Non-pipelined & Pipelined processors
Cycle Time for Non-pipelined & Pipelined processors Fetch Decode Execute Memory Writeback 250ps 350ps 150ps 300ps 200ps For a non-pipelined processor, the clock cycle is the sum of the latencies of all
Let!s go back to a course goal... Let!s go back to a course goal... Question? Lecture 22 Introduction to Memory Hierarchies
1 Lecture 22 Introduction to Memory Hierarchies Let!s go back to a course goal... At the end of the semester, you should be able to......describe the fundamental components required in a single core of
BBM371- Data Management. Lecture 1: Course policies, Introduction to DBMS
BBM371- Data Management Lecture 1: Course policies, Introduction to DBMS 26.09.2017 Today Introduction About the class Organization of this course Introduction to Database Management Systems (DBMS) About
Algorithm Performance Factors. Memory Performance of Algorithms. Processor-Memory Performance Gap. Moore s Law. Program Model of Memory I
Memory Performance of Algorithms CSE 32 Data Structures Lecture Algorithm Performance Factors Algorithm choices (asymptotic running time) O(n 2 ) or O(n log n) Data structure choices List or Arrays Language
1/3/2015. Column-Store: An Overview. Row-Store vs Column-Store. Column-Store Optimizations. Compression Compress values per column
//5 Column-Store: An Overview Row-Store (Classic DBMS) Column-Store Store one tuple ata-time Store one column ata-time Row-Store vs Column-Store Row-Store Column-Store Tuple Insertion: + Fast Requires
USC Viterbi School of Engineering
Introduction to Computational Thinking and Data Science USC Viterbi School of Engineering http://www.datascience4all.org Term: Fall 2016 Time: Tues- Thur 10am- 11:50am Location: Allan Hancock Foundation