from bits to systems
|
|
|
- Jade Andrews
- 6 years ago
- Views:
Transcription
1 class 2 from bits to systems prof. Stratos Idreos
2 today logistics, goals, etc big data & systems (cont d) designing a data system algorithm: what can go wrong quick overview: logical/physical/system design Stratos Idreos 2 /50
3 guest lecture Marcin Zukowski previously 10/17 Stratos Idreos 3 /50
4 guest lecture Johannes Gehrke Microsoft previously prof at Cornell 10/24 Stratos Idreos 4 /50
5 CS Colloquium HV Jagadish Prof University of Michigan 10/6 Stratos Idreos 5 /50
6 CS Colloquium Magdalena Balazinska Prof University of Washington 10/13 Stratos Idreos 6 /50
7 research oriented, discussion based, material based on research papers Stratos Idreos 7 /50
8 labs, sections, and cs nights Sections: video only / 1-2 videos a week first 2 sections on Project 0 and basics of C are online Labs (& CS Night): every day help with debugging and everything else starting Tuesday Sep 6 OH: every day 3-4pm with Stratos at MD139 away next week but most likely I will be able to hold OH remotely most days monitor Piazza and learn to use Zoom Stratos Idreos 8 /50
9 code and testing infrastructure available as of today treat it as a good starting point you have complete design freedom no libraries allowed collaboration encouraged! final deliverable is personal face-to-face/hands-on evaluation stay in touch! Stratos Idreos 9 /50
10 James & Wilson spring 2014, somewhere at Harvard, late at night working on cs165 project Stratos Idreos 10/50
11 James & Wilson spring 2014, somewhere at Harvard, late at night working on cs165 project several months later This class is invaluable. One of the greatest intellectual journeys to date. Stratos Idreos 10/50
12 Project is brutal and rewarding. student testimonials (from Q) I absolutely, wholeheartedly loved 165!!!! TAKE THIS CLASS IF YOU WANT TO ACTUALLY LEARN SOMETHING AND BECOME A TRUE COMPUTER SCIENTIST! The course is amazing! It's relaxed, it teaches you about modern database systems, it introduces you to current research topics, it's freaking incredible! CS is awesome and Databases rock! I came in not being very con0ident in my cs skills or my understanding of how memory hierarchy and large systems work and now everything is changed! I also just feel very inspired to explore more in the area of databases in the future! Loved this class! Stratos Idreos 11/50
13 FAQ should I take the class now or next year (or never)? it depends (but mostly up to you and your energy levels) delaying can be better if you not senior unless you have research ambitions can I take it pass fail? no can I audit? yes (but limited slots) Stratos Idreos 12/50
14 cs165 diff from 2015 cleaned up DSL cleaned up starter code give out parser code revamped testing infrastructure performance tests (=bigger data) benchmark implementation project 0 midway check-in 10% class participation 20% more TFs (lab class status) addition of new research papers from 2015/2016 Stratos Idreos 13/50
15 we purposely build up slowly in class from your side: Project 0 next week and basic environment/tools for project then start with P1 the week after once we discuss the basics of storage Stratos Idreos 14/50
16 astronomer data star(id, name, distance, density, ) [1, star1, x1, y1, ] [2, star2, x2, y2, ] [3, star3, x3, y3, ] [4, star4, x4, y4, ] Stratos Idreos 15/50
17 astronomer data size 10s/100s store - access paper - just look at it! data collection is the key data star(id, name, distance, density, ) [1, star1, x1, y1, ] [2, star2, x2, y2, ] [3, star3, x3, y3, ] [4, star4, x4, y4, ] Stratos Idreos 15/50
18 astronomer data size 10s/100s store - access paper - just look at it! data collection is the key K/M PC files - shell/excel data learn a bit how computers work star(id, name, distance, density, ) [1, star1, x1, y1, ] [2, star2, x2, y2, ] [3, star3, x3, y3, ] [4, star4, x4, y4, ] Stratos Idreos 15/50
19 astronomer data size 10s/100s store - access paper - just look at it! data collection is the key K/M PC files - shell/excel data star(id, name, distance, density, ) [1, star1, x1, y1, ] [2, star2, x2, y2, ] [3, star3, x3, y3, ] [4, star4, x4, y4, ] learn a bit how computers work need a bit more tailored analysis B PC files - custom need serious programming skills Stratos Idreos 15/50
20 astronomer data size 10s/100s store - access paper - just look at it! data collection is the key K/M PC files - shell/excel data star(id, name, distance, density, ) [1, star1, x1, y1, ] [2, star2, x2, y2, ] [3, star3, x3, y3, ] [4, star4, x4, y4, ] learn a bit how computers work need a bit more tailored analysis B PC files - custom need serious programming skills exploration - many users/updates T data sys. - declarative data-driven analysis Stratos Idreos 15/50
21 Increasingly, scientific breakthroughs will be powered by advanced computing capabilities that help researchers manipulate and explore massive datasets. The speed at which any given scientific discipline advances will depend on how well its researchers collaborate with one another, and with technologists, in areas of escience such as databases, workflow management, visualization, and cloud computing technologies. Stratos Idreos 16/50
22 big data V s (it is not about size only) volume velocity variety veracity actually none of that is really new new: our ability to gather and store machine generated data broad understanding that we cannot just manually get value out of data Stratos Idreos 17/50
23 declarative interface ask what you want db system the system decides how to best store and access data Stratos Idreos 18/50
24 a data system stores data and provides access to data & makes knowledge generation easy Stratos Idreos 19/50
25 a data system stores data and provides access to data & makes knowledge generation easy data system data analysis knowledge Stratos Idreos 19/50
26 a data system stores data and provides access to data & makes knowledge generation easy data system X data analysis knowledge Stratos Idreos 19/50
27 (here is where all the magic happens!) data system kernel cs165/265 student Stratos Idreos 20/50
28 applications sql database kernel algorithms/operators cpu memory data data data disk Stratos Idreos 21/50
29 a data system is a massive collection of algorithms and data structures more than one ways to do the same thing optimization: a smart way to dynamically decide which way to choose for each query no good way to model a system Stratos Idreos 22/50
30 a data system is a massive collection of algorithms and data structures more than one ways to do the same thing optimization: a smart way to dynamically decide which way to choose for each query no good way to model a system Research Problem 1 Stratos Idreos 22/50
31 so what is a good data system? it depends conflicting goals application requirements performance budget hardware energy profile Stratos Idreos 23/50
32 Three things are important in the database world: performance, performance, and performance Bruce Lindsay, IBM ACM SIGMOD Edgar F. Codd Innovations award 2012 Stratos Idreos 24/50
33 a simple example assume an array of N integers: find all positions where value>x qualifying positions select operator exists in all systems: sql, nosql, newsql data even the simplest tasks are actually far from trivial no obvious solutions; just a taste of what to come Stratos Idreos 25/50
34 assume an array of N integers: find all positions where value>x res=new array[data.size] j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i Stratos Idreos 26/50
35 assume an array of N integers: find all positions where value>x res=new array[data.size] j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i res[j]=i; j+=(data[i]>x) Stratos Idreos 26/50
36 assume an array of N integers: find all positions where value>x res=new array[data.size] j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i what if only 1% qualifies? Stratos Idreos 26/50
37 assume an array of N integers: find all positions where value>x res=new array[data.size] what if only 1% qualifies? j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i memory data Stratos Idreos 26/50
38 assume an array of N integers: find all positions where value>x res=new array[data.size] what if only 1% qualifies? j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i memory data res Stratos Idreos 26/50
39 assume an array of N integers: find all positions where value>x res=new array[data.size] what if only 1% qualifies? j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i memory data Stratos Idreos 26/50
40 assume an array of N integers: find all positions where value>x res=new array[data.size] what if only 1% qualifies? j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i memory data Stratos Idreos 26/50
41 assume an array of N integers: find all positions where value>x res=new array[data.size] what if only 1% qualifies? j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i memory data Stratos Idreos 26/50
42 assume an array of N integers: find all positions where value>x res=new array[data.size] what if only 1% qualifies? j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i memory data Stratos Idreos 26/50
43 assume an array of N integers: find all positions where value>x res=new array[data.size] what if only 1% qualifies? j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i memory data Stratos Idreos 26/50
44 assume an array of N integers: find all positions where value>x res=new array[data.size] what if only 1% qualifies? j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i memory data Stratos Idreos 26/50
45 assume an array of N integers: find all positions where value>x res=new array[data.size] what if only 1% qualifies? j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i memory data Stratos Idreos 26/50
46 assume an array of N integers: find all positions where value>x res=new array[data.size] what if only 1% qualifies? j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i memory data copy res Stratos Idreos 26/50
47 assume an array of N integers: find all positions where value>x res=new array[data.size] what if only 1% qualifies? j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i data but how can we know? memory copy res Stratos Idreos 26/50
48 assume an array of N integers: find all positions where value>x res=new array[data.size] j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i what if 90% qualifies? result size= qualifying values*w bytes Stratos Idreos 27/50
49 assume an array of N integers: find all positions where value>x res=new array[data.size] what if 90% qualifies? j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i result size= qualifying values*w bytes bit vector for res? Stratos Idreos 27/50 1 vs
50 assume an array of N integers: find all positions where value>x res=new array[data.size] what if 90% qualifies? j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i result size= qualifying values*w bytes bit vector for res? if statements= bad, bad, bad Stratos Idreos 27/ vs
51 assume an array of N integers: find all positions where value>x and we haven t even started discussing about how to find the qualifying values res=new array[data.size] what if 90% qualifies? j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i result size= qualifying values*w bytes bit vector for res? if statements= bad, bad, bad Stratos Idreos 27/ vs
52 assume an array of N integers: find all positions where value>x thread1, core1 res=new array[data.size] thread2, core2 j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i logically partition thread3, core3 Stratos Idreos 28/50
53 assume an array of N integers: find all positions where value>x thread1, core1 res=new array[data.size] thread2, core2 j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i logically partition thread3, core3 NUMA architectures? SIMD functionality? & what about result writing? not as simple as spinning off N threads Stratos Idreos 28/50
54 assume an array of N integers: find all positions where value>x N>>1 queries in parallel res=new array[data.size] j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i q1,q2,q3 Stratos Idreos 29/50
55 assume an array of N integers: find all positions where value>x N>>1 queries in parallel res=new array[data.size] j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i q1,q2,q3 Stratos Idreos 29/50
56 assume an array of N integers: find all positions where value>x N>>1 queries in parallel res=new array[data.size] j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i q1,q2,q3 Stratos Idreos 29/50
57 assume an array of N integers: find all positions where value>x N>>1 queries in parallel res=new array[data.size] j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i q1,q2,q3 q4 Stratos Idreos 29/50
58 assume an array of N integers: find all positions where value>x N>>1 queries in parallel res=new array[data.size] j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i q1,q2,q3 q4 q4 Stratos Idreos 29/50
59 assume an array of N integers: find all positions where value>x res=new array[data.size] j=0 for (i=0; i<data.size; i++) if data[i]>x res[j++]=i vs res=new array[10] j=0 if data[0]>x res[j++]=0 if data[1]>x res[j++]=1 if data[2]>x res[j++]=2 if data[3]>x res[j++]=3 if data[4]>x res[j++]=4 if data[5]>x res[j++]=5 if data[6]>x res[j++]=6 if data[7]>x res[j++]=7 if data[8]>x res[j++]=8 if data[9]>x res[j++]=9 Stratos Idreos 30/50
60 cost: data touched & computation speed cpu mem time Stratos Idreos 31/50
61 random access & page-based access need to only read x but have to read all of page 1 CPU registers data value x on chip cache page1 page2 page3 data move on board cache memory disk Stratos Idreos 32/50
62 assume an array of N integers: find all positions where value>x option 1: scan all data option 2: use a tree (do not consider tree generation costs) which one is best? groups 3-4 students: try to keep noise level down, take turns speaking, etc. Stratos Idreos 33/50
63 it is not obvious that the tree would give the best result even if we ignore the building cost random access at the granularity of one page at a time data movement is a HUGE cost component it depends on the query and the data >1 ways to have a tree (implicit tree, linked leaves or not, etc.) and happens with updates, >>1 concurrent queries, etc. Stratos Idreos 34/50
64 actually this was your second open research problem of the day! systems are so fast= too little time to optimize we will do all of these in great detail: the examples so far are meant to demonstrate the kinds of problems and discussion we will be doing Stratos Idreos 35/50
65 design space it all starts with how we store data every bit matters Stratos Idreos 36/50
66 C and no libraries Stratos Idreos 37/50
67 MAIN-MEMORY OPTIMIZED DATA SYSTEMS Stratos Idreos 38/50
68 MAIN-MEMORY OPTIMIZED DATA SYSTEMS Stratos Idreos 38/50
69 MAIN-MEMORY OPTIMIZED DATA SYSTEMS Stratos Idreos 38/50
70 philosophy/ sciences ~1960s declarative/ physical/logical independence recovery/ consistency ~2015 papyrus/ paper dbs dbs dbs vs OSs dbs dbs vs nosql dbs vs hand code history/timeline Stratos Idreos 39/50
71 db complex legacy tuning expensive fast nosql simple clean just enough cheap/free slow why nosql as apps become more complex as apps need to be more scalable newsql Stratos Idreos 40/50
72 db complex legacy tuning expensive fast nosql simple clean just enough cheap/free slow why nosql as apps become more complex as apps need to be more scalable regardless of system name/data model it is the same design space and the same design principles newsql Stratos Idreos 40/50
73 design logical design physical design system design Stratos Idreos 41/50
74 piazza forum all announcements & discussions (link on class website) Stratos Idreos 42/50
75 slides are not notes! slides are mainly there to trigger discussion note keeping is your task: starting class 3 we will do collaborative note taking: Stratos Idreos 43/50
76 and check syllabus/website! Stratos Idreos 44/50
77 stratos away next week NO CLASS ON SEP 7 TFs will hold a lab NEXT CLASS ON SEP 12 4pm, Pierce 301 NO OH NEXT WEEK every day labs + OH with Stratos on demand via Skype or Zoom Stratos Idreos 45/50
78 reading textbook: chapters 1, 3 (-3.5), 5 (-5.8,-5.9) intro + relational model + SQL browse the Fourth Paradigm next couple of classes data models low level details - system design data layouts and column-stores Stratos Idreos 46/50
79 readings for next 3 classes Architecture of a Database System (Sections 1,2,3,4) by J. Hellerstein, M. Stonebraker and J. Hamilton The Design and Implementation of Modern Column-store Database Systems by D. Abadi, P. Boncz, S. Harizopoulos, S. Idreos, S. Madden Stratos Idreos 47/50
80 games thousands of users - tons of data frequent updates - tons of queries analytics in the background for game intelligence W. White, A. Demers, C. Koch, J. Gehrke and R. Rajagopalan Scaling games to epic proportions ACM SIGMOD International Conference on Management of Data, 2007 Stratos Idreos 48/50
81 Notes to remember traditional algorithm analysis does not capture all costs both data movement and processing matter mundane steps such as incrementing a loop counter or materializing results can have a huge effect there are no simple tasks all systems are the same : just data and access patterns Stratos Idreos 49/50
82 class 2 from bits to dbs DATA SYSTEMS prof. Stratos Idreos
data systems 101 prof. Stratos Idreos class 2
class 2 data systems 101 prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS265/ 2 classes per week - OH/Labs every day 1 presentation/discussion lead - 2 reviews each week research (or systems)
data systems 101 prof. Stratos Idreos class 2
class 2 data systems 101 prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS265/ big data V s (it is not about size only) volume velocity variety veracity actually none of that is really new
HOW INDEX TO STORE DATA DATA
Stratos Idreos HOW INDEX DATA TO STORE DATA ALGORITHMS data structure decisions define the algorithms that access data INDEX DATA ALGORITHMS unordered [7,4,2,6,1,3,9,10,5,8] INDEX DATA ALGORITHMS unordered
column-stores basics
class 3 column-stores basics prof. HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS265/ Goetz Graefe Google Research guest lecture Justin Levandoski Microsoft Research projects option 1: systems project (now
Models & Intro to DB Architectures
class 3 Models & Intro to DB Architectures prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ welcome brave cs165 students! 42+44 Stratos Idreos 2 /49 NO LAPTOP/PHONE POLICY class is based
Models & Intro to DB Architectures
class 3 Models & Intro to DB Architectures prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ welcome brave cs165 students! Stratos Idreos 2 /55 NO LAPTOP/PHONE POLICY class is based on
basic db architectures & layouts
class 4 basic db architectures & layouts prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ videos for sections 3 & 4 are online check back every week (1-2 sections weekly) there is a schedule
SQL & intro to db architectures
class 3 SQL & intro to db architectures prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ welcome brave cs165 students! 35+62 Stratos Idreos 2 /55 guest lecture Laura Haas Data Systems
column-stores basics
class 3 column-stores basics prof. HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS265/ project description is now online First background info will be given this Friday and detailed lecture on Feb 21 Basic Readings
class 5 column stores 2.0 prof. Stratos Idreos
class 5 column stores 2.0 prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ worth thinking about what just happened? where is my data? email, cloud, social media, can we design systems
class 6 more about column-store plans and compression prof. Stratos Idreos
class 6 more about column-store plans and compression prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ query compilation an ancient yet new topic/research challenge query->sql->interpet
class 9 fast scans 1.0 prof. Stratos Idreos
class 9 fast scans 1.0 prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ 1 pass to merge into 8 sorted pages (2N pages) 1 pass to merge into 4 sorted pages (2N pages) 1 pass to merge into
class 10 b-trees 2.0 prof. Stratos Idreos
class 10 b-trees 2.0 prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ CS Colloquium HV Jagadish Prof University of Michigan 10/6 Stratos Idreos /29 2 CS Colloquium Magdalena Balazinska
class 17 updates prof. Stratos Idreos
class 17 updates prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ early/late tuple reconstruction, tuple-at-a-time, vectorized or bulk processing, intermediates format, pushing selects
class 11 b-trees prof. Stratos Idreos
class 11 b-trees prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ Midway check-in: Two design docs tmr (Canvas) & tests on Sunday Next weekend: Lab marathon for midway check-in & tests
class 17 updates prof. Stratos Idreos
class 17 updates prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ UPDATE table_name SET column1=value1,column2=value2,... WHERE some_column=some_value INSERT INTO table_name VALUES (value1,value2,value3,...)
CMPT 354: Database System I. Lecture 1. Course Introduction
CMPT 354: Database System I Lecture 1. Course Introduction 1 Outline Motivation for studying this course Course admin and set up Overview of course topics 2 Trend 1: Data grows exponentially 1 ZB = 1,
class 20 updates 2.0 prof. Stratos Idreos
class 20 updates 2.0 prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ UPDATE table_name SET column1=value1,column2=value2,... WHERE some_column=some_value INSERT INTO table_name VALUES
EECS3421 Introduction to Database Management Systems. Thanks to John Mylopoulos and Ryan Johnson for material in these slides
EECS3421 Introduction to Database Management Systems Thanks to John Mylopoulos and Ryan Johnson for material in these slides Overview What is a database? Course administrivia The relational model 2 What
class 13 scans vs indexes prof. Stratos Idreos
class 13 scans vs indexes prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ b-tree - dynamic tree - always balanced 35,50 35, 12,20 50, 1,2,3 12,15,17 20, Stratos Idreos 2 /24 select from
systems & research project
class 4 systems & research project prof. HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS265/ index index knows order about the data data filtering data: point/range queries index data A B C sorted A B C initial
Overview of Data Exploration Techniques. Stratos Idreos, Olga Papaemmanouil, Surajit Chaudhuri
Overview of Data Exploration Techniques Stratos Idreos, Olga Papaemmanouil, Surajit Chaudhuri data exploration not always sure what we are looking for (until we find it) data has always been big volume
Evolution of Database Systems
Evolution of Database Systems Krzysztof Dembczyński Intelligent Decision Support Systems Laboratory (IDSS) Poznań University of Technology, Poland Intelligent Decision Support Systems Master studies, second
CS564: Database Management Systems. Lecture 1: Course Overview. Acks: Chris Ré 1
CS564: Database Management Systems Lecture 1: Course Overview Acks: Chris Ré 1 2 Big science is data driven. 3 Increasingly many companies see themselves as data driven. 4 Even more traditional companies
CSE 544 Principles of Database Management Systems. Alvin Cheung Fall 2015 Lecture 8 - Data Warehousing and Column Stores
CSE 544 Principles of Database Management Systems Alvin Cheung Fall 2015 Lecture 8 - Data Warehousing and Column Stores Announcements Shumo office hours change See website for details HW2 due next Thurs
CTL.SC4x Technology and Systems
in Supply Chain Management CTL.SC4x Technology and Systems Key Concepts Document This document contains the Key Concepts for the SC4x course, Weeks 1 and 2. These are meant to complement, not replace,
Course Introduction. CSC343 - Introduction to Databases Manos Papagelis
Course Introduction CSC343 - Introduction to Databases Manos Papagelis Thanks to Ryan Johnson, John Mylopoulos, Arnold Rosenbloom and Renee Miller for material in these slides Overview 2 What is a database?
CS639: Data Management for Data Science. Lecture 1: Intro to Data Science and Course Overview. Theodoros Rekatsinas
CS639: Data Management for Data Science Lecture 1: Intro to Data Science and Course Overview Theodoros Rekatsinas 1 2 Big science is data driven. 3 Increasingly many companies see themselves as data driven.
CS370 Operating Systems
CS370 Operating Systems Colorado State University Yashwant K Malaiya Spring 2018 Lecture 2 Slides based on Text by Silberschatz, Galvin, Gagne Various sources 1 1 2 What is an Operating System? What is
CAS CS 460/660 Introduction to Database Systems. Fall
CAS CS 460/660 Introduction to Database Systems Fall 2017 1.1 About the course Administrivia Instructor: George Kollios, gkollios@cs.bu.edu MCS 283, Mon 2:30-4:00 PM and Tue 1:00-2:30 PM Teaching Fellows:
Introduction to Database Systems CS432. CS432/433: Introduction to Database Systems. CS432/433: Introduction to Database Systems
Introduction to Database Systems CS432 Instructor: Christoph Koch koch@cs.cornell.edu CS 432 Fall 2007 1 CS432/433: Introduction to Database Systems Underlying theme: How do I build a data management system?
CSE544 Database Architecture
CSE544 Database Architecture Tuesday, February 1 st, 2011 Slides courtesy of Magda Balazinska 1 Where We Are What we have already seen Overview of the relational model Motivation and where model came from
CSC 261/461 Database Systems Lecture 20. Spring 2017 MW 3:25 pm 4:40 pm January 18 May 3 Dewey 1101
CSC 261/461 Database Systems Lecture 20 Spring 2017 MW 3:25 pm 4:40 pm January 18 May 3 Dewey 1101 Announcements Project 1 Milestone 3: Due tonight Project 2 Part 2 (Optional): Due on: 04/08 Project 3
CS 245: Principles of Data-Intensive Systems. Instructor: Matei Zaharia cs245.stanford.edu
CS 245: Principles of Data-Intensive Systems Instructor: Matei Zaharia cs245.stanford.edu Outline Why study data-intensive systems? Course logistics Key issues and themes A bit of history CS 245 2 My Background
CS145: Intro to Databases. Lecture 1: Course Overview
CS145: Intro to Databases Lecture 1: Course Overview 1 The world is increasingly driven by data This class teaches the basics of how to use & manage data. 2 Key Questions We Will Answer How can we collect
CS 564: DATABASE MANAGEMENT SYSTEMS. Spring 2018
CS 564: DATABASE MANAGEMENT SYSTEMS Spring 2018 DATA IS EVERYWHERE! Our world is increasingly data driven scientific discoveries online services (social networks, online retailers) decision making Databases
complex plans and hybrid layouts
class 7 complex plans and hybrid layouts prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ essential column-stores features virtual ids late tuple reconstruction (if ever) vectorized execution
CMPSCI 645 Database Design & Implementation
Welcome to CMPSCI 645 Database Design & Implementation Instructor: Gerome Miklau Overview of Databases Gerome Miklau CMPSCI 645 Database Design & Implementation UMass Amherst Jan 19, 2010 Some slide content
Fundamentals of Database Systems
Fundamentals of Database Systems Semester 1, 2017 Fundamentals of Database Systems COMPSCI/SOFTENG 351 COMPSCI 751 Instructors: Gill Dobbie, Miika Hannula, Sebastian Link, Gerald Weber Department of Computer
CMPT 354: Database System I. Lecture 11. Transaction Management
CMPT 354: Database System I Lecture 11. Transaction Management 1 Why this lecture DB application developer What if crash occurs, power goes out, etc? Single user à Multiple users 2 Outline Transaction
Course Overview, Python Basics
CS 1110: Introduction to Computing Using Python Lecture 1 Course Overview, Python Basics [Andersen, Gries, Lee, Marschner, Van Loan, White] Interlude: Why learn to program? (which is subtly distinct from,
CS 245: Database System Principles
CS 245: Database System Principles Notes 01: Introduction Peter Bailis CS 245 Notes 1 1 This course pioneered by Hector Garcia-Molina All credit due to Hector All mistakes due to Peter CS 245 Notes 1 2
61A LECTURE 1 FUNCTIONS, VALUES. Steven Tang and Eric Tzeng June 24, 2013
61A LECTURE 1 FUNCTIONS, VALUES Steven Tang and Eric Tzeng June 24, 2013 Welcome to CS61A! The Course Staff - Lecturers Steven Tang Graduated L&S CS from Cal Back for a PhD in Education Eric Tzeng Graduated
Introduction to Data Management. Lecture #1 (The Course Trailer )
Introduction to Data Management Lecture #1 (The Course Trailer ) Instructor: Mike Carey mjcarey@ics.uci.edu Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Today s Topics v Welcome to
Introduction to Data Management. Lecture #2 (Big Picture, Cont.)
Introduction to Data Management Lecture #2 (Big Picture, Cont.) Instructor: Mike Carey mjcarey@ics.uci.edu Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Announcements v Still hanging
Introduction to Data Management. Lecture #1 (Course Trailer )
Introduction to Data Management Lecture #1 (Course Trailer ) Instructor: Mike Carey mjcarey@ics.uci.edu Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Today s Topics v Welcome to one
Massive Scalability With InterSystems IRIS Data Platform
Massive Scalability With InterSystems IRIS Data Platform Introduction Faced with the enormous and ever-growing amounts of data being generated in the world today, software architects need to pay special
Lecture 1 Introduction (Chapter 1 of Textbook)
Bilkent University Department of Computer Engineering CS342 Operating Systems Lecture 1 Introduction (Chapter 1 of Textbook) Dr. İbrahim Körpeoğlu http://www.cs.bilkent.edu.tr/~korpe 1 References The slides
Course Introduction & History of Database Systems
Course Introduction & History of Database Systems CMPT 843, SPRING 2018 JIANNAN WANG https://sfu-db.github.io/dbsystems/ CMPT 843-2018 SPRING - SFU 1 Introduce Yourself What s your name? Where are you
EECS 647: Introduction to Database Systems
EECS 647: Introduction to Database Systems Instructor: Luke Huan Spring 2009 Queries for Today What is a database? What is a database management system? Why take a database course? Who will teach? How
TA hours and labs start today. First lab is out and due next Wednesday, 1/31. Getting started lab is also out
Announcements TA hours and labs start today. First lab is out and due next Wednesday, 1/31. Getting started lab is also out Get you setup for project/lab work. We ll check it with the first lab. Stars
CompSci 516 Database Systems
CompSci 516 Database Systems Lecture 20 NoSQL and Column Store Instructor: Sudeepa Roy Duke CS, Fall 2018 CompSci 516: Database Systems 1 Reading Material NOSQL: Scalable SQL and NoSQL Data Stores Rick
CSE 544 Principles of Database Management Systems
CSE 544 Principles of Database Management Systems Lecture 1 - Introduction and the Relational Model 1 Outline Introduction Class overview Why database management systems (DBMS)? The relational model 2
Lecture 12. Lecture 12: The IO Model & External Sorting
Lecture 12 Lecture 12: The IO Model & External Sorting Announcements Announcements 1. Thank you for the great feedback (post coming soon)! 2. Educational goals: 1. Tech changes, principles change more
Databasesystemer, forår 2005 IT Universitetet i København. Forelæsning 8: Database effektivitet. 31. marts Forelæser: Rasmus Pagh
Databasesystemer, forår 2005 IT Universitetet i København Forelæsning 8: Database effektivitet. 31. marts 2005 Forelæser: Rasmus Pagh Today s lecture Database efficiency Indexing Schema tuning 1 Database
Traditional RDBMS Wisdom is All Wrong -- In Three Acts "
Traditional RDBMS Wisdom is All Wrong -- In Three Acts "! The Stonebraker Says Webinar Series! The first three acts:! 1. Why the elephants are toast and why main memory is the answer for OLTP! Today! 2.
Overview of the ECE Computer Software Curriculum. David O Hallaron Associate Professor of ECE and CS Carnegie Mellon University
Overview of the ECE Computer Software Curriculum David O Hallaron Associate Professor of ECE and CS Carnegie Mellon University The Fundamental Idea of Abstraction Human beings Applications Software systems
CISC 7610 Lecture 2b The beginnings of NoSQL
CISC 7610 Lecture 2b The beginnings of NoSQL Topics: Big Data Google s infrastructure Hadoop: open google infrastructure Scaling through sharding CAP theorem Amazon s Dynamo 5 V s of big data Everyone
Introduction CPS343. Spring Parallel and High Performance Computing. CPS343 (Parallel and HPC) Introduction Spring / 29
Introduction CPS343 Parallel and High Performance Computing Spring 2018 CPS343 (Parallel and HPC) Introduction Spring 2018 1 / 29 Outline 1 Preface Course Details Course Requirements 2 Background Definitions
745: Advanced Database Systems
745: Advanced Database Systems Yanlei Diao University of Massachusetts Amherst Outline Overview of course topics Course requirements Database Management Systems 1. Online Analytical Processing (OLAP) vs.
Introduction to Data Management. Lecture #1 (Course Trailer )
Introduction to Data Management Lecture #1 (Course Trailer ) Instructor: Mike Carey mjcarey@ics.uci.edu Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Today s Topics! Welcome to my biggest
CS 378 (Spring 2003) Linux Kernel Programming. Yongguang Zhang. Copyright 2003, Yongguang Zhang
Department of Computer Sciences THE UNIVERSITY OF TEXAS AT AUSTIN CS 378 (Spring 2003) Linux Kernel Programming Yongguang Zhang (ygz@cs.utexas.edu) Copyright 2003, Yongguang Zhang Read Me First Everything
CS370 Operating Systems
CS370 Operating Systems Colorado State University Yashwant K Malaiya Fall 2016 Lecture 2 Slides based on Text by Silberschatz, Galvin, Gagne Various sources 1 1 2 System I/O System I/O (Chap 13) Central
Lecture 1. Course Overview, Python Basics
Lecture 1 Course Overview, Python Basics We Are Very Full! Lectures and Labs are at fire-code capacity We cannot add sections or seats to lectures You may have to wait until someone drops No auditors are
BBM371- Data Management. Lecture 1: Course policies, Introduction to DBMS
BBM371- Data Management Lecture 1: Course policies, Introduction to DBMS 26.09.2017 Today Introduction About the class Organization of this course Introduction to Database Management Systems (DBMS) About
Introduction to Database Systems CSE 444. Lecture 1 Introduction
Introduction to Database Systems CSE 444 Lecture 1 Introduction 1 About Me: General Prof. Magdalena Balazinska (magda) At UW since January 2006 PhD from MIT Born in Poland Grew-up in Poland, Algeria, and
Introduction to Data Management. Lecture #2 (Big Picture, Cont.) Instructor: Chen Li
Introduction to Data Management Lecture #2 (Big Picture, Cont.) Instructor: Chen Li 1 Announcements v We added 10 more seats to the class for students on the waiting list v Deadline to drop the class:
Modern Database Concepts
Modern Database Concepts Introduction to the world of Big Data Doc. RNDr. Irena Holubova, Ph.D. holubova@ksi.mff.cuni.cz What is Big Data? buzzword? bubble? gold rush? revolution? Big data is like teenage
CSC369 Lecture 9. Larry Zhang, November 16, 2015
CSC369 Lecture 9 Larry Zhang, November 16, 2015 1 Announcements A3 out, due ecember 4th Promise: there will be no extension since it is too close to the final exam (ec 7) Be prepared to take the challenge
CSE 333 Lecture 1 - Systems programming
CSE 333 Lecture 1 - Systems programming Hal Perkins Department of Computer Science & Engineering University of Washington Welcome! Today s goals: - introductions - big picture - course syllabus - setting
Page 1. Goals for Today" TLB organization" CS162 Operating Systems and Systems Programming Lecture 11. Page Allocation and Replacement"
Goals for Today" CS162 Operating Systems and Systems Programming Lecture 11 Page Allocation and Replacement" Finish discussion on TLBs! Page Replacement Policies! FIFO, LRU! Clock Algorithm!! Working Set/Thrashing!
What We Have Already Learned. DBMS Deployment: Local. Where We Are Headed Next. DBMS Deployment: 3 Tiers. DBMS Deployment: Client/Server
What We Have Already Learned CSE 444: Database Internals Lectures 19-20 Parallel DBMSs Overall architecture of a DBMS Internals of query execution: Data storage and indexing Buffer management Query evaluation
Performance Profiling
Performance Profiling Minsoo Ryu Real-Time Computing and Communications Lab. Hanyang University msryu@hanyang.ac.kr Outline History Understanding Profiling Understanding Performance Understanding Performance
CMPT 354 Database Systems I. Spring 2012 Instructor: Hassan Khosravi
CMPT 354 Database Systems I Spring 2012 Instructor: Hassan Khosravi Textbook First Course in Database Systems, 3 rd Edition. Jeffry Ullman and Jennifer Widom Other text books Ramakrishnan SILBERSCHATZ
Outline. Database Management Systems (DBMS) Database Management and Organization. IT420: Database Management and Organization
Outline IT420: Database Management and Organization Dr. Crăiniceanu Capt. Balazs www.cs.usna.edu/~adina/teaching/it420/spring2007 Class Survey Why Databases (DB)? A Problem DB Benefits In This Class? Admin
CS 241 Data Organization using C
CS 241 Data Organization using C Fall 2018 Instructor Name: Dr. Marie Vasek Contact: Private message me on the course Piazza page. Office: Farris 2120 Office Hours: Tuesday 2-4pm and Thursday 9:30-11am
CS 445 Introduction to Database Systems
CS 445 Introduction to Database Systems TTh 2:45-4:20pm Chadd Williams Pacific University 1 Overview Practical introduction to databases theory + hands on projects Topics Relational Model Relational Algebra/Calculus/
CS162 Operating Systems and Systems Programming Lecture 10 Caches and TLBs"
CS162 Operating Systems and Systems Programming Lecture 10 Caches and TLBs" October 1, 2012! Prashanth Mohan!! Slides from Anthony Joseph and Ion Stoica! http://inst.eecs.berkeley.edu/~cs162! Caching!
CS 471 Operating Systems. Yue Cheng. George Mason University Fall 2017
CS 471 Operating Systems Yue Cheng George Mason University Fall 2017 Introduction o Instructor of Section 002 Dr. Yue Cheng (web: cs.gmu.edu/~yuecheng) Email: yuecheng@gmu.edu Office: 5324 Engineering
Two Success Stories - Optimised Real-Time Reporting with BI Apps
Oracle Business Intelligence 11g Two Success Stories - Optimised Real-Time Reporting with BI Apps Antony Heljula October 2013 Peak Indicators Limited 2 Two Success Stories - Optimised Real-Time Reporting
Introduction to Distributed HTC and overlay systems
Introduction to Distributed HTC and overlay systems Tuesday morning session Igor Sfiligoi University of California San Diego Logistical reminder It is OK to ask questions - During
CS 405G: Introduction to Database Systems. Lecture 1: Introduction
CS 405G: Introduction to Database Systems Lecture 1: Introduction Topics Topics for Today Introduction What is a database? What is a database management system? Why take a database course? How to take
class 8 b-trees prof. Stratos Idreos
class 8 b-trees prof. Stratos Idreos HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/ I spend a lot of time debugging am I doing something wrong? maybe but probably not 1. learn to use gdb 2. after spending
COS 318: Operating Systems. Introduction
COS 318: Operating Systems Introduction Kai Li Computer Science Department Princeton University (http://www.cs.princeton.edu/courses/cs318/) Today Course staff and logistics What is operating system? Evolution
Internet Scale Storage
Internet Scale Storage SIGMOD 2011 James Hamilton, 2011/6/14 VP & Distinguished Engineer, Amazon Web Services email: James@amazon.com web: mvdirona.com/jrh/work blog: perspectives.mvdirona.com Agenda Cloud
Crescando: Predictable Performance for Unpredictable Workloads
Crescando: Predictable Performance for Unpredictable Workloads G. Alonso, D. Fauser, G. Giannikis, D. Kossmann, J. Meyer, P. Unterbrunner Amadeus S.A. ETH Zurich, Systems Group (Funded by Enterprise Computing
Persistence & State. SWE 432, Fall 2016 Design and Implementation of Software for the Web
Persistence & State SWE 432, Fall 2016 Design and Implementation of Software for the Web Today What s state for our web apps? How do we store it, where do we store it, and why there? For further reading:
Engineering Robust Server Software
Engineering Robust Server Software Scalability Other Scalability Issues Database Load Testing 2 Databases Most server applications use databases Very complex pieces of software Designed for scalability
CSC 443: Web Programming
1 CSC 443: Web Programming Haidar Harmanani Department of Computer Science and Mathematics Lebanese American University Byblos, 1401 2010 Lebanon Today 2 Course information Course Objectives A Tiny assignment
Introduction to K2View Fabric
Introduction to K2View Fabric 1 Introduction to K2View Fabric Overview In every industry, the amount of data being created and consumed on a daily basis is growing exponentially. Enterprises are struggling
Goals for Today. CS 133: Databases. Final Exam: Logistics. Why Use a DBMS? Brief overview of course. Course evaluations
Goals for Today Brief overview of course CS 133: Databases Course evaluations Fall 2018 Lec 27 12/13 Course and Final Review Prof. Beth Trushkowsky More details about the Final Exam Practice exercises
EECS 482 Introduction to Operating Systems
EECS 482 Introduction to Operating Systems Winter 2018 Baris Kasikci barisk@umich.edu (Thanks, Harsha Madhyastha for the slides!) 1 About Me Prof. Kasikci (Prof. K.), Prof. Baris (Prof. Barish) Assistant
CSCI455: Introduction to Programming System Design
CSCI455: Introduction to Programming System Design Claire Bono bono@usc.edu Spring 2019 http://bytes.usc.edu/cs455/ 455 Intro [Bono] 1 Today s topics Course overview and logistics Academic integrity Java
Why are you here? Introduction. Course roadmap. Course goals. What do you want from a DBMS? What is a database system? Aren t databases just
Why are you here? 2 Introduction CPS 216 Advanced Database Systems Aren t databases just Trivial exercises in first-order logic (says AI)? Bunch of out-of-fashion I/O-efficient indexes and algorithms (says
Traditional RDBMS Wisdom is All Wrong -- In Three Acts. Michael Stonebraker
Traditional RDBMS Wisdom is All Wrong -- In Three Acts Michael Stonebraker The Stonebraker Says Webinar Series The first three acts: 1. Why main memory is the answer for OLTP Recording available at VoltDB.com
Data! CS 133: Databases. Goals for Today. So, what is a database? What is a database anyway? From the textbook:
CS 133: Databases Fall 2018 Lec 01 09/04 Introduction & Relational Model Data! Need systems to Data is everywhere Banking, airline reservations manage the data Social media, clicking anything on the internet
Jens Bollmann. Welcome! Performance 101 for Small Web Apps. Principal consultant and trainer within the Professional Services group at SkySQL Ab.
Welcome! Jens Bollmann jens@skysql.com Principal consultant and trainer within the Professional Services group at SkySQL Ab. Who is SkySQL Ab? SkySQL Ab is the alternative source for software, services
Introduction to Data Management. Lecture #1 (Course Trailer ) Instructor: Chen Li
Introduction to Data Management Lecture #1 (Course Trailer ) Instructor: Chen Li 1 Today s Topics v Welcome to one of my biggest classes ever! v Read (and live by) the course wiki page: http://www.ics.uci.edu/~cs122a/
CSC Operating Systems Spring Lecture - II OS Structures. Tevfik Ko!ar. Louisiana State University. January 17 th, 2007.
CSC 4103 - Operating Systems Spring 2008 Lecture - II OS Structures Tevfik Ko!ar Louisiana State University January 17 th, 2007 1 Announcements Teaching Assistant: Asim Shrestrah Email: ashres1@lsu.edu
Announcements. Operating System Structure. Roadmap. Operating System Structure. Multitasking Example. Tevfik Ko!ar
CSC 4103 - Operating Systems Spring 2008 Lecture - II OS Structures Tevfik Ko!ar Teaching Assistant: Asim Shrestrah Email: ashres1@lsu.edu Announcements All of you should be now in the class mailing list.