# Lecture 15. Lecture 15: Bitmap Indexes

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Lecture 5 Lecture 5: Bitmap Indexes

2 Lecture 5 What you will learn about in this section. Bitmap Indexes 2. Storing a bitmap index 3. Bitslice Indexes 2

3 Lecture 5. Bitmap indexes 3

4 Motivation Consider the following table: CREATE TABLE Tweets ( uniquemsgid INTEGER, -- unique message id tstamp TIMESTAMP, -- when was the tweet posted uid INTEGER, -- unique id of the user msg VARCHAR (4), -- the actual message zip INTEGER, -- zipcode when posted retweet BOOLEAN -- retweeted? ); Consider the following query, Q: And, the following query, Q2: SELECT * FROM Tweets WHERE uid = 45; SELECT * FROM Tweets WHERE zip BETWEEN 53 AND Speed-up queries using a B+-tree for the uid and the zip values.

5 Motivation Consider the following table: CREATE TABLE Tweets ( uniquemsgid INTEGER, -- unique message id tstamp TIMESTAMP, -- when was the tweet posted uid INTEGER, -- unique id of the user msg VARCHAR (4), -- the actual message zip INTEGER, -- zipcode when posted retweet BOOLEAN -- retweeted? ); In a B+-tree, how many bytes do we use for each record? At least key + rid, so key-size+rid-size Can we do better, i.e. an index with lower storage overhead? Especially for attributes with small domain cardinalities? Bit-based indices: Two flavors a) Bitmap indices and b) Bitslice indices

6 Bitmap Indices Consider building an index to answer equality queries on the retweet attribute Issues with building a B-tree: Three distinct values: True, False, NULL Lots of duplicates for each distinct value Sort of an odd B-tree with three long rid lists Bitmap Index: Build three bitmap arrays (stored on disk), one for each value. The i th bit in each bitmap correspond to the i th tuple (need to map i th position to a rid)

7 Bitmap Example Table (stored in a heapfile) uniquemsgid zip retweet 324 Y Y N NULL 5 92 N,,, 5375 Y Bitmap index on retweet R-Yes R-No R-Null SELECT * FROM Tweets WHERE retweet = N. Scan the R-No Bitmap file 2. For each bit set to, compute the tuple # 3. Fetch the tuple # (s)

8 Critical Issue Need an efficient way to compute a bit position Layout the bitmap in page id order. Need an efficient way to map a bit position to a record id. How?. If you fix the # records per page in the heapfile 2. And lay the pages out so that page #s are sequential and increasing 3. Then can construct rid (page-id, slot#) page-id = Bit-position / #records-per-page slot# = Bit-position % #records-per-page Implications of #? With variable length records, have to set the limit based on the size of the largest record, which may result in under-filled pages.

9 Other Queries Table (stored in a heapfile) uniquemsgid zip retweet Bitmap index on retweet R-Yes R-No R-Null 324 Y Y N NULL 5 92 N,,, 5375 Y SELECT COUNT(*) FROM Tweets WHERE retweet = N SELECT * FROM Tweets WHERE retweet IS NOT NULL

10 Lecture 5 2. Storing a bitmap index

11 Storing the Bitmap index One bitmap for each value, and one for Nulls Need to store each bitmap Simple method: file for each bitmap Can compress the bitmap! Index size? #tuples * (cardinality of the domain + ) bits When is a bitmap index more space efficient than a B+-tree? #distinct values < data entry size in the B+-tree

12 Lecture 5 3. Bit-sliced Index 2

13 Bit-sliced Index: Motivation (Re)consider the following table: CREATE TABLE Tweets ( uniquemsgid INTEGER, -- unique message id tstamp TIMESTAMP, -- when was the tweet posted uid INTEGER, -- unique id of the user msg VARCHAR (4), -- the actual message zip INTEGER, -- zipcode when posted retweet BOOLEAN -- retweeted? ); SELECT * FROM Tweets WHERE zip = 5376 Would we build a bitmap index on zipcode?

14 Bit-sliced index Table uniquemsgid zip retweet 324 Y Y N NULL 5 92 N,,, 5375 Y Why do we have 7 bits for zipcode? Slice 6 Bit-sliced index Slice Slice ( slice per bit) Query evaluation: Walk through each slice constructing a result bitmap e.g. zip 324, skip entries that have in the first three slices (6, 5, 4) (Null bitmap is not shown)

15 Bitslice Indices Can also do aggregates with Bitslice indices E.g. SUM(attr): Add bit-slice by bit-slice. First, count the number of s in the slice7, and multiply the count by 2 7 Then, count the number of s in the slice6, and multiply the count by Store each slice using methods like what you have for a bitmap. Note once again can use compression

16 Bitmap v/s Bitslice Bitmaps better for low cardinality domains Bitslice better for high cardinality domains Generally easier to do the math with bitmap indices

### Spring 2017 EXTERNAL SORTING (CH. 13 IN THE COW BOOK) 2/7/17 CS 564: Database Management Systems; (c) Jignesh M. Patel,

Spring 2017 EXTERNAL SORTING (CH. 13 IN THE COW BOOK) 2/7/17 CS 564: Database Management Systems; (c) Jignesh M. Patel, 2013 1 Motivation for External Sort Often have a large (size greater than the available

### Spring 2017 B-TREES (LOOSELY BASED ON THE COW BOOK: CH. 10) 1/29/17 CS 564: Database Management Systems, Jignesh M. Patel 1

Spring 2017 B-TREES (LOOSELY BASED ON THE COW BOOK: CH. 10) 1/29/17 CS 564: Database Management Systems, Jignesh M. Patel 1 Consider the following table: Motivation CREATE TABLE Tweets ( uniquemsgid INTEGER,

### STORING DATA: DISK AND FILES

STORING DATA: DISK AND FILES CS 564- Fall 2016 ACKs: Dan Suciu, Jignesh Patel, AnHai Doan MANAGING DISK SPACE The disk space is organized into files Files are made up of pages s contain records 2 FILE

### Lecture Data layout on disk. How to store relations (tables) in disk blocks. By Marina Barsky Winter 2016, University of Toronto

Lecture 01.04 Data layout on disk How to store relations (tables) in disk blocks By Marina Barsky Winter 2016, University of Toronto How do we map tables to disk blocks Relation, or table: schema + collection

### Disks & Files. Yanlei Diao UMass Amherst. Slides Courtesy of R. Ramakrishnan and J. Gehrke

Disks & Files Yanlei Diao UMass Amherst Slides Courtesy of R. Ramakrishnan and J. Gehrke DBMS Architecture Query Parser Query Rewriter Query Optimizer Query Executor Lock Manager for Concurrency Access

### Indexing. Jan Chomicki University at Buffalo. Jan Chomicki () Indexing 1 / 25

Indexing Jan Chomicki University at Buffalo Jan Chomicki () Indexing 1 / 25 Storage hierarchy Cache Main memory Disk Tape Very fast Fast Slower Slow (nanosec) (10 nanosec) (millisec) (sec) Very small Small

### Database Applications (15-415)

Database Applications (15-415) DBMS Internals: Part II Lecture 10, February 17, 2014 Mohammad Hammoud Last Session: DBMS Internals- Part I Today Today s Session: DBMS Internals- Part II Brief summaries

### Evaluation of Relational Operations: Other Techniques. Chapter 14 Sayyed Nezhadi

Evaluation of Relational Operations: Other Techniques Chapter 14 Sayyed Nezhadi Schema for Examples Sailors (sid: integer, sname: string, rating: integer, age: real) Reserves (sid: integer, bid: integer,

### Midterm 1: CS186, Spring I. Storage: Disk, Files, Buffers [11 points] SOLUTION. cs186-

Midterm 1: CS186, Spring 2016 Name: Class Login: SOLUTION cs186- You should receive 1 double-sided answer sheet and an 10-page exam. Mark your name and login on both sides of the answer sheet, and in the

### Understanding the Optimizer

Understanding the Optimizer 1 Global topics Introduction At which point does the optimizer his work Optimizer steps Index Questions? 2 Introduction Arno Brinkman BISIT engineering b.v. ABVisie firebird@abvisie.nl

### Data Storage and Query Answering. Data Storage and Disk Structure (4)

Data Storage and Query Answering Data Storage and Disk Structure (4) Introduction We have introduced secondary storage devices, in particular disks. Disks use blocks as basic units of transfer and storage.

### Storing Data: Disks and Files

Storing Data: Disks and Files Module 2, Lecture 1 Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet Database Management Systems, R. Ramakrishnan 1 Disks and

### Storing Data: Disks and Files

Storing Data: Disks and Files Chapter 9 CSE 4411: Database Management Systems 1 Disks and Files DBMS stores information on ( 'hard ') disks. This has major implications for DBMS design! READ: transfer

### Implementing Relational Operators: Selection, Projection, Join. Database Management Systems, R. Ramakrishnan and J. Gehrke 1

Implementing Relational Operators: Selection, Projection, Join Database Management Systems, R. Ramakrishnan and J. Gehrke 1 Readings [RG] Sec. 14.1-14.4 Database Management Systems, R. Ramakrishnan and

### CS220 Database Systems. File Organization

CS220 Database Systems File Organization Slides from G. Kollios Boston University and UC Berkeley 1.1 Context Database app Query Optimization and Execution Relational Operators Access Methods Buffer Management

### Database Applications (15-415)

Database Applications (15-415) DBMS Internals- Part VI Lecture 14, March 12, 2014 Mohammad Hammoud Today Last Session: DBMS Internals- Part V Hash-based indexes (Cont d) and External Sorting Today s Session:

### Database Applications (15-415)

Database Applications (15-415) DBMS Internals: Part II Lecture 11, February 17, 2015 Mohammad Hammoud Last Session: DBMS Internals- Part I Today Today s Session: DBMS Internals- Part II A Brief Summary

### Storing Data: Disks and Files. Storing and Retrieving Data. Why Not Store Everything in Main Memory? Chapter 7

Storing : Disks and Files Chapter 7 base Management Systems, R. Ramakrishnan and J. Gehrke 1 Storing and Retrieving base Management Systems need to: Store large volumes of data Store data reliably (so

### Data Structures and Algorithms Sorting

Data Structures and Algorithms Sorting Chris Brooks Department of Computer Science University of San Francisco Department of Computer Science University of San Francisco p.1/23 12-0: Sorting Sorting is

### Unit 3 Disk Scheduling, Records, Files, Metadata

Unit 3 Disk Scheduling, Records, Files, Metadata Based on Ramakrishnan & Gehrke (text) : Sections 9.3-9.3.2 & 9.5-9.7.2 (pages 316-318 and 324-333); Sections 8.2-8.2.2 (pages 274-278); Section 12.1 (pages

### Storing and Retrieving Data. Storing Data: Disks and Files. Solution 1: Techniques for making disks faster. Disks. Why Not Store Everything in Tapes?

Storing and Retrieving Storing : Disks and Files Chapter 9 base Management Systems need to: Store large volumes of data Store data reliably (so that data is not lost!) Retrieve data efficiently Alternatives

### DATABASE PERFORMANCE AND INDEXES. CS121: Relational Databases Fall 2017 Lecture 11

DATABASE PERFORMANCE AND INDEXES CS121: Relational Databases Fall 2017 Lecture 11 Database Performance 2 Many situations where query performance needs to be improved e.g. as data size grows, query performance

### External Sorting. Chapter 13. Comp 521 Files and Databases Fall

External Sorting Chapter 13 Comp 521 Files and Databases Fall 2012 1 Why Sort? A classic problem in computer science! Advantages of requesting data in sorted order gathers duplicates allows for efficient

### Midterm 1: CS186, Spring I. Storage: Disk, Files, Buffers [11 points] cs186-

Midterm 1: CS186, Spring 2016 Name: Class Login: cs186- You should receive 1 double-sided answer sheet and an 11-page exam. Mark your name and login on both sides of the answer sheet, and in the blanks

### Evaluation of Relational Operations

Evaluation of Relational Operations Chapter 14 Comp 521 Files and Databases Fall 2010 1 Relational Operations We will consider in more detail how to implement: Selection ( ) Selects a subset of rows from

### Storing Data: Disks and Files

Storing Data: Disks and Files Chapter 9 Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke Disks

### Storage hierarchy. Textbook: chapters 11, 12, and 13

Storage hierarchy Cache Main memory Disk Tape Very fast Fast Slower Slow Very small Small Bigger Very big (KB) (MB) (GB) (TB) Built-in Expensive Cheap Dirt cheap Disks: data is stored on concentric circular

### Parser. Select R.text from Report R, Weather W where W.image.rain() and W.city = R.city and W.date = R.date and R.text.

Select R.text from Report R, Weather W where W.image.rain() and W.city = R.city and W.date = R.date and R.text. Lifecycle of an SQL Query CSE 190D base System Implementation Arun Kumar Query Query Result

### CSE 190D Database System Implementation

CSE 190D Database System Implementation Arun Kumar Topic 1: Data Storage, Buffer Management, and File Organization Chapters 8 and 9 (except 8.5.4 and 9.2) of Cow Book Slide ACKs: Jignesh Patel, Paris Koutris

### Basic SQL. Basic SQL. Basic SQL

Basic SQL Dr Fawaz Alarfaj Al Imam Mohammed Ibn Saud Islamic University ACKNOWLEDGEMENT Slides are adopted from: Elmasri & Navathe, Fundamentals of Database Systems MySQL Documentation Basic SQL Structured

### Query optimization. Elena Baralis, Silvia Chiusano Politecnico di Torino. DBMS Architecture D B M G. Database Management Systems. Pag.

Database Management Systems DBMS Architecture SQL INSTRUCTION OPTIMIZER MANAGEMENT OF ACCESS METHODS CONCURRENCY CONTROL BUFFER MANAGER RELIABILITY MANAGEMENT Index Files Data Files System Catalog DATABASE

### Review 1-- Storing Data: Disks and Files

Review 1-- Storing Data: Disks and Files Chapter 9 [Sections 9.1-9.7: Ramakrishnan & Gehrke (Text)] AND (Chapter 11 [Sections 11.1, 11.3, 11.6, 11.7: Garcia-Molina et al. (R2)] OR Chapter 2 [Sections 2.1,

### Column Stores vs. Row Stores How Different Are They Really?

Column Stores vs. Row Stores How Different Are They Really? Daniel J. Abadi (Yale) Samuel R. Madden (MIT) Nabil Hachem (AvantGarde) Presented By : Kanika Nagpal OUTLINE Introduction Motivation Background

### Overview of Query Evaluation

Overview of Query Evaluation Chapter 12 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Overview of Query Evaluation Plan: Tree of R.A. operators, with choice of algorithm for each operator.

### Disks and Files. Storage Structures Introduction Chapter 8 (3 rd edition) Why Not Store Everything in Main Memory?

Why Not Store Everything in Main Memory? Storage Structures Introduction Chapter 8 (3 rd edition) Sharma Chakravarthy UT Arlington sharma@cse.uta.edu base Management Systems: Sharma Chakravarthy Costs

### Lecture 4. ISAM and B + -trees. Database Systems. Tree-Structured Indexing. Binary Search ISAM. B + -trees

Lecture 4 and Database Systems Binary Multi-Level Efficiency Partitioned 1 Ordered Files and Binary How could we prepare for such queries and evaluate them efficiently? 1 SELECT * 2 FROM CUSTOMERS 3 WHERE

### Indexing. Chapter 8, 10, 11. Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1

Indexing Chapter 8, 10, 11 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Tree-Based Indexing The data entries are arranged in sorted order by search key value. A hierarchical search

### Track Join. Distributed Joins with Minimal Network Traffic. Orestis Polychroniou! Rajkumar Sen! Kenneth A. Ross

Track Join Distributed Joins with Minimal Network Traffic Orestis Polychroniou Rajkumar Sen Kenneth A. Ross Local Joins Algorithms Hash Join Sort Merge Join Index Join Nested Loop Join Spilling to disk

### Relational Query Optimization

Relational Query Optimization Module 4, Lectures 3 and 4 Database Management Systems, R. Ramakrishnan 1 Overview of Query Optimization Plan: Tree of R.A. ops, with choice of alg for each op. Each operator

### Scan and its Uses. 1 Scan. 1.1 Contraction CSE341T/CSE549T 09/17/2014. Lecture 8

CSE341T/CSE549T 09/17/2014 Lecture 8 Scan and its Uses 1 Scan Today, we start by learning a very useful primitive. First, lets start by thinking about what other primitives we have learned so far? The

### Disks, Memories & Buffer Management

Disks, Memories & Buffer Management The two offices of memory are collection and distribution. - Samuel Johnson CS3223 - Storage 1 What does a DBMS Store? Relations Actual data Indexes Data structures

### L9: Storage Manager Physical Data Organization

L9: Storage Manager Physical Data Organization Disks and files Record and file organization Indexing Tree-based index: B+-tree Hash-based index c.f. Fig 1.3 in [RG] and Fig 2.3 in [EN] Functional Components

### COLUMN-STORES VS. ROW-STORES: HOW DIFFERENT ARE THEY REALLY? DANIEL J. ABADI (YALE) SAMUEL R. MADDEN (MIT) NABIL HACHEM (AVANTGARDE)

COLUMN-STORES VS. ROW-STORES: HOW DIFFERENT ARE THEY REALLY? DANIEL J. ABADI (YALE) SAMUEL R. MADDEN (MIT) NABIL HACHEM (AVANTGARDE) PRESENTATION BY PRANAV GOEL Introduction On analytical workloads, Column

### MTA Database Administrator Fundamentals Course

MTA Database Administrator Fundamentals Course Session 1 Section A: Database Tables Tables Representing Data with Tables SQL Server Management Studio Section B: Database Relationships Flat File Databases

### Data Structure Lecture#15: Non-Binary Trees 2 (Chapter 6) U Kang Seoul National University

Data Structure Lecture#15: Non-Binary Trees 2 (Chapter 6) U Kang Seoul National University U Kang (2016) 1 In This Lecture Main ideas in implementations of general trees Compare advantages and disadvantages

### Hash-Based Indexing 165

Hash-Based Indexing 165 h 1 h 0 h 1 h 0 Next = 0 000 00 64 32 8 16 000 00 64 32 8 16 A 001 01 9 25 41 73 001 01 9 25 41 73 B 010 10 10 18 34 66 010 10 10 18 34 66 C Next = 3 011 11 11 19 D 011 11 11 19

### Relational Query Optimization. Highlights of System R Optimizer

Relational Query Optimization Chapter 15 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Highlights of System R Optimizer v Impact: Most widely used currently; works well for < 10 joins.

### Overview of Query Evaluation. Overview of Query Evaluation

Overview of Query Evaluation Chapter 12 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Overview of Query Evaluation v Plan: Tree of R.A. ops, with choice of alg for each op. Each operator

### REPORT ON SQL TUNING USING INDEXING

REPORT ON SQL TUNING USING INDEXING SUBMITTED BY SRUNOKSHI KANIYUR PREMA NEELAKANTAN CIS -798 INDEPENDENT STUDY COURSE PROFESSOR Dr.TORBEN AMTOFT Kansas State University Page 1 of 38 TABLE OF CONTENTS

### Sorting a file in RAM. External Sorting. Why Sort? 2-Way Sort of N pages Requires Minimum of 3 Buffers Pass 0: Read a page, sort it, write it.

Sorting a file in RAM External Sorting Chapter 13 Three steps: Read the entire file from disk into RAM Sort the records using a standard sorting procedure, such as Shell sort, heap sort, bubble sort, Write

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY Database Systems: Fall 2009 Quiz I Solutions

Department of Electrical Engineering and Computer Science MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.830 Database Systems: Fall 2009 Quiz I Solutions There are 15 questions and 12 pages in this quiz booklet.

### CS 253. January 14, 2017

CS 253 Department of Computer Science College of Engineering Boise State University January 14, 2017 1/30 Motivation Most programming tasks can be implemented using abstractions (e.g. representing data

### Implementation of Relational Operations. Introduction. CS 186, Fall 2002, Lecture 19 R&G - Chapter 12

Implementation of Relational Operations CS 186, Fall 2002, Lecture 19 R&G - Chapter 12 First comes thought; then organization of that thought, into ideas and plans; then transformation of those plans into

### Lecture 2: Streaming Algorithms for Counting Distinct Elements

Lecture 2: Streaming Algorithms for Counting Distinct Elements 20th August, 2008 Streaming Algorithms Streaming Algorithms Streaming algorithms have the following properties: 1 items in the stream are

### Storing Data: Disks and Files. Administrivia (part 2 of 2) Review. Disks, Memory, and Files. Disks and Files. Lecture 3 (R&G Chapter 7)

Storing : Disks and Files Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet Lecture 3 (R&G Chapter 7) Administrivia Greetings Office Hours Prof. Franklin

### Overview of Implementing Relational Operators and Query Evaluation

Overview of Implementing Relational Operators and Query Evaluation Chapter 12 Motivation: Evaluating Queries The same query can be evaluated in different ways. The evaluation strategy (plan) can make orders

### Managing Storage: Above the Hardware

Managing Storage: Above the Hardware 1 Where we are Last time: hardware HDDs and SSDs Today: how the DBMS uses the hardware to provide fast access to data 2 How DBMS manages storage "Bottom" two layers

### Schema for Examples. Query Optimization. Alternative Plans 1 (No Indexes) Motivating Example. Alternative Plans 2 With Indexes

Schema for Examples Query Optimization (sid: integer, : string, rating: integer, age: real) (sid: integer, bid: integer, day: dates, rname: string) Similar to old schema; rname added for variations. :

### Database Systems. Announcement. December 13/14, 2006 Lecture #10. Assignment #4 is due next week.

Database Systems ( 料 ) December 13/14, 2006 Lecture #10 1 Announcement Assignment #4 is due next week. 2 1 Overview of Query Evaluation Chapter 12 3 Outline Query evaluation (Overview) Relational Operator

### Lecture Query evaluation. Combining operators. Logical query optimization. By Marina Barsky Winter 2016, University of Toronto

Lecture 02.03. Query evaluation Combining operators. Logical query optimization By Marina Barsky Winter 2016, University of Toronto Quick recap: Relational Algebra Operators Core operators: Selection σ

### QUERY OPTIMIZATION E Jayant Haritsa Computer Science and Automation Indian Institute of Science. JAN 2014 Slide 1 QUERY OPTIMIZATION

E0 261 Jayant Haritsa Computer Science and Automation Indian Institute of Science JAN 2014 Slide 1 Database Engines Main Components Query Processing Transaction Processing Access Methods JAN 2014 Slide

### Chapter 12: Indexing and Hashing

Chapter 12: Indexing and Hashing Basic Concepts Ordered Indices B+-Tree Index Files B-Tree Index Files Static Hashing Dynamic Hashing Comparison of Ordered Indexing and Hashing Index Definition in SQL

### Representing Data Elements

Representing Data Elements Week 10 and 14, Spring 2005 Edited by M. Naci Akkøk, 5.3.2004, 3.3.2005 Contains slides from 18.3.2002 by Hector Garcia-Molina, Vera Goebel INF3100/INF4100 Database Systems Page

### R & G Chapter 13. Implementation of single Relational Operations Choices depend on indexes, memory, stats, Joins Blocked nested loops:

Relational Query Optimization R & G Chapter 13 Review Implementation of single Relational Operations Choices depend on indexes, memory, stats, Joins Blocked nested loops: simple, exploits extra memory

### Data Compression for Bitmap Indexes. Y. Chen

Data Compression for Bitmap Indexes Y. Chen Abstract Compression Ratio (CR) and Logical Operation Time (LOT) are two major measures of the efficiency of bitmap indexing. Previous works by [5, 9, 10, 11]

### Deep Dive Into Storage Optimization When And How To Use Adaptive Compression. Thomas Fanghaenel IBM Bill Minor IBM

Deep Dive Into Storage Optimization When And How To Use Adaptive Compression Thomas Fanghaenel IBM Bill Minor IBM Agenda Recap: Compression in DB2 9 for Linux, Unix and Windows New in DB2 10 for Linux,

### Query Optimization. Schema for Examples. Motivating Example. Similar to old schema; rname added for variations. Reserves: Sailors:

Query Optimization Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Schema for Examples (sid: integer, sname: string, rating: integer, age: real) (sid: integer, bid: integer, day: dates,

### SQL Structured Query Language Introduction

SQL Structured Query Language Introduction Rifat Shahriyar Dept of CSE, BUET Tables In relational database systems data are represented using tables (relations). A query issued against the database also

### Selection Statement ( if )

Islamic University Of Gaza Faculty of Engineering Computer Engineering Department Lab 4 Selection Statement ( if ) Eng. Ibraheem Lubbad October 10, 2016 In this lab we will constructs program that allow

### Database Design and Programming

Database Design and Programming Jan Baumbach jan.baumbach@imada.sdu.dk http://www.baumbachlab.net JDBC Java Database Connectivity (JDBC) is a library similar for accessing a DBMS with Java as the host

### Why Is This Important? Overview of Storage and Indexing. Components of a Disk. Data on External Storage. Accessing a Disk Page. Records on a Disk Page

Why Is This Important? Overview of Storage and Indexing Chapter 8 DB performance depends on time it takes to get the data from storage system and time to process Choosing the right index for faster access

### Database System Concepts, 6 th Ed. Silberschatz, Korth and Sudarshan See for conditions on re-use

Chapter 11: Indexing and Hashing Database System Concepts, 6 th Ed. See www.db-book.com for conditions on re-use Chapter 12: Indexing and Hashing Basic Concepts Ordered Indices B + -Tree Index Files Static

### SQL. Lecture 4 SQL. Basic Structure. The select Clause. The select Clause (Cont.) The select Clause (Cont.) Basic Structure.

SL Lecture 4 SL Chapter 4 (Sections 4.1, 4.2, 4.3, 4.4, 4.5, 4., 4.8, 4.9, 4.11) Basic Structure Set Operations Aggregate Functions Null Values Nested Subqueries Derived Relations Modification of the Database

Lecture IV Query Processing Kyumars Sheykh Esmaili Basic Steps in Query Processing 2 Query Optimization Many equivalent execution plans Choosing the best one Based on Heuristics, Cost Will be discussed

### Firebird 3.0 statistics and plans

1 Firebird 3.0 statistics and plans Dmitry Kuzmenko, IBSurgeon 2 Firebird 2017 Tour: Performance Optimization Firebird Tour 2017 is organized by Firebird Project, IBSurgeon and IBPhoenix, and devoted to

### COMP 430 Intro. to Database Systems. Indexing

COMP 430 Intro. to Database Systems Indexing How does DB find records quickly? Various forms of indexing An index is automatically created for primary key. SQL gives us some control, so we should understand

### Insertion Sort: an algorithm for sorting an array

Insertion Sort: an algorithm for sorting an array Let s use arrays to solve a problem that comes up often in programming, namely sorting. Suppose we have an array of objects that is in no particular order

### Indexing and Searching

Indexing and Searching Berlin Chen Department of Computer Science & Information Engineering National Taiwan Normal University References: 1. Modern Information Retrieval, chapter 8 2. Information Retrieval:

### PostgreSQL/Jsonb. A First Look

PostgreSQL/Jsonb A First Look About Me Started programming in 1981 Owner of Enoki Solutions Inc. Consulting and Software Development Running VanDev since Oct 2010 Why PostgreSQL? Open Source Feature Rich

### CMSC424: Database Design. Instructor: Amol Deshpande

CMSC424: Database Design Instructor: Amol Deshpande amol@cs.umd.edu Databases Data Models Conceptual representa1on of the data Data Retrieval How to ask ques1ons of the database How to answer those ques1ons

### SQL STORED ROUTINES. CS121: Relational Databases Fall 2017 Lecture 9

SQL STORED ROUTINES CS121: Relational Databases Fall 2017 Lecture 9 SQL Functions 2 SQL queries can use sophisticated math operations and functions Can compute simple functions, aggregates Can compute

### Chapter 5: Physical Database Design. Designing Physical Files

Chapter 5: Physical Database Design Designing Physical Files Technique for physically arranging records of a file on secondary storage File Organizations Sequential (Fig. 5-7a): the most efficient with

### MySQL B+ tree. A typical B+tree. Why use B+tree?

MySQL B+ tree A typical B+tree Why use B+tree? B+tree is used for an obvious reason and that is speed. As we know that there are space limitations when it comes to memory, and not all of the data can reside

### Chapter 12: Indexing and Hashing

Chapter 12: Indexing and Hashing Database System Concepts, 5th Ed. See www.db-book.com for conditions on re-use Chapter 12: Indexing and Hashing Basic Concepts Ordered Indices B + -Tree Index Files B-Tree

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY Database Systems: Fall 2017 Quiz I

Department of Electrical Engineering and Computer Science MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.830 Database Systems: Fall 2017 Quiz I There are 15 questions and 12 pages in this quiz booklet. To receive

### Column-Stores vs. Row-Stores: How Different Are They Really?

Column-Stores vs. Row-Stores: How Different Are They Really? Daniel J. Abadi, Samuel Madden and Nabil Hachem SIGMOD 2008 Presented by: Souvik Pal Subhro Bhattacharyya Department of Computer Science Indian

### Querying Data with Transact-SQL

Querying Data with Transact-SQL Course: 20761 Course Details Audience(s): IT Professional(s) Technology: Microsoft SQL Server 2016 Duration: 24 HRs. ABOUT THIS COURSE This course is designed to introduce

### Database Applications (15-415)

Database Applications (15-415) DBMS Internals- Part V Lecture 15, March 15, 2015 Mohammad Hammoud Today Last Session: DBMS Internals- Part IV Tree-based (i.e., B+ Tree) and Hash-based (i.e., Extendible

### QUERY EXECUTION: How to Implement Relational Operations?

QUERY EXECUTION: How to Implement Relational Operations? 1 Introduction We ve covered the basic underlying storage, buffering, indexing and sorting technology Now we can move on to query processing Relational

### Storing Data: Disks and Files

Storing Data: Disks and Files Lecture 3 (R&G Chapter 7) Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet Administrivia Greetings Office Hours Prof. Franklin

### Data Warehousing and Data Mining

Data Warehousing and Data Mining Lecture 3 Efficient Cube Computation CITS3401 CITS5504 Wei Liu School of Computer Science and Software Engineering Faculty of Engineering, Computing and Mathematics Acknowledgement:

### Data Structure. Measuring Input size. Composite Data Structures. Linear data structures. Data Structure is: Abstract Data Type 1/9/2014

Data Structure Measuring Input size Last lecture recap. A Data Structure is an aggregation of atomic and composite data into a set with defined relationships. Structure means a set of rules that holds

### Last Class Carnegie Mellon Univ. Dept. of Computer Science /615 - DB Applications

Last Class Carnegie Mellon Univ. Dept. of Computer Science 15-415/615 - DB Applications C. Faloutsos A. Pavlo Lecture#12: External Sorting (R&G, Ch13) Static Hashing Extendible Hashing Linear Hashing Hashing

### 8. Binary Search Tree

8 Binary Search Tree Searching Basic Search Sequential Search : Unordered Lists Binary Search : Ordered Lists Tree Search Binary Search Tree Balanced Search Trees (Skipped) Sequential Search int Seq-Search

### Midterm 1: CS186, Spring 2015

Midterm 1: CS186, Spring 2015 Prof. J. Hellerstein You should receive a double-sided answer sheet and a 7-page exam. Mark your name and login on both sides of the answer sheet, and in the blanks above.

### Efficiency. Efficiency: Indexing. Indexing. Efficiency Techniques. Inverted Index. Inverted Index (COSC 488)

Efficiency Efficiency: Indexing (COSC 488) Nazli Goharian nazli@cs.georgetown.edu Difficult to analyze sequential IR algorithms: data and query dependency (query selectivity). O(q(cf max )) -- high estimate-

### Outlines. Chapter 2 Storage Structure. Structure of a DBMS (with some simplification) Structure of a DBMS (with some simplification)

Outlines Chapter 2 Storage Structure Instructor: Churee Techawut 1) Structure of a DBMS 2) The memory hierarchy 3) Magnetic tapes 4) Magnetic disks 5) RAID 6) Disk space management 7) Buffer management

### Guided Analytics with large Detail Data

Guided Analytics with large Detail Data When we think of guided analytics, we need to think of it as two pieces of the same puzzle. 1. What is it we want the user to be able to accomplish? 2. How do we