NCSS: Databases and SQL

Transcription

1 NCSS: Databases and SQL Tim Dawborn Lecture 1, January, 2016

2 Motivation SQLite SELECT WHERE JOIN Tips 2 Outline 1 Motivation 2 SQLite 3 Searching for Data 4 Filtering Results 5 Joining multiple tables together 6 Summing it up

3 Motivation SQLite SELECT WHERE JOIN Tips 3 Databases are everywhere Who has ever purchased something on the internet? Who has a Facebook page? Who uses Wikipedia?

4 Motivation SQLite SELECT WHERE JOIN Tips 4 The Big Picture There are two ways to build a website: 1 A static website is a fixed set of html files 2 A dynamic website can display information based on user-input Static websites cannot change with user interaction. and require change to the static files for updates

5 Motivation SQLite SELECT WHERE JOIN Tips 5 A dynamic website keeps user- and content-specific data in a database... uses several template files to define the look and feel... runs code on the webserver that personalises the templates (using the database) and displays the resulting page to users HTML & template files Client Web-Server Database

6 Motivation SQLite SELECT WHERE JOIN Tips 6 SQLite There are many different database implementations Some of the major players: MySQL, PostgreSQL, Oracle SQLite is another implementation SQLite is an embedded database system: It is a software library that provides an application with a self-contained SQL database engine.

7 Motivation SQLite SELECT WHERE JOIN Tips 7 Databases A SQLite database consists of a number of tables (relations) The data in a table is defined by the columns (fields) The data entries in a table are called rows (records) id fname lname gender age Column (field) Barry Prue Andrew Mathew Mara Scott Alec Karen Grant Schultz Robinson Varvel Nemes Barber Herdman Newton Barber Ovzinsky M F M M F M M F M Row (record) Table (relation)

8 Motivation SQLite SELECT WHERE JOIN Tips 8 Data Types SQLite is a lightweight database system Only four different data types are supported TEXT for textual data INTEGER for whole integer values REAL for floating point values BLOB for blob-like (binary) data NULL can be used in place of any of these data types Like the special None value in Python

9 Motivation SQLite SELECT WHERE JOIN Tips 9 Implementations We will be interacting with SQLite in two different ways There is an interpreter where you can type commands directly 1 sqlite> We can also access a SQLite database from within Python 1 import sqlite3

10 Motivation SQLite SELECT WHERE JOIN Tips 10 SQL Syntax SQL (Structured Query Language) is the standard language for interacting with databases Syntax basics All statements end in a semicolon Strings are enclosed in single quotes Comments are between /* and */ or from -- to EOL Whitespace independent 1 -- this is our example query 2 SELECT fruit, price 3 FROM /* comments can go in here too */ inventory 4 WHERE fruit!= 'apple' AND price > 1.23;

11 Motivation SQLite SELECT WHERE JOIN Tips 11 Case Study: Sports Carnival Consider the database system used for a sports carnival which is run at your high school This database needs to keep track of all of the results for all of the events which occurred during the day Simplified example database: people id fname lname gender age results event person result events id name age gender at

12 Motivation SQLite SELECT WHERE JOIN Tips 12 Searching for Data The most frequent operation performed on a database is to ask for some data contained within it The SQL SELECT keyword allows us to query the database for data which meets some criterion The basic syntax for the SELECT clause is 1 SELECT <field_names> 2 FROM <table_name> <field_names> is either a comma-separated list of fields, or the special * character

13 Motivation SQLite SELECT WHERE JOIN Tips 13 Examples of SELECT How to we list everything from the people table?

14 Motivation SQLite SELECT WHERE JOIN Tips 13 Examples of SELECT How to we list everything from the people table? 1 SELECT * 2 FROM people;

15 Motivation SQLite SELECT WHERE JOIN Tips 13 Examples of SELECT How to we list everything from the people table? 1 SELECT * 2 FROM people; The wildcard character means every field in this table 1 SELECT id, fname, lname, gender, age 2 FROM people;

16 Motivation SQLite SELECT WHERE JOIN Tips 13 Examples of SELECT How to we list everything from the people table? 1 SELECT * 2 FROM people; The wildcard character means every field in this table 1 SELECT id, fname, lname, gender, age 2 FROM people; What if we wanted to only list people s first and last names?

17 Motivation SQLite SELECT WHERE JOIN Tips 13 Examples of SELECT How to we list everything from the people table? 1 SELECT * 2 FROM people; The wildcard character means every field in this table 1 SELECT id, fname, lname, gender, age 2 FROM people; What if we wanted to only list people s first and last names? 1 SELECT fname, lname 2 FROM people;

18 Motivation SQLite SELECT WHERE JOIN Tips 14 Functions and Aggregation The values in the columns in a SELECT clause can be transformed via functions Database implementations have a number of builtin functions SQLite: corefunc.html Example: Return all of the surnames in uppercase

19 Motivation SQLite SELECT WHERE JOIN Tips 14 Functions and Aggregation The values in the columns in a SELECT clause can be transformed via functions Database implementations have a number of builtin functions SQLite: corefunc.html Example: Return all of the surnames in uppercase 1 SELECT fname, UPPER(lname) 2 FROM people;

20 Motivation SQLite SELECT WHERE JOIN Tips 14 Functions and Aggregation The values in the columns in a SELECT clause can be transformed via functions Database implementations have a number of builtin functions SQLite: corefunc.html Example: Return all of the surnames in uppercase 1 SELECT fname, UPPER(lname) 2 FROM people; Aggregate functions reduce a whole table or column to just one value, e.g. for counting: 1 SELECT COUNT(*) 2 FROM people;

21 Motivation SQLite SELECT WHERE JOIN Tips 15 Columns as Expressions Column names can also be any valid expression What would we expect this query to produce? 1 SELECT fname, age AS birthyear, 3 age / 10 AS decade 4 FROM people;

22 Motivation SQLite SELECT WHERE JOIN Tips 16 Distinct Values Sometimes you want to list only the unique values in a column How would we list all surnames?

23 Motivation SQLite SELECT WHERE JOIN Tips 16 Distinct Values Sometimes you want to list only the unique values in a column How would we list all surnames? 1 SELECT lname 2 FROM people;

24 Motivation SQLite SELECT WHERE JOIN Tips 16 Distinct Values Sometimes you want to list only the unique values in a column How would we list all surnames? 1 SELECT lname 2 FROM people; How about all unique surnames?

25 Motivation SQLite SELECT WHERE JOIN Tips 16 Distinct Values Sometimes you want to list only the unique values in a column How would we list all surnames? 1 SELECT lname 2 FROM people; How about all unique surnames? The SQL DISTINCT keyword is used in this situation 1 SELECT DISTINCT lname 2 FROM people;

26 Motivation SQLite SELECT WHERE JOIN Tips 17 Ordering the Results Often we would like the result ordered in some specific way In SQL, this is done with the ORDER BY clause For example: List the people in the database alphabetically by last name

27 Motivation SQLite SELECT WHERE JOIN Tips 17 Ordering the Results Often we would like the result ordered in some specific way In SQL, this is done with the ORDER BY clause For example: List the people in the database alphabetically by last name 1 SELECT * 2 FROM people 3 ORDER BY lname;

28 Motivation SQLite SELECT WHERE JOIN Tips 17 Ordering the Results Often we would like the result ordered in some specific way In SQL, this is done with the ORDER BY clause For example: List the people in the database alphabetically by last name 1 SELECT * 2 FROM people 3 ORDER BY lname; How about in descending order?

29 Motivation SQLite SELECT WHERE JOIN Tips 17 Ordering the Results Often we would like the result ordered in some specific way In SQL, this is done with the ORDER BY clause For example: List the people in the database alphabetically by last name 1 SELECT * 2 FROM people 3 ORDER BY lname; How about in descending order? 1 SELECT * 2 FROM people 3 ORDER BY lname DESC;

30 Motivation SQLite SELECT WHERE JOIN Tips 18 Multiple Ordering Criteria Order all people by capitalised last name in ascending order, and break ties using their first name in descending order

31 Motivation SQLite SELECT WHERE JOIN Tips 18 Multiple Ordering Criteria Order all people by capitalised last name in ascending order, and break ties using their first name in descending order 1 SELECT * 2 FROM people 3 ORDER BY UPPER(lname), fname DESC;

32 Motivation SQLite SELECT WHERE JOIN Tips 18 Multiple Ordering Criteria Order all people by capitalised last name in ascending order, and break ties using their first name in descending order 1 SELECT * 2 FROM people 3 ORDER BY UPPER(lname), fname DESC; Note that the ordering values can be expressions Note also that the ordering modifier keyword applies only to the previous ordering criterion

33 Motivation SQLite SELECT WHERE JOIN Tips 19 Limiting no. Results We can place an upper bound on the number of rows returned using the LIMIT SQL keyword 1 SELECT * 2 FROM people 3 LIMIT 3; Normally used in conjunction with ORDER BY Example: Find the 3 people whose surnames are last alphabetically

34 Motivation SQLite SELECT WHERE JOIN Tips 19 Limiting no. Results We can place an upper bound on the number of rows returned using the LIMIT SQL keyword 1 SELECT * 2 FROM people 3 LIMIT 3; Normally used in conjunction with ORDER BY Example: Find the 3 people whose surnames are last alphabetically 1 SELECT * 2 FROM people 3 ORDER BY lname DESC 4 LIMIT 3;

35 Motivation SQLite SELECT WHERE JOIN Tips 20 Filtering Results using WHERE Always returning every row from a table is not always desirable imagine how big Facebook s database is... You often want to return only rows which meet some condition The SELECT clause can contain a WHERE clause Syntax is WHERE <condition> Six standard comparison operators for constructing conditional expressions: =,!=, <, <=, >, and >=

36 Motivation SQLite SELECT WHERE JOIN Tips 21 Filtering Results We can list only the people in the database who are male by creating a condition expression on the gender column 1 SELECT * 2 FROM people 3 WHERE gender = 'M';

37 Motivation SQLite SELECT WHERE JOIN Tips 21 Filtering Results We can list only the people in the database who are male by creating a condition expression on the gender column 1 SELECT * 2 FROM people 3 WHERE gender = 'M'; What (and why) will the following query return? 1 SELECT * 2 FROM people 3 WHERE 1 = 0;

38 Motivation SQLite SELECT WHERE JOIN Tips 22 Comparing Data Types Comparison operators with different data types 1 sqlite> SELECT 'one' = 'one'; sqlite> SELECT 'one' = 'One'; sqlite> SELECT 'a' > 'b'; sqlite> SELECT 'a' < 'b'; sqlite> SELECT 1 > -1; sqlite> SELECT 1 > 1; sqlite> SELECT 1.0 = 1; 14 1

39 Motivation SQLite SELECT WHERE JOIN Tips 23 Comparing Data Types How about when we compare NULL values? 1 sqlite> SELECT NULL = NULL; 2 NULL 3 sqlite> SELECT NULL!= NULL; 4 NULL

40 Motivation SQLite SELECT WHERE JOIN Tips 23 Comparing Data Types How about when we compare NULL values? 1 sqlite> SELECT NULL = NULL; 2 NULL 3 sqlite> SELECT NULL!= NULL; 4 NULL SQL provides the special IS NULL syntax for NULL checking 1 sqlite> SELECT NULL IS NULL; sqlite> SELECT NULL IS NOT NULL; 4 0

41 Motivation SQLite SELECT WHERE JOIN Tips 24 Filtering Results List all results where the result field is NULL

42 Motivation SQLite SELECT WHERE JOIN Tips 24 Filtering Results List all results where the result field is NULL 1 SELECT * 2 FROM results 3 WHERE result IS NULL;

43 Motivation SQLite SELECT WHERE JOIN Tips 24 Filtering Results List all results where the result field is NULL 1 SELECT * 2 FROM results 3 WHERE result IS NULL; List only the fastest result from event with ID 0

44 Motivation SQLite SELECT WHERE JOIN Tips 24 Filtering Results List all results where the result field is NULL 1 SELECT * 2 FROM results 3 WHERE result IS NULL; List only the fastest result from event with ID 0 1 SELECT DISTINCT result 2 FROM results 3 WHERE event = 0 4 AND result IS NOT NULL 5 ORDER BY result 6 LIMIT 1;

45 Motivation SQLite SELECT WHERE JOIN Tips 25 Joining Tables Sometimes, we need to combine data from multiple tables... E.g. List all personal details about competitors who achieved a result of 20 seconds or less in some event The results and people tables are linked by the foreign id values

46 Motivation SQLite SELECT WHERE JOIN Tips 26 Joining Two Tables Who achieved the different times of each event?

47 Motivation SQLite SELECT WHERE JOIN Tips 26 Joining Two Tables Who achieved the different times of each event? 1 SELECT fname, lname, event, result 2 FROM results, people 3 WHERE person = id;

48 Motivation SQLite SELECT WHERE JOIN Tips 26 Joining Two Tables Who achieved the different times of each event? 1 SELECT fname, lname, event, result 2 FROM results, people 3 WHERE person = id; Note: Whenever you list more than one table under FROM, you probably want an explicit join condition in your WHERE clause What happens in the above query if we didn t have the WHERE condition in place?

49 Motivation SQLite SELECT WHERE JOIN Tips 27 Inner Join There is a more efficient notation via the JOIN operator 1 SELECT * 2 FROM results 3 JOIN people ON results.person = people.id; Table aliases just like column aliases 1 SELECT * 2 FROM results r 3 JOIN people p ON r.person = p.id;

50 Motivation SQLite SELECT WHERE JOIN Tips 28 Some Notes on Schema Creation and Updates CREATE TABLE creates a new table in a database (cf..schema command) INSERT a new row to a table: 1 INSERT INTO results VALUES (3, 2, '00:22'); UPDATE an existing entry: 1 UPDATE results 2 SET result = '00:21' 3 WHERE event = 3 AND person = 2; DELETE (one or more careful!) rows: 1 DELETE FROM results 2 WHERE event = 3 AND person = 2;

51 Motivation SQLite SELECT WHERE JOIN Tips 29 More Complex SQL List all of the females ordered by last name

52 Motivation SQLite SELECT WHERE JOIN Tips 29 More Complex SQL List all of the females ordered by last name 1 SELECT * 2 FROM people 3 WHERE gender = 'F' 4 ORDER BY lname;

53 Motivation SQLite SELECT WHERE JOIN Tips 29 More Complex SQL List all of the females ordered by last name 1 SELECT * 2 FROM people 3 WHERE gender = 'F' 4 ORDER BY lname; List all females, plus all males who are over 18

54 Motivation SQLite SELECT WHERE JOIN Tips 29 More Complex SQL List all of the females ordered by last name 1 SELECT * 2 FROM people 3 WHERE gender = 'F' 4 ORDER BY lname; List all females, plus all males who are over 18 1 SELECT * 2 FROM people 3 WHERE gender = 'F' 4 OR (gender = 'M' AND age > 18);

55 Motivation SQLite SELECT WHERE JOIN Tips 30 More Complex SQL List the name and completion result for each competitor in the male 16 years 100m event, ordering the results increasing in completion result.

56 Motivation SQLite SELECT WHERE JOIN Tips 30 More Complex SQL List the name and completion result for each competitor in the male 16 years 100m event, ordering the results increasing in completion result. 1 SELECT p.fname, p.lname, r.result 2 FROM results r 3 JOIN people p ON r.person = p.id 4 JOIN events e ON r.event = e.id 5 WHERE e.name = '100m' 6 AND e.gender = 'M' 7 AND e.age = 16 8 AND r.result IS NOT NULL 9 ORDER BY r.result ASC;

57 Motivation SQLite SELECT WHERE JOIN Tips 31 Summing up Databases are used to efficiently store data We can query them using the SQL language SQLite is a database implementation that we can use from Python Next time: SQL from Python, and good database design