CSC 120 Worksheet 12 Databases

Transcription

1 CSC 120 Worksheet 12 Databases 1 Format for SQLite Commands We will create tables and retrieve data from the tables using Python and SQLite. You can find a list of Python and SQLite commands at the end of this handout as well as the tables we will use as examples. You may even find it helpful to pull it o so that you can have it at your side. As always, to get more/better information on what is available to you, browse the documentation! 2 Creating Tables Let s write a function to read in a file and populate the table Precipitation that we saw earlier. def create_precipitation(db, data_file): (str, file open for reading) -> NoneType Populate the database with name/path db with the contents of data_file as follows: create a table called Precipitation, with four columns: City (text), Snow (real), Total (integer), Days (integer). (data_file contains one city, snowfall amount, total precipitation amount, and number of days, per line, separated by comma.) # Connect to database db. # Get cursor. # Create a new table. # Populate the Precipitation Table. # Loop through each line in the file: # Get the data from the file and insert into the table. # Commit the changes to the database. # Close cursor and connection. 1

2 3 Accessing Data How can we access the data in the database? def print_all_precipitation(db): (str) -> NoneType Print all records in database db (name/path to a database) in Precipitation table, one per line. # Open database. # Get cursor. # Select desired data. # Fetch and print everything selected. # Close cursor and database. 4 Simple Queries Let s write the following functions to retrieve data from the table. def city_snow(db): (str) -> list of tuple Return a list of tuples [(city, snow),...] from the Precipitation table in the database with name db. con = sqlite3.connect(db) cur = con.cursor() # Execute query. # Get all results. cur.close() con.close() return result 2

3 def city_total(db): (str) -> list of tuple Return a list of tuples [(city, total),...] from the Precipitation table in the database db. con = sqlite3.connect(db) cur = con.cursor() # Execute query. # Get all results. cur.close() con.close() return result Notice that the only line changing in the two functions is the one with cur.execute(). We can take advantage of this by writing a general run query function! def run_query(db, db_query): (str, str) -> list of tuple Return the results of executing query db_query on database with name db. con = sqlite3.connect(db) cur = con.cursor() # Execute query result = cur.fetchall() cur.close() con.close() return result We can now re-write our functions using run query(): def city_snow_v2(db): (str) -> list of tuple Return a list of tuples [(city, snow),...] from the Precipitation table in the database with name db. 3

4 def city_total_v2(db): (str) -> list of tuple Return a list of tuples [(city, total),...] from the Precipitation table in the database named db. Much better! 4.1 Adding a WHERE condition We can restrict the rows that we select by adding query conditions using WHERE. Think of this as adding a filter on the rows that the SELECT query returns. For example, it might be that we only want those cities with precipitation on more than 180 days of the year. We would execute the statement: cur.execute('select City, Days FROM Precipitation WHERE Days > 180') or (notice the tuple of one element!): cur.execute('select City, Days FROM Precipitation WHERE Days >?', (180,)) Let s write a function that returns all cities from Precipitation table where the total snowfall is greater than 200cm: def snow_over_200(db): (str) -> list of tuple Return a list of cities from the Precipitation table in the database with name db, where the amount of snowfall is > 200 cm. It might be nice to be able to make a more generalized function that allows us to specify amounts for the snow and total precipitation as arguments to the function. Complete the function snow total over below. def snow_total_over(db, x, y): (str, number, number) -> list of tuple Return the city names, snowfall amounts, and total precipitation of the cities with at least x cm of snowfall and y total precipitation from the Precipitation table of database with name db. con = sqlite3.connect(db) cur = con.cursor() 4

5 # write query here result = cur.fetchall() cur.close() con.close() return result Q. Can we use the helper function run query in this function? A. Let s improve run query() to allow us to pass query arguments. Notice the header: we are introducing a new way to pass parameters, args=none means if there isn t a third argument passed to run query() then assume that the value is None. That is, the third argument is optional. def run_query_v2(db, q, args=none): (str, str [, tuple]) -> list of tuple Return the results of running query q with arguments args on database db. con = sqlite3.connect(db) cur = con.cursor() if args is None: else: data = cur.fetchall() cur.close() con.close() return data Let s use our new run query v2 function to rewrite snow total over. def snow_total_over_v2(db, x, y): (str, number, number) -> list of tuple Return the cities and snowfall amounts of the cities with at least x cm of snowfall and y total precipitation from the Precipitation table of database db. Notice that we can still use run query v2 as we did run query for function city snow v2: def city_snow_v2(db): (str) -> list of tuple Return a list of tuples [(city, snow),...] from the Precipitation table in the database db (name/path to a database). 5

6 5 Joins Sometimes we want to select data from two or more di erent tables. Consider the following two tables, one which lists library card ID numbers with names and one which lists library card ID numbers with books on loan from the library. Table: Library ID ID Name Homer Marge Bart Lisa Table: Library Loans ID Book NULL Updos for the Modern Woman The Etymology of Slang Quantum Physics for Minors LSAT Prep Guide Suppose that we want to list the books taken out of the library for each student. This requires using one table to look up into another. We can do this by joining the tables. The join of two tables is their cross product. This means you match up every row of the left table with every row of the right table. The symbol for join is on. Here is the result of Library ID on Library Loans: ID Name ID Book Homer NULL Homer Updos for the Modern Woman Homer The Etymology of Slang Homer Quantum Physics for Minors Homer LSAT Prep Guide Marge NULL Marge Updos for the Modern Woman Marge The Etymology of Slang Marge Quantum Physics for Minors Marge LSAT Prep Guide Bart NULL Bart Updos for the Modern Woman Bart The Etymology of Slang Bart Quantum Physics for Minors Bart LSAT Prep Guide Lisa NULL Lisa Updos for the Modern Woman Lisa The Etymology of Slang Lisa Quantum Physics for Minors Lisa LSAT Prep Guide Notice that in this case, the only lines that are useful to us are the ones where the library IDs match. How can we get the list of Books taken out by a person, given their Name? 6

7 Let s do a concrete example. Consider another table Temperature related to our Precipitation table (see page 12 of this handout) populated from the file temperature.txt. First, let s add it to our database: def setup_temp(db, data_file): (str, file open for reading) -> Nonetype Populate the database db with the contents of data_file. Create a table called Temperature, with nine columns: City (text), AvgHigh (real), AvgLow (real), ColdMonth (text), ColdAvgHigh (real), ColdAvgLow (real), WarmMonth (text), WarmAvgHigh (real), WarmAvgLow (real). # Connect to database. con = sqlite3.connect(db) # Get cursor. cur = con.cursor() # Create the Temperature table. cur.execute('''create TABLE Temperature( City TEXT, AvgHigh REAL, AvgLow REAL, ColdMonth TEXT, ColdAvgHigh REAL, ColdAvgLow REAL, WarmMonth TEXT, WarmAvgHigh REAL, WarmAvgLow REAL)''') # Populate the Temperature Table. for line in data_file: data = line.split(",") cur.execute('''insert INTO Temperature VALUES(?,?,?,?,?,?,?,?,?)''', \ (data[0].strip(), float(data[1].strip()), float(data[2].strip()), data[3].strip(), float(data[4].strip()), float(data[5].strip()), data[6].strip(), float(data[7].strip()), float(data[8].strip()))) # Commit the changes to the database. con.commit() # Close cursor and connection. cur.close() con.close() We will also use a table Geography populated by the data in the file geography.txt. Can you do this one yourself? 7

8 def create_geography(db, data_file): (str, file open for reading) -> NoneType Populate the database db with the contents of data_file. Create a table called Geography, with two columns: City (text), Province (text). 5.1 Using JOINs with SELECT We will now practise writing SELECT statements with JOINs. Write a SELECT statement to pass to run query v2 that returns the names of cities which have an average low temperature less than 1 and at least 250cm of snow. What tables do you need to join here? Write a SELECT statement to pass to run query v2 that returns the names of cities and provinces, such that the city has average high temperature over 10 and total precipitation less than 900. What tables do you need to join here? We can also join a table with itself! What problem do you have when trying to write such a SELECT statement? We can solve this by... 8

9 Q. Write a query that returns the names of cities which have the same average low temperatures. Notice that there are duplicates, but the order is permuted we can get rid of these by doing some post processing (working with the results after). 5.2 Using DISTINCT Now consider the following example: Q. Write a query that returns the names of the provinces which have cities where the average low temperature is negative. Notice that this returns duplicates. We can get rid of them by using SELECT DISTINCT. 5.3 Aggregation There are several built-in column functions that we can use: they are AVG, MIN, MAX, SUM, COUNT. They are called aggregate functions and are used as SELECT FUNCTION NAME(column) etc... Q. Example: Use AVG to find the average snowfall amount in table Precipitation. 5.4 Using GROUP BY Suppose that we want to get the average days of precipitation for each province. This means we want to group the Days values according to the province the city is in, and then take the average of the Days belonging to the same province. For example, Ontario cities are Ottawa and Toronto and the total days precipitation are 159 and 139, giving an average of 149. We use the GROUP BY statement: SELECT Geography.Province, AVG(Precipitation.Days) FROM Precipitation JOIN Geography ON Precipitation.City = Geography.City GROUP BY Geography.Province 9

10 10

11 Python Commands import sqlite3: use SQLite to access a database. con = sqlite3.connect("weather.db"): connect to the database stored in file weather.db and name this connection con. cur = con.cursor(): Get a cursor to work with the database and name it cur. cur.execute("sql command"): Execute the given SQL command (string) on the database accessed with the cursor cur. cur.fetchall(): Get a list of rows of the table (tuples) from the columns that have been selected. cur.fetchone(): Get one row of the table (tuple) from the columns that have been selected. cur.close(): Close the cursor. con.commit(): Make our changes to the database permanent by committing them. con.close(): Close the connection. SQL Commands CREATE TABLE TableName(ColumnName 1 TYPE 1, ColumnName 2 TYPE 2,..., ColumnName N TYPE N) DROP TABLE TableName S INSERT INTO TableName VALUES(Val 1, Val 2,..., Val n) INSERT INTO TableName VALUES(?,?,...,?), (Val 1, Val 2,..., Val n) SELECT * FROM TableName SELECT Col 1, Col 2,..., Col n FROM TableName SELECT Col 1, Col 2,..., Col n FROM TableName WHERE condition 1 AND condition 2 AND... AND condition n SELECT TableName1.ColName, TableName2.ColName FROM TableName1 JOIN TableName2 ON condition WHERE condition 1 AND condition 2 AND... AND condition n SELECT DISTINCT TableName1.ColName, TableName2.ColName FROM TableName1 JOIN TableName2 ON condition WHERE condition 1 AND condition 2 AND... AND condition n SELECT TableName1.ColName1, TableName2.ColName2 FROM TableName1 JOIN TableName2 ON condition WHERE condition 1 AND condition 2 AND... AND condition n GROUP BY ColName i UPDATE TableName SET Col 1 = val 1, Col 2 = val 2,... WHERE condition 1 AND condition 2 AND... DELETE FROM TableName WHERE condition

12 NOTE: In the files given on the website, the headers have been stripped and the values are separated by commas. precipitation.txt geography.txt City Snow Total Days St.John s Charlottetown Halifax Fredericton Quebec Montreal Ottawa Toronto Winnipeg Regina Edmonton Calgary Vancouver Victoria Whitehorse Yellowknife City St.John s Charlottetown Halifax Fredericton Quebec Montreal Ottawa Toronto Winnipeg Regina Edmonton Calgary Vancouver Victoria Whitehorse Yellowknife Province Newfoundland PEI Nova Scotia New Brunswick Quebec Quebec Ontario Ontario Manitoba Saskatchewan Alberta Alberta BC BC Yukon NWT temperature.txt City AvgHigh AvgLow ColdMonth ColdAvgHigh ColdAvgLow WarmMonth WarmAvgHigh WarmAvgLow St.John s February July Charlottetown January July Halifax February July Fredericton January July Quebec January July Montreal January July Ottawa January July Toronto January July Winnipeg January July Regina January July Edmonton January July Calgary January July Vancouver January August Victoria January July Whitehorse January July Yellowknife January July