Traffic violations revisited

Transcription

1 Traffic violations revisited November 9, 2017 In this lab, you will once again extract data about traffic violations from a CSV file, but this time you will use SQLite. First, download the following files CSV file from project 2 wk11thurs.py 1 Creating a database 1. Open a Python shell in the directory containing your two lab files. 2. import sqlite3 3. Create a database wk11thurs.db in your lab directory. To do this, simply create a connection to the desired database: conn = sqlite3.connect("wk11thurs.db") If the database doesn t exist already, Python will create it for you. There should now be a file called wk11thurs.db in your lab directory. 4. When you are done, close the connection: conn.close() 2 Creating a table In wk11thurs.py, complete the function createtable() as follows: 1. In the Python shell, connect to wk11thurs.db again. 1

2 2. Create a cursor object: c = conn.cursor() This is the type of object you use to execute SQL commands in your database. 3. Create a table violations with the columns specified by the first line of the CSV file: Coumn Type intersection TEXT camera id INT address TEXT date TEXT num violations INT (You can ignore the remaining 5 columns in the CSV.) The SQLite syntax for creating this table is CREATE TABLE violations( intersection text, camera id int, address text, date text, num violations int ) Create a string containing this command and store it in a variable cmd. Then use your cursor to execute the command: c.execute(cmd) If you did it correctly, Python will print something like <sqlite3.cursor at 0x109dece30>. Otherwise, you ll get an error. 4. When you have successfully created the table, save your changes to the database by committing them: conn.commit() You must do this whenever you modify a database; otherwise, any changes you make will be discarded when you close the database (just like when you forget to save your changes to a Word document before closing it). 5. Close the connection when you re done. 2

3 3 Populating the table Open the Python script wk11thurs.py. You will add code to read the input file line by line just as in project 2, but this time you will add each line as an entry in your database. 1. Inside the for loop, split each line on,. 2. We only care about the first 5 columns, so use slicing to get just the first 5 entries of your list. Print the result. Observe that the indices of the resulting list go in the order of the columns of the violations table: intersection, camera id, address, date, num violations. 3. You must now construct the SQLite command to add an entry to violations. The SQLite syntax to add a line to the table looks like this: INSERT INTO violations VALUES( intersection, camera id, address, date, num violations ) You can execute this command in Python as follows: c.execute("insert INTO violations VALUES (?,?,?,?,?)", values) where values is a list containing containing the values you want to add. sqlite3 will replace each? with the corresponding element of values. What variable in your code should you use for values? Hint: you can simply use your list from step 2, since it contains the values in the correct order! 3

4 4 Querying your database One of the advantages of databases is that they allow you to retrieve data from your database in many different formats using the simple syntax SELECT columns FROM table WHERE condition This returns the selected columns from the specified table, but only the rows satisfying the specified condition. Let s practice doing this in Python! 1. Open the Python shell, connect to your database, and create a cursor. 2. Construct a query string. For example, SELECT date, num violations WHERE camera id = 2121 This query will return the date and number of violations from every violation at camera Execute the query: results = c.execute(query) If you did it correctly, Python won t print anything. 4. Get the results of the query as a Python list: results.fetchall() Here are some tips for constructing queries: You can use * to select all columns: SELECT * WHERE camera id =

5 You can use LIKE keyword to select all entries that fit a certain pattern. For example, to get all violations that happened in January: SELECT * WHERE date LIKE 01% The % is a wildcard that matches with any string. Thus, this query will return all entries in which the date starts with 01, i.e. all the violations in January. You can use aggregate functions like SUM(), AVG(), and COUNT() to perform various operations on the data. For example, to get the total number of violations that happened in January (as in project 2): SELECT SUM(num violations) WHERE date LIKE 01% Try experimenting with some queries. Get creative! 5