Databases and SQL programming overview

Transcription

1 Databases and SQL programming overview

2 Databases: Digital collections of data A database system has: Data + supporting data structures The management system (DBMS) Popular DBMS Commercial: Oracle, IBM, Microsoft GUI: Microsoft Access, OpenOffice Base Terminal: MySQL, Postgres, SQLite

3 Databases: Digital collections of data Why use a database system? Large amounts of diverse data Example: sequence identifiers and expression data

4 Databases: Digital collections of data

5 Databases: Digital collections of data Why use a database system? Large amounts of diverse data Example: sequence identifiers and expression data Many concurrent end-users or collaborators

6 Databases: Digital collections of data

7 Databases: Digital collections of data

8 Biological databases Complex data Relationships (1-1, 1-many, many-many) End users

9 Biological databases

10 SQL (Structured Query Language) Language for building, accessing, and manipulating a relational database management systems (RDBMS) Database Schema: tables, relationships, permissions The data Access the data with SQL queries

11 PostgreSQL Install postgres (version 8.4 is the current one in Ubuntu) $ psql --help Log into your psql terminal $ psql -h localhost -U postgres -d my_database Get help with SQL commands my_database=>\h Get help with psql commands my_database=>\?

12 SQL server basics Keywords are not case sensitive. Table and column names are stored as they are entered (use lower case as naming convention). SQL statements terminated with a semicolon. Elements are comma separated. Comments are enclosed between /* and */ or preceded by --.

13 SQL (Structured Query Language) Language for building, accessing, and manipulating a relational database management systems (RDBMS) Database Create a new database from your psql terminal Create or drop database from your linux terminal (createdb and dropdb commands) * Mind owner of the database, and permissions!

14 SQL (Structured Query Language) Language for building, accessing, and manipulating a relational database management systems (RDBMS) Database Schema: tables, relationships, permissions (DDL) The data Access the data with SQL queries

15 SQL (Structured Query Language) Data definition (DDL) Data manipulation SELECT statements Joins Row functions Aggregate functions Subqueries Views * PostgreSQL, and database connection with Perl

16 Data definition Data types

17 Data definition Create table Defines the structure of the table: CREATE TABLE sample ( sample_id serial NOT NULL PRIMARY KEY, sample_name character varying NOT NULL );

18 Data definition Create table Defines the structure of the table: Table name CREATE TABLE sample ( sample_id serial NOT NULL PRIMARY KEY, sample_name character varying NOT NULL ); SQL command Column name Data type Constraint(s)

19 Data definition Constraints are used to enforce valid data in columns modifies the structure of the table: NOT NULL CHECK PRIMARY KEY FOREIGN KEY

20 Data definition Create table CREATE TABLE sample ( sample_id serial NOT NULL PRIMARY KEY, sample_name character varying NOT NULL, species character varying ); Or: CREATE TABLE sample ( sample_id serial NOT NULL PRIMARY KEY, sample_name character varying NOT NULL ); ALTER TABLE sample ADD COLUMN species character varying ;

21 Data definition Alter table modifies the structure of the table: ALTER TABLE sample ALTER COLUMN species SET NOT NULL; ALTER TABLE sample DROP COLUMN species; ALTER TABLE sample ADD COLUMN species text DEFAULT NULL;

22 Data definition Drop tables DROP TABLE sample; Oops!

23 Data definition Drop tables DROP TABLE sample; Oops! ALWAYS USE TRANSACTIONS!! => BEGIN; => SQL statement 1; SQL statement 2... ; -- I made some mistake... => ROLLBACK;

24 Data definition ALWAYS USE TRANSACTIONS!! => BEGIN; => SQL statement 1; SQL statement 2... ; -- Looks good! => COMMIT;

25 Data definition Foreign keys CREATE TABLE species ( species_id serial PRIMARY KEY, species_name character varying NOT NULL ); CREATE TABLE sample ( sample_id serial NOT NULL PRIMARY KEY, sample_name character varying NOT NULL, species_id integer REFERENCES species(species_id) );

26 Foreign key constraint species animal species_id (PK) animal_id (PK) species_name animal_name species_id (FK)

27 SQL (Structured Query Language) Language for building, accessing, and manipulating a relational database management systems (RDBMS) Database Schema: tables, relationships, permissions (DDL) The data (INSERT, UPDATE, DELETE) Access the data with SQL queries

28 Data manipulation Insert add new rows to your table INSERT INTO species (species_name) VALUES ('Solanum lycopersicum'); Update modify column value/s of existing rows UPDATE species SET species_name = 'Solanum tuberosum' WHERE species_id = 1; Delete -remove rows from table DELETE FROM species WHERE species_id = 1;

29 Data manipulation Transactions BEGIN; UPDATE...; DELETE...; INSERT...; COMMIT; or ROLLBACK;

30 Data manipulation - transactions

31 Use transactions, sanitize your input

32 Data manipulation copy command Large dataset? Write SQL file with INSERT commands Use COPY with a delimited text file

33 Data manipulation copy command

34 Data manipulation copy command BEGIN; INSERT INTO species (species_name) VALUES ('Solanum melongena'); INSERT INTO species (species_name) VALUES ('Solanum tuberosum'); INSERT INTO species (species_name) VALUES ('Capsicum annuum');.. COMMIT; => \copy species (species_name) FROM 'species_list.txt'

35 SQL (Structured Query Language) Language for building, accessing, and manipulating a relational database management systems (RDBMS) Database Schema: tables, relationships, permissions (DDL) The data (INSERT, UPDATE, DELETE) Access the data with SQL queries (SELECT)

36 SQL (Structured Query Language) Data definition Data manipulation (DML) SELECT statements Joins Row functions Aggregate functions Subqueries Views * PostgreSQL, and database connection with Perl

37 SELECT statements

38 SELECT statements Select everything: SELECT * FROM sample; Select with a condition, sort results: SELECT sample_name FROM sample WHERE species_id=1 ORDER BY sample_name ASC; Count rows, group by column, with a condition: SELECT count(sample_id), species.species_name FROM sample JOIN species USING (species_id) GROUP BY species_name HAVING species_name like 'Solanum%';

39 SELECT statements Conditional operators: SELECT... FORM... WHERE...

40 SQL (Structured Query Language) Data definition Data manipulation (DML) SELECT statements Joins Row functions Aggregate functions Subqueries Views * PostgreSQL, and database connection with Perl

41 Joins

42 Joins Inner joins are default, so the word 'inner' can be omitted Natural joins (when foreign key column has the same name as the referenced column) SELECT * FROM sample JOIN species USING (species_id);

43 Joins

44 Joins

45 Row functions

46 Row functions Math functions String functions

47 Row functions Date and time Data type conversion # SELECT now(); --this is a timestamp! # SELECT cast (now() AS text) ; --the output is now a 'text' data type

48 PostgreSQL the basics $ psql -h hostname -U username -d dbname Users: postgres, your_user_name, other_user *user permissions postgres is the database superuser Grant permissions to other users as required

49 Resources In your psql terminal: =>\h [SQL command name] =>\?

50 SQL advanced querying Data definition Data manipulation (DML) SELECT statements Joins Row functions Aggregate functions Subqueries Views * PostgreSQL, and database connection with Perl

51 Row functions - Case SELECT * FROM test; The SQL CASE expression is a generic conditional expression, similar to if/else statements in other languages: CASE WHEN condition THEN result [WHEN...] [ELSE result] END a SELECT a, CASE WHEN a=1 THEN 'one' WHEN a=2 THEN 'two' ELSE 'other' END FROM test; a case one 2 two 3 other

52 Row functions more conditional expressions The COALESCE function returns the first of its arguments that is not null. Null is returned only if all arguments are null. It is often used to substitute a default value for null values when data is retrieved for display, for example: SELECT COALESCE(description, short_description, '(none)')... The NULLIF function returns a null value if value1 and value2 are equal; otherwise it returns value1. This can be used to perform the inverse operation of the COALESCE example given above: SELECT NULLIF(value, '(none)')... If value1 is (none), return a null, otherwise return value1.

53 Aggregate functions

54 Aggregate functions - grouping GROUP BY will condense into a single row all selected rows that share the same values for the grouped expressions

55 Aggregate functions - grouping HAVING clause use with aggregate functions instead of 'WHERE'

56 Subqueries

57 Subqueries

58 Views A view is a virtual table based on the results of an SQL statement CREATE VIEW myview AS SELECT city, temp_lo, temp_hi, prcp, date, location FROM weather, cities WHERE city = name;

59 postgresql Programming with a procedural language Write your own postgresql functions! PL/pgSQL (similar to Oracle's PL/SQL ) Many other languages: PL/Perl PL/Java, plphp, PL/Python, PL/R and more...