Introduction to Databases [p.3]

Transcription

1 Object Oriented Programming and Internet Application Development Unit 5 The Back-end in Internet Software Introduction to Databases Relational Databases Designing a Relational Database Manipulating Data in a Relational Database Database Programming in Java Programming Language 05 The Back-end in Internet Software 1 Introduction to Databases [p.3] A database refers to a collection of data structured so that the data can be manipulated with a unified approach. Refer to for details. 05 The Back-end in Internet Software 2 1

2 Problems of Storing Data in Text Files such as.csv file [p.4 to 6] Tight integration of data storage operations and business operations in source code. Lack of reusability of source code. Lack of division of labour people familiar with data manipulation operations may not be familiar with business operations. 3 Three Database Models 1. Hierarchical model Record are linked in a tree format obsolete 2. Network model Records are linked in a graph format 3. Relational model Records (or tuples) are stored in tables Most popular 4 2

3 Hierarchical Database Model [p.7] Records are connected in inverted tree-like structure. Figure 5.1 Company and director records organized in hierarchical way. Cons: some data cannot be organized in hierarchical way All records must start from a dummy root record. Obsolete due to popularity of other database models. 5 Network Database Model [p.7] Records are connected in a graph format. The same record can be referred by more than one record. Figure 5.2 Records stored in the network model for an inventory system. Product records can be referred by invoice records and supplier records. Enables different software applications to use data in their own ways. 6 3

4 Relational Database Model [p.8] Records (or tuples) are stored in a table that is similar to spreadsheet. Each record has the same set of properties. Each record can be uniquely identified by a property value that is key of the table. Figure 5.3 Data stored in the relational model for a course enrolment system Most popular database model. It has commercial and open-source implementations. 7 Relational Database Servers [p.14] Popular relational database servers Relational database servers Commercial database servers Open-source database servers Rationale Needs $ to purchase Most enterprises have high expectations on support, reliability and maintenance Closed-source software Free of charge Clears worry on Trojan horse in closed-source software Open-source Examples Oracle (Oracle, Most popular) DB2 (IBM) MS SQL Server (Microsoft) Sybase (Sybase) MySQL (most popular) Derby (pure Java, installed with NetBeans, used in this course) PostgreSQL SQLite 8 4

5 of Relational Databases Advantages [p.10 to 11] 5 advantages are listed below. For details, pls refer to Study Unit. Control of data redundancy: via database design techniques. Improved data integrity: define constraints on data to prevent from addition of invalid data. Improved security: authentication, authorization Increased concurrency Improved back-up and recovery services 9 of Relational Databases Disadvantages [p.12] Complexity of Relational Database Management System, RDMS (as it has lots of functionalities) makes people need more time to understand it thoroughly. Large size of RDBMS needs more computing power (disk space, CPU and main memory) to run it efficiently. Higher cost : Initial and ongoing software license and server hardware. Hire trained support staff for database administrations. High cost of converting legacy file-base system to databases. Poorer performance when compared legacy file-based system Greater impact on database failure. E.g. failure of bank database 10 5

6 Design a Relational Database Needs techniques such as normalization when designing a relational database. But these techniques need time to master, it is covered in database design course. Cover practical side of database design in this course via Informal approach to data modeling [p.15 to 21] 11 Data Model of a Travel Agency [p.16 to 20] Travel agency maintains 4 worksheets Routes worksheet stores all the routes. Tours worksheet stores all possible tours for a particular route that starts on a particular date. That is, each row specifies one tour that customers can join. Orders worksheet stores the details for particular orders by customers. Customers worksheet stores the list of customers for a particular order. Each worksheet is modeled by a table in relational database. 12 6

7 Database Concepts Database Schema Table Tuple (or record, row) Attribute (or field, column) Primary key Index Foreign key View Stored procedure SQL DDL DML DCL Focus of this course 13 Relationship of Database Concepts One database can have many schemas. One schema can have many tables. One table can have many tuples (or records, rows). Each tuple consists of many attributes (or fields, columns) Database 0..m 0..m 0..m 0..m Schema Table Tuple Attribute 14 7

8 Table 5.5 [p.28] Travel Agency Worksheets Modeled by Tables - I Primary Key (PK) uniquely identifies a record in Routes table. It cannot be NULL. NULL means unknown or unspecified. 15 Travel Agency Worksheets Modeled by Tables - II Table 5.5 [p.28] 16 8

9 Using Java DB (Derby Database) Refer to : (Optional) Configuring the Database (Optional) Registering the Database in NetBeans IDE Starting the Server and Creating a Database Connecting to the Database Creating Tables Adding Table Data Deleting Tables Using an External SQL Script (i.e. via a SQL file) (Optional) Recreating Tables from a Different Database 17 Start Derby Database Server Select Services tab Right-click Java DB under Databases Click Start Server How to stop a started server? Answer in this slide. 18 9

10 Create a New Database - I Select Services tab Right-click Java DB under Databases Click Create Database 19 Create a New Database - II In Create Java DB Database pop-up screen, type travelagent as Database Name dbuser as User Name dbpassword as Password and Confirm Password Stores travelagent as a folder under this Database Location. It is useful when you cannot delete this database

11 Connect to an Existing Database, travelagent Right-click jdbc:derby://localhost:1527/travelagent[dbuser on DBUSER] Click Connect. How to disconnect the database? dbuser is user name. If there is NO user name, this will be empty. DBUSER is schema. 21 Structured Query Language (SQL) [p.37] SQL is a programming language designed to manipulate data in a Relational Database Management System (RDBMS). Refer to SQL is case insensitive E.g. insert into is regarded the same as INSERT INTO Every RDBMS has a similar SQL as it needs to conform SQL standard such as SQL-92, SQL:

12 Sub-types of SQL DDL (Data Definition Language) E.g. create table, create index, alter table DML (Data Manipulation Language) E.g. Select, Insert, Update, Delete Select statement examples select * from tablea where column1 = XXX ; select tablea.columna1, tableb.columnb1 from tablea, tableb where. DCL (Data Control Language) e.g. Grant, revoke Focus of this course 23 Check Location of Java DB Installation and your Database Right-click JavaDB in Services tab page Select Properties 24 12

13 Create a Table, routes via a SQL file - I Right-click a package in NetBeans project Select SQL file under New Other Categories, click Next 25 Create a Table, routes via a SQL file - II Type create_table_routes to save a create_table_routes.sql Click Finish No need to add.sql as it will be appended to create_table_routes 26 13

14 Right-click APP Create a Table, routes via a SQL file - III Select Set as Default Schema as default schema 27 Create a Table, routes via a SQL file - IV 1. Select jdbc:derby://localhost:1527/travelagent[dbuser on DBUSER] in Connection 1. Select the 2. Type SQL, and connection in the drop-down list. 3. click Run SQL icon 2. Type SQL 3. Click Run SQL icon 28 14

15 Create a Table, routes via a SQL file - V Click + on the left-hand-side of APP, Tables, Routes Examine table structure of routes Left-hand-side icon: Red: primary key Green: index field Blue: field 29 DDL for Travel Agency - I CREATE TABLE Routes ( ) Route_number VARCHAR(10) NOT NULL PRIMARY KEY, Route_Name VARCHAR(30) NOT NULL, Route_Description LONG VARCHAR CREATE TABLE Tours ( ) Route_number VARCHAR(10) NOT NULL, Start_date DATE NOT NULL, Return_date DATE, Tea_gathering_date TIMESTAMP, PRIMARY KEY (Route_number, Start_date) Primary key has only 1 column for this DDL. Primary key has more than 1 columns for this DDL

16 DDL for Travel Agency - II Run this DDL to create table Orders under APP schema CREATE TABLE Orders ( Order_number VARCHAR(12) NOT NULL PRIMARY KEY, Order_date DATE NOT NULL, Route_number VARCHAR(10) NOT NULL, Start_date DATE NOT NULL, Number_of_person INTEGER NOT NULL, Total DECIMAL(8,2) NOT NULL, Payment_method VARCHAR(10) NOT NULL ) 31 DDL for Travel Agency - III Run this DDL to create table Customers under APP schema CREATE TABLE Customers ( Order_number VARCHAR(12) NOT NULL, Customer_name VARCHAR(30) NOT NULL, Package_type VARCHAR(10) NOT NULL, Travel_document VARCHAR(20) NOT NULL, PRIMARY KEY (Order_number,Customer_name) ) 32 16

17 Create a Table, tours via GUI - I Right-click Tables under APP schema Select Create Table Type TOURS as Table name, click Add column 33 Create a Table, tours via GUI - II Type details of column (or field), Route_number Constraints on column route_number NOT NULL 34 17

18 Create a Table, tours via GUI - III Do the same for remaining fields NOT NULL 35 Create a Table, tours via GUI - IV Click OK to create table process. NULL 36 18

19 Create a Table, tours via GUI - V Add primary key on (route_number, start_date) by following steps: Right-click Tours table, select Execute Command, type the following DDL. Alter table tours add primary key(route_number, start_date); Click Run SQL icon Right-click Tours table, select Refresh to see these 2 fields become primary key. If primary key consists of 2 or more columns, it could not be added to a table via GUI. 37 Data Type of Columns Table 5.1 Common data type categories supported by a relational database [p.23] Table 5.2 Descriptions of commonly used data types [p.24] Full list of data type, refer to 68.html 38 19

20 Constraints on Table Primary key (refer to A unique key that identifies each row in a table. Index (refer to a data structure that improves the speed of data retrieval operations on a database table at the cost of slower writes and increased storage space. Indexes can be created using one or more columns of a database table 39 Constraints on Column NULL [p.25 to 26] Denotes unknown or unspecified value Unique Every column value in the table should be unqiue, i.e. no duplicate values

21 Home Exercise on Creation of Database and Tables Create a database, travelagent in your computer with user name = dbuser, password = dbpassword Refer to previous slides on database creation Create the following tables under APP schema in database travelagent with DDL statements. (i.e Create Table statements given in previous slides) Routes, Tours, Orders, Customers Answer can be found in create_table SQL files in NetBeans project Unit5ExamplesBy [p.37 to 40] Inserting Data with DML INSERT Statement - I Meaning: add a record into a table in default schema. INSERT Syntax: (note: [ ] means optional) INSERT INTO Table_Name [(Field_List)] VALUES (Value_List) Examples: INSERT INTO Routes (Route_Number, Route_Name, Route_Description) VALUES ('HK2008', 'Hong Kong one-day tour', 'Visiting various famous sight-seeing scenes in Hong Kong in one day') -- The following insert is identical to the above one INSERT INTO Routes VALUES ('HK2008', 'Hong Kong one-day tour', 'Visiting various famous sight-seeing scenes in Hong Kong in one day') -- If a column allows NULL, can omit it in insert statement INSERT INTO Routes (Route_Number, Route_Name) VALUES ( TH2012', Thailand three-day tour') -- denotes single line comment

22 [p.37 to 40] Inserting Data with DML INSERT Statement - II If textual data specified by the SQL statement may contain an apostrophe, use any method below: -- Method 1: Use double apostrophe to escape apostrophe INSERT INTO Routes (Route_Number, Route_Name) VALUES ( HK2012', HK s two-day tour') -- Method 2: use \ to escape apostrophe INSERT INTO Routes (Route_Number, Route_Name) VALUES ( HK2012', HK\ s two-day tour') Exercise on Insert statement, Activity 5.2 [p.39] Answer can be found in NetBeans project Unit5Examples 43 [p.48] Updating Data with DML UPDATE Statement UPDATE Syntax: (Note: [ ] means optional, { } means repeating) Example UPDATE Table_Name SET Field_Name=New_Value {,Field_Name=New_Value} [WHERE Condition] UPDATE Tours SET Tea_gathering_date=' :00' WHERE Route_number= HK2008' AND Start_date=' Update record(s) in one table only If you need to update records in more than one table, you have to execute UPDATE statement for each table one by one

23 [p.48] Deleting Data with DML DELETE Statement DELETE Syntax: (Note: [ ] means optional) DELETE FROM Table_Name [WHERE Condition] Examples -- Delete record(s )with route_number = HK1997 DELETE FROM Routes WHERE Route_number='HK1997 ; Delete from one table only. -- delete all records in routes DELETE FROM routes; 45 View Data of an Existing Table, ROUTES DML SELECT Statement - I Click + on the left-hand-side of APP Click + on the left-hand-side of Tables Right-click ROUTES Select View Data Select statement of View Data.. No. of rows and no. of pages Content of table, ROUTES 46 23

24 DML SELECT Statement - II Syntax of SELECT statement [ ] denotes optional SELECT target_column_list FROM table_list [WHERE where_conditions] [GROUP BY group_by_column_list] [HAVING having_conditions] [ORDER BY order_by_column_list] List means that it can have more than 1 items. 47 Examples of SELECT Statement [p.40-44] - I -- select all fields from table routes, * denotes all fields Select * from routes; Run SQL Content of select statement 48 24

25 Examples of SELECT Statement [p.40-44] - II -- Get route name for non-null route description Select route_name from routes where route_description is not null; -- Get information for all HK routes, % denotes >= 0 characters Select * from routes where route_number like HK% ; -- Get order information in descending order of order date -- and ascending order of start date (keyword asc is optional) Select * from orders order by order_date desc, start_date asc; 49 Aggregate Functions of SELECT Statement [p.44-45] Table 5.8 A summary of aggregate functions [p.45] Aggregate functions: count, max, min, sum, avg Examples -- Get total no. of routes SELECT COUNT(*) AS Route_Total FROM Routes; -- Get no. of distinct routes that forms the tours SELECT COUNT(DISTINCT Route_Number) FROM Tours; Table 5.9 Examples of SQL SELECT queries that use aggregate functions [p.45] 50 25

26 Complex Joins of SELECT Statement [p.47-48] Join is a SELECT statement that involving more than 1 tables. -- Get route name of all routes that have orders SELECT routes.route_name FROM Routes, orders WHERE routes.route_number = orders.route_number; Sub-query means a SELECT in another one. -- identical with above statement SELECT route_name FROM Routes WHERE route_number in (select route_number from orders); 51 Database Programming in Java Programming Language ODBC (Open Database Connectivity) [p.51] Standard approach to access a database, especially a relational database. Introduced by Microsoft in 1992, but limited to Microsoft Windows only. Refer to JDBC (Java Database Connectivity) [p.51 to 52] Introduced as standard classes since Java 1.1 Refer to

27 4 Types of JDBC Driver [p.52, 80] Refer to JDBC Driver Type 1 driver (JDBC-ODBC bridge) Type 2 driver (Partial Java driver) Type 3 driver (Network-protocol driver) Type 4 driver (Native-protocol driver) Description uses a JDBC-ODBC bridge to connect to the desired DBMS. As ODBC is a proprietary to Windows, software applications that use this driver are not platform independent. Also, as there is extra overhead between the interaction of the JVM and the ODBC module, performance will be inferior to other types of JDBC drivers. calls platform-dependent binary code for database manipulations. Therefore, software applications that call this platform-dependent binary code are no longer platform independent. software applications call pure Java code to communicate with the DBMS. As pure Java code is platform independent, the software applications are platform independent. Needs to communicate to dedicated software module that manipulates the DBMS. Calls pure Java code to communicate with the DBMS. No need to communicate to dedicated software that manipulates the DBMS. 53 Connecting Database with JDBC JDBC driver [p.52] Type 1 and 2 are interim solutions, type 3 and 4 are for software deployment 2 methods to establish a database connection in Java program 1. Use DriverManager (simpler method) 2. Use DataSource (preferred method) Will discuss in Unit

28 Important JDBC jar file for JavaDB, Derby derbyclient.jar Ensure the DerbyClient.jar is in the libraries of your project. If not, do the following: Right-click Libraries Select Add Jar/Folder See slide #24 Select derbyclient.jar in <JavaDB installation folder>\lib 55 [p.53] Steps of Using DriverManager to Establish a Connection - I 1. Load a JDBC driver into the JVM. E.g. derby Class.forName("org.apache.derby.jdbc.ClientDriver"); 2. Call DriverManager.getConnection(url ) to get a connection object. // If Derby database, e.g. sample has no user name and password // Note: If the database is at another IP, just use the IP to replace localhost Connection conn = DriverManager.getConnection( " jdbc:derby://localhost:1527/sample"); In DerbyClient.jar // If Derby database, e.g. travelagent has user name and password Connection conn = DriverManager.getConnection( jdbc:derby://localhost:1527/travelagent, dbuser, dbpassword ); 56 28

29 [p.54] Steps of Using DriverManager to Establish a Connection - II 3. Get data from database. Read content in next slide 4. Release resources held by Connection object. conn.close(); 57 [p.56] Steps of Getting Data from Database (i.e. for SELECT) 1. Create a statement object Statement stmt = conn.createstatement(); 2. Execute Select statement with executequery(). ResultSet rs = stmt.executequery( select * from routes ); 3. Use rs.next() and rs.getobject( ) to get rows in result set. 4. Release resources held by ResultSet object and Statement object. rs.close(); stmt.close(); 58 29

30 Examples of Getting Data from Database Demonstrate Fig 5.18 DbmsDataGetter.java [p.58] with the following argument. org.apache.derby.jdbc.clientdriver jdbc:derby://localhost:1527/travelagent dbuser dbpassword app.routes Route_Number Route_Name Route_Description Note: routes is in APP schema. Needs to add schema name if it is different from user name of database Self-test 5.5 [p.60, 80-81] DisplayRoutes.java with try/catch, more robust than DbmsDataGetter.java. // Get no. of columns in table routes ResultSetMetaData metadata = resultset.getmetadata(); int numberofcolumns = metadata.getcolumncount(); 59 [p.60] Steps of Manipulating Data in a Database (i.e. for DDL or DML) 1. Create a prepared statement object. PreparedStatement is subclass of Statement. PreparedStatement stmt = conn.preparestatement( "DELETE FROM Routes WHERE Route_number=?"); /* use setxxx() method to update different placeholders according to XXX data type */ stmt.setstring(1, routenumber); // replace 1 st? (i.e. placeholder) by routenumber 2. Execute insert, update, delete with executeupdate(). stmt.executeupdate(); 3. Release resources held by PreparedStatement object. stmt.close(); 60 30

31 Why preparestatement is preferred? On submitting DML (select, insert, update, delete) statements, conn.preparestatement rather than createstatement should be used because: Better performance: preparestatement pre-compiles the query into execution plan, but createstatement does not. So preparestatement performs better than createstatement. For details, pls read this URL. Better security: automatic prevention of SQL injection attacks by built-in escaping the quotes and other special characters. For details, pls read this URL. 61 Home Exercise Familiarize with PreparedStatement, Statement, ResultSet Go to Read methods of PreparedStatement, Statement and ResultSet. For class ResultSet, read all getxxx() methods for different data types. Data type boolean int double Date XXX getxxx( ) getboolean(int columnindex), getboolean(string columnlabel) getint(int columnindex), getint(string columnlabel) getdouble(int columnindex), getdouble(string columnlabel) getdate(int columnindex), getdate(string columnlabel) 62 31

32 Case Study POJO - I [p.63 to 64] POJO (Plain Old Java Object) Refer to POJO is a Java object which does not follow any of the major Java object models, conventions, or frameworks. In this course, each POJO object can be used to store a record in a table. Strictly speaking, this kind of POJO is a Java Bean (refer to 63 Case Study POJO II [p.63 to 64] POJO example on routes table POJO Route.java in MT801_Unit5_Ans NetBeans project. Setter and getter methods defined on the above columns

33 Case Study DAO Concept [p.64-65] DAO (Data Access object) For SELECT : DAO populates values from database to POJO object. For INSERT, UPDATE, DELETE : DAO uses POJO object to update database. Refer to DAO usage diagram Database DAO for SELECT POJO object DAO for INSERT, UPDATE, DELETE 65 Case Study DAO Example [p.64-67] Table 5.11 Behaviours of the RouteDAO object 66 33

34 RouteDAO.java in MT801_Unit5_Ans RouteDAO.java in MT801_Unit5_Ans defines following methods void insert(route route) void update(route route) void delete(route route) List<Route> selectall() Return a list of route records. Route selectbyroutenumber(string routenumber) Return a single route record 67 Appendix Further Exploration [p.75] Optional (out of scope for this course) Data warehouses A large repository of data Data mining or data analytics Analytic skills to find pattern stored in data warehouse Business intelligence Use data mining technique to find useful information stored in data warehouse to assist management to make decision 68 34

35 Java DB (Derby) Reference Refer to Select a version of Derby (e.g. 10.8) first in the above URL ( Getting started with Derby Derby Reference Manual (i.e. SQL, JDBC) Derby Developer s Guide (i.e. JDBC Tuning Derby