SQLite Perl tutorial. Perl DBI. Table of Contents

Transcription

1 This is a Perl programming tutorial for the SQLite database. It covers the basics of SQLite programming with the Perl language. Table of Contents SQLite Perl tutorial...6 Perl DBI...1 Related tutorials...2 Perl DBI...3 Prerequisites...3 SQLite database...3 Perl DBI...3 Common DBI methods...4 Conventions...5 Connecting to the SQLite database with Perl...6 Version...6 Inserting data...8 The last inserted row id...9 Fetching data...10 SQLite Perl error handling...12 Default error handling...12 Raising exceptions...13 Error subroutines...14 The kosher example...15 SQLite queries with Perl...18 The fetch methods...18 Dumping data...21 The convenience methods...22 Parameterized queries...24 Binding SQLite parameters & columns with Perl...26 Quoting parameters...27 Binding columns...28 Working with images in SQLite with Perl...30 Inserting images...30 Reading images...31 Getting SQLite metadata with Perl...32 SQLite transactions with Perl...36 Definitions...36 Examples...36 Perl DBI The Perl DBI (Database Interface) is a database access module for the Perl programming language. It defines a set of methods, variables and conventions that provide a standard database interface. Other languages have created similar universal interfaces for working with databases. Java has JDBC, PHP PDO. 1

2 Related tutorials SQLite Perl tutorial The SQLite tutorial covers the SQLite embedded database engine. The SQLite Ruby tutorial is a SQLite programming tutorial for the Ruby language. The SQLite C# tutorial is a SQLite programming tutorial for the C# language. 2

3 Perl DBI SQLite Perl tutorial In the first chapter of the SQLite Perl tutorial, we will introduce the Perl DBI module and the SQLite database. We will provide some definitions and show how to install the necessary elements. Prerequisites To work with this tutorial, we must have Perl language, SQLite database, sqlite3 command line tool, Perl DBI and DBD::SQLite modules installed. The DBI is the standard Perl database interface. Each database has its driver. In our case, DBD::SQLite is the driver for the SQLite database. $ sudo perl -MCPAN -e shell cpan> install DBI cpan[2]> install DBD::SQLite The above commands show, how to install Perl DBI and DBD::SQLite modules. SQLite database SQLite is an embedded relational database engine. It is a self-contained, serverless, zero-configuration and transactional SQL database engine. SQLite implements most of the SQL-92 standard for SQL. The SQLite engine is not a standalone process. Instead, it is statically or dynamically linked into the application. An SQLite database is a single ordinary disk file that can be located anywhere in the directory hierarchy. The SQLite comes with the sqlite3 command line utility. It can be used to issue SQL commands against a database. Now we are going to use the sqlite3 command line tool to create a new database. $ sqlite3 test.db SQLite version Enter ".help" for instructions Enter SQL statements terminated with a ";" We provide a parameter to the sqlite3 tool. The test.db is a database name. It is a single file on our disk. If it is present, it is opened. If not, it is created. sqlite>.tables sqlite>.exit $ ls test.db The.tables command gives a list of tables in the test.db database. There are currently no tables. The.exit command terminates the interactive session of the sqlite3 command line tool. The ls Unix command shows the contents of the current working directory. We can see the test.db file. All data will be stored in this single file. Perl DBI The Perl DBI (Database Interface) is a database access module for the Perl programming language. It defines a set of methods, variables and conventions that provide a standard database interface. The DBI is 3

4 also responsible for the dynamic loading of drivers, error checking and handling, providing default implementations for methods, and many other non-database specific duties. The DBI dispatches method calls to the appropriate database driver. The DBD (Database Driver) is a Perl module which translates the DBI methods for a specific database engine. The database drivers are supplied by database vendors. = DBI->available_drivers(); print "\n"; The code example lists all available drivers on our system. We import the DBI module for our script. = DBI->available_drivers(); The available_drivers() class method gets all the current available drivers on our system. print "\n"; This line prints the drivers to the console, each on a separate line. $./available_drivers.pl DBM ExampleP File Gofer Proxy SQLite Sponge mysql Example output. Common DBI methods The following table lists some common DBI methods. Method name Description available_drivers() Returns a list of all available drivers connect() Establishes a connection to the requested data source disconnect() Disconnects from the database server prepare() Prepares an SQL statement for execution execute() Executes the prepared statement do() Prepares and executes an SQL statement bind_param() Associates a value with a placeholder in a prepared statement bind_col() Binds a Perl variable to an output field of a SELECT statement 4

5 begin_work() Starts a new transaction commit() Writes the most recent series of uncommitted database changes to the database rollback() Undoes the most recent series of uncommitted database changes quote() Quotes a string literal for use as a literal value in an SQL statement dump_results() Fetches all the rows and prints them fetchrow_array() Fetches the next row as an array of fields fetchrow_arrayref() Fetches the next row as a reference array of fields fetchrow_hashref() Fetches the next row as a reference to a hashtable fetchall_arrayref() Fetches all data as an array of arrays finish() Finishes a statement and lets the system free resources rows() Returns the number of rows affected column_info() Provides information about columns table_info() Provides information about tables primary_key_info() Provides information about primary keys in tables foreign_key_info() Provides information about foreign keys in tables Conventions Perl programmers usually use the following variable names when working with Perl DBI. In this tutorial we will adhere to these conventions too. Variable name Description $dbh Database handle object $sth Statement handle object $drh Driver handle object (rarely seen or used in applications) $h Any of the handle types above ($dbh, $sth, or $drh) $rc General Return Code (boolean: true=ok, false=error) $rv General Return Value (typically an List of values returned from the database, typically a row of data $rows Number of rows processed (if available, else -1) $fh A filehandle undef NULL values are represented by undefined values in Perl \%attr Reference to a hash of attribute values passed to methods This chapter of the SQLite Perl tutorial was an introduction to the Perl DBI module and the SQLite database. 5

6 Connecting to the SQLite database with Perl This part of the SQLite Perl tutorial will show, how to create a database connection to the database. The first step is to connect to the database. We use the connect() DBI method to establish a connection. The disconnect() method is used to close the database connection. $dbh = DBI->connect($dsn, $username, $password) or die $DBI::errstr; $dbh = DBI->connect($dsn, $username, $password, \%attr) or die $DBI::errstr; The connect() method establishes a database connection to the requested data source. It returns a database handle object if the connection succeeds. We use the disconnect() method to terminate the connection. The $dsn is the data source name. It is a string that tells the Perl DBI module, what kind of driver it should load and the location of the database to which the connection is going to be created. dbi:drivername:database_name dbi:drivername:database_name@hostname:port dbi:drivername:database=database_name;host=hostname;port=port The above strings are examples of data source names in Perl DBI. dbi:sqlite:dbname=test.db We are going to use this data source name. The dsn starts always with the dbi: substring. Then we have the driver name. In our case the driver name is SQLite. The third part is the database name. We will work with test.db throughout this tutorial. dbi:sqlite:dbname=:memory: We can also create a database in the memory with the above data source name. We do not give $username and $password for the SQLite database. The database does not support it. We left two empty strings there. The final parameter is a reference to hash, in which we can set attributes to alter the default settings of a connection. For example the RaiseError attribute can be used to force errors to raise exceptions rather than return error codes. The HandleError attribute can be used to provide a subroutine which is called in case of error. The AutoCommit attribute sets or unsets the autocommit mode. The $DBI::errstr is a DBI dynamic attribute which returns the native database engine error message. In case the connection fails, this message is displayed and the script is aborted. Version In the first code example, we will get the version of the SQLite database. 6

7 { RaiseError => 1, SQLite Perl tutorial my $sth = $dbh->prepare("select SQLITE_VERSION()"); my $ver = $sth->fetch(); print "\n"; $sth->finish(); In the above Perl script we connect to the previously created test.db database. We execute an SQL statement which returns the version of the SQLite database. We use the Perl DBI module to connect to the SQLite database. { RaiseError => 1, Here we connect to the test.db database. The first parameter is the data source name. In the string we specify the database driver and the database name. The second and third parameters are empty. In other cases we provide the user name and the password there. The last parameter is the database options. We set the RaiseError option to 1. This will cause exceptions to be raised instead of returning error codes. my $sth = $dbh->prepare("select SQLITE_VERSION()"); The prepare() method prepares an SQL statement for later execution. The execute() method executes the SQL statement. my $ver = $sth->fetch(); We fetch the data. print "\n"; We print the data that we have retrieved to the console. $sth->finish(); Here we indicate that no more data will be fetched from this statement handle. 7

8 We close the connection to the database. $./version.pl SQLite Perl tutorial Executing the verion.pl script we get the version of the SQLite database. Inserting data We will create a Cars table and insert several rows to it. { RaiseError => 1 $dbh->do("drop TABLE IF EXISTS Cars"); $dbh->do("create TABLE Cars(Id INT PRIMARY KEY, Name TEXT, Price INT)"); $dbh->do("insert INTO Cars VALUES(1,'Audi',52642)"); $dbh->do("insert INTO Cars VALUES(2,'Mercedes',57127)"); $dbh->do("insert INTO Cars VALUES(3,'Skoda',9000)"); $dbh->do("insert INTO Cars VALUES(4,'Volvo',29000)"); $dbh->do("insert INTO Cars VALUES(5,'Bentley',350000)"); $dbh->do("insert INTO Cars VALUES(6,'Citroen',21000)"); $dbh->do("insert INTO Cars VALUES(7,'Hummer',41400)"); $dbh->do("insert INTO Cars VALUES(8,'Volkswagen',21600)"); The above script creates a Cars table and inserts 8 rows into the table. $dbh->do("create TABLE Cars(Id INT PRIMARY KEY, Name TEXT, Price INT)"); The do() method executes the SQL statements. It combines two method calls, prepare() and execute() into one single call. The do() method is used for non-select statements. $dbh->do("insert INTO Cars VALUES(1,'Audi',52642)"); $dbh->do("insert INTO Cars VALUES(2,'Mercedes',57127)"); These two lines insert two cars into the table. Note that by default, we are in the autocommit mode, where all changes to the table are immediately effective. sqlite>.mode column sqlite>.headers on We verify the written data with the sqlite3 tool. First we modify the way the data is displayed in the console. We use the column mode and turn on the headers. sqlite> SELECT * FROM Cars; Id Name Price 8

9 Audi Mercedes Skoda Volvo Bentley Citroen Hummer Volkswagen SQLite Perl tutorial This is the data that we have written to the Cars table. The last inserted row id Sometimes, we need to determine the id of the last inserted row. In Perl DBI, we use the last_insert_id() method to find it. "dbi:sqlite:dbname=:memory:", { RaiseError => 1, $dbh->do("create TABLE Friends(Id INTEGER PRIMARY KEY, Name TEXT)"); $dbh->do("insert INTO Friends(Name) VALUES ('Tom')"); $dbh->do("insert INTO Friends(Name) VALUES ('Rebecca')"); $dbh->do("insert INTO Friends(Name) VALUES ('Jim')"); $dbh->do("insert INTO Friends(Name) VALUES ('Robert')"); $dbh->do("insert INTO Friends(Name) VALUES ('Julian')"); my $id = $dbh->last_insert_id( "Friends", ""); print "The last Id of the inserted row is $id\n"; We create a Friends table in memory. The Id is automatically incremented. $dbh->do("create TABLE Friends(Id INTEGER PRIMARY KEY, Name TEXT)"); In SQLite, INTEGER PRIMARY KEY column is auto incremented. There is also an AUTOINCREMENT keyword. When used in INTEGER PRIMARY KEY AUTOINCREMENT a slightly different algorithm for Id creation is used. $dbh->do("insert INTO Friends(Name) VALUES ('Tom')"); $dbh->do("insert INTO Friends(Name) VALUES ('Rebecca')"); $dbh->do("insert INTO Friends(Name) VALUES ('Jim')"); $dbh->do("insert INTO Friends(Name) VALUES ('Robert')"); $dbh->do("insert INTO Friends(Name) VALUES ('Julian')"); 9

10 These five SQL statements insert five rows into the Friends table. my $id = $dbh->last_insert_id( "Friends", ""); Using the last_insert_id() method, we get the last inserted row id. $./lastrowid.pl The last Id of the inserted row is 5 We see the output of the script. Fetching data In the last example of this chapter we fetch some data. More about data fetching will be discussed in the Queries chapter. { RaiseError => 1 my $sth = $dbh->prepare( "SELECT * FROM Cars WHERE Id=1" ); my ($id, $name, $price) = $sth->fetchrow(); print "$id $name $price\n"; my $fields = $sth->{num_of_fields; print "We have selected $fields field(s)\n"; my $rows = $sth->rows(); print "We have selected $rows row(s)\n"; $sth->finish(); In the example we fetch a row from the Cars table. We will also find out how many fields & rows we have selected. my $sth = $dbh->prepare( "SELECT * FROM Cars WHERE Id=1" ); We prepare an SQL statement with the prepare() method. The SQL string is sent to the SQLite database engine for processing. Its syntax and validity is checked. The method returns a statement handle. Then the SQL statement is executed. The data is prepared to be sent to the client program. my ($id, $name, $price) = $sth->fetchrow(); print "$id $name $price\n"; The data is retrieved from the database with the fetchrow() method. The method returns one row from the table in form of a Perl list. 10

11 my $fields = $sth->{num_of_fields; SQLite Perl tutorial The NUM_OF_FIELDS is a statement handle attribute which gives us the number of returned fields. In our case we have three fields returned: Id, Name, and Price. my $rows = $sth->rows(); We get the number of selected rows. We have retrieved only one row from the table. The rows() method returns the number of affected rows. It can be used for SELECT, UPDATE, and DELETE SQL statements. $./fetchrow.pl 1 Audi We have selected 3 field(s) We have selected 1 row(s) Output of the fetchrow.pl script. In this chapter of the SQLite Perl tutorial, we have shown how to establish a database connection to the SQLite database. We have explained scripts which do some basic work with a database. 11

12 SQLite Perl error handling In this chapter we will show, how we can handle errors. Method name Description $h->err() Returns the native database engine error code from the last driver method called. $h->errstr() Returns the native database engine error message from the last DBI method called. $h->state() Returns a state code in the standard SQLSTATE five character format The above three methods deal with error messages. DBI dynamic attribute Description $DBI::err Equivalent to $h->err() $DBI::errstr Equivalent to $h->errstr() $DBI::state Equivalent to $h->state() The second table gives a list of DBI dynamic attributes, which are related to error handling. These attributes have a short lifespan. They should be used immediately after the method that might cause an error. Default error handling By default, the errors are returned by Perl DBI methods. my $dsn = "dbi:sqlite:dbname=test.db"; my $user = ''; my $password = ''; $dsn, $user, $password) or die "Can't connect to database: $DBI::errstr"; my $sth = $dbh->prepare( q{ SELECT Id, Name, Price FROM Cars ) or die "Can't prepare statement: $DBI::errstr"; my $rc = $sth->execute() or die "Can't execute statement: $DBI::errstr"; while (my($id, $name, $price) = $sth->fetchrow()) { print "$id $name $price\n"; # check for problems which may have terminated the fetch early warn $DBI::errstr if $DBI::err; $sth->finish(); 12

13 In the first script we deal with the default behaviour of returning error codes. $dsn, $user, $password) or die "Can't connect to database: $DBI::errstr"; We call the connect() method to create a database connection. If the attempt fails, the method returns undef and sets both $DBI::err and $DBI::errstr attributes. The die() method prints the error message in case of a failure and terminates the script. my $sth = $dbh->prepare( q{ SELECT Id, Name, Price FROM Cars ) or die "Can't prepare statement: $DBI::errstr"; We call the prepare() statement. If the method fails, the die() method prints an error message and terminates the script. my $rc = $sth->execute() or die "Can't execute statement: $DBI::errstr"; Again. We call the execute() method and check for errors. The method returns undef if it fails. warn $DBI::errstr if $DBI::err; We check for problems which may have terminated the fetch method early. Raising exceptions Checking for errors each time we call a DBI method may be tedious. We could easily forget to do so if we have a larger script. The preferred way of dealing with possible errors is to raise exceptions. To raise exceptions, we set the RaiseError attribute to true. my $dsn = "dbi:sqlite:dbname=test.db"; my $user = ''; my $password = ''; my %attr = ( RaiseError => 1 ); $dsn, $user, $password, \%attr) or die "Can't connect to database: $DBI::errstr"; my $sth = $dbh->prepare("select * FROM Cars LIMIT 5"); while (my($id, $name, $price) = $sth->fetchrow()) { print "$id $name $price\n"; $sth->finish(); In the connection attributes, we set the RaiseError attribute to 1. When an error occurs, the exceptions 13

14 are raised rather than error codes returned. my %attr = ( RaiseError => 1 ); We set the RaiseError attribute to 1. SQLite Perl tutorial $dsn, $user, $password, \%attr) or die "Can't connect to database: $DBI::errstr"; The connect() method is the only method that we check for the return code. my $sth = $dbh->prepare("select * FROM Cars LIMIT 5"); The prepare() and execute() methods do not check for the return error codes. If they fail, an exception is thrown and the Perl DBI will call the die() method and print the error message. Error subroutines With the HandleError connection handle attribute, we can set a reference to a subroutine, which is called when an error is detected. The subroutine is called with three parameters: the error message string that RaiseError and "PrintError" would use, the DBI handle being used, and the first value being returned by the method that failed (typically undef). If the subroutine returns a false value then the RaiseError or PrintError attributes are checked and acted upon as normal. my $dsn = "dbi:sqlite:dbname=test.db"; my $user = ''; my $password = ''; my %attr = ( RaiseError => 1, AutoCommit => 0, HandleError => \&handle_error ); $dsn, $user, $password, \%attr) or die "Can't connect to database: $DBI::errstr"; $dbh->do("update Cars SET Price=52000 WHERE Id=1"); $dbh->do("update Car SET Price=22000 WHERE Id=8"); $dbh->commit(); sub handle_error { $dbh->rollback(); my $error = shift; print "An error occurred in the script\n"; print "Message: $error\n"; return 1; 14

15 Our own subroutine will handle the error. SQLite Perl tutorial my %attr = ( RaiseError => 1, AutoCommit => 0, HandleError => \&handle_error ); The HandleError attribute provides a reference to a handle_error() subroutine that is called, when an error is detected. The AutoCommit is turned off, which means that we work with transactions. $dbh->do("update Car SET Price=22000 WHERE Id=8"); There is an error in the SQL statement. There is no Car table. sub handle_error { $dbh->rollback(); my $error = shift; print "An error occurred in the script\n"; print "Message: $error\n"; return 1; This is the handle_error() subroutine. We rollback the changes. We print the error message and return 1. If we returned 0 instead, additional error messages would appear. Returning 1 error messages associated with RaiseError attribute are supressed. $./errsub.pl An error occurred in the script Message: DBD::SQLite::db do failed: no such table: Car Output of the example. The kosher example According to the Perl DBI documentation, the most robust way to deal with DBI errors is to use the eval() method. use DBI qw(:sql_types); my $dsn = "dbi:sqlite:dbname=test.db"; my $user = ''; my $password = ''; my %attr = ( RaiseError => 1, AutoCommit => 0 ); $dsn, $user, $password, \%attr) or die "Can't connect to database: $DBI::errstr"; = ( [ 1, "Audi", 52642], [ 2, "Mercedes", 57127], [ 3, "Skoda", 9000], [ 4, "Volvo", 29000], [ 5, "Bentley", ], 15

16 ); [ 6, "Citroen", 21000], [ 7, "Hummer", 41400], [ 8, "Volkswagen", 21601] eval { $dbh->do("drop TABLE IF EXISTS Cars"); $dbh->do("create TABLE Cars(Id INTEGER PRIMARY KEY, Name TEXT, Price INT)"); ; my $sql = qq{ INSERT INTO Cars VALUES (?,?,? ) ; my $sth = $dbh->prepare( $sql ); foreach my $row (@data) { eval { $sth->bind_param( SQL_INTEGER ); $sth->bind_param( SQL_VARCHAR ); $sth->bind_param( SQL_INTEGER ); $dbh->commit(); ; if( $@ ) { warn "Database error: $DBI::errstr\n"; $dbh->rollback(); $sth->finish(); The above code example is the most correct way of dealing with errors. my %attr = ( RaiseError => 1, AutoCommit => 0 ); We raise exceptions rather than check for return codes. We turn the autocommit mode off and manually commit or rollback the changes. my $sql = qq{ INSERT INTO Cars VALUES (?,?,? ) ; my $sth = $dbh->prepare( $sql ); We guard against errors and security issues by using the placeholders. eval { $sth->bind_param( SQL_INTEGER ); $sth->bind_param( SQL_VARCHAR ); $sth->bind_param( SQL_INTEGER ); $dbh->commit(); ; Inside the eval() method we put the error prone code. The method traps exceptions and fills the $@ special variable with error messages. We bind the variables to the placeholders with the bind_param() method. 16

17 if( ) { warn "Database error: $DBI::errstr\n"; $dbh->rollback(); SQLite Perl tutorial In case of an error, we print the error message and rollback the changes. In this part of the SQLite Perl tutorial, we were discussing error handling. 17

18 SQLite queries with Perl SQLite Perl tutorial We have already established a connection to the database. Now we are going modify and fetch the data from the database. Data is retrieved from the database with the SELECT statement. In Perl DBI, first we prepare the SQL statement with the prepare() method. The SQL string is sent to the database engine, which checks the statement validity, syntax and in some databases also the user permissions to perform certain queries. If all is OK, a reference to the statement handle is returned to the Perl script. The next step is the call to the execute() method. The method executes the query within the database. At this moment the result stays in the database. The Perl script does not contain the data yet. For non-select statements, the execute() method returns the number of rows affected if known. In the last step the data is fetched from the database. The data is pulled row by row and populated into the Perl data structures. The Perl DBI has several methods to fetch data from database tables. Method Description fetchrow_arrayref() Fetches the next row of data and returns a reference to an array. fetchrow_array() Fetches the next row of data and returns it as a list. fetchrow_hashref() Fetches the next row of data and returns it as a reference to a hash. Fetches all data & returns a reference to an array that has one reference per fetchall_arrayref() row. fetch() The method is an alias for fetchrow_arrayref(). fetchrow() The method is an alias for fetchrow_array(). After the SQL statement was prepared and executed, we call one of the available fetch methods. Method Description selectrow_arrayref( Combines prepare(), execute() and fetchrow_arrayref() into a single call ) selectrow_hashref() Combines prepare(), execute() and fetchrow_hashref() into a single call select_row_array() Combines prepare(), execute() and fetchrow_array() into a single call. selectall_arrayref( Combines prepare(), execute() and fetchall_arrayref() into a single call. ) selectall_hashref() Combines prepare(), execute() and fetchall_hashref() into a single call. selectcol_arrayref( Combines prepare(), execute() and fetching one col from all rows into a ) single call. In the second table we have a list of utility methods which combine three methods into one call. They are convenience methods. The fetch methods In the first example we will demonstrate the usage of the fetchrow_arrayref() method. 18

19 { RaiseError => 1, my $sth = $dbh->prepare("select * FROM Cars LIMIT 5"); my $row; while ($row = $sth->fetchrow_arrayref()) { $sth->finish(); In the example we select 5 rows from the Cars table. The data is retrieved with the fetchrow_arrayref() method. my $sth = $dbh->prepare("select * FROM Cars LIMIT 5"); These are the first two phases of the data retrieval process. We prepare and execute the SELECT statement. my $row; while ($row = $sth->fetchrow_arrayref()) { Now we are fetching the data. The fetchrow_arrayref() method fetches the next row of data and returns a reference to an array holding the field values. We put the method in the while loop which terminates, when there are no more rows left. $./fetchrow_arrayref.pl 1 Audi Mercedes Skoda Volvo Bentley Example output. In the second example, we will use the fetchrow_array() method. 19

20 { RaiseError => 1, my $sth = $dbh->prepare( "SELECT * FROM Cars LIMIT 5" ); while (@row = $sth->fetchrow_array()) { print "@row\n"; $sth->finish(); In this script we connect to the database and fetch 5 rows of the Cars table one by one using the fetchrow_array() method. while (@row = $sth->fetchrow_array()) { print "@row\n"; The fetchrow_array() method fetches the next row of data and returns it as a list containing the field values. We use the while loop to go through all 5 rows. In the next example, we will fetch data by their column names. For this we will utilise the fetchrow_hashref() method. { RaiseError => 1, my $sth = $dbh->prepare( "SELECT * FROM Cars LIMIT 5" ); my $row; while($row = $sth->fetchrow_hashref()) { print "$row->{id $row->{name $row->{price\n"; $sth->finish(); In the example, the data is returned in the form of a reference to a Perl hash. my $row; while($row = $sth->fetchrow_hashref()) { print "$row->{id $row->{name $row->{price\n"; 20

21 The fetchrow_hashref() method fetches the next row of data and returns it as a reference to a hash containing field name and field value pairs. With this method, we can retrieve the values by their column names. In the last example of this section, we fetch all data from the SELECT statement at one step. We use the fetchall_arrayref() method. { RaiseError => 1, my $sth = $dbh->prepare("select * FROM Cars LIMIT 5"); my $all = $sth->fetchall_arrayref(); foreach my $row (@$all) { my ($id, $name, $price) print "$id $name $price\n"; $sth->finish(); The example selects and prints five rows from the Cars table. my $all = $sth->fetchall_arrayref(); We fetch all data in one method call. The fetchall_arrayref() method returns a reference to an array that contains one reference per row. foreach my $row (@$all) { my ($id, $name, $price) print "$id $name $price\n"; We use the foreach loop to go through the retrieved data. Dumping data Perl DBI has a special method called dump_results(). This method is designed as a handy utility for prototyping and testing queries. It uses a neat_list() method to format and edit the string for reading by humans. It is not recommended for data transfer applications. 21

22 { RaiseError => 1 my $sth = $dbh->prepare( "SELECT * FROM Cars LIMIT 5" ); $sth->dump_results(); $sth->finish(); In the example we will dump all the data from the result set. my $sth = $dbh->prepare( "SELECT * FROM Cars LIMIT 5" ); The SQL statement selects five rows from the Cars table. And all three columns. $sth->dump_results(); The dump_results() selects all rows from the statement handle and prints them. It is a method used for prototyping and testing. $./dump.pl 1, 'Audi', , 'Mercedes', , 'Skoda', , 'Volvo', , 'Bentley', rows This is the output of the example. The convenience methods We will show two examples that will use the aforementioned convenience methods. { RaiseError => 1, my $ary = $dbh->selectrow_arrayref("select * FROM Cars WHERE Id = 5"); print join(" "\n"; 22

23 In the first code example, we will call the selectrow_arrayref() method. We select the fifth row from the Cars table. my $ary = $dbh->selectrow_arrayref("select * FROM Cars WHERE Id = 5"); The selectrow_arrayref() method combines the prepare(), execute() and fetchrow_arrayref() into a single call. It returns a reference to the first row of data from the statement. Note that we do not use the statement handle. We use the $dbh database handle object. print join(" "\n"; We print the row to the console. The following example shows the selectall_arrayref() method. { RaiseError => 1, my $all = $dbh->selectall_arrayref("select * FROM Cars LIMIT 5"); foreach my $row (@$all) { my ($id, $name, $price) print "$id $name $price\n"; We again retrieve 5 rows from the Cars table. my $all = $dbh->selectall_arrayref("select * FROM Cars LIMIT 5"); The selectall_arrayref() method returns a reference to an array containing a reference to an array for each row of data fetched. The supplied SQL statement selects 5 rows from the Cars table. Note that we did not call neither the prepare(), nor the execute() method. It is because the selectall_arrayref() method combines prepare(), execute() and fetchall_arrayref() into one single call. foreach my $row (@$all) { my ($id, $name, $price) print "$id $name $price\n"; We go through the fetched array of arrays and print the data to the terminal. $./retrieve.pl 23

24 1 Audi Mercedes Skoda Volvo Bentley This is the output of the example. Parameterized queries Now we will concern ourselves with parameterized queries. When we use parameterized queries, we use placeholders instead of directly writing the values into the statements. Parameterized queries increase security and performance. A programmer must be always cautious when the program receives an input from the user. Instead of building a string from the user input, we bind the value later to the prepared statement. { RaiseError => 1, my $id = 3; my $sth = $dbh->prepare( "SELECT * FROM Cars WHERE Id =?" ); $sth->execute($id); my $ret = $sth->fetch(); foreach my $row (@$ret) { print "$row "; print "\n"; $sth->finish(); In the code example, we select a specific row from the table. The SQL statement has one placeholder, which is filled later in the code. my $id = 3; This could be an input from the user. my $sth = $dbh->prepare( "SELECT * FROM Cars WHERE Id =?" ); The question mark? is a placeholder for a value. The value is added later. $sth->execute($id); 24

25 The execute() statement takes one parameter, which is bound to the placeholder. $./parameterized.pl 3 Skoda 9000 We have retrieved one row from the Cars table using a parameterized query. In the second example, we will use a parameterized query using one of the convenience select methods. { RaiseError => 1, my $id = 2; = $dbh->selectrow_array("select * FROM Cars WHERE Id =?", undef, $id); print join(" "\n"; We have one placeholder to be filled in the SELECT query. = $dbh->selectrow_array("select * FROM Cars WHERE Id =?", undef, $id); The third parameter of the selectrow_array() method takes a value for the placeholder. In this part of the SQLite Perl tutorial, we have demonstrated how to fetch data from the database using various Perl DBI methods. 25

26 Binding SQLite parameters & columns with Perl SQL statements are often dynamically built. A user provides some input and it is built into the statement. A programmer must be cautious every time he deals with an input from a user. It has some serious security implications. The recommended way to dynamically build SQL statements is to use parameter binding. Binding parameters guards the program against SQL injections. It automatically escapes some special characters and allows them to be handled correctly. Many databases also increase significantly their performance, when we prepare the statements and bind the parameters. { RaiseError => 1, my $name = "Volkswagen"; my $sth = $dbh->prepare("select * FROM Cars WHERE Name =?"); $sth->execute($name); my ($id, $name, $price) = $sth->fetchrow(); print "$id $name $price\n"; $sth->finish(); The example selects a row from the Cars table for a specific car name. my $name = "Volkswagen"; This is a value that could come from a user. For example from a HTML form. my $sth = $dbh->prepare("select * FROM Cars WHERE Name =?"); The question mark? is a placeholder for a value. It is added later in the script. $sth->execute($name); In the execute() method we bind the value to the placeholder. The following example is the same as the previous one; this time we use the bind_param() method. 26

27 { RaiseError => 1, my $name = "Volkswagen"; SQLite Perl tutorial my $sth = $dbh->prepare("select * FROM Cars WHERE Name =?"); $sth->bind_param(1, $name); my ($id, $name, $price) = $sth->fetchrow(); print "$id $name $price\n"; $sth->finish(); Retrieving a row with a parameter binding, using the bind_param() method. $sth->bind_param(1, $name); The bind_param() method takes a value and associates it with the placeholder inside the SQL statement. There can be more placeholders. The placeholders are numbered from 1. Quoting parameters Using placeholders and binding parameters to them is the best way to deal with dynamic SQL statement building. Sometimes placeholders cannot be used. For example when we want to dynamically choose a table name. In such cases, we can concatenate the SQL string and use the quote() and quote_identifier() methods. We must use these methods for the variables, otherwise we introduce serious security bugs. The quote() method quotes a string literal for use as a literal value in an SQL statement. It escapes any special characters (such as quotation marks) contained within the string and adds the required type of outer quotation marks. The quote_identifier() method quotes an identifier (table name etc.) for use in an SQL statement. It escapes any special characters (such as double quotation marks) it contains and adds the required type of outer quotation marks. { RaiseError => 1, my $table = "Cars"; my $name = "Volkswagen"; my $sql = sprintf "SELECT * FROM %s WHERE Name = %s", 27

28 $dbh->quote_identifier($table), $dbh->quote($name); my $sth = $dbh->prepare($sql); while (@row = $sth->fetchrow_array()) { print "@row\n"; $sth->finish(); In the example we build the SQL statement string dynamically using the quote() and quote_identifier() methods. my $table = "Cars"; my $name = "Volkswagen"; These are the Perl scalars to be used in the SQL statement. my $sql = sprintf "SELECT * FROM %s WHERE Name = %s", $dbh->quote_identifier($table), $dbh->quote($name); We have a more complex SQL statement. It is not possible to build this statement using placeholders. We use the quote methods to quote the supplied scalars. Binding columns When using the fetch methods, we copy the returned values to Perl variables. This process may be simplified and made faster by binding columns. The Perl DBI has bind_col() and bind_columns() methods, which associate Perl variables with table columns. { RaiseError => 1, my $sth = $dbh->prepare( "SELECT * FROM Cars LIMIT 4" ); $sth->bind_columns(\my($id, $name, $price)); while ($sth->fetchrow_arrayref()) { print "$id $name $price\n"; $sth->finish(); 28

29 In the example, we bind three columns of the Cars table to the $id, $name, and $price variables. $sth->bind_columns(\my($id, $name, $price)); We bind the variables to the colums of the Cars table with the bind_columns() method. while ($sth->fetchrow_arrayref()) { print "$id $name $price\n"; We traverse the returned data and print the values to the console. In this part of the SQLite Perl tutorial we talked about binding parameters. 29

30 Working with images in SQLite with Perl In this chapter of the SQLite Perl tutorial, we will work with image files. Note that some people argue against putting images into databases. Here we only show how to do it. We do not talk about whether to save images in databases or not. sqlite> CREATE TABLE Images(Id INTEGER PRIMARY KEY, Data BLOB); For this example, we create a new table called Images. For the images, we use the BLOB data type, which stands for Binary Large Object. Inserting images In the first example, we are going to insert an image to the SQLite database. { RaiseError => 1 open IMAGE, "mushrooms.jpg" or die $!; my ($image, $buff); while(read IMAGE, $buff, 1024) { $image.= $buff; my $stm = $dbh->prepare("insert INTO Images(Data) VALUES (?)"); $stm->bind_param(1, $image, DBI::SQL_BLOB); $stm->execute(); close(image); $stm->finish(); We read an image from the current working directory and write it into the Images table of the SQLite test.db database. open IMAGE, "mushrooms.jpg" or die $!; We open an image. It is a JPG image called mushrooms.jpg. my ($image, $buff); while(read IMAGE, $buff, 1024) { $image.= $buff; We read binary data from the image file. 30

31 my $sth = $dbh->prepare("insert INTO Images(Data) VALUES (?)"); $sth->bind_param(1, $image, DBI::SQL_BLOB); The three code lines prepare the SQL statement, bind the image data to the statement and execute it. close(image); $sth->finish(); Finally, we are releasing the resources. Reading images In this section, we are going to perform the reverse operation. We will read an image from the database table. { RaiseError => 1 my $stm = $dbh->prepare("select Data FROM Images WHERE Id=1"); $stm->execute(); my $image = $stm->fetch(); print "Image: $image"; open IMAGE, ">mushrooms2.jpg" or die $!; print close(image); $stm->finish(); We read image data from the Images table and write it to another file, which we call mushrooms2.jpg. my $sth = $dbh->prepare("select Data FROM Images WHERE Id=1"); my $image = $sth->fetch(); These three lines select the image data from the table. open IMAGE, ">mushrooms2.jpg" or die $!; print close(image); We open a new image file and write the retrieved data into that file. Then we close the file. This part of the SQLite Perl tutorial was dedicated to reading and writing images. 31

32 Getting SQLite metadata with Perl Metadata is information about the data in the database. Metadata in SQLite contains information about the tables and columns, in which we store data. Number of rows affected by an SQL statement is a metadata. Number of rows and columns returned in a result set belong to metadata as well. Metadata in SQLite can be obtained using the PRAGMA command. SQLite objects may have attributes, which are metadata. Finally, we can also obtain specific metatada from querying the SQLite system sqlite_master table. Method name Description column_info() Provides information about columns table_info() Provides information about tables primary_key_info() Provides information about primary keys in tables foreign_key_info() Provides information about foreign keys in tables The above table lists four Perl DBI methods, which are used to retrieve metadata. { RaiseError => 1 my $sth = $dbh->primary_key_info(undef, "main", "Cars"); = $sth->fetchrow_array(); print join(" "\n"; $sth->finish(); In the first example, we will find out information about a primary key in the Cars table. my $sth = $dbh->primary_key_info(undef, "main", "Cars"); The primary_key_info() returns an active statement handle that can be used to fetch information about columns that make up the primary key for a table. = $sth->fetchrow_array(); From the statement handle, we retrieve the information. $./pkinfo.pl main Cars Id 1 PRIMARY KEY From the output we can see that there is a primary key in the Cars table. The primary key is the first column, named Id. 32

33 In the next example, we will find out some data about the Cars table. { RaiseError => 1 my $sth = $dbh->prepare( "PRAGMA table_info(cars)" ); while (@row = $sth->fetchrow_array()) { print "@row\n"; $sth->finish(); In this example, we issue the PRAGMA table_info(tablename) command, to get some metadata info about our Cars table. my $sth = $dbh->prepare( "PRAGMA table_info(cars)" ); The PRAGMA table_info(cars) command returns one row for each column in the Cars table. Columns in the result set include the column order number, column name, data type, whether or not the column can be NULL, and the default value for the column. while (@row = $sth->fetchrow_array()) { print "@row\n"; We print the selected data. $./pragma_table.pl 0 Id INT Name TEXT Price INT 0 0 Output of the example. Next we will print all rows from the Cars table with their column names. 33

34 { RaiseError => 1, SQLite Perl tutorial my $sth = $dbh->prepare( "SELECT * FROM Cars LIMIT 8" ); my $headers = $sth->{name; my ($id, $name, $price) printf "%s %-10s %s\n", $id, $name, $price; my $row; while($row = $sth->fetchrow_hashref()) { printf "%2d %-10s %d\n", $row->{id, $row->{name, $row->{price; $sth->finish(); We print the contents of the Cars table to the console. Now, we include the names of the columns too. The records are aligned with the column names. my $headers = $sth->{name; We get the column names from the statement object. my ($id, $name, $price) printf "%s %-10s %s\n", $id, $name, $price; The column names are printed to the console. We apply some formatting with the printf function. my $row; while($row = $sth->fetchrow_hashref()) { printf "%2d %-10s %d\n", $row->{id, $row->{name, $row->{price; The data is retrieved, formatted, and printed to the terminal. $./columnheaders.pl Id Name Price 1 Audi Mercedes Skoda Volvo Bentley Citroen Hummer Volkswagen Output. In our last example related to the metadata, we will list all tables in the test.db database. 34

35 { RaiseError => 1, = $dbh->tables(); foreach my $table ) { print "Table: $table\n"; The code example prints all available tables in the current database to the terminal. = $dbh->tables(); The table names are retrieved with the tables() method. $./list_tables.pl Table: "main"."sqlite_master" Table: "temp"."sqlite_temp_master" Table: "main"."cars" Table: "main"."friends" Table: "main"."images" These were the tables on our system. In this part of the SQLite Perl tutorial, we have worked with database metadata. 35

36 SQLite transactions with Perl In this chapter, we will work with transactions. First we provide some basic definitions. Then we present Perl scripts that show, how to work with transactions in Perl DBI. We will also talk about the autocommit mode, which is essential to understand when dealing with transactions. Definitions A transaction is an atomic unit of database operations against the data in one or more databases. The effects of all the SQL statements in a transaction can be either all committed to the database or all rolled back. In the autocommit mode the changes are immediately effective. To work with transactions we either turn the autocommit mode off or start a transaction with the begin_work() method. The transaction is ended with either the commit() or rollback() methods. The database connection is in the autocommit mode by default. The AutoCommit database handle attribute is used to set or read the autocommit mode. When the AutoCommit is on, the call to the begin_work() turns the AutoCommit off. The commit() and rollback() methods turn the AutoCommit back on. If we turn the AutoCommit attribute off and then later call the begin_work() method, we receive an error message that we are already in a transaction. In SQLite, any command other than the SELECT will start an implicit transaction. Also, within a transaction a command like CREATE TABLE..., VACUUM, PRAGMA, will commit previous changes before executing. Manual transactions are started with the BEGIN TRANSACTION statement and finished with the COMMIT or ROLLBACK statements. SQLite supports three non-standard transaction levels: DEFERRED, IMMEDIATE, and EXCLUSIVE. Examples Now we will have some scripts that work with transactions. { RaiseError => 1, AutoCommit => 0, $dbh->do("drop TABLE IF EXISTS Friends"); $dbh->do("create TABLE Friends(Id INTEGER PRIMARY KEY, Name TEXT)"); $dbh->do("insert INTO Friends(Name) VALUES ('Tom')"); $dbh->do("insert INTO Friends(Name) VALUES ('Rebecca')"); $dbh->do("insert INTO Friends(Name) VALUES ('Jim')"); $dbh->do("insert INTO Friends(Name) VALUES ('Robert')"); $dbh->do("insert INTO Friends(Name) VALUES ('Julian')"); 36

37 We create a Friends table and try to fill it with data. However, as we will see, the data will not be committed. { RaiseError => 1, AutoCommit => 0, We have set the AutoCommit parameter to 0. Changes are not automatically committed. And there is no commit statement. So the changes are not written to the database. $./noautocommit.pl $ sqlite3 test.db SQLite version :49:22 Enter ".help" for instructions Enter SQL statements terminated with a ";" sqlite>.tables Cars Images There is no Friends table in our database. In the second example we will write the data into the database with the commit() method. { RaiseError => 1, AutoCommit => 0, $dbh->do("drop TABLE IF EXISTS Friends"); $dbh->do("create TABLE Friends(Id INTEGER PRIMARY KEY, Name TEXT)"); $dbh->do("insert INTO Friends(Name) VALUES ('Tom')"); $dbh->do("insert INTO Friends(Name) VALUES ('Rebecca')"); $dbh->do("insert INTO Friends(Name) VALUES ('Jim')"); $dbh->do("insert INTO Friends(Name) VALUES ('Robert')"); $dbh->do("insert INTO Friends(Name) VALUES ('Julian')"); $dbh->commit(); When the autocommit mode is turned off, every statement is within a transaction until we call the commit() method. $dbh->commit(); All changes are written to the database. sqlite> SELECT * FROM Friends; 1 Tom 2 Rebecca 3 Jim 37

38 4 Robert 5 Julian We verify with the sqlite3 command line tool that the changes were written. When there is an error in the transaction, the transaction is rolled back an no changes are committed to the database. { RaiseError => 1, AutoCommit => 0, $dbh->do("update Friends SET Name='Thomas' WHERE Id=1"); $dbh->do("update Friend SET Name='Bob' WHERE Id=4"); $dbh->commit(); In the code example the autocommit is turned off. There are two statements which form a transaction. There is an error in the second SQL statement. Therefore the transaction is rolled back. $dbh->do("update Friend SET Name='Bob' WHERE Id=4"); The name of the table is incorrect. There is no Friend table in the database. $./rollingback.pl DBD::SQLite::db do failed: no such table: Friend at./rollingback.pl line 15. DBD::SQLite::db do failed: no such table: Friend at./rollingback.pl line 15. Issuing rollback() due to DESTROY without explicit disconnect() of DBD::SQLite::db handle dbname=test.db at./rollingback.pl line 15. Running the example will display this error message. The transaction is rolled back. $ sqlite3 test.db SQLite version :49:22 Enter ".help" for instructions Enter SQL statements terminated with a ";" sqlite> SELECT * FROM Friends; 1 Tom 2 Rebecca 3 Jim 4 Robert 5 Julian sqlite> No changes took place in the Friends table, even if the first UPDATE statement was correct. As we have already mentioned in the tutorial, the default mode is autocommit. In this mode we can start a 38

39 new transaction with the begin_work() method and finish it with either commit() or rollback(). The begin_work() method will turn off the autocommit, the commit() and the rollback() methods will turn the autocommit back on. { RaiseError => 1, HandleError=>\&handle_error, $dbh->begin_work(); $dbh->do("update Friends SET Name='Thomas' WHERE Id=1"); $dbh->do("update Friend SET Name='Bob' WHERE Id=4"); $dbh->commit(); $dbh->do("insert INTO Friends(Name) VALUES('Ronald')"); sub handle_error { my $error = shift; print "An error occurred in the script\n"; print "Message: $error\n"; return 1; Again we have an incorrect second SQL statement. This time we do not explicitly turn off the autocommit. { RaiseError => 1, HandleError=>\&handle_error, We will delegate error handling to the handle_error() subroutine. $dbh->begin_work(); With the begin_work() method, we start a new transaction. The autocommit is turned off. $dbh->do("update Friends SET Name='Thomas' WHERE Id=1"); $dbh->do("update Friend SET Name='Bob' WHERE Id=4"); These two statements form a transaction. The second one is incorrect. sub handle_error { my $error = shift; print "An error occurred in the script\n"; print "Message: $error\n"; 39