Tango with Django. Chapter 1: Overview. 1. Overview. 2. The N tier architecture of web applications

Transcription

1 1 Tango with Django Chapter 1: Overview 1. Overview Aim of this tutorial: provides a pratical guide to web development using Django. What you will learn: Setup a development environment Setup the Django project Configure the Django project to serve static media Work with Django's Model View Template (MTV) design pattern Create database models Create forms Use the user authentication service Incorporate external services Include CSS and JavaScript Design and apply CSS Work with cookies and sessions Include more advanced functionality like AJAX Deploy your application Technoligies and services involved: Python Pip Django Git and github HTML CSS JavaScript, jquery Postgres Twitter Bootstrap Heroku 2. The N tier architecture of web applications The client tier: a web browser 1 Source: 1

2 The processing tier: a web server (maybe with a framework) The datastorage tier: a database Various products/packages: Browser: Google chrome, Firefox, Safari, Opera, Internet Explorer (IE), Edge, Web server: Apache, Nginx, Gunicorn, IIS, Server side language: PHP, JSP, ASP, Perl, Python, Ruby, Browser side language: JavaScript (jquery), Flash, Database: Postgres, MySQL, MS SQL, Oracle, Web frameworks: PHP framework :Laravel, CakePHP, CodeIgniter, Prado, Symfony, Yii, Zend, JSP framework :Struts 2, JSF, Spring MVC, Wicket, Python framework :Django, Grok, Pylons, TurboGears, web2py, Zope2, Ruby framework :Camping, Ruby on Rails, Ramaze, Google App Engine framework :webapp, webapp2 MS ASP.NET Front end framework: Bootstrap, AngularJS, CSS extension language (SASS, LESS, Stylus, ) 3. What is a framework? A framework is a layered structure indicating what kind of programs can or should be built and how they would interrelate. It forces programmers to implement code in a way that promotes consistent coding. It also provides common libraries to facilitate/speed up web development. Conceptually, a framework is different from libraries : our programs call the libraries; however, the framework calls our programs. For a web framework, the framework do most of the work for handling an HTTP request and calls our program when necessary. Such an architecture is also called Handler Callback Pattern " (very common in object oriented programming), in which we hand over most of the work to the framework programs and let the framework call our program back to handle some of the tasks. 2

3 4. Initial design and specification of the website The rango project will provide categorized information of programming languages and tutorials. The followings are the main features. (a) The website Let users browse through user defined categories to access various web pages. (b) Main page: the visitor is able to see the 5 most viewed pages (or all pages); the 5 most rango'ed categories (or all categories); and some way for visitors to browse or search through categories (c) When a user views a category page, they would like to display: the category name, the number of visits, the number of likes; along with the list of associated pages in that category (showing the page s title and linking to its url); and some search functionality (via Bing s Search API) to find other pages that can be linked to this category (d) For a particular category, the client would like the name of the category to be recorded, the number of times each category page has been visited, and how many users have clicked a like button (i.e. the page gets rango ed, and voted up the social hierarchy). (e) Each category should be accessible via a readable URL for example, /rango/books about django/ (f) Only registered users will be able to search and add pages to categories. And so, visitors to the site should be able to register for an account. 5. Wireframes Wireframe: provides clients with some idea of what the application should look like when complete. The main page of the rango project will look like Fig. 1.1, and the category page is shown in Fig

4 6. Pages and URL mappings Fig. 1.1 Fig. 1.2 The URL and page mappings are as follows: /rango/ : main page view /rango/about/ : about page view /rango/category/<categoryname>/ : category page view, <categoryname> might be games, python recipies, or code and compilers /rango/etc/ : other page views, etc could be replaced with another URL 7. Models There are two entities, a category and a page. A category can contain many pages, and one page may be assigned to only one category. The fields and types of the Category Table and Page Table are as follows Category: name (String) views (Intege) likes (Integer) Page: category (ForeignKey) title (String) url (URL) views (Integer) in which Str, Int, FK, and URL are the string, integer, foriegn key, and URL types, respectively. There is another table, User Table, which will be discussed later. 4

5 8. The font convention used in this tutorial This font means user entered texts or commands This font means system messages This font means directories or files This font means codes This font means modified codes This font means emphasized codes 5

6 Chapter 2: Getting Ready to Tango In this project, the software stack we will be using is as follows: Ubuntu LTS Python 3.4 Django 1.8 Postgres Gunicorn 19.3 Nginx Note: Pronunciation: Ubuntu: "oo boon too" Gunicorn: "Green unicorn" or "G unicorn" Ngnix: "Engine Ex" Python3 is pre installed in Ubuntu. For Windows or Mac, python3 would have to be installed manually (see: official python website ). Check package versions: Ubuntu: $ lsb_release a Python 3: $ python3 V Django: $ source <venv>/bin/activate <venv>$ python3 >>> import django >>> django.version Postgress: $ psql version Gunicorn: $ source <venv>/bin/activate <venv>$ gunicorn v Nigix: $ nginx v 1. Creating a virtual environment and installing libraries: We would like to layout our project as follows (under some directory): project/ deploy/ git/ virtualenv/ workspace/ where deploy : contains projects for deploying to Heroku git : the Git repository virtualenv : contains the installed virtual environment workspace : contains projects created from Eclipse Before developing the project, we have to create a virtual environment for our project. A virtual environment contains all the libraries required for the Django project, and its main purpose is to specify the libraries and their versions in the execution 6

7 environment and isolate them from other libraries. Install pip3 and virtualenv: $ sudo apt get install python3 pip $ sudo pip3 install virtualenv Pip is a package management system for python. Create the directory strucutre and virtual environment, tangovenv : $ cd <> # change directory to the parent of directory project $ mkdir project $ cd project $ mkdir virtualenv $ cd virtualenv $ virtualenv p /usr/bin/python3 tangovenv The following short hand names will be used: <ursername> : the username of the computer <projroot> : the project root ( <>/project/workspace/tango ) <projgitroot> : the project root under Git ( <>/project/git/tango/tango ) <venv> : <>/project/virtualenv/tangovenv 2. Installing Postgress on the operating system Install postgres on Ubuntu using the native package: $ sudo apt get update $ sudo apt get y upgrade $ sudo apt get install libpq dev postgresql postgresql contrib After installaton, the postgres daemon will be running. Working with postgres: Check postgres's version: $ psql version Start postgres $ sudo /etc/init.d/postgresql start Stop postgres $ sudo /etc/init.d/postgresql stop Check whether the postgres server is running: $ ps f grep postgres <username> :09 pts/1 color=auto postgres 00:00:00 grep Note: Postgres may require the permission of directory 7

8 /var/lib/postgresql/9.3/main to be 700 $ sudo chmod 700 /var/lib/postgresql/9.3/main Postgres's log: /var/log/postgresql/postgresql 9.3 main.lo g The default database is postgres. To create a database newdb : $ sudo i u postgres [sudo] password for <username>: xxxxxxxx postgres@<username> $ createdb newdb Drop a database: postgres@<username> $ dropdb newdb Create a database user: postgres@<username> $ createuser P newuser Enter password for new role: xxxxxxxx Enter it again: xxxxxxxx Delete a user: postgres@<username>$ dropuser newuser To grant privileges to a user: postgres@<username>$ psql postgres=# grant all privileges on database newdb to newuser; An error message will show up because postgres change every character into lower case. ERROR: database "newdb" does not exist Solution: add double quotes: postgres=# grant all privileges on database " newdb " to newuser; List all databases: postgres=# \list Connect to a database: postgres=# \c <databasename> (The prompt becomes <databasename>=# ) List all tables in the current database: <databasename>=# \dt Change a user's password: postgres=# \password newuser Exit from postgres=# prompt: postgres=# \q Exit from postgres=# prompt: postgres=# exit 8

9 3. Installing Django and Postgres in the virtual environment Now we install Django and Postgres using the command pip3 installed in <venv>. $ cd <venv> $ source bin/activate (venv)$ pip3 install django (venv)$ pip3 install psycopg2 List all the installed libraries: (venv)$ pip3 freeze Django==1.8.2 psycopg2==2.6 Chapter 3: Django Basics and Version Control In this chapter, we will create and setup a new project with Django, and it will be be put in version control with Github. 1. Installing and setting up Eclipse (a) Installing Eclpse Skip (b) Setting up Eclipse Run Eclipse and choose the workspace as <>/project/workspace. Install PyDev in Eclipse: Help Install New Software Add Name: PyDev, Location: OK Check PyDev Next Finish (afterwards: check Brainy Software; PyDev; Brainwy) Create a new python interpreter from the venv: Window Preferences (Left panel) Expand PyDev Expand Interpreters Python Interpreter (Right panel) New Interpreter Name: tangopython3, Interpreter Executable: <venv>/bin/python3 OK Check everything OK OK Install HTML and JavaScript editors Help Install New Software Work with: <Enter> (Left panel) Expand Web, XML, Java EE and OSGi Enterprise Development 9

10 Check Eclipse Web Developer Tools, JavaScript Development Tools and JSF Tools Web Page Editor Next Next Agree with the terms Finish Set file associatiion Window Preferences Expand General and Editor Select File Association File types: *.html, *.htm Associated editors: Add : HTML Editor File types: *.js Associated editors: JavaScript Editor Set monospace font Window Preference General Editors Text Editors (Right panel) Color and Fonts Edit Fonts: DejaVu Sans Mono, Size: 12 OK Show line number in the editor Window Preference General Editors Text Editors (Right panel) Check Show line numbers OK Set character encoding as UTF 8 Window Preference General Editors Text Editors Spelling Encoding: UTF 8 2. Creating a Django project in Eclipse (a) Creating a new django project in Eclipse File New Project PyDev PyDev Django Project Next Project Name: tango, Grammar Version: 3.0, Interpreter: tangopython3 Next Next Finish Note: If there is an error message Django not found, exit eclipse and restart again If the Next and Finish buttons are grayed out, create a dummy PeDev project first and then create the django project. Do not delete the PyDev project. (This is a PyDev bug) CLI (Command line interface) for creating a django project: $ source <venv>/bin/activate (venv)$ django admin startproject tango Project thus created will have the following file structure (under <>/project/workspace/ ): tango/ manage.py tango/ init.py settings.py urls.py wsgi.py 10

11 where manage.py : main Django execution command init.py : indicates that the directory is a python package settings.py : project's settings urls.py : URL patterns wsgi.py : runs the development server and deploy to production environment (b) Setting up the project Edit <projroot>/tango/settings.py and make the following changes: ALLOWED_HOSTS = [ '*' ] LANGUAGE_CODE = ' zh tw ' TIME_ZONE = ' Asia/Taipei ' DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': ' xxxxxxxx ', 'USER': ' xxxxxx ', 'PASSWORD': ' xxxxxxxx ', 'HOST': 'localhost', 'PORT': '', } } Note: the database name and user will be set to tangodb and tango, respectively. Create a tangodb database and grant all privileges to user tango : $ sudo i u postgres [sudo] password for <username>: xxxxxxxx postgres@<username> $ createdb tangodb postgres@<username> $ createuser P tango Enter password for new role: xxxxxxxx Enter it again: xxxxxxxx postgres@<username>$ psql postgres=# grant all privileges on database " tangodb " to tango ; GRANT postgres=# \q postgres@<username> $ exit Note: The double quotes for tangodb is necessary because psql lowercases the command. Otherwise, an error message will show up: ERROR: database "tango db " does not exist. If the username contains capital letters, double quotes are 11

12 also necessary. There is a semicolon at the end of the command. Reset the password: postgres@<username>$ psql postgres=# \password tango After creating the database, it's time to migrate it. Database migration is the process of taking the schema specified in the application and actually creating the corresponding data tables in the database. Django (or South ) provides two commands for database migration: makemigrations and migrate. The purpose of makemigrations is to prepare everything for database migration. Further readings: What are migrations? The basics of south To perform makemigration: Right click project Django Make Migrations Specify the name of the app Because we have not created/altered any models, there is no need to perform makemigrations. Now, migrate the database: Right click project Django Migrate Operations to perform: Synchronize unmigrated apps: messages, staticfiles Apply all migrations: sessions, admin, contenttypes, auth Synchronizing apps without migrations: Creating tables Running deferred SQL Installing custom SQL Running migrations: Rendering model states DONE Applying contenttypes.0001_initial OK Applying auth.0001_initial OK Applying admin.0001_initial OK Applying contenttypes.0002_remove_content_type_name OK Applying auth.0002_alter_permission_name_max_length OK Applying auth.0003_alter_user_ _max_length OK Applying auth.0004_alter_user_username_opts OK Applying auth.0005_alter_user_last_login_null OK Applying auth.0006_require_contenttypes_0002 OK Applying sessions.0001_initial OK CLI for migation: (venv)$ python3 manage.py makemigrations (venv)$ python3 manage.py migrate 12

13 We are now ready to run the vanilla project. 3. Running the project Right click project Run As PyDev: Django Performing system checks System check identified no issues (0 silenced). June 16, :57:53 Django version 1.8.2, using settings 'tango.settings' Starting development server at Quit the server with CONTROL C. Open a browser and test: localhost:8000 Setup the run configuration: Right click project Run As Run Configurations (Left panel) PyDev Django tango tango (note: there are usually repeated two names, this is a bug of Eclipse) (Right panel) Name: tango, Main tab, Project: tango, Main Module: ${workspace_loc:tango}/${django_manage_location}, Arguments tab, Program arguments: runserver Apply Close CLI for running the project: (venv)$ cd <projroot> (venv)$ python3 manage.py runserver 4. Creating Django applications Create a main app: Right click project Django Create Application Name of the appdjango: main OK CLI for creating an app: (venv)$ python3 manage.py startapp <appname> Under the proejct's root folder, a new folder main will be created with the following files: main/ migrations/ init.py admin.py models.py tests.py views.py where migrations : contains migration information for this app init.py : same as before admin.py : contains the models that are avaiable in the 13

14 admin's page models.py : stores the app's data models test.py : stores functions for testing the app views.py : stores functions for handling requests Register the app so that Django knows this app has been installed. Edit settings.py and add main into INSTALLED_APPS : INSTALLED_APPS = ( 'django.contrib.messages', 'django.contrib.staticfiles', 'main', ) 5. Creating a view Model View Controller (MVC) is a software architecure pattern with the main purpose to separate the following three main building blocks of a software application: Model: the data (i.e., the database schema) View: the page (i.e., the HTML files) Controller: the program (business logic) Django practices MVC, too, but it is called model tamplate view (MTV): Model: the data (i.e., the database schema) Tamplate: the page (i.e., the HTML files) View: the program (business logic) Best practice: Fat model, thin view, stupid template. Now we are ready to create a view which print out the string "Hello world!" on the web page. Edit /main/views.py with the following content: from django.shortcuts import render from django.http import HttpResponse def main(request): return HttpResponse('Hello world!') In the above: Lines 1, 2: import objects from django.shortcuts and django.http for handing HTTP requests ( render is not used now and will be used later) Line 4: the view function, main, which handles the HTTP request. It takes an input argument request. Each view takes at least one argument an HTTPRequest object, with the conventional name request. But other names may be used. Line 5: each view must return an HTTPResponse object. A 14

15 simple HTTPResponse object takes a string parameter, e.g. "Hello world!", to send to the client. Good practice: use the same names for all objects within the same app, e.g., request: /main/, template: main.html, view function: main(), CSS: main.css, With the view created, next thing is to specify which kind of requests will be handled by this view. 6. Mapping URLs It is necessary to specify which URL is handled by which app, and such mappings are stored in the urls.py file. Create a file main/urls.py with the following contents: from django.conf.urls import patterns, url from main import views urlpatterns = patterns('', url(r'^$', views.main, name='main'), ) In the above: Line 1: import Django's URL pattern Line 2: import the view function from package main Lines 4~6: Django uses Python tuples to set up URL mappings. The name of the tuple must be urlpatterns, which contains a series of calls to the django.conf.urls.url() function (we have only one mapping here). The arguments to the url() function: r'' : a raw string (i.e., there is no escape characters in the string) in which a regular expression is used to match the HTTP request ^ : begin with the expression to the right of ^ $ : end with the expression to the left of $ ^$ : means an empty string views.main : the matched request will be handle my views.main module name='main' : this URL mapping is given the name main for later reference In the proejct's URL mapping <projroot>/urls.py, insert the main 's URL mapping: from django.conf.urls import patterns, include, url from django.contrib import admin urlpatterns = patterns('', url(r'^admin/', include(admin.site.urls)), url(r'^.*', include('main.urls')), ) 15

16 In the above: r'^.*' : the request format is followed by anything. It uses the regular expression (explained later), in which dot means any character and star means the character to its left may occur zero or many times. The request will be handled according to the settings in the main.urls module. For example, we may enter in the browser's url field. The string, whatever/andwhatever/ matches r'^.*'. The rest of the string (i.e., an empty string) will be sent to main.urls for further matching. Finally, an empty string matches r'^$' in the url mapping of the main app, hence the request will be handled by the main.view.main() function. A separate urls.py file for each application allows you to set URLs for individual applications, which maintains minimal coupling among apps. Enter in the browser's URL field, the result: 7. Creating a superuser Every project needs a superuser to manage its data, and Django provides powerful admin functions to manipulate the data. To create a superuser: CLI: $ source <venv>/bin/activate (venv)$ cd <projroot> (venv)$ python3 manage.py createsuperuser Username (leave blank to use '<username>'): xxxxxx address: xxxxxxx@xxxxx.xxx Password: xxxxxxxx Password (again): xxxxxxxx Superuser created successfully. Note: we will use admin as the name of the superuser. The admin page: 16

17 8. Distributed version control with Git A project is usually developed by a group of team members. Version control is a good tool for team collaboration. We will use the Git version control on GitHub. Eclipse has a built in Git tool called EGit. (a) Setting up Git repository Set up egit folder: Window Preferences (Left panel) Expand Team Git (Right panel) Default repository folder: <>/project/git OK Note: Eclipse suggests not to set the git repository inside the project in order to prevent interference. (b) Creating a local Git repository Right click project Team Share Project Select Git Next Click Create button Parent directory (or Repository directory): <>/project/git/tango Finish Finish The project will be moved to the <>/project/git/tango directory (one more level down) and a.git directory will be created: project/ git/ tango/.git/ tango/ manage.py 17

18 tango/ The content of.git/config : [core] symlinks = false repositoryformatversion = 0 filemode = true logallrefupdates = true Note : after commiting the project, the development server needs to be restarted. (c) Creating a GitHub account Go to Create a GitHub account (The username will be referenced by <gitusername> ) Create a Repository Repository name: tango Create repository (Registration confirm in ) You may add collaborators of the project: (Right panel) Settings (Left panel) Collaborators Enter the username and then select it Add collaborator (d) Pushing to GItHub Right click project Team Commit (Eclipse may ask for name and ) Name: xxxxxxxx, xxxxxx@xxx.xxx.xxx (Don't show this dialog again) OK Commit message: xxxxxxxx Select all files by clicking the check box at the right Commit and Push URI: (Host: github.com, Repository path: /<gitusername>/tango.git ) Next User: xxxxxxxx, Password: xxxxxxxx OK Add create/update specification, Source ref: master [BRANCH] ( refs/heads/master ), 18

19 Destination ref: refs/heads/master + Add Spec Finish User: xxxxxxxx ( <gitusername> ), Password: xxxxxxxx OK OK (f) Creating a.gitignore file Some files/directories do not need version control and they can be specified in the.gitignore file. Create.gitignore file in the project root directory which lists the file or directory patterns that will not be committed into version control:.* *~ *.pyc Note: Don't create.gitignore before the first push because the Eclipse's.project and.pydevproject files have to be there for other collaborators to pull. Other cllaborators may create their own.gitignore file after first pull. (g) Collabortors import the project File Import (Left) Expand Git Projects from Git Next Clone URI Next URI: Next Check master Next Make sure the destination directory is correct Next Check Import existing Eclipse projects Next Check tango Finish (h) Configuring push and pull Right click project Team Push Branch Remote name: origin, URI: Host: github.com, Repository path: /<gitusername>/tango.git Next Branch name: master, Check "Configure upstream for push and pull", When doing a pull, check "Merge upstream commits into local branch" Next Input user and PSW OK Finish Input user and PSW OK OK The content of <>/project/git/tango/.git/config : [core] symlinks = false repositoryformatversion = 0 filemode = true logallrefupdates = true [remote "origin"] url = <gitusername> /tango.git fetch = +refs/heads/*:refs/remotes/origin/* [branch "master"] 19

20 remote = origin merge = refs/heads/master (i) Follow up push to GitHub Right click project Team Commit Commit Message: xxxxxxxx, Check the files to push Commit and Push Enter user and PSW Next OK (j) Follow up pull from GitHub Right click project Team Pull OK 20

21 Chapter 4: Templates and Static Media In this chapter, we'll discuss the template engine as well as how to serve static media within your web pages. 1. Responding with the HTML format In the previous chapter, the main view only returns a string to the user's browser. Normally, we would like to return a complete HTML page, and the HTML page may be coded in the program and sent directly to the user. Change the content of main/views.py as follows and run the app again: from django.http import HttpResponse def main(request): html = ''' <!doctype html> <html> <head> <title>tango with Django</title> <meta charset=utf 8> </head> <body> <p>this is the HTML version of Hello World!</p> </body> </html> ''' return HttpResponse(html) Such a method is not practical because Writing HTML codes in the program is a bad practice, which violates the paradigm of separation the view from the template. The HTML could have a great number of codes Solution: use HTML files. 2. Creating a template folder and add a template The best way of handling HTML codes is to use HTML files, which are called templates in Django. The most important function of a template is that we may create dynamic web pages, in which the view provide variable values to the template. Django requires the structure of templates in an app as follows: <appname>/ templates/ <appname>/ <somename>.html 21

22 For our project, it will be: main/ templates/ main/ main.html Note: Why should there be one more folder also called main? Because prefixing every HTML file with its app is a good idea, so the format of the render() function that returns a page to the user is preferred to be render(request, ' <appname>/<somename>.html ', ). Now create the folder structure: Right click main app New Folder Folder name: templates Right click the main/templates folder New Folder Folder name: main Right click the main/templates/main folder New File File name: main.html Finish. Add the following contents in main.html : <!doctype html> <html> <head> <title>tango with Django</title> <meta charset=utf 8> </head> <body> <h1>django says hello world!</h1> <p><strong>{{message}}</strong></p> </body> </html> In the above: {{message}} is a Django template variable which will be substituted by the value supplied by the view. A template variable begins with {{ and ends with }}, with the variable name between them. Modify main/views.py such that it will render an HTML template and supply the value of the template variable message : from django.shortcuts import render from django.http import HttpResponse def main(request): context = {'message':'django is great.'} return render(request, 'main/main.html', context) In the above: context : Django uses Python's dictionary to map the template variable and its value 22

23 render() : a function to respond to the request, which has three arguments: the HTTP request object: request the template: main.html a dictionary which contains template variables and their values: context render() function replace all template variables with the corresponding values specified in the context dictionary and returns the resulting page to the user: Result: 3. Creating another app 'rango' Create another app rango : Right click project Django Create Application Name of the django app to be created: rango OK Register the apps: edit tango/settings.py and add rango into INSTALLED_APPS : INSTALLED_APPS = ( 'django.contrib.staticfiles', 'main', 'rango', ) Edit rango/views.py with the following content: from django.shortcuts import render def rango(request): return render(request, 'rango/rango.html') Create folders rango/templates and 23

24 rango/templates/rango. And then, create file rango/templates/rango/rango.html with the following contents: <!doctype html> <html> <head> <title>tango with Django</title> <meta charset=utf 8> </head> <body> <h1>rango says hello world!</h1> <a href=/rango/about/>about</a><br><br> </body> </html> Create file rango/urls.py : from django.conf.urls import patterns, url from rango import views urlpatterns = patterns('', url(r'^$', views.rango, name='rango'), ) Register the urls in tango/urls.py : url(r'^admin/', include(admin.site.urls)), url(r'^rango/', include('rango.urls')), url(r'^.*', include('main.urls')) Note: the main app's url has to be at the last line. Now, test the rango app by entering localhost:8000/rango/ in the browser's URL field. Note: the About link in rango.html currently has no corresponding view, hence is handled by main view due to url(r'^.*', ). We may add a link in the main page, which goes to the rango page. Add the following line to main.html : <p><strong>{{message}}</strong></p> <br> <p><a href=/rango/>go to the rango page</a></p> </body> </html> 4. Serving static media 24

25 Static media: images, videos, audios, JavaScripts, static HTML, CSS, The contents of static data will not change, therefore they are handled differently from dynamic templates. To get static media up and running, you will need to set up a directory in which static media files are stored. Now, we will add an image into the template. First we need to configure the static media directory: create folder /rango/static. Django will look for a static subdirectory in each app in INSTALLED_APPS. Create another two folders: /rango/static/rango/ and /rango/static/rango/img/, and then put the image rango.png in the folder. So, we have the following folder structure: rango/ static/ rango/ img/ rango.png Edit rango/templates/rango/rango.html file and make the following changes: <!doctype html> {% load staticfiles %} <html> <a href=/rango/about/>about</a><br><br> <img src="{% static 'rango/img/rango.png' %}" alt="picture of Rango"> </body> </html> Note: HTML requrires <!doctype html> to be at the first line, so {% load staticfiles %} is at the second line. {% %} is a template statement, which will be explained further later. The resulting page is on the right: The function of {% load staticfiles %} and {% static 'rango/img/rango.png' %} is to use STATIC_URL in settings.py to compose the correct path of the static file. Because STATIC_URL is /static/, so the resulting path is /static/rango/img/rango.png. Now add a CSS file in main app: create the folders main/static, main/static/main, and main/static/main/css and create file main/static/main/css/main.css with the follow contents: body { background color: lightblue; 25

26 } Add the CSS link in main.html : {% load staticfiles %} <html> <head> <link rel=stylesheet href="{% static 'main/css/main.css' %}"> </head> After adding the CSS link, the server may need to restart. Also add CSS link in rango.html : <head> <link rel=stylesheet href="{% static 'main/css/main.css' %}"> </head> The resulting page: Request localhost:8000/static/main/css/main.css can access the CSS file. Same as image file localhost:8000/static/rango/img/rango.png. See: Deploying static files into production. Another type of static media is JavaScript. T locate a JavaScript is similar: create folder main/static/main/js and file main/static/main/js/main.js, and add the following line in the template. <script src="{% static 'main/js/main.js' %}"></script> 5. Exercises Now, fix the About link by adding a view and a template. Create a new view about in rango/views.py : 26

27 def about(request): return render(request, 'rango/about.html') Create a new template rango/templates/rango/about.html. <!doctype html> {% load staticfiles %} <html> <head> <title>tango with Django</title> <meta charset=utf 8> </head> <body> <h1>rango says: Here is the about page.</h1> </body> </html> Create a new URL mapping in rango/urls.py : url(r'^$', views.rango, name='rango'), url(r'^about/$', views.about, name='about'), Now test the system by clicking the About link in the rango.html page: Now, we may push the new version of this project to Github. 27

28 Chapter 5: Models and Databases Django's database module: object relational mapping (ORM). ORM is a programming technique for converting data between incompatible type systems in object oriented programming languages. SQL languages are no longer needed when ORM is introduced. A Django model is a Python object that describes the data model/table. Instead of directly working on the database table via SQL, all you have to do is manipulate the corresponding Python object. 1. Rango's requirement Rango's data requirements: Rango is a web page directory a site containing links to other websites There are a number of different webpage categories, and each category contains a number of links A category has a name, number of visits, and number of likes A page refers to a category and has a title, URL, and a number of views 2. Creating models About our database, we have already set the following in settings.py : DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'xxxxxxxx', 'USER': 'xxxxxx', 'PASSWORD': 'xxxxxxxx', 'HOST': 'localhost', 'PORT': '', } } The database engine is Postgres backend. We don't need to alter our application if later we decided to use another database, say MySQL. All we have to do is to change the database engine. For example, MySQL: DATABASES = { 'default': { 'ENGINE': ' mysql.connector.django ', 'NAME': 'xxxxxxxx', 'USER': 'xxxxxx', 28

29 } } 'PASSWORD': 'xxxxxxxx', 'HOST': 'localhost', 'PORT': '3306', 'OPTIONS': { 'autocommit': True, }, For the Rango site, we need two initial models: Category and Page. Category: name Page: category title url views Edit rango/models.py and add the following: from django.db import models class Category(models.Model): name = models.charfield(max_length=128, unique=true) def str (self): return self.name # unicode (self): for python2 class Page(models.Model): category = models.foreignkey(category) title = models.charfield(max_length=128) url = models.urlfield() views = models.integerfield(default=0) def str (self): # unicode (self): for python2 return self.title Note: It is a good practice to use the lowercase of the model name as a foreign key. For example, category = models.foreignkey( Category ) A Django model is a Python class and it subclasses Django's module models.models. 29

30 Django provides a number of built in fields. Some of the most commonly used are listed below. CharField : a field for storing character data (e.g. strings). Specify max_length to provide a maximum number of characters the field can store URLField : much like a CharField, but designed for storing resource URLs. You may also specify a max_length parameter IntegerField : stores an integes DateField : stores a Python datetime.date If a field value has to be unique, you can specify unique=true Django also provides simple mechanisms to relate models/database tables together. These mechanisms are encapsulated in three further field types:. ForeignKey : a field type that allows to create a one to many relationship OneToOneField : afield type that allows to define a strict one to one relationship ManyToManyField : a field type which allows to define a many to many relationship In the Rango application, several pages may belong to one catogory, hence, it's a one to many relationship, and the ForeignKey is used to designate such relationship. The str (self): method is used in the admin page, return specifies which field should be displayed. 3. Creating and migrating the database Now, let Django create the database tables. Migration lets you alter the model fields later without deleting the entire model and recreate it again. We've done the intial migrate, and whenever we make changes to the models, we need to register the changes, via makemigrations command for the particular app. Right click project Django Make Migrations Name of the app: rango OK Migrations for 'rango': 0001_initial.py: Create model Category Create model Page CLI for make migrations: (venv)$ python3 manage.py makemigrations rango Makemigrations command creates file 0001_initial.py in rango/migrations folder Now, migrate the data models: 30

31 Right click project Django Migrate Operations to perform: Synchronize unmigrated apps: staticfiles, messages Apply all migrations: contenttypes, rango, sessions, auth, admin Synchronizing apps without migrations: Creating tables Running deferred SQL Installing custom SQL Running migrations: Rendering model states DONE Applying rango.0001_initial OK CLI for migrate: (venv)$ python3 manage.py migrate Add one more item 00*_* in.gitignore : 00*_* Note: 00*_* files are migration information and they may be different for each team member. Hence, they don't need version control during development. However, when the system goes production, migration information need to be retained; therefore, the 00*_* item in.gitignore has to be removed. To delete all data in the database: drop the database and delete all 00* files: $ sudo i u postgress # and then enter password $ dropdb tangodb $ find. type f name 00* exec rm {} \; 4. Configure the admin interface Django provides a build in, web based administrative interface for manipulating databases. To be able to edit rango's data, make sure that the item 'django.contrib.admin' is in the INSTALLED_APPS in file settings.py, and there is the item url(r'^admin/', include(admin.site.urls)), in urlpatterns in the project's urls.py. Register rango 's models such they may be manipulated by Django's admin interface. Edit rango/admin.py and add the following contents: from django.contrib import admin from rango.models import Category, Page admin.site.register(category) admin.site.register(page) The admin page: 31

32 It can be seen that Django changes the name of models by adding s to make them plural. 5. Creating a popluating script Entering data into the database one by one is time consuming. We would like to create a population script to automatically populate the database with initial test data. First, define the popcategory() function to retrieve or create a category record: def popcategory(name): category = Category.objects.get_or_create(name=name)[0] return category This function takes one arguments (the name of the category: name ) and uses get_or_create() method to retrieve or create a record in the model. And then it returns the record. Second, define the popcategory() function to retrieve or add a page record: def poppage(category, title, url, views=0): page = Page.objects.get_or_create(category=category, title=title, url=url, views=views)[0] page.save() This function takes several arguments ( category, title, url, and views ) and also uses get_or_create() method to retrieve or create a record in the model. With the above two functions, we may perform a series of calls to create the initial data. For example, adding a Python category and an Official Python Tutorial page can be done as follows: pythoncategory = popcategory('python') poppage(category=pythoncategory, title='official Python Tutorial', url=' Edit rango/views.py and add the following: 32

33 from rango.models import Category, Page def populate(request): # Python pythoncategory = popcategory('python') poppage(category=pythoncategory, title='official Python Tutorial', url=' poppage(category=pythoncategory, title='how to Think like a Computer Scientist', url=' poppage(category=pythoncategory, title='learn Python in 10 Minutes', url=' # Django djangocategory = popcategory('django') poppage(category=djangocategory, title='official Django Tutorial', url=' poppage(category=djangocategory, title='django Rocks', url=' poppage(category=djangocategory, title='how to Tango with Django', url=' # Other frameword framecategory = popcategory('other Frameworks') poppage(category=framecategory, title='bottle', url=' poppage(category=framecategory, title='flask', url=' def popcategory(name): category = Category.objects.get_or_create(name=name)[0] return category def poppage(category, title, url, views=0): page = Page.objects.get_or_create(category=category, title=title, url=url, views=views)[0] page.save() 33

34 In the admin page, clicking Categorys or Pages, the category names or page titles will appear, respectively, because we specified such in the model declaration. Category Page def str (self): return self.name def str (self): return self.title 6. Django ORM An object relational mapping (ORM) is a tool that lets you query and manipulate data from a database using an object paradigm. That is, map object oriented language to relational database. With ORM, SQL is not needed anymore. Common database operations: CRUD create, read, upate, and delete. Examples: (a) Create: Category.object.create() Category.objects.get_or_create()[0] (b) Read: Category.object.all() # Retrieve all data Category.objects.get() # Retrieve only one record Category.objects.filter() # Retrieve one or more records as a list (c) Update: category = Category.object.get() category.name = category.save() (d) Delete: 34

35 category = Category.object.get() category.delete() 7. Basic workflow Steps of setting up models and databases. 1. Configure DATABASES in settings.py 2. Add a model: (a) Create the new model in the app's models.py (b) Register the model in the app's admin.py (c) Migrate DB (d) Enter data or run the populate script 8. Exercises There are two fields missing in class Category : views and likes. Now, fix it. Add two more fields in class Category : class Category(models.Model): name = models.charfield(max_length=128, unique=true) views = models.integerfield(default=0) likes = models.integerfield(default=0) And then populate the two fields by modifying rango/views.py : import random def popcategory(name): category = Category.objects.get_or_create(name=name)[0] category.views = random.randint(0, 20) category.likes = random.randint(0, 20) category.save() return category Perform makemigrations and migrate (there will be an additional file 0002_*.py in the rango/migrations folder). Now, populate the data again: Note: When performing migration, eclipse may terminate the development server. However, running the server again, you may encounter the error message: Error: That port is already in use. It means the server was not terminated properly and the port 8000 is occupied. Click Terminate to stop server and run the server again. Or use the command $ fuser k 8000/tcp to clear the port: It's time to push the project to Github. 35

36 Chapter 6: Models, Templates, and Views Now that we have the models set up and populated with some data, we are ready to to retrieve the data from the database and present the data within the templates. 1. Basic workflow: data driven pages Django's steps to create a data driven webpage. 1. Import the models in the views.py 2. Query the model to get data 3. Pass the results into the template's context 4. Setup the template to present the data to the user 2. Showing categories on Rango's page Now, we would like to show the categories on the page. All records in model Category will be retrieved via database query and listed in rango.html as follows: Django Python Other Frameworks Edit rango/view.py : retrieve data form Category and pass the data to template rango.html as a python list def rango(request): categories = Category.objects.order_by(' likes') context = {'categories':categories} return render(request, 'rango/rango.html', context ) Note: It is a good practice to use the lowercase, plural form of the model name as the name of the retrieved list. For example, categories = Category.object So, later we may iterate each of the item as follows: for category in categories: It is also a good practice to use the same name for a template variable and its value. For example, context = {' categories '= categories } So, later we may iterate each of the item in the template as follows: {% for category in categories %} A template statement is enclosed by {% and %}, and there should be one space after {% and before %}. A template variable is enclosed by {{ and }} ; however, if the variable is in a statement, it does not need to be enclosed. We will use the decision ( if ) and iteration ( for loop) statements 36

37 to list the data. The format of the if statement: {% if <condition> %} {% endif %} {% if <condition> %} {% else %} {% endif %} {% if <condition> %} {% elif <condition> %} {% elif <condition> %} {% else %} {% endif %} The format of the for statement: {% for <item> in <itemlist> %} {% endfor %} {% for i in 'xxxxxxxxxx' %} {{forloop.counter0}} {% endfor %} views: render(,, {'range':range(100)}) template: {% for i in range %} {% endfor %} Edit rango.html : display the category list and remove the image element <h1>rango says hello world!</h1> {% if categories %} 37

38 <ul> {% for category in categories %} <li>{{category.name}}</li> {% endfor %} </ul> {% else %} <p><strong>there are no categories present.</strong></p> {% endif %} <a href=/rango/about/>about</a><br><br> <img src="{% static 'rango/img/rango.png' %}" alt="picture of Rango"> Result of 3. Creating a details page According to Rango s specification, we also need to show a list of pages that are associated with each category. A new view must be created, which should be parameterised, i.e., there will be parameters in the request. We also need to create URL patterns and URL strings that encode category names. (a) URL design and mapping There should be a way for the user to tell Rango which category he/she wants to see. Solution 1: Design the URL patterns such as /rango/category/1/ or /rango/category/2/, in which the numbers 1 and 2 are the category's ID. However, the user wouldn't know what it means by 1 or 2. Solution 2: Use the category name as part of the URL pattern such as /rango/category/python/ or /rango/category/django/. This patter is simple, readable, and meaningful. Problem: the category name may have multiple words with white spaces, like 'Other Frameworks', which is inappropriate because URLs does not like spaces. This can be solved by Django's slugify function. Also see the solution. 38

39 (b) Add a slug field in the Category model To make clean urls, we are going to include a slug field in the Category model. First import the slugify function, which will replace white space with hyphens, e.g., "how do i create a slug in django" turns into "how do i create a slug in django". Second, override the save method of Category model, which we will call the slugify method and update the slug field. Modify rango/models.py as follows: from django.template.defaultfilters import slugify class Category(models.Model): name = models.charfield(max_length=128, unique=true) views = models.integerfield(default=0) likes = models.integerfield(default=0) slug = models.slugfield(unique=true) def save(self, *args, **kwargs): self.slug = slugify(self.name) super(category, self).save(*args, **kwargs) def str (self): return self.name In the above: One more field, slug, is added with field type SlugField. Django models has a default save method, but here we override it with our own save method. The save method: Use Slugify() function to replace white spaces with hyphens in the name field and store the result in slug field Save the model using the save method, which calls the save mothod of the parent class ( models.model ) Python function arguments: *args : positional arguments, for examples: # Example 1 def foo(x, y, z): print(x, y, z) foo(1, 2, 3) # (x, y, z take values 1, 2, 3, respectively) # Example 2 def foo(x, y, z=3): print(x, y, z) # Default arg may not follow non default arg 39

40 foo(1, 2) # x, y take values 1, 2,respectively; z is 3 by default # Example 3 def foo(*args): print(args) foo(1, 2, 3) # (1, 2, 3), a tuple **kwargs : keyword arguments, for example: def bar(**kwargs): print(kwargs) foo(x=1, y=2, z=3) # {'x':1, 'y':2, 'z':3}, a dictionary Now make migration for the model: Click project Django Make Migrations Name of the app: rango OK You are trying to add a non nullable field 'slug' to category without a default; we can't do that (the database needs something to populate existing rows). Please select a fix: 1) Provide a one off default now (will be set on all existing rows) 2) Quit, and let me add a default in models.py Select an option: 1 Please enter the default value now, as valid Python The datetime and django.utils.timezone modules are available, so you can do e.g. timezone.now() >>> '' Migrations for 'rango': 0003_category_slug.py: Add field slug to category In the above, because we have added a new field and the existing records don't have any value in that field. We have two options: to provide a default value once for all, or quit this makemigration. We choose to enter an empty string as a default value. Now migrate the model: Click project Django Migrate Operations to perform: Synchronize unmigrated apps: staticfiles, messages Apply all migrations: contenttypes, rango, auth, sessions, admin Synchronizing apps without migrations: Creating tables Running deferred SQL Installing custom SQL Running migrations: Rendering model states DONE Applying rango.0003_category_slugtraceback (most recent call last): File 40

41 "<venv>/lib/python3.4/site packages/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) psycopg2.integrityerror: could not create unique index "rango_category_slug_key" DETAIL: Key (slug)=() is duplicated. Migrate failed because we specified unique=true in the slug field; however, we provided an empty string for all existing records. Try the following solutions: Remove unique=true and migrate again Fail Remove slug field and migrate again Fail Enter admin page to set values Fail The only solution: delete the database? During development, it's OK to delete the database; but in production, you cannot do this! Correct solution: 1. Remove unique=true in the slug field 2. Delete the failed migration files in rango (in our case, files with the starting number greater than 0002) 3. Perform makemigrations and migrate 4. Run localhost:8000/rango/populate/ to fill in the slug field for all existing records (because name is unique, the derived slug will be unique) You may want to check in the admin page to see if everything is OK 5. Set unique=true 6. Perform makemigrations and migrate again Lesson learned: When adding a new field in a model, never set any constraints. Constraints: unique=true, blank=false, null=false, Set the constraint after correct data have been entered to the new field for each existing records. (c) Display categories and pages After designing the slug URL, we will start to write a view to retrieve all data in Category and Page models and display them in a template. The steps: 1. Create a new view in rango/view.py, called category, which will take an additional parameter, categorynameurl which will store the encoded category name 2. Create a new template templates/rango/category.html 3. Update Rango's urlpatterns to map the new category view to a URL pattern in rango/urls.py 41

42 (c.1) Create a category() view to retrieve the data Edit rango/views.py : add a new view category() def category(request, categorynameslug): context = {} try: category = Category.objects.get(slug=categoryNameSlug) context['category'] = category pages = Page.objects.filter(category=category) context['pages'] = pages except Category.DoesNotExist: context['categoryname'] = categorynameslug.replace(' ', ' ') return render(request, 'rango/category.html', context) In the above, categorynameslug is an argument obtained from the URL request context = {} or context = dict initializes an empty dictionary. context[''] = or context.update({'':}) adds or modifies an item in the dictionary try: retrieve data from models Category and Page, and put them into template variables. If the category is not found, except part will takeover without crashing the program. except : replace hyphens with white spaces in the categorynameslug such that the name looks normal in the template (c.2) Create a category.html template to display the data Create template rango/templates/rango/category.html and enter the following contents: <!doctype html> {% load staticfiles %} <html> <head> <title>tango with Django</title> <meta charset=utf 8> <link rel=stylesheet href="{% static 'main/css/main.css' %}"> </head> <body> {% if category %} <h1>{{category.name}}</h1> {% if pages %} <ul> {% for page in pages %} <li><a href={{page.url}}>{{page.title}}</a></li> 42

43 {% endfor %} </ul> {% else %} <p><strong>no pages currently in category.</strong></p> {% endif %} {% else %} <p>the specified category "{{categoryname}}" does not exist.</p> {% endif %} </body> </html> In the above: First print out the category name If categories is not empty If there are pages associated with it then print the pages, otherwise "no pages " else print " category does not exist. " (c.3) Create a URL mapping Modify rango/urls.py : urlpatterns = patterns('', url(r'^category/(?p<categorynameslug>[\w\ ]+)/$', views.category, name='category'), ) In the above: (?P<name>pattern) is a named regular expression group in which name is the name of the argument passed to the view (remember we have such an argument in category() view) and pattern is some pattern to match. For example, (?P<categoryNameSlug>Python) will pass the argument categorynameslug with the value Python to the request handler. \w matches any character, number, or underscore (i.e., a~z, A~Z, 0~9, _ ) \ matches a hyphen + is a "one or more" quentifier. Hence, [\w\ ]+ means one or more character, number, underscore, or hyphen The followings are some of the regular expresions' symbols/expressions and their meanings: Symbol/ Expression Matched String. Any character ^ Start of string 43

44 $ End of string * 0 or more repetitions + 1 or more repetitions? 0 or 1 repetitions A B means A or B [a z] [A Z] [0 9] Any lowercase character Any uppdrcase character Any one digit number \w Any alphanumeric character or _ \d Any digit (c.3) Add a category link in rango.html Finally, modify the rango.html template: add a link to a category <ul> {% for category in categories %} <li> <a href=/rango/category/{{category.slug}}/> {{category.name}} </a> </li> {% endfor %} </ul> Result of Click Python Also note, the link of "Other Frameworks" is actually " o ther f rameworks". 4. Exercises We would like to see the populated data immediately after the request localhost:8000/rango/populate/ as follows. Make it happen. 44

45 It's time to push the project to Github. 45

46 Chapter 7: Fun with Forms Forms are used to gather information from users. Django's form handling functionality allows you to: display an HTML form with automatically generated form widgets (like a text field or date picker) check submitted data against a set of validation rules redisplay a form in case of validation errors convert submitted form data to the relevant Python data types Using Django's form functionality saves a lot of time and you don't have to write too much HTML codes because forms are generated automatically. A form takes the following format: <form method = action=> <input > <input type=submit value=submit> </form> For example: <h1>sign up</h1> <form method=post action=/rango/signup> Username: <input type=text name=account><br> Password: <input type=password name=password><br> Password (enter again): <input type=password name=password2><br> <input type=submit value=submit> </form> Some of the form fields: Text input: <input type=text name=> Password input: <input type=password name=> Text area: <textarea rows= cols=> </textarea> Radio choice: <input type=radio name= value= checked> <input type=radio name= value=> <input type=radio name= value=> In the above, the values of the name has to be the same Check box: <input type=checkbox name= value=> <input type=checkbox name= value= checked> <input type=checkbox name= value=> 46

47 In the above, the values of the name has to be the same Drop down select box: <select name=> <option value=> </option> <option value=> </option> <option value= selected> </option> <option value=> </option> </select> File upload form <form enctype=multipart/form data > <input type=file value=> Image button: <input type=image src=> Hidden: <input type=hidden name= value=> Button: <button> </button> Form submit: <input type=submit value=> 1. Basic workflow Steps of creating a form: 1. Create a file forms.py in the application folder 2. Create a ModelForm class for each model that you wish to represent as a form 3. Customise the forms as you desire 4. Create or update a view to handle the form including displaying the form, saving the form data, and flagging up errors which may occur when the user enters incorrect data (or no data at all) in the form 5. Create or update a template to display the form 6. Add a urlpattern to map to the new view (if you created a new one) 2. Page and Category Forms Create the file rango/forms.py and enter the following contents: from django import forms from rango.models import Page, Category class CategoryForm(forms.ModelForm): name = forms.charfield(max_length=128, help_text="please enter the category name.") views = forms.integerfield(widget=forms.hiddeninput(), initial=0) likes = forms.integerfield(widget=forms.hiddeninput(), initial=0) slug = forms.charfield(widget=forms.hiddeninput(), required=false) class Meta: model = Category fields = ('name',) 47

48 class PageForm(forms.ModelForm): title = forms.charfield(max_length=128, help_text="please enter the title of the page.") url = forms.urlfield(max_length=200, help_text="please enter the URL of the page.") views = forms.integerfield(widget=forms.hiddeninput(), initial=0) class Meta: model = Page exclude = ('category',) In the above, In CategoryForm, fields views and likes are set to be hidden ( forms.hiddeninput() ) because users won't have to fill them or even have to see them. They are here because their values cannot be NULL in the model, so we have to use form fields to populate their values ( initial=0 ). Question : their values are set default ( = 0) in the model, we don't seem to have to include them in the form. Inner class Meta is a class container with some options (metadata) attached to the model. It is anything that s not a field, such as ordering options, available permissions, associated database table name, whether the model is abstract or not, singular and plural versions of the name etc. A model is specified first. Including a model in the Meta class is important, because thus Django may check if the model and the form is consistent Some fields may allow NULL in the model definition, so we don't have to include them in the form It is required to put fields = () or exclude = () in Meta Use fields = ' all ' to include all fields, fields = () to include the needed fields, or exclude = () to exclude certain fields Besides the CharField, IntegerField widget, others are also available, such as Field, ChoiceField, DateField, etc. Check: the official Django documentation on widgets. Check out the official Django documentation on forms for further information. 3. Creating an addcategory view, template, and URL mapping With the CategoryForm defined, we are ready to create a new view to display the form and handle the posting of the form data. (a) Create a view for adding categories Add the following to rango/views.py : 48

49 from rango.forms import CategoryForm def addcategory(request): template = 'rango/addcategory.html' if request.method=='get': return render(request, template, {'form':categoryform()}) # request.method=='post' form = CategoryForm(request.POST) if not form.is_valid(): return render(request, template, {'form':form}) form.save(commit=true) return rango(request) # Call function rango() In the above, First import CategoryForm class and define the template If the HTTP method is GET, return with an empty form CategoryForm(), which is also called an unbound form. Else (the HTTP method is POST ) set form to CategoryForm(request.POST), which is a bound form populated wiith user input data ( request.post ). If the form is not valid, display a form with error messages automatically. Django's form handling not only save the form data, but also check if the form is valid. It will not save an incomplete form. Else (a valid form) save the form data into the model. commit=true means save the data immediately. Finally, invoke the rango view (i.e., return to the rango page). (b) Create a template for adding categories Create rango/templates/rango/addcategory.html : <!doctype html> {% load staticfiles %} <html> <head> <title>tango with Django</title> <meta charset=utf 8> <link rel=stylesheet href={% static 'main/css/main.css' %}> </head> <body> <h1>add a Category</h1> <form id=categoryform method=post action=/rango/addcategory/> {% csrf_token %} {% for hidden in form.hidden_fields %} {{hidden}} {% endfor %} {% for field in form.visible_fields %} {{field.errors}} 49

50 {{field.help_text}} {{field}} {% endfor %} <input type=submit value="create Category"> </form> </body> </html> In the above, {% csrf_token %} : Django form security to prevent Cross Site Request Forgery {% for hidden %} : output the hidden fields {% for field %} : output the visible fields defined in class Meta: fields() Hidden form fields are necessary because HTTP is a stateless protocol, and you can t persist state between different HTTP requests. To overcome this limitation, hidden HTML form fields allow to pass information to a client (which cannot be seen on the rendered page), and to be sent back to the server when the user submits the form. (c) Create a URL mapping Add the following line into rango/urls.py : url(r'^addcategory/$', views.addcategory, name='addcategory'), Now test with localhost:8000/rango/addcategory/. Django generates the following form: <form id=categoryform method=post action=/rango/addcategory/> <input type='hidden' name='csrfmiddlewaretoken' value='34gjrzmxx4xzfbpjojltup46zzaedugk' /> <input id="id_views" name="views" type="hidden" value="0" /> <input id="id_likes" name="likes" type="hidden" value="0" /> <input id="id_slug" name="slug" type="hidden" /> Please enter the category name. <input id="id_name" maxlength="128" name="name" type="text" /> <input type=submit value="create Category"> </form> Django form actions: Generate form automatically Generate CSRF middleware token Loop through invisible and visible fields and ceneratereate corresponding input fields Add an id for each field with the format id_<fieldname> Now, put a link on the rango page so that we can easily add 50

51 categories. Edit rango/templates/rango/rango.html and add the following link before About link: {% else %} <strong>there are no categories present.</strong> {% endif %} <br> <a href=/rango/addcategory/>add a New Category</a><br><br> <a href=/rango/about/>about</a><br><br> Result: click Add a New Category Enter Ruby on Rails and click Create Category 4. Cleaner forms Purpose: to help users to enter form data with correct formats. Add a clean() function to class PageForm() in rango/forms.py file to check if the URL is in the correct format (should start with ) class PageForm(forms.ModelForm): class Meta: def clean(self): cleaneddata = self.cleaned_data url = cleaneddata.get('url') 51

52 if url and not url.startswith(' url = ' + url cleaneddata['url'] = url return cleaneddata In the above: Our clean() method overrides Django form's default clean() method Form data is obtained from the ModelForm dictionary attribute cleaned_data Use get() method to obtain a particular field Use startswith() methd to detect if the string matches what we specified If the data format is incorrect, correct it Finally return the cleaned result 5. Exercise: creating an addpage view, template, and URL mapping (a) Create a view for adding pages Create a new view addpage() in /rango/views.py, which takes two parameters as inputs: categoryname and pagename. from rango.forms import CategoryForm, PageForm def addpage(request, categorynameslug): template = 'rango/addpage.html' try: cat = Category.objects.get(slug=categoryNameSlug) except Category.DoesNotExist: return category(request, categorynameslug) context = {'category':cat} if request.method=='get': context['form'] = PageForm() return render(request, template, context) # request.method=='post' form = PageForm(request.POST) context['form'] = form if not form.is_valid(): return render(request, template, context) page = form.save(commit=false) page.category = cat page.views = 0 page.save() return category(request, categorynameslug) commit=false means don't actually save the object (i.e., page ) into the database (something more needs to be set), just obtain an object for now. Django will delay saving it until page.save() is called. After performing necessary setting ( page.category, 52

53 page.view s), finally call page.save() to actually save it. (b) Create a template for adding pages Create a new template /rango/templates/rango/addpage.html : <!doctype html> {% load staticfiles %} <html> <head> <title>tango with Django</title> <meta charset=utf 8> <link rel=stylesheet href={% static 'main/css/main.css' %}> </head> <body> <h1>add a new page in {{category.name}}</h1> <form method=post action=/rango/{{category.slug}}/addpage/> {% csrf_token %} {% for hidden in form.hidden_fields %} {{hidden}} {% endfor %} {% for field in form.visible_fields %} {{field.errors}} {{field.help_text}} {{field}}<br> {% endfor %} <input type=submit value="create Page"> </form> </body> </html> (c) Create a URL mapping Add the following lines in category.html for the user to add a new page: <p><strong>no pages currently in category.</strong></p> {% endif %} <br><br> <a href="/rango/{{category.slug}}/addpage/">add a New Page</a><br><br> {% else %} <p>the specified category "{{categoryname}}" does not exist.</p> {% endif %} Add the following line into rango/urls.py : url(r'^(?p<categorynameslug>[\w\ ]+)/addpage/$', views.addpage, name='addpage'), Now, test the app 53

54 Click Django Click Add a New Page Eenter Writing you first Django app and click Create Page Take a good look at: Working with forms Again, it's time to push the project to Github. 54

55 Chapter 8: User Authentication The auth application in Django's django.contrib.auth package handles user authentication. It consists of the following aspects: Users Permissions: binary flags (e.g. yes/no) determining what a user may or may not do Groups: a method of applying permissions to more than one user A configurable password hashing system: a must for ensuring data security Forms and view tools for logging in users, or restricting content A pluggable backend system, allowing you to provide your own authentication related functionality 1. Setting up authentication Make sure that django.contrib.auth and django.contrib.conttenttypes are in the INSTALLED_APP in settings.py. In Django, passwords are stored using PBKDF2 algorithm 2. The User model The main object for authentication is the User model, which contains the following fields: username password first_name last_name groups user_permissions is_staff is_active is_superuser last_login date_joined For this project, we need two additional fields: URLField and ImageField for a user. Modify rango/models.py : from django.contrib.auth.models import User class UserProfile(models.Model): user = models.onetoonefield(user) # The additional attributes we wish to include. 55

56 website = models.urlfield(blank=true) picture = models.imagefield(upload_to='profileimages', blank=true) def str (self): return self.user.username One UserProfile model maps to only one User model, based on the one to one relationship. Setting blank=true allows the field to be empty. To use ImageField, we have to install Pillow : (venv)$ pip3 install Pillow The upload_to attribute takes the value of MEDIA_ROOT specified in settings.py. Add the UserProfile model into rango/admin.py such that it is modifiable in the administration interface. from rango.models import Category, Page, UserProfile admin.site.register(userprofile) Now, perform makemigrations : Migrations for 'rango': 0005_userprofile.py: Create model UserProfile and migrate : Operations to perform: Synchronize unmigrated apps: messages, staticfiles Apply all migrations: sessions, auth, admin, rango, contenttypes Synchronizing apps without migrations: Creating tables Running deferred SQL Installing custom SQL Running migrations: Rendering model states DONE Applying rango.0005_userprofile OK 3. Creating a user registration view and template We would like to let users register to our site. Steps: 1. Create a UserForm and UserProfileForm 2. Add a view to handle the creation of a user 3. Create a template that displays the UserForm and UserProfileForm 4. Map a URL 5. Link the rango page to the register page 56

57 (a) Create UserForm and UserProfileForm Create two classes in rango/forms.py : from django.contrib.auth.models import User from rango.models import Page, Category, UserProfile class UserForm(forms.ModelForm): password = forms.charfield(widget=forms.passwordinput()) class Meta: model = User fields = ('username', ' ', 'password') class UserProfileForm(forms.ModelForm): class Meta: model = UserProfile fields = ('website', 'picture') The UserForm takes three fields, username, , and password, from the User model. And a widget ( forms.passwordinput() ) is specified for field password. The UserProfileForm takes two fields, website and picture, from the UserProfile model. (b) Create the register() view There are two form class in our registration form. In rango/views.py, add the following: from rango.forms import UserForm, UserProfileForm def register(request): template = 'rango/register.html' if request.method=='get': return render(request, template, {'userform': UserForm(), 'profileform': UserProfileForm(), 'registered': False}) # request.method == 'POST': userform = UserForm(request.POST) profileform = UserProfileForm(request.POST) if not (userform.is_valid() and profileform.is_valid()): #print(userform.errors, profileform.errors) return render(request, template, 57

58 {'userform': userform, 'profileform': profileform, 'registered': False}) # Save the user's form data to the database user = userform.save() user.set_password(user.password) user.save() profile = profileform.save(commit=false) profile.user = user if 'picture' in request.files: profile.picture = request.files['picture'] profile.save() return render(request, template, {'userform': userform, 'profileform': profileform, 'registered': True}) In the above: If the request method is GET, return an unbound form containing two form classes, UserForm and UserProfileForm, and a flag registered. Else, bound the two form classes. Test if both of the bound forms are valid, if not return the bound forms Else, save the userform form, following by encypting the password by method set_password() and another save. Next, save userprofileform without commit, followed by setting the OneToOne field and uploaded file if necessary. And then, save again. Finally, return the bound forms. (c) Create the registration template Create a new template rango/templates/rango/register.html : <!doctype html> {% load staticfiles %} <html> <head> <title>tango with Django</title> <meta charset=utf 8> <link rel=stylesheet href={% static 'main/css/main.css' %}> </head> <body> <h1>register with Tango</h1> {% if registered %} <p><strong>thank you for registering!</strong><br> <a href=/>return to the homepage.</a></p> {% else %} <p><strong>register here!</strong></p> <form id=userform method=post action=/rango/register/ enctype=multipart/form data> {% csrf_token %} {{ userform.as_p }} 58

59 {{ profileform.as_p }} <input type=submit name=submit value=register> </form> {% endif %} </body> </html> In the above: If the registered successfully, print "thank you" message and provide a link to the main page Else, provide a registration form, the user may upload an image, so the form has to have the enctype=multipart/form data attribute. The method.as_p sets the form in a <p> element so every field is in a paragraph. (d) Map the URL In rango/urls.py, add the new mapping: url(r'^register/$', views.register, name='register'), (e) Create a link in rango.html : <a href=/rango/addcategory/>add a New Category</a><br><br> <a href=/rango/register/>register Here</a><br><br> <a href=/rango/about/>about</a><br><br> Now test the system and click Register Here. There are Chinese characters because we set LANGUAGE_CODE = 'zh tw' in settings.py and Django's model User has internationalization (I18N) property. If we set LANGUAGE_CODE = 'en', then everything will be English. 4. Add login functionality Once users may register our site, we need to provide a login page. Steps: 59

60 1. Create a login view 2. Create a login template 3. Map the URL 4. Provide a login link (a) Create the login view Add the userlogin() view in rango/views.py : from django.http import HttpResponse, HttpResponseRedirect from django.contrib.auth import authenticate, login def userlogin(request): if request.method=='get': return render(request, 'rango/userlogin.html') # request.method=='post': username = request.post['username'] password = request.post['password'] user = authenticate(username=username, password=password) if not user: return HttpResponse('Invalid login details supplied.') if not user.is_active: return HttpResponse('Your Tango account is disabled.') login(request, user) return HttpResponseRedirect('/') In the above, If the request method is GET, return the template Get the POST data: username and password Authenticate the user with function authentication() : see if the user has already registered and if the password is correct Something wrong with logging in: return and show the corresponding message Everything is OK: login the user using function login() and redirect to the main page (b) Create the template rango/templates/rango/userlogin.html: <!doctype html> {% load staticfiles %} <html> <head> <! Is anyone getting tired of repeatedly entering the header over and over?? > <title>tango with Django</title> <meta charset=utf 8> <link rel=stylesheet href={% static 'main/css/main.css' %}> </head> <body> <h1>login to Tango</h1> <form id=loginform method=post action=/rango/userlogin/> 60

61 {% csrf_token %} Username: <input type=text name=username size=50> <br> Password: <input type=password name=password size=50> <br> <input type=submit value=submit> </form> </body> </html> In the above, Display a login form with two input fields: username and password. (c) Map the URL in rango/urls/py : url(r'^userlogin/$', views.userlogin, name='userlogin'), (d) Create a login link in rango.thml and change the greeting message at the top: <body> {% if user.is_authenticated %} <h1>rango says hello {{user.username}}!</h1> {% else %} <h1>rango says hello world!</h1> {% endif %} <a href=/rango/register/>register Here</a><br><br> <a href=/rango/userlogin/>login</a><br><br> <a href=/rango/about/>about</a><br><br> In the above, Use Django template statement user.is_authenticated() to test if the user is logged in and display different welcome message Provide a link for login Now test the system and click Login. 5. Restrict access Restrict some part of the application as per the specification: only registered users can add categories and pages. There are two ways to check if the user is authenticated: examining the request directly or using a decorator function. 61

62 (a) Examine the request object to check if the user is authenticated Check to see if the user is logged in via the user.is_authenticated() method. For example: def someview(request): if request.user.is_authenticated(): return HttpResponse('You are logged in.') return HttpResponse('You are not logged in.') (b) Use Python decorator to check if the user is authenticated Python decorators can dynamically alter the functionality of a function, method, or class without having to change the source code. Django provides a decorator called login_required() which can be attached to a view that requires the user to be logged in. Add the following to rango() in rango/views.py : from django.contrib.auth.decorators import def restricted(request): return HttpResponse("Since you're logged in, you can see this text!") The decorator is placed directly above the function signature, and is put before the decorator name. Python will execute the decorator before executing the decorated function/method. Add an url pattern in rango's urls.py : urlpatterns = patterns('', url(r'^restricted/', views.restricted, name='restricted'), ) For an un loggend user who attemps to acces the restricted() view, we will redirect him/her to the login page. Add the following line in settings.py so that login_required() decorator will redirect not logged user to the URL /rango/userlogin/. LOGIN_URL = '/rango/userlogin/' 6. Log out Django has a built in logout() function. Add a view called userlogout() in rango/views.py file with the following: from django.contrib.auth import authenticate, login, 62

63 def userlogout(request): logout(request) return HttpResponseRedirect('/') Map the URL: url(r'^userlogout/', views.userlogout, name='userlogout'), Now provide a logout link for the logged in user. Modify rango/templates/rango/rango.html : to check if user is logged in. If yes, provide a logout link, otherwise, provide a login link. <a href=/rango/addcategory/>add a New Category</a><br><br> {% if user.is_authenticated %} <a href=/rango/restricted/>restricted Page</a><br><br> <a href=/rango/userlogout/>logout</a><br><br> {% else %} <a href="/rango/register/">register Here</a><br><br> <a href=/rango/userlogin/>login</a><br><br> {% endif %} <a href=/rango/about/>about</a><br><br> In the above, If user is logged in, present Restricted Page and Logout links Else, present Register Here and Login links Now test the system: (1) Login and enter in the browser's URL field. (2) Logout and enter in the browser's URL field. 7. Create a superuser with an initialization view Normally, a superuser has to be created to manage the data and this can be done by the following as we have done before: $ python manage.py createsuperuser The superuser can also be created during the system deployment. The procedure of system deployment could be as follows: (a) Copy all system programs and data to the production server (b) Collect static data and copy them to the destination for the static file server (c) Run the init() app to create necessary initial data, including the admin account (d) Set DEBUG = False in settings.py Implement the init() app: 63

64 In main.views.py, add the init() fucntion: import os from django.contrib.auth.models import User from rango.models import Category def main(request): def init(request): if User.objects.filter(username='admin'): return render(request, 'main/init.html', {'abort':true}) # Create 'admin' account User.objects.all().delete() user = User() user.username = 'admin' user.set_password('admin') user.is_staff = True user.is_superuser = True user.is_active = True user.save() # Init Category Category.objects.all().delete() infile = open(os.path.dirname(os.path.realpath( file ))+'/category.txt', 'r') for line in infile: category = Category() category.name = line category.save() infile.close() # Init other data # Retrieve everything users = User.objects.all() categories = Category.objects.all() context = {'users':users, 'categories':categories} return render(request, 'main/init.html', context) # end of init() In the above, Check if admin is already in the User model, if yes, don't initialize the project Delete everything in User model and then create an admin user Delete everything in the models that we would like to initialize, e.g., Category, and then populate the initial data using the data from a file we provide ( category.txt ). The format of the categories in category.txt should be one category per line, for example: 64

65 Django Python Other frameworks Finally, after initialization, retrieve everything and display them on the page init.html. Create a template main/templates/init.html : <!doctype html> {% load staticfiles %} <html> <head> <title>tango with Django</title> <meta charset=utf 8> <link rel=stylesheet href={% static 'main/css/main.css' %}> </head> <body> {% if abort %} <h1>initialization aborted.</h1> {% else %} <h1>initial data:</h1> <h1>users:</h1> {% for user in users %} {{user.username}}<br> {% endfor %} <h1>categories:</h1> {% for category in categories %} {{category.name}}<br> {% endfor %} {% endif %} </body> </html> Add another URL mapping in the project's tango/urls.py (again, url(r'^.*', ) has to be at the last line): url(r'^rango/', include('rango.urls')), url(r'^main/', include('main.urls')), url(r'^.*', include('main.urls')), Add a URL mapping in main/urls.py : urlpatterns = patterns('', url(r'^init/$', views.init, name='init'), url(r'^.*', views.main, name='main'), ) Test by entering the url in the browser: 8. Exercises 65

66 (a) Write codes such that only registered users can add/edit, while non registered can only view/use the categories/pages. Add decorator on addpage() and addcategory() views. In category.html, check if the user is logged in. If yes, show the Add a Page link; otherwise show nothing: {% if user.is_authenticated %} <a href="/rango/{{ categoryname }}/add_page/">add a Page</a> {% endif %} (b) Provide informative error messages when users incorrectly enter their username or password. We shouldn't really want to do this! However, since the tutorial ask us to do it, here we go. from django.contrib.auth.models import User # The following try: user = User.objects.get(username=username) except User.DoesNotExist: return HttpResponse('Invalid username: {0}'.format(username)) user = authenticate(username=username, password=password) if not user: return HttpResponse('Invalid password: {0}'.format(password)) if not user.is_active: return HttpResponse("Your Rango account is disabled.") login(request, user) return HttpResponseRedirect('/rango/') (c) Visit registration.readthedocs.org/en/latest/ to find out more about using the user registration package. Now push your project. 66

67 Chapter 9: Working with Templates There are a lot of same HTML codes in various HTML files. We will use templates to minimize the copy and paste of HTML codes. Steps 1. Identify the re occurring parts of each HTML pages (i.e. header bar, sidebar, footer, content pane). 2. In a base template, provide the skeleton structure of a standard page along with any common content. And then define a number of blocks which are subject to change depending on which page the user is viewing. 3. Create specific templates which inherit from the base tempalte, and specify the contents of each block. 1. Re occuring HTML and the base template The common part (color blue) of various templates in the project: <!doctype html> {% load staticfiles %} <html> <head> <title>tango with Django</title> <meta charset=utf 8> <link rel=stylesheet href={% static 'main/css/main.css' %}> </head> <body> <! Page specific content goes here > </body> </html> These common part should not be inherited. Supposing a template base.html has the following contents: <!doctype html> {% load staticfiles %} <html> <head> <title>tango with Django</title> <meta charset=utf 8> <link rel=stylesheet href={% static 'main/css/main.css' %}> </head> <body> {% block content %}{% endblock %} </body> </html> In the above, a template block is denoted by {% block <blockname> %}, where <blockname> is the name of the block, and {% endblock %} is an ending tag. You may also include default content of the block, for example: 67

68 {% block content %} This is the default content. {% endblock %} If the content of the {% block content %} is not replaced with anything, it's result will be empty: <body> </body> </html> If a template inherits from base.html, it can supply the content of block content. For example, the content of index.html : {% extends "main/base.html" %} {% block content %}<p>this is the text to insert.</p>{% endblock %} The resulting page would be: <!doctype html> {% load staticfiles %} <html> <head> <title>tango with Django</title> <meta charset=utf 8> <link rel=stylesheet href={% static 'main/css/main.css' %}> </head> <body> <p>this is the text to insert.</p> </body> </html> 2. More blocks Modify main/templates/main/main.html with the following contents: <!doctype html> {% load staticfiles %} <html> <head> <title>tango with Django</title> <meta charset=utf 8> <link rel=stylesheet href={% static 'main/css/main.css' %}> </head> <body> <h1>tango with Django{% block heading %}{% endblock %}</h1> <div> {% block content %}{% endblock %} </div> <br> <hr> <div> <ul> {% if user.is_authenticated %} <li><a href=/rango/restricted/>restricted Page</a></li> 68

69 <li><a href=/rango/userlogout/>logout</a></li> <li><a href=/rango/addcategory/>add a New Category</a></li> {% else %} <li><a href=/rango/register/>register Here</a></li> <li><a href=/rango/userlogin/>login</a></li> {% endif %} <li><a href=/rango/about/>about</a></li> <li><a href=/rango/>back to Rango</a></li> </ul> </div> </body> </html> In the above: The first block is called heading without default content The second block is called content without default content A series of links are at the bottom of the page 3. Template inheritance Since main.html is ready, other templates can inherit from it. For example, we can modify category.html as follows. The beginning part: <!doctype html> {% load staticfiles %} <html> <head> <title>tango with Django</title> <meta charset=utf 8> <link rel=stylesheet href={% static 'main/css/main.css' %}> </head> <body> <h1>tango with Django {% block heading %}{% endblock %}</h1> is replace by: {% extends "main/main.html" %} {% block content %} And the ending part: </body> </htm> is replace by: {% endblock %} The resulting code becomes: {% extends "main/main.html" %} 69

70 {% block heading %} List Pages{% endblock %} {% block content %} {% if category %} <h1>{{category.name}}</h1> {% if pages %} <ul> {% for page in pages %} <li><a href={{page.url}}>{{page.title}}</a></li> {% endfor %} </ul> {% else %} <p><strong>no pages currently in category.</strong></p> {% endif %} <br><br> <a href="/rango/{{category.slug}}/addpage/">add a New Page</a><br><br> {% else %} <p>the specified category "{{categoryname}}" does not exist.</p> {% endif %} {% endblock %} Important! In main.html, there is user.is_authenticated statement, which means main.html has to have access to the content of the request. Since all other templates now inherit from main.html, that means all HTMLs have to have access to the request. Using render(request, ) in the views can achieve that, however, HttpResponse() will not. Therefore, HttpResponse() should not be used anymore. Modify userlogin() in rango/views.py : def userlogin(request): template = 'rango/userlogin.html' if request.method == 'GET': return render(request, template ) if not user: return render(request, template, {'errormessage':'incorrect login information. Try again.'}) return HttpResponse("Invalid login details supplied.") if not user.is_active: return render(request, template, {'errormessage':'the user is diabled. Please contact the site manager.'}) return HttpResponse("Your Rango account is disabled.") Add a line for displaying an error message in userlogin.html : <h1>login to Rango</h1> {{errormessage}}<br><br> <form id=loginform method=post action=/rango/userlogin/> 70

71 4. Further works (a) Update all other templates to extends from main/main.html. about.html : {% extends 'main/main.html' %} {% block heading %} About{% endblock %} {% block content %} <h2> Rango Says: Here is the about page. </h2> {% endblock %} AddCategory.html : {% extends 'main/main.html' %} {% block heading %} Add a Category{% endblock %} {% block content %} <h2> Add a Category </h2> <form id=categoryform method=post action=/rango/addcategory/> {% csrf_token %} {% for hidden in form.hidden_fields %} {{hidden}} {% endfor %} {% for field in form.visible_fields %} {{field.errors}} {{field.help_text}} {{field}} {% endfor %} <input type=submit value="create Category"> </form> {% endblock %} addpage.html : {% extends 'main/main.html' %} {% block heading %} Add a Page{% endblock %} {% block content %} <h2> Add a new page in {{category.name}} </h2> <form method=post action=/rango/{{category.slug}}/addpage/> {% csrf_token %} {% for hidden in form.hidden_fields %} {{hidden}} {% endfor %} {% for field in form.visible_fields %} {{field.errors}} {{field.help_text}} {{field}}<br> {% endfor %} <input type=submit value="create Page"> </form> 71

72 {% endblock %} category.html : {% extends "main/main.html" %} {% block heading %} List Pages{% endblock %} {% block content %} {% if category %} <h2> {{category.name}} </h2> {% if pages %} <ul> {% for page in pages %} <li><a href={{page.url}}>{{page.title}}</a></li> {% endfor %} </ul> {% else %} <p><strong>no pages currently in category.</strong></p> {% endif %} <br><br> <a href="/rango/{{category.slug}}/addpage/">add a New Page</a><br><br> {% else %} <p>the specified category "{{categoryname}}" does not exist.</p> {% endif %} {% endblock %} rango.html : {% extends "main/main.html" %} {% block heading %} Rango{% endblock %} {% block content %} {% if user.is_authenticated %} <h2> Rango says hello {{user.username}}! </h2> {% else %} <h2> Rango says hello world! </h2> {% endif %} {% if categories %} <ul> {% for category in categories %} <li><a href=/rango/category/{{category.slug}}/>{{category.name}}</a></li> {% endfor %} </ul> {% else %} <p><strong>there are no categories present.</strong></p> {% endif %} <br> <a href=/rango/addcategory/>add a New Category</a><br><br> {% endblock %} 72

73 register.html : {% extends "main/main.html" %} {% block heading %} Register{% endblock %} {% block content %} <h2> Register with Tango </h2> {% if registered %} <p><strong>thank you for registering!</strong><br> <a href=/>return to the homepage.</a></p> {% else %} <p><strong>register here!</strong></p> <form id=userform method=post action=/rango/register/ enctype=multipart/form data> {% csrf_token %} {{ userform.as_p }} {{ profileform.as_p }} <input type=submit name=submit value=register> </form> {% endif %} {% endblock %} userlogin.html : {% extends 'main/main.html' %} {% block heading %} Login{% endblock %} {% block content %} <h2> Login to Tango </h2> {{errormessage}}<br><br> <form id=loginform method=post action=/rango/userlogin/> {% csrf_token %} Username: <input type=text name=username size=50> <br> Password: <input type=password name=password size=50> <br> <input type=submit value=submit> </form> {% endblock %} Now push your project. 73

74 2 Chapter 11: Deploy to Heroku The current project layout under GitHub's version control is as follows: project/ git/ tango/.git/ tango/ tango/.gitignore manage.py virtualenv/ tangovenv/ workspace/ 1. Install django toolbelt for Heroku Install system toolbelt package on Ubuntu: $ wget qo ubuntu.sh sh Create a new virtual environment herokuvenv : $ cd <>/project/virtualenv $ virtualenv p /usr/bin/python3 herokuvenv $ cd herokuvenv $ source bin/activate (venv)$ pip3 install django (venv)$ pip3 install django toolbelt $ pip3 freeze Django==1.8.2 dj database url==0.3.0 dj static==0.0.6 django toolbelt==0.0.1 gunicorn== psycopg2==2.6.1 static3==0.6.1 Note: django toolbelt includes posgres Install a new python interpreter in Eclipse: Window Preferences (Left panel) Expand PyDev Expand 2 Source: started with django 74

75 Interpreters Python Interpreter (Right panel) New Interpreter Name: herokupython3, Interpreter Executable: <>/project/virtualenv/herokuvenv/bin/python3 OK OK OK Set the new interpreter for the tango project: Right click project Properties (Left panel) PyDev Interpreter/Grammar (Right panel) Interpreter: herokupython3 OK 2. Create files to specify the runtime environment for Heroku Create the following 3 files in the project root directory : Specify the application server: Procfile web: gunicorn tango.wsgi log file Specify the virtual environment: requirements.txt by: (venv)$ pip3 freeze > requirements.txt Django==1.8.2 dj database url==0.3.0 dj static==0.0.6 django toolbelt==0.0.1 gunicorn== psycopg2==2.6 static3==0.6.0 Specify the runtime Python version: runtime.txt python The project layout under git directory becomes: project/ git/ tango/.git/ tango/ tango/.gitignore manage.py Procfile requirements.txt runtime.txt 3. Configure settings.py for Heroku Edit settings.py 75

76 DEBUG = True if 'DYNO' in os.environ: DEBUG = False ALLOWED_HOSTS = [ '*' ] # Running on Heroku # Database # if DEBUG==True: # Running on the development environment DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'tangodb', 'USER': 'tango', 'PASSWORD': 'xxxxxxxx', 'HOST': ' ', 'PORT': '', # Set to empty string for default. } } else: # Running on Heroku # Parse database configuration from $DATABASE_URL import dj_database_url DATABASES = {'default':dj_database_url.config()} # Honor the 'X Forwarded Proto' header for request.is_secure() SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https') Edit wsgi.py: application = get_wsgi_application() if 'DYNO' in os.environ: # Running on Heroku from dj_static import Cling application = Cling(get_wsgi_application()) else: application = get_wsgi_application() Now, test in the development enrironment: Note : Heroku requires that a Django project have at least one user created app. 4. Register a Heroku account and generate ssh2 credentials Register a Heroku account: 76

77 ID: PSW: xxxxxxxx Heroku ssh2 credentials: Window Preferences General Network Connections SSH2 (Right) Key Management tab Generate RSA Key Save Private Key OK Name: id_rsa OK OK ( OK) Add key to Heroku: $ heroku login ID: xxxxxxxx PSW: xxxxxxxx $ heroku keys:add Found an SSH public key at <usrhome>/.ssh/id_rsa.pub Would you like to upload it to Heroku? [Yn] y Uploading SSH public key <usrhome>/.ssh/id_rsa.pub done 5. Perform local makemigrations: $ cd <project root> $ source <venv>/bin/activate (venv)$ python3 manage.py makemigrations Database migrate will be performed remotely on Heroku. 6. Create a deployment folder, create a new Git repository, and deploy the project Because Heroku also uses Git for file upload, we need to clone the project and create a new Git repository in order not to interfere with the original. 1. Create a new folder <>/project/deploy 2. Clone the project 3. Create a git repository in the Git project root $ cd <>/project $ mkdir deploy $ cd deploy $ cp r <projgitroot>. Make sure the content of.gitignore is as follows:.* 77

78 *~ *.pyc Now the project layout becomes: <>/project/ git/ tango/.git/ tango/ manage.py deploy/ tango/ tango/.git/ manage.py virtualenv/ herokuvenv/ workspace/ Perform Git init and commit: $ cd tango $ git init $ git add. $ git commit m "First push" Login to heroku: $ heroku login Enter your Heroku credentials. xxxxxx@xxx.xxx Password (typing will be hidden): xxxxxxxx Authentication successful. Create a Heroku app: $ heroku create <appname> (E.g., <appname>=tango12345 as tango may not be available) This command also creates a config file in.git directory with the content: [core] repositoryformatversion = 0 filemode = true bare = false logallrefupdates = true [remote "heroku"] url = <appname>.git fetch = +refs/heads/*:refs/remotes/heroku/* Push to Heroku: $ git push heroku master 78

79 On the Heroku's Resources page, check to see if a postgres database is created. Perform migrate remotely on Heroku: $ heroku run python3 manage.py migrate Set Heroku dyno (Heroku may have done this automatically): $ heroku ps:scale web=1 Create the superuser: $ heroku run python3 manage.py createsuperuser Test: 7. Get more addons On Heroku's Resources page, click "Get more addons" and add the following addons: (1) Data Store Utilities: PostgreSQL Studio (beta) (2) Monitoring: New Relic APM (3) Logging: Papertrail (4) PostgreSQL Studio 8. Other Heroku commands (a) Clear database $ heroku pg:reset DATABASE_URL (b) Check logs $ heroku logs tail (b) Run bash $ heroku run bash 9. Deployment scripts The process of deploying to Heroku is very complex, so it's better to automate it using a script. 79

80 There are two types of deployment: Clean deploy: deletes all migration information and data tables on Heroku Follow up deploy: retains all migration information and data tables on Heroku Before deployment: 1. Create the app on Heroku if it is the first deploy 2. Login heroku 3. Make sure the contents of.gitignore, requirements.txt, and runtime.txt are correct.gitignore Clean Deploy.* *~ *.pyc 00*_* Follow up deploy.* *~ *.pyc Run the following deploy script (set file mode to executable: $ chmod +x deploy ) #!/bin/bash # # Deploy the project to Heroku. # # Prerequisites: # 1. The heroku app has already been created # 2. Login heroku # 3. Make sure the contents of ".gitignore", "requirements.txt", and # "runtime.txt" are correct # # Project layout: # project/ # deploy/ # tango/ # tango/ #.git/ # tango/ # manage.py # deploy # git/ # tango/ #.git/ # tango/ # tango/ # manage.py # virtualenv/ # herokuvenv/ # bin/ # activate 80

81 # workspace/ # # Variables (no spaces before or after "="): # projname=<name of the project> # appname=<name of the app on Heroku> # root=<>/project # projroot=$root/git/$projname/$projname # activate=<path of the activate file> # projname=tango appname=tango12345 root=<>/project deploy=$root/deploy/$projname projroot=$root/git/$projname/$projname activate=$root/virtual.env/herokuvenv/bin/activate read p "Perform (1)follow up or (2)clean deployment? (1/2)? " deploytype if [ "$deploytype" = "1" ]; then cleandeploy=false else if [ "$deploytype" = "2" ]; then echo e "\ndanger!! All data on Heroku will be deleted!" read p "Are you ABSOLUTELY sure (yes)? " yes if [ "$yes" = "yes" ]; then cleandeploy=true else echo e "Abort.\n" exit fi else echo e "Abort.\n" exit fi fi if [! f requirements.txt ]; then echo e "File 'requirements.txt' does not exit." exit fi if [! f runtime.txt ]; then echo e "File 'runtime.txt' does not exit." exit fi if [! f gitignore ]; then echo e "File 'gitignore' does not exit." exit fi echo e "\nclone the project." if [! d $projroot ]; then echo e "$projroot does not exist.\n" exit fi rm rf./$projname cp r $projroot. 81

82 cd $projname cp../requirements.txt. cp../runtime.txt. cp../gitignore.gitignore if [ "$cleandeploy" = true ]; then echo e "\ndelete '00*_*' files." find. name "00*_*" exec rm {} \; fi echo e "\nrun 'makemigrations'." source $activate python3 manage.py makemigrations echo e "\ncreate git repository " rm rf.git git init git remote add heroku echo e "\ncommit all files " git add. git commit m "Deploying" echo e "\npush to heroku " git push force heroku master if [ "$cleandeploy" = true ]; then echo e "\nclear Heroku's database " echo $appname heroku pg:reset DATABASE_URL fi echo e "\nrun 'migrate' on Heroku " heroku run python3 manage.py migrate if [ "$cleandeploy" = true ]; then echo e "\nset 'web dyno'=1. Adjust if necessary." heroku ps:scale web=1 fi echo e "\ndeployment completed." After the system goes production, when a model needs to be changed, make sure only one member may make the change at a time. Follow the steps below: 1. Make sure no other member is making any model changes 2. Change the model and perform makemigrations 3. Push to Git as soon as possible (include all 00*_* migration files) 4. Other members pull form Git (also include all 00*_* migration files) 5. Members perform migrate on his/her machine 6. And then proceed to work on other code 82

83 83

84 Chapter 10: Deploying to an Ubuntu 3 server 1. Install database and vsftpd on the production server Install postgres on Ubuntu using the native package: $ sudo apt get install libpq dev postgresql postgresql contrib Install vsftpd: $ sudo apt get install vsftpd Edit the setting file: /etc/vsftpd.conf anonymous_enable=no local_enable=yes write_enable=yes chroot_local_user=yes 2. Why use Nginx as a reverse proxy in front of an application server (Gunicorn)? Many frameworks and application servers (including Gunicorn) can serve static files (e.g. javascript, css, images etc.) together with responses. However, the better thing to do is to let a (reverse proxy) server such as Nginx handle the task of serving these files and managing connections (requests). This relieves a lot of the load from the application servers, granting you a much better overall performance. As your application grows, you will want to optimize it and when the time comes, distribute it across servers (VPS) to be able to handle more connections simultaneously (and have a generally more robust architecture). Having a reverse proxy in front of your application server(s) helps you with this from the very beginning. Example of a basic server architecture: 3 Source: 1. up an FTP Server in Ubuntu L inux 2. nginx and gunicornconfiguration for djang/ 84

85 3. Create the virtual environment and install Gunicorn and Nginx on the production server On the development computer, create requirements.txt. $ cd <venv> $ source bin/activate (venv)$ pip3 freeze > requirements.txt On the production server, install python dev, pip, and virtualenv $ sudo apt get update $ sudo apt get y upgrade $ sudo apt get install python3 dev $ sudo apt get install python3 pip $ sudo pip3 install virtualenv We would like to arrange the directory structure as follows: /webapps /tango /logs /venv /tango manage.py /main /rango /tango Now create the directory structure: $ sudo mkdir /webapps $ sudo chown <loginuser> /webapps $ sudo chgrp <loginuser> /webapps $ cd /webapps $ mkdir tango $ cd tango $ mkdir logs $ mkdir tango $ virtualenv p /usr/bin/python3 venv $ cd venv Copy requirements.txt from the development platform to /webapps/tango. Clone the development environment: 85

86 $ source <venv>/bin/activate (venv)$ pip3 install r requirements.txt Install Gunicorn: (venv)$ pip3 install gunicorn Now, copy the entire project to /webapps/tango/tango (or ftp the project files). Turn off debugging in /webapps/tango/tango/tango/settings.py : DEBUG = False TEMPLATE_DEBUG = False 4. Set up the database $ sudo i u postgres postgres@<username> $ createdb "tangodb" postgres@<username> $ createuser P tango Enter password for new role: xxxxxxxx Enter it again: xxxxxxxx postgres@<username>$ psql postgres=# grant all privileges on database "tangodb" to tango; 5. Serving Python Web Applications with Gunicorn Web Server Gunicorn is a WSGI compliant server and you need to pass your application object to it. Django provides an application object wsgi.py. Run gunicorn by passing it the wsgi application and set the port to 8001: $ source <venv>/bin/activate (venv)$ cd /webapps/tango/tango (venv)$ gunicorn tango.wsgi:application bind=localhost:8001 & To stop Gunicorn, use the following command: $ killall gunicorn To further enhance the server, we may create multiple workers. We don't want the server to be trapped in I/O bound processes. Therefore, when a worker is waiting for I/O, another server may take over other requests (see here ). For example: (venv)$ gunicorn tango.wsgi:application bind=localhost:8001 workers=9 & The suggested amount of workers is set according to the following simple formula: Total workers = (2 workers * CPU cores) + 1 For example: For 1 core (2*1) + 1 = 3 86

87 For 2 cores (2*2) + 1 = 5 For 4 cores (2*4) + 1 = 9 If access logs are necessary: (venv)$ cd /webapps/tango/tango (venv)$ gunicorn tango.wsgi:application bind=localhost:8001 workers=5 access logfile /webapps/tango/logs/gunicornaccess.log & If error logs are necessary: (venv)$ cd /webapps/tango/tango (venv)$ gunicorn tango.wsgi:application bind= :8001 workers=5 access logfile /webapps/tango/logs/gunicornaccess.log log file /webapps/tango/logs/gunicornerror.log & If daemon is necessary: (venv)$ gunicorn tango.wsgi:application bind= :8001 workers=5 access logfile /webapps/tango/logs/gunicornaccess.log log file /webapps/tango/logs/gunicornerror.log daemon Test: : Note, the page is served without static files because we haven't run nginx yet. 6. Install and configure Nginx (Read this ) Install Nginx: $ sudo apt get install nginx Nginx acts as a reverse proxy. All the requests will initially come to Nginx and it will decide what requests it wants to serve and what requests it wants to pass to some other server. In our case, we 87

88 want Nginx to serve requests for static resources and to pass any other request to Gunicorn. We need to tell nginx the location of our static resources and for that we need to make sure all our static resources are at a single location: /webapps/tango/ static : /main /css /img /js /rango /css /img /js Create a file /etc/nginx/sites enabled/tango : $ cd /etc/nginx/sites enabled $ sudo vi tango Enter the following content: access_log /webapps/tango/logs/nginxaccess.log; error_log /webapps/tango/logs/nginxerror.log; server { listen xxx.xxx.xxx:80 ; location / { proxy_pass ; } location /static/ { root /webapps/tango/ ; } } where xxxx.xxx.xxx is a real domain name. Or, enter the following content ( not tested yet ): access_log /webapps/tango/logs/nginxaccess.log; error_log /webapps/tango/logs/nginxerror.log; server { listen 80; server_name my.domain.name; location / { proxy_pass ; } location /static/ { alias /webapps/tango/static/ ; } } Modify privileges: $ cd /etc/nginx/sites enabled 88

89 $ sudo chown <loginuser> tango $ sudo chgrp <loginuser> tango (d) Comment out access_log and error_log in /etc/nginx/nginx.conf : $ cd /etc/nginx $ sudo vi nginx.conf # access_log /var/log/nginx/access.log; # error_log /var/log/nginx/error.log; Start, reload, and stop Nginx $ sudo nginx $ sudo nginx s reload # After modifying nginx's configurations, perform reload $ sudo nginx s stop 7. Configure settings.py for production and perform collectstatic (a) Edit file settings.py : 1. Disable debug: DEBUG = False 2. Set error reporting to admin's account ( ): ADMINS = ( ('tango', 'tango@gmail.com'), ) 3. Set gmail smtp ( via gm ail in django/ ): _USE_TLS = True _HOST = 'smtp.gmail.com' _HOST_USER = 'tango@gmail.com' _HOST_PASSWORD = 'xxxxxxxx' _PORT = Set error logging (Examples: a simple configuration in ): LOGGING = { 'version': 1, 'disable_existing_loggers': False, 'handlers': { 'file': { 'level': 'DEBUG', 'class': 'logging.filehandler', 'filename': '/webapps/tango/logs/django.log', }, 89

90 } }, 'loggers': { 'django.request': { 'handlers': ['file'], 'level': 'DEBUG', 'propagate': True, }, }, 5. Set static root directory: STATIC_ROOT = '/webapps/tango/static/' (b) Collect static files Django's django.contrib.staticfiles provides a command for gathering static files in a single directory. Run the collectstatic management command $ source <venv>/bin/activate (venv)$ cd /webapps/tango/tango (venv)$ python3 manage.py collectstatic You have requested to collect static files at the destination location as specified in your settings: /webapps/tango/static This will overwrite existing files! Are you sure you want to do this? Type 'yes' to continue, or 'no' to cancel: yes <n> static files copied to '/webapps/tango/static'. C. Migrate the database (venv)$ cd /webapps/tango/tango (venv)$ python3 manage.py makemigrations (venv)$ python3 manage.py migrate Now test: 90

91 Summary Steps of first time system deployment: 1. Install nginx, gunicorn, and vsftpd 2. Install Postgres 3. Create a database (tangodb) and the database user (tango) and grant all privileges 4. Create the whole file structure 5. FTP all application files 6. Create the virtual environment according to requirements.txt 7. Execute: source bin/activate and python3 manage.py collectstatic 8. Make database migrations (makemigrations, migrate) 9. Edit settings.py (a) Modify: DEBUG = False TEMPLATE_DEBUG = False (b) Uncomment LOGGING setting 10.Perform system initialization: 11.Run gunicorn 91