Server-Side Web Programming: Python (Part 2) Copyright 2017 by Robert M. Dondero, Ph.D Princeton University

Server-Side Web Programming: Python (Part 2) Copyright 2017 by Robert M. Dondero, Ph.D Princeton University 1

Objectives You will learn about: Python WSGI programming Web app frameworks in general (briefly) The Bottle web app framework The MVC architecture 2

Agenda 1. WSGI Programming 2. Web Application Frameworks 3. The Bottle Web Application Framework 3

CGI Problem Problem: CGI apps are slow For each request, HTTP server forks/execs a new process OK for low-volume website Maybe not OK for high-volume website 4

CGI Example Browser HTTP Server 5

CGI Example Browser request for searchform page HTTP Server Browser process asks HTTP server process for searchform page 6

CGI Example Browser HTTP Server request for searchform page searchform HTTP server process forks child process, execs searchform program 7

CGI Example Browser HTTP Server searchform page searchform Searchform process passes searchform page to HTTP server process Searchform process exits 8

CGI Example Browser searchform page HTTP Server HTTP server process passes searchform page to browser 9

CGI Example Browser request for searchresults page HTTP Server Browser process asks HTTP server process for searchresults page 10

CGI Example Browser HTTP Server request for searchresults page searchresults HTTP server process forks child process, execs searchresults program 11

CGI Example Browser HTTP Server searchresults page searchresults Searchresults process passes searchresults page to HTTP server process Searchresults process exits 12

CGI Example Browser searchresults page HTTP Server HTTP server process passes searchresults page to browser 13

Embedding Solution 1: Embed apps in HTTP server process HTTP server does not fork processes HTTP server spawns threads in its process (See Threads lecture later in course) 14

Embedding Example Browser request for searchform page HTTP Server Browser process asks HTTP server process for searchform page 15

Embedding Example Browser HTTP Server searchform Thread HTTP server process spawns new thread (See upcoming Threads lecture) Thread generates searchform page 16

Embedding Example Browser searchform page HTTP Server searchform Thread Searchform thread exits HTTP server process sends searchform page to browser process 17

Embedding Example Browser request for searchresults page HTTP Server Browser process asks HTTP server process for searchresults page 18

Embedding Example Browser HTTP Server searchresults Thread HTTP server process spawns new thread Thread generates searchresults page 19

Embedding Example Browser searchresults page HTTP Server searchresults Thread Searchresults thread exits HTTP server process sends searchresults page to browser process 20

Embedding Assessment Embedding assessment (simplified): (pro) Faster than CGI Spawning a tread is faster than forking a process (con) Inflexible Poor load balancing; better for HTTP server to delegate work to pool of app servers, maybe running on distinct computers (con) Insecure HTTP server typically runs as root Apps run with same permissions as HTTP server 21

Fast CGI Solution 2: Fast CGI HTTP server forks processes As with CGI Typically one for each app es persist Unlike CGI 22

Fast CGI Example Browser HTTP Server 23

Fast CGI Example Browser request for searchform page HTTP Server Browser process asks HTTP server process for searchform page 24

Fast CGI Example Browser HTTP Server request for searchform page Penny HTTP server process forks child process, execs Penny program 25

Fast CGI Example Browser HTTP Server searchform page Penny Penny process passes searchform page to HTTP server process Penny process persists 26

Fast CGI Example Browser searchform page HTTP Server Penny HTTP server process passes searchform page to browser 27

Fast CGI Example Browser request for searchresults page HTTP Server Penny Browser process asks HTTP server process for searchresults page 28

Fast CGI Example Browser HTTP Server request for searchresults page Penny HTTP server process sends request to existing Penny process 29

Fast CGI Example Browser HTTP Server searchresults page Penny Penny process passes searchresults page to HTTP server process Penny process persists 30

Fast CGI Example Browser searchresults page HTTP Server Penny HTTP server process passes searchresults page to browser 31

Fast CGI Assessment Fast CGI assessment (simplified): (pro) Faster than CGI (pro) Flexible Good load balancing; HTTP server can delegate work to pool of app servers, maybe running on different computers Etc. (pro) Secure Apps can run with more limited permissions than HTTP server has 32

Python Solution: WSGI Python solution: Web Server Gateway Interface (WSGI) A specification of two interfaces 33

Python Solution: WSGI Environment info Callback function Browser HTTP Server Your App Response via callback function WSGI server-side Defines messages your app Can send to HTTP server WSGI client-side Defines messages HTTP server can send to your app 34

Python Solution: WSGI Browser Recall Decorator pattern HTTP Server WSGI Middleware WSGI Middleware Your App Optional middleware provides: App persistence (as with fast CGI) Load balancing among multiple app processes Maybe on multiple computers Authentication 35

Python Solution: WSGI Web Server Gateway Interface (WSGI) Allows architectural flexibility Unknown to web app developer Facilitates separation of concerns: Programmers (and others) compose web apps System administrators deploy web apps The way to structure Python web apps 36

PennyPythonWsgi App See PennyPythonWsgi app penny.sql, penny.sqlite, book.py, database.py runserver runserver.bat common.py penny.py 37

PennyPythonWsgi App Observation: Programmers don t use WSGI directly Instead: Programmers use web app frameworks Many Python web app frameworks use WSGI Suggestion: Don t be concerned about WSGI details Unless you intend to compose a Python web app framework! 38

Agenda 1. WSGI Programming 2. Web Application Frameworks 3. The Bottle Web Application Framework 39

Motivation for Frameworks Ancient approach to building a website: Write ad hoc client-side code In HTML, CSS, JavaScript, etc. Write ad hoc server-side code In Python, Java, PHP, etc. Write ad hoc code to access DB 40

Motivation for Frameworks Problem: Common patterns within and among web apps Replicated code within and among web apps Web app development involves lots of tedious/ replicated effort Solution: Web app frameworks Web app frameworks mechanize (parts of) the development process 41

Popular Frameworks According to h,ps://ho1rameworks.com/ on 6/21/17 Stack Github Overflow Overall Score Score Score ASP.NET (C#) 100 100 AngularJS (Javascript) 97 97 97 Ruby on Rails (Ruby) 93 98 95 ASP.NET MVC (C#) 94 94 React (JavaScript) 100 84 92 Django (Python) 90 93 91 Laravel (PHP) 92 87 89 Angular (JavaScript) 89 86 87 Spring (Java) 84 91 87 Express (JavaScript) 92 82 87 Meteor (JavaScript) 93 80 86 CodeIgniter (PHP) 84 86 85 Symfony (PHP) 84 85 84 Vue.js (JavaScript) 98 69 83 Flask (Python) 90 76 83 42

Popular Frameworks According to h,ps://ho1rameworks.com/ on 6/21/17 Stack Github Overflow Overall Score Score Score Ember.js (JavaScript) 86 78 82 JSF (Java) 82 82 Flex (ActionScript) 79 79 CakePHP (PHP) 76 80 78 Google Web Toolkit (Java) 78 78 Play (Scala) 79 75 77 Zend (PHP) 74 78 76 Sails.js (JavaScript) 85 67 76 Yii (PHP) 72 76 74 Tornado (Python) 83 62 72 Sinatra (Ruby) 79 66 72 Grails (Groovy) 64 80 72 Aurelia (JavaScript) 79 61 70 Phoenix (Elixir) 79 60 69 Phalcon (PHP) 78 58 68 43

Popular Frameworks According to h,ps://ho1rameworks.com/ on 6/21/17 Stack Github Overflow Overall Score Score Score Koa (JavaScript) 84 49 66 Dropwizard (Java) 74 56 65 Struts (Java) 64 64 Bottle (Python) 74 55 63 Wicket (Java) 63 63 Dojo (JavaScript) 56 71 63 Nancy (C#) 73 54 63 Vert.x (Java) 75 51 63 web.py (Python) 71 53 62 Elm (Elm) 70 54 62 Pyramid (Python) 65 59 62 Vapor (Swift) 80 40 60 beego (Go) 81 40 60 Gin (Go) 80 41 60 Kohana (PHP) 61 60 60 44

Popular Python Frameworks According to https://hotframeworks.com/ on 6/21/17 Some popular Python frameworks: Django, Flask*, Tornado, Bottle*, web.py, Pyramid, web2py, CherryPy*, Grok, Zope * micro-framework 45

Popular Java Frameworks According to https://hotframeworks.com/ on 6/21/17 Some popular Java frameworks: Spring MVC, JSF, Google Web Toolkit, Dropwizard, Struts, Wicket, Vert.x, Vaadin 46

Popular PHP Frameworks According to https://hotframeworks.com/ on 6/21/17 Some popular PHP frameworks: Laravel, CodeIgniter, Symfony, CakePHP, Zend, Yii 47

Framework Assessment Positives Yield reliable code Framework code has been thoroughly tested Yield consistent code Code has uniform structure across apps Make efficient use of programmer time Framework handles repetitive parts DRY (don t repeat yourself) is encouraged 48

Framework Assessment Negatives Can yield systems that are: Large Slow See Joel Spolsky s Why I Hate Frameworks http://discuss.joelonsoftware.com/default.asp? joel.3.219431.12 49

Agenda 1. WSGI Programming 2. Web Application Frameworks 3. The Bottle Web Application Framework 50

The Bottle Web App Framework Who: Marcel Hellkamp Descrip: Bottle is a fast, simple and lightweight WSGI micro web-framework for Python. It is distributed as a single file module and has no dependencies other than the Python Standard Library. -- Wikipedia 51

Why Bottle? Why study Bottle? (Instead of some other Python framework) Easy to learn Simple ( micro-framework ) Good documentation and tutorial Reasonable to use for Assignments 3 & 4 Installed on CourseLab Easy to install on your personal computer Has test web server Allows flat directory structure; easy to submit P.S. I have Penny in Django; ask if you want it 52

PennyBottle1Fund App See PennyBottle1Fund App penny.sql, penny.sqlite, book.py, database.py runserver runserver.bat common.py penny.py Generalizing... 53

PennyBottle1Fund App Bottle separates URL from file name (Unlike CGI) Can use pretty URLs Bottle separates URL from function name Can map multiple URLs to same function Can change function name without changing URL, or vice versa 54

PennyBottle1Fund App Bottle provides utility functions Functions to fetch name=value pairs Functions to fetch/create cookies Function to implement HTTP redirect 55

Aside: Python Decorators def sqr(i): return i * i def main(): print sqr(5) if name == ' main ': main() Wanted: sqr() prints sqr was called each time it is called 56

Aside: Python Decorators def sqr(i): print 'sqr was called' return i * i def main(): print sqr(5) if name == ' main ': main() OK, but Doesn t generalize Requires edit of def of sqr() 57

Aside: Python Decorators One approach: def printnamedecorator(f): def fwrapper(i): print f. name, 'was called' return f(i) return fwrapper def sqr(i): return i * i sqr = printnamedecorator(sqr) Prints: sqr was called 25 def main(): print sqr(5) if name == ' main ': main() 58

Aside: Python Decorators Using a decorator def printnamedecorator(f): def fwrapper(i): print f. name, 'was called' return f(i) return fwrapper @printnamedecorator def sqr(i): return i * i Prints: sqr was called 25 def main(): print sqr(5) if name == ' main ': main() 59

PennyBottle1Fund App Bottle uses Python decorators Using a decorator... @route('/searchform') def searchform():...... Same code without using a decorator... def searchform():... searchform = default_app().route( \ path='/searchform', callback=searchform)... 60

Toward PennyBottle2Templates Problem: Code contains many assignment statements to compose HTML code Bulky; awkward; error prone Solution: Templates 61

PennyBottle2Templates App See PennyBottle2Templates App runserver, runserver.bat, penny.sql, penny.sqlite, book.py, database.py header.tpl footer.tpl index.tpl searchform.tpl searchresults.tpl penny.py Generalizing... 62

PennyBottle2Templates App Template (informally) HTML doc with placeholders Each placeholder is identified by a key hello.tpl Hello <strong>{{username}}</strong> and welcome 63

PennyBottle2Templates App To instantiate a template: template('somefile.tpl', dict) dict contains key/value pairs For each placeholder identified by key in somefile.tpl, replaces the placeholder with the value associated with key in dict Returns the resulting str 64

PennyBottle2Templates App To instantiate a template: Example: hello.tpl Hello <strong>{{username}}</strong> and welcome dict = {'username': 'Bob'} s = template('hello.tpl', dict) equivalent to s = 'Hello <strong>bob</strong> and welcome' 65

PennyBottle2Templates App Template can contain: Inline expressions {{expr}} expr usually is a key in the provided dict... {{prevauthor}}... expr can be any expression that evaluates to a str or has a str representation... {{book.getauthor()}}... 66

PennyBottle2Templates App Embedded pseudo-python code % line of Python code Indentation is ignored Blocks that are normally indented must be closed with end keyword % for book in books: {{book.getauthor()}}, {{book.gettitle()}}, ${{book.getprice()}}<br> % end 67

PennyBottle3Templates App Includes of other templates % include('other.tpl')... % include('header.tpl')... 68

Toward PennyBottle3Templates Problem??? Computation of morning/afternoon is in penny.py; better to move it to header.tpl? Computation of current time is in penny.py; better to move it to footer.tpl? Solution??? Move more Python code to templates 69

PennyBottle3Templates App See PennyBottle3Templates App runserver, runserver.bat, penny.sql, penny.sqlite, book.py, database.py, common.py, index.tpl, searchform.tpl, searchresults.tpl header.tpl footer.tpl penny.py Generalizing... 70

MVC Architecture Bottle encourages... The Model-View-Controller (MVC) architecture Model: The system s data access code View: The system s data presentation code Controller: (vague) The system s business code; the logic that connects the model with its view(s) 71

MVC Architecture In a Bottle app Views: template files In Penny apps: *.tpl files Controller: main Python file In Penny apps: penny.py Model: DB-related Python files In Penny apps: database.py, book.py Many non-micro frameworks facilitate model dev 72

MVC Assessment Positives Allows separation of concerns: Model: DB administrator (SQL, normalization,...) Views: Web designer (HTML, CSS,...) Controller: Programmer (Python, Java, ) 73

MVC Assessment Positives (cont.) Facilitates parallel development Can develop model, views, controller concurrently Facilitates maintenance Framework provides loose coupling (To some extent) can change model, views, controller independently 74

MVC Suggestions Suggestions: Use MVC!!! But keep views simple Avoid embedded code in views/templates when you reasonably can Is PennyBottle3Templates app better or worse than PennyBottle2Templates app? 75

Aside: ORMs Problem: Impedence mismatch Relational DB data model: tables, rows, columns OO data model: objects, object composition, classes, class inheritance Very different! 76

Aside: ORMs Solution 1: Cursor Relational DB driver fetches (sub)table from DB, presents it to OO code as a cursor OO code uses cursor to traverse table 77

Aside: ORMs Solution 2: Object-relational mappers (ORMs) Provided stand-alone (see Wikipedia) Provided by many full-featured frameworks Not provided by micro-frameworks (e.g., Bottle) Through config files, map relational DB tables to classes/objects OO code fetches/stores data by sending messages to objects, not by executing SQL statements Facilitates model development 78

Aside: Django and COS 333 Django Use for COS 333 assignments? (pro) The most popular Python framework (pro) Has ORM (con) More complex/powerful than necessary (con) Requires non-flat directory structure (con) ORM requires control of DB Django s ORM requires control fields in tables Could not use Django s ORM to access Penny DB Limitation beyond COS 333??? 79

More Bottle! There is much more to Bottle Bottle user s guide: http://bottlepy.org/docs/dev/ Bottle tutorial: http://bottlepy.org/docs/dev/tutorial.html 80

Summary We have covered: Python WSGI programming Web app frameworks in general (briefly) The Bottle web app framework The MVC architecture 81