Server-Side Web Programming: Python (Part 1) Copyright 2017 by Robert M. Dondero, Ph.D. Princeton University

Size: px

Start display at page:

Peregrine Gilbert
5 years ago
Views:

2 Objectives You will learn about Server-side web programming in Python Common Gateway Interface (CGI) programming Fundamentals Stateful programming HTTP redirection 2

3 Agenda 1. CGI Programming Fundamentals 2. Stateful CGI Programming 3. HTTP Redirection in CGI Programming

4 Motivating Example Demo of PennyPython1Fund app HTML server does more than deliver static HTML pages HTML server generates HTML pages dynamically, and then delivers them Based upon argument supplied by browser Using data retrieved from a database 4

5 Problem Generalizing Problem: Often static HTML pages are insufficient Often HTTP server must generate HTML pages dynamically Based upon arguments supplied by browser Using data retrieved from a database 5

6 Solution One solution: Common Gateway Interface (CGI) protocol 6

7 URLs (Revisited) URL format: protocol://host:port/file.py? name1=value1&name2=value2&... Same as previously described, except... Browser passes program name file.py and name/value pair(s) to HTTP server HTTP server forks/execs process running file.py and passes name/value pair(s) to process 7

8 HTML Links (Revisited) <a href=" Same as previously described, except... Browser passes program name file.py and name/value pair(s) to HTTP server HTTP server forks/execs process running file.py and passes name/value pair(s) to process 8

9 HTML Forms (Revisited) Or could be "post"; see upcoming slides <form action=" method="get"> <input type="text" name="name1" value="value1"><br> <input type="text" name="name2" value="value2"><br> <input type="submit"> </form> Same as previously described, except... Browser passes program name file.py and name/ value pair(s) to HTTP server HTTP server forks/execs process running file.py and passes name/value pair(s) to process 9

10 Aside: URL Encoding Names and values are URL encoded Each special character (", ', =, &, etc.) is encoded as %nn (hex) Each space is encoded as + Many standard libraries provide functions/ methods to encode and decode... 10

11 Aside: URL Encoding in Python See urlencode.py Try command line arguments: one 't wo' 'th" ree' 11

12 Aside: URL Encoding in Java See UrlEncode.java Try command line arguments: one 't wo' 'th" ree' 12

13 CGI Details: GET Method Browser Or could be POST; see upcoming slides Socket GET file.py?name1=value1&name2=value2 HTTP/1.1 Host: host <Blank line> HTTP Server Environment variable fork/exec Pipe (usually) QUERY_STRING: name1=value1&name2=value2 13

14 CGI Details: GET Method file.py Uses QUERY_STRING Writes to stdout database wait Pipe (usually) Content-Type: text/html <Blank line> <HTML page> somefile HTTP Server Socket Browser HTTP/ OK Date: date Server: server Content-Type: text/html <Blank line> <HTML page> There are some additional headers 14

15 Hello Get in Python See HelloPythonGet app runserver runserver.bat index.html hello.py 15

16 CGI Example: GET Method Browser User types Bob Dondero and clicks form submit button Socket GET hello.py?person=bob+dondero HTTP/1.1 Host: localhost <Blank line> HTTP Server Environment variable fork/exec Pipe (usually) QUERY_STRING: person=bob+dondero 16

17 CGI Example: GET Method hello.py Content-Type: text/html wait Pipe (usually) HTTP Server Socket Browser <!DOCTYPE html> <html> <body> <p>hello, Bob Dondero</p> </body> </html> HTTP/ OK Content-Type: text/html <!DOCTYPE html> <html> <body> <p>hello, Bob Dondero</p> </body> </html> Uses QUERY_STRING Writes to stdout There are some additional headers 17

18 Review: How to GET Question: How can user command browser to generate a GET request? Answer: 3 ways 18

19 Review: How to GET Answer 1: Enter a URL at top of browser The default protocol is http There is no default host The default port is 80 The default file name is index.html or index.php, depending upon HTTP server configuration 19

20 Review: How to GET Answer 2: Click on a page link <a href=" </a> The default protocol is http The default host is the host that delivered this page The default port is the port from which this page was delivered 20

21 Review: How to GET Answer 3: Submit a form User enters Bob Dondero <form action=" method="get" <input type="text" name="person" value="somedefault"> <input type="submit"> </form> The default protocol is http The default host is the host that delivered this page The default port is the port from which this page was delivered 21

22 CGI Details: POST Method Browser Socket POST file.py HTTP/1.1 Host: host Content-type: application/x-www-form-urlencoded Content-length: length <Blank line> name1=value1&name2=value2 HTTP Server fork/exec Pipe (usually) stdin: name1=value1&name2=value2 22

23 CGI Details: POST Method file.py Reads from stdin Writes to stdout database wait Pipe (usually) Content-Type: text/html <Blank line> <HTML page> somefile HTTP Server Socket Browser HTTP/ OK Date: date Server: server Content-Type: text/html <Blank line> <HTML page> There are some additional headers 23

24 Hello Post in Python See HelloPythonPost app runserver, runserver.bat index.html hello.py 24

25 CGI Example: POST Method Browser Socket User types Bob Dondero and clicks form submit button POST hello.py HTTP/1.1 Host: localhost Content-type: application/x-www-form-urlencoded Content-length: 18 person=bob+dondero HTTP Server fork/exec Pipe (usually) stdin: person=bob+dondero 25

26 CGI Example: POST Method wait file.py Pipe (usually) HTTP Server Socket Browser Content-Type: text/html <!DOCTYPE html> <html> <body> <p>hello, Bob Dondero</p> </body> </html> HTTP/ OK Content-Type: text/html <!DOCTYPE html> <html> <body> <p>hello, Bob Dondero</p> </body> </html> Reads from stdin Writes to stdout There are some additional headers 26

27 Review: How to POST Question: How can user command browser to generate a POST request? Answer: Submit a form User enters Bob Dondero <form action=" method="post" <input type="text" name="person" value="somedefault"> <input type="submit"> </form> The default protocol is http The default host is the host that delivered this page The default port is the port from which this page was delivered 27

28 FancyForm in Python See FancyForm app runserver, runserver.bat index.html fancyform.py input tag Type text, password, radio, checkbox, hidden, reset, submit textarea tag select and option tags 28

29 GET vs. POST GET method name=value pairs passed in request header POST method name=value pairs passed in request body Same power When to use which? 29

30 GET vs. POST Technical Criteria Use POST when: There are very many name=value pairs, and/or... Some name=value pairs are very long Keep HTTP request headers under 2000 chars name=value pairs contain non-ascii chars 30

31 GET vs. POST Convention Use GET iff request is idempotent The request does not change server-side state Processing the same name=value pairs twice has same effect as processing them once Use POST iff request is not idempotent The request does change server-side state Processing the same name=value pairs twice had different effect from processing them once 31

32 GET vs. POST Convention With that convention... Browser sees POST request => Browser assumes request is not idempotent Browser warns about page refresh Browser sees GET request => Browser assume request is idempotent Browser does not warn about page refresh 32

33 GET vs. POST Example Example: Web page asks for the price of a car Form should use GET Browser: GET => idempotent Refresh page => no problem Example: Web page purchases a car Form should use POST Browser: POST => not idempotent Refresh page => generate warning Illustrated by FancyForm app 33

34 GET vs. POST Recommendation Generally: Use GET to query server-side data And maybe for learning Use POST to change server-side data 34

35 PennyPython1Fund App Browser Author name prefix HTTP Server Author name prefix CGI Program database Author names, book titles, book prices HTTP Server Author names, book titles, book prices Browser 35

36 PennyPython1Fund App See PennyPython1Fund app runserver, runserver.bat penny.sql penny.sqlite (not shown in hard copy) book.py database.py common.py index.html searchform.py searchresults.py 36

37 Agenda 1. CGI Programming Fundamentals 2. Stateful CGI Programming 3. HTTP Redirection in CGI Programming 37

38 Motivating Example Demo of PennyPython2State application Displays name of previous searched-for author in searchform page But how??? Application must remember previous author name when generating searchform page 38

39 Motivating Example Browser <form action= "searchresults.py" method="get"> Prev search: (None) Browser Click here to do another author search <a href="searchform.py"> Browser <form action= "searchresults.py" method="get"> Prev search:??? User enters "Ker" HTTP Server HTTP Server HTTP Server HTTP Server searchresults.py searchform.py Doesn't know previous author 39

40 Problem Generalizing Problem: HTTP is a stateless protocol Neither browser nor HTTP server remembers previous interactions CGI program exits; so it can t remember either! 40

41 Examples Penny app Want to remember previous search Search engine Want to remember previous searches Commercial app Want to remember shopping cart Secure app Want to remember that user logged in 41

42 State via URL Rewriting Solution 1: URL rewriting Append state data to end of URL 42

43 State via URL Rewriting Example Browser <form action= "searchresults.py" method="get"> Prev search: (None) Browser Click here to do another author search <a href="searchform.py? prevauthor=ker"> Browser <form action= "searchresults.py" method="get"> Prev search: Ker User enters "Ker" HTTP Server HTTP Server HTTP Server HTTP Server searchresults.py Performs URL rewriting searchform.py Does know previous author 43

44 State via Hidden Form Fields Solution 2: Hidden form fields Place state data into form input tag of type hidden Browser does not display 44

45 Browser State via Form Fields Example <form action="searchresults.py" method="get">... Prev search: (None) Browser User enters "Ker" HTTP Server searchresults.py Click here to do another author search <form action="searchform.py" method="get"> <input type="hidden" name="prevauthor" value="ker"> <input type="submit">... HTTP Server HTTP Server Generates hidden form field Browser <form action="searchresults.py" method="get">... Prev search: Ker HTTP Server searchform.py Does know previous author 45

46 State via Cookies Solution 3: Cookies HTTP server: Places state in a cookie Passes cookie to browser as header And then... 46

47 State via Cookies Browser: Retains cookie in memory or client file system Until specified cookie expiration date Until human user explicitly deletes it Passes cookie to HTTP server as header But only to same web app that sent it originally 47

48 Cookie Attributes Cookie attributes: Name Content Host & path The host to which the browser should send the cookie The directory where the cookie is active Default is the directory of page that created cookie Host & path define the web app that sent & should receive the cookie Expiration date Default is this browser session Etc. 48

49 State via Cookies Example Browser <form action="searchresults.py" method="get">... Prev search: (None) User enters "Ker" HTTP Server GET searchresults.py?author=ker HTTP/1.1 Host: localhost <blank line> searchresults.py QUERY_STRING: author=ker Set-Cookie: prevauthor=ker Content-type: Text/html <blank line> <HTML page> 49

50 State via Cookies Example HTTP Server Browser Click here to do another author search <a href="searchform.py"> HTTP/ OK Date: date Server: localhost Set-Cookie: prevauthor=ker Content-Type: Text/html <Blank line> <HTML page> Browser retains cookie HTTP Server GET searchform.py HTTP/1.1 Host: localhost Cookie: prevauthor=ker <blank line> HTTP_COOKIE: prevauthor=ker 50

51 State via Cookies Example searchform.py Knows previous author via HTTP_COOKIE env var Content-type: Text/html <blank line> <HTML page containing Ker> HTTP Server HTTP/ OK Date: date Server: localhost Content-Type: Text/html <Blank line> <HTML page containing Ker> Browser <form action="searchresults.py" method="get">... Prev search: Ker 51

52 PennyPython2State App See PennyPython2State app penny.sql, penny.sqlite, runserver, runserver.bat, book.py, database.py, common.py, index.html searchresults.py Asks browser to store prevauthor cookie searchform.py Uses prevauthor cookie stored by browser 52

53 PennyPython2State App Try viewing cookies in Firefox: Tools Page Info Security View Cookies Try viewing cookies in Chrome: Settings Advanced Privacy & security Content settings Cookies 53

54 Cookie Issue: Size Problem: Cookie size is limited to 4K Solution: Cookie content stored on server-side (in database), indexed by a unique key Cookie contains key only See sessions as implemented in frameworks and PHP 54

55 Cookie Issue: Disabled Cookies Problem: User may disable browser cookies Solution: Ask the user to enable them! Or... Add more logic Fall back to URL rewriting or hidden form fields if necessary 55

56 Cookie Issue: 3rd Party Cookies Problem: Third-party cookies can invade privacy (See Cookie_stuffing) 56

57 Cookie Issue: 3rd Party Cookies Browser AdConsultant contracts with Company1 & Company2 Company1.com Page which includes (hidden) request to fetch blank image from AdConsultant.com User visited Company1 Browser AdConsultant.com Image Set AdConsultant cookie: User visited Company1 AdConsultant knows that user visited Company1 57

58 Cookie Issue: 3rd Party Cookies Company2.com Page which includes (hidden) request to fetch blank image from AdConsultant.com User visited Company2 AdConsultant cookie: User visited Company1 Browser AdConsultant.com Image Set AdConsultant cookie: User visited Company2 AdConsultant knows that user visited Company1 and Company2 AdConsultant provides user profile to Company1 & Company2 58

59 Cookie Issue: 3rd Party Cookies Solution: Tell browser to refuse third-party cookies In Firefox: Edit Preferences Privacy Firefox will Use custom settings for history Accept thirdparty cookies Never But then can t use CAS authentication! Edit Preferences Privacy Firefox will Use custom settings for history Accept thirdparty cookies From visited 59

60 Cookie Issue: 3rd Party Cookies In Chrome: Settings Advanced Privacy & security Content settings Cookies Block third-party cookies 60

61 Agenda 1. CGI Programming Fundamentals 2. Stateful CGI Programming 3. HTTP Redirection in CGI Programming 61

62 Motivating Example Demo of PennyPython3Red app Distinguishes between: Name of author who has no books in database Displays (none) Missing author name Causes redirection back to searchform page Displays error message, prompting user to enter a name 62

63 Problem Generalizing Problem: Often web app must redirect to a page other than requested one Original page Login page Error page 63

64 Solution Solution: HTTP redirection CGI program writes headers to stdout: Status: 307 Temporary Redirect Location: newurl <blank line> HTTP server forwards headers to browser Browser accepts headers, requests newurl 64

65 PennyPython3Red App See PennyPython3Red app penny.sql, penny.sqlite, runserver, runserver.bat, book.py, database.py, common.py, index.html searchform.py searchresults.py 65

66 HTTP Status Codes HTTP Status Code Description 200 OK Standard response for successful HTTP requests. 307 Temporary Redirect The request should be repeated with another URI; however, future requests should still use the original URI. 404 Not Found The requested resource could not be found but may be available in the future. Subsequent requests by the client are permissible. Descriptions from: 66

67 Summary We have covered: Server-side web programming in Python Common Gateway Interface (CGI) programming Fundamentals Stateful programming HTTP redirection 67

CS105 Perl: Perl CGI. Nathan Clement 24 Feb 2014

CS105 Perl: Perl CGI. Nathan Clement 24 Feb 2014 CS105 Perl: Perl CGI Nathan Clement 24 Feb 2014 Agenda We will cover some CGI basics, including Perl-specific CGI What is CGI? Server Architecture GET vs POST Preserving State in CGI URL Rewriting, Hidden