Context-Oriented Programming with Python Martin v. Löwis Hasso-Plattner-Institut an der Universität Potsdam
Agenda Meta-Programming Example: HTTP User-Agent COP Syntax Implicit Layer Activation Django 2
Meta-Programming
Introspection and Reflection in Python Everything is an object classes/types (Python s class/type dichotomy) functions, bound and unbound methods code objects Implicit attributes for reflection: o. class (also: type(o)), for arbitrary objects o. dict, for arbitrary objects C. name, for classes and functions C. bases, for classes f.func_code, for functions Using standard container types dict is a dictionary bases is a tuple 4
Modifying the Program Creating new program elements: call the meta-type types.functiontype(code_objects, globals_dictionary) types.codetype(argcount, nlocals, stacksize, flags, codestring, names, varnames, filename, name, firstlineno, lnotab) type(name, bases, dict) # a.k.a. types.typetype Adding to and deleting from classes: modify classes dict Changing inheritance hierarchy: modify bases Adding to and deleting from objects: modify dict Changing the type of an object: assign to class 5
Protocols Indexed access (o[k]): o. getitem (k), o. setitem (k,v) Field access (o.a): o. getattr (a), o. setattr (o, a, v) also getattribute Iteration (for e in o:): o. iter (), then i.next() Call (o(args)): o. call (args) Hooks for instance creation: new and init... 6
Meta-Classes Standard meta-class: type (new-style classes) also: types.classtype, for old-style classes Meta-class usage on class definition: metaclass attribute in the class metaclass attribute in the module defining the class meta-class of the first base class new-style classes inherit from object, which is a new-style class 7
Decorators Decorator: Annotation on functions @foo def bar(arguments): body @baz(arguments) def foobar(more_arguments): body Operational semantics: Decorators must be callable bar = foo(bar) foobar = baz(arguments)(foobar) Various use cases: Declaration of static and class methods synchronized, logged, memoized,... functions 8
COP Example: User-Agent
HTTP Libraries httplib: Protocol implementation (full access to all protocol elements Little default behaviour (default: HTTP/1.1, Host:, Accept-Encoding:) def get1(): print "Using httplib" import httplib conn = httplib.httpsconnection('www.dcl.hpi.uni-potsdam.de') #conn.set_debuglevel(1) conn.request('get', '/env') r = conn.getresponse() filter_response(r) 10
HTTP Libraries (2) urllib, urllib2: general URL access automatically sets User-Agent to Python-urllib/2.5 def get2(): print "Using urllib" import urllib2 f = urllib2.urlopen('https://www.dcl.hpi.uni-potsdam.de/env') filter_response(f) def filter_response(f): for line in f.read().splitlines(): if line.find("user_agent")!= -1: print line return print "No user-agent found" 11
Using Method Layers with statement: automatic enter/leave semantics from useragent import HTTPUserAgent with HTTPUserAgent("WebCOP"): print "Using useragent layer" get1() get2() Importing useragent module automatically defines the layer and the layered methods Disabling layers from layers import Disabled with Disabled(Layer): code 12
Defining Layers Inherit from class Layer Class can have arbitrary methods, instance variables, etc class HTTPUserAgent(layers.Layer): def init (self, agent): self.agent = agent 13
Defining Layered Methods Inherit a class (with arbitrary name) from both the layer and the class to augment Define methods with the same name as the original methods Each method has automatic second parameter "context" (after self, before explicit method parameters) Decorate each method with either before, after, or instead Context: Object indicating the layer activation.layer: reference to the layer object.result: result of the original method (for after-methods).proceed: callable object denoting the continuation to the original method (or the next layer) 14
class HTTPConnection(HTTPUserAgent, httplib.httpconnection): # Always add a User-Agent header @before def endheaders(self, context): with layers.disabled(httpuseragent): self.putheader("user-agent", context.layer.agent) # suppress other User-Agent headers added @instead def putheader(self, context, header, value): if header.lower() == 'user-agent': return return context.proceed(header, value) 15
Context-L Layer definition: Layers are classes access to layer instance's state through 'find-layer (only?) Layered methods and functions Layered class: may add layer-specific slots Layer Activation: explicit nested explicit deactivation 16
Comparing PyContext with Context-L Layer definition: Layers are classes Layer activation operates on instances Layered methods: put methods in class block no intuitive syntax for putting methods outside of classes method definitions refer to layer classes Layered functions: not supported yet Layered classes: not necessary Layered methods can add any slots they like Layer activation: both explicit and implicit 17
Implicit Activation
Context 19
Context State and history of the environment often not directly observable to a computer who is the current user? what is the temperature outside? what piece of code called this method? 19
Context State and history of the environment often not directly observable to a computer who is the current user? what is the temperature outside? what piece of code called this method? Sensor: device to observe context os.getuid(), os.environ["remote_user"] pymetar.reportfetcher("lszb")...gettemperaturecelsius() inspect.currentframe(1).f_code.co_filename 19
Context State and history of the environment often not directly observable to a computer who is the current user? what is the temperature outside? what piece of code called this method? Sensor: device to observe context os.getuid(), os.environ["remote_user"] pymetar.reportfetcher("lszb")...gettemperaturecelsius() inspect.currentframe(1).f_code.co_filename Condition: predicate on the context should a certain algorithm be executed? 19
Context-Awareness Selection of algorithm depending on conditions target of behavior layers not necessarily just on methods More generally: behavior is function of sensor readings control systems: context is input to formula part of context is index into some data structure (user name, object number, database transaction) currently not supported in COP 20
Conditions in PyContext Activation of layer depends on condition (predicate) predicate given as imperatively-defined function When to evaluate the predicate? whenever it changes? not feasible Solution: at method invocation After predicate evaluation, layer is activated just for the period Need to keep global list of layer instances whose conditions need checking Composition semantics: how to combine multiple layered methods? Solution: at activation time, the layered method conceptually replaced the original method multiple implicit layers: activate by "precedence", else in registration order 21
Demo: Configuration Files Assumption: Application reads configuration file at start up, then runs Context-orientation: changes to the configuration file should be taken into account Sensors: 1. Determination of change: look at modification time 2. Detection of change: multiple sensors, OS dependent Time-triggered checking (alarm(2), SIGALRM) Explicit signal by operator (SIGHUP) File-system change notifications Check on every method invocation use alarm mechanism for demo (check every 5 seconds) 22
class FileWatcher(Layer): all_instances = [] def init (self, filename):... def read(self): self.mtime = os.stat(filename).st_mtime self.signalled = False def signal(self): if self.mtime < os.stat(filename).st_mtime: self.signalled = True def active(self): return self.signalled def signalled(sig, frame): for obj in FileWatcher.all_instances: obj.signal() signal.alarm(10) signal.signal(signal.sigalrm, signalled) signal.alarm(10) 23
class AppExtension(FileWatcher, Application): @before def do_msg(self, context): if context.layer.filename!= self.confname: # It's not our configuration file that changed return # update configuration self.read_conf() # deactivate layer context.layer.read() 24
Django
Overview Web framework O-R-Mapper HTML templating engine Webapp component framework HTTP server Webapp packaging and deployment 26
Context in Django: Server-originated ("System") context Configuration files: settings.py Database connection List of installed applications (and various plugins) change will automatically cause server reset Configuration files: URL space configuration mapping from URL regexes to Python functions (callables) re-read on every request? Model-View (applications) re-read on every request? need explicit synchronization with database schema 27
Context in Django: Request Context During processing, a HTTPRequest object is passed along Template engine has explicit notion of Context object context variables can be read in the template Middleware: Configurable software layers that each request goes through, affecting overall behaviour: sessions caching decorators to specify whether to cache, and how long transactions decorators for autocommit, commit_on_success, commit_manually 28
Further work Study Django implementation Try to identify places in the Django code or Django applications where COP could be applied Write paper 29