features of Python 1.5, including the features earlier described in [2]. Section 2.6 summarizes what is new in Python The class and the class

Transcription

1 A note on reection in Python 1.5 Anders Andersen y AAndersen@ACM.Org March 13, 1998 Abstract This is a note on reection in Python 1.5. Both this and earlier versions of Python has an open implementation where we can access and alter the internal structures of language components. Python 1.5 takes this one step further making it even easier to adapt a reective programming model. These dynamic features of Python were one of the reasons for chosing it as the language used to implement an experimental reective middleware platform in the SUMO II project. We will in this note describe the open implementation in Python 1.5 and show how it can be used to achieve a reective programming model. The note also includes a description of our reection module for Python. 1 Introduction Python 1.4, which was the previous release of Python, was recognized as a good choice for a language implementing a experimental reective middleware platform in the SUMO II project [1]. All the dynamic features in the language made it possible to go behind the scene and inspect and change the behavior of the language components [2]. In the latest version of the language, version 1.5 1, these dynamic features have been developed even further. We will in this note try to describe how Python 1.5 can be used to adapt a reective programming model. Be aware that we are not talking about the Python interpreter itself as reective 2. We are focusing on a programming model with classes, objects (instances of classes) and its methods and attributes. We will rst describe the reective features of Python. We will then give some comments about how this can be used in our and related work. Add the end a short description of the Python module reection is attached. We will in the rest of this note talk about Python 1.5 when nothing else is specied. We will use the phrase `earlier versions' about version 1.4 and older. 2 The reective features in Python 1.5 Michael Papathomas describes reective features of earlier versions of Python in \Using Reection in Middleware Platforms" [2]. We will in this note try to describe all the reective Distributed Multimedia Research Group Report MPG-98-05, Lancaster University, UK. y This work was carried out while being a visiting researcher at the Lancaster University, UK. 1 Python version 1.5 is available at 2 A description of a reective interpreter for CLOS can be found in \The Art of the Metaobject Protocol" [3]. 1

2 features of Python 1.5, including the features earlier described in [2]. Section 2.6 summarizes what is new in Python The class and the class hierarchy Each instance of a class (an object) has a class attribute which gives the class of this instance. However, types dened in C and other built in types do not have this class attribute. The bases attribute in each class is a tuple containing base classes (direct parent classes), and it can be used to inspect the inheritance hierarchy of a class. In Python 1.5 it is also possible to change the class of an instance at runtime. You just give the class attribute a new value and the instance has changed class. Michael Papathomas developed a neat C extension module called reect to implement this for earlier versions of Python. In Python 1.5 this module is no longer needed. 2.2 Attributes and methods The attributes of instances and classes are stored in dictionaries in Python. The class dictionary contains all attributes (and methods) dened for this particular class. Inherited attributes are not copied over to this dictionary. Instead, they are looked up dynamically when referred. The instance dictionary holds attributes assigned or changed after it is created. You may also say that the instance dictionary holds attributes that vary between dierent instances of a class. The dictionary of a class or an instance is available in its dict attribute. You can change, add and remove attributes from both classes and instances by directly accessing their dict attribute or by using the normal Python syntax. For instances can you even change the dictionary completely by assigning a new dictionary to the dict attribute. You can also add methods to a class using its dict attribute, but you have to remember that a class method needs a reference to the relevant instance as the rst argument (the self argument). Because of this self argument, there is no direct way to add methods to an instance. A workaround is however easy to implement. We can hold this information in an object and use the possibility in Python to call an object as a method if it has its call method dened. 2.3 Attribute access By providing the class methods getattr and setattr it is possible to control access to attributes in classes and instances in Python. If getattr is dened, it will be called whenever an attribute that does not exist in the instance or any of its classes is red. Normally this will give an undened-name error, but the getattr method will catch these. The setattr and delattr methods are triggered under the same conditions when you respectively try to assign a value to an attribute or delete an attribute. 2.4 The Don Beaudry and the metaclass hook In earlier versions of the Python interpreter there is something called the \Don Beaudry hook". This, which you still will nd in Python, is used to provide an alternate class 2

3 behavior to C extensions 3, allowing the Python class syntax to be used to dene other classlike entities. This has been used in the MESS [4] and the Extension Class [5] packages. The Don Beaudry hook can apply when the class creation syntax is used. It checks if the type of the base class is callable, and if so, it is called to create the new class. The Don Beaudry hook never applies when we are only using core Python, since the type of a core object is never callable. We can however notice that this gives us the possibility to control the creation of some special types of classes in a very powerfull way. In Python 1.5 another similar hook is introduced [6]. We have chosen to call this hook the \class metaclass hook" and it can also be applied when the class creation syntax is used (both hooks are \metaclass hooks" 4 ). The class metaclass hook checks if the base class has a class attribute. If so, an instance of the class given in the class attribute is created, and this instance will be the new class. So an instance has now become a class. The class in the class attribute will be referred to as the metaclass and the instance of this metaclass is a class. Confused? Well, be patient and we will soon show you some code. First, we have to take a closer look on what is happening when the class, also known as the instance of the metaclass, is created. So far have we only said that when the Don Beaudry hook or the class metaclass hook applies, either the type of the base class is called (for the Don Beaudry hook) or an instance of the class in the class attribute of the base class is created (for the class metaclass hook). Let us illustrate this with a simple class C: class C(D): 1 a = 1 2 b = "text" 3 This class has a base class D and two attributes a and b. If one of the hooks above applies, you can think of this as equivalent to C = creator('c', (D,), f'a': 1, 'b': "text"g) 1 where f'a': 1, 'b': 2g is the namespace of the class statement, (D,) is a tuple containing the base classes, 'C' is the name of the new class and creator is either the callable type of D (when the Don Beaudry hook is applied) or the class of the class attribute in D (when the class metaclass hook is applied). We will now concentrate on the class metaclass hook, since this is the most interesting hook in our use of Python. Since the the instance of the metaclass is created with the arguments shown above, we allready know about the arguments for the init method of the metaclass. Since this 5 new instance is going to act as a class, we also know the metaclass has to have a call method used when when someone makes a new instance of this class. This call method behaves like the init method in ordinary classes, except that it has to return something (an instance). Here is an example of a simple metaclass: class MyMetaClass: 1 def init (self, name, bases, namespace): 2 3 Classes, built-in-types and C extension types do not share a common behavior in Python. Built-in-types and C extension types can for example not be inherited. 4 I suggested this name for the new hook on the PSA (The Python Software Activity) mailing list, and Guido van Rossum replied \Yes, I think this is a perfect name (though it should apply equally to the C version and the Python version of the hook)". So we now use the term \metaclass hook" for both hooks and \class metaclass hook" for the new hook. 5 A call method of a class is applied when an instance of the class is called like a method. 3

4 self. name = name 3 self. bases = bases 4 self. namespace = namespace 5 def call (self): 6 return Instance(self) 7 This metaclass saves the information about a class when the class is created. We can now make a class which is an instance of the metaclass MyMetaClass. An example of such a class can be: class MyClass(BaseClass): 1 a = 1 2 b = "text" 3 BaseClass has to be a class that connects MyClass to MyMetaClass with the metaclass hook. The metaclass hook will apply if there is a class attribute in BaseClass. If this attribute is MyMetaClass MyClass will be created as an instance of of MyMetaClass. This can be translated to MyClass = BaseClass. class ('MyClass', (BaseClass,), f'a': 1, 'b': "text"g) 1 where BaseClass. class is MyMetaClass. Then what is BaseClass? It has to have a class attribute which is MyMetaClass, so we can simply make it a dummy instance of MyMetaClass: BaseClass = MyMetaClass('BaseClass', (), fg) 1 MyClass is an instance of MyMetaClass, and when we later create an instance of MyClass the call method of MyMetaClass will be called. This method will return an instance of the Instance class. The class attribute of MyClass and BaseClass is the same since both are an instance of MyMetaClass. However, MyClass and BaseClass are not equal, since they do not have the same name, base classes and namespace. Still confused? Another description and more examples of the use of this hook is available in Guido van Rossum's \Metaclasses in Python 1.5" essay [6]. 2.5 Other reective features of Python There is also other reective features in Python, for example concerning the namespace of modules. We have only concentrated on reection on classes and objects since we want to achieve reection at this level in the programming model. 2.6 What is new in Python 1.5? The possibility to change the value of the class attribute of an instance is new in Python 1.5. The same is the possibility to assign the dict attribute in an instance to a new dictionary. The class metaclass hook is also new in Python 1.5. There is also a lot of other interesting changes in Python, like a new regular expression module, a more ecient interpreter implementation, package support and so on, but none of this directly inuence the reective programming model and are therefore not discussed here. 4

5 3 What is in it for us? 3.1 A reective programming model The reective features in Python 1.5 made it easy to achieve a reective programming model at the object (and class) level. Appendix A describes an implementation of this programming model. 3.2 A reective middleware We have also started to investigate and implement a reective middleware platform based on this reective programming model. Some of the ideas and some of the work done are discussed in Gordon Blair's \Notes on Reective Middleware" [1] and in Michael Papathomas' \Using Reection in Middleware Platforms" [2]. Our approach is to have a functional level which consits of objects and bindings between objects, and a metalevel where non-functional aspects are taken into concern. This model has a lot common with the computational and then engineering viewpoints in the ODP reference model [7] but the reective approach gives you a strong tool to separate these concerns and alter the behavior through the metalevel. While you on the functional level only will see objects and bindings, you will on the metalevel nd capsules giving a location for objects, dierent scheduling policies for dierent needs, dierent bindings for dierent needs and so on. For example, you introduce explicit bindings and quality of service parameters by accessing the metalevel through the metaobject of an object. A further description and a more complete implementation of this approach are under development. 3.3 ATOM In ATOM [8, 9], a concurrent object-oriented programming model, one of the goals is to separate the concurrency mechanisms from the object-oriented mechanism. An implementation of this programming model has been done in Python. It is possible that a new implementation of ATOM could gain from taking a reective approach. The separation of concern for concurrency mechanisms and object-oriented mechanisms has a lot in common with the separation of concern in our middleware approach, and a similar approach where concurrency mechanisms is managed on the metalevel seems like a good idea. ATOM could also use the new metaclass hook when creating an active object class 6. In the current implementation you have to create an active object class by applying the function ActiveObject to a class that inherits from ActiveObjectSupport. In a new implementation the ActiveObjectSupport could trigger the metaclass hook and create the class without using the ActiveObject function (the ActiveObject function builds the necessary infrastructure for active objects in ATOM). 6 Active objects, or ATOM objects, is multi-threaded objects, where a new thread is created for each method call. Other threads can also exists for the execution of internal activities. 5

6 4 A note at the end My hope is that you now are ready to play with this, so I have attached a note at the end describing where to nd the software and some documentation. Python and a lot of more Python related stu is available from the Python home: On this site will you nd Guido van Rossum's essay on metaclasses [6] and also references to other related documents like [4] and [5]. Another good source for software and documentation is the Python Starship: 7 This is also the place where the reection module and this note is available from: References [1] G. Blair, \Notes on reective midleware," tech. rep., Department of Computing, Lancaster University, [2] M. Papathomas, \Using reection in middleware platforms," tech. rep., Computing Department, Lancaster University, June Draft. [3] G. Kiczales, J. des Rivieres, and D. G. Bobrow, The Art of the Metaobject Protocol. The MIT Press, [4] D. Beaudry, MESS Tutorial, [5] J. Fulton, \Extension classes, Python extension types become classes," tech. rep., Digital Creations, [6] G. van Rossum, \Metaclasses in Python 1.5," essay, Corporation for National Research Initiatives, [7] K. Raymond, \Reference model of open distributed processing (RM-ODP): Introduction," tech. rep., CRC fo Distributed System Technology, Centre for Information Technology Research, University of Queensland, [8] M. Papathomas and A. Andersen, \Concurrent object-orieted programming in Python with ATOM," in The Sixth International Python Conference, (San Jose, California), pp. 77{87, October [9] M. Papathomas, \ATOM: An active object model for enhancing reuse in the development of concurrent software," Research Report 963-I-LSR-2, IMAG-LSR, Grenoble, France, November The Python Starship will probably change its URL to in the near future. 6

7 A The Python reection module The Python reection module exposes and extends the reective features in Python. It provide reection at the object level, meaning that you can inspect, modify and extend objects. The module also includes some functions to manipulate classes (inspectclass, addmethodclass and delmethodclass), but we have focused on the possibility to manipulates objects. The functions provided to manipulate objects are: inspectobject inspects the given object and return a dictionary describing it, addmethodobject adds a new method to an object, delmethodobject deletes a method from an object, addpremethodobject adds a pre-method to a method in an object, delpremethodsobject deletes pre-methods from a method in an object, addpostmethodobject adds a post-method to a method in an object, delpostmethodsobject deletes post-methods from a method in an object, and changeclassobject changes the class of an object. These features are either provided by manipulating the object itself directly or through a metaobject of the object. If the object have a metaobject (accessible through the meta attribute of the object), the metaobject is in control of the object. If there is no metaobject for the object, the manipulation is done directly on the object it self. This module provides two metaobjects, BasicMeta and PlainMeta. BasicMeta implements the object manipulation by directly manipulating the object itself. PlainMeta does not change the object itself, but builds new structures around it. These simple classes could easily be extended or changed through inheritance (PlainMeta is in fact an extension to BasicMeta). Access to a metaobject of an object is given through the function getmetaobject. If the object does not have a metaobject, this is created on the y. You can remove a metaobject from an object with the function restoreobject. If the metaobject was an instance of BasicMeta the reective changes to the object will still exist after this call. However, if the metaobject was an instance of PlainMeta, the changes through reection will disappear when you restore the object (because the changes has not been done directly on the object). However, changes in the object because of its normal use will of course survive. A.1 Reection exception ReectionException(Exception): All new exceptions or error-types introduced by the reection module is handled by this exception class. A.2 Attributes ignored when inspecting IgnoreAttr: Attributes to ignore when inspecting an object or a class. The class has 2 members: inobject is the list of attributes ignored when inspecting an object and inclass is list of attributes ignored when inspecting a class. A.3 Inspect an object inspectobject(o): Inspects the object o. The result is a dictionary with 4 members: 'class' is the class of the object, 'vars' are the attributes in the object (not including the class 7

8 attributes), 'allattr' is all the attributes of the object (not including methods, but including the class attributes) and 'allmethods' is all methods for the object (including the inherited methods). A.4 Inspect a class inspectclass(c): Inspects the class c. The result is a dictionary with 4 members: 'bases' is the base classes (direct parent classes) of the class, 'vars' are the attributes in the class (not including attributes from the base classes), 'allattr' is all the attributes of the class (not including methods, but including the inherited attributes) and 'allmethods' is all methods for the class (including the inherited methods). A.5 Add a method to an object addmethodobject(o, name, function, override=0): Adds a method with the key name and the implementation function to the object o. The rst argument of the function should be the object it self (the self argument). If override is false (default) an exception will occur if a method with the key name exists. A.6 Delete a method from an object delmethodobject(o, name, completely=0): Deletes a method with the key name from object o. A ReectionException exception is raised if the method does not exists. If the completely argument is true (not default) the method will be hidden completely (also inherited methods with this key). A.7 Add a method to a class addmethodclass(c, name, function, override=0): Adds a method with the key name and the implementation function to the class c. The rst argument of the function should be the object it self (the self argument). If override is false (default) an exception will occur if a method with the key name exists. A.8 Delete a method from a class delmethodclass(c, name, completely=0): Deletes a method with the key name from class c. A ReectionException exception is raised if the method does not exists. If the completely argument is true (not default) the method will be hidden completely (also inherited methods with this key). A.9 Add a pre-method to a method addpremethodobject(o, name, function): Adds a pre-method with the implementation function for the method with the key name in the object o. The new method will be inserted rst in the list of pre-methods. 8

9 A.10 Delete pre-methods from a method delpremethodsobject(o, name, function=none): Deletes the pre-methods of the method with the key name in the object o. If function is given, only the given function is deleted. Otherwise all pre-methods are deleted. A.11 Add a post-method to a method addpostmethodobject(o, name, function): Adds a post-method with the implementation function for the method with the key name in the object o. The new method will be appended last to the list of post-methods. A.12 Delete post-methods from a method delpostmethodsobject(o, name, function=none): Deletes the post-methods of the method with the key name in the object o. If function is given, only the given function is deleted. Otherwise all the post-methods are deleted. A.13 Change the class of an object changeclassobject(o, newclass): Change the class of the object o to the class newclass. A.14 The basic metaobject BasicMeta: This is the class for the simplest metaobject. All reection is done in the actual object, which means that the changes through reection are still visible after the object is restored. The creation of a metaobject has an optional argument o which is the object under control of this metaobject. You can create a metaobject without an object, and then later connect the object to the metaobject with the metaobjects settoobject method. An exception will be raised if you try to use this on an object which allready has a metaobject. A.14.1 Connect the metaobject to an object settoobject(self, o): Make this the metaobject for o. A.14.2 Inspect the object inspect(self): Returns the dictionary representing the inspected object. A.14.3 Return a string representing the object repr(self): Returns a string representing the object. A.14.4 Get the value of an attribute getattr(self, key): Gets the attribute key from the object. 9

10 A.14.5 Set the value of an attribute setattr(self, key, value): Sets the attribute key in the object to the given value. A.14.6 Delete an attribute delattr(self, key): Deletes the attribute key in the object. A.14.7 Add a method addmethod(self, name, function, override=0): Adds the method function with the key name to the object. A.14.8 Delete a method delmethod(self, name, completely=0): Deletes the method withe the key name from the object. A.14.9 Add a pre-method addpremethod(self, name, function): Adds the pre-method function to the method with the key name in the object. A Delete pre-methods delpremethods(self, name, function=none): Deletes the pre-methods of the method with the key name or only the pre-method function if it is given. A Add a post-method addpostmethod(self, name, function): Adds the post-method function to the method with the key name in the object. A Delete post-methods delpostmethods(self, name, function=none): Deletes the post-methods of the method with the key name or only the post-method function if it is given. A Change the class changeclass(self, newclass): Change the class of the object to newclass. 10

11 A.15 The plain metaobject PlainMeta(BasicMeta): A metaobject of this class will do all the reection in its own internal datastructures, without inuencing the actual object. All changes through reection will disappear when the object is restored. Normal changes in the object, like the values of attributes, will of course still be there. A.15.1 Connect the metaobject to an object settoobject(self, o): Make this the metaobject for o. A.15.2 Inspect the object inspect(self): Returns a dictionary representing the inspected object. A.15.3 Call the method message(self, msg): Calls a method of the object using the information in msg. A.15.4 Add a method addmethod(self, name, function, override=0): Adds the method function with the key name to the object. A.15.5 Delete a method delmethod(self, name, completely=0): Deletes the method with the key name from the object. A.15.6 Add a pre-method addpremethod(self, name, function): Adds the pre-method function to the method with the key name in the object. A.15.7 Delete pre-methods delpremethods(self, name, function=none): Deletes the pre-methods of the method with the key name or only the pre-method function if it is given. A.15.8 Add a post-method addpostmethod(self, name, function): Adds the post-method function to the method with the key name in the object. 11

12 A.15.9 Delete post-methods delpostmethods(self, name, function=none): Deletes the post-methods of the method with the key name or only the post-method function if it is given. A Change the class changeclass(self, newclass): Change the class of the object to newclass. A.16 Get the metaobject of an object getmetaobject(o, metaclass = None): Returns the metaobject of the object o. If the object doesn't have a metaobject, a new metaobject is created on the y. You can give the class of the new metaobject in the optional metaclass argument. Otherwise a default one is chosen based on the defaultmeta attribute of the object or the implementation of getmetaobject. A.17 Restore an object restoreobject(o): Removes the metaobject from the object o. 12