Adaptive Object-Model Metadata Evolver

Transcription

1 Adaptive Object-Model Metadata Evolver Atzmon Hen-Tov Pontis David H. Lorenz Open University Lena Nikolaev Pontis Lior Schachter Open University Rebecca Wirfs-Brock Wirfs-Brock Associates Joseph W. Yoder The Refactory, Inc. Introduction An Adaptive Object-Model (AOM) system represents user-defined domain classes (and their relationships) as metadata [YBJ01; YJ02]. The AOM system is a model represented by objects rather than classes. The object model is constructed at run time by interpreting externally stored definitions (metadata). Users change the object model (or the metadata) to reflect changes in the domain. One way to support model evolution is to take the system off-line, evolve the metadata and the domain objects, and perform a clean restart of the system after a careful release and upgrade. However, when a system needs to keep running, or when updating the entire model at the time a new version needs to come online is too time consuming, then the evolution of the metadata and the domain objects can be included as a part of the model loader. Target Audience The pattern presented in this paper focuses on updating the metadata as either the core meta-architecture or the domain model of an AOM evolves. This pattern is particularly relevant to developers of AOM systems. It may also be relevant to those who need to evolve other types of meta-architectures. We assume the reader has some background knowledge of AOM systems (mainly TYPE OBJECT [JW98], PROPERTIES [JW98], and TYPE-SQUARE [YBJ01] or DYNAMIC OBJECT MODEL [RTJ05] based architectures). 1

2 AOM Metadata Evolver Context You are upgrading a 24x7 production system that is implemented using an AOM. You find that implementing new functionality requires changes to the AOM architecture. For example, a core update to the AOM architecture may be required. You want to make these changes available to a running production system where there is already considerable metadata and data in the system based upon a previous version of the AOM architecture. Your model relies on a TYPE OBJECT [JW98] to dynamically define new business entities for the system. Entities have attributes, which are implemented with the PROPERTIES [JW98] pattern. The TYPE OBJECT pattern is used a second time to define the legal types of attributes, called AttributeTypes. Thus you are using TYPE SQUARE [YBJ01], a combination of TYPE OBJECT [JW98] and PROPERTIES [JW98] patterns. Additionally you might be using these patterns: ENTITYRELATIONSHIP (ACCOUNTABILITY) [WYW09], RULEOBJECTS [Ars00], HISTORY OF OPERATIONS [FCW08], SYSTEM MEMENTO [FCW08], and MIGRATION [FCW08]. Problem How can you evolve at runtime the metadata and the data associated with a live and running AOM as its core architecture changes? Forces Polymorphism: The rules for creating an entity may vary according to its type or according to rules that apply to its data. A monolithic upgrade logic will not manage model evolution well, i.e., a solution that doesn t support polymorphism in the upgrade logic will be hard to manage against a polymorphic model.. Customisability: There are many customers using the product, each customer may modify its own AOM model. The new architecture needs to be able to understand different versions of the AOM model. A solution that doesn t let the customer team resolve conflicts autonomously will require reimplementation of the customization. Legacy Data: There may be large amounts of stored instances, excluding the possibility for off-line data migration in a short maintenance window. Solution Use an AOM BUILDER [WYW09] to create entities (Figure 1). Add as the first step in your AOM-Builder an UpgradeBuildStep to check whether the data associated with an entity-type has been defined with the current version of the AOM. If not, invoke the necessary upgrade scripts that will evolve the data to map to the current version of the AOM. Each entity-type may have several upgrade scripts (Figure 2), some of which are supplied with the new product version and some that are added manually by the user (per customized installation of the AOM system). 2

3 Figure 1: Upgrade Builder Step This process can include migrating metadata and data from many versions before so there may be a chain of version upgrades that needs to be applied. For that to work, every persistent object (Entity, EntityType) has a version field (Figure 2). A system-wide parameter holds the currently installed version. When upgrading the system a collection of ModelVersion entities and the associated upgrade scripts are available to the UpgradeManager. Figure 2: Upgrade Framework Class Diagram 3

4 The UpgradeManager is responsible for determining which scripts are necessary to invoke. This is carried out by consulting the EntityType and the migration converstions each Upgrader handles. This logic includes querying a flag (overridesuper) to determine whether or not the EntityType superclass Upgraders should be used. An Upgrader edits the raw AOM data (e.g., XML) and transforms it to the new version format. The raw data transformation is the responsibility of the Upgrader.upgrade method (Figure 2). Updating the raw data values of AOM objects in the BuildContext is carried out by the UpgradeManager (Figure 1). The BuildContext is part of the AOMBuilder. The upgrade framework provides an API (using the PROXY pattern) for manipulating raw data format. This lets the script programmer concentrate on the transformation from the old version to the new one, rather than dealing with the raw data format [BMR+96]. Dynamics The overall process requires a staging server (Figure 3) in which the new version is installed and the AOM metadata from the production system is brought into the running system. Conflicts between modification done by developers in the new version of the product and modifications done at production by end users are reported in the Problems view (see BREAK AND CORRECT [HNS+10]) and the user needs to resolve them. For example, if an entity type has a new property with the same name of a property defined in an AOM type, the user will need to change the dynamically defined property name. The product team prepares placeholder update scripts for cases where they think a customer specific upgrade script is needed (for example, when a new mandatory property is defined and a simple default value will not suffice). After all conflicts are resolved and all placeholder scripts are written, the customer team can test the new version along with the entire set of upgrade scripts. During production deployment, the upgrade framework is invoked as a first step in the AOM-BUILDER and performs just-in-time instance upgrades as described above (invoked as script hook points during the build process). 4

5 Figure 3. AOM Evolution Process Example For example, consider an AOM implementation of an online marketing system, designed for running Telephony campaigns targeted at different subscriber segments [HLPS09]. Figure 4 shows several classes in the AOM and several instantiated entities. 5

6 Figure 4. The AOM Model and Instances. A generic product comprises several general-purpose classes, such as Benefit and Event. Benefit describes specific campaigns. Event describes the trigger for activating Benefits. Each Benefit is associated with an EventType to define which type of Event can trigger grant of this Benefit type. BenefitType and EventType are part of the AOM TYPE OBJECT pattern. These generic classes can be specialized for a particular customer. In our example, the customer defined two kinds of Benefits: FreeSms and FreeAstrologicalPrediction; and three kinds of Events: SmsEvent, TopupEvent, and VoiceCallEvent. FreeSms is activated by an SmsEvent and grants the user with free units of Sms sends. The number of units is defined in the Benefit class. FreeAstrologicalPrediction is activated by a TopUpEvent and grants the user with free units of daily predictions. For simplicity we omit details such as the conditioning for granting benefits, e.g., send 200 text messages 6

7 this week and get 100 more sends for free. In addition, the customer took advantage of the AOM architecture and added a CAP instance variable in FreeAstrologicalPrediction. This variable is used by the customer to set a special cap on the number of free units that limits the overall cost of the benefit to the service provider in keeping with the marketing budget for this campaign. At runtime, instantiations of specific benefits represent specific products. For example, 100FreeSms sets the benefit to 100 units of free SMS's. WeekFreeAstrologicalPrediction sets the benefit to 7 days of free predictions with a maximum cap of 1904 days total for all users collectively to keep within a $10K budget. Upgrade Scenario The next version of the AOM system needs to incorporate two improvements (Figure 5). First, the cardinality of the association between BenefitType and EventType needs to change from 1:1 to 1:M. In this new version each benefit may be triggered by several event types. Second, to improve budget control, a new monetary CAP is introduced in the generic framework (in the Benefit class), along with a costperunit (in BenefitType). When granting benefits to end users, the system checks that the CAP is not exceeded, otherwise it fails the grant. This is implemented in the new version of the Benefit class. Next we consider the conflicts that may occur during the upgrade and their resolution. Generic upgrade scripts: The cardinality change is a change to the type of benefits, thus affecting FreeSMS and FreeAstrologicalPrediction. This is resolved by providing an upgrade script that converts an event type value to an event type list containing a single event type value. The script is provided as part of the generic system upgrade: if (oldinstance.gettriggertype()!= null) { newinstance.gettriggertypes().add(oldinstance.gettriggertype()); } The generic product can contain a library of recurring refactoring scenarios and their solutions (e.g., property rename). Customer upgrade scripts: Not all upgrade scenarios can be resolved automatically. In some cases specific business logic should be applied in order to resolve the conflict. For example, the cost-per-unit modification cannot be resolved generically. The new property is mandatory and therefore must be initialized. Therefore, the customer needs to supply the cost on case-by-case basis. This is achieved by defining a placeholder script that the customer team needs to implement. The addition of the CAP variable clashes with pre-existing customizations performed to the model by the customer. Specifically, the addition conflicts with the CAP introduced earlier by the customer. To resolve this conflict, the customer needs to rename the original CAP to oldcap and provide an upgrade script that calculates the CAP value for existing model instances. This can be accomplished by multiplying the oldcap by the costperunit and the number of units. newinstance.setoldcap(oldinstance.getcap()); 7

8 newinstance.setcap( oldinstance.getoldcap() * oldinstancenewinstance.gettype().getcostperunit() * oldinstance.getunits()); The FreeSMS is simpler: the customer just needs to define a default value. Although the example presented here is of an application domain change, the pattern applies as well to core AOM architecture changes, such as changing the serialization format of AOM objects. Figure 5. New Version of the AOM Model. 8

9 Alternative Solution The core solution presented in this pattern is based upon lazily evolving the metadata as the system evolves; primarily for modifying systems that need to be running 24x7. However there is a very common alternative (or possible pattern) to the core solution mentioned in this paper that can be used when you need to Evolve your Metadata and you don t have such execution constraints. This is to suspend your AOM system and completely migrate all of your metadata before resuming the system. This alternative is premised on the ability to completely shut down the running AOM system. This allows for a script to completely evolve all of the metadata before bringing your AOM system back online. This solution has the benefit of allowing the metadata to always be consistent with the AOM and will also perform better as the system will not need to invoke checks and hooks in order to possibly evolve the metadata as it is loaded and instantiated. There are definitely tradeoffs to either solution but the core solution mentioned in this paper requires interesting enhancements to the core of the AOM architecture, while this alternative solution does not affect the architecture of the AOM. Resulting Context High Availability: Changes to the AOM architecture can be made without having to stop a running system and then evolve all the previous metadata and data at the same time. Amortized Cost: The overhead of converting data to the new AOM architecture occurs incrementally. Automatic Migration: Default migration can be provided for the changes to the core AOM metadata and architecture using generic scripts. An AOM system may be upgraded from a much earlier version to the latest version without going through the intermediate versions Reuse: Generic upgrades can be provided by the upgrade framework (e.g., for rename property). Usability: Users are supported in resolving version conflicts. Higher Complexity: The AOM Builder code is more complex as hook methods and evolution code will become part of the Builder. Performance Hit: The AOM Builders can be slower at first as the upgrade to the metadata is invoked. Running version checks every time an entity is created can degrade performance. Fragility: Cached AOM data may need to be flushed to force data evolution of existing in-memory AOM instances. Related Patterns History Patterns [AND98] allows for different version of the rules to be available thus going back in time but can add a lot of complexity to the system. Also, in some systems you only want the current version of the AOM architecture and rules. AOM BUILDER [WYW09] is an evolution of the BUILDER [GoF95] pattern. 9

10 AOM Builder is used to invoke the Upgrade Scripts [HNS+10]. Customer upgrade scripts are implemented using the PLACEHOLDER PATTERN [FJS99]. HOOKS [FPR01] are invoked in the AOM loader to enable customers to provide default values for AOM attributes when migrating to a new version. Known Uses Pontis Ltd. ( is a provider of Online Marketing solutions for Communication Service Providers. Pontis Marketing Delivery Platform allows for onsite customization and model evolution by non-programmers. The system is developed using ModelTalk [HLPS09] based on AOM patterns. Pontis system is deployed in over 20 customer sites including Tier I Telcos. A typical customer system handles tens of millions of transactions a day exhibiting Telco-Grade performance and robustness. There is a mutation technique developed for migrating objects in Smalltalk-80 [CW86] that is a known use of the core of this pattern. These authors made an enhancement to Smalltalk that changed how class redefinitions worked. The original Smalltak-80 system mutated all existing instances to use the new representation whenever a class was redefined. To eliminate the overhead of this bulk object mutation, the new approach used lazy mutation. Lazy mutation defers object mutation until it is actually used in a computation. Lazy mutation is accomplished by replacing the method dictionary of the old class definition with a dictionary which defines only the message doesnotunderstand:. In addition, the superclass of the old class definition is set to nil and a reference to the new class definition is stored within the old definition. When a message is sent to such an instance of the old class, a response to the message will not be found and the doesnotunderstand: method will be activated. This method contains the code to mutate the instance into an instance of the new class. The difference between this approach and the pattern described in this paper is that lazy mutation only applies a single evolution to an object, not a chain of replacements. An AOM system developed for the Illinois Department of Public Health used the alternative solution mentioned above. There are also many other well-known uses of the alternative solution done by some of the authors. 10

11 References [AOM] [And98] [Ars00] Adaptive Object-Models. Anderson, F. A Collection of History Patterns. Proceedings of the 6 th Pattern Language of Programs Conference (PLoP 1998), Monticello, Illinois, USA, Arsanjani, A. Rule Object Pattern Language. Proceedings of the 8 th Pattern Language of Programs Conference (PLoP 2000). Technical Report WUCS-00-29, Dept. of Computer Science, Washington University Department of Computer Science. (2000). [BMR+96] Buschman, F., Meunier, R., Rohnert, H., Sommerlad, P., and Stal, M. Pattern Oriented Software Architecture, Volume 1: A System of Patterns. Wiley & Sons [BR98] [CW86] [FCW08] [Fow97] [Fow02] [FPR01] [FY98] [GoF95] [HNS+10] [HLPS09] [Jon99] Bäumer, D., D. Riehle. Product Trader. Pattern Languages of Program Design 3. Edited by Robert Martin, Dirk Riehle, and Frank Buschmann. Addison-Wesley, Caudill, P., Wirfs-Brock A. "A Third Generation Smalltalk-80 Implementation.", p , OOPSLA '86 Conference Proceedings, Portland Oregon, September 29-October 2, Ferreira, H. S., Correia, F. F., and Welicki, L Patterns for data and metadata evolution in adaptive object-models. Proceedings of the 15th Conference on Pattern Languages of Programs (Nashville, Tennessee, October 18-20, 2008). PLoP '08, vol ACM, New York, NY, 1-9. Fowler, M. Analysis Patterns: Reusable Object Models. Addison- Wesley Fowler, M. Patterns of Enterprise Application Architecture. Addison-Wesley Fontura, M., Pree, W., Rump, B. The UML Profile for Framework Architectures. Addison-Wesley Foote B, J. Yoder. Metadata and Active Object Models. Proceedings of Plop98. Technical Report #wucs-98-25, Dept. of Computer Science, Washington University Department of Computer Science, October Gamma, E., R. Helm, R. Johnson, J. Vlissides. Design Patterns: Elements of Reusable Object Oriented Software. Addison-Wesley Atzmon Hen-Tov, Lena Nikolaev, Lior Schachter, Joseph W. Yoder, Rebecca Wirfs-Brock. Adaptive Object-Model Evolution Patterns, SugarLoafPLoP To appear. Atzmon Hen-Tov, David H. Lorenz, Assaf Pinhasi, Lior Schachter: ModelTalk: When Everything Is a Domain-Specific Language, IEEE Software, vol. 26, no. 4, pp , July/Aug Jones, S. A Framework Recipe. Building Application Frameworks: Object-Oriented Foundations of Framework Design. Edited by Fayed, M., Johnson, R., Schmidt,.D. John Wiley & Sons

12 [JW98] [KJ04] [KSS05] [Mar02] [RFBO01] [RTJ05] [RY01] [WYWJ07] [WYW09] [WYW07] [YBJ01] [YJ02] [YR00] Johnson, R., R. Wolf. Type Object. Pattern Languages of Program Design 3. Addison-Wesley, Kircher, M.; P. Jain. Pattern Oriented Software Architecture, Volume 3: Patterns for Resource Management. Wiley & Sons Krishna, A., D.C. Schmidt, M. Stal. Context Object: A Design Pattern for Efficient Middleware Request Processing. 13 th Pattern Language of Programs Conference (PLoP 2005), Monticello, Illinois, USA, Martin, R. Agile Software Development: Principles, Patterns, and Practices. Prentice Hall, Riehle, D., Fraleigh S., Bucka-Lassen D., Omorogbe N. The Architecture of a UML Virtual Machine. Proceedings of the 2001 Conference on Object-Oriented Program Systems, Languages and Applications (OOPSLA 01), October Riehle D., M. Tilman, and R. Johnson. Dynamic Object Model. Pattern Languages of Program Design 5. Edited by Dragos Manolescu, Markus Völter, James Noble. Reading, MA: Addison- Wesley, Revault, N, J. Yoder. Adaptive Object-Models and Metamodeling Techniques Workshop Results. Proceedings of the 15th European Conference on Object Oriented Programming (ECOOP 2001). Budapest, Hungary Welicki, L.; J. Yoder; R. Wirfs-Brock; R. Johnson. Towards a Pattern Language for Adaptive Object-Models. Companion of the ACM SIGPLAN Conference on Object Oriented Programming, Systems, Languages and Applications (OOPSLA 2007), Montreal, Canada, Welicki, L.; J. Yoder; R. Wirfs-Brock. Adaptive Object-Model Builder. 16 th Pattern Language of Programs Conference (PLoP 2009), Chicago, Illinois, USA, Welicki, L, J. Yoder, R. Wirfs-Brock. Rendering Patterns for Adaptive Object Models. 14 th Pattern Language of Programs Conference (PLoP 2007), Monticello, Illinois, USA, 2007 Yoder, J.; F. Balaguer; R. Johnson. Architecture and Design of Adaptive Object-Models. Proceedings of the ACM SIGPLAN Conference on Object Oriented Programming, Systems, Languages and Applications (OOPSLA 2001), Tampa, Florida, USA, Yoder, J.; R. Johnson. The Adaptive Object-Model Architectural Style. IFIP 17th World Computer Congress - TC2 Stream / 3rd IEEE/IFIP Conference on Software Architecture: System Design, Development and Maintenance (WICSA 2002), Montréal, Québec, Canada, Yoder, J.; R. Razavi. Metadata and Adaptive Object-Models. ECOOP Workshops (ECOOP 2000), Cannes, France,