Let me begin by introducing myself. I have been a Progress Application Partner since 1986 and for many years I was the architect and chief developer

Let me begin by introducing myself. I have been a Progress Application Partner since 1986 and for many years I was the architect and chief developer for our ERP application. In recent years, I have refocused on the problems of transforming and modernizing legacy ABL applications. To modernize an application, one needs to define the desired target architecture for the new application, and that has led me to focus on Object Orientation as a key tool in designing modern, OERA compliant applications. 1

Here s our agenda for today. We ll take a quick look at why we are talking about OO ABL in the first place and then dive into specific considerations for the Data Access layer, Business Logic Layer, and the interface between them. 2

The first question we might ask ourselves is why are we interested in OO in the first place. It is commonly believed that the OO paradigm improves quality by encapsulating logic in easily tested units, promoting code re-use, providing a mechanism for easy and well-controlled extensibility, reducing production error rates by catching errors at compilation, creating a more maintainable system, reducing large problems to simpler components, and providing a more natural relationship between code and the real world. I realize that sounds like a pretty extraordinary set of expectations, but it is all achievable if one follows the best OO practice. 4

Some of you have been around long enough to remember the V6 to V7 transition where the recommended programming paradigm changed from a more or less linear procedural model to an event driven model. Lots of programmers had trouble making that transition and had trouble getting used to the idea of letting the user determine the sequence of events. That was a major shift in mindset of the programmer but moving to OO is actually a bigger change in mindset. Learning the syntax is relatively simple, but learning how to think in an OO way is a much harder issue and one that is complicated by there being a lot of examples in 3GL practice which are really bad OO. Getting to think about the problem in the right way is key. 5

Select your guidance carefully. Ask for recommendations on books and courses although books oriented toward ABL are nearly non-existent. The OO reference manual in the documentation has good material and deserves careful study. Consider training and mentoring from someone that you trust. Start off being pretty religious about doing things right rather than accepting early sloppiness with the idea you will clean it up later. Chances are you won t clean up and you will end up having to live with the mess. Even if you don t understand yet why a recommended approach is better, try to use it. Through use, you may discover why it works. One place to start are three whitepapers at the URL shown which cover basic OO vocabulary, OO design patterns, and OO design principles. Each could easily be a book, but these are intended for a quick reference and a place to get started. 6

While the focus of this talk in OO, there is great synergy between good OO and OERA, so we are going to be talking in OERA terms as well. Everyone may have seen one or more versions of the OERA diagram by now. Here is a version which emphasizes the disconnectedness of the layers. What I am going to be talking about today is patterns for the Data Access layer and the Business Logic layer. Data Access is responsible for obtaining data from data stores and persisting it in those data stores. It may contain relational integrity logic, but does not contain operational business logic. Business Logic is contained in the Business Logic layer which obtains data and control inputs from Data Access, Presentation, and a Service Interface. Note that this apparently simple division into layers actually is a simplification of a complex network of relationships between objects, but is still a useful abstraction. In particular, we should regard each layer as a subsystem and therefore strive for the loose coupling between subsystems which we expect in OO, while objects within a layer can be somewhat more tightly coupled. 7

The goal for today is to move beyond this simple OERA diagram and examine patterns and techniques for implementing an OERA architecture using Object Orientation. 8

Let s start with to the Data Access Layer. 9

In traditional ABL, programmers often make direct data accesses wherever they need the data, mixed in with user interface statements and business logic. This mixture isn t a particularly good programming structure even in traditional ABL, but in OERA we are definitely trying to move to a more layered approach. Layering into distinct subsystems or domains is very consistent with OO thinking where we want Separation of Responsibility. I.e., we want each object to have its own distinct responsibility which does not overlap with other objects and we want each collection of related objects in a layer or subsystem to also be coherent in the responsibility it covers and separate from the responsibilities of other subsystems or layers. 10

The Data Access layer allows us to abstract the source of the data from its use. The Business Logic layer should be unaware of where the data is stored, what kind of database was used, whether there was a database access or a request to a remote service, how the data was stored, or basically anything about the persisted form. 11

The Data Access layer helps us to separate what is efficient and practical for processing from what is efficient and practical for storage. In OO processing, we normally want to handle one object at a time. Just like in traditional ABL programming where we handle one record at a time, so in most cases we want the Business Logic layer to ask for one object at a time. But, if we know that we are going to need a whole set of data, e.g., all orders for a customer, there is no reason not to efficiently prefetch that data, even though it gets passed along one object at a time. 12

Those of you who have heard or read my earlier talks might be familiar that I have criticized many uses of temp-tables and ProDataSets. So, what about Temp-tables and ProDataSets in the context of Data Access? Temp-tables are embodiment of 13

relational thinking. Temp-tables are about data arranged in tuples and accessed by keys. Objects are about data and behavior accessed by navigating relationships. Temp-tables thus seem almost antithetical to OO thinking. 13

But, temp-tables and ProDataSets have very attractive features for interfacing to the database. In the DA layer, our subject matter is interfacing to the database. A temptable within one object used as interface to DB is quite different than use as a surrogate for collections or passed among multiple objects in business logic. So, the objections which I might have to some uses of 14

temp-tables and ProDataSets in the context of OO don t necessarily apply in the context of DA objects. 14

Why do we want to consider TTs and PDSs? Query and FILL() very performant and flexible. PDSs can provide a local surrogate to the DB. PDS before-image features provide the information needed for optimistic locking management. 15

What s this about optimistic locking? To minimize locking contention, we normally want to use optimistic locking, but that implies tracking the before image. Is that something the Data Access layer does or is it the responsibility of the objects in the Business Logic layer? The answer is, needing that before image data is really an issue about storage, so it belongs in the Data Access layer. Again, ProDataSets are indicated because all the structures are there for holding the before image data, resolving it with the updated data, and managing the optimistic locking logic. 16

Optimistic Locking Handling Requires before-image copy to compare to current database copy. If unchanged, then no problem; just commit new copy. If changed, need to decide on what to do simple return to BL or attempt merge. Merge may be more appropriate in BL. I will come back to this a bit when we get to the BL and look at the whole cycle. 17

The Data Access layer helps us to separate what is efficient and practical for processing from what is efficient and practical for storage. There is a great role here for ProDataSets since they are very efficient ways to get data to and from the database. Even when we are fetching individual records, we need to keep track of them somewhere until they are ready to be persisted or discarded and a ProDataSet is ideal for that as well. In OO processing, we normally want to handle one object at a time, just like in traditional ABL programming we handle one record at a time, so in most cases we want the Business Logic layer to ask for one object at a time. But, if we know that we are going to need a whole set of data, e.g., all orders for a customer, there is no reason not to efficiently prefetch that data, even though it gets passed along one object at a time. There are some uses, of course, where we are genuinely dealing with a set of data, e.g., a sales matrix we are going to process for forecasting or a set of data used for validation or passing a set of data to the UI. In those cases it makes sense to pass through a set at a time. 18

What else belongs in the subject matter of the DA layer? Relational integrity checks. Computations and manipulations which are part of OO to RDB mapping. Composition and analysis of messages from BL. 19

Now that we have a pretty good idea of what needs to be in the DA, let s look at the interface between the DA and the BL. 20

A key OO principle is Separation of Concerns, i.e., putting each responsibility into its own object -- or subsystem in the case of larger, more complex responsibilities -- and minimizing the interface between objects or subsystems. This clear separation -- related things inside and a minimal necessary connection to things outside -- is one of the key factors which leads to ease of maintenance, ease of modification, and simplifying testing. 21

The Data Access to Business Logic interface is a very fundamental subsystem barrier, so we are going to expect this to be a minimal interface, i.e., a real clear separation between what happens on one side and what happens on the other. 22

In fact, since data can come from multiple sources, this means that the business logic shouldn t really even be aware of where the data comes from except to the extent it is necessary for providing feedback. All of this data should come to the BL components in the same form. 23

The normal way of isolating subsystems in OO is by the means of messages. In this case, the normal practice is to package data in a Message Data Packet or MDP. An MDP is a simple value object, i.e., it has no behavior, only data. To supply data to the BL layer, the DA layer creates an MDP, sends it across, and the BL consumes the data and destroys the message object. Incidentally, this is one of the exceptions to the usual rule of you create it; you destroy it since the BL is the one that knows when the MDP is no longer needed. While it might seem that creating and reading an MDP is pure overhead compared to passing the data as parameters, this approach provides a major decoupling between the subsystems for which you will be grateful when you get to debugging and maintenance. It also means that other subsystems can provide identical data packets to be consumed in the same way. 25

What does this look like in practice? Each subsystem will have a façade object responsible for communication to other subsystems. The façade object will know about the objects inside the subsystem and the façade objects of other subsystems, but otherwise subsystems will not interact directly with each other. 26

Alternatively, we can use a bridge structure in which a single object links two subsystems or layers. A bridge is less flexible and may require more maintenance than a façade, but is particularly appropriate when generating the interface. 27

A factory is the standard pattern for an object which creates other objects, possibly one object type per factory, possibly more than one. It centralizes all of the logic needed to create that type of object in one place. Some people will be tempted to put their object factories in the DA layer because of the direct access to the retrieved data, but the factory belongs in the BL layer since passing a completed Business Entity object between layers would create strong coupling and the DA layer isn t the only possible source of data, for example, order data can come from a service interface or the UI as well as from data access. 32

The immediate source for the factory is the MDP that I talked about earlier. Note, that good OO design dictates that one knows one has valid data before one builds the object. It is bad design to build the object and have it fail because of bad data. 33

Several people have written about patterns which use ProDataSets to hold data in the Business Logic layer. These include John Sadd s writings on PSDN from a few years ago, the Model-Set-Entity pattern of Progress Professional Services, and NSRA from NomadeSoft. I think these approaches are misdirected because the fundamental building block in OO is a single entity, not a whole set of data. Moreover, passing a TT or PDS creates strong coupling between the layers. Model-Set-Entity admittedly works very hard to make it appear as if the Business Entity and Set objects appear to their clients like traditional separate objects exactly for this reason. I have written about this topic specifically in a whitepaper on my website. 34

OO behavior matches OO data, i.e., behavior about an individual entity is packaged with the data for that entity and behavior about a set is packaged with the set 35

There are times when one genuinely has sets of data like a sales history matrix on which one is going to do forecasting or a validation table like a list of states. For these, a ProDataSet or Temp-table are perfectly appropriate, but note that any logic on the object should relate to the set, not the elements. If you need to operate on an individual element, bring it out into its own business entity object where it has the entity logic. Each object type should have its clear responsibilities. If those responsibilities are the set, then they should only be about the set. If they are about the individual, then they should only be about the individual. Another possible rôle might be passing set data to the UI, but there are multiple solutions to that problem, including XML and JSON, so some experimentation is required and the best approach might vary according to specifics. 36

In traditional linear ABL programs issues of lifecycle and persistence can often seem simple. A program runs until it ends. Persistent procedures and dynamic objects can make this more complex. In an OO world, it might seem that these decisions are more complex because of the separation into subsystems, but the principles remain the same. 37

Like traditional ABL, there are some guidelines like you make it, you destroy it. Just as there are exceptions in traditional ABL, there are exceptions for OO, notably for MDP objects since they are created in one subsystem and consumed in another. In the end the real guideline in OO is no different than the considerations one should have in non-oo programming. I.e., the reason one is done needing an object is because one has reached a point where it is no longer needed for business reasons. In OO terms, this means that the object which is controlling the scope of the task, which may be the business entity or, more likely, the task object using a business entity, decides that it has gotten to an important event state changes completed or a phase of processing finished i.e., it is some version of Done. So it sends off a message to the Data Access layer or other appropriate destination announcing that it is done. It may need to wait for a confirmation message, but at some point it knows that it is finished using that entity object. There is a natural point where the object is no longer needed. Likewise, for any given business task or workflow, there are natural beginning and end points, and those define the creation and deletion points for the objects which implement the aspects of that task. 38

So, who controls persistence in an OERA environment? BL is not only not connected to the DB, but doesn t even know if there is a local DB, so it can t make decisions about persistence. DA is connected to the DB or other persistent store, but knows nothing about object state. 39

Lifecycle is not the same as persistence. Persistence is about having reached a state where it is appropriate to preserve that state. This might happen once or many times in the life of an object and it may happen for different reasons at different points in that lifecycle. So, it is the BL s job to decide when an appropriate state has been reached and to signal that this state has been reached and what should be preserved. It is the DA s job to act on that message. 40

Back in talking about the DA layer we talked about this cycle for optimistic locking. So, we know what will happen in the DA layer, but what is going to happen on the BL side? 41

Typically we will use a state machine to wait for confirmation. Hint state machines are a key part of managing behavior in objects! If the Done message is confirmed, then we terminate the process or go on to the next step. If it is not confirmed, we have to decide whether to merge or 42

reprocess. That decision is based on business requirements first and what data is different second. For example, if we are editing the customer record and have made a change to the address, but find that while we were doing that the credit limit changed, we are going to merge those changes and try again. But, if we were adjusting the discount on an order line and find that the item has changed, we probably need to reprocess. If we don t use a state machine to wait for confirmation, we have to provide handling of reversing transactions when the persistence was not successful. That has potential fan out issues if we act on the supposedly modified data and 42

then have to reverse our changes, so we want to partition data carefully to minimize this potential. 42

What about Transactions? Historically, ABL programmers think of transactions closely tied to the database. DB transactions are very important because they insure nothing incomplete gets in the DB. Practice has migrated toward very brief DB transactions when 43

everything is ready to commit. But, we need to fix and log errors as much as possible. Thus, we have business transactions that are larger in scope than DB transactions. 43

With SOA distributed across a DB, the business transaction can often span more than one machine. With DA/BL layer separation, we have a similar situation since the BL can t know what is happening in the DA. Consequently, BL transactions are business transactions which 44

need to operate independently of the DB transaction mechanism. 44

The same is true of transaction scope, except, in non-oo code we often get used to thinking of a transaction as a DB transaction and tying everything to that. In distributed ESB environments we learn that we have to step back from the DB transaction a bit not that DB transactions are any less important, but that the business transaction may exist at a higher level and thus need to be handled in a different way, often by providing fire and forget messaging for loose coupling associated with reversing transactions to provide the undo when necessary. The same is true of Business Layer transactions in OO, where we need to provide more explicit save point and restart or undo mechanisms. One such mechanism is the Memento pattern which captures the state of an object at a particular point so that one can restore to that state if necessary. 45

Here are some links for more information on my website. 48

And here are some additional resources. You don t need to scramble to write these down as the slides will be on my website in a few days. You also might want to catch Peter Judge s talk in the last breakout slot for a down and dirty, lots of code example of making this separation. 49

Thank you. 50

And now for questions. 51