2008 WebSphere System z Podcasts - Did you say Mainframe?

Transcription

1 TITLE: WebSphere Extended Deployment for z/os HOST: Hi, and welcome to the Did you say mainframe? podcast series. This is where we regularly interview IBM technical experts who can help you to understand important IBM mainframe hardware and software issues. I'm your host Sherrie Abshire. Today we re going to talk about WebSphere Extended Deployment for z/os. Our guest today is Snehal Antani from IBM Software Services. Snehal it s great to have you here. SME: I m delighted to be here. HOST: Before we begin, I d like to mention to our listeners that there is a WebSphere Application Infrastructure and Smart SOA on System z teleconference with additional information on this topic. I ll be giving out more information at the end of this podcast. HOST: Snehal, What is WebSphere XD? SME: WebSphere XD is actually composed of 3 sub-products: Compute Grid, a java-based batch and grid infrastructure; Operations Optimization, which improves the resiliency of the WAS z/os infrastructure via a health management feature and continuous availability of the application; and data grid, which delivers data caching technologies and enables XTP-style applications HOST: Talk about batch and WebSphere XD Compute Grid. What is batch and why is Compute Grid important? SME: First let s define batch processing. Batch processing is really the task of processing a tremendous amount of data within a narrowing window of time. This task has nonnegotiable, enterprise-oriented requirements of high availability, scalability, reliability, security, and operational management. Unlike OLTP workloads, there is no user sitting at a terminal waiting for a response. Batch happens behind the scenes. Core business processescalculating interest and credit scores, payments processing, billing systems, end of month/quarter/year reporting and so on all rely on batch as the execution model. WebSphere XD Compute Grid delivers a java-based batch execution environment built on WebSphere z/os. For mainframe customers, this is important for a few reasons: first, since we re dealing with batch applications written in java, these workloads (which traditionally were written in non-java) are zaap eligible. This affects the cost of executing the batch; second, z/os customers mitigated the lack of a java-based batch engine by building their own, homegrown environment, Compute Grid provides a vendor-managed replacement; third, -1-

2 Compute Grid extends the qualities of service provided by System Z, z/os, and WAS on z/os and in addition provides batch qualities of service like checkpointing, restartability and operational control. What makes compute grid interesting is how the technology has brought together the worlds of High Performance Computing (HPC), Extreme Transaction Processing (XTP), traditional grid computing, and utility computing which (in its current definition) is derived from the mainframe. HOST: Interesting. How are these technologies relevant to Compute Grid? Let s start with HPC. HPC s origins are in scientific computing. Several important patterns emerged from this paradigm, two in particular are related here: scalability via the dispatcher/worker pattern; divide-and-conquer through parallelization. The dispatcher/worker pattern is composed of two pieces: the dispatcher that manages the execution of the workload; and 1 or more workers that actually execute the business logic. This architecture is naturally scalable- if there are not enough workers to complete the task per some SLA, start more workers. Those familiar with WAS z/os will recognize the application of this pattern via the Controller-Servant architecture. Add to this scalability the pattern of divide-and-conquer, and you re able to build a high-performance execution environment for workloads of very large size. HOST: Can you tell me more about the divide-and-conquer approach? The divide-and-conquer approach is the idea of taking a very large job, say composed of 100 million records, and dividing that job up in to smaller partitions of 1 million records. You can then execute these partitions concurrently across the collection of workers. This can significantly influence elapsed time, where originally it would be f(100 million) whereas with the parallel model it becomes f(1 million). A key challenge in this area is how to provide operational control of the many hundreds or thousands of job partitions. Compute Grid s architecture fits the dispatcher worker pattern, where we have a Job Scheduler, which is the dispatcher, and 1 or more Grid Execution Environments (which are full WAS z/os application servers), which are the workers. We then have the Parallel Job Manager (PJM), where we can define partitioning rules for a job. The PJM will then create partitions per the rules and manage their execution, providing full operational control over the N sub jobs. From an administrators point of view, you just start/stop/restart/cancel the single logical job. The PJM will manage executing that operation on all of the sub-jobs. HOST: Wow! And what about XTP, Grid and Utility computing? -2-

3 With XTP, the key challenge is narrowing the distance between the data and the business logic. If the data is on z/os, the batch business logic will run faster if on z/os. More over, we can do clever things like affinity routing, where we intelligently route the batch jobs to the servers that already have the data preloaded. This can provide enormous performance gains. Grid computing taught us about providing operational control over thousands of jobs within an infrastructure. A key value-add of compute grid is its ability to provide this complete control. Compute Grid builds on this and provides: full operational controls over the jobs; log management and aggregation of logs for parallel jobs; integration with external enterprise schedulers. Finally Utility Computing. In a nutshell, utility computing is about keeping track of the resources consumed by a unit of work and charging the user of that job for exactly the resources consumed. Compute Grid integrates with z workload management to provide this feature. So you can do things like deliver batch as a service. Suddenly datacenters are not just cost centers, but you can build a business model around them! HOST: What are some specific use-cases for compute grid? How are customers using the technology today? 6 common use-cases: 1. Batch Modernization -- Pursuing batch modernization projects has been of particular interest of z/os customers. The purpose of these projects is to migrate from a native z/os batch runtime, typically developed in programming languages like C, C++, PL/I, and COBOL, to Java. 2. Highly Parallel Batch Jobs -- Leveraging the Dispatcher-worker architecture of Compute Grid to execute highly parallel batch jobs. A highly-parallel batch job is defined as a single, large batch job that can be broken into discrete chunks; each chunk can be executed concurrently across a grid of resources. With Compute Grid, a single job to process 100,000,000 bank account records can be broken down into jobs that process a separate partition of the original data. For example, that job of 100,000,000 records can be broken into 100 jobs of 1,000,000 records. 3. Dynamic OLTP + Batch Runtime -- On z/os, XD Compute Grid integrates with WebSphere for z/os and the underlying z/os operating system and leverages its inherent dynamic runtime. For distributed platforms, XD Compute Grid integrates with WebSphere XD Operations Optimization to deliver a virtualized and goals-oriented infrastructure. 4. Batch as a Service -- WebSphere XD delivers features for resource usage accounting and reporting, on z/os, these features integrate with the features already -3-

4 available on the platform (workload management, RMF, SMF, and so on). On distributed platforms, XD Compute Grid integrates with XD Operations Optimization to provide usage accounting. 5. Replace Existing, Homegrown Batch Frameworks -- IT Operations, prior to the introduction of enterprise batch runtimes like WebSphere XD Compute Grid, took it upon themselves to build their own homegrown batch solutions. These home-grown infrastructures are being replaces with XD Compute Grid in an effort to eliminate code maintenance and execution costs. 6. Sharing business logic across OLTP and Batch -- Business services, defined as discrete tasks responsible for executing some business function, can be shared across both the batch and OLTP runtimes. Batch execution environments are typically independent of the OLTP runtime, especially within the context of J2EE middleware like WebSphere; WebSphere XD Compute Grid delivers a batch execution runtime built on J2EE application servers, and this technology therefore brings together two traditionally disparate worlds, the effect of which is the ability to share business services across them HOST: As you mentioned, WebSphere XD Operations Optimization is another component of XD that z/os customers are looking at. What is XD Operations Optimization? Operations Optimization delivers the following features for improving resiliency and management of a middleware infrastructure: 1. A health management infrastructure, where customizable policies and corrective actions can be defined and enforced. 2. Continuous availability interruption-free application updates of applications with the Application Edition Manager. 3. Checkpointing the configuration of the WebSphere runtime to improve recoverability from administrative changes 4. Visualization technologies for viewing the relative health of the applications and infrastructure. HOST: You mentioned improves resiliency. What do mean by this, and how does XD help? XD can monitor servers for common health problems and take corrective action Conditions include: Memory Leaks or Excessive Memory Usage Hung Servers Excessive Requests Timeouts or Response Time Storm Drains Extended Service Policy Violation -4-

5 Server Age and Max Requests When detected an action plan can be put into effect automatically Notify administrators (including via ) Capture diagnostics information (java thread or heap dump) Restart server Server restarts are smart and done in a way to prevent outage and service policy violations Definition: A Resilient SOA Infrastructure is one that, despite unexpected changes within the end-to-end architecture, continues to serve requests and meet stated service-level agreements and qualities of service Antonym: Fragile SOA Infrastructure - Improving Resiliency is about understanding and mitigating the risks to the SOA Infrastructure HOST: Snehal, that was really interesting - thanks for joining us. We ll be looking to learn more. SME: Thanks for giving me the opportunity to talk about WebSphere Extended Deployment for z/os or XDz/OS. HOST: That wraps up this segment. To find out more about the WebSphere Application Infrastructure and Smart SOA on System z teleconference I mentioned earlier, go to the description of this podcast at: Join us next time to talk about another important mainframe topic. For now, this is Sherrie Abshire saying Thanks for listening. -5-