Web Serving Architectures

Transcription

1 Web Serving Architectures Paul Dantzig IBM Global Services 2000 without the express written consent of the IBM Corporation is prohibited

2 Contents Defining the Problem e-business Solutions e-business Architectures Scaling Techniques How do we scale e-business Solutions?

3 Defining the Problem without the express written consent of the IBM Corporation is prohibited

4 Definitions Performance How long does it take to get a response to a request from the system Capacity How many transactions can a system completed in a specified unitoftime. Scalability The capability of a system to adapt readily to a greater or lesser intensity of use, volume, or demand while still meeting its business objectives (acceptable levels of performance, availability, manageability etc.)

5 Scalability, Performance and Capacity A truly scalable architecture will allow an IT System to provide an acceptable level of performance under increased loads Original Performance Curve Improved Performance Curve Response Time Acceptable Response Time Number of Operations Current Load Scaling Target

6 Resource Utilization vs.load A scalable architecture will allow an IT System to have performance characteristics like the blue or green line Resource Utilization Utilization increases faster than the load - Typical Utilization increase linearly with load - Good Situation Ideal - Gracefully degrade as load increases.seldom happens Bad situation - Think it's OK until load increases. Poor design Load

7 e-business Solutions without the express written consent of the IBM Corporation is prohibited

8 e-business Solutions e-business solutions allow an organization to leverage Network Computing technologies to re-engineer business processes, enhance communications and lower organizational boundaries with their customers and shareholders (across the Internet), employees and stakeholders (across the corporate Intranet) and its vendors, suppliers and partners (across its Extranet). While providing many benefits, these solutions present many unique challenges. This includes: since these solutions reach beyond the enterprise to users across the public Internet any failures or poor performance results in bad publicity for the organization require a higher amount of security than applications that existed within an enterprise have usage patterns, transaction volumes and transaction volume swings that cannot be predicted accurately.

9 Workload Characteristics To better understand the factors that influence the scalability of e-business solutions, we need to focus on the workload characteristics of these solutions. These fall into 3 major categories: Application functionality Types of Interaction - Inquiry (Static and Dynamic) vs.transactions Volume of Transactions Volume of User-Specific Responses (Personalization) Amount of Cross-Session Info Transaction Complexity Data Volatility Integration with legacy systems Usage Patterns Number of Unique Items Number of Page Views Volume of Dynamic Searches Transaction Volumes Swing Infrastructure Constraints % Secure Pages (privacy) Security: Authentication, Integrity, Non-repudiation, Regulations

10 Types of e-business Solutions To understand the implications of these workload characteristics let use consider four (4) specific types of e-business applications that have high transaction volumes. They are: Publish and Subscribe Shopping Customer Self Service Trading

11 Publish and Subscribe These applications publish different types of content on a regular basis and make this content available to the users of these applications. There are several applications that fall into this category. They include: Web Portals such as yahoo.com, excite.com Media Sites such as cnn.com, zdnet.com and Events such as

12 Shopping These applications allow users to browse or search through a catalog of items, select one or more items from the catalog, electronically pay for these items in a secure manner, fill the order and complete the transactions by having these items shipped to one or more addresses. There are several applications that fall into this category. They include: Exact Inventory Sites - Victoriassecret.com, Abercrombie.com Inexact Inventory Sites - buy.com, dvdexpress.com

13 Customer Self Service These applications empower customers of an organization to access information and perform operations that were traditionally performed by dedicated service professionals (agent, CSR etc.). These applications are available for access across the web on a 24x7 basis. There are several applications that fall into this category. They include: Home banking - bankone.com, wingspanbank.com Travel Sites - Travelocity Insurance - amica.com

14 Trading These applications allow users to access information and perform secure transactions across the web on highly reliable environment. They typically have high volume of transactions. There are several applications that fall into this category. They include: Online Brokerages - schwab.com, fidelity.com, etrade.com Auction Sites - ebay.com, priceline.com

15 Workload Characteristics of e-business Solutions System Workload Characteristics Transaction Volumes Dynamic Content Dynamic Searches User Specific Responses (Personalization) Publish & Subscribe Shopping Customer Self Svc. Trading Cross-session Information Transaction Complexity Legacy Integration Transaction Volume Swings Data Volatility Number of content Publishers/Sources Page Content Volatility Number of Unique Items per page Number of Page Views Percentage of Secure Pages Security, Authentication etc. Low Medium High

16 e-business Architecture without the express written consent of the IBM Corporation is prohibited

17 e-business Architecture The Thin-Client Reference Architecture provided by Enterprise Solutions Structure (ESS) provides a strong foundation for the operational model of an e-business architecture. The operational model consists of a logical view which describes the major components in the architecture and a description of the hardware nodes that support these components. For the sake of this presentation we have simplified some of the details of this model.

18 e-business Architecture Directory & Security Services Other Device Browser Internet Firewall HTTP HTTP Web Intranet Firewall Web App Connectors Enterprise Information Systems Business Partners and External Services Network Desktop Application UI Data Business Databases Systems Management Content Management

19 e-business Architectures The e-business Architecture described on the previous slide consists of a few major architectural components. They are: Web Web Application Security Services Business Databases Enterprise Information Systems Internet Firewall Intranet Firewall and the Network It is important to note again that this is a simplified version of an e-business Architecture. Each of these components are logical services provided by the architecture that can be deployed on one to many physical nodes. Let us now look at how the workload characteristics impact the individual components of this architecture

20 Impact of Workload Characteristics on Components System Workload Characteristics Web Web App s Network Security s Firewalls Ent.Inf. Systems Database Transaction Volumes Dynamic Content Dynamic Searches User Specific Responses (Personalization) Cross-session Information Transaction Complexity Legacy Integration Transaction Volume Swings Data Volatility Number of content Publishers/Sources Page Content Volatility Number of Unique Items per page Number of Page Views Percentage of Secure Pages Security, Authentication etc. Low Medium High

21 Scaling Techniques without the express written consent of the IBM Corporation is prohibited

22 Best Practices A look at the techniques used by the high volume web sites highlight best practices for achieving application scalability. These include: Faster Machines Replicated Machines Specialized Machines Segmented Workloads Request Batching User Data Aggregation Connection Management and Caching It is important to note again that a detailed understanding of the application is key to the successful implementation of these best practices on an architecture.

23 Faster Machines Scalability can be achieved through the use of faster machines. This technique can include: moving to hardware that is bigger than current environment. For example: moving a web server from and PC based server running NT to a UNIX based server using machines with more CPUs to leverage the operating system's multitasking and multiprocessing capabilities using machines that leverage other computing paradigms such as parallel computing using better software that is optimized for the CPU using faster hardware components such as memory, cache, disk and I/O devices etc.

24 Replicated Machines The application can also be scaled by adding more machines of the same type and load balancing processing requests across these machines. In order to implement this technique we have to implement additional components in the architecture such as: Dispatcher node that can monitor and load balance processing requests across the replicated machines A synchronization node that synchronizes the content and data across the machines A mechanism for managing sessions across replicated machines

25 Specialized Machines Individual components of the architecture can be scaled by using specialized machines that perform a certain function much faster. This technique is typically used in architectures to facilitate: intelligent routing of traffic and data across replicated machines dynamic caching, used extensively by event sites and other media sites to speed up access to frequently accessed content security and encryption, used by high volume sites to speed up the SSL encryption and decryption

26 Segmented Workload This is a technique that is typically used in conjunction with replicated machines. It involves the partitioning of the workload of an application to achieve optimum performance. There are several ways of implementing this technique, they vary from: URL references, which is the most simplistic form of segmenting the workload by analyzing the URL and directing the requests to appropriate servers Application Segmentation, which looks at the application and builds the partitioning of the workload in through custom programming Function 1 Function 2 Function 3

27 Request Batching Multi-tier communication places a large computational load on both the client-tier (requester) and the Client server-tier. It also introduces considerable latency. Furthermore, the overhead costs of virtually all cross-tier requests are equal, therefore it is much better to make fewer, but larger requests. The goal of this technique is to reduce the number of requests that are sent between requesters and responders (such as between tiers or processes) by allowing the requester to define new requests that combine multiple requests. Client Client Command Client

28 User Data Aggregation This technique aggregates most commonly accessed data from multiple backend systems to speed up the overall performance of the architecture. This technique is typically implemented using: Custom Programming Intelligent Middleware and Data replication Client Client Client Client

29 Connection Management This technique aims to achieve scalability by reducing the most expensive operations within an application's workflows. This includes connections to legacy systems, databases and other servers Client 1 WEB Application Servlet /App Connection 7 Incoming Request 2 3 Connection Manager Pool 1.WAS passes a user request to a servlet/app 2.The servlet requests a connection from the Mgr. 3.The Mgr get a connection from the pool and gives the servlet/app a connection. 4.The servlet uses the connection to the resource 5.The resource returns data back 6.The servlet return the connection to the Manager and the connection is returned to the pool 7.The servlet/app sends the response back A B Resource If a connection is not available A The CM requests a new connection B Adds the connection to the pool

30 Caching Caching is the most commonly used scaling technique. It can be described very simply as the ability to store most commonly accessed information and data so that it can be accessed and used by multiple users in a rapid and timely manner. This technique can include: caching static data such as images and pages for easy, rapid and repeated access and intelligent caching which looks at dynamic data and looks at how this data can be viewed as a set of static pages. For example, in a sports event web site a table of scores, which is typically generated dynamically, needs to be accessed frequently. Today these sites build these pages periodically and make them available on high speed caches on the web Client (Browser) Client (Browser) Client (Browser) Srv Cache Web

31 Caching Sources of Data Sinks of Data File System HTTP Trigger Monitor Socket Protocol 2216 Configuration Lotus Notes HTTP Object Store Assembly Object Dependency Graph Socket Protocol Cache Manager HTTP IBM DB2 Standard I/O File System

32 Dimensions of Scaling There are 3 primary dimensions for achieving the scalability techniques outlined in the previous slides. They are: 1. Increase the power of the node that supports the architecture component. For example increase the size of the server that supports the database. 2. Improve the efficiency of the component or the node that supports it and 3. Shift the load or reduce the load on the component

33 Dimensions of the Scaling Techniques Scaling Technique Increase Power Improve Efficiency Shift / Reduce Load Faster Machine X Replicate Machines X Specialized Machines X X Segmented Workload X X Request Batching User Data Aggregation Connection Management X X X Caching X X Note: all of these techniques are in use somewhere, yet our top customers are not using all of the techniques that would be useful.

34 Applying the scalability techniques to the architecture This diagram shows how the scalability techniques outlined earlier and apply to the individual components of the e-business architecture Directory & Security Services 1 8 HTML Client App Client 8 8 Internet Firewall Dispatcher/Router 2 3 HTTP HTTP UI Data UI Logic Intranet Firewall Business Logic Connectors 1 8 Existing Apps and Data Business Partners and External Services Business Data Techniques 1 Faster Machine 2 3 Replicate Machines Specialized Machines 4 5 User Data Aggregation 6 Segmented Workload Request Batching 7 8 Connection Management Caching

35 So how do we scale e-business Solutions? without the express written consent of the IBM Corporation is prohibited

36 Scaling the Application In order to develop a truly scalable architecture, it is extremely important to understand both the structure and the functionality of the application that will be supported by it. This includes: Identify all the component parts of the application Understand component interaction and placement Identify the application flows and understand what can and cannot be altered Understand the flow of data and control among the components of the application The application is key to the scalability of any solution, so a detailed understanding is mandatory to effectively scale any solution.

37 Plan for scalability True scalability can be achieved only when it has been planned and built into the architecture of the solution. To achieve this goal, an architect should: Use of standards and open technologies facilitates the portability of the solution to bigger and faster platforms Consider various architecture topologies and select the one that can adequately address the operational requirements of the solution Evaluate and select approaches to place data and functions across the nodes of the architecture Identify and understand the service level characteristics of external systems and the implications of these on the overall performance and scalability of the proposed solution One of the first steps in this process is the establishment of a Performance Budget for the solution.

38 "The Performance Budget" It is critical to establish a performance budget by measuring end user interactive response time 1. Divide into three major buckets time spent in client time spent in network time spent in servers Total X% or xmilliseconds Y% or y Milliseconds Z% or z Milliseconds 100% or Total Time Budget 2. Define Goal for response time/ scale - e.g. 2 seconds, 500 Users 3. Assign budget to each bucket e.g. - X% * goal or fraction of budget 4. Use as target for detailed data flow analysis some of the budget is fixed (e.g. network latency) the variable portion is your "opportunity"

39 Example Response Time Budget Client Response Processing 15% Client Request 5% Request Network Latency 5% Response Network Latency 20% Time 55%

40 Design the application for scalability It is critical to use sound application development practices and techniques to facilitate the overall scalability of the solution. These include the: Use of a layered architecture that encapsulates and isolates functions and provides the flexibility to place or relocate these functions on different nodes of the architecture Development of appropriate instrumentation (such as multithreading, automatic load balancing, fault-tolerance and automatic restart) in application components. These will allow the application to scale both vertically across the same node and horizontally across multiple nodes in the architecture Consider the use of appropriate data structures, access mechanisms and other application development techniques to achieve optimal performance of individual components of the architecture Understand and design the interfaces between the components to achieve maximum scalability. This includes determining the number, the frequency, and the size of these requests

41 Determine the optimum environment Once the application is designed with the scalability in mind it is then possible to determine the optimal scaling environment by: Deploying the application on the actual hardware nodes Tuning the individual hardware nodes to achieve the best performance from each component and Applying the scaling techniques and best practices identified earlier in this document. This process should be one of continuous improvement where the workload characteristics of the application are monitored and evaluated periodically and appropriate changes are made to the operational architecture of the overall solution.