Chapter 4. Adaptive Self-tuning : A Neural Network approach. 4.1 Introduction

Transcription

1 Chapter 4 Adaptive Self-tuning : A Neural Network approach 4.1 Introduction Machine learning is a method of solving real world problems by employing the hidden knowledge present in the past data or data patterns. Several expert systems have been developed that are widely used in the industry. The objective of machine learning technique is to provide higher levels of automation in the domain of engineering application. This is a replacement for the time consuming, error prone human intervention with automatic techniques that enhance productivity, performance and also result in cost saving. The machine learning can be categorized as Inductive and Deductive learning. Inductive machine learning techniques use computer programs that act on massive data- sets by extracting rules and patterns. This method 75

2 takes examples of a concept as input and generalizes the concept rather than starting with existing knowledge. On the other hand deductive learning takes existing facts and knowledge as input to deduce new knowledge. Machine learning relies heavily on large volume of data for deducing new knowledge and hence, its working overlaps with statistical approach of solving real world problems. Use of machine learning is demonstrated in a wide variety of applications [17] [55] [61], these include search engines, bio-informatics, stock market analysis, medical diagnosis, gaming, robotic automation etc. One important characteristic of machine learning technique is that, it effectively learns how to estimate the desired output by learning from a given training data-set Neural network : A machine learning technique Some of the important machine learning techniques are, statistical approach, case based reasoning, classification and regression, genetic algorithms, Artificial Intelligence using Neural network etc. Among the machine learning techniques neural network based machine learning approach is the best choice for systems that are highly complex [61] [62] for which there is no analytical solution possible. The primary advantage of using neural network approach is that they are adaptable. A typical neural network setup comprises of an input layer, a hidden layer and an output layer. The nodes in the neural network are connected through edges with suitable weights assigned. As the neural network is trained using past data the weights are adjusted using an appropriate learning algorithm so that the desired target at the output node is reached for a given set of inputs. The output of each node depends on the weighted sum of the values of the preceding nodes, the type of activation function and the firing angle. Some of the activation functions are binary, bipolar, sigmoidal etc. The convergence of the output of the neural network depends on the learning rate, acceptable error, momentum factor and the number of iterations. The size of training data-set, number training iteration(epochs), and acceptable error, determine the accuracy of the result and hence must be chosen very carefully. In the following section, the use of neural network approach for estimating the tuning parameters is presented Neural network setup for Self-tuning Neural network provides an elegant way to setup a self-tuning system that can learn from past values of tuning parameters to predict the future values based on the sensor inputs like Buffer-Hit-Ratio, User-load and Database size. In this approach, Neural network is used to estimate the tuning parameter values based on the above three inputs. Neural networks are best suited to provide solutions to problems in complex systems that are intrinsically non-linear in 76

3 nature. Since, DBMS is one such complex software system and also the response time of the user queries exhibit non-linear relationship with the tuning parameters. Neural network can be trained to learn the complex relationship between the input and the output from training dataset. Machine learning technique can be leveraged as an effective method of predicting the estimated values of the tuning parameters from a given training data-set. The training dataset comprises of tuning parameter values corresponding to Buffer-Hit-ratio, User-load and Database size under different load conditions. For the purpose of estimating the tuning parameter values namely, Buffer_cache_size(BCS), Shared_Pool_size(SHP) and Large_Pool_size(LRP) are used as input to the neural network setup as shown in figure 4.1. It is a three layer neural network with three nodes in the input layer(one for each of the 3 sensor inputs), 10 nodes in the hidden layer and 3 nodes in the output layer(one for each of the three tuning parameter). BHR N user load Large_Pool_Size Shared_Pool_Size Database size DB_Cache_Size Figure 4.1 A 3-layer Neural network setup The neural network uses a Feedforward Backpropagation learning algorithm wherein, the learning is decided by difference in the error between the target and estimated values and the gradient of the activation function. The activation function used is sigmoidal, for all the inner nodes. It is this function that gives the neural network the ability to learn and produce an output for which it is not trained. This algorithm provides a procedure for changing the weights of the edges in the network during the training process. The basic idea in this learning method is the use of gradient descent method to propagate the error back to the nodes in the previous layers. Though, the primary objective of this algorithm is to achieve faster convergence it must also balance between the ability of the neural network to quickly respond to the changing inputs and also achieve generalization. However, for the proper functioning of the neural 77

4 network, a well defined training data set, appropriate values of learning rate, momentum coefficient etc. are needed. The output nodes have a pure-linear activation function to generate the final estimated values for a given test input data Properties of Neural networks Following are some of the very important parameters that need to be carefully chosen for the successful implementation of the neural network for the purpose of estimating the correct set of tuning parameters depending on the external factors like workload type, user-load and database size. 1. Initial weights : The choice of initial weights decides the quality of the solution and also how fast the network converges. The initial weights can t be very high because the sigmoidal activation function will saturate and would not respond to the changing inputs. The initial weight is usually done using a method called Nyigen-Widrow method [61] as it leads to faster convergence. In the above setup, the weights are randomly assigned through this method. 2. Learning rate : The learning rate affects the convergence of the neural network output, and hence, must be carefully chosen. A higher value may lead to faster convergence, but also may result into overshooting of the desired output. The typical range of learning rate is in the range 10-3 to 10 as has been established through successful implementation. In the above setup a learning rate of has been found to be most suitable. 3. Momentum factor : The gradient descent technique used in Backpropagation approach becomes very slow if the learning rate chosen is small and oscillates widely if it is chosen to be large. By choosing an appropriate value of momentum factor, oscillation can be avoided even if a larger value of learning rate is chosen for faster convergence. The momentum factor used in the above setup is Generalization : A network is said to generalize the learning, if it produces the correct expected output for the inputs that were not present in the training data set. The back propagation is known to be the best generalization network. However, if the number of nodes are large then this network is not only computationally expensive, it also fails to generalize on the input values. Hence, the number nodes should be kept to the minimum. In the above setup, only 10 nodes are used for the network to be computationally faster and effective in generalization. 5. Training data-set : For the above setup to generate meaningful and useful output without generating arbitrarily large or absurd(-ve values) values in the output, the 78

5 training data setup must be sufficient and properly framed. The general rule in choosing the training data-set is that the data-set must cover the entire input data space and training values must be randomly chosen from this set. It would also be appropriate to separate the input training data sets into N disjoint regions with each disjoint set has different effects on the output values. In the above setup, 4 different disjoint sets were defined, one for each of the 4 different workload types. 6. Number of hidden nodes : The number and size of each hidden layer has to experimentally established as there are no rules of thumb to decide these two. The number of nodes in the hidden layers may have to be increased in small steps, if the result doesn t converge faster. On the other hand, if the network converges swiftly, then the number of nodes have to be reduced. In the above setup, the neural network with only 4 nodes was not converging fast and also was not generating meaningful output. Hence, the nodes in the final setup were increased to 8. With 8 nodes though convergence was faster, the results for some of the inputs was overshooting the target. The nodes in the network were increased to 12 nodes. 4.2 MAPE : A General framework for self-tuning architecture A general self-optimizing technique that works on the principle of Monitor, Analyze, Plan and Execute (MAPE) feedback control loop and is part of the autonomic computing architecture proposed by IBM. The general self-tuning architecture is modified to implement a self-tuning architecture for improving the performance of DBMS and is shown in figure 4.2. In this architecture the sensor inputs that indicate the degradation in system performance are extracted by a sensor module and are then fed to an Analyzer. The analyzer module examines the sensor inputs and other external factors and estimates the extent of changes needed in the system. The planner generates a plan of action based on the input received from the Analyzer. In some specific applications, for example, the Neural network based self-tuning system as shown in figure 4.2, the analyzer & planner could be one single module which does both the analysis of input data and generation of a plan of action. The plan of action is given to the execution unit of MAPE to effect the changes to the effectors. 79

6 Knowledge-Base : Training Dataset Monitor Neural Network to estimate the tuning parameters (Analyze + Plan) Execute Sensor Managed Resource Touch Point Effector Database Management System DBMS Memory Components End User App. Database Figure 4.2 A Self-tuning Architecture based on MAPE autonomic computing paradigm Figure 4.3 shows the details of Neural network based tuning architecture setup that works on the principle of MAPE. In this setup the neural network setup that is presented in the previous section forms an important part of the entire self-tuning setup. As can be seen from the diagram, the inputs to the Neural network block are extracted from the DBMS using appropriate SQL commands[a.1] and are preprocessed before being fed into the neural network proper. In this setup three parameters that act as sensors, namely the user-load N, Buffer Hit Ratio(BHR) and the Database size(dbs) are extracted from the DBMS by sensor data extraction module. This module uses simple SQL commands or scripts to extract the sensor data. The neural network uses the training dataset stored in the knowledgebase for learning and estimate the extent of change in the tuning parameters need for changed workload scenario and/or performance degradation. The estimated values are used to change the tuning parameter values, namely Buffer_Cache_Size(BCS), Shared_Pool_size(SHP) and Large_Pool_size(LRP) again using appropriate SQL commands. 80

7 DBMS Db_Cache_Size(DCS) Large_Pool_Size(LPS) Shared_Pool_Size(SPS) Tuner N BHR DBS Extraction Module Pre- Processing Neural Network Post- Processing DBA Database Database Training Dataset Training Dataa Set Figure 4.3. Neural Network based tuning system As discussed earlier, for the Neural network to generate consistent and meaningful output, it is important to choose an appropriate activation function, learning rate, number of training loops and sizeablee number of nodes in the hidden layer. In the proposed auto-tuning based on neural network has 10 nodes in the hidden layer and the learning rate & accuracy are set to 0.05 and respectively for fast convergence and accurate results. For the Feedforwardd Backpropagation Network(FBN), it is also important to preprocess the input data as the FBN uses a bipolar activation function. After preprocessing the values fed to the network would be in the range [-1, +1] as required by the FBN. Similarly, the output from the neural network must be given to a post processor module for the data to be mapped into actual tuning values from the output values in the range [-1, +1]. The tuner after receiving the estimated values of the tuning parameter executes certain commands to effect the changes to the dynamic tuning parameters namely, the DB_Cache, Shared_pool and Large_pool. In some cases if the estimated values are arbitrarily large due to sudden change in the input workload pattern or surge in user load, the tuner module may have to moderate the values before it changes the values of the tuning parameters just to ensure system stability. 81

8 4.2.1 Training Data-Set generation methodology As discussed in section 4.2.1, the role of framing training dataset is extremely important for the neural network based self-tuning method to accurately predict the values of the tuning parameters that are appropriate for the given user-load, workload type and database size. Hence, an experimental based method to evolve the training dataset has been presented in this section. The method starts by varying the value of a tuning parameter Tp for a given workload type and user-load and the corresponding response-time is noted. For instance, as shown in table 4.1, DB_CACHE tuning parameter is varied from 32MB to 1296MB for a user load of 5 and workload type TPC-C. In the next step the value of DB_CACHE tuning parameter where the response time reaches a minimum value is noted. In this case, the response time settles at 7 msecs. For a DB_Cache_size of 784MB. This value would be higher for higher user loads and scaling factors for obvious reasons. As can been seen from table 4.1, for 20 user-load, scaling factor of 2 and workload type TPC-C, the value of response-time that reaches saturation is 824MB and that for 80 user-load is 1264MB etc. This experiment was repeated for the scaling factors 4, 8 and 10 and similar tables were constructed. From these tables, the training data-set was constructed as shown in table 4-II, wherein the entries were derived from table 4.1. The process constructing training data-set from the experimental data is explained through the following algorithm. ALGORITHM Gen_Training_DataSet(Exp_Data_Table, maxnum, maxusld, NTps, wkldtypes) // Constructs the Training data set from the Exp_data that has maxnum of rows and maxsf number of columns //Input : Exp_Data_Table : table of response time values for different tuning parameters and scaling factors, the number of tuning parameters and workload types //Output: Training Dataset : Contains NTps columns one for each tuning parameter and number of rows depends on number of readings in Exp_Data_Table and workload types. Begin for i = 1 to wkldtypes // for each workload for j = 1 to NTps // for each tuning parameter for k = 1 to maxusload // for each scaling factor tp_min= min(exp_data_table, k) // Compute the tp value for which response time is minimum in the column 82

9 Insert_Data(TrainingDataset, j, tp_min) // Record the min value in Training dataset next k // repeat for each user load next j // repeat for each tuning parameter next k // repeat for each workload type End The self-tuning database system architecture proposed in this work uses three tuning parameters, that supports four workload types and the user-load varies from 1-10 or depending on the workload type. Table 4.1 Effect on Response-time due to increase in Db_Cache Db_Cache (MB) R-time(ms) (5 Users) R-time(ms) (10 Users) R-time(ms) (20 Users) R-time(ms) (40 users) R-time(ms) (80 Users) R-time(ms) (100 Users) Application of the above procedure to the experimental data in table 4.1 results in training dataset in Table 4.2 and Table 4.3. Though, a single table is used for storing the training data-set of all the workload types, they are shown in two separate tables for clarity. For instance, the first entry in Table 4.2 for user load 5, BHR=88.34 and scaling factor of 2 is selected from Table 4.1 that corresponds to the minimum response time entry at Db_cache_size value of 784. Table The Neural Network Training Data Set for the Workload Type TPC-C N BHR DBS DB_CACHE(MB) SH_POOL(MB) LRG_POOL(MB)

10 Similarly for user-load of 20, the corresponding Db_cache value of 824 from table 4.1 is chosen and entered in table 4.2. Since, the training data set in the table are derived from the experimental outcomes and not from empirical formulas or approximate models, the neural network after training with these set of values would accurately predict the tuning parameter values. Table 4.2 shows the sample training dataset for TPC-C workload type and scaling factors 2 and 5. As can be seen from the table, the DBS, SHP tuning parameters are set to progressively higher values as the user-load varies from and 1-10 for TPC-C and TPC-H workloads respectively. However, LARGE_POOL is kept constant as it has no effect on the response-time for this type of workload. It is to be noted that the training data-set shown in figure 4.2 and 4.3 is only the sample dataset. The co,mplete dataset used in the experimentation has 234 rows. Table Training dataset for TPC-H workload type N BHR DBS DB_CACHE(MB) SH_POOL(MB) LRG_POOL(MB) It is also important to invoke the self-tuning system at regular intervals based on certain criteria. The self-tuning can be invoked when the user-load increases by a constant. For example, at every 5 user-interval the self-tuning module can be run to check for noticeable change in performance degradation. However, an increase in user-load alone may not result in performance deterioration as the users the queries generated by users may not result in major disk activity. On the other hand, change in BHR may be a good indication that lot of disk activity is happening. Hence, a user-load change of 5-10 and 10 or 20% change in BHR can be used to trigger the tuning action. A user-load change of 10 and 20% change in BHR is used to trigger the tuning in the validation of results presented in the next section. 84

11 4.4.2 Result & Analysis The proposed adaptive self-tuning technique based on Neural network approach has been tested for two workload types namely TPC-C and TPC-E. 250 Response time in ms Responsetime V/s User load Scaling factor of 2 NeuroTech Auto-Tun Userload User Load Figure 4.4 : Query response time v/s User-load for TPC-C Workload SF=2 As can be seen from figure 4.4, the performance of Neural Network based tuning was found to be 27% and 16.3% better as compared to the auto-tuning feature of Oracle 10g for the entire user range of users for TPC-C Workload type and a scaling factors of 2 and 5 respectively. The up-down behavior in response-time in figure 4.4 and 4.5 is due to individual transactions of the workload take different amount of time to execute according to their level of complexity. This improvement in performance can be ascribed to the ability of the Neural network to learn from past data to predict appropriate values of the tuning parameters and also its ability to effectively utilize all the subcomponents of the system memory areas. However, the drop in performance from 27% to 16.3% is on expected lines, as the database size increases, the number of disk accesses will also increase. 85

12 % improvement in Response-time TPC-C SF=5 Response-time (msec) Auto NN User Load Figure 4.5 : Query response time v/s User-load for TPC-C Workload, SF=5 Figure 4.6 Query Response time V/s User load : TPC-E workload SF=1 Figure 4.6 shows the result of applying the proposed tuning method when the DBMS is presented with TPC-E workload type of scaling factor =1. Here again there is 32.8% improvement. This a significant performance improvement as compared to auto-tuning feature and also throughout the user-load range of users. Figure 4.6 shows the effect of scaling factor on response time. It is evident that the proposed method to self-tune the system is scalable. Summary In this chapter, a new adaptive self-tuning technique based on machine learning approach using Neural network is presented. In this setup, the sensor values after extraction are 86

13 pre-processed before being fed to the Neural network input. A three layer neural network with Feedforward Backpropagation based learning is used in the implementation. The experimental setup used though small compared to large databases[114] found in business enterprises, the self-tuning is most important in small and medium business enterprises where employing and expert DBA for manual tuning is very expensive. And in small and medium enterprises the database sizes are comparable to that used in the experimental setup. The training dataset is derived from a series of experiments and the methodology to construct the training data-set is presented. The self-tuning method has been validated for two workload types namely TPC-C and TPC-E and the results were found to be better as compared to auto-tuning feature of Oracle 10g. The performance improvement varies from workload to workload and also depends on the scaling factor. The performance improvement for TPC-C and TPC-E workload types was 27% and 32.8% respectively. 87

14 88