Design and evaluation of a system that coordinate clients to use the same server

Transcription

1 Linköpings universitet/linköping University IDA Department of Computer and Information Science Bachelor Thesis Information Technology Spring term 2017 LIU-IDA/LITH-EX-G--17/067--SE Design and evaluation of a system that coordinate clients to use the same server Erica Gavefalk Per Gustavsson Tutor: Mikael Asplund Examinator: Nahid Shahmehri

2 Linköpings universitet/linköping University IDA Department of Computer and Information Science Bachelor Thesis Information Technology Spring term 2017 LIU-IDA/LITH-EX-G--17/067--SE Upphovsrätt Detta dokument hålls tillgängligt på Internet eller dess framtida ersättare under 25 år från publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida Copyright The publishers will keep this document online on the Internet or its possible replacement for a period of 25 years starting from the date of publication barring exceptional circumstances. The online availability of the document implies permanent permission for anyone to read, to download, or to print out single copies for his/hers own use and to use it unchanged for non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional upon the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its www home page: Erica Gavefalk, Per Gustavsson

3 Students in the 5 year Information Technology program complete a semester-long software development project during their sixth semester (third year). The project is completed in mid-sized groups, and the students implement a mobile application intended to be used in a multi-actor setting, currently a search and rescue scenario. In parallel they study several topics relevant to the technical and ethical considerations in the project. The project culminates by demonstrating a working product and a written report documenting the results of the practical development process including requirements elicitation. During the final stage of the semester, students create small groups and specialise in one topic, resulting in a bachelor thesis. The current report represents the results obtained during this specialisation work. Hence, the thesis should be viewed as part of a larger body of work required to pass the semester, including the conditions and requirements for a bachelor thesis.

4 Abstract This thesis present a system which examine when clients should select a new server to maximize the number of packets received with the fewest number of server selections. The ratio between the number of packets received and the number of server selections should therefore be as high as possible. To help minimize the number of server selections, the system should be adaptive if the conditions would change. We construct a server selection mechanism which guarantees agreement between clients when called upon by the system. Furthermore, four policies have been created for when the mechanism is to be called. The purpose of this is to find out when and how it is best to switch server and how they can adapt to varying connection failure rates between server and clients. We evaluate the system by running simulations concluding that the policy we call on demand is best, which calls the server selection mechanism when clients have no connection to the server. This policy is the best if the aim is to send as many packets as possible and that the server selection mechanism is called few times with adaption to varying connection failure rates. This would enable the system to provide information even if the conditions will change to the better or to the worse.

5 Contents 1 Introduction Motivation Aim Approach Delimitations Background Consensus Replication Passive replication Active replication Failure model Related work Model System architecture Server selection mechanism Coordination policies Method Evaluation overview Good performance Simulator Experiment setup Implementation Implementation structure Simulation Server Client

6 6 Result The effect of different number of servers and clients Comparison between policies Varying connection failure Discussion and conclusion Discussion Conclusion Method Future work

7 List of Figures 1.1 The volcano with deployed clients and three base stations surrounding Passive replication Active replication System architecture depicting clients placed on the volcano An example of a connection matrix for a system of three servers and three clients Packets being sent to different servers Server evaluation will occur after a t time The classes and dependencies for the implementation An example of how the array may look for one client. The different boxes represent if the client has a connection (true) or not (false) Shows the correlation between the number of packets received with different number of servers and clients Shows the correlation between the number of coordinations with different number of servers and clients. Note that the Y-axis is logarithmic Showing the comparison of server selections (X-axis) and packets received (Y-axis) between policies with 3 servers and 9 clients (Note that the X-axis is logarithmic). n is the number of clients The policies shown are policy time interval and policy on demand

8 List of Tables 4.1 Default parameter values Default parameter values for policy time interval Number of servers and clients Shows the ratio, for different policies, between the number of packets received and the number of coordinations

9 1 Introduction Living near an active volcano involves many dangers and if an eruption occurs, time will be crucial. With the help of modern technology and warning signals, the safety for people living close have increased. We consider a scenario in which we monitor a volcanic island, with a network of wireless and portable seismometers, which we refer to as clients, as in Figure 1.1. These clients regularly send information to any of the base stations, which acts as servers, also seen in Figure 1.1. The information sent by the clients, is shared among the servers and used to detect if an eruption is on its way. For the clients to save energy, we want them to coordinate and agree on which server to use. Therefore, we implement a server selection mechanism and evaluate how often the server selection should be done. Figure 1.1: The volcano with deployed clients and three base stations surrounding 1

10 1.1 Motivation We have implemented a server selection mechanism which can coordinate which server the clients should use. The mechanism could be implemented in various scenarios and the focus was on how often the coordination needs to be done. To ease the replication of information among servers the clients should, as far as possible, always use the same server, i.e. the clients should have reached consensus regarding which server to use. Because of that, the replication of information among the servers do not need to be executed as often and will not put as much unnecessary load on the network as otherwise. Coordinating which server to use has a cost and we will investigate the problem of how to make the deviation of how many times the coordination should be carried out. We have constructed a scenario with clients and servers, as seen in Figure 1.1, which purpose is to monitor a volcano. We assume that all the clients are situated within close range of each other and because of that have a reliable communication and can always reach each other. The connections between the clients and the servers on the other hand, are unsafe and can sometimes fail, i.e. crash. This is due to the longer distance between the clients and the servers. 1.2 Aim When creating a system with the purpose of monitoring for peoples safety, multiple factors will be vital. Among these are the reliability of the system and that the information will reach its supposed target. The monitoring of a volcano benefits from being executed proactive, i.e. round-the-clock, since this will provide more time for action before the eruption starts [1]. Due to this, we do not want our clients to use more battery than necessary and every time the clients need to select a new server to use it will drain battery. We also want to minimize the need of frequent replication among servers and to achieve this, we want our clients to reach consensus regarding which server to use. Therefore, we want to measure how many packets that are sent from the clients to the servers and how often the server selection mechanism is called, i.e. number of coordinations. We want to maximize the total of packets received using as few coordinations as possible. This is to ease the replication among servers since each coordination implies that the clients have switched server. Problem statement In this thesis, we consider the problem of how clients can reach consensus regarding which server to use. We evaluate how many packets in total are sent from all clients to a server in relation to how often the system needs to run the server selection mechanism. We decided on four policies that would provide different settings for our evaluation: 2

11 Clients never coordinate their server selections. Clients coordinate which server to use every time a packet is to be sent from a client to a server. Clients coordinate which server to use after a pre-decided time interval. Clients coordinate which server to use on demand. The first two policies were chosen because they represent extremes of how often clients coordinate which server to use, never or for every request. The third was chosen to investigate how well it works to perform coordination at certain time intervals, regardless of how many clients are in contact with the server currently in use and without regard to the prevailing conditions. The last policy was chosen to investigate how well the system works when coordinating based on if clients have connection or not. 1.3 Approach We began with a study of different solutions on how to reach consensus and how we could use them in our implementation. We decided on a solution where the server which a majority of the clients have a connection to will be the chosen one. We continued with implementing the system and the server selection mechanism and then evaluated the system according to how many packets received, how often the server selection mechanism was called and how the different policies performed during varying connection failure rates. We used a script to perform simulations of our program and collect data about the number of packets received and the number of times the server selection mechanism was called. 1.4 Delimitations We consider a system composed of n clients and investigate how they may reach consensus regarding which server to use. Considering an implementation where the clients may decide between different servers. We assume that continuous exchange of information among clients is possible. We do not consider that clients nor servers may fail. We do not investigate the impact of processes sending faulty messages or any other similar arbitrary failures. Failures such as channel omission failures, i.e. a client not having connection to a server will be considered. We do consider a system where clients in all cases can send messages among each other and that the clients never will change. 3

12 2 Background This chapter introduce some theories as well as provide details concerning how clients can reach consensus, server replication. We present how our system can fail and lastly related work and how our work differs. 2.1 Consensus Consensus is a term which Coulouris et al. [2] defines as that multiple processes, p, will agree upon a unanimous value for which they will decide on the next action. To achieve this, every process, p i, in the system will start in an undecided state. From this state, every process will propose a single value v i from a pre-defined set D (i = 1, 2...N). The processes will then proceed by exchanging the values between each other in the system. Following the exchange, each process will set a decision variable d i (i = 1, 2...N), when doing so it goes from the undecided state to the decided state. When a process is in the decided state, it is not allowed to change d i anymore [2]. A consensus algorithm should always meet the following requirements for every time it executes (in which a correct process is defined as a process which do not suffers from any failures during execution): Termination: Eventually each correct process sets its decision variable. Agreement: The decision value of all correct processes is the same: if p i and p j are correct and have entered the decided state, d i = d j. Integrity: If the correct processes all proposed the same value, then any correct process in the decided state has chosen that value. 2.2 Replication Replication will make identical copies of data objects and provide the system with a backup. There are two kinds of replication, passive and active. 4

13 2.2.1 Passive replication Wiesmann et al. [3] describes passive replication as that clients will send all their requests to the primary replica which will handle the request and afterwards send updates to the other replicas, see Figure 2.1. That is, the replicas will not perform the request but implement eventual changes that is a result of that request. If the primary fails, passive replication will need a high reconfiguration cost to determine which replica to use instead [3]. Figure 2.1: Passive replication Active replication Wiesmann et al. [3] describes that the main approach for active replication is that all replicas receive and process the same sequence of client requests. Since all replicas are provided with the same information in the same order, they will produce the same output and consistency is guaranteed [3]. Figure 2.2 show a simple picture of the process. Figure 2.2: Active replication 2.3 Failure model We only consider channel omission failures between a server and a client, not between a client and a client. An omission failure occurs when a communication channel cannot fulfill its purpose. 5

14 Channel omission Channel omission failure occurs if a client has no connection to the server. That is, the client cannot send packets to the selected server. These failures could in a real-life scenario be caused by bad weather or similar causes or network transmission error. We will not differentiate between the different reasons for a channel omission failure. 2.4 Related work There exists much research regarding server selection and when it should be executed. Wendell et al. [4] propose a method for decentralized server selection for cloud services, with the goal to create a protocol designed to help server selection for clients. The protocol is meant to operate as a load balancer between the servers, which acts as a cloud service, and the clients using the cloud. The aim is to implement replica selection which achieves scalability and reliability from both server and client-based selection. The performance was evaluated through different methods of choosing replica, the measurement is given in the distance travelled by the packets when looking at a global scale of distribution. The conclusion is that because of using a selection protocol as a middleware, the geo-distance travelled can be significantly lower than using a "Round-Robin"-protocol. 1 Boru et al. [5] present an investigation regarding energy-efficiency in cloud computing using datacenters. The underlying theory is that datacenters or databases that are closer in location to the clients is more energy-efficient than centers located far away from the clients. They tested this theory using Green Cloud 2 with mathematical models to calculate the improvements possible with using nearby datacenters. The conclusion is that by using replicas close to the users can improve not only the energy consumption but also the required bandwidth as well as the delay regarding the network traffic. Replica selection in the cloud environments using an ant colony algorithm [6] propose an algorithm for replica selection mechanism to achieve the best performance. The model is based on ant colony optimization to improve average access. They present the Ant Colony Optimization algorithm (ACO) which uses a graph in form G = (V, E) where V represent vertexes and E edges. Initialization start with one ant moving from the start node, in the graph, and then select the best node to move to. The algorithm starts by (after initialization) calculating probability of replicas to be selected. The algorithm continues with selecting the replica by using the response time to choose the file replica. Then it will update the pheromone information, which tells how likely the file replica is to be chosen. The pheromone information is used by the ants to decide which replica to select. The first round there are no pheromone values spread across the graph and this round is used to plant pheromone in graph edges. After 1 Round-Robin is an algorithm which chooses the next server in line until there exist no more servers and thereby start over. Going through all servers equally. 2 Green Cloud is a packet-level simulator which evaluates energy-consumption in cloud services. 6

15 the update of values the algorithm will terminate if finished or start again by calculating probability and follow the steps once more. The results from the simulations showed better performance in average data access time and bandwidth than compared protocols. Wei et al. [7] propose a model to capture the relationship between how available the replica is and its replica number. They introduce a dynamic replication management protocol, CDRM, which leverages this model to calculate and maintain minimal replica numbers for given availability requirements. The location of a replica is based on its capacity and the likelihood that data codes are blocked. If you change the replica number and location based on node capacity and changes in workload, it is possible for CDRM to reset the load dynamically between data nodes in the heterogeneous cloud. The evaluation shows that the CDRM is cost effective in terms of performance and load balancing of large-scale cloud storage and can adapt to changes in environment regarding data node failure and changes in workload, while the number of replicas do not rise to high. This creates accessibility to replicas and improves access latency, load balance and keeps the entire storage system stable. These articles relate to our project in terms of investigating different aspects of coordinating which server replica to use. They take into consideration the location, energy consumption and availability. We have instead considered if the clients have connection to the replica or not and evaluating the performance of the server selection mechanism in terms of how many coordinations that have been done in correlation to how many packets that were sent. As in Wei et al. [7], we also chose to use the best server with the lowest replica number available. 7

16 3 Model In this chapter, a detailed description of the system model will be presented. Including the system architecture and the server selection mechanism. As well as a, thorough description of the four policies regarding when the system should call the server selection mechanism. 3.1 System architecture We have constructed a system implementation consisting of m servers and n clients. The servers and clients have communication links between them and are therefore able to communicate. These links may fail, illustrated with the solid lines in Figure 3.1. The clients have links between them, illustrated with the dotted lines in Figure 3.1 and are able to communicate among each other. We assume that the client-client links are stable and clients may never leave the network. Figure 3.1 shows a simplified picture over how the servers and clients are linked together. The red client, which as well is surrounded by a circle, is the leader client which is chosen by the program on initialization. A leader client is an extension of a regular client and thus has the same functionality as other clients. What differs the leader client from a regular client is the ability to decide which server to use, the leader client will contain the server selection mechanism which determines the new server. The leader client will be able to gather the other clients connections by sending a broadcast message telling the clients to respond with their current connections. Since we will not be able to guarantee connection between servers in our scenario we consider passive replication to be the most effective. This will also provide us with a simple way to see which server that handle the request from the client. Even though passive replication need a high reconfiguration cost we value more that the servers may work independently and send updates when there exists connection. The clients could also be able to notify the server when they decide on a new server and thus the cost for servers to reconfigure which server to be used will be non-existing, unlike for active replication where the servers need constant connection to each other. To use active replication between the servers will also increase the network load which we want to avoid, as well as it makes the use of coordination between the clients unnecessary. By using few coordinations, the reconfiguration cost of using passive replication will be lower than using active replication since the servers do not need to continuously send data to 8

17 each other. Figure 3.1: System architecture depicting clients placed on the volcano 3.2 Server selection mechanism Our server selection mechanism should achieve consensus among clients regarding which server to use next. The system will initiate the process of reaching consensus at different times depending on which of the four policies that is currently in use. This process will find the server which as many clients as possible have connection to: 1. If there exists a server which every client has a connection to, it will be the new selected server. 2. If there exists more than one server which have the same number of clients with connection to, the first server in order will be chosen. 3. If there do not exist a server which all clients have connection to, the mechanism will choose the server with as many working connections as possible Since no client or server never may leave the network, the group composition will always be the same. Because of this, we are not in need of a more complex selection mechanism than the mechanism described, which chooses the best suitable server for the moment. Server selecting is executed in the method coordinate, seen in Algorithm 1, which returns the chosen server. The method will loop through the matrix connections, which contains the connections provided from each client, this can, for example, look as in Figure

18 Figure 3.2: An example of a connection matrix for a system of three servers and three clients Algorithm 1 Server selection mechanism Result: The new chosen server for all servers do for all clients do if client has connection with server then increase the number of clients with connection to that server end end if new server has more connections than currently chosen server then new server becomes new primarily chosen server end end 3.3 Coordination policies We will examine four different policies for when the clients should call the server selection mechanism. These are policy never, for every request, time interval and on demand. In this section, we describe the policies more thoroughly and provide pseudocode for each one. Policy 1: Never The first policy, never, will always try to send packets to the same server. For this policy, when the clients do not have a connection to the server, they will not send any packets and will only send when there is a connection. 10

19 Figure 3.3: Packets being sent to different servers Algorithm 2 shows the pseudocode for policy never. A client will send packets if it has connection to the server it is using. Algorithm 2 Show how policy never calls the server selection mechanism while do if Client has connection to current server then Send packet to the server Wait to send next packet else Wait for reconnection end end Policy 2: For every request The second policy, for every request, is that clients call the server selection mechanism every time they are about to send a packet. This will guarantee the consensus between the clients but will also need to use the server selection mechanism for each packet to be sent. The pseudocode for policy for every request is shown in Algorithm 3 and during the whole run the client will for every packet it is about to send first call the server selection algorithm. Algorithm 3 Show how policy for every request calls the server selection mechanism while do Call for server selection Receive new selected server Send packet to new server Wait to send next packet end 11

20 Policy 3: Time interval The third policy, time interval, is that clients will call the server selection mechanism after a pre-defined time interval t regardless if the clients have connections to the server in use or not. The markings in Figure 3.4 represent every time the call the server selection mechanism is appointed. This will not consider whether clients have connections which may result in clients not being able to send packets to a server. A benefit of using this implementation is that the leader client does not need to ask the other clients for their connections, i.e. broadcast for connections, because the clients will know when to send their connections to the leader client. Figure 3.4: Server evaluation will occur after a t time In Algorithm 4, pseudocode for policy time interval is provided. The policy will for the whole run, if it is in the range of the pre-defined given time interval, send packets to the server. When the time interval exceeds, the server selection will be called and the policy will continue send packets to the new server until the next time interval exceeds. If the client loses the connection to the server before the time interval exceeds it will stop sending packets until a new server is chosen. Algorithm 4 Show how policy time interval calls the server selection mechanism in a time interval while do while Within time interval do if Client has connection to current server then Send packet to the server Wait to send next packet else Wait until time runs out end end Call server selection Receive new selection end Policy 4: On demand The fourth policy, on demand, is that the leader client will call the server selection mechanism when 12

21 clients with connection α n where α is a real value in the range 0 to 1. This will be beneficial when the environment is unstable and the connections change often. Algorithm 5 provides pseudocode for policy on demand. The policy will examine that a certain number of clients have a connection to the server. Then the code provides the client view, sending packets to the current server. If the client do not has a connection, it will report that it has no connection and wait until a new server is given by the selection mechanism. The selection mechanism will be called when a certain number of clients does not have a connection to the current server. Algorithm 5 Show how policy on demand calls the server selection mechanism while do if Clients with connection is more than α n then if Client has connection to current server then Send packet to the server Wait to send next packet else Report that there is no connection Wait for new server end else Receive new server end end 13

22 4 Method In this chapter, we describe the method used to create, simulate and evaluate the protocol. Furthermore, in this chapter we also present our experiment setup with descriptions for the choices made in the implementation of the protocol. Finally, the default parameters values and system parameter values is presented. 4.1 Evaluation overview To evaluate our system, we have performed simulations. We have evaluated the four different policies and how they perform. In this section, we describe what we consider to be good performance, our experiment setup, default parameter values and system parameter values. We consider two parameters to evaluate, the number of packets received by all servers and the number of coordinations that have been performed by the system, i.e. how many times the server selection mechanism were called. Each client count the number of packets they have sent which at the end of the test is summarized, resulting in all packets received by the servers that is sent by the clients in the system. The second parameter, coordinations, is a number which increments each time the server selection mechanism is called upon. 4.2 Good performance We evaluate how good the policies perform and base our conclusion upon this. Good performance for a policy equal many packets received and that the server selection mechanism is called few times, in comparison to the other policies. The policy should also be reliable if the link conditions change and adapt with a rising number of coordinations with a higher connection failure rate. We try each policy for different connection failure rate, from 0 % 90 % failure. This failure rate will change the connections between the clients and servers after a pre-defined time interval. Thus, the ratio between the number of packages sent and the number of coordinations should be as high as possible and the policy should be adaptive to changing circumstances. We consider the number of packets received and the number of coordinations equally important and therefore, we have not weighted them. 14

23 4.3 Simulator To be able to test our protocol, we built a simulator which creates clients and servers. In the simulator, each server runs in its own thread, which allows the program to run concurrently and create multiple connections with the same characteristics which can execute in parallel. As well, each client runs in its own thread with a connection to the server which is currently in use. The connection also runs in its own thread. Packets cannot be lost but a client can lose its connection to a server. Every cycle of the simulation there is a probability x that one of the connections will be lost. 4.4 Experiment setup To provide a method with replicability we have decided on some features and values for how we implement our model. The parameters for the system is chosen with a real-life scenario in mind [8]. Since we are monitoring a system for a long period of time, the system has been modified to simulate approximately one year of testing. The packet send rate has been set to be as if the clients send packets 4 times a day, i.e. every sixth hour. This value is estimated for what should be a good rate in a real-life scenario [8]. To be able to efficiently investigate performance of the system, some parameters have been chosen as default values and therefore will be fixed; these are shown in Table 4.1. Table 4.1: Default parameter values One run One year Packet send rate (R) 4 packets/day The failure rate will be 20 % for all simulation except when we vary the connection failure rate. For the policy time interval, we have chosen some extra parameters regarding when to investigate which server to use. These parameters can be seen in Table 4.2. Table 4.2: Default parameter values for policy time interval t 1 Every day Every week t 2 For policy on demand we have chosen to investigate which new server to use for when 0.5n, half of the, and 0.25n, quarter of the, clients do not have connection. If there is an uneven number of clients the results are rounded up to the closest integer. System parameter values Since the clients might need to cover areas with different sizes we evaluate different configurations regarding multiple clients. The same is applied for servers, since we might have/or need more servers depending on the conditions. Every policy is tested for the values presented in Table

24 Table 4.3: Number of servers and clients Number of servers m Number of clients n The number of servers and clients can be combined and varied as preferred. We also consider a change of conditions, i.e. a higher or lower connection failure, and investigate which of our four policies that performs best regarding how many packets received and how many times the server selection mechanism is called. 16

25 5 Implementation In this chapter, we describe how we have implemented our model by using our previously described method. We begin with a description of the implementation structure, continue with describing how we define a packet and how the classes are constructed 5.1 Implementation structure In the implementation of our system, we have created five classes: Simulation Client Leader Client Server Server Thread The Simulation class which is the environment where the clients and servers are created upon initialization. It enables the system to have the connection failures which is independent of the clients and servers. The Simulation class also keeps track of time as well as tracking how many of the clients that is in a waiting state, i.e. is waiting for a new server. The Client class connects to the Server class, visualized with a solid line in Figure 5.1. The server class creates a thread which responds the Client class, the dotted line in Figure 5.1 represent the internal communication between the thread and the server class. Since the server selection mechanism is in the Leader client class, the leader client and the regular client will have to communicate with each other, but doing so without sending packets, this is visualized with a dotted line i Figure

26 Figure 5.1: The classes and dependencies for the implementation Packets We have defined a packet as a datagram sent between a client and a server. The packets only contain an incrementing number representing the packets ID. Packets are sent between sockets. We guarantee that these packets reach their destination, i.e. if a client has sent a packet it will be delivered to the server. 5.2 Simulation Simulation is the class which creates the environment that enables the evaluation to be executed. In Algorithm 6, the pseudocode for the Simulation class is shown, it starts with creating the clients and servers and start up the simulation. After the initialization, the simulation run for 365 seconds and it calls the server selection mechanism if the condition for the current policy is fulfilled. The Simulation class also change the connections between the servers and the clients each cycle of the while loop. The Simulation class will also tell the clients which server they should connect to, based on the result from server selection mechanism, as well as keep track of how many clients that is in a waiting state. 18

27 Algorithm 6 Show the pseudocode for the simulation algorithm Create clients Create server for t in 1,...,365 do for Every client do Change network connections end if Policy condition is fulfilled then Call for server selection at the leader client Tell clients to switch to the new server end end Server The Server class is a class which run in its own thread. It uses a socket to listen for new clients trying to connect. Each server has a unique port number for the clients to use when connecting. When a client is trying to establish a new connection to a server, the server will create a new thread with the class Server thread which provides the functionality for the client to communicate with. This is to allow for multiple clients to connect to the same server. Pseudocode for the server can be seen in Algorithm 7. Algorithm 7 Show the pseudocode for the server class if Client trying to connect then Create new thread Respond the client else Wait for a new client to try to connect end Client The Client class runs in its own thread and uses a socket with a unique port number to connect with the server. Each client also has an array with its current connections, an example can be seen in Figure 5.2 which determines if the client has a connection to a server or not. A client without a connection to the current server will be in a waiting state to get a new server to use instead. The array with connections are changed by the Simulation class previously described. Pseudocode for the server can be seen in Algorithm 8. 19

28 Figure 5.2: An example of how the array may look for one client. The different boxes represent if the client has a connection (true) or not (false). Algorithm 8 Show the pseudocode for the client class Connect to server while Has connection to the server do Wait 250 milliseconds Send packet to the server end Wait for new server Leader client The Leader Client class is an extension of the Client class. This provides the leader client with the same functionality as a client. The difference between the two classes is that the leader client has the selection mechanism, which will be called upon by the Simulation class if needed. This method return the result of the mechanism. 20

29 6 Result In this chapter, we present the results from our evaluation of the system. We implemented a script that could run our program 30 times for each policy and provide data which could be visualized. We describe how the number of servers and clients affect the performance. We also show comparisons of the four different policies and how they perform during varying connection failure rates. 6.1 The effect of different number of servers and clients Figure 6.1 show the results for how many packets received for the four policies and Figure 6.2 show the number of coordinations for all policies except policy never, which never coordinate among each other which server to use. Therefore, the result is zero no matter how many clients or servers that participate. As seen in Figure 6.1 the three bars for each number of clients (9, 29 and 99) shows the three different number of servers (3, 9 and 12) when the packet loss rate is at 20 %. From the figure, it is possible to see that the number of servers participating have a small effect on the number of packets received. There exist differences but we consider these to be too small to be significant. However, the number of clients participating affects how many packages are sent. That is, the effect on how many packets that are sent and number of coordinations is made is only related to how many clients that participate and not to the number of servers participating. In Figure 6.2, it is possible to see that the number of coordinations are not affected by neither the number of clients nor the number of servers participating. For policy for every request the number of coordinations is decreasing with an increasing number of servers. This is due to the time-limit in the test resulting in that the system does not have enough time to coordinate with so many coordinations in such short amount of time, this would not happen in a real-life scenario. Thus, we do consider that the number of clients or the number of servers participating do not affect the number of coordinations 21

30 Figure 6.1: Shows the correlation between the number of packets received with different number of servers and clients 22

31 Figure 6.2: Shows the correlation between the number of coordinations with different number of servers and clients. Note that the Y-axis is logarithmic 6.2 Comparison between policies Figure 6.3 show the four policies using 9 clients and 3 servers, with the connection failure rate 20 %. Figure 6.3 show the relationship between the total number of packets received during the simulation and the total number of times the policy has called the server selection mechanism. The figure also show that the performance varies regarding the number of packets received between the policies. 23

32 Figure 6.3: Showing the comparison of server selections (X-axis) and packets received (Yaxis) between policies with 3 servers and 9 clients (Note that the X-axis is logarithmic). n is the number of clients. Table 6.1: Shows the ratio, for different policies, between the number of packets received and the number of coordinations Policy Ratio Policy: For every request 3.5 Policy: Time interval once a day 30.8 Policy: Time interval once a week Policy: On demand 0.5n 86.3 Policy: On demand 0.25n 73.0 As seen in Figure 6.3, policy never has no server selections 1 and achieve a moderate number of sent packets. Since the clients do not need to communicate with each other, their battery life will be dedicated to sending packets regarding the state of the volcano. This policy performs well with packets received but since the policy only will use the same server, it will be vulnerable to major faults at that server. The opposite of this is policy for every request, where information sharing between servers only is needed when the clients change server. However, policy for every request has a lot more coordinations than the other policies and even if the clients have reached consensus for every sent packet the coordinations drain the clients battery. The ratio, seen in Table 6.1 between the number of packets received and the number of coordinations is low compared to other policies and thus it will not achieve good performance. Also, policy time interval once a day have a low ratio, even if it is higher than the one for policy for every request, and does not achieve good performance. 1 On initialization the server selection mechanism will select the first server in order for the clients which gives policy never a value of 1 in the Figure, but is not accounted for as a coordination. 24

33 Policy time interval for once a day has fewer packets received and more coordinations, even though the difference is not that big it affects the overall performance and the ratio is about half compared to policy on demand for both 0.5n and 0.25n. Policy time interval for once a week and policy on demand for 0.5n have the lowest number of packets received and the number of coordinations is about half to policy on demand 0.25n. Policy on demand 0.25n on the other hand, has the most packets received out of all policies. Both policy time interval once a week and policy on demand for both configurations, have a high ratio, as seen in Table 6.1, and thus achieve good performance. 6.3 Varying connection failure Figure 6.4 shows how the number of packets received, Figure 6.4a, and the number of times the server selection mechanism is called, 6.4b, are effected by the connection failure rate. The result is tested for 9 clients and 3 servers, with different probability for connection failures. Both representing policy time interval with the configuration of once a day and once a week as well as policy on demand with configurations 0.5n and 0.25n. For policy time interval, the Figure 6.4, show a steady number of selections, disregarding the failure rate of connections. For policy on demand the figure shows a logarithmic growth resulting in a similar amount of times for selections as policy time interval at connection failures at 60% or more. The number of packets received is decreasing with an increased connection failure rate. Policy time interval with the configuration once a week has a steeper decrease compared to the other three configurations. (a) Shows how the number of packets received (b) Shows the number of coordinations when the vary with a varying connection failure rate connection failure rate varies Figure 6.4: The policies shown are policy time interval and policy on demand During varying conditions, the policies will perform differently. During conditions that change often, a policy should adapt and change the number of coordinations. However, this is not the case for policy time interval which is sensitive to changes in the 25

34 client-server link condition, as seen in Figure 6.4b. The number of coordinations do not change with the connection failure rate, no matter if it is for the interval once a week or once a day. If the conditions would deteriorate distinctly and the time interval does not match this, important information may not arrive fast enough since the coordination is executed with to long interval between. Policy time interval once a week achieved a high ratio but since it is not adaptive to varying connection failure rate it does not achieve good performance. Policy on demand will however be able to adapt to scenarios where the conditions changes over time, where the dynamics of the system will adapt without the need to change the systems parameters. This can also be seen in Figure 6.4b where policy on demand, for both 0.5n and 0.25n, have higher number of coordinations for higher connection failure rate. Thus, both 0.5n and 0.25n achieve good performance regarding adapting to varying connection failure rate. 26

35 7 Discussion and conclusion In this chapter, we discuss how our result achieve good performance, how the performance has effected consensus as well as the replication needed. We also present our conclusion of this thesis, discuss our method and propose some future work. 7.1 Discussion We have evaluated how many coordinations each policy needed to do and how many packets they send and according to our definition of good performance, we can see that the policy on demand performance the best for both stable and varying connection failure rates. When the link conditions are good, policy on demand will not drain battery to do unnecessary coordinations. This is especially for policy on demand with both configurations which achieve a high ratio between the amount of packet sent and the number of coordinations as well as it is adaptive. The configuration 0.25n also have the highest number of packets received for failure rate at 20 %. The other policies tested for, either put a lot of unnecessary load on the network during lower connection failure rates or will send few packets during a higher connection failure rate due to that multiple clients are in the waiting state. We have not weighted the ratio between how many packets received and the number of coordinations since we have thought them to be equally important. But depending on the aim of the system it is possible to do so and all policies could be suitable depending on what you want to achieve. In this thesis, the implementation has strived towards sending a maximum number of packets to the same server by achieving consensus between the clients. Our result however has not tested for if a particular server has received more packets than others. But, from our result we can see that the number of packets received without consensus among clients can be even higher with consensus among clients. This is the case for policy on demand for 0.25n which then also will not put as much load on the network since it requires less frequent replication among servers. Also, the replication could be coordinated to be done when the clients change server. 27

36 7.2 Conclusion The aim of this thesis was to investigate how many packets that in total are sent from all clients to a server in relation to how often the system needs to run the server selection mechanism. We have done this by implementing a server selection mechanism. The system has been tested for four different policies for when to run the server selection mechanism. The policies could all be useful when monitoring a volcano. If the aim is a reliable and adaptive system policy on demand is preferable and use approximately 0.25n as a parameter. This will achieve a high ratio between how many packets received and how many times the server selection mechanism is called if compared to other policies. Policy on demand is also most reliable if the link conditions suddenly get worse than expected since it will adapt Method When creating the simulations for the system, the narrow time frame limits the system in a couple of aspects. Since the test is only for 365 seconds, some methods used in the implementation executes for a long time and therefore creates a bottleneck in the system, which occurs when using policy for every request. This would not happen in a real-life scenario when time between packets is 6 hours, which makes plenty of time for coordinations and calculations to be executed. A solution to this problem could be to increase the time running the simulations, but this would have increased the time needed to test all policies which was not possible with our time-frame. Another alternative could have been to use a round-based system which is not dependent on the execution time of methods, this however would have increased the time for the tests as well, but would have eliminated the problem. Another disadvantage of using a round-based implementation is that in a real-time analysis of the simulation, the round-based implementation will not reflect the reality as well as using a time-based implementation. Policy time interval and on demand could have been simulated with more values, i.e. more values for t and α. This would show how the behavior for each policy changes when the values change and could also have enabled to find an optimal value. When performing our simulations, we could have measured how many packets the system would have sent if all clients always have connection to the server currently used. We could then instead of using how many packets received used how large percentage of the packets tried to be sent that could be sent. With this implementation, regardless of which server selections mechanism that is used, the result should be the same as ours since the test concluded when the coordination should be performed and have nothing to do with how the selection is made. The evaluation has been executed on several different computers with identical results and thus should our program produce the same results if the evaluation should be done again. If implementing a new similar program the results should present the same pattern as ours. 28