Provably Secure Competitive Routing against Proactive Byzantine Adversaries via Reinforcement Learning

Transcription

1 Provably Secure Competitive Routing against Proactive Byzantine Adversaries via Reinforcement Learning Baruch Awerbuch David Holmer Herbert Rubens Abstract An ad hoc wireless network is an autonomous selforganizing system of mobile nodes connected by wireless links where nodes not in direct range communicate via intermediary nodes. Routing in ad hoc networks is a challenging problem as a result of highly dynamic topology as well as bandwidth and energy constraints. The Swarm Intelligence paradigm has recently been demonstrated as an effective approach for routing in small static network configurations with no adversarial intervention. These algorithms have also been proven to be robust and resilient to changes in node configuration. However, none of the existing routing algorithms can withstand a dynamic proactive adversarial attack, where the network may be completely controlled by byzantine adversaries. The routing protocol presented in this work attempts to provide throughput competitive route selection against an adversary which is essentially unlimited; more specifically, the adversary benefits from complete collusion of adversarial nodes, can engage in arbitrary byzantine behavior and can mount arbitrary selective adaptive attacks, dynamically changing its attack with each new packet. In this work, we show how to use the Swarm Intelligence paradigm and Distributed Reinforcement Learning in order to develop provably secure routing against byzantine adversaries. A proof of the convergence time of our algorithm is presented as well as preliminary simulation results. 1 Introduction The motivation for this work is to design robust routing for wired overlay networks or mobile ad hoc wireless networks (MANET s). Although routing in ad hoc wireless networks has unique aspects, many of the security problems faced in ad Authors are with the Computer Science Department, Johns Hopkins University, Baltimore, Maryland. {baruch, dave, herb}@cs.jhu.edu Technical Report 5/16/2003. hoc routing protocols are similar to those faced by wired networks. Wireless security issues are more acute, due to the inherent vulnerabilities of a wireless environment and are the focus of this paper; however our ideas are applicable to wired overlay networks as well. In general, routing protocols are susceptible to a wide variety of attacks. A malicious node may advertise false routing information, try to redirect routes, or perform a denial of service attack by engaging a node in resource consuming activities such as routing packets in a loop. Furthermore, due to their cooperative nature and the broadcast medium, ad hoc wireless networks are more vulnerable to attacks in practice [27]. A great deal of work has been done in terms of guaranteeing practical security considerations in existing network protocols. In practice, adversarial attacks observed and documented in ad hoc networks might not be overly sophisticated. The ease of access to the medium has allowed extremely basic attacks to cause a great deal of damage. Consequently, such attacks can be thwarted by simple yet effective methods. For example: a mis-routing attack can be easily detected by authenticating the packet path; consistent traffic blocking can eventually be detected; a single adversary that does not collude can be detected by its neighbors; trusted servers or sensors an be used to monitor truthfulness of link state databases, etc. For each of these important special cases, a lot of great work was done, which is extremely important in practice. However, existing work does not come anywhere close in either addressing the sophisticated attacks arising in our dynamic adversarial model, or attempting to prove analytical bounds on packet loss. Our Contribution: The goal of this paper is to design an on-demand flooding-free routing protocol in a dynamic byzantine adversarial environ- 1

2 ment. We propose a generic framework for routing protocols which are appropriate for networks operating under this extremely strong adversarial model. Such strong models have not been considered in the literature to the best of our knowledge. In fact, one does not even need to consider the full power of byzantine attacks. Even relatively benign adaptive dynamic denial of service attacks are already sufficient to break most existing algorithmic work. (In section 2.6, we briefly describe why DOS attacks are so devastating.) What is remarkable about our result is the ability to prove near-optimality bounds under completely arbitrary adversarial behavior, with essentially no assumptions about either the network, or the underlying security infrastructure. We use techniques similar to the Swarm Intelligence and Distributed Reinforcement Learning paradigm. Swarm Intelligence is a set of learning and biologically-inspired approaches to solve hard optimization problems using distributed cooperative agents. Motivation comes from work which explored the behaviors of ants and how they coordinate each others selections of routes based on a pheromone secretion. As in [50], one can imagine a model of network routing, such that the network is populated by artificial ants (packets) that make use of the trail laying principle; at each node an ant encounters on the journey to its destination, it leaves an amount of pheromone which evaporates with time, but is an increasing function of the frequency of traversal of that location. The ant then selects the next node on its journey on the basis of the local pheromone distribution [50]. The routing decision is determined by these pheromone distributions. In our Byzantine routing approach, we act similarly: the process of route detection and fault avoidance is carried out by a distributed process of learning fault free paths, in spite of deceptive techniques pursued by adversaries. The routing process creates and adjusts a probability distribution at each node for the node s neighbors. The probability associated with a neighbor reflects the relative likelihood of that neighbor forwarding and eventually delivering the packet to the destination. It may appear that our Byzantine-inspired routing model may lead to impractical algorithms in typical (non-byzantine) settings. However, antinspired routing algorithms were developed and tested in real network environments by British Telecomm and NTT for both wired and wireless networks with superior results [13, 56, 18, 14, 48, 50, 11, 21]. AntNet, a particular such algorithm, was tested in routing for data communication networks [13]. The algorithm performed better than OSPF, distributed Bellman-Ford with various dynamic metrics, and various modifications of shortest path with a dynamic cost metric [9, 48]. 2 Network and Security Model In the following section we describe both our network and security models and any assumptions that we make. We also fully explain the adversarial model that we consider in this work and the challenges to mitigating the adversarial threats from both an algorithmic and security stand point. 2.1 Network Assumptions Bi-directional communication channels. Our protocol does not require bi-directional communication on all links in the network, but it will seek only bi-directional routes from senders to receivers. Bi-directional communication is also a requirement of most wireless MAC protocols, including [2] and MACAW [10]. We focus on providing a secure routing protocol, which addresses threats to the ISO/OSI network layer. No knowledge of the network topology. As mentioned before, topology databases may be completely corrupt. We only assume that nodes that join the network flood their authenticated identity thru the whole network, so each node knows the potential membership in the network. We do not specifically address attacks against lower layers. For example, the physical layer can be disrupted by jamming, and MAC protocols such as can be disrupted by attacks using the special RTS/CTS packets. Though MAC protocols can detect packet corruption, we do not consider this a substitute for cryptographic integrity checks [55]. While our protocol is not trying to prevent any of these attacks, it should be able to overcome such attacks. 2

3 2.2 Security Assumptions Authentication and Symmetric Cryptographic Primitives. We assume existence of efficient cryptographic primitives. This requires pairwise shared keys between the sender and other nodes in the network 1 which are established on-demand. The public-key infrastructure used for authentication can be either completely distributed (as described in [29]), or Certificate Authority (CA) based. In the latter case, a distributed cluster of peer CA s sharing a common certificate and revocation list can be deployed to improve the CA s availability. However, the issues of establishing authentication mechanisms are beyond the scope of this paper. No trusted parties assumed. This work does not assume the existence of any trusted servers or routers. In this work we consider only the source and the destination to be trusted. The only assumption is that any intermediate node on the path between the source and destination can be authenticated and can participate in the protocol. Nodes that can not be authenticated do not participate in the protocol. 2.3 Considered Attacks Even authenticated nodes may exhibit byzantine behavior at times. The goal of our protocol is not to explicitly detect byzantine behavior (which is essentially impossible) but rather to avoid byzantine participants in such a way that the traffic losses are nearly optimal. We define byzantine behavior as any action by an authenticated node that results in disruption or degradation of the routing service. We assume that an intermediate node can exhibit such behavior either alone or in collusion with other nodes. More generally, we use the term fault to refer to any disruption that causes significant loss or delay in the network. A fault can be caused by byzantine behavior, external adversaries, lower layer influences, and certain types of normal network behavior such as bursting traffic. 1 We discourage group shared keys since this is an invitation for impersonation in a cooperative environment. An adversary or group of adversaries can intercept, modify, or fabricate packets, create routing loops, drop packets selectively (often referred to as a black hole), artificially delay packets, route packets along non-optimal paths, or make a path look either longer or shorter than it is. All the above attacks result in disruption or degradation of the routing service. In addition, they can induce excess resource consumption which is particularly problematic in wireless networks. One strong attack, referred to as a wormhole [28], is where two attackers establish a path and tunnel packets from one to another. Our protocol addresses this attack by treating the wormhole as a single link which will be avoided if it exhibits byzantine behavior, but does not prevent the wormhole formation. Also, we do address traditional denial of service attacks which are characterized by packet injection with the goal of resource consumption. The following Section 2.4 provides systematic description of all attacks handled. 2.4 Adversary models overview We now provide the list of all the possible adversarial models captured by our work. We would like to classify adversaries based on a number of dimensions. In each dimension, we rank the adversaries based on its power. We then address the adversary which is the strongest in each dimension. No prior work provided a bounds on packets loss under such a stringent adversarial model. Malignancy of the attacks. A benign adversary can drop packets, e.g. exercise a remote Denial of Service (DOS) attack, but cannot infiltrate routers and cannot not engage in active attacks such as modifying or mis-routing (tunnelling) messages or acknowledgements. Static malicious adversary is the next level up in the adversarial hierarchy. This class of attacks allows the adversary to permanently control a static subset of the routers. An adversarial network node (router) may employ the following attacks: Libel: assigning the blame to its neighbors for the failure to transmit or receive packets. Impersonation: else. pretending to be somebody 3

4 Mis-routing: sending a message in the wrong direction. Topology-distortion: incorrectly advertising topology information to draw traffic into itself (dropping it later). Ack-distortion: attempting to send fake acknowledgements to confuse the protocol and, basically do anything else an adversary can think of to confuse and disrupt routing algorithm. Proactive malicious adversaries: is the ultimate adversary that we would like to confront. It may take control of different routers at different times, getting in and out, completely at will, in spite of these routers being previously authenticated. While a router or edge is possessed by an adversary, it executes any of the byzantine or DOS attacks above. Collusion power of adversary. The easiest case involves no collusion: each adversarial router acts on its own. The hardest possible case that we would like to handle in this paper is complete collusion: all adversarial routers act as a team, engaging in simultaneous collusion with other routers which are under adversarial control (e.g., enabling tunnelling or wormhole attacks, as well as switching blame between compromised routers for the purpose of avoiding detection.) Adaptivity/Intelligence of the adversary. The easiest model is that of a completely static adversary. This adversary designs a static fault pattern ahead of time, while knowing the algorithm, but without knowing any coin flips used by the algorithm. It applies some pattern of attacks to all the packets on a given router (edge). It completely destroys all static deterministic or randomized algorithms, such as min-hop routing path, random path, random selection among all edge-disjoint paths, etc.) However, some of the existing work on byzantine routing handles such an adversary, e.g, [7]. Oblivious dynamic adversary designs a dynamic fault pattern ahead of time, with full knowledge of the routing algorithm, but without knowledge of any coin flips used by the algorithm. It applies the same attacks to all the packets on a given router (edge). This adversary can completely destroy any deterministic routing algorithm e.g, [7], that attempts to collect traffic statistics and choose the next path based on lowest fault rate on previously sampled paths, e.g., RON [5]. The only known algorithm that can succeed against this type of adversary is a learning style algorithm as in [8]; however it only works for DOS attacks and fails in the case of byzantine behavior. Adaptive adversary adapts the fault pattern based on the past, after gaining full knowledge of all past decisions, including the past coin flips used by the algorithm. It applies same attacks to all the current packets on a given router (edge). This type of adversary will destroy all existing routing algorithms including [8]. Adaptive selective adversary is the ultimate adversary that we would like to consider in this paper. This type of adversary selectively attacks packets on a given router (edge). This adversary benefits from knowledge of the traffic pattern (including packet contents) in all compromised and and non-compromised routers; this includes all current traffic and all past traffic history 2.5 System and Security challenges Authentication is of little use. Although one might assume that, once authenticated, a node should be trusted, there are many scenarios where this is not appropriate. For example, when ad hoc networks are used in a public Internet access system (airports or conferences), users are authenticated by the Internet service provider, but this authentication does not imply trust between the individual users of the service. Also, mobile wireless devices can be more easily compromised (e.g. lack of physical security), so complete trust should not be assumed. Thus, we focus on providing routing survivability under a dynamic proactive [22] adversarial model, where dynamically changing subsets of intermediate nodes can perform arbitrary byzantine attacks such as creating routing loops, misrouting packets on non-optimal paths, or selectively dropping packets (black hole). Only the source and destination nodes are assumed to be trusted. 4

5 Maintaining link-state databases is expensive and not useful. While this sounds counterintuitive, the standard proactive link-state routing model, where topology updates are being broadcast through the network, is not applicable to this type of byzantine setting. Indeed, nothing prevents byzantine nodes from claiming perfect connectivity with every node in the network and then completely destroying all traffic. It is provably impossible under certain circumstances, for example when a majority of the nodes are malicious, to attribute a byzantine fault occurring along a path to a specific node, even using expensive and complex byzantine agreement. Consequently, the topology database used in link state methods cannot be trusted and, strictly speaking, is useless in this context. On-demand operation needed. On-demand routing protocols [44, 30, 7] are an attractive alternative since such protocols can test the reliability of network components based on actual traffic. They are also attractive, regardless of trust issues, for wireless environments because they initiate a route discovery process only when data packets need to be routed, thus saving energy in the unused areas of the network. Discovered routes are then typically cached until they go unused for a period of time, or break as a result of topology changes. Flooding should be avoided. One unattractive feature of on-demand protocols [44, 30, 7] is the occasional flooding of broadcast messages through the entire network in order to discover routes from the sender to the receiver. In networks with infrequent traffic or low mobility the cost of an on-demand flood is extremely small compared to the constant overhead of link-state updates. However, it would be extremely desirable to eliminate flooding altogether from on-demand protocols; we will refer to such solutions as floodingfree. 2.6 Algorithmic challenges Past performance is no guarantee of future success. The problem is that we are making decisions online, where the online algorithm needs to service requests by selecting actions while having only information regarding past requests, and no (or very limited) information regarding the future requests. There are no stochastic assumptions about the way the adversary is acting or the sequence of fault patterns generated. Moreover, it is also assumed that there is a powerful adversary that generates the worst input sequence for the online algorithm. A fundamental question regarding online algorithms is how to evaluate their performance. It is rare, and in some cases outright impossible that one deterministic algorithm always outperforms another deterministic algorithm. One classical method has been to assume a stochastic model regarding the environment, and compare the performance of different algorithms on the same model. However, in an adversarial setting, there is no reason why the adversary should follow the rules of the stochastic model we invented - if anything, the adversary will do exactly the the opposite of what our model would predict. (There is no reason to hope that the adversary will not know our model.) One may be tempted to think that converging to the best fixed route may be easily accomplished by utilizing a greedy heuristic: keep track of past error rates on different links, and simply select the path with the smallest overall failure rate, namely the sum of the failure rates of its links. For example, existing work on routing in overlay networks, such as RON [5] runs a greedy strategy of windows of a specific size. The intuition is very strong: extrapolate the past failure pattern into the future. This may work if failure pattern is static or if there is a statistical model of failures. However this method fails quite spectacularly in the case of dynamic adversaries. For example, consider 100 options of choosing a relay point between the sender and receiver, which define 100 different paths. At time i path i (modulo 100) is under control of the adversary, and is failing all packets; at all other times the path is perfect. The greedy algorithm will always pick the worst path, even though at any moment in time only 1% of the paths are faulty. To see the principle flaw in this greedy strategy, consider the performance of an investor who tries to follow the best performing stock on the stock market, with the naive assumption that past per- 5

6 formance is a guarantee of future results. In fact, in an adversarial setting, e.g., the stock market, it may very well be the case that past performance correlates negatively with future results, and algorithms must not be fooled into this easy trap. Our path selection must withstand competitive analysis. In general, there may not be an ideal fault-free path, but yet there may be the best path in terms of accumulated loss on that path. Our goal is to make path selections, such that our overall performance is comparable to that of the best path. Our goal is thus to find a robust randomized algorithm that works well on all inputs, in the sense that the expected behavior of our algorithm is comparable to the optimum fixed path on each input. Our goal is to design a distributed algorithm for sending messages on paths with the overall smallest loss ratio. The contribution of this paper is a novel distributed algorithm for route selection and its proof of optimality in the sense that the total number of messages lost in our algorithm exhibits a very small additive gap with optimum prescient route assignment. The optimum assignment is made with complete knowledge of adversary actions, in the past, current, and future, and has infinite computational power. The only restriction on the optimum route is that it must change somewhat less frequently. The loss of performance of the algorithm that we are seeking should be based on competitive analysis [51]. We compare the performance of our online algorithms to the best static offline. Namely the offline algorithm selects a fixed path and uses it during all time steps. Our results bound the difference between the cost of the best static offline and our online algorithms. This difference is also referred to in the learning literature as regret. For example, consider a wireless network with potentially 200 users, around 1000 potential wireless links and a guarantee of existence of one fixed path of 7 hops between sender and receiver that has an average link fault rate of 0.01 %, i.e % of the time the whole path is reliable. The only problem is selecting one out of around possible paths, while only having information about past experiments over these paths. In this case, one can construct a counter-example in which greedy may never succeed in delivering a single message, i.e. has a 100% fault rate, in spite of thoroughly selecting the best of the paths so far! The explanation to this counter-intuitive fact is that any deterministic algorithm can be easily fooled by an adversary, forcing it to pick the currently worst path that always fails. In contrast, the competitive randomized algorithm that we are seeking should be able to zero in on the reliable path, or at least get comparable performance. The algorithm we are seeking should guarantee, for any adversarial behavior (subject to the adversary s pledge to keep some path of length 7 hops being 99.99% reliable on each link), a fault rate of just below % = 0.07% over long sequences (this is averaged over the coin tosses of our algorithm). Randomness is not a guarantee of success. Another quick fix is to try to select a random path. There is a misconception in the community that selecting one of many node-disjoint paths, or selecting a random path will somehow guarantee success. (The stock-market equivalent of this belief is asserting that a stock index such as the NAS- DAQ 100 cannot go down by much.) There is something appealing about random strategies in the sense that they allow the spreading of risk; what is wrong with the above quick fix is that there is no attempt to adapt the probabilities. In fact, it is easy to generate counter-examples to either one of these policies. For example, consider a faulty edge leading into a dense subnetwork; generating paths at random pretty much guarantees hitting this faulty edge. Multiple paths do not guarantee success. In has been generally recognized that sending packets along multiple paths is more faulttolerant than sending packets over a single path. However, it is easy to come up with a example involving a collection of edge-disjoint paths (e.g., disjoint paths in an expander where the error probability is constant) such that each path has at least one fault, and yet a random path is likely to not have any fault with high probability. 6

7 2.7 Contribution of this paper General statement. The main result of this paper is a routing protocol that combines all the following features: on-demand operation: energy is not wasted learning/maintaining topology information for the areas which are not affected, or while there is no traffic in the network. flooding-free routing: no flooding is taking place even during heavy traffic. Each packet is sent over a selected path of certain bounded length to its destination. near-optimum loss: for every adversarial pattern our algorithms achieves near-optimal loss on that sequence. The routing protocol presented in this work attempts to provide throughput competitive route selection against an adversary which is (essentially) not limited; more specifically, the adversary benefits from complete collusion of adversarial nodes, can engage in arbitrary byzantine behavior and can mount arbitrary selective adaptive attacks, dynamically changing its attack with each new packet. It appears from the description of the model that no possible solution exists against such a strong adversary. Indeed our model does allow the adversary to compromise the whole network. Now, once the adversary is completely in control of everybody, it appears that our algorithm cannot possibly accomplish any packet deliveries. If the adversary wiped out the whole network than surely the failure to communicate is not being attributed to a naive routing algorithm, but rather to the information-theoretic inability to accomplish communication by any algorithm. In this case, our algorithm should be off the hook. More dangerous to us are situations where a post-mortem analysis indicates that it was indeed possible to route the messages by a reasonable benchmark strategy, but our algorithm in fact failed to match the performance of such reasonable strategy in terms of overall packet loss. Quantitative definition of optimality. Informally, we pick as a benchmark a fixed strategy (best fixed path) that knows the pattern of the attack of the dynamic adversary, and has optimized its selection in advance. In fact, a stronger version of our results can be proved, by picking a somewhat stronger dynamic benchmark, which will in fact be allowed to change, albeit less frequently than the algorithm (this is described below). We still need to formally define the best path. Definition 2.1 We consider a path to be instance optimal if, for a given schedule of adversarial infiltration/attacks on routers/network edges, processors, it minimizes, among all possible paths, the average number of faults, attacks, message distortions or other adversarial action on a packet s path. Let this sum be the called hypothetical loss of a path. Definition 2.2 The Hypothetical regretof a routing strategy is the worst case normalized difference, over all possible adversarial actions between the following two factors: the actual loss of our routing strategy, and the above hypothetical loss of instanceoptimal path. Notice that the benchmark strategy and our strategy are both evaluated on the same input instance, namely the particular instance of adversarial behavior. The difference is that our algorithm does not know the adversarial behavior, while the benchmark strategy was custom-tailored for this instance. Our algorithm is outlined in Sec. 3 and formally described in section B. The following theorem is an informal statement of Thm. A.1. Theorem 2.3 For all ɛ > 0, the algorithm accomplishes regret upper-bounded by ɛ. Extension to dynamic benchmarks. The statement of Thm. 2.3 can be strengthened to dynamic benchmarks. Let us make the following notations: E is the set of the edges in the network, and ɛ sa is an arbitrarily chosen constant, d is degree of the network, and H is the length of routing paths being considered, and ɛ sa, ɛ ex is the 7

8 error tolerance parameters that we choose (e.g,, ɛ sa = ɛ ex = 0.01). More precisely, we will allow our dynamic benchmark to change every τ ben = T τ pha packets, where τ pha = Ω( E ) and T = H2 log d. ɛ sa ɛ 2 ex In contrast, our algorithm will changes its policy each τ pha packets. In other words, we expect our algorithm to learn the best path after T phases, each phase lasting τ pha packets. Consider a schedule of possessed processors (i.e., a time-table indicating at which time each router is being possessed), which is unknown to our routing strategy, but may be known to somebody else. The statement of Thm. 2.3 applies in this case as well. 3 Major ideas of our algorithm The algorithm we present is executed at each source node with respect to a specific destination. The actual data packets are source routed and the source route is protected using an onion encryption technique described in Section 4. Every source is maintaining its own graph and probabilities of reaching specific destinations. This information is not shared between nodes because of the byzantine adversarial model which is assumed. Source nodes only rely on other nodes in the network to forward packets and return acknowledgments. It is also assumed that the nodes may choose not to forward the packets or not to return acknowledgements and the source adjusts its probability distribution accordingly. It is important to note that this approach does not rely on intermediary nodes to make hop by hop routing decisions as in other approaches, since those approaches are sure to fail under this strong model. The following description provides an overview of the major ideas of our algorithm. The main idea of this algorithm is the use of a Distributed Reinforcement Learning technique. Specifically, each node is attempting to pick a parent edge towards the source. The set of all parent edges forms a tree rooted at the source. The packets are sent on the unique path in this tree from the sender (the root) to the receiver, and are being acknowledged by the receiver. However, a potential failure may decompose the path into two parts: the part closest to the source that succeeded to acknowledge the packet, and the rest of the path that failed to return an acknowledgement. We now adjust the choice of parent edges as follows. The part of the path that succeeds in acking reinforces its confidence in the parent, while the other part of the path reduces its confidence. The parent will be chosen probabilistically based on the confidences acquired. The intuition is that confidence in the parent reflects not only the reliability of the link between child and parent, but also the fact that the parent is intelligent enough to pick the right (grand)- parent. In this context, there is no way to distinguish byzantine nodes from nodes that are unlucky in choosing their parent. Notice that by authenticating all messages, byzantine nodes are limited in their ability to mislead the source since some edge controlled by the adversary must be exposed to the source, each time an edge fails. The adversary can choose not to expose itself, and indeed this complicates the proof somewhat. We need to show that the adversary has nothing to gain from taking control of an edge, and then not killing packets on this edge. By using a probabilistic distribution over the parent edges we generate a probability distribution over all source-rooted trees, such that probability of each tree is growing exponential in its performance. This discriminates edges controlled by the adversary, and reinforces edges where the adversary is absent. More intuition can be obtained by work on non-stochastic multi-armed bandit [6] and its distributed analog in [8]. We outline below the main components of our algorithm, postponing formal presentation to Section B and its proof to section C. Transforming a graph into a layered directed graph. Without loss of generality, we assume that we can transform the original undirected network graph (e.g., see Fig. 1) into a directed layered graph, with the receiver being in layer 0 of this graph (Fig, 2). All the nodes that can potentially reach the receiver in i hops (or less), for 0 i H are represented in layer i of the graph. Here H is the upper bound on hop count of a routing path. The directed edges connect these representative nodes in layer i to representative nodes in layer i 1. If the topology of the graph is not 8

9 a b sender to receiver beyond the separator edge are called unlucky. (See Sec. B.2.) s r Figure 1: The original network. s 4 s 3 s 2 s 1 a 3 a 2 Figure 2: Layered graph w.r.t. receiver r (in layer 0). known, the assumption is that the topology of the graph is full interconnection, in which case all the nodes have representatives in all the layers. If the topology is partially known, then a node may still have representatives in more than one layer. In order to get an intuition of our algorithm, it is easiest to imagine the original graph as a layered graph with respect to the destination. However, our algorithm and claims of optimality do apply for any graph. (See Section B.1.) Establishing a feedback mechanism for the packets. We request that each packet be acknowledged using a secure acknowledgement scheme by the destination. The adversary can definitely interfere with either the packet or the ack propagation process. However, any adversarial action can be summarized as follows: the source has received an ack from some intermediate node, indicating the first parent edge on the path (we call it the separator that appears to have failed. All the edges leading to parents on the path from sender to receiver prior to the separator are called lucky. All the edges leading to parents on the path from c 3 b 2 c c 1 r 0 Quantitatively ranking the incoming edges. Imagine that instead trying to optimize the process of the source trying to reach destination, we try to optimize the process of all the nodes in the network sending ack packets ( ants ) trying to reach the source in the reverse direction of the edges. Each node will be selecting this best edge on its path towards the sender. Let us call such edge a parent edge. The set of parent edges forms a routing tree rooted at the source. Routing from source to the destination is performed over the tree from its root (source) towards the destination, from parents to their children on the path. As in [8], the nodes rank their adjacent edges based on the percentage of luck they bring. The greedy strategy (which does not work) would be picking edges in decreasing order of luckiness. However, picking edges based on the exponent of luckiness, say β x, where β < 1 is a small number, and x is luckiness percentage, will essentially yield optimal performance. The value of β is crucial for the performance of our algorithm. Low values of β result in our decision sequence resembling that of the greedy algorithm, and thus will be vulnerable to adversarial attacks. On the other hand, values of β which are close to 1 will cause the algorithm to respond slowly to changes. The optimal value of β can be determined using the best expert framework of [33, 15]. (See Sec. B.3.) Final policy: distribution over outgoing edges. Our final policy is determined by each node making a probabilistic decision on an outgoing edge, selecting the edge which has the best chance of reaching the receiver. Our policy will discriminate packets based on the packet s hop count. The routing policy is maintaining a time-to-live count c, initially set to H. When the packet arrives at a node v with hop count c, the receiving node v decrements c by 1, and then chooses the probability distribution over its outgoing edges. The idea is to set the probability over the outgoing edges to emulate the probability distribution over incoming edges defining a probability distribution of directed rooted trees, with the source being the root of each tree. While this provides a route towards 9

10 the source, we want to route towards the receiver. However, to formally specify the routing function from the sender to the receiver, we need to define a probability distribution over the outgoing edges of each node. Routing works as follows: start at the source and iteratively pick an outgoing edge from the current node according to the prescribed edge probabilities until the sink is reached. One might think that the same probabilities assigned to incoming edges can not be used for going in the opposite direction, which is definitely incorrect. This transformation from incoming to outgoing probabilities is performed via matrix multiplication, similar to Bellman-Ford algorithms. This algorithm is called the Weight Pushing algorithm. It was developed by Mehryar Mohri in the context of speech recognition [39], and used in this context in [58]. (See Sec. B.4). Sampling unexplored territory. A subtle but important component of the algorithm is its ability to sample edges in the network, since this is the only way it is able to detect changes in the adversarial fault pattern, and to sample relatively stale portions of the network. This means that the algorithm is occasionally (and probabilistically) deviating from the optimal routes in order to make sure that feedback is obtained from each edge in the network. This is reminiscent of the classical multi-armed bandit algorithms in [6] and its extension to network routing in [8], but our implementation here is quite different. This deviation is accomplished by forcing the algorithm to pass, with small probability, which is completely under our control, over a completely random edge, using a different set of outgoing probabilities. This set of probabilities is computed via the same Weight Pushing algorithm, but starting with the edge being sampled as the sink, and moving back towards the source. Once we cross the edge, we start moving towards the receiver with the usual outgoing probabilities. Let us give some intuition on why this is desirable. By allowing β to be small the algorithm will move quickly to the least fault path, but will only utilize even slightly less reliable paths with low probability. In order to ensure that the algorithm is making correct decisions, it continuously needs to have updated feedback on the other paths in the network. This is accomplished by random sampling. Ideally, if every data packet was used as a sampling packet, meaning the sampling rate was 100%, the algorithm would have extremely high loss rates from exploring faulty paths, but would have extremely accurate estimates of the current reliability of paths in the network. On the other hand if the algorithm only samples paths with 1% of its traffic, then it will be slightly slower in detecting changes, but will be sending a higher percentage of its packets over links that do not fail. It is important to note that the algorithm samples with actual data packets and not small exploratory test packets. This is critically important since we are working under a strong byzantine adversarial model. It is assumed that if we send test packets, the adversaries will be aware that we are testing links and allow the packets to pass through the network. This would increase our probability of selecting the adversarial controlled links. By utilizing real data packets, the only way for the adversary to draw us in would be to allow enough data packets to successfully pass through its links, that its links appeared more reliable then other links in the network. However, our goal is to deliver packets from the source to the destination. If the adversary chooses to deliver these packets, then it is not exhibiting adversarial behavior. If the adversary decides to increase its fault rate as we increase the rate at which we utilize its links, the algorithm will immediately respond since it would not be receiving acks for packets. 4 Byzantine Fault Detection Our detection algorithm is based on using acknowledgments (acks) of the data packets. If a valid ack is not received within a timeout, it is assumed that the packet has been lost. Note that this definition of loss includes both malicious and nonmalicious causes. A loss can be caused by packet drop due to buffer overflow, packet corruption due to interference, a malicious attempt to modify the packet contents, or any other event that prevents either the packet or the ack from being received and verified within the timeout. We would like to emphasize that our protocol provides fault avoidance and never disconnects nodes from the net- 10

11 work. Thus, the impact of false positives, due to normal events such as bursting traffic, is drastically reduced. This provides a much more flexible solution than one where nodes are declared faulty and excluded from the network. 4.1 Fault Detection Overview Our fault detection protocol requires the destination and each intermediate node to return an ack to the source, for every successfully received data packet. We use shared keys between the source and each intermediate node as a basis for our cryptographic primitives in order to avoid the prohibitively high cost of using public key cryptography on a per packet basis. These pairwise shared keys can be established on-demand via a key exchange protocol such as Diffie-Hellman [20], authenticated using digital signatures. 4.2 Source Route Specification The mechanism for specifying the source route on a packet is essential for the correct operation of the protocol. The source route is specified as a list of addresses, where the order of the addresses indicates the path which the packet will traverse. The list is onion encrypted [57]. Each route is specified by an address indicating the next hop, an HMAC of the packet (not including the list), and the encrypted remaining list. Both the HMAC and the encrypted remaining list are computed with the shared key between the source and that node. An HMAC [4] using a one-way hash function such as SHA1 [1] and a standard block cipher encryption algorithm such as AES [3] can be used. When a node receives a packet it verifies the HMAC of the packet and decrypts the remainder of the source route using its shared key with the source. If the HMAC verifies, the node proceeds to forward the packet to the next hop. Since the source route is onion encrypted the intermediate nodes are only aware of the next hop and do not know the full path that the packet will traverse. This prevents the adversary from strategically dropping packets which are intended to traverse specific nodes. It can drop packets based on the selected next hop, but this will only incriminate the adversary, thus decreasing the probability that the source selects it as a forwarding node. 4.3 Acknowledgment Specification If the adversary can drop individual acks, it can incriminate any arbitrary link along the path. In order to prevent this, each node does not send its ack immediately, but waits for the ack from the next node and combines them into one ack. In other words, acks are accumulated from the destination back to the source. For any successfully delivered data packet, the source should receive a single acknowledgement allowing it to verify the successful delivery of the packet as well as the path which the packet traversed. Each ack consists of the identifier of the node, the identifier of the data packet that is being acknowledged, the ack received from the next node encrypted with the key shared by this node and the source, and an HMAC of the new combined ack. If no ack is received within a timeout, the node stops waiting, and creates and sends its ack. The timeouts are set up in such a way that if there is a failure, all the acks before the failure point can be combined without other timeouts occurring. This is accomplished by setting the ack timeout at each node to be the upper bound of the round-trip from it to the destination. Upon receipt of an ack, the source checks the acks from each node by successively verifying the HMACs and decrypting the next ack. The source either verifies all the acks up through the destination, or discovers a loss on the link following the last ack. By using the above specification, it is simple to attribute losses to links. A loss is attributed to a set of links when the source successfully received and verified an ack from an intermediary node, but does not from the destination. Blame is assigned to the nodes between the last received ack and the destination node. Those nodes are penalized for poor parent selection. 5 Implementation Results In order to substantiate the claims made in this work, simulations were conducted to investigate the convergence time of the algorithm. The simulations were conducted by developing a simple pro- 11

12 gram which would simulate the decision making process of the algorithm and examine its performance against adversarial inputs. The simulation consisted of a source selecting a path to the destination at each unit of time. An adversarial model would then select which nodes at the current unit of time were faulty. The packet would traverse the graph and receive positive feedback from the destination if there were no faulty nodes on the path, or from the last non-faulty node before the packet was dropped. Using this feedback the algorithm would adjust its probabilities and compute a new path for the next packet Percent on Link Beta = 0.1 Sample Rate = Packets Sent Figure 5: Simple Configuration Results Beta = 0.05 Sample Rate = to 1 0 to 2 1 to 3 1 to 4 2 to 3 2 to 4 3 to 5 4 to Source 0 5 Destination Figure 3: Simple Simulation Topology Percent on Link to 1 0 to 2 1 to 3 1 to 4 2 to 3 2 to 4 3 to 5 4 to Beta = 0.5 Sample Rate = Packets Sent Figure 6: Simple Configuration Results Percent on Link Packets Sent Figure 4: Simple Configuration Results 5.1 Simple Configuration 0 to 1 0 to 2 1 to 3 1 to 4 2 to 3 2 to 4 3 to 5 4 to 5 The first set of simulations consisted of 10,000 packets which were sent from the source to the destination. The nodes were arranged as indicated in figure 3. The intermediary nodes are labelled 0 through 5 and their fault rates are indicated. At every unit of time each node was probabilistically selected to fail based on the nodes fault rate. On this particular simple configuration the optimal path would be from the source to node 1, from node 1 to node 3, and then from node 3 to node 5. The results of the simulation are the sources estimated link preference metrics at every unit of time. The experiment was run for different values of β, which is the base of the exponent described in the algorithm specification. As the value of β decreases, the algorithm responds faster to changes and is able to converge faster. However, as the algorithm responds faster it begins to resemble the greedy algorithm which has vulnerabilities. In order to explore the effects of β on the convergence time of the algorithm, multiple experiments were run with the same adversarial input. The results are indicated in Figures 4,5, and 6. These figures show the probability of the source selecting a specific edge at every unit of time. With β=0.5 the results show that the source is slowly realizing which path is correct and is sending a majority of its traffic over the correct sequence of nodes. Notice that once the algorithm converged it was sending approximately 90% of 12

13 Beta = 0.05 Sample Rate = 0.01 Beta=0.05 Sample Rate=0.01 Percent on Link Packets Sent 0 to 1 0 to 2 1 to 3 1 to 4 2 to 3 2 to 4 3 to 5 4 to 5 Percent on Link Packets Sent Figure 7: Simple Configuration Results Figure 9: Large Simulation Results its traffic across the first hop to node 1 which was experiencing a 5% loss rate and only 10% of its traffic to node 2 which had a 10% loss rate. While this seems reasonable, notice the convergence when β=0.1 or better yet With these values the source is able to completely differentiate between nodes and converge quickly on the best path. Since the algorithm is continuously exploring sub-optimal paths with a small fraction of its traffic, this low value of β allows it to react quickly as the adversary changes its fault pattern. While the optimal algorithm might know the fault pattern ahead of time, the algorithm we present is able to follow it very closely. If the speed at which the adversary moves is slightly slower then our decision making process, then our algorithm would in fact follow the optimal at every step. In order to investigate the effects of the sampling percentage on the convergence time an additional experiment was done using the same simple configuration described above, but with the sampling rate decreased from 10% to 1%. The results of this experiment are indicated in Figure 7. It appears that the lower sampling rate inhibits the algorithms ability to converge as well as the 10% sampling rate indicated in the Figure 6. Source Figure 8: Large Simulation Topology Destination Percent on Link Beta=0.05 Sample Rate= Packets Sent Figure 10: Large Simulation Results 5.2 Large Configuration The previous example provided a demonstration of the effects of various parameters on the performance of our algorithm. While the previous example showed the convergence of the algorithm, it is somewhat less interesting since the example consisted of a small set of both nodes and links. In this second example we will consider a network with 25 nodes forming a 10 layered graph, with 3 nodes at each layer (except at the source and destination). The topology of the network is indicated in Figure 8. In this example there is exists an optimal path from the source to the destination which experiences no loss. This optimal path is indicated in Figure 8 by the bold line. All other links in the network exhibit 10% loss, meaning that when we sample them they will successfully forward the packet 90% of the time. As a result, this should make it more difficult for our algorithm to discover the optimal path since the non-optimal paths in the 13