P2P Control for ISPs Dr. Lawrence G. Roberts Anagran Oct. 2008

Transcription

1 P2P Control for ISPs Dr. Lawrence G. Roberts Anagran Oct Executive Summary P2P applications have become a serious problem for ISPs since they generate multiple flows for long periods, consuming much of the pooled capacity intended for a large collection of subscribers. It was hoped that could detect the P2P and slow it down, but encryption has made that approach miss sufficient P2P applications that the problem remains. Also, prying into the users data is viewed by many as an invasion of privacy. Thus a new technique is required, one which does not need to recognize P2P applications, but which looks at the subscribers total activity (# of flows plus their total data rate) and then lowers the data rates of subs with excessive usage so that all subs that paid the same have equal capacity during congestion. This requires the ability to slow a data flow smoothly without modifying the packets and not stalling the flows. The Anagran FR Flow Manager is the first system that can do this. One system can manage 10 Gbps with far less size, power and cost than the systems. This paper will show why multi-flow applications must be controlled and that P2P is only the start of the problem. My contention is that networks must implement a new fairness rule equal capacity for equal pay. Currently, when congestion occurs, we live with the default rule of equal capacity per flow. This default rule is why P2P has become a problem. Other applications like FTP could easily start using multiple flows to compete with P2P and without the new fairness rule; the network quality would deteriorate rapidly. The paper will also examine the behavior of P2P and various control methods. P2P applications add more and more flows so long as they can expand total throughput. Thus, there is a complex feedback between the rate and number of flows P2P applications utilize and the control method. For example, a fixed reduction of capacity per P2P sub is generally insufficient to improve the speed and quality of the average users being affected. Also, adding capacity may harm the average user, not improve things. These issues will be explored. The Simulation Model After examining P2P activity and observing its behavior, the following model was constructed. The network is assumed to be FTTX, DSL, Cable, Satellite, or Wireless. In each case the total upstream and downstream capacity per fiber hub, DSLAM, cable channel, transponder, or base station is pooled capacity shared between some number of subs. The peak capacity per sub and the average usage per sub are other parameters. Also, the maximum number of flows and minimum rate per flow of the P2P applications are important parameters. The cases shown here have been chosen to be typical cases and ones where significant information can be learned.

2 In all cases, the % of P2P subs in a pooled capacity is a variable from 5% down to zero. Generally, it has been observed that 5% of the population tends to use P2P. If is being used to reduce P2P activity, the best current estimate of detection and control can be found in the recent French ETAC study where 7 of the P2P was controlled. This leaves 3 of the P2P using subs still operating, or out of 5%, about 1.5% P2P active and unconstrained. In all cases modeled this reduction of the P2P subs from 5% to 1.5% makes no difference in the service received by the other subs since the remaining P2P subs generate more flows and still saturate the channel. Generally, the P2P subs compete among themselves for capacity until they are reduced to 0.5% of the population, at which point things start to get better. Asymmetric Channels An example may clarify this. Figure 1 shows a DSL or Cable case with 4:1 asymmetry. 4 6 Mbps Downstream,.75 Mbps upstream Wasted Capacity % % % % % 0. % User s P2P 6 Mbps Dow nstream,.75 Mbps upstream Figure 1: 100 Mbps Pooled Capacity shared among 800 subs P2P tends to use the same capacity upstream as downstream, in this case about 23.6 Mbps in each direction. The P2P applications sense the congestion of the upstream channel and until there are less than 0.5% P2P subs active, the upstream is congested. This means the ACK s for the other 99.5% of the users who might be downloading, are greatly delayed, thereby slowing their downstream traffic to 650 Kbps per sub. The four P2P subs also use the downstream at 6 Mbps each. This leaves about 45% of the downstream capacity unused due to the congested upstream. The performance for the average user is identical for any number of P2P subs above 0.5% but the P2P subs must share the 23.5 Mbps in the upstream direction and thus as more P2P subs are active, each will receive less and less of the capacity until at 4.6% P2P subs, each P2P sub receives less downstream than the average user. Thus has no impact on the performance for the majority of the subscribers. Not only do they have poor performance, but since ACK s may be discarded in the congested upstream channel, they may have a larger instance of stalled flows; flows that never seem to complete % % % % % 0. % Use rs with P2P Symmetric Channels One might think that by using a symmetric channel design, the congestion of the upstream channel would be solved and the situation would improve. However, the reverse is true, now both the upstream and the downstream channel will be congested, and the problem is worse. Figure 2 shows the same DSL or Cable system with 100 Mbps upstream and downstream.

3 4 6 Mbps Dow nstre am, 6 Mbps Upstream % % % % % 0. % Us ers P2P 6 Mbps Downstream, 6 Mbps Upstream Figure 2: Symmetric DSL -100 Mbps Pooled Capacity shared among 800 subs Here without the 5% P2P subs consume 98.6 % of both the upstream and downstream channels! This is because the P2P subs are not competing so hard between themselves and each one can use more capacity. If can reduce the P2P subs that remain active to 1.5%, then the average subs each receive 592 Kbps, 9% less than with the asymmetric system. Thus, unless detects and controls more than 7 of the P2P applications, the service is either worse or much worse than the asymmetric case. 4 Idle Capacity % % % % % 0. Higher Capacity The next thought one might have is to raise the capacity and perhaps the P2P would not hurt. However, this is not the case again. In Figure 3 we consider a FTTX system with 1 Gbps downstream and upstream channels serving 5000 subs with each sub having 100 Mbps in each direction. The P2P applications now have 100 Mbps available and if they can find enough sources and sinks, then they would use it all. However, now with 5000 subs, there are more P2P subs and they still must share between themselves Mbps Downstream, 100 Mbps Upstream % % % % % 0. % Us ers P2P 100 Mbps Downstream, 100 Mbps Upstream Figure 3: FTTX with 100 Mbps maximum each direction per sub Here, with 5% P2P subs, they get about 4 Mbps per sub whereas the other 95% of the subs get 30 Kbps in each direction. The reason for the 30 Kbps is that is the rate the P2P subs have chosen to use and if under congestion every flow gets equal capacity, the average sub with one flow gets 30 Kbps where the P2P sub is using 131 flows at 30 Kbps. The best strategy for the P2P applications is to use more flows at lower rates, thus driving down the average rate of the other users. 30 Kbps is about as low as we see today, but even lower is quite feasible. The lower it is, the worse the performance is for the average sub. In fact here with 100 Mbps service, the average sub is getting less than dialup speeds % % % % % 0.

4 Problems As the prior cases show, fails to improve the low performance of the average user since it cannot recognize al the encrypted P2P traffic. Another option some have used is to downgrade all traffic that is not recognized as good (not P2P). This has the impact of downgrading good traffic like HTTPS and other encrypted traffic. This also is not acceptable to the average user. In a test where this was being done at a University, the Anagran system showed many false negatives produced by this approach. However, suffers from many other problems as well: Recognized P2P: Too many false positives bad user experience Recognized good: Too many false negatives bad user experience Method to slow P2P traffic: Either resets (bad press) or excessive stalls produced Privacy invasion: Negative reaction from users, press, and regulators High processing load: High cost, size, and power 10 Gbps trunks: 1 Gbps units need efficient load balancer The Anagran system is probably the best load balancer since it carefully balances 10 Gbps streams to multiple smaller systems, flow by flow. However, the same system could also be managing the P2P more effectively thus this approach is mainly valuable where is used for other issues like security. Rate Control instead of We have seen that has many problems and cannot adequately control P2P. However, other solutions exist. First one needs to watch all the traffic and discover the total capacity being used by each subscriber in all flows in each direction. Next, if there is congestion, the subs using excessive capacity can have their maximum rate reduced or their individual flow rates reduced. Now we will examine these options. Subscriber Rate Cap Most subscriber management systems are designed to provide for several levels of service based on the DSL capacity or the payment level. In cable and wireless this may be in the CMTS systems and for DSL it may be in the DSLAMs. If these rate caps can be adjusted by policy, they can be reduced when excessive use is determined. The process here is to either measure the traffic per subscriber before the management system or use the management systems own measurement to determine subscriber usage over a period. 4 Dow nstream Capacity Usage 6 Mbps Downstream, 1.5 Mbps upstream % % % % % 0. % Users P2P 6 Mbps Downstream, 1.5 Mbps upstream % % % % % 0. Figure 4: Asymmetric DSL service with excessive subs limited to 1/3 rate Then, when congestion is seen, the subscribers with the highest total usage can be downgraded temporally to a lower maximum rate cap. For example a 12 Mbps subscriber 4

5 might be reduced to 4 Mbps, a 3:1 reduction. The result is to limit the P2P users ability to use as much capacity and thus, hopefully to relieve congestion for the average subs. However, this does not succeed that well. Going back to the first example, asymmetric DSL, figure 4 shows the result of capping the excessive subs to 1/3 their maximum rate. As figure 4 shows, the 1/3 rate reduction moves the breakpoint for improvement of the average subs performance from 0.5% to 1.5% P2P active. However, even if is still deployed, there is no improvement with a 3:1 reduction. If does not lose ground as encryption is more widely deployed, a larger rate reduction would improve the situation, but congestion has not been avoided, s privacy invasion is still required, and s use of crude methods to kill or slow P2P are still necessary. Looking at the FTTX example, an even worse situation occurs. Figure 5 shows the FTTX system with 100 Mbps to the sub when a 3:1 rate reduction is imposed on the maximum rate. 4 Average Traffic % % % % % 0. % Users P2P Figure 5: FTTX with a 3:1 cap rate reduction % % % % % 0. In figure 5 there is no improvement although improvement now starts if P2P subs can be reduced below 1%, not 0.1%. In fact no matter what the rate reduction, would still be required, and anything more than zero P2P subs is less than perfect. Thus, the impact of rate cap reduction would only help if is continued to be used and major rate cap reductions like 20:1 were available and used. Such large rate cap settings are not likely to be available, and if they were, a light P2P user or subs with many videos, would be subjected to excessive rate reduction which is likely to cause serious complaints. 4 Variable Rate Reduction What the examples above have shown is that fixed rate reductions are unfair, insufficient, and still require. However, if the rate reduction can be applied on a basis proportional to the rate each sub is using then, to first order the could be eliminated and the correction made fair. It still requires that this only be done when there is congestion and then only to the extent needed to bring each channel to just under congestion. This cannot be pre-computed easily since the P2P applications will respond to congestion by dropping the rate of the flows or the number of flows, or both. Thus, the situation is dynamic and must be dealt with dynamically. It also must use a method of rate control that can smoothly reduce the rate of each flow without discarding more than one packet per RTT so that the flow does not stall or return to slow start. This is exactly the unique sort of flow rate control that Anagran has built into its flow manager. The flow manager can set a priority for each flow and when congestion is foreseen, it can

6 dynamically adjust the rate of each flow proportional to the priority. That is, if the priority is 1/27 the flow will be controlled to 1/27 the rate of other flows in the same congestion group (cable channel, DSLAM, base station, etc.). Thus, by observing the number of flows and the traffic over some period of each sub, a priority can be applied to all the flows of that sub which will reduce its total rate to be equal with all other fully active subs when there is congestion. This is all done at very high speed since the flow manager is streaming packets through, and only managing the higher level, flows. Thus, one small 1RU system can manage 10 Gbps of traffic for many congestion groups in both directions. The cost size and power is far less than and there is no dependence on some built-in rate caps in the subscriber management systems. Thus, multi-flow control (including P2P) is possible with no unfairness, no privacy invasion, and no confusion with encrypted traffic. This is Anagran Equalize. The result can be seen in figure % % % % % 0. % Users P2P Figure 6: FTTX with Anagran Equalize This FTTX case was the most difficult, but all cases now look the same. The P2P subs get the same capacity as any average sub. There is no congestion, the maximum traffic level allowed in any congestion group is 99%, thus not requiring any packet discards at the subscriber management systems. Also, video and voice flows are protected from packet discard or delay. Priorities can be applied for subscribers who pay more and when this is applied the result is FTP, HTTP, and other applications are allowed to run at higher per flow rates, not just less delay or more flows. But most of all, the result is easy to describe to all subs; all subs the paid the same will get the same capacity if congestion does or would occur. 4 Idle Capacity % % % % % 0. The Importance of a New Fairness Rule As we have seen, P2P can cause severe disruption of IP service, largely due to the default fairness rule that results from TCP operation and network equipments output queues; equal capacity per flow. If this continues, more and more applications will use multiple flows and IP service will deteriorate. However, by introducing flow managers which change this rule at each congestion point to a new and much fairer rule; equal capacity for equal pay.