A Custom Fair Usage System for RESNET: Design and Implementa>on

Size: px

Start display at page:

Download "A Custom Fair Usage System for RESNET: Design and Implementa>on"

Phoebe Montgomery
5 years ago
Views:

1 A Custom Fair Usage System for RESNET: Design and Implementa>on Ma#hew Weaver Network Specialist Lancaster University

2 Overview About Lancaster s RESNET The problem with our exis>ng topology Introducing our Fair Usage Policy A new network topology to support the policy How to determine what a fair quota amount is How to enforce the policy with our exis>ng network and systems Tes>ng the system and migra>ng it into service opera>on Encouraging fair usage amongst our users Results and conclusions Ques>ons

3 RESNET Background ~6800 bedrooms connected to RESNET today 100Mbps Ethernet connec>ons Exis>ng in house network access control system Hosts on private IP addresses no inbound connec>ons Web proxy used for off site h[p/h[ps access NAT for outside connec>vity to the Internet A default deny firewall provided limited outbound connec>vity for only the most common applica>ons

Exis>ng network topology Campus Core Network 10G LU Border Router Juniper M320 3G CANLMAN JANET Internet 10G 10G Campus Core Routers Juniper MX 960s 1G RESNET Border Router Juniper M7i Single

4 Exis>ng network topology Campus Core Network 10G LU Border Router Juniper M320 3G CANLMAN JANET Internet 10G 10G Campus Core Routers Juniper MX 960s 1G RESNET Border Router Juniper M7i Single connec>on point between campus core network and RESNET NAT 1G RESNET Core Router Brocade BigIron RX4 RESNET 400Mbps NAT limit on the M7i pladorm adap>ve services PIC Default deny firewall is a natural bandwidth limiter 10G 1G 10G RESNET Aggrega:on Routers Brocade BigIron RX4s Brocade FastIron X Series

5 The need to make RESNET more open June 2008 The Internet was no longer just about browsing the web increases in home broadband speeds and availability fuelled the development of media rich services in the last few years Some mul> media applica>ons could not be supported e.g. Skype for 2 way voice and video chat increase in the number of complaints from students Increased support overhead dealing with requests to allow ports through the RESNET firewall for individual applica>ons on average 6 requests per week September 2008 change was needed the RESNET firewall and NAT service was changed from default deny to default allow as a trial

Impact of opening firewall [1] Media rich

communicate using voice and video to friends and

with requests for opening specific ports on the

traffic levels RESNET Border Router reaches upper

performance Rapid increase in the number of

6 Impact of opening firewall [1] Media rich applica>ons could be supported, allowing students to communicate using voice and video to friends and family back home Reduced support overhead in dealing with requests for opening specific ports on the firewall Massive increase in University outbound traffic levels RESNET Border Router reaches upper capacity limits Ad hoc reports of slow network performance Rapid increase in the number of copyright infringement reports due to peer to peer applica>ons no longer being blocked

Impact of opening firewall [2] 180,000 Lancaster University Traffic to/from JANET August 2002 March 2010 * Traffic (GBytes) 160,000 140,000 120,000 100,000 80,000 60,000 Sep 2008 RESNET firewall

7 Impact of opening firewall [2] 180,000 Lancaster University Traffic to/from JANET August 2002 March 2010 * Traffic (GBytes) 160, , , ,000 80,000 60,000 Sep 2008 RESNET firewall changed to default allow University outbound traffic profile increases by ~ 435% (May 2008: 17 TBytes vs Nov 2008: 91 TBytes) Increased outbound traffic pa[ern follows term boundaries, i.e. this traffic is from RESNET users 40,000 20,000 0 * Data courtesy of JISC Monitoring Unit Inbound Outbound

8 Introducing a Fair Usage Policy April 2010 a new Fair Usage Policy for RESNET is approved h[ps://resnet.lancs.ac.uk/fair_usage The University s recommenda>on was to implement the policy without needing any capital investment integrate a fair usage system into our exis>ng systems and network Key points from the policy: access to university resources and general web browsing should be unlimited a monthly per user quota allowance should be introduced to ensure a fair share of the available Internet bandwidth amongst all users

9 Changing the network topology [1] Campus Core Network 10G LU Border Router Juniper M320 3G CANLMAN JANET Internet 10G 10G Campus Core Routers Juniper MX 960s /16 A new 10G link was provisioned directly between RESNET and Campus Core Routers Link brought into OSPF as point to point /30 Traffic from RESNET clients to campus servers prefers the more specific route Juniper firewall policy implemented on both sides of the link to control traffic flow 10G 1G NAT 10G 1G RESNET Border Router Juniper M7i 1G 10G Default route sends all other traffic to RESNET Border Router via NAT RESNET Core Router Brocade BigIron RX /16 ResNet Aggrega:on Routers Brocade BigIron RX4s Brocade FastIron X Series RESNET

Changing the network topology [2] Exis:ng 1G link via RESNET Border Router and NAT Mid April 2010 sudden increase in RESNET download traffic ~500M

provisioned 10G link between RESNET and Campus ~500M download traffic seen on new 10G link, plus ~200M remains on 1G link we had a serious

10 Changing the network topology [2] Exis:ng 1G link via RESNET Border Router and NAT Mid April 2010 sudden increase in RESNET download traffic ~500M maximum download traffic limit is observed on the 1G link Mid May 2010 a new 10G link is provisioned for traffic between Campus and RESNET Newly provisioned 10G link between RESNET and Campus ~500M download traffic seen on new 10G link, plus ~200M remains on 1G link we had a serious bo[leneck Upload traffic on 1G link remains about the same majority of our RESNET upload traffic is from clients uploading to the Internet Download Upload

11 Deciding on a fair quota RESNET traffic data was gathered from April 2010 onwards Fair usage policies from comparable ins>tu>ons were evaluated lack of clarity as to exactly what traffic was governed by a quota varying quota sizes and representa>on, e.g. monthly, weekly Analysis of data showed a 30GB monthly quota for combined upload and download would only affect about 8% of users Over the period April to June 2010 a 30GB quota was exceed 1566 >mes Of these, 185 (12%) used GB, 411 (26%) used 35 50GB and 970 (62%) exceeded 50GB Month Total Users Over Quota April % May % June % Average % %

12 Design features and decisions Easy for users to track their quota usage online portal with clear graphs showing cumula>ve usage helpful e mail warnings at key usage milestones Easy for Service Desk staff to see when users are over or near their quota limit for efficient handling of related queries at 1 st line >ght integra>on with exis>ng helpdesk tools Flexible quota amounts temporary per user increases for academic use when appropriate Rate limi>ng should be non prohibi>ve important for voice/video chat and the student experience

Enforcing the policy/quota [1] 1 Nedlow data

Server 2 Hourly bandwidth usage exported to RESNET

of users who are over quota are added to a rate

13 Enforcing the policy/quota [1] 1 Nedlow data collected RESNET Border Router NAT RESNET Flows Server 2 Hourly bandwidth usage exported to RESNET database and merged with user records 3 IP addresses of users who are over quota are added to a rate limi>ng firewall RESNET Database Server RESNET Network Access Control Servers

14 Enforcing the policy/quota [2] Per user traffic data for upload and download is aggregated each hour and added to a running total for the current month Once users exceed 30GB, their IP addresses are automa>cally applied to a firewall rule on the RESNET Border Router line speed is reduced to 500Kbps download, 100Kbps upload e mail sent to no>fy user that their connec>on has been limited connec>ons remain limited for the remainder of the month all devices validated against the user are limited, so the system cannot be bypassed by simply valida>ng a new computer Only traffic going via NAT to off site is limited so 100Mbps line speed is maintained to vital university services, including browsing via the web proxy, filestore access and VLE

Tes>ng the fair usage system The system was switched on transparently for 5 days from Monday 9 th August 2010 to gather real data, in a period of low residence occupancy to isolate and fix any

15 Tes>ng the fair usage system The system was switched on transparently for 5 days from Monday 9 th August 2010 to gather real data, in a period of low residence occupancy to isolate and fix any sovware bugs before deployment to address any user feedback Results ~1200 users ac>ve on RESNET at that >me traffic levels reduced no>ceably 35% reduc>on in peak download and upload no incidents raised to our Service Desk effec>vely users did not no>ce Download Upload

Encouraging fair use [1] Users can track their quota usage in near real >me via a web portal E mails are sent to users to no>fy them of their quota usage At 25% and 50% u>lisa>on, but only if they

16 Encouraging fair use [1] Users can track their quota usage in near real >me via a web portal E mails are sent to users to no>fy them of their quota usage At 25% and 50% u>lisa>on, but only if they are likely to exceed the 30GB quota during the month (i.e. they have used more than 7/14GB in the first 7/14 days) At 75%, 90% and 100% u>lisa>on Each night if their u>lisa>on for the past 24 hours is greater than 1GB, and if more than 50% of that was used for uploading

Encouraging fair use [2] Interac>ve graphs show users their cumula>ve monthly usage Download and upload breakdown helps users iden>fy applica>ons that may be running in the background and consuming

check their usage Greetings Joe Bloggs, This is an automated e mail from the Lancaster University ResNet System. We would like to inform you that you have now used 76.90% (23.

Did you know that peer to peer applications like BitTorrent and edonkey use large quantities of bandwidth? Turning these applications off can significantly reduce your quota usage.

17 Encouraging fair use [2] Interac>ve graphs show users their cumula>ve monthly usage Download and upload breakdown helps users iden>fy applica>ons that may be running in the background and consuming their quota An example quota warning shows how much quota has been used a reminder that peer topeer clients can use large amounts of bandwidth a pointer to the RESNET portal where the user can check their usage Greetings Joe Bloggs, This is an automated e mail from the Lancaster University ResNet System. We would like to inform you that you have now used 76.90% (23.07GB) of your total ResNet quota allowance for the calendar month of December. Your total ResNet quota allowance for this month is 30GB. Did you know that peer to peer applications like BitTorrent and edonkey use large quantities of bandwidth? Turning these applications off can significantly reduce your quota usage. Remember, you can track your quota usage in real time online. Visit and click the 'My ResNet' tab at the top of the page, then choose the 'My Quota' option. For full details about the ResNet quota visit If you need any further information or assistance, please contact the ISS Service Desk at iss service desk@lancaster.ac.uk or call x Kind Regards, Information Systems Services

Using graphs to convey informa>on The quota graph of a user who went over quota on 1 st March is shown here Their quota usage increases aver the limit, download more

quota usage at a glance This is a 7 day overview of the same user s traffic data taken on 4 th April When limited in March, their download usage is about 1.

18 Using graphs to convey informa>on The quota graph of a user who went over quota on 1 st March is shown here Their quota usage increases aver the limit, download more steeply than upload due to the 5:1 ra>o between D:U rate limi>ng The greater the red area, the heavier the user Very easy for the user or Service Desk staff to gauge quota usage at a glance This is a 7 day overview of the same user s traffic data taken on 4 th April When limited in March, their download usage is about 1.3GB per day As soon as the restric>on is lived we see a massive increase in both download & upload followed by heavily reduced usage once they exceed 30GB and are rate limited

19 Going live The system went live on 1 st September month before the arrival of our new year intake to allow for tes>ng of the remaining systems (e.g. auto e mails) to address user feedback E mail sent specifically to RESNET users who were ac>ve on the network during the previous 2 weeks 718 users were targeted by e mail to advise them of the change to the network to highlight the new My Quota area on our RESNET portal Within 5 minutes of sending the e mail 328 unique people were seen logging into the My Quota area 45% of targeted users took note! 1 user even replied to apologise for their excessive behaviour

20 Results traffic reduc>on Term 1 Term 2 Term 3 Term 1 Term 2 1G link NAT link from RESNET Border Router 10G bypass link provisioned Fair usage system went live 45% reduc>on in download traffic 75% reduc>on in upload traffic

Overall impact on Lancaster s traffic 200,000 180,000 160,000 140,000 Lancaster University Traffic to/from JANET 2002 Feb 2011 *

indicated by an exponen>al trend of outbound traffic from Aug 2002 Aug 2008) Traffic (GBytes) 120,000 100,000 80,000 60,000 40,000

21 Overall impact on Lancaster s traffic 200, , , ,000 Lancaster University Traffic to/from JANET 2002 Feb 2011 * Sep 2010 Fair Usage Policy implemented on RESNET Feb 2011 University outbound traffic profile decreases to an>cipated levels (as indicated by an exponen>al trend of outbound traffic from Aug 2002 Aug 2008) Traffic (GBytes) 120, ,000 80,000 60,000 40,000 20,000 0 * Data courtesy of JISC Monitoring Unit Inbound Outbound Source for trend Outbound Trend based on data from Aug 2002 to Aug 2008

B E F O R E Results heavy users: before and aver 1 2 1 1.

8TB Upload (1779GB) (94%) Equivalent to 63 monthly 30GB

With the Fair Usage System in place, the impact on heavy

slower rate, with the same ra>o for upload/download But,

over quota The heaviest users usage con>nues to increase more

22 B E F O R E Results heavy users: before and aver TB Total usage (1901GB) 0.1TB Download (121GB) (6%) 1.8TB Upload (1779GB) (94%) Equivalent to 63 monthly 30GB quotas! With the Fair Usage System in place, the impact on heavy users varies: A F T E R Some users usage con>nues at a slower rate, with the same ra>o for upload/download But, there is a no>ceable reduc>on in usage once the user has gone over quota The heaviest users usage con>nues to increase more aggressively Upload usage con>nues linearly, sugges>ng that the upload connec>on is maxed out Download usage increases and overtakes the upload

23 Results quota usage analysis For March 2011 % of Quota Used Number of Users % of Users < 25% % 25% to 50% % 50% to 75% % 75% to 90% % 90% to 100% % > 100% % Most users actually use less than 25% of their quota % of users who use more than 30GB is only 6% 2% less compared to the es>mated 8% from ini>al research But, this number is rising

25 Conclusions Some users just do not care about the quota limita>on they repeatedly go over quota on 1 st of every month suggests that heavy users leave their peer to peer clients running The system has been very effec>ve in increasing our bandwidth capacity to RESNET both from internal University servers and the Internet Monitoring quota usage allows us to be[er understand how our users are using their connec>ons an important tool to aid future capacity planning for RESNET and LU Li[le or no effect on the number of copyright infringement no>ces received by the University a possible reduc>on in copyright no>ces was an>cipated in reality this has not been achieved as a result of introducing fair usage

26 So, what did it cost? Nothing.. well, not exactly Engineer resource >me = approx. 50 man days network planning and maintenance to relieve conges>on analysis of traffic data to determine a fair quota amount sovware development to design and integrate the fair usage system into our exis>ng RESNET systems Network cost associated interface cost of provisioning a new 10G link Sovware license for ChartDirector graphing sovware = 75

27 Benefits Cost no capital investment was needed lifespan of our exis>ng network hardware has been extended Capacity we can analyse our RESNET traffic usage and shape it accordingly User educa>on hopefully we are encouraging our users to use their bandwidth more responsibly Flexibility we can adapt the system to meet any changes to our Fair Usage Policy we can integrate the Fair Usage System into our exis>ng helpdesk and service management tools

28 Ques>ons?

Conges'on. Last Week: Discovery and Rou'ng. Today: Conges'on Control. Distributed Resource Sharing. Conges'on Collapse. Conges'on

Conges'on. Last Week: Discovery and Rou'ng. Today: Conges'on Control. Distributed Resource Sharing. Conges'on Collapse. Conges'on Last Week: Discovery and Rou'ng Provides end-to-end connectivity, but not necessarily good performance Conges'on logical link name Michael Freedman COS 461: Computer Networks Lectures: MW 10-10:50am in