The Challenges of Measuring Wireless Networks David Kotz Dartmouth College August 2005
Why measure? For better management Large WLANs need scalable management To manage you must monitor To monitor you must measure Where is there trouble... or trouble-makers? Where should your network grow? Where are your users? How and when do they move? What do they use the network for? How is it abused?
Why measure? For research and innovation To improve our understanding of user and network behavior. This understanding leads to better models. Better models are critical to innovation: network protocols distributed algorithms and applications deployment strategy
Wireless is not Wired Many inherent differences Wired medium: clear points of connection Wireless medium: physically dispersed Not everything is visible for measurement Mobility inspires new usage patterns Novel devices inspire new usage patterns 802.11 protocol inspires new attacks
Dartmouth s wireless network
Active cards per week Cards 5000 4500 4000 3500 3000 2500 2000 1500 1000 500 2001 trace 1706 cards 7134 cards 0 Apr01 Jul01 Nov01 Mar02 Jun02 Oct02 Feb03 Jun03 Sep03 Jan04 May04 Date 2003/4 trace
GB 14 12 10 8 6 4 2 0 Average hourly traffic, by hour Fall 2003 Winter 2004 Inbound Outbound 0 6 12 18 24
VoIP devices in use Fall 2003 Winter 2004 100 80 Cisco SoftPhone Wired Cisco VoIP phone Vocera VoIP badge Wireless Cisco VoIP phone Telesym PocketPC Softphone 60 40 20 0
1 0.1 VoIP call duration Fall 2003 Winter 2004 All VoIP calls are significantly shorter than non-voip calls Total Wired Wireless P[X>x] 0.01 0.001 Median wired call duration: 42 seconds Median wireless call duration: 31 seconds 0.0001 1 10 100 1000 10000 x = call duration (seconds)
Session diameter by device Fall 2003 Winter 2004 VoIP and PDA devices tend to move farther during a single wireless session 1 0.9 0.8 0.7 A B All devices Laptops PDAs VoIP devices 0.6 diameter = max inter-ap 0.5 0.4 0.3 E distance Median All: 15.3m Laptops: 14.0m PDAs: 17.5m VoIP devices: 27.8m C 0.2 0.1 D 0 0 100 200 300 400 500 600 700 800 distance (meters)
www.cs.dartmouth.edu/~map Security New attacks on the 802.11 or VoIP protocols Denial of Service = DoS Denial of Quality Service = DoQS Measure Analyze Protect
Measurement challenges
Measuring real networks: Practical challenges Lack of portable tools for collection, analysis Lack of information about network hardware Lack of common data formats Reluctance of network administrators Avoiding, identifying, and handling data holes
Capturing traffic: sniffing the wire One sniffer captures traffic from many APs to Internet But... Does not capture intra-ap traffic Does not capture 802.11 control frames Does not capture collisions, drops Sniffer Ethernet Switch AP AP AP
Capturing traffic: switched wireless to Internet Can capture nearly all MAC-layer traffic But... Needs explicit support from switch Today s switches are underpowered Sniffer Wireless Switch AP GRE tunnel AP AP
to Internet sniffing the air Ethernet Switch client Sniffer Sniffer client Sniffer AP client client
sniffing the air Can capture full MAC-layer traffic But... Need multiple radios, multiple locations Still might not capture all traffic Frames may be encrypted
Understanding traffic Encryption limits what we can see Identifying P2P traffic is hard Identifying VoIP traffic is hard WEP and WPA encrypt traffic in the air VPNs and other tunnels encrypt the wire Firewalls and NAT may block traffic
Correlating sources Hard to match data across sources syslog: device movements CDR: call detail records tcpdump or other packet traces RADIUS: authentication records SNMP: traffic counts (poor granularity)
Network structure Must know network structure and history location of all APs subnet structure switched wireless AP structure channel assignments, power levels?
Devices Tracking device movements syslog, SNMP: lack of common formats no clean way to identify device departure no clean way to tie cards to users no standards for localization Device type: laptop, palmtop, phone, psp...?
Human subjects Laws governing research and privacy For research in US universities, need approval of Institutional Review Board In any case, we must protect privacy: secure the collection infrastructure encrypt the data storage anonymize data where possible
Real user behavior Ultimately, it is hard to map observed network activity to real user behavior.
Mesh networks WiFi infrastructure network WiFi mesh network uses wireless backbone network serves mobile WiFi clients with fixed APs serves mobile WiFi clients with fixed APs How and where to collect the data? What, additionally, to monitor?
Modeling wireless networks
Building models Models should be derived from real data Then, how are the models validated? What is modeled, and what is assumed? Carefully define assumptions Clearly identify usable range of the model Is the model portable? translates to other places, other times, other sizes
Mobility models The MANET world is full of fake models! We need new models from real users Mobility model: the path of users in geographic space Association model: sequence of APs associated by a device
The needs of our community
We need... Data collection Standardized formats and interfaces Portable tools for data collection Portable tools for network mapping Documentation from network vendors
We need... Data processing Effective, portable tools for anonymization avoid data-mining attacks but don t lose the tail of the data Identifying holes and cleaning data
We need... Data analysis Portable tools for data analysis Precise definitions: e.g., session Standardized metrics: e.g., diameter, prevalence
We need... Wireless testbeds Ideal testbed should allow repeatable experiments be flexible to variety of experiments be remotely accessible represent indoor, outdoor, or both recognize its own limitations
We need... A scientific process Every wireless-network paper should clearly state its assumptions clearly state its experimental conditions identify how these assumptions and conditions limit the conclusions contribute its data to an archive make source code available
We need... Data sharing Archives of measurement data for research... and the tools to manage archives... and encourage contributions of new data... and staff to manage archives and tools
CRAWDAD.cs.dartmouth.edu Community Resource for Archiving Wireless Data At Dartmouth
CRAWDAD.cs.dartmouth.edu Archive of wireless-network traces Tools for trace collection and analysis Support for the research community Event calendar Bibliography Annual workshop at Mobicom Co-led by David Kotz and Tristan Henderson Hosted by the Center for Mobile Computing at Dartmouth College Funded by US National Science Foundation
Summary Large WLANs require scalable management Management requires monitoring Monitoring requires measurement Measurement is hard! We need better tools and data formats We need shared archives and testbeds Researchers and providers must cooperate
David Kotz Dartmouth College Center for Mobile Computing cmc.cs.dartmouth.edu