CN R G. Empirical Evaluation of Upstream Throughput in a DOCSIS Access Network. Swapnil Bhatia (with Radim Bartoš and Chaitanya Godsay)

Transcription

1 Empirical Evaluation of Upstream Throughput in a DOCSIS Access Network Swapnil Bhatia (with Radim Bartoš and Chaitanya Godsay) Computer Networks Research Group Department of Computer Science and CN R G The InterOperability Laboratory Research Computing Center University of New Hampshire Durham, NH 03824

2 Objectives of this Talk Report measurement results from our DOCSIS testbed Describe our approach to interpreting results Promote discussion of practical aspects of access networks Solicit feedback and ideas from the audience about each of the above Promote further collaborative study of access networks 2 of 30 MSAN 2005

3 Outline Introduction DOCSIS architecture, protocol, enhancers (piggybacking, concatenation, fragmentation etc.). Background of this study InterOperability Lab., vendors and providers, complexity of standard. Overview of Experiments Testbed, variables and data interpretation. Results Subset of conclusions. Summary and discussion 3 of 30 MSAN 2005

4 DOCSIS Introduction (Source: DOCSIS 1.1 RFI Specification) DOCSIS Data Over Cable Service Interface Specification MAC protocol utilizing existing CATV network Developed by CableLabs (Louisville, CO) Version 1.0 (pre-1999), 1.1, (1999-), 2.0 ( ) 4 of 30 MSAN 2005

5 DOCSIS Introduction (contd.) Tree topology Downstream vs. upstream Separate frequencies Broadcast, unicast (resp.) TDMA upstream To WAN CMTS Splitter/Combiner CATV Plant CM Cable Modem CMTS CM Termination System CM 1 User 1 data CM 2 User 2 data CM 3 User 3 data CM n User n data MAP: Periodic downstream control message Describes upstream transmission schedule Who: Which CM transmits? When: Starting when and how long? What: What can it transmit? Different types of transmission windows BW Request (BWR), BWR or Data, Short Data, Long Data, Maintenance. 5 of 30 MSAN 2005

6 DOCSIS Introduction (contd.) (Source: DOCSIS 1.1 RFI Specification) 6 of 30 MSAN 2005

7 DOCSIS Introduction (contd.) Basic Data Transmission Cycle Wait for contention-based BW Request window Send Request (with retries) Retry until MAP received Wait for start of MAPped window Send data (Source: DOCSIS 1.1 RFI Specification) Alternatives Unicast data or request windows 7 of 30 MSAN 2005

8 DOCSIS Introduction (contd.) Performance Enhancers Piggybacking Use part of data transmission window to make new requests Concatenation Transmit more than one data PDU in a single transmission window Fragmentation Divide large data PDU to fit into current transmission window Header Suppression Header of data PDU suppressed at CM, regenerated at CMTS 8 of 30 MSAN 2005

9 DOCSIS Introduction (contd.) Performance Enhancers Piggybacking Use part of data transmission window to make new requests Concatenation Transmit more than one data PDU in a single transmission window Fragmentation Divide large data PDU to fit into current transmission window Header Suppression Header of data PDU suppressed at CM, regenerated at CMTS 9 of 30 MSAN 2005

10 Outline Introduction DOCSIS architecture, protocol, enhancers (piggybacking, concatenation, fragmentation etc.). Background of this study InterOperability Lab., CableLabs, complexity of standard. Overview of Experiments Testbed, variables and data interpretation. Results Subset of conclusions. Summary and discussion 10 of 30 MSAN 2005

11 Background of this Study Supported by the UNH InterOperability Laboratory Largest standards compliance testing facility in the country 19 consortia (including: iscsi, SATA, IPv6, WiMax, EFM... ) Industry supported, driven testing and applied research Conformance, interoperability and performance Previously verified, but in isolation Bottomline for vendors and service providers Configuration design Measurements with real devices Benefits to Protocol designers Equipment manufacturers Service providers 11 of 30 MSAN 2005

12 Outline Introduction DOCSIS architecture, protocol, enhancers (piggybacking, concatenation, fragmentation etc.). Background of this study InterOperability Lab., CableLabs, complexity of standard. Overview of Experiments Testbed, variables and data interpretation. Results Subset of conclusions. Summary and discussion 12 of 30 MSAN 2005

13 Overview of Experiments Goal Characterize upstream performance to answer deployment design questions of the type: When is it better to piggyback than concatenate? How much is the improvement using concatenation? Dependent or independent of CMTS scheduling algorithm Independent variables Upstream channel rate Input packet length Performance enhancer CMTS Dependent variables Throughput Latency CMTS Ethernet Traffic generator and analyzer Upstream data RF analyzer CM CM CM Coaxial cable 13 of 30 MSAN 2005

14 Overview of Experiments Independent variables Upstream channel rate {0.64, 1.28, 2.56, 5.12, 10.24} Mpbs. Packet length {64, 128, 256, 512, 768, 1262, 1500} bytes. Performance enhancer {Concatenation, Piggybacking, Both, Neither} allowed. CMTS {Vendor-A, Vendor-B}. Load Constant load of 8 Mbps (saturation). 14 of 30 MSAN 2005

15 Overview of Experiments Define a configuration as a tuple < rate, length, enhancer, cmts > Define a transition as a doubleton of configurations {< v 1, v 2, v 3, v 4 >, < u 1, u 2, u 3, u 4 >} such that! (1 i 4) (v i u i ) Consider a k tuple of n 1,..., n k -valued attributes each Total number of transitions is N = k ( ni (n i 1) i=1 2 ) n j = j i k n i i=1 k i=1 (n i 1) 2 15 of 30 MSAN 2005

16 Overview of Experiments 2400 cases An experiment for each transition Capture effect of a single change 25 runs per experiment Decide whether change improves or worsens performance Statistically robust, unbiased data interpretation Between and across CMTS 16 of 30 MSAN 2005

17 Overview of Experiments Wilcoxon Signed Rank Sum Test (WSRS) A popular hypothesis test independent of distribution of data Calculates probability of median of sorted ranks being zero Null hypothesis (NH): no change in throughput due to a transition (T original T changed = 0) Test provides probability P of NH being true Fix desired significance level α = 0.05 If P α, reject NH (T original T changed 0) i.e., transition affects throughput Check one-sided alternative (T original T changed > 0, T original T changed < 0?) Actual α = 0.05/2400 (Bonferroni correction) 17 of 30 MSAN 2005

18 Outline Introduction DOCSIS architecture, protocol, enhancers (piggybacking, concatenation, fragmentation etc.). Background of this study InterOperability Lab., CableLabs, complexity of standard. Overview of Experiments Testbed, variables and data interpretation. Results Subset of conclusions. Summary and discussion 18 of 30 MSAN 2005

19 Results Throughput (Mbps) Channel 0.64Mbps 1.28Mbps 2.56Mbps 5.12Mbps 10.24Mbps Per CMTS (99% confidence) Maximum throughput < 3 Mbps per CM Packet length (bytes) (a) No enhancers Throughput (Mbps) Channel 0.64Mbps 1.28Mbps 2.56Mbps 5.12Mbps 10.24Mbps Enhancers effective for smaller packets Packet length (bytes) (b) Both enhancers 19 of 30 MSAN 2005

20 Results Throughput (Mbps) Channel 0.64Mbps 1.28Mbps 2.56Mbps 5.12Mbps 10.24Mbps Per CMTS (99% confidence) Concatenation very effective for smaller packets Packet length (bytes) (c) Concatenation Throughput (Mbps) Channel 0.64Mbps 1.28Mbps 2.56Mbps 5.12Mbps 10.24Mbps Piggybacking largely ineffective Need more CMs to see effect Packet length (bytes) (d) Piggybacking 20 of 30 MSAN 2005

21 Results When is Piggybacking useful? Throughput (normalized) No enhancers on 1.28Mbps Piggybacking on 1.28Mbps Larger packet lengths at 1.28 Mbps Fewer request windows due to large packets Packet Length (bytes) Throughput (normalized) No enhancers on 2.56Mbps Piggybacking on 2.56Mbps No enhancers on 5.12Mbps Piggybacking on 5.12Mbps Packet Length (bytes) 21 of 30 MSAN 2005

22 Results Is Piggybacking Ever Better than Concatenation? Throughput (normalized) Concatenation on 0.64Mbps Piggybacking on 0.64Mbps Yes. With large packets on small channels Packet Length (bytes) Throughput (normalized) Concatenation on 1.28Mbps Piggybacking on 1.28Mbps Packet Length (bytes) 22 of 30 MSAN 2005

23 Results Is having both enhancers always useful? No. With small packets it is detrimental Concatenation on 0.64Mbps Both Enhancers on 0.64Mbps Concatenation on 1.28Mbps Both Enhancers on 1.28Mbps Throughput (normalized) Throughput (normalized) Packet Length (bytes) Packet Length (bytes) Concatenation on 2.56Mbps Both Enhancers on 2.56Mbps Concatenation on 5.12Mbps Both Enhancers on 5.12Mbps Throughput (normalized) Throughput (normalized) Packet Length (bytes) Packet Length (bytes) 23 of 30 MSAN 2005

24 Results Will an increase in channel rate always help? No, with small packets an increase can be detrimental. Throughput (normalized) Mbps channel Mbps channel Enhancer combination 24 of 30 MSAN 2005

25 Results Suppose small packets on a small channel What is the most economical way to increase throughput? Enable concatenation alone. Mid-sized packets? Must increase channel rate. Large packets? Enable piggybacking, or increase channel rate 25 of 30 MSAN 2005

26 Results Do both CMTS agree on all responses? No None on 0.64 Mbps None on 1.28 Mbps None on 1.28 Mbps None on 0.64 Mbps Throughput (normalized) Throughput (normalized) Packet Length (bytes) Packet Length (bytes) Anomalies excluded from results Useful to respective CMTS vendors 26 of 30 MSAN 2005

27 Outline Introduction DOCSIS architecture, protocol, enhancers (piggybacking, concatenation, fragmentation etc.). Background of this study InterOperability Lab., CableLabs, complexity of standard. Overview of Experiments Testbed, variables and data interpretation. Results Subset of conclusions. Summary and Discussion 27 of 30 MSAN 2005

28 Summary Characterized upstream performance of a DOCSIS system Channel rate, packet sizes, enhancers and CMTS Presented only a subset of results Empirical Results Exhaustive, measurement-based, real system-level Valuable tool for configuration design Black box approach Data currently being analyzed by vendors Also available at Future work Multi-CM characterization Other QoS enhancers Comparison with analytical models 28 of 30 MSAN 2005

29 Future Work Classification and Regression Tree Model 29 of 30 MSAN 2005

30 Discussion and Questions 30 of 30 MSAN 2005