Sibyl A Practical Internet Route Oracle

Transcription

1 Sibyl A Practical Internet Route Oracle Ítalo Cunha1, Pietro Marchetta2, Matt Calder3, Yi-Ching Chiu3 Brandon Schlinker3, Bruno Machado1, Antonio Pescapè2 Vasileios Giotsas4, Harsha Madhyastha5, Ethan Katz-Bassett3

2 The Art of Network Troubleshooting 1. Operator has routing problem 2. Runs traceroute to the destination 3. Needs to identify where the problem is 2

3 The Art of Network Troubleshooting 1. Operator has routing problem? 2. Runs traceroute to the destination 3. Needs to identify where the problem is Are routes through GTT in Seattle experiencing problems? Are routes through Level3 in LA experiencing problems?? 3

4 The Art of Network Troubleshooting 1. Operator has routing problem 2. Runs traceroute to the destination 3. Needs to identify where the problem is Are routes through GTT in Seattle experiencing problems? Are routes through Level3 in LA experiencing problems? 4

5 Operators Have Complex Questions Someone suggests problem is on link between NTT and GTT in Seattle. Which routes use that link and could be impacted? 5

6 Operators Have Complex Questions Someone suggests problem is on link between NTT and GTT in Seattle. Which routes use that link and could be impacted? If route to destination D is impacted, which of my providers have a route that avoids that link? Providers in my region with routes that avoid that link? 6

7 The Art of Network Troubleshooting 1. Operator has routing problem? 2. Runs traceroute to the destination 3. Needs to identify where the problem is Are routes through GTT experiencing problems? Are routes through Level3 experiencing problems?? 7

8 Problems with the Current Approach 1. Takes time 2. Requires help from others a. Needs operator with right VP to respond b. Destination to measure may not be obvious 3. Limits analyses 8

9 The Art of Internet Measurement 1. Researcher wants to study, for example, boomerang routes 9

10 Researchers Have Complex Questions What routes from the US to the US detour through Canada? 10

11 Researchers Have Complex Questions What routes from the US to the US detour through Canada? Eastward routes detouring through Canada? Routes from Southern US through Canada? Mexico? Routes from Africa to Africa through Europe? 11

12 The Art of Internet Measurement 1. Researcher wants to study, for example, boomerang routes 2. Collect (a lot of) traceroute measurements 12

13 The Art of Internet Measurement 1. Researcher wants to study, for example, boomerang routes 2. Collect (a lot of) traceroute measurements Peering relationship? 13

14 The Art of Internet Measurement 1. Researcher wants to study, for example, boomerang routes 2. Collect (a lot of) traceroute measurements PlanetLab lacks diversity. Other platforms cannot make exhaustive measurements. 14

15 The number one go-to tool is traceroute. NANOG Troubleshooting Tutorial, NANOG Traceroute Tutorial,

16 The number one go-to tool is traceroute. NANOG Troubleshooting Tutorial, NANOG Traceroute Tutorial, But traceroute only answers one simple question: What is the path from vantage point s to destination d? 16

17 The number one go-to tool is traceroute. NANOG Troubleshooting Tutorial, NANOG Traceroute Tutorial, But traceroute only answers one simple question: What is the path from vantage point s to destination d? We need a new tool to answer our questions 17

18 Sibyl Supports high-level queries over Internet routes

19 Goal: Serve High-Level Routing Queries Users submit high-level queries Sibyl Queries Traceroutes that match queries 19

20 Query Internet Routes Using Regular Expressions Symbols representing boolean expressions that IP addresses must satisfy Level3&LA GTT&Seattle 20

21 Query Internet Routes Using Regular Expressions Symbols representing boolean expressions that IP addresses must satisfy Level3&LA GTT&Seattle Routes traversing Level3 in LA and GTT in Seattle? ^ * Level3&LA * GTT&Seattle * $ 21

22 Regular Expressions Capture Operational Questions Someone suggests problem is on link between NTT and GTT in Seattle. Which routes use that link and could be impacted? ^ * NTT&Seattle GTT&Seattle * $ 22

23 Regular Expressions Capture Operational Questions Someone suggests problem is on link between NTT and GTT in Seattle. Which routes use that link and could be impacted? ^ * NTT&Seattle GTT&Seattle * $ If route to destination D is impacted, which of my providers have a route that avoids that link? ^ USC (?!NTT GTT&Seattle) * D $ Providers in my region with routes that avoid that link? ^ * LA (?!NTT GTT&Seattle) * D $ 23

24 Regular Expressions Capture Research Questions What routes from the US to the US detour through Canada? ^ US* Canada* US* $ Eastward routes detouring through Canada? ^ EastCoast * Canada * WestCoast $ Routes from Southern US through Canada? Mexico? ^ SouthernUS * (Canada Mexico) * US $ Routes from Africa to Africa through Europe? ^ Africa * Europe * Africa $ 24

25 How do we achieve route coverage to satisfy complex and diverse queries? Users submit high-level queries Sibyl Queries Traceroutes that match queries Measure Paths 25

26 Combining Platforms Provides Great Coverage Vantage points in 100% of very large networks and 50% of small and regional networks 26

27 Combining Platforms Provides Great Coverage Vantage points in 100% of very large networks and 50% of small and regional networks Measurement Platform Vantage Points Vantage Point ASes PlanetLab ~400 ~250 Traceroute servers ~500 ~500 ~9000 ~2700 RIPE Atlas 27

28 Vantage Points Are Resource Constrained Want to use network coverage to answer queries, but cannot issue all measurements Measurement Platform Vantage Points Vantage Point ASes Traces per Day PlanetLab ~400 ~250 16,000K Traceroute servers ~500 ~ K ~9000 ~ K RIPE Atlas 28

29 Vantage Points Are Resource Constrained Total coverage Number of ASes Seen toward Each Destination 29

30 Resource Constraints Limit Visibility Total coverage Rate limits Number of ASes Seen toward Each Destination 30

31 Resource Constraints Limit Visibility Goal: satisfy queries as if we have total coverage, despite rate limits Total coverage Rate limits Number of ASes Seen toward Each Destination 31

32 Resource Constraints Limit Visibility Goal: satisfy queries as if we have total coverage, despite rate limits Total coverage Approach: carefully choose measurements Rate limits Number of ASes Seen toward Each Destination 32

33 Too many possible measurements. How do we prune the set of measurements to consider? Users submit high-level queries Queries Sibyl Generate candidates Combine multiple platforms PlanetLab Traceroutes that match queries Measure Paths RIPE Atlas Traceroute servers 33

34 Destinations Vantage Point B ^.* A.* B.*$ A Vantage Point 34

35 Destinations Vantage Point B ^.* A.* B.*$ A Vantage Point 35

36 Destinations Vantage Point B ^.* A.* B.*$ A Vantage Point 36

37 Vantage Point Destinations D1 V2 D2 X B ^.* A.* B.*$ A Vantage Point V1 37

38 Vantage Point Destinations D1 V2 D2 X B ^.* A.* B.*$ A Vantage Point V1 38

39 Vantage Point Destinations D1 V2 D2 X B ^.* A.* B.*$ A Vantage Point Many possible splices! V1 39

40 How to estimate which candidate measurements will match queries? Users submit high-level queries Queries Sibyl Generate candidates Match likelihood Combine multiple platforms PlanetLab Traceroutes that match queries Measure Paths RIPE Atlas Traceroute servers 40

41 Destination B A ^.* A.* B.*$ Vantage Point 41

42 Destination 1. B Predict all paths for candidate measurements A ^.* A.* B.*$ Vantage Point 42

43 Destination 5% Predict all paths for candidate measurements Compute likelihood of prediction being correct 80% A 10% Vantage Point B 5% ^.* A.* B.*$ 43

44 Computing Likelihood that Prediction is Correct Input set of predicted paths between source and destination Output likelihood of each prediction being correct Approach Compute route features and use machine learning 44

45 Computing Likelihood that Prediction is Correct Input set of predicted paths between source and destination Output likelihood of each prediction being correct Approach Compute route features and use machine learning Most important features include AS peering relationships Route length 45

46 Destination 5% Predict all paths for candidate measurements Compute likelihood of prediction being correct Compute likelihood actual path matches 80% A 10% 90% Vantage Point B 5% ^.* A.* B.*$ Path matches query Path doesn t match query 46

47 Vantage Point 45% Destination 10% 45% Predict all paths for candidate measurements Compute likelihood of prediction being correct Compute likelihood actual path matches B A 10% ^.* A.* B.*$ 90% Vantage Point Path matches query Path doesn t match query 47

48 Match Likelihood Estimation Is Accurate 48

49 Match Likelihood Estimation Is Accurate Chosen measurements match queries 49

50 Optimize budget use to maximize expected utility over all queries Users submit high-level queries Queries Sibyl Generate candidates Match likelihood Optimize budget Update topology Traceroutes that match queries Combine multiple platforms PlanetLab Measure Paths RIPE Atlas Traceroute servers 50

51 Optimize budget use to maximize expected utility over all queries Users submit high-level queries Queries Sibyl Generate candidates Match likelihood Optimize budget Update topology Traceroutes that match queries PlanetLab Measure Paths Opportunistically update knowledge base Combine multiple platforms RIPE Atlas Traceroute servers 51

52 Evaluation Evaluated Sibyl over multiple rounds. Each round: System has fixed, limited probing budget to allocate Generate random (but satisfiable) queries Evaluate the fraction of queries Sibyl can satisfy 52

53 How Accurate Is Sibyl? PlanetLab only matches 52% of queries 53

54 How Accurate Is Sibyl? Adding random measurements helps little (56%) PlanetLab only matches 52% of queries 54

55 How Accurate Is Sibyl? Need intelligence to effectively match (75%) Adding random measurements helps little (56%) PlanetLab only matches 52% of queries 55

56 Conclusions Supports high-level queries over Internet routes Combines platforms to improve coverage and budget Smartly chooses which measurements to issue Overcome probing budget constraints 56

57 Future Work Improve path prediction and likelihood estimation Long-term budget allocation Balance satisfying current queries vs benefit to serve future queries Adapt probing rate as a function of query load Unify queries over historical and live data Give me paths that looked like X but now look like Y 57

58 Query Language 58

59 We Have Complex Monitoring Systems ispy: diverse routes to a given prefix? Reverse traceroute: routes through Level3 to monitoring node? RocketFuel: routes through Verizon between Chicago and LA? 59

60 Regular Expressions Support Existing Systems ispy: diverse routes to a given prefix? ^ { *} /19 $ by AS Reverse traceroute: routes through Level3 to monitoring node? ^ * Level3.* USC $ RocketFuel: route through Verizon between Chicago and LA? ^ * Verizon&Chicago * Verizon&LA * $ 60

61 Internal Representation of Regular Expressions Paths that go through Sprint s Chicago PoP on the way to Brazil: ^.*S.*U$ where S := Sprint&Chicago U := USC From NANOG: Problem between Level3 in LA and GTT in Seattle: ^.*L.*G.*$ where L := Level3&LA G := GTT&Seattle 61

62 Utility Optimization 62

63 Maximize Global Utility The union of the set of traceroutes chosen from each platform 63

64 Maximize Global Utility The union of the set of traceroutes chosen from each platform Choose the set of traceroutes that maximizes utility over all queries 64

65 Maximize Global Utility The union of the set of traceroutes chosen from each platform Choose the set of traceroutes that maximizes utility over all queries 65

66 Maximize Global Utility The union of the set of traceroutes chosen from each platform Choose the set of traceroutes that maximizes utility over all queries Subject to budget constraints of each platform 66

67 Utility functions Existence queries find one path that matches a query 67

68 Utility functions Existence queries find one path that matches a query Diversity queries find all different paths that match a part of the query 68

69 Utility functions Existence queries find one path that matches a query Diversity queries find all different paths that match a part of the query 69

70 RuleFit and Likelihood Estimation 70

71 Most Important Features 71

72 RuleFit Jaccard Index Predictions Correlate 72

73 SFSA Splicing 73

74 Query FSA Splicing to Generate Candidates ^.* A.* B.*$ * S1 * * A S2 B S3 Initial 74

75 Query FSA Splicing to Generate Candidates Hop Transition ^.* A.* B.*$ * V1 * S1 * * A S2 S1 B A S1 A S2 Hop Transition V2 X S3 X B * D2 S2 Initial D1 * S2 75

76 Query FSA Splicing to Generate Candidates Hop Transition ^.* A.* B.*$ * V1 * S1 * A S2 S1 * B A S1 A S2 Hop Transition * V2 S2 * X S2 S3 X B * D2 S2 Initial D1 * S2 S3 B S2 * S3 76

77 Query FSA Splicing to Generate Candidates Hop Transition ^.* A.* B.*$ * V1 V1 A X B D2 A X D1 S1 S1 A S2 * V2 S2 * X * B * D2 S2 S2 Hop Transition S2 S3 B S2 * S3 77

78 Evaluation 78

79 Queries Types Used in Evaluation Traverse AS toward destination ^.* A.* B $ ^ [^A]+ A+.* B $ Traverse link toward destination ^.* A B.* D $ Traverse three different locations in sequence ^.* A.* B.* C.* $ 79

80 How Much Better Can We Do? Unlimited budget would Sibyl to match 88% of queries Sibyl matches 75% of queries Current approach matches 52% of queries 80

81 How Much Does Each Module Contribute Could satisfy 84% with unlimited budget Sibyl satisfies 75% of queries Each module contributes 5-8% accuracy 81

82 Sibyl Is Effective at Different Granularities 82

83 Combining Platforms Helps Satisfy More Queries 83

84 Sibyl Uncovers More Path Diversity 84

85 Staleness 85

86 Patching and Pruning Stale Measurements New measurements may detect path changes Whenever we detect a change toward a destination, update all paths to that destination that overlap Whenever we detect a path change from a source, update all paths from that source that overlap 86

87 Staleness Has Small Impact 87

88 Path Change Properties 88

89 Uphill Path Changes Are Less Likely 89