The Internet: A Complex System at its Limits

Transcription

1 The Inter: A Complex System at its Limits Anja Feldmann TU Berlin

2 About myself: Education Diplom: University Paderborn, Germany Ph.D.: CMU, Pittsburgh, US Work experience Researcher at AT&T, Murray Hill, Florham Park, US Professor at 00 02: University des Saarlandes, Germany 02 06: TU München, Germany 06 : TU Berlin, Germany 12 : SAP SE Supervisory Board member Honors Leibnizpreis Member of Academies: Leopoldina, BBAW, Europaea 2

3 The Inter: A Complex System at its Limits Anja Feldmann TU Berlin

4 The Inter: Man made system An highly engineered system Well specified and documented (Inter Engineering Task Force (IETF)) Simple components Best effort Dump core packet switches An evolutionary system Explosive growth A moving target Many heterogeneous players with different goals Smart end-system computer 4

5 Many different players Network of autonomous systems (AS) operated by Inter Service Providers (ISP) Networks buy connectivity from other works (AS) AS A AS B AS C 5

6 Many different players ASs can peer directly or via Inter Exchange Points (IXPs): DE-CIX, AMS-IX, AS A Inter exchange point IXP IXP AS B AS C peering link 6

7 Many different players Geo-distributed data centers: Equinix, Often used by cloud providers: Amazon, Microsoft, Data centers AS A Inter exchange point IXP IXP AS B AS C peering link 7

8 Many different players Content provider works: Google, Dropbox Content distribution works (CDN): Akamai AS C AS A Data centers AS B IXP Content provider works IXP Inter exchange point peering link 8

9 Many different players Infrastructure Application service providers Applications: Games, YouTube, Web, Social Networks, IoT, Dropbox Devices: Sensors over mobiles to supercomputers Users: Novice to experts Individuals to companies Each player has his/her own goals 9

10 The Inter: Challenges Designed to be an open, cooperating system Highly interacting layered hardware/software systems Temporal: Spatial: Vertical: Between users, hosts, and works Among different components Across different working layers Mission critical operational system pushed to its limits New applications that demand better service More users, devices, data streams, 10

11 Research approach Instrumentation: Collect data from operational works Identify novel measurement opportunities New analysis techniques for data correlation Analysis: Use big-data analytics to Find invariants or revisit assumptions Detect bottlenecks Simulation: Build mirror world to Verify explanations Enable what-if analysis Shape the Inter: Optimizations, alternative designs, and deployment Protocol enhancements Network management tools Software defined working 11

12 Overview Network invariants Revisiting assumptions Root cause detection for outages/bottlenecks Improvements via software defined working Outlook 12

13 Overview Network invariants Revisiting assumptions Root cause detection for outages/bottlenecks Improvements via software defined working Outlook 13

14 Network Invariants early work Motivation: Background: Algorithm Tool: Competitive analysis worst case analysis Question? Is that needed? Statistical properties of arrival processes? Exponential ideal for average case analysis Heavy-tail need worst case analysis (Skewed distribution (some large events and many small items)) 14

15 Self-similarity in Inter traffic Heavy-tail arrival distributions almost everywhere TCP connections (Wiley&Sons 00, ACM CCR 98) IP flows (IEEE INFCOM 98) IP-packets (Leland et al. ACM Sigcomm 94) Exponential arrivals For sessions (ACM CCR 98) Self-similar behavior not only at packet level but across all protocol levels (with heavy-tail durations) Self-similarity looks the same at multiple time scales 15

16 From self-similarity to multifractal Self-similarity results from additivity Yet, where is the effect of the work? Small time scale multifractal (ACM Sigcomm 98, 99) Multifractal fractal system with continuous spectrum of exponents 16

17 From self-similarity to multifractal Self-similarity results from additivity Yet, where is the effect of the work? Small time scale multifractal (ACM Sigcomm 98, 99) (Multifractal different local dimensions) LAN trace (self-similar, large scales)» Color corresponds to local dimension WAN trace (multifractal, small scales)» Construction Cascades Simulations to validate observations 17

18 Overview Network invariants Essential for Protocol and work evolution Choosing evaluation setups Performance analysis Revisiting assumptions Root cause detection for outages/bottlenecks Improvements via software defined working Outlook 18

19 Overview Network invariants Revisiting assumptions Root cause detection for outages/bottlenecks Improvements via software defined working Outlook 19

20 Revisiting assumptions Motivation: The Inter is more than the sum of its pieces Only constant in the Inter is change Questions? Are our mental models still appropriate? Examples: Application mix Topology 20

21 Revisiting assumptions Motivation: The Inter is more than the sum of its pieces Only constant in the Inter is change Questions? Are our mental models still appropriate? Examples: Application mix Topology 21

22 Inter Killer application? Web traffic (WWW) 1993:... Hardly any 1994:... About 10% 95/96:... About 60-70% (ToN 98) Peer to peer and the Inter 2000: Hardly any 2001: Significant fraction 2009: Bittorrent designer claim dominance 22

23 Application Usage Application mix: 2009 / 2011 today Web dominates : 60% of bytes (ACM IMC 09) flash-video 25.2% RAR 14.7% image 11.5% video 7.6% other 23.4% unclass. 17.6% Peer to peer less than 14% (unclassified: 11%) Web and secure Web still dominate ~ 60% of overall traffic (PAM 15) Erman et al. found very similar results in cotemporaneous work presented at WWW'09 Arbor Network found very similar results in cotemporaneous work presented at Sigcomm 10 Dito for Sandvine and Ipoque 23

24 Application mix: Location dependent (PAM 15) Content/Content distribution: Almost 100% Web Peer to peer not only between Eyeballs Hoster/Infrastructure as a Service (IaaS) too! Different Vantage point different Application mix 24

25 Revisiting assumptions Motivation: The Inter is more than the sum of its pieces Only constant in the Inter is change Questions? Are our mental models still appropriate? Examples: Application mix Topology 25

26 Mental model pre

27 Mental model 2011 Google, Akamai, YouTube, Flattening of the AS topology 27

28 State of the art Autonomous Systems (AS): ~ AS to As links (relationships): ~ CustomerProvider: Peering: ~ (LHDGC IMC 13) ~ (LHDGC IMC 13) Agreements between two ASs: CustomerProvider: Paid Inter connectivity Peering: Exchange traffic of customers at no cost 28

29 Question What about IXPs Google, Akamai, YouTube, IXP What about Inter Exchange Points (IXP) 29

30 Inter Exchange Points (IXPs) IXP: Physical work infrastructure to facilitate exchange of Inter traffic among Autonomous Systems (AS). Idea: Keep local traffic local Improve performance at no extra costs EU: Non-profit US: For-profit Content Provider 1 AS2 Content Provider 2 AS1 Layer-2 switch AS3 AS5 AS4 30

31 Inter topology changes due to IXPs (ACM Sigcomm 12) Inter Exchange Points (IXP): Generally known: # IXPs ~ 350 worldwide Somewhat known: # ASs per IXP up to 700 Less known: # ASs ~ worldwide Even less known: IXPs = ~ Tier-1 ISP traffic (5 7 Tb/s) Our insight: Single IXP: > (Peering rate 60%) (Inter peering rate < 0.002%) => Peerings at IXPs: > Previously known # of peerings < (Packet Clear House 2016 reports > 2,000,000 peerings) 31

32 Overview Network invariants Revisiting assumptions Essential for Evaluation setups Protocol and infrastructure evolution Performance analysis Root cause detection for outages/bottlenecks Improvements via software defined working Outlook 32

33 Overview Network invariants Revisiting assumptions Root cause detection for outages/bottlenecks Improvements via software defined working Outlook 33

34 The physical Inter: Co-Location facilities/ixp interdependencies FranceIX 34

35 Infrastructure outages: Why? Causes: Power failures, human errors, bugs, cyber-attacks, natural disasters, (SIGCOMM 2017) Impact: Average cost: > $740K Application-layer failures: VOIP, banking, streaming,... Myth: Local Outages have local impact 35

36 IXP and Co-location facilities outages (SIGCOMM 2017) Many outages are not reported! Many facilities below 99.99% uptime, Even more below % uptime 36

37 Infrastructure-level outage detection How? Mine mailing lists/social s Use reachability information (route update info) as signal Passive measurement Inter-AS routing protocol: BGP Active probes traceroute Challenges: Distinguish normal route changes from outages Distributed signal with hidden information 37

38 Detecting outages (SIGCOMM 2017) Paths before outage: 38

39 Detecting outages (SIGCOMM 2017) Paths before outage: BGP AS routes: AS1 AS2 AS3 AS2 AS4 AS4 39

40 Detecting outages (SIGCOMM 2017) Paths before outage: 40

41 Detecting outages (SIGCOMM 2017) Paths before outage: Path after outage: AS paths remain the same! Problem 1: Infer the PoP-level hops between ASes Challenge: BGP hides information 41

42 Detecting outages (SIGCOMM 2017) F 2 failed IX 1 or F 2 failed Paths before outage: IX 1 still active Path after outage: AS paths remain the same! Problem 2: Correlate across vantage points via facility map Challenge: How to gather facility maps 42

43 Our approach Insights from Inter-AS Routing (BGP) Global signal at many vantage points Has additional info: BGP communities How: Contain location information Are crowdsource info Find correlated routing updates Use communities in route updates for localization Use facility level map to pinpoint outages Use public traceroute data for validation Big data analytics of many different data sources 43

44 Unexpected global dependencies Outages can have unexpected (SIGCOMM 2017) consequences Outage at some remote IXP caused traffic drop at this IXP Reason: Asymmetric routing Local dependencies Outages can have global impact! 44

45 Overview Network invariants Revisiting assumptions Root cause detection for outages/bottlenecks Essential for Critical infrastructures Regulation and law makers Long term evolution of the infrastructure Improvements via software defined working Outlook 45

46 Overview Network invariants Revisiting assumptions Root cause detection for outages/bottlenecks Improvements via software defined working Outlook 46

47 Improvements via SDN Software defined working (SDN): Separation of Control plane Software Data plane Hardware To overcome architectural limits Example projects: CloudNets Wireless works Novel attack surface 47

48 Improvements via SDN Software defined working (SDN): Separation of Control plane Software Data plane Hardware To overcome architectural limits Example projects: CloudNets Wireless works Novel attack surface 48

49 Next natural virtualization step: CloudNets Cloud computing is a big success! But what about to the cloud? Successful node virtualization Successful link virtualization Unified, fully virtualized works: CloudNets Combine working with cloud resources (Network + CPU, memory, storage,...)! 49

50 CloudNets CloudNets Flexibly specifiable virtual works Executing different protocol stacks On same substrate Why CloudNets: Facilitate innovation Add programmability Service-tailored works Research challenges: Migration, embedding, flexibility, specify, manage, (ERCIM News 12, Sigcomm Visa 09/10,ICCCN 12,ICDCN 12,UCC 12/12, ITC 13,IPTComm 11,ICDCN 13,CloudNet 13,ATC 14,IPDPS 14,ToN 14, ) 50

51 Improvements via SDN Software defined working (SDN): Separation of Control plane Software Data plane Hardware To overcome architectural limits Example projects: CloudNets Wireless works Novel attack surface 51

52 Commodity WiFi Challenges Almost no support for work management Client management mobility, authentication Deployment unplanned and uncoordinated No control over data path no user participation Goal: Seamless integration of wired, cellular, and WiFi Inter HotSpot Home Networks

53 Software defined working Load Balancing Routing Application X Control Plane Separation of Control and Data plane 53

54 Software defined working Load Balancing Routing Application X Control Plane Separation of Control and Data plane OpenFlow well suited for wired works 54

55 Software defined wireless working Mobility Mgnt. Load Balancing Application X Control Plane WiFi AP WiFi AP WiFi AP WiFi AP Not applicable to WiFi works 55

56 Software defined wireless working Odin: Virtualizing WiFi Access Points Enables orchestration of WiFi works Converged WiFi/wired Infrastructure (Usenix ATC 14, HotSDN 12) OpenSDWN: WiFi data path programmability and NFV management Per-Client virtual middle box abstraction (ACM SOSR 15) LegoFi: Functional decomposition (ACM MobiArch 16) 56

57 Improvements via SDN Software defined working (SDN): Separation of Control plane Software Data plane Hardware To overcome architectural limits Example projects: CloudNets Wireless works Novel attack surface 57

58 SDN: Novel attack surfaces (ongoing work) Software defined working and data centers: Data center provide virtual machines Networking via virtual (software) switches Attacker surface Virtual switches attackable, e.g., protocol parser Trusted Input (no extra firewall between VMs and switch) Co-Location of hypervisor and kernel SDN uses centralized control with direct communication Attacker with cloud VM and weaponized bug in virtual switch Attacker has full control of the cloud Need to reduce attack surface Typical data center setup 58

59 Overview Network invariants Revisiting assumptions Root cause detection for outages/bottlenecks Improvements via software defined working Essential for Outlook Long term evolution of the Inter and data analytic systems Unified management of all work types (WiFi/5G/fixed) Infrastructure security 59

60 Overview Network invariants Revisiting assumptions Root cause detection for outages/bottlenecks Improvements via software defined working Outlook 60

61 Overview Network invariants Revisiting assumptions Root cause detection for outages/bottlenecks Improvements via software defined working Outlook Distributed wide area data analytics Inter challenges Summary 61

62 Motivation: Inter of Things Integration of physical and digital world Motivating example: Car Has many parts of many different companies Needs local processing due to intermitted and limited work Interacts with local environment, e.g., at intersection Goals: Improve throughput, pollution, accident rate, Needs global (city-/country-wide) context Goals: Support for self-driving and driven cars 62

63 Motivation: Inter of Things Integration of physical and digital world Challenges Many different players (different from today s Inter) Some controlled some uncontrollable Continuous data flow processing Requires hierarchical abstractions and structure Needs real-time and cannot tolerate prolonged outages Distributed wide area data analytics 63

64 Distributed wide area data analytics: How? Single consumer (target): Many distributed data sources Need joins Heterogeneous sources Preprocessed data Need splits Data flow processing Processing Pipeline Idea: Collaborative Data Processing Pipelines 64

65 Distributed wide area data analytics: How? (ICDCS 17) Collaborative Data Processing Pipelines (CDPPs) Bundles all resources to derive information from data Builds upon CloudNets and SDN Overlays on top of the Inter Challenges: CDPP data and control plane Data plane: Fast, efficient, security, tamper resistant, anonymous Keeps data and information local yet enables global analysis Control plane has to answer questions such as: Based on which data elements was this information derived? Did anyone tamper with the data? Interactions with the Inter 65

66 Inter challenges Manage services rather than components Consistent unified control plane for Network (, mobile, backbone) Cloud, fog (compute and storage) Data (raw and derived) both within and across ISPs Enable debugging: Build in failure detection/localization Build in feedback to effected services Security: Eliminate misconfiguration opportunities 66

67 The Inter: A Complex System at its Limits My research approach Instrument Analyze Simulate, what-if Challenges: Find appropriate protocol enhancements Network management: CloudNets and CDPPs Enable efficient, scalable, distributed wide area big data analytics 67

68 Summary: My research approach Instrumentation: Collect data from operational works Identify novel measurement opportunities New analysis techniques for data correlation Analysis: Use big-data analytics to Find invariants or revisit assumptions Detect bottlenecks Simulation: Build mirror world to Verify explanations Enable what-if analysis Shape the Inter and big data analytics: Optimizations, alternative designs, and deployment Protocol enhancements Network management: CloudNets and CDPPs Enable efficient, scalable, distributed wide area big data analytics 68

69 Overview Network invariants Revisiting assumptions Root cause detection for outages/bottlenecks Improvements via software defined working Outlook Distributed wide area data analytics Inter challenges Summary 69