Networking Seminar Stanford University. Madan Jampani 3/12/2015

Size: px
Start display at page:

Download "Networking Seminar Stanford University. Madan Jampani 3/12/2015"

Transcription

1 Networking Seminar Stanford University Madan Jampani 3/12/2015

2 Can SDN control plane scale without sacrificing abstractions and performance?

3 Control Plane Data Plane

4 Simple yet powerful abstraction Global Network View Scaling strong consistency Managing Complexity

5 What is the simplest controller? Observe Program

6 Abstractions Single Instance Performance Correctness Availability Scale

7 How to improve availability?

8 How to improve availability? Primary Standby

9 How to improve availability? Standby Primary

10 Abstractions Single Instance Primary/ Standby Performance Correctness Availability Scale

11 Why you might need to scale? Primary Standby

12 Why you might need to scale? Primary Standby

13 Why you might need to scale? Primary Standby

14 Why you might need to scale? Primary Standby

15 Fully Distributed Control Plane

16

17

18

19

20

21

22 Abstractions Single Instance Primary/ Standby MultiInstance Performance Correctness Availability Scale

23

24

25

26 Abstractions Single Instance Primary/ Standby Multi Instance Performance Correctness Availability Scale

27 Tell me about your slice?

28 Abstractions Single Instance Primary/ Standby Multi Instance Performance Correctness Availability Scale

29 Tell me about your slice? Cache

30 Abstractions Single Instance Primary/ Standby Multi Instance Performance Correctness Availability Scale

31 Can we build a complete solution? Let s look at prior art...

32 Write Topology State Topology Events Distributed Topology Store

33 Read Topology State Distributed Topology Store

34 Read Topology State Cache Distributed Topology Store

35 Read Topology State Cache Distributed Topology Store

36 Can we build a solution that meets all criteria?

37 Topology as a State Machine Current Topology apply event Events are Switch/Port/Link up/down Updated Topology

38

39

40 E

41 E E E

42 Pros Simple Fast Cons Dropped messages Reordered messages

43 E

44 F

45 F E

46 We have a single writer problem

47

48

49 Switch Mastership Terms C1 C2 C We track this in a strongly consistent store

50 Switch Event Numbers C1 C2 C

51 Partial Ordering of Topology Events Each event has a unique logical timestamp (Switch ID, Term Number, Event Number)

52 E (S1, 1, 2)

53 E (S1, 1, 2) E (S1, 1, 2) (S1, 1, 2) E

54 F (S1, 2, 1)

55 F (S1, 2, 1) F (S1, 2, 1)

56 F (S1, 2, 1) F (S1, 2, 1) E (S1, 1, 2)

57 F (S1, 2, 1) F (S1, 2, 1) E (S1, 1, 2)

58 To summarize Each instance has a full copy of network topology Events are timestamped on arrival and broadcasted Stale events are dropped on receipt

59 There is one additional step... What about dropped messages?

60 Did you hear about the port that went offline?

61

62 Anti-Entropy Lightweight, Gossip style, peer-to-peer Quickly bootstraps newly joined nodes

63 Abstractions Single Instance Primary/ Standby Multi Instance Performance Correctness Availability Scale

64 A model for state tracking If you are the observer, Eventual Consistency is the best option View should be consistent with the network, not with other views

65 Hiding the complexity EventuallyConsistentMap<K, V, T> Plugin your own timestamp generation

66 What about other Control Plane state... Switch to Controller mapping Resource Allocations Flows Various application generated state

67 What about other Control Plane state... Switch to Controller mapping Resource Allocations Flows Various application generated state In all case strong consistency is either required or highly desirable

68 Consistency => Coordination 2PC All participants need to be available Consensus Only a majority need to participate

69 State Consistency through Consensus LOG LOG LOG

70 State Consistency through Consensus LOG LOG LOG LOG LOG LOG LOG

71 Scaling Consistency

72 Scaling Consistency

73 Scaling Consistency

74 Consistency Cost Atomic updates within a shard are cheap Atomic updates spanning shards use 2PC

75 Hiding Complexity ConsistentMap<K, V>

76 To Conclude SDN at scale is possible if we can take care of state management Abstractions are important

77 Thank you

Intuitive distributed algorithms. with F#

Intuitive distributed algorithms. with F# Intuitive distributed algorithms with F# Natallia Dzenisenka Alena Hall @nata_dzen @lenadroid A tour of a variety of intuitivedistributed algorithms used in practical distributed systems. and how to prototype

More information

EECS 498 Introduction to Distributed Systems

EECS 498 Introduction to Distributed Systems EECS 498 Introduction to Distributed Systems Fall 2017 Harsha V. Madhyastha Dynamo Recap Consistent hashing 1-hop DHT enabled by gossip Execution of reads and writes Coordinated by first available successor

More information

Distributed Systems 8L for Part IB. Additional Material (Case Studies) Dr. Steven Hand

Distributed Systems 8L for Part IB. Additional Material (Case Studies) Dr. Steven Hand Distributed Systems 8L for Part IB Additional Material (Case Studies) Dr. Steven Hand 1 Introduction The Distributed Systems course covers a wide range of topics in a variety of areas This handout includes

More information

Cassandra - A Decentralized Structured Storage System. Avinash Lakshman and Prashant Malik Facebook

Cassandra - A Decentralized Structured Storage System. Avinash Lakshman and Prashant Malik Facebook Cassandra - A Decentralized Structured Storage System Avinash Lakshman and Prashant Malik Facebook Agenda Outline Data Model System Architecture Implementation Experiments Outline Extension of Bigtable

More information

Important Lessons. Today's Lecture. Two Views of Distributed Systems

Important Lessons. Today's Lecture. Two Views of Distributed Systems Important Lessons Replication good for performance/ reliability Key challenge keeping replicas up-to-date Wide range of consistency models Will see more next lecture Range of correctness properties L-10

More information

TECHNICAL WHITE PAPER

TECHNICAL WHITE PAPER TECHNICAL WHITE PAPER This white paper, intended for a technical audience, describes the challenges of architecting a carrier grade SDN control plane, provides an effective set of metrics to evaluate its

More information

Rendezvous Point Engineering

Rendezvous Point Engineering Rendezvous Point Engineering Last updated: November 2008 Introduction A Rendezvous Point (RP) is a router in a multicast network domain that acts as a shared root for a multicast shared tree. Any number

More information

Outline. Introduction Background Use Cases Data Model & Query Language Architecture Conclusion

Outline. Introduction Background Use Cases Data Model & Query Language Architecture Conclusion Outline Introduction Background Use Cases Data Model & Query Language Architecture Conclusion Cassandra Background What is Cassandra? Open-source database management system (DBMS) Several key features

More information

ONOS OVERVIEW. Architecture, Abstractions & Application

ONOS OVERVIEW. Architecture, Abstractions & Application ONOS OVERVIEW Architecture, Abstractions & Application WHAT IS ONOS? Open Networking Operating System (ONOS) is an open source SDN network operating system (controller). Mission: to enable Service Providers

More information

Spotify. Scaling storage to million of users world wide. Jimmy Mårdell October 14, 2014

Spotify. Scaling storage to million of users world wide. Jimmy Mårdell October 14, 2014 Cassandra @ Spotify Scaling storage to million of users world wide! Jimmy Mårdell October 14, 2014 2 About me Jimmy Mårdell Tech Product Owner in the Cassandra team 4 years at Spotify

More information

CS505: Distributed Systems

CS505: Distributed Systems Cristina Nita-Rotaru CS505: Distributed Systems Protocols. Slides prepared based on material by Prof. Ken Birman at Cornell University, available at http://www.cs.cornell.edu/ken/book/ Required reading

More information

Dynamo: Amazon s Highly Available Key-value Store

Dynamo: Amazon s Highly Available Key-value Store Dynamo: Amazon s Highly Available Key-value Store Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall and

More information

SCALABLE DATABASES. Sergio Bossa. From Relational Databases To Polyglot Persistence.

SCALABLE DATABASES. Sergio Bossa. From Relational Databases To Polyglot Persistence. SCALABLE DATABASES From Relational Databases To Polyglot Persistence Sergio Bossa sergio.bossa@gmail.com http://twitter.com/sbtourist About Me Software architect and engineer Gioco Digitale (online gambling

More information

Building Replication Systems with PRACTI and PADRE

Building Replication Systems with PRACTI and PADRE Building Replication Systems with PRACTI and PADRE Discussing Two Papers PRACTI Replication N. Belaramani, M. Dahlin, L. Gao, A. Nayate, A. Venkataramani, P. Yalagandula, J. Zheng NSDI 2006 PADRE: A Policy

More information

Broadcast and Multicast Routing

Broadcast and Multicast Routing Broadcast and Multicast Routing Daniel Zappala CS 460 Computer Networking Brigham Young University Group Communication 2/34 How can the Internet provide efficient group communication? send the same copy

More information

Big Data Development CASSANDRA NoSQL Training - Workshop. November 20 to (5 days) 9 am to 5 pm HOTEL DUBAI GRAND DUBAI

Big Data Development CASSANDRA NoSQL Training - Workshop. November 20 to (5 days) 9 am to 5 pm HOTEL DUBAI GRAND DUBAI Big Data Development CASSANDRA NoSQL Training - Workshop November 20 to 24 2016 (5 days) 9 am to 5 pm HOTEL DUBAI GRAND DUBAI ISIDUS TECH TEAM FZE PO Box 9798 Dubai UAE, email training-coordinator@isidusnet

More information

Distributed Systems 8L for Part IB

Distributed Systems 8L for Part IB Distributed Systems 8L for Part IB Handout 3 Dr. Steven Hand 1 Distributed Mutual Exclusion In first part of course, saw need to coordinate concurrent processes / threads In particular considered how to

More information

Message Authentication and Hash function

Message Authentication and Hash function Message Authentication and Hash function Concept and Example 1 Approaches for Message Authentication Encryption protects message against passive attack, while Message Authentication protects against active

More information

Consistency-Based SLA for Cloud Storage

Consistency-Based SLA for Cloud Storage Consistency-Based SLA for Cloud Storage Patryk Hes Distributed Systems course, 2016 Based on: Consistency-Based Service Level Agreements for Cloud Storage Douglas B. Terry, Vijayan Prabhakaran, Ramakrishna

More information

Peer-to-peer Sender Authentication for . Vivek Pathak and Liviu Iftode Rutgers University

Peer-to-peer Sender Authentication for  . Vivek Pathak and Liviu Iftode Rutgers University Peer-to-peer Sender Authentication for Email Vivek Pathak and Liviu Iftode Rutgers University Email Trustworthiness Sender can be spoofed Need for Sender Authentication Importance depends on sender Update

More information

Dynamo. Smruti R. Sarangi. Department of Computer Science Indian Institute of Technology New Delhi, India. Motivation System Architecture Evaluation

Dynamo. Smruti R. Sarangi. Department of Computer Science Indian Institute of Technology New Delhi, India. Motivation System Architecture Evaluation Dynamo Smruti R. Sarangi Department of Computer Science Indian Institute of Technology New Delhi, India Smruti R. Sarangi Leader Election 1/20 Outline Motivation 1 Motivation 2 3 Smruti R. Sarangi Leader

More information

Inter-controller Traffic in ONOS Clusters for SDN Networks

Inter-controller Traffic in ONOS Clusters for SDN Networks Inter-controller Traffic in Clusters for SDN Networks Abubakar Siddique Muqaddas, Andrea Bianco, Paolo Giaccone, Guido Maier Dip. di Elettronica e Telecomunicazioni, Politecnico di Torino, Italy Dip. di

More information

Large-Scale Data Stores and Probabilistic Protocols

Large-Scale Data Stores and Probabilistic Protocols Distributed Systems 600.437 Large-Scale Data Stores & Probabilistic Protocols Department of Computer Science The Johns Hopkins University 1 Large-Scale Data Stores and Probabilistic Protocols Lecture 11

More information

Cloudy Weather for P2P

Cloudy Weather for P2P Cloudy Weather for P2P with a Chance of Gossip Alberto Montresor Luca Abeni Best paper award in P2P'11 Presented by: amir@sics.se 1 Introduction 2 Cloud Computing vs. P2P Similarity: Providing the infinite

More information

Causal Order Multicast Protocol Using Different Information from Brokers to Subscribers

Causal Order Multicast Protocol Using Different Information from Brokers to Subscribers , pp.15-19 http://dx.doi.org/10.14257/astl.2014.51.04 Causal Order Multicast Protocol Using Different Information from Brokers to Subscribers Chayoung Kim 1 and Jinho Ahn 1, 1 Dept. of Comp. Scie., Kyonggi

More information

Distributed System. Gang Wu. Spring,2018

Distributed System. Gang Wu. Spring,2018 Distributed System Gang Wu Spring,2018 Lecture4:Failure& Fault-tolerant Failure is the defining difference between distributed and local programming, so you have to design distributed systems with the

More information

Advanced Peer to Peer Discovery and Interaction Framework

Advanced Peer to Peer Discovery and Interaction Framework Advanced Peer to Peer Discovery and Interaction Framework Peeyush Tugnawat J.D. Edwards and Company One, Technology Way, Denver, CO 80237 peeyush_tugnawat@jdedwards.com Mohamed E. Fayad Computer Engineering

More information

Dynamo: Key-Value Cloud Storage

Dynamo: Key-Value Cloud Storage Dynamo: Key-Value Cloud Storage Brad Karp UCL Computer Science CS M038 / GZ06 22 nd February 2016 Context: P2P vs. Data Center (key, value) Storage Chord and DHash intended for wide-area peer-to-peer systems

More information

Radix - Tempo. Dan Hughes Abstract

Radix - Tempo. Dan Hughes   Abstract Radix - Tempo Monday 25 t h September, 2017 Dan Hughes www.radix.global Abstract In this paper we present a novel method for implementing a Distributed Ledger that preserves total order of events allowing

More information

Radix - Tempo. Dan Hughes

Radix - Tempo. Dan Hughes Radix - Tempo Dan Hughes 25 th September 2017 Abstract In this paper we present a novel method for implementing a Distributed Ledger that preserves total order of events allowing for the trustless transfer

More information

Replica Placement. Replica Placement

Replica Placement. Replica Placement Replica Placement Model: We consider objects (and don t worry whether they contain just data or code, or both) Distinguish different processes: A process is capable of hosting a replica of an object or

More information

Virtual Private Networks Advanced Technologies

Virtual Private Networks Advanced Technologies Virtual Private Networks Advanced Technologies Petr Grygárek rek Agenda: Supporting Technologies (GRE, NHRP) Dynamic Multipoint VPNs (DMVPN) Group Encrypted Transport VPNs (GET VPN) Multicast VPNs (mvpn)

More information

10. Replication. Motivation

10. Replication. Motivation 10. Replication Page 1 10. Replication Motivation Reliable and high-performance computation on a single instance of a data object is prone to failure. Replicate data to overcome single points of failure

More information

Advanced Computer Networks. Network Virtualization

Advanced Computer Networks. Network Virtualization Advanced Computer Networks 263 3501 00 Network Virtualization Patrick Stuedi Spring Semester 2014 1 Oriana Riva, Department of Computer Science ETH Zürich Outline Last week: Portland VL2 Today Network

More information

Axway API Management 7.5.x Cassandra Best practices. #axway

Axway API Management 7.5.x Cassandra Best practices. #axway Axway API Management 7.5.x Cassandra Best practices #axway Axway API Management 7.5.x Cassandra Best practices Agenda Apache Cassandra - Overview Apache Cassandra - Focus on consistency level Apache Cassandra

More information

Availability versus consistency. Eventual Consistency: Bayou. Eventual consistency. Bayou: A Weakly Connected Replicated Storage System

Availability versus consistency. Eventual Consistency: Bayou. Eventual consistency. Bayou: A Weakly Connected Replicated Storage System Eventual Consistency: Bayou Availability versus consistency Totally-Ordered Multicast kept replicas consistent but had single points of failure Not available under failures COS 418: Distributed Systems

More information

Configuring Port Channels

Configuring Port Channels CHAPTER 5 This chapter describes how to configure port channels and to apply and configure the Link Aggregation Control Protocol (LACP) for more efficient use of port channels in Cisco DCNM. For more information

More information

Last time. Distributed systems Lecture 6: Elections, distributed transactions, and replication. DrRobert N. M. Watson

Last time. Distributed systems Lecture 6: Elections, distributed transactions, and replication. DrRobert N. M. Watson Distributed systems Lecture 6: Elections, distributed transactions, and replication DrRobert N. M. Watson 1 Last time Saw how we can build ordered multicast Messages between processes in a group Need to

More information

CS6450: Distributed Systems Lecture 13. Ryan Stutsman

CS6450: Distributed Systems Lecture 13. Ryan Stutsman Eventual Consistency CS6450: Distributed Systems Lecture 13 Ryan Stutsman Material taken/derived from Princeton COS-418 materials created by Michael Freedman and Kyle Jamieson at Princeton University.

More information

MySQL Group Replication. Bogdan Kecman MySQL Principal Technical Engineer

MySQL Group Replication. Bogdan Kecman MySQL Principal Technical Engineer MySQL Group Replication Bogdan Kecman MySQL Principal Technical Engineer Bogdan.Kecman@oracle.com 1 Safe Harbor Statement The following is intended to outline our general product direction. It is intended

More information

Distributed Systems. Characteristics of Distributed Systems. Lecture Notes 1 Basic Concepts. Operating Systems. Anand Tripathi

Distributed Systems. Characteristics of Distributed Systems. Lecture Notes 1 Basic Concepts. Operating Systems. Anand Tripathi 1 Lecture Notes 1 Basic Concepts Anand Tripathi CSci 8980 Operating Systems Anand Tripathi CSci 8980 1 Distributed Systems A set of computers (hosts or nodes) connected through a communication network.

More information

Distributed Systems. Characteristics of Distributed Systems. Characteristics of Distributed Systems. Goals in Distributed System Designs

Distributed Systems. Characteristics of Distributed Systems. Characteristics of Distributed Systems. Goals in Distributed System Designs 1 Anand Tripathi CSci 8980 Operating Systems Lecture Notes 1 Basic Concepts Distributed Systems A set of computers (hosts or nodes) connected through a communication network. Nodes may have different speeds

More information

Reprise: Stability under churn (Tapestry) A Simple lookup Test. Churn (Optional Bamboo paper last time)

Reprise: Stability under churn (Tapestry) A Simple lookup Test. Churn (Optional Bamboo paper last time) EECS 262a Advanced Topics in Computer Systems Lecture 22 Reprise: Stability under churn (Tapestry) P2P Storage: Dynamo November 20 th, 2013 John Kubiatowicz and Anthony D. Joseph Electrical Engineering

More information

Distributed Systems. 19. Spanner. Paul Krzyzanowski. Rutgers University. Fall 2017

Distributed Systems. 19. Spanner. Paul Krzyzanowski. Rutgers University. Fall 2017 Distributed Systems 19. Spanner Paul Krzyzanowski Rutgers University Fall 2017 November 20, 2017 2014-2017 Paul Krzyzanowski 1 Spanner (Google s successor to Bigtable sort of) 2 Spanner Take Bigtable and

More information

Virtual Private Networks Advanced Technologies

Virtual Private Networks Advanced Technologies Virtual Private Networks Advanced Technologies Petr Grygárek rek Agenda: Supporting Technologies (GRE, NHRP) Dynamic Multipoint VPNs (DMVPN) Group Encrypted Transport VPNs (GET VPN) Multicast VPNs (mvpn)

More information

Simple 2-Trunk Intermediate System (S2IS)

Simple 2-Trunk Intermediate System (S2IS) Simple 2-Trunk Intermediate System (S2IS) Norman Finn Rev 2 July 3, 2013 1 ISIS is increasingly important to IEEE 802.1 networks Ø Basis for optimal forwarding for SPBV and SPBM networks. Ø Basis for topology

More information

416 Distributed Systems. Mar 3, Peer-to-Peer Part 2

416 Distributed Systems. Mar 3, Peer-to-Peer Part 2 416 Distributed Systems Mar 3, Peer-to-Peer Part 2 Scaling Problem Millions of clients server and network meltdown 2 P2P System Leverage the resources of client machines (peers) Traditional: Computation,

More information

Introduction to IP Routing. Geoff Huston

Introduction to IP Routing. Geoff Huston Introduction to IP Routing Geoff Huston Routing How do packets get from A to B in the Internet? A Internet B Connectionless Forwarding Each router (switch) makes a LOCAL decision to forward the packet

More information

Pangaea: Wide-area File System. Taming Aggressive Replication in the Pangaea Wide-area File System. Pangaea Design Goals

Pangaea: Wide-area File System. Taming Aggressive Replication in the Pangaea Wide-area File System. Pangaea Design Goals Taming ggressive Replication in the Pangaea Wide-area ile System Pangaea: Wide-area ile System o Support the daily storage needs of distributed users. Y. Saito,. Kaamanolis, M. Karlsson, M. Mahalingam

More information

NoSQL systems: sharding, replication and consistency. Riccardo Torlone Università Roma Tre

NoSQL systems: sharding, replication and consistency. Riccardo Torlone Università Roma Tre NoSQL systems: sharding, replication and consistency Riccardo Torlone Università Roma Tre Data distribution NoSQL systems: data distributed over large clusters Aggregate is a natural unit to use for data

More information

Basic vs. Reliable Multicast

Basic vs. Reliable Multicast Basic vs. Reliable Multicast Basic multicast does not consider process crashes. Reliable multicast does. So far, we considered the basic versions of ordered multicasts. What about the reliable versions?

More information

ONOS. Open Network Operating System. Ali Al-Shabibi and Andrea Campanella. ON.Lab 13/09/2016 TIM Labs, Turin. #ONOSProject

ONOS. Open Network Operating System. Ali Al-Shabibi and Andrea Campanella. ON.Lab 13/09/2016 TIM Labs, Turin. #ONOSProject ONOS Open Network Operating System Ali Al-Shabibi and Andrea Campanella ON.Lab 13/09/2016 TIM Labs, Turin Outline Introduction to ONOS and ON.Lab Architecture Northbound interface Southbound interface

More information

Control as LCD for future networking

Control as LCD for future networking IETF 96 IRTF SDNRG Berlin, Germany July 22, 2016 Control as LCD for future networking Artur Hecker and Zoran Despotovic European Research Center, Munich Huawei Technologies Programmable networks: change

More information

Design and Implementation of Modern Programming Languages (Seminar)

Design and Implementation of Modern Programming Languages (Seminar) Design and Implementation of Modern Programming Languages (Seminar) Outline Administrivia Intro Schedule Topics GENERAL INFORMATION Intro Introduce students to the core techniques of scientific work Give

More information

Scalable overlay Networks

Scalable overlay Networks overlay Networks Dr. Samu Varjonen 1 Contents Course overview Lectures Assignments/Exercises 2 Course Overview Overlay networks and peer-to-peer technologies have become key components for building large

More information

Taming the Flood: How I Learned to Stop Worrying and Love the Swarm Yogesh Vedpathak Cleversafe

Taming the Flood: How I Learned to Stop Worrying and Love the Swarm Yogesh Vedpathak Cleversafe Taming the Flood: How I Learned to Stop Worrying and Love the Swarm Yogesh Vedpathak Cleversafe Topics Popular data Creating broadcasting storage system Bittorrent protocol Creating swarms and destroying

More information

Horizontal or vertical scalability? Horizontal scaling is challenging. Today. Scaling Out Key-Value Storage

Horizontal or vertical scalability? Horizontal scaling is challenging. Today. Scaling Out Key-Value Storage Horizontal or vertical scalability? Scaling Out Key-Value Storage COS 418: Distributed Systems Lecture 8 Kyle Jamieson Vertical Scaling Horizontal Scaling [Selected content adapted from M. Freedman, B.

More information

Scaling Out Key-Value Storage

Scaling Out Key-Value Storage Scaling Out Key-Value Storage COS 418: Distributed Systems Logan Stafman [Adapted from K. Jamieson, M. Freedman, B. Karp] Horizontal or vertical scalability? Vertical Scaling Horizontal Scaling 2 Horizontal

More information

CONFIGURATION DU SWITCH

CONFIGURATION DU SWITCH Current configuration : 2037 bytes version 12.2 no service pad service timestamps debug uptime service timestamps log uptime no service password-encryption hostname Switch no aaa new-model ip subnet-zero

More information

Ethane: taking control of the enterprise

Ethane: taking control of the enterprise Ethane: taking control of the enterprise Martin Casado et al Giang Nguyen Motivation Enterprise networks are large, and complex, and management is distributed. Requires substantial manual configuration.

More information

Compiling Path Queries

Compiling Path Queries Compiling Path Queries Princeton University Srinivas Narayana Mina Tahmasbi Jen Rexford David Walker Management = Measure + Control Network Controller Measure Control Software-Defined Networking (SDN)

More information

Corrected Gossip Algorithms for Fast Reliable Broadcast on Unreliable Systems

Corrected Gossip Algorithms for Fast Reliable Broadcast on Unreliable Systems TORSTEN HOEFLER, AMNON BARAK, AMNON SHILOH, ZVI DREZNER Corrected Gossip Algorithms for Fast Reliable Broadcast on Unreliable Systems Failures in large-scale computing system The number of components grows

More information

Early Measurements of a Cluster-based Architecture for P2P Systems

Early Measurements of a Cluster-based Architecture for P2P Systems Early Measurements of a Cluster-based Architecture for P2P Systems Balachander Krishnamurthy, Jia Wang, Yinglian Xie I. INTRODUCTION Peer-to-peer applications such as Napster [4], Freenet [1], and Gnutella

More information

VXLAN Overview: Cisco Nexus 9000 Series Switches

VXLAN Overview: Cisco Nexus 9000 Series Switches White Paper VXLAN Overview: Cisco Nexus 9000 Series Switches What You Will Learn Traditional network segmentation has been provided by VLANs that are standardized under the IEEE 802.1Q group. VLANs provide

More information

Underlying Technologies -Continued-

Underlying Technologies -Continued- S465 omputer Networks Spring 2004 hapter 3 (Part B) Underlying Technologies -ontinued- Dr. J. Harrison These slides were produced from material by Behrouz Forouzan for the text TP/IP Protocol Suite (2

More information

Top-Down Network Design

Top-Down Network Design Top-Down Network Design Chapter Seven Selecting Switching and Routing Protocols Original slides by Cisco Press & Priscilla Oppenheimer Selection Criteria for Switching and Routing Protocols Network traffic

More information

Megastore: Providing Scalable, Highly Available Storage for Interactive Services & Spanner: Google s Globally- Distributed Database.

Megastore: Providing Scalable, Highly Available Storage for Interactive Services & Spanner: Google s Globally- Distributed Database. Megastore: Providing Scalable, Highly Available Storage for Interactive Services & Spanner: Google s Globally- Distributed Database. Presented by Kewei Li The Problem db nosql complex legacy tuning expensive

More information

RapidChain: Scaling Blockchain via Full Sharding

RapidChain: Scaling Blockchain via Full Sharding RapidChain: Scaling Blockchain via Full Sharding Mahdi Zamani Visa Research Join work with Mahnush Movahedi, Dfinity Mariana Raykova, Yale University Stanford Blockchain Seminar August 2018 Agenda! Part

More information

Process groups and message ordering

Process groups and message ordering Process groups and message ordering If processes belong to groups, certain algorithms can be used that depend on group properties membership create ( name ), kill ( name ) join ( name, process ), leave

More information

Overlay Networks for Multimedia Contents Distribution

Overlay Networks for Multimedia Contents Distribution Overlay Networks for Multimedia Contents Distribution Vittorio Palmisano vpalmisano@gmail.com 26 gennaio 2007 Outline 1 Mesh-based Multicast Networks 2 Tree-based Multicast Networks Overcast (Cisco, 2000)

More information

OPAX - An Open Peer-to-Peer Architecture for XML Message Exchange

OPAX - An Open Peer-to-Peer Architecture for XML Message Exchange OPAX - An Open Peer-to-Peer Architecture for XML Message Exchange Bernhard Schandl, University of Vienna bernhard.schandl@univie.ac.at Users wishing to find multimedia material about interesting events

More information

Data Replication CS 188 Distributed Systems February 3, 2015

Data Replication CS 188 Distributed Systems February 3, 2015 Data Replication CS 188 Distributed Systems February 3, 2015 Page 1 Some Other Possibilities What if the machines sharing files are portable and not always connected? What if the machines communicate across

More information

Recap. CSE 486/586 Distributed Systems Google Chubby Lock Service. Recap: First Requirement. Recap: Second Requirement. Recap: Strengthening P2

Recap. CSE 486/586 Distributed Systems Google Chubby Lock Service. Recap: First Requirement. Recap: Second Requirement. Recap: Strengthening P2 Recap CSE 486/586 Distributed Systems Google Chubby Lock Service Steve Ko Computer Sciences and Engineering University at Buffalo Paxos is a consensus algorithm. Proposers? Acceptors? Learners? A proposer

More information

CSC458 Lecture 6. Administrivia. Inter-domain Routing IP Addressing. Midterm will Cover Following Topics (2) Midterm will Cover Following Topics

CSC458 Lecture 6. Administrivia. Inter-domain Routing IP Addressing. Midterm will Cover Following Topics (2) Midterm will Cover Following Topics CSC458 Lecture 6 Inter-domain Routing IP Addressing Administrivia Homework: #2 due today #3 out today, due in two weeks (same date as midterm) No lecture next week Reading Week Midterm in two weeks 60

More information

Today. Architectural Styles

Today. Architectural Styles Today Architectures for distributed systems (Chapter 2) Centralized, decentralized, hybrid Middleware Self-managing systems Lecture 2, page 1 Architectural Styles Important styles of architecture for distributed

More information

Project Midterms: March 22 nd : No Extensions

Project Midterms: March 22 nd : No Extensions Project Midterms: March 22 nd : No Extensions Team Presentations 10 minute presentations by each team member Demo of Gateway System Design What choices did you make for state management, data storage,

More information

DYNAMO: AMAZON S HIGHLY AVAILABLE KEY-VALUE STORE. Presented by Byungjin Jun

DYNAMO: AMAZON S HIGHLY AVAILABLE KEY-VALUE STORE. Presented by Byungjin Jun DYNAMO: AMAZON S HIGHLY AVAILABLE KEY-VALUE STORE Presented by Byungjin Jun 1 What is Dynamo for? Highly available key-value storages system Simple primary-key only interface Scalable and Reliable Tradeoff:

More information

Building Consistent Transactions with Inconsistent Replication

Building Consistent Transactions with Inconsistent Replication Building Consistent Transactions with Inconsistent Replication Irene Zhang, Naveen Kr. Sharma, Adriana Szekeres, Arvind Krishnamurthy, Dan R. K. Ports University of Washington Distributed storage systems

More information

ZooKeeper & Curator. CS 475, Spring 2018 Concurrent & Distributed Systems

ZooKeeper & Curator. CS 475, Spring 2018 Concurrent & Distributed Systems ZooKeeper & Curator CS 475, Spring 2018 Concurrent & Distributed Systems Review: Agreement In distributed systems, we have multiple nodes that need to all agree that some object has some state Examples:

More information

IXP Policy Considerations

IXP Policy Considerations IXP Policy Considerations IXP MODELS Institutional and Operational Models for IXPs A variety of institutional models have been adopted to operate IXPs. They fall into four categories: Nonprofit industry

More information

CPSC 426/526. P2P Lookup Service. Ennan Zhai. Computer Science Department Yale University

CPSC 426/526. P2P Lookup Service. Ennan Zhai. Computer Science Department Yale University CPSC / PP Lookup Service Ennan Zhai Computer Science Department Yale University Recall: Lec- Network basics: - OSI model and how Internet works - Socket APIs red PP network (Gnutella, KaZaA, etc.) UseNet

More information

The Scalability of Swarming Peer-to-Peer Content Delivery

The Scalability of Swarming Peer-to-Peer Content Delivery The Scalability of Swarming Peer-to-Peer Content Delivery Daniel Zappala Brigham Young University zappala@cs.byu.edu with Daniel Stutzbach Reza Rejaie University of Oregon Page 1 Motivation Small web sites

More information

Networking Recap Storage Intro. CSE-291 (Cloud Computing), Fall 2016 Gregory Kesden

Networking Recap Storage Intro. CSE-291 (Cloud Computing), Fall 2016 Gregory Kesden Networking Recap Storage Intro CSE-291 (Cloud Computing), Fall 2016 Gregory Kesden Networking Recap Storage Intro Long Haul/Global Networking Speed of light is limiting; Latency has a lower bound (.) Throughput

More information

How does a router know where to send a packet next?

How does a router know where to send a packet next? How does a router know where to send a packet next? The Problem Which path should packets take from A to B? A B R2 R R4 R3 C D 2 The Internet forwards packets hop-by-hop Data IP Address Next-hop A R B

More information

Application Editioning in WebSphere 8.5

Application Editioning in WebSphere 8.5 IBM Software Group Application Editioning in WebSphere 8.5 Robert Outlaw (routlaw@us.ibm.com) Christopher Hutcherson (cmhutche@us.ibm.com) WebSphere Intelligent Management Level 2 Support 2 May 2013 WebSphere

More information

Consistency and Replication

Consistency and Replication Topics to be covered Introduction Consistency and Replication Consistency Models Distribution Protocols Consistency Protocols 1 2 + Performance + Reliability Introduction Introduction Availability: proportion

More information

Fabric Development Update & Discussion. Binh Nguyen

Fabric Development Update & Discussion. Binh Nguyen Fabric Development Update & Discussion Binh Nguyen New Inspiration: Simple but Effective 2 And can make $ 3 Background: Architecture membership No SPoF No SPoT peer Endorser application SDK Keys 1 Endorse

More information

Recall: Sequential Consistency. CS 258 Parallel Computer Architecture Lecture 15. Sequential Consistency and Snoopy Protocols

Recall: Sequential Consistency. CS 258 Parallel Computer Architecture Lecture 15. Sequential Consistency and Snoopy Protocols CS 258 Parallel Computer Architecture Lecture 15 Sequential Consistency and Snoopy Protocols arch 17, 2008 Prof John D. Kubiatowicz http://www.cs.berkeley.edu/~kubitron/cs258 ecall: Sequential Consistency

More information

Distributed systems. Lecture 6: distributed transactions, elections, consensus and replication. Malte Schwarzkopf

Distributed systems. Lecture 6: distributed transactions, elections, consensus and replication. Malte Schwarzkopf Distributed systems Lecture 6: distributed transactions, elections, consensus and replication Malte Schwarzkopf Last time Saw how we can build ordered multicast Messages between processes in a group Need

More information

Scalable Control Plane Substrate. Sachin Ka3, John Ousterhout, Guru Parulkar, Marcos Aguilera, Curt Kolovson ONRC + ON.Lab + RAMCloud, VMWare

Scalable Control Plane Substrate. Sachin Ka3, John Ousterhout, Guru Parulkar, Marcos Aguilera, Curt Kolovson ONRC + ON.Lab + RAMCloud, VMWare Scalable Control Plane Substrate Sachin Ka3, John Ousterhout, Guru Parulkar, Marcos Aguilera, Curt Kolovson ONRC + ON.Lab + RAMCloud, VMWare MoEvaEon SeparaEon of control plane is a common trend: networks/systems

More information

ONLINE REGISTRATION: A STEP-BY-STEP GUIDE

ONLINE REGISTRATION: A STEP-BY-STEP GUIDE ONLINE REGISTRATION: A STEP-BY-STEP GUIDE We encourage all of our Walkers to register online at diabetes.org/stepout. It s quick. It s easy. And, you ll have the opportunity to take advantage of our online

More information

Managing Update Conflicts in Bayou. Lucy Youxuan Jiang, Hiwot Tadese Kassa

Managing Update Conflicts in Bayou. Lucy Youxuan Jiang, Hiwot Tadese Kassa Managing Update Conflicts in Bayou Lucy Youxuan Jiang, Hiwot Tadese Kassa Outline! Background + Motivation! Bayou Model Dependency checking for conflict detection Merge procedures for conflict resolution

More information

Distributed Systems. replication Johan Montelius ID2201. Distributed Systems ID2201

Distributed Systems. replication Johan Montelius ID2201. Distributed Systems ID2201 Distributed Systems ID2201 replication Johan Montelius 1 The problem The problem we have: servers might be unavailable The solution: keep duplicates at different servers 2 Building a fault-tolerant service

More information

Summarizing Existing Conversation For Newly Added Participants In Online Communication

Summarizing Existing Conversation For Newly Added Participants In Online Communication Technical Disclosure Commons Defensive Publications Series March 13, 2017 Summarizing Existing Conversation For Newly Added Participants In Online Communication Victor Carbune Sandro Feuz Thomas Deselaers

More information

MongoDB and Mysql: Which one is a better fit for me? Room 204-2:20PM-3:10PM

MongoDB and Mysql: Which one is a better fit for me? Room 204-2:20PM-3:10PM MongoDB and Mysql: Which one is a better fit for me? Room 204-2:20PM-3:10PM About us Adamo Tonete MongoDB Support Engineer Agustín Gallego MySQL Support Engineer Agenda What are MongoDB and MySQL; NoSQL

More information

Recap. CSE 486/586 Distributed Systems Google Chubby Lock Service. Paxos Phase 2. Paxos Phase 1. Google Chubby. Paxos Phase 3 C 1

Recap. CSE 486/586 Distributed Systems Google Chubby Lock Service. Paxos Phase 2. Paxos Phase 1. Google Chubby. Paxos Phase 3 C 1 Recap CSE 486/586 Distributed Systems Google Chubby Lock Service Steve Ko Computer Sciences and Engineering University at Buffalo Paxos is a consensus algorithm. Proposers? Acceptors? Learners? A proposer

More information

Goals. Facebook s Scaling Problem. Scaling Strategy. Facebook Three Layer Architecture. Workload. Memcache as a Service.

Goals. Facebook s Scaling Problem. Scaling Strategy. Facebook Three Layer Architecture. Workload. Memcache as a Service. Goals Memcache as a Service Tom Anderson Rapid application development - Speed of adding new features is paramount Scale Billions of users Every user on FB all the time Performance Low latency for every

More information

This presentation covers Gen Z Buffer operations. Buffer operations are used to move large quantities of data between components.

This presentation covers Gen Z Buffer operations. Buffer operations are used to move large quantities of data between components. This presentation covers Gen Z Buffer operations. Buffer operations are used to move large quantities of data between components. 1 2 Buffer operations are used to get data put or push up to 4 GiB of data

More information

Source-Route Bridging

Source-Route Bridging 25 CHAPTER Chapter Goals Describe when to use source-route bridging. Understand the difference between SRB and transparent bridging. Know the mechanism that end stations use to specify a source-route.

More information

Layering and Addressing CS551. Bill Cheng. Layer Encapsulation. OSI Model: 7 Protocol Layers.

Layering and Addressing CS551.  Bill Cheng. Layer Encapsulation. OSI Model: 7 Protocol Layers. Protocols CS551 Layering and Addressing Bill Cheng Set of rules governing communication between network elements (applications, hosts, routers) Protocols define: Format and order of messages Actions taken

More information