FRANCESCO CESARINI. presents ERLANG/OTP. Francesco Cesarini Erlang

Size: px
Start display at page:

Download "FRANCESCO CESARINI. presents ERLANG/OTP. Francesco Cesarini Erlang"

Transcription

1 FRANCESCO CESARINI presents Francesco Cesarini Erlang Solutions

2

3 WHAT IS SCALABILITY?

4

5 WHAT IS (MASSIVE) CONCURRENCY?

6

7 WHAT IS HIGH AVAILABILITY?

8

9 WHAT IS FAULT TOLERANCE?

10

11 WHAT IS DISTRIBUTION TRANSPARENCY?

12

13 Do you need a distributed system? Do you need a scalable system? Do you need a reliable system? Do you need a fault-tolerant system? Do you need a massively concurrent system? Do you need a distributed system? Do you need a scalable YES, PLEASE!!! system? Do you need a reliable system? Do you need a fault-tolerant system? Do distributed system? Do you need a scalable system? Do you need a reliable system? Do you need a fault-tolerant system? Do you need a massively

14 TO THE RESCUE

15 OPEN SOURCE CONCURRENCY-ORIENTED LIGHTWEIGHT PROCESSES ASYNCHRONOUS MESSAGE PASSING SHARE-NOTHING MODEL PROCESS LINKING / MONITORING SUPERVISION TREES AND RECOVERY STRATEGIES TRANSPARENT DISTRIBUTION MODEL SOFT-REAL TIME LET-IT-FAIL PHILOSOPHY HOT-CODE UPGRADES WHAT IS ERLANG

16 WELL, IN FACT YOU NEED MORE.

17 ERLANG IS JUST A PROGRAMMING LANGUAGE.

18 YOU NEED ARCHITECTURE PATTERNS. YOU NEED MIDDLEWARE. YOU NEED LIBRARIES. YOU NEED TOOLS.

19 YOU NEED OTP.

20 SOME TEXT

21 WHAT IS MIDDLEWARE?

22 DESIGN PATTERNS FAULT TOLERANCE DISTRIBUTION UPGRADES PACKAGING MIDDLEWARE

23 WHAT ARE LIBRARIES?

24 STORAGE O&M INTERFACES COMMUNICATION LIBRARIES

25 WHAT TOOLS?

26 DEVELOPMENT TEST FRAMEWORKS RELEASE & DEPLOYMENT DEBUGGING & MONITORING OTP TOOLS

27 OPEN SOURCE OTP IS PART OF THE ERLANG DISTRIBUTION

28 Less Code Servers OTP Less Bugs More Solid Code More Tested Code More Free Time Finite State Machines Event Handlers Supervisors Applications

29 BEHAVIOURS

30 GENERIC BEHAVIOUR MODULE Server process SPECIFIC CALLBACK MODULE

31 Less Code Servers OTP Less Bugs More Solid Code More Tested Code More Free Time Finite State Machines Event Handlers Supervisors Applications

32 Client {request, Pid, Message} {reply, Reply} Server call(name, Message) -> Name! {request, self(), Message}, receive {reply, Reply} -> Reply end. reply(pid, Reply) -> Pid! {reply, Reply}.

33 Client {request, {request, {Ref, Pid, self()}, Message} Message} {reply, Ref, Reply} Reply} Server {reply,???, Reply} Reply} Server 2 call(name, Msg) -> Ref = make_ref(), Name! {request, {Ref, self()}, Msg}, receive {reply, Ref, Reply} -> Reply end. reply({ref, Pid}, Reply) -> Pid! {reply, Ref, Reply}.

34 {request, {Ref, PidA}, Msg} PidA PidB call(name, Msg) -> Ref = erlang:monitor(process, Name), Name! {request, {Ref, self()}, Msg}, receive! {reply, Ref, Reply} ->! erlang:demonitor(ref),! Reply;! {'DOWN', Ref, process, _Name, _Reason} ->! {error, no_proc} end.

35 PidA {'DOWN', Ref, process, PidB, Reason} {request, {Ref, PidA}, Msg} {reply, Ref, Reply} PidB call(name, Msg) -> Ref = erlang:monitor(process, Name), Name! {request, {Ref, self()}, Msg}, receive! {reply, Ref, Reply} ->! erlang:demonitor(ref, [flush]),! Reply;! {'DOWN', Ref, process, _Name, _Reason} ->! {error, no_proc} end.

36 TIMEOUTS DEADLOCKS TRACING MONITORING DISTRIBUTION BEHAVIOURS

37 Your Heading

38 convert(day) -> case Day of monday -> 1; tuesday -> 2; wednesday -> 3; thursday -> 4; friday -> 5; saturday -> 6; sunday -> 7; Other -> {error, unknown_day} end. Let It Fail

39 convert(day) -> case Day of monday -> 1; tuesday -> 2; wednesday -> 3; thursday -> 4; friday -> 5; saturday -> 6; sunday -> 7 Let It Fail end.

40 ISOLATE THE ERROR!

41 PidA PROPAGATING EXIT SIGNALS {'EXIT', PidA, Reason} {'EXIT', PidB, Reason} PidB Exit Signals PidC

42 PidA TRAPPING AN EXIT SIGNAL {'EXIT', PidA, Reason} PidB Trap Exit PidC

43 PidA Supervisor PidB Supervisors Workers Application PidC

44 Release Mongoose IM folsom lager Releases snmp mnesia stdlib SASL kernel ERTS

45 AUTOMATIC TAKEOVER AND FAILOVER

46 {myapp, 2000, N1 N2 N3 Application Master Application dies Application Masters on failover nodes

47 {myapp, 2000, N2 N3 Application is restarted on dies

48 {myapp, 2000, N1 N3 comes back up Application is restarted on

49 {myapp, 2000, N1 N3 N1 takes over N3

50 RELEASE STATEMENT OF AIMS To scale the radical concurrency-oriented programming paradigm to build reliable general-purpose software, such as serverbased systems, on massively parallel machines (10^5 cores).

51 Erlang VM Scheduler #1 Scheduler #2 Scheduler #N run queue run queue run queue migration logic The Runtime Queues

52

53 LIMITATIONS ARE PRESENT AT THREE LEVELS WP4 Scalable Infrastructure WP3 SD Erlang Language WP2 Virtual Machine WP5 Tools WP6 Case Studies

54 VM LANGUAGE INFRASTRUCTURE PUSH THE RESPONSIBILITY FOR SCALABILITY FROM THE PROGRAMMER TO THE VM ANALYZE PERFORMANCE AND SCALABILITY IDENTIFY BOTTLENECKS AND PRIORITIZE CHANGES AND EXTENSIONS TACKLE WELL-KNOWN SCALABILITY ISSUES ETS TABLES (SHARED GLOBAL DATA STRUCTURE) MESSAGE PASSING, COPYING AND FREQUENTLY COMMUNICATING PROCESSES

55 VM LANGUAGE INFRASTRUCTURE TWO MAJOR ISSUES FULLY CONNECTED CLUSTERS EXPLICIT PROCESS PLACEMENT SCALABLE DISTRIBUTED (SD) ERLANG NODES GROUPING NON-TRANSITIVE CONNECTIONS IMPLICIT PROCESS PLACEMENT PART OF THE STANDARD ERLANG/OTP PACKAGE NEW CONCEPTS INTRODUCED LOCALITY, AFFINITY AND DISTANCE

56 VM LANGUAGE INFRASTRUCTURE Wombat O&M MIDDLEWARE LAYER SET OF ERLANG APPLICATIONS CREATE AND MANAGE CLUSTERS OF (HETEROGENEOUS) ERLANG NODES API TO MONITOR AND CONTROL ERLANG DISTRIBUTED SYSTEMS EXISTING TRACING/LOGGING/DEBUGGING TOOLS PLUGGABLE BROKER LAYER BETWEEN USERS AND CLOUD PROVIDERS AUTO-SCALING... AND MUCH MORE

57 CONCLUSIONS

58 Do you need a distributed system? Do you need a scalable system? Do you need a reliable system? Do you need a fault-tolerant system? Do you need a massively concurrent system? Do you need a distributed system? Do you need a scalable USE ERLANG system? Do you need a reliable system? Do you need a fault-tolerant system? Do distributed system? Do you need a scalable system? Do you need a reliable system? Do you need a fault-tolerant system? Do you need a massively

59 Do you need a distributed system? Do you need a scalable system? Do you need a reliable system? Do you need a fault-tolerant system? Do you need a massively concurrent system? Do you need a distributed system? Do you need a scalable USE ERLANG/OTP system? Do you need a reliable system? Do you need a fault-tolerant system? Do distributed system? Do you need a scalable system? Do you need a reliable system? Do you need a fault-tolerant system? Do you need a massively

60

Thinking in a Highly Concurrent, Mostly-functional Language

Thinking in a Highly Concurrent, Mostly-functional Language Thinking in a Highly Concurrent, Mostly-functional Language Build Stuff Vilnius, November 19th 2014 Francesco Cesarini Founder & Technical Director @francescoc francesco@erlang-solutions.com Thinking in

More information

2015 Erlang Solutions Ltd

2015 Erlang Solutions Ltd Thinking in a Highly Concurrent, Mostly-functional Language GOTO Berlin Berlin, December 4 th 2015 Francesco Cesarini Founder & Technical Director @francescoc francesco@erlang-solutions.com Thinking in

More information

An Introduction to Erlang

An Introduction to Erlang Erlang Solutions Ltd An Introduction to Erlang From behind the trenches GOTO Copenhagen May 13 th, 2011 Francesco Cesarini Founder, Technical Director @FrancescoC francesco@erlang-solutions.com So Here

More information

An Introduction to Erlang

An Introduction to Erlang Erlang Solutions Ltd An Introduction to Erlang From behind the trenches GOTO Amsterdam Amsterdam, May 25 th 2012 Francesco Cesarini Founder, Technical Director @FrancescoC francesco@erlang-solutions.com

More information

An Introduction to Erlang

An Introduction to Erlang Erlang Solutions Ltd An Introduction to Erlang From behind the trenches Erlang Factory Lite Zurich, April 23 rd 2012 Francesco Cesarini Founder, Technical Director @FrancescoC francesco@erlang-solutions.com

More information

Francesco Cesarini. Concurrency + Distribution = Availability + Scalability.

Francesco Cesarini. Concurrency + Distribution = Availability + Scalability. Concurrency + Distribution = Availability + Scalability Francesco Cesarini francesco@erlang-solutions.com www.erlang-solutions.com @francescoc Chapter 13 Ch 13: Node Architecture Chapter 13 Ch 13: Distributed

More information

The Actor Model applied to the Raspberry Pi and the Embedded Domain. Omer

The Actor Model applied to the Raspberry Pi and the Embedded Domain. Omer The Actor Model applied to the Raspberry Pi and the Embedded Domain Omer Kilic @OmerK omer@erlang-solutions.com Agenda Current state of Embedded Systems Overview of the Actor Model Erlang Embedded Project

More information

The Actor Model applied to the Raspberry Pi and the Embedded Domain. The Erlang Embedded Project. Omer

The Actor Model applied to the Raspberry Pi and the Embedded Domain. The Erlang Embedded Project. Omer The Actor Model applied to the Raspberry Pi and the Embedded Domain. The Erlang Embedded Project Omer Kilic @OmerK omer@erlang-solutions.com Outline Current state of Embedded Systems Overview of Erlang

More information

Erlang. Joe Armstrong

Erlang. Joe Armstrong Erlang Joe Armstrong Though OOP came from many motivations, two were central. The large scale one was to find a better module scheme for complex systems involving hiding of details, and the small scale

More information

The Golden Trinity of Erlang How Something Simple Has Real Business Value

The Golden Trinity of Erlang How Something Simple Has Real Business Value The Golden Trinity of Erlang How Something Simple Has Real Business Value Torben Hoffmann CTO, Erlang Solutions torben.hoffmann@erlang-solutions.com @LeHoff Why this talk? Why this talk? Introduce The

More information

Introduction. version 5.0

Introduction. version 5.0 Introduction version 5.0 Typeset in L A TEX from SGML source using the DOCBUILDER 3.0 Document System. Contents 1 Introduction 1 1.1 Introduction.......................................... 2 Erlang and

More information

Erlang. Joe Armstrong.

Erlang. Joe Armstrong. Erlang Joe Armstrong joe.armstrong@ericsson.com 1 Who is Joe? Inventor of Erlang, UBF, Open Floppy Grid Chief designer of OTP Founder of the company Bluetail Currently Software Architect Ericsson Current

More information

Erlang: distributed programming

Erlang: distributed programming Erlang: distributed May 11, 2012 1 / 21 Fault tolerance in Erlang links, exit signals, system process in Erlang OTP Open Telecom Platform 2 / 21 General idea Links Exit signals System processes Summary

More information

Styling your Architecture in an Evolving Concurrent World

Styling your Architecture in an Evolving Concurrent World Styling your Architecture in an Evolving Concurrent World Erlang Training and Consulting Ltd JAOO Aarhus, October 5 th, 2009 Francesco Cesarini francesco@erlang-consulting.com @FrancescoC Erlang Forces

More information

ERLANG EVOLVES FOR MULTI-CORE AND CLOUD ENVIRONMENTS

ERLANG EVOLVES FOR MULTI-CORE AND CLOUD ENVIRONMENTS ERLANG EVOLVES FOR MULTI-CORE AND CLOUD ENVIRONMENTS Torben Hoffmann Erlang Solutions Ltd. @LeHoff http://musings-of-an-erlang-priest.blogspot.dk/ 1 Agenda Erlang fundamentals Challenges 2 2 Warning 1:

More information

Erlang 101. Google Doc

Erlang 101. Google Doc Erlang 101 Google Doc Erlang? with buzzwords Erlang is a functional concurrency-oriented language with extremely low-weight userspace "processes", share-nothing messagepassing semantics, built-in distribution,

More information

concurrent programming XXL

concurrent programming XXL concurrent programming XXL Industrial Use of Erlang Introduction Karol Ostrovský (karol.ostrovsky@gmail.com) Motivation Machine Ericsson Blade System 3 sub-racks 14 blades 2 routers 12 compute nodes 6

More information

WOMBATOAM OPERATIONS & MAINTENANCE FOR ERLANG & ELIXIR SYSTEMS

WOMBATOAM OPERATIONS & MAINTENANCE FOR ERLANG & ELIXIR SYSTEMS version 3.0.0 3.0.0 IMPROVEMENTS OVER 2.0.0 MONITORING ++ Extensive dashboard redesign with a new, more intuitive user interface using GridStack. ++ Improved Mnesia netsplit service to detect and fix partitions

More information

WOMBATOAM OPERATIONS & MAINTENANCE FOR ERLANG & ELIXIR SYSTEMS

WOMBATOAM OPERATIONS & MAINTENANCE FOR ERLANG & ELIXIR SYSTEMS version 3.0.0 3.0.0 IMPROVEMENTS OVER 2.0.0 MONITORING ++ Extensive dashboard redesign with a new, more intuitive user interface using GridStack. ++ Improved Mnesia netsplit service to detect and fix partitions

More information

Windows Azure Services - At Different Levels

Windows Azure Services - At Different Levels Windows Azure Windows Azure Services - At Different Levels SaaS eg : MS Office 365 Paas eg : Azure SQL Database, Azure websites, Azure Content Delivery Network (CDN), Azure BizTalk Services, and Azure

More information

Adaptive Cluster Computing using JavaSpaces

Adaptive Cluster Computing using JavaSpaces Adaptive Cluster Computing using JavaSpaces Jyoti Batheja and Manish Parashar The Applied Software Systems Lab. ECE Department, Rutgers University Outline Background Introduction Related Work Summary of

More information

Programming Language Impact on the Development of Distributed Systems

Programming Language Impact on the Development of Distributed Systems Programming Language Impact on the Development of Distributed Systems Steve Vinoski Architect, Basho Technologies Cambridge, MA USA vinoski@ieee.org @stevevinoski http://steve.vinoski.net/ Co-Authors Debasish

More information

All you need is fun. Cons T Åhs Keeper of The Code

All you need is fun. Cons T Åhs Keeper of The Code All you need is fun Cons T Åhs Keeper of The Code cons@klarna.com Cons T Åhs Keeper of The Code at klarna Architecture - The Big Picture Development - getting ideas to work Code Quality - care about the

More information

Cloud Programming James Larus Microsoft Research. July 13, 2010

Cloud Programming James Larus Microsoft Research. July 13, 2010 Cloud Programming James Larus Microsoft Research July 13, 2010 New Programming Model, New Problems (and some old, unsolved ones) Concurrency Parallelism Message passing Distribution High availability Performance

More information

Experiments in OTP-Compliant Dataflow Programming

Experiments in OTP-Compliant Dataflow Programming Experiments in OTP-Compliant Dataflow Programming Introducing Erlang Services Platform (Erlang/SP) San Francisco Erlang Factory, March 21, 2013 Jay Nelson Twitter: @duomark Email: Jay@duomark.com Manycore

More information

Designing Database Solutions for Microsoft SQL Server (465)

Designing Database Solutions for Microsoft SQL Server (465) Designing Database Solutions for Microsoft SQL Server (465) Design a database structure Design for business requirements Translate business needs to data structures; de-normalize a database by using SQL

More information

The Actor Model, Part Two. CSCI 5828: Foundations of Software Engineering Lecture 18 10/23/2014

The Actor Model, Part Two. CSCI 5828: Foundations of Software Engineering Lecture 18 10/23/2014 The Actor Model, Part Two CSCI 5828: Foundations of Software Engineering Lecture 18 10/23/2014 1 Goals Cover the material presented in Chapter 5, of our concurrency textbook In particular, the material

More information

<Urban.Boquist. com> Rev PA

<Urban.Boquist. com> Rev PA FP in industry - Erlang Urban Boquist Ericsson AB com> 1 Outline Who Am I Mobile Telecommunications Networks Packet Core Network GPRS & SGSN Use of Erlang in SGSN SGSN Design

More information

Erlang. Functional Concurrent Distributed Soft real-time OTP (fault-tolerance, hot code update ) Open. Check the source code of generic behaviours

Erlang. Functional Concurrent Distributed Soft real-time OTP (fault-tolerance, hot code update ) Open. Check the source code of generic behaviours Lecture 9 Erlang Erlang Functional Concurrent Distributed Soft real-time OTP (fault-tolerance, hot code update ) Open Check the source code of generic behaviours 2 Functional Haskell call-by-need (lazy)

More information

Scaling Erlang to 10,000 cores.!!! Simon Thompson, University of Kent

Scaling Erlang to 10,000 cores.!!! Simon Thompson, University of Kent Scaling Erlang to 10,000 cores!!! Simon Thompson, University of Kent Multicore and many-core The inexorable rise in core numbers growing exponentially just as processors used to.! These are becoming the

More information

Kent Academic Repository

Kent Academic Repository Kent Academic Repository Full text document (pdf) Citation for published version Cesarini, Francesco and Thompson, Simon (2010) Erlang Behaviours: Programming With Process Design Patterns. In: Central

More information

Scalable ejabberd. Konstantin Tcepliaev. Moscow Erlang Factory Lite June 2012

Scalable ejabberd. Konstantin Tcepliaev. Moscow Erlang Factory Lite June 2012 Scalable ejabberd Konstantin Tcepliaev Moscow Erlang Factory Lite June 2012 ejabberd XMPP (previously known as Jabber) IM server Erlang/OTP Mnesia for temporary data (sessions, routes, etc.) Mnesia or

More information

Erlang: An Overview. Part 2 Concurrency and Distribution. Thanks to Richard Carlsson for most of the slides in this part

Erlang: An Overview. Part 2 Concurrency and Distribution. Thanks to Richard Carlsson for most of the slides in this part Erlang: An Overview Part 2 Concurrency and Distribution Thanks to Richard Carlsson for most of the slides in this part Processes P1 fib(0) -> 1; fib(1) -> 1; fib(n) when N > 0 -> fib(n-1) + fib(n-2). Whenever

More information

ERLANG TRACING HELLO! Lukas Larsson lukas.larsson@erlang-solutions.com @garazdawi Tracing today Overview Trace receivers Tracing events Meta tracing Match Specifications Sequence Tracing dtrace/systemtap

More information

MQ on z/os - Vivisection. Lyn Elkins IBM Advanced Technical Skills

MQ on z/os - Vivisection. Lyn Elkins IBM Advanced Technical Skills MQ on z/os - Vivisection Lyn Elkins elkinsc@us.ibm.com IBM Advanced Technical Skills Agenda One of these things is not like the other How are messages stored? Private Queues Shared Queues First line managers

More information

IBM WebSphere MQ for HP NonStop Update

IBM WebSphere MQ for HP NonStop Update IBM WebSphere MQ for HP NonStop Update Gerry Reilly Development Director and CTO, IBM Messaging greilly@uk.ibm.com 5 th December 2013 2013 IBM Corporation Trademark Statement IBM, WebSphere and the IBM

More information

DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S. TANENBAUM MAARTEN VAN STEEN. Chapter 3 Processes

DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S. TANENBAUM MAARTEN VAN STEEN. Chapter 3 Processes DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S. TANENBAUM MAARTEN VAN STEEN Chapter 3 Processes Context Switching Processor context: The minimal collection of values stored in the

More information

Outline. INF3190:Distributed Systems - Examples. Last week: Definitions Transparencies Challenges&pitfalls Architecturalstyles

Outline. INF3190:Distributed Systems - Examples. Last week: Definitions Transparencies Challenges&pitfalls Architecturalstyles INF3190:Distributed Systems - Examples Thomas Plagemann & Roman Vitenberg Outline Last week: Definitions Transparencies Challenges&pitfalls Architecturalstyles Today: Examples Googel File System (Thomas)

More information

Application Monitor Application (APPMON)

Application Monitor Application (APPMON) Application Monitor Application (APPMON) version 1.0 Magnus Fröberg 1997-05-02 Typeset in L A TEX from SGML source using the DOCBUILDER 3.0 Document System. Contents 1 APPMON Reference Manual 1 1.1 appmon

More information

Introduction to Erlang. Franck Petit / Sebastien Tixeuil

Introduction to Erlang. Franck Petit / Sebastien Tixeuil Introduction to Erlang Franck Petit / Sebastien Tixeuil Firstname.Lastname@lip6.fr Hello World % starts a comment. ends a declaration Every function must be in a module one module per source file source

More information

Understanding and Evaluating Kubernetes. Haseeb Tariq Anubhavnidhi Archie Abhashkumar

Understanding and Evaluating Kubernetes. Haseeb Tariq Anubhavnidhi Archie Abhashkumar Understanding and Evaluating Kubernetes Haseeb Tariq Anubhavnidhi Archie Abhashkumar Agenda Overview of project Kubernetes background and overview Experiments Summary and Conclusion 1. Overview of Project

More information

Implementing Riak in Erlang: Benefits and Challenges

Implementing Riak in Erlang: Benefits and Challenges Implementing Riak in Erlang: Benefits and Challenges Steve Vinoski Basho Technologies Cambridge, MA USA http://basho.com @stevevinoski vinoski@ieee.org http://steve.vinoski.net/ Erlang Erlang Started in

More information

Achieving Scalability and High Availability for clustered Web Services using Apache Synapse. Ruwan Linton WSO2 Inc.

Achieving Scalability and High Availability for clustered Web Services using Apache Synapse. Ruwan Linton WSO2 Inc. Achieving Scalability and High Availability for clustered Web Services using Apache Synapse Ruwan Linton [ruwan@apache.org] WSO2 Inc. Contents Introduction Apache Synapse Web services clustering Scalability/Availability

More information

Akka: Simpler Concurrency, Scalability & Fault-tolerance through Actors. Jonas Bonér Viktor Klang

Akka: Simpler Concurrency, Scalability & Fault-tolerance through Actors. Jonas Bonér Viktor Klang Akka: Simpler Concurrency, Scalability & Fault-tolerance through Actors Jonas Bonér Viktor Klang We believe that... Writing correct concurrent applications is too hard Scaling out applications is too hard

More information

Robust Erlang (PFP Lecture 11) John Hughes

Robust Erlang (PFP Lecture 11) John Hughes Robust Erlang (PFP Lecture 11) John Hughes Genesis of Erlang Problem: telephony systems in the late 1980s Digital More and more complex Highly concurrent Hard to get right Approach: a group at Ericsson

More information

Beyond 1001 Dedicated Data Service Instances

Beyond 1001 Dedicated Data Service Instances Beyond 1001 Dedicated Data Service Instances Introduction The Challenge Given: Application platform based on Cloud Foundry to serve thousands of apps Application Runtime Many platform users - who don

More information

Scaling Up from 1000 to 10 Nodes

Scaling Up from 1000 to 10 Nodes Scaling Up from 1000 to 10 Nodes Anton Lavrik Alert Logic, Inc. 1 Alert Logic vs Twitter: daily numbers (2012) Alert Logic Twitter Events received 11.5B 0.5B Volume received 5TB 70GB Logs produced? 100TB

More information

Fault Tolerance. Goals: transparent: mask (i.e., completely recover from) all failures, or predictable: exhibit a well defined failure behavior

Fault Tolerance. Goals: transparent: mask (i.e., completely recover from) all failures, or predictable: exhibit a well defined failure behavior Fault Tolerance Causes of failure: process failure machine failure network failure Goals: transparent: mask (i.e., completely recover from) all failures, or predictable: exhibit a well defined failure

More information

Inside Broker How Broker Leverages the C++ Actor Framework (CAF)

Inside Broker How Broker Leverages the C++ Actor Framework (CAF) Inside Broker How Broker Leverages the C++ Actor Framework (CAF) Dominik Charousset inet RG, Department of Computer Science Hamburg University of Applied Sciences Bro4Pros, February 2017 1 What was Broker

More information

The Little Elixir & OTP Guidebook by Tan Wei Hao

The Little Elixir & OTP Guidebook by Tan Wei Hao SAMPLE CHAPTER The Little Elixir & OTP Guidebook by Tan Wei Hao Chapter 9 Copyright 2016 Manning Publications brief contents PART 1 GETTING STARTED WITH ELIXIR AND OTP...1 1 Introduction 3 2 A whirlwind

More information

MongooseIM - Messaging that Scales

MongooseIM - Messaging that Scales MongooseIM - Messaging that Scales Michał Ślaski What is MongooseIM? 2 What is MongooseIM? Instant messaging for Social Media, Gaming and Telecommunications 2 What is MongooseIM? Instant messaging for

More information

Distributed Systems Principles and Paradigms. Chapter 01: Introduction. Contents. Distributed System: Definition.

Distributed Systems Principles and Paradigms. Chapter 01: Introduction. Contents. Distributed System: Definition. Distributed Systems Principles and Paradigms Maarten van Steen VU Amsterdam, Dept. Computer Science Room R4.20, steen@cs.vu.nl Chapter 01: Version: February 21, 2011 1 / 26 Contents Chapter 01: 02: Architectures

More information

Erlang. Functional Concurrent Distributed Soft real-time OTP (fault-tolerance, hot code update ) Open. Check the source code of generic behaviours

Erlang. Functional Concurrent Distributed Soft real-time OTP (fault-tolerance, hot code update ) Open. Check the source code of generic behaviours Lecture 10 Erlang Erlang Functional Concurrent Distributed Soft real-time OTP (fault-tolerance, hot code update ) Open Check the source code of generic behaviours 2 Functional Haskell call-by-need (lazy)

More information

Distributed Systems COMP 212. Lecture 18 Othon Michail

Distributed Systems COMP 212. Lecture 18 Othon Michail Distributed Systems COMP 212 Lecture 18 Othon Michail Virtualisation & Cloud Computing 2/27 Protection rings It s all about protection rings in modern processors Hardware mechanism to protect data and

More information

Distributed Systems Principles and Paradigms. Chapter 01: Introduction

Distributed Systems Principles and Paradigms. Chapter 01: Introduction Distributed Systems Principles and Paradigms Maarten van Steen VU Amsterdam, Dept. Computer Science Room R4.20, steen@cs.vu.nl Chapter 01: Introduction Version: October 25, 2009 2 / 26 Contents Chapter

More information

Simplified Operating Instructions T105-C / T108-C / T106-C / T17B-C DISPLAYS DAY OR DAY BLOCK PROG 00:0000 PROG PROG ALWAYS OFF

Simplified Operating Instructions T105-C / T108-C / T106-C / T17B-C DISPLAYS DAY OR DAY BLOCK PROG 00:0000 PROG PROG ALWAYS OFF Button Operations Simplified Operating Instructions T105-C / T108-C / T106-C / T17B-C R - Reset (with pen or other pointed instrument). Y - Enters function setup. +/- Buttons to scroll through icons. Y

More information

Enters system mode. Example The following example creates a scheduler named maintenancesched and commits the transaction:

Enters system mode. Example The following example creates a scheduler named maintenancesched and commits the transaction: Deployment, on page 1 Deployment Creating a Schedule UCS-A /system # create scheduler Creates a scheduler and enters scheduler mode. UCS-A /system/scheduler # commit-buffer The following example creates

More information

AMQP and Beyond. Messaging by Extending RabbitMQ. Tony Garnock-Jones

AMQP and Beyond. Messaging by Extending RabbitMQ. Tony Garnock-Jones AMQP and Beyond Messaging by Extending RabbitMQ Tony Garnock-Jones Why messaging? Database is to Filesystem as Messaging is to Network Messaging abstracts away from the details of

More information

Chapter 1: Distributed Systems: What is a distributed system? Fall 2013

Chapter 1: Distributed Systems: What is a distributed system? Fall 2013 Chapter 1: Distributed Systems: What is a distributed system? Fall 2013 Course Goals and Content n Distributed systems and their: n Basic concepts n Main issues, problems, and solutions n Structured and

More information

Fault Tolerance Causes of failure: process failure machine failure network failure Goals: transparent: mask (i.e., completely recover from) all

Fault Tolerance Causes of failure: process failure machine failure network failure Goals: transparent: mask (i.e., completely recover from) all Fault Tolerance Causes of failure: process failure machine failure network failure Goals: transparent: mask (i.e., completely recover from) all failures or predictable: exhibit a well defined failure behavior

More information

Scripting with Luerl

Scripting with Luerl Scripting with Luerl Luerl Luerl is an implementation of standard Lua written in Erlang/OTP. Lua is a powerful, efficient, lightweight, embeddable scripting language common in games, machine learning,

More information

Building loosely coupled and scalable systems using Event-Driven Architecture. Jonas Bonér Patrik Nordwall Andreas Källberg

Building loosely coupled and scalable systems using Event-Driven Architecture. Jonas Bonér Patrik Nordwall Andreas Källberg Building loosely coupled and scalable systems using Event-Driven Architecture Jonas Bonér Patrik Nordwall Andreas Källberg Why is EDA Important for Scalability? What building blocks does EDA consists of?

More information

MQ High Availability and Disaster Recovery Implementation scenarios

MQ High Availability and Disaster Recovery Implementation scenarios MQ High Availability and Disaster Recovery Implementation scenarios Sandeep Chellingi Head of Hybrid Cloud Integration Prolifics Agenda MQ Availability Message Availability Service Availability HA vs DR

More information

The vsphere 6.0 Advantages Over Hyper- V

The vsphere 6.0 Advantages Over Hyper- V The Advantages Over Hyper- V The most trusted and complete virtualization platform SDDC Competitive Marketing 2015 Q2 VMware.com/go/PartnerCompete 2015 VMware Inc. All rights reserved. v3b The Most Trusted

More information

02 - Distributed Systems

02 - Distributed Systems 02 - Distributed Systems Definition Coulouris 1 (Dis)advantages Coulouris 2 Challenges Saltzer_84.pdf Models Physical Architectural Fundamental 2/60 Definition Distributed Systems Distributed System is

More information

DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S. TANENBAUM MAARTEN VAN STEEN. Chapter 1. Introduction

DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S. TANENBAUM MAARTEN VAN STEEN. Chapter 1. Introduction DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S. TANENBAUM MAARTEN VAN STEEN Chapter 1 Introduction Modified by: Dr. Ramzi Saifan Definition of a Distributed System (1) A distributed

More information

Massive IM Scalability using WebSockets Michał Ślaski

Massive IM Scalability using WebSockets Michał Ślaski Erlang Solutions Ltd. Massive IM Scalability using WebSockets Michał Ślaski What am I chatting about? 1999-2011 Erlang Solutions Ltd. 2 What am I chatting about? Chat features 1999-2011 Erlang Solutions

More information

High Availability for Enterprise Clouds: Oracle Solaris Cluster and OpenStack

High Availability for Enterprise Clouds: Oracle Solaris Cluster and OpenStack High Availability for Enterprise Clouds: Oracle Solaris Cluster and OpenStack Eve Kleinknecht Principal Product Manager Thorsten Früauf Principal Software Engineer November 18, 2015 Safe Harbor Statement

More information

Top five Docker performance tips

Top five Docker performance tips Top five Docker performance tips Top five Docker performance tips Table of Contents Introduction... 3 Tip 1: Design design applications as microservices... 5 Tip 2: Deployment deploy Docker components

More information

Above the Clouds: Introducing Akka. Jonas Bonér Scalable Solutions

Above the Clouds: Introducing Akka. Jonas Bonér Scalable Solutions Above the Clouds: Introducing Akka Jonas Bonér CEO @ Scalable Solutions Twitter: @jboner The problem It is way too hard to build: 1. correct highly concurrent systems 2. truly scalable systems 3. fault-tolerant

More information

CA464 Distributed Programming

CA464 Distributed Programming 1 / 25 CA464 Distributed Programming Lecturer: Martin Crane Office: L2.51 Phone: 8974 Email: martin.crane@computing.dcu.ie WWW: http://www.computing.dcu.ie/ mcrane Course Page: "/CA464NewUpdate Textbook

More information

VMware vsphere 4.0 The best platform for building cloud infrastructures

VMware vsphere 4.0 The best platform for building cloud infrastructures VMware vsphere 4.0 The best platform for building cloud infrastructures VMware Intelligence Community Team Rob Amos - Intelligence Programs Manager ramos@vmware.com (703) 209-6480 Harold Hinson - Intelligence

More information

3C05 - Advanced Software Engineering Thursday, April 29, 2004

3C05 - Advanced Software Engineering Thursday, April 29, 2004 Distributed Software Architecture Using Middleware Avtar Raikmo Overview Middleware What is middleware? Why do we need middleware? Types of middleware Distributed Software Architecture Business Object

More information

BUILDING A SCALABLE MOBILE GAME BACKEND IN ELIXIR. Petri Kero CTO / Ministry of Games

BUILDING A SCALABLE MOBILE GAME BACKEND IN ELIXIR. Petri Kero CTO / Ministry of Games BUILDING A SCALABLE MOBILE GAME BACKEND IN ELIXIR Petri Kero CTO / Ministry of Games MOBILE GAME BACKEND CHALLENGES Lots of concurrent users Complex interactions between players Persistent world with frequent

More information

MapReduce: Simplified Data Processing on Large Clusters 유연일민철기

MapReduce: Simplified Data Processing on Large Clusters 유연일민철기 MapReduce: Simplified Data Processing on Large Clusters 유연일민철기 Introduction MapReduce is a programming model and an associated implementation for processing and generating large data set with parallel,

More information

CloudI Integration Framework. Chicago Erlang User Group May 27, 2015

CloudI Integration Framework. Chicago Erlang User Group May 27, 2015 CloudI Integration Framework Chicago Erlang User Group May 27, 2015 Speaker Bio Bruce Kissinger is an Architect with Impact Software LLC. Linkedin: https://www.linkedin.com/pub/bruce-kissinger/1/6b1/38

More information

DESIGNING DATABASE SOLUTIONS FOR MICROSOFT SQL SERVER CERTIFICATION QUESTIONS AND STUDY GUIDE

DESIGNING DATABASE SOLUTIONS FOR MICROSOFT SQL SERVER CERTIFICATION QUESTIONS AND STUDY GUIDE 70-465 DESIGNING DATABASE SOLUTIONS FOR MICROSOFT SQL SERVER CERTIFICATION QUESTIONS AND STUDY GUIDE Designing Database Solutions for Microsoft SQL Server (70-465) WWW.ANALYTICSEXAM.COM Contents Designing

More information

Cloud-Native Applications. Copyright 2017 Pivotal Software, Inc. All rights Reserved. Version 1.0

Cloud-Native Applications. Copyright 2017 Pivotal Software, Inc. All rights Reserved. Version 1.0 Cloud-Native Applications Copyright 2017 Pivotal Software, Inc. All rights Reserved. Version 1.0 Cloud-Native Characteristics Lean Form a hypothesis, build just enough to validate or disprove it. Learn

More information

Distributed KIDS Labs 1

Distributed KIDS Labs 1 Distributed Databases @ KIDS Labs 1 Distributed Database System A distributed database system consists of loosely coupled sites that share no physical component Appears to user as a single system Database

More information

Distributed Systems Principles and Paradigms

Distributed Systems Principles and Paradigms Distributed Systems Principles and Paradigms Chapter 01 (version September 5, 2007) Maarten van Steen Vrije Universiteit Amsterdam, Faculty of Science Dept. Mathematics and Computer Science Room R4.20.

More information

Starting the System & Basic Erlang Exercises

Starting the System & Basic Erlang Exercises Starting the System & Basic Erlang Exercises These exercises will help you get accustomed with the Erlang development and run time environments. Once you have set up the Erlang mode for emacs, you will

More information

CHAPTER - 4 REMOTE COMMUNICATION

CHAPTER - 4 REMOTE COMMUNICATION CHAPTER - 4 REMOTE COMMUNICATION Topics Introduction to Remote Communication Remote Procedural Call Basics RPC Implementation RPC Communication Other RPC Issues Case Study: Sun RPC Remote invocation Basics

More information

Leveraging Azure Services for a Scalable Windows Remote Desktop Deployment

Leveraging Azure Services for a Scalable Windows Remote Desktop Deployment WEBINAR Leveraging Azure Services for a Scalable Windows Remote Desktop Deployment May 16 2018 About Me 18+ years in IT Blog at www.ciraltos.com, Twitter @ciraltos Work at Bowman and Brooke LLP as IT Infrastructure

More information

Akka: Simpler Concurrency, Scalability & Fault-tolerance through Actors. Jonas Bonér Scalable Solutions

Akka: Simpler Concurrency, Scalability & Fault-tolerance through Actors. Jonas Bonér Scalable Solutions Akka: Simpler Concurrency, Scalability & Fault-tolerance through Actors Jonas Bonér Scalable Solutions jonas@jonasboner.com twitter: @jboner The problem It is way too hard to build: 1. correct highly concurrent

More information

OpenSAF More than HA. Jonas Arndt. HP - Telecom Architect OpenSAF - TCC

OpenSAF More than HA. Jonas Arndt. HP - Telecom Architect OpenSAF - TCC OpenSAF More than HA Jonas Arndt HP - Telecom Architect OpenSAF - TCC Presentation Layout OpenSAF Background OpenSAF Advantages Where are we and how did we get here? High Level Architecture Use Cases What

More information

a.k.a. Introducing the IBM MQ Appliance

a.k.a. Introducing the IBM MQ Appliance Understanding MQ Deployment Choices and Use Cases a.k.a. Introducing the IBM MQ Appliance Morag Hughson hughson@uk.ibm.com Session # 17060 Introduction Introducing IBM MQ Appliance The scalability and

More information

20 Years of Commercial Functional Programming

20 Years of Commercial Functional Programming 20 Years of Commercial Functional Programming Ulf Wiger Senior Software Architect Ericsson AB 2004-07-01 1 History of Erlang How to design SW for future telecoms systems? 1995: Several new projects 1998:

More information

CS 350 Winter 2011 Current Topics: Virtual Machines + Solid State Drives

CS 350 Winter 2011 Current Topics: Virtual Machines + Solid State Drives CS 350 Winter 2011 Current Topics: Virtual Machines + Solid State Drives Virtual Machines Resource Virtualization Separating the abstract view of computing resources from the implementation of these resources

More information

HP Designing and Implementing HP Enterprise Backup Solutions. Download Full Version :

HP Designing and Implementing HP Enterprise Backup Solutions. Download Full Version : HP HP0-771 Designing and Implementing HP Enterprise Backup Solutions Download Full Version : http://killexams.com/pass4sure/exam-detail/hp0-771 A. copy backup B. normal backup C. differential backup D.

More information

GFS Overview. Design goals/priorities Design for big-data workloads Huge files, mostly appends, concurrency, huge bandwidth Design for failures

GFS Overview. Design goals/priorities Design for big-data workloads Huge files, mostly appends, concurrency, huge bandwidth Design for failures GFS Overview Design goals/priorities Design for big-data workloads Huge files, mostly appends, concurrency, huge bandwidth Design for failures Interface: non-posix New op: record appends (atomicity matters,

More information

CS370 Operating Systems

CS370 Operating Systems CS370 Operating Systems Colorado State University Yashwant K Malaiya Fall 2016 Lecture 2 Slides based on Text by Silberschatz, Galvin, Gagne Various sources 1 1 2 System I/O System I/O (Chap 13) Central

More information

Programming Paradigms

Programming Paradigms PP 2017/18 Unit 15 Concurrent Programming with Erlang 1/32 Programming Paradigms Unit 15 Concurrent Programming with Erlang J. Gamper Free University of Bozen-Bolzano Faculty of Computer Science IDSE PP

More information

Chapter 5. The MapReduce Programming Model and Implementation

Chapter 5. The MapReduce Programming Model and Implementation Chapter 5. The MapReduce Programming Model and Implementation - Traditional computing: data-to-computing (send data to computing) * Data stored in separate repository * Data brought into system for computing

More information

Software Architecture Patterns

Software Architecture Patterns Software Architecture Patterns *based on a tutorial of Michael Stal Harald Gall University of Zurich http://seal.ifi.uzh.ch/ase www.infosys.tuwien.ac.at Overview Goal Basic architectural understanding

More information

OSE Copyright Ericsson AB. All Rights Reserved. OSE 1.0 June 23, 2014

OSE Copyright Ericsson AB. All Rights Reserved. OSE 1.0 June 23, 2014 OSE Copyright 2014-2014 Ericsson AB. All Rights Reserved. OSE 1.0 June 23, 2014 Copyright 2014-2014 Ericsson AB. All Rights Reserved. The contents of this file are subject to the Erlang Public License,

More information

VMware vsphere 5.5 Professional Bootcamp

VMware vsphere 5.5 Professional Bootcamp VMware vsphere 5.5 Professional Bootcamp Course Overview Course Objectives Cont. VMware vsphere 5.5 Professional Bootcamp is our most popular proprietary 5 Day course with more hands-on labs (100+) and

More information

02 - Distributed Systems

02 - Distributed Systems 02 - Distributed Systems Definition Coulouris 1 (Dis)advantages Coulouris 2 Challenges Saltzer_84.pdf Models Physical Architectural Fundamental 2/58 Definition Distributed Systems Distributed System is

More information

CS370 Operating Systems

CS370 Operating Systems CS370 Operating Systems Colorado State University Yashwant K Malaiya Spring 2018 Lecture 2 Slides based on Text by Silberschatz, Galvin, Gagne Various sources 1 1 2 What is an Operating System? What is

More information

Detecting and Visualising Process Relationships in Erlang

Detecting and Visualising Process Relationships in Erlang Procedia Computer Science Volume 29, 2014, Pages 1525 1534 ICCS 2014. 14th International Conference on Computational Science Melinda Tóth and István Bozó Eötvös Loránd University, Budapest, Hungary {tothmelinda,

More information

Distributed Object-Based Systems The WWW Architecture Web Services Handout 11 Part(a) EECS 591 Farnam Jahanian University of Michigan.

Distributed Object-Based Systems The WWW Architecture Web Services Handout 11 Part(a) EECS 591 Farnam Jahanian University of Michigan. Distributed Object-Based Systems The WWW Architecture Web Services Handout 11 Part(a) EECS 591 Farnam Jahanian University of Michigan Reading List Remote Object Invocation -- Tanenbaum Chapter 2.3 CORBA

More information