ERLANG EVOLVES FOR MULTI-CORE AND CLOUD ENVIRONMENTS
|
|
- Mercy Crystal Skinner
- 6 years ago
- Views:
Transcription
1 ERLANG EVOLVES FOR MULTI-CORE AND CLOUD ENVIRONMENTS Torben Hoffmann Erlang Solutions 1
2 Agenda Erlang fundamentals Challenges 2 2
3 Warning 1: The Truth I will do a few simplifications in order to get the main points across. 3
4 Warning 1: The Truth 3 I will do a few simplifications in order to get the main points across. 3
5 Warning 1: The Truth Will you tell the truth? 3 I will do a few simplifications in order to get the main points across. 3
6 Warning 1: The Truth Will you tell the truth? Yes 3 I will do a few simplifications in order to get the main points across. 3
7 Warning 1: The Truth Will you tell the truth? Yes The whole truth? 3 I will do a few simplifications in order to get the main points across. 3
8 Warning 1: The Truth Will you tell the truth? The whole truth? Yes No 3 I will do a few simplifications in order to get the main points across. 3
9 Warning 1: The Truth Will you tell the truth? The whole truth? Yes No So help you OTP? 3 I will do a few simplifications in order to get the main points across. 3
10 Warning 1: The Truth Will you tell the truth? The whole truth? So help you OTP? Yes No Yes 3 I will do a few simplifications in order to get the main points across. 3
11 Warning 2: Serious Love Ahead I love Erlang! 4 It was part of a major career shift and I have never looked back. Apologies if I get too intense. 4
12 Realities Of Software Development Time-to-market pressure Scaling successes Maintenance burden Utilisation of computing resources 5 5
13 What Could Be x productivity over C++/Java Seamless scaling on multicore Scaling nicely over machines Less code per feature The future is here today - it s called Erlang! 6
14 What Could Be x productivity over C++/Java Seamless scaling on multicore Scaling nicely over machines Less code per feature The future is here today - it s called Erlang! 6
15 What Could Be x productivity over C++/Java Seamless scaling on multicore Scaling nicely over machines Less code per feature The future is here today - it s called Erlang! 6
16 What Could Be x productivity over C++/Java Seamless scaling on multicore Scaling nicely over machines Less code per feature The future is here today - it s called Erlang! 6
17 What Could Be x productivity over C++/Java Seamless scaling on multicore Scaling nicely over machines Less code per feature The future is here today - it s called Erlang! 6
18 What Could Be... The future is here x productivity over C++/Java Seamless scaling on multicore Scaling nicely over machines Less code per feature The future is here today - it s called Erlang! 6
19 What Could Be... The future is here... The future is Erlang! 6 3x productivity over C++/Java Seamless scaling on multicore Scaling nicely over machines Less code per feature The future is here today - it s called Erlang! 6
20 Erlang s Original Requirements 7 7
21 Erlang s Original Requirements Large scale concurrency 7 7
22 Erlang s Original Requirements Large scale concurrency Soft real-time 7 7
23 Erlang s Original Requirements Large scale concurrency Soft real-time Distributed systems 7 7
24 Erlang s Original Requirements Large scale concurrency Soft real-time Distributed systems Hardware interaction 7 7
25 Erlang s Original Requirements Large scale concurrency Soft real-time Distributed systems Hardware interaction Very large software systems 7 7
26 Erlang s Original Requirements Large scale concurrency Soft real-time Distributed systems Hardware interaction Complex functionality Very large software systems 7 7
27 Erlang s Original Requirements Large scale concurrency Soft real-time Distributed systems Hardware interaction Complex functionality Very large software systems Continuous operation for many years 7 7
28 Erlang s Original Requirements Large scale concurrency Soft real-time Distributed systems Hardware interaction Complex functionality Very large software systems Continuous operation for many years Software maintenance on-the-fly 7 7
29 Erlang s Original Requirements Large scale concurrency Soft real-time Distributed systems Hardware interaction Complex functionality Very large software systems Continuous operation for many years Software maintenance on-the-fly High quality and reliability 7 7
30 Erlang s Original Requirements Large scale concurrency Soft real-time Distributed systems Hardware interaction Complex functionality Very large software systems Continuous operation for many years Software maintenance on-the-fly High quality and reliability Fault tolerance 7 7
31 Erlang s Original Requirements Large scale concurrency Soft real-time Distributed systems Hardware interaction Complex functionality Very large software systems Continuous operation for many years Software maintenance on-the-fly High quality and reliability Sounds familiar? Fault tolerance 7 7
32 Erlang s Original Requirements Large scale concurrency Soft real-time Distributed systems Hardware interaction Complex functionality Very large software systems Continuous operation for many years Software maintenance on-the-fly High quality and reliability Sounds familiar? Sounds good, right? Fault tolerance 7 7
33 General vs Domain Specific Small semantic gap Telecom 8 8
34 General vs Domain Specific Small semantic gap Telecom C++/Java 8 8
35 General vs Domain Specific Small semantic gap Telecom C++/Java 8 8
36 General vs Domain Specific Small semantic gap Telecom C++/Java Erlang 8 8
37 General vs Domain Specific Small semantic gap Telecom C++/Java Erlang 8 8
38 General vs Domain Specific Small semantic gap Telecom C++/Java Erlang Smaller gap = benefits! 8 8
39 Erlang s Sweet Spot GUI Middleware Erlang Coordination Control Drivers 9 Erlang was intended to deal with the control plane in telecom, which is all about orchestration of what goes on. GUI and low-level things are not what Erlang was created for - hence Erlang has good support for integration with other languages. Read the wonderful doctor thesis by Bjarne Däcker if you want to learn more: 9
40 Other Erlang Domains Messaging - XMPP et al - ejabberd, MongooseIM Webservers - Yaws, Chicago Boss Payment switches & soft switches - Vocalink, OpenFlow/LINC Distributed Databases - Riak, CouchDB, Scalaris 10 10
41 Other Erlang Domains Messaging - XMPP et al - ejabberd, MongooseIM Webservers - Yaws, Chicago Boss Payment switches & soft switches - Vocalink, OpenFlow/LINC Distributed Databases - Riak, CouchDB, Scalaris If the tool fits, you must select! Tech Mesh Conference 4-5 December London 10 10
42 To Share Or Not To Share 11 Death propagates in shared memory unless you do a ton of defensive programming. Due to the actor model with no shared memory it is custom in Erlang to do fail-fast programming. No shared memory allows you to fail fast when suitable. Erlang uses message passing between processes to exchange information. 11
43 To Share Or Not To Share Memory 11 Death propagates in shared memory unless you do a ton of defensive programming. Due to the actor model with no shared memory it is custom in Erlang to do fail-fast programming. No shared memory allows you to fail fast when suitable. Erlang uses message passing between processes to exchange information. 11
44 To Share Or Not To Share Memory P1 11 Death propagates in shared memory unless you do a ton of defensive programming. Due to the actor model with no shared memory it is custom in Erlang to do fail-fast programming. No shared memory allows you to fail fast when suitable. Erlang uses message passing between processes to exchange information. 11
45 To Share Or Not To Share Memory P1 P2 11 Death propagates in shared memory unless you do a ton of defensive programming. Due to the actor model with no shared memory it is custom in Erlang to do fail-fast programming. No shared memory allows you to fail fast when suitable. Erlang uses message passing between processes to exchange information. 11
46 To Share Or Not To Share Memory Corrupt P2 11 Death propagates in shared memory unless you do a ton of defensive programming. Due to the actor model with no shared memory it is custom in Erlang to do fail-fast programming. No shared memory allows you to fail fast when suitable. Erlang uses message passing between processes to exchange information. 11
47 To Share Or Not To Share Memory Corrupt 11 Death propagates in shared memory unless you do a ton of defensive programming. Due to the actor model with no shared memory it is custom in Erlang to do fail-fast programming. No shared memory allows you to fail fast when suitable. Erlang uses message passing between processes to exchange information. 11
48 To Share Or Not To Share Memory Corrupt Memory 11 Death propagates in shared memory unless you do a ton of defensive programming. Due to the actor model with no shared memory it is custom in Erlang to do fail-fast programming. No shared memory allows you to fail fast when suitable. Erlang uses message passing between processes to exchange information. 11
49 To Share Or Not To Share Memory Corrupt Memory P1 11 Death propagates in shared memory unless you do a ton of defensive programming. Due to the actor model with no shared memory it is custom in Erlang to do fail-fast programming. No shared memory allows you to fail fast when suitable. Erlang uses message passing between processes to exchange information. 11
50 To Share Or Not To Share Memory Corrupt Memory Memory P1 11 Death propagates in shared memory unless you do a ton of defensive programming. Due to the actor model with no shared memory it is custom in Erlang to do fail-fast programming. No shared memory allows you to fail fast when suitable. Erlang uses message passing between processes to exchange information. 11
51 To Share Or Not To Share Memory Corrupt Memory Memory P1 P2 11 Death propagates in shared memory unless you do a ton of defensive programming. Due to the actor model with no shared memory it is custom in Erlang to do fail-fast programming. No shared memory allows you to fail fast when suitable. Erlang uses message passing between processes to exchange information. 11
52 To Share Or Not To Share Memory Corrupt Corrupt Memory P2 11 Death propagates in shared memory unless you do a ton of defensive programming. Due to the actor model with no shared memory it is custom in Erlang to do fail-fast programming. No shared memory allows you to fail fast when suitable. Erlang uses message passing between processes to exchange information. 11
53 To Share Or Not To Share Memory Corrupt Memory P2 11 Death propagates in shared memory unless you do a ton of defensive programming. Due to the actor model with no shared memory it is custom in Erlang to do fail-fast programming. No shared memory allows you to fail fast when suitable. Erlang uses message passing between processes to exchange information. 11
54 Dealing With Failures P1 P2 link = die together P1 P2 monitor = notification of death 12 The ability to link processes and monitor them is the key to adopting fail-fast fully. Link & monitor works across machines! 12
55 Supervision Trees supervisor supervisor worker worker worker worker worker The OTP library is built on this principle 13 Robust systems does not happen by accident - even in Erlang! You have to think about the consequences of a worker process that fails and let the supervisor take appropriate actions. Using the OTP library s components makes it straightforward to implement the supervision tree, which has the added benefit that all things are started in the right order. 13
56 Distribution Over Cores Core Core Scheduler Scheduler Processes Processes 14 There is 1 scheduler per core. The VM tries to load balance across the available cores. Scales extremely well with the addition of extra cores - WITHOUT changing the programs! 14
57 Distribution Over Machines 15 Each instance of the Erlang runtime is called a node. There can be several nodes on one machine if you fancy that. Nodes detect when other nodes are not around any more - the programmer can then decide what to do. If you have the PID (Process Identifier) of a process you do not care which node it is on. You 15
58 Distribution Over Machines Node 15 Each instance of the Erlang runtime is called a node. There can be several nodes on one machine if you fancy that. Nodes detect when other nodes are not around any more - the programmer can then decide what to do. If you have the PID (Process Identifier) of a process you do not care which node it is on. You 15
59 Distribution Over Machines Node Node 15 Each instance of the Erlang runtime is called a node. There can be several nodes on one machine if you fancy that. Nodes detect when other nodes are not around any more - the programmer can then decide what to do. If you have the PID (Process Identifier) of a process you do not care which node it is on. You 15
60 Distribution Over Machines Node Node Node 15 Each instance of the Erlang runtime is called a node. There can be several nodes on one machine if you fancy that. Nodes detect when other nodes are not around any more - the programmer can then decide what to do. If you have the PID (Process Identifier) of a process you do not care which node it is on. You 15
61 Distribution Over Machines Node Node Node Node 15 Each instance of the Erlang runtime is called a node. There can be several nodes on one machine if you fancy that. Nodes detect when other nodes are not around any more - the programmer can then decide what to do. If you have the PID (Process Identifier) of a process you do not care which node it is on. You 15
62 Staying Alive... Learn New Moves! 16 As style changed a new group entered the top of the hip scale. For most it meant a serious restart or a stop altogether. 16
63 Staying Alive... Learn New Moves! 16 As style changed a new group entered the top of the hip scale. For most it meant a serious restart or a stop altogether. 16
64 Staying Alive... Learn New Moves! 16 As style changed a new group entered the top of the hip scale. For most it meant a serious restart or a stop altogether. 16
65 Staying Alive... Learn New Moves! 16 As style changed a new group entered the top of the hip scale. For most it meant a serious restart or a stop altogether. 16
66 Staying Alive Erlang Style 17 With Erlang you can survive upgrades without loosing service. Along with the code change signal you specify how the internal state of the process should be updated before continuing. 17
67 Staying Alive Erlang Style Process running v1 17 With Erlang you can survive upgrades without loosing service. Along with the code change signal you specify how the internal state of the process should be updated before continuing. 17
68 Staying Alive Erlang Style Process running v1 Code loaded: v2 17 With Erlang you can survive upgrades without loosing service. Along with the code change signal you specify how the internal state of the process should be updated before continuing. 17
69 Staying Alive Erlang Style Process running Code change signal v1 Code loaded: v2 17 With Erlang you can survive upgrades without loosing service. Along with the code change signal you specify how the internal state of the process should be updated before continuing. 17
70 Staying Alive Erlang Style Process running Code change signal v1 v2 Code loaded: v2 17 With Erlang you can survive upgrades without loosing service. Along with the code change signal you specify how the internal state of the process should be updated before continuing. 17
71 Challenges 18 18
72 How Will I Know If It Really Scales? Performance??? Cores/Machines 19 A Scalability Benchmark Suite for Erlang/OTP. A number of synthetic benchmarks plus real-world (dialyzer and scalaris). Extendable to test your own application. 19
73 How Will I Know If It Really Scales? Performance Bencherl Cores/Machines 19 A Scalability Benchmark Suite for Erlang/OTP. A number of synthetic benchmarks plus real-world (dialyzer and scalaris). Extendable to test your own application. 19
74 Who Is Doing What? Core Core Scheduler Scheduler Processes Processes 20 20
75 Who Is Doing What? Core Core Scheduler When is this Scheduler process running? Processes Processes 20 20
76 Who Is Doing What? Core Core Scheduler When is this Scheduler process running? Processes Processes Who started this guy? 20 20
77 Percept2 To The Rescue Active functions 21 Percept: Erlang Concurrency Profiling Tool, utilizes trace informations and profiler events to form a picture of the processes's and ports runnability. Percept2 is an extension of Percept (part of the OTP release). Extensions: # of schedulers active, active functions, process migration, message passing stats, inter-node communication 21
78 Memory Alloc Previously R12B-1 Scheduler N Scheduler 2 Scheduler 1 Thread N Thread 1 sl_alloc sl_alloc sl_alloc eheap_alloc eheap_alloc eheap_alloc binary_alloc binary_alloc binary_alloc ets_alloc ets_alloc ets_alloc mseg_alloc 22 One central memory allocator for all schedulers on the same machine 22
79 Memory Alloc Now R15B Scheduler N Scheduler 2 Scheduler 1 Thread N Thread 1 sl_alloc sl_alloc sl_alloc sl_alloc eheap_alloc eheap_alloc eheap_alloc eheap_alloc binary_alloc binary_alloc binary_alloc binary_alloc ets_alloc ets_alloc ets_alloc ets_alloc mseg_alloc mseg_alloc mseg_alloc mseg_alloc 23 One central memory allocator for all schedulers on the same machien 23
80 Upgrading Blocks Current Code Next Code Load code All schedulers blocked 24 24
81 Upgrade Without Blocking Current Code Next Code Last Code Load code Each scheduler does it when needed 25 Coming in R16! 25
82 Fully Connected 26 Erlang connects all nodes fully. So you get a lot of connections. 26
83 Scaling Over Machines Node Node Node Node Node 27 A fully connected system might not be right for every problem. s_groups allows you to create clusters of nodes. Nodes inside a cluster are fully connected. Connections between clusters can be arbitrary. 27
84 Managing Erlang Systems 28 Basic Erlang has the ability to go in and monitor what is going on in any node you can attach yourself to. But no tool exists to manage a big number of nodes in a coherent fashion. This is no different from any other language/technology! 28
85 Managing Erlang Systems Provision machines 28 Basic Erlang has the ability to go in and monitor what is going on in any node you can attach yourself to. But no tool exists to manage a big number of nodes in a coherent fashion. This is no different from any other language/technology! 28
86 Managing Erlang Systems Provision machines Deploy Erlang application 28 Basic Erlang has the ability to go in and monitor what is going on in any node you can attach yourself to. But no tool exists to manage a big number of nodes in a coherent fashion. This is no different from any other language/technology! 28
87 Managing Erlang Systems Provision machines Attach to node Deploy Erlang application 28 Basic Erlang has the ability to go in and monitor what is going on in any node you can attach yourself to. But no tool exists to manage a big number of nodes in a coherent fashion. This is no different from any other language/technology! 28
88 Managing Erlang Systems Provision machines Attach to node Dig out metrics Deploy Erlang application 28 Basic Erlang has the ability to go in and monitor what is going on in any node you can attach yourself to. But no tool exists to manage a big number of nodes in a coherent fashion. This is no different from any other language/technology! 28
89 Managing Erlang Systems Provision machines Attach to node Dig out metrics Deploy Erlang application Memory usage CPU load Process hierarchy 28 Basic Erlang has the ability to go in and monitor what is going on in any node you can attach yourself to. But no tool exists to manage a big number of nodes in a coherent fashion. This is no different from any other language/technology! 28
90 Managing Big Systems CCL /sɪˈsɪlɪ/ 29 Erlang Solutions are building a tool as part of the RELEASE project to manage and operate big Erlang systems. CCL = Cloud Computing Lace or Cloud Cuckoo Land depending on your mood. 29
91 Erlang And Parallelism 30 30
92 Erlang And Parallelism Created for 30 30
93 Erlang And Parallelism Created for - explicit concurrency 30 30
94 Erlang And Parallelism Created for - - explicit concurrency fault tollerance 30 30
95 Erlang And Parallelism Created for explicit concurrency fault tollerance highly concurrent systems 30 30
96 Erlang And Parallelism Created for explicit concurrency fault tollerance highly concurrent systems No direct support for 30 30
97 Erlang And Parallelism Created for explicit concurrency fault tollerance highly concurrent systems No direct support for - matrix multiplication 30 30
98 Erlang And Parallelism Created for explicit concurrency fault tollerance highly concurrent systems No direct support for - - matrix multiplication ray tracing 30 30
99 Erlang And Parallelism Created for explicit concurrency fault tollerance highly concurrent systems No direct support for matrix multiplication ray tracing coarse grained parallel problems 30 30
100 Intensional Parallelism 31 We are taking the good things from what has been learnt in the Haskell & Data-flow language communities and building a DSL which helps us leverage these types of parallel optimisations" 31
101 Intensional Parallelism Lucid like: demand-driven data computation Find short comings in the Erlang VM Variables are infinite streams of values 31 We are taking the good things from what has been learnt in the Haskell & Data-flow language communities and building a DSL which helps us leverage these types of parallel optimisations" 31
102 Intensional Parallelism Lucid like: demand-driven data computation Find short comings in the Erlang VM Variables are infinite streams of values running_avg where sum = first(input) fby sum + next(input); n = 1 fby n + 1; running_avg = sum / n; end; 31 We are taking the good things from what has been learnt in the Haskell & Data-flow language communities and building a DSL which helps us leverage these types of parallel optimisations" 31
103 Intensional Parallelism Lucid like: demand-driven data computation Find short comings in the Erlang VM Variables are infinite streams of values running_avg Black magic where sum = first(input) fby sum + next(input); n = 1 fby n + 1; running_avg = sum / n; end; 31 We are taking the good things from what has been learnt in the Haskell & Data-flow language communities and building a DSL which helps us leverage these types of parallel optimisations" 31
104 Going Forward 32 32
105 Going Forward Consider Erlang when the problem fits 32 32
106 Going Forward Consider Erlang when the problem fits More focus on right tool for the job 32 32
107 Going Forward Consider Erlang when the problem fits More focus on right tool for the job 32 32
The Golden Trinity of Erlang How Something Simple Has Real Business Value
The Golden Trinity of Erlang How Something Simple Has Real Business Value Torben Hoffmann CTO, Erlang Solutions torben.hoffmann@erlang-solutions.com @LeHoff Why this talk? Why this talk? Introduce The
More informationAn Introduction to Erlang
Erlang Solutions Ltd An Introduction to Erlang From behind the trenches GOTO Amsterdam Amsterdam, May 25 th 2012 Francesco Cesarini Founder, Technical Director @FrancescoC francesco@erlang-solutions.com
More informationScaling Erlang to 10,000 cores.!!! Simon Thompson, University of Kent
Scaling Erlang to 10,000 cores!!! Simon Thompson, University of Kent Multicore and many-core The inexorable rise in core numbers growing exponentially just as processors used to.! These are becoming the
More information2015 Erlang Solutions Ltd
Thinking in a Highly Concurrent, Mostly-functional Language GOTO Berlin Berlin, December 4 th 2015 Francesco Cesarini Founder & Technical Director @francescoc francesco@erlang-solutions.com Thinking in
More informationFRANCESCO CESARINI. presents ERLANG/OTP. Francesco Cesarini Erlang
FRANCESCO CESARINI presents Francesco Cesarini Erlang Solutions ERLANG/OTP @FrancescoC francesco@erlang-solutions.com www.erlang-solutions.com WHAT IS SCALABILITY? WHAT IS (MASSIVE) CONCURRENCY? WHAT
More informationThinking in a Highly Concurrent, Mostly-functional Language
Thinking in a Highly Concurrent, Mostly-functional Language Build Stuff Vilnius, November 19th 2014 Francesco Cesarini Founder & Technical Director @francescoc francesco@erlang-solutions.com Thinking in
More informationErlang. Joe Armstrong.
Erlang Joe Armstrong joe.armstrong@ericsson.com 1 Who is Joe? Inventor of Erlang, UBF, Open Floppy Grid Chief designer of OTP Founder of the company Bluetail Currently Software Architect Ericsson Current
More informationAn Introduction to Erlang
Erlang Solutions Ltd An Introduction to Erlang From behind the trenches GOTO Copenhagen May 13 th, 2011 Francesco Cesarini Founder, Technical Director @FrancescoC francesco@erlang-solutions.com So Here
More informationAn Introduction to Erlang
Erlang Solutions Ltd An Introduction to Erlang From behind the trenches Erlang Factory Lite Zurich, April 23 rd 2012 Francesco Cesarini Founder, Technical Director @FrancescoC francesco@erlang-solutions.com
More informationFunctional Programming In Real Life
Functional Programming In Real Life Dr. Erik Stenman CTO Kreditor Europe AB Creative Payment Solutions Introduction I will talk about KREDITOR, a company that bet it's future on Erlang, a functional programming
More informationconcurrent programming XXL
concurrent programming XXL Industrial Use of Erlang Introduction Karol Ostrovský (karol.ostrovsky@gmail.com) Motivation Machine Ericsson Blade System 3 sub-racks 14 blades 2 routers 12 compute nodes 6
More informationDesigning and debugging real-time distributed systems
Designing and debugging real-time distributed systems By Geoff Revill, RTI This article identifies the issues of real-time distributed system development and discusses how development platforms and tools
More informationCloud Programming James Larus Microsoft Research. July 13, 2010
Cloud Programming James Larus Microsoft Research July 13, 2010 New Programming Model, New Problems (and some old, unsolved ones) Concurrency Parallelism Message passing Distribution High availability Performance
More informationTaking a Virtual Machine Towards Many-Cores. Rickard Green - Patrik Nyblom -
Taking a Virtual Machine Towards Many-Cores Rickard Green - rickard@erlang.org Patrik Nyblom - pan@erlang.org What we all know by now Number of cores per processor AMD Opteron Intel Xeon Sparc Niagara
More informationCloudI Integration Framework. Chicago Erlang User Group May 27, 2015
CloudI Integration Framework Chicago Erlang User Group May 27, 2015 Speaker Bio Bruce Kissinger is an Architect with Impact Software LLC. Linkedin: https://www.linkedin.com/pub/bruce-kissinger/1/6b1/38
More informationAll you need is fun. Cons T Åhs Keeper of The Code
All you need is fun Cons T Åhs Keeper of The Code cons@klarna.com Cons T Åhs Keeper of The Code at klarna Architecture - The Big Picture Development - getting ideas to work Code Quality - care about the
More information<Urban.Boquist. com> Rev PA
FP in industry - Erlang Urban Boquist Ericsson AB com> 1 Outline Who Am I Mobile Telecommunications Networks Packet Core Network GPRS & SGSN Use of Erlang in SGSN SGSN Design
More informationSQL Azure as a Self- Managing Database Service: Lessons Learned and Challenges Ahead
SQL Azure as a Self- Managing Database Service: Lessons Learned and Challenges Ahead 1 Introduction Key features: -Shared nothing architecture -Log based replication -Support for full ACID properties -Consistency
More informationMessage Passing. Advanced Operating Systems Tutorial 7
Message Passing Advanced Operating Systems Tutorial 7 Tutorial Outline Review of Lectured Material Discussion: Erlang and message passing 2 Review of Lectured Material Message passing systems Limitations
More informationWOMBATOAM OPERATIONS & MAINTENANCE FOR ERLANG & ELIXIR SYSTEMS
version 3.0.0 3.0.0 IMPROVEMENTS OVER 2.0.0 MONITORING ++ Extensive dashboard redesign with a new, more intuitive user interface using GridStack. ++ Improved Mnesia netsplit service to detect and fix partitions
More informationWOMBATOAM OPERATIONS & MAINTENANCE FOR ERLANG & ELIXIR SYSTEMS
version 3.0.0 3.0.0 IMPROVEMENTS OVER 2.0.0 MONITORING ++ Extensive dashboard redesign with a new, more intuitive user interface using GridStack. ++ Improved Mnesia netsplit service to detect and fix partitions
More informationHiTune. Dataflow-Based Performance Analysis for Big Data Cloud
HiTune Dataflow-Based Performance Analysis for Big Data Cloud Jinquan (Jason) Dai, Jie Huang, Shengsheng Huang, Bo Huang, Yan Liu Intel Asia-Pacific Research and Development Ltd Shanghai, China, 200241
More informationDistributed Scheduling for the Sombrero Single Address Space Distributed Operating System
Distributed Scheduling for the Sombrero Single Address Space Distributed Operating System Donald S. Miller Department of Computer Science and Engineering Arizona State University Tempe, AZ, USA Alan C.
More informationLightweight Streaming-based Runtime for Cloud Computing. Shrideep Pallickara. Community Grids Lab, Indiana University
Lightweight Streaming-based Runtime for Cloud Computing granules Shrideep Pallickara Community Grids Lab, Indiana University A unique confluence of factors have driven the need for cloud computing DEMAND
More informationBEAMJIT: An LLVM based just-in-time compiler for Erlang. Frej Drejhammar
BEAMJIT: An LLVM based just-in-time compiler for Erlang Frej Drejhammar 140407 Who am I? Senior researcher at the Swedish Institute of Computer Science (SICS) working on programming languages,
More informationResources and Services Virtualization without Boundaries (ReSerVoir)
Resources and Services Virtualization without Boundaries (ReSerVoir) Benny Rochwerger April 14, 2008 IBM Labs in Haifa The Evolution of the Power Grid The Burden Iron Works Water Wheel http://w w w.rootsw
More informationDEMYSTIFYING BIG DATA WITH RIAK USE CASES. Martin Schneider Basho Technologies!
DEMYSTIFYING BIG DATA WITH RIAK USE CASES Martin Schneider Basho Technologies! Agenda Defining Big Data in Regards to Riak A Series of Trade-Offs Use Cases Q & A About Basho & Riak Basho Technologies is
More informationFrancesco Cesarini. Concurrency + Distribution = Availability + Scalability.
Concurrency + Distribution = Availability + Scalability Francesco Cesarini francesco@erlang-solutions.com www.erlang-solutions.com @francescoc Chapter 13 Ch 13: Node Architecture Chapter 13 Ch 13: Distributed
More informationAdaptive Cluster Computing using JavaSpaces
Adaptive Cluster Computing using JavaSpaces Jyoti Batheja and Manish Parashar The Applied Software Systems Lab. ECE Department, Rutgers University Outline Background Introduction Related Work Summary of
More informationDeveloping MapReduce Programs
Cloud Computing Developing MapReduce Programs Dell Zhang Birkbeck, University of London 2017/18 MapReduce Algorithm Design MapReduce: Recap Programmers must specify two functions: map (k, v) * Takes
More informationUsing the SDACK Architecture to Build a Big Data Product. Yu-hsin Yeh (Evans Ye) Apache Big Data NA 2016 Vancouver
Using the SDACK Architecture to Build a Big Data Product Yu-hsin Yeh (Evans Ye) Apache Big Data NA 2016 Vancouver Outline A Threat Analytic Big Data product The SDACK Architecture Akka Streams and data
More informationThe Actor Model applied to the Raspberry Pi and the Embedded Domain. Omer
The Actor Model applied to the Raspberry Pi and the Embedded Domain Omer Kilic @OmerK omer@erlang-solutions.com Agenda Current state of Embedded Systems Overview of the Actor Model Erlang Embedded Project
More informationSoftware Architecture
Software Architecture Architectural Design and Patterns. Standard Architectures. Dr. Philipp Leitner @xleitix University of Zurich, Switzerland software evolution & architecture lab Architecting, the planning
More informationMongooseIM - Messaging that Scales
MongooseIM - Messaging that Scales Michał Ślaski What is MongooseIM? 2 What is MongooseIM? Instant messaging for Social Media, Gaming and Telecommunications 2 What is MongooseIM? Instant messaging for
More informationErlang. Joe Armstrong
Erlang Joe Armstrong Though OOP came from many motivations, two were central. The large scale one was to find a better module scheme for complex systems involving hiding of details, and the small scale
More informationEmbedded Systems: Projects
December 2015 Embedded Systems: Projects Davide Zoni PhD email: davide.zoni@polimi.it webpage: home.dei.polimi.it/zoni Research Activities Interconnect: bus, NoC Simulation (component design, evaluation)
More informationA Holistic View of Telco Clouds
A Holistic View of Telco Clouds Cloud Computing in the Telecom environment, bridging the gap Miyazaki, 4 March 2012 (A workshop in conjunction with World Telecom Congress 2012) Authors: Lóránt Németh,
More informationCluster-Based Scalable Network Services
Cluster-Based Scalable Network Services Suhas Uppalapati INFT 803 Oct 05 1999 (Source : Fox, Gribble, Chawathe, and Brewer, SOSP, 1997) Requirements for SNS Incremental scalability and overflow growth
More informationSoft shadows. Steve Marschner Cornell University CS 569 Spring 2008, 21 February
Soft shadows Steve Marschner Cornell University CS 569 Spring 2008, 21 February Soft shadows are what we normally see in the real world. If you are near a bare halogen bulb, a stage spotlight, or other
More informationClaude TADONKI. MINES ParisTech PSL Research University Centre de Recherche Informatique
Got 2 seconds Sequential 84 seconds Expected 84/84 = 1 second!?! Got 25 seconds MINES ParisTech PSL Research University Centre de Recherche Informatique claude.tadonki@mines-paristech.fr Séminaire MATHEMATIQUES
More informationCourse Syllabus: Lifting Off into Space-Based Architecture with Magic xpi 4.x
Course Syllabus: Lifting Off into Space-Based Architecture with Magic xpi 4.x TABLE OF CONTENTS: 1 COURSE OBJECTIVES AND GOALS... 2 2 GENERAL COURSE DETAILS... 2 3 TECHNICAL MATERIAL... 4 4 LESSONS...
More informationWHITE PAPER NGINX An Open Source Platform of Choice for Enterprise Website Architectures
ASHNIK PTE LTD. White Paper WHITE PAPER NGINX An Open Source Platform of Choice for Enterprise Website Architectures Date: 10/12/2014 Company Name: Ashnik Pte Ltd. Singapore By: Sandeep Khuperkar, Director
More informationAWS Lambda: Event-driven Code in the Cloud
AWS Lambda: Event-driven Code in the Cloud Dean Bryen, Solutions Architect AWS Andrew Wheat, Senior Software Engineer - BBC April 15, 2015 London, UK 2015, Amazon Web Services, Inc. or its affiliates.
More informationGavin Payne Senior Consultant.
Gavin Payne Senior Consultant gavin@coeo.com Getting Started with SQL Server and Virtualisation Designing successful virtual database environments Monitoring a virtual database environments Summary The
More informationBuilding a government cloud Concepts and Solutions
Building a government cloud Concepts and Solutions Dr. Gabor Szentivanyi, ULX Open Source Consulting & Distribution Background Over 18 years of experience in enterprise grade open source Based in Budapest,
More informationChapter 5. The MapReduce Programming Model and Implementation
Chapter 5. The MapReduce Programming Model and Implementation - Traditional computing: data-to-computing (send data to computing) * Data stored in separate repository * Data brought into system for computing
More informationEUC 2015 MongooseIM The Right Tool for Scalable Messaging. Michał Piotrowski
EUC 2015 MongooseIM The Right Tool for Scalable Messaging Michał Piotrowski michal.piotrowski@erlang-solutions.com MongooseIM - The Right Tool for Scalable Messaging Agenda What is MongooseIM Motivation
More informationAbove the Clouds: Introducing Akka. Jonas Bonér Scalable Solutions
Above the Clouds: Introducing Akka Jonas Bonér CEO @ Scalable Solutions Twitter: @jboner The problem It is way too hard to build: 1. correct highly concurrent systems 2. truly scalable systems 3. fault-tolerant
More informationTHE VMTURBO CLOUD CONTROL PLANE
THE VMTURBO CLOUD CONTROL PLANE Software-Driven Control for the Software-Defined Data Center EXECUTIVE SUMMARY The Software-Defined Datacenter (SDDC) has the potential to extend the agility, operational
More informationArchitekturen für die Cloud
Architekturen für die Cloud Eberhard Wolff Architecture & Technology Manager adesso AG 08.06.11 What is Cloud? National Institute for Standards and Technology (NIST) Definition On-demand self-service >
More informationStreamBox: Modern Stream Processing on a Multicore Machine
StreamBox: Modern Stream Processing on a Multicore Machine Hongyu Miao and Heejin Park, Purdue ECE; Myeongjae Jeon and Gennady Pekhimenko, Microsoft Research; Kathryn S. McKinley, Google; Felix Xiaozhu
More informationIntroduction to Virtualization. From NDG In partnership with VMware IT Academy
Introduction to Virtualization From NDG In partnership with VMware IT Academy www.vmware.com/go/academy Why learn virtualization? Modern computing is more efficient due to virtualization Virtualization
More informationLinux multi-core scalability
Linux multi-core scalability Oct 2009 Andi Kleen Intel Corporation andi@firstfloor.org Overview Scalability theory Linux history Some common scalability trouble-spots Application workarounds Motivation
More informationParallelism. Parallel Hardware. Introduction to Computer Systems
Parallelism We have been discussing the abstractions and implementations that make up an individual computer system in considerable detail up to this point. Our model has been a largely sequential one,
More informationThe New Enterprise Network In The Era Of The Cloud. Rohit Mehra Director, Enterprise Communications Infrastructure IDC
The New Enterprise Network In The Era Of The Cloud Rohit Mehra Director, Enterprise Communications Infrastructure IDC Agenda 1. Dynamics of the Cloud Era 2. Market Landscape 3. Implications for the new
More informationSystematic Cooperation in P2P Grids
29th October 2008 Cyril Briquet Doctoral Dissertation in Computing Science Department of EE & CS (Montefiore Institute) University of Liège, Belgium Application class: Bags of Tasks Bag of Task = set of
More informationTITLE: PRE-REQUISITE THEORY. 1. Introduction to Hadoop. 2. Cluster. Implement sort algorithm and run it using HADOOP
TITLE: Implement sort algorithm and run it using HADOOP PRE-REQUISITE Preliminary knowledge of clusters and overview of Hadoop and its basic functionality. THEORY 1. Introduction to Hadoop The Apache Hadoop
More informationImplementing the Twelve-Factor App Methodology for Developing Cloud- Native Applications
Implementing the Twelve-Factor App Methodology for Developing Cloud- Native Applications By, Janakiram MSV Executive Summary Application development has gone through a fundamental shift in the recent past.
More informationPerformance Tools for Technical Computing
Christian Terboven terboven@rz.rwth-aachen.de Center for Computing and Communication RWTH Aachen University Intel Software Conference 2010 April 13th, Barcelona, Spain Agenda o Motivation and Methodology
More informationThe Next Generation of Extreme OLTP Processing with Oracle TimesTen
The Next Generation of Extreme OLTP Processing with TimesTen Tirthankar Lahiri Redwood Shores, California, USA Keywords: TimesTen, velocity scaleout elastic inmemory relational database cloud Introduction
More informationMinimizing the Risks of OpenStack Adoption
Minimizing the Risks of OpenStack Adoption White Paper Minimizing the Risks of OpenStack Adoption Introduction Over the last five years, OpenStack has become a solution of choice for enterprise private
More informationRobust Erlang (PFP Lecture 11) John Hughes
Robust Erlang (PFP Lecture 11) John Hughes Genesis of Erlang Problem: telephony systems in the late 1980s Digital More and more complex Highly concurrent Hard to get right Approach: a group at Ericsson
More informationConcurrency: what, why, how
Concurrency: what, why, how May 28, 2009 1 / 33 Lecture about everything and nothing Explain basic idea (pseudo) vs. Give reasons for using Present briefly different classifications approaches models and
More informationProgrammable Peer-to-Peer Systems
Programmable Peer-to-Peer Systems p. 1/18 Programmable Peer-to-Peer Systems Dimitris Vyzovitis vyzo@media.mit.edu Viral Communications MIT Media Laboratory Programmable Peer-to-Peer Systems p. 2/18 Overview
More informationHalf full or half empty? William Gropp Mathematics and Computer Science
Half full or half empty? William Gropp Mathematics and Computer Science www.mcs.anl.gov/~gropp MPI on Multicore Processors Work of Darius Buntinas and Guillaume Mercier 340 ns MPI ping/pong latency More
More informationMySQL As A Service. Operationalizing 19 Years of Infrastructure at GoDaddy
MySQL As A Service Operationalizing 19 Years of Infrastructure at GoDaddy WHOAMI Nathan Northcutt Senior Software Engineer MySQL DevOps ~10 years performance engineering & distributed data services. Email:
More informationHSA Foundation! Advanced Topics on Heterogeneous System Architectures. Politecnico di Milano! Seminar Room (Bld 20)! 15 December, 2017!
Advanced Topics on Heterogeneous System Architectures HSA Foundation! Politecnico di Milano! Seminar Room (Bld 20)! 15 December, 2017! Antonio R. Miele! Marco D. Santambrogio! Politecnico di Milano! 2
More informationTask Scheduling of Real- Time Media Processing with Hardware-Assisted Virtualization Heikki Holopainen
Task Scheduling of Real- Time Media Processing with Hardware-Assisted Virtualization Heikki Holopainen Aalto University School of Electrical Engineering Degree Programme in Communications Engineering Supervisor:
More informationCUDA GPGPU Workshop 2012
CUDA GPGPU Workshop 2012 Parallel Programming: C thread, Open MP, and Open MPI Presenter: Nasrin Sultana Wichita State University 07/10/2012 Parallel Programming: Open MP, MPI, Open MPI & CUDA Outline
More informationVMware - VMware vsphere: Install, Configure, Manage [V6.7]
VMware - VMware vsphere: Install, Configure, Manage [V6.7] Code: Length: URL: EDU-VSICM67 5 days View Online This five-day course features intensive hands-on training that focuses on installing, configuring,
More informationMission-Critical Databases in the Cloud. Oracle RAC in Microsoft Azure Enabled by FlashGrid Software.
Mission-Critical Databases in the Cloud. Oracle RAC in Microsoft Azure Enabled by FlashGrid Software. White Paper rev. 2017-10-16 2017 FlashGrid Inc. 1 www.flashgrid.io Abstract Ensuring high availability
More informationLecture Topics. Announcements. Today: Advanced Scheduling (Stallings, chapter ) Next: Deadlock (Stallings, chapter
Lecture Topics Today: Advanced Scheduling (Stallings, chapter 10.1-10.4) Next: Deadlock (Stallings, chapter 6.1-6.6) 1 Announcements Exam #2 returned today Self-Study Exercise #10 Project #8 (due 11/16)
More informationAssignment 5. Georgia Koloniari
Assignment 5 Georgia Koloniari 2. "Peer-to-Peer Computing" 1. What is the definition of a p2p system given by the authors in sec 1? Compare it with at least one of the definitions surveyed in the last
More informationCS370 Operating Systems
CS370 Operating Systems Colorado State University Yashwant K Malaiya Spring 2018 Lecture 2 Slides based on Text by Silberschatz, Galvin, Gagne Various sources 1 1 2 What is an Operating System? What is
More informationCache Coherence. CMU : Parallel Computer Architecture and Programming (Spring 2012)
Cache Coherence CMU 15-418: Parallel Computer Architecture and Programming (Spring 2012) Shared memory multi-processor Processors read and write to shared variables - More precisely: processors issues
More informationOVERVIEW OF DIFFERENT APPLICATION SERVER MODELS
OVERVIEW OF DIFFERENT APPLICATION SERVER MODELS Before you start Objectives: learn what is application server, what is thin-client and what is fat-client, and about different types of application server
More informationPBS PROFESSIONAL VS. MICROSOFT HPC PACK
PBS PROFESSIONAL VS. MICROSOFT HPC PACK On the Microsoft Windows Platform PBS Professional offers many features which are not supported by Microsoft HPC Pack. SOME OF THE IMPORTANT ADVANTAGES OF PBS PROFESSIONAL
More informationGustavo Alonso, ETH Zürich. Web services: Concepts, Architectures and Applications - Chapter 1 2
Chapter 1: Distributed Information Systems Gustavo Alonso Computer Science Department Swiss Federal Institute of Technology (ETHZ) alonso@inf.ethz.ch http://www.iks.inf.ethz.ch/ Contents - Chapter 1 Design
More informationSetting up Kubernetes with Day 2 in Mind. Angela Chin, Senior Software Engineer, Pivotal Urvashi Reddy, Senior Software Engineer, Pivotal
Setting up Kubernetes with Day 2 in Mind Angela Chin, Senior Software Engineer, Pivotal Urvashi Reddy, Senior Software Engineer, Pivotal About Us Angela Software Engineer @ Pivotal Based in Santa Monica,
More informationThe 7 deadly sins of cloud computing [2] Cloud-scale resource management [1]
The 7 deadly sins of [2] Cloud-scale resource management [1] University of California, Santa Cruz May 20, 2013 1 / 14 Deadly sins of of sin (n.) - common simplification or shortcut employed by ers; may
More informationOnline Course Evaluation. What we will do in the last week?
Online Course Evaluation Please fill in the online form The link will expire on April 30 (next Monday) So far 10 students have filled in the online form Thank you if you completed it. 1 What we will do
More informationGhaffari, Amir (2015) The scalability of reliable computation in Erlang. PhD thesis.
Ghaffari, Amir (2015) The scalability of reliable computation in Erlang. PhD thesis. http://theses.gla.ac.uk/6789/ Copyright and moral rights for this thesis are retained by the author A copy can be downloaded
More informationIntroduction to Parallel Programming
Introduction to Parallel Programming David Lifka lifka@cac.cornell.edu May 23, 2011 5/23/2011 www.cac.cornell.edu 1 y What is Parallel Programming? Using more than one processor or computer to complete
More informationCSCI-GA Multicore Processors: Architecture & Programming Lecture 3: The Memory System You Can t Ignore it!
CSCI-GA.3033-012 Multicore Processors: Architecture & Programming Lecture 3: The Memory System You Can t Ignore it! Mohamed Zahran (aka Z) mzahran@cs.nyu.edu http://www.mzahran.com Memory Computer Technology
More informationExperiments in OTP-Compliant Dataflow Programming
Experiments in OTP-Compliant Dataflow Programming Introducing Erlang Services Platform (Erlang/SP) San Francisco Erlang Factory, March 21, 2013 Jay Nelson Twitter: @duomark Email: Jay@duomark.com Manycore
More informationMesosphere and the Enterprise: Run Your Applications on Apache Mesos. Steve Wong Open Source Engineer {code} by Dell
Mesosphere and the Enterprise: Run Your Applications on Apache Mesos Steve Wong Open Source Engineer {code} by Dell EMC @cantbewong Open source at Dell EMC {code} by Dell EMC is a group of passionate open
More informationScalable Tools - Part I Introduction to Scalable Tools
Scalable Tools - Part I Introduction to Scalable Tools Adisak Sukul, Ph.D., Lecturer, Department of Computer Science, adisak@iastate.edu http://web.cs.iastate.edu/~adisak/mbds2018/ Scalable Tools session
More informationSuper-Peer Architectures for Distributed Computing
Whitepaper Super-Peer Architectures for Distributed Computing Fiorano Software, Inc. 718 University Avenue, Suite 212 Los Gatos, CA 95032 U.S.A. +1.408.354.3210 email info@fiorano.com www.fiorano.com Entire
More informationProgramming Paradigms
PP 2017/18 Unit 15 Concurrent Programming with Erlang 1/32 Programming Paradigms Unit 15 Concurrent Programming with Erlang J. Gamper Free University of Bozen-Bolzano Faculty of Computer Science IDSE PP
More informationErlang and Concurrency. André Pang Rising Sun Research
Erlang and Concurrency André Pang Rising Sun Research Games. Unreal 3 engine. Amazing graphics. (Unreal 3 video demo.) 2004/2005? Still state-of-the-art. What does that game have to do with this talk?
More informationErlang 101. Google Doc
Erlang 101 Google Doc Erlang? with buzzwords Erlang is a functional concurrency-oriented language with extremely low-weight userspace "processes", share-nothing messagepassing semantics, built-in distribution,
More informationErlang and VoltDB TechPlanet 2012 H. Diedrich
TechPlanet 2012 H. Diedrich http://www.eonblast.com twitter @hdiedrich 1 Your Host Henning Diedrich Founder, CEO CTO Freshworks CTO, Producer at Newtracks Team Lead, Producer at Bigpoint OS Maintainer
More informationErlang in the battlefield. Łukasz Kubica Telco BSS R&D Department Cracow Erlang Factory Lite, 2013
Erlang in the battlefield Łukasz Kubica Telco BSS R&D Department Cracow Erlang Factory Lite, 2013 Agenda Introduction to the SCM Erlang vm and upgrades Tracing Mnesia Final thoughts Questions 2 The Session
More informationVisual Design Flows for Faster Debug and Time to Market FlowTracer White Paper
Visual Design Flows for Faster Debug and Time to Market FlowTracer White Paper 2560 Mission College Blvd., Suite 130 Santa Clara, CA 95054 (408) 492-0940 Introduction As System-on-Chip (SoC) designs have
More informationThe 7 Habits of Highly Effective API and Service Management
7 Habits of Highly Effective API and Service Management: Introduction The 7 Habits of Highly Effective API and Service Management... A New Enterprise challenge has emerged. With the number of APIs growing
More informationScaling DreamFactory
Scaling DreamFactory This white paper is designed to provide information to enterprise customers about how to scale a DreamFactory Instance. The sections below talk about horizontal, vertical, and cloud
More informationVmware VCP550PSE. VMware Certified Professional on vsphere 5.
Vmware VCP550PSE VMware Certified Professional on vsphere 5 http://killexams.com/exam-detail/vcp550pse QUESTION: 108 A virtual machine fails to migrate during a Storage DRS event. What could cause this
More informationNetwork Slicing Supported by Dynamic VIM Instantatiation. Stuart Clayman Dept of Electronic Engineering University College London
Network Slicing Supported by Dynamic Instantatiation Stuart Clayman Dept of Electronic Engineering University College London Overview Here we present an overview of some of the mechanisms, components,
More informationBUILDING the VIRtUAL enterprise
BUILDING the VIRTUAL ENTERPRISE A Red Hat WHITEPAPER www.redhat.com As an IT shop or business owner, your ability to meet the fluctuating needs of your business while balancing changing priorities, schedules,
More informationCSE 120 Principles of Operating Systems
CSE 120 Principles of Operating Systems Spring 2018 Lecture 15: Multicore Geoffrey M. Voelker Multicore Operating Systems We have generally discussed operating systems concepts independent of the number
More information