Inside Broker How Broker Leverages the C++ Actor Framework (CAF)
|
|
- Phillip Cain
- 5 years ago
- Views:
Transcription
1 Inside Broker How Broker Leverages the C++ Actor Framework (CAF) Dominik Charousset inet RG, Department of Computer Science Hamburg University of Applied Sciences Bro4Pros, February
2 What was Broker again? 2
3 Problem at Hand Bro A State Updates Events Logs Bro B User App. 3
4 Traditional Approach Bro A Child Process Child Process Bro B Events libbroccoli 4 Image source: Robin Sommer, BroCon 2015
5 Traditional Issues Persistency issues Possible race conditions with &synchronized Limited control over data flow 5
6 Broker Approach Bro A Broker Broker Bro B Broker Application C C++ Python 6 Image source: Robin Sommer, BroCon 2015
7 Broker Benefits Grant unified access to Bro events Empower users to manage state Provide a global, persistent key/value store 7
8 How does CAF relate to Bro? 8
9 Broker in Context Bro: monitor the network Broker: distribute network insights Events M CAF C C C M C C 9
10 Broker's Goals Provide flexible pub/sub data distribution Enable distributed, deep detection Support data-intense algorithms on realtime events 10
11 Broker's Requirements Efficient communication layer Expressive data model Persistent storage 11
12 Fueling Broker Broker uses CAF to meet its requirements: Structure: endpoints & messages Communication: send & receive Network: connect peers & distribute data 12
13 CAF in a Nutshell Programming interface based on the actor model Configurable runtime for infrastructure software* Emphasis on reliability, efficiency & maintainability *following def. in: Bjarne Stroustroup, Software Development for Infrastructure, IEE Computer 45,
14 What is our vision for a next-gen Bro? 14
15 Deep Detection Correlation in multi-hop processing pipelines Distribution with pub/sub data access Resilience through replicated data stores 15
16 Bro Cluster Internet Tap Local Network Vision Goal: monitor for a next-gen critical communication Bro with CAF. path. Frontend #1: split agile traffic rebalancing into many via netcontrol streams/flows. & broker. Worker Worker Worker #2: #2: (stateful) pub/sub traffic & consensus monitoring instead and protocol of shared analysis. state. Proxy Packets Logs State Manager #3: combine fault-tolerance and post-process & failover worker-generated through snapshotting. logs. 16
17 Leveraging CAF Bro has to grow with user demands Scaling up and out is key to meet future work loads CAF provides building blocks for a next-gen Bro 17
18 What is CAF, exactly? 18
19 Scalable Abstractions Actors avoid race conditions by design Unified API for concurrency & distribution Compose large systems from small components Scale runtime from the IoT up to HPC Microcontrollers Servers Supercomputers 19
20 The Actor Model Asynchronous message passing No shared state Divide & conquer work flow Actor FIFO mailbox Hierarchical failure handling & propagation 20
21 Anatomy of an Actor Actor Processing (Control Loop) Storage (State) Dequeue Message Internal Variables int count; string foo;... Communication (via FIFO mailbox) Invoke Behavior no Message Handlers (Behavior) done? yes [=](int x) { count += x; }... Address to an actor (allows enqueueing of messages) 21
22 CAF's Architecture Node Node Process Actor Message Network CPU GPGPU GPU Actor System Actor System Network Middleman Cooperative Scheduler GPGPU Wrapper Distribution Layer Socket API Thread API OpenCL 22
23 Communication Patterns CAF offers various messaging primitives: Asynchronous "fire & forget" messages Request/response messaging (with timeouts) Pub/sub-based group communication Streaming pipelines (soon-ish) 23
24 CAF Facts Sheet Developed at inet research group First commit: March 4, 2011 Active international community > 40,000 lines of code ( 24
25 What is next? 25
26 Streaming Streams as first-class citizen in CAF Priority-aware message processing Re-deployable actor pipelines with back pressure 26
27 Streaming Concept data flows downstream source stage sink demand flows upstream errors are propagated both ways 27
28 Streaming Bro Events Critical realtime data import. CAF Application Besteffort file imports Server Parser Worker 28
29 High-level Clustering Declarative API for deploying actors & pipelines Dynamic redeployment & -configuration Monitoring of running CAF applications 29
30 Debugging Support Debugging distributed applications is challenging CAF's logs can reproduce causal ordering Visualization helps devs understand their system, e.g., with ShiViz: 30 Image source:
31 ShiViz* UI with CAF App. 31 * see:
32 Tracing Lightweight monitoring of data flows Captures causal and temporal ordering of events Recording (debugging) or sampling (monitoring) 32
33 Tracing: Example Monitor Inject Annotated Request N4 (N4, N5, N6, N2, N1) (N4) N1 N5 (N4, N5, N6, N2) N3 (N4, N5) N7 N2 (N4, N5, N6) N6 33
34 Tracing: Visualization Path in the system Causal and temporal relationship User X Response (time) Frontend Request A Request rpc1 rpc2 rpc1 Management B C rpc2 rpc3 rpc4 rpc3 Backend D E rpc4 Fig. mod. from: Benjamin Sigelman et al., Dapper, a Large-Scale Distributed Systems Tracing Infrastructure, Google Technical Report,
35 Thanks for Listening bro/broker actor-framework actor_framework 35
Broker. Matthias Vallentin UC Berkeley International Computer Science Institute (ICSI) BroCon '16
Broker Matthias Vallentin UC Berkeley International Computer Science Institute (ICSI) BroCon '16 Communication in Bro Exploiting Independent State For Network Intrusion Detection Tap Broccoli, Independent
More informationSoftware Architecture Patterns
Software Architecture Patterns *based on a tutorial of Michael Stal Harald Gall University of Zurich http://seal.ifi.uzh.ch/ase www.infosys.tuwien.ac.at Overview Goal Basic architectural understanding
More informationAchieving Scalability and High Availability for clustered Web Services using Apache Synapse. Ruwan Linton WSO2 Inc.
Achieving Scalability and High Availability for clustered Web Services using Apache Synapse Ruwan Linton [ruwan@apache.org] WSO2 Inc. Contents Introduction Apache Synapse Web services clustering Scalability/Availability
More informationC++ Actor Framework. Transparent Scaling from IoT to Datacenter Apps. Matthias Vallentin. UC Berkeley. RISElab seminar November 21, 2016
C++ Actor Framework Transparent Scaling from IoT to Datacenter Apps Matthias Vallentin UC Berkeley RISElab seminar November 21, 2016 Heterogeneity More cores on desktops and mobile Complex accelerators/co-processors
More informationThe Stream Processor as a Database. Ufuk
The Stream Processor as a Database Ufuk Celebi @iamuce Realtime Counts and Aggregates The (Classic) Use Case 2 (Real-)Time Series Statistics Stream of Events Real-time Statistics 3 The Architecture collect
More informationThe NIDS Cluster: Scalable, Stateful Network Intrusion Detection on Commodity Hardware
The NIDS Cluster: Scalable, Stateful Network Intrusion Detection on Commodity Hardware Matthias Vallentin 1, Robin Sommer 2,3, Jason Lee 2, Craig Leres 2 Vern Paxson 3,2, and Brian Tierney 2 1 TU München
More informationStreaming data Model is opposite Queries are usually fixed and data are flows through the system.
1 2 3 Main difference is: Static Data Model (For related database or Hadoop) Data is stored, and we just send some query. Streaming data Model is opposite Queries are usually fixed and data are flows through
More informationBUILDING A SCALABLE MOBILE GAME BACKEND IN ELIXIR. Petri Kero CTO / Ministry of Games
BUILDING A SCALABLE MOBILE GAME BACKEND IN ELIXIR Petri Kero CTO / Ministry of Games MOBILE GAME BACKEND CHALLENGES Lots of concurrent users Complex interactions between players Persistent world with frequent
More informationBuilding Microservices with the 12 Factor App Pattern
Building Microservices with the 12 Factor App Pattern Context This documentation will help introduce Developers to implementing MICROSERVICES by applying the TWELVE- FACTOR PRINCIPLES, a set of best practices
More informationHi! NET Developer Group Braunschweig!
Hi! NET Developer Group Braunschweig! Über Tobias Dipl. Informatiker (FH) Passionated Software Developer Clean Code Developer.NET Junkie.NET User Group Lead Microsoft PFE Software Development Twitter @Blubern
More informationAbove the Clouds: Introducing Akka. Jonas Bonér Scalable Solutions
Above the Clouds: Introducing Akka Jonas Bonér CEO @ Scalable Solutions Twitter: @jboner The problem It is way too hard to build: 1. correct highly concurrent systems 2. truly scalable systems 3. fault-tolerant
More informationUsing the SDACK Architecture to Build a Big Data Product. Yu-hsin Yeh (Evans Ye) Apache Big Data NA 2016 Vancouver
Using the SDACK Architecture to Build a Big Data Product Yu-hsin Yeh (Evans Ye) Apache Big Data NA 2016 Vancouver Outline A Threat Analytic Big Data product The SDACK Architecture Akka Streams and data
More informationData Acquisition. The reference Big Data stack
Università degli Studi di Roma Tor Vergata Dipartimento di Ingegneria Civile e Ingegneria Informatica Data Acquisition Corso di Sistemi e Architetture per Big Data A.A. 2016/17 Valeria Cardellini The reference
More informationTime and Space. Indirect communication. Time and space uncoupling. indirect communication
Time and Space Indirect communication Johan Montelius In direct communication sender and receivers exist in the same time and know of each other. KTH In indirect communication we relax these requirements.
More informationDistributed Data on Distributed Infrastructure. Claudius Weinberger & Kunal Kusoorkar, ArangoDB Jörg Schad, Mesosphere
Distributed Data on Distributed Infrastructure Claudius Weinberger & Kunal Kusoorkar, ArangoDB Jörg Schad, Mesosphere Kunal Kusoorkar Director Solutions Engineering, ArangoDB @neunhoef Jörg Schad Claudius
More informationTechnical Brief. A Checklist for Every API Call. Managing the Complete API Lifecycle
Technical Brief A Checklist for Table of Contents Introduction: The API Lifecycle 2 3 Security professionals API developers Operations engineers API product or business owners Apigee Edge 7 A Checklist
More informationDynamic Fine Grain Scheduling of Pipeline Parallelism. Presented by: Ram Manohar Oruganti and Michael TeWinkle
Dynamic Fine Grain Scheduling of Pipeline Parallelism Presented by: Ram Manohar Oruganti and Michael TeWinkle Overview Introduction Motivation Scheduling Approaches GRAMPS scheduling method Evaluation
More informationAbstract. Introduction
Highly Available In-Memory Metadata Filesystem using Viewstamped Replication (https://github.com/pkvijay/metadr) Pradeep Kumar Vijay, Pedro Ulises Cuevas Berrueco Stanford cs244b-distributed Systems Abstract
More informationReactive Microservices Architecture on AWS
Reactive Microservices Architecture on AWS Sascha Möllering Solutions Architect, @sascha242, Amazon Web Services Germany GmbH Why are we here today? https://secure.flickr.com/photos/mgifford/4525333972
More informationIndirect Communication
Indirect Communication To do q Today q q Space and time (un)coupling Common techniques q Next time: Overlay networks xkdc Direct coupling communication With R-R, RPC, RMI Space coupled Sender knows the
More informationCloud Programming James Larus Microsoft Research. July 13, 2010
Cloud Programming James Larus Microsoft Research July 13, 2010 New Programming Model, New Problems (and some old, unsolved ones) Concurrency Parallelism Message passing Distribution High availability Performance
More informationExam : Implementing Microsoft Azure Infrastructure Solutions
Exam 70-533: Implementing Microsoft Azure Infrastructure Solutions Objective Domain Note: This document shows tracked changes that are effective as of January 18, 2018. Design and Implement Azure App Service
More informationIndirect Communication
Indirect Communication Vladimir Vlassov and Johan Montelius KTH ROYAL INSTITUTE OF TECHNOLOGY Time and Space In direct communication sender and receivers exist in the same time and know of each other.
More information05 Indirect Communication
05 Indirect Communication Group Communication Publish-Subscribe Coulouris 6 Message Queus Point-to-point communication Participants need to exist at the same time Establish communication Participants need
More informationAkka: Simpler Concurrency, Scalability & Fault-tolerance through Actors. Jonas Bonér Viktor Klang
Akka: Simpler Concurrency, Scalability & Fault-tolerance through Actors Jonas Bonér Viktor Klang We believe that... Writing correct concurrent applications is too hard Scaling out applications is too hard
More informationMillWheel:Fault Tolerant Stream Processing at Internet Scale. By FAN Junbo
MillWheel:Fault Tolerant Stream Processing at Internet Scale By FAN Junbo Introduction MillWheel is a low latency data processing framework designed by Google at Internet scale. Motived by Google Zeitgeist
More informationLoosely Coupled Actor Systems
Loosely Coupled Actor Systems for the Internet of Things Raphael Hiesgen Internet Technologies Group Hamburg University of Applied Sciences Agenda Introduction Where We Are Next Steps Risks and Conclusion
More informationMindLink Desktop. Technical Overview. Version 17.3
Desktop Technical Overview Version 17.3 Table of Contents 1 Overview... 3 1.1 Browser Support... 3 1.2 High-level Architecture... 3 2 lication Lifecycle... 4 2.1 Configuration Bootstrapping... 4 2.2 Logging
More informationTransformation-free Data Pipelines by combining the Power of Apache Kafka and the Flexibility of the ESB's
Building Agile and Resilient Schema Transformations using Apache Kafka and ESB's Transformation-free Data Pipelines by combining the Power of Apache Kafka and the Flexibility of the ESB's Ricardo Ferreira
More informationAdaptive Cluster Computing using JavaSpaces
Adaptive Cluster Computing using JavaSpaces Jyoti Batheja and Manish Parashar The Applied Software Systems Lab. ECE Department, Rutgers University Outline Background Introduction Related Work Summary of
More informationPayPal Delivers World Class Customer Service, Worldwide
PayPal Delivers World Class Customer Service, Worldwide Greg Gates, VP of Enterprise Ops Engineering Ramki Rosanuru, Sr. Engineering Manager-COE PayPal PEGA in PayPal Why we choose PEGA? Bridge the gap
More informationarxiv: v1 [cs.dc] 30 Sep 2018
If you cite this paper, please use the AGERE!@SPLASH reference: Raphael Hiesgen, Dominik Charousset and Thomas C. Schmidt. A Configurable Transport Layer for CAF. In Proc. of ACM SIGPLAN SPLASH, ACM, 2018.
More informationCLUSTERING HIVEMQ. Building highly available, horizontally scalable MQTT Broker Clusters
CLUSTERING HIVEMQ Building highly available, horizontally scalable MQTT Broker Clusters 12/2016 About this document MQTT is based on a publish/subscribe architecture that decouples MQTT clients and uses
More informationData Acquisition. The reference Big Data stack
Università degli Studi di Roma Tor Vergata Dipartimento di Ingegneria Civile e Ingegneria Informatica Data Acquisition Corso di Sistemi e Architetture per Big Data A.A. 2017/18 Valeria Cardellini The reference
More informationExecutive Summary. It is important for a Java Programmer to understand the power and limitations of concurrent programming in Java using threads.
Executive Summary. It is important for a Java Programmer to understand the power and limitations of concurrent programming in Java using threads. Poor co-ordination that exists in threads on JVM is bottleneck
More informationBUILDING MICROSERVICES ON AZURE. ~ Vaibhav
BUILDING MICROSERVICES ON AZURE ~ Vaibhav Gujral @vabgujral About Me Over 11 years of experience Working with Assurant Inc. Microsoft Certified Azure Architect MCSD, MCP, Microsoft Specialist Aspiring
More informationA Tracing Technique for Understanding the Behavior of Large-Scale Distributed Systems
A Tracing Technique for Understanding the Behavior of Large-Scale Distributed Systems Yuichi Bando NTT Software Innovation Center Who am I? Research engineer at NTT Software Innovation Center (SIC) SIC
More informationCommunication Paradigms
Communication Paradigms Nicola Dragoni Embedded Systems Engineering DTU Compute 1. Interprocess Communication Direct Communication: Sockets Indirect Communication: IP Multicast 2. High Level Communication
More informationDREMS: A Toolchain and Platform for the Rapid Application Development, Integration, and Deployment of Managed Distributed Real-time Embedded Systems
DREMS: A Toolchain and Platform for the Rapid Application Development, Integration, and Deployment of Managed Distributed Real-time Embedded Systems William Emfinger, Pranav Kumar, Abhishek Dubey, William
More informationManaging your microservices with Kubernetes and Istio. Craig Box
Managing your microservices with Kubernetes and Istio Craig Box Agenda What is a Service Mesh? How we got here: a story Architecture and details Q&A 2 What is a service mesh? A network for services, not
More informationChapter Outline. Chapter 2 Distributed Information Systems Architecture. Distributed transactions (quick refresh) Layers of an information system
Prof. Dr.-Ing. Stefan Deßloch AG Heterogene Informationssysteme Geb. 36, Raum 329 Tel. 0631/205 3275 dessloch@informatik.uni-kl.de Chapter 2 Distributed Information Systems Architecture Chapter Outline
More informationIEMS 5780 / IERG 4080 Building and Deploying Scalable Machine Learning Services
IEMS 5780 / IERG 4080 Building and Deploying Scalable Machine Learning Services Lecture 11 - Asynchronous Tasks and Message Queues Albert Au Yeung 22nd November, 2018 1 / 53 Asynchronous Tasks 2 / 53 Client
More informationDistributed Coordination with ZooKeeper - Theory and Practice. Simon Tao EMC Labs of China Oct. 24th, 2015
Distributed Coordination with ZooKeeper - Theory and Practice Simon Tao EMC Labs of China {simon.tao@emc.com} Oct. 24th, 2015 Agenda 1. ZooKeeper Overview 2. Coordination in Spring XD 3. ZooKeeper Under
More informationIBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM s sole discretion.
Please note Copyright 2018 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM IBM s statements
More informationPushyDB. Jeff Chan, Kenny Lam, Nils Molina, Oliver Song {jeffchan, kennylam, molina,
PushyDB Jeff Chan, Kenny Lam, Nils Molina, Oliver Song {jeffchan, kennylam, molina, osong}@mit.edu https://github.com/jeffchan/6.824 1. Abstract PushyDB provides a more fully featured database that exposes
More informationAWS Lambda: Event-driven Code in the Cloud
AWS Lambda: Event-driven Code in the Cloud Dean Bryen, Solutions Architect AWS Andrew Wheat, Senior Software Engineer - BBC April 15, 2015 London, UK 2015, Amazon Web Services, Inc. or its affiliates.
More informationMicroservices on AWS. Matthias Jung, Solutions Architect AWS
Microservices on AWS Matthias Jung, Solutions Architect AWS Agenda What are Microservices? Why Microservices? Challenges of Microservices Microservices on AWS What are Microservices? What are Microservices?
More informationModern Stream Processing with Apache Flink
1 Modern Stream Processing with Apache Flink Till Rohrmann GOTO Berlin 2017 2 Original creators of Apache Flink da Platform 2 Open Source Apache Flink + da Application Manager 3 What changes faster? Data
More informationOpenShift Roadmap Enterprise Kubernetes for Developers. Clayton Coleman, Architect, OpenShift
OpenShift Roadmap Enterprise Kubernetes for Developers Clayton Coleman, Architect, OpenShift What Is OpenShift? Application-centric Platform INFRASTRUCTURE APPLICATIONS Use containers for efficiency Hide
More informationIndirect Communication
Indirect Communication Today l Space and time (un)coupling l Group communication, pub/sub, message queues and shared memory Next time l Distributed file systems xkdc Indirect communication " Indirect communication
More informationWide-area Migration with Monterey, AS7, Seam and jclouds
Wide-area Migration with Monterey, AS7, Seam and jclouds Alex Heneveld, CTO & Aled Sage, VP Engineering Cloudsoft Corporation Company Intro Who are Cloudsoft? Venture-backed software company headquartered
More informationVortex Whitepaper. Simplifying Real-time Information Integration in Industrial Internet of Things (IIoT) Control Systems
Vortex Whitepaper Simplifying Real-time Information Integration in Industrial Internet of Things (IIoT) Control Systems www.adlinktech.com 2017 Table of Contents 1. Introduction........ P 3 2. Iot and
More informationEdge Foundational Training
Edge Foundational Training Give your team the tools to get up and running with Edge Edge Foundational Training provides the tools and information needed to start using Edge whether in the cloud or on premises.
More informationEnd to End Optimization Stack for Deep Learning
End to End Optimization Stack for Deep Learning Presenter: Tianqi Chen Paul G. Allen School of Computer Science & Engineering University of Washington Collaborators University of Washington AWS AI Team
More informationStorm. Distributed and fault-tolerant realtime computation. Nathan Marz Twitter
Storm Distributed and fault-tolerant realtime computation Nathan Marz Twitter Storm at Twitter Twitter Web Analytics Before Storm Queues Workers Example (simplified) Example Workers schemify tweets and
More informationIntelligent Interconnect for Autonomous Vehicle SoCs. Sam Wong / Chi Peng, NetSpeed Systems
Intelligent Interconnect for Autonomous Vehicle SoCs Sam Wong / Chi Peng, NetSpeed Systems Challenges Facing Autonomous Vehicles Exploding Performance Requirements Real-Time Processing of Sensors Ultra-High
More informationStorm. Distributed and fault-tolerant realtime computation. Nathan Marz Twitter
Storm Distributed and fault-tolerant realtime computation Nathan Marz Twitter Basic info Open sourced September 19th Implementation is 15,000 lines of code Used by over 25 companies >2700 watchers on Github
More informationOverview SENTINET 3.1
Overview SENTINET 3.1 Overview 1 Contents Introduction... 2 Customer Benefits... 3 Development and Test... 3 Production and Operations... 4 Architecture... 5 Technology Stack... 7 Features Summary... 7
More informationApplications of Paxos Algorithm
Applications of Paxos Algorithm Gurkan Solmaz COP 6938 - Cloud Computing - Fall 2012 Department of Electrical Engineering and Computer Science University of Central Florida - Orlando, FL Oct 15, 2012 1
More informationGriddable.io architecture
Griddable.io architecture Executive summary This whitepaper presents the architecture of griddable.io s smart grids for synchronized data integration. Smart transaction grids are a novel concept aimed
More information"#' %#& Lecture 7: Organizing Game Clients and Servers. Socket: Network communication endpoints. IP address: IP-level name of a machine
Lecture 7: Organizing Game s and Servers! Socket: communication endpoints Analogous to a file descriptor Apps read/write to/from sockets system handles delivery IP address: IP-level name of a machine One
More informationChelsio Communications. Meeting Today s Datacenter Challenges. Produced by Tabor Custom Publishing in conjunction with: CUSTOM PUBLISHING
Meeting Today s Datacenter Challenges Produced by Tabor Custom Publishing in conjunction with: 1 Introduction In this era of Big Data, today s HPC systems are faced with unprecedented growth in the complexity
More informationMOM MESSAGE ORIENTED MIDDLEWARE OVERVIEW OF MESSAGE ORIENTED MIDDLEWARE TECHNOLOGIES AND CONCEPTS. MOM Message Oriented Middleware
MOM MESSAGE ORIENTED MOM Message Oriented Middleware MIDDLEWARE OVERVIEW OF MESSAGE ORIENTED MIDDLEWARE TECHNOLOGIES AND CONCEPTS Peter R. Egli 1/25 Contents 1. Synchronous versus asynchronous interaction
More informationLecture 5: Process Description and Control Multithreading Basics in Interprocess communication Introduction to multiprocessors
Lecture 5: Process Description and Control Multithreading Basics in Interprocess communication Introduction to multiprocessors 1 Process:the concept Process = a program in execution Example processes:
More informationAssignment 5. Georgia Koloniari
Assignment 5 Georgia Koloniari 2. "Peer-to-Peer Computing" 1. What is the definition of a p2p system given by the authors in sec 1? Compare it with at least one of the definitions surveyed in the last
More informationTake Risks But Don t Be Stupid! Patrick Eaton, PhD
Take Risks But Don t Be Stupid! Patrick Eaton, PhD preaton@google.com Take Risks But Don t Be Stupid! Patrick R. Eaton, PhD patrick@stackdriver.com Stackdriver A hosted service providing intelligent monitoring
More informationHigh Availability Distributed (Micro-)services. Clemens Vasters Microsoft
High Availability Distributed (Micro-)services Clemens Vasters Microsoft Azure @clemensv ice Microsoft Azure services I work(-ed) on. Notification Hubs Service Bus Event Hubs Event Grid IoT Hub Relay Mobile
More informationTyphoon: An SDN Enhanced Real-Time Big Data Streaming Framework
Typhoon: An SDN Enhanced Real-Time Big Data Streaming Framework Junguk Cho, Hyunseok Chang, Sarit Mukherjee, T.V. Lakshman, and Jacobus Van der Merwe 1 Big Data Era Big data analysis is increasingly common
More informationUpgrade Your MuleESB with Solace s Messaging Infrastructure
The era of ubiquitous connectivity is upon us. The amount of data most modern enterprises must collect, process and distribute is exploding as a result of real-time process flows, big data, ubiquitous
More informationHYBRID TRANSACTION/ANALYTICAL PROCESSING COLIN MACNAUGHTON
HYBRID TRANSACTION/ANALYTICAL PROCESSING COLIN MACNAUGHTON WHO IS NEEVE RESEARCH? Headquartered in Silicon Valley Creators of the X Platform - Memory Oriented Application Platform Passionate about high
More informationNext Paradigm for Decentralized Apps. Table of Contents 1. Introduction 1. Color Spectrum Overview 3. Two-tier Architecture of Color Spectrum 4
Color Spectrum: Next Paradigm for Decentralized Apps Table of Contents Table of Contents 1 Introduction 1 Color Spectrum Overview 3 Two-tier Architecture of Color Spectrum 4 Clouds in Color Spectrum 4
More informationArchitecting Microsoft Azure Solutions (proposed exam 535)
Architecting Microsoft Azure Solutions (proposed exam 535) IMPORTANT: Significant changes are in progress for exam 534 and its content. As a result, we are retiring this exam on December 31, 2017, and
More informationIntroduction to Distributed Systems
Introduction to Distributed Systems Other matters: review of the Bakery Algorithm: why can t we simply keep track of the last ticket taken and the next ticvket to be called? Ref: [Coulouris&al Ch 1, 2]
More informationNetworks and distributed computing
Networks and distributed computing Abstractions provided for networks network card has fixed MAC address -> deliver message to computer on LAN -> machine-to-machine communication -> unordered messages
More informationScale-out Storage Solution and Challenges Mahadev Gaonkar igate
Scale-out Solution and Challenges Mahadev Gaonkar igate 2013 Developer Conference. igate. All Rights Reserved. Table of Content Overview of Scale-out Scale-out NAS Solution Architecture IO Workload Distribution
More informationlibcppa Now: High-Level Distributed Programming Without Sacrificing Performance
libcppa Now: High-Level Distributed Programming Without Sacrificing Performance Matthias Vallentin matthias@bro.org University of California, Berkeley C ++ Now May 14, 2013 Outline 1. Example Application:
More informationSupercomputing and Mass Market Desktops
Supercomputing and Mass Market Desktops John Manferdelli Microsoft Corporation This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.
More informationThis tutorial will give you a quick start with Consul and make you comfortable with its various components.
About the Tutorial Consul is an important service discovery tool in the world of Devops. This tutorial covers in-depth working knowledge of Consul, its setup and deployment. This tutorial aims to help
More informationEvent Streams using Apache Kafka
Event Streams using Apache Kafka And how it relates to IBM MQ Andrew Schofield Chief Architect, Event Streams STSM, IBM Messaging, Hursley Park Event-driven systems deliver more engaging customer experiences
More informationHadoop File System S L I D E S M O D I F I E D F R O M P R E S E N T A T I O N B Y B. R A M A M U R T H Y 11/15/2017
Hadoop File System 1 S L I D E S M O D I F I E D F R O M P R E S E N T A T I O N B Y B. R A M A M U R T H Y Moving Computation is Cheaper than Moving Data Motivation: Big Data! What is BigData? - Google
More informationImplementing the Twelve-Factor App Methodology for Developing Cloud- Native Applications
Implementing the Twelve-Factor App Methodology for Developing Cloud- Native Applications By, Janakiram MSV Executive Summary Application development has gone through a fundamental shift in the recent past.
More informationTHE DATACENTER AS A COMPUTER AND COURSE REVIEW
THE DATACENTER A A COMPUTER AND COURE REVIEW George Porter June 8, 2018 ATTRIBUTION These slides are released under an Attribution-NonCommercial-hareAlike 3.0 Unported (CC BY-NC-A 3.0) Creative Commons
More informationData Communication in LabVIEW
An Overview of Data Communication in LabVIEW Elijah Kerry LabVIEW Product Manager Certified LabVIEW Architect (CLA) Data Communication Options in LabVIEW 1. TCP and UDP 2. Network Streams 3. Shared Variables
More informationBuilding a Real-time Notification System
Building a Real-time Notification System September 2015, Geneva Author: Jorge Vicente Cantero Supervisor: Jiri Kuncar CERN openlab Summer Student Report 2015 Project Specification Configurable Notification
More informationMOC 6461A C#: Visual Studio 2008: Windows Communication Foundation
MOC 6461A C#: Visual Studio 2008: Windows Communication Foundation Course Number: 6461A Course Length: 3 Days Certification Exam This course will help you prepare for the following Microsoft exam: Exam
More informationChapter 2 Distributed Information Systems Architecture
Prof. Dr.-Ing. Stefan Deßloch AG Heterogene Informationssysteme Geb. 36, Raum 329 Tel. 0631/205 3275 dessloch@informatik.uni-kl.de Chapter 2 Distributed Information Systems Architecture Chapter Outline
More informationAWS Lambda. 1.1 What is AWS Lambda?
Objectives Key objectives of this chapter Lambda Functions Use cases The programming model Lambda blueprints AWS Lambda 1.1 What is AWS Lambda? AWS Lambda lets you run your code written in a number of
More informationChapter 5: Processes & Process Concept. Objectives. Process Concept Process Scheduling Operations on Processes. Communication in Client-Server Systems
Chapter 5: Processes Chapter 5: Processes & Threads Process Concept Process Scheduling Operations on Processes Interprocess Communication Communication in Client-Server Systems, Silberschatz, Galvin and
More informationThe Bro Cluster The Bro Cluster
The Bro Cluster The Bro Cluster Intrusion Detection at 10 Gig and A High-Performance beyond using the NIDS Bro Architecture IDS for the Lawrence Berkeley National Lab Robin International Computer Science
More informationJBoss Users & Developers Conference. Boston:2010
JBoss Users & Developers Conference Boston:2010 Next Gen. Web Apps with GWT & JBoss Mike Brock (cbrock@redhat.com) The Browser is a Platform! Beyond Hypertext Web browsers now have very fast and very usable
More informationThe ProActive Parallel Suite. Fabrice Huet, INRIA-University of Nice Joint work with ActiveEon
1 The ProActive Parallel Suite Fabrice Huet, INRIA-University of Nice Joint work with ActiveEon 2 Outline Overview of ProActive Parallel Suite Active objects GCM Deployment ProActive Scheduler Resource
More informationNODE.JS MOCK TEST NODE.JS MOCK TEST I
http://www.tutorialspoint.com NODE.JS MOCK TEST Copyright tutorialspoint.com This section presents you various set of Mock Tests related to Node.js Framework. You can download these sample mock tests at
More informationChapter 5. The MapReduce Programming Model and Implementation
Chapter 5. The MapReduce Programming Model and Implementation - Traditional computing: data-to-computing (send data to computing) * Data stored in separate repository * Data brought into system for computing
More informationApache Flink. Alessandro Margara
Apache Flink Alessandro Margara alessandro.margara@polimi.it http://home.deib.polimi.it/margara Recap: scenario Big Data Volume and velocity Process large volumes of data possibly produced at high rate
More informationData Management in Application Servers. Dean Jacobs BEA Systems
Data Management in Application Servers Dean Jacobs BEA Systems Outline Clustered Application Servers Adding Web Services Java 2 Enterprise Edition (J2EE) The Application Server platform for Java Java Servlets
More informationMultithreading and Interactive Programs
Multithreading and Interactive Programs CS160: User Interfaces John Canny. Last time Model-View-Controller Break up a component into Model of the data supporting the App View determining the look of the
More informationReactive App using Actor model & Apache Spark. Rahul Kumar Software
Reactive App using Actor model & Apache Spark Rahul Kumar Software Developer @rahul_kumar_aws About Sigmoid We build realtime & big data systems. OUR CUSTOMERS Agenda Big Data - Intro Distributed Application
More informationCommunication. Distributed Systems Santa Clara University 2016
Communication Distributed Systems Santa Clara University 2016 Protocol Stack Each layer has its own protocol Can make changes at one layer without changing layers above or below Use well defined interfaces
More informationCoherence & WebLogic Server integration with Coherence (Active Cache)
WebLogic Innovation Seminar Coherence & WebLogic Server integration with Coherence (Active Cache) Duško Vukmanović FMW Principal Sales Consultant Agenda Coherence Overview WebLogic
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
CNA2080BU Deep Dive: How to Deploy and Operationalize Kubernetes Cornelia Davis, Pivotal Nathan Ness Technical Product Manager, CNABU @nvpnathan #VMworld #CNA2080BU Disclaimer This presentation may contain
More information