Go Circuit: Distributing the Go Language and Runtime. Petar Maymounkov
|
|
- Johnathan McDowell
- 6 years ago
- Views:
Transcription
1 Go Circuit: Distributing the Go Language and Runtime Petar Maymounkov
2 Problem: DEV OPS isolation
3 App complexity vs manual involvement Distribute cloud apps How to describe complex deploy topologies Elastic apps Admin manual included? Yes. Not elastic then. Cognitive load grows Every app requires attention Job control Visibility into process ecosystem. Abstract map of available hosts. Provision negotiation Universal code format Write once, run on any cluster Provides for cluster-specific operations Adopt incrementally Co-habitable, minimal Don t solve any app problems Just abstract underlying capabilities
4 Too much humans in the stack. SaaS? APP NET MON APP OS HOST DPLY CFG FAIL BKP RPL OS HOST DPLY CFG FAIL BKP RPL Key operations (like re-shard, replicate, etc.) should be in the app if you want consistency and automation Generic apps cannot provision hosts or self-deploy remotely Frameworks usually suffer: Heavy-weight, cross-integration External technologies (Catch 22)
5 True division of responsibilities USER APP OPS ENG Networking Job execution Hardware negotiation & provision
6 Solution concept
7 Overview Universal distribution format = Go source Open source executables Control app exposure to libraries Easy source-to-source transform for instrumentation Tiny source footprint for hard apps OPS vs APP isolation OPS need not understand how app works (black box) OPS controls networking, execution, provisioning, failure response, etc. Elementary to deploy Compile entire solution into one smart binary Deploy sloppily App self-organizes at runtime
8 Minimal Single System Image (MSSI) APP APP Abstraction Stack Level SSI Interface Linux POSIX Binary OS HOST OS HOST OS HOST Goal: Abstract away and standardize capabilities/resources and behaviors (failures) of system. Not: Conceal failures, add scheduling, provide reliability, etc. The circuit is just a tool for building reliable systems efficiently and portably. OpenSSI, etc. monolithic Plan9 modular Erlang/OTP monolithic Go Circuit modular minimal POSIX File Protocol, 9P CSP Functional Language CSP Imperative Language OS Network Language Language
9 Concurrency inside, concurrency outside Non-CSP CSP PROCESS THREAD Comm Exec Protocols (DEV stack) Framework (OPS stack) Sync Libraries Thread Async Libraries Lang Lang Go Language * No cognitive switch * Imperative * Meaningful stack traces * Debug, trace UI unchanged Concurrent Sequentially-communicating Processes (CSP)
10 Solution stack MSSI Circuit Language Go Process Networking & Authentication Dial, Listen, Spawn,... DRIVER APP CIRCUIT TRANSLATOR RUNTIME DRIVER DRIVER OPS interacts only with CIRCUIT Debug, profile, trace, monitor,... OPS ENG OS OS OS HOST HOST HOST
11 Solution translates language thru net High-level Linguistic Communication APP APP APP package main import circuit circuit.spawn( Low-level Networking ops CIR DRIVER CIR DRIVER CIR DRIVER package circuit import drv/n import drv/job OS OS OS HOST HOST HOST
12 Example: Circuit + Docker = SSI Linux Legacy Linux binaries Within containers instrumented with consistent view of whole cluster MSSI Circuit Language Process Networking & Authentication Dial, Listen, Spawn,... P DRIVER P P APP DRIVER P CIRCUIT TRANSLATOR RUNTIME P DRIVER OPS ENG SSI Linux, POSIX, Plan9 OS OS OS HOST HOST HOST
13 Circuit Programming
14 Circuitizing a Go app and build Add circuit runtime to any legacy Go program package main import _ circuit/load func main() { } Build ~/gopath/src/app/cmd$ go build
15 Go + spawn = distributed Go Circuit Go slow.host HOST PROCESS WORKER slow.host GOROUTINE GOROUTINE
16 Spawn Run a function in this process Compute(arg) Run a function in a new process on a negotiated host Spawn( locus, []string{ /db/shard, /critical }, SpawnCompute{}, arg, ) Declare spawn-able functions type SpawnCompute struct{} func (SpawnCompute) Compute(arg Arg) { }
17 Anchor file system (side note) Global file system of live workers, arranged by the user $ 4ls /... / /critical /critical/r1a4124aa276f4a4e /db /db/shard /db/shard/r1a4124aa276f4a4e /db/shard/r8efd95a09bcde325 More on this later.
18 Spawn semantics Returns RETURN VALUES WORKER ADDRESS ERROR func Spawn( ) ([]interface{}, n.addr, error) Worker dies as soon as function exits... func (SpawnCompute) Compute(arg Arg) (r1, r2 ReturnType) { return } unless, we defer its death to the end of another goroutine. func (SpawnCompute) Compute(arg Arg) { ExitAfter(func() { return }) return }
19 Values vs cross-interfaces Communicate by value or cross-interface func Compute(int, Struct, *T, []byte, map[key]value, interface{}, X) type T struct { Cross X Value float64 } Create a cross-interface to a local receiver x := Ref(r) Use a cross-interface: call its methods reply = x.call( MethodName, arg1, arg2) // Returns []interface{}
20 Cross-worker garbage collection Cross-interfaces func (SpawnCompute) Compute(X) (X, error) { return Ref(v), nil } Permanent cross-interfaces func (SpawnCompute) Compute(XPerm) (XPerm, error) { return PermRef(v), nil } Spawn( host, nil, SpawnCompute{}, Ref(v), ) Spawn( host, nil, SpawnCompute{}, PermRef(v), )
21 Cross-call semantics Cross-calls are not RPC They return out-of-order, just like Go function calls do var x X // Cross-interface go func() { ch < x.call( Find, needle ) }() go func() { ch < x.call( Find, haystack ) }() < ch // Wait for whoever returns first
22 Error propagation Retain natural method signature func (s *Server) ServeHTML(url string) (string, error) { } Decouple application and system errors at call site var x X // x is a cross-interface to a remote *Server instance defer recover() // Catch system errors as panics result := x.call( ServerHTML, url) // Get app errors in return
23 Worker services Expose a cross-interface to a worker object Listen( ingest, r) Recall that spawn returns a worker address result, addr, err := Spawn() Dial a worker and obtain an exposed cross-interface x := Dial(addr, ingest )
24 Circuit Architecture
25 The circuit translates Go + Spawn import _ circuit/load CIRCUIT Network + Spawn Instructions type S struct { A float64 B []X } func (R) F(s *S, x X) (X, error) { return Ref(v), nil } defer recover() result := x.call( F, s1, y) Reflect on in/out types of F Recurse within composite types to find all cross-interface fields Read input arguments Check type safety Dial remote worker address Encode and send cross-call request Read response or error etc.
26 Modules decouple. Universal code. OPS USER CODE import ( use/circuit use/afs _ load/petar ) APP DEV load/petar use/circuit use/afs import ( _ sys/lang _ sys/zafs ) sys/lang sys/zafs import ( use/n use/afs ) func init() { afs.bind() } CIRCUIT DEV
27 Circuit Facilities
28 Anchor FS + cli tools = debug, profile, Global file system of live workers, arranged by the user $ 4ls /... / /critical /critical/r1a4124aa276f4a4e /db /db/shard /db/shard/r1a4124aa276f4a4e /db/shard/r8efd95a09bcde325 Kill $ 4kill /critical/... Profile for 30 sec $ 4cpu /critical/ 30 Stack traces $ 4ls /db/shard/ xargs 4stk Monitor worker vitals (mem, cpu, ) $ 4top /critical/
29 The end. Disclaimer. Talk describes soon-to-be-released R2. Skipped Teleport Transport. See github.com/petar/goteleport gocircuit.org
30 Appendix
31 Historical perspective on CSP systems Servers + Hierarchy of exported channels Workers + Crossinterfaces Cluster Level Plan9 Interfaces are more general than channels. Channels are interfaces with Send and Receive. Go Circuit Process Level Go Lang Goroutines + Channels
Building Microservices with the 12 Factor App Pattern
Building Microservices with the 12 Factor App Pattern Context This documentation will help introduce Developers to implementing MICROSERVICES by applying the TWELVE- FACTOR PRINCIPLES, a set of best practices
More informationKnit, Chisel, Hack: Crafting with Guile Scheme. Andy Wingo ~ wingolog.org
Knit, Chisel, Hack: Crafting with Guile Scheme Andy Wingo ~ wingo@igalia.com wingolog.org ~ @andywingo I love craft! Woodworking Gardening Grow-your-own Brew-your-own Knit-your-own Sew-your-own Roast-your-own
More informationCSci Introduction to Distributed Systems. Communication: RPC
CSci 5105 Introduction to Distributed Systems Communication: RPC Today Remote Procedure Call Chapter 4 TVS Last Time Architectural styles RPC generally mandates client-server but not always Interprocess
More informationDebugging and Profiling
Debugging & Profiling Dr.-Ing. Thomas Springer M.Sc. Martin Weißbach Errors in Swift conditions can occur that require a deviation from the predefined control flow in order to handle e.g. a file does not
More informationRemote Invocation. Today. Next time. l Indirect communication. l Request-reply, RPC, RMI
Remote Invocation Today l Request-reply, RPC, RMI Next time l Indirect communication Data representation and marshalling Processes information kept as data structures but sent in msgs as sequence of bytes
More information416 Distributed Systems. RPC Day 2 Jan 12, 2018
416 Distributed Systems RPC Day 2 Jan 12, 2018 1 Last class Finish networks review Fate sharing End-to-end principle UDP versus TCP; blocking sockets IP thin waist, smart end-hosts, dumb (stateless) network
More informationWhat are some common categories of system calls? What are common ways of structuring an OS? What are the principles behind OS design and
What are the services provided by an OS? What are system calls? What are some common categories of system calls? What are the principles behind OS design and implementation? What are common ways of structuring
More informationEmbedded type method, overriding, Error handling, Full-fledged web framework, 208 Function defer, 31 panic, 32 recover, 32 33
Index A Alice package, 108, 110 App Engine applications configuration file, 258 259 goapp deploy command, 262 Google Developers Console project creation, 261 project details, 262 HTTP server, 257 258 task
More informationfor Multi-Services Gateways
KURA an OSGi-basedApplication Framework for Multi-Services Gateways Introduction & Technical Overview Pierre Pitiot Grenoble 19 février 2014 Multi-Service Gateway Approach ESF / Increasing Value / Minimizing
More informationGo Forth and Code. Jonathan Gertig. CSC 415: Programing Languages. Dr. Lyle
J o n a t h a n G e r t i g P a g e 1 Go Forth and Code Jonathan Gertig CSC 415: Programing Languages Dr. Lyle 2013 J o n a t h a n G e r t i g P a g e 2 Go dogs Go or A Brief History of Go 6 years ago
More informationUsing DC/OS for Continuous Delivery
Using DC/OS for Continuous Delivery DevPulseCon 2017 Elizabeth K. Joseph, @pleia2 Mesosphere 1 Elizabeth K. Joseph, Developer Advocate, Mesosphere 15+ years working in open source communities 10+ years
More informationThe benefits and costs of writing a POSIX kernel in a high-level language
1 / 38 The benefits and costs of writing a POSIX kernel in a high-level language Cody Cutler, M. Frans Kaashoek, Robert T. Morris MIT CSAIL Should we use high-level languages to build OS kernels? 2 / 38
More informationKnative: Building serverless platforms on top of Kubernetes
Knative: Building serverless platforms on top of Kubernetes Ahmet Alp Balkan @ahmetb Thanks to Mark Chmarny, Ryan Gregg, DeWitt Clinton and Bret McGowen for some of the slides used in this presentation.
More informationOS Virtualization. Linux Containers (LXC)
OS Virtualization Emulate OS-level interface with native interface Lightweight virtual machines No hypervisor, OS provides necessary support Referred to as containers Solaris containers, BSD jails, Linux
More informationSystem support for adaptation and composition of applications
System support for adaptation and composition of applications Stephen Kell Stephen.Kell@cl.cam.ac.uk Computer Laboratory University of Cambridge... adaptation and composition... p. 1 First, a video Note:
More informationIntroduzione a Go e RPC in Go
Università degli Studi di Roma Tor Vergata Dipartimento di Ingegneria Civile e Ingegneria Informatica Introduzione a Go e RPC in Go Corso di Sistemi Distribuiti e Cloud Computing A.A. 2017/18 Valeria Cardellini
More informationErlang 101. Google Doc
Erlang 101 Google Doc Erlang? with buzzwords Erlang is a functional concurrency-oriented language with extremely low-weight userspace "processes", share-nothing messagepassing semantics, built-in distribution,
More informationExam Questions. Give an example network topology where GPSR cannot find a way from a source to a sink. Explain your answer.
Exam Questions Type 1 Consider a 1000m2 area with a uniform distribution of 1000 sensor nodes, each equipped with a temperature sensor. You want to constitute a temperature map of the area at various resolutions.
More informationOpenShift Roadmap Enterprise Kubernetes for Developers. Clayton Coleman, Architect, OpenShift
OpenShift Roadmap Enterprise Kubernetes for Developers Clayton Coleman, Architect, OpenShift What Is OpenShift? Application-centric Platform INFRASTRUCTURE APPLICATIONS Use containers for efficiency Hide
More information416 Distributed Systems. RPC Day 2 Jan 11, 2017
416 Distributed Systems RPC Day 2 Jan 11, 2017 1 Last class Finish networks review Fate sharing End-to-end principle UDP versus TCP; blocking sockets IP thin waist, smart end-hosts, dumb (stateless) network
More informationOverview. This Lecture. Interrupts and exceptions Source: ULK ch 4, ELDD ch1, ch2 & ch4. COSC440 Lecture 3: Interrupts 1
This Lecture Overview Interrupts and exceptions Source: ULK ch 4, ELDD ch1, ch2 & ch4 COSC440 Lecture 3: Interrupts 1 Three reasons for interrupts System calls Program/hardware faults External device interrupts
More informationSmashing Node.JS: JavaScript Everywhere
Smashing Node.JS: JavaScript Everywhere Rauch, Guillermo ISBN-13: 9781119962595 Table of Contents PART I: GETTING STARTED: SETUP AND CONCEPTS 5 Chapter 1: The Setup 7 Installing on Windows 8 Installing
More informationImplementing the Twelve-Factor App Methodology for Developing Cloud- Native Applications
Implementing the Twelve-Factor App Methodology for Developing Cloud- Native Applications By, Janakiram MSV Executive Summary Application development has gone through a fundamental shift in the recent past.
More informationHeroku. Rimantas Kybartas
Heroku Rimantas Kybartas Salesforce platform (PaaS) Facts about Heroku Has been in development since June 2007, 2010 acquired by Salesforce Open platform Languages and frameworks: Ruby and Rails Node.js
More informationSantiago Documentation
Santiago Documentation Release 1.2.0 Top Free Games November 07, 2016 Contents 1 Overview 3 1.1 Getting started.............................................. 3 1.2 Features..................................................
More informationLessons learnt building Kubernetes controllers. David Cheney - Heptio
Lessons learnt building Kubernetes controllers David Cheney - Heptio g day Contour A Kubernetes Ingress Controller Connaissez-vous Kubernetes? Kubernetes in one slide Replicated data store; etcd API
More informationYou can also launch the instances on different machines by supplying IPv4 addresses and port numbers in the format :3410
CS 3410: Paxos Introduction In this assignment, you will implement a simple in-memory database that is replicated across multiple hosts using the Paxos distributed consensis protocol. You can download
More informationBeyond 1001 Dedicated Data Service Instances
Beyond 1001 Dedicated Data Service Instances Introduction The Challenge Given: Application platform based on Cloud Foundry to serve thousands of apps Application Runtime Many platform users - who don
More informationBuilding a Data-Friendly Platform for a Data- Driven Future
Building a Data-Friendly Platform for a Data- Driven Future Benjamin Hindman - @benh 2016 Mesosphere, Inc. All Rights Reserved. INTRO $ whoami BENJAMIN HINDMAN Co-founder and Chief Architect of Mesosphere,
More informationSeptember 15th, Finagle + Java. A love story (
September 15th, 2016 Finagle + Java A love story ( ) @mnnakamura hi, I m Moses Nakamura Twitter lives on the JVM When Twitter realized we couldn t stay on a Rails monolith and continue to scale at the
More informationBest Practices for Developing & Deploying Java Applications with Docker
JavaOne 2017 CON7957 Best Practices for Developing & Deploying Java Applications with Docker Eric Smalling - Solution Architect, Docker Inc. @ericsmalling Who Am I? Eric Smalling Solution Architect Docker
More informationdbdeployer the future of MySQL sandboxes Giuseppe Maxia Software explorer
dbdeployer the future of MySQL sandboxes Giuseppe Maxia Software explorer About me Who's this guy? Giuseppe Maxia, a.k.a. "The Data Charmer" Software Explorer at VMware Several decades development and
More informationOracle NoSQL Database 3.0
Oracle NoSQL Database 3.0 Installation, Cluster Topology Deployment, HA and more Seth Miller, Oracle ACE Robert Greene, Product Management / Strategy Oracle Server Technologies July 09, 2014 Safe Harbor
More informationCMPSC 311- Introduction to Systems Programming Module: Systems Programming
CMPSC 311- Introduction to Systems Programming Module: Systems Programming Professor Patrick McDaniel Fall 2013 Patrick McDaniel Professor of Computer Science and Engineering Co-head of Security Group
More informationUsing Industry Standards to Exploit the Advantages and Resolve the Challenges of Multicore Technology
Using Industry Standards to Exploit the Advantages and Resolve the Challenges of Multicore Technology September 19, 2007 Markus Levy, EEMBC and Multicore Association Enabling the Multicore Ecosystem Multicore
More informationThe Slide does not contain all the information and cannot be treated as a study material for Operating System. Please refer the text book for exams.
The Slide does not contain all the information and cannot be treated as a study material for Operating System. Please refer the text book for exams. Operating System Services User Operating System Interface
More informationMaking Cloud Easy: Components of a Distributed. Design Considerations and First. Operating System for Cloud
Making Cloud Easy: Design Considerations and First Components of a Distributed Operating System for Cloud James Kempf for the Nefele Development Team ER RACSP July 09, 2018 James Kempf ER RACSP 2018-07-09
More informationAgile CI/CD with Jenkins and/at ZeroStack. Kiran Bondalapati CTO, Co-Founder & Jenkins Admin ZeroStack, Inc. (
Agile CI/CD with Jenkins and/at ZeroStack Kiran Bondalapati CTO, Co-Founder & Jenkins Admin ZeroStack, Inc. (www.zerostack.com) Outline ZeroStack Hybrid Cloud Platform Jenkins and ZeroStack Jenkins at
More informationChapter 4: Threads. Chapter 4: Threads
Chapter 4: Threads Silberschatz, Galvin and Gagne 2009 Chapter 4: Threads Overview Multithreading Models Thread Libraries Threading Issues Operating System Examples Windows XP Threads Linux Threads 4.2
More informationECE 550D Fundamentals of Computer Systems and Engineering. Fall 2017
ECE 550D Fundamentals of Computer Systems and Engineering Fall 2017 The Operating System (OS) Prof. John Board Duke University Slides are derived from work by Profs. Tyler Bletsch and Andrew Hilton (Duke)
More informationCS 333 Introduction to Operating Systems. Class 3 Threads & Concurrency. Jonathan Walpole Computer Science Portland State University
CS 333 Introduction to Operating Systems Class 3 Threads & Concurrency Jonathan Walpole Computer Science Portland State University 1 The Process Concept 2 The Process Concept Process a program in execution
More informationWHITE PAPER. RedHat OpenShift Container Platform. Benefits: Abstract. 1.1 Introduction
WHITE PAPER RedHat OpenShift Container Platform Abstract Benefits: Applications are designed around smaller independent components called microservices. Elastic resources: Scale up or down quickly and
More informationLeveraging the Serverless Architecture for Securing Linux Containers
Leveraging the Serverless Architecture for Securing Linux Containers Nilton Bila, Paolo Dettori, Ali Kanso, Yuji Watanabe*, Alaa Youssef IBM T.J. Watson Research Center New York *IBM Research - Tokyo,
More informationWho is Docker and how he can help us? Heino Talvik
Who is Docker and how he can help us? Heino Talvik heino.talvik@seb.ee heino.talvik@gmail.com What is Docker? Software guy view: Marriage of infrastucture and Source Code Management Hardware guy view:
More informationSystems I: Programming Abstractions
Systems I: Programming Abstractions Course Philosophy: The goal of this course is to help students become facile with foundational concepts in programming, including experience with algorithmic problem
More informationKubernetes Integration with Virtuozzo Storage
Kubernetes Integration with Virtuozzo Storage A Technical OCTOBER, 2017 2017 Virtuozzo. All rights reserved. 1 Application Container Storage Application containers appear to be the perfect tool for supporting
More informationOPERATING SYSTEM. Chapter 4: Threads
OPERATING SYSTEM Chapter 4: Threads Chapter 4: Threads Overview Multicore Programming Multithreading Models Thread Libraries Implicit Threading Threading Issues Operating System Examples Objectives To
More informationErlang and Go (CS262a, Berkeley Fall 2016) Philipp Moritz
Erlang and Go (CS262a, Berkeley Fall 2016) Philipp Moritz The Problem Distributed computation is hard! State Hard to do recovery, dependency on order of execution Concurrency and Synchronization Hard to
More informationExercise (could be a quiz) Solution. Concurrent Programming. Roadmap. Tevfik Koşar. CSE 421/521 - Operating Systems Fall Lecture - IV Threads
Exercise (could be a quiz) 1 2 Solution CSE 421/521 - Operating Systems Fall 2013 Lecture - IV Threads Tevfik Koşar 3 University at Buffalo September 12 th, 2013 4 Roadmap Threads Why do we need them?
More informationGo Cheat Sheet. Operators. Go in a Nutshell. Declarations. Basic Syntax. Hello World. Functions. Comparison. Arithmetic. Credits
Credits Go Cheat Sheet Most example code taken from A Tour of Go, which is an excellent introduction to Go. If you're new to Go, do that tour. Seriously. Original HTML Cheat Sheet by Ariel Mashraki (a8m):
More informationPontoon An Enterprise grade serverless framework using Kubernetes Kumar Gaurav, Director R&D, VMware Mageshwaran R, Staff Engineer R&D, VMware
Pontoon An Enterprise grade serverless framework using Kubernetes Kumar Gaurav, Director R&D, VMware Mageshwaran R, Staff Engineer R&D, VMware Serverless: a quick review Enables running back end logic
More informationTDDI04, K. Arvidsson, IDA, Linköpings universitet Operating System Structures. Operating System Structures Overview. Operating System Services
TDDI04 Concurrent Programming, Operating Systems, and Real-time Operating Systems Operating System Structures [SGG7] Chapter 2 Copyright Notice: The lecture notes are mainly based on Silberschatz s, Galvin
More informationLessons learnt building Kubernetes controllers. David Cheney - Heptio
Lessons learnt building Kubernetes controllers David Cheney - Heptio g day Craig McLuckie and Joe Beda 2/3rds of a pod Connaissez-vous Kubernetes? Kubernetes is an open-source system for automating deployment,
More informationRemote Invocation. To do. Request-reply, RPC, RMI. q Today q. q Next time: Indirect communication
Remote Invocation To do q Today q Request-reply, RPC, RMI q Next time: Indirect communication Beyond message passing In DS, all IPC is based on low-level msg passing A bit too low level Modern distributed
More informationOperating System. Operating System Overview. Structure of a Computer System. Structure of a Computer System. Structure of a Computer System
Overview Chapter 1.5 1.9 A program that controls execution of applications The resource manager An interface between applications and hardware The extended machine 1 2 Structure of a Computer System Structure
More informationAgenda Process Concept Process Scheduling Operations on Processes Interprocess Communication 3.2
Lecture 3: Processes Agenda Process Concept Process Scheduling Operations on Processes Interprocess Communication 3.2 Process in General 3.3 Process Concept Process is an active program in execution; process
More information70-532: Developing Microsoft Azure Solutions
70-532: Developing Microsoft Azure Solutions Objective Domain Note: This document shows tracked changes that are effective as of January 18, 2018. Create and Manage Azure Resource Manager Virtual Machines
More informationInterSystems Cloud Manager & Containers for InterSystems Technologies. Luca Ravazzolo Product Manager
InterSystems Cloud Manager & Containers for InterSystems Technologies Luca Ravazzolo Product Manager InterSystems Cloud Manager 1. What is it? 2. How does it work & How do I use it? 3. Why is it interesting?
More information" Qué me estás container?" Docker for dummies
" Qué me estás container?" Docker for dummies Sara Arjona @sara_arjona Pau Ferrer @crazyserver Developer at Moodle HQ Moodle Mobile developer at Moodle HQ #MootES18 Who uses Docker for development? Who
More informationOperating Systems. Lecture 4. Nachos Appetizer
CMSC 321: Operating Systems Lecture 4 Processes & Threads; Nachos Appetizer Before Threads Recall that a process consists of: program(s) data stack PCB all stored in the process image Process (context)
More informationCHAPTER 2: SYSTEM STRUCTURES. By I-Chen Lin Textbook: Operating System Concepts 9th Ed.
CHAPTER 2: SYSTEM STRUCTURES By I-Chen Lin Textbook: Operating System Concepts 9th Ed. Chapter 2: System Structures Operating System Services User Operating System Interface System Calls Types of System
More informationAllowing Users to Run Services at the OLCF with Kubernetes
Allowing Users to Run Services at the OLCF with Kubernetes Jason Kincl Senior HPC Systems Engineer Ryan Adamson Senior HPC Security Engineer This work was supported by the Oak Ridge Leadership Computing
More informationGFS: The Google File System
GFS: The Google File System Brad Karp UCL Computer Science CS GZ03 / M030 24 th October 2014 Motivating Application: Google Crawl the whole web Store it all on one big disk Process users searches on one
More informationProblem Set: Processes
Lecture Notes on Operating Systems Problem Set: Processes 1. Answer yes/no, and provide a brief explanation. (a) Can two processes be concurrently executing the same program executable? (b) Can two running
More informationCSCC24 Functional Programming Scheme Part 2
CSCC24 Functional Programming Scheme Part 2 Carolyn MacLeod 1 winter 2012 1 Based on slides from Anya Tafliovich, and with many thanks to Gerald Penn and Prabhakar Ragde. 1 The Spirit of Lisp-like Languages
More informationDemystifying the Cloud With a Look at Hybrid Hosting and OpenStack
Demystifying the Cloud With a Look at Hybrid Hosting and OpenStack Robert Collazo Systems Engineer Rackspace Hosting The Rackspace Vision Agenda Truly a New Era of Computing 70 s 80 s Mainframe Era 90
More informationIntroduction to pthreads
CS 220: Introduction to Parallel Computing Introduction to pthreads Lecture 25 Threads In computing, a thread is the smallest schedulable unit of execution Your operating system has a scheduler that decides
More informationAn introduction to Docker
An introduction to Docker Ing. Vincenzo Maffione Operating Systems Security Container technologies on Linux Several light virtualization technologies are available for Linux They build on cgroups, namespaces
More informationgo get my/vulnerabilities Green threads are not eco friendly threads
go get my/vulnerabilities Green threads are not eco friendly threads 1 Who ( Web Mobile ) penetration tester Code reviewer Programmer Roberto Clapis @empijei 2 Go Google s language Born in 2007 (quite
More informationRio-2 Hybrid Backup Server
A Revolution in Data Storage for Today s Enterprise March 2018 Notices This white paper provides information about the as of the date of issue of the white paper. Processes and general practices are subject
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
CNA1612BU Deploying real-world workloads on Kubernetes and Pivotal Cloud Foundry VMworld 2017 Fred Melo, Director of Technology, Pivotal Merlin Glynn, Sr. Technical Product Manager, VMware Content: Not
More informationLast class: Today: Thread Background. Thread Systems
1 Last class: Thread Background Today: Thread Systems 2 Threading Systems 3 What kind of problems would you solve with threads? Imagine you are building a web server You could allocate a pool of threads,
More informationunitrends.com MYTH VS. REALITY What you need to know about Microsoft Office 365 Backup
MYTH VS. REALITY What you need to know about Microsoft Office 365 Backup Is Your Office 365 Data Fully Protected? ebook Contents Introduction /1 How data loss occurs in cloud applications Protecting Email:
More informationOperating Systems (2INC0) 2018/19. Introduction (01) Dr. Tanir Ozcelebi. Courtesy of Prof. Dr. Johan Lukkien. System Architecture and Networking Group
Operating Systems (2INC0) 20/19 Introduction (01) Dr. Courtesy of Prof. Dr. Johan Lukkien System Architecture and Networking Group Course Overview Introduction to operating systems Processes, threads and
More informationApplication Centric Microservices Ken Owens, CTO Cisco Intercloud Services. Redhat Summit 2015
Application Centric Microservices Ken Owens, CTO Cisco Intercloud Services Redhat Summit 2015 Agenda Introduction Why Application Centric Application Deployment Options What is Microservices Infrastructure
More informationMultithreading and Interactive Programs
Multithreading and Interactive Programs CS160: User Interfaces John Canny. Last time Model-View-Controller Break up a component into Model of the data supporting the App View determining the look of the
More informationContainer Adoption for NFV Challenges & Opportunities. Sriram Natarajan, T-Labs Silicon Valley Innovation Center
Container Adoption for NFV Challenges & Opportunities Sriram Natarajan, T-Labs Silicon Valley Innovation Center Virtual Machine vs. Container Stack KVM Container-stack Libraries Guest-OS Hypervisor Libraries
More informationRunning Databases in Containers.
Running Databases in Containers. How to Overcome the Challenges of Data Frank Stienhans CTO Prepared for Evolution of Enterprise IT Subjective Perspective CONTAINERS 1. More Choices CLOUD 2. Faster Delivery
More informationGo for Java Developers
Go for Java Developers Stoyan Rachev May 26-27 16, Sofia 1 Agenda Introduction Variables and Control Flow Types and Data Structures Functions Methods and Interfaces Concurrency Conclusion 2 What is Go?
More informationServerless in the Java ecosystem
Serverless in the Java ecosystem Pratik Patel Pratik PateL CTO Triplingo Java Champion JavaScript Troublemaker Python Hacker Founder, PERL recovery group WHAT IS SERVERLESS? ARCHITECTURE ECOSYSTEM SERVERLESS
More informationMerging Enterprise Applications with Docker* Container Technology
Solution Brief NetApp Docker Volume Plugin* Intel Xeon Processors Intel Ethernet Converged Network Adapters Merging Enterprise Applications with Docker* Container Technology Enabling Scale-out Solutions
More informationCMPE-655 Fall 2013 Assignment 2: Parallel Implementation of a Ray Tracer
CMPE-655 Fall 2013 Assignment 2: Parallel Implementation of a Ray Tracer Rochester Institute of Technology, Department of Computer Engineering Instructor: Dr. Shaaban (meseec@rit.edu) TAs: Jason Lowden
More informationChapter 4: Threads. Overview Multithreading Models Thread Libraries Threading Issues Operating System Examples Windows XP Threads Linux Threads
Chapter 4: Threads Overview Multithreading Models Thread Libraries Threading Issues Operating System Examples Windows XP Threads Linux Threads Chapter 4: Threads Objectives To introduce the notion of a
More informationRemote Procedure Call
Remote Procedure Call Remote Procedure Call Integrate network communication with programming language Procedure call is well understood implementation use Control transfer Data transfer Goals Easy make
More informationArmon HASHICORP
Nomad Armon Dadgar @armon Distributed Optimistically Concurrent Scheduler Nomad Distributed Optimistically Concurrent Scheduler Nomad Schedulers map a set of work to a set of resources Work (Input) Resources
More informationCNA1699BU Running Docker on your Existing Infrastructure with vsphere Integrated Containers Martijn Baecke Patrick Daigle VMworld 2017 Content: Not fo
CNA1699BU Running Docker on your Existing Infrastructure with vsphere Integrated Containers VMworld 2017 Content: Not for publication #VMworld #CNA1699BU CNA1699BU Running Docker on your Existing Infrastructure
More informationPluggable Rule Engine Architecture
Pluggable Rule Engine Architecture ABSTRACT Hao Xu DICE Center University of North Carolina at Chapel Hill, NC 27599, USA xuh@email.unc.edu Jason Coposky Renaissance Computing Institute (RENCI) 100 Europa
More informationAchieving Horizontal Scalability. Alain Houf Sales Engineer
Achieving Horizontal Scalability Alain Houf Sales Engineer Scale Matters InterSystems IRIS Database Platform lets you: Scale up and scale out Scale users and scale data Mix and match a variety of approaches
More informationRemote Procedure Calls (RPC)
Distributed Computing Remote Procedure Calls (RPC) Dr. Yingwu Zhu Problems with Sockets Sockets interface is straightforward [connect] read/write [disconnect] BUT it forces read/write mechanism We usually
More informationLogging, Monitoring, and Alerting
Logging, Monitoring, and Alerting Logs are a part of daily life in the DevOps world In security, we focus on particular logs to detect security anomalies and for forensic capabilities A basic logging pipeline
More informationGo Tutorial. To do. A brief, gentle intro to Go. Next Networking. q Today
Go Tutorial To do q Today A brief, gentle intro to Go q Next Networking About Go Developed by Google Webpage: https://golang.org/ Concurrency was a priority in the language design A bit of a mix between
More informationChapter 4: Threads. Chapter 4: Threads. Overview Multicore Programming Multithreading Models Thread Libraries Implicit Threading Threading Issues
Chapter 4: Threads Silberschatz, Galvin and Gagne 2013 Chapter 4: Threads Overview Multicore Programming Multithreading Models Thread Libraries Implicit Threading Threading Issues 4.2 Silberschatz, Galvin
More informationChapter 2: Operating-System Structures
Chapter 2: Operating-System Structures Chapter 2: Operating-System Structures Operating System Services User Operating System Interface System Calls Types of System Calls System Programs Operating System
More informationDistributed Systems. How do regular procedure calls work in programming languages? Problems with sockets RPC. Regular procedure calls
Problems with sockets Distributed Systems Sockets interface is straightforward [connect] read/write [disconnect] Remote Procedure Calls BUT it forces read/write mechanism We usually use a procedure call
More informationObject Interaction. Object Interaction. Introduction. Object Interaction vs. RPCs (2)
Introduction Objective To support interoperability and portability of distributed OO applications by provision of enabling technology Object interaction vs RPC Java Remote Method Invocation (RMI) RMI Registry
More informationGFS: The Google File System. Dr. Yingwu Zhu
GFS: The Google File System Dr. Yingwu Zhu Motivating Application: Google Crawl the whole web Store it all on one big disk Process users searches on one big CPU More storage, CPU required than one PC can
More informationProfessional Multicore Programming. Design and Implementation for C++ Developers
Professional Multicore Programming Design and Implementation for C++ Developers Cameron Hughes Tracey Hughes WILEY Wiley Publishing, Inc. Introduction xxi Chapter 1: The New Architecture 1 What Is a Multicore?
More informationpblk the OCSSD FTL Linux FAST Summit 18 Javier González Copyright 2018 CNEX Labs
pblk the OCSSD FTL Linux FAST Summit 18 Javier González Read Latency Read Latency with 0% Writes Random Read 4K Percentiles 2 Read Latency Read Latency with 20% Writes Random Read 4K + Random Write 4K
More informationFour times Microservices: REST, Kubernetes, UI Integration, Async. Eberhard Fellow
Four times Microservices: REST, Kubernetes, UI Integration, Async Eberhard Wolff @ewolff http://ewolff.com Fellow http://continuous-delivery-buch.de/ http://continuous-delivery-book.com/ http://microservices-buch.de/
More informationAmazon EC2 Container Service: Manage Docker-Enabled Apps in EC2
Amazon EC2 Container Service: Manage Docker-Enabled Apps in EC2 Ian Massingham AWS Technical Evangelist @IanMmmm 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Agenda Containers
More information