Design of MPI Passive Target Synchronization for a Non-Cache- Coherent Many-Core Processor
|
|
|
- Dora Houston
- 5 years ago
- Views:
Transcription
1 Design of MPI Passive Target Synchronization for a Non-Cache- Coherent Many-Core Processor 27th PARS Workshop, Hagen, Germany, May Steffen Christgau, Bettina Schnor Operating Systems and Distributed Systems Institute for Computer Science University of Potsdam, Germany
2 Motivation: Distributed Hash Table (DHT) hash table as cache for computational results in MPI application PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 1 / 14
3 Motivation: Distributed Hash Table (DHT) hash table as cache for computational results in MPI application large amount of data distribute across processes DHT PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 1 / 14
4 Motivation: Distributed Hash Table (DHT) hash table as cache for computational results in MPI application large amount of data distribute across processes DHT local DHT part local DHT part local DHT part DHT rank 0 rank 1... rank n 1 PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 1 / 14
5 Motivation: Distributed Hash Table (DHT) hash table as cache for computational results in MPI application large amount of data distribute across processes DHT local DHT part local DHT part local DHT part DHT rank 0 rank 1... rank n 1 accessing distributed data: hash function returns arbitrary process and address difficult to program with two-sided message passing MPI passive target one-sided communication to the rescue synchronization required PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 1 / 14
6 Motivation: ncc Systems Future many-cores may not provide (global) cache coherence. PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 2 / 14
7 Motivation: ncc Systems Future many-cores may not provide (global) cache coherence. Intel Knights Landing: coherent multi-socket systems not feasible PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 2 / 14
8 Motivation: ncc Systems Future many-cores may not provide (global) cache coherence. Intel Knights Landing: coherent multi-socket systems not feasible HPE "The Machine", EuroServer: coherence islands PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 2 / 14
9 Research Platform ncc many-core research system: Intel SCC 48 Pentium cores with L1/2 caches no HW cache coherence L2$ MC 2 MC 3 MIU L2$ R Tile Core MPB Core MC 0 MC 1 PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 3 / 14
10 Research Platform ncc many-core research system: Intel SCC 48 Pentium cores with L1/2 caches no HW cache coherence L2$ MC 2 MC 3 MIU L2$ R Tile Core MPB Core MC 0 MC 1 This talk: design of synchronization on ncc platform. PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 3 / 14
11 Agenda MPI Passive Target One-Sided Communication Design for Passive Target Synchronization on the SCC Data Structures and Algorithms Data Placement Outlook and Future Work PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 4 / 14
12 MPI One-Sided Communication process memory exposed via windows process address space local DHT part rank 0 local DHT part rank 1... local DHT part rank n 1 DHT PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 5 / 14
13 MPI One-Sided Communication process memory exposed via windows process address space local DHT part local DHT part local DHT part (window) (window) (window) rank 0 rank 1... rank n 1 DHT PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 5 / 14
14 MPI One-Sided Communication process memory exposed via windows access to windows with window object (handle) process address space window object window object window object local DHT part local DHT part local DHT part (window) (window) (window) rank 0 rank 1... rank n 1 DHT PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 5 / 14
15 MPI One-Sided Communication process memory exposed via windows access to windows with window object (handle) process address space window object window object window object local DHT part local DHT part local DHT part (window) (window) (window) rank 0 rank 1... rank n 1 DHT key concept: only one communication partner issues communication operations PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 5 / 14
16 MPI One-Sided Communication process memory exposed via windows access to windows with window object (handle) process address space window object window object window object local DHT part local DHT part local DHT part (window) (window) (window) rank 0 rank 1... rank n 1 DHT key concept: only one communication partner issues communication operations origin processes issue communication operations PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 5 / 14
17 MPI One-Sided Communication process memory exposed via windows access to windows with window object (handle) process address space window object window object window object local DHT part local DHT part local DHT part (window) (window) (window) rank 0 rank 1... rank n 1 DHT key concept: only one communication partner issues communication operations origin processes issue communication operations target processes are addressed by operations PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 5 / 14
18 MPI One-Sided Communication process memory exposed via windows access to windows with window object (handle) process address space window object window object window object local DHT part local DHT part local DHT part (window) (window) (window) rank 0 rank 1... rank n 1 DHT key concept: only one communication partner issues communication operations origin processes issue communication operations target processes are addressed by operations typical RMA operations: PUT, GET,... PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 5 / 14
19 MPI One-Sided Communication process memory exposed via windows access to windows with window object (handle) process address space window object window object window object local DHT part local DHT part local DHT part (window) (window) (window) rank 0 rank 1... rank n 1 DHT key concept: only one communication partner issues communication operations origin processes issue communication operations target processes are addressed by operations typical RMA operations: PUT, GET,... explicit synchronization required PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 5 / 14
20 MPI Passive Target Synchronization locks as means for synchronization, used by origins only no participation of targets in synchronization (passive targets) PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 6 / 14
21 MPI Passive Target Synchronization locks as means for synchronization, used by origins only no participation of targets in synchronization (passive targets) usage similar to shared memory locks 1. acquire lock for target window WIN_LOCK(win, rank,...) 2. perform operations PUT(win, rank,...) 3. release lock WIN_UNLOCK(win, rank) PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 6 / 14
22 MPI Passive Target Synchronization locks as means for synchronization, used by origins only no participation of targets in synchronization (passive targets) usage similar to shared memory locks 1. acquire lock for target window WIN_LOCK(win, rank,...) 2. perform operations PUT(win, rank,...) 3. release lock WIN_UNLOCK(win, rank) MPI defines two lock types: shared concurrent accesses on target window allowed exclusive prevent concurrent accesses on same target window PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 6 / 14
23 Distributed Hash Table with MPI OSC process address space window object local DHT part (window) rank 0 window object local DHT part (window) rank 1... window object local DHT part (window) rank n 1 DHT PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 7 / 14
24 Distributed Hash Table with MPI OSC process address space window object local DHT part (window) rank 0 window object local DHT part (window) rank 1... window object local DHT part (window) rank n 1 DHT DHT_read LOCK(window_obj, target, SHARED) GET(window_obj, target, &data) UNLOCK(window_obj, target) PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 7 / 14
25 Distributed Hash Table with MPI OSC process address space window object local DHT part (window) rank 0 window object local DHT part (window) rank 1... window object local DHT part (window) rank n 1 DHT DHT_read LOCK(window_obj, target, SHARED) GET(window_obj, target, &data) UNLOCK(window_obj, target) DHT_write LOCK(window_obj, target, EXCLUSIVE) PUT(window_obj, target, data) UNLOCK(window_obj, target) PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 7 / 14
26 Synchronization for the DHT observation: high latency for synchronization in SCC s MPI previous work (PASA 2016): 5x lower latency with shared memory and uncached accesses instead of messages PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 8 / 14
27 Synchronization for the DHT observation: high latency for synchronization in SCC s MPI previous work (PASA 2016): 5x lower latency with shared memory and uncached accesses instead of messages synchronization semantics undefined by MPI: "much freedom for implementors" PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 8 / 14
28 Synchronization for the DHT observation: high latency for synchronization in SCC s MPI previous work (PASA 2016): 5x lower latency with shared memory and uncached accesses instead of messages synchronization semantics undefined by MPI: "much freedom for implementors" assumption: (far) more DHT reads than writes Readers & Writers Synchronization (Courtois et al.) advantageous writer precedence recent data for readers PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 8 / 14
29 Synchronization for the DHT observation: high latency for synchronization in SCC s MPI previous work (PASA 2016): 5x lower latency with shared memory and uncached accesses instead of messages synchronization semantics undefined by MPI: "much freedom for implementors" assumption: (far) more DHT reads than writes Readers & Writers Synchronization (Courtois et al.) advantageous writer precedence recent data for readers design of MPI passive target synchronization scheme with R&W semantics for SCC PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 8 / 14
30 Data Structures for Synchronization use lock data structure as proposed by Mellor-Crummey/Scott ( 91) shared memory lock L 0 writer queue reader queue state window rank 0 rank 1 rank 2 PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 9 / 14
31 Data Structures for Synchronization use lock data structure as proposed by Mellor-Crummey/Scott ( 91) distributed lists of waiting readers and writers shared memory blocked = 0 blocked = 1 blocked = 1 lock L 0 writer queue reader queue state window rank 0 rank 1 rank 2 PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 9 / 14
32 Data Structures for Synchronization use lock data structure as proposed by Mellor-Crummey/Scott ( 91) distributed lists of waiting readers and writers no centralized object to spin on (avoid memory contention) instead: per-process list entry for spinning shared memory blocked = 0 blocked = 1 blocked = 1 lock L 0 writer queue reader queue state window rank 0 rank 1 rank 2 PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 9 / 14
33 Data Structures for Synchronization use lock data structure as proposed by Mellor-Crummey/Scott ( 91) distributed lists of waiting readers and writers no centralized object to spin on (avoid memory contention) instead: per-process list entry for spinning state variable: counts active/interested readers/writers shared memory blocked = 0 blocked = 1 blocked = 1 lock L 0 writer queue reader queue state window rank 0 rank 1 rank 2 PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 9 / 14
34 Data Structures for Synchronization use lock data structure as proposed by Mellor-Crummey/Scott ( 91) distributed lists of waiting readers and writers no centralized object to spin on (avoid memory contention) instead: per-process list entry for spinning state variable: counts active/interested readers/writers one lock variable per process and window shared memory blocked = 0 blocked = 1 blocked = 1 lock L 0 writer queue reader queue state lock L 1 writer queue reader queue state lock L 2 writer queue reader queue state window window window rank 0 rank 1 rank 2 PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 9 / 14
35 Synchronization Operations according to Mellor-Crummey/Scott processes enter either list of readers or writers Readers start_read blocks as long as writers are active or waiting, allows multiple active readers PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 10 / 14
36 Synchronization Operations according to Mellor-Crummey/Scott processes enter either list of readers or writers Readers start_read blocks as long as writers are active or waiting, allows multiple active readers end_read wake first waiting writer if no active reader left PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 10 / 14
37 Synchronization Operations according to Mellor-Crummey/Scott processes enter either list of readers or writers Readers start_read blocks as long as writers are active or waiting, allows multiple active readers end_read wake first waiting writer if no active reader left Writers start_write blocks when readers are active PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 10 / 14
38 Synchronization Operations according to Mellor-Crummey/Scott processes enter either list of readers or writers Readers start_read blocks as long as writers are active or waiting, allows multiple active readers end_read wake first waiting writer if no active reader left Writers start_write blocks when readers are active end_write wake up next writer (if any) or all waiting readers PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 10 / 14
39 R&W Synchronization inside MPI Library MPI_Win_lock(type, target_rank, win_obj) { entry = alloc_list_entry(); } win_obj.entry[target_rank] = entry; win_obj.entry[target_rank].lock_type = type; if (type == SHARED) start_read(win_obj.lock[target_rank], entry); else start_write(win_obj.lock[target_rank], entry); PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 11 / 14
40 R&W Synchronization inside MPI Library MPI_Win_lock(type, target_rank, win_obj) { entry = alloc_list_entry(); } win_obj.entry[target_rank] = entry; win_obj.entry[target_rank].lock_type = type; if (type == SHARED) start_read(win_obj.lock[target_rank], entry); else start_write(win_obj.lock[target_rank], entry); unlock operation straight forward PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 11 / 14
41 Data Placement synchronization data located in shared memory danger of contention on memory interface PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 12 / 14
42 Data Placement synchronization data located in shared memory danger of contention on memory interface speedup of memory-bound application with different synchronization data locations: speedup distributed controller 3 controller 2 controller 1 controller number of MPI processes PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 12 / 14
43 Data Placement synchronization data located in shared memory danger of contention on memory interface speedup of memory-bound application with different synchronization data locations: speedup distributed controller 3 controller 2 controller 1 controller number of MPI processes bring spinning object close to process/core allocate list entry in closest memory controller local uncached spinning PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 12 / 14
44 Discussion design characteristics: concurrent window access: one lock per window and process per-window Readers & Writers semantic contention avoidance: spin on local object only truly passive: no participation of the remote process in synchronization operations and communication Christgau, Schnor: Exploring One-Sided Communication and Synchronization on a non-cache-coherent Many-Core Architecture. Concurrency and Computation: Practice and Experience PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 13 / 14
45 Summary and Outlook Summary presented design for implementing MPI passive target synchronization on ncc many-core applied concepts from Mellor-Crummey/Scott to ncc processor distributed data structures critical PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 14 / 14
46 Summary and Outlook Summary presented design for implementing MPI passive target synchronization on ncc many-core applied concepts from Mellor-Crummey/Scott to ncc processor distributed data structures critical Future Work implement the presented scheme evaluate performance by comparison against message-based implementation and other designs PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 14 / 14
47 Summary and Outlook Summary presented design for implementing MPI passive target synchronization on ncc many-core applied concepts from Mellor-Crummey/Scott to ncc processor distributed data structures critical Future Work implement the presented scheme evaluate performance by comparison against message-based implementation and other designs Questions!? PARS 2017 S. Christgau (U Potsdam): MPI Passive Target Synchronization 14 / 14
High-Performance Distributed RMA Locks
High-Performance Distributed RMA Locks PATRICK SCHMID, MACIEJ BESTA, TORSTEN HOEFLER NEED FOR EFFICIENT LARGE-SCALE SYNCHRONIZATION spcl.inf.ethz.ch LOCKS An example structure Proc p Proc q Inuitive semantics
CSE 153 Design of Operating Systems
CSE 153 Design of Operating Systems Winter 2018 Lecture 10: Monitors Monitors A monitor is a programming language construct that controls access to shared data Synchronization code added by compiler, enforced
Enforcing Mutual Exclusion Using Monitors
Enforcing Mutual Exclusion Using Monitors Mutual Exclusion Requirements Mutually exclusive access to critical section Progress. If no process is executing in its critical section and there exist some processes
Operating Systems ECE344
Operating Systems ECE344 Ding Yuan Announcement & Reminder Lab 0 mark posted on Piazza Great job! One problem: compilation error I fixed some for you this time, but won t do it next time Make sure you
Opera&ng Systems ECE344
Opera&ng Systems ECE344 Lecture 6: Synchroniza&on (II) Semaphores and Monitors Ding Yuan Higher- Level Synchroniza&on We looked at using locks to provide mutual exclusion Locks work, but they have some
High-Performance Distributed RMA Locks
High-Performance Distributed RMA Locks PATRICK SCHMID, MACIEJ BESTA, TORSTEN HOEFLER Presented at ARM Research, Austin, Texas, USA, Feb. 2017 NEED FOR EFFICIENT LARGE-SCALE SYNCHRONIZATION spcl.inf.ethz.ch
Evaluation and Improvements of Programming Models for the Intel SCC Many-core Processor
Evaluation and Improvements of Programming Models for the Intel SCC Many-core Processor Carsten Clauss, Stefan Lankes, Pablo Reble, Thomas Bemmerl International Workshop on New Algorithms and Programming
CSE 120 Principles of Operating Systems Spring 2016
CSE 120 Principles of Operating Systems Spring 2016 Condition Variables and Monitors Monitors A monitor is a programming language construct that controls access to shared data Synchronization code added
Understanding Hardware Transactional Memory
Understanding Hardware Transactional Memory Gil Tene, CTO & co-founder, Azul Systems @giltene 2015 Azul Systems, Inc. Agenda Brief introduction What is Hardware Transactional Memory (HTM)? Cache coherence
Advance Operating Systems (CS202) Locks Discussion
Advance Operating Systems (CS202) Locks Discussion Threads Locks Spin Locks Array-based Locks MCS Locks Sequential Locks Road Map Threads Global variables and static objects are shared Stored in the static
Threads. Concurrency. What it is. Lecture Notes Week 2. Figure 1: Multi-Threading. Figure 2: Multi-Threading
Threads Figure 1: Multi-Threading Figure 2: Multi-Threading Concurrency What it is 1. Two or more threads of control access a shared resource. Scheduler operation must be taken into account fetch-decode-execute-check
Outline for Today. 5 Dining Philosophers. Template for Philosopher. Naive Solution. Objective: Administrative details:
Outline for Today Objective: 5 Dining Philosophers Reader-writer problem Message Passing Administrative details: Check for demo location with grader TA s and me: come to our offices UTA s if they don t
Spinlocks. Spinlocks. Message Systems, Inc. April 8, 2011
Spinlocks Samy Al Bahra Devon H. O Dell Message Systems, Inc. April 8, 2011 Introduction Mutexes A mutex is an object which implements acquire and relinquish operations such that the execution following
Advanced Topic: Efficient Synchronization
Advanced Topic: Efficient Synchronization Multi-Object Programs What happens when we try to synchronize across multiple objects in a large program? Each object with its own lock, condition variables Is
Lecture 26: Multiprocessing continued Computer Architecture and Systems Programming ( )
Systems Group Department of Computer Science ETH Zürich Lecture 26: Multiprocessing continued Computer Architecture and Systems Programming (252-0061-00) Timothy Roscoe Herbstsemester 2012 Today Non-Uniform
Today. SMP architecture. SMP architecture. Lecture 26: Multiprocessing continued Computer Architecture and Systems Programming ( )
Lecture 26: Multiprocessing continued Computer Architecture and Systems Programming (252-0061-00) Timothy Roscoe Herbstsemester 2012 Systems Group Department of Computer Science ETH Zürich SMP architecture
Concurrent Programming Issues & Readers/Writers
Concurrent Programming Issues & Readers/Writers 1 Summary of Our Discussions! Developing and debugging concurrent programs is hard Ø Non-deterministic interleaving of instructions! Safety: isolation and
CS 318 Principles of Operating Systems
CS 318 Principles of Operating Systems Fall 2018 Lecture 7: Semaphores and Monitors Ryan Huang Slides adapted from Geoff Voelker s lectures Administrivia HW2 is out Do the exercise to check your understanding
Locks and Condition Variables Recap. Introducing Monitors. Hoare Monitors: Semantics. Coke Machine Example. Locks. Condition variables
Introducing Monitors Separate the concerns of mutual exclusion and conditional synchronization What is a monitor? " One lock, and " Zero or more condition variables for managing concurrent access to shared
CSE 120 Principles of Operating Systems Spring 2016
CSE 120 Principles of Operating Systems Spring 2016 Semaphores and Monitors Higher-Level Synchronization We looked at using locks to provide mutual exclusion Locks work, but they have limited semantics
CS 318 Principles of Operating Systems
CS 318 Principles of Operating Systems Fall 2017 Lecture 7: Semaphores and Monitors Ryan Huang Higher-Level Synchronization We looked at using locks to provide mutual exclusion Locks work, but they have
Evaluation of Hardware Synchronization Support of the SCC Many-Core Processor
Evaluation of Hardware Synchronization Support of the SCC Many-Core Processor Pablo eble, Stefan Lankes, Florian Zeitz and Thomas Bemmerl Chair for Operating Systems, WTH Aachen University, Germany {reble,lankes,zeitz,bemmerl}@lfbs.rwth-aachen.de
Outline for Today. Readers/Writers Problem
Objective: Outline for Today Reader-writer problem Message Passing Administrative details: Check that you know where your demo will be For me, Microsoft Lab on 2 nd floor LSRC. 1 Readers/Writers Problem
Efficient and Truly Passive MPI-3 RMA Synchronization Using InfiniBand Atomics
1 Efficient and Truly Passive MPI-3 RMA Synchronization Using InfiniBand Atomics Mingzhe Li Sreeram Potluri Khaled Hamidouche Jithin Jose Dhabaleswar K. Panda Network-Based Computing Laboratory Department
CS377P Programming for Performance Multicore Performance Cache Coherence
CS377P Programming for Performance Multicore Performance Cache Coherence Sreepathi Pai UTCS October 26, 2015 Outline 1 Cache Coherence 2 Cache Coherence Awareness 3 Scalable Lock Design 4 Transactional
CS4410, Fall 2016 Homework 2 Deadline: 09/28/2016, 11:59AM
Instructions for Homework 2: This is the second k out of n homeworks CS 4410. The homework may be done in pairs, or individually. If doing in pairs, one of you should upload to gradescope and add your
Message Passing Improvements to Shared Address Space Thread Synchronization Techniques DAN STAFFORD, ROBERT RELYEA
Message Passing Improvements to Shared Address Space Thread Synchronization Techniques DAN STAFFORD, ROBERT RELYEA Agenda Background Motivation Remote Memory Request Shared Address Synchronization Remote
Semaphores and Monitors: High-level Synchronization Constructs
1 Synchronization Constructs Synchronization Coordinating execution of multiple threads that share data structures Semaphores and Monitors High-level Synchronization Constructs A Historical Perspective
Towards scalable RDMA locking on a NIC
TORSTEN HOEFLER spcl.inf.ethz.ch Towards scalable RDMA locking on a NIC with support of Patrick Schmid, Maciej Besta, Salvatore di Girolamo @ SPCL presented at HP Labs, Palo Alto, CA, USA NEED FOR EFFICIENT
Multiprocessors II: CC-NUMA DSM. CC-NUMA for Large Systems
Multiprocessors II: CC-NUMA DSM DSM cache coherence the hardware stuff Today s topics: what happens when we lose snooping new issues: global vs. local cache line state enter the directory issues of increasing
High-Performance Distributed RMA Locks
TORSTEN HOEFLER High-Performance Distributed RMA Locks with support of Patrick Schmid, Maciej Besta @ SPCL presented at Wuxi, China, Sept. 2016 ETH, CS, Systems Group, SPCL ETH Zurich top university in
CS 471 Operating Systems. Yue Cheng. George Mason University Fall 2017
CS 471 Operating Systems Yue Cheng George Mason University Fall 2017 1 Review: Sync Terminology Worksheet 2 Review: Semaphores 3 Semaphores o Motivation: Avoid busy waiting by blocking a process execution
Synchronization for Concurrent Tasks
Synchronization for Concurrent Tasks Minsoo Ryu Department of Computer Science and Engineering 2 1 Race Condition and Critical Section Page X 2 Algorithmic Approaches Page X 3 Hardware Support Page X 4
Important Lessons. A Distributed Algorithm (2) Today's Lecture - Replication
Important Lessons Lamport & vector clocks both give a logical timestamps Total ordering vs. causal ordering Other issues in coordinating node activities Exclusive access to resources/data Choosing a single
Two Types of Semaphores
Two Types of Semaphores Counting semaphore integer value can range over an unrestricted domain. Binary semaphore integer value can range only between 0 and 1; can be simpler to implement. Can implement
CPSC/ECE 3220 Summer 2018 Exam 2 No Electronics.
CPSC/ECE 3220 Summer 2018 Exam 2 No Electronics. Name: Write one of the words or terms from the following list into the blank appearing to the left of the appropriate definition. Note that there are more
Example of use. Shared var mutex: semaphore = 1; Process i. begin.. P(mutex); execute CS; V(mutex);.. End;
Example of use Shared var mutex: semaphore = 1; Process i P(mutex); execute CS; V(mutex); End; Other synchronization problems Semaphore can be used in other synchronization problems besides Mutual Exclusion
Utilizing Linux Kernel Components in K42 K42 Team modified October 2001
K42 Team modified October 2001 This paper discusses how K42 uses Linux-kernel components to support a wide range of hardware, a full-featured TCP/IP stack and Linux file-systems. An examination of the
Distributed Operating Systems
Distributed Operating Systems Synchronization in Parallel Systems Marcus Völp 2009 1 Topics Synchronization Locking Analysis / Comparison Distributed Operating Systems 2009 Marcus Völp 2 Overview Introduction
Lecture 19: Coherence and Synchronization. Topics: synchronization primitives (Sections )
Lecture 19: Coherence and Synchronization Topics: synchronization primitives (Sections 5.4-5.5) 1 Caching Locks Spin lock: to acquire a lock, a process may enter an infinite loop that keeps attempting
Concurrency. Chapter 5
Concurrency 1 Chapter 5 2 Concurrency Is a fundamental concept in operating system design Processes execute interleaved in time on a single processor Creates the illusion of simultaneous execution Benefits
CSE 120. Fall Lecture 6: Semaphores. Keith Marzullo
CSE 120 Principles of Operating Systems Fall 2007 Lecture 6: Semaphores Keith Marzullo Announcements Homework #2 out Homework #1 almost graded... Discussion session on Wednesday will entertain questions
Chapter 6: Process Synchronization
Chapter 6: Process Synchronization Objectives Introduce Concept of Critical-Section Problem Hardware and Software Solutions of Critical-Section Problem Concept of Atomic Transaction Operating Systems CS
main read Identical static threads Dynamic threads with Concurrent and
CHAPTER 3: CONCURRENT PROCESSES AND PROGRAMMING Chapter outline Thread implementations Process models The client/server model Time services Language constructs for synchronization Concurrent programming
Software and Tools for HPE s The Machine Project
Labs Software and Tools for HPE s The Machine Project Scalable Tools Workshop Aug/1 - Aug/4, 2016 Lake Tahoe Milind Chabbi Traditional Computing Paradigm CPU DRAM CPU DRAM CPU-centric computing 2 CPU-Centric
Advanced Operating Systems (CS 202)
Advanced Operating Systems (CS 202) Memory Consistency, Cache Coherence and Synchronization (Part II) Jan, 30, 2017 (some cache coherence slides adapted from Ian Watson; some memory consistency slides
Operating Systems (2INC0) 2017/18
Operating Systems (2INC0) 2017/18 Condition Synchronization (07) Dr. Courtesy of Prof. Dr. Johan Lukkien System Architecture and Networking Group Agenda Condition synchronization motivation condition variables
Concurrency: Deadlock and Starvation
Concurrency: Deadlock and Starvation Chapter 6 E&CE 354: Processes 1 Deadlock Deadlock = situation in which every process from a set is permanently blocked, i.e. cannot proceed with execution Common cause:
MULTI-THREADED QUERIES
15-721 Project 3 Final Presentation MULTI-THREADED QUERIES Wendong Li (wendongl) Lu Zhang (lzhang3) Rui Wang (ruiw1) Project Objective Intra-operator parallelism Use multiple threads in a single executor
5 Dining Philosophers. Template for Philosopher. Naive Solution. while (food available) /*pick up forks*/ eat; /*put down forks*/ think awhile;
5 Dining Philosophers Philosopher 4 Philosopher 3 Philosopher 0 while(food available) pick up 2 adj. forks; put down forks; Philosopher 1 Philosopher 2 90 Template for Philosopher /*pick up forks*/ /*put
Parallelization and Synchronization. CS165 Section 8
Parallelization and Synchronization CS165 Section 8 Multiprocessing & the Multicore Era Single-core performance stagnates (breakdown of Dennard scaling) Moore s law continues use additional transistors
Adaptive Lock. Madhav Iyengar < >, Nathaniel Jeffries < >
Adaptive Lock Madhav Iyengar < miyengar@andrew.cmu.edu >, Nathaniel Jeffries < njeffrie@andrew.cmu.edu > ABSTRACT Busy wait synchronization, the spinlock, is the primitive at the core of all other synchronization
Systèmes d Exploitation Avancés
Systèmes d Exploitation Avancés Instructor: Pablo Oliveira ISTY Instructor: Pablo Oliveira (ISTY) Systèmes d Exploitation Avancés 1 / 32 Review : Thread package API tid thread create (void (*fn) (void
Review of last lecture. Peer Quiz. DPHPC Overview. Goals of this lecture. Lock-based queue
Review of last lecture Design of Parallel and High-Performance Computing Fall 2016 Lecture: Linearizability Motivational video: https://www.youtube.com/watch?v=qx2driqxnbs Instructor: Torsten Hoefler &
1995 Paper 10 Question 7
995 Paper 0 Question 7 Why are multiple buffers often used between producing and consuming processes? Describe the operation of a semaphore. What is the difference between a counting semaphore and a binary
Concept of a process
Concept of a process In the context of this course a process is a program whose execution is in progress States of a process: running, ready, blocked Submit Ready Running Completion Blocked Concurrent
Linux kernel synchronization. Don Porter CSE 506
Linux kernel synchronization Don Porter CSE 506 The old days Early/simple OSes (like JOS): No need for synchronization All kernel requests wait until completion even disk requests Heavily restrict when
Advanced Operating Systems (CS 202) Memory Consistency, Cache Coherence and Synchronization
Advanced Operating Systems (CS 202) Memory Consistency, Cache Coherence and Synchronization (some cache coherence slides adapted from Ian Watson; some memory consistency slides from Sarita Adve) Classic
RWLocks in Erlang/OTP
RWLocks in Erlang/OTP Rickard Green rickard@erlang.org What is an RWLock? Read/Write Lock Write locked Exclusive access for one thread Read locked Multiple reader threads No writer threads RWLocks Used
Lecture 18: Coherence and Synchronization. Topics: directory-based coherence protocols, synchronization primitives (Sections
Lecture 18: Coherence and Synchronization Topics: directory-based coherence protocols, synchronization primitives (Sections 5.1-5.5) 1 Cache Coherence Protocols Directory-based: A single location (directory)
Concurrent programming: From theory to practice. Concurrent Algorithms 2015 Vasileios Trigonakis
oncurrent programming: From theory to practice oncurrent Algorithms 2015 Vasileios Trigonakis From theory to practice Theoretical (design) Practical (design) Practical (implementation) 2 From theory to
Interprocess Communication By: Kaushik Vaghani
Interprocess Communication By: Kaushik Vaghani Background Race Condition: A situation where several processes access and manipulate the same data concurrently and the outcome of execution depends on the
Semaphore. Originally called P() and V() wait (S) { while S <= 0 ; // no-op S--; } signal (S) { S++; }
Semaphore Semaphore S integer variable Two standard operations modify S: wait() and signal() Originally called P() and V() Can only be accessed via two indivisible (atomic) operations wait (S) { while
CSE 5306 Distributed Systems
CSE 5306 Distributed Systems Consistency and Replication Jia Rao http://ranger.uta.edu/~jrao/ 1 Reasons for Replication Data is replicated for the reliability of the system Servers are replicated for performance
Constant RMR Transformation to Augment Reader-Writer Locks with Atomic Upgrade/Downgrade Support
Constant RMR Transformation to Augment Reader-Writer Locks with Atomic Upgrade/Downgrade Support Jake Stern Leichtling Thesis Advisor: Prasad Jayanti With significant contributions from Michael Diamond.
HETEROGENEOUS MEMORY MANAGEMENT. Linux Plumbers Conference Jérôme Glisse
HETEROGENEOUS MEMORY MANAGEMENT Linux Plumbers Conference 2018 Jérôme Glisse EVERYTHING IS A POINTER All data structures rely on pointers, explicitly or implicitly: Explicit in languages like C, C++,...
PROJECT 6: PINTOS FILE SYSTEM. CS124 Operating Systems Winter , Lecture 25
PROJECT 6: PINTOS FILE SYSTEM CS124 Operating Systems Winter 2015-2016, Lecture 25 2 Project 6: Pintos File System Last project is to improve the Pintos file system Note: Please ask before using late tokens
Distributed Operating Systems
Distributed Operating Systems Synchronization in Parallel Systems Marcus Völp 203 Topics Synchronization Locks Performance Distributed Operating Systems 203 Marcus Völp 2 Overview Introduction Hardware
Operating Systems. Operating Systems Summer 2017 Sina Meraji U of T
Operating Systems Operating Systems Summer 2017 Sina Meraji U of T More Special Instructions Swap (or Exchange) instruction Operates on two words atomically Can also be used to solve critical section problem
Lecture Topics. Announcements. Today: Concurrency (Stallings, chapter , 5.7) Next: Exam #1. Self-Study Exercise #5. Project #3 (due 9/28)
Lecture Topics Today: Concurrency (Stallings, chapter 5.1-5.4, 5.7) Next: Exam #1 1 Announcements Self-Study Exercise #5 Project #3 (due 9/28) Project #4 (due 10/12) 2 Exam #1 Tuesday, 10/3 during lecture
CSE 451: Operating Systems Winter Lecture 7 Synchronization. Steve Gribble. Synchronization. Threads cooperate in multithreaded programs
CSE 451: Operating Systems Winter 2005 Lecture 7 Synchronization Steve Gribble Synchronization Threads cooperate in multithreaded programs to share resources, access shared data structures e.g., threads
Synchronization I. Jo, Heeseung
Synchronization I Jo, Heeseung Today's Topics Synchronization problem Locks 2 Synchronization Threads cooperate in multithreaded programs To share resources, access shared data structures Also, to coordinate
Fortran 2008: what s in it for high-performance computing
Fortran 2008: what s in it for high-performance computing John Reid, ISO Fortran Convener, JKR Associates and Rutherford Appleton Laboratory Fortran 2008 has been completed and is about to be published.
GLocks: Efficient Support for Highly- Contended Locks in Many-Core CMPs
GLocks: Efficient Support for Highly- Contended Locks in Many-Core CMPs Authors: Jos e L. Abell an, Juan Fern andez and Manuel E. Acacio Presenter: Guoliang Liu Outline Introduction Motivation Background
CS140 Operating Systems and Systems Programming Midterm Exam
CS140 Operating Systems and Systems Programming Midterm Exam October 31 st, 2003 (Total time = 50 minutes, Total Points = 50) Name: (please print) In recognition of and in the spirit of the Stanford University
Linearizability of Persistent Memory Objects
Linearizability of Persistent Memory Objects Michael L. Scott Joint work with Joseph Izraelevitz & Hammurabi Mendes www.cs.rochester.edu/research/synchronization/ Workshop on the Theory of Transactional
CS 153 Design of Operating Systems Winter 2016
CS 153 Design of Operating Systems Winter 2016 Lecture 9: Semaphores and Monitors Some slides from Matt Welsh Summarize Where We Are Goal: Use mutual exclusion to protect critical sections of code that
CSE Traditional Operating Systems deal with typical system software designed to be:
CSE 6431 Traditional Operating Systems deal with typical system software designed to be: general purpose running on single processor machines Advanced Operating Systems are designed for either a special
Synchronization. Coherency protocols guarantee that a reading processor (thread) sees the most current update to shared data.
Synchronization Coherency protocols guarantee that a reading processor (thread) sees the most current update to shared data. Coherency protocols do not: make sure that only one thread accesses shared data
More Types of Synchronization 11/29/16
More Types of Synchronization 11/29/16 Today s Agenda Classic thread patterns Other parallel programming patterns More synchronization primitives: RW locks Condition variables Semaphores Message passing
CSC2/458 Parallel and Distributed Systems Scalable Synchronization
CSC2/458 Parallel and Distributed Systems Scalable Synchronization Sreepathi Pai February 20, 2018 URCS Outline Scalable Locking Barriers Outline Scalable Locking Barriers An alternative lock ticket lock
Introduction. Coherency vs Consistency. Lec-11. Multi-Threading Concepts: Coherency, Consistency, and Synchronization
Lec-11 Multi-Threading Concepts: Coherency, Consistency, and Synchronization Coherency vs Consistency Memory coherency and consistency are major concerns in the design of shared-memory systems. Consistency
Cache Coherence and Atomic Operations in Hardware
Cache Coherence and Atomic Operations in Hardware Previously, we introduced multi-core parallelism. Today we ll look at 2 things: 1. Cache coherence 2. Instruction support for synchronization. And some
Synchronization. Jinkyu Jeong Computer Systems Laboratory Sungkyunkwan University
Synchronization Jinkyu Jeong (jinkyu@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Types of Synchronization Mutual Exclusion Locks Event Synchronization Global or group-based
MultiJav: A Distributed Shared Memory System Based on Multiple Java Virtual Machines. MultiJav: Introduction
: A Distributed Shared Memory System Based on Multiple Java Virtual Machines X. Chen and V.H. Allan Computer Science Department, Utah State University 1998 : Introduction Built on concurrency supported
CPSC/ECE 3220 Fall 2017 Exam Give the definition (note: not the roles) for an operating system as stated in the textbook. (2 pts.
CPSC/ECE 3220 Fall 2017 Exam 1 Name: 1. Give the definition (note: not the roles) for an operating system as stated in the textbook. (2 pts.) Referee / Illusionist / Glue. Circle only one of R, I, or G.
Synchronization. Jinkyu Jeong Computer Systems Laboratory Sungkyunkwan University
Synchronization Jinkyu Jeong (jinkyu@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) Types of Synchronization
Hybrid MPI - A Case Study on the Xeon Phi Platform
Hybrid MPI - A Case Study on the Xeon Phi Platform Udayanga Wickramasinghe Center for Research on Extreme Scale Technologies (CREST) Indiana University Greg Bronevetsky Lawrence Livermore National Laboratory
Lecture 10: Multi-Object Synchronization
CS 422/522 Design & Implementation of Operating Systems Lecture 10: Multi-Object Synchronization Zhong Shao Dept. of Computer Science Yale University Acknowledgement: some slides are taken from previous
250P: Computer Systems Architecture. Lecture 14: Synchronization. Anton Burtsev March, 2019
250P: Computer Systems Architecture Lecture 14: Synchronization Anton Burtsev March, 2019 Coherence and Synchronization Topics: synchronization primitives (Sections 5.4-5.5) 2 Constructing Locks Applications
Scalable Reader-Writer Synchronization for Shared-Memory Multiprocessors
Scalable Reader-Writer Synchronization for Shared-Memory Multiprocessors John M. Mellor-Crummey' (johnmcgrice.edu) Center for Research on Parallel Computation Rice University, P.O. Box 1892 Houston, TX
Distributed Systems Synchronization. Marcus Völp 2007
Distributed Systems Synchronization Marcus Völp 2007 1 Purpose of this Lecture Synchronization Locking Analysis / Comparison Distributed Operating Systems 2007 Marcus Völp 2 Overview Introduction Hardware
Readers-Writers Problem. Implementing shared locks. shared locks (continued) Review: Test-and-set spinlock. Review: Test-and-set on alpha
Readers-Writers Problem Implementing shared locks Multiple threads may access data - Readers will only observe, not modify data - Writers will change the data Goal: allow multiple readers or one single
Modernizing NetBSD Networking Facilities and Interrupt Handling. Ryota Ozaki Kengo Nakahara
Modernizing NetBSD Networking Facilities and Interrupt Handling Ryota Ozaki Kengo Nakahara Overview of Our Work Goals 1. MP-ify NetBSD networking facilities 2.
Mutual Exclusion and Synchronization
Mutual Exclusion and Synchronization Concurrency Defined Single processor multiprogramming system Interleaving of processes Multiprocessor systems Processes run in parallel on different processors Interleaving
Mutual Exclusion Algorithms with Constant RMR Complexity and Wait-Free Exit Code
Mutual Exclusion Algorithms with Constant RMR Complexity and Wait-Free Exit Code Rotem Dvir 1 and Gadi Taubenfeld 2 1 The Interdisciplinary Center, P.O.Box 167, Herzliya 46150, Israel rotem.dvir@gmail.com
Implementing Transactional Memory in Kernel space
Implementing Transactional Memory in Kernel space Breno Leitão Linux Technology Center leitao@debian.org leitao@freebsd.org Problem Statement Mutual exclusion concurrency control (Lock) Locks type: Semaphore
Advanced Operating Systems (CS 202) Memory Consistency, Cache Coherence and Synchronization
Advanced Operating Systems (CS 202) Memory Consistency, Cache Coherence and Synchronization Jan, 27, 2017 (some cache coherence slides adapted from Ian Watson; some memory consistency slides from Sarita
CSE 451: Operating Systems Winter Lecture 7 Synchronization. Hank Levy 412 Sieg Hall
CSE 451: Operating Systems Winter 2003 Lecture 7 Synchronization Hank Levy Levy@cs.washington.edu 412 Sieg Hall Synchronization Threads cooperate in multithreaded programs to share resources, access shared
Module 14: "Directory-based Cache Coherence" Lecture 31: "Managing Directory Overhead" Directory-based Cache Coherence: Replacement of S blocks
Directory-based Cache Coherence: Replacement of S blocks Serialization VN deadlock Starvation Overflow schemes Sparse directory Remote access cache COMA Latency tolerance Page migration Queue lock in hardware