DISTRIBUTED HIGH-SPEED COMPUTING OF MULTIMEDIA DATA

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "DISTRIBUTED HIGH-SPEED COMPUTING OF MULTIMEDIA DATA"

Transcription

1 DISTRIBUTED HIGH-SPEED COMPUTING OF MULTIMEDIA DATA M. GAUS, G. R. JOUBERT, O. KAO, S. RIEDEL AND S. STAPEL Technical University of Clausthal, Department of Computer Science Julius-Albert-Str. 4, Clausthal-Zellerfeld, Germany Distributed platforms are not necessarily well-suited for systems which handle large data sets, such as processed in multimedia applications. In this paper a specialised computation model, based on asynchronous transmission, is presented. As the necessary functions are encapsulated this system can be used without detailed knowledge of the system architecture. A dynamic strategy of task execution is utilised to adjust the number and size of the distributed data packages according to the computational load of the processing elements at transmission time. Thus more powerful PE s, or those whose resources are not fully utilised, will either receive packages more frequently or will be given larger packages. In large networks some nodes can be replaced by others or only a few data blocks may be sent to (a) particular node(s). The efficiency of the method is evaluated with a variety of practical run time measurements. 1 Introduction Distributed systems consisting of a network of workstations are increasingly being used for solving compute intensive problems. Distributed platforms are, however, not always well-suited for systems which handle large data sets as can be found in, for example, multimedia applications. The limiting factor for processing of large data sets is usually network bandwidth. Thus, the distribution of huge amounts of data bounds the overall processing speed. This situation is made worse by the fact that data is transmitted only when requested or sent by the parallel processes. In order to reduce this effect, the transmission of data should be separated from the process synchronisation. Well-known software systems for parallel/distributed processing on existing computer networks are PVM, MPI, PVMPI, Condor, Mosix [1-5] and Treadmarks. An advantage of the PVM is its availability on nearly all important architectures and operating systems. On the other hand synchronous data transfers and type conversions are time consuming, making it unsuitable for the processing of large multimedia data sets. 1.1 Multimedia data Multimedia data has become an important component of modern software systems. Static media (images, graphics, text) are combined with dynamic media (audio, video, animations) to obtain realistic representations of natural processes, for the visualisation of complex results or to depict dynamic processes. In spite of the increases in memory sizes, processing and communication speeds, the processing and communication of multimedia data is still time and submitted to World Scientific : : 13:47 1/1

2 compute intensive. Some of the initial problems, essentially data compression, could be solved by the development of efficient compression algorithms, e.g. JPEG, MPEG, MP3. Many of these algorithms have been implemented in hardware, offering the possibility of real time encoding. The next step resulted in parallelising numerous procedures for processing multimedia data. Static media, such as encountered in image processing applications, are usually subdivided into independent data fragments, which are then distributed among a number of processing elements. The results are gathered and combined to form the final result. In dynamic media, interdependencies between the different data blocks must be considered and resolved. An example for this is MPEG compression, which is based on finding and eliminating redundant information in consecutive frames. Parallelisation by means of data segmentation is well-suited for parallel computers with shared memory, since little or no time is spent on communicating the data. Software for distributed computing in heterogeneous networks will have less of a performance gain, because of the slow synchronous transfers and greater variances in client resources. If the operations executed are simple these delay effects can be seen quite clearly. An example for this is the calculation of correlation coefficients for short term series [8]. Considering all combinations between 100 shares and a time difference of 5 days resulted in correlation terms and 27 megabytes of data. The performance gain by parallelising the algorithm with the PVM among 4 DEC Alphas was negated by the resulting administration overhead. This resulted in the run time on a single workstation being up to 6 times faster than the parallel PVM version. These requirements (large data sets, simple operations) are also found in the management, retrieval and processing of multimedia data. Current approaches to multimedia databases are based on the extraction and management of specific characteristics. Queries compare the extracted characteristics with all images stored in the database, and return the most similar images. Each archival and retrieval process results in the computation of huge amounts of data. Performance gains through parallelisation are negated by transfer times and administration of the data, as described in the correlation example. This results in the necessity of a specialised model for parallel processing of huge amounts of data. 2 Processing model for static multimedia data The proposed processing model aims to make development of parallel programs by non-experienced users easy, and minimise the communication and management effort, by using TCP/IP sockets directly. Similar to the work pile model [6], this model is based on the creation of pools of tasks, which are controlled by three special processes (distribution and collection manager, computation client). The information is divided into sections which are distributed to a number of processing elements (Figure 1). submitted to World Scientific : : 13:47 2/2

3 PE 1 Pool of Tasks Distribution manager PE 2 : Collection manager Pool of Results PE n Figure 1: Schematic representation of the processing model 2.1 Distribution manager The distribution manager is responsible for the division and management of the data packets to be processed. Push technology is used to minimise the transfer cost between server and clients. The responsibility of the distribution manager includes data packets definition, management of data packets in the local pool of tasks, processing of client requests and distribution of the data packets among the processing elements. The distribution strategy is set within this process. Essential requirements include the efficient use of available resources, as well as being failure tolerant. To circumvent problems related to processing element failures the data packets are subdivided into three groups: the first group consists of packages which were not yet distributed, the second group comprises transmitted, but unprocessed data, whereas the third group consists of processed data packets. A simple distribution strategy of available data packets increases computing efficiency. If the first group is empty, but non-processed data blocks are still in the second group, then these are dispatched to idle clients, which have already completed their computation tasks. This can be achieved by generating a list of all available active nodes and of the status of their local pools of tasks. The number of distributed but not yet processed packets can be calculated from the number of packets sent, but not yet received by the collection task. This requires a direct connection between the distributor and the collector. The difference is analysed and compared to a given threshold values. If it is below the threshold the distribution manager sends new packets to the client. This strategy requires a time and/or workload oriented distribution of the data packets as well, since processing can only occur if the processing element has a low CPU load. A blocked client that does not satisfy this requirement is regarded as a node that has failed. The server will redistribute the data packets sent to this client. 2.2 Computation client This component performs the computation on each processing element. A simple and compact structure reduces the management overhead and enables an important performance increase. The computation client consists of a local pool of tasks, a submitted to World Scientific : : 13:47 3/3

4 processing object and a local pool of results. In this pool the processed data packets are temporarily stored until a connection for the transfer to the collection manager becomes available. 2.3 Collection manager This process accepts processed data packets from the computation clients and stores them until all data packets have been received in the pool of results. Once this occurs, it composes the processed original from the received data packets. A picture or a series of pictures would be composed at this point during e.g. JPEG-encoding. Furthermore, the collector sends a message giving the number of received data packets to the distributor. From this information the distribution manager determines the current workload of each client and redefines the distribution strategy. The distributor is also notified when all data packets have reached the collector and the processing is completed. 2.4 Arraying in multiple hierarchical levels The described model consists of two hierarchical levels, containing the distribution and collection processes on one level, and computation clients on the other. This model will reach its capacities quickly with a large number of non-local processing elements. An alternative is to arrange servers hierarchically. The lower levels of this hierarchy contain not only clients, but subordinated servers as well, which distribute the data packets to lower level clients. An example for the application of such a model are data distributions in corporate or university networks: a super server sends data packets to subordinate servers in each division. Each of these servers initiates the computation in its own domain. This significantly reduces the communication complexity, or at least binds it locally. The processed packets are still sent to a central collector making dynamic regrouping possible. The clients of a new group will then receive their packets from the server of the new group. Marking the processed data packets with the id of the group which processed them is mandatory. This allows the collector to find out which group processed each data packet so that this group is resupplied with data to process once it drops below a given threshold. 3 An adaptive distribution strategy Heterogeneous networks consist of processing elements with different performance capabilities (CPU, memory etc). Information about the complexity of tasks being processed is usually not available. Furthermore, the number of users working on a particular workstation are continuously changing. Thus it is impossible to predict the performance of any particular workstation in a network at a given time. This submitted to World Scientific : : 13:47 4/4

5 makes it impossible to a priori schedule task processing. A dynamic distribution strategy of processing tasks is thus needed. The number and size of the distributed data packages must be adapted to the work load of the processing element at transmission time. Even this strategy may not be near optimal, as additional tasks can be started on the PE between the determination of the current load and the arrival of data packages. More powerful PE s or those with a small performance utilisation will receive packets more frequently or will be allocated larger packages. In large networks some low performance nodes can be skipped and the work distributed to more powerful PE s. If this is not possible the data blocks sent to the low performance nodes will automatically be adapted. For the concrete realisation of this method a performance ranking must be generated. This can be done by calculating the difference between sent and processed packages as described above. In the first distribution run each processing element is supplied with n packages. After a certain time interval a performance rank list is created. The number of packets for the respective processing elements are then increased or decreased. This operation is repeated until the collector has received all data. Alternatively the packet size can be adapted. Larger packets are sent to the PE s at the top of the performance list. This can minimise the communication and network traffic. However, this is not always possible. For example, an image is usually subdivided into n sections. If all sections are distributed during the first run a change of the package size is not possible without a loss of already processed data. This performance information can only be used if the image has large dimensions, or if a whole image sequence is to be processed. A disadvantage of this model is that additional logic for the management of dynamic block sizes is necessary in the clients. Furthermore the complexity of the model tasks and the requirements regarding the user knowledge are increased. 4 The usage of the system The data flow of the proposed model for the parallel processing of multimedia data involves the following steps: The generated data packets are put into the pool of tasks when processing starts and the distribution manager is initialised. The data packets, received by the clients, are stored in the local pool of tasks, which is essentially a queue. Afterwards the computation starts. Processed data is stored in the local pool of results and sent to the collection manager. The collection manager informs the distribution manager of the receipt of the processed packets. When all data packets have been received, the so-called NULL-packet is distributed. Every processing element which receives a NULL-packet immediately terminates processing. An object oriented system design will help making system components reusable and lessens the difficulty of using the distribution models. The most submitted to World Scientific : : 13:47 5/5

6 important class is the processing class. It does the actual processing and is the focal point of the model. All other classes support it by managing the administration, reception and distribution of data. The parameter of its run()-method contains the data to be processed. The packet is processed in this method, stored in the local pool of results by means of a return call and is then sent back. The usage of this system merely requires an overloading of the run()-method of the processing class, adjusting the class for special problems. The distribution and collection manager have to be initialised at the beginning of a session. Furthermore, the required processes need to be launched in the processing nodes. These will then contact the distributor and collector on their own. At this stage the system will be idle. The pool of tasks is now filled with the required packets. Once this has been done, the distributor is activated and the data is processed. All processed packets are stored in the pool of results. Manipulating the packet size requires overloading of the methods that split and merge the packets. 5 Performance measurements The measurements were performed on a cluster of Linux K6, 300 MHz PCs connected over a 10Mbit Ethernet. In a first attempt different block sizes and number of iterations as well as various configurations of the processing model were examined in order to obtain data about the efficiency and the run time behaviour of the proposed system. Table 1: Measurement results (run times, speedup and efficiency) with the implemented prototype Iterations Time[s]: 1 PE Time[s]/Sp/Ep : 2 PE Time[s]/Sp/Ep : 3 PE Time[s]/Sp/Ep : 4 PE / 1.407/ / 1.615/ / 1.590/ / 1.586/ / 1.751/ / 1.939/ / 1.637/ / 1.852/ / 2.136/ / 1.758/ / 2.036/ / 2.409/ / 1.768/ / 2.215/ / 2.501/ / 1.805/ / 2.218/ / 2.514/ / 1.829/ / 2.388/ / 2.797/ / 1.855/ / 2.400/ / 2.921/ / 1.846/ / 2.429/ / 2.991/ / 1.825/ / 2.511/ / 3.155/ / 1.938/ / 2.529/ / 3.301/ Table 1 shows the run times needed for iterations of a simple inverting operation performed on a 10 Mbyte large block as well as the speedup factor S P submitted to World Scientific : : 13:47 6/6

7 and the efficiency E P. The data is subdivided into byte large subsections and according to the strategy described distributed to the single PE clients. Speedup values between 1.4 and 3.3 are reached in this simple application. At the beginning the network communication is the most influencing factor resulting in speedups between (2 PE s) and 1.59 (4 PE s). With larger numbers of iterations a linear increase of the speedup values can be observed reaching top speedup values of 3.3 in case of 4 PE s and 200 iterations. The efficiency decreases only slightly, e.g. there is a difference of 0.24 between the mean values of the two and four PE systems. Thus the scalability of the system model appears to be good. A clearer description of the results is given in figure 2. The right hand diagram shows the run times of the different system configurations, the left hand diagram contains the mean speedup and efficiency values for the parallel configurations. 3,0 2,5 Speedup and Efficiency (mean values ) Speedup Efficiency Client 2 Clients 3 Clients 4 Clients 2,0 1,5 1,0 0,5 Time [s] , Processing Elements Iterations Figure 2: A diagram of the speedup and efficiency values achieved (left); run times for 1-4 PEs (right) The achieved results are compared to the mean speedup and efficiency values of the PVM, which are shown in figure 3. The measurements are performed on the same configurations (K6 with Linux, distribution of byte large blocks) and type conversion disabled. Speedup and Efficiency PVM (mean values) 3 2,5 2 Speedup Efficiency 1,5 1 0, Processing Elements Figure 3: A diagram of the PVM average speedup and efficiency values submitted to World Scientific : : 13:47 7/7

8 An analysis of the PVM results shows slightly better speedup and efficiency values in case of two processing elements. These decrease when larger numbers of PE s are used. The effort of management and transfer clearly reduces the performance. Thus the proposed system model reaches a five times better speedup and efficiency in case of configurations with four PEs. 6 Conclusions In this paper a specialised computation model based on asynchronous transmission is presented, which automatically adapts to the workload of the elements in the parallel environment at transmission time, enables easy development of parallel programs and minimises the communication and management effort by direct use of TCP/IP sockets. It is based on the creation of pools of tasks, which are controlled by three special modules. A simple distribution strategy of the available packages increases the computing efficiency. More powerful processing elements or such with a small workload will more frequently receive packages. Additionally the package size can be adapted. The efficiency of the proposed method is evaluated through a variety of performance measurements. The results are compared with the results of the PVM. Future work includes extensions, which primarily concern improving the system s performance. Storing the packets in the local file system, similar to a spool-directory, makes it possible to save all packets of the same type that are to be processed in a special directory. Furthermore, comparative benchmarks with other systems are to be performed. References: 1. PVM Home page: Documentation, comparison between various packages, 2. CONDOR Project description, documentation, 3. MPI Project Home page: Documentation, tutorials, etc, 4. Mosix Home page: 5. Information about PVMPI: 6. S. Keinman, D. Shah, Programming with Threads, Prentice Hall, B. Wilkinson, M. Allen: Parallel Programming: Techniques and Applications Using Networked Workstations and Parallel Computers, Prentice Hall, O. Sachs, Analyse von Aktienreihen mittels paralleler Korrelationsberechnungen, Master thesis, TU Clausthal, 1998 submitted to World Scientific : : 13:47 8/8

Study of Load Balancing Schemes over a Video on Demand System

Study of Load Balancing Schemes over a Video on Demand System Study of Load Balancing Schemes over a Video on Demand System Priyank Singhal Ashish Chhabria Nupur Bansal Nataasha Raul Research Scholar, Computer Department Abstract: Load balancing algorithms on Video

More information

LAPI on HPS Evaluating Federation

LAPI on HPS Evaluating Federation LAPI on HPS Evaluating Federation Adrian Jackson August 23, 2004 Abstract LAPI is an IBM-specific communication library that performs single-sided operation. This library was well profiled on Phase 1 of

More information

6.1 Internet Transport Layer Architecture 6.2 UDP (User Datagram Protocol) 6.3 TCP (Transmission Control Protocol) 6. Transport Layer 6-1

6.1 Internet Transport Layer Architecture 6.2 UDP (User Datagram Protocol) 6.3 TCP (Transmission Control Protocol) 6. Transport Layer 6-1 6. Transport Layer 6.1 Internet Transport Layer Architecture 6.2 UDP (User Datagram Protocol) 6.3 TCP (Transmission Control Protocol) 6. Transport Layer 6-1 6.1 Internet Transport Layer Architecture The

More information

Fractals. Investigating task farms and load imbalance

Fractals. Investigating task farms and load imbalance Fractals Investigating task farms and load imbalance Reusing this material This work is licensed under a Creative Commons Attribution- NonCommercial-ShareAlike 4.0 International License. http://creativecommons.org/licenses/by-nc-sa/4.0/deed.en_us

More information

Fractals exercise. Investigating task farms and load imbalance

Fractals exercise. Investigating task farms and load imbalance Fractals exercise Investigating task farms and load imbalance Reusing this material This work is licensed under a Creative Commons Attribution- NonCommercial-ShareAlike 4.0 International License. http://creativecommons.org/licenses/by-nc-sa/4.0/deed.en_us

More information

The latency of user-to-user, kernel-to-kernel and interrupt-to-interrupt level communication

The latency of user-to-user, kernel-to-kernel and interrupt-to-interrupt level communication The latency of user-to-user, kernel-to-kernel and interrupt-to-interrupt level communication John Markus Bjørndalen, Otto J. Anshus, Brian Vinter, Tore Larsen Department of Computer Science University

More information

Transactions on Information and Communications Technologies vol 15, 1997 WIT Press, ISSN

Transactions on Information and Communications Technologies vol 15, 1997 WIT Press,  ISSN Balanced workload distribution on a multi-processor cluster J.L. Bosque*, B. Moreno*", L. Pastor*" *Depatamento de Automdtica, Escuela Universitaria Politecnica de la Universidad de Alcald, Alcald de Henares,

More information

Distributed Scheduling for the Sombrero Single Address Space Distributed Operating System

Distributed Scheduling for the Sombrero Single Address Space Distributed Operating System Distributed Scheduling for the Sombrero Single Address Space Distributed Operating System Donald S. Miller Department of Computer Science and Engineering Arizona State University Tempe, AZ, USA Alan C.

More information

Chapter 3. Design of Grid Scheduler. 3.1 Introduction

Chapter 3. Design of Grid Scheduler. 3.1 Introduction Chapter 3 Design of Grid Scheduler The scheduler component of the grid is responsible to prepare the job ques for grid resources. The research in design of grid schedulers has given various topologies

More information

Job Re-Packing for Enhancing the Performance of Gang Scheduling

Job Re-Packing for Enhancing the Performance of Gang Scheduling Job Re-Packing for Enhancing the Performance of Gang Scheduling B. B. Zhou 1, R. P. Brent 2, C. W. Johnson 3, and D. Walsh 3 1 Computer Sciences Laboratory, Australian National University, Canberra, ACT

More information

Configuration Guideline for CANopen Networks

Configuration Guideline for CANopen Networks Configuration Guideline for CANopen Networks Martin Rostan, Beckhoff Unlike most other fieldbus systems, CANopen provides many degrees of freedom to configure the communication behaviour of the network.

More information

Introduction to parallel computing

Introduction to parallel computing Introduction to parallel computing Introduction to Parallel Computing with MPI and OpenMP P. Ramieri 2014 Course agenda May 12, 2014 9.30-11.00 01 - Introduction to parallel computing (P. Ramieri) 11.15

More information

The former pager tasks have been replaced in 7.9 by the special savepoint tasks.

The former pager tasks have been replaced in 7.9 by the special savepoint tasks. 1 2 3 4 With version 7.7 the I/O interface to the operating system has been reimplemented. As of version 7.7 different parameters than in version 7.6 are used. The improved I/O system has the following

More information

PROCESS VIRTUAL MEMORY. CS124 Operating Systems Fall , Lecture 18

PROCESS VIRTUAL MEMORY. CS124 Operating Systems Fall , Lecture 18 PROCESS VIRTUAL MEMORY CS124 Operating Systems Fall 2017-2018, Lecture 18 2 Programs and Memory Programs perform many interactions with memory Accessing variables stored at specific memory locations Calls

More information

PROCESS VIRTUAL MEMORY. CS124 Operating Systems Winter , Lecture 18

PROCESS VIRTUAL MEMORY. CS124 Operating Systems Winter , Lecture 18 PROCESS VIRTUAL MEMORY CS124 Operating Systems Winter 2015-2016, Lecture 18 2 Programs and Memory Programs perform many interactions with memory Accessing variables stored at specific memory locations

More information

OVERHEADS ENHANCEMENT IN MUTIPLE PROCESSING SYSTEMS BY ANURAG REDDY GANKAT KARTHIK REDDY AKKATI

OVERHEADS ENHANCEMENT IN MUTIPLE PROCESSING SYSTEMS BY ANURAG REDDY GANKAT KARTHIK REDDY AKKATI CMPE 655- MULTIPLE PROCESSOR SYSTEMS OVERHEADS ENHANCEMENT IN MUTIPLE PROCESSING SYSTEMS BY ANURAG REDDY GANKAT KARTHIK REDDY AKKATI What is MULTI PROCESSING?? Multiprocessing is the coordinated processing

More information

G Robert Grimm New York University

G Robert Grimm New York University G22.3250-001 Receiver Livelock Robert Grimm New York University Altogether Now: The Three Questions What is the problem? What is new or different? What are the contributions and limitations? Motivation

More information

Network-Adaptive Video Coding and Transmission

Network-Adaptive Video Coding and Transmission Header for SPIE use Network-Adaptive Video Coding and Transmission Kay Sripanidkulchai and Tsuhan Chen Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA 15213

More information

Comparing Centralized and Decentralized Distributed Execution Systems

Comparing Centralized and Decentralized Distributed Execution Systems Comparing Centralized and Decentralized Distributed Execution Systems Mustafa Paksoy mpaksoy@swarthmore.edu Javier Prado jprado@swarthmore.edu May 2, 2006 Abstract We implement two distributed execution

More information

Distributed Computing: PVM, MPI, and MOSIX. Multiple Processor Systems. Dr. Shaaban. Judd E.N. Jenne

Distributed Computing: PVM, MPI, and MOSIX. Multiple Processor Systems. Dr. Shaaban. Judd E.N. Jenne Distributed Computing: PVM, MPI, and MOSIX Multiple Processor Systems Dr. Shaaban Judd E.N. Jenne May 21, 1999 Abstract: Distributed computing is emerging as the preferred means of supporting parallel

More information

Why Study Multimedia? Operating Systems. Multimedia Resource Requirements. Continuous Media. Influences on Quality. An End-To-End Problem

Why Study Multimedia? Operating Systems. Multimedia Resource Requirements. Continuous Media. Influences on Quality. An End-To-End Problem Why Study Multimedia? Operating Systems Operating System Support for Multimedia Improvements: Telecommunications Environments Communication Fun Outgrowth from industry telecommunications consumer electronics

More information

Last Class: RPCs and RMI. Today: Communication Issues

Last Class: RPCs and RMI. Today: Communication Issues Last Class: RPCs and RMI Case Study: Sun RPC Lightweight RPCs Remote Method Invocation (RMI) Design issues Lecture 9, page 1 Today: Communication Issues Message-oriented communication Persistence and synchronicity

More information

Benchmarking results of SMIP project software components

Benchmarking results of SMIP project software components Benchmarking results of SMIP project software components NAILabs September 15, 23 1 Introduction As packets are processed by high-speed security gateways and firewall devices, it is critical that system

More information

Database Server. 2. Allow client request to the database server (using SQL requests) over the network.

Database Server. 2. Allow client request to the database server (using SQL requests) over the network. Database Server Introduction: Client/Server Systems is networked computing model Processes distributed between clients and servers. Client Workstation (usually a PC) that requests and uses a service Server

More information

I/O in the Gardens Non-Dedicated Cluster Computing Environment

I/O in the Gardens Non-Dedicated Cluster Computing Environment I/O in the Gardens Non-Dedicated Cluster Computing Environment Paul Roe and Siu Yuen Chan School of Computing Science Queensland University of Technology Australia fp.roe, s.chang@qut.edu.au Abstract Gardens

More information

Chapter 7 Multimedia Operating Systems

Chapter 7 Multimedia Operating Systems MODERN OPERATING SYSTEMS Third Edition ANDREW S. TANENBAUM Chapter 7 Multimedia Operating Systems Introduction To Multimedia (1) Figure 7-1. Video on demand using different local distribution technologies.

More information

Final Project Writeup

Final Project Writeup Jitu Das Bertha Lam 15-418 Final Project Writeup Summary We built a framework that facilitates running computations across multiple GPUs and displaying results in a web browser. We then created three demos

More information

Parallel Algorithms for the Third Extension of the Sieve of Eratosthenes. Todd A. Whittaker Ohio State University

Parallel Algorithms for the Third Extension of the Sieve of Eratosthenes. Todd A. Whittaker Ohio State University Parallel Algorithms for the Third Extension of the Sieve of Eratosthenes Todd A. Whittaker Ohio State University whittake@cis.ohio-state.edu Kathy J. Liszka The University of Akron liszka@computer.org

More information

Introduction to Real-time Systems. Advanced Operating Systems (M) Lecture 2

Introduction to Real-time Systems. Advanced Operating Systems (M) Lecture 2 Introduction to Real-time Systems Advanced Operating Systems (M) Lecture 2 Introduction to Real-time Systems Real-time systems deliver services while meeting some timing constraints Not necessarily fast,

More information

Advanced Topics UNIT 2 PERFORMANCE EVALUATIONS

Advanced Topics UNIT 2 PERFORMANCE EVALUATIONS Advanced Topics UNIT 2 PERFORMANCE EVALUATIONS Structure Page Nos. 2.0 Introduction 4 2. Objectives 5 2.2 Metrics for Performance Evaluation 5 2.2. Running Time 2.2.2 Speed Up 2.2.3 Efficiency 2.3 Factors

More information

Operating System Support for Multimedia. Slides courtesy of Tay Vaughan Making Multimedia Work

Operating System Support for Multimedia. Slides courtesy of Tay Vaughan Making Multimedia Work Operating System Support for Multimedia Slides courtesy of Tay Vaughan Making Multimedia Work Why Study Multimedia? Improvements: Telecommunications Environments Communication Fun Outgrowth from industry

More information

CC MPI: A Compiled Communication Capable MPI Prototype for Ethernet Switched Clusters

CC MPI: A Compiled Communication Capable MPI Prototype for Ethernet Switched Clusters CC MPI: A Compiled Communication Capable MPI Prototype for Ethernet Switched Clusters Amit Karwande, Xin Yuan Dept. of Computer Science Florida State University Tallahassee, FL 32306 {karwande,xyuan}@cs.fsu.edu

More information

Differential Compression and Optimal Caching Methods for Content-Based Image Search Systems

Differential Compression and Optimal Caching Methods for Content-Based Image Search Systems Differential Compression and Optimal Caching Methods for Content-Based Image Search Systems Di Zhong a, Shih-Fu Chang a, John R. Smith b a Department of Electrical Engineering, Columbia University, NY,

More information

Distributed Information Processing

Distributed Information Processing Distributed Information Processing 6 th Lecture Eom, Hyeonsang ( 엄현상 ) Department of Computer Science & Engineering Seoul National University Copyrights 2016 Eom, Hyeonsang All Rights Reserved Outline

More information

Distributed Information Processing

Distributed Information Processing Distributed Information Processing 6 th Lecture Eom, Hyeonsang ( 엄현상 ) Department of Computer Science & Engineering Seoul National University Copyrights 2017 Eom, Hyeonsang All Rights Reserved Outline

More information

Execution Architecture

Execution Architecture Execution Architecture Software Architecture VO (706.706) Roman Kern Institute for Interactive Systems and Data Science, TU Graz 2018-11-07 Roman Kern (ISDS, TU Graz) Execution Architecture 2018-11-07

More information

MULTIMEDIA ADAPTATION FOR DYNAMIC ENVIRONMENTS

MULTIMEDIA ADAPTATION FOR DYNAMIC ENVIRONMENTS MULTIMEDIA ADAPTATION FOR DYNAMIC ENVIRONMENTS Maija Metso, Antti Koivisto and Jaakko Sauvola Machine Vision and Media Processing Group Infotech Oulu, University of Oulu PO BOX 444, 90570 Oulu, FINLAND

More information

Multimedia Systems 2011/2012

Multimedia Systems 2011/2012 Multimedia Systems 2011/2012 System Architecture Prof. Dr. Paul Müller University of Kaiserslautern Department of Computer Science Integrated Communication Systems ICSY http://www.icsy.de Sitemap 2 Hardware

More information

1995 Paper 10 Question 7

1995 Paper 10 Question 7 995 Paper 0 Question 7 Why are multiple buffers often used between producing and consuming processes? Describe the operation of a semaphore. What is the difference between a counting semaphore and a binary

More information

Assignment 5. Georgia Koloniari

Assignment 5. Georgia Koloniari Assignment 5 Georgia Koloniari 2. "Peer-to-Peer Computing" 1. What is the definition of a p2p system given by the authors in sec 1? Compare it with at least one of the definitions surveyed in the last

More information

SEDA: An Architecture for Well-Conditioned, Scalable Internet Services

SEDA: An Architecture for Well-Conditioned, Scalable Internet Services SEDA: An Architecture for Well-Conditioned, Scalable Internet Services Matt Welsh, David Culler, and Eric Brewer Computer Science Division University of California, Berkeley Operating Systems Principles

More information

6.1 Multiprocessor Computing Environment

6.1 Multiprocessor Computing Environment 6 Parallel Computing 6.1 Multiprocessor Computing Environment The high-performance computing environment used in this book for optimization of very large building structures is the Origin 2000 multiprocessor,

More information

CS 344/444 Computer Network Fundamentals Final Exam Solutions Spring 2007

CS 344/444 Computer Network Fundamentals Final Exam Solutions Spring 2007 CS 344/444 Computer Network Fundamentals Final Exam Solutions Spring 2007 Question 344 Points 444 Points Score 1 10 10 2 10 10 3 20 20 4 20 10 5 20 20 6 20 10 7-20 Total: 100 100 Instructions: 1. Question

More information

Frank Miller, George Apostolopoulos, and Satish Tripathi. University of Maryland. College Park, MD ffwmiller, georgeap,

Frank Miller, George Apostolopoulos, and Satish Tripathi. University of Maryland. College Park, MD ffwmiller, georgeap, Simple Input/Output Streaming in the Operating System Frank Miller, George Apostolopoulos, and Satish Tripathi Mobile Computing and Multimedia Laboratory Department of Computer Science University of Maryland

More information

Grid Computing Systems: A Survey and Taxonomy

Grid Computing Systems: A Survey and Taxonomy Grid Computing Systems: A Survey and Taxonomy Material for this lecture from: A Survey and Taxonomy of Resource Management Systems for Grid Computing Systems, K. Krauter, R. Buyya, M. Maheswaran, CS Technical

More information

Some popular Operating Systems include Linux Operating System, Windows Operating System, VMS, OS/400, AIX, z/os, etc.

Some popular Operating Systems include Linux Operating System, Windows Operating System, VMS, OS/400, AIX, z/os, etc. An Operating System (OS) is an interface between computer user and computer hardware. An operating system is software which performs all the basic tasks like file management, memory management, process

More information

CHAPTER 7 CONCLUSION AND FUTURE SCOPE

CHAPTER 7 CONCLUSION AND FUTURE SCOPE 121 CHAPTER 7 CONCLUSION AND FUTURE SCOPE This research has addressed the issues of grid scheduling, load balancing and fault tolerance for large scale computational grids. To investigate the solution

More information

Multimedia Communications. Transform Coding

Multimedia Communications. Transform Coding Multimedia Communications Transform Coding Transform coding Transform coding: source output is transformed into components that are coded according to their characteristics If a sequence of inputs is transformed

More information

CS455: Introduction to Distributed Systems [Spring 2018] Dept. Of Computer Science, Colorado State University

CS455: Introduction to Distributed Systems [Spring 2018] Dept. Of Computer Science, Colorado State University CS 455: INTRODUCTION TO DISTRIBUTED SYSTEMS [NETWORKING] Shrideep Pallickara Computer Science Colorado State University Frequently asked questions from the previous class survey Why not spawn processes

More information

Module 6 STILL IMAGE COMPRESSION STANDARDS

Module 6 STILL IMAGE COMPRESSION STANDARDS Module 6 STILL IMAGE COMPRESSION STANDARDS Lesson 19 JPEG-2000 Error Resiliency Instructional Objectives At the end of this lesson, the students should be able to: 1. Name two different types of lossy

More information

Continuous Real Time Data Transfer with UDP/IP

Continuous Real Time Data Transfer with UDP/IP Continuous Real Time Data Transfer with UDP/IP 1 Emil Farkas and 2 Iuliu Szekely 1 Wiener Strasse 27 Leopoldsdorf I. M., A-2285, Austria, farkas_emil@yahoo.com 2 Transilvania University of Brasov, Eroilor

More information

Design of Parallel Algorithms. Course Introduction

Design of Parallel Algorithms. Course Introduction + Design of Parallel Algorithms Course Introduction + CSE 4163/6163 Parallel Algorithm Analysis & Design! Course Web Site: http://www.cse.msstate.edu/~luke/courses/fl17/cse4163! Instructor: Ed Luke! Office:

More information

The Transport Layer: User Datagram Protocol

The Transport Layer: User Datagram Protocol The Transport Layer: User Datagram Protocol CS7025: Network Technologies and Server Side Programming http://www.scss.tcd.ie/~luzs/t/cs7025/ Lecturer: Saturnino Luz April 4, 2011 The UDP All applications

More information

PowerVR Series5. Architecture Guide for Developers

PowerVR Series5. Architecture Guide for Developers Public Imagination Technologies PowerVR Series5 Public. This publication contains proprietary information which is subject to change without notice and is supplied 'as is' without warranty of any kind.

More information

Page 1 SPACEWIRE SEMINAR 4/5 NOVEMBER 2003 JF COLDEFY / C HONVAULT

Page 1 SPACEWIRE SEMINAR 4/5 NOVEMBER 2003 JF COLDEFY / C HONVAULT Page 1 SPACEWIRE SEMINAR 4/5 NOVEMBER 2003 JF COLDEFY / C HONVAULT INTRODUCTION The SW IP was developped in the frame of the ESA 13345/#3 contract "Building block for System on a Chip" This presentation

More information

The MPI Message-passing Standard Practical use and implementation (I) SPD Course 2/03/2010 Massimo Coppola

The MPI Message-passing Standard Practical use and implementation (I) SPD Course 2/03/2010 Massimo Coppola The MPI Message-passing Standard Practical use and implementation (I) SPD Course 2/03/2010 Massimo Coppola What is MPI MPI: Message Passing Interface a standard defining a communication library that allows

More information

Receive Livelock. Robert Grimm New York University

Receive Livelock. Robert Grimm New York University Receive Livelock Robert Grimm New York University The Three Questions What is the problem? What is new or different? What are the contributions and limitations? Motivation Interrupts work well when I/O

More information

Module objectives. Integrated services. Support for real-time applications. Real-time flows and the current Internet protocols

Module objectives. Integrated services. Support for real-time applications. Real-time flows and the current Internet protocols Integrated services Reading: S. Keshav, An Engineering Approach to Computer Networking, chapters 6, 9 and 4 Module objectives Learn and understand about: Support for real-time applications: network-layer

More information

B.H.GARDI COLLEGE OF ENGINEERING & TECHNOLOGY (MCA Dept.) Parallel Database Database Management System - 2

B.H.GARDI COLLEGE OF ENGINEERING & TECHNOLOGY (MCA Dept.) Parallel Database Database Management System - 2 Introduction :- Today single CPU based architecture is not capable enough for the modern database that are required to handle more demanding and complex requirements of the users, for example, high performance,

More information

Performance of Various Levels of Storage. Movement between levels of storage hierarchy can be explicit or implicit

Performance of Various Levels of Storage. Movement between levels of storage hierarchy can be explicit or implicit Memory Management All data in memory before and after processing All instructions in memory in order to execute Memory management determines what is to be in memory Memory management activities Keeping

More information

What s New in VMware vsphere 4.1 Performance. VMware vsphere 4.1

What s New in VMware vsphere 4.1 Performance. VMware vsphere 4.1 What s New in VMware vsphere 4.1 Performance VMware vsphere 4.1 T E C H N I C A L W H I T E P A P E R Table of Contents Scalability enhancements....................................................................

More information

Network Systems for Emerging WAN Applications

Network Systems for Emerging WAN Applications Network Systems for Emerging WAN Applications Hitachi Review Vol. 48 (1999), No. 4 169 Akihiko Takase, D.Sc. OVERVIEW: This paper describes wide-area-network architecture from the viewpoints of networking

More information

CCNA Exploration1 Chapter 3: Application Layer Functionality and Protocols

CCNA Exploration1 Chapter 3: Application Layer Functionality and Protocols CCNA Exploration1 Chapter 3: Application Layer Functionality and Protocols LOCAL CISCO ACADEMY ELSYS TU INSTRUCTOR: STELA STEFANOVA 1 Objectives Functions of the three upper OSI model layers, network services

More information

Architectural Styles II

Architectural Styles II Architectural Styles II Software Architecture VO/KU (707.023/707.024) Denis Helic, Roman Kern KMI, TU Graz Nov 21, 2012 Denis Helic, Roman Kern (KMI, TU Graz) Architectural Styles II Nov 21, 2012 1 / 66

More information

Introduction 1.1 SERVER-CENTRIC IT ARCHITECTURE AND ITS LIMITATIONS

Introduction 1.1 SERVER-CENTRIC IT ARCHITECTURE AND ITS LIMITATIONS 1 Introduction The purpose of this chapter is to convey the basic idea underlying this book. To this end we will first describe conventional server-centric IT architecture and sketch out its limitations

More information

An Optimized Search Mechanism for Large Distributed Systems

An Optimized Search Mechanism for Large Distributed Systems An Optimized Search Mechanism for Large Distributed Systems Herwig Unger 1, Thomas Böhme, Markus Wulff 1, Gilbert Babin 3, and Peter Kropf 1 Fachbereich Informatik Universität Rostock D-1051 Rostock, Germany

More information

The BANDIT can also concentrate and switch multiple sources of Frame Relay traffic simultaneously.

The BANDIT can also concentrate and switch multiple sources of Frame Relay traffic simultaneously. encor! enetworks TM Version A, March 2008 2013 Encore Networks, Inc. All rights reserved. Routing with Frame Relay This chapter discusses Frame Relay routing. 4.1 Frame Relay You can configure use of synchronous

More information

Chapter-6. SUBJECT:- Operating System TOPICS:- I/O Management. Created by : - Sanjay Patel

Chapter-6. SUBJECT:- Operating System TOPICS:- I/O Management. Created by : - Sanjay Patel Chapter-6 SUBJECT:- Operating System TOPICS:- I/O Management Created by : - Sanjay Patel Disk Scheduling Algorithm 1) First-In-First-Out (FIFO) 2) Shortest Service Time First (SSTF) 3) SCAN 4) Circular-SCAN

More information

Memory-Link Compression Schemes: A Value Locality Perspective

Memory-Link Compression Schemes: A Value Locality Perspective Memory-Link Compression Schemes: A Value Locality Perspective Martin Thuresson, Lawrence Spracklen and Per Stenström IEEE Presented by Jean Niklas L orange and Caroline Sæhle, for TDT01, Norwegian University

More information

Real-Time Protocol (RTP)

Real-Time Protocol (RTP) Real-Time Protocol (RTP) Provides standard packet format for real-time application Typically runs over UDP Specifies header fields below Payload Type: 7 bits, providing 128 possible different types of

More information

Syllabus for Computer Science General Part I

Syllabus for Computer Science General Part I Distribution of Questions: Part I Q1. (Compulsory: 20 marks). Any ten questions to be answered out of fifteen questions, each carrying two marks (Group A 3 questions, Group B, Group C and Group D 4 questions

More information

Quality of Service II

Quality of Service II Quality of Service II Patrick J. Stockreisser p.j.stockreisser@cs.cardiff.ac.uk Lecture Outline Common QoS Approaches Best Effort Integrated Services Differentiated Services Integrated Services Integrated

More information

Protocols SPL/ SPL

Protocols SPL/ SPL Protocols 1 Application Level Protocol Design atomic units used by protocol: "messages" encoding reusable, protocol independent, TCP server, LinePrinting protocol implementation 2 Protocol Definition set

More information

Multiprocessor Systems. Chapter 8, 8.1

Multiprocessor Systems. Chapter 8, 8.1 Multiprocessor Systems Chapter 8, 8.1 1 Learning Outcomes An understanding of the structure and limits of multiprocessor hardware. An appreciation of approaches to operating system support for multiprocessor

More information

Chapter 4 NETWORK HARDWARE

Chapter 4 NETWORK HARDWARE Chapter 4 NETWORK HARDWARE 1 Network Devices As Organizations grow, so do their networks Growth in number of users Geographical Growth Network Devices : Are products used to expand or connect networks.

More information

ENSC 427: COMMUNICATION NETWORKS (Spring 2011) Final Report

ENSC 427: COMMUNICATION NETWORKS (Spring 2011) Final Report ENSC 427: COMMUNICATION NETWORKS (Spring 2011) Final Report Video Streaming over the 802.11g WLAN Technologies http://www.sfu.ca/~zxa7/ Zhenpeng Xue 301062408 zxa7@sfu.ca Page 2 of 16 Table of Contents

More information

High Availability through Warm-Standby Support in Sybase Replication Server A Whitepaper from Sybase, Inc.

High Availability through Warm-Standby Support in Sybase Replication Server A Whitepaper from Sybase, Inc. High Availability through Warm-Standby Support in Sybase Replication Server A Whitepaper from Sybase, Inc. Table of Contents Section I: The Need for Warm Standby...2 The Business Problem...2 Section II:

More information

L1: Introduction. Hui Chen, Ph.D. Dept. of Engineering & Computer Science Virginia State University Petersburg, VA 23806

L1: Introduction. Hui Chen, Ph.D. Dept. of Engineering & Computer Science Virginia State University Petersburg, VA 23806 L1: Introduction Hui Chen, Ph.D. Dept. of Engineering & Computer Science Virginia State University Petersburg, VA 23806 8/15/2016 CSCI 445 Fall 2016 1 Acknowledgements Some pictures used in this presentation

More information

Real-time grid computing for financial applications

Real-time grid computing for financial applications CNR-INFM Democritos and EGRID project E-mail: cozzini@democritos.it Riccardo di Meo, Ezio Corso EGRID project ICTP E-mail: {dimeo,ecorso}@egrid.it We describe the porting of a test case financial application

More information

PARALLEL TRAINING OF NEURAL NETWORKS FOR SPEECH RECOGNITION

PARALLEL TRAINING OF NEURAL NETWORKS FOR SPEECH RECOGNITION PARALLEL TRAINING OF NEURAL NETWORKS FOR SPEECH RECOGNITION Stanislav Kontár Speech@FIT, Dept. of Computer Graphics and Multimedia, FIT, BUT, Brno, Czech Republic E-mail: xkonta00@stud.fit.vutbr.cz In

More information

Design Patterns. SE3A04 Tutorial. Jason Jaskolka

Design Patterns. SE3A04 Tutorial. Jason Jaskolka SE3A04 Tutorial Jason Jaskolka Department of Computing and Software Faculty of Engineering McMaster University Hamilton, Ontario, Canada jaskolj@mcmaster.ca November 18/19, 2014 Jason Jaskolka 1 / 35 1

More information

Chapter 8: Memory-Management Strategies

Chapter 8: Memory-Management Strategies Chapter 8: Memory-Management Strategies Chapter 8: Memory Management Strategies Background Swapping Contiguous Memory Allocation Segmentation Paging Structure of the Page Table Example: The Intel 32 and

More information

Towards Quality of Service Based Resource Management for Cluster-Based Image Database

Towards Quality of Service Based Resource Management for Cluster-Based Image Database Towards Quality of Service Based Resource Management for Cluster-Based Image Database A. Brüning 1,F.Drews 2,M.Hoefer 1,O.Kao 3, and U. Rerrer 3 1 Department of Computer Science, Technical, University

More information

Lecture 13. Quality of Service II CM0256

Lecture 13. Quality of Service II CM0256 Lecture 13 Quality of Service II CM0256 Types of QoS Best Effort Services Integrated Services -- resource reservation network resources are assigned according to the application QoS request and subject

More information

Scalable Performance Analysis of Parallel Systems: Concepts and Experiences

Scalable Performance Analysis of Parallel Systems: Concepts and Experiences 1 Scalable Performance Analysis of Parallel Systems: Concepts and Experiences Holger Brunst ab and Wolfgang E. Nagel a a Center for High Performance Computing, Dresden University of Technology, 01062 Dresden,

More information

Improving Http-Server Performance by Adapted Multithreading

Improving Http-Server Performance by Adapted Multithreading Improving Http-Server Performance by Adapted Multithreading Jörg Keller LG Technische Informatik II FernUniversität Hagen 58084 Hagen, Germany email: joerg.keller@fernuni-hagen.de Olaf Monien Thilo Lardon

More information

QoS-Aware IPTV Routing Algorithms

QoS-Aware IPTV Routing Algorithms QoS-Aware IPTV Routing Algorithms Patrick McDonagh, Philip Perry, Liam Murphy. School of Computer Science and Informatics, University College Dublin, Belfield, Dublin 4. {patrick.mcdonagh, philip.perry,

More information

Running High Performance Computing Workloads on Red Hat Enterprise Linux

Running High Performance Computing Workloads on Red Hat Enterprise Linux Running High Performance Computing Workloads on Red Hat Enterprise Linux Imed Chihi Senior Technical Account Manager Red Hat Global Support Services 21 January 2014 Agenda 2 The case of HPC on commodity

More information

Data and Computer Communications. Protocols and Architecture

Data and Computer Communications. Protocols and Architecture Data and Computer Communications Protocols and Architecture Characteristics Direct or indirect Monolithic or structured Symmetric or asymmetric Standard or nonstandard Means of Communication Direct or

More information

Internetworking With TCP/IP

Internetworking With TCP/IP Internetworking With TCP/IP Vol II: Design, Implementation, and Internais DOUGLAS E. COMER and DAVID L. STEVENS Department of Computer Sciences Purdue University West Lafayette, IN 47907 PRENTICE HALL

More information

Optimizing TCP in a Cluster of Low-End Linux Machines

Optimizing TCP in a Cluster of Low-End Linux Machines Optimizing TCP in a Cluster of Low-End Linux Machines ABDALLA MAHMOUD, AHMED SAMEH, KHALED HARRAS, TAREK DARWICH Dept. of Computer Science, The American University in Cairo, P.O.Box 2511, Cairo, EGYPT

More information

Adaptive Cluster Computing using JavaSpaces

Adaptive Cluster Computing using JavaSpaces Adaptive Cluster Computing using JavaSpaces Jyoti Batheja and Manish Parashar The Applied Software Systems Lab. ECE Department, Rutgers University Outline Background Introduction Related Work Summary of

More information

Communication. Overview

Communication. Overview Communication Chapter 2 1 Overview Layered protocols Remote procedure call Remote object invocation Message-oriented communication Stream-oriented communication 2 Layered protocols Low-level layers Transport

More information

From Cluster Monitoring to Grid Monitoring Based on GRM *

From Cluster Monitoring to Grid Monitoring Based on GRM * From Cluster Monitoring to Grid Monitoring Based on GRM * Zoltán Balaton, Péter Kacsuk, Norbert Podhorszki and Ferenc Vajda MTA SZTAKI H-1518 Budapest, P.O.Box 63. Hungary {balaton, kacsuk, pnorbert, vajda}@sztaki.hu

More information

Introduction to Parallel Computing. CPS 5401 Fall 2014 Shirley Moore, Instructor October 13, 2014

Introduction to Parallel Computing. CPS 5401 Fall 2014 Shirley Moore, Instructor October 13, 2014 Introduction to Parallel Computing CPS 5401 Fall 2014 Shirley Moore, Instructor October 13, 2014 1 Definition of Parallel Computing Simultaneous use of multiple compute resources to solve a computational

More information

Intra-MIC MPI Communication using MVAPICH2: Early Experience

Intra-MIC MPI Communication using MVAPICH2: Early Experience Intra-MIC MPI Communication using MVAPICH: Early Experience Sreeram Potluri, Karen Tomko, Devendar Bureddy, and Dhabaleswar K. Panda Department of Computer Science and Engineering Ohio State University

More information

Future Generation Computer Systems. Task granularity policies for deploying bag-of-task applications on global grids

Future Generation Computer Systems. Task granularity policies for deploying bag-of-task applications on global grids Future Generation Computer Systems 29 (2012) 170 181 Contents lists available at SciVerse ScienceDirect Future Generation Computer Systems journal homepage: www.elsevier.com/locate/fgcs Task granularity

More information

Lecture 9: Load Balancing & Resource Allocation

Lecture 9: Load Balancing & Resource Allocation Lecture 9: Load Balancing & Resource Allocation Introduction Moler s law, Sullivan s theorem give upper bounds on the speed-up that can be achieved using multiple processors. But to get these need to efficiently

More information

Introduction to Parallel Computing

Introduction to Parallel Computing Introduction to Parallel Computing Introduction to Parallel Computing with MPI and OpenMP P. Ramieri Segrate, November 2016 Course agenda Tuesday, 22 November 2016 9.30-11.00 01 - Introduction to parallel

More information

Query Answering Using Inverted Indexes

Query Answering Using Inverted Indexes Query Answering Using Inverted Indexes Inverted Indexes Query Brutus AND Calpurnia J. Pei: Information Retrieval and Web Search -- Query Answering Using Inverted Indexes 2 Document-at-a-time Evaluation

More information