Towards Jungle Computing with Ibis/Constellation
|
|
- Timothy Payne
- 5 years ago
- Views:
Transcription
1 Towards Jungle Computing with Ibis/Constellation Jason Maassen, Niels Drost Henri Bal, Frank Seinstra Department of Computer Science VU University, Amsterdam, The Netherlands
2 Introduction HPC is entering many domains Not just: physics / chemistry / climate modelling Also: semantic web / medical / multimedia analysis / neuroinformatics / remote sensing / astronomy /... HPC is becoming more complex Not just large SMP or clusters, instead: Clusters of SMPs / Grids / Clouds / Supers /... Heterogenous machines using GPU / Cell / FPGA It s a jungle out there 3DAPAS Workshop
3 Example Domain Computational Astrophysics (amusecode.org)
4 Jungle Computing Worst case computing... as required by users Arbitrary combination of distributed, hierarchical, and heterogenous computing 3DAPAS Workshop
5 Many Task Computing According to Raicu, Foster, et al [SC 08] High-performance computations comprising multiple distinct activities, coupled via file system operations or message passing. Tasks may be small or large, uni-processor or multi-processor, compute-intensive or data-intensive. The set of tasks may be static or dynamic, homogeneous or heterogeneous, loosely coupled or tightly coupled. The aggregate number of tasks, quantity of computing, and volumes of data may be extremely large. Applications are dynamic and heterogeneous workflows / DAGs of activities 3DAPAS Workshop
6 MTC in the Jungle MTC has advantages for Jungle Computing Many distinct activities Can be implemented independently using the tools and targeted to the HPC architecture, that best suit them Reduced programming complexity Complete applications are constructed using sequences and combinations of activities 3DAPAS Workshop
7 Constellation MTC system for Jungle Computing Model based on: activities (tasks) executors (resources) contexts (matchmaking) events (communication) 3DAPAS Workshop
8 Constellation Model Application Application: set of activities Distinct tasks Size and complexity may vary Targeted at specific HPC platform (Loosly) Coupled using events Often wrapper around existing code Similar to workflow or DAG of tasks Dynamic and unlimited in size 3DAPAS Workshop
9 Constellation Model Hardware Hardware: set of executors Capable of running activities May represent anything from a single core to an entire cluster, a GPU, etc. May be application specific Provides an application specific heterogeneous resource pool 3DAPAS Workshop
10 Constellation Model Context Both activities and executors are tagged with a context Application defined label (+ rank) Used to defines relationship between activites and executors, e.g.: Data dependencies, hardware requirements,... May combine contexts Executors may have preference for label or rank 3DAPAS WorkShop
11 Constellation Model Matchmaking RTS performs load-balancing and match-making Ensures activities are forwarded to a suitable executor Tries to keep all executors busy Uses context-aware work-stealing RTS also performs event routing Based on unique activity identifier ComplexHPC Spring School
12 Constellation API 3DAPAS Workshop
13 Constellation API 3DAPAS Workshop
14 DACH 2008 Data Challenge in conjunction with IEEE Cluster/Grid 2008 Supernova detection Analyse 1052 image pairs on 11 clusters (Intrigger) Sequential executable provided 3DAPAS Workshop
15 DACH 2008 Problem Main problems: Data distribution Heterogeneity of work and hardware Load balancing 3DAPAS Workshop
16 DACH 2008 Workflow Winning approach in 2008: Parallelize workflow to improve hardware utilization Create hierarchical master worker framework Scheduling heuristics using data location and size 3DAPAS Workshop
17 Constellation Version Option 1: Monolythic Wrap entire application in a single activity One activity per image pair Wrap each machine in one executor Multiple cores per executor Use context to influence order and placement of each of activities 3DAPAS Workshop
18 Evaluation Intrigger not available Instead we use DAS3+DAS4 5+6 clusters in the Netherlands Mix of 2/4/8/12/48 core machines Various types of GPUs Three Scenarios Data locality (Executor granularity) Heterogeneous processing 3DAPAS Workshop
19 Scenario 1 Data Locality Data distributed over 4 clusters of DAS3 + DAS4 Use context to express data locality and preferred processing order Adapt context to tune application No change in application 3DAPAS Workshop
20 Scenario 1 Results Activity Executor Effect any any Random order any,50 VU3, VU4,50 any, biggest VU3, biggest Sorted by size Local only Sorted by size VU3, VU4, any,50 VU3, any, biggest Preference for local Fallback to any, Sorted by size 3DAPAS Workshop
21 Constellation Version Option 2: Workflow Wrap each stage in activity Wrap each core executor Use context to influence order and placement of each of the jobs 3DAPAS Workshop
22 Scenario 3: Heterogeneous System 18 node GPU cluster 8 cores + 1 GPU per node Activity: single task Executor: 1 core (top) 1 core or GPU (bottom) Replaced activity 7.2 with GPU version. Label activities and executors accordingly Significant performance gain. ComplexHPC Spring School
23 Conclusions We think Jungle Computing is a neccesity for some application areas. Constellation offers a suitable model (MTC) to create such applications. Initial experiments show that Constellation works well for a wide range of hardware configurations Easy to reconfigure applications to match resources Allows integration of specialized accellerator codes Suitable basis for a Jungle Computing model 3DAPAS Workshop
24 Future Work Application development AMUSE Remote Sensing Climate modelling Platform improvements Easier integration of existing codes Smart/automatic deployment/tuning of executors Improve data handling Better monitoring 3DAPAS Workshop
25 Questions? 3DAPAS Workshop
26 Scenario 2 Executor Granularity 30 largest images only Single 48 core machine Activity: entire application (a-c) single task (d) Executor: [n]-cores No change in application for experiment (a-c) Only change executor config. Completely ported application in (d) Significant performance gain! 3DAPAS Workshop
MOHA: Many-Task Computing Framework on Hadoop
Apache: Big Data North America 2017 @ Miami MOHA: Many-Task Computing Framework on Hadoop Soonwook Hwang Korea Institute of Science and Technology Information May 18, 2017 Table of Contents Introduction
More informationSynonymous with supercomputing Tightly-coupled applications Implemented using Message Passing Interface (MPI) Large of amounts of computing for short
Synonymous with supercomputing Tightly-coupled applications Implemented using Message Passing Interface (MPI) Large of amounts of computing for short periods of time Usually requires low latency interconnects
More informationDistributed ASCI Supercomputer DAS-1 DAS-2 DAS-3 DAS-4 DAS-5
Distributed ASCI Supercomputer DAS-1 DAS-2 DAS-3 DAS-4 DAS-5 Paper IEEE Computer (May 2016) What is DAS? Distributed common infrastructure for Dutch Computer Science Distributed: multiple (4-6) clusters
More informationTypically applied in clusters and grids Loosely-coupled applications with sequential jobs Large amounts of computing for long periods of times
Typically applied in clusters and grids Loosely-coupled applications with sequential jobs Large amounts of computing for long periods of times Measured in operations per month or years 2 Bridge the gap
More informationDistributed ASCI Supercomputer DAS-1 DAS-2 DAS-3 DAS-4 DAS-5
Distributed ASCI Supercomputer DAS-1 DAS-2 DAS-3 DAS-4 DAS-5 Paper IEEE Computer (May 2016) What is DAS? Distributed common infrastructure for Dutch Computer Science Distributed: multiple (4-6) clusters
More informationAutomating Real-time Seismic Analysis
Automating Real-time Seismic Analysis Through Streaming and High Throughput Workflows Rafael Ferreira da Silva, Ph.D. http://pegasus.isi.edu Do we need seismic analysis? Pegasus http://pegasus.isi.edu
More informationGrid Scheduling Architectures with Globus
Grid Scheduling Architectures with Workshop on Scheduling WS 07 Cetraro, Italy July 28, 2007 Ignacio Martin Llorente Distributed Systems Architecture Group Universidad Complutense de Madrid 1/38 Contents
More informationMATE-EC2: A Middleware for Processing Data with Amazon Web Services
MATE-EC2: A Middleware for Processing Data with Amazon Web Services Tekin Bicer David Chiu* and Gagan Agrawal Department of Compute Science and Engineering Ohio State University * School of Engineering
More informationZorilla: a peer-to-peer middleware for real-world distributed systems
CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE Concurrency Computat.: Pract. Exper. (2011) Published online in Wiley Online Library (wileyonlinelibrary.com)..1713 Zorilla: a peer-to-peer middleware
More informationPERFORMANCE ANALYSIS AND OPTIMIZATION OF MULTI-CLOUD COMPUITNG FOR LOOSLY COUPLED MTC APPLICATIONS
PERFORMANCE ANALYSIS AND OPTIMIZATION OF MULTI-CLOUD COMPUITNG FOR LOOSLY COUPLED MTC APPLICATIONS V. Prasathkumar, P. Jeevitha Assiatant Professor, Department of Information Technology Sri Shakthi Institute
More informationWrite a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical
Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or
More informationSemantic Web in a Constrained Environment
Semantic Web in a Constrained Environment Laurens Rietveld and Stefan Schlobach Department of Computer Science, VU University Amsterdam, The Netherlands {laurens.rietveld,k.s.schlobach}@vu.nl Abstract.
More informationWorkloads Programmierung Paralleler und Verteilter Systeme (PPV)
Workloads Programmierung Paralleler und Verteilter Systeme (PPV) Sommer 2015 Frank Feinbube, M.Sc., Felix Eberhardt, M.Sc., Prof. Dr. Andreas Polze Workloads 2 Hardware / software execution environment
More informationUVA HPC & BIG DATA COURSE INTRODUCTORY LECTURES. Adam Belloum
UVA HPC & BIG DATA COURSE INTRODUCTORY LECTURES Adam Belloum Introduction to Parallel programming distributed systems Parallel programming MPI/openMP/RMI Service Oriented Architecture and Web Service Grid
More informationIntroduction & Motivation Problem Statement Proposed Work Evaluation Conclusions Future Work
Introduction & Motivation Problem Statement Proposed Work Evaluation Conclusions Future Work Introduction & Motivation Problem Statement Proposed Work Evaluation Conclusions Future Work Today (2014):
More informationAdaptive Cluster Computing using JavaSpaces
Adaptive Cluster Computing using JavaSpaces Jyoti Batheja and Manish Parashar The Applied Software Systems Lab. ECE Department, Rutgers University Outline Background Introduction Related Work Summary of
More informationEFFICIENT ALLOCATION OF DYNAMIC RESOURCES IN A CLOUD
EFFICIENT ALLOCATION OF DYNAMIC RESOURCES IN A CLOUD S.THIRUNAVUKKARASU 1, DR.K.P.KALIYAMURTHIE 2 Assistant Professor, Dept of IT, Bharath University, Chennai-73 1 Professor& Head, Dept of IT, Bharath
More informationOverview Past Work Future Work. Motivation Proposal. Work-in-Progress
Overview Past Work Future Work Motivation Proposal Work-in-Progress 2 HPC: High-Performance Computing Synonymous with supercomputing Tightly-coupled applications Implemented using Message Passing Interface
More informationFACULTY OF ENGINEERING B.E. 4/4 (CSE) II Semester (Old) Examination, June Subject : Information Retrieval Systems (Elective III) Estelar
B.E. 4/4 (CSE) II Semester (Old) Examination, June 2014 Subject : Information Retrieval Systems Code No. 6306 / O 1 Define Information retrieval systems. 3 2 What is precision and recall? 3 3 List the
More informationGrid Scheduler. Grid Information Service. Local Resource Manager L l Resource Manager. Single CPU (Time Shared Allocation) (Space Shared Allocation)
Scheduling on the Grid 1 2 Grid Scheduling Architecture User Application Grid Scheduler Grid Information Service Local Resource Manager Local Resource Manager Local L l Resource Manager 2100 2100 2100
More informationThe Use of Cloud Computing Resources in an HPC Environment
The Use of Cloud Computing Resources in an HPC Environment Bill, Labate, UCLA Office of Information Technology Prakashan Korambath, UCLA Institute for Digital Research & Education Cloud computing becomes
More informationCollaboration Support in Open Hypermedia Environments
Collaboration Support in Open Hypermedia Environments Jörg M. Haake & Weigang Wang GMD - German National Research Center for Information Technology Integrated Publication and Information Systems Institute
More informationParallel VS Distributed
Parallel VS Distributed The distributed systems tend to be multicomputers whose nodes made of processor plus its private memory whereas parallel computer refers to a shared memory multiprocessor. In Parallel
More informationGrid-Based Genetic Algorithm Approach to Colour Image Segmentation
Grid-Based Genetic Algorithm Approach to Colour Image Segmentation Marco Gallotta Keri Woods Supervised by Audrey Mbogho Image Segmentation Identifying and extracting distinct, homogeneous regions from
More informationAdvanced School in High Performance and GRID Computing November Introduction to Grid computing.
1967-14 Advanced School in High Performance and GRID Computing 3-14 November 2008 Introduction to Grid computing. TAFFONI Giuliano Osservatorio Astronomico di Trieste/INAF Via G.B. Tiepolo 11 34131 Trieste
More informationMULTI-THREADED QUERIES
15-721 Project 3 Final Presentation MULTI-THREADED QUERIES Wendong Li (wendongl) Lu Zhang (lzhang3) Rui Wang (ruiw1) Project Objective Intra-operator parallelism Use multiple threads in a single executor
More informationHybrid Model Parallel Programs
Hybrid Model Parallel Programs Charlie Peck Intermediate Parallel Programming and Cluster Computing Workshop University of Oklahoma/OSCER, August, 2010 1 Well, How Did We Get Here? Almost all of the clusters
More informationModule 1: Introduction
Module 1: Introduction What is an operating system? Simple Batch Systems Multiprogramming Batched Systems Time-Sharing Systems Personal-Computer Systems Parallel Systems Distributed Systems Real -Time
More informationParallel DBMS. Parallel Database Systems. PDBS vs Distributed DBS. Types of Parallelism. Goals and Metrics Speedup. Types of Parallelism
Parallel DBMS Parallel Database Systems CS5225 Parallel DB 1 Uniprocessor technology has reached its limit Difficult to build machines powerful enough to meet the CPU and I/O demands of DBMS serving large
More informationPegasus. Automate, recover, and debug scientific computations. Rafael Ferreira da Silva.
Pegasus Automate, recover, and debug scientific computations. Rafael Ferreira da Silva http://pegasus.isi.edu Experiment Timeline Scientific Problem Earth Science, Astronomy, Neuroinformatics, Bioinformatics,
More informationGrid Computing. Lectured by: Dr. Pham Tran Vu Faculty of Computer and Engineering HCMC University of Technology
Grid Computing Lectured by: Dr. Pham Tran Vu Email: ptvu@cse.hcmut.edu.vn 1 Grid Architecture 2 Outline Layer Architecture Open Grid Service Architecture 3 Grid Characteristics Large-scale Need for dynamic
More informationOverview of research activities Toward portability of performance
Overview of research activities Toward portability of performance Do dynamically what can t be done statically Understand evolution of architectures Enable new programming models Put intelligence into
More informationDynamic Load-Balanced Multicast for Data-Intensive Applications on Clouds 1
Dynamic Load-Balanced Multicast for Data-Intensive Applications on Clouds 1 Contents: Introduction Multicast on parallel distributed systems Multicast on P2P systems Multicast on clouds High performance
More informationAuto Management for Apache Kafka and Distributed Stateful System in General
Auto Management for Apache Kafka and Distributed Stateful System in General Jiangjie (Becket) Qin Data Infrastructure @LinkedIn GIAC 2017, 12/23/17@Shanghai Agenda Kafka introduction and terminologies
More informationWorkflows and Scheduling
Workflows and Scheduling Frank Röder Arbeitsbereich Wissenschaftliches Rechnen Fachbereich Informatik Fakultät für Mathematik, Informatik und Naturwissenschaften Universität Hamburg 14-12-2015 Frank Röder
More informationConnecting the e-infrastructure chain
Connecting the e-infrastructure chain Internet2 Spring Meeting, Arlington, April 23 rd, 2012 Peter Hinrich & Migiel de Vos Topics - About SURFnet - Motivation: Big data & collaboration - Collaboration
More informationGeneric Framework for Parallel and Distributed Processing of Video-Data
Generic Framework for Parallel and Distributed Processing of Video-Data Dirk Farin and Peter H. N. de With,2 University Eindhoven, Signal Processing Systems, LG., 56 MB Eindhoven, Netherlands d.s.farin@tue.nl
More informationOVERHEADS ENHANCEMENT IN MUTIPLE PROCESSING SYSTEMS BY ANURAG REDDY GANKAT KARTHIK REDDY AKKATI
CMPE 655- MULTIPLE PROCESSOR SYSTEMS OVERHEADS ENHANCEMENT IN MUTIPLE PROCESSING SYSTEMS BY ANURAG REDDY GANKAT KARTHIK REDDY AKKATI What is MULTI PROCESSING?? Multiprocessing is the coordinated processing
More informationChapter 1: Introduction
Chapter 1: Introduction What is an Operating System? Mainframe Systems Desktop Systems Multiprocessor Systems Distributed Systems Clustered System Real -Time Systems Handheld Systems Computing Environments
More informationParallel Architectures
Parallel Architectures CPS343 Parallel and High Performance Computing Spring 2018 CPS343 (Parallel and HPC) Parallel Architectures Spring 2018 1 / 36 Outline 1 Parallel Computer Classification Flynn s
More informationECE 574 Cluster Computing Lecture 1
ECE 574 Cluster Computing Lecture 1 Vince Weaver http://web.eece.maine.edu/~vweaver vincent.weaver@maine.edu 22 January 2019 ECE574 Distribute and go over syllabus http://web.eece.maine.edu/~vweaver/classes/ece574/ece574_2019s.pdf
More informationParallel Computing with MATLAB
Parallel Computing with MATLAB CSCI 4850/5850 High-Performance Computing Spring 2018 Tae-Hyuk (Ted) Ahn Department of Computer Science Program of Bioinformatics and Computational Biology Saint Louis University
More informationMediaTek CorePilot 2.0. Delivering extreme compute performance with maximum power efficiency
MediaTek CorePilot 2.0 Heterogeneous Computing Technology Delivering extreme compute performance with maximum power efficiency In July 2013, MediaTek delivered the industry s first mobile system on a chip
More informationDRYAD / DRYADLINQ OVERVIEW. Xavier Pillons, Principal Program Manager, Technical Computing Customer Advocate Team
DRYAD / DRYADLINQ OVERVIEW Xavier Pillons, Principal Program Manager, Technical Computing Customer Advocate Team Data Intensive Scalable Computing (DISC) Market Customer needs for DISC lie on a spectrum
More informationA Fully Automated Faulttolerant. Distributed Video Processing and Off site Replication
A Fully Automated Faulttolerant System for Distributed Video Processing and Off site Replication George Kola, Tevfik Kosar and Miron Livny University of Wisconsin-Madison June 2004 What is the talk about?
More informationEnergy Efficient Computing Systems (EECS) Magnus Jahre Coordinator, EECS
Energy Efficient Computing Systems (EECS) Magnus Jahre Coordinator, EECS Who am I? Education Master of Technology, NTNU, 2007 PhD, NTNU, 2010. Title: «Managing Shared Resources in Chip Multiprocessor Memory
More informationIntroduction to Parallel Programming
Introduction to Parallel Programming David Lifka lifka@cac.cornell.edu May 23, 2011 5/23/2011 www.cac.cornell.edu 1 y What is Parallel Programming? Using more than one processor or computer to complete
More informationDistributed Systems. Thoai Nam Faculty of Computer Science and Engineering HCMC University of Technology
Distributed Systems Thoai Nam Faculty of Computer Science and Engineering HCMC University of Technology Chapter 1: Introduction Distributed Systems Hardware & software Transparency Scalability Distributed
More informationL3.4. Data Management Techniques. Frederic Desprez Benjamin Isnard Johan Montagnat
Grid Workflow Efficient Enactment for Data Intensive Applications L3.4 Data Management Techniques Authors : Eddy Caron Frederic Desprez Benjamin Isnard Johan Montagnat Summary : This document presents
More informationMultiple Broker Support by Grid Portals* Extended Abstract
1. Introduction Multiple Broker Support by Grid Portals* Extended Abstract Attila Kertesz 1,3, Zoltan Farkas 1,4, Peter Kacsuk 1,4, Tamas Kiss 2,4 1 MTA SZTAKI Computer and Automation Research Institute
More informationDAS 1-4: Experiences with the Distributed ASCI Supercomputers
DAS 1-4: Experiences with the Distributed ASCI Supercomputers Henri Bal bal@cs.vu.nl Kees Verstoep versto@cs.vu.nl Vrije Universiteit Amsterdam Introduction DAS: shared distributed infrastructure for experimental
More informationIntroduction to Parallel Programming
Introduction to Parallel Programming January 14, 2015 www.cac.cornell.edu What is Parallel Programming? Theoretically a very simple concept Use more than one processor to complete a task Operationally
More informationStorage and Compute Resource Management via DYRE, 3DcacheGrid, and CompuStore Ioan Raicu, Ian Foster
Storage and Compute Resource Management via DYRE, 3DcacheGrid, and CompuStore Ioan Raicu, Ian Foster. Overview Both the industry and academia have an increase demand for good policies and mechanisms to
More informationSUPPORTING EFFICIENT EXECUTION OF MANY-TASK APPLICATIONS WITH EVEREST
SUPPORTING EFFICIENT EXECUTION OF MANY-TASK APPLICATIONS WITH EVEREST O.V. Sukhoroslov Centre for Distributed Computing, Institute for Information Transmission Problems, Bolshoy Karetny per. 19 build.1,
More informationBUYING SERVER HARDWARE FOR A SCALABLE VIRTUAL INFRASTRUCTURE
E-Guide BUYING SERVER HARDWARE FOR A SCALABLE VIRTUAL INFRASTRUCTURE SearchServer Virtualization P art 1 of this series explores how trends in buying server hardware have been influenced by the scale-up
More informationIntroduction to Parallel Computing
Introduction to Parallel Computing Bootcamp for SahasraT 7th September 2018 Aditya Krishna Swamy adityaks@iisc.ac.in SERC, IISc Acknowledgments Akhila, SERC S. Ethier, PPPL P. Messina, ECP LLNL HPC tutorials
More informationCloudKon: a Cloud enabled Distributed task execution framework
CloudKon: a Cloud enabled Distributed task execution framework Iman Sadooghi, Ioan Raicu isadoogh@iit.edu, iraicu@cs.iit.edu Department of Computer Science, Illinois Institute of Technology, Chicago IL,
More informationJob-Oriented Monitoring of Clusters
Job-Oriented Monitoring of Clusters Vijayalaxmi Cigala Dhirajkumar Mahale Monil Shah Sukhada Bhingarkar Abstract There has been a lot of development in the field of clusters and grids. Recently, the use
More informationFederated XDMoD Requirements
Federated XDMoD Requirements Date Version Person Change 2016-04-08 1.0 draft XMS Team Initial version Summary Definitions Assumptions Data Collection Local XDMoD Installation Module Support Data Federation
More informationChapter 1: Distributed Information Systems
Chapter 1: Distributed Information Systems Contents - Chapter 1 Design of an information system Layers and tiers Bottom up design Top down design Architecture of an information system One tier Two tier
More informationFLAT DATACENTER STORAGE. Paper-3 Presenter-Pratik Bhatt fx6568
FLAT DATACENTER STORAGE Paper-3 Presenter-Pratik Bhatt fx6568 FDS Main discussion points A cluster storage system Stores giant "blobs" - 128-bit ID, multi-megabyte content Clients and servers connected
More informationUse cases. Faces tagging in photo and video, enabling: sharing media editing automatic media mashuping entertaining Augmented reality Games
Viewdle Inc. 1 Use cases Faces tagging in photo and video, enabling: sharing media editing automatic media mashuping entertaining Augmented reality Games 2 Why OpenCL matter? OpenCL is going to bring such
More informationTechnology for a better society. SINTEF ICT, Applied Mathematics, Heterogeneous Computing Group
Technology for a better society SINTEF, Applied Mathematics, Heterogeneous Computing Group Trond Hagen GPU Computing Seminar, SINTEF Oslo, October 23, 2009 1 Agenda 12:30 Introduction and welcoming Trond
More informationIntroduction to Operating Systems
Introduction to Operating Systems B. Ramamurthy (adapted from C. Egert s and W. Stallings slides) 1/25/02 CSE421, Spring 2002 1 Introduction A computer system consists of hardware system programs application
More informationMapReduce for Data Intensive Scientific Analyses
apreduce for Data Intensive Scientific Analyses Jaliya Ekanayake Shrideep Pallickara Geoffrey Fox Department of Computer Science Indiana University Bloomington, IN, 47405 5/11/2009 Jaliya Ekanayake 1 Presentation
More informationMediaTek CorePilot. Heterogeneous Multi-Processing Technology. Delivering extreme compute performance with maximum power efficiency
MediaTek CorePilot Heterogeneous Multi-Processing Technology Delivering extreme compute performance with maximum power efficiency In July 2013, MediaTek delivered the industry s first mobile system on
More informationForming an ad-hoc nearby storage, based on IKAROS and social networking services
Forming an ad-hoc nearby storage, based on IKAROS and social networking services Christos Filippidis1, Yiannis Cotronis2 and Christos Markou1 1 Institute of Nuclear & Particle Physics, NCSR Demokritos,
More informationThe Cray Rainier System: Integrated Scalar/Vector Computing
THE SUPERCOMPUTER COMPANY The Cray Rainier System: Integrated Scalar/Vector Computing Per Nyberg 11 th ECMWF Workshop on HPC in Meteorology Topics Current Product Overview Cray Technology Strengths Rainier
More informationScheduling Algorithms in Large Scale Distributed Systems
Scheduling Algorithms in Large Scale Distributed Systems Prof.dr.ing. Florin Pop University Politehnica of Bucharest, Faculty of Automatic Control and Computers (ACS-UPB) National Institute for Research
More informationModels for model integration
Models for model integration Oxford, UK, June 27, 2017 Daniel S. Katz Assistant Director for Scientific Software & Applications, NCSA Research Associate Professor, CS Research Associate Professor, ECE
More informationAn Introduction to GPFS
IBM High Performance Computing July 2006 An Introduction to GPFS gpfsintro072506.doc Page 2 Contents Overview 2 What is GPFS? 3 The file system 3 Application interfaces 4 Performance and scalability 4
More informationOperating Systems Fundamentals. What is an Operating System? Focus. Computer System Components. Chapter 1: Introduction
Operating Systems Fundamentals Overview of Operating Systems Ahmed Tawfik Modern Operating Systems are increasingly complex Operating System Millions of Lines of Code DOS 0.015 Windows 95 11 Windows 98
More informationCloud Programming. Programming Environment Oct 29, 2015 Osamu Tatebe
Cloud Programming Programming Environment Oct 29, 2015 Osamu Tatebe Cloud Computing Only required amount of CPU and storage can be used anytime from anywhere via network Availability, throughput, reliability
More informationDisk Cache-Aware Task Scheduling
Disk ache-aware Task Scheduling For Data-Intensive and Many-Task Workflow Masahiro Tanaka and Osamu Tatebe University of Tsukuba, JST/REST Japan Science and Technology Agency IEEE luster 2014 2014-09-24
More informationOvercoming Data Locality: an In-Memory Runtime File System with Symmetrical Data Distribution
Overcoming Data Locality: an In-Memory Runtime File System with Symmetrical Data Distribution Alexandru Uta a,, Andreea Sandu a, Thilo Kielmann a a Dept. of Computer Science, VU University Amsterdam, The
More informationComputing over the Internet: Beyond Embarrassingly Parallel Applications. BOINC Workshop 09. Fernando Costa
Computing over the Internet: Beyond Embarrassingly Parallel Applications BOINC Workshop 09 Barcelona Fernando Costa University of Coimbra Overview Motivation Computing over Large Datasets Supporting new
More informationCOMP528: Multi-core and Multi-Processor Computing
COMP528: Multi-core and Multi-Processor Computing Dr Michael K Bane, G14, Computer Science, University of Liverpool m.k.bane@liverpool.ac.uk https://cgi.csc.liv.ac.uk/~mkbane/comp528 2X So far Why and
More informationIBM Data Science Experience White paper. SparkR. Transforming R into a tool for big data analytics
IBM Data Science Experience White paper R Transforming R into a tool for big data analytics 2 R Executive summary This white paper introduces R, a package for the R statistical programming language that
More informationMPI Optimizations via MXM and FCA for Maximum Performance on LS-DYNA
MPI Optimizations via MXM and FCA for Maximum Performance on LS-DYNA Gilad Shainer 1, Tong Liu 1, Pak Lui 1, Todd Wilde 1 1 Mellanox Technologies Abstract From concept to engineering, and from design to
More informationProcessor Architecture and Interconnect
Processor Architecture and Interconnect What is Parallelism? Parallel processing is a term used to denote simultaneous computation in CPU for the purpose of measuring its computation speeds. Parallel Processing
More informationPortable Heterogeneous High-Performance Computing via Domain-Specific Virtualization. Dmitry I. Lyakh.
Portable Heterogeneous High-Performance Computing via Domain-Specific Virtualization Dmitry I. Lyakh liakhdi@ornl.gov This research used resources of the Oak Ridge Leadership Computing Facility at the
More informationWhite. Paper. EMC Isilon Scale-out Without Compromise. July, 2012
White Paper EMC Isilon Scale-out Without Compromise By Terri McClure, Senior Analyst July, 2012 This ESG White Paper was commissioned by EMC and is distributed under license from ESG. 2012, The Enterprise
More informationAdventures in Load Balancing at Scale: Successes, Fizzles, and Next Steps
Adventures in Load Balancing at Scale: Successes, Fizzles, and Next Steps Rusty Lusk Mathematics and Computer Science Division Argonne National Laboratory Outline Introduction Two abstract programming
More informationIan Foster, An Overview of Distributed Systems
The advent of computation can be compared, in terms of the breadth and depth of its impact on research and scholarship, to the invention of writing and the development of modern mathematics. Ian Foster,
More informationMark Sandstrom ThroughPuter, Inc.
Hardware Implemented Scheduler, Placer, Inter-Task Communications and IO System Functions for Many Processors Dynamically Shared among Multiple Applications Mark Sandstrom ThroughPuter, Inc mark@throughputercom
More informationNFS, GPFS, PVFS, Lustre Batch-scheduled systems: Clusters, Grids, and Supercomputers Programming paradigm: HPC, MTC, and HTC
Segregated storage and compute NFS, GPFS, PVFS, Lustre Batch-scheduled systems: Clusters, Grids, and Supercomputers Programming paradigm: HPC, MTC, and HTC Co-located storage and compute HDFS, GFS Data
More informationExecuting dynamic heterogeneous workloads on Blue Waters with RADICAL-Pilot
Executing dynamic heterogeneous workloads on Blue Waters with RADICAL-Pilot Research in Advanced DIstributed Cyberinfrastructure & Applications Laboratory (RADICAL) Rutgers University http://radical.rutgers.edu
More informationPBS PROFESSIONAL VS. MICROSOFT HPC PACK
PBS PROFESSIONAL VS. MICROSOFT HPC PACK On the Microsoft Windows Platform PBS Professional offers many features which are not supported by Microsoft HPC Pack. SOME OF THE IMPORTANT ADVANTAGES OF PBS PROFESSIONAL
More informationCase Studies in Storage Access by Loosely Coupled Petascale Applications
Case Studies in Storage Access by Loosely Coupled Petascale Applications Justin M Wozniak and Michael Wilde Petascale Data Storage Workshop at SC 09 Portland, Oregon November 15, 2009 Outline Scripted
More informationQOS BASED SCHEDULING OF WORKFLOWS IN CLOUD COMPUTING UPNP ARCHITECTURE
QOS BASED SCHEDULING OF WORKFLOWS IN CLOUD COMPUTING UPNP ARCHITECTURE 1 K. Ramkumar, 2 Dr.G.Gunasekaran 1Research Scholar, Computer Science and Engineering Manonmaniam Sundaranar University Tirunelveli
More informationPervasive DataRush TM
Pervasive DataRush TM Parallel Data Analysis with KNIME www.pervasivedatarush.com Company Overview Global Software Company Tens of thousands of users across the globe Americas, EMEA, Asia ~230 employees
More informationBUILDING A GPU-FOCUSED CI SOLUTION
BUILDING A GPU-FOCUSED CI SOLUTION Mike Wendt @mike_wendt github.com/nvidia github.com/mike-wendt Need for CPU CI Challenges of GPU CI Methods to Implement GPU CI AGENDA Improving GPU CI Today Demo Lessons
More informationKarthik Narayanan, Santosh Madiraju EEL Embedded Systems Seminar 1/41 1
Karthik Narayanan, Santosh Madiraju EEL6935 - Embedded Systems Seminar 1/41 1 Efficient Search Space Exploration for HW-SW Partitioning Hardware/Software Codesign and System Synthesis, 2004. CODES + ISSS
More informationParallel Programming. Presentation to Linux Users of Victoria, Inc. November 4th, 2015
Parallel Programming Presentation to Linux Users of Victoria, Inc. November 4th, 2015 http://levlafayette.com 1.0 What Is Parallel Programming? 1.1 Historically, software has been written for serial computation
More informationChapter 1: Introduction
Chapter 1: Introduction What is an operating system? Simple Batch Systems Multiprogramming Batched Systems Time-Sharing Systems Personal-Computer Systems Parallel Systems Distributed Systems Real -Time
More informationHPC learning using Cloud infrastructure
HPC learning using Cloud infrastructure Florin MANAILA IT Architect florin.manaila@ro.ibm.com Cluj-Napoca 16 March, 2010 Agenda 1. Leveraging Cloud model 2. HPC on Cloud 3. Recent projects - FutureGRID
More informationCees de Laat University of Amsterdam
GreenClouds Cees de Laat University of Amsterdam Towards Hybrid Networking! Costs of photonic equipment 10% of switching 10 % of full routing for same throughput! Photonic vs Optical (optical used for
More informationIoan Raicu Distributed Systems Laboratory Computer Science Department University of Chicago
Falkon, a Fast and Light-weight task execution framework for Clusters, Grids, and Supercomputers Ioan Raicu Distributed Systems Laboratory Computer Science Department University of Chicago In Collaboration
More informationCSD3 The Cambridge Service for Data Driven Discovery. A New National HPC Service for Data Intensive science
CSD3 The Cambridge Service for Data Driven Discovery A New National HPC Service for Data Intensive science Dr Paul Calleja Director of Research Computing University of Cambridge Problem statement Today
More informationSpark and HPC for High Energy Physics Data Analyses
Spark and HPC for High Energy Physics Data Analyses Marc Paterno, Jim Kowalkowski, and Saba Sehrish 2017 IEEE International Workshop on High-Performance Big Data Computing Introduction High energy physics
More information