MPI: A Message-Passing Interface Standard

Transcription

1 MPI: A Message-Passing Interface Standard Version 2.1 Message Passing Interface Forum June 23, 2008

2 Contents Acknowledgments xvl1 1 Introduction to MPI Overview and Goals Background of MPI Background of MPI-1.1, MPI-1.2, and MPI Background of MPI-1.3 and MPI Who Should Use This Standard? What Platforms Are Targets For Implementation? What Is Included In The Standard? What Is Not Included In The Standard? Organization of this Document fi 2 MPI Terms and Conventions Document Notation Naming Conventions Procedure Specification Semantic Terms Data Types Opaque Objects Array Arguments State Named Constants Choice Addresses File Offsets Language Binding Deprecated Names and Functions Fortran Binding Issues C Binding Issues C++ Binding Issues Functions and Macros Processes Error Handling Implementation Issues Independence of Basic Runtime Routines Interaction with Signals 24 v

3 2.10 Examples 24 3 Point-to-Point Communication Introduction Blocking Send and Receive Operations Blocking Send Message Data Message Envelope Blocking Receive Return Status Passing MPLSTATUSJGNORE for Status Data Type Matching and Data Conversion Type Matching Rules 34 Type MPLCHARACTER Data Conversion Communication Modes Semantics of Point-to-Point Communication Buffer Allocation and Usage Model Implementation of Buffered Mode Nonblocking Communication Communication Request Objects Communication Initiation Communication Completion Semantics of Nonblocking Communications Multiple Completions Non-destructive Test of status Probe and Cancel Persistent Communication Requests Send-Receive Null Processes 75 4 Datatypes Derived Datatypes Type Constructors with Explicit Addresses Datatype Constructors Subarray Datatype Constructor Distributed Array Datatype Constructor Address and Size Functions Lower-Bound and Upper-Bound Markers Extent and Bounds of Datatypes True Extent of Datatypes Commit and Free Duplicating a Datatype Use of General Datatypes in Communication Correct Use of Addresses Decoding a Datatype Examples Ill 4.2 Pack and Unpack 120 vi

4 4.3 Canonical MPLPACK and MPLUNPACK Collective Communication Introduction and Overview Communicator Argument Specifics for Intiacommunicator Collective Operations Applying Collective Operations to Intercommunicators Specifics for Intercommunicator Collective Operations Barrier Synchronization Broadcast Example using MPLBCAST Gather Examples using MPLGATHER, MPLGATHERV Scatter Examples using MPI.SCATTER, MPI.SCATTERV Gather-to-all Examples using MPLALLGATHER, MPLALLGATHERV All-to-All Scatter/Gather Global Reduction Operations Reduce Predefined Reduction Operations Signed Characters and Reductions MINLOC and MAXLOC User-Defined Reduction Operations 168 Example of User-defined Reduce All-Reduce Reduce-Scatter Scan Inclusive Scan Exclusive Scan Example using MPLSCAN Correctness , 6 Groups, Contexts, Communicators, and Caching Introduction Features Needed to Support Libraries MPI's Support for Libraries Basic Concepts Groups Contexts Intra-Communicators Predefined Intra-Communicators Group Management Group Accessors Group Constructors Group Destructors Communicator Management Communicator Accessors 193 vii

5 6.4.2 Communicator Constructors Communicator Destructors Motivating Examples Current Practice # Current Practice # (Approximate) Current Practice # Example # Library Example # Library Example # Inter-Communication Inter-communicator Accessors Inter-communicator Operations Inter-Communication Examples 214 Example 1: Three-Group "Pipeline" 214 Example 2: Three-Group "Ring" 216 Example 3: Building Name Service for Intercommunication Caching Functionality Communicators Windows Datatypes Error Class for Invalid Keyval Attributes Example Naming Objects Formalizing the Loosely Synchronous Model Basic Statements Models of Execution 239 Static communicator allocation 239 Dynamic communicator allocation 240 The General cose Process Topologies Introduction Virtual Topologies Embedding in MP I Overview of the Functions Topology Constructors Cartesian Constructor Cartesian Convenience Function: MPI_DIMS_CREATE General (Graph) Constructor Topology Inquiry Functions Cartesian Shift Coordinates Partitioning of Cartesian structures Low-Level Topology Functions An Application Example 256 viii

6 8 MPI Environmental Management Implementation Information Version Inquiries Environmental Inquiries 260 Tag Values 260 Host Rank Rank 261 Clock Synchronization Memory Allocation Error Handling Error Handlers for Communicators Error Handlers for Windows Error Handlers for Files Freeing Errorhandlers and Retrieving Error Strings Error Codes and Classes Error Classes, Error Codes, and Error Handlers Timers and Synchronization Startup Allowing User Functions at Process Termination Determining Whether MPI Has Finished Portable MPI Process Startup The Info Object Process Creation and Management Introduction The Dynamic Process Model Starting Processes The Runtime Environment Process Manager Interface Processes in MPI Starting Processes and Establishing Communication Starting Multiple Exccutables and Establishing Communication Reserved Keys Spawn Example 304 Manager-worker Example, Using MPI.COMM.SPAWN Establishing Communication Names, Addresses, Ports, and All That Server Routines Client Routines Name Publishing Reserved Key Values Client/Server Examples Simplest Example Completely Portable 313 Ocean/Atmosphere Relies on Nome Publishing 314 Simple Client-Server Example Other Functionality Universe Size ix

7 Singleton MPUNIT MPI.APPNUM Releasing Connections Another Way to Establish MPI Communication One-Sided Communications Introduction Initialization Window Creation Window Attributes Communication Calls 32^ Put Get Examples Accumulate Functions Synchronization Calls Fence General Active Target Synchronization Lock Assertions Miscellaneous Clarifications Examples Error Handling Error Handlers Error Classes Semantics and Correctness Atomicity Progress Registers and Compiler Optimizations External Interfaces Introduction Generalized Requests Examples Associating Information with Status MPI and Threads General Clarifications Initialization I/O Introduction Definitions File Manipulation Opening File a Closing a File Deleting a File Resizing a File 379 x

8 Preallocating Space for a File Querying the Size of a File Querying File Parameters File Info File Views Reserved File Hints Data Access Data Access Routines 387 Positioning 388 Synchronism 389 Coordination 389 Data Access Conventions Data Access with Explicit Offsets Data Access with Individual File Pointers Data Access with Shared File Pointers 399 Noncollective Operations 400 Collective Operations 402 Seek Split Collective Data Access Routines File Interoperability Datatypes for File Interoperability External Data Representation: "external32" User-Defined Data Representations 415 Extent Callback 417 Datarep Conversion Functions Matching Data Representations Consistency and Semantics File Consistency Random Access vs. Sequential Files Progress Collective File Operations Type Matching Miscellaneous Clarifications MPLOffset Type Logical vs. Physical File Layout Pile Size Examples 425 Asynchronous I/O I/O Error Handling I/O Error Classes Examples Double Buffering with Split Collective I/O Subarray Filetype Constructor xi

9 14 Profiling Interface Requirements Discussion Logic of the Design Miscellaneous Control of Profiling Examples Profiler Implementation MPI Library Implementation 438 Systems with Weak Symbols 438 Systems Without Weak Symbols Complications 439 Multiple Counting 439 Linker Oddities Multiple Levels of Interception Deprecated Functions Deprecated since MPI Language Bindings C Overview Design C++ Classes for MPI Class Member Functions for MPI Semantics C++ Datatypes Communicators Exceptions Mixed-Language Operability Profiling Fortran Support Overview Problems With Fortran Bindings for MPI 462 Problems Due to Strong Typing 463 Problems Due to Data Copying and Sequence Association 463 Special Constants 465 Fortran 90 Derived Types 465 A Problem with Register Optimization Basic Fortran Support Extended Fortran Support 469 The mpi Module 469 No Type Mismatch Problems for Subroutines with Choice Arguments Additional Support for Fortran Numeric Intrinsic Types 470 Parameterized Datatypes with Specified Precision and Exponent Range471 Support for Size-specific MPI Datatypes 474 Communication With Size-specific Types 476 1G.3 Language Interoperability Introduction 478 xii

10 Assumptions Initialization Transfer of Handles Status MPI Opaque Objects 483 Datatypes 483 Callback Functions 485 Error Handlers 485 Reduce Operations 485 Addresses Attributes Extra State Constants Interlanguage Communication A Language Bindings Summary 491 A.l Defined Values and Handles 491 A.l.l Defined Constants 491 A.1.2 Types 499 A.l.3 Prototype definitions 500 A.1.4 Deprecated prototype definitions 504 A.1.5 Info Keys 504 A. 1.6 Info Values 505 A.2 C Bindings ; 506 A.2.1 Point-to-Point Communication C Bindings 506 A.2.2 Datatypes C Bindings 507 A.2.3 Collective Communication C Bindings 509 A.2.4 Groups, Contexts, Communicators, and Caching C Bindings 510 A.2.5 Process Topologies C Bindings 513 A.2.6 MPI Environmenta Management C Bindings 513 A.2.7 The Info Object C Bindings 514 A.2.8 Process Creation and Management C Bindings 515 A.2.9 One-Sided Communications C Bindings 515 A.2.10 External Interfaces C Bindings 516 A.2.11 I/O C Bindings 516 A.2.12 Language Bindings C Bindings 519 A.2.13 Profiling Interface C Bindings 520 A.2.14 Deprecated C Bindings 520 A.3 Fortran Bindings 521 A.3.1 Point-to-Point Communication Fortran Bindings 521 A.3.2 Datatypes Fortran Bindings 523 A.3.3 Collective Communication Fortran Bindings 526 A.3.4 Groups, Contexts, Communicators, and Caching Fortran Bindings 527 A.3.5 Process Topologies Fortran Bindings 532 A.3.6 MPI Environmenta Management Fortran Bindings 533 A.3.7 The Info Object Fortran Bindings 534 A.3.8 Process Creation and Management Fortran Bindings 535 A.3.9 One-Sided Communications Fortran Bindings 536 xiii

11 A.3.10 External Interfaces Fortran Bindings 537 A.3.11 I/O Fortran Bindings 538 A.3.12 Language Bindings Fortran Bindings 542 A.3.13 Profiling Interface Fortran Bindings 542 A.3.14 Deprecated Fortran Bindings 542 A. 4 C++ Bindings 545 A.4.1 Point-to-Point Communication C++ Bindings 545 A.4.2 Datatypes C++ Bindings 547 A.4.3 Collective Communication C++ Bindings 549 A.4.4 Groups, Contexts, Communicators, and Caching C++ Bindings A.4.5 Process Topologies C++ Bindings 552 A.4.6 MPI Environmenta Management C++ Bindings 553 A.4.7 The Info Object C++ Bindings 554 A.4.8 Process Creation and Management C++ Bindings 554 A.4.9 One-Sided Communications C++ Binding's 555 A.4.10 External Interfaces C++ Bindings 556 A.4.11 I/O C++ Bindings 556 A.4.12 Language Bindings C++ Bindings 560 A.4.13 Profiling Interface C++ Bindings 560 A.4.14 Deprecated C++ Bindings 560 A.4.15 C++ Bindings on all MPI Classes 560 A.4.16 Construction / Destruction 561 A.4.17 Copy / Assignment 561 A.4.18 Comparison 561 A.4.19 Inter-language Operability 561 B Change-Log 562 B. l Changes from Version 2.0 to Version Bibliography 567 Examples Index 571 MPI Constant and Predefined Handle Index 574 MPI Declarations Index 578 MPI Callback Function Prototype Index 580 MPI Function Index 581 xiv