mpi-report.tex (original) (raw)
MPI: A Message-Passing Interface Standard Version 3.1
Message Passing Interface Forum
This document describes the Message-Passing Interface ( MPI) standard, version 3.1. The MPI standard includes point-to-point message-passing, collective communications, group and communicator concepts, process topologies, environmental management, process creation and management, one-sided communications, extended collective operations, external interfaces, I/O, some miscellaneous topics, and a profiling interface. Language bindings for C and Fortran are defined.
Historically, the evolution of the standards is from MPI-1.0 (May 5, 1994) to MPI-1.1 (June 12, 1995) to MPI-1.2 (July 18, 1997), with several clarifications and additions and published as part of the MPI-2 document, to MPI-2.0 (July 18, 1997), with new functionality, to MPI-1.3 (May 30, 2008), combining for historical reasons the documents 1.1 and 1.2 and some errata documents to one combined document, and to MPI-2.1 (June 23, 2008), combining the previous documents. Version MPI-2.2 (September 4, 2009) added additional clarifications and seven new routines. Version MPI-3.0 (September 21, 2012) is an extension of MPI-2.2. This version, MPI-3.1, adds clarifications and minor extensions to MPI-3.0
Comments. Please send comments on MPI to the MPI Forum as follows:
1. Subscribe to http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-comments
2. Send your comment to: mpi-comments@mpi-forum.org, together with the URL of the version of the MPI standard and the page and line numbers on which you are commenting. Only use the official versions.
Your comment will be forwarded to MPI Forum committee members for consideration. Messages sent from an unsubscribed e-mail address will not be considered.
(c) 1993, 1994, 1995, 1996, 1997, 2008, 2009, 2012, 2015 University of Tennessee, Knoxville, Tennessee. Permission to copy without fee all or part of this material is granted, provided the University of Tennessee copyright notice and the title of this document appear, and notice is given that copying is by permission of the University of Tennessee.
Version 3.1: June 4, 2015. This document contains mostly corrections and clarifications to the MPI-3.0 document. The largest change is a correction to the Fortran bindings introduced in MPI-3.0. Additionally, new functions added include routines to manipulate MPI_Aint values in a portable manner, nonblocking collective I/O routines, and routines to get the index value by name for MPI_T performance and control variables.
Version 3.0: September 21, 2012. Coincident with the development of MPI-2.2, the MPI Forum began discussions of a major extension to MPI. This document contains the MPI-3 Standard. This draft version of the MPI-3 standard contains significant extensions to MPI functionality, including nonblocking collectives, new one-sided communication operations, and Fortran 2008 bindings. Unlike MPI-2.2, this standard is considered a major update to the MPI standard. As with previous versions, new features have been adopted only when there were compelling needs for the users. Some features, however, may have more than a minor impact on existing MPI implementations.
Version 2.2: September 4, 2009. This document contains mostly corrections and clarifications to the MPI-2.1 document. A few extensions have been added; however all correct MPI-2.1 programs are correct MPI-2.2 programs. New features were adopted only when there were compelling needs for users, open source implementations, and minor impact on existing MPIimplementations.
Version 2.1: June 23, 2008. This document combines the previous documents MPI-1.3 (May 30, 2008) and MPI-2.0 (July 18, 1997). Certain parts of MPI-2.0, such as some sections of Chapter 4, Miscellany, and Chapter 7, Extended Collective Operations, have been merged into the Chapters of MPI-1.3. Additional errata and clarifications collected by the MPI Forum are also included in this document.
Version 1.3: May 30, 2008. This document combines the previous documents MPI-1.1 (June 12, 1995) and the MPI-1.2 Chapter in MPI-2(July 18, 1997). Additional errata collected by the MPI Forum referring to MPI-1.1 and MPI-1.2 are also included in this document.
Version 2.0: July 18, 1997. Beginning after the release of MPI-1.1, the MPI Forum began meeting to consider corrections and extensions. MPI-2 has been focused on process creation and management, one-sided communications, extended collective communications, external interfaces and parallel I/O. A miscellany chapter discusses items that do not fit elsewhere, in particular language interoperability.
Version 1.2: July 18, 1997. The MPI-2 Forum introduced MPI-1.2 as Chapter 3 in the standard `` MPI-2: Extensions to the Message-Passing Interface'', July 18, 1997. This section contains clarifications and minor corrections to Version 1.1 of the MPI Standard. The only new function in MPI-1.2 is one for identifying to which version of the MPI Standard the implementation conforms. There are small differences between MPI-1 and MPI-1.1. There are very few differences between MPI-1.1 and MPI-1.2, but large differences between MPI-1.2and MPI-2.
Version 1.1: June, 1995. Beginning in March, 1995, the Message-Passing Interface Forum reconvened to correct errors and make clarifications in the MPI document of May 5, 1994, referred to below as Version 1.0. These discussions resulted in Version 1.1. The changes from Version 1.0 are minor. A version of this document with all changes marked is available.
Version 1.0: May, 1994. The Message-Passing Interface Forum (MPIF), with participation from over 40 organizations, has been meeting since January 1993 to discuss and define a set of library interface standards for message passing. MPIF is not sanctioned or supported by any official standards organization.
The goal of the Message-Passing Interface, simply stated, is to develop a widely used standard for writing message-passing programs. As such the interface should establish a practical, portable, efficient, and flexible standard for message-passing.
This is the final report, Version 1.0, of the Message-Passing Interface Forum. This document contains all the technical features proposed for the interface. This copy of the draft was processed by LaTeX on May 5, 1994.
Acknowledgments
This document is the product of a number of distinct efforts in three distinct phases: one for each of MPI-1, MPI-2, and MPI-3. This section describes these in historical order, starting with MPI-1. Some efforts, particularly parts of MPI-2, had distinct groups of individuals associated with them, and these efforts are detailed separately.
This document represents the work of many people who have served on the MPI Forum. The meetings have been attended by dozens of people from many parts of the world. It is the hard and dedicated work of this group that has led to the MPI standard.
The technical development was carried out by subgroups, whose work was reviewed by the full committee. During the period of development of the Message-Passing Interface ( MPI), many people helped with this effort.
Those who served as primary coordinators in MPI-1.0 and MPI-1.1 are:
- Jack Dongarra, David Walker, Conveners and Meeting Chairs
- Ewing Lusk, Bob Knighten, Minutes
- Marc Snir, William Gropp, Ewing Lusk, Point-to-Point Communication
- Al Geist, Marc Snir, Steve Otto, Collective Communication
- Steve Otto, Editor
- Rolf Hempel, Process Topologies
- Ewing Lusk, Language Binding
- William Gropp, Environmental Management
- James Cownie, Profiling
- Tony Skjellum, Lyndon Clarke, Marc Snir, Richard Littlefield, Mark Sears, Groups, Contexts, and Communicators
- Steven Huss-Lederman, Initial Implementation Subset
The following list includes some of the active participants in the MPI-1.0 and MPI-1.1 process not mentioned above.
Ed Anderson | Robert Babb | Joe Baron | Eric Barszcz |
---|---|---|---|
Scott Berryman | Rob Bjornson | Nathan Doss | Anne Elster |
Jim Feeney | Vince Fernando | Sam Fineberg | Jon Flower |
Daniel Frye | Ian Glendinning | Adam Greenberg | Robert Harrison |
Leslie Hart | Tom Haupt | Don Heller | Tom Henderson |
Alex Ho | C.T. Howard Ho | Gary Howell | John Kapenga |
James Kohl | Susan Krauss | Bob Leary | Arthur Maccabe |
Peter Madams | Alan Mainwaring | Oliver McBryan | Phil McKinley |
Charles Mosher | Dan Nessett | Peter Pacheco | Howard Palmer |
Paul Pierce | Sanjay Ranka | Peter Rigsbee | Arch Robison |
Erich Schikuta | Ambuj Singh | Alan Sussman | Robert Tomlinson |
Robert G. Voigt | Dennis Weeks | Stephen Wheat | Steve Zenith |
The University of Tennessee and Oak Ridge National Laboratory made the draft available by anonymous FTP mail servers and were instrumental in distributing the document.
The work on the MPI-1 standard was supported in part by ARPA and NSF under grant ASC-9310330, the National Science Foundation Science and Technology Center Cooperative Agreement No. CCR-8809615, and by the Commission of the European Community through Esprit project P6643 (PPPE).
Contents
- MPI-1.2 and MPI-2.0:
- MPI-1.3 and MPI-2.1:
- MPI-2.2:
- MPI-3.0:
- MPI-3.1:
- Introduction to MPI
- Overview and Goals
- Background of MPI-1.0
- Background of MPI-1.1, MPI-1.2, and MPI-2.0
- Background of MPI-1.3 and MPI-2.1
- Background of MPI-2.2
- Background of MPI-3.0
- Background of MPI-3.1
- Who Should Use This Standard?
- What Platforms Are Targets For Implementation?
- What Is Included In The Standard?
- What Is Not Included In The Standard?
- Organization of this Document
- MPI Terms and Conventions
- Document Notation
- Naming Conventions
- Procedure Specification
- Semantic Terms
- Data Types
* Opaque Objects
* Array Arguments
* State
* Named Constants
* Choice
* Absolute Addresses and Relative Address Displacements
* File Offsets
* Counts - Language Binding
* Deprecated and Removed Names and Functions
* Fortran Binding Issues
* C Binding Issues
* Functions and Macros - Processes
- Error Handling
- Implementation Issues
* Independence of Basic Runtime Routines
* Interaction with Signals - Examples
- Point-to-Point Communication
- Introduction
- Blocking Send and Receive Operations
* Blocking Send
* Message Data
* Message Envelope
* Blocking Receive
* Return Status
* Passing MPI_STATUS_IGNORE for Status - Data Type Matching and Data Conversion
* Type Matching Rules
* Type MPI_CHARACTER
* Data Conversion - Communication Modes
- Semantics of Point-to-Point Communication
- Buffer Allocation and Usage
* Model Implementation of Buffered Mode - Nonblocking Communication
* Communication Request Objects
* Communication Initiation
* Communication Completion
* Semantics of Nonblocking Communications
* Multiple Completions
* Non-destructive Test of status - Probe and Cancel
* Probe
* Matching Probe
* Matched Receives
* Cancel - Persistent Communication Requests
- Send-Receive
- Null Processes
- Datatypes
- Derived Datatypes
* Type Constructors with Explicit Addresses
* Datatype Constructors
* Subarray Datatype Constructor
* Distributed Array Datatype Constructor
* Address and Size Functions
* Lower-Bound and Upper-Bound Markers
* Extent and Bounds of Datatypes
* True Extent of Datatypes
* Commit and Free
* Duplicating a Datatype
* Use of General Datatypes in Communication
* Correct Use of Addresses
* Decoding a Datatype
* Examples - Pack and Unpack
- Canonical MPI_PACK and MPI_UNPACK
- Derived Datatypes
- Collective Communication
- Introduction and Overview
- Communicator Argument
* Specifics for Intracommunicator Collective Operations
* Applying Collective Operations to Intercommunicators
* Specifics for Intercommunicator Collective Operations - Barrier Synchronization
- Broadcast
* Example using MPI_BCAST - Gather
* Examples using MPI_GATHER, MPI_GATHERV - Scatter
* Examples using MPI_SCATTER, MPI_SCATTERV - Gather-to-all
* Example using MPI_ALLGATHER - All-to-All Scatter/Gather
- Global Reduction Operations
* Reduce
* Predefined Reduction Operations
* Signed Characters and Reductions
* MINLOC and MAXLOC
* User-Defined Reduction Operations
* Example of User-defined Reduce
* All-Reduce
* Process-Local Reduction - Reduce-Scatter
* MPI_REDUCE_SCATTER_BLOCK
* MPI_REDUCE_SCATTER - Scan
* Inclusive Scan
* Exclusive Scan
* Example using MPI_SCAN - Nonblocking Collective Operations
* Nonblocking Barrier Synchronization
* Nonblocking Broadcast
* Example using MPI_IBCAST
* Nonblocking Gather
* Nonblocking Scatter
* Nonblocking Gather-to-all
* Nonblocking All-to-All Scatter/Gather
* Nonblocking Reduce
* Nonblocking All-Reduce
* Nonblocking Reduce-Scatter with Equal Blocks
* Nonblocking Reduce-Scatter
* Nonblocking Inclusive Scan
* Nonblocking Exclusive Scan - Correctness
- Groups, Contexts, Communicators, and Caching
- Introduction
* Features Needed to Support Libraries
* MPI's Support for Libraries - Basic Concepts
* Groups
* Contexts
* Intra-Communicators
* Predefined Intra-Communicators - Group Management
* Group Accessors
* Group Constructors
* Group Destructors - Communicator Management
* Communicator Accessors
* Communicator Constructors
* Communicator Destructors
* Communicator Info - Motivating Examples
* Current Practice #1
* Current Practice #2
* (Approximate) Current Practice #3
* Example #4
* Library Example #1
* Library Example #2 - Inter-Communication
* Inter-communicator Accessors
* Inter-communicator Operations
* Inter-Communication Examples
* Example 1: Three-Group ``Pipeline''
* Example 2: Three-Group ``Ring'' - Caching
* Functionality
* Communicators
* Windows
* Datatypes
* Error Class for Invalid Keyval
* Attributes Example - Naming Objects
- Formalizing the Loosely Synchronous Model
* Basic Statements
* Models of Execution
* Static Communicator Allocation
* Dynamic Communicator Allocation
* The General Case
- Introduction
- Process Topologies
- Introduction
- Virtual Topologies
- Embedding in MPI
- Overview of the Functions
- Topology Constructors
* Cartesian Constructor
* Cartesian Convenience Function: MPI_DIMS_CREATE
* Graph Constructor
* Distributed Graph Constructor
* Topology Inquiry Functions
* Cartesian Shift Coordinates
* Partitioning of Cartesian Structures
* Low-Level Topology Functions -
* Neighborhood Gather
* Neighbor Alltoall -
* Nonblocking Neighborhood Gather
* Nonblocking Neighborhood Alltoall - An Application Example
- MPI Environmental Management
- Implementation Information
* Version Inquiries
* Environmental Inquiries
* Tag Values
* Host Rank
* IO Rank
* Clock Synchronization
* Inquire Processor Name - Memory Allocation
- Error Handling
* Error Handlers for Communicators
* Error Handlers for Windows
* Error Handlers for Files
* Freeing Errorhandlers and Retrieving Error Strings - Error Codes and Classes
- Error Classes, Error Codes, and Error Handlers
- Timers and Synchronization
- Startup
* Allowing User Functions at Process Termination
* Determining Whether MPI Has Finished - Portable MPI Process Startup
- Implementation Information
- The Info Object
- Process Creation and Management
- Introduction
- The Dynamic Process Model
* Starting Processes
* The Runtime Environment - Process Manager Interface
* Processes in MPI
* Starting Processes and Establishing Communication
* Starting Multiple Executables and Establishing Communication
* Reserved Keys
* Spawn Example
* Manager-worker Example Using MPI_COMM_SPAWN - Establishing Communication
* Names, Addresses, Ports, and All That
* Server Routines
* Client Routines
* Name Publishing
* Reserved Key Values
* Client/Server Examples
* Simplest Example --- Completely Portable.
* Ocean/Atmosphere --- Relies on Name Publishing
* Simple Client-Server Example - Other Functionality
* Universe Size
* Singleton MPI_INIT
* MPI_APPNUM
* Releasing Connections
* Another Way to Establish MPI Communication
- One-Sided Communications
- Introduction
- Initialization
* Window Creation
* Window That Allocates Memory
* Window That Allocates Shared Memory
* Window of Dynamically Attached Memory
* Window Destruction
* Window Attributes
* Window Info - Communication Calls
* Put
* Get
* Examples for Communication Calls
* Accumulate Functions
* Accumulate Function
* Get Accumulate Function
* Fetch and Op Function
* Compare and Swap Function
* Request-based RMA Communication Operations - Memory Model
- Synchronization Calls
* Fence
* General Active Target Synchronization
* Lock
* Flush and Sync
* Assertions
* Miscellaneous Clarifications - Error Handling
* Error Handlers
* Error Classes - Semantics and Correctness
* Atomicity
* Ordering
* Progress
* Registers and Compiler Optimizations - Examples
- External Interfaces
- I/O
- Introduction
* Definitions - File Manipulation
* Opening a File
* Closing a File
* Deleting a File
* Resizing a File
* Preallocating Space for a File
* Querying the Size of a File
* Querying File Parameters
* File Info
* Reserved File Hints - File Views
- Data Access
* Data Access Routines
* Positioning
* Synchronism
* Coordination
* Data Access Conventions
* Data Access with Explicit Offsets
* Data Access with Individual File Pointers
* Data Access with Shared File Pointers
* Noncollective Operations
* Collective Operations
* Seek
* Split Collective Data Access Routines - File Interoperability
* Datatypes for File Interoperability
* External Data Representation: ``external32''
* User-Defined Data Representations
* Extent Callback
* Datarep Conversion Functions
* Matching Data Representations - Consistency and Semantics
* File Consistency
* Random Access vs. Sequential Files
* Progress
* Collective File Operations
* Nonblocking Collective File Operations
* Type Matching
* Miscellaneous Clarifications
* MPI_Offset Type
* Logical vs. Physical File Layout
* File Size
* Examples
* Asynchronous I/O - I/O Error Handling
- I/O Error Classes
- Examples
* Double Buffering with Split Collective I/O
* Subarray Filetype Constructor
- Introduction
- Tool Support
- Introduction
- Profiling Interface
* Requirements
* Discussion
* Logic of the Design
* Miscellaneous Control of Profiling
* Profiler Implementation Example
* MPI Library Implementation Example
* Systems with Weak Symbols
* Systems Without Weak Symbols
* Complications
* Multiple Counting
* Linker Oddities
* Fortran Support Methods
* Multiple Levels of Interception - The MPI Tool Information Interface
* Verbosity Levels
* Binding MPI Tool Information Interface Variables to MPI Objects
* Convention for Returning Strings
* Initialization and Finalization
* Datatype System
* Control Variables
* Control Variable Query Functions
* Example: Printing All Control Variables
* Handle Allocation and Deallocation
* Control Variable Access Functions
* Example: Reading the Value of a Control Variable
* Performance Variables
* Performance Variable Classes
* Performance Variable Query Functions
* Performance Experiment Sessions
* Handle Allocation and Deallocation
* Starting and Stopping of Performance Variables
* Performance Variable Access Functions
* Example: Tool to Detect Receives with Long Unexpected Message Queues
* Variable Categorization
* Return Codes for the MPI Tool Information Interface
* Profiling Interface
- Deprecated Functions
- Removed Interfaces
- Language Bindings
- Fortran Support
* Overview
* Fortran Support Through the mpi_f08 Module
* Fortran Support Through the mpi Module
* Fortran Support Through the mpif.h Include File
* Interface Specifications, Procedure Names, and the Profiling Interface
* MPI for Different Fortran Standard Versions
* Requirements on Fortran Compilers
* Additional Support for Fortran Register-Memory-Synchronization
* Additional Support for Fortran Numeric Intrinsic Types
* Parameterized Datatypes with Specified Precision and Exponent Range
* Support for Size-specific MPI Datatypes
* Communication With Size-specific Types
* Problems With Fortran Bindings for MPI
* Problems Due to Strong Typing
* Problems Due to Data Copying and Sequence Association with Subscript Triplets
* Problems Due to Data Copying and Sequence Association with Vector Subscripts
* Special Constants
* Fortran Derived Types
* Optimization Problems, an Overview
* Problems with Code Movement and Register Optimization
* Nonblocking Operations
* Persistent Operations
* One-sided Communication
* MPI_BOTTOM and Combining Independent Variables in Datatypes
* Solutions
* The Fortran ASYNCHRONOUS Attribute
* Calling MPI_F_SYNC_REG
* A User Defined Routine Instead of MPI_F_SYNC_REG
* Module Variables and COMMON Blocks
* The (Poorly Performing) Fortran VOLATILE Attribute
* The Fortran TARGET Attribute
* Temporary Data Movement and Temporary Memory Modification
* Permanent Data Movement
* Comparison with C - Language Interoperability
* Introduction
* Assumptions
* Initialization
* Transfer of Handles
* Status
* MPI Opaque Objects
* Datatypes
* Callback Functions
* Error Handlers
* Reduce Operations
* Attributes
* Extra-State
* Constants
* Interlanguage Communication
- Fortran Support
- Language Bindings Summary
- Defined Values and Handles
* Defined Constants
* Types
* Prototype Definitions
* C Bindings
* Fortran 2008 Bindings with the mpi_f08 Module
* Fortran Bindings with mpif.h or the mpi Module
* Deprecated Prototype Definitions
* Info Keys
* Info Values - C Bindings
* Point-to-Point Communication C Bindings
* Datatypes C Bindings
* Collective Communication C Bindings
* Groups, Contexts, Communicators, and Caching C Bindings
* Process Topologies C Bindings
* MPI Environmental Management C Bindings
* The Info Object C Bindings
* Process Creation and Management C Bindings
* One-Sided Communications C Bindings
* External Interfaces C Bindings
* I/O C Bindings
* Language Bindings C Bindings
* Tools / Profiling Interface C Bindings
* Tools / MPI Tool Information Interface C Bindings
* Deprecated C Bindings - Fortran 2008 Bindings with the mpi_f08 Module
* Point-to-Point Communication Fortran 2008 Bindings
* Datatypes Fortran 2008 Bindings
* Collective Communication Fortran 2008 Bindings
* Groups, Contexts, Communicators, and Caching Fortran 2008 Bindings
* Process Topologies Fortran 2008 Bindings
* MPI Environmental Management Fortran 2008 Bindings
* The Info Object Fortran 2008 Bindings
* Process Creation and Management Fortran 2008 Bindings
* One-Sided Communications Fortran 2008 Bindings
* External Interfaces Fortran 2008 Bindings
* I/O Fortran 2008 Bindings
* Language Bindings Fortran 2008 Bindings
* Tools / Profiling Interface Fortran 2008 Bindings - Fortran Bindings with mpif.h or the mpi Module
* Point-to-Point Communication Fortran Bindings
* Datatypes Fortran Bindings
* Collective Communication Fortran Bindings
* Groups, Contexts, Communicators, and Caching Fortran Bindings
* Process Topologies Fortran Bindings
* MPI Environmental Management Fortran Bindings
* The Info Object Fortran Bindings
* Process Creation and Management Fortran Bindings
* One-Sided Communications Fortran Bindings
* External Interfaces Fortran Bindings
* I/O Fortran Bindings
* Language Bindings Fortran Bindings
* Tools / Profiling Interface Fortran Bindings
* Deprecated Fortran Bindings
- Defined Values and Handles
- Change-Log
- Bibliography
- Index
- Index
- Index
- Index
- Index
- Index
Return to MPI-3.1 Standard Index
Return to MPI Forum Home Page
(Unofficial) MPI-3.1 of June 4, 2015
HTML Generated on June 4, 2015