mpi4py (original) (raw)

Project description
Project details
Release history
Download files

Project description

This package provides Python bindings for the Message Passing Interface (MPI) standard. It is implemented on top of the MPI specification and exposes an API which grounds on the standard MPI-2 C++ bindings.

Features

This package supports:

Convenient communication of any picklable Python object
- point-to-point (send & receive)
- collective (broadcast, scatter & gather, reductions)
Fast communication of Python object exposing the Python buffer interface (NumPy arrays, builtin bytes/string/array objects)
- point-to-point (blocking/nonblocking/persistent send & receive)
- collective (broadcast, block/vector scatter & gather, reductions)
Process groups and communication domains
- creation of new intra/inter communicators
- creation/query of Cartesian & graph topologies
Parallel input/output:
- read & write
- blocking/nonblocking & collective/noncollective
- individual/shared file pointers & explicit offset
Dynamic process management
- spawn & spawn multiple
- accept/connect
- name publishing & lookup
One-sided operations
- remote memory access (put, get, accumulate)
- passive target synchronization (start/complete & post/wait)
- active target synchronization (lock & unlock)

Install

Wheel packages

The mpi4py project builds and publishes binary wheels able to run in a variety of:

operating systems: Linux, macOS, Windows;
processor architectures: AMD64, ARM64;
MPI implementations: MPICH, Open MPI,MVAPICH, Intel MPI, HPE Cray MPICH, Microsoft MPI;
Python implementations: CPython, PyPy.

These mpi4py wheels are distributed via the Python Package Index (PyPI) and can be installed with Python package managers like pip:

python -m pip install mpi4py

The mpi4py wheels can be installed in standard Python virtual environments. The MPI runtime can be provided by other wheels installed in the same virtual environment.

The mpi4py wheels can also be installed (with pip) in condaenvironments and they should work out of the box, without any special tweak to environment variables, for any of the MPI packages provided by conda-forge.

Externally-provided MPI implementations may come from a system package manager, sysadmin-maintained builds accessible via module files, or customized user builds. Such usage is supported and encouraged. However, there are a few platform-specific considerations to take into account.

Linux

The Linux (x86_64/aarch64) wheels require one of

MPICH or any other ABI-compatible derivative, like MVAPICH, Intel MPI, HPE Cray MPICH
Open MPI or any other ABI-compatible derivative, like NVIDIA HPC-X

Users may need to set the LD_LIBRARY_PATH environment variable such that the dynamic linker is able to find at runtime the MPI shared library file (libmpi.so.*).

Fedora/RHEL

On Fedora/RHEL systems, both MPICH and Open MPI are available for installation. There is no default or preferred MPI implementation. Instead, users must select their favorite MPI implementation by loading the proper MPI module.

module load mpi/mpich-$(arch)    # for MPICH
module load mpi/openmpi-$(arch)  # for Open MPI

After loading the requested MPI module, the LD_LIBRARY_PATHenvironment variable should be properly setup.

Debian/Ubuntu

On Debian/Ubuntu systems, Open MPI is the default MPI implementation and most of the MPI-based applications and libraries provided by the distribution depend on Open MPI. Nonetheless, MPICH is also available to users for installation.

In Ubuntu 22.04 and older, due to legacy reasons, the MPICH ABI is slightly broken: the MPI shared library file is namedlibmpich.so.12 instead of libmpi.so.12 as required by theMPICH ABI Compatibility Initiative.

Users without sudo access can workaround this issue creating a symbolic link anywhere in their home directory and appending toLD_LIBRARY_PATH.

mkdir -p ~/.local/lib
libdir=/usr/lib/$(arch)-linux-gnu
ln -s $libdir/libmpich.so.12 ~/.local/lib/libmpi.so.12
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.local/lib

A system-wide fix for all users requires sudo access:

libdir=/usr/lib/$(arch)-linux-gnu
sudo ln -sr $libdir/libmpi{ch,}.so.12

HPE Cray OS

On HPE Cray systems, users must load the cray-mpich-abi module. For further details, refer to man intro_mpi.

macOS

The macOS (arm64/x86_64) wheels require

MPICH or Open MPI installed (either manually or via a package manager) in the standard system prefix /usr/local
MPICH or Open MPI installed via Homebrew in the default prefix /opt/homebrew
MPICH or Open MPI installed via MacPorts in the default prefix /opt/local

Windows

The Windows (AMD64) wheels require one of

User may need to set the I_MPI_ROOT or MSMPI_BIN environment variables such that the MPI dynamic link library (DLL) (impi.dllor msmpi.dll) can be found at runtime.

Intel MPI is under active development and supports recent versions of the MPI standard. Intel MPI can be installed with pip (see theimpi-rt package on PyPI), being therefore straightforward to get it up and running within a Python environment. Intel MPI can also be installed system-wide as part of the Intel oneAPI HPC Toolkit for Windows or via standalone online/offline installers.

Conda packages

The conda-forge community provides ready-to-use binary packages from an ever growing collection of software libraries built around the multi-platform conda package manager. Four MPI implementations are available on conda-forge: Open MPI (Linux and macOS), MPICH (Linux and macOS), Intel MPI (Linux and Windows), and Microsoft MPI (Windows). You can install mpi4py and your preferred MPI implementation using theconda package manager:

to use MPICH do:

conda install -c conda-forge mpi4py mpich

to use Open MPI do:

conda install -c conda-forge mpi4py openmpi

to use Intel MPI do:

conda install -c conda-forge mpi4py impi_rt

to use Microsoft MPI do:

conda install -c conda-forge mpi4py msmpi

MPICH and many of its derivatives are ABI-compatible. You can provide the package specification mpich=X.Y.*=external_* (where X andY are the major and minor version numbers) to request the conda package manager to use system-provided MPICH (or derivative) libraries. Similarly, you can provide the package specificationopenmpi=X.Y.*=external_* to use system-provided Open MPI libraries.

The openmpi package on conda-forge has built-in CUDA support, but it is disabled by default. To enable it, follow the instruction outlined during conda install. Additionally, UCX support is also available once the ucx package is installed.

System packages

mpi4py is readily available through system package managers of most Linux distributions and the most popular community package managers for macOS.

Linux

On Fedora Linux systems (as well as RHEL and their derivatives using the EPEL software repository), you can install binary packages with the system package manager:

using dnf and the mpich package:

sudo dnf install python3-mpi4py-mpich

using dnf and the openmpi package:

sudo dnf install python3-mpi4py-openmpi

Please remember to load the correct MPI module for your chosen MPI implementation:

for the mpich package do:

module load mpi/mpich-$(arch)  
python -c "from mpi4py import MPI"

for the openmpi package do:

module load mpi/openmpi-$(arch)  
python -c "from mpi4py import MPI"

On Ubuntu Linux and Debian Linux systems, binary packages are available for installation using the system package manager:

sudo apt install python3-mpi4py

On Arch Linux systems, binary packages are available for installation using the system package manager:

sudo pacman -S python-mpi4py

macOS

macOS users can install mpi4py using the Homebrew package manager:

brew install mpi4py

Note that the Homebrew mpi4py package uses Open MPI. Alternatively, install the mpich package and next install mpi4py from sources using pip.

Alternatively, mpi4py can be installed from MacPorts:

sudo port install py-mpi4py

Building from sources

Installing mpi4py from pre-built binary wheels, conda packages, or system packages is not always desired or appropriate. For example, the mpi4py wheels published on PyPI may not be interoperable with non-mainstream, vendor-specific MPI implementations; or a system mpi4py package may be built with a alternative, non-default MPI implementation. In such scenarios, mpi4py can still be installed from its source distribution (sdist) using pip:

python -m pip install --no-binary=mpi4py mpi4py

You can also install the in-development version with:

python -m pip install git+https://github.com/mpi4py/mpi4py

or:

python -m pip install https://github.com/mpi4py/mpi4py/tarball/master

Citation

If MPI for Python been significant to a project that leads to an academic publication, please acknowledge that fact by citing the project.

M. Rogowski, S. Aseeri, D. Keyes, and L. Dalcin,mpi4py.futures: MPI-Based Asynchronous Task Execution for Python, IEEE Transactions on Parallel and Distributed Systems, 34(2):611-622, 2023.https://doi.org/10.1109/TPDS.2022.3225481
L. Dalcin and Y.-L. L. Fang,mpi4py: Status Update After 12 Years of Development, Computing in Science & Engineering, 23(4):47-54, 2021.https://doi.org/10.1109/MCSE.2021.3083216
L. Dalcin, P. Kler, R. Paz, and A. Cosimo,Parallel Distributed Computing using Python, Advances in Water Resources, 34(9):1124-1139, 2011.https://doi.org/10.1016/j.advwatres.2011.04.013
L. Dalcin, R. Paz, M. Storti, and J. D’Elia,MPI for Python: performance improvements and MPI-2 extensions, Journal of Parallel and Distributed Computing, 68(5):655-662, 2008.https://doi.org/10.1016/j.jpdc.2007.09.005
L. Dalcin, R. Paz, and M. Storti,MPI for Python, Journal of Parallel and Distributed Computing, 65(9):1108-1115, 2005.https://doi.org/10.1016/j.jpdc.2005.03.010

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Built Distributions

File details

Details for the file mpi4py-4.1.0.tar.gz.

File metadata

Download URL: mpi4py-4.1.0.tar.gz
Upload date: Jun 25, 2025
Size: 495.4 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for mpi4py-4.1.0.tar.gz | Algorithm | Hash digest | | | ----------- | ---------------------------------------------------------------- | | | SHA256 | 817492796bce771ccd809a6051cf68d48689815493b567a696ce7679260449cd | | | MD5 | 3a0adf4804cb62bd25140e387ee28e70 | | | BLAKE2b-256 | 7b0c2fc9e1f48226ddbc472d1c94580836c59fc24ae97592f8ad4fc9ffdb5373 | |

mpi4py (original) (raw)

Project description

Features

Install

Wheel packages

Linux

Fedora/RHEL

Debian/Ubuntu

HPE Cray OS

macOS

Windows

Conda packages

System packages

Linux

macOS

Building from sources

Citation

Project details

Download files

Source Distribution

Built Distributions

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes