MPJ: A Proposed Java Message Passing API and Environment for High Performance Computing (original) (raw)

F-MPJ: scalable Java message-passing communications on parallel systems

The Journal of Supercomputing, 2012

This paper presents F-MPJ (Fast MPJ), a scalable and efficient Message-Passing in Java (MPJ) communication middleware for parallel computing. The increasing interest in Java as the programming language of the multi-core era demands scalable performance on hybrid architectures (with both shared and distributed memory spaces). However, current Java communication middleware lacks efficient communication support. F-MPJ boosts this situation by: (1) providing efficient non-blocking communication, which allows communication overlapping and thus scalable performance; (2) taking advantage of shared memory systems and highperformance networks through the use of our high-performance Java sockets implementation (named JFS, Java Fast Sockets); (3) avoiding the use of communication buffers; and (4) optimizing MPJ collective primitives. Thus, F-MPJ significantly improves the scalability of current MPJ implementations. A performance evaluation on an InfiniBand multi-core cluster has shown that F-MPJ communication primitives outperform representative MPJ libraries up to 60 times. Furthermore, the use of F-MPJ in communication-intensive MPJ codes has increased their performance up to seven times.

MPJ: MPI-like message passing for Java

2000

Description/Abstract Recently, there has been a lot of interest in using Java for parallel programming. Efforts have been hindered by lack of standard Java parallel programming APIs. To alleviate this problem, various groups started projects to develop Java message passing systems modeled on the successful Message Passing Interface (MPI). Official MPI bindings are currently defined only for C, Fortran, and C++, so early MPI-like environments for Java have been divergent.

JMPI: Implementing the message passing standard in Java

ipdps, 2002

The Message Passing Interface (MPI) standard provides a uniform Application Programmers Interface (API) that abstracts the underlying hardware from the parallel applications. Recent research efforts have extended the MPI standard to Java either through wrapper implementations or as subsets of larger parallel infrastructures. In this paper, we describe JMPI, a reference implementation of MPI developed at the Architecture and Real-Time Laboratory at the University of Massachusetts Amherst. In this implementation, we explore using Java's Remote Method Invocation (RMI), Object Serialization and Introspection technologies for message passing. We discuss the architecture of our implementation, adherence to emerging standards for MPI bindings for Java, our performance results, and future research direction.

latteMPI: Constructing a Message Passing Interface Implementation in Java

2011

The latteMPI project is a pure Java implementation of a subset of the Message Passing Interface (MPI) Standard. The MPI Standard defines a number of methods that facilitate writing programs using the message-passing model of computing, where processes share data using explicit calls to send and receive functions. Performance of parallel programs running on both latteMPI and a C implementation of the MPI Standard is investigated. The performance of latteMPI is slower than the equivalent C implementation, ...

Device level communication libraries for high-performance computing in Java

Concurrency and Computation: Practice and Experience, 2011

Since its release, the Java programming language has attracted considerable attention from the highperformance computing (HPC) community because of its portability, high programming productivity, and built-in multithreading and networking support. As a consequence, several initiatives have been taken to develop a high-performance Java message-passing library to program distributed memory architectures, such as clusters. The performance of Java message-passing applications relies heavily on the communications performance. Thus, the design and implementation of low-level communication devices that support messagepassing libraries is an important research issue in Java for HPC. MPJ Express is our Java message-passing implementation for developing high-performance parallel Java applications. Its public release currently contains three communication devices: the first one is built using the Java New Input/Output (NIO) package for the TCP/IP; the second one is specifically designed for the Myrinet Express library on Myrinet; and the third one supports thread-based shared memory communications. Although these devices have been successfully deployed in many production environments, previous performance evaluations of MPJ Express suggest that the buffering layer, tightly coupled with these devices, incurs a certain degree of copying overhead, which represents one of the main performance penalties. This paper presents a more efficient Java message-passing communications device, based on Java Input/Output sockets, that avoids this buffering overhead. Moreover, this device implements several strategies, both in the communication protocol and in the HPC hardware support, which optimizes Java message-passing communications. In order to evaluate its benefits, this paper analyzes the performance of this device comparatively with other Java and native message-passing libraries on various high-speed networks, such as Gigabit Ethernet, Scalable Coherent Interface, Myrinet, and Infini-Band, as well as on a shared memory multicore scenario. The reported communication overhead reduction encourages the upcoming incorporation of this device in MPJ Express (http://mpj-express.org). Copyright DEVICE LEVEL COMMUNICATION LIBRARIES FOR HPC IN JAVA 2383 programming productivity, platform independence, portability, and security. These significant benefits motivated the appearance of the Java Grande Forum , an initiative devoted to promote the use of Java for Grande applications, those with large requirements of computational resources, and proposed modifications to the Java language specification to make it more suitable for these codes. Furthermore, in the era of multicore processors, the use of Java threads is considered to be a feasible option to harness the performance of these processors. Another interesting argument in favor of Java is the large pool of developers, especially due to its significant presence in academia.

MPJava: High-Performance Message Passing in Java Using Java.nio

Lecture Notes in Computer Science, 2004

We explore advances in Java Virtual Machine (JVM) technology along with new high performance I/O libraries in Java 1.4, and find that Java is increasingly an attractive platform for scientific clusterbased message passing codes. We report that these new technologies allow a pure Java implementation of a cluster communication library that performs competitively with standard C-based MPI implementations.

mpiJava: A Java interface to MPI

First UK Workshop on Java for High Performance Network …, 1999

The overall aim of this paper is to introduce mpiJava|a Java interface to the widely used Message Passing Interface (MPI). In the rst part of the paper we discuss the design of the mpiJava API and issues associated with its development. In the second part of the paper we brie y describe an implementation of mpiJava on NT using the WMPI environment. We then discuss some measurements made of communications performance to compare mpiJava with C and Fortran bindings of MPI. In the nal part of the paper we summarize our ndings and brie y mention work we plan to undertake in the near future.

CCJ: object-based message passing and collective communication in Java

Concurrency and Computation: Practice and Experience, 2003

CCJ is a communication library that adds MPI-like message passing and collective operations to Java. Rather than trying to adhere to the precise MPI syntax, CCJ aims at a clean integration of communication into Java's object-oriented framework. For example, CCJ uses thread groups to support Java's multithreading model and it allows any data structure (not just arrays) to be communicated. CCJ is implemented entirely in Java, on top of RMI, so it can be used with any Java virtual machine. The paper discusses three parallel Java applications that use collective communication. It compares the performance (on top of a Myrinet cluster) of CCJ, RMI and mpiJava versions of these applications, and also compares the code complexity of the CCJ and RMI versions. The results show that the CCJ versions are significantly simpler than the RMI versions and obtain a good performance. A detailed performance comparison between CCJ and mpiJava is given using the Java Grande Forum MPJ benchmark suite.

MpiJava: An Object-Oriented Java Interface to MPI

Parallel and Distributed …, 1999

A basic prerequisite for parallel programming is a good communication API. The recent i n terest in using Java for scienti c and engineering application has led to several international e orts to produce a message passing interface to support parallel computation. In this paper we describe and then discuss the syntax, functionality and performance of one such i n terface, mpiJava, an object-oriented Java i n terface to MPI. We rst discuss the design of the mpiJava API and the issues associated with its development. We then move on to brie y outline the steps necessary to 'port' mpiJava onto a range of operating systems, including Windows NT, Linux and Solaris. In the second part of the paper we present and then discuss some performance measurements made of communications bandwidth and latency to compare mpiJava on these systems. Finally, we summarise our experiences and then brie y mention work that we plan to undertake. many aspects of Java and so ensure that its future development makes it more appropriate for scienti c programmers.

MPJ Express: Towards Thread Safe Java HPC

2006 IEEE International Conference on Cluster Computing, 2006

MPJ Express is a thread-safe Java messaging library that provides a full implementation of the mpiJava 1.2 API specification. This specification defines a MPI-like bindings for the Java language. We have implemented two communication devices as part of our library, the first, called niodev is based on the Java New I/O package and the second, called mxdev is based on the Myrinet eXpress library. MPJ Express comes with an experimental runtime, which allows portable bootstrapping of Java Virtual Machines across a cluster or network of computers. In this paper we describe the implementation of MPJ Express. Also, we present a qualitative and quantitative comparison against various other C and Java messaging systems.