Performance Portability of Parallel Programs on Networks of Workstations (original) (raw)
Related papers
Parallel Programming on top of Networks of Heterogeneous Workstations (NHW)
This paper considers a new approach for distributing and executing applications in networks of heterogeneous workstations (NHW). Many problems and challenges still exist in distributing and executing applications in NHW such as: a. Creating and developing a programming environment that allows a single application task to be efficiently executed, shared, and ported among the various computers in the heterogeneous suite. b. A suitable mapping technique in which one can map an arbitrary heterogeneous tasks onto an arbitrary heterogeneous suite. c. Tasks need to be assigned so that the overall execution time is minimized, load imbalance is avoided, and accuracy of the result is maintained. d. Since the users can select a group of computers to run their own applications, the load on some machines can vary and an intelligent technique is necessary for migrating tasks between machines. In this paper, most of the problems of programming in a NHW will be addressed and a new approach that overcomes some of these problems will be explained. The new approach supports parallel programming on top of the clusters of NHW where some of the clusters are running a shared memory technique and others do not. The approach uses a messagepassing paradigm to communicate between machines of different clusters.
A parallel language and its programming system for heterogeneous networks
Concurrency: Practice and Experience, 2000
The paper presents a new parallel language, mpC, designed specially for programming high-performance computations on heterogeneous networks of computers, as well as its supportive programming environment. The main idea underlying mpC is that an mpC application explicitly defines an abstract network and distributes data, computations and communications over the network. The mpC programming environment uses at run time this information as well as information about any real executing network in order to map the application to the real network in such a way that ensures efficient execution of the application on this real network. Experience of using mpC for solving both regular and irregular real-life problems on networks of heterogeneous computers is also presented.
EC : A Language for Distributed Computing
Computer Networks, Architecture and Applications, 1995
Computation intensive programs can utilise idle workstations in a cluster by exploiting the parallelism inherent in the problem being solved. A programming language for distributed computing offers advantages like early detection of type mismatch in communication and offers structured mechanisms to specify possible overlap in communication and computation, and exception handling for catching run time errors. Success of a language depends on its ease of use, expressibility and efficient implementation of its constructs. EC is a super set of C supporting process creation, a message passing mechanism, and exception handling. The pipelined communication constructs and multiple process instances help in expressing concurrency between computation and communication. Data driven activation of EC processes is used for scheduling. EC has been implemented in a Sun-3 workstation cluster. An inter-node message passing mechanism has been built on top of the socket interface using the TCP protocol, and intra-node message passing is done by passing pointers to improve efficiency. However, message_type variables hide the implementation details, improve type safety and location transparency of a program.
Abstractions for portable, scalable parallel programming
IEEE Transactions on Parallel and Distributed Systems, 1998
In parallel programming, the need to manage communication costs, load imbalance, and irregularities in the computation puts substantial demands on the programmer. Key properties of the architecture, such as the number of processors and the costs of communication, must be exploited to achieve good performance. Coding these properties directly into a program compromises the portability and exibility of the code because signicant changes are usually needed to port or enhance the program. We describe a parallel programming model that supports the concise, independent description of key aspects of a parallel program|such as data distribution, communication, and boundary conditions|without reference to machine idiosyncrasies. The independence of such components improves portability by allowing the components of a program to be tuned independently, and encourages reuse by supporting the composition of existing components. The architecture-sensitive aspects of a computation are isolated from the rest of the program, reducing the need to make extensive changes to port a program. This model is eective in exploiting both data parallelism and functional parallelism. This paper provides programming examples, compares this work to related languages, and presents performance results.
MAGE: A Distributed Programming Model
2001
This thesis explores computation mobility. We view mobility, the selection of an execution environment, as an attribute of a computation. To capture this attribute, we introduce a novel programming abstraction, which we call a mobility attribute, that specifies where computations should occur within a distributed system. Programmers dynamically bind mobility attributes to program components, nonempty sets of functions and their state. Once bound to a component, mobility attributes apply prior to each execution of that component.
HeteroMPI: Towards a Message-Passing Library for Heterogeneous Networks of Computers
Journal of Parallel and Distributed Computing, 2006
The paper presents Heterogeneous MPI (HMPI), an extension of MPI for programming high-performance computations on heterogeneous networks of computers. It allows the application programmer to describe the performance model of the implemented algorithm. This model allows for all the main features of the underlying parallel algorithm, which have an impact on its execution performance, such as the total number of parallel processes, the total volume of computations to be performed by each process, the total volume of data to be transferred between each pair of the processes, and how exactly the processes interact during the execution of the algorithm. Given the description of the performance model, HMPI creates a group of processes executing the algorithm faster than any other group of processes. The most principal extensions to MPI are presented. Parallel simulation of the interaction of electric and magnetic fields and parallel matrix multiplication are used to demonstrate the features of the library.
Message-passing environments for metacomputing
Future Generation Computer Systems, 1999
In this paper, we present the three libraries PACX-MPI, PLUS, and PVMPI that provide message-passing between different high-performance computers in metacomputing environments. Each library supports the development and execution of distributed metacomputer applications. The PACX-MPI approach offers a transparent interface for the communication between two or more MPI environments. PVAMPI allows the user spawning parallel processes under the MPI environment. The PLUS protocol bridges the gap between vendor-specific (e.g., MPL, NX, and PARIX) and vendor-independent message-passing environments (e.g., PVM and MPI). Moreover, it offers the ability to create and control processes at application runtime.
Programming and Computer Software, 2000
An mpC language designed specifically for programming high-performance computations on heterogeneous networks is described. An mpC program explicitly defines an abstract computing network and distributes data, computations, and communications over it. At runtime, the mpC programming environment uses this information and that about the actual network to distribute the processes over the actual network so as to execute the program in the most efficient way. Experience in using mpC for solving problems on local networks consisting of heterogeneous workstations is discussed. /