Parallel Algorithm Models (original) (raw)

Thinking in Parallel: Some Basic Data-Parallel Algorithms and Techniques

2008

* Copyright 2007, Uzi Vishkin. These class notes reflect the theorertical part in the Parallel Algorithms course at UMD. The parallel programming part and its computer architecture context within the PRAM-On-Chip Explicit Multi-Threading (XMT) platform is provided through the XMT home page www.umiacs.umd.edu/users/vishkin/XMT and the class home page. Comments are welcome: please write to me using my last name at umd.edu

Approaches to Data Parallel Programming

In this paper efforts have been put efforts to illustrate in the best way how to start with the programming of data parallel model. The various things that need to be taken care before starting up with this approach of programming. Data Parallel Approach means split the data on which the instruction is to be applied and assign the same task to different processing elements for the processing over the individual data that has been assigned to them. Hence after all the processing elements are done with the assigned task then the whole result is then accommodated back at one place.

Task Parallelism and Data Distribution: An Overview of Explicit Parallel Programming Languages

Lecture Notes in Computer Science, 2013

Programming parallel machines as effectively as sequential ones would ideally require a language that provides high-level programming constructs to avoid the programming errors frequent when expressing parallelism. Since task parallelism is considered more error-prone than data parallelism, we survey six popular and efficient parallel language designs that tackle this difficult issue: Cilk, Chapel, X10, Habanero-Java, OpenMP and OpenCL. Using as single running example a parallel implementation of the computation of the Mandelbrot set, this paper describes how the fundamentals of task parallel programming, i.e., collective and point-to-point synchronization and mutual exclusion, are dealt with in these languages. We discuss how these languages allocate and distribute data over memory. Our study suggests that, even though there are many keywords and notions introduced by these languages, they all boil down, as far as control issues are concerned, to three key task concepts: creation, synchronization and atomicity. Regarding memory models, these languages adopt one of three approaches: shared memory, message passing and PGAS (Partitioned Global Address Space). The paper is designed to give users and language and compiler designers an upto-date comparative overview of current parallel languages. Recent programming models explore the best trade-offs between expressiveness and performance when addressing parallelism. Traditionally, there are two general ways to break an application into concurrent parts in order to take advantage of a parallel computer and execute them simultaneously on different CPUs: data and task parallelisms.

A mathematical formalization of data parallel operations

2016

We give a mathematical formalization of ‘generalized data parallel’ operations, a concept that covers such common scientific kernels as matrix-vector multiplication, multi-grid coarsening, load distribution, and many more. We show that from a compact specification such computational aspects as MPI messages or task dependencies can be automatically derived.

Parallel programming models: a survey

Parallel programming and the design of efficient parallel programs is a development area of growing importance. Parallel programming models are almost used to integrate parallel software concepts into a sequential code. These models represent an abstraction of the hardware capabilities to the programmer. In fact, a model is a bridge between the application to be parallelized and the machine organization. Up to now, a variety of programming models have been developed, each having its own approach. This paper enumerates various existing parallel programming models in the literature. The purpose is to perform a comparative evaluation of the mostly used ones, namely MapReduce, Cilk, Cilk++, OpenMP and MPI, within some extracted features.

A Survey on the Techniques and Algorithms For Data Parallelism

Grid computing basically to make a super computer virtually by connecting the different machines remotely. The major objective of Grid computing resource sharing by combining the different administrative domains to obtain the common goal. In grid computing each node contribute and could get resources to resource pool. But unfortunately grid computing can't get big place in academic circles because it's considered to be a hard to implement technology. Grid are constructed through the universally useful Grid programming libraries are called middleware. A particular Grid container be used for different application. A Grid can be dedicated for some special applications.

Concurrent approach to Data Parallel Model using Java

Parallel programming models exist as an abstraction of hardware and memory architectures. There are several parallel programming models in commonly use; they are shared memory model, thread model, message passing model, data parallel model, hybrid model, Flynn's models, embarrassingly parallel computations model, pipelined computations model. These models are not specific to a particular type of machine or memory architecture. This paper expresses the model program for concurrent approach to data parallel model through java programming.

A Data-Parallel Formulation for Divide and Conquer Algorithms

The Computer Journal, 2001

This paper presents a general data-parallel formulation for a class of problems based on the divide and conquer strategy. A combination of three techniques-mapping vectors, index-digit permutations and space-filling curves-are used to reorganize the algorithmic dataflow, providing great flexibility to efficiently exploit data locality and to reduce and optimize communications. In addition, these techniques allow the easy translation of the reorganized dataflows into HPF (High Performance Fortran) constructs. Finally, experimental results on the Cray T3E validate our method.

Chapter 2 Parallel Programming Considerations

The principal goal of this chapter is to introduce the common issues that a programmer faces when implementing a parallel application. The treatment assumes that the reader is familiar with programming a uniprocessor using a conventional language, such as Fortran. The principal challenge of parallel programming is to decompose the program into subcomponents that can be run in parallel. However, to understand some of the low-level issues of decomposition, the programmer must have a simple view of parallel machine architecture. Thus we begin our treatment with a discussion of this topic. This discussion, found in Section 2.2, focuses on two main parallel machine organizations — shared memory and distributed memory — that characterize most current machines. The section also treats clusters, which are hybrids of the two main memory designs. The standard parallel architectures support a variety of decomposition strategies, such as decomposition by task (task parallelism) and decompositio...