Trace Generation and Deterministic Execution for Concurrent Programs (original) (raw)
Related papers
Testing concurrent programs to achieve high synchronization coverage
… on Software Testing and …, 2012
The effectiveness of software testing is often assessed by measuring coverage of some aspect of the software, such as its code. There is much research aimed at increasing code coverage of sequential software. However, there has been little research on increasing coverage for concurrent software. This paper presents a new technique that aims to achieve high coverage of concurrent programs by generating thread schedules to cover uncovered coverage requirements. Our technique first estimates synchronization-pair coverage requirements, and then generates thread schedules that are likely to cover uncovered coverage requirements. This paper also presents a description of a prototype tool that we implemented in Java, and the results of a set of studies we performed using the tool on a several open-source programs. The results show that, for our subject programs, our technique achieves higher coverage faster than random testing techniques; the estimation-based heuristic contributes substantially to the effectiveness of our technique.
Data Flow Testing in Concurrent Programs with Message Passing and Shared Memory Paradigms
Procedia Computer Science, 2013
This paper proposes a novel structural test model to capture information about flows of data, control, communication and synchronization of parallel and distributed applications interacting through both message passing and shared memory paradigms. Our main focus is the data flow among threads of different processes, considering the operations of communication and synchronization related to such flow. This inter-process data-flow represents a challenge to the tester, who must consider the heterogeneity of programming paradigms and languages. The information captured by the test model is used by new testing criteria, which improve the testing activity quality. The proposed model and criteria were applied to an object-oriented and distributed application developed in Java. The results suggest that the model and the criteria are able to represent distributed applications developed with message passing and shared memory paradigms. The main contribution of this paper is to present a more flexible test model capable of improving the structural test activity related to the data flow on processes and threads simultaneously.
Science of Computer Programming, 2013
Concurrent programs exhibit nondeterministic behavior in that multiple executions thereof with the same input might produce different sequences of synchronization events and different results. This is because different executions of a concurrent program with the same input may exhibit different interleavings. Thus, one of the major issues in the testing of concurrent programs is how to explore different interleavings or exhaust all the possible interleavings of the target programs. However, for terminating concurrent programs that have cyclic state spaces due to using iterative statements such as busywaiting loops, they might have an infinite number of feasible synchronization sequences; that is, there is an infinite number of possible interleavings, which makes it impossible to explore all the possible interleavings for this type of concurrent program. To overcome this problem, we propose a testing scheme called dynamic effective testing that can perform state-cover testing for nondeterministic terminating concurrent programs with an infinite number of synchronization sequences. Dynamic effective testing does not require static analysis of the target concurrent program or the assistance of a model checker, and thus is loosely coupled to the syntax of the target concurrent program. It only needs to analyze sequences of synchronization events produced by the execution of the concurrent programs for race detection and state-traversal control. Therefore, the method is easy to port to different programming languages. In addition, only reiterated states discovered in a single SYN-sequence need to be stored. The implementation and experimental results obtained with real code demonstrate that source-code-level dynamic testing can be systematically performed on nondeterministic concurrent programs with infinite synchronization sequences.
Race directed random testing of concurrent programs
2008
Bugs in multi-threaded programs often arise due to data races. Numerous static and dynamic program analysis techniques have been proposed to detect data races. We propose a novel randomized dynamic analysis technique that utilizes potential data race information obtained from an existing analysis tool to separate real races from false races without any need for manual inspection. Specifically, we use potential data race information obtained from an existing dynamic analysis technique to control a random scheduler of threads so that real race conditions get created with very high probability and those races get resolved randomly at runtime. Our approach has several advantages over existing dynamic analysis tools. First, we can create a real race condition and resolve the race randomly to see if an error can occur due to the race. Second, we can replay a race revealing execution efficiently by simply using the same seed for random number generation-we do not need to record the execution. Third, our approach has very low overhead compared to other precise dynamic race detection techniques because we only track all synchronization operations and a single pair of memory access statements that are reported to be in a potential race by an existing analysis. We have implemented the technique in a prototype tool for Java and have experimented on a number of large multi-threaded Java programs. We report a number of previously known and unknown bugs and real races in these Java programs.
Coverage guided systematic concurrency testing
2011
Shared-memory multi-threaded programs are notoriously difficult to test, and because of the often astronomically large number of thread schedules, testing all possible interleavings is practically infeasible. In this paper we propose a coverage-guided systematic testing framework, where we use dynamically learned ordering constraints over shared object accesses to select only high-risk interleavings for test execution. An interleaving is of high-risk if it has not be covered by the ordering constraints, meaning that it has concurrency scenarios that have not been tested. Our method consists of two components. First, we utilize dynamic information collected from good test runs to learn ordering constraints over the memory-accessing and synchronization statements. These ordering constraints are treated as likely invariants since they are respected by all the tested runs. Second, during the process of systematic testing, we use the learned ordering constraints to guide the selection of interleavings for future test execution. Our experiments on public domain multithreaded C/C++ programs show that, by focusing on only the high-risk interleavings rather than enumerating all possible interleavings, our method can increase the coverage of important concurrency scenarios with a reasonable cost and detect most of the concurrency bugs in practice.
Statement-Coverage Testing for Nondeterministic Concurrent Programs
2012 Sixth International Symposium on Theoretical Aspects of Software Engineering, 2012
In this paper we propose a scheme for reachability testing to obtain statement coverage in the dynamic testing of concurrent programs. The proposed scheme derives inputs from SYN-sequences obtained in reachability testing and uses these inputs to perform reachability testing multiple times in order to achieve statement-coverage testing for a concurrent program.
A Framework for Testing Concurrent Programs
Incremental, test-driven development is sweeping the software industry, elevating testing from an ancillary activity to an integral part of the programming process. Unfortunately, in our recent experience developing production programs in Java, unit testing has only proven effective in assuring the reliability of code with a single thread of control; it is much less effective in concurrent programs. To facilitate the development of concurrent programs, we are developing: i) An extension of the JUnit framework that actively supports the developer by treating tests that could silently ignore failures in auxiliary threads as test errors; ii) A lightweight Java annotation language that can be used to specify and check the threading invariants of both existing and new code; iii) A testing framework that can record and analyze the schedules of unit tests, detect deadlocks, and run the tests using modified schedules, increasing the likelihood that concurrency problems are discovered.
Systematic Testing of Multithreaded Programs
1999
Concurrent programs are becoming common, while testing techniques that can adequatelytest such programs are not widely available. Due to the nondeterministic nature of concurrentprograms, program errors resulting from unintended timing dependencies can beextremely difficult to track down. We have designed, proved correct, and implemented atesting algorithm called ExitBlock that systematically and deterministically finds such errors.ExitBlock executes a program or a
Structural testing for message-passing concurrent programs: an extended test model
Concurrency and Computation: Practice and Experience, 2014
Developing high-quality, error-free message-passing concurrent programs is not trivial. Although a number of different primitives with associated semantics are available to assist such development, they often increase the complexity of the testing process. In this paper, we extend our previous test model for messagepassing programs and present new structural testing criteria, taking into account additional features used in this paradigm, such as collective communication, non-blocking sends, distinct semantics for non-blocking receives, and persistent operations. Our new model also recognizes that sender primitives cannot always be matched with every receive primitive. This improvement allows us to remove statically a significant number of infeasible synchronization edges that would otherwise have to be analyzed later by the tester. In this paper, the test model is presented using the Message-Passing Interface standard; however, our new model has been designed to be flexible, and it can be configured to support a range of different messagepassing environments or languages. We have carried out case studies showing the applicability of the new test model to represent message-passing programs and also to reveal errors, mainly those errors related to inter-process communication. In addition to increasing the number of features supported by the test model, we have also reduced the overall cost of testing significantly. Our case studies suggest that the number of synchronization edges can be reduced by up to 93%, mainly by eliminating infeasible edges between unmatchable communication primitives. The main contribution of the paper is to present a more flexible test model that provides improved coverage for message-passing programs and at the same time reduces the cost of testing significantly. use of structural aspects from the source code to guide the selection of test cases. They are usually based on a control flow graph (CFG) and the definition or use of program variables .
ConcJUnit: Unit Testing for Concurrent Programs
In test-driven development, tests are written for each program unit before the code is written, ensuring that the code has a comprehensive unit testing harness. Unfortunately, unit testing is much less effective for concurrent programs than for conventional sequential programs, partly because extant unit testing frameworks provide little help in addressing the challenges of testing concurrent code. In this paper, we present ConcJUnit, an extension of the popular unit testing framework JUnit that simplifies the task of writing tests for concurrent programs by handling uncaught exceptions and failed assertions in all threads, and by detecting child threads that were not forced to terminate before the main thread ends.