Feedback-Directed Random Test Generation (original) (raw)
Related papers
Combined static and dynamic automated test generation
2011
ABSTRACT In an object-oriented program, a unit test often consists of a sequence of method calls that create and mutate objects, then use them as arguments to a method under test. It is challenging to automatically generate sequences that are legal and behaviorally-diverse, that is, reaching as many different program states as possible. This paper proposes a combined static and dynamic automated test generation approach to address these problems, for code without a formal specification.
Evotec: Evolving the Best Testing Strategy for Contract-Equipped Programs
… Conference (APSEC), 2011 …, 2011
Automated random testing is efficient at detecting faults but it is certainly not an optimal testing strategy for every given program. For example, an automated random testing tool ignores that some routines have stronger preconditions, they use certain literal values, or they are more error-prone. Taking into account such characteristics may increase testing effectiveness. In this article, we present Evotec, an enhancement of random testing which relies on genetic algorithms to evolve a best testing strategy for contract-equipped ...
GenRed: A Tool for Generating and Reducing Object-Oriented Test Cases
2010 IEEE 34th Annual Computer Software and Applications Conference, 2010
An important goal of automatic testing techniques, including random testing is to achieve high code coverage with a minimum set of test cases. To meet this goal, random testing researchers have proposed many techniques to generate test inputs and method call sequences that yield higher code coverage. However, most proposed random testing techniques are only suitable for toy systems, and they achieve low code coverage rates while generating too many unnecessary test cases on large-scale software systems. We propose GENRED, a tool that utilizes three approaches: input on demand creation and coverage-based method selection techniques that enhance Randoop, a state-of-the-art feedback-directed random testing technique, and finally, a sequence-based reduction technique that removes redundant test cases without executing them. We evaluate GENRED as a tool to test four open-source systems. The results show that these techniques achieve branch coverage improvement by 13.7% and prune 51.8% of the test cases without sacrificing code coverage.
Testful: automatic unit-test generation for Java classes
2010
Abstract This paper presents TestFul, an Eclipse plugin for the generation of tests for Java classes. It is based on the idea of search-based testing, working both at class and method level. The former puts objects in useful states, used by the latter to exercise the uncovered parts of the class.
ARTOO: adaptive random testing for object-oriented software
2008
Intuition is often not a good guide to know which testing strategies will work best. There is no substitute for experimental analysis based on objective criteria: how many faults a strategy finds, and how fast. "Random" testing is an example of an idea that intuitively seems simplistic or even dumb, but when assessed through such criteria can yield better results than seemingly smarter strategies. The efficiency of random testing is improved if the generated inputs are evenly spread across the input domain. This is the idea of Adaptive Random Testing (ART).
Using Contracts and Boolean Queries to Improve the Quality of Automatic Test Generation
Lecture Notes in Computer Science
Since test cases cannot be exhaustive, any effective test case generation strategy must identify the execution states most likely to uncover bugs. The key issue is to define criteria for selecting such interesting states. If the units being tested are classes in object-oriented programming, it seems attractive to rely on the boolean queries present in each class, which indeed define criteria on the states of the corresponding objects, and-in contract-equipped O-O software-figure prominently in preconditions, postconditions and invariants. As these queries are part of the class specification and hence relevant to its clients, one may conjecture that the resulting partition of the state space is also relevant for tests. We explore this conjecture by examining whether relying on the boolean queries of a class to extract abstract states improves the results of black-box testing. The approach uses proof techniques to generate objects that satisfy the class invariants, then performs testing by relying on postconditions as test oracles.
Stateful testing: Finding more errors in code and contracts
2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011), 2011
Automated random testing has shown to be an effective approach to finding faults but still faces a major unsolved issue: how to generate test inputs diverse enough to find many faults and find them quickly. Stateful testing, the automated testing technique introduced in this article, generates new test cases that improve an existing test suite. The generated test cases are designed to violate the dynamically inferred contracts (invariants) characterizing the existing test suite. As a consequence, they are in a good position to detect new errors, and also to improve the accuracy of the inferred contracts by discovering those that are unsound. Experiments on 13 data structure classes totalling over 28,000 lines of code demonstrate the effectiveness of stateful testing in improving over the results of long sessions of random testing: stateful testing found 68.4% new faults and improved the accuracy of automatically inferred contracts to over 99%, with just a 7% time overhead.
2012
Abstract—This paper presents an automatic technique for generating maintainable regression unit tests for programs. We found previous test generation techniques inadequate for two main reasons. First. they were designed for and evaluated upon libraries rather than applications. Second, they were designed to find bugs rather than to create maintainable regression test suites: the test suites that they generated were brittle and hard to understand. This paper presents a suite of techniques that address these problems by enhancing an existing unit test generation system. In experiments using an industrial system, the generated tests achieved good coverage and mutation kill score, were readable by the product’s developers, and required few edits as the system under test evolved. While our evaluation is in the context of one test generator, we are aware of many research systems that suffer similar limitations, so our approach and observations are more generally relevant. I.