11 Evaluating Coverage Based Testing (original) (raw)
Related papers
Torx: Automated model-based testing
2003
Systematic testing is very important for assessing and improving the quality of software systems. Yet, testing turns out to be expensive, laborious, time-consuming and error-prone. The Dutch research and development project Côte de Resyste worked on methods, techniques and tools for automating specification based testing using formal methods. The main achievement of the project is a test tool, baptized TorX, which integrates automatic test generation, test execution, and test analysis in an on-the-fly manner. On the one hand, TorX is based on well-defined theory, viz. the ioco-test theory, which has its roots in the theory of testing-and refusal-equivalences for transition systems. On the other hand, the applicability of TorX has been demonstrated by testing several academic and industrial case studies. This paper summarizes the main results of the project Côte de Resyste.
QSIC09-Model-DrivenTestingTechniques.pdf
The model-driven approach to software development has not only changed the way software systems are built and maintained but also the way they are tested. For such systems, a model-based testing approach is much recommended since it is aligned with the new model-driven development paradigm that favors models over code with the objective being to reduce time to market while improving product quality. There has been a noticeable increase in the number of model-driven testing techniques in recent years. Although these techniques have a common objective they tend to vary significantly in their design. In this paper, we discuss the model-driven testing techniques presented in 15 different studies. We compare these techniques according to specific criteria including the modeling language used to represent the system design artifacts, the ability to automatically generate test cases, the testing target, and tool support.
2011
This paper provides a comprehensive introduction to a framework for formal testing using labelled transition systems, based on an extension and reformulation of the ioco theory introduced by Tretmans. We introduce the underlying models needed to specify the requirements, and formalise the notion of test cases. We discuss conformance, and in particular the conformance relation ioco. For this relation we prove several interesting properties, and we provide algorithms to derive test cases (either in batches, or on the fly).
Software testing is as old as the hills in the history of digital computers. The testing of software is an important means of assessing the software to determine its quality. Since testing typically consumes 40~50% of development efforts, and consumes more effort for systems that require higher levels of reliability, it is a significant part of the software engineering. With the development of Fourth generation languages (4GL), which speeds up the implementation process, the proportion of time devoted to testing increased. As the amount of maintenance and upgrade of existing systems grow, significant amount of testing will also be needed to verify systems after changes are made [12]. Despite advances in formal methods and verification techniques, a system still needs to be tested before it is used. Testing remains the truly effective means to assure the quality of a software system of non-trivial complexity [13], as well as one of the most intricate and least understood areas in software engineering [19]. Testing, an important research area within computer science is likely to become even more important in the future.
Specification-based testing of reactive software: A case study in technology transfer
Journal of Systems and Software, 1998
Testing commercial software is expensive and time consuming. Automated testing methods promise to save a great deal of time and money throughout the software industry. One approach that is well-suited for the reactive systems found in telephone switching systems is speci cation-based testing. We have built a set of tools to automatically test software applications for violations of safety properties expressed in temporal logic. Our testing system automatically constructs nite state machine oracles corresponding to safety properties, builds test harnesses, and integrates them with the application. The test harness then generates inputs automatically to test the application. We describe a study examining the feasibility of this approach for testing industrial applications. To conduct this study we formally modeled an Automatic Protection Switching system (APS), which is an application common to many telephony systems. We then asked a number of computer science graduate students to develop several versions of the APS and use our tools to test them. We found that the tools are very e ective, save signi cant amounts of human e ort (at the expense of machine resources), and are easy to use. We also dis
2006
Software testing is regarded by practitioners as the central activity used for ensuring that a system behaves as expected. It consists of the following activities: test generation, execution, evaluation, coverage analysis and regression testing. Testing is often a neglected activity that has the potential and the need for benefiting from automation. In fact, testing solutions currently available automate some testing activities. However, the majority of the automated solutions are code-based. Code-based testing uses the implementation to derive test cases. It can be used to test the system. It cannot, however, be used to test that the system built is what was intended. Testing activities that compare the system's implementation to their expected behavior are more informative because they show the system satisfies its original requirements. This kind of activity is characteristic of specification-based testing.