Software integration testing based on communication coverage criteria and partial model generation (original) (raw)
Related papers
Software Quality Journal, 2008
This paper considers the problem of integration testing the components of a timed distributed software system. We assume that communication between the components is specified using timed interface automata and use computational tree logic (CTL) to define communication-based coverage criteria that refer to send-and receive-statements and communication paths. The proposed method enables testers to focus during component integration on such parts of the specification, e.g. behaviour specifications or Markovian usage models, that are involved in the communication between components to be integrated. A more specific application area of this approach is the integration of test-models, e.g. a transmission gear can be tested based on separated models for the driver behaviour, the engine condition, and the mechanical and hydraulical transmission states. Given such a state-based specification of a distributed system and a concrete coverage goal, a model checker is used in order to determine the coverage or generate test sequences that achieve the goal. Given the generated test sequences we derive a partial test-model of the components from which the test sequences were derived. The partial model can be used to drive further testing and can also be used as the basis for producing additional partial models in incremental integration testing. While the process of deriving the test sequences
Model-Based Testing of Distributed Systems
2008
This paper provides an overview of Model-Based Testing (MBT) and its activities. A classification of MBT based on different criteria is also presented. Furthermore, several difficulties of MBT are highlighted in this paper. A survey that provides a detailed description of how MBT is effective in testing different quality attributes of distributed systems such as security, performance, reliability, and correctness is given. A comparison between different MBT tools based on the classification is also given at the end of this paper.
Testing Distributed Systems Through Symbolic Model Checking
Lecture Notes in Computer Science, 2007
The observation of a distributed system's finite execution can be abstracted as a partial ordered set of events generally called finite trace. In practice, this trace can be obtained through a standard code instrumentation, which takes advantage of existing communications between processes to partially order events of different processes. We show that testing that such a distributed execution satisfies some global property amounts therefore to model check the corresponding trace. This work can be time consuming; we therefore provide an efficient symbolic Ctl modelchecking algorithm for traces. This method is based on a symbolic data structure, called Interval Sharing Trees, allowing to efficiently represent and manipulate sets of k-uples of naturals. Efficient symbolic operations are defined on this data structure in order to deal with all Ctl modalities. We show that in practice this data structure is well adapted for Ctl model checking of traces.
Two Complementary Tools for the Formal Testing of Distributed Systems with Time Constraints
2008 12th IEEE/ACM International Symposium on Distributed Simulation and Real-Time Applications, 2008
The complexity and the variety of the deployed timedependent systems, as well as the high degree of reliability required for their global functioning, justify the care provided to the design of the best possible tests. Moreover, it is significant to automate these steps with an aim of reducing the time and the development cost and especially of increasing the reliability of the offered products. In this paper, we present two different tools to test systems with time constraints. The first one allows to automatically generate test cases based on model-based active testing techniques. Whereas the second tool is based on passive testing approach to check that the collected system traces respect a set of formal properties called Invariants.
Proceedings of the 12th International Conference on Software Technologies
The provisioning of a growing number of services depends on the proper interoperation of multiple products, forming a new distributed system, often subject to timing requirements. To ensure the interoperability and timely behavior of this new distributed system, it is important to conduct integration tests that verify the interactions with the environment and between the system components. Integration test scenarios for that purpose may be conveniently specified by means of UML sequence diagrams (SDs) enriched with time constraints. The automation of such integration tests requires that test components are also distributed, with a local tester deployed close to each system component, coordinated by a central tester. The distributed observation of execution events, combined with the impossibility to ensure clock synchronization in a distributed system, poses special challenges for checking the conformance of the observed execution traces against the specification, possibly yielding inconclusive verdicts. Hence, in this paper we investigate decision procedures and criteria to check the conformance of observed execution traces against a specification set by a UML SD enriched with time constraints. The procedures and criteria are specified in a formal language that allows executing and validating the specification. Examples are presented to illustrate the approach.
Model Driven Testing of Time Sensitive Distributed Systems
In this paper we demonstrate an approach to model structure and behavior of distributed systems, to map those models to a lightweight execution engine by using a functional programming language and to systematically define and execute tests for these models within the same technology. This is a prerequisite for a smooth integration of model based development into an agile method. The novelty of this paper is the demonstration, how composition and state machine models for distributed asynchronously communicating systems can easily be mapped to a lazy functional language and then using standard testing techniques to define test on those programs. In particular distributed timing aspects and underspecification can be treated accordingly within a functional language, using a certain style of functions.
Model checking, testing and verification working together
Formal Aspects of Computing, 2005
We present a symbolic model checking approach that allows verifying a unit of code, e.g., a single procedure or a collection of procedures that interact with each other. We allow temporal specifications that assert over both theprogram countersand theprogram variables. We decompose the verification into two parts: (1) a search that is based on the temporal behavior of theprogram counters, and (2) the formulation and refutation of a path condition, which inherits conditions constraining theprogram variablesfrom the temporal specification. This verification approach is modular, as we do not require that all the involved procedures are provided. Furthermore, we do not request that the code is based on a finite domain. The presented approach can also be used for automating the generation of test cases for unit testing.
A model checker for performance and dependability properties
2001
Abstract—Markov chains are widely used in the context of performance and reliability evaluation of systems of various nature. Model checking of such chains with respect to a given (branching) temporal logic formula has been proposed for both the discrete [8] and the continuous time setting [1],[3]. In this short paper, we describe the prototype model checker E TMC2 for discrete and continuous-time Markov chains, where properties are expressed in appropriate extensions of CTL.
Towards Decentralized Conformance Checking in Model-Based Testing of Distributed Systems
2017
In a growing number of domains, the provisioning of end-to-end services to the users depends on the proper interoperation of multiple products, forming a new distributed system. To ensure interoperability and the integrity of this new distributed system, it is important to conduct integration tests that verify not only the interactions with the environment but also the interactions between the system components. Integration test scenarios for that purpose may be conveniently specified by means of UML sequence diagrams, possibly allowing multiple execution paths. The automation of such integration tests requires that test components are also distributed, with a local tester deployed close to each system component, and a central tester coordinating the local testers. In such a test architecture, it is important to minimize the communication overhead during test execution. Hence, in this paper we investigate conditions upon which conformance errors can be detected locally (local observability) and test inputs can be decided locally (local controllability) by the local testers, without the need for exchanging coordination messages between the test components during test execution. The conditions are specified in a formal specification language that allows executing and validating the specification. Examples of test scenarios are also presented, illustrating local observability and controllability problems associated with optional messages without corresponding acknowledgment messages, races and nonlocal choices.