Automata Based Monitoring and Mining of Execution Traces (original) (raw)

Generating Properties for Runtime Monitoring from Software Specification Patterns

International Journal of Software Engineering and Knowledge Engineering, 2007

This paper presents an approach to support run-time verification of software systems that combines two existing tools, Prospec and Java-MaC, into a single framework. Prospec can be used to clarify natural language specifications for sequential, concurrent, and nondeterministic behavior. In addition, Prospec assists the user in reading, writing, and understanding formal specifications through the use of property patterns and visual abstractions. Prospec automatically generates specifications written in Future Interval Logic (FIL). Java-MaC monitors Java programs at runtime to ensure adherence to a set of formally specified properties. Safety properties of a program are specified in the formal language Meta-Event Definition Language (MEDL). Java-MaC generates runtime components from specifications. The components are used to instrument the target program and determine whether the execution of the program violates any of the safety properties. This paper describes an algorithm for tran...

A Pattern-Based Approach to Parametric Specification Mining

This paper presents a technique for using execution traces to mine parametric temporal specifications in the form of quantified event automata (QEA) - previously introduced as an expressive and efficient formalism for runtime verification. We consider a pattern-based mining approach that uses a pattern library to generate and check potential properties over given traces, and then combines successful patterns. By using predefined models to measure the tool’s precision and recall we demonstrate that our approach can effectively and efficiently extract specifications in realistic scenarios.

Specification and error pattern based program monitoring

2001

We briefly present Java PathExplorer (JPAX), a tool developed at NASA Ames for monitoring the execution of Java programs../PAX can be used not only during program testing to reveal subtle errors, but also can be applied during operation to surJey safety critical systems. The tool facilitates automated instrumentation of a program in order to property observe its execution. The instrumentation can be either at the bytecode level or at the source level when the tource code is available. JPaX is an instance of a more general project, called PathExplorer (PAX), which is a basis for experiments rather than a fixed system, capable of monitoring various programming languages and ex!: erimenting with other logics and analysis techniques.

Towards Fully Automatic Execution Monitoring

2008

Abstract. UFO is a new application framework in which programs written in FORMAN, a declarative assertion language, are compiled into execution monitors that run on a virtual machine with extensive monitoring capabilities provided by the Alamo monitor architecture. FORMAN provides an event trace model in which precedence and inclusion relations define a DAG structure that abstracts execution behavior. Compiling FORMAN assertions into hybrid run-time/post-mortem monitors allows substantial speed and size improvements over post-mortem analyzers. The UFO compiler generates code that computes the minimal projection of the DAG necessary for a given set of assertions. UFO enables fully automatic execution monitoring of real programs. The approach is non-intrusive with respect to program source code and provides a high level of abstraction for monitoring and debugging activities. The ability to compile suites of debugging rules into efficient monitors, and apply them generically to differe...

Interactive Semi-automated Specification Mining for Debugging: An Experience Report

Information and Software Technology, 2019

Context: Specification mining techniques are typically used to extract the specification of a software in the absence of (up-to-date) specification documents. This is useful for program comprehension, testing, and anomaly detection. However, specification mining can also potentially be used for debugging, where a faulty behavior is abstracted to give developers a context about the bug and help them locating it. Objective: In this project, we investigate this idea in an industrial setting. We propose a very basic semi-automated specification mining approach for debugging and apply that on real reported issues from an AutoPilot software system from our industry partner, MicroPilot Inc. The objective is to assess the feasibility and usefulness of the approach in a real-world setting. Method: The approach is developed as a prototype tool, working on C code, which accept a set of relevant state fields and functions, per issue, and generates an extended finite state machine that represents the faulty behavior, abstracted with respect to the relevant context (the selected fields and functions). Results: We qualitatively evaluate the approach by a set of interviews (including observational studies) with the company's developers on their real-world reported bugs. The results show that a) our approach is feasible, b) it can be automated to some extent, and c) brings advantages over only using their code-level debugging tools. We also compared this approach with traditional fully automated state-merging algorithms and reported several issues when applying those techniques on a real-world debugging context. Conclusion: The main conclusion of this study is that the idea of an "interactive" specification mining rather than a fully automated mining tool is NOT impractical and indeed is useful for the debugging use case.

Execution Traces: A New Domain That Requires the Creation of a Standard Metamodel

Communications in Computer and Information Science, 2009

Despite the fact dynamic analysis techniques of software systems have been shown to be useful in many software engineering activities such as software maintenance, software performance, testing, etc., there is no standard format for representing run-time information, which hinders interoperability and sharing of data. Runtime information is typically represented in the form of execution traces. Traces can contain different information, and can contain different types of information depending on what is being traced and the purpose of the trace. In this paper, we argue that traces represent vital knowledge about software that needs to be organized and modeled. We support our arguments by discussing the various types of traces used in the literature. We also discuss the challenges when dealing with execution traces and why a trace metamodel has to be carefully designed to overcome these challenges. We also discuss existing attempts to model execution traces. Finally, we discuss how the Knowledge Discovery Metamodel can be extended to support efficiently the modeling of large and complex execution traces.

Static Specification Mining Using Automata-Based Abstractions

IEEE Transactions on Software Engineering, 2000

We present a novel approach to client-side mining of temporal API specifications based on static analysis. Specifically, we present an interprocedural analysis over a combined domain that abstracts both aliasing and event sequences for individual objects. The analysis uses a new family of automata-based abstractions to represent unbounded event sequences, designed to disambiguate distinct usage patterns and merge similar usage patterns. Additionally, our approach includes an algorithm that summarizes abstract traces based on automata clusters, and effectively rules out spurious behaviors.

Detecting Implied Scenarios from Execution Traces

2007

A scenario describes how one or more system components interact to provide a certain set of functionalities. Because each scenario only represents a partial view of the overall system behavior, scenario-based specifications may hide unexpected interactions (called implied scenarios) which are not part of any scenario individually. Implied scenarios may either indicate gaps in the original scenario specification, or undesired behavior that should be avoided during scenario implementation. This paper presents a reverse engineering environment to support extraction and detection of implied scenarios from execution traces. The main contribution of the paper is to demonstrate how the concept of implied scenarios, thus far restricted to early phases of the software life-cycle, such as requirements elaboration and reliability prediction, can also be applied to support comprehension and testing of existing systems.

A Review on Specification Mining Architecture

2014

To aid program testing efforts and help program verification tools to find bugs and ensure correctness of systems the mined specifications can be used. Mining of specification process starts with a program under analysis and/or a set of test cases. By applying static or dynamic analysis method traces should be produced from the program. Techniques employing dynamic analysis require the running of the test cases to produce a set of traces which is later analyzed. Program traces are often long and involve several phases. Semantic clustering can be used to split the traces vertically into phases based on comments or annotations on source code. This paper shows a review on the architecture of the automaton based specification mining process.