Michael Whalen - Academia.edu (original) (raw)

Papers by Michael Whalen

Research paper thumbnail of Validity-Guided Synthesis of Reactive Systems from Assume-Guarantee Contracts

arXiv (Cornell University), Sep 14, 2017

Automated synthesis of reactive systems from specifications has been a topic of research for deca... more Automated synthesis of reactive systems from specifications has been a topic of research for decades. Recently, a variety of approaches have been proposed to extend synthesis of reactive systems from propositional specifications towards specifications over rich theories. We propose a novel, completely automated approach to program synthesis which reduces the problem to deciding the validity of a set of ∀∃-formulas. In spirit of IC3 / PDR, our problem space is recursively refined by blocking out regions of unsafe states, aiming to discover a fixpoint that describes safe reactions. If such a fixpoint is found, we construct a witness that is directly translated into an implementation. We implemented the algorithm on top of the JKind model checker, and exercised it against contracts written using the Lustre specification language. Experimental results show how the new algorithm outperforms JKind's already existing synthesis procedure based on-induction and addresses soundness issues in the-inductive approach with respect to unrealizable results.

Research paper thumbnail of The JKind Model Checker

arXiv (Cornell University), Dec 4, 2017

JKind is an open-source industrial model checker developed by Rockwell Collins and the University... more JKind is an open-source industrial model checker developed by Rockwell Collins and the University of Minnesota. JKind uses multiple parallel engines to prove or falsify safety properties of infinite state models. It is performance competitive with other state-of-the-art model checkers and serves as the back-end for various industrial applications.

Research paper thumbnail of On Implementing Real-Time Specification Patterns Using Observers

Lecture Notes in Computer Science, 2016

English language requirements are often used to specify the behavior of complex cyber-physical sy... more English language requirements are often used to specify the behavior of complex cyber-physical systems. The process of transforming these requirements to a formal specification language is often challenging, especially if the specification language does not contain constructs analogous to those used in the original requirements. For example, requirements often contain real-time constraints, but many specification languages for model checkers have discrete time semantics. Work in specification patterns helps to bridge these gaps, allowing straightforward expression of common requirements patterns in formal languages. In this work we demonstrate how we support real-time specification patterns in the Assume Guarantee Reasoning Environment (AGREE) using observers. We demonstrate that there are subtle challenges, not mentioned in previous literature, to express real-time patterns accurately using observers. We then demonstrate that these patterns are sufficient to model real-time requirements for a real-world avionics system.

Research paper thumbnail of Design Considerations for Modeling Modes in Cyber–Physical Systems

IEEE Design & Test, 2015

The behavior of control systems is frequently defined in terms of operational modes. The modes an... more The behavior of control systems is frequently defined in terms of operational modes. The modes and the rules defining the transitions between them is called mode logic. Lucent and understandable modeling of mode logic can be challenging due to the plurality of modes and the complexity of the rules that govern the transitions. In this paper we discuss the challenges and present a modeling pattern to structure the mode logic of cyber physical systems, using an infusion pump as a case example. The intent of this article is to provide the initial steps towards a catalog of solutions for various modeling problems and build a repertoire of modeling patterns.

Research paper thumbnail of From Requirements to Code: Model Based Development of a Medical Cyber Physical System

Lecture Notes in Computer Science, 2017

The advanced use of technology in medical devices has improved the way health care is delivered t... more The advanced use of technology in medical devices has improved the way health care is delivered to patients. Unfortunately, the increased complexity of modern medical devices poses challenges for development, assurance, and regulatory approval. In an effort to improve the safety of advanced medical devices, organizations such as FDA have supported exploration of techniques to aid in the development and regulatory approval of such systems. In an ongoing research project, our aim is to provide effective development techniques and exemplars of system development artifacts that demonstrate state of the art development techniques. In this paper we present an end-to-end model-based approach to medical device software development along with the artifacts created in the process. While outlining the approach, we also describe our experiences, challenges, and lessons learned in the process of formulating and analyzing the requirements, modeling the system, formally verifying the models, generating code, and executing the generated code in the hardware for generic patient controlled analgesic infusion pump (GPCA). We believe that the development artifacts and techniques presented in this paper could serve as a generic reference to be used by researchers, practitioners, and authorities while developing and evaluating cyber physical medical devices.

Research paper thumbnail of Java Ranger at SV-COMP 2020 (Competition Contribution)

Tools and Algorithms for the Construction and Analysis of Systems, 2020

Path-merging is a known technique for accelerating symbolic execution. One technique, named “veri... more Path-merging is a known technique for accelerating symbolic execution. One technique, named “veritesting” by Avgerinos et al. uses summaries of bounded control-flow regions and has been shown to accelerate symbolic execution of binary code. But, when applied to symbolic execution of Java code, veritesting needs to be extended to summarize dynamically dispatched methods and exceptional control-flow. Such an extension of veritesting has been implemented in Java Ranger by implementing as an extension of Symbolic PathFinder, a symbolic executor for Java bytecode. In this paper, we briefly describe the architecture of Java Ranger and describe its setup for SV-COMP 2020.

Research paper thumbnail of Architectural Modeling and Analysis for Safety Engineering

Model-Based Safety and Assessment, 2017

Architecture description languages such as AADL allow systems engineers to specify the structure ... more Architecture description languages such as AADL allow systems engineers to specify the structure of system architectures and perform several analyses over them, including schedulability, resource analysis, and information flow. In addition, they permit system-level requirements to be specified and analyzed early in the development process of airborne and ground-based systems. These tools can also be used to perform safety analysis based on the system architecture and initial functional decomposition. Using AADL-based system architecture modeling and analysis tools as an exemplar, we extend existing analysis methods to support system safety objectives of ARP4754A and ARP4761. This includes extensions to existing modeling languages to better describe failure conditions, interactions, and mitigations, and improvements to compositional reasoning approaches focused on the specific needs of system safety analysis. We develop example systems based on the Wheel Braking System in SAE AIR6110 to evaluate the effectiveness and practicality of our approach.

Research paper thumbnail of The JKind Model Checker

Computer Aided Verification, 2018

JKind is an open-source industrial model checker developed by Rockwell Collins and the University... more JKind is an open-source industrial model checker developed by Rockwell Collins and the University of Minnesota. JKind uses multiple parallel engines to prove or falsify safety properties of infinite state models. It is portable, easy to install, performance competitive with other state-of-the-art model checkers, and has features designed to improve the results presented to users: inductive validity cores for proofs and counterexample smoothing for test-case generation. It serves as the back-end for various industrial applications.

Research paper thumbnail of Hierarchical multi-formalism proofs of cyber-physical systems

2015 ACM/IEEE International Conference on Formal Methods and Models for Codesign (MEMOCODE), 2015

To manage design complexity and provide verification tractability, models of complex cyber-physic... more To manage design complexity and provide verification tractability, models of complex cyber-physical systems are typically hierarchically organized into multiple abstraction layers. High-level analysis explores interactions of the system with its physical environment, while embedded software is developed separately based on derived requirements. This separation of lowlevel and high-level analysis also gives hope to scalability, because we are able to use tools that are appropriate for each level. When attempting to perform compositional reasoning in such an environment, care must be taken to ensure that results from one tool can be used in another to avoid errors due to "mismatches" in the semantics of the underlying formalisms. This paper proposes a formal approach for linking high-level continuous time models and lower-level discrete time models. Specifically, we lift a discrete-time controller specified using synchronous observer properties into continuous time for proof using timed automata (UPPAAL). To define semantic compatibility between the models, we propose a direct semantics for a network of timed automata with a discrete-time component called Contract-Extended Network of Timed Automata (CENTA) and examine semantic issues involving timing and events with the combination. We then propose a translation of the discrete-time controller into a timed automata state machine and show the equivalence of the translation with the CENTA formulation. We demonstrate the usefulness of the approach by proving that a complex medical infusion pump controller is safe with respect to a continuous time clinical scenario.

Research paper thumbnail of Complexity-reducing design patterns for cyber-physical systems

Research paper thumbnail of Proof-based coverage metrics for formal verification

2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE), 2017

When using formal verification on critical software, an important question involves whether we ha... more When using formal verification on critical software, an important question involves whether we have we specified enough properties for a given implementation model. To address this question, coverage metrics for property-based formal verification have been proposed. Existing metrics are usually based on mutation, where the implementation model is repeatedly modified and re-analyzed to determine whether mutant models are "killed" by the property set. These metrics tend to be very expensive to compute, as they involve many additional verification problems. This paper proposes an alternate family of metrics that can be computed using the recently introduced idea of Inductive Validity Cores (IVCs). IVCs determine a minimal set of model elements necessary to establish a proof. One of the proposed metrics is both rigorous and substantially cheaper to compute than mutation-based metrics. In addition, unlike the mutationbased techniques, the design elements marked as necessary by the metric are guaranteed to preserve provability. We demonstrate the metrics on a large corpus of examples.

Research paper thumbnail of One-Click Formal Methods

Research paper thumbnail of Efficient observability-based test generation by dynamic symbolic execution

2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE), 2015

Structural coverage metrics have been widely used to measure test suite adequacy as well as to ge... more Structural coverage metrics have been widely used to measure test suite adequacy as well as to generate test cases. In previous investigations, we have found that the fault-finding effectiveness of tests satisfying structural coverage criteria is highly dependent on program syntax-even if the faulty code is exercised, its effect may not be observable at the output. To address these problems, observability-based coverage metrics have been defined. Specifically, Observable MC/DC (OMC/DC) is a criterion that appears to be both more effective at detecting faults and more robust to program restructuring than MC/DC. Traditional counterexample-based test generation for OMC/DC, however, can be infeasible on large systems. In this study, we propose an incremental test generation approach that combines the notion of observability with dynamic symbolic execution. We evaluated the efficiency and effectiveness of our approach using seven systems from the avionics and medical device domains. Our results show that the incremental approach requires much lower generation time, while achieving even higher fault finding effectiveness compared with regular OMC/DC generation.

Research paper thumbnail of The Effect of Program and Model Structure on the Effectiveness of MC/DC Test Adequacy Coverage

ACM Transactions on Software Engineering and Methodology, 2016

Test adequacy metrics defined over the structure of a program, such as Modified Condition and Dec... more Test adequacy metrics defined over the structure of a program, such as Modified Condition and Decision Coverage (MC/DC), are used to assess testing efforts. However, MC/DC can be “cheated” by restructuring a program to make it easier to achieve the desired coverage. This is concerning, given the importance of MC/DC in assessing the adequacy of test suites for critical systems domains. In this work, we have explored the impact of implementation structure on the efficacy of test suites satisfying the MC/DC criterion using four real-world avionics systems. Our results demonstrate that test suites achieving MC/DC over implementations with structurally complex Boolean expressions are generally larger and more effective than test suites achieving MC/DC over functionally equivalent, but structurally simpler, implementations. Additionally, we found that test suites generated over simpler implementations achieve significantly lower MC/DC and fault-finding effectiveness when applied to comple...

Research paper thumbnail of Efficient generation of all minimal inductive validity cores

2017 Formal Methods in Computer Aided Design (FMCAD), 2017

Symbolic model checkers can construct proofs of safety properties over complex models, but when a... more Symbolic model checkers can construct proofs of safety properties over complex models, but when a proof succeeds, the results do not generally provide much insight to the user. Recently, proof cores (alternately, for inductive model checkers, Inductive Validity Cores (IVCs)) were introduced to trace a property to a minimal set of model elements necessary for proof. Minimal IVCs facilitate several engineering tasks, including performing traceability and analyzing requirements completeness, that usually rely on the minimality of IVCs. However, existing algorithms for generating an IVC are either expensive or only able to find an approximately minimal IVC. Besides minimality, computing all minimal IVCs of a given property is an interesting problem that provides several useful analyses, including regression analysis for testing/proof, determination of the minimum (as opposed to minimal) number of model elements necessary for proof, the diversity examination of model elements leading to proof, and analyzing fault tolerance. This paper proposes an efficient method for finding all minimal IVCs of a given property proving its correctness and completeness. We benchmark our algorithm against existing IVCgenerating algorithms and show, in many cases, the cost of finding all minimal IVCs by our technique is similar to finding a single minimal IVC using existing algorithms.

Research paper thumbnail of Towards Synthesis from Assume-Guarantee Contracts involving Infinite Theories: A Preliminary Report

In previous work, we have introduced a contract-based real- izability checking algorithm for assu... more In previous work, we have introduced a contract-based real- izability checking algorithm for assume-guarantee contracts involving infinite theories, such as linear integer/real arith- metic and uninterpreted functions over infinite domains. This algorithm can determine whether or not it is possible to con- struct a realization (i.e. an implementation) of an assume- guarantee contract. The algorithm is similar to k-induction model checking, but involves the use of quantifiers to deter- mine implementability. While our work on realizability is inherently useful for vir- tual integration in determining whether it is possible for sup- pliers to build software that meets a contract, it also provides the foundations to solving the more challenging problem of component synthesis. In this paper, we provide an initial synthesis algorithm for assume-guarantee contracts involv- ing infinite theories. To do so, we take advantage of our realizability checking procedure and a skolemization solver...

Research paper thumbnail of In a ubiquitous world requirements are ubiquitous too

2011 IEEE 19th International Requirements Engineering Conference, 2011

Abstract Summary form only given. The soaring presence of devices that can sense the environment,... more Abstract Summary form only given. The soaring presence of devices that can sense the environment, human activity and social interactions in a ubiquitous fashion, opens the doors to potentially very effective multi-disciplinary research. Battery-powered tiny sensors can be distributed across an area to monitor conditions with very fine granularity. Moreover, mobile phones are powerful sensors that we voluntarily carry throughout our daily life. However, as well as introducing exciting opportunities, these technologies offer many challenges: ...

Research paper thumbnail of A Flexible and Non-intrusive Approach for Computing Complex Structural Coverage Metrics

2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, 2015

Research paper thumbnail of IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, TBD 1 Automated Oracle Data Selection Support

Abstract—The choice of test oracle—the artifact that determines whether an application under test... more Abstract—The choice of test oracle—the artifact that determines whether an application under test executes correctly—can significantly impact the effectiveness of the testing process. However, despite the prevalence of tools that support test input selection, little work exists for supporting oracle creation. We propose a method of supporting test oracle creation that automatically selects the oracle data—the set of variables monitored during testing—for expected value test oracles. This approach is based on the use of mutation analysis to rank variables in terms of fault-finding effectiveness, thus automating the selection of the oracle data. Experimental results obtained by employing our method over six industrial systems (while varying test input types and the number of generated mutants) indicate that our method—when paired with test inputs generated either at random or to satisfy specific structural coverage criteria—may be a cost-effective approach for producing small, effecti...

Research paper thumbnail of Introduction to the Guardol Programming Language and Verification System

Guardol is a high-level programming language intended to facilitate the construction of correct n... more Guardol is a high-level programming language intended to facilitate the construction of correct network guards. The Guardol system generates Ada code from Guardol programs. It also provides specification and automated verification support: guard specifications are formally translated to SMT format and passed to a new decision procedure dealing with functions over tree-structured data. The result is that difficult properties of Guardol programs can be proved fully automatically.

Research paper thumbnail of Validity-Guided Synthesis of Reactive Systems from Assume-Guarantee Contracts

arXiv (Cornell University), Sep 14, 2017

Automated synthesis of reactive systems from specifications has been a topic of research for deca... more Automated synthesis of reactive systems from specifications has been a topic of research for decades. Recently, a variety of approaches have been proposed to extend synthesis of reactive systems from propositional specifications towards specifications over rich theories. We propose a novel, completely automated approach to program synthesis which reduces the problem to deciding the validity of a set of ∀∃-formulas. In spirit of IC3 / PDR, our problem space is recursively refined by blocking out regions of unsafe states, aiming to discover a fixpoint that describes safe reactions. If such a fixpoint is found, we construct a witness that is directly translated into an implementation. We implemented the algorithm on top of the JKind model checker, and exercised it against contracts written using the Lustre specification language. Experimental results show how the new algorithm outperforms JKind's already existing synthesis procedure based on-induction and addresses soundness issues in the-inductive approach with respect to unrealizable results.

Research paper thumbnail of The JKind Model Checker

arXiv (Cornell University), Dec 4, 2017

JKind is an open-source industrial model checker developed by Rockwell Collins and the University... more JKind is an open-source industrial model checker developed by Rockwell Collins and the University of Minnesota. JKind uses multiple parallel engines to prove or falsify safety properties of infinite state models. It is performance competitive with other state-of-the-art model checkers and serves as the back-end for various industrial applications.

Research paper thumbnail of On Implementing Real-Time Specification Patterns Using Observers

Lecture Notes in Computer Science, 2016

English language requirements are often used to specify the behavior of complex cyber-physical sy... more English language requirements are often used to specify the behavior of complex cyber-physical systems. The process of transforming these requirements to a formal specification language is often challenging, especially if the specification language does not contain constructs analogous to those used in the original requirements. For example, requirements often contain real-time constraints, but many specification languages for model checkers have discrete time semantics. Work in specification patterns helps to bridge these gaps, allowing straightforward expression of common requirements patterns in formal languages. In this work we demonstrate how we support real-time specification patterns in the Assume Guarantee Reasoning Environment (AGREE) using observers. We demonstrate that there are subtle challenges, not mentioned in previous literature, to express real-time patterns accurately using observers. We then demonstrate that these patterns are sufficient to model real-time requirements for a real-world avionics system.

Research paper thumbnail of Design Considerations for Modeling Modes in Cyber–Physical Systems

IEEE Design & Test, 2015

The behavior of control systems is frequently defined in terms of operational modes. The modes an... more The behavior of control systems is frequently defined in terms of operational modes. The modes and the rules defining the transitions between them is called mode logic. Lucent and understandable modeling of mode logic can be challenging due to the plurality of modes and the complexity of the rules that govern the transitions. In this paper we discuss the challenges and present a modeling pattern to structure the mode logic of cyber physical systems, using an infusion pump as a case example. The intent of this article is to provide the initial steps towards a catalog of solutions for various modeling problems and build a repertoire of modeling patterns.

Research paper thumbnail of From Requirements to Code: Model Based Development of a Medical Cyber Physical System

Lecture Notes in Computer Science, 2017

The advanced use of technology in medical devices has improved the way health care is delivered t... more The advanced use of technology in medical devices has improved the way health care is delivered to patients. Unfortunately, the increased complexity of modern medical devices poses challenges for development, assurance, and regulatory approval. In an effort to improve the safety of advanced medical devices, organizations such as FDA have supported exploration of techniques to aid in the development and regulatory approval of such systems. In an ongoing research project, our aim is to provide effective development techniques and exemplars of system development artifacts that demonstrate state of the art development techniques. In this paper we present an end-to-end model-based approach to medical device software development along with the artifacts created in the process. While outlining the approach, we also describe our experiences, challenges, and lessons learned in the process of formulating and analyzing the requirements, modeling the system, formally verifying the models, generating code, and executing the generated code in the hardware for generic patient controlled analgesic infusion pump (GPCA). We believe that the development artifacts and techniques presented in this paper could serve as a generic reference to be used by researchers, practitioners, and authorities while developing and evaluating cyber physical medical devices.

Research paper thumbnail of Java Ranger at SV-COMP 2020 (Competition Contribution)

Tools and Algorithms for the Construction and Analysis of Systems, 2020

Path-merging is a known technique for accelerating symbolic execution. One technique, named “veri... more Path-merging is a known technique for accelerating symbolic execution. One technique, named “veritesting” by Avgerinos et al. uses summaries of bounded control-flow regions and has been shown to accelerate symbolic execution of binary code. But, when applied to symbolic execution of Java code, veritesting needs to be extended to summarize dynamically dispatched methods and exceptional control-flow. Such an extension of veritesting has been implemented in Java Ranger by implementing as an extension of Symbolic PathFinder, a symbolic executor for Java bytecode. In this paper, we briefly describe the architecture of Java Ranger and describe its setup for SV-COMP 2020.

Research paper thumbnail of Architectural Modeling and Analysis for Safety Engineering

Model-Based Safety and Assessment, 2017

Architecture description languages such as AADL allow systems engineers to specify the structure ... more Architecture description languages such as AADL allow systems engineers to specify the structure of system architectures and perform several analyses over them, including schedulability, resource analysis, and information flow. In addition, they permit system-level requirements to be specified and analyzed early in the development process of airborne and ground-based systems. These tools can also be used to perform safety analysis based on the system architecture and initial functional decomposition. Using AADL-based system architecture modeling and analysis tools as an exemplar, we extend existing analysis methods to support system safety objectives of ARP4754A and ARP4761. This includes extensions to existing modeling languages to better describe failure conditions, interactions, and mitigations, and improvements to compositional reasoning approaches focused on the specific needs of system safety analysis. We develop example systems based on the Wheel Braking System in SAE AIR6110 to evaluate the effectiveness and practicality of our approach.

Research paper thumbnail of The JKind Model Checker

Computer Aided Verification, 2018

JKind is an open-source industrial model checker developed by Rockwell Collins and the University... more JKind is an open-source industrial model checker developed by Rockwell Collins and the University of Minnesota. JKind uses multiple parallel engines to prove or falsify safety properties of infinite state models. It is portable, easy to install, performance competitive with other state-of-the-art model checkers, and has features designed to improve the results presented to users: inductive validity cores for proofs and counterexample smoothing for test-case generation. It serves as the back-end for various industrial applications.

Research paper thumbnail of Hierarchical multi-formalism proofs of cyber-physical systems

2015 ACM/IEEE International Conference on Formal Methods and Models for Codesign (MEMOCODE), 2015

To manage design complexity and provide verification tractability, models of complex cyber-physic... more To manage design complexity and provide verification tractability, models of complex cyber-physical systems are typically hierarchically organized into multiple abstraction layers. High-level analysis explores interactions of the system with its physical environment, while embedded software is developed separately based on derived requirements. This separation of lowlevel and high-level analysis also gives hope to scalability, because we are able to use tools that are appropriate for each level. When attempting to perform compositional reasoning in such an environment, care must be taken to ensure that results from one tool can be used in another to avoid errors due to "mismatches" in the semantics of the underlying formalisms. This paper proposes a formal approach for linking high-level continuous time models and lower-level discrete time models. Specifically, we lift a discrete-time controller specified using synchronous observer properties into continuous time for proof using timed automata (UPPAAL). To define semantic compatibility between the models, we propose a direct semantics for a network of timed automata with a discrete-time component called Contract-Extended Network of Timed Automata (CENTA) and examine semantic issues involving timing and events with the combination. We then propose a translation of the discrete-time controller into a timed automata state machine and show the equivalence of the translation with the CENTA formulation. We demonstrate the usefulness of the approach by proving that a complex medical infusion pump controller is safe with respect to a continuous time clinical scenario.

Research paper thumbnail of Complexity-reducing design patterns for cyber-physical systems

Research paper thumbnail of Proof-based coverage metrics for formal verification

2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE), 2017

When using formal verification on critical software, an important question involves whether we ha... more When using formal verification on critical software, an important question involves whether we have we specified enough properties for a given implementation model. To address this question, coverage metrics for property-based formal verification have been proposed. Existing metrics are usually based on mutation, where the implementation model is repeatedly modified and re-analyzed to determine whether mutant models are "killed" by the property set. These metrics tend to be very expensive to compute, as they involve many additional verification problems. This paper proposes an alternate family of metrics that can be computed using the recently introduced idea of Inductive Validity Cores (IVCs). IVCs determine a minimal set of model elements necessary to establish a proof. One of the proposed metrics is both rigorous and substantially cheaper to compute than mutation-based metrics. In addition, unlike the mutationbased techniques, the design elements marked as necessary by the metric are guaranteed to preserve provability. We demonstrate the metrics on a large corpus of examples.

Research paper thumbnail of One-Click Formal Methods

Research paper thumbnail of Efficient observability-based test generation by dynamic symbolic execution

2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE), 2015

Structural coverage metrics have been widely used to measure test suite adequacy as well as to ge... more Structural coverage metrics have been widely used to measure test suite adequacy as well as to generate test cases. In previous investigations, we have found that the fault-finding effectiveness of tests satisfying structural coverage criteria is highly dependent on program syntax-even if the faulty code is exercised, its effect may not be observable at the output. To address these problems, observability-based coverage metrics have been defined. Specifically, Observable MC/DC (OMC/DC) is a criterion that appears to be both more effective at detecting faults and more robust to program restructuring than MC/DC. Traditional counterexample-based test generation for OMC/DC, however, can be infeasible on large systems. In this study, we propose an incremental test generation approach that combines the notion of observability with dynamic symbolic execution. We evaluated the efficiency and effectiveness of our approach using seven systems from the avionics and medical device domains. Our results show that the incremental approach requires much lower generation time, while achieving even higher fault finding effectiveness compared with regular OMC/DC generation.

Research paper thumbnail of The Effect of Program and Model Structure on the Effectiveness of MC/DC Test Adequacy Coverage

ACM Transactions on Software Engineering and Methodology, 2016

Test adequacy metrics defined over the structure of a program, such as Modified Condition and Dec... more Test adequacy metrics defined over the structure of a program, such as Modified Condition and Decision Coverage (MC/DC), are used to assess testing efforts. However, MC/DC can be “cheated” by restructuring a program to make it easier to achieve the desired coverage. This is concerning, given the importance of MC/DC in assessing the adequacy of test suites for critical systems domains. In this work, we have explored the impact of implementation structure on the efficacy of test suites satisfying the MC/DC criterion using four real-world avionics systems. Our results demonstrate that test suites achieving MC/DC over implementations with structurally complex Boolean expressions are generally larger and more effective than test suites achieving MC/DC over functionally equivalent, but structurally simpler, implementations. Additionally, we found that test suites generated over simpler implementations achieve significantly lower MC/DC and fault-finding effectiveness when applied to comple...

Research paper thumbnail of Efficient generation of all minimal inductive validity cores

2017 Formal Methods in Computer Aided Design (FMCAD), 2017

Symbolic model checkers can construct proofs of safety properties over complex models, but when a... more Symbolic model checkers can construct proofs of safety properties over complex models, but when a proof succeeds, the results do not generally provide much insight to the user. Recently, proof cores (alternately, for inductive model checkers, Inductive Validity Cores (IVCs)) were introduced to trace a property to a minimal set of model elements necessary for proof. Minimal IVCs facilitate several engineering tasks, including performing traceability and analyzing requirements completeness, that usually rely on the minimality of IVCs. However, existing algorithms for generating an IVC are either expensive or only able to find an approximately minimal IVC. Besides minimality, computing all minimal IVCs of a given property is an interesting problem that provides several useful analyses, including regression analysis for testing/proof, determination of the minimum (as opposed to minimal) number of model elements necessary for proof, the diversity examination of model elements leading to proof, and analyzing fault tolerance. This paper proposes an efficient method for finding all minimal IVCs of a given property proving its correctness and completeness. We benchmark our algorithm against existing IVCgenerating algorithms and show, in many cases, the cost of finding all minimal IVCs by our technique is similar to finding a single minimal IVC using existing algorithms.

Research paper thumbnail of Towards Synthesis from Assume-Guarantee Contracts involving Infinite Theories: A Preliminary Report

In previous work, we have introduced a contract-based real- izability checking algorithm for assu... more In previous work, we have introduced a contract-based real- izability checking algorithm for assume-guarantee contracts involving infinite theories, such as linear integer/real arith- metic and uninterpreted functions over infinite domains. This algorithm can determine whether or not it is possible to con- struct a realization (i.e. an implementation) of an assume- guarantee contract. The algorithm is similar to k-induction model checking, but involves the use of quantifiers to deter- mine implementability. While our work on realizability is inherently useful for vir- tual integration in determining whether it is possible for sup- pliers to build software that meets a contract, it also provides the foundations to solving the more challenging problem of component synthesis. In this paper, we provide an initial synthesis algorithm for assume-guarantee contracts involv- ing infinite theories. To do so, we take advantage of our realizability checking procedure and a skolemization solver...

Research paper thumbnail of In a ubiquitous world requirements are ubiquitous too

2011 IEEE 19th International Requirements Engineering Conference, 2011

Abstract Summary form only given. The soaring presence of devices that can sense the environment,... more Abstract Summary form only given. The soaring presence of devices that can sense the environment, human activity and social interactions in a ubiquitous fashion, opens the doors to potentially very effective multi-disciplinary research. Battery-powered tiny sensors can be distributed across an area to monitor conditions with very fine granularity. Moreover, mobile phones are powerful sensors that we voluntarily carry throughout our daily life. However, as well as introducing exciting opportunities, these technologies offer many challenges: ...

Research paper thumbnail of A Flexible and Non-intrusive Approach for Computing Complex Structural Coverage Metrics

2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, 2015

Research paper thumbnail of IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, TBD 1 Automated Oracle Data Selection Support

Abstract—The choice of test oracle—the artifact that determines whether an application under test... more Abstract—The choice of test oracle—the artifact that determines whether an application under test executes correctly—can significantly impact the effectiveness of the testing process. However, despite the prevalence of tools that support test input selection, little work exists for supporting oracle creation. We propose a method of supporting test oracle creation that automatically selects the oracle data—the set of variables monitored during testing—for expected value test oracles. This approach is based on the use of mutation analysis to rank variables in terms of fault-finding effectiveness, thus automating the selection of the oracle data. Experimental results obtained by employing our method over six industrial systems (while varying test input types and the number of generated mutants) indicate that our method—when paired with test inputs generated either at random or to satisfy specific structural coverage criteria—may be a cost-effective approach for producing small, effecti...

Research paper thumbnail of Introduction to the Guardol Programming Language and Verification System

Guardol is a high-level programming language intended to facilitate the construction of correct n... more Guardol is a high-level programming language intended to facilitate the construction of correct network guards. The Guardol system generates Ada code from Guardol programs. It also provides specification and automated verification support: guard specifications are formally translated to SMT format and passed to a new decision procedure dealing with functions over tree-structured data. The result is that difficult properties of Guardol programs can be proved fully automatically.