Testing Static Analyzers with Randomly Generated Programs (original) (raw)
Related papers
An Automated Framework for Testing Source Code Static Analysis Tools
Proceedings of the Institute for System Programming of the RAS, 2021
Automated testing frameworks are widely used for assuring quality of modern software in secure software development lifecycle. Sometimes it is needed to assure quality of specific software and, hence specific approach should be applied. In this paper, we present an approach and implementation details of automated testing framework suitable for acceptance testing of static source code analysis tools. The presented framework is used for continuous testing of static source code analyzers for C, C++ and Python programs.
Static Code Analysis tools can reduce the number of bugs in one program therefore it can reduce the cost of this program. Many developers don’t use these tools losing a lot of time with manual code analysis (in some cases there are no analysis at all) and a lot of money with resources to do the analysis. In this paper we will test and study the results of three static code analysis tools that by being inexpensive can efficiently remove the most common vulnerabilities in a software. It can be difficult to compare tools with different characteristics but we can get interesting results by testing the tools together.
Testing static analyses for precision and soundness
2020
Static analyses compute properties of programs that are true in all executions, and compilers use these properties to justify optimizations such as dead code elimination. Each static analysis in a compiler should be as precise as possible while remaining sound and being sufficiently fast. Unsound static analyses typically lead to miscompilations, whereas imprecisions typically lead to missed optimizations. Neither kind of bug is easy to track down. Our research uses formal methods to help compiler developers create better static analyses. Our contribution is the design and evaluation of several algorithms for computing sound and maximally precise static analysis results using an SMT solver. These methods are too slow to use at compile time, but they can be used offline to find soundness and precision errors in a production compiler such as LLVM. We found no new soundness bugs in LLVM, but we can discover previously-fixed soundness errors that we reintroduced into the code base. We identified many imprecisions in LLVM's static analyses, some of which have been fixed as a result of our work. CCS Concepts • Software and its engineering → Compilers; Formal software verification; Automated static analysis; • Theory of computation → Design and analysis of algorithms.
Finding and understanding bugs in C compilers
2011
Compilers should be correct. To improve the quality of C compilers, we created Csmith, a randomized test-case generation tool, and spent three years using it to find compiler bugs. During this period we reported more than 325 previously unknown bugs to compiler developers. Every compiler we tested was found to crash and also to silently generate wrong code when presented with valid input. In this paper we present our compiler-testing tool and the results of our bug-hunting study. Our first contribution is to advance the ...
Loopfrog: A static analyzer for ANSI-C programs
2009 IEEE/ACM …, 2009
Practical software verification is dominated by two major classes of techniques. The first is model checking, which provides total precision, but suffers from the state space explosion problem. The second is abstract interpretation, which is usually much less demanding, but often returns a high number of false positives. We present LOOPFROG, a static analyzer that combines the best of both worlds: the precision of model checking and the performance of abstract interpretation. In contrast to traditional static analyzers, it also provides 'leaping' counterexamples to aid in the diagnosis of errors.
StaDy: Deep Integration of Static and Dynamic Analysis in Frama-C
We present StaDy, a new integration of the concolic test generator PathCrawler within the software analysis platform Frama-C. When executing a dynamic analysis of a C code, the integrated test generator also exploits its formal specification, written in an executable fragment of the acsl specification language shared with other analyzers of Frama-C. The test generator provides the user with accurate verdicts, that other Frama-C plugins can reuse to improve their own analyses. This tool is designed to be the foundation stone of static and dynamic analysis combinations in the Frama-C platform. Our first experiments confirm the benefits of such a deep integration of static and dynamic analysis within the same platform.
High Performance Static Analysis for Industry
Static source code analysis for software bug detection has come a long way since its early beginnings as a compiler technology. However, with the introduction of more sophisticated algorithmic techniques, such as model checking and constraint solving, questions about performance are a major concern. In this work we present an empirical study of our industrial strength source code analysis tool Goanna that uses a model checking core for static analysis of C/C++ code. We present the core technology and abstraction mechanism with a focus on performance, as guided by experience from having analyzed millions of lines of code. In particular, we present results from our recent study within the NIST/DHS SAMATE program. The results show that, maybe surprisingly, formal verification techniques can be used successfully in practical industry applications scaling roughly linearly, even for millions of lines of code.
An evaluation of free/open source static analysis tools applied to embedded software
LATW2010 - 11th Latin-American Test Workshop, 2010
Static analysis can be a valuable strategy to improve the quality of embedded software at a lower development cost. In this paper, we have surveyed ten different free/open source tools that perform static software analysis and evaluated their use in embedded software. Experimental results show that the studied tools present widely different results, and most of them are not ready to be applied to embedded systems. Furthermore, we discuss possible directions to improve the use of static analysis tools in the embedded domain.