Statistical Debugging: A Hypothesis Testing-Based Approach (original) (raw)
Related papers
Statistical Debugging Effectiveness as a Fault Localization Approach: Comparative Study
Journal of Software Engineering and Applications, 2016
Fault localization is an important topic in software testing, as it enables the developer to specify fault location in their code. One of the dynamic fault localization techniques is statistical debugging. In this study, two statistical debugging algorithms are implemented, SOBER and Cause Isolation, and then the experimental works are conducted on five programs coded using Python as an example of well-known dynamic programming language. Results showed that in programs that contain only single bug, the two studied statistical debugging algorithms are very effective to localize a bug. In programs that have more than one bug, SOBER algorithm has limitations related to nested predicates, rarely observed predicates and complement predicates. The Cause Isolation has limitations related to sorting predicates based on importance and detecting bugs in predicate condition. The accuracy of both SOBER and Cause Isolation is affected by the program size. Quality comparison showed that SOBER algorithm requires more code examination than Cause Isolation to discover the bugs.
SOBER: statistical model-based bug localization
2005
Automated localization of software bugs is one of the essential issues in debugging aids. Previous studies indicated that the evaluation history of program predicates may disclose important clues about underlying bugs. In this paper, we propose a new statistical model-based approach, called SOBER, which localizes software bugs without any prior knowledge of program semantics. Unlike existing statistical debugging approaches that select predicates correlated with program failures, SOBER models evaluation patterns of predicates in both correct and incorrect runs respectively and regards a predicate as bug-relevant if its evaluation pattern in incorrect runs differs significantly from that in correct ones. SOBER features a principled quantification of the pattern difference that measures the bug-relevance of program predicates.
Statistical bug localisation by supervised clustering of program predicates
International Journal of Information Systems and Change Management, 2018
Regarding the fact that the majority of faults may be revealed as joint effect of program predicates on each other, a new method for localising complex bugs of programs is proposed in this article. The presented approach attempts to identify and select groups of interdependent predicates which altogether may affect the program failure. To find these groups, we suggest the use of a supervised algorithm that is based on penalised logistic regression analysis. To provide the failure context, faulty sub-paths are recognised as sequences of fault relevant predicates. Estimating the grouping effect of program predicates on the failure helps programmers in the multiple-bug setting. Several case studies have been designed to evaluate the proposed approach on well-known test suites. The evaluations show that our method produces more precise results compared with prior fault localisation techniques.
Statistical software debugging: From bug predictors to the main causes of failure
2009
Detecting latent errors is a key challenging issue in the software testing process. Latent errors could be best detected by bug predictors. A bug predictor manifests the effect of a bug on the program execution state. The aim has been to find the smallest reasonable subset of the bug predictors, manifesting all possible bugs within a program. In this paper, a new algorithm for finding the smallest subset of bug predictors is presented. The algorithm, firstly, applies a LASSO method to detect program predicates which have relatively higher effect on the termination status of the program. Then, a ridge regression method is applied to select a subset of the detected predicates as independent representatives of all the program predicates. Program control and data dependency graphs can be best applied to find the causes of bugs represented by the selected bug predictors. Our proposed approach has been evaluated on two well-known test suites. The experimental results demonstrate the effectiveness and accuracy of the proposed approach.
Heuristics for automatic localization of software faults
1992
Abstract Developing effective debugging strategies to guarantee the reliability of software is important. By analyzing the debugging process used by experienced programmers, four distinct tasks are found to be consistently performed:(1) determining statements involved in program failures,(2) selecting suspicious statements that might contain faults,(3) making hypotheses about suspicious faults (variables and locations), and (4) restoring program state to a specific statement for verification.
IJERT-Fine Grained Statistical Debugging for the Identification of Multiple Bugs
International Journal of Engineering Research and Technology (IJERT), 2021
https://www.ijert.org/fine-grained-statistical-debugging-for-the-identification-of-multiple-bugs https://www.ijert.org/research/fine-grained-statistical-debugging-for-the-identification-of-multiple-bugs-IJERTV10IS050308.pdf Commercial software ships with undetected bugs despite the combined efforts of programmers, sophisticated bug detection tools and extensive testing. So, the identification and localization of the bugs in the software becomes essential issues in program debugging. Traditional software debugging is a difficult task to accomplish which requires a lot of time, effort and very good understanding of the source code. Given the scale and complexity of the job, automating the process of program debugging is very essential. Our approach aims at automating the process of program debugging. The earlier proposed approaches namely, statistical debugging and decision tree were able to identify only the most frequently occurring bugs and they failed to identify masked, simultaneously and non-frequently occurring bugs. We propose two approaches: one is decision tree based and the other uses bi-clustering for the task. The results obtained by our proposed approaches showed great improvements in the results in terms of purity, mis-classification rate and over splitting. Our proposed approaches were able to identify all the bugs present in the software including the masked and nonfrequently occurring bugs.
FPA-Debug: Effective Statistical Fault Localization Considering Fault-proneness Analysis
2016
The aim is to identify faulty predicates which have strong effect on program failure. Statistical debugging techniques are amongst best methods for pinpointing defects within the program source code. However, they have some drawbacks. They require a large number of executions to identify faults, they might be adversely affected by coincidental correctness, and they do not take into consideration fault-proneness associated with different parts of the program code while constructing behavioral models. Additionally, they do not consider the simultaneous impact of predicates on program termination status. To deal with mentioned problems, a new fault-proneness-aware approach based on elastic net regression, namely FPA-Debug has been proposed in this paper. FPA-Debug employs a clustering-based strategy to alleviate coincidental correctness in fault localization and finds the smallest effective subset of program predicates known as bug predictors. Moreover, the approach considers fault-pro...
Evaluating the Impact of Experimental Assumptions in Automated Fault Localization
2023
Much research on automated program debugging often assumes that bug fix location(s) indicate the faults' root causes and that root causes of faults lie within single code elements (statements). It is also often assumed that the number of statements a developer would need to inspect before finding the first faulty statement reflects debugging effort. Although intuitive, these three assumptions are typically used (55% of experiments in surveyed publications make at least one of these three assumptions) without any consideration of their effects on the debugger's effectiveness and potential impact on developers in practice. To deal with this issue, we perform controlled experimentation, split testing in particular, using 352 bugs from 46 open-source C programs, 19 Automated Fault Localization (AFL) techniques (18 statistical debugging formulas and dynamic slicing), two (2) stateof-the-art automated program repair (APR) techniques (GenProg and Angelix) and 76 professional developers. Our results show that these assumptions conceal the difficulty of debugging. They make AFL techniques appear to be (up to 38%) more effective, and make APR tools appear to be (2X) less effective. We also find that most developers (83%) consider these assumptions to be unsuitable for debuggers and, perhaps worse, that they may inhibit development productivity. The majority (66%) of developers prefer debugging diagnoses without these assumptions twice as much as with the assumptions. Our findings motivate the need to assess debuggers conservatively, i.e., without these assumptions.
How Developers Debug Software — The DBGBENCH Dataset
2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C), 2017
How do professional software engineers debug computer programs? In an experiment with 27 real bugs that existed in several widely used programs, we invited 12 professional software engineers, who together spent one month on localizing, explaining, and fixing these bugs. This did not only allow us to study the various tools and strategies used to debug the same set of errors. We could also determine exactly which statements a developer would localize as faults, how a developer would diagnose and explain an error, and how a developer would fix an errorall of which software engineering researchers seek to automate. Until now, it has been difficult to evaluate the effectiveness and utility of automated debugging techniques without a user study. We publish the collected data, called DBGBENCH, to facilitate the effective evaluation of automated fault localization, diagnosis, and repair techniques w.r.t. the judgement of human experts.
Statistical Debugging with Elastic Predicates
cs.virginia.edu
An important class of software, including simulations and computational models, employs stochastic distributions to represent, or support evaluation of, uncertainty in an underlying model. This class of software presents three interesting analysis challenges: 1) effective localization of the sources of unexpected outcomes; 2) effective treatment of stochastic distributions and the floating-point computations that generally accompany them and 3) separation of unexpected outcomes: disassociating valid, but unexpected, results from those reflecting software failure. Traditional debugging and fault localization methods [2-6, 11-15, 17-20, 23, 28, 30-32] have addressed primarily the first challenge, namely localization of sources of faults while assuming character, Boolean, integer and pointer operations. While these methods are effective in general, they are not tailored to software that uses stochastic distributions and floating-point computations. Thus there is opportunity. We introduce ESP, a novel approach to predicate-based statistical debugging. ESP localizes sources of unexpected outcomes in software using stochastic distributions and floating-point operations, thus addressing the first two aforementioned challenges. ESP predicates are elastic rather than uniform and static; each predicate adapts to variable values observed at its program instrumentation point. We present experimental results for established fault localization benchmarks and widely used simulations. For benchmarks employing stochastic distributions, ESP outperforms the best alternatives: Interesting Value Map Pair (IVMP), Cooperative Bug Isolation (CBI) and Tarantula. For traditional benchmarks, ESP performs similar to IVMP and outperforms CBI and Tarantula.
Loading Preview
Sorry, preview is currently unavailable. You can download the paper by clicking the button above.
Related papers
Proceedings of the 32nd Annual International Computer Software and Applications Conference (COMPSAC '08), IEEE Computer Society, Los Alamitos, CA, USA, pp. 128-135, 2008
Proceedings of the 8th International Conference on Quality Software (QSIC '08), IEEE Computer Society, Los Alamitos, CA, USA, pp. 385-395, 2008