R. Gore - Academia.edu (original) (raw)

Papers by R. Gore

Stochastic simulations frequently exhibit behaviors that are difficult to recreate and analyze, o... more Stochastic simulations frequently exhibit behaviors that are difficult to recreate and analyze, owing largely to the stochastics themselves, and consequent program dependency chains that can defy human reasoning capabilities. We present a novel approach called Markov Chain Execution Traces (MCETs) for efficiently representing sampled stochastic simulation execution traces and ultimately driving semiautomated analysis methods that require accurate, efficiently generated candidate execution traces. The MCET approach is evaluated, using new and established measures, against both additional novel and existing approaches for computing dynamic program slices in stochastic simulations. 0&(7 ¶V VXSHULRU SHrformance is established. Finally, a description of how users can apply MCETs to their own stochastic simulations and a discussion of the new analyses MCETs can enable are presented.

2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011), 2011

Page 1. Statistical Debugging with Elastic Predicates Ross Gore, Paul F. Reynolds, Jr., David Kam... more Page 1. Statistical Debugging with Elastic Predicates Ross Gore, Paul F. Reynolds, Jr., David Kamensky {rjg7v, pfr, dmk3d}@virginia.edu University of Virginia* Tech Report CS-2011-02 Page 2. ABSTRACT An important class ...

Simulations and computational models have become the common tool of subject matter experts (SMEs)... more Simulations and computational models have become the common tool of subject matter experts (SMEs) in a variety of disciplines to explore systems with inherent uncertainty. Predictions from these models and simulations have entered the mainstream of critical public policy and research decision-making practices. Unfortunately, methods for gaining insight into unexpected simulation outcomes have not kept pace. SMEs need to understand and explain unexpected behaviors in exploratory simulations to determine if the behaviors reflect an error or if they represent new knowledge in the discipline. Common practice is to apply classic debugging techniques to identify the program statements and interactions that lead to the unexpected behaviors. This practice is largely manual, it can consume years of effort, and it will not scale as models increase in complexity. Automation of at least a portion of the process has become essential. The automated process proposed here, Bayesian Program Slicing (BPS), will combine program slicing and Bayesian networks in a novel manner to identify program statements that are relevant to understanding unexpected behaviors. BPS will facilitate focusing SME attention on understanding and explaining the interactions of program statements whose execution results in variable state changes that are most relevant to the unexpected behaviors. Issues that make the proposed approach research challenging include: identifying prior knowledge in exploratory simulation that can be employed by Bayesian networks, efficiently sampling variable states and dynamic program slices and identifying an approach to cluster similar dynamic program slices. Evaluation of BPS will employ established methods for evaluating emerging software tools and quantitative metrics. The effectiveness of BPS will be compared to that of established, leading tools. The thesis of the proposed work is that by these measures BPS will be deemed more effective for facilitating SME explanation and understanding of unexpected behaviors than existing tools and will be a useful contribution by bringing automation to a challenging task.

Abstract—Quantitative methods of analysis have progressed faster than quantitative methods of cap... more Abstract—Quantitative methods of analysis have progressed faster than quantitative methods of capturing, representing, propagating and analyzing uncertainty in the realm of computational thinking, adversely affecting the quality of both scientific computational ...

cs.virginia.edu

Page 1. Statistical Debugging with Elastic Predicates Ross Gore, Paul F. Reynolds, Jr., David Kam... more Page 1. Statistical Debugging with Elastic Predicates Ross Gore, Paul F. Reynolds, Jr., David Kamensky {rjg7v, pfr, dmk3d}@virginia.edu University of Virginia* Tech Report CS-2011-02 Page 2. ABSTRACT An important class ...

ABSTRACT Unexpected behaviours in simulations require explanation, so that decision-makers and su... more ABSTRACT Unexpected behaviours in simulations require explanation, so that decision-makers and subject matter experts can separate valid behaviours from design or coding errors. Validation of unexpected behaviours requires accumulation of insight into the behaviour and the conditions under which it arises. Agent-based simulations are known for unexpected behaviours that emerge as the simulation executes. To facilitate user exploration, analysis, understanding and insight of unexpected behaviours, we have developed a novel semi-automated methodology, INSIGHT. INSIGHT provides: (1) semi-automatic hypothesis testing for exploring an unexpected behaviour, and (2) automatic identification of statements in an agent-based simulation's source code which have the strongest influence on an unexpected behaviour. INSIGHT is applicable to both deterministic and stochastic agent-based simulations and extends the state of the art in agent-based simulation analysis.

Computational Science–ICCS …, 2007

We seek to increase user confidence in simulations as they are adapted to meet new requirements. ... more We seek to increase user confidence in simulations as they are adapted to meet new requirements. Our approach includes formal representation of uncertainty, lightweight validation, and novel techniques for exploring emergent behavior. Uncertainty representation, using formalisms such as Dempster-Shafer theory, can capture designer insight about uncertainty, enabling formal analysis and improving communication with decision and policy makers. Lightweight validation employs targeted program analysis and automated regression testing to maintain user confidence as adaptations occur. Emergent behavior validation exploits the semi-automatic adaptation capability of COERCE to make exploration of such behavior efficient and productive. We describe our research on these three technologies and their impact on validating dynamically evolving simulations.

We present a study of the dimensionality and factorial invariance of religiosity for 26 countries... more We present a study of the dimensionality and factorial invariance of religiosity for 26 countries with a Christian heritage, based on the 1998 and 2008 rounds of the International Social Survey Programme (ISSP) Religion survey, using both exploratory and multi-group confirmatory factor analyses. The results of the exploratory factor analysis showed that three factors, common to Christian and religiously unaffiliated respondents, could be extracted from our initially selected items and suggested the testing of four different three-factor models using multi-group confirmatory factor analysis. For the model with the best fit and measurement invari-ance properties, we labeled the three resulting factors as "Beliefs in afterlife and miracles", "Belief and importance of God" and "Religious involvement." The first factor is measured by four items related to the Supernatural Beliefs Scale (SBS-6); the second by three items related to belief in God and God's perceived roles as a supernatural agent; and the third one by three items with the same structure found in previous cross-cultural analyses of religiosity using the European Values Survey (ESS) and also by belief in God. Unexpectedly, we found that one item, belief in God, cross-loaded on to the second and third factors. We discussed possible interpretations for this finding, together with the potential limitations of the ISSP Religion questionnaire for revealing the structure of religiosity. Our tests of measurement invari-ance across gender, age, educational degree and religious (un)affiliation led to acceptance of the hypotheses of metric-and scalar-invariance for these groupings (units of analysis). However, in the measurement invariance tests across the countries, the criteria for metric invariance were met for twenty-three countries only, and partial scalar invariance was accepted for fourteen countries only. The present work shows that the exploration of large multinational and cross-cultural datasets for studying the dimensionality and invariance of social constructs (in our case, religiosity) yields useful results for cross-cultural comparisons, but is also limited by the structure of these datasets and the way specific items are coded.

Stochastic simulations frequently exhibit behaviors that are difficult to recreate and analyze, o... more Stochastic simulations frequently exhibit behaviors that are difficult to recreate and analyze, owing largely to the stochastics themselves, and consequent program dependency chains that can defy human reasoning capabilities. We present a novel approach called Markov Chain Execution Traces (MCETs) for efficiently representing sampled stochastic simulation execution traces and ultimately driving semiautomated analysis methods that require accurate, efficiently generated candidate execution traces. The MCET approach is evaluated, using new and established measures, against both additional novel and existing approaches for computing dynamic program slices in stochastic simulations. 0&(7 ¶V VXSHULRU SHrformance is established. Finally, a description of how users can apply MCETs to their own stochastic simulations and a discussion of the new analyses MCETs can enable are presented.

2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011), 2011

Page 1. Statistical Debugging with Elastic Predicates Ross Gore, Paul F. Reynolds, Jr., David Kam... more Page 1. Statistical Debugging with Elastic Predicates Ross Gore, Paul F. Reynolds, Jr., David Kamensky {rjg7v, pfr, dmk3d}@virginia.edu University of Virginia* Tech Report CS-2011-02 Page 2. ABSTRACT An important class ...

Simulations and computational models have become the common tool of subject matter experts (SMEs)... more Simulations and computational models have become the common tool of subject matter experts (SMEs) in a variety of disciplines to explore systems with inherent uncertainty. Predictions from these models and simulations have entered the mainstream of critical public policy and research decision-making practices. Unfortunately, methods for gaining insight into unexpected simulation outcomes have not kept pace. SMEs need to understand and explain unexpected behaviors in exploratory simulations to determine if the behaviors reflect an error or if they represent new knowledge in the discipline. Common practice is to apply classic debugging techniques to identify the program statements and interactions that lead to the unexpected behaviors. This practice is largely manual, it can consume years of effort, and it will not scale as models increase in complexity. Automation of at least a portion of the process has become essential. The automated process proposed here, Bayesian Program Slicing (BPS), will combine program slicing and Bayesian networks in a novel manner to identify program statements that are relevant to understanding unexpected behaviors. BPS will facilitate focusing SME attention on understanding and explaining the interactions of program statements whose execution results in variable state changes that are most relevant to the unexpected behaviors. Issues that make the proposed approach research challenging include: identifying prior knowledge in exploratory simulation that can be employed by Bayesian networks, efficiently sampling variable states and dynamic program slices and identifying an approach to cluster similar dynamic program slices. Evaluation of BPS will employ established methods for evaluating emerging software tools and quantitative metrics. The effectiveness of BPS will be compared to that of established, leading tools. The thesis of the proposed work is that by these measures BPS will be deemed more effective for facilitating SME explanation and understanding of unexpected behaviors than existing tools and will be a useful contribution by bringing automation to a challenging task.

Abstract—Quantitative methods of analysis have progressed faster than quantitative methods of cap... more Abstract—Quantitative methods of analysis have progressed faster than quantitative methods of capturing, representing, propagating and analyzing uncertainty in the realm of computational thinking, adversely affecting the quality of both scientific computational ...

cs.virginia.edu

Page 1. Statistical Debugging with Elastic Predicates Ross Gore, Paul F. Reynolds, Jr., David Kam... more Page 1. Statistical Debugging with Elastic Predicates Ross Gore, Paul F. Reynolds, Jr., David Kamensky {rjg7v, pfr, dmk3d}@virginia.edu University of Virginia* Tech Report CS-2011-02 Page 2. ABSTRACT An important class ...

ABSTRACT Unexpected behaviours in simulations require explanation, so that decision-makers and su... more ABSTRACT Unexpected behaviours in simulations require explanation, so that decision-makers and subject matter experts can separate valid behaviours from design or coding errors. Validation of unexpected behaviours requires accumulation of insight into the behaviour and the conditions under which it arises. Agent-based simulations are known for unexpected behaviours that emerge as the simulation executes. To facilitate user exploration, analysis, understanding and insight of unexpected behaviours, we have developed a novel semi-automated methodology, INSIGHT. INSIGHT provides: (1) semi-automatic hypothesis testing for exploring an unexpected behaviour, and (2) automatic identification of statements in an agent-based simulation's source code which have the strongest influence on an unexpected behaviour. INSIGHT is applicable to both deterministic and stochastic agent-based simulations and extends the state of the art in agent-based simulation analysis.

Computational Science–ICCS …, 2007

We seek to increase user confidence in simulations as they are adapted to meet new requirements. ... more We seek to increase user confidence in simulations as they are adapted to meet new requirements. Our approach includes formal representation of uncertainty, lightweight validation, and novel techniques for exploring emergent behavior. Uncertainty representation, using formalisms such as Dempster-Shafer theory, can capture designer insight about uncertainty, enabling formal analysis and improving communication with decision and policy makers. Lightweight validation employs targeted program analysis and automated regression testing to maintain user confidence as adaptations occur. Emergent behavior validation exploits the semi-automatic adaptation capability of COERCE to make exploration of such behavior efficient and productive. We describe our research on these three technologies and their impact on validating dynamically evolving simulations.

We present a study of the dimensionality and factorial invariance of religiosity for 26 countries... more We present a study of the dimensionality and factorial invariance of religiosity for 26 countries with a Christian heritage, based on the 1998 and 2008 rounds of the International Social Survey Programme (ISSP) Religion survey, using both exploratory and multi-group confirmatory factor analyses. The results of the exploratory factor analysis showed that three factors, common to Christian and religiously unaffiliated respondents, could be extracted from our initially selected items and suggested the testing of four different three-factor models using multi-group confirmatory factor analysis. For the model with the best fit and measurement invari-ance properties, we labeled the three resulting factors as "Beliefs in afterlife and miracles", "Belief and importance of God" and "Religious involvement." The first factor is measured by four items related to the Supernatural Beliefs Scale (SBS-6); the second by three items related to belief in God and God's perceived roles as a supernatural agent; and the third one by three items with the same structure found in previous cross-cultural analyses of religiosity using the European Values Survey (ESS) and also by belief in God. Unexpectedly, we found that one item, belief in God, cross-loaded on to the second and third factors. We discussed possible interpretations for this finding, together with the potential limitations of the ISSP Religion questionnaire for revealing the structure of religiosity. Our tests of measurement invari-ance across gender, age, educational degree and religious (un)affiliation led to acceptance of the hypotheses of metric-and scalar-invariance for these groupings (units of analysis). However, in the measurement invariance tests across the countries, the criteria for metric invariance were met for twenty-three countries only, and partial scalar invariance was accepted for fourteen countries only. The present work shows that the exploration of large multinational and cross-cultural datasets for studying the dimensionality and invariance of social constructs (in our case, religiosity) yields useful results for cross-cultural comparisons, but is also limited by the structure of these datasets and the way specific items are coded.