Rafael Oliveira | Universidade de São Paulo (original) (raw)

Papers by Rafael Oliveira

Anais do III Workshop em Modelagem e Simulação de Sistemas Intensivos em Software (MSSiS 2021)

Software testing automation is one of the most challenging activities in Software Engineering sce... more Software testing automation is one of the most challenging activities in Software Engineering scenarios. Moden-Based Testing (MBT) is a feasible strategy to alleviate efforts on automating testing activities. Trough a model that specifies the behavior of the Software Under Testing (SUT), MBT approaches are useful strategies to generate test cases and run them. However, some domains such as, web applications require extra efforts on applying MBT approaches. Due to this, in this study we propose and validate Teasy a Domain Specification Language (DSL) that makes MBT feasible for web application. Through the conduction of a Proof-of-Concept on testing a real-world web application, we noticed Teasy has potential to evolve to effectively support software development environments. Using a real-world application and projects with manually seeded faults, Teasy testing scenarios have detected 78,57% of the functional inconsistencies.

This paper describes a novel application of Content-Based Image Retrieval (CBIR) to search a medi... more This paper describes a novel application of Content-Based Image Retrieval (CBIR) to search a medical database consisting of 3D models for diagnosis purposes. The 3D models, which are generated using Magnetic Resonance Imaging and include depth information, are used to search for similarity a database of 3D annotated medical cases using their pairwise feature similarity. The 3D models consist of both local and global feature descriptors that consider the surface of the 3D model and the overall geometry of the medical artifact. The models are then matched using spectral clustering that embeds the Euclidean distance for affinity and partitions the models into two groups, Congestive Heart Failure (CHF) and non-CHF. This suffices to demarcate using pairwise similarity the existence of CHF for the left ventricle. Experimental results using thirty 3D models show the utility of the new 3D method compared to existing methods. In particular, the novel method yields 83% overall accuracy.

In this paper we present the results of a two-phase empirical study where we evaluate and compare... more In this paper we present the results of a two-phase
empirical study where we evaluate and compare the applicability of automated component-based Graphical User Interface (GUI) testing and Visual GUI Testing (VGT) in the tools GUITAR and a prototype tool we refer to as VGT GUITAR. First, GUI mutation operators are defined to create 18 faulty versions
of an application on which both tools are then applied in
an experiment. Results from 456 test case executions in each
tool show, with statistical significance, that the component-based approach reports more false negatives than VGT for acceptance tests but that the VGT approach reports more false positives for system tests. Second, a case study is performed with larger open source applications, ranging from 8,803-55,006 lines of code. Results show that GUITAR is applicable in practice but has some challenges related to GUI component states. The results also show that VGT GUITAR is currently not applicable in practice and therefore requires further research and development. Based on the study’s results we present areas of future work for both test approaches and conclude that the approaches have different benefits and drawbacks. The component-based approach is robust and executes tests faster than the VGT approach, with a factor of 3. However, the VGT approach can perform visual assertions and is perceived more flexible than the component-based approach. These conclusions let us hypothesize that a combination of the two approaches is the most suitable in practice and therefore warrants future research.

Automated testing has become essential in software industry to meet market demands for faster del... more Automated testing has become essential in software
industry to meet market demands for faster delivery and higher
quality software. Testing is performed on many levels of system
abstraction, from tests on source code to Graphical User Interface
(GUI) tests. New testing techniques and frameworks are also
continuously released to the market.
Mutation analysis has been proposed as a way of assessing
the quality of these new test techniques/frameworks as well as
existing test suites in practice. The analysis is performed by
seeding defects, referred to as mutants, into the system under test
with the assumption that a technique/test suite of high quality
will “kill” the mutants. However, whilst support for mutation
analysis exists for test techniques that operate on on lower levels
of system abstraction, i.e. method-level mutation operators, the
support for GUI-level mutation analysis is currently lacking.
In this paper we perform an empirical analysis of 18 GUI-level
mutation operators defined in our previous work and compare
their efficiency and comprehensiveness to state-of-practice lower
level mutation operators. The main findings of our analysis are
(1) that traditional method-level mutation operators are not
precise enough for GUI-level mutation; (2) the defined GUI-based
mutation operators provide comprehensive support for
GUI-level mutation; and (3) GUI-based mutation operators can
be automated but are challenged by the dependencies between
GUI widgets.

Test oracle methods have changed signi ficantly over time, which has resulted in clear shifts in ... more Test oracle methods have changed signi ficantly over time, which has resulted in clear shifts in the research literature. Over the years, the testing techniques, strategies, and criteria utilized by researchers went through technical developments due to the
improvement of technologies and programming languages. Software testing designers, known as testers, currently have several resources to increase their con fidence in the software under test correctness. All of these Software Testing resources are supposed to include a mechanism to decide whether a particular execution is considered a failure or not. In Software Testing environments, this decision is the responsibility of the test oracle. Despite the evolution and adaptation of testing techniques over more than 30 years, test oracles remain a particular and relevant issue. In this chapter, using literary evidence from a pool of about 300 studies directly related to test oracles, we present a classifi cation of test oracles based on a taxonomy that considers their source of information and notations. Based on this classifi cation, we perform a quantitative analysis to highlight the shifts in (evolution of) research on test oracles. Exploring geographical and quantitative information, we analyzed the maturity of this field using co-authorship networks among studies published between 1978 and 2013. Further, we determine the most prolifi c authors and their countries, main conferences and journals, supporting tools, academic e fforts, and use a comparative analysis between academia and industry. Finally, from these analyzes we draw an analytic reflection about contemporary test oracle approaches and a criticism about oracle trends.

The 26th International Conference on Software Engineering and Knowledge Engineering, Jun 30, 2014

Test oracles evaluate the execution of SUTs (Systems Under Test) supporting testers to decide ab... more Test oracles evaluate the execution of SUTs (Systems
Under Test) supporting testers to decide about correct outputs and behaviors. “Non-testable” systems are cases in which the testers must spend extraordinary efforts to judge SUT’s outputs. Currently, some contemporary non-testable programs are represented by systems with complex outputs such as GUIs (Graphical User Interface), Web applications, and Text-to-speech (TTS) systems. Currently, there is a lack of knowledge associated with automated test oracles and SUTs with complex outputs. Extensible testing frameworks are necessary to provide the reuse of components and the sharing of knowledge in this field. This paper presents an extensible framework to support the development of test oracles for non-testable programs with complex outputs. In addition, we present an alternative to reuse software engineering components through plug-ins-based frameworks. The framework adapts CBIR (Content-Based Image Retrieval) concepts to enable testers to specify test oracles. The framework matches concepts of signal feature extraction, similarity functions, and object comparisons to obtain a Java program that compares two objects, responding how similar they are, according to a threshold. We performed proofs of concept using two empirical studies and the results showed our framework is useful to alleviate human-oracle efforts supporting human decisions. In addition, the plug-ins based framework we present is a contribution toward a reusing of components on test oracles for systems with complex outputs.

In Software Testing, systems with complex outputs (GUIs, web applications, and Text-to-speech (TT... more In Software Testing, systems with complex outputs (GUIs, web applications, and Text-to-speech (TTS) systems) make the test automation a difficult job and may require from the tester a large amount of time to determine whether the current output is correct. Even so, there is still no known general method to define test oracles for such systems. Regarding TTS systems, in which audio files are given as output, the literature describes several techniques useful for measure their quality, but they mostly involve opinion scores, informal interpretations and human intervention. This Ph.D. research aim to use test oracles to deal with this problem, extending the framework O-FIm that uses CBIR concepts to automate test oracles. Using feature extractors implemented specifically for systems with complex outputs (image and audio), this research provides an effective automated test oracle technique, supporting the quality assessment of such systems. We expect this new approach to reduce the human efforts of testing systems with complex outputs.

This research aims to explore CBAR concepts to implement test oracles to support testing activiti... more This research aims to explore CBAR concepts to implement test oracles to support testing activities of TTS Systems, helping the human in quality evaluations. In an automated software testing environment, Test Oracles represent the mechanism to evaluate whether an SUT (Software Under Testing) execution is correct or not.

Over the past 25 years the Brazilian Symposium on Software Engineering (SBES) has evolved to beco... more Over the past 25 years the Brazilian Symposium on Software Engineering (SBES) has evolved to become the most important event on software engineering in Brazil. Throughout these years, SBES has gathered a large body of studies in software testing. Aimed at providing an insightful understanding of what has already been published in such event, we have synthesized its 25-year history of research on software testing. Using information drawn from this overview we highlighted which software testing topics have been the most extensively surveyed in SBES literature. We have also devised a co-authorship network to depict the most prolific research groups and researchers. Moreover, by performing a citation analysis of the selected studies we have tried to ascertain the importance of SBES in a wider scenario. Finally, using the information extracted from the studies, we have shed light on the state-of-the-art of software testing in Brazil and provided an outlook on its foreseeable future.

Oracles are mechanisms used to determine whether the behavior of an application under testing is ... more Oracles are mechanisms used to determine whether the behavior of an application under testing is correct or incorrect. This work presents a prototype of a tool, the O-FIm, which uses the concept of CBIR to configure an environment to support testing of programs with graphic processing through automating oracle-related mechanisms. In early approaches, the tool has been enhanced to support the test of some specific subdomains of CAD systems.

Automation of the testing activity is an essential requirement to render it viable in the softwar... more Automation of the testing activity is an essential requirement to render it viable in the software development process. A complete automation depends on a solution for the oracle mechanism, used to determine whether the behavior of an execution of a system under testing (SUT) is correct or not. Consequently, as the complexity of the outputs increases, more efforts are required in the automation of the oracle. This paper explores the OFIm (Oracle For Images) framework, that uses CBIR (Content-Based Image Retrieval) concepts to configure an environment that supports testing programs with complex output formats, making possible to devise testing oracle that supports systems with graphical user interface (GUI). We provide results of experiments which indicate that the framework can be useful to support the software testing of systems with GUIs.

Over the past 25 years the Brazilian Symposium on Software Engineering (SBES) has evolved to beco... more Over the past 25 years the Brazilian Symposium on Software Engineering (SBES) has evolved to become the most important event on software engineering in Brazil. Throughout these years, SBES has gathered a large body of studies in software testing. Aimed at providing an insightful understanding of what has already been published in such event, we synthesized its rich 25-year history of research on software testing. Using information drawn from this overview we attempted to highlight which types of study have been the most applied for conveying software testing efforts. We also devised a co-authorship network to obtain a bird's-eye view of which research groups and scholars have been the most prolific ones. Moreover, by performing a citation analysis of the selected studies we set out to ascertain the importance of SBES in a wider scenario. Finally, borne out by the information extracted from the studies, we shed some light on the state-of-the-art of software testing in Brazil and provide an outlook on its foreseeable future.

Automation of testing is an essential requirement to render it viable for software development. A... more Automation of testing is an essential requirement to render it viable for software development. Although there are several testing techniques and criteria in many different domains, developing methods to test programs with complex outputs remains an unsolved challenge. This setting includes programs with graphical output, which produce images or interface windows. One possible approach towards automating the testing activity is the use of automatic oracles in which a reference image, taken as correct, can be used to establish a correctness measure in the tested program execution. A method that uses concepts of content-based image retrieval to facilitate oracle automation in the domain of programs with graphics output is presented. Two case studies, one using a computer-aided diagnostic system and one using a Web application, are presented, including some reflections and discussions that demonstrate the feasibility of the proposed approach. Copyright © 2011 John Wiley & Sons, Ltd.

Oracles are a challenge for the automation of software testing. To define an oracle means setting... more Oracles are a challenge for the automation of software testing. To define an oracle means setting a structure capable of providing a verdict on the correctness of an execution of the program under test. Automating test mechanisms is not a trivial work. A negligence may lead to the production of poor quality software. In this paper we propose an approach that uses CBIR concepts, which is the application of image processing to recover digital images in databases, to set up a supportive environment for defining testing oracles for programs with a graphical interface.

Anais do III Workshop em Modelagem e Simulação de Sistemas Intensivos em Software (MSSiS 2021)

The 26th International Conference on Software Engineering and Knowledge Engineering, Jun 30, 2014

Over the past 25 years the Brazilian Symposium on Software Engineering (SBES) has evolved to beco... more Over the past 25 years the Brazilian Symposium on Software Engineering (SBES) has evolved to become the most important event on software engineering in Brazil. Throughout these years, SBES has gathered a large body of studies in software testing. Aimed at providing an insightful understanding of what has already been published in such event, we synthesized its rich 25-year history of research on software testing. Using information drawn from this overview we attempted to highlight which types of study have been the most applied for conveying software testing efforts. We also devised a co-authorship network to obtain a bird's-eye view of which research groups and scholars have been the most prolific ones. Moreover, by performing a citation analysis of the selected studies we set out to ascertain the importance of SBES in a wider scenario. Finally, borne out by the information extracted from the studies, we shed some light on the state-of-the-art of software testing in Brazil and provide an outlook on its foreseeable future.