www.isr.uci.edu/tech-reports.html Scenario-based and State Machine-based Testing: An Evaluation of Automated Approaches (original) (raw)
Related papers
Software testing is as old as the hills in the history of digital computers. The testing of software is an important means of assessing the software to determine its quality. Since testing typically consumes 40~50% of development efforts, and consumes more effort for systems that require higher levels of reliability, it is a significant part of the software engineering. With the development of Fourth generation languages (4GL), which speeds up the implementation process, the proportion of time devoted to testing increased. As the amount of maintenance and upgrade of existing systems grow, significant amount of testing will also be needed to verify systems after changes are made [12]. Despite advances in formal methods and verification techniques, a system still needs to be tested before it is used. Testing remains the truly effective means to assure the quality of a software system of non-trivial complexity [13], as well as one of the most intricate and least understood areas in software engineering [19]. Testing, an important research area within computer science is likely to become even more important in the future.
Model of Execution based Software Testing
When a program is implemented to provide a concrete representation of an algorithm, the developers of this program are naturally concerned with the correctness and performance of the implementation. Software engineers must ensure that their software systems achieve an appropriate level of quality. Software verification is the process of ensuring that a program meets its intended specification [Kaner et al., 1993]. One technique that can assist during the specification, design, and implementation of a software system is software verification through correctness proof. Software testing, or the process of assessing the functionality and correctness of a program through execution or analysis, is another alternative for verifying a software system. As noted by Bowen, Hinchley, and Geller, software testing can be appropriately used in conjunction with correctness proofs and other types of formal approaches in order to develop high quality software systems [Bowen and Hinchley, 1995, Geller, 1978]. Yet, it is also possible to use software testing techniques in isolation from program correctness proofs or other formal methods.
Special issue on automation of software testing: improving practical applicability
Software Quality Journal, 2018
An important challenge in software testing has been to improve the degree of attainable automation, in developing advanced techniques for generating the test inputs as well as in finding innovative support procedures to automate the testing process. Test automation is a very active field of research, covering different aspects such as tools, application domains, the lifecycle development process, project dimensions, and empirical work reporting successful results or failures (Graham and Fewster 2012). Test automation is also highly relevant in practice: the market for automated test support tools is expanding, opening relevant business opportunities for new innovative testing platforms. The ultimate challenge is the development of a powerful integrated test environment that goes beyond automated test execution and by itself can automatically take care Software Qual J
A Review Paper on Software Testing
2021
Testing is a task, which is performed to check the quality of the software and also this process is done for the improvement in software at the same time. Software testing is just testing software for the purpose of checking whether the behavior of the software is correctly working or not according to the user expectations. For knowing the full details of any software development there is a concept of software development life cycle (SDLC) which describes all the stages of the software development. There are various phases i.e., Analysis (ideation), Requirements (Documentation), Design, Development (implementation), Testing, Deployment (release), maintenance which is used to actual development of software according to the users requirement. The main goal of the software development life cycle is to provide error free software and fulfill the user’s expectations within the time limit. Testing of any software is basically a process which is used to assist in recognizing the completene...
Software Test Automation in Practice
Challenges and Improvements, 2011
The objective of this industry study was to shed light on the current situation and improvement needs in software test automation. To this end, 55 industry specialists from 31 organizational units were interviewed. In parallel with the survey, a qualitative study was conducted in 12 selected software development organizations. The results indicated that the software testing processes usually follow systematic methods to a large degree, and have only little immediate or critical requirements for resources. Based on the results, the testing processes have approximately three fourths of the resources they need, and have access to a limited, but usually sufficient group of testing tools. As for the test automation, the situation is not as straightforward: based on our study, the applicability of test automation is still limited and its adaptation to testing contains practical difficulties in usability. In this study, we analyze and discuss these limitations and difficulties.
Advances in Software Engineering, 2010
Test redundancy detection reduces test maintenance costs and also ensures the integrity of test suites. One of the most widely used approaches for this purpose is based on coverage information. In a recent work, we have shown that although this information can be useful in detecting redundant tests, it may suffer from large number of false-positive errors, that is, a test case being identified as redundant while it is really not. In this paper, we propose a semiautomated methodology to derive a reduced test suite from a given test suite, while keeping the fault detection effectiveness unchanged. To evaluate the methodology, we apply the mutation analysis technique to measure the fault detection effectiveness of the reduced test suite of a real Java project. The results confirm that the proposed manual interactive inspection process leads to a reduced test suite with the same fault detection ability as the original test suite. 4 Advances in Software Engineering public void testAlleleOrderDoesntMatter () { Genotype g1 = new Genotype(new double [ ] {0,1}); Genotype g2 = new Genotype(new double [ ] [13]); assertTrue (g1.getAdjustedAlleleValues (2). equals(g2.getAdjustedAlleleValues (2))); } public void testOffset (){ Genotype g = new Genotype(new double [ ]{0,1}); g.offsetBy (0.5); List adjusted = g.getAdjustedAlleleValues ; assertEquals (2, adjusted.size ()); assertEquals (0.5, adjusted.get (0)); assertEquals (1.5, adjusted.get (1)); g.clearOffset(); adjusted = g.getAdjustedAlleleValues (2); assertEquals (0.0, adjusted.get (0)); assertEquals (1.0, adjusted.get (1));
Technology for Testing of Software Modules Based on Use Cases
Odes’kyi Politechnichnyi Universytet Pratsi, 2017
тестування програмних модулів на основі варіантів використання. Розроблено автоматизовану технологію, що поєднує процеси опису варіанта використання (прецеденту) і складання наборів тест-кейсів. Для цього запропонована математична модель прецеденту, що представляє його у вигляді орієнтованого графа. Кожна вершина графа відповідає пункту прецеденту, а кожна дуга визначає умови переходу і дані, що визначають ці умови. Застосована класифікація пунктів сценаріїв прецеденту, що дозволила виділити 7 типів пунктів. Для кожного типу пункту прецеденту розроблен окремий шаблон тест-кейса. Шаблони мають 3 розділа: дані, що вводяться в даному пункті, дані що раніше надійшли в систему, та результати виконання пункту. Розроблен алгоритм виявлення незалежних шляхів і процедура визначення завершення обходу. Розроблені програмні засоби, що підтримують запропоновану технологію складання тест-кейсів. Проведені випробування показали істотне скорочення часу при використанні даної технології в порівнянні з існуючими рішеннями, які передбачають роздільні процеси опису прецедентів і складання тест-кейсів. Ключові слова: варіанти використання, математична модель, тестування, шаблон тест-кейса O.B. Kungurtsev, Nguyen Tran Quoc Vinh, N.O. Novikova. Technology for testing of software modules based on use cases. An automated technology is developed that combines the processes of describing of the use case (precedent) and compiling of test cases sets. For this purpose, a mathematical model of the precedent is proposed, representing it in the form of an oriented graph. Each vertex of the graph corresponds to a precedent item, and each edge defines the transition conditions and data that define these conditions. The classification of the test case scenarios was used, which made it possible to distinguish 7 types of items. For each type of test case item, a separate test case template has been developed. Templates have 3 sections: data entered in this item; data previously received in the system; and the results of the implementation of the item. An algorithm for identifying independent paths and a procedure for determining the completion of traversal is developed. Software tools that support the proposed technology of drawing test cases have been developed. The tests showed a significant reduction in the time when this technology was used in comparison with existing solutions which provide the separate processes for describing of use cases and drawing up of test cases.
DEVELOPMENT OF AN AUTOMATED FRAMEWORK TO RESOLVE SOFTWARE TESTING ISSUES
TJPRC, 2014
The growing importance of Software in the present scenario can be attributed to the fact that software is utilized in several different ways for different issues to provide different types of solutions. The subject of Software Engineering is so vast and varied that it encompasses several areas such as manufacturing of Refrigerators, Air Conditioning Systems, Automobiles, Space Engineering, Wireless and Mobile Communications. Therefore, in most cases, software usually need to adhere to a specifications so that they can perform to produce only those results that are expected and enunciated. In general, keeping in view of the present scenario, a software professional goes through a certain process to establish that the software conforms to a given specification. This process, verification and validation (V & V), ensures that the software conforms to its specification and that the customers ultimately receive what they ordered. Software testing is one of the techniques to use during V & V. To be able to use resources in a better way, computers should be able to help out in the “art of software testing” to a higher extent, than is currently the case today. The main issue is to not retrench human resources from the process of software testing itself altogether but to consider the fact that software engineering is an art and science of identifying suitable solutions to critical problems by not just involving computers alone to solve the testing issues but software engineers should themselves participate and come out with better and suitable solutions to those problems where computers are clearly not that good to provide appropriate solutions. This research work presents a systematic and methodical approach aimed at examining, classifying and improving the concept of automated software testing and is built upon the assumption that software testing could be automated to a higher extent. Throughout this thesis an emphasis has been put on “real life” applications and the testing of these applications. One of the contributions in this thesis is the research aimed at uncovering different issues with respect to automated software testing. The research is performed through a series of case studies and experiments which ultimately also leads to another contribution—a model for expressing, clarifying and classifying software testing and the automated aspects thereof. An additional contribution in this thesis is the development of framework which in turns acts as a broad substratum for a framework for object message pattern analysis of intermediate code representations. This is achieved keeping in view of the procedures relating to code optimization and effective code generation at all levels. 260 Chittineni Aruna & R. Siva Ram Prasad Impact Factor (JCC): 6.8785 Index Copernicus Value (ICV): 3.0 The results that are expected in this thesis indicate how software testing can be improved, extended and better classified with respect to automation aspects. The main contribution lays in the investigations and the improvements related issues with regard to automated software testing. A comprehensive study has been made by the researchers to tackle the very crust of the problem of software testing methodologies. It has to be noted that testing software in an automated way has been a goal for researchers and industry during the last few decades. Nevertheless, the success rate has not been readily available. Some tools that are partly automated have evolved, while at the same time the methodologies for testing software has improved, thus leading to an overall improvement. Even today, much of the software testing procedures are performed manually which is not reliable, prone to errors, inefficient and also a costly affair. Even today several questions still remain unanswered. For instance, we have the question of when and how to stop the process of testing software when is it not economically feasible to continue to test software? It is very much evident that investing so much of time and money for testing an application which will be used only for a few times, in a non-critical environment, is probably a waste of resources.[1] At the same time, when software engineers develop software that will be placed in a critical domain and extensively used, an answer to that question needs to be found. Secondly, there is the problem of resources. It is to be noted that small- and medium-sized companies are today, as always, challenged by their resources, or to be more precise, the lack thereof. Deadlines must be kept at all costs even when, in some cases, the cost turns out to be the actual reliability of their products. In addition to the growing complexity of software, the size of the problem becomes more and more complex and huge. Keeping in view of the question of complexity and to be more precise based on the fact that software has grown in complexity and size which have become very much part of the problem of software testing and the type of research work that needs to be done to improvise on these issues is to be given a methodical and systematic approach.
Criteria for generating specification-based tests
Engineering of Complex Computer …, 1999
This paper presents general criteria for generating test inputs from state-based s p eci cations. Software testing can only be formalized and quanti ed w h e n a solid basis for test generation can be d e n e d. Formal speci cations of complex systems represent a signicant opportunity for testing because they precisely describe what functions the software i s s u p p osed t o p r ovide in a form that can easily be manipulated. These techniques provide coverage criteria that are b ased o n the speci cations, and are made up of several parts, including test pre xes that contain inputs necessary to put the software into the appropriate state for the test values. The test generation process includes several steps for transforming speci cations to tests. Empirical results from a comparative case study application of these criteria are p r esented.
IAEME, 2019
Programming testing is the stage which makes programming as usable quality scholarly amount. Programming testing under experiences distinctive stages. The accompanying stages according to the examination are investigation test, test arranging, experiment or test information or test condition creation, test execution, bugs logging, following and test strategy. Past research has been improved the situation advance test process in nature of programming. All accessible testing forms incorporate distinctive advancement models and diverse programming testing procedures are performed. Each organization chooses their testing procedure dependent on the basic condition of the applications each organization selects their testing procedure. The security, execution and utilitarian parts are most basic in every application these are altogether to be tried and carrying on obviously. This paper will clarify and guaranteeing about programming applications quality to do enhanced testing forms. The real programming testing systems are Security, Performance and Functional are handled by Analysis, Preparation and Execution will be finished up.