Wasif Afzal | Blekinge Institute of Technology (original) (raw)

Papers by Wasif Afzal

Research paper thumbnail of RESAMPLING METHODS IN SOFTWARE QUALITY CLASSIFICATION

In the presence of a number of algorithms for classification and prediction in software engineeri... more In the presence of a number of algorithms for classification and prediction in software engineering, there is a need to have a systematic way of assessing their performances. The performance assessment is typically done by some form of partitioning or resampling of the original data to alleviate biased estimation. For predictive and classification studies in software engineering, there is a lack of a definitive advice on the most appropriate resampling method to use.

Research paper thumbnail of SEARCH-BASED PREDICTION OF SOFTWARE QUALITY

ABSTRACT Software verification and validation (V&V) activities are critical for achieving softwar... more ABSTRACT Software verification and validation (V&V) activities are critical for achieving software quality; however, these activities also constitute a large part of the costs when developing software. Therefore efficient and effective software V&V activities are both a priority and a necessity considering the pressure to decrease time-to-market and the intense competition faced by many, if not all, companies today.

Research paper thumbnail of On the application of genetic programming for software engineering predictive

a b s t r a c t 18 The objective of this paper is to investigate the evidence for symbolic regres... more a b s t r a c t 18 The objective of this paper is to investigate the evidence for symbolic regression using genetic program-19 ming (GP) being an effective method for prediction and estimation in software engineering, when com-20 pared with regression/machine learning models and other comparison groups (including comparisons 21 with different improvements over the standard GP algorithm). We performed a systematic review of lit-22 erature that compared genetic programming models with comparative techniques based on different 23 independent project variables. A total of 23 primary studies were obtained after searching different infor-24 mation sources in the time span 1995-2008. The results of the review show that symbolic regression 25 using genetic programming has been applied in three domains within software engineering predictive 26 modeling: (i) Software quality classification (eight primary studies). (ii) Software cost/effort/size estima-27 tion (seven primary studies). (iii) Software fault prediction/software reliability growth modeling (eight 28 primary studies). While there is evidence in support of using genetic programming for software quality 29 classification, software fault prediction and software reliability growth modeling; the results are incon-30 clusive for software cost/effort/size estimation.

Research paper thumbnail of Lessons from applying experimentation in software engineering prediction systems

Asia-Pacific Software Engineering Conference- …

Within software engineering prediction systems, experiments are undertaken primarily to investiga... more Within software engineering prediction systems, experiments are undertaken primarily to investigate relationships and to measure/compare models' accuracy. This paper discusses our experience and presents useful lessons/guidelines in experimenting with software engineering prediction systems. For this purpose, we use a typical software engineering experimentation process as a baseline. We found that the typical experimentation process in software engineering is supportive in developing prediction systems and have highlighted issues more central to the domain of software engineering prediction systems.

Research paper thumbnail of A Systematic Review of Non-Functional Search-Based Software Testing

... A Systematic Review of Non-Functional Search-Based Software Testing (2008). Download: http://... more ... A Systematic Review of Non-Functional Search-Based Software Testing (2008). Download: http://drfeldt.googlepages.com/afzal_submitted0805 CACHED: Download as a PDF. by Wasif Afzal , Richard Torkar , Robert Feldt. Add ...

Research paper thumbnail of Search-based resource scheduling for bug fixing tasks

2nd International Symposium on Search Based …, Jan 1, 2010

The software testing phase usually results in a large number of bugs to be fixed. The fixing of t... more The software testing phase usually results in a large number of bugs to be fixed. The fixing of these bugs require executing certain activities (potentially concurrent) that demand resources having different competencies and workloads. Appropriate resource allocation to these bug-fixing activities can help a project manager to schedule capable resources to these activities, taking into account their availability and skill requirements for fixing different bugs. This paper presents a multi-objective search-based resource scheduling method for bug-fixing tasks. The inputs to our proposed method include i) a bug model, ii) a human resource model, iii) a capability matching method between bug-fixing activities and human resources and iv) objectives of bug-fixing. A genetic algorithm (GA) is used as a search algorithm and the output is a bug-fixing schedule, satisfying different constraints and value objectives. We have evaluated our proposed scheduling method on an industrial data set and have discussed three different scenarios. The results indicate that GA is able to effectively schedule resources by balancing different objectives. We have also compared the effectiveness of using GA with a simple hillclimbing algorithm. The comparison shows that GA is able to achieve statistically better fitness values than hill-climbing.

Research paper thumbnail of Genetic programming versus other techniques for software engineering predictive modeling: A systematic review

Research paper thumbnail of Research Track Secondary Reviewers

ieeexplore.ieee.org

Wasif Afzal Prasanth Anbalagan Dejan Baca Christian Bird Peter Bokor Olof Bridal Erwan Brottier J... more Wasif Afzal Prasanth Anbalagan Dejan Baca Christian Bird Peter Bokor Olof Bridal Erwan Brottier Jeremy Bruce Brett Daniel Romain Delamare Hongbo Deng Cecilia Ekelin Marwa El-Ghali Anuraj Ennai Robert Feldt Harald Gall Michael Gegick László Gönczy Madhu Gopinathan Bhargav Gulavani Sarah Heckman Martin Hiller Chih-Wei Ho Chris Hobbs Kaizhu Huang Vilas Jagannath Sang-Uk Jeon Sheetal Kalyani Sarfraz Khurshid Mitsuhiro Kimura Zhifeng Lai Steven Lauterburg Yun Young Lee Otávio Lemos Henrik Lönn Hao Ma István Majzik Cesar Marcondes

Research paper thumbnail of Prediction of Faults-Slip-Through in Large Software Projects: An Empirical Evaluation

Research paper thumbnail of Proposal for master thesis in software engineering

Software engineering course credits completed at BTH (total): 22.5 ECTS credit points + 30 points... more Software engineering course credits completed at BTH (total): 22.5 ECTS credit points + 30 points in E-Government (highly Software Engineering related)

Research paper thumbnail of Search-based approaches to software fault prediction and software testing

Software verification and validation activities are essential for software quality but also const... more Software verification and validation activities are essential for software quality but also constitute a large part of software development costs. Therefore efficient and costeffective software verification and validation activities are both a priority and a necessity considering the pressure to decrease time-to-market and intense competition faced by many, if not all, companies today. It is then perhaps not unexpected that decisions related to software quality, when to stop testing, testing schedule and testing resource allocation needs to be as accurate as possible.

Research paper thumbnail of Evolution of Software Reliability Growth Model Using Genetic Programming

Building reliability growth models to predict software reliability and identify and remove errors... more Building reliability growth models to predict software reliability and identify and remove errors is both a necessity and a challenge for software testing engineers and project managers. Being able to predict the number of faults in software helps significantly in determining the software release date and in effectively managing project resources. Most of the growth models consider two or three parameters to estimate the accumulated faults in the testing process. Interest in using evolutionary computation to solve prediction and modeling problems has grown in recent years. In this paper, we explore the use of genetic programming (GP) as a tool to help in building growth models that can accurately predict the number of faults in software early on in the testing process. The proposed GP model is based on a recursive relation derived from the history of measured faults. The developed model is tested on real-time control, military, and operating system applications. The dataset was developed by John Musa of Bell Telephone Laboratories. The results of a comparison of the GP model with the traditional and simpler auto-regression model are presented.

Research paper thumbnail of Search-based Software Verification and Validation

Research paper thumbnail of Search-based prediction of fault count data

Search Based Software …, Jan 1, 2009

Symbolic regression, an application domain of genetic programming (GP), aims to find a function w... more Symbolic regression, an application domain of genetic programming (GP), aims to find a function whose output has some desired property, like matching target values of a particular data set. While typical regression involves finding the coefficients of a pre-defined function, symbolic regression finds a general function, with coefficients, fitting the given set of data points. The concepts of symbolic regression using genetic programming can be used to evolve a model for fault count predictions. Such a model has the advantages that the evolution is not dependent on a particular structure of the model and is also independent of any assumptions, which are common in traditional time-domain parametric software reliability growth models. This research aims at applying experiments targeting fault predictions using genetic programming and comparing the results with traditional approaches to compare efficiency gains.

Research paper thumbnail of Search-based prediction of fault-slip-through in large software projects

Search Based Software …

A large percentage of the cost of rework can be avoided by finding more faults earlier in a softw... more A large percentage of the cost of rework can be avoided by finding more faults earlier in a software testing process. Therefore, determination of which software testing phases to focus improvements work on, has considerable industrial interest. This paper evaluates the use of five different techniques, namely particle swarm optimization based artificial neural networks (PSO-ANN), artificial immune recognition systems (AIRS), gene expression programming (GEP), genetic programming (GP) and multiple regression (MR), for predicting the number of faults slipping through unit, function, integration and system testing phases. The objective is to quantify improvement potential in different testing phases by striving towards finding the right faults in the right phase. We have conducted an empirical study of two large projects from a telecommunication company developing mobile platforms and wireless semiconductors. The results are compared using simple residuals, goodness of fit and absolute relative error measures. They indicate that the four search-based techniques (PSO-ANN, AIRS, GEP, GP) perform better than multiple regression for predicting the fault-slip-through for each of the four testing phases. At the unit and function testing phases, AIRS and PSO-ANN performed better while GP performed better at integration and system testing phases. The study concludes that a variety of search-based techniques are applicable for predicting the improvement potential in different testing phases with GP showing more consistent performance across two of the four test phases.

Research paper thumbnail of Using faults-slip-through metric as a predictor of fault-proneness

2010 Asia Pacific Software Engineering Conference, Jan 1, 2010

Research paper thumbnail of On the application of genetic programming for software engineering predictive modeling: A systematic review

Expert Systems with Applications, Jan 1, 2011

a b s t r a c t 18 The objective of this paper is to investigate the evidence for symbolic regres... more a b s t r a c t 18 The objective of this paper is to investigate the evidence for symbolic regression using genetic program-19 ming (GP) being an effective method for prediction and estimation in software engineering, when com-20 pared with regression/machine learning models and other comparison groups (including comparisons 21 with different improvements over the standard GP algorithm). We performed a systematic review of lit-22 erature that compared genetic programming models with comparative techniques based on different 23 independent project variables. A total of 23 primary studies were obtained after searching different infor-24 mation sources in the time span 1995-2008. The results of the review show that symbolic regression 25 using genetic programming has been applied in three domains within software engineering predictive 26 modeling: (i) Software quality classification (eight primary studies). (ii) Software cost/effort/size estima-27 tion (seven primary studies). (iii) Software fault prediction/software reliability growth modeling (eight 28 primary studies). While there is evidence in support of using genetic programming for software quality 29 classification, software fault prediction and software reliability growth modeling; the results are incon-30 clusive for software cost/effort/size estimation.

Research paper thumbnail of Predicting software test effort in iterative development using a dynamic Bayesian network

21st IEEE International Symposium …, Jan 1, 2010

Projects following iterative software development methodologies must still be managed in a way as... more Projects following iterative software development methodologies must still be managed in a way as to maximize quality and minimize costs. However, there are indications that predicting test effort in iterative development is challenging and currently there seem to be no models for test effort prediction.

Research paper thumbnail of Metrics in software test planning and test design processes

Software metrics plays an important role in measuring attributes that are critical to the success... more Software metrics plays an important role in measuring attributes that are critical to the success of a software project. Measurement of these attributes helps to make the characteristics and relationships between the attributes clearer. This in turn supports informed decision making.

Research paper thumbnail of Genetic programming for cross-release fault count predictions in large and complex software projects

Evolutionary Computation and …, Jan 1, 2010

Software fault prediction can play an important role in ensuring software quality through efficie... more Software fault prediction can play an important role in ensuring software quality through efficient resource allocation. This could, in turn, reduce the potentially high consequential costs due to faults. Predicting faults might be even more important with the emergence of short-timed and multiple software releases aimed at quick delivery of functionality. Previous research in software fault prediction has indicated that there is a need i) to improve the validity of results by having comparisons among number of data sets from a variety of software, ii) to use appropriate model evaluation measures and iii) to use statistical testing procedures. Moreover, cross-release prediction of faults has not yet achieved sufficient attention in the literature. In an attempt to address these concerns, this paper compares the quantitative and qualitative attributes of 7 traditional and machine-learning techniques for modeling the cross-release prediction of fault count data. The comparison is done using extensive data sets gathered from a total of 7 multirelease open-source and industrial software projects. These software projects together have several years of development and are from diverse application areas, ranging from a web browser to a robotic controller software. Our quantitative analysis suggests that genetic programming (GP) tends to have better consistency in terms of goodness of fit and accuracy across majority of data sets. It also has comparatively less model bias. Qualitatively, ease of configuration and complexity are less strong points for GP even though it shows generality and gives transparent models. Artificial neural networks did not perform as well as expected while linear regression gave average predictions in terms of goodness of fit and accuracy. Support vector machine regression and traditional software reliability growth models performed below average on most of the quantitative evaluation criteria while remained on average for most of the qualitative measures.

Research paper thumbnail of RESAMPLING METHODS IN SOFTWARE QUALITY CLASSIFICATION

In the presence of a number of algorithms for classification and prediction in software engineeri... more In the presence of a number of algorithms for classification and prediction in software engineering, there is a need to have a systematic way of assessing their performances. The performance assessment is typically done by some form of partitioning or resampling of the original data to alleviate biased estimation. For predictive and classification studies in software engineering, there is a lack of a definitive advice on the most appropriate resampling method to use.

Research paper thumbnail of SEARCH-BASED PREDICTION OF SOFTWARE QUALITY

ABSTRACT Software verification and validation (V&V) activities are critical for achieving softwar... more ABSTRACT Software verification and validation (V&V) activities are critical for achieving software quality; however, these activities also constitute a large part of the costs when developing software. Therefore efficient and effective software V&V activities are both a priority and a necessity considering the pressure to decrease time-to-market and the intense competition faced by many, if not all, companies today.

Research paper thumbnail of On the application of genetic programming for software engineering predictive

a b s t r a c t 18 The objective of this paper is to investigate the evidence for symbolic regres... more a b s t r a c t 18 The objective of this paper is to investigate the evidence for symbolic regression using genetic program-19 ming (GP) being an effective method for prediction and estimation in software engineering, when com-20 pared with regression/machine learning models and other comparison groups (including comparisons 21 with different improvements over the standard GP algorithm). We performed a systematic review of lit-22 erature that compared genetic programming models with comparative techniques based on different 23 independent project variables. A total of 23 primary studies were obtained after searching different infor-24 mation sources in the time span 1995-2008. The results of the review show that symbolic regression 25 using genetic programming has been applied in three domains within software engineering predictive 26 modeling: (i) Software quality classification (eight primary studies). (ii) Software cost/effort/size estima-27 tion (seven primary studies). (iii) Software fault prediction/software reliability growth modeling (eight 28 primary studies). While there is evidence in support of using genetic programming for software quality 29 classification, software fault prediction and software reliability growth modeling; the results are incon-30 clusive for software cost/effort/size estimation.

Research paper thumbnail of Lessons from applying experimentation in software engineering prediction systems

Asia-Pacific Software Engineering Conference- …

Within software engineering prediction systems, experiments are undertaken primarily to investiga... more Within software engineering prediction systems, experiments are undertaken primarily to investigate relationships and to measure/compare models' accuracy. This paper discusses our experience and presents useful lessons/guidelines in experimenting with software engineering prediction systems. For this purpose, we use a typical software engineering experimentation process as a baseline. We found that the typical experimentation process in software engineering is supportive in developing prediction systems and have highlighted issues more central to the domain of software engineering prediction systems.

Research paper thumbnail of A Systematic Review of Non-Functional Search-Based Software Testing

... A Systematic Review of Non-Functional Search-Based Software Testing (2008). Download: http://... more ... A Systematic Review of Non-Functional Search-Based Software Testing (2008). Download: http://drfeldt.googlepages.com/afzal_submitted0805 CACHED: Download as a PDF. by Wasif Afzal , Richard Torkar , Robert Feldt. Add ...

Research paper thumbnail of Search-based resource scheduling for bug fixing tasks

2nd International Symposium on Search Based …, Jan 1, 2010

The software testing phase usually results in a large number of bugs to be fixed. The fixing of t... more The software testing phase usually results in a large number of bugs to be fixed. The fixing of these bugs require executing certain activities (potentially concurrent) that demand resources having different competencies and workloads. Appropriate resource allocation to these bug-fixing activities can help a project manager to schedule capable resources to these activities, taking into account their availability and skill requirements for fixing different bugs. This paper presents a multi-objective search-based resource scheduling method for bug-fixing tasks. The inputs to our proposed method include i) a bug model, ii) a human resource model, iii) a capability matching method between bug-fixing activities and human resources and iv) objectives of bug-fixing. A genetic algorithm (GA) is used as a search algorithm and the output is a bug-fixing schedule, satisfying different constraints and value objectives. We have evaluated our proposed scheduling method on an industrial data set and have discussed three different scenarios. The results indicate that GA is able to effectively schedule resources by balancing different objectives. We have also compared the effectiveness of using GA with a simple hillclimbing algorithm. The comparison shows that GA is able to achieve statistically better fitness values than hill-climbing.

Research paper thumbnail of Genetic programming versus other techniques for software engineering predictive modeling: A systematic review

Research paper thumbnail of Research Track Secondary Reviewers

ieeexplore.ieee.org

Wasif Afzal Prasanth Anbalagan Dejan Baca Christian Bird Peter Bokor Olof Bridal Erwan Brottier J... more Wasif Afzal Prasanth Anbalagan Dejan Baca Christian Bird Peter Bokor Olof Bridal Erwan Brottier Jeremy Bruce Brett Daniel Romain Delamare Hongbo Deng Cecilia Ekelin Marwa El-Ghali Anuraj Ennai Robert Feldt Harald Gall Michael Gegick László Gönczy Madhu Gopinathan Bhargav Gulavani Sarah Heckman Martin Hiller Chih-Wei Ho Chris Hobbs Kaizhu Huang Vilas Jagannath Sang-Uk Jeon Sheetal Kalyani Sarfraz Khurshid Mitsuhiro Kimura Zhifeng Lai Steven Lauterburg Yun Young Lee Otávio Lemos Henrik Lönn Hao Ma István Majzik Cesar Marcondes

Research paper thumbnail of Prediction of Faults-Slip-Through in Large Software Projects: An Empirical Evaluation

Research paper thumbnail of Proposal for master thesis in software engineering

Software engineering course credits completed at BTH (total): 22.5 ECTS credit points + 30 points... more Software engineering course credits completed at BTH (total): 22.5 ECTS credit points + 30 points in E-Government (highly Software Engineering related)

Research paper thumbnail of Search-based approaches to software fault prediction and software testing

Software verification and validation activities are essential for software quality but also const... more Software verification and validation activities are essential for software quality but also constitute a large part of software development costs. Therefore efficient and costeffective software verification and validation activities are both a priority and a necessity considering the pressure to decrease time-to-market and intense competition faced by many, if not all, companies today. It is then perhaps not unexpected that decisions related to software quality, when to stop testing, testing schedule and testing resource allocation needs to be as accurate as possible.

Research paper thumbnail of Evolution of Software Reliability Growth Model Using Genetic Programming

Building reliability growth models to predict software reliability and identify and remove errors... more Building reliability growth models to predict software reliability and identify and remove errors is both a necessity and a challenge for software testing engineers and project managers. Being able to predict the number of faults in software helps significantly in determining the software release date and in effectively managing project resources. Most of the growth models consider two or three parameters to estimate the accumulated faults in the testing process. Interest in using evolutionary computation to solve prediction and modeling problems has grown in recent years. In this paper, we explore the use of genetic programming (GP) as a tool to help in building growth models that can accurately predict the number of faults in software early on in the testing process. The proposed GP model is based on a recursive relation derived from the history of measured faults. The developed model is tested on real-time control, military, and operating system applications. The dataset was developed by John Musa of Bell Telephone Laboratories. The results of a comparison of the GP model with the traditional and simpler auto-regression model are presented.

Research paper thumbnail of Search-based Software Verification and Validation

Research paper thumbnail of Search-based prediction of fault count data

Search Based Software …, Jan 1, 2009

Symbolic regression, an application domain of genetic programming (GP), aims to find a function w... more Symbolic regression, an application domain of genetic programming (GP), aims to find a function whose output has some desired property, like matching target values of a particular data set. While typical regression involves finding the coefficients of a pre-defined function, symbolic regression finds a general function, with coefficients, fitting the given set of data points. The concepts of symbolic regression using genetic programming can be used to evolve a model for fault count predictions. Such a model has the advantages that the evolution is not dependent on a particular structure of the model and is also independent of any assumptions, which are common in traditional time-domain parametric software reliability growth models. This research aims at applying experiments targeting fault predictions using genetic programming and comparing the results with traditional approaches to compare efficiency gains.

Research paper thumbnail of Search-based prediction of fault-slip-through in large software projects

Search Based Software …

A large percentage of the cost of rework can be avoided by finding more faults earlier in a softw... more A large percentage of the cost of rework can be avoided by finding more faults earlier in a software testing process. Therefore, determination of which software testing phases to focus improvements work on, has considerable industrial interest. This paper evaluates the use of five different techniques, namely particle swarm optimization based artificial neural networks (PSO-ANN), artificial immune recognition systems (AIRS), gene expression programming (GEP), genetic programming (GP) and multiple regression (MR), for predicting the number of faults slipping through unit, function, integration and system testing phases. The objective is to quantify improvement potential in different testing phases by striving towards finding the right faults in the right phase. We have conducted an empirical study of two large projects from a telecommunication company developing mobile platforms and wireless semiconductors. The results are compared using simple residuals, goodness of fit and absolute relative error measures. They indicate that the four search-based techniques (PSO-ANN, AIRS, GEP, GP) perform better than multiple regression for predicting the fault-slip-through for each of the four testing phases. At the unit and function testing phases, AIRS and PSO-ANN performed better while GP performed better at integration and system testing phases. The study concludes that a variety of search-based techniques are applicable for predicting the improvement potential in different testing phases with GP showing more consistent performance across two of the four test phases.

Research paper thumbnail of Using faults-slip-through metric as a predictor of fault-proneness

2010 Asia Pacific Software Engineering Conference, Jan 1, 2010

Research paper thumbnail of On the application of genetic programming for software engineering predictive modeling: A systematic review

Expert Systems with Applications, Jan 1, 2011

a b s t r a c t 18 The objective of this paper is to investigate the evidence for symbolic regres... more a b s t r a c t 18 The objective of this paper is to investigate the evidence for symbolic regression using genetic program-19 ming (GP) being an effective method for prediction and estimation in software engineering, when com-20 pared with regression/machine learning models and other comparison groups (including comparisons 21 with different improvements over the standard GP algorithm). We performed a systematic review of lit-22 erature that compared genetic programming models with comparative techniques based on different 23 independent project variables. A total of 23 primary studies were obtained after searching different infor-24 mation sources in the time span 1995-2008. The results of the review show that symbolic regression 25 using genetic programming has been applied in three domains within software engineering predictive 26 modeling: (i) Software quality classification (eight primary studies). (ii) Software cost/effort/size estima-27 tion (seven primary studies). (iii) Software fault prediction/software reliability growth modeling (eight 28 primary studies). While there is evidence in support of using genetic programming for software quality 29 classification, software fault prediction and software reliability growth modeling; the results are incon-30 clusive for software cost/effort/size estimation.

Research paper thumbnail of Predicting software test effort in iterative development using a dynamic Bayesian network

21st IEEE International Symposium …, Jan 1, 2010

Projects following iterative software development methodologies must still be managed in a way as... more Projects following iterative software development methodologies must still be managed in a way as to maximize quality and minimize costs. However, there are indications that predicting test effort in iterative development is challenging and currently there seem to be no models for test effort prediction.

Research paper thumbnail of Metrics in software test planning and test design processes

Software metrics plays an important role in measuring attributes that are critical to the success... more Software metrics plays an important role in measuring attributes that are critical to the success of a software project. Measurement of these attributes helps to make the characteristics and relationships between the attributes clearer. This in turn supports informed decision making.

Research paper thumbnail of Genetic programming for cross-release fault count predictions in large and complex software projects

Evolutionary Computation and …, Jan 1, 2010

Software fault prediction can play an important role in ensuring software quality through efficie... more Software fault prediction can play an important role in ensuring software quality through efficient resource allocation. This could, in turn, reduce the potentially high consequential costs due to faults. Predicting faults might be even more important with the emergence of short-timed and multiple software releases aimed at quick delivery of functionality. Previous research in software fault prediction has indicated that there is a need i) to improve the validity of results by having comparisons among number of data sets from a variety of software, ii) to use appropriate model evaluation measures and iii) to use statistical testing procedures. Moreover, cross-release prediction of faults has not yet achieved sufficient attention in the literature. In an attempt to address these concerns, this paper compares the quantitative and qualitative attributes of 7 traditional and machine-learning techniques for modeling the cross-release prediction of fault count data. The comparison is done using extensive data sets gathered from a total of 7 multirelease open-source and industrial software projects. These software projects together have several years of development and are from diverse application areas, ranging from a web browser to a robotic controller software. Our quantitative analysis suggests that genetic programming (GP) tends to have better consistency in terms of goodness of fit and accuracy across majority of data sets. It also has comparatively less model bias. Qualitatively, ease of configuration and complexity are less strong points for GP even though it shows generality and gives transparent models. Artificial neural networks did not perform as well as expected while linear regression gave average predictions in terms of goodness of fit and accuracy. Support vector machine regression and traditional software reliability growth models performed below average on most of the quantitative evaluation criteria while remained on average for most of the qualitative measures.