The dangers of performance-based research funding in non-competitive higher education systems (original) (raw)
Related papers
Journal of Informetrics, 2012
Higher education systems in competitive environments generally present top universities, that are able to attract top scientists, top students and public and private financing, with notable socioeconomic benefits in their region. The same does not hold true for noncompetitive systems. In this study we will measure the dispersion of research performance within and between universities in the Italian university system, typically non-competitive. We will also investigate the level of correlation that occurs between performance in research and its dispersion in universities. The findings may represent a first benchmark for similar studies in other nations. Furthermore, they lead to policy indications, questioning the effectiveness of selective funding of universities based on national research assessment exercises. The field of observation is composed of all Italian universities active in the hard sciences. Research performance will be evaluated using a bibliometric approach, through publications indexed in the Web of Science between 2004 and 2008.
Journal of Informetrics, 2016
The ability to attract and retain talented professors is a distinctive competence of world-class universities and a source of competitive advantage. The ratio of top scientists to academic staff could therefore be an indicator of the competitive strength of the universities. This work identifies the Italian top scientists in over 200 fields, by their research productivity. It then ranks the relative universities by the ratio of top scientists to overall faculty. Finally, it contrasts this list with the ranking list by average productivity of the overall faculty. The analysis is carried out at the field, discipline, and overall university levels. The paper also explores the secondary question of whether the ratio of top scientists to faculty is related to the size of the university.
Scientometrics, 2008
This paper presents a methodology for measuring the technical efficiency of research activities. It is based on the application of data envelopment analysis to bibliometric data on the Italian university system. For that purpose, different input values (research personnel by level and extra funding) and output values (quantity, quality and level of contribution to actual scientific publications) are considered. Our study aims at overcoming some of the limitations connected to the methodologies that have so far been proposed in the literature, in particular by surveying the scientific production of universities by authors’ name.
The different responses of universities to introduction of performance-based research funding
Research Evaluation, 2021
Governments and organizations design performance-based research funding systems (PBRFS) for strategic aims, such as to selectively allocate scarce resources and stimulate research efficiency. In this work, we analyze the relative change in research productivity of Italian universities after the introduction of such a system, featuring financial and reputational incentives. Using a bibliometric approach, we compare the relative research performance of universities before and after the introduction of PBRFS, at the overall, discipline and field levels. The findings show convergence in the universities’ performance, due above all to the remarkable improvement of the lowest performers. Geographically, the universities of the south (vs central and northern Italy) achieved the greatest improvement in relative performance. The methodology, and results, should be of use to university management and policy-makers.
Scientific output scales with resources. A comparison of US and European universities
PLOS ONE, 2019
By using a comprehensive dataset of US and European universities, we demonstrate super-linear scaling between university revenues and their volume of publications and (fieldnormalized) citations. We show that this relationship holds both in the US and in Europe. In terms of resources, our data show that three characteristics differentiate the US system: (1) a significantly higher level of resources for the entire system, (2) a clearer distinction between education-oriented institutions and doctoral universities and (3) a higher concentration of resources among doctoral universities. Accordingly, a group of US universities receive a much larger amount of resources and have a far higher number of publications and citations when compared to their European counterparts. These results demonstrate empirically that international rankings are by and large richness measures and, therefore, can be interpreted only by introducing a measure of resources. Implications for public policies and institutional evaluation are finally discussed.
Journal of Informetrics, 2010
The practice of collaboration, and particularly international collaboration, is becoming ever more widespread in scientific research, and is likewise receiving greater interest and stimulus from policy-makers. However, the relation between research performance and degree of internationalization at the level of single researchers still presents unresolved questions. The present work, through a bibliometric analysis of the entire Italian university population working in the hard sciences over the period [2001][2002][2003][2004][2005], seeks to answer some of these questions. The results show that the researchers with top performance with respect to their national colleagues are also those who collaborate more abroad, but that the reverse is not always true. Also, interesting differences emerge at the sectorial level. Finally, the effect of the nation involved in the international partnership plays a role that should not be ignored.
Scientometrics, 2011
The study presents a time-series analysis of field-standardized average impact of Italian research compared to the world average. The approach is purely bibliometric, based on census of the full scientific production from all Italian public research organizations active in 2001-2006 (hard sciences only). The analysis is conducted both at sectorial level (aggregated, by scientific discipline and for single fields within disciplines) and at organizational level (by type of organization and for single organizations). The essence of the methodology should be replicable in all other national contexts. Its offers support to policy-makers and administrators for strategic analysis aimed at identifying strengths and weaknesses of national research systems and institutions.
A sensitivity analysis of research institutions’ productivity rankings to citation window length
One of the critical issues in bibliometric research assessments is the time required to achieve maturity in citations. Citation counts can be considered a reliable proxy of the real impact of a work only if they are observed after sufficient time has passed from publication date. In the present work the authors investigate the effect of varying the time of citation observation on accuracy of productivity rankings for research institutions. Research productivity measures are calculated for all Italian universities active in the hard sciences in the 2001-2003 period, by individual field and discipline, with the time of the citation observation varying from 2004 to 2008. The objective is to support policy-makers in choosing a citation window that optimizes the tradeoff between accuracy of rankings and timeliness of the exercise.
In this paper, we analyze the effects of research collaborations on the scientific output of academic institutions, drawing on data from the first official Italian research assessment exercise. We measure the scientific performance of a research unit as the number of publications that received an excellent grade in the evaluation process. Different aspects of scientific collaboration are taken into account, such as the degree of openness of a research team towards other institutions and/or countries, the frequency of co-authorships, and the average size of co-authoring teams. Using econometric models for count data, we find that greater and more frequent knowledge exchange resulting from collaboration with external or foreign colleagues increase researchers' productivity.