on the use and abuse of bibliometric performance indicators: a critique of Hix's ‘global ranking of political science departments’ (original) (raw)

Abstract

Bibliometric measures, as provided by the Social Science Citation Index of the Institute for Scientific Information, certainly represent a useful tool for librarians and researchers. However, although librarian scientists have shown that the use of journal impact factors to evaluate the performance of academics is misleading, some authors continue to promote bibliometric metrics to assess the productivity of academic departments and even the entire European academic community. Taking an ambitious ‘global ranking of political science departments’ as a reference, this article questions both the reliability and desirability of bibliometric performance indicators. The article concludes that the development of a panopticon-like audit culture in universities will not enhance their quality, but rather undermine the classical idea and purpose of the university.

Access this article

Log in via an institution

Subscribe and save

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Notes

1. For instance, according to Article 12 of the Irish Universities Act, 1997, the objectives of universities are: ‘(a) to advance knowledge through teaching, scholarly research and scientific investigation, (b) to promote learning in its student body and in society generally, (c) to promote the cultural and social life of society, while fostering and respecting the diversity of the university's traditions, (d) to foster a capacity for independent critical thinking amongst its students, (e) to promote the official languages of the State, with special regard to the preservation, promotion and use of the Irish language and the preservation and promotion of the distinctive cultures of Ireland, (f) to support and contribute to the realisation of national economic and social development, (g) to educate, train and retrain higher level professional, technical and managerial personnel, (h) to promote the highest standards in, and quality of, teaching and research, (i) to disseminate the outcomes of its research in the general community, (j) to facilitate lifelong learning through the provision of adult and continuing education, and (k) to promote gender balance and equality of opportunity among students and employees of the university’ (Irish Statute Book, 2007).
2. Incidentally, the following journals, which all figured among the top twenty ‘political science’ journals according to the ISI impact factor analysis in 2003, have not been included in Hix's list of the main political science journals: Political Geography (second ranked), Annual Review of Political Science (fifth), Public Opinion Quarterly (seventh), Political Physiology (eleventh), New Left Review (twelfth), Survival (thirteenth), Review of International Political Economy (seventeenth) and Policy and Politics (nineteenth).
3. It goes without saying that the €320,000 p.a. pay claim by the presidents of the seven Irish universities (University College, Dublin (UCD), Trinity, Dublin City University, Cork, Maynooth, Galway and Limerick) – representing a 55 per cent pay rise of up to €135,000 p.a. – is corroding staff morale and the above cited, egalitarian objectives of the Irish university sector. Incidentally, UCD president, Hugh Brady, justified his wage claim by suggesting that his role was now more ‘akin to that of the corporate chief executive who must develop and drive strategically and position their business to grow’ (Gleeson, 2007). While the Irish universities have always had a majority of unelected non-academics on their governing bodies, it is noteworthy that a corresponding, corporate takeover of Oxford University dramatically failed in December 2006, as a clear majority of its academic staff rejected a set of controversial governance proposals in an internal referendum, despite huge governmental and corporate pressures (MacNamara, 2006).

References

• Benninghaus, H. (1994) Einführung in die sozialwissenschaftliche Datenanalyse, München: R Oldenbourg Verlag.
  Google Scholar
• Buchstein, H. (1992) Politikwissenschaft und Demokratie. Wissenschaftskonzeption und Demokratietheorie sozialdemokratischer Nachkriegspolitologen in Berlin, Nomos: Baden-Baden.
  Google Scholar
• Bull, M. and Espíndola, R. (2005) ‘European universities in a global ranking of political science departments: a comment on Hix’, European Political Science 4(1): 27–29.
  Article Google Scholar
• Cameron, B.D. (2005) ‘Trends in the usage of ISI bibliometric data: uses, abuses and implications’, Libraries and the Academy 5(1): 105–125.
  Article Google Scholar
• Clark, J. and Newman, J. (1997) The Managerial State, London: Sage.
  Google Scholar
• Crouch, C. (2003) Commercialisation or Citizenship. Education Policy and the Futures of Public Services, London: Fabian Society.
  Google Scholar
• Fehér, I. (2001) ‘The Humboldtian idea of an university’, Neohelicon 28(2): 33–37.
  Article Google Scholar
• Galtung, J. (1990) ‘Theory Formation is Social Research: A Plea for Pluralism’, in E. Øyen (ed.) Comparative Methodology: Theory and Practice in International Social Research, London: Sage, pp. 96–112.
  Google Scholar
• Gleeson, C. (2007) ‘Because we’er worth it!’, College Tribune 20(6): 1, available at: http://www.ucd.ie/tribune, accessed 8 March 2007.
  Google Scholar
• Goldthorpe, J.H. (2000) ‘Social Class and the Differentiation of Employment Contracts’, in J.H. Goldthorpe (ed.) On Sociology: Numbers, Narratives and the Integration of Research and Theory, Oxford: Oxford University Press, pp. 206–229.
  Google Scholar
• Hix, S. (2004a) ‘A global ranking of political science departments’, Political Studies Review 2(3): 293–313.
  Article Google Scholar
• Hix, S. (2004b) ‘European universities in a global ranking of political science departments’, European Political Science 3(2): 5–24.
  Article Google Scholar
• Hix, S. (2005) ‘European universities in a global ranking of political science departments: a reply to Bull and Espíndola’, European Political Science 4(1): 30–32.
  Article Google Scholar
• Hollingsworth, J.R. (2000) ‘Doing institutional analysis: implications for the study of innovations’, Review of International Political Economy 7(4): 595–644.
  Article Google Scholar
• IREG International Ranking Expert Group. (2006) Berlin principles on ranking of Higher Education Institutions, available at: http://www.cepes.ro/hed/meetings/berlin06/Berlin%20Principles.pdf, accessed 8 March 2007.
• Irish Statute Book. (2007) Universities Act, 1997, available at: http://www.irishstatutebook.ie/ZZA24Y1997.html, accessed 8 March 2007.
• Kostoff, R.N. (1998) ‘The use and misuse of citation analysis in research evaluation’, Scientometrics 43(1): 27–43.
  Article Google Scholar
• Lascoumes, P. and Le Gales, P. (2007) ‘Introduction: understanding public policy through its instruments – from the nature of instruments to the sociology of public policy instrumentation’, Governance 20(1): 1–21.
  Article Google Scholar
• Lewinson, G. (2002) ‘Researchers’ and users’ perceptions of the relative standing of biomedical papers in different journals’, Scientometrics 53(2): 229–240.
  Article Google Scholar
• Lynch, K. (2006) ‘Neo-liberalism and marketisation: the implications for higher education’, European Educational Research Journal 5(1): 1–17.
  Article Google Scholar
• MacNamara, W. (2006) ‘Dons reject proposals for reforms of Oxford’, Financial Times, 20 December, 4.
• Mattern, F. (2002) ‘Zur Evaluation der Informatik mittels bibliometrischer Analyse’, Informatik Spektrum 25(1): 22–32.
  Article Google Scholar
• Power, M. (1997) The Audit Society. Rituals of Verification, Oxford: Oxford University Press.
  Google Scholar
• Power, M. (2003) ‘Auditing and the production of legitimacy’, Accounting, Organisations and Society 28: 379–394.
  Article Google Scholar
• Schneider, G., Steunenberg, B., Holzinger, K. and Gleditsch, P. (2006) ‘Why is European political science so unproductive and what should be done about this? available at: http://www.uni-konstanz.de/FuF/Verwiss/GSchneider/downloads/papers/EPS.pdf, accessed 8 March 2007.
• Seglen, P.O. (1997) ‘Why the impact factor of journals should not be used for evaluation research’, British Medical Journal 314: 497.
  Article Google Scholar
• Shore, C. and Wright, S. (2000) ‘Coercive Accountability. The Rise of Audit Culture in Higher Education’, in M. Strathern (ed.) Audit Cultures. Anthropological Studies in Accountability, Ethics and the Academy, London: Routledge, pp. 59–89.
  Google Scholar
• Terrier, J. (2002) ‘Le processus d’autonomisation des universités suisses. Principes et problèmes’, Carnets de bords en sciences humaines 2(4): 13–21, available at: http://www.carnets-de-bord.ch, accessed 8 March.
  Google Scholar
• Turner, D. (2005) ‘Benchmarking in universities: league tables revisited’, Oxford Review of Education 31(3): 353–371.
  Article Google Scholar

Download references

Acknowledgements

The author would like to thank John Baker, Colin Crouch, Stefan Klein, Oscar Molina, Sabina Stan and Tobias Theiler for valuable comments on an earlier version of this paper. The usual disclaimer applies.

Author information

Authors and Affiliations

1. Industrial Relations and Human Resources Group, UCD Business School, University College Dublin, Dublin, 4, Belfield, Ireland
   Roland Erne

Authors

1. Roland Erne
   You can also search for this author inPubMed Google Scholar

Rights and permissions

About this article

Cite this article

Erne, R. on the use and abuse of bibliometric performance indicators: a critique of Hix's ‘global ranking of political science departments’.Eur Polit Sci 6, 306–314 (2007). https://doi.org/10.1057/palgrave.eps.2210136

Download citation

• Published: 02 August 2007
• Issue Date: 01 September 2007
• DOI: https://doi.org/10.1057/palgrave.eps.2210136

Keywords