A performance measurement framework for IT service management (original) (raw)

Abstract

The study provides a comprehensive literature review on the performance measurement of ITSM. A review of the existing industry and academic literature showed a gap in theory for performance measurement of ITSM. There was also a lack of a contextualised performance measurement framework for ITSM. The study developed categories for types of organisation level and process level ITSM, and categories for types of ITSM performance metrics. The performance measurement framework developed by the study is structured using the Balanced Scorecard (BSC) and can be used to quantify benefits and link organisational level benefits and metrics with process level metrics. The developed framework includes a consolidated ITSM metrics catalogue structure. The study identified the internal and external organisational factors that influence the selection of ITSM performance metrics and proposes a contingency theory for the performance measurement of ITSM. The study makes theoretical and practical contributions in ITSM performance measurement by extending ITSM performance measurement theory, IS design theory and developing a holistic multi-level ITSM performance measurement framework that can be used by organisations. This dissertation is a result of a study funded by an Australian Research Council (ARC) linkage project grant in partnership with Queensland Health (QH) and the IT Service Management Forum (itSMF) Australia. The study contributes to the linkage project by addressing the complex interactions of benefits, performance metrics and methods to enable Chief Information Officers (CIOs) and IT service managers to measure the performance of IT service management.

Loading...

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

References (234)

  1. Aazadnia, M & Fasanghari, M 2008, 'Improving the information technology service management with six sigma', International Journal of Computer Science and Network Security, vol. 8, no. 3, pp. 144-50.
  2. Abran, A & Buglione, L 2003, 'A multidimensional performance model for consolidating Balanced Scorecards', Advances in Engineering Software, vol. 34, no. 6, pp. 339-49.
  3. Akatsu, M 2007, 'Identifying the value for service management', 9th IEEE International Conference on E-Commerce Technology and the 4th IEEE International Conference on Enterprise Computing, E-Commerce, and E-Services, 2007., Tokyo, Japan, July 23-26. pp. 491-2.
  4. Alter, S 2008, 'Seeking synergies between four views of service in the IS field', Proceedings of the Fourteenth Americas Conference on Information Systems, Toronto, Ontario, August 14-17.
  5. Anderson, K & McAdam, R 2004, 'A critique of benchmarking and performance: lead or lag', Benchmarking: An International Journal, vol. 11, no. 5, pp. 465-83.
  6. Azzone, G, Masella, C & Bertelè, U 1991, 'Design of Performance Measures for Time-based Companies', International Journal of Operations & Production Management, vol. 11, no. 3, pp. 77-85.
  7. Barafort, B, DiRenzo, B & Merlan, O 2002, 'Benefits resulting from the combined use of ISO/IEC 15504 with the Information Technology Infrastructure Library (ITIL)', Proceedings of the 4th International Conference on Product Focussed Software Development and Process Improvement (PROFES), Rovaniemi, Finland, December 9-11. pp. 314-25.
  8. Barafort, B, DiRenzo, B, Lejeune, V, Prime, S & Simon, J-M 2005, 'ITIL based service management measurement and ISO/IEC 15504 process assessment: a win- win opportunity', Proceedings of the 5th International SPICE Conference on Process Assessment and Improvement, Klagenfurt, Austria, April 27-29.
  9. Bardhan, IR, Demirkan, H, Kannan, PK, Kauffman, RJ & Sougstad, R 2010, 'An Interdisciplinary Perspective on IT Services Management and Service Science', Journal of Management Information Systems, vol. 26, no. 4, pp. 13-64.
  10. Bartolini, C 2009, 'Business-driven IT management', Doctor of Research thesis, Università degli Studi di Ferrara.
  11. Bartolini, C, Salle, M & Trastour, D 2006, 'IT service management driven by business objectives an application to incident management', Proceedings of the 10th IEEE/IFIP Network Operations and Management Symposium, NOMS. , Osaka, Japan, April 3-7. pp. 45-55.
  12. Basili, V, Caldiera, G, Rombach, D & Marciniak, J 1994, 'The goal question metric approach', in Encyclopedia of Software Engineering, Wiley.
  13. Belo de Oliveira, P 2009, 'The value of ITIL', Univerisdade Tecnica de Lisboa.
  14. Belson, WA 1981, The design and understanding of survey questions, Gower, Aldershot, England.
  15. Bhattacherjee, A 2012, Social Science Research: Principles, Methods, and Practices, vol. 3, USF Open Access Textbooks Collection.
  16. Black, J, Draper, C, Lococo, T, Matar, F & Ward, C 2007, 'An integration model for organizing IT service management', IBM Systems Journal, vol. 46, no. 3, pp. 405-22.
  17. Botha, JH 2004, 'Improving customer satisfaction and operational effectiveness with the use of an ICT service management best practice framework: action research in the shared service centre', Masters Degree thesis, Oxford Brookes University.
  18. Bradley, G 2010, Benefit realisation management a practical guide to achieving benefits through change, 2nd edn, Ashgate Pub, Farnham. viewed 24 November 2012, <http://ezproxy.usq.edu.au/login?url=http://site.ebrary.com/lib/unisouthernqld/Doc?i d=10378028%3E.
  19. Brignall, S & Ballantine, J 1996, 'Performance measurement in service businesses revisited', International Journal of Service Industry Management, vol. 7, no. 1, pp. 6-
  20. Brito e Abreu, F, de Braganca, V, Freitas, JM & Costa, JC 2010, 'Definition and validation of metrics for ITSM process models', Proceedings of the Seventh International Conference on the Quality of Information and Communications Technology (QUATIC), September 29 -October 2. pp. 79-88.
  21. Brooks, P 2006, Metrics for IT service management, 1st edn, Van Haren Pub., Zaltbommel, Netherlands.
  22. Brown, DH 2009, 'Indirect perceptual realism and demonstratives', Philosophical Studies, vol. 145, no. 3, pp. 377-94.
  23. Brown, MG 1996, Keeping score using the right metrics to drive world-class performance, Quality Resources: Distributed by AMACOM Books, New York. viewed 18 March 2010.
  24. Bruton, N 2004, The ITIL Experience has it been worth it?, Bruton Consultancy, Cardigan, Wales.
  25. Brynjolfsson, E & Hitt, LM 1995, 'Paradox lost? : firm-level evidence on the returns to information systems spending', Management Science, vol. 42, no. 4, pp. 541-58.
  26. Brynjolfsson, E & Hitt, LM 1998, 'Beyond the productivity paradox', Communications of the ACM, vol. 41, no. 8, pp. 49-55.
  27. Cater-Steel, A 2009, 'IT service departments struggle to adopt a service-oriented philosophy', International Journal of Information Systems in the Service Sector (IJISSS), vol. 1, no. 2, pp. 69-77.
  28. Cater-Steel, A & Tan, W-G 2005, 'Implementation of IT infrastructure library (ITIL) in Australia: progress and success factors', Proccedings of the IT Governance International Conference, Auckland, New Zealand, November 14-16. pp. 39-52.
  29. Cater-Steel, A & McBride, N 2007, 'IT service management improvement -actor network perspective', Proceedings of the 15th European Conference on Information Systems, ECIS Paper 96, St Gallen, Switzerland, June 7-9. pp. 1202-13.
  30. Cater-Steel, A & Toleman, M 2009, 'Education for IT service management standards', Selected readings on information technology management: contemporary issues, pp. 293-306.
  31. Cater-Steel, A, Tan, W-G & Toleman, M 2006a, 'Challenge of adopting multiple process improvement frameworks', Proceedings of the 14th European Conference on Information Systems, ECIS Paper 177, Goteborg, Sweden, June 12-14. pp. 1375-86.
  32. Cater-Steel, A, Toleman, M & Tan, W-G 2006b, 'Transforming IT service management the ITIL impact', Proceedings of the 17th Australasian Conference on Information Systems, ACIS 2006 Paper 81, Adelaide, Australia, December 6-8.
  33. Cater-Steel, A, Tan, W-G & Toleman, M 2007, itSMF Australia 2006 Conference: summary of ITIL adoption survey responses, University of Southern Queensland, Toowoomba, Australia.
  34. ----2008, itSMF Australia 2007 Conference: summary of ITSM standards and frameworks survey responses, University of Southern Queensland, Toowoomba, Australia.
  35. ----2009a, 'Using institutionalism as a lens to examine ITIL adoption and diffusion', Proceedings of the 20th Australasian Conference on Information Systems, ACIS Paper 73, Melbourne, Australia, December 2-4. pp. 321-30.
  36. ----2009b, itSMF Australia 2008 Conference: summary of ITSM standards and frameworks survey responses, University of Southern Queensland, Toowoomba, Australia.
  37. ----2011, itSMF Australia 2010 Conference: summary report of ITSM standards and frameworks survey, University of Southern Queensland, Toowoomba, Australia.
  38. Chan, P, Durant, S, Gall, V & Raisinghani, M 2008, 'Aligning six sigma and ITIL implications for IT service management', Proceedings of the International Conference on Information Resources Management, CON-FIRM, Paper 7, Ontario, Canada, May 18-20.
  39. Chang, JCJ & King, WR 2005, 'Measuring the performance of information systems: A functional scorecard', Journal of Management Information Systems, vol. 22, no. 1, pp. 85-115.
  40. Chesbrough, H & Spohrer, J 2006, 'A research manifesto for services science', Communications of the ACM, vol. 49, no. 7, pp. 35-40.
  41. Chiapello, E & Lebas, M 1996, 'The tableau de bord, a French approach to management information', Proceedings of the 19th Annual Congress of the European Accounting Association, Bergen, Norway, May 2-4.
  42. Chiu, D & Nouri, H 2009, Metrics that matter -selecting metrics that drive service improvement, Nouri Associates Incorporated, viewed 24 February 2012, <http://www.on-demand-itsm.com/ITSM-Metrics-That-Matter\_8-13-09\_v2.pdf%3E.
  43. Chow, R & Jonas, W 2008, 'Beyond dualisms in methodology: an integrative design research medium "MAPS" and some reflections', Proceedings of the 4th biennial conference of the Design Research Society, Sheffield, UK, July 16-19. pp. 1-18.
  44. Cleven, A, Gubler, P & Huner, KM 2009, 'Design alternatives for the evaluation of design science research artifacts', Proceedings of the 4th International Conference on Design Science Research in Information Systems and Technology, DESRIST Paper 19, Philadelphia, Pennsylvania, May 7-8. pp. 1-8.
  45. Conger, S, Winniford, M & Erickson-Harris, L 2008, 'Service management in operations', Proceedings of the 14th Americas Conference on Information Systems, AMCIS Paper 362, Toronto, August 14 -17. pp. 1-10.
  46. Cooper, HM 1984, 'A taxonomy of literature reviews', Proceedings of the 69th Annual Meeting of the American Educational Research Association, Chicago, Illinois, March 31-April 4.
  47. Creswell, JW 2009, Research design: qualitative, quantitative, and mixed methods approaches, 3rd edn, Sage Publications, Thousand Oaks.
  48. Creswell, JW, Clark, P & Vicki, L 2011, Designing and conducting mixed methods research, 2nd edn, Sage Publications, Los Angeles.
  49. Cronin Jr, JJ & Taylor, SA 1992, 'Measuring service quality: A reexamination and extension', Journal of Marketing, vol. 56, no. 3, pp. 55-68.
  50. Cross, N 1982, 'Designerly ways of knowing', Design Studies, vol. 3, no. 4, pp. 221- 7.
  51. De Toni, A & Tonchia, S 2001, 'Performance measurement systems -models, characteristics and measures', International Journal of Operations & Production Management, vol. 21, no. 1/2, pp. 46-71.
  52. Dedrick, J, Gurbaxani, V & Kraemer, KL 2003, 'Information technology and economic performance: A critical review of the empirical evidence', ACM Computing Surveys, vol. 35, no. 1, pp. 1-28.
  53. Dehning, B & Richardson, VJ 2002, 'Returns on investments in information technology: a research synthesis', Journal of Information Systems, vol. 16, no. 1, pp. 7-30.
  54. DeLone, WH & McLean, ER 1992, 'Information Systems Success: The Quest for the Dependent Variable', Information Systems Research, vol. 3, no. 1, pp. 60-95.
  55. ----2003, 'The DeLone and McLean model of information systems success: a ten- year update', Journal of Management Information Systems, vol. 19, no. 4, pp. 9-30.
  56. Dobson, PJ 2001, 'The philosophy of critical realism -An opportunity for information systems research', Information Systems Frontiers, vol. 3, no. 2, pp. 199- 210. Dobson, PJ 2002, 'Critical realism and information systems research: why bother with philosophy?', Information Research, vol. 7, no. 2, viewed 22 September 2009, http://InformationR.net/ir/7-2/paper124.html
  57. Donko, D & Traljic, I 2006, 'IT service management and normatively regulated activities', Proceedings of the 5th WSEAS International Conference on Telecommunications and Informatics, Istanbul, Turkey, May 27-29, 2006 pp. 116-
  58. ----2009, 'Measurement of service effectiveness and establishment of baselines', WSEAS Transactions on Information Science and Applications, vol. 6, no. 8, pp. 1310-9. EAB. Group Pty. Ltd. 2009, SmartKPIs.com: the smart choice in performance managemement, viewed 26 April 2011, <http://www.smartkpis.com/%3E.
  59. Edvinsson, L 1997, 'Developing intellectual capital at Skandia', Long Range Planning, vol. 30, no. 3, pp. 366-73.
  60. Eisenhardt, KM 1991, 'Better stories and better constructs: the case for rigor and comparative logic', Academy of Management Review, vol. 16, no. 3, pp. 620-7.
  61. England, R, 2007, 'Don't swim against the tide: ten alternatives to itSMF for ITIL practitioners', The IT Skeptic, 13 October 2007, <http://www.itskeptic.org/node/392%3E.
  62. ----, 2011, 'ITSM's fixation with capability maturity', The IT Skeptic, 26 April 2011, <http://www.itskeptic.org/%3E.
  63. Foddy, W 1993, Constructing questions for interviews and questionnaires: theory and practice in social research, New York: Cambridge University Press, Cambridge, England. Gable, G 1994, 'Integrating case study and survey research methods: an example in information systems', European Journal of Information Systems, vol. 3, no. 2, pp. 112-26.
  64. Galliers, R 1992, Information systems research: issues, methods, and practical guidelines, Blackwell Scientifc Publications, Oxford, Boston.
  65. Galup, S, Quan, J, Dattero, R & Conger, S 2007, 'Information technology service management: an emerging area for academic research and pedagogical development', Proceedings of the 2007 ACM SIGMIS CPR conference on Computer personnel research: The global information technology workforce, St. Louis, Missouri, USA, April 19-21. pp. 46-52.
  66. Galup, SD, Dattero, R, Quan, JJ & Conger, S 2009, 'An Overview of IT Service Management', Communications of the ACM, vol. 52, no. 5, pp. 124-7.
  67. Gartner Inc. 2012, Quarterly IT spending forecast, Gartner Inc., viewed 6 March 2012, <http://www.gartner.com/it/page.jsp?id=1888514%3E.
  68. Gershon, P 2008, Review of the Australian government's use of information and communication technology, Commonwealth of Australia.
  69. Goldratt, EM 1990, What is this thing called theory of constraints and how should it be implemented?, North River Press, Great Barrington, Mass.
  70. Gomes, CF, Yasin, MM & Lisboa, JV 2004, 'A literature review of manufacturing performance measures and measurement in an organizational context: a framework and direction for future research', Journal of Manufacturing Technology Management, vol. 15, no. 6, pp. 511-30.
  71. Graves, D 2010, 'IT services management IT service inventory', Proceedings of the 43rd Hawaii International Conference on System Sciences Koloa, Kauai, HI, USA, January 5-8. pp. 1-9.
  72. Gregor, S 2002, 'Design theory in information systems', Australian Journal of Information Systems, no. Special Issue, pp. 14-22.
  73. ----2006, 'The nature of theory in information systems', MIS Quarterly, vol. 30, no. 3, pp. 611-42.
  74. ----2009, 'Building theory in the sciences of the artificial', Proceedings of the 4th International Conference on Design Science Research in Information Systems and Technology, Philadelphia, Pennsylvania, May 7-8. pp. 1-10.
  75. Gregor, S & Jones, D 2007, 'The anatomy of a design theory', Journal of the Association for Information Systems, vol. 8, no. 5, pp. 312-35.
  76. Grönroos, C 1994, 'From scientific management to service management', International Journal of Service Industry Management, vol. 5, no. 1, pp. 5-20.
  77. ----2000, Service management and marketing: a customer relationship management approach, 2nd edn, Wiley, Chichester.
  78. ----2008, 'Service logic revisited: who creates value? And who co-creates?', European Business Review, vol. 20, no. 4, pp. 298-314.
  79. Gummesson, E 1999, Total relationship marketing : from the 4Ps-product, price, promotion, place-of traditional marketing management to the 30Rs-the thirty relationships-of the new marketing paradigm, Butterworth-Heinemann, Boston Mass. Hall, RH 1980, 'Effectiveness Theory and Organizational Effectiveness', The Journal of Applied Behavioral Science, vol. 16, no. 4, pp. 536-45.
  80. Henri, J-F 2004, 'Performance measurement and organizational effectiveness: bridging the gap', Managerial Finance, vol. 30, no. 6, pp. 93-123.
  81. Hevner, AR, March, ST, Park, J & Ram, S 2004, 'Design science in Information Systems research', MIS Quarterly, vol. 28, no. 1, pp. 75-105.
  82. Hochstein, A 2004, 'Managing IT service quality as perceived by the customer: the service oriented IT SERVQUAL', Proceedings of the International Telecommunications Society (ITS) 15th Biennial Conference, Berlin, September 4- 7.
  83. Hochstein, A, Zarnekow, R & Brenner, W 2005a, 'Evaluation of service-oriented IT management in practice', Proceedings of the International Conference on Services Systems and Services Management (ICSSSM). June 13 -15. pp. 80-4.
  84. ----2005b, 'ITIL as common practice reference model for IT service management: formal assessment and implications for practice', Proceedings of the 2005 IEEE International Conference on e-Technology, e-Commerce and e-Service, Hong Kong, 29 March -1 April. pp. 704-10.
  85. Hochstein, A, Tamm, G & Brenner, W 2005c, 'Service-oriented IT management: benefit, cost and success factors', Proceedings of the 13th European Conference on Information Systems, ECIS 2005 Paper 98, Regensburg, Germany, May 26 -28.
  86. IBM Corporation 2012, 'PureSystems: Solving the complexity of IT', PureSystems: Solving the complexity of IT, http://www-304.ibm.com/businesscenter/, viewed 22 September 2012, <ibm.com/businesscenter/forwardview> IBM Research 2012, Services Defintion, viewed 5 March 2012, <http://www.research.ibm.com/ssme/services.shtml%3E.
  87. Iden, J & Langeland, L 2010, 'Setting the stage for a successful ITIL adoption: A delphi study of IT experts in the Norwegian armed forces', Information Systems Management, vol. 27, no. 2, pp. 103-12.
  88. ISO/IEC 2005, ISO/IEC 20000-1:2005 Information technology service management, Part 1: Specification, International Organization for Standardization, Geneva.
  89. itSMF Australia 2011a, About itSMF Australia, itSMF, viewed 01 February 2011, <http://www.itsmf.org.au/%3E.
  90. ----2011b, itSMF Australia Conference Demographic History and Research Information, itSMF Australia.
  91. itSMF Germany 2008, Total Value of IT (TVITEM), itSMF Deutschland e.V., viewed 07 February 2011, <http://www.itsmf.de/ak\_tv\_it.html%3E. itSMF International 2008, Self assessment (ITIL V2), viewed 07 February 2011, <http://www.itsmfi.org/content/self-assessment-itil-v2%3E.
  92. Ittner, CD & Larcker, DF 2003, 'Coming up short on nonfinancial performance measurement', Harvard Business Review, vol. 81, no. 11, pp. 88-95.
  93. Ittner, CD & Lacker, DF 2003, 'Coming Up Short on Nonfinancial Performance Measurement', Harvard Business Review, no. November, pp. 1-9.
  94. Jiang, JJ, Klein, G & Crampton, SM 2000, 'A note on SERVQUAL reliability and validity in information system service quality measurement', Decision Sciences, vol. 31, no. 3, pp. 725-44.
  95. Jiang, JJ, Klein, G & Carr, CL 2002, 'Measuring information system service quality: SERVQUAL from the other side', MIS Quarterly, vol. 26, no. 2, pp. 145-66.
  96. Jonas, W 2007, 'Research through DESIGN through research: A cybernetic model of designing design foundations', Kybernetes, vol. 36, no. 9/10, pp. 1362 -80.
  97. Kaplan, R & Norton, D 2004, 'The strategy map: guide to aligning intangible assets', Strategy and Leadership, vol. 32, no. 5, pp. 10-7.
  98. Kaplan, RS & Norton, DP 1992, 'The Balanced Scorecard -Measures That Drive Performance', Harvard Business Review, vol. 70, no. 1, pp. 71-9.
  99. Katzan, H 2008, Service science concepts, technology, management, iUniverse, Inc., Bloomington.
  100. Keegan, D, Eiler, RG & Jones, CR 1989, 'Are your performance measures obsolete?', Management Accounting, vol. 70, no. 12, pp. 45-60.
  101. Keel, AJ, Orr, MA, Hernandez, RR, Patrocinio, EA & Bouchard, J 2007, 'From a technology-oriented to a service-oriented approach to IT management', IBM Systems Journal, vol. 46, no. 3, pp. 549-64.
  102. Kettinger, WJ & Lee, CC 1994, 'Perceived Service Quality and User Satisfaction with the Information-Services Function', Decision Sciences, vol. 25, no. 5-6, pp. 737-66.
  103. Kettinger, WJ & Lee, CC 1999, 'Replication of Measures in Information Systems Research: The Case of IS SERVQUAL', Decision Sciences, vol. 30, no. 3, pp. 893-9.
  104. Khazanchi, D & Munkvold, BE 2000, 'Is information systems a science? an inquiry into the nature of the information systems discipline', ACM SIGMIS Database, vol. 31, no. 2, pp. 24-42.
  105. King, WR & Jun, H 2005, 'External validity in IS survey research', Communications of AIS, vol. 2005, no. 16, pp. 880-94.
  106. Kitchenham, B 2004, Procedures for performing systematic reviews, NICTA Technical Report 0400011T.1, Keele University and National ICT Australia Ltd.
  107. Kitchenham, B, Brereton, OP, Budgen, D, Turner, M, Bailey, J & Linkman, S 2009, 'Systematic literature reviews in software engineering -A systematic literature review', Information and Software Technology, vol. 51, no. 1, pp. 7-15.
  108. Koch, H & Gierschner, C 2007, 'Advantages of an ITIL-based process framework in a complex SAP® system landscape', 4th IEEE Workshop on Intelligent Data Acquisition and Advanced Computing Systems. pp. 431-3.
  109. Krippendorff, K 1980, Content analysis: an introduction to its methodology, Sage, Beverly Hills.
  110. ----2004, Content analysis: an introduction to its methodology, 2nd edn, Sage, Thousand Oaks, CA.
  111. Lahtela, A, Jäntti, M & Kaukola, J 2010, 'Implementing an ITIL-based IT service management measurement system', Fourth International Conference on Digital Society, St. Maarten, Netherlands Antilles. pp. 249-54.
  112. Landrum, H, Prybutok, V, Zhang, X & Peak, D 2009, 'Measuring IS system service quality with SERVQUAL: Users' perceptions of relative importance of the five SERVPERF dimensions', The International Journal of an Emerging Transdiscipline, vol. 12, pp. 18-35.
  113. Lee, A (ed.) 2004, Thinking about social theory and philosophy for information systems, Social Theory and Philosophy for Information Systems, John Wiley & Sons Chichester, UK.
  114. Lee, AS 1999, 'Rigor and relevance in MIS research: Beyond the approach of positivism alone', MIS Quarterly, vol. 23, no. 1, pp. 29-33.
  115. Lowe, W 2003, Content Analysis Software: A Review, Harvard University.
  116. Luftman, J & Ben-Zvi, T 2010, 'Key Issues for It Executives 2010: Judicious It Investments Continue Post-Recession', MIS Quarterly Executive, vol. 9, no. 4, pp. 263-73.
  117. Luftman, J & Ben-Zvi, T 2011, 'Judicious IT investments continue post-recession ', CIO Net, Key Issues for IT Executives 2010, viewed 31 January, http://blog.cionet.com/2010/12/15/key-issues-for-it-executives-2010/ Luthans, F & Stewart, T 1977, 'A general contingency theory of management', Academy of Management Review, vol. 2, no. 2, pp. 181-95.
  118. Lynch, RL & Cross, KF 1993, Measure up!: yardsticks for continuous improvement, Brackwell Business, Cambridge, Massachusetts.
  119. Lynge, B & Schou, C 2008, 'Discovering archetypal patterns in the journey toward service oriented management -an empirical analysis of an emerging area', Masters thesis, University of Aarhus.
  120. Macquarie Library 2012, The Macquarie dictionary, Macquarie Library, North Ryde NSW. <http://resguide.usq.edu.au/index.php?type=databases&route=direct&ID=314%3E.
  121. Maglio, PP, Srinivasan, S, Kreulen, JT & Spohrer, J 2006, 'Service systems, service scientists, SSME, and innovation', Communications of the ACM, vol. 49, no. 7, pp. 81-5.
  122. Malcolm, C 2009, 'ITIL: the de facto framework for IT service management?', University of Portsmouth.
  123. Manuel de Oliveira, V 2007, 'Monitoring IT services for supporting business processes', Universidad Tecnica de Lisboa.
  124. Marchand, M & Raymond, L 2008, 'Researching performance measurement systems -An information systems perspective', International Journal of Operations & Production Management, vol. 28, no. 7-8, pp. 663-86.
  125. Marrone, M 2009, ITIL state of the nation survey findings, Hornbill, Service Management Consultancy (SMCG) Ltd, Georg-August-Universität Göttingen.
  126. Marrone, M & Kolbe, L 2010, 'ITIL: providing more than just operational benefits: an empirical research', Multikonferenz Wirtschaftsinformatik 2010, Göttingen, February 23-25. pp. 281-92.
  127. Martinsons, M, Davison, R & Tse, D 1999, 'The balanced scorecard: a foundation for the strategic management of information systems', Decision Support Systems, vol. 25, no. 1, pp. 71-88.
  128. McLaren, TS, Head, MM, Yuan, YF & Chan, YE 2011, 'A Multilevel Model for Measuring Fit between a Firm's Competitive Strategies and Information Systems Capabilities', MIS Quarterly, vol. 35, no. 4, pp. 909-29.
  129. McNamara, C 1997, Basics of developing case studies, viewed 09 September 2009, <<http://managementhelp.org/evaluatn/casestdy.htm%3E%3E.
  130. McNaughton, B, Ray, P & Lewis, L 2010, 'Designing an evaluation framework for IT service management', Information & Management, vol. 47, no. 4, pp. 219-25.
  131. Medori, D & Steeple, D 2000, 'A framework for auditing and enhancing performance measurement systems', International Journal of Operations & Production Management, vol. 20, no. 5-6, pp. 520-33.
  132. Miles, MB & Huberman, AM 1994, Qualitative data analysis: an expanded sourcebook, 2nd edn, SAGE, Thousand Oaks, California.
  133. Moura, A, Sauve, J, Jornada, J & Radziuk, E 2006a, 'A quantitative approach to IT investment allocation to improve business results', Proceedings of the Seventh IEEE International Workshop on Policies for Distributed Systems and Networks (POLICY), 5-7 June 2006. pp. 85-95.
  134. ----2006b, 'A Quantitative Approach to IT Investment Allocation to Improve Business Results', in Proceedings of the Seventh IEEE International Workshop on Policies for Distributed Systems and Networks, pp. 87-95.
  135. Myers, B, Kappelman, L & Prybutok, V 1997, 'A comprehensive model for assessing the quality and productivity of the information systems function: toward a contingency theory for information systems assessment', Information Resources Management Journal, vol. 10, no. 1, pp. 6-25.
  136. Neale, P, Thapa, S & Boyce, C 2006, 'Preparing a case study: a guide for designing and conducting a case study for evaluation input', Pathfinder International: Writing a case study, viewed 09 September 2009, <http://www.pathfind.org/site/DocServer/m\_e\_tool\_series\_case\_study.pdf http://www.pathfind.org/site/DocServer/m\_e\_tool\_series\_case\_study.pdf%3E
  137. Neely, A, Adams, C & Crowe, P 2001, 'The performance prism in practice', Measuring Business Excellence, vol. 5, no. 2, pp. 6-13.
  138. Neely, A, Adams, C & Kennerley, M 2002, The performance prism: the scorecard for measuring and managing business success, Prentice Hall Financial Times, London.
  139. Neely, A, Gregory, M & Platts, K 2005, 'Performance measurement system design - A literature review and research agenda', International Journal of Operations & Production Management, vol. 25, no. 12, pp. 1228-63.
  140. Neely, A, Mills, J, Platts, K, Richards, H, Gregory, M, Bourne, M & Kennerley, M 2000, 'Performance measurement system design: developing and testing a process- based approach', International Journal of Operations & Production Management, vol. 20, no. 9-10, pp. 1119-45.
  141. Nicho, M 2008, 'Information technology audit: systems alignment and effectiveness measures', AUT University.
  142. Normann, R 2000, Service management: strategy and leadership in service business, 3rd edn, Wiley, Chichester; New York.
  143. Norreklit, H 2000, 'The balance on the balanced scorecard a critical analysis of some of its assumptions', Management Accounting Research, vol. 11, no. 1, pp. 65-88.
  144. O'Callaghan, KM 2010, ' Incident Management: Human Factors and Minimising Mean Time to Restore', Australian Catholic University. OGC 2002a, ICT infrastructure management, TSO, London.
  145. ----2002b, Planning to implement service management, TSO, London.
  146. ----2007a, Continual service improvement, TSO, London.
  147. ----2007b, Service strategy, TSO, London.
  148. ----2009, ITIL V3 foundation handbook, TSO, London.
  149. ----2011, ITIL Continual service improvement, 2nd edn, TSO, London.
  150. Osterle, H, Becker, J, Frank, U, Hess, T, Karagiannis, D, Krcmar, H, Loos, P, Mertens, P, Oberweis, A & Sinz, EJ 2010, 'Memorandum on design-oriented information systems research', European Journal of Information Systems, pp. 7-10.
  151. OUP 2000, The Oxford English dictionary online, Oxford University Press, Oxford. <http://gateway.library.qut.edu.au/login?url=http://dictionary.oed.com/entrance.dtl%
  152. Parasuraman, A 2004, 'Assessing and improving service performance for maximum impact: insights from a two-decade-long research journey', Performance Measurement and Metrics, vol. 5, no. 2, pp. 45-52.
  153. Parasuraman, A, Zeithaml, VA & Berry, LL 1985, 'A Conceptual-Model of Service Quality and Its Implications for Future-Research', Journal of Marketing, vol. 49, no. 4, pp. 41-50.
  154. Patton, MQ 2002, Qualitative research and evaluation methods, 3rd edn, Sage Publications, Thousand Oaks, California.
  155. Paxson, V 1996, 'Towards a framework for defining Internet performance metrics', paper presented to iNet Canada: transforming our society now, Montreal, Canada, 24-28 June.
  156. Peffers, K, Tuunanen, T, Rothenberger, M & Chatterjee, S 2008, 'A design science research methodology for information systems research', Journal of Management Information Systems, vol. 24, no. 3, pp. 45-77.
  157. Peppard, J, Ward, J & Daniel, E 2007, 'Managing the Realization of Business Benefits from IT Investments', MIS Quarterly Executive, vol. 6, no. 1, pp. 1-11.
  158. Pinsonneault, A & Kraemer, KL 1993, 'Survey research methodology in management information systems: an assessment', Journal of Management Information Systems, vol. 10, no. 2, pp. 75-105.
  159. Pitt, LF, Watson, RT & Kavan, CB 1995, 'Service Quality -a Measure of Information-Systems Effectiveness', MIS Quarterly, vol. 19, no. 2, pp. 173-87.
  160. Pollard, C & Cater-Steel, A 2009, 'Justifications, strategies, and critical success factors in successful ITIL implementations in U.S. and Australian companies: an exploratory study', Information Systems Management, vol. 26, no. 2, pp. 164 -75.
  161. Potgieter, B, Botha, J & Lew, C 2005, 'Evidence that use of the ITIL framework is effective', Proceedings of the 18th Annual Conference of the National Advisory Committee on Computing Qualifications, Tauranga, New Zealand, July 10-13. pp. 160-7.
  162. Praeg, C & Schnabel, U 2006, 'IT-service cachet -managing IT-service performance and IT-service quality', Proceedings of the 39th Hawaii International Conference on System Sciences, Hawaii, Jan 04-07. pp. 34-43.
  163. Pultorak, D, Henry, C & Leenards, P 2008, MOF 4.0: Microsoft operations framework 4.0, Van Haren Publishing, Zaltbommel.
  164. Ratcliff, D 2009, 15 Methods of data analysis in qualitative research, viewed 28 February 2011, <http://qualitativeresearch.ratcliffs.net/15methods.pdf%3E.
  165. Raymond, L 1990, 'Organizational context and information systems success: a contingency approach', Journal of Management Information Systems, vol. 6, no. 4, pp. 5-20.
  166. Reinecke, I, Independent review of implementation of the ICT reform program, 2010, Commonwealth of Australia.
  167. Rejc, A 2004, 'Toward contingency theory of performance measurement', Journal for East European Management Studies, vol. 9, no. 3, pp. 243-364.
  168. Rolstadås, A 1998, 'Enterprise performance measurement', International Journal of Operations & Production Management, vol. 18, no. 9/10, pp. 989-99.
  169. Roozenburg, NFM & Eekels, J 1995, Product design: fundamentals and methods, Wiley, Chichester, New York.
  170. Rosemann, M & Vessey, I 2008, 'Toward improving the relevance of information systems research to practice: The role of applicability checks', MIS Quarterly, vol. 32, no. 1, pp. 1-22.
  171. Ryan, GW & Bernard, HR 2003, 'Techniques to Identify Themes', Field Methods, vol. 15, no. 1, pp. 85-109.
  172. Salle, M 2004, 'IT service management and IT governance: review, comparative analysis and their impact on utility computing', HP Research Labs, viewed 29 September 2009, http://www.hpl.hp.com/techreports/2004/HPL-2004-98.html
  173. Saunders, C & Jones, J 1992, 'Measuring performance of the information systems function', Journal of Management Information Systems, vol. 8, pp. 63-82.
  174. Schneberger, S, Pollard, C & Watson, H 2009, 'Theories: For Academics and Practitioners', Information Systems Management, vol. 26, no. 1, pp. 52-60.
  175. Scott, JE 1995, 'The Measurement of Information-Systems Effectiveness - Evaluating a Measuring Instrument', Data Base for Advances in Information Systems, vol. 26, no. 1, pp. 43-61.
  176. Seddon, PB, Graeser, V & Willcocks, LP 2002, 'Measuring organizational IS effectiveness', ACM SIGMIS Database, vol. 33, no. 2, p. 11.
  177. Sheehan, KB 2001, 'E-mail survey response rates: a review ', Journal of Computer- Mediated Communication, vol. 6, no. 2, viewed 02 February 2011, http://jcmc.indiana.edu/vol6/issue2/sheehan.html
  178. Sills, SJ & Song, CY 2002, 'Innovations in survey research -An application of Web- based surveys', Social Science Computer Review, vol. 20, no. 1, pp. 22-30.
  179. Šimková, E & Basl, J 2006, 'Business value of IT', Proceedings of the 14th International Conference on Systems Integration, Prague, Czech Republic, June 11- 13. pp. 421-7.
  180. Sink, DS & Tuttle, TC 1989, Planning and measurement in your organization of the future, Industrial Engineering and Management Press, Norcross, Georgia.
  181. Sivo, SA, Saunders, C, Chang, Q & Jiang, JJ 2006, 'How low should you go? Low response rates and the validity of inference in IS questionnaire research', Journal of the Association for Information Systems, vol. 7, no. 6, pp. 351-414.
  182. Smith, DA 2008, Implementing metrics for IT service management, Van Haren Publishing.
  183. Son, S, Weitzel, T & Laurent, F 2005, 'Designing a process-oriented framework for IT performance management systems', The Electronic Journal of Information Systems Evaluation, vol. 8, no. 3, pp. 219-28.
  184. Spohrer, J & Riecken, D 2006, 'Introduction', Communications of the ACM, vol. 49, no. 7, p. 30.
  185. Spremic, M, Zmirak, Z & Kraljevic, K 2008, 'IT and business process performance management: case study of ITIL implementation in finance service industry', Proceedings of the ITI 2008 30th International Conference on Information Technology Interfaces, Cavtat, June 23-26. pp. 243-50.
  186. Steinberg, RA 2006, Measuring ITIL: measuring, reporting and modeling the IT service management metrics that matter most to IT senior executives, Trafford, Victoria, B.C. Sveiby, KE 1997, The new organizational wealth: managing & measuring knowledge-based assets, 1st edn, Berrett-Koehler Publishers, San Francisco.
  187. Tan, W, Cater-Steel, A & Toleman, M 2010, 'Implementing IT service management: a case study focusing on critical success factors', Journal of Computer Information Systems, vol. 50, no. 2, pp. 1-12.
  188. Tangen, S 2004, 'Performance measurement: from philosophy to practice', International Journal of Productivity and Performance Management, vol. 53, no. 8, pp. 726-37.
  189. ----2005, 'Demystifying productivity and performance', International Journal of Productivity and Performance Management, vol. 54, no. 1, pp. 34-46.
  190. Tellis, W 1997, Introduction to case study, viewed 21 September 2010, <http://www.nova.edu/ssss/QR/QR3-2/tellis1.html%3E.
  191. Tiong, C, Cater-Steel, A & Tan, W-G 2009, 'Measuring return on investment from implementing ITIL: a review of the literature', in A Cater-Steel (ed.), Information Technology Governance and Service Management: Frameworks and Adaptations, Information Science Reference (IGI Global), Hershey PA, USA, pp. 408-22.
  192. Tranfield, D, Denyer, D & Smart, P 2003, 'Towards a methodology for developing evidence-informed management knowledge by means of systematic review', British Journal of Management, vol. 14, no. 3, pp. 207-22.
  193. U.S. General Accounting Office, Content analysis: a methodology for structuring and analyzing written material, 1996, USGA Office, Program Evaluation and Methodology Division, Washington, D.C. Umanath, NS 2003, 'The concept of contingency beyond "It depends": illustrations from IS research stream', Information & Management, vol. 40, no. 6, pp. 551-62.
  194. Valdés, O, St-Jean, M, Renault, A, Picard, M, Cortina, S, Betry, V & Barafort, B 2009, ITSM process assessment supporting ITIL, Van Haren Publishing, Amersfoot.
  195. Valiente, MC, Garcia-Barriocanal, E & Sicilia, MA 2012, 'Applying an ontology approach to IT service management for business-IT integration', Knowledge-Based Systems, vol. 28, no. 0, pp. 76-87.
  196. Van Bon, J 2007, Foundations of IT service management based on ITIL v3, 1st edn, Van Haren Publishing, Zaltbommel, Netherlands.
  197. Van Bon, J & Van Selm, L 2008, ISO/IEC 20000: an introduction, 1st edn, Van Haren Publishing, Zaltbommel.
  198. Van der Zee, JTM & de Jong, B 1999, 'Alignment is not enough: integrating business and information technology management with the balanced business scorecard', Journal of Management Information Systems, vol. 16, no. 2, pp. 137-56.
  199. Van Grembergen, W 2000, 'The balanced scorecard and IT governance', Proceedings of the 2000 Information Resources Management Association International Conference on Challenges of Information Technology Management in the 21st Century. pp. 1123-4.
  200. Van Grembergen, W, De Haes, W & Amelinckx, SI 2003, 'Using COBIT and the balanced scorecard as instruments for service level management', Information Systems Control Journal, vol. 4, no. , pp. 56 -62.
  201. Vargo, S & Lusch, R 2004a, 'The Four Service Marketing Myths: Remnants of a Goods-Based, Manufacturing Model', Journal of Service Research, vol. 6, no. 4, pp. 324-35.
  202. ----2004b, 'Evolving to a new dominant logic for marketing', Journal of Marketing, vol. 68, no. 1, pp. 1-17.
  203. Vessey, I, Ramesh, V & Glass, RL 2002, 'Research in information systems: An empirical study of diversity in the discipline and its journals', Journal of Management Information Systems, vol. 19, no. 2, pp. 129-74.
  204. Wagner, H 2006, 'Managing the impact of IT on firm success: the link between the resource-based view and the IT infrastructure library', Proceedings of the 39th Annual Hawaii International Conference on System Sciences, HICSS '06., Kauai, Hawaii, USA, January 4-6. pp. 1-10.
  205. Walsham, G 1995, 'Interpretive Case-Studies in Is Research -Nature and Method', European Journal of Information Systems, vol. 4, no. 2, pp. 74-81.
  206. Ward, J & Daniel, E 2006, Benefits management: delivering value from IS & IT investments, Wiley, Hoboken, New Jersey.
  207. Weber, RP 1990a, Basic content analysis, 2nd edn, Sage Publications, Newbury Park, Calif.
  208. ----1990b, Basic content analysis, 2nd edn, Sage university papers, Sage, Newbury Park, California.
  209. Webster, J & Watson, RT 2002, 'Analyzing the past to prepare for the future: writing a literature review', MIS Quarterly, vol. 26, no. 2, pp. 13-23.
  210. Weill, P 1992, 'The Relationship Between Investment in Information Technology and Firm Performance: A Study of the Valve Manufacturing Sector', Information Systems Research, vol. 3, no. 4, pp. 307-33.
  211. Weill, P & Olson, M 1989, 'An assessment of the contingency theory of management information systems', Journal of Management Information Systems, vol. 6, no. 1, pp. 59-85.
  212. Winniford, M, Conger, S & Erickson-Harris, L 2009, 'Confusion in the Ranks: IT Service Management Practice and Terminology', Information Systems Management, vol. 26, no. 2, pp. 153-63.
  213. Yin, RK 2009, Case study research: design and methods, 4th edn, Sage, Beverly Hills, California.
  214. Yixin, D & Bhattacharya, K 2008, 'Estimating business value of IT services through process complexity analysis', Proceedings of the 12th IEEE Network Operations and Management Symposium, NOMS, Salvador, Bahia, Brazil, April 7-11. pp. 208-15.
  215. Yuthas, K & Young, ST 1998, 'Material matters: Assessing the effectiveness of materials management IS', Information & Management, vol. 33, no. 3, pp. 115-24.
  216. Zuboff, S 1988, In the age of the smart machine: the future of work and power, Basic Books, New York.
  217. B2. Can you give me examples of metrics that you use related to ITSM? B.3 Does your IT organisation (division, department, unit, team) maintain a catalogue/dictionary of current ITSM performance metrics? Please provide me with the metrics (service) catalogue (definitions and attributes).
  218. B.4 What factors (industry, size, ownership, legislation, competitors, customers, historical) influenced the selection of the metrics used to measure the performance of ITSM? B.5 How are the metrics organised (for example areas of focus such as people, process, partners and technology) within the ITSM performance measurement framework? Why are the metrics organised in this way? B.6 Are the metrics classified into categories for example lead vs. lag, qualitative vs. quantitative, internal vs. external, financial vs. nonfinancial? Why do you use this specific categorisation? Please provide documentation of the various classifications? B.7 Who is involved in collecting metrics and at what levels of the organisations are they positioned? Please provide me with an organisation chart.
  219. B.8 Do you review the effectiveness of the ITSM performance metrics collected? B.9 Do you think that some of the metrics are more effective than others? Why? Table D.3 Publications from the study Journal Articles (
  220. Gacenga, F, Cater-Steel, A & Toleman, M 2010, 'An International analysis of IT service management benefits and performance measurement', Journal of Global Information Technology Management, vol. 13, no. 4, pp. 28-63.
  221. Gacenga, F, Cater -Steel, A, Toleman, M & Tan, W-G 2011, 'Measuring the performance of service orientated IT management', Sprouts: working papers on information environments, systems and organisations, vol. 11, no. 162.
  222. Gacenga, F, Cater-Steel, AP & Toleman, M 2011, 'Measuring the performance of IT service management', Communications of Global Information Technology, vol. 3, pp. 13-25.
  223. Gacenga, F, Cater -Steel, A, Toleman, M & Tan, W-G 2012, 'A proposal and evaluation of a design method in design science research', The Electronic Journal of Business Research Methods, vol. 10, no. 2, pp. 89-100. Book Chapter
  224. Gacenga, F, Cater -Steel, A, Toleman, M & Tan, W-G 2012, 'Measuring IT service management performance: a model development', in Z Belkhamza & SA Wafa (eds), Measuring Organisational Information Systems Success: New Technologies and Practices, Business Science Reference (an imprint of IGI Global), Hershey PA, pp. 102-19. Conference Papers
  225. Gacenga, F & Cater -Steel, A 2011, Performance measurement of IT service management: a case study of an Australian university, Brisbane, Australia, July 8-11, SBN: [978-1-86435-644-1];
  226. Gacenga, F, Cater-Steel, A & Toleman, M 2011, 'Measuring the performance of IT service management', paper presented to 12th Global Information Technology Management Association World Conference, Las Vegas, USA, 5 -7 June.
  227. Gacenga, F, Cater -Steel, A, Tan, W-G & Toleman, M 2011, IT service management: towards a contingency theory of performance measurement, Shanghai, December 4-7, 978-0-615-55907-0, http://aisel.aisnet.org/icis2011/proceedings/servicescience/5.
  228. Gacenga, F, Cater -Steel, A, Tan, W-G & Toleman, M 2012, Alleviating Design Silence in Design Science Research: a Proposal of a Design Method, Academic Publishing International, University of Bolton, UK, 28-29 June 2012, <http://www.academic-bookshop.com/ourshop/prod\_2105844-lt095gtECRM-2012-Proceedings-of-the-11th-European- Conference-on-Research-Methodology-for-Business-and-Management-Studies-Bolton-UK-Print-version.html>.
  229. Gacenga, F, Cater-Steel, AP & Tan, W-G 2011, Towards a framework and contingency theory for performance measurement: a mixed-method approach, Brisbane, Australia, July 8-11, ISBN: [978-1-86435-644-1];
  230. Gacenga, F & Cater -Steel, A 2010, 'Delivering value through IT service management metrics', Informed Intelligence, Bulletin of itSMF Australia, Summer, pp. 8-9.
  231. Gacenga, F, Cater -Steel, A & Toleman, M 2011, 'Cut once, measure twice: a case study of performance measurement practices', Informed Intelligence, Bulletin of itSMF Australia, Autumn.
  232. Gacenga, F & Cater -Steel, A 2010, 'What's your PMF challenge', Informed Intelligence, Bulletin of itSMF Australia, Winter, pp. 14-7. Industry Conference Proceedings (1)
  233. Gacenga, F, Cater-Steel, A, Toleman, M & Tan, W-G 2011, 'Keeping score: measuring ITSM performance', paper presented to 14th National Conference of IT Service Management Forum (itSMF) Australia, Perth, Australia, 17-19 August. Industry Forum Presentation (
  234. Cater-Steel, A, Kolbe, LM, Marrone, M & Gacenga, F 2010, Achieving value through IT service management transformation: an international perspective, itSMF Australia, State Library of Queensland, Brisbane, December, http://www.itsmf.org.au/modules/resource/file.php?id=699.