Assessing Scholarly Influence: Proposing New Metrics (original) (raw)
Related papers
The Hirsch Family of Bibliometric Indices as an Improved Measure of IS Academic Journal Impact
2008
This study examines the use of journal rankings and proposes a new method of measuring IS journal impact based on the Hirsch family of indices . Journal rankings are a very important exercise in academia since they impact tenure and promotion decisions. Current methods employed to rank journal influence are shown to be subjective. We propose that the Hirsch Index and Contemporary Hirsch Index ) based on data from Publish or Perish be adopted as a more objective journal ranking method. To demonstrate the results of using this methodology, it is applied to the "pure MIS" journals ranked by . The authors find substantial differences between the scholar rankings and those obtained using the Hirsch family of indices. They also find that the contemporary Hirsch Index allows researchers to identify journals that are rising or declining in influence.
Use of Hirsch Index for Measuring the Impact of IS Academic Research and Journals
2008
This study examines the use of journal rankings and a relatively new method of measuring impact of research as a surrogate of scholarly impact: the Hirsch Index . Journal rankings are a very important exercise in academia since they impact tenure and promotion decisions. Current methods employed to rank journal influence are shown to be subjective. We propose that the Hirsch Index be adopted as a more objective journal ranking method. To demonstrate the results of using the Hirsch Index, it is applied to the "pure MIS" journals ranked by . The authors find substantial differences between the scholar rankings and those obtained using the Hirsch index. This provides weak support for the current journal ranking system but also suggests that other factors are at play.
Assessing Scholarly Influence: Using the Hirsch Indices to Reframe the Discourse
Journal of the Association for Information Systems, 2009
This study is part of a program aimed at creating measures enabling a fairer and more complete assessment of a scholar’s contribution to a field, thus bringing greater rationality and transparency to the promotion and tenure process. It finds current approaches toward the evaluation of research productivity to be simplistic, atheoretic, and biased toward reinforcing existing reputation and power structures. This study examines the use of the Hirsch family of indices, a robust and theoretically informed metric, as an addition to prior approaches to assessing the scholarly influence of IS researchers. It finds that while the top tier journals are important indications of a scholar’s impact, they are neither the only nor, indeed, the most important sources of scholarly influence. Other ranking studies, by narrowly bounding the venues included in those studies, distort the discourse and effectively privilege certain venues by declaring them to be more highly influential than warranted. The study identifies three different categories of scholars: those who publish primarily in North American journals, those who publish primarily in European journals, and a transnational set of authors who publish in both geographies. Excluding the transnational scholars, for the scholars who published in these journal sets during the period of this analysis, we find that North American scholars tend to be more influential than European scholars, on average. We attribute this difference to a difference in the publication culture of the different geographies. This study also suggests that the influence of authors who publish in the European journal set is concentrated at a moderate level of influence, while the influence of those who publish in the North American journal set is dispersed between those with high influence and those with relatively low influence. Therefore, to be a part of the top European scholar list requires a higher level of influence than to be a part of the top North American scholar list.
Contributors to the High-Impact IS Journals (1977-2014): An Aid for Setting Research Standards
Communications of the Association for Information Systems
Interest in the rankings of contributors to academic literature is evidenced by the numerous publications across most business and economic disciplines. This study presents the most prolific authors 1) over the entire history of the 11 high-impact IS journals and 2) over the ten most recent years for each of the journals. We include the number of authors who have published in the journals but who may not be considered prolific based on our classification; this data is important and especially critical for IS departments that set research standards. Identifying and ranking authors in the IS discipline is interesting for several reasons. While some may be curious to see how they perform compared to these researchers, a more beneficial application of the findings in this paper pertains to establishing realistic promotion and tenure standards. Although 11,204 authors published in the 11 high-impact journals, 7,734 (69%) of those authors published only once in these journals in the 1977-2014 period. This fact is essential for any IS department that sets promotion and tenure guidelines. Using our findings will help colleges and IS departments establish reasonable and attainable promotion and tenure standards based on the actual performance of others in the discipline.
2010
Following previous research findings, this paper argues that the current method of evaluating scholar performance, publication counts in "quality" journals is flawed by subjectivity in generating the list of approved journals and the definition of quality. Truex, sought to improve on this method by substituting the measurement of "influence" using the Hirsch statistics to measure ideational influence. Since the h-family statistics are a measure of productivity and the uptake of a scholar's ideas this methodology privileges the uptake of a scholar's ideas over the venue of publication. But influence is built through other means than by having one's papers read and cited. This paper argues that interaction between scholars resulting in co-authored papers is another way to build academic influence. This academic influence, which we term social influence, can be assessed by Social Network Analysis (SNA) metrics which examine the nature and strength of coauthoring networks among IS Scholars. The paper demonstrates the method of assessing social influence by analysis of the social network of AMCIS scholars and compares the results of this analysis with other coauthorship networks from the ECIS and ICIS community.
AC 2007-687: RANKING SCHOLARLY OUTLETS FOR INFORMATION TECHNOLOGY
2007
Many well-established disciplines have a number of outlets for scholarly work, including archival journals, conference proceedings, periodicals, and others. These outlets are commonly known within the discipline, and many have an established reputation and even ranking. Faculty seeking to publish in one of these disciplines, and seeking to advance in rank and tenure status, are well served by knowing the most common scholarly outlets and their rankings. The relatively new discipline of Information Technology does not yet have a well established ranking of scholarly outlets. This paper presents the findings of a survey conducted among the members of the IT community about their perceptions of the quality of various journals and conference proceedings. The rankings of the 20 publications were not widely separated, ranging from 5.0 (of 5.0) for Computerworld, to 4.07 for Dr. Dobbs Journal. Likewise, the rankings of the seven conference proceedings were also not widely separated, ranging from 4.97 for SIGITE, to 4.33 for SIGMetrics. Scholars and professionals in IT will be well served by this study, which is a first effort to establish the reputation and ranking of scholarly outlets in the IT discipline.
Scholarly Influence Research (SIR)
International Journal of Social and Organizational Dynamics in IT, 2012
Following previous research findings, this paper argues that the currently predominant method of evaluating scholar performance - publication counts in “quality” journals - is flawed due to the subjectivity inherent in the generation of the list of approved journals and absence of a definition of quality. Truex, Cuellar, and Takeda (2009) improved on this method by substituting a measurement of “influence” using the Hirsch statistics to measure ideational influence. Since the h-family statistics are a measure of productivity and the uptake of a scholar’s ideas expressed in publications, this methodology privileges the uptake of a scholar’s ideas over the venue of publication. Influence is built through other means than by having one’s papers read and cited. The interaction between scholars resulting in co-authored papers is another way to build scholarly influence. This aspect of scholarly influence, which the authors term social influence, can be assessed by Social Network Analysis...