Multidimensional impact assessment of a large collection of books using PlumX: methodology, technical limitations and indicators analysis [Complementary material to manuscript] (original) (raw)
Related papers
Journal of Informetrics, 2021
The surge in the number of books published makes the manual evaluation methods (e.g. peer review) difficult to efficiently evaluate books. The use of books' citations and alternative evaluation metrics (e.g. library holdings, social media mentions, book reviews) can assist manual evaluation and reduce the cost of evaluation. However, most existing evaluation research was based on a single evaluation source with coarse-grained analysis, which may obtain incomprehensive or one-sided evaluation results of book impact. Meanwhile, relying on a single resource for book assessment may lead to the risk that the evaluation results cannot be obtained due to the lack of the evaluation data, especially for newly published books. Hence, this paper measured book impact based on an evaluation system constructed by integrating multiple evaluation sources. Specifically, we conducted finer-grained mining on the multiple evaluation sources, including books' internal evaluation resources (e.g. books' contents) and external evaluation resources (e.g. books' reviews, books' citations and books' usages). Various technologies (e.g. topic extraction, sentiment analysis, text classification) were used to extract corresponding evaluation metrics from the internal and external evaluation resources. Then, Expert evaluation combined with analytic hierarchy process was used to integrate the evaluation metrics and construct a book impact evaluation system. Finally, the reliability of the evaluation system was verified by comparing with the results of expert evaluation, detailed and diversified evaluation results were then obtained. The experimental results reveal that differential evaluation resources can measure the books' impacts from different dimensions, and the integration of multiple evaluation data can assess books more comprehensively. Meanwhile, the book impact evaluation system can provide personalized evaluation results according to the users' evaluation purposes. In addition, the disciplinary differences should be considered for assessing books' impacts.
Book impact assessment: A quantitative and text-based exploratory analysis
Journal of Intelligent & Fuzzy Systems, 2018
Books are an important source of knowledge to disseminate information. Researchers and academicians write books to propagate their innovative research or teachings amongst academic as well as non-academic audience. The number of books written every year is increasing rapidly. According to International Publisher Association (IPA) annual report 2015-2016, around 150 million different books were published worldwide in 2014-2015. Many e-commerce websites are also involved in selling books. A recent addition to book publishing world is e-books, which have really made it very simple to publish. While, availability of large number of books is good for readers, at the same time it is challenging to find a good book, particularly in scholarly settings. Researchers in the area of Scientometrics have attempted to view assessment of goodness of a scholarly book by measuring citations that a book receive. However, citations alone are not a true measure of a book's impact. Many a times people use the knowledge in a book without actually citing it. Also use of books in classroom settings or for general reading often is not reflected in terms of citations. Therefore, it is important to obtain users's opinion about a book from other forms of data. Fortunately, we have now some data of this sort available in form of reviews, downloads and social media mentions etc. Amazon and Goodreads, both of which provide the readers' views about a book, are two good examples. This paper presents an exploratory research work on using these non-traditional data about books to assess impact of a book. A set of Scopus-indexed computer science books with good citations as well as some other popular books in computer science domain are used for analysis. The reviews of books have been crawled in an automated fashion from Amazon and Goodreads. Thereafter sentiment analysis is carried out the text of reviews. Results of sentiment analysis are compared and correlated with traditional impact assessment metrics. The experimental analysis does not show a coherent relationship between citation and online reviews. Also, majority of the online reviews are found to be positive for large number of books in the dataset. As a related exercise, the Scopus citation data and Google scholar citation data for books are also compared. A high value of correlation is observed in these two. Overall the exploratory analysis provides a useful insight into the problem of book impact assessment.
Metric assessments of books as families of works
Journal of the Association for Information Science and Technology, 2017
We describe the intellectual and physical properties of books as manifestations, expressions, and works and assess the current indexing and metadata structure of monographs in the Book Citation Index (BKCI). Our focus is on the interrelationship of these properties in light of the Functional Requirements for Bibliographic Records (FRBR). Data pertaining to monographs were collected from the Danish PURE repository system as well as the BKCI (2005-2015) via their International Standard Book Numbers (ISBNs). Each ISBN was then matched to the same ISBN and family-related ISBNs cataloged in two additional databases: OCLC-WorldCat and Goodreads. With the retrieval of all family-related ISBNs, we were able to determine the number of monograph expressions present in the BKCI and their collective relationship to one work. Our results show that the majority of missing expressions from the BKCI are emblematic (i.e., first editions of monographs) and that both the indexing and metadata structure of this commercial database could significantly improve with the introduction of distinct expression IDs (i.e., for every distinct edition) and unifying work-related IDs. This improved metadata structure would support the collection of more accurate publication and citation counts for monographs and has implications for developing new indicators based on bibliographic levels.
Alternative Sources of Information to Support a Set of Statistical Indicators of the Book Publishing
International Journal of Innovative Technologies in Economy
Issues of supporting the book publishing as an economic activity by a set of statistical indicators are investigated. It is found out that the existing set of statistical indicators does not meet the needs of researchers and practitioners, which is the case of not only Ukraine, but the global book publishing area. The case of the Ukrainian book publishing is taken for analysis to identify core problems faced by this industry. It is emphasized that a comprehensive study of the book publishing industry and presentation of the statistical information with high level of quality and aggregation requires the involvement of new alternative sources of data, of which big data should be highlighted. The component of scientific novelty is that an updated system of statistical indicators is proposed for the first time, with eight modules of sources of statistical information as alternative ones: questionnaires, electronic books, digital libraries, websites of publishers and bookstores, electron...
Scientometrics, 2021
Scholarly books are important outputs in some fields and their many publishing formats seem to introduce opportunities to scrutinize their impact. As there is a growing interest in the publisher-enforced massive collection of ebooks in libraries in the past decade, this study examined how this influences the relationship that library print holdings (LPH), library electronic holdings (LEH) and total library holdings (TLH) have with other metrics. As a follow up study to a previous research on OCLC library holdings, the relationship between library holdings and twelve other metrics including Scopus Citations, Google Books (GB) Citations, Goodreads engagements, and Altmetric indicators were examined for 119,794 Scopus-indexed book titles across 26 fields. Present study confirms the weak correlation levels observed between TLH and other indicators in previous studies and contributes additional evidence that print holdings can moderately reflect research, educational and online impact of books consistently more efficient than eholdings and total holdings across fields and over time, except for Mendeley for which eholdings slightly prevailed. Regression models indicated that along with other dimensions, Google Books Citations frequently best explained LPH (in 14 out of 26 fields), whereas Goodreads User counts were weak, but the best predictor of both LEH and TLH (in 15 fields out of 26), suggesting significant association of eholdings with online uptake of books. Overall, findings suggest that inclusion of eholdings overrides the more impactful counts of print holdings in Total Library Holdings metric and therefore undermines the statistical results, whilst print holdings has both statistically and theoretically promising underlying assumptions for prediction of impact of books and shows greater promise than the general Library Holding metric for book impact assessment. Thus, there is a need for a distinction between print and electronic holding counts to be made, otherwise total library holding data need to be interpreted with caution.
Measuring the impact: Springer Book Archives at Melbourne
Collection Building
Purpose The purpose of this paper is to examine usage trends during the first four years of the implementation of the Springer Book Archives (SBA) at the University of Melbourne. The paper assesses the benefits of the SBA against perceptions at the time of purchase and seeks to evaluate the long-term value of the purchase. Design/methodology/approach The methodology included a literature search to identify issues in the adoption of large backlists of ebooks, examination of detailed usage data supplied in COUNTER complaint spreadsheets and tables by Springer, validating findings with librarians and academics and positing next steps. Findings Usage of ebooks, like other electronic resources, is difficult to predict. Resources expected to be used, may not be and vice versa. Access to large aggregations of electronic content creates new opportunities for teaching and research, additional economies and benefits, as well as unexpected outcomes. Research limitations/implications Detailed d...
Scientometrics, 2017
More than five years after their emergence, altmetrics are still seen as a promise to complement traditional citation-based indicators. However, no study has focused on their potential usefulness to capture the impact of scholarly books. While recent literature shows that citation indicators cannot fully capture the impact of books, other studies have suggested alternative indicators such as usage, publishers' prestige or library holdings. In this paper, we calculate 18 indicators which range from altmetrics to library holdings, views, downloads or citations to the production of monographs of a Spanish university using the bibliometric suite PlumX from EBSCO. The objective of the study is to adopt a multidimensional perspective on the analysis of books and understand the level of complementarity between these different indicators. Also, we compare the overview offered by this range of indicators when applied to monographs with the traditional bibliometric perspective focused on journal articles and citation impact. We observe a low presence of altmetric indicators for monographs, even lower than for journal articles and a predominance of library holdings, confirming this indicator as the most promising one towards the analysis of the impact of books.
The insoluble problems of books: what does Altmetric.com have to offer?
Aslib Journal of Information Management
Purpose The purpose of this paper is to analyze the capabilities, functionalities and appropriateness of Altmetric.com as a data source for the bibliometric analysis of books in comparison to PlumX. Design/methodology/approach The authors perform an exploratory analysis on the metrics the Altmetric Explorer for Institutions, platform offers for books. The authors use two distinct data sets of books. On the one hand, the authors analyze the Book Collection included in Altmetric.com. On the other hand, the authors use Clarivate’s Master Book List, to analyze Altmetric.com’s capabilities to download and merge data with external databases. Finally, the authors compare the findings with those obtained in a previous study performed in PlumX. Findings Altmetric.com combines and orderly tracks a set of data sources combined by DOI identifiers to retrieve metadata from books, being Google Books its main provider. It also retrieves information from commercial publishers and from some Open Acc...
Collection Building, 2012
Purpose-This paper seeks to determine the extent of e-book availability and how existing e-books might meet library needs as defined by monograph circulation. Design/methodology/approach-This study identified highly circulated print titles and searched for e-book versions available from aggregators or publishers. Findings-The results indicate inadequate coverage levels to allow for a discontinuance of print purchasing, but offer clues as to where e-books might have the greatest impact in this library's collection plans. Originality/value-The paper introduces a method whereby libraries may determine where e-book collections best meets user needs.