The Pros and Cons of the Use of Altmetrics in Research Assessment

• Main Author: Mike Thelwall
• Format: Article
• Language: English
• Published: Levy Library Press 2020-05-01
• Series: Scholarly Assessment Reports
• Subjects: altmetrics; webometrics; alternative indicators; responsible metrics; scientometrics
• Online Access: https://www.scholarlyassessmentreports.org/articles/10

Many indicators derived from the web have been proposed to supplement citation-based indicators in support of research assessments. These indicators, often called altmetrics, are available commercially from Altmetric.com and Elsevier’s Plum Analytics or can be collected directly. These organisations can also deliver altmetrics to support institutional self-evaluations. The potential advantages of altmetrics for research evaluation are that they may reflect important non-academic impacts and may appear before citations when an article is published, thus providing earlier impact evidence. Their disadvantages often include susceptibility to gaming, data sparsity, and difficulties translating the evidence into specific types of impact. Despite these limitations, altmetrics have been widely adopted by publishers, apparently to give authors, editors and readers insights into the level of interest in recently published articles. This article summarises evidence for and against extending the adoption of altmetrics to research evaluations. It argues that whilst systematically-gathered altmetrics are inappropriate for important formal research evaluations, they can play a role in some other contexts. They can be informative when evaluating research units that rarely produce journal articles, when seeking to identify evidence of novel types of impact during institutional or other self-evaluations, and when selected by individuals or groups to support narrative-based non-academic claims. In addition, Mendeley reader counts are uniquely valuable as early (mainly) scholarly impact indicators to replace citations when gaming is not possible and early impact evidence is needed. Organisations using alternative indicators need recruit or develop in-house expertise to ensure that they are not misused, however.   Policy highlights Altmetrics, or alternative indicators for research outputs, have been proposed as a partial solution to two research management problems: (a) assessing the societal impacts of research, and (b) obtaining early impact evidence. This article reviews the evidence and finds limited support for (a) but strong support for (b). Organisations will need to assess whether the value provided by alternative indicators in terms of helping to provide data so support research assessments is sufficient for their financial and time costs. Those using alternative indicators will deed to develop in-house expertise so that they can be used responsibly and interpreted effectively.