Measuring research impact: beyond the ivory tower

Beyond the impact factor, new metrics called “altmetrics” were introduced to survey whether works are talked about in the media and online. Our colleague Linda Leger tells you what you should know about them in the following article.

Altmetrics or alternative metrics are a new generation of statistical measures used to evaluate scientific production. They appeared with the emergence of the Open science movement, as researchers started using different ways to share their work online.

These metrics measure the attention a resource generates on the web, which allows estimating research impact beyond the scholarly community. The objective of these metrics is not to replace more traditional options but rather to complement them.

What do they measure?

Altmetric indicators can be sorted in 6 categories:

  1. Publications, which refer to the number of documents accessible to the public
  2. Citations, provided by several databases and platforms as Web of Sciences, Scopus, ResearchGate, Academia.edu, REPEC, etc
  3. Usage, which includes visualisation of the abstracts or documents, downloads, and number of captures in bookmarking tools such as Diigo or references tools such as Mendeley
  4. Dissemination, comments or discussion indicators measure the number of times a document has been shared, commented on, mentioned or retweeted
  5. Rating includes actions where users emit a value judgment about a publication through a like or recommendation
  6. Social connectivity refers to what extent a researcher is connected with the rest of the scientific, academic or  professional community. It is measured by the number of followers or contacts

With which tools?

You can obtain altmetric indicators directly from social bookmarking platforms such as Diigo, reference manager platforms such as Mendeley, social networks platforms such as Facebook, blogs and microblogging, online digital libraries or repositories such as Scopus or SSRN; you can also consult altmetric aggregators which harvest information from varied platforms. The most widely known free services are Altmetric.com and Impactstory.

Altmetric.com offers a free bookmarklet for researchers, and Institutional repository-embeddable badges for free.

Impactstory is a free web tool that only requires registration to produce personal reports with metrics.

Figures 1-3: Using the Altmetrics bookmarklet

Benefits and issues

Altmetrics are a good way to know what others think about your research and to document the attention it gathered on the web. In comparison with traditional bibliometrics:

  • They take less time to be calculated, as altmetric indicators can be measured as soon as a work becomes public.
  • They cover more diverse research outputs such as theses, dissertations, teaching materials and not only articles
  • They include a broader non-academic audience
  • They also have the advantage of being openly accessible

Nevertheless, altmetrics face several challenges such as:

  • a higher risk of manipulation
  • the invisibility of some research outputs as the more accessible publications are more cited
  • the lack of stability of data sources as some of them disappear as soon as they appear
  • data quality issues

These metrics can not always measure the impact or quality of a research: the most controversial articles are widely cited because they attract more attention, and not because of their scientific quality.

Do not forget that altmetric indicators need to be completed by the analysis of the context in which they evolve and cannot replace evaluation methods such as informed peer review. Besides, altmetrics are relatively new and more research is needed to better use them.


More information about altmetrics and research evaluation is available on our dedicated guide.

Cover picture (cropped): Social Media, by Geralt (Pixabay CC0)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.