Contact
Related Guides
Overview
Bibliographic research metrics, quantitative metrics rooted in publication counts, have increasingly been used and sometimes misused to make claims about the impact, quality, and prestige of research, publications, journals, and researchers’ work. Many quantitative research impact indicators are available, but there can be mismatch between what is meant to be assessed—and the indicator used for assessment. This guide is intended to steer you to resources available for equitable evaluation of scholarly work, particularly in the wake of unpredictable challenges.
Statements on Responsible Metrics
The following resources describe how and why researchers and evaluators should take responsible metrics into account when assessing the impact of research:
- The Leiden ManifestoTen principles emanating from the International Conference on Science and Technology Indicators (Leiden University, Netherlands). Encourages the use of robust quantitative and qualitative data "with sensitivity to the aim and nature of the research that is evaluated."
- The San Francisco Declaration on Research Assessment (DORA)DORA has gained attention in the research community in recent years for its criticism of the Impact Factor.
Frameworks for Responsible Evaluation of Research
The following resources outline specific approaches that one might take to responsibly evaluate research or a researcher. Most of these approaches are specific to a particular purpose or discipline. Not every resource will be appropriate in every case. These are examples of ways in which responsible research impact assessment might be put into practice in various contexts.
- Assessing and Communicating the Value of Biomedical Research (2016)Developed by the Association of American Medical Colleges
- Becker Medical Library Model for Assessment of Research Impact (2014)Provides a wealth of ideas for assessing your research impact broadly, considering a range of quantitative and qualitative data sources. It includes a list of indicators for documenting evidence of impact, and an established framework to aid in your information gathering process.
- Guides to the Responsible Use of SciVal and InCites (2020)Developed by members of the LIS-Bibliometrics Committee
- The Hong Kong Principles for assessing researchers: Fostering research integrity (2020)The Hong Kong Principles (HKPs) were developed as part of the 6th World Conference on Research Integrity with a specific focus on the need to drive research improvement through ensuring that researchers are explicitly recognized and rewarded for behaviors that strengthen research integrity. There are five principles: responsible research practices; transparent reporting; open science (open research); valuing a diversity of types of research; and recognizing all contributions to research and scholarly activity. For each principle, there is a rationale for its inclusion and examples where these principles are already being adopted.
- HUMetrics HSS Initiative (2016)HuMetricsHSS is an initiative that creates and supports values-enacted frameworks for understanding and evaluating all aspects of the scholarly life well-lived and for promoting the nurturing of these values in scholarly practice. Comprised of individuals working in academic and nonprofit academic–adjacent sectors, the HuMetricsHSS team is committed to establishing humane indicators of excellence in academia, focused particularly on the humanities and social sciences (HSS).
- Making FAIReR Assessments Possible (2021)This report argues for creating FAIReR academic assessments. Such assessments are rooted in both the FAIR guidelines for data management and policies for the responsible assessment of research (FAIReR = FAIR + Responsible). Specifically, FAIReR assessments emphasize diversity, communities, and dialogue. This report is a deliverable of EOSC Co-Creation projects (i) “European overview of career merit systems’’ and (ii) “Vision for research data in research careers”, funded by the EOSC Co-Creation funding.
- Project TARAProject TARA is supported by a generous three-year grant from Arcadia, a charitable fund of Lisbet Rausing and Peter Baldwin. It will help DORA identify, understand, and make visible the criteria and standards universities use to make hiring, promotion, and tenure decisions. This information will be used to create resources and practical guidance on research assessment reform for academic and scholarly institutions.
- Position Paper: Room for Everyone's Talent (2019)Published by the Dutch Research Council (NWO), this paper makes a case for the Council's efforts to more broadly recognize and reward the work of academic staff. This includes placing less emphasis on the number of publications, and a greater emphasis on the other domains in which the academic is active, such as education and impact. This broader form of recognition and appreciation is better suited to the current core tasks of knowledge and educational institutions and what society requires of these establishments.
- Rethinking Research Assessment: Unintended Cognitive & System BiasDeveloped by DORA, Unintended Cognitive and Systems Biases identifies seven personal biases that can influence hiring, promotion, and tenure decisions. It also reveals four institutional and infrastructural implications of these biases and provides strategies to develop new institutional conditions that reduce bias. Additional context for the brief is provided in a four-part blog post series.
- The SCOPE Model (2019)Developed by the The International Network of Research Management Societies (INORMS) Research
Evaluation Working Group (REWG). SCOPE
moves beyond principles for responsible evaluation and provides a five-stage process by which good evaluations
can be implemented. - Translational Science Benefits Model (2019)Developed by the Institute of Clinical and Translational Sciences at the Washington University School of Medicine