Library Contact
For more information or to schedule an individual or group consultation, contact the THL Research Impact Core.
Benefits and Limitations of Impact Metrics
Quantitative publication metrics offer a relatively quick and seemingly concrete measure of research impact and are used widely in the assessment of academic health sciences research. However, metrics can lead to over-simplification and sometimes serve as direct proxies for impact at the expense of other valuable considerations.
Several efforts have led to frameworks that promote the application of assessment metrics as one important aspect of a broader assessment process.
-
The Leiden ManifestoTen principles emanating from the International Conference on Science and Technology Indicators (Leiden University, Netherlands). Encourages the use of robust quantitative and qualitative data "with sensitivity to the aim and nature of the research that is evaluated."
-
The Metric TideRecommendations from a 2014 multidisciplinary study of the United Kingdom research system. Encourages "responsible metrics," with quantitative indicators, expert judgment, and qualitative measures serving as complements to decision making.
-
The San Francisco Declaration on Research Assessment (DORA)DORA has gained attention in the research community in recent years for its critique of the use of Journal Impact Factor in research evaluations.
NEW! Resource
-
ReformscapeDORA is committed to supporting the development of new policies and practices for responsible research assessment (RRA). Reformscape is a searchable collection of criteria and standards for hiring, review, promotion, and tenure from academic institutions.
Assessment Frameworks & Resources
Quantitative citation metrics (such as citation counts) should not be relied upon as the sole indicator of scientific impact, but instead serve as one aspect of a broader evaluation framework including peer review and other qualitative considerations.
The resources below provide examples of additional indicators that can provide insight into research impact beyond qualitative publication metrics.
-
Becker Medical Library Model for Assessment of Research ImpactProvides a wealth of ideas for assessing your research impact broadly, considering a range of quantitative and qualitative data sources. It includes a list of indicators for documenting evidence of impact, and an established framework to aid in your information gathering process.
-
Translational Science Benefit Model ToolkitNine tools informed by the Translational Science Benefits Model to help public health and clinical scientists track the impact of their work across various domains and audiences.
-
"100 Metrics to Assess and Communicate the Value of Biomedical Research" (PDF)This idea book by the AAMC and Rand Europe offers an extensive list of metrics and indicators corresponding to various domains (e.g., policy, health, economic, teaching, research, dissemination).
-
Metrics ToolkitThe Metrics Toolkit is a resource for researchers and evaluators that provides guidance for demonstrating and evaluating claims of research impact.
-
U.S. Centers for Disease Control and Prevention Scientific Impact FrameworkThe Science Impact Project is an adaptation and extension of the Institute of Medicine (IOM) Degrees of Impact framework.
-
Article: A Review of the Characteristics of 108 Author-Level Bibliometric Indicators“A Review of the Characteristics of 108 Author-Level Bibliometric Indicators.” Scientometrics 101, no. 1 (October 1, 2014): https://doi.org/10.1007/s11192-014-1423-3.
-
Article: Assessing the impact of healthcare research: A systematic review of methodological frameworksCruz Rivera S, Kyte DG, Aiyegbusi OL, Keeley TJ, Calvert MJ (2017) Assessing the impact of healthcare research: A systematic review of methodological frameworks. PLOS Medicine 14(8): e1002370. https://doi.org/10.1371/journal.pmed.1002370
-
Canadian Academy of Health Sciences FrameworkThis document proposes a new impacts framework and a preferred menu of indicators and metrics that can be used for evaluating the returns on investment in health research.
-
Introducing SCOPE – a process for evaluating responsiblySCOPE is a model that provides step-by-step guidance as to how to test your existing evaluation procedures or go about establishing new ones.
Example Research Impact Statements
-
The Research Excellence Framework IndexThe REF Index offers browsable impact studies written by researchers for the United Kingdom's Research Excellence Framework, which assesses the quality of research in UK higher education institutions. See "About" for more information.
-
DORA Case Studies ArchiveBrowse case studies highlighting several global university research assessment initiatives.