NLM® Resources for Informing Comparative Effectiveness Research
Federal Initiatives
The American Recovery and Reinvestment Act (ARRA) of 2009 created the [link removed] Federal Coordinating Council for Comparative Effectiveness Research to coordinate comparative effectiveness research across the federal government. The ARRA provides for $1.1 billion in funding for Comparative Effectiveness Research (CER) primarily through the Agency for Healthcare Research and Quality (AHRQ), the National Institutes of Health (NIH), and the Office of the Secretary of the Department of Health and Human Services (OS).
The Council defines CER as the:
- Conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat, and monitor health conditions in "real world" settings. The purpose of this research is to improve health by developing and disseminating information to patients, providers, and decision-makers, responding to their expressed needs about which interventions are most effective for which patients under specific circumstances. To provide this information, comparative effectiveness research must assess a comprehensive array of health-related outcomes for diverse patient populations and sub-groups. Defined interventions compared may include medications, procedures, medical and assistive devices, and technologies, diagnostic testing, behavioral change, and delivery system strategies. This research necessitates the development, expansion, and use of a variety of data sources and methods to assess comparative effectiveness and actively disseminate the results.
NLM Response
In response to ARRA, the National Library of Medicine® (NLM) has identified or developed new resources in the following domains: new MeSH® (Medical Subject Headings) terms, new coordinated search queries, and Webinars for the library/information intermediary community.
Additions to MeSH
NLM introduced the following MeSH terms in 2010 to assist in identification and retrieval of citations relating to ARRA, CER, and related research support:
- American Recovery and Reinvestment Act
Public Law No: 111-5, enacted February 2009, makes supplemental appropriations for job preservation and creation, infrastructure investment, energy efficiency and science, assistance to the unemployed, and State and local fiscal stabilization, for fiscal year ending September 30, 2009.
Year introduced: 2010 - Comparative Effectiveness Research
Conduct and synthesis of systematic research comparing interventions and strategies to prevent, diagnose, treat, and monitor health conditions. The purpose of this research is to inform patients, providers, and decision-makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances. ([link removed] hhs.gov/recovery/programs/cer/draftdefinition.html accessed 6/12/2009)
Year introduced: 2010 -
Research Support, American Recovery and Reinvestment Act [Publication Type]
Year introduced: 2010
New Coordinated Search Queries for PubMed®, Health Services Research Projects in Progress (HSRProj), and ClinicalTrials.gov
The NLM National Information Center on Health Services Research and Health Care Technology (NICHSR) is responsible for developing information services for the research and library communities involved in health services research and for coordinating clinical terminology and health data standards within the Department of Health and Human Services. Building on a recent initial analysis of the field of CER by AcademyHealth and the excellent work of information professionals who support the AHRQ Effective Health Care program as well as the work of the InterTASC Information Specialists' Sub-Group and existing PubMed queries, search filters to identify studies, both on-going and completed, for informing comparative effectiveness research in the NLM ClinicalTrials.gov, HSRProj, and PubMed databases have been developed (see Figure 1). This approach not only supports queries of the published literature, but also supports examination of current funding trends and research activities in the health and health services fields.
The URL for accessing these search filters is: //www.nlm.nih.gov/nichsr/cer/cerqueries.html. A link is also available from the Special Queries page.
The audience for these search filters includes:
- Policy/Decision Makers — to help inform CER
- Practitioners — to help increase awareness of CER
- Health Services Researchers — to help frame CER questions
- Librarians/Information Specialists — to assist in systematic review and summarization efforts and in teaching students and users about information resources
PubMed CER Searches
The study designs used for conducting comparative effectiveness research in practice settings are evolving and at present there are few studies conducted that have wide applicability beyond the settings in which they are based. Efficacy studies are generally at one end of the continuum of clinical study designs and are conducted in the "ideal world" in carefully selected patients or populations. Effectiveness studies are conducted under real conditions and involve patients seen in physician or hospital settings or are from specific populations. The purpose of the CER search strategies is to help inform discussions on or relating to comparative effectiveness by providing retrieval of a full-range of studies on the efficacy-effectiveness-comparative effectiveness in the "real world" continuum. Additionally, CER employs a range of research data methods and sources, e.g., randomized controlled trials (RCTs), observational studies (including natural experiments, disease registries, administrative data, and electronic health records (EHR)), and secondary research such as systematic reviews, meta-analyses and mathematical models, as shown in the conceptual model below (see Figure 2).
To use the PubMed CER search queries, select a Research or Selected Topics category, enter a word or phrase in the search box, and click on the Go button (see Figure 3).
Because federal efforts on CER focus on priority populations (populations underrepresented in clinical research), the PubMed special query search on Health Disparities has also been included, as have topic searches on Costs and Cost Analysis and citations "about" Comparative Effectiveness Research. Figure 3 shows the search box and the research categories and topic selections for PubMed. After running a search, the PubMed Limits, e.g., English language, can be easily applied to the retrieval if desired. Note also that the PubMed History feature can be used to combine search results and that the All search selection covers all research categories and topics combined.
An alternate search technique using a simplified one-word search of effectiveness, with the search tag, [sb], which corresponds to the All search category has been developed and can be used singly or in combination with other terms and/or MeSH headings when searching PubMed directly.
- Example: atrial fibrillation AND effectiveness [sb]
On-going or Recently Completed Research for CER
HSRProj is a database of research project descriptions relating to access, costs, and quality issues in health care and public health. ClinicalTrials.gov contains descriptions of on-going or recently completed clinical trials and related research. Search boxes for HSRProj and ClinicalTrials.gov appear below the PubMed search area (see Figure 4). To use the CER search queries for HSRProj or for ClinicalTrials.gov, go to the respective search box, enter a word or phrase, and click the Go button.
Strategies Used for the Searches
- Clinical effectiveness studies cover a range of study designs with wide variation in terminology; they may be drawn from a variety of data sources. For this reason, the Clinical Studies and Systematic Reviews search strategies have been designed for sensitivity (that is, searches will retrieve many relevant citations but probably a few less relevant ones, too) while the Observational Studies search strategy has been designed for specificity (that is, searches will retrieve mostly relevant citations but may not include all citations that could be potentially relevant to a particular topic).
- The Clinical Studies search strategy has been developed to retrieve citations that discuss comparisons of treatments OR that discuss treatment outcomes.
- The Observational Studies search strategy has been developed to retrieve citations that discuss comparisons of treatments AND that also discuss treatment outcomes.
- The CER Systematic Reviews, Simulations, Models strategy uses and expands on the existing PubMed Systematic Reviews search strategy to include simulations and models.
- The Health Disparities search uses the existing PubMed Health Disparities search strategy.
- The Costs and Cost Analysis search strategy is based on the Health Services Research query on Costs (Broad scope). It has been expanded beyond title and abstract words to include all text words.
- The search strategy on Comparative Effectiveness Research as Topic has been designed to retrieve citations "about" CER.
- The search strategy for HSRProj includes Program Evaluation studies in addition to health services research studies focusing on CER.
- The search strategy for ClinicalTrials.gov includes Phase III or IV clinical trials or observational studies with an active comparator.
- The search strategies discussed above are available here.
Note:
Indexing status affects retrieval. The PubMed CER searches have not been designed to retrieve in-process citations. This means that searches may not retrieve some relevant citations until they are indexed. Work to expand the searches will be undertaken in the future.
Webinar and Feedback
NICHSR, working with AcademyHealth and the Medical Library Association, has developed a Webinar that provides details about the CER search queries, including sample searches. The archived version of the Webinar is available from http://www.academyhealth.org/Training/ResourceDetail.cfm?ItemNumber=4792&navItemNumber=2102.
The field of comparative effectiveness research is evolving. These search strategies have been developed as an initial starting point for people interested in CER. We hope they will be useful to help identify and gauge the volume of literature, to help frame questions, and until more advanced enhancements and mechanisms are in place to readily identify research about "what works, for whom, and under what conditions." We welcome feedback on the CER search queries and provide a short online feedback form from the CER search page. The queries will continue to have this feedback feature so that the searches can be refined as the field and its needs develop and evolve.
Work Cited:
Federal Coordinating Council for Comparative Effectiveness Research. Report to the President and the Congress, June 30, 2009. [Internet]. Washington, DC: The Council; 2009 [cited 2009 Oct. 14]. 73 p. [link removed] http://www.hhs.gov/recovery/programs/cer/cerannualrpt.pdf
Holve, E. and P. Pittman. A first look at the volume and cost of comparative effectiveness research in the United States [Internet]. Washington, DC: AcademyHealth; 2009 June [cited 2009 Oct. 14]. 20 p. http://www.academyhealth.org/files/FileDownloads/AH_Monograph_09FINAL7.pdf
For further discussion see also:
Knecht LS, Marcetich J. Indexing the American Recovery and Reinvestment Act (ARRA) for MEDLINE®. NLM Tech Bull. 2009 Mar-Apr;(367):e14. //www.nlm.nih.gov/pubs/techbull/ma09/ma09_arra_tech_memo.html
Florance V. American Recovery and Reinvestment Act (ARRA) of 2009. NLM Tech Bull. 2009 Mar-Apr;(367):e2 . //www.nlm.nih.gov/pubs/techbull/ma09/ma09_arra.html
Auston I. NLM® Resources for Informing Comparative Effectiveness Research. NLM Tech Bull. 2010 Mar-Apr;(373):e17.
