Published on March 10, 2014
March 5-6, 2014 Vancouver, British Columbia
CADTH CADTH is an independent, not-for-profit producer and broker of health technology assessments. Federal, provincial, and territorial health care decision-makers rely on our evidence-based information to make informed policy and practice decision about drugs and other health technologies. www.cadth.ca
Health Technologies • Pharmaceuticals • Medical, surgical and dental devices and procedures • Diagnostics
Ann Vosilla CADTH Liaison Officer firstname.lastname@example.org (604) 787-9945
About Us • Amanda Hodgson (email@example.com) Manager, Information Services • Caitlyn Ford (firstname.lastname@example.org) Information Specialist • Our Team Literature searching for rapid reviews, health technology assessments and CADTH Common Drug Review
Objectives PART 1 • Role of the Information Specialist in HTA • Creating your literature search • Applying search filters • Peer review of search strategies BREAK PART 2 • Grey literature searching • Grey Matters and Grey Matters Light • Closing and evaluation
Finding the Evidence • Finding the evidence: literature searching tools in support of systematic reviews http://cadth.ca/en/resources/finding-evidence-is
What is a Systematic Search? “Systematic reviews of interventions require a thorough, objective and reproducible search of a range of sources to identify as many relevant studies as possible (within resource limits). This is a major factor in distinguishing systematic reviews from traditional narrative reviews and helps to minimize bias and therefore assist in achieving reliable estimates of effects.” Higgins et al. Cochrane handbook for systematic reviews of interventions. 2008. Section 220.127.116.11
What is a Systematic Search? “…the aspiration of the HTA literature search should not be comprehensiveness but rather the minimization of bias.” Booth. Int J Technol Assess Health Care. 2010 Oct;26(4):431-5.
Fundamental Challenge of Searching Balancing Recall and Precision = getting "everything" while avoiding "junk" http://www.creighton.edu/fileadmin/user/HSL/docs/ref/Searching_-_Recall_Precision.pdf
How Does This Translate Into Literature Searches? Simple search for “diabetes” in Medline Search Fields Number of Results • Subject heading • Title, abstract, all free-text fields 400,000 + • Title • 2+ times in abstract • Major subject heading 280,000 + • Title • Major subject heading 160,000+
Standards for Systematic Searchers • Finding what works in health care: standards for systematic reviews. Washington D.C.: Institute of Medicine of the National Academies; 2011. See Chapter 3: Standards for Finding and Assessing Individual Studies. Standard 3.1 – • Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions. Version 5.1.0. [Updated March 2011]. The Cochrane Collaboration; 2011. See Part 2: General methods for Cochrane Reviews. Chapter 6: searching for studies.( Authors: Carol Lefebvre, Eric Manheimer and Julie Glanville on behalf of the Cochrane Information Retrieval Methods Group)
Who are “Systematic Searchers”? • Experts in technical and methodological issues of searching • MLIS/MLS/MISt – educational requirements • Work with the research team at every stage of review WHY ARE THEY IMPORTANT? • Help to mitigate potential biases in search • Know search resources and how to use them • Can advise on size and quality of search results
Role of the Information Specialist • Help define scope of review • Perform complex database and grey literature searches • Document the search • Manage citations, ensure quality of references • Perform supplemental searches • Provide advice on copyright & obtaining permissions • Maintain alerts & media scanning
Creating YOUR Literature Search • Understanding your research question • PICO • Scoping • Setting your parameters
Understand Research Question • Work with research team to refine questions – Scoping searches, identify main concepts • Determine PICO • Define search parameters – age, gender, date or language limits, study types • Get suggestions on terminology or sources from subject experts
What is your question? P population I intervention C comparator O outcome
Understand the Research Question Patient Population Adults with type 2 diabetes who have experienced inadequate glycemic control on combination therapy with metformin and a sulfonylurea Intervention Saxagliptin (2.5 mg once daily or 5 mg once daily) in combination with metformin and a sulfonylurea Comparators Other DPP-4 inhibitors, basal insulin, or placebo in combination with metformin and a sulfonylurea Outcomes Key efficacy outcomes • mortality • diabetes-related morbidity (macrovascular, microvascular) • glycemic control (A1C, FBG) • QoL (measured by any validated scale) Harms outcomes hypoglycemia total AEs SAEs WDAEs changes in body weight Other outcomes health care resource utilization Study design Published and unpublished RCTs ≥ 12 weeks duration
Population Intervention Comparison Outcome Intensive Care Units [mh] and others Dexmedetomidine [mh] None None ICU;PICU; intensive care; etc. precedex; dexdor; dexmedotomidin*; 113775-47-6 None None Adapted from:http://www.oxfordradcliffe.nhs.uk/research/ researchers/news/documents/LiteratureSearchingGuidelinesChecklist.pdf Keywords Keywords Keywords Keywords Controlled terms Controlled terms Controlled terms Controlled terms AND O R Translating PICO
When PICO Doesn’t Cut It…
Understand the Research Question • Search Protocol
Translating the Search • CADTH PubMed cheat sheet http://cadth.ca/media/is/Search- dev/PubMed_Cheat_Sheet_external.pdf
Scoping – Finding Terms • Look at indexed articles on a similar topic • Sources for drug names/terms - Chem ID - Pubmed/Medline - Embase • Synonyms in Medical thesauri/dictionaries/online sites • MeSH thesaurus for synonyms and entry terms
Use both controlled vocabulary and keywords Uncontrolled Terms (keywords in PubMed) [ti] or [tiab] or [tw] • Words in title & abstract fields • Synonyms/Acronyms • Spelling variants • Old and new terminology • Brand and generic names • Lay and medical terminology • Singular & plural variations Controlled terms (in PubMed) • MeSH subject headings [mh] or [major] or [mh:NoExp] or [major:NoExp] • Substance name field [nm] • Pharmaceutical Action field [pa] • Registry number field [rn]
Major Medical Databases Medline/PubMed Embase PsycINFO CINAHL Cochrane Central Focus Biomedical and health sciences Biomedical and pharmaceutical database Behavioural sciences and mental health Nursing and allied health Controlled clinical trials Specialization Strong in clinical research; North American focus Strong in pharmaceutical information; European focus; index conference abstracts Includes fields such as psychiatry, management, business, education, social science, neuroscience, law, medicine, and social work Includes fields such as nursing (clinical and theory), nurse education, physiotherapy, occupational therapy Contains only citations of clinical trials; no need to use an RCT filter Thesaurus MeSH EMTREE Thesaurus of Psychological Index Terms CINAHL Headings MeSH Date Coverage Begins 1946 1974 1840 1981 circa 1966 (some earlier) Journals Indexed 5,516 7,500 2,450 3,000 2,400 handsearched Overlap with other databases Yes Yes Yes Yes Yes
Know Database Syntax Need to know indexing and “syntax” for each database searched Example: Subject Heading Search OVID Medline Diabetes Mellitus, Type 2/ OVID Embase Non insulin dependent diabetes mellitus/ PubMed "Diabetes Mellitus, Type 2"[Mesh] CINAHL MH diabetes mellitus, type 2
Visual Example of Typical Search Strategy INDICATION TERMS INTERVENTION TERMS STUDY DESIGN FILTERS
Filters Why use filters? • Reduce the number of irrelevant results • Extract the higher level evidence (i.e. RCTs) • Extract a specific concept (ex. QOL, adverse events) • Remove off-topic results (ex. animals, editorials) • Saves time for IS and researcher
Types of Filters • Specific methodology – RCTs, Systematic reviews • Clinical queries – Diagnosis, Prognosis, Adverse events • Population specific – Humans, Pediatrics • CADTH Filters http://www.cadth.ca/en/resources/finding- evidence-is/string
Filters: Warning • No filter is infallible • Filters are regularly being tested, tweaked and updated but they almost always capture irrelevant records and sometimes exclude relevant ones
RCT filter would pick up this article…
And this article… “Randomized” in title, but not a study:
But not this one. Study design NOT found anywhere:
CADTH Peer Review Checklist for Search Strategies http://www.cadth.ca/en/resources/finding-evidence-is/peer- review-search-strat
CADTH Peer Review Checklist for Search Strategies • History and development • Current checklist • New research on the value of peer review • How can you get your search peer reviewed?
CADTH Peer Review Checklist for Search Strategies • Developed from: Sampson M, McGowan J, Lefebvre C, Moher D, Grimshaw J. PRESS: Peer review of electronic search strategies. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2008. http://www.cadth.ca/media/pdf/477_PRESS-Peer-Review- Electronic-Search-Strategies_tr_e.pdf
CADTH Peer Review Checklist for Search Strategies Key Elements Translation Appropriate search filters Operators Limits Subject headings Fields Natural language Adapted databases Spelling & syntax Additional suggestions/comments
Exercise Sample Searches
Best Practices for Search Strategies 1) Understand research question(s) 2) Use keywords and controlled vocabulary 3) Know database indexing and syntax 4) Use filters where appropriate 5) Have searches peer reviewed 6) Use caution: don’t overcomplicate
What is Grey Literature? • Grey/gray • Unpublished • Non-commercial material • In-house, vertical files • Information on the deep web • “Hard to find” • Fugitive
Grey Literature Definition INTERNATIONAL CONFERENCE ON GREY LITERATURE “Information produced on all levels of government, academics, business and industry in electronic and print formats not controlled by commercial publishing i.e. where publishing is not the primary activity of the producing body.” (Luxembourg, 1997 - Expanded in New York, 2004) http://www.greynet.org/greynethome.html Source: GreyNet: grey literature network service. Amsterdam: GreyNet; 2008.
Examples of Grey Literature • Government Documents • Policy Documents • Clinical Guidelines • Conference Abstracts/Proceedings/Papers • Open Access Journals (some un-indexed) • Health Technology Assessments • Personal Communication • Clinical Trials
Why is Searching Grey Literature Important? • Essential component of a comprehensive search • Overcomes bias of commercial publications • Part of evidence base • Most HTAs created for healthcare decision-makers are considered “grey literature” and are not found in traditional databases
Publication Bias • “Exclusion of grey literature from meta-analyses can lead to exaggerated estimates of intervention effectiveness” McAuley et al. Lancet 2000; 356(Oct 7): 1228-31. • “Published literature tend to be larger and show larger effects of a health care intervention than those trials found in the grey literature. There was limited evidence to show whether grey trials are of poorer methodological quality than published trials. This means that those carrying out systematic reviews need to search for trials in both the published and grey literature in order to help minimise the effects of publication bias in their review.” Hopewell et al. Cochrane Database Syst Rev 2007; 18(2): MR000010.
Grey Literature as Evidence • Agency for Healthcare Research and Quality (AHRQ): Finding Grey Literature Evidence and Assessing for Outcome and Analysis Reporting Biases When Comparing Medical Interventions: AHRQ and the Effective Health Care Program. Methods Guide for Comparative Effectiveness Reviews. (Prepared by the Oregon Health and Science University and the University of Ottawa Evidence-based Practice Centers under Contract Nos. 290-2007-10057-I and 290-2007-10059-I.) AHRQ Publication No. 13(14)- EHC096-EF. Rockville, MD: Agency for Healthcare Research and Quality. November 2013. http://effectivehealthcare.ahrq.gov/ehc/products/486/1751/methods-guidance-reporting-bias-131118.pdf • Institute of Medicine (IOM): Institute of Medicine. Finding what works in health care: standards for systematic reviews. Washington D.C.: National Academy of Sciences; March 2011. http://www.iom.edu/Reports/2011/Finding-What-Works-in-Health-Care- Standards-for-Systematic-Reviews/
Grey Literature as Evidence Clinical Practice Guidelines Health Technology Assessments (HTAs) and/or Meta-analysis Safety advisories or adverse event reports Source: Trip Database http://www.tripdatabase.com/
Grey Matters: A Practical Deep-Web Search Tool for Evidence- based Medicine http://www.cadth.ca/en/resources/fi nding-evidence-is/grey-matters
Health Technology Assessments (HTAs) KEY RESOURCES • Databases: – Centre for Reviews and Dissemination (CRD) – The Cochrane Library ($) – Trip Database • Stand-alone websites: – Health Quality Ontario Publications and OHTAC Recommendations – L’Institut national d’excellence en santé et en services sociaux (INESSS) – Canadian Agency for Drugs and Technologies in Health (CADTH)
CRD Database http://www.crd.york.ac.uk/crdweb/SearchPage.asp
The Cochrane Library http://www.thecochranelibrary.com/
Trip Database http://www.tripdatabase.com/
Health Quality Ontario http://www.hqontario.ca/evidence/publications-and-ohtac-recommendations
CADTH – Health Technology Assessment Search http://www.cadth.ca/en/products/health-technology-assessment
AHRQ Technology Assessments http://www.ahrq.gov/clinic/techix.htm
Safety/Advisory KEY RESOURCES • European Medicines Agency (EMA) Patient Safety • New Zealand Medicines and Medical Devices Safety Authority Medsafe Prescriber Update articles • Medicines and Healthcare Products Regulatory Agency (MHRA) Drug Safety Update • MedWatch: The FDA Safety Information and Adverse Event Reporting Program
Health Canada • MedEffect Canada http://www.hc-sc.gc.ca/dhp- mps/medeff/advisories-avis/index-eng.php • CARN http://www.hc-sc.gc.ca/dhp- mps/medeff/bulletin/carn-bcei_index-eng.php • Canada Vigilance Online Database http://webprod3.hc- sc.gc.ca/arquery-rechercheei/index-eng.jsp
EMA Patient Safety http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/landing/pha_listing.jsp&mid=WC0b01ac058001 d126
Medsafe Prescriber Update Articles http://www.medsafe.govt.nz/profs/PUarticles.asp
MHRA Drug Safety Update http://www.mhra.gov.uk/Publications/Safetyguidance/DrugSafetyUpdate/index.htm
MedWatch: The FDA Safety Information and Adverse Event Reporting Program http://www.fda.gov/Safety/MedWatch/default.htm
Clinical Practice Guidelines • A search for clinical practice guidelines may extend to association websites or organizations dedicated to a particular disease or patient group, depending on the requirements of the search – Guidelines – Protocols – Positions statements – Patient information sheets and resources
Clinical Practice Guidelines KEY RESOURCES • Canadian Medical Association CMA Infobase: Clinical Practice Guidelines (CPGs) • National Institute for Health and Clinical Excellence NICE Guidelines • National Guideline Clearing House (NGC)
CMA Infobase http://www.cma.ca/index.php/ci_id/54316/la_id/1.htm
NICE Guidance http://www.nice.org.uk/guidance/index.jsp?action=find
National Guideline Clearinghouse http://www.guideline.gov/
Policy Health Systems Evidence http://www.mcmasterhealthforum.org/hse/
Closing and Q&A • Further questions? • Evaluations • Draw
Follow us on: @CADTH_ACMTS
Finding Medical Evidence - Vancouver Supporting patient care – Using the internet to your advantage
Home Resources Finding the Evidence: Literature Searching Tools in Support of ... 2011 Apr 3-5; Vancouver, BC ... Standards for Finding and Assessing ...
Finding Medical Evidence - Computer Workshop - Vancouver Supporting Patient Care – Using the internet to your advantage
Inside City Hall Mayor and City Council Agendas and Minutes 2016-2021 Strategic Plan
Finding Medical Evidence- Vancouver (ME8848A) Start date: April 11, 2015 End date: April 11, 2015 Location: SFU Harbour Centre, 515 West Hastings Street ...
Finding Medical Evidence - Vancouver (ME8848C) Start date: September 26, 2015, 0830 End date: September 26, 2015, 1230 Location: SFU Harbour Centre, 515 ...
Collecting and authenticating online evidence ... Vancouver’s Stanley Cup riots offered a ... Identifying and finding online evidence.
CANADIAN BIGFOOT SASQUATCH ATTACK! ~ ANALYZING THE EVIDENCE 2015 The WTF Files ... The Sasquatch of Vancouver Island (Documentary) - Duration: ...
International weekly science journal, published by the American Association for the Advancement of Science (AAAS).
Finding Bigfoot. #FindingBigfoot / Finding Bigfoot. Evidence Map. Home; Videos; Schedule; Bios; Personality Quiz; Bigfoot Game; About; Bigfoot Cam; The ...