Finding the Evidence (Vancouver)

50 %
50 %
Information about Finding the Evidence (Vancouver)
Health & Medicine

Published on March 10, 2014




Presentation from the 2014 CADTH Information Services workshop held in Vancouver.

March 5-6, 2014 Vancouver, British Columbia

CADTH CADTH is an independent, not-for-profit producer and broker of health technology assessments. Federal, provincial, and territorial health care decision-makers rely on our evidence-based information to make informed policy and practice decision about drugs and other health technologies.

Health Technologies • Pharmaceuticals • Medical, surgical and dental devices and procedures • Diagnostics

Ann Vosilla CADTH Liaison Officer (604) 787-9945

About Us • Amanda Hodgson ( Manager, Information Services • Caitlyn Ford ( Information Specialist • Our Team Literature searching for rapid reviews, health technology assessments and CADTH Common Drug Review

Objectives PART 1 • Role of the Information Specialist in HTA • Creating your literature search • Applying search filters • Peer review of search strategies BREAK PART 2 • Grey literature searching • Grey Matters and Grey Matters Light • Closing and evaluation

Finding the Evidence • Finding the evidence: literature searching tools in support of systematic reviews

What is a Systematic Search? “Systematic reviews of interventions require a thorough, objective and reproducible search of a range of sources to identify as many relevant studies as possible (within resource limits). This is a major factor in distinguishing systematic reviews from traditional narrative reviews and helps to minimize bias and therefore assist in achieving reliable estimates of effects.” Higgins et al. Cochrane handbook for systematic reviews of interventions. 2008. Section

What is a Systematic Search? “…the aspiration of the HTA literature search should not be comprehensiveness but rather the minimization of bias.” Booth. Int J Technol Assess Health Care. 2010 Oct;26(4):431-5.

Fundamental Challenge of Searching Balancing Recall and Precision = getting "everything" while avoiding "junk"

How Does This Translate Into Literature Searches? Simple search for “diabetes” in Medline Search Fields Number of Results • Subject heading • Title, abstract, all free-text fields 400,000 + • Title • 2+ times in abstract • Major subject heading 280,000 + • Title • Major subject heading 160,000+

Standards for Systematic Searchers • Finding what works in health care: standards for systematic reviews. Washington D.C.: Institute of Medicine of the National Academies; 2011. See Chapter 3: Standards for Finding and Assessing Individual Studies. Standard 3.1 – • Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions. Version 5.1.0. [Updated March 2011]. The Cochrane Collaboration; 2011. See Part 2: General methods for Cochrane Reviews. Chapter 6: searching for studies.( Authors: Carol Lefebvre, Eric Manheimer and Julie Glanville on behalf of the Cochrane Information Retrieval Methods Group)

Who are “Systematic Searchers”? • Experts in technical and methodological issues of searching • MLIS/MLS/MISt – educational requirements • Work with the research team at every stage of review WHY ARE THEY IMPORTANT? • Help to mitigate potential biases in search • Know search resources and how to use them • Can advise on size and quality of search results

Role of the Information Specialist • Help define scope of review • Perform complex database and grey literature searches • Document the search • Manage citations, ensure quality of references • Perform supplemental searches • Provide advice on copyright & obtaining permissions • Maintain alerts & media scanning

Creating YOUR Literature Search • Understanding your research question • PICO • Scoping • Setting your parameters

Understand Research Question • Work with research team to refine questions – Scoping searches, identify main concepts • Determine PICO • Define search parameters – age, gender, date or language limits, study types • Get suggestions on terminology or sources from subject experts

What is your question? P population I intervention C comparator O outcome

Understand the Research Question Patient Population Adults with type 2 diabetes who have experienced inadequate glycemic control on combination therapy with metformin and a sulfonylurea Intervention Saxagliptin (2.5 mg once daily or 5 mg once daily) in combination with metformin and a sulfonylurea Comparators Other DPP-4 inhibitors, basal insulin, or placebo in combination with metformin and a sulfonylurea Outcomes Key efficacy outcomes • mortality • diabetes-related morbidity (macrovascular, microvascular) • glycemic control (A1C, FBG) • QoL (measured by any validated scale) Harms outcomes  hypoglycemia  total AEs  SAEs  WDAEs  changes in body weight Other outcomes  health care resource utilization Study design Published and unpublished RCTs ≥ 12 weeks duration

Population Intervention Comparison Outcome Intensive Care Units [mh] and others Dexmedetomidine [mh] None None ICU;PICU; intensive care; etc. precedex; dexdor; dexmedotomidin*; 113775-47-6 None None Adapted from: researchers/news/documents/LiteratureSearchingGuidelinesChecklist.pdf Keywords Keywords Keywords Keywords Controlled terms Controlled terms Controlled terms Controlled terms AND O R Translating PICO

When PICO Doesn’t Cut It…

Understand the Research Question • Search Protocol

Translating the Search • CADTH PubMed cheat sheet dev/PubMed_Cheat_Sheet_external.pdf

Scoping – Finding Terms • Look at indexed articles on a similar topic • Sources for drug names/terms - Chem ID - Pubmed/Medline - Embase • Synonyms in Medical thesauri/dictionaries/online sites • MeSH thesaurus for synonyms and entry terms

Use both controlled vocabulary and keywords Uncontrolled Terms (keywords in PubMed) [ti] or [tiab] or [tw] • Words in title & abstract fields • Synonyms/Acronyms • Spelling variants • Old and new terminology • Brand and generic names • Lay and medical terminology • Singular & plural variations Controlled terms (in PubMed) • MeSH subject headings [mh] or [major] or [mh:NoExp] or [major:NoExp] • Substance name field [nm] • Pharmaceutical Action field [pa] • Registry number field [rn]

Major Medical Databases Medline/PubMed Embase PsycINFO CINAHL Cochrane Central Focus Biomedical and health sciences Biomedical and pharmaceutical database Behavioural sciences and mental health Nursing and allied health Controlled clinical trials Specialization Strong in clinical research; North American focus Strong in pharmaceutical information; European focus; index conference abstracts Includes fields such as psychiatry, management, business, education, social science, neuroscience, law, medicine, and social work Includes fields such as nursing (clinical and theory), nurse education, physiotherapy, occupational therapy Contains only citations of clinical trials; no need to use an RCT filter Thesaurus MeSH EMTREE Thesaurus of Psychological Index Terms CINAHL Headings MeSH Date Coverage Begins 1946 1974 1840 1981 circa 1966 (some earlier) Journals Indexed 5,516 7,500 2,450 3,000 2,400 handsearched Overlap with other databases Yes Yes Yes Yes Yes

Know Database Syntax Need to know indexing and “syntax” for each database searched Example: Subject Heading Search OVID Medline Diabetes Mellitus, Type 2/ OVID Embase Non insulin dependent diabetes mellitus/ PubMed "Diabetes Mellitus, Type 2"[Mesh] CINAHL MH diabetes mellitus, type 2


Filters Why use filters? • Reduce the number of irrelevant results • Extract the higher level evidence (i.e. RCTs) • Extract a specific concept (ex. QOL, adverse events) • Remove off-topic results (ex. animals, editorials) • Saves time for IS and researcher

Types of Filters • Specific methodology – RCTs, Systematic reviews • Clinical queries – Diagnosis, Prognosis, Adverse events • Population specific – Humans, Pediatrics • CADTH Filters evidence-is/string

Filters: Warning • No filter is infallible • Filters are regularly being tested, tweaked and updated but they almost always capture irrelevant records and sometimes exclude relevant ones

RCT filter would pick up this article…

And this article… “Randomized” in title, but not a study:

But not this one. Study design NOT found anywhere:

CADTH Peer Review Checklist for Search Strategies review-search-strat

CADTH Peer Review Checklist for Search Strategies • History and development • Current checklist • New research on the value of peer review • How can you get your search peer reviewed?

CADTH Peer Review Checklist for Search Strategies • Developed from: Sampson M, McGowan J, Lefebvre C, Moher D, Grimshaw J. PRESS: Peer review of electronic search strategies. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2008. Electronic-Search-Strategies_tr_e.pdf

CADTH Peer Review Checklist for Search Strategies Key Elements Translation Appropriate search filters Operators Limits Subject headings Fields Natural language Adapted databases Spelling & syntax Additional suggestions/comments

Exercise Sample Searches

Best Practices for Search Strategies 1) Understand research question(s) 2) Use keywords and controlled vocabulary 3) Know database indexing and syntax 4) Use filters where appropriate 5) Have searches peer reviewed 6) Use caution: don’t overcomplicate


What is Grey Literature? • Grey/gray • Unpublished • Non-commercial material • In-house, vertical files • Information on the deep web • “Hard to find” • Fugitive

Grey Literature Definition INTERNATIONAL CONFERENCE ON GREY LITERATURE “Information produced on all levels of government, academics, business and industry in electronic and print formats not controlled by commercial publishing i.e. where publishing is not the primary activity of the producing body.” (Luxembourg, 1997 - Expanded in New York, 2004) Source: GreyNet: grey literature network service. Amsterdam: GreyNet; 2008.

Examples of Grey Literature • Government Documents • Policy Documents • Clinical Guidelines • Conference Abstracts/Proceedings/Papers • Open Access Journals (some un-indexed) • Health Technology Assessments • Personal Communication • Clinical Trials

Why is Searching Grey Literature Important? • Essential component of a comprehensive search • Overcomes bias of commercial publications • Part of evidence base • Most HTAs created for healthcare decision-makers are considered “grey literature” and are not found in traditional databases

Publication Bias • “Exclusion of grey literature from meta-analyses can lead to exaggerated estimates of intervention effectiveness” McAuley et al. Lancet 2000; 356(Oct 7): 1228-31. • “Published literature tend to be larger and show larger effects of a health care intervention than those trials found in the grey literature. There was limited evidence to show whether grey trials are of poorer methodological quality than published trials. This means that those carrying out systematic reviews need to search for trials in both the published and grey literature in order to help minimise the effects of publication bias in their review.” Hopewell et al. Cochrane Database Syst Rev 2007; 18(2): MR000010.

Grey Literature as Evidence • Agency for Healthcare Research and Quality (AHRQ): Finding Grey Literature Evidence and Assessing for Outcome and Analysis Reporting Biases When Comparing Medical Interventions: AHRQ and the Effective Health Care Program. Methods Guide for Comparative Effectiveness Reviews. (Prepared by the Oregon Health and Science University and the University of Ottawa Evidence-based Practice Centers under Contract Nos. 290-2007-10057-I and 290-2007-10059-I.) AHRQ Publication No. 13(14)- EHC096-EF. Rockville, MD: Agency for Healthcare Research and Quality. November 2013. • Institute of Medicine (IOM): Institute of Medicine. Finding what works in health care: standards for systematic reviews. Washington D.C.: National Academy of Sciences; March 2011. Standards-for-Systematic-Reviews/

Grey Literature as Evidence Clinical Practice Guidelines Health Technology Assessments (HTAs) and/or Meta-analysis Safety advisories or adverse event reports Source: Trip Database

Grey Matters: A Practical Deep-Web Search Tool for Evidence- based Medicine nding-evidence-is/grey-matters

Health Technology Assessments (HTAs) KEY RESOURCES • Databases: – Centre for Reviews and Dissemination (CRD) – The Cochrane Library ($) – Trip Database • Stand-alone websites: – Health Quality Ontario Publications and OHTAC Recommendations – L’Institut national d’excellence en santé et en services sociaux (INESSS) – Canadian Agency for Drugs and Technologies in Health (CADTH)

CRD Database

The Cochrane Library

Trip Database

Health Quality Ontario


CADTH – Health Technology Assessment Search

AHRQ Technology Assessments

Safety/Advisory KEY RESOURCES • European Medicines Agency (EMA) Patient Safety • New Zealand Medicines and Medical Devices Safety Authority Medsafe Prescriber Update articles • Medicines and Healthcare Products Regulatory Agency (MHRA) Drug Safety Update • MedWatch: The FDA Safety Information and Adverse Event Reporting Program

Health Canada • MedEffect Canada mps/medeff/advisories-avis/index-eng.php • CARN mps/medeff/bulletin/carn-bcei_index-eng.php • Canada Vigilance Online Database http://webprod3.hc-

EMA Patient Safety d126

Medsafe Prescriber Update Articles

MHRA Drug Safety Update

MedWatch: The FDA Safety Information and Adverse Event Reporting Program

Clinical Practice Guidelines • A search for clinical practice guidelines may extend to association websites or organizations dedicated to a particular disease or patient group, depending on the requirements of the search – Guidelines – Protocols – Positions statements – Patient information sheets and resources

Clinical Practice Guidelines KEY RESOURCES • Canadian Medical Association CMA Infobase: Clinical Practice Guidelines (CPGs) • National Institute for Health and Clinical Excellence NICE Guidelines • National Guideline Clearing House (NGC)

CMA Infobase

NICE Guidance

National Guideline Clearinghouse

Policy Health Systems Evidence

Closing and Q&A • Further questions? • Evaluations • Draw

Follow us on: @CADTH_ACMTS

Add a comment

Related presentations

Related pages

Finding Medical Evidence - Vancouver | UBC CPD

Finding Medical Evidence - Vancouver Supporting patient care – Using the internet to your advantage
Read more

Finding the Evidence: Literature Searching Tools in ...

Home Resources Finding the Evidence: Literature Searching Tools in Support of ... 2011 Apr 3-5; Vancouver, BC ... Standards for Finding and Assessing ...
Read more

Finding Medical Evidence - Computer Workshop - Vancouver ...

Finding Medical Evidence - Computer Workshop - Vancouver Supporting Patient Care – Using the internet to your advantage
Read more

Finding Evidence of Kinship in Military Records | City of ...

Inside City Hall Mayor and City Council Agendas and Minutes 2016-2021 Strategic Plan
Read more

Finding Medical Evidence- Vancouver (ME8848A)

Finding Medical Evidence- Vancouver (ME8848A) Start date: April 11, 2015 End date: April 11, 2015 Location: SFU Harbour Centre, 515 West Hastings Street ...
Read more

Finding Medical Evidence - Vancouver (ME8848C)

Finding Medical Evidence - Vancouver (ME8848C) Start date: September 26, 2015, 0830 End date: September 26, 2015, 1230 Location: SFU Harbour Centre, 515 ...
Read more

Canadian Bar Association - Collecting and authenticating ...

Collecting and authenticating online evidence ... Vancouver’s Stanley Cup riots offered a ... Identifying and finding online evidence.
Read more


CANADIAN BIGFOOT SASQUATCH ATTACK! ~ ANALYZING THE EVIDENCE 2015 The WTF Files ... The Sasquatch of Vancouver Island (Documentary) - Duration: ...
Read more

Science | AAAS

International weekly science journal, published by the American Association for the Advancement of Science (AAAS).
Read more

Evidence Map | Finding Bigfoot | Animal Planet

Finding Bigfoot. #FindingBigfoot / Finding Bigfoot. Evidence Map. Home; Videos; Schedule; Bios; Personality Quiz; Bigfoot Game; About; Bigfoot Cam; The ...
Read more