Finding the Evidence (Ottawa)

60 %
40 %
Information about Finding the Evidence (Ottawa)
Health & Medicine

Published on March 10, 2014




Presentation from the 2014 CADTH Information Services workshop held in Ottawa.

March 6, 2014 Ottawa, Ontario

CADTH CADTH is an independent, not-for-profit producer and broker of health technology assessments. Federal, provincial, and territorial health care decision-makers rely on our evidence-based information to make informed policy and practice decision about drugs and other health technologies.

Health Technologies • Pharmaceuticals • Medical, surgical and dental devices and procedures • Diagnostics

Jeanette Smith CADTH Liaison Officer 613-226-2553 ext:1265

About Us • David Kaunelis ( Methods Specialist, Information Services • Hayley Fitzsimmons ( Manager, Information Services • Our Team Literature searching for rapid reviews, health technology assessments and CADTH Common Drug Review

Agenda • Introduction to CADTH’s search tools • Creating your literature search BREAK • PubMed tips • Applying search filters • Peer review of search strategies BREAK • Grey literature searching • Grey Matters and Grey Matters Light • Closing and evaluation

Medical Literature as Evidence Clinical Practice Guidelines Health Technology Assessments (HTAs) and/or Meta-analysis Source: Trip Database Safety advisories or adverse event reports

CADTH’s Information Services Major role: Ensure researchers have all the available evidence on a topic • failure to retrieve and analyze all relevant evidence on a topic can seriously compromise the quality of CADTH reports. However… • retrieving too much irrelevant literature can seriously compromise the timeliness and costs of CADTH reports

How Does This Translate Into Literature Searches? Simple search for “diabetes” in Medline Search Fields Number of Results • Subject heading • Title, abstract, all free-text fields 400,000 + • Title • 2+ times in abstract • Major subject heading 280,000 + • Title • Major subject heading 160,000+

Solutions to the Recall versus Precision Conundrum?

Finding the Evidence • Finding the evidence: literature searching tools in support of systematic reviews

Creating YOUR Literature Search • Scoping • Understanding your research question • PICO • Setting your parameters

Scoping – Finding Terms • Look at indexed articles on a similar topic • Sources for drug names/terms - Chem ID - Pubmed/Medline - Embase • Synonyms in Medical thesauri/dictionaries/online sites • MeSH database for synonyms and controlled terms

Use both controlled vocabulary and keywords Uncontrolled Terms (keywords in PubMed) [ti] or [tiab] or [tw] • Words in title & abstract fields • Synonyms/Acronyms • Spelling variants • Old and new terminology • Brand and generic names • Lay and medical terminology • Singular & plural variations Controlled terms (in PubMed) • MeSH subject headings [mh] or [major] or [mh:NoExp] or [major:NoExp] • Substance name field [nm] • Pharmaceutical Action field [pa] • Registry number field [rn]

What is Your Question? P population I intervention C comparator O outcome

What is the clinical effectiveness of the six minute walk test in patients with chronic heart failure? P I C O Adapted with permission: Sampson, M. Chronic heart failure Six minute walk test {none} Clinical effectiveness

Population Intervention Comparison Outcome Heart failure[mh] Exercise test [mh] None Meta-analysis[pt] Randomized controlled trial[pt] (Cardiac OR heart OR myocardial OR congestive) AND Failure 6MWT; SMWT; Six minute; 6M; 6 Minute; 6Min; 6min None Systematic review; RCT; CCT Adapted from: researchers/news/documents/LiteratureSearchingGuidelinesChecklist.pdf Keywords Keywords Keywords Keywords Controlled terms Controlled terms Controlled terms Controlled terms AND O R Translating PICO

Search Translation Could eliminate ‘test’ concept to be more systematic! “AND” with Systematic Review plus RCT filters for best evidence (represents “clinical effectiveness” outcome)

Define Search Parameters

Major Medical Databases Medline/PubMed Embase PsycINFO CINAHL Cochrane Central Focus Biomedical and health sciences Biomedical and pharmaceutical database Behavioural sciences and mental health Nursing and allied health Controlled clinical trials Specialization Strong in clinical research; North American focus Strong in pharmaceutical information; European focus; index conference abstracts Includes fields such as psychiatry, management, business, education, social science, neuroscience, law, medicine, and social work Includes fields such as nursing (clinical and theory), nurse education, physiotherapy, occupational therapy Contains only citations of clinical trials; no need to use an RCT filter Thesaurus MeSH EMTREE Thesaurus of Psychological Index Terms CINAHL Headings MeSH Date Coverage Begins 1946 1974 1840 1981 circa 1966 (some earlier) Journals Indexed 5,516 7,500 2,450 3,000 2,400 handsearched Overlap with other databases Yes Yes Yes Yes Yes

Know Database Syntax Need to know indexing and “syntax” for each database searched Example: Subject Heading Search OVID Medline Diabetes Mellitus, Type 2/ OVID Embase Non insulin dependent diabetes mellitus/ PubMed "Diabetes Mellitus, Type 2"[Mesh] CINAHL MH diabetes mellitus, type 2

Translating the Search • CADTH PubMed cheat sheet dev/PubMed_Cheat_Sheet_external.pdf • PubMed

How PubMed Works • PubMed title and abstract fields are indexed with both single words and select phrases • Not all phrases are included and therefore not all phrases are searchable in PubMed's’ title/abstract fields!

How Most Databases Work • The title and abstract fields of most other biomedical databases (including Medline below) are only single term indexed Phrases are searchable because position of each word in each record’s title or abstract is numbered and the system recognizes if words are beside each other in those fields

Automatic Term Mapping (ATM) • If no field tags ([ti], [tiab]), “quotes” or truncation (*) are used when typing in a word or phrase (i.e.: breast cancer), PubMed will attempt to map keywords to a medical (MeSH) subject heading Exception: if a phrase is searched using quotes and it is not indexed as a phrase in the title or abstract indexes it may map to a corresponding MeSH heading despite the quotes! • If no MeSH synonyms are found, PubMed may map to Journal titles or Authors which contain the search term(s) searched

Automatic Term Mapping (ATM) • PubMed’s automated back-end compensates for no “true” phrase searching in title/abstract fields by “ANDing” terms in the phrase being searched • The automatic “ANDing” of phrase terms is turned off if a phrase is indexed in the title and/or abstract index and one of the following things is true: ● “quotes” are used around the phrase ● truncation * is used at the end of the phrase ● a field tag is used at the end of the phrase [tiab] or [ti] (field tags will also stop phrases that are not indexed in the title and/or abstract index from being “ANDed” together unless truncation * is added)

Keyword Phrase Searching in PubMed Examples of recall with a phrase not indexed in title or abstract indexes


Top Ten PubMed Search Tips

PubMed Tip #1 To find a couple of good articles on a topic: 1. Search the ‘Title’ field for one or more specific keywords COPD[ti] AND (dietary[ti] OR diet[ti]) 2. Search using the appropriate ‘Major’ MeSH subject heading and specific subheading "Pulmonary Disease, Chronic Obstructive/diet therapy"[Majr] 3. If have one perfect article on your topic, use PubMed’s ‘Related citations’ feature to find similar studies (related articles are displayed in ranked order from most to least relevant)

PubMed Tip #2 Always check search details when using PubMed’s automatic term mapping system (ATM) or truncation (*)

PubMed Tip #3 Avoid using phrases when searching in title [ti] and title/abstract [tiab] fields. PubMed has limited ability for “true” phrase searching in these fields and no way to search adjacency. PubMed phrase search in Title/abstract fields: Medline phrase search in Title/abstract fields:

Be careful of punctuation when searching title/abstract fields! PubMed Tip #4 Punctuation/characters with specific functions in PubMed: help&rendertype=table&id=pubmedhelp.T43 PubMed Tip #5 The following stopwords are not searchable in title/abstract fields These terms cannot be searched unless they are part of a ‘phrase indexed’ field such as MeSH

Search with MeSH heading(s) if available. Assigning articles with the appropriate MeSH headings helps to compensate for inconsistent language & many different synonyms for the same concept All of the above articles can be retrieve using the following MeSH term: Stockings, Compression[MH] PubMed Tip #6 *Important: new non- indexed articles will not be retrieved if only MeSH terms are searched

Set #2: Depression Depression[MeSH] OR Depressive disorder[mh:noexp] OR Depressive disorder, major[mh] OR depression*[tiab] OR depressive[tiab] OR depressed[tiab] MeSH Tree: Whenever adding MeSH terms make sure to look at where they sit within the MeSH tree. PubMed automatically explodes MeSH headings. PubMed Tip #7

When searching for drug classes (i.e. antifungal agents) always double check in MeSH Thesaurus to see if there is both a [Pharmacological action] and [MeSH] heading PubMed Tip #8 The Pharmacological heading compiles all the [Substance name] and [MeSH] headings (available in PubMed) of the drugs that fall into that particular drug class

PubMed Tip #9 Use PubMed filters/limits cautiously! Filters/limits circled in red exclude "in process" and "supplied by publisher" citations because they have not been indexed for MEDLINE Other pre-loaded PubMed filters are available in the Clinical Queries screen: nical

PubMed Tip #10 May copy URL for your complex PubMed search strategy and add link to company website for anyone to run Click on URL box then copy search hyperlink and paste where required

Filters Why use filters? • Reduce the number of irrelevant results • Extract the higher level evidence (i.e. RCTs) • Extract a specific concept (ex. QOL, adverse events) • Remove off-topic results (ex. animals, editorials) • Saves time for IS and researcher


Types of Filters • Specific methodology – RCTs, Systematic reviews • Clinical queries – Diagnosis, Prognosis, Adverse events • Population specific – Humans, Pediatrics • CADTH Filters • InterTASC Search Filter Resource

Example Filter: RCTs 1. Randomized Controlled 2. Pragmatic Clinical 3. Randomized Controlled Trials as Topic/ 4. Randomized Controlled Trial/ 5. Randomization/ 6. Double-Blind Method/ 7. Single-Blind Method/ 8. Placebos/ 9. (random* or sham or placebo*).ti,ab,hw. 10. ((singl* or doubl*) adj (blind* or dumm* or mask*)).ti,ab,hw. 11. ((tripl* or trebl*) adj (blind* or dumm* or mask*)).ti,ab,hw. 12. or/1-11

Filters: Warning • No filter is infallible • Filters are regularly being tested, tweaked and updated but they almost always capture irrelevant records and sometimes exclude relevant ones

RCT filter would pick up this article…

And this article… “Randomized” in title and abstract, but not a study!

But not this one. Study design NOT found anywhere:

CADTH Peer Review Checklist for Search Strategies review-search-strat

CADTH Peer Review Checklist for Search Strategies • Developed from: Sampson M, McGowan J, Lefebvre C, Moher D, Grimshaw J. PRESS: Peer review of electronic search strategies. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2008. Electronic-Search-Strategies_tr_e.pdf

CADTH Peer Review Checklist for Search Strategies Key Elements Question translation Appropriate search filters Operators Limits Subject headings Fields Keywords Database selection & adaptation Spelling & syntax Additional suggestions/comments

Peer Review: Does it Make a Difference? • Recent research conducted to determine whether the peer review of literature search strategies has an effect on the number and quality of articles included in CADTH rapid review reports • What were the results?

Getting Your Search Reviewed • Colleagues • Listservs • Your librarian • PRESS Forum website. A forum for librarians to request and conduct peer reviews of search strategies.


What is Grey Literature? • Grey/gray • Unpublished • Non-commercial material • In-house, vertical files • Information on the deep web • “Hard to find” • Fugitive

Grey Literature Definition INTERNATIONAL CONFERENCE ON GREY LITERATURE “Information produced on all levels of government, academics, business and industry in electronic and print formats not controlled by commercial publishing i.e. where publishing is not the primary activity of the producing body.” (Luxembourg, 1997 - Expanded in New York, 2004) Source: GreyNet: grey literature network service. Amsterdam: GreyNet; 2008.

Examples of Grey Literature • Government Documents • Policy Documents • Clinical Guidelines • Conference Abstracts/Proceedings/Papers • Open Access Journals (some un-indexed) • Health Technology Assessments • Personal Communication • Clinical Trials

Why is Searching Grey Literature Important? • Essential component of a comprehensive search • Overcomes bias of commercial publications • Part of evidence base • Most HTAs created for healthcare decision-makers are considered “grey literature” and are not found in traditional databases

Publication Bias • “Exclusion of grey literature from meta-analyses can lead to exaggerated estimates of intervention effectiveness” McAuley et al. Lancet 2000; 356(Oct 7): 1228-31. • “Published literature tend to be larger and show larger effects of a health care intervention than those trials found in the grey literature. There was limited evidence to show whether grey trials are of poorer methodological quality than published trials. This means that those carrying out systematic reviews need to search for trials in both the published and grey literature in order to help minimise the effects of publication bias in their review.” Hopewell et al. Cochrane Database Syst Rev 2007; 18(2): MR000010.

Grey Literature as Evidence Clinical Practice Guidelines Health Technology Assessments (HTAs) and/or Meta-analysis Source: Trip Database Safety advisories or adverse event reports

Grey Literature: Benefits and Challenges Benefits Challenges • Can reduce publication bias • Large volume of material • Current • Time-consuming to find • Usually free • Not always available online • Unique content (i.e.: HTAs) • Quality of evidence varies • Information on non- mainstream topics • Bibliographic information may be missing • Helps in identifying terms for search strategies • Inconsistent language, no subject indexing

Grey Matters: A Practical Deep-Web Search Tool for Evidence- based Medicine nding-evidence-is/grey-matters

Health Technology Assessments (HTAs) KEY RESOURCES • Databases: – Centre for Reviews and Dissemination (CRD) – The Cochrane Library ($) – Trip Database • Stand-alone websites: – Health Quality Ontario Publications and OHTAC Recommendations – L’Institut national d’excellence en santé et en services sociaux (INESSS) – Canadian Agency for Drugs and Technologies in Health (CADTH)

CRD Database

The Cochrane Library

Trip Database

Health Quality Ontario


CADTH – Search

AHRQ Technology Assessments

Safety/Advisory KEY RESOURCES • European Medicines Agency (EMA) Patient Safety • New Zealand Medicines and Medical Devices Safety Authority Medsafe Prescriber Update articles • Medicines and Healthcare Products Regulatory Agency (MHRA) Drug Safety Update • MedWatch: The FDA Safety Information and Adverse Event Reporting Program

Health Canada • MedEffect Canada mps/medeff/advisories-avis/index-eng.php • CARN mps/medeff/bulletin/carn-bcei_index-eng.php • Canada Vigilance Online Database http://webprod3.hc-

EMA Patient Safety d126

Medsafe Prescriber Update Articles

MHRA Drug Safety Update

MedWatch: The FDA Safety Information and Adverse Event Reporting Program

Clinical Practice Guidelines • A search for clinical practice guidelines may extend to association websites or organizations dedicated to a particular disease or patient group, depending on the requirements of the search – Guidelines – Protocols – Positions statements – Patient information sheets and resources

Clinical Practice Guidelines KEY RESOURCES • Canadian Medical Association CMA Infobase: Clinical Practice Guidelines (CPGs) • National Institute for Health and Clinical Excellence NICE Guidelines • National Guideline Clearing House (NGC)

CMA Infobase

NICE Guidance

National Guideline Clearinghouse

Policy Health Systems Evidence (free registration required)

Closing and Q&A • Further questions? • Evaluations

Follow us on: @CADTH_ACMTS

Add a comment

Related presentations

Related pages

University of Ottawa | Canada's University | National

Medical students among Canada’s ... Finding the evidence to confirm best medical practices. The University of Ottawa has been selected as an Evidence ...
Read more

University of Ottawa | Canada's University | Print

Medical students among Canada’s ... Finding the evidence to confirm best medical practices. The University of Ottawa has been selected as an Evidence ...
Read more

Evidence supports Mohamed Mahjoub's security certificate ...

Evidence supports Mohamed Mahjoub's security certificate, judge says. ... seen here at Federal Court in Ottawa in 2012, after finding the evidence ...
Read more

Top 10 Chiropractors in Ottawa, ON - Opencare

Here is the definitive list of Ottawa's top chiropractors as rated by the Ottawa, ON community. Want to see who made the cut?
Read more

CADTH Peer Review Checklist for Search Strategies |

Finding the Evidence: Literature Searching Tools in Support of Systematic Reviews. ... CADTH Peer Review Checklist for Search Strategies. Date: March 3, 2016.
Read more

Where to find evidence | Ontario Centre of Excellence for ...

The links below can assist you in preparing a literature review, finding evidence-informed programs or practices, finding information about evaluating ...
Read more

ADR Ottawa

ADR Ottawa offers a wide variety of dispute and conflict resolution services, ranging from modest to large-scale, from simple to complex. • mediation
Read more

Ottawa ankle rules for the injured ankle - National Center ...

This became known as the Ottawa ankle ... reproduced Stiell's findings and established the Ottawa ankle rules ... ankle rules for the injured ankle .
Read more

Clinical Usefulness of the Ottawa Ankle Rules for ...

Clinical Usefulness of the Ottawa Ankle Rules for ... What is the evidence for the accuracy of ... a negative test finding is a reasonable ...
Read more