Published on July 23, 2014
Google Scholar as a research and evaluation toolevaluation tool Álvaro Cabezas-Clavijo
INDEX • Goals • GS as a research tool – Search engine – Alerts – Updates – Library + Cite • GS as a tool for evaluating research• GS as a tool for evaluating research – GS Citations (or Profiles) – GS Metrics – Bibliometric analysis • Data manipulation • Comparison to WOS/ Scopus • Conclusions
To show GS possibilities to do research To show GS possibilities to evaluate research To point out strengths and weaknesses of GS as a source to perform bibliometric analysis GOALS To introduce Google Scholar as a valid source for carrying out research assessments (with caution!) in Social Sciences and Humanities (SSH)
GS AS A TOOL FOR RESEARCHERSGS AS A TOOL FOR RESEARCHERS
• GS is a widespread tool for doing research • Comprehensive source of academic literature • Widely known by scholars and students • Ease of use (just like google) GS AS A TOOL FOR RESEARCHERS • • Free and fast • Links to primary source (when it is freely available on the web or can be accessed from our institution)
• Non-selective source (covers all material which meet the technical requirements) • No restrictions on language • No restrictions on document type • No discipline/areas bias GS AS A TOOL FOR RESEARCHERS (II) • No discipline/areas bias • Estimated size is 100 million documents100 million documents (Web of Science 50 million; Scopus 53 million)
GS AS A TOOL FOR RESEARCHERS (III)
• GS is not just an academic search engine but a “family” of tools for doing and evaluating research GS AS A TOOL FOR RESEARCHERS (IV) ALERTS UPDATES LIBRARY + CITE
• Set up by user just clicking on the “Create alert” link when performing a search • Delivered by 2-4 days if there is fresh info to show • Authors or thematic searches can be configured (but not for journals) ALERTS journals) 1 2 3
• Configure an alert for citations to particular papers or authors (in this case, if they have already set up a profile) ALERTS Following Christian’s new papers or citations
• Alerts about papers of our interest automatically generated by Google • You only need to set up a GS profile. Algorithm takes into account: – Who we cite in our papers – Who cite us in their papers UPDATES – Who cite us in their papers – Titles of our papers – Co-authors – Content • A red bell indicates the number of new papers that might be of your interest
UPDATESSome paper recommendations Ok, I have published about these topics, so I may be interested in these papers Ok, this paper cites me so it should be related to my research
• Kind of reference manager • Save references in one click • Arrange references by using labels • Edit references & export them (but one by one!) • Can’t import references from other bibliographic software LIBRARY + CITE • Can’t import references from other bibliographic software •Cite easily from GS results: formats MLA, APA, Chicago •Get the formatted reference and copy/paste it into a Bibliography.
• By default; 2 folders: My citations (actually My papers) and Cited by me • You can set up other LIBRARY + CITE • You can set up other labels/folders and save new references from GS searches
GS as a tool for evaluating researchGS as a tool for evaluating research
• GS provides different tools which can help in assessments of individuals, journals and more GS AS A TOOL FOR EVALUATORS (II) PROFILES METRICS SOURCE FOR BIBLIOMETRIC ANALYSES
• Individual Web page with publication list and basic bibliometric indicators • Papers are ranked (by default) according to their number of citations • Also links to co-authors (if they have set up a profile) and PROFILES • Also links to co-authors (if they have set up a profile) and keywordskeywords Ranking of researchers by keywords/areas are generated through this classification TIP: there is always a field where you can be the most cited author! ☺
• Easy to set up and edit • Strongly recommended for young researchers as it can increase their visibility • You will get an alert (maximum 3 times a week) every time you are cited PROFILES • 72,579 researchers with profile (March 2012) (Estimates by Ortega & Aguillo, 2013) • GS profiles-derived products – Some Spanish institutions are ranking their researchers according to GSC (Univ. Granada, CSIC,…) stimulus to set up a profile?
COMMUNICATION: 886 RESEARCHERS
SCIENTOMETRICS: 265 RESEARCHERS
CSIC RANKING OF RESEARCHERS ACCORDING TO GSC
• Easy to set up (just search for your papers) • Terrific tool for comparing researchers within a field or department • Automatically updated basic bibliometric data PROFILES • Data can be easily manipulated (a researcher can self-claim non-authored papers) • Can stimulate vanity and ego • Can generate no fair comparisons (for example,bibliometric data • No restrictions on source, language or area • Everyone can measure their performance (and their colleagues') comparisons (for example, researchers from different areas in a single univ) • Can generate unfair analysis by non-bibliometric experts STRENGTHS WEAKNESSES
• GS service devoted to measure journal’s impact according to their h-index (est. April 2012) • Rankings are provided by language (nine languages) areas (8) and subcategories (313) (just for English-language journals) and journal title’s words (i.e, oncology) • Subject/area rankings are limited to top 20 journals and language METRICS • Subject/area rankings are limited to top 20 journals and language rankings are limited to top 100 journals. • Calculations take into account output for 5 years (currently 2008-2012) and citations received until july 2013. A new update is expected to be released in July 2014. • Journals with fewer than 100 papers published in 5 years or with no citation (h-index=0) are excluded • arXiv and other repositories’ collections are also included
• Free product to compare and rank journals • We can get impact information about non-JCR journals and about national and SSH publications • Transparency: citations for every paper that contributes to the h-index can be checked • Methodological inconsistencies such as comparing journals from different areas • Lack of proper bibliographic control (duplicates, “dirty” data,…) • No selection criteria • No action against data manipulation METRICS can be checked • High correlation with JCR Impact factor (0.82) • Simple to use and understand by any scholar • Can be easily replicated • No action against data manipulation • No info about new updates (expected to be updated yearly) • Just top results are presented • H index favours journals with high production STRENGTHS WEAKNESSES
• GS is estimated to contain about 100 million documents (87% of scholarly papers on the web) Khabsa & Giles (2014) • So, it is the largest source for bibliometric analysis that we can use SOURCE FOR BIBLIOMETRIC ANALISYS can use • GS-based products can also be used to help with research assessment – Publish or Perish (software for bibliometric analysis) – H-Index Scholar (ranking of spanish researchers) – Google Chrome add-ons – …
PUBLISH OR PERISH
GOOGLE CHROME ADD-ONS
• Many things to sort out (unknown coverage, duplicates, “dirty” data) but GS can be useful for: Non-covered areas by WoS… Non-English research… SOURCE FOR BIBLIOMETRIC ANALISYS Non-English research… …Which means basically Social Sciences and Humanities institutions and individuals which lack access to WoS/Scopus
GS data can be easily manipulated • Just need to upload an “academic” paper to a institutional repository, university webpage or similar • “Academic” means a paper with title, authors, abstract and of course, bibliographic references • GS will count the citations from these fake papers, modifying• GS will count the citations from these fake papers, modifying indicators of all cited scholars and journals. Delgado, Robinson & Torres (2014)
• Small diferences between rankings generated by GS, WoS and Scopus • Intensive Care Medicine Spearman correlation of 0.99 for journals and 0.93 for top researchers Comparison to Wos / Scopus JOURNALSJOURNALS Cabezas & Delgado (2013) • Spearman correlation for Communication Journals: 0.895 (GSM-WoS); 0.879 (GSM-Scopus) • Spearman correlation for Economics & Business Journals: 0.718 (GS-WoS) JOURNALS Delgado & Repiso (2013) Harzing & der Wal (2009)
• Easy to use, fast and free • Largest source for assessment purposes • Provides information about areas and materials not covered by • Lack of bibliographic control • Data can be manipulated • No possibilities to massive info download • Time-consuming, data needs to SOURCE FOR BIBLIOMETRIC ANALISYS and materials not covered by traditional bibliometric products • Rankings are very similar to those offered by WoS / Scopus • Possibility to build new products based on GS data • Time-consuming, data needs to be “cleaned” • GS and their services could be closed down in any moment STRENGTHS WEAKNESSES
• Bibliometricians need to find other products to complement WoS / Scopus-based evaluations (GS, ERIH, national products,…) • What if subscriptions to traditional databases are cancelled? – Spain paying 4-6 million Euros to Thomson Reuters yearly for national access to WoS from public research organizations CONCLUSIONS – Spain paying 4-6 million Euros to Thomson Reuters yearly for national access to WoS from public research organizations • SSH deserve to be measured (prominent rankings such as Shanghai or Leiden ignore Humanities and a huge part of Social Sciences!) • GS tools are far from perfect but provide us with some valuable information
• Bibliometricians take usually the “easy way”; only measure what can be easily measured with databases such as WoS or Scopus • A challenge for bibliometricians is to get the most out of the existing tools to evaluate SSH (and to develop new ones!) • GS can be used as a complementary or alternative source to CONCLUSIONS • GS can be used as a complementary or alternative source to shed some light on SSH • Use with extreme caution, especially when assessing individuals • Keep an eye on the future: Institutions might be demanding GS-based analysis soon.
Further readingsFurther readings
• Delgado-López-Cózar, E., & Cabezas-Clavijo, Á. (2013). Ranking journals: could Google Scholar Metrics be an alternative to Journal Citation Reports and Scimago Journal Rank?. Learned publishing, 26(2), 101-113. • Delgado López-Cózar, E., Robinson-García, N., & Torres-Salinas, D. (2014). The Google Scholar Experiment: how to index false papers and manipulate bibliometric indicators. Journal of the Association for Information Science and Technology, 65(3), 446-454. • Harzing, A. W., & Van Der Wal, R. (2009). A Google Scholar h-index for journals: An FURTHER READINGS • Harzing, A. W., & Van Der Wal, R. (2009). A Google Scholar h-index for journals: An alternative metric to measure journal impact in economics and business. Journal of the American Society for Information Science and Technology, 60(1), 41-46. • Khabsa, M., & Giles, C. L. (2014). The Number of Scholarly Documents on the Public Web. PloS one, 9(5), e93949. • Ortega, J. L., & Aguillo, I. F. (2013). Institutional and country collaboration in an online service of scientific profiles: Google Scholar Citations. Journal of Informetrics, 7(2), 394-403.
• BLOG: Google Scholar Digest: analysis of papers dealing with GS http://googlescholardigest.blogspot.com.es/ • TWITTER: Follow the #googlescholar hashtag • SOFTWARE: Publish or Perish http://www.harzing.com/pop.htm • PRODUCTS: FURTHER MATERIALS • PRODUCTS: • GOOGLE SCHOLAR CITATIONS http://scholar.google.com/citations • GOOGLE SCHOLAR METRICS http://scholar.google.com/citations?view_op=top_venues&hl=en
Google Scholar as a research and evaluation toolevaluation tool Álvaro Cabezas-Clavijo Thank you!
... Visiting Scholar, New York University. Publicado por ... 01/06/2005 Agencia Veritas ... http://www.google.es; http://www.hotmail.com;
Who is John Baran - (434) 244-3740 - Charlottesville - VA - waatp.com.See also John Baran: pictures, social networks profiles, videos, weblinks, at blogs ...
Managing Director M&T Bank Pittsburgh, Pennsylvania Richard F. Fiore Sr. '73 President Leonard S ... Debra S. Bender First Summit Bank Kip Benko, FACEP ...