Published on March 12, 2014
Discovery or Displacement? A Major Longitudinal Study of the Eﬀect of Web-‐Scale Discovery Services on Online (Journal) Usage SCELC Colloquium March 5, 2014 Michael Levine-‐Clark, University of Denver John McDonald, University of Southern California Jason Price, SCELC ConsorNum
“…a steep increase in full text downloads and link resolver click‐throughs suggests Summon had a dramatic impact on user behavior and the use of library collections during this time period.” The Impact ofWeb-scale Discovery on the Use of a Library Collection Doug Way (2010) http://scholarworks.gvsu.edu/ library_sp/9/
h"p://www.oclc.org/partnerships/econtent/solu4ons.en.html Vendor marke5ng
Does implementa4on of a discovery service impact usage of publisher-‐ hosted journal content?
What did we measure? • Whether there is an eﬀect • NOT why that eﬀect exists (that’s a future study!)
• “Society will need to shed some of its obsession for causality in exchange for simple correla5ons: not knowing why, but only what” • Cukier & Mayer-‐Schonberger. 2013. Big data: A revolu4on that will transform how we live, work, and think.
Data collec5on • List of libraries with discovery services > Searched on lib-‐web-‐cats • Surveyed Libraries > Discovery service Implemented > ImplementaNon Date (month/year) > Search box locaNon > MarkeNng eﬀort • 149 Libraries Gave Approval > 33 libraries selected for this phase > 6 for each of the 4 major discovery services and a group of 9 libraries with no service
Dataset • 33 Libraries – 28 US, 2 CA, 1 each from UK, AUS, NZ – WorldCat book holdings > Average: 1,114,193 ; Range: ~300k to ~2.6mil • ImplementaNon dates (Discovery Libraries): > 2010 (3), 2011 (19), 2012 (2) • 6 Publishers • 9,206 Journals • 163,545 Usable ObservaNons
Methodology Compared COUNTER JR1 total full text arNcle views for the 12 months before vs 12 months aeer implementaNon date June 2010 Start ImplementaNon May 2011 May 2012 End Year 1 Year 2 Included implementaNon month in Year 1 to ensure that both periods included an enNre academic year
Examine Data for Outliers
Observa5ons by Library & Service
Observa5ons by Publisher
Average Usage Change By Discovery & Publisher
Analyzing Usage Change: % vs Total Use 12 months before Use 12 months aRer % Change Total Change Journal A 500 600 20% 100 Journal B 5 15 200% 10 Which is the beier measure? Is it the same for publisher-‐ & journal-‐level data?
Reducing varia5on due to ins5tu5on size Currently converNng to change per FTE Values are shown as x 1,000 to bring the change metric back per journal-‐library combinaNon to a minimum of 0.1 2013 JISC Discovery study took a similar approach
Average Usage Change By Discovery & Publisher Per Journal & Per 10,000 FTE
Full Model Including Discovery Service, Publisher, and Library Including Discovery Service, Publisher, and Library
Does the effect of discovery service differ across libraries? Library 10-‐15 Library 16-‐21 Library 22-‐27 Library 28-‐33 Library 1-‐9
Nested ANOVA Model [all three factors – preliminary results]
Does usage change vary across libraries? Institution (sorted by Mean Change)
Does usage change vary across publishers? Publisher (sorted by Mean Change)
Does usage change vary across discovery services?
Publisher Does the eﬀect of discovery service diﬀer across publishers?
Results Can we detect diﬀerences between Discovery Services, Publishers, and/or Libraries and/or their interac4ons? • Library – Yes • Publisher – No • Discovery Service – Yes • DiﬀerenNal discovery service eﬀect by publisher – Yes
Next Steps • Design & test for eﬀects of: – Aggregator full text availability – Publisher Size – Journal Subject – Overall usage trends (Requires Disc Srvc ‘control’) – ConﬁguraNon opNons in Discovery services • Expand pool of libraries • Perhaps explore WHY
Sharing Data • With par5cipa5ng libraries – Customized reports for each library • With par5cipa5ng publishers – Customized reports for each publisher – PresentaNons as requested • With discovery vendors – PresentaNons as requested • In publica5ons and presenta5ons – Maintaining anonymity of data
Doing “Resarch”, SCELC Style! • Why SCELC? • SCELC Funding – staNsNcs consultant – research & wriNng retreats – See hip://bit.ly/1dNMDL3 for more detail • SCELC libraries encouraged to parNcipate in next round – Survey: hip://bit.ly/DSparNcipaNon
Home; Education; Discovery impact scelc colloquium 2014mar05; Discovery impact scelc colloquium 2014mar05 Oct 21, 2014 Education
Home; Education; Discovery impact scelc colloquium 2014mar05; Download. of 31 ×
Discovery impact scelc colloquium 2014mar05 ...
A Major Longitudinal Study of the Effect of Web-Scale Discovery Services on Online (Journal) ... Usage SCELC Colloquium March 5, Published by ...
Impact & Assessment of Summon @ USC Beth Namei, University of Southern California 3/5/2014 ... Discovery impact scelc colloquium 2014mar05.
SCELC Colloquium Lightning Round 2016 Speaker Contact ... Behind the Scenes With the SCELC Electronic ... what do librarians really think of our discovery
Jason Price, PhD. Director of ... 2015.02.24 Discovery System Impact on Usage of ... Panel Discussion with Dan Tonkery and Gordon Tibbets at the SCELC ...
... and implement projects that will have an impact on SCELC ... of the SPIF awards. SCELC staff ... projects at a SCELC event (Colloquium, ...