Not Dead Yet! A Longitudinal Study of Query Type and Ready Reference Accuracy in Live Chat and IM Reference.

100 %
0 %
Information about Not Dead Yet! A Longitudinal Study of Query Type and Ready Reference...
Education

Published on March 3, 2014

Author: LynnConnaway

Source: slideshare.net

Description

Presented at ALISE Conference, January 24, 2013, Seattle, Washington.
Received ALISE/Bohdan S. Wynar Research Paper Award, 2013.

Marie L. Radford Chair, Department of Library & Information Science Rutgers University mradford@rutgers.edu Lynn Silipigni Connaway Senior Research Scientist OCLC ALISE/Bohdan S. Wynar Research Paper Award connawal@oclc.org The world’s libraries. Connected. ALISE, January 22-25, 2012 http://www.oclc.org/research/activities/synergy.html

Seeking Synchronicity & Cyber Synergy The world’s libraries. Connected.

Transcript Analysis Seeking Synchronicity 2004-2006 (QP1) Live Chat Transcripts 2004-2006 (QP1) • Total QuestionPoint & 24/7 = 651,687 • 850 randomly selected • 550 QP & 300 24/7 Cyber Synergy 2010 (QP2 & QW) Live Chat & Qwidget Transcripts 2010 (QP2 & QW) • Total QP2 & QW = 296,797 • QP2 221,158 & QW 75,639 • 560 randomly selected • 350 QP2 & 210 QW The world’s libraries. Connected.

Research Questions – Cyber Synergy • How have the mix & frequency of types of VRS questions changed? • How frequently are ready reference questions asked in VRS? • How has the rate of accuracy of answers to VRS ready reference questions changed? • What differences are there in mix, frequency, & accuracy for live chat compared to IM/QW VRS modes? The world’s libraries. Connected.

Seeking Synchronicity & Cyber Synergy Transcript Analysis Results Types of Questions The world’s libraries. Connected.

Types of VR Questions • Subject Search • Ready Reference • Procedural • No Question • Holdings • Research • Inappropriate • Directional • Reader‟s Advisory The world’s libraries. Connected.

Subject Search or Specific Search “Almost always takes the form of giving the user a document, for example, a list of citations, a book, or a report” (Arnold & Kaske, 2005). How was the labeling theory important in deinstitutionalization? (QW–023) can you help me find poems by Maya Angelou and criticism on them? (QP1–230) The world’s libraries. Connected. Where can i find information about ADHD? (QP2–005)

Ready Reference “These are the typical ready-reference or data queries that require only a single, usually uncomplicated, straightforward answer...Who? What? When? Why? Where?” (Arnold & Kaske, 2005). who won the world cup game between south africa and france? (QW–024) Who was Bentonville, NC named after? (QP1–147) The world’s libraries. Connected. How do i cite a political talk show in MLA format? (QP2–013)

Procedural “Questions pertaining to the policies or procedures within the library system” (Arnold & Kaske, 2005). Can I get summer long term renewals online? How? (QP1–004) what is the max for checkout on blu ray dvids? (QW–033) The world’s libraries. Connected. need access logon info to lexis from scool library website (QP2– 027)

Holdings “Questions about specific holdings of a library in print or digital form” (Arnold & Kaske, 2005). I was wondering if you have textbooks to rent for an hour or 2? (QP2–172) Do you have any books on Paris Hilton? (doing a project on her) (QP 1–195) The world’s libraries. Connected. hi there; i was wondering if you guys have Moby Dick in stock? (QW–014)

Research “Research questions… involve trial-and-error searching or browsing… [and] are usually identified as coming from an adult specialist who is seeking detailed information to assist in specific work” (Arnold & Kaske, 2005). Looking for information on national traffic survey and incident reports. (QP2–169) Hi, I am trying to find out information on how the Learn Direct initiative was started, the criteria that was set and if local government had to be involved in the first stage. (QP 1–155) The world’s libraries. Connected. Hi I‟m looking for studies similar to one I‟ve already found. How do I go about this? (QW–013)

Inappropriate “Questions which are not appropriate for a reference service including personal questions” (Radford, 2005). How do I have sex? (QP1–159) WAT DOES it mean when a guy said he like me more as a friend doesnt it mean like a gf? (QP2–153) The world’s libraries. Connected. Are You Typing War and Peace (QW–020)

Directional “The general information or directional question is of the information booth variety…” (Arnold & Kaske, 2005). what is the URL for the summer reading program for teens? (QP2–032) I heard that you have adapted book kits for ESL speakers. Where can I find these online? (QW–008) The world’s libraries. Connected. Where is the 67th street library? Is it on the west or east side? (QP1–387)

Reader‟s Advisory Reader‟s Advisory questions are “focused on helping readers find materials they want to read, listen to, or view for pleasure” (Ross, Nilsen, & Radford, 2009). Hello I am looking for recommendations for the author jonathan kellerman. (QP1–218) I was wondering if u would happen to know any good fantasy books… (QP2 – 050) The world’s libraries. Connected. Hi, my 7 year old son is looking for a copy of the Hobbit suitable for children, but we can‟t seem to find anything in the library catalogue. Can you please offer any advice? Thanks. (QW– 120) (from Australia)

Query Type: 2004-2006 35% 32% n=293 30% 27% n=243 25% '04 - '06 (n = 915) 18% 20% n=162 15% 11% n=104 8% 10% n=77 5% 0% Subject Search Ready Reference The world’s libraries. Connected. Procedural No Question Holdings

Query Type: 2010 35% 31.1% n=179 31.1% n=181 30% 25% '10 (n=575) 20% 16.9% n=97 15% 8.5% 10% n=49 4.3% 5% n=25 0% Ready Reference Procedural The world’s libraries. Connected. Subject Search Holdings No Question

Query Type: 2004-2006 vs. 2010 35% 32% n=293 31% n=179 30% 31% n=181 27% n=243 25% '04 - '06 (n = 915) 20% 18% 17% n=162 n=97 15% '10 (n = 575) 11% n=104 8% 10% n=77 9% n=49 4% n=25 5% 0% Subject Search Ready Reference The world’s libraries. Connected. Procedural No Question Holdings

Ready Reference Accuracy The world’s libraries. Connected.

Accuracy in VR Ready Reference • How has the rate of accuracy of answers to VRS ready reference questions changed? • Is the 55% rule in effect? (Hernon & McClure, 1987) The world’s libraries. Connected.

Accuracy Transcript Analysis • Subset of ready reference questions • 8/2004 - 10/2006 • n=162 ready reference queries • 6/2010 - 12/2010 • n=168 ready reference queries The world’s libraries. Connected.

Checking for Accuracy You were interested in the kids‟ (up to age 12) summer reading, right? • Teams of coders checked responses • Authoritative websites • Subscription-based databases Yes Yes Here is a link to info on it. http://host8.evanced.info/yor klibraries/sr/homepage.asp Hopefully, that page will answer your questions. (QP2-024) The world’s libraries. Connected. • Any citations/links provided by the librarian/staff member

Categories 1. Correct with Citation 2. Correct without Citation 3. Correct without Citation, No Citation Needed 4. Incorrect with Citation 5. Incorrect without Citation 6. Incorrect (without specific information requested) 7. Other The world’s libraries. Connected.

Ready Reference Questions (2004-2006 vs. 2010) Received by Type of Chat Service 45% 40% n=68 40% 39% n=65 36% n=59 '04-'06 (n=162) 35% 30% 28% '10 (n=168) n=45 25% 21% n=34 20% 20% n=33 12% 15% n=20 10% 5% <1% n=1 2% n=4 <1% n=1 0% Consortium Academic & Law The world’s libraries. Connected. Public National Other

Ready Reference Accuracy Results The world’s libraries. Connected.

Improving Accuracy: 2004-2006 vs. 2010 100% 90% 80% 90% 78% n=151 n=141 '04 - '06 (n=180) 70% '10 (n = 168) 60% 50% 40% 30% 20% n=36 20% 6% n=10 4% 10% n=7 2% n=3 0% Correct The world’s libraries. Connected. Incorrect Other

Ready Reference Accuracy Correct Answers and Citations for 2004-2006 vs. 2010 100% 90% 89% n=125 75% 80% n=114 '04 - '06 (n=141) 70% '10 (n = 151) 60% 50% 40% 30% 14% 20% 6% 10% 11% n=21 n=9 5% n=16 n=7 0% Correct w/ Citation The world’s libraries. Connected. Correct w/o Citation Correct No Citation Needed

Ready Reference is Alive! • % of Ready Reference Queries • Relatively stable • Slight increase over time (4%) • Accuracy much higher than 55% benchmark • 78% („04-‟06) to („10) 90% • Both ready reference accuracy & percentages for correct w/ citation improved The world’s libraries. Connected.

Explanations for Increase • VRS service providers more facile in chat & IM environment • Providers feel less rushed • Have followed authors‟ previous recommendations? The world’s libraries. Connected.

Where to Go from Here The world’s libraries. Connected.

Recommendations to Improve Accuracy • Clarify question • Provide citations • Check for broken links • Make sure all parts of a query are answered • Make sure specific information requested in presented before pushing a general website The world’s libraries. Connected.

Future Research Directions Investigate: • Accuracy in other question types & SQA • Data-derived taxonomy of accuracy ask a librarian • SQA accuracy • Level of difficulty (using READ model) • Staffing models • Both FtF & VR desks • Managing multiple chat windows • Lean staffing in tight budgets • More collaboration/crowd-sourcing • Mobile access The world’s libraries. Connected.

References Arnold, J., & Kaske, N. (2002). Chat with a Librarian. Computers in Libraries Annual Conference and Exhibition Usa-, 1-6. Arnold , J., & Kaske, N. K. (2005). Evaluating the quality of a chat service. portal: Libraries and the Academy, 5(2), 177-193. Connaway, L. S., Dickey, T. J., & Radford, M. L. (2011) “If it is too inconvenient I'm not going after it:” Convenience as a critical factor in information-seeking behaviors. Library & Information Science Research, 33(3), 179-190. Connaway, L. S., Radford, M. L., & OCLC Research. (2011). Seeking synchronicity: Revelations and recommendations for virtual reference. Dublin, Ohio: OCLC Research. http://www.oclc.org/research/activities/synchronicity/default.htm Hernon, P., & McClure, C. R. (1987). Unobtrusive testing and library reference services. Norwood, New Jersey: Ablex Publishing. Janes, J. (2003). What is reference for? Reference Services Review, 31(1), 22-25. Nummien, P., & Vakkari, P. (2009). Question types in public libraries' digital reference service in Finland: Comparing 1999 and 2006. Journal of the American Society for Information Science and Technology, 60(6), 1249-1257. Pearce, A., Collard, S., & Whatley, K. (2010). SMS reference: Myths, markers, and modalities. Reference Services Review, 38(2), 250-263. Pew Internet & American Life Project. (2012a). Nearly half of American adults are smartphone owners. Retrieved on April 13, 2012 from http://www.pewinternet.org/Reports/2012/Smartphone-Update-2012/Findings.aspx The world’s libraries. Connected.

References Pew Internet & American Life Project. (2012b). Teens, smartphones & texting. Retrieved on April 13, 2012 from http://www.pewinternet.org/~/media//Files/Reports/2012/PIP_Teens_Smartphones_and_Texting.pdf Radford, M. L., & Connaway, L. S. 2005–2008. Seeking Synchronicity: Evaluating virtual reference services from user, non-user, and librarian perspectives. Funded by the Institute of Museum and Library Services (IMLS), Rutgers University, and OCLC http://www.oclc.org/research/activities/synchronicity/default.htm Radford, M. L., & Connaway, L. S. (2013). Not dead yet! A longitudinal study of query type and ready reference accuracy in live chat and IM reference. Library & Information Science Research, 35(1), 2-13. [Available: http://www.oclc.org/resources/research/publications/library/2012/radford-connaway-lisr.pdf] Radford, M. L., Connaway, L. S., Confer, P., Sabolsci-Boros, S., & Kwon, H. (2011). “Are we getting warmer?” Query clarification in live chat virtual reference. Reference & User Services Quarterly, 50(3), 259-279. Radford, M.L., Connaway,L.S. & Shah, C. (2011-2013). Cyber synergy: Seeking sustainability through collaboration between virtual reference and social Q&A sites. Funded by the Institute of Museum and Library Services (IMLS). http://www.oclc.org/research/activities/synergy/default.htm Radford, M. L., & Vine, S. (2011). An exploration of the hybrid service model: Keeping what works. In: Diane Zabel (Ed.), Reference Reborn: Breathing New Life into Public Services Librarianship, pp. 79-89. CO: Libraries Unlimited. Ross, C. S., Nilsen, K., & Radford, M. L. (2009). Conducting the reference interview (2nd ed.). New York: NealSchuman. Stacy-Bates, K. (2003). E-mail reference responses from academic ARL libraries. Reference & User Services Quarterly, 43(1), 59-70. The world’s libraries. Connected.

Funding & Acknowledgements Cyber Synergy: Seeking Sustainability through Collaboration between Virtual Reference and Social Q&A Sites • $250,000 for 2011-2013 • Funded by IMLS, OCLC, & Rutgers University • Co-PIs Marie Radford (RU), Lynn Silipigni Connaway (OCLC), & Chirag Shah (RU) http://www.oclc.org/research/activities/synergy.html • We thank Alyssa Darden, Lisa DeLuca, Kathy Juliano, Stephanie Mikitish, Janet Torsney, & Jacqueline Woolcott for assistance in coding, analysis, & data presentation. The world’s libraries. Connected.

Questions & Discussion Marie Radford mradford@rutgers.edu Lynn Silipigni Connaway connawal@oclc.org The world’s libraries. Connected.

Add a comment

Related presentations

Related pages

OCLC Senior Research Scientist Lynn Silipigni Connaway ...

OCLC Senior Research Scientist Lynn Silipigni ... “Not dead yet! A longitudinal study of query type and ready reference accuracy in live chat and ...
Read more

Conference Presentation: “Not Dead Yet: A Longitudinal ...

Conference Presentation: “Not Dead Yet: A Longitudinal Study of Query Type and Ready Reference Accuracy in Live Chat and IM Reference”
Read more

Seeking Synchronicity: Evaluating Virtual Reference ...

"Not Dead Yet! A Longitudinal Study of Query Type and Ready Reference Accuracy in Live Chat and ... http://www.oclc.org/research/publications/library ...
Read more

Marie L. Radford, Ph. D., Rutgers University

Not dead yet! A longitudinal study of query type and ready reference accuracy in live chat and IM reference. ... Marie L. Radford and R. David Lankes ...
Read more

OCLC Research 2012: and the winner is ...

OCLC Research 2012: and the winner is ... for her article “Not dead yet! A longitudinal study of query type and ready reference accuracy in live chat and ...
Read more

Social question and answer services versus library virtual ...

... but not dead yet! Revitalizing reference for the ... Not dead yet! A longitudinal study of query type and ready reference accuracy in live chat and IM ...
Read more

Marie L. Radford Wins ALISE/Bohdan S. Wynar Research Paper ...

Marie L. Radford Wins ALISE/Bohdan S. Wynar Research Paper ... Dead Yet! A Longitudinal Study of Query Type and Ready Reference Accuracy in Live Chat ...
Read more

Wynar Research Paper Competition - Alise Home

ALISE/Bohdan S. Wynar Research Paper ... OCLC for Not dead yet! A longitudinal study of query type and ready reference accuracy in live chat and IM ...
Read more

Five-Factor Model - Great Ideas in Personality--Theory and ...

... not types, so people vary ... The Big Five personality traits and the life course: A 45-year longitudinal study. ... Five-Factor Model: Reference Source.
Read more