Ellen Wagner: Putting Data to Work

50 %
50 %
Information about Ellen Wagner: Putting Data to Work
Education

Published on February 27, 2014

Author: alexandrapickett

Source: slideshare.net

Description

Ellen Wagner, Executive Director, WCET.
Putting Data to Work
This session explores changing data sensibilities at US post-secondary institutions with particular attention paid to how predictive analytics are changing expectations for institutional accountability and student success. Results from the Predictive Analytics Reporting Framework show that predictive modeling can identify students at risk and that linking behavioral predictions of risk with interventions to mitigate those risks at the point of need is a powerful strategy for increasing rates of student retention, academic progress and completion.
presentation at the 15th annual SLN SOLsummit February 27, 2014
http://slnsolsummit2014.edublogs.org/

From Reporting to Insight to Action: How Data are Changing Post-secondary Education Ellen D. Wagner Chief Strategy Officer, PAR Framework

Session Overview • This session explores changing data sensibilities at US postsecondary institutions. Particular attention is paid to how predictive analytics are changing expectations for institutional accountability and student success. • Results from current work in postsecondary education show that predictive modeling can effectively identify students at risk. • Is predicting risk enough to move the needle on risk mitigation to improve student success? • What does this mean for online learning?

Setting the Context: Data Are Changing Everything

“Meh… education researchers have always worked with data.” • • • • We do qualitative research with data We do quantitative research with data We do evaluations with data We develop surveys and instruments and experiments to collect more data • We pull data from LMSs, SISs, ERPs, CRMs … • We write reports, summaries, make presentations, develop articles and books and webcasts….

What is the one thing we don’t do??? Data mining

Data Optimize Online Experience The “digital breadcrumbs” that online technology users leave about viewing, engagement and behaviors, interests preferences provide massive amounts of information that can be mined to better optimize online experience. It’s about convenience, personalization, recommendations, just-in-time, just-the-right-device.

What do we want? The RIGHT Answers!! When do we want them? NOW!!

Getting to the right answer takes work • Analysis and model building is an iterative process • Around 70-80% efforts are spent on data exploration and understanding. SAS Analysis/Modeling Process

Three Opportunities on our Horizons • Linking predictions to action • Creating new insights and opportunities with data • Delivering on the promise of what online learning can be

Link Predictions to Action • Predictive analytics refer to a wide varieties of methodologies. There is no single “best” way of doing predictives. You need to know what you are looking for. • Simply knowing who is at risk is simply not enough. Predictions have value when they are tied to what you can do about it. • Linking behavioral predictions of risk with interventions at the best points of fit offers a powerful strategy for increasing rates of student retention, academic progress and completion.

Create new insights and opportunities for data in our practices • • • • • • • Enrollment management Student services Program and learning experience design Content creation Retention, completion Gainful employment Institutional Culture

Delivering on the promises of what Online Learning can be • Online learning is more than MOOCs, but that has now become the public perception of what we do. • We are watching our our practice disaggregated into 3rd party platforms and apps. • We find ourselves at the center of strategic conversations, but the ways in which we are evaluated continue to miss the mark. • We need to get out of the way of what we’ve been and embrace where we need to go.

Online Learning as a Catalyst for Changing Data Sensibilities Sloan-C Babson, 2013

And yet…… Sloan-C Babson, 2013

Faculty still don’t trust it Sloan-C Babson, 2013

16

How Are We Doing So Far? • Data analytics are still emerging. Many organization still rely on traditional technology (e.g. spreadsheets) and methods (e.g. inferential statistics). • Analytics tend to be used narrowly within departments and business units, not integrated across the institutions. • Intuition based on experience is still the driving factor in datadriven decision-making. Analytics are used as a part of the process. 17 Bloomberg BusinessWeek Research Services Analytics Insights (2014):

How Are We Doing So Far? • Data is the number 1 challenge in the adoption and use of analytics. Organization continue to struggle with data accuracy, consistency, access. • Analytics to solve big issues, with the primary focus on reducing costs, improving the bottom line, managing risk. • Many organizations lack the proper analytical talent. Organizations that struggle with making good use of analytics often don’t know how to apply the results. • Culture plays a critical role in the effective use of data analytics. 18

National Non-profit Multi-institutional Collaborative Institutional Effectiveness + Student Success

About PAR Framework • Established, growing non-profit collaborative focused on using existing institutional data to improve institutional effectiveness and student outcomes • Funded by Bill & Melinda Gates Foundation 2011, 2012, 2013 • Managed at the Western Interstate Commission for Higher Education • Engagement with more than 39 forward thinking US institutions • Small, high functioning team with partner, subject and domain expertise • In-kind donations to date ▫ IBM Tableau ▫ Blackboard iData ▫ Starfish

DATA STATISTICS • Total Counts Time Frame • August 2009 – May 2013 – 13,090,351 course records – 1,842,917 student records

PAR Objectives Creating scalable solutions for institutional effectiveness and student success through -common data definitions -common measures -institutional collaboration

Structured, Readily Available Data • Common data definitions = reusable predictive models and meaningful comparisons. • Openly published via a cc license @ https://public.datacook book.com/public/institu tions/par

Speak the same language

PAR Core Competencies, helping members IDENTIFY Benchmarks Provide insight into how institutions compare to their peers through common measures by scaling multiinstitutional datasets for benchmarking and research purposes. TARGET Models Identify which students need assistance, by using in-depth, institutional specific predictive models Models are unique to the needs and priorities of our member institutions based on their specific data. TREAT Interventions Determine the best way to address areas of weakness identified in benchmarks and models by scaling and leveraging a member and literature validated framework for examining interventions within and across institutions (SSMx)

Different Levels of Insight PAR Benchmarks Descriptive Analytics PAR Models Predictive Analytics Cross Institutional Student/degree/major level insight into: 1. What did the retention look like for students entering in the same cohort 2. How does your institution compare to peer institutions / institutions in other sectors 3. What was the relationship of student attributes 4. What were the attributes and performance outcomes Institutional Specific insight into: 1. What students are being retained over time? 2. Which students are currently at risk for completing and why? 3. Which factors are directly correlated to student success?

DATA DELIVERY AND QA TOOLS • • Automated response and self service Q1 2014 300 automated tests

BENCHMARKS • • • • • • • Available now Unlimited institutional users Released November / May Member driven report development Expanded report sets Releasing on tablets/devices Dynamic institution

INSTITUTIONAL PREDICTIVE MODELS • • • • • Delivered a a limited beta Member driven institutional model targets selection Migrating SAS Visual Analytics delivery with next data set Unlimited number of institutional logins Delivered up to 3x a year year, within 21 days of data acceptance Rapid turn model publications for watch lists

INTERVENTION INVENTORY TOOLS • • • • • • • Online application built on SSMx Lays groundwork for intervention benchmarks Enables institutional snapshot on expanded intervention dataset Ver. 1 launches Q1 ‘14 Ver. 1.1 with reporting Q3 ‘14 Designed for integration with benchmarks and models Full integration of reporting Q3 ‘14

Thank you for interest Ellen D. Wagner, Ph.D. edwsonoma@gmail.com ellen.wagner@parframework.org http://.parframework.org http://twitter.com/edwsonoma +1.415.613.2690 mobile 31

Add a comment

Related presentations

Related pages

Putting Data to Work (Ellen Wagner)

Putting Data to Work (Ellen Wagner) SUNY Presenter; Powered by Mediasite - webcasting platform
Read more

Putting to your Robots to Work V1.1 - Technology

Ellen Wagner: Putting Data to Work ... Putting It to Work in your Organization. Whitepaper: Putting Big Data & Analytics to Work for Your Business.
Read more

The eLearning Guild : Data & Analytics Summit 2016 - Program

... Putting Data Analytics in Context. JD Dillon, ... Ellen Wagner, Chief Research ... you will explore work being done to fulfill the promise of the ...
Read more

What’s Next for Learning, Technology, and our Profession ...

... by Ellen Wagner Five Steps for Achieving Learning Analytics Success slides by Ellen Wagner. Performance Support Big Data ... Putting Curation to Work ...
Read more

PAR at Hobsons: Joining Research and Predictive Models ...

CT talks with Ellen Wagner, ... Joining Research and Predictive Models with Real Time Data. A Q&A with Ellen Wagner. ... the ability to work with data ...
Read more

A Collaborative for Student Success and Institutional ...

A Collaborative for Student Success and Institutional Comparisons. A Q&A with PAR Chief Research and Strategy Officer Ellen Wagner. ... We are not putting ...
Read more

The eLearning Guild : Curated Content

Content provided for these publications comes from ... Putting Data Analytics in ... (host), Ellen Wagner, Sean Putman, JD Dillon 9.21.16 | Online ...
Read more

The Predictive Analytics Reporting (PAR) Framework, WCET

The Predictive Analytics Reporting (PAR) Framework, ... undergraduate student data and contribute an inventory ... Ellen Wagner is executive director ...
Read more