Published on May 7, 2014
What We Know and How We Know it FIPSE Results Prepared by: Jill Perry Debby Zambo Susan Wunder Contributors: Ray R. Buss, Ron Zambo, and Tiffany R. Williams
Fund for the Improvement of Post- secondary Education (FIPSE) grant • received in 2010 • focused on 21 original Phase I members • sought to document and evaluate: 1. change in the structure of graduate schools 2. change in the signature learning processes, learning environments, and patterns of engagement of faculty and candidates in CPED- influenced EdD programs 3. fidelity to a set of guiding principles developed in Phase 1 Disseminate lessons learned and best practices.
Overall Data and Analytical Process DATA: 21 cases and 3 surveys (student, faculty, researcher) with both close- and open-ended items Analysis of all measures aimed at ensuring credibility/trustworthiness/validity/reliability ANALYSIS OF CASES: Focused on finding commonalities and complexities within and across CPED institutions. • performed f-2-f and virtually • entailed multiple iterations • cases read reread, examined through theoretical framework (Rogers) for answers to RQs • cases coded and re-coded • matrix created for each case • from matrices themes developed and from these claims/assertions made
Understanding How Schools of Education have Redesigned the Doctorate of Education Jill Alexa Perry Debby Zambo Susan Wunder Paper presented at the 2014 American Educational Researchers Association Annual Meeting
Theoretical Frame and Methodology • Rogers (1995) Diffusion of Innovation • original data collected and analyzed by by 38 researchers – wrote 21 cases • cross-case analysis conducted by 3 researchers • proceeded through multiple levels
Prior to Joining CPED Institutions Experienced Issues and Pressures Internal issues and confusion: • coursework not distinct • low quality dissertations • students in wrong programs • declining enrollments • ABDs External pressures: • state level – improved leadership preparation • districts and organizations – better prepared employees and research partners • students – programs to prepare them to take on leadership roles
CPED Influenced Policy • time to degree • number of required degree credits • dissertation format • dissertation oversight
CPED Influenced Programs Programs incorporated and used (in varied ways) • CPED’s six principles • CPED’s six design features scholarly practitioner Cohorts Courses • Content, sequence and focused on practice New pedagogies Collaborative learning environments Intensified patterns of engagement Dissertations in practice
CPED had an Impact on Deans • new ways to bargain and collaborate • communication opportunities and status with upper administration and other deans • cache that allowed them to introduce the idea of programmatic change • support required
CPED had an Impact on Faculty • shift in workload and faculty positions • shift in pedagogy • shift in relationships with students • a national network (convenings) • cache • not all faculty open to change and could slow change • junior tenure-track faculty fit into tenure • some practitioners hired as clinical faculty did not feel welcome
CPED had an Impact on Students • clearer direction - sequence of courses • focused on their own problems of practice and professional goals • respect for their practitioner knowledge • extended communication and interaction with faculty • cohorts and support groups • satisfaction with their programs
Cross Case Conclusions • Schools of education adopted the CPED design features and principles and diffused them throughout their organizations to create innovative and distinct EdD programs. • Changes occurred in the signature learning processes, learning environments, and patterns of engagement. • Lessons learned and best practices are emerging.
Seven Years After the Call: Students’ and Graduates’ Perceptions of the Re-envisioned Ed.D. Ron Zambo Debby Zambo Ray R. Buss Jill Alexai Perry Tiffany R. Williams Innovative Higher Education 2013
Student Survey 1. What are students in newly designed Ed.D. programs learning? Does what they are learning align with CPED’s principles? 2. How are students in Ed.D. programs learning? Does this type of learning/teaching align with CPED’s design concepts? 3. Do students in Ed.D. programs see themselves as scholarly practitioners? If so, what does this mean? 4. Why are students pursuing an Ed.D.?
Instrument Online questionnaire 6-point Likert scale (Strongly Agree to Strongly Disagree) • 32 close-ended items based on CPED’s vision of a scholarly practitioner, its principles, and design features • 1 open-ended item asking participants why they pursued an Ed.D. from a CPED-influenced program
Participants • 296 respondents - 14 (67%) of the 21 institutions • 266 students currently enrolled in a program • 30 recent graduates • 64% female - 36% male • held education related positions for 14 years 143 (65.9%)PK-12 69 (31.8%) post secondary education (2.3%) professions outside of education
Construct with CPED Principle Mean SD Alpha Range by Institution Learning to collaborate and form partnerships (Prin. #3) 5.26 0.81 0.78 4.75-5.71 Learning to apply what they learn to solve problems of practice (Prin. #6) 5.11 0.84 0.80 4.42-5.68 Learning to connect theory to their practice (Prin. #5) 5.08 0.88 0.83 4.27-5.82 Becoming leaders working toward positive change (Prin. #2) 5.06 0.83 0.65 4.36-5.45 Becoming scholarly practitioners (Broad Goal) 5.02 0.76 0.79 4.31-5.54 Learning to engage with diverse communities and work toward social justice (Prin. #1) 4.73 0.91 0.74 4.25-5.70 Learning through authentic experiences (Prin. #4) 4.55 1.21 0.72 3.53-5.36
Open-Ended Item: Why an Ed.D.? • professional, career related advancement • personal reasons • development and growth • because of the degree itself
Faculty Members’ Responses to Implementing New EdD programs Ray R. Buss Ron Zambo Debby Zambo Jill Alexia Perry Tiffany R. Williams under review
Faculty Survey 1. How and to what extent have variables associated with Rogers’ theory on diffusion and adoption of an innovation influenced program changes, implementation, and outcomes in newly designed/redesigned EdD programs? 1. How and to what extent have CPED principles related to EdD program improvement influenced program changes, implementation, and outcomes in newly designed/redesigned EdD programs?
Instrument Online questionnaire 6-point Likert scale ( Strongly Agree to Strongly Disagree) 55 close-ended items that asked about: • changes to their programs • benefits resulting from their participation in CPED • use of the six CPED principles • conceptualizations and outcomes of various redesign efforts • communication channels, time, social system of faculty members and adoption of redesign efforts two open-ended items: • Describe two important changes that occurred in your program because of your participation in CPED • Do you have any comments or questions?
Participants 61 faculty members from 12 institutions who had shaped, developed, and worked in their EdD program
Criterion variable df, F test statistic and p Adjusted R2 Individual Predictor Variables that Were Statistically Significant Program changes F(2, 58) = 76.65, p < .001 .72 Communication channels, CPED Principle 4 Innovation implementation F(3, 57) = 40.65, p < .001 .67 Social system of faculty members, Communication channels, CPED Principle 1 Program orientation F(3, 57) = 44.13, p < .001 .68 CPED Principle 1, Social system of faculty members, CPED Principle 4 Program attractiveness F(2, 58) = 11.20, p < .001 .25 Social system of faculty members, Time Program learning environment F (2, 58) = 29.97, p < .001 .49 Communication channels, CPED Principle 4 Program benefits from participating in CPED F(3, 57) = 40.23, p < .001 .66 CPED Principle 2, Communication channels, CPED Principle 4 All construct scales demonstrated acceptable levels of reliability (0.70 or higher) except for principle #4
Open-ended Items Changes and Comments Six themes: • program focus/orientation • program changes • logistics of program implementation • faculty members’ perspectives • program outcomes because of changes • value of participating in CPED
Researching the Researchers: The Influence of a Sense of Belonging on Faculty and Student Research Volunteers Debby Zambo Ray R. Buss Ron Zambo Jill Alexia Perry Paper presented at the 2014 American Educational Researchers Association Annual Meeting Paper under review
Researcher Survey 1. What were the CPED-FIPSE researchers’ motivations to volunteer? 2. What did the researchers learn through their participation in the research project? 3. What was the greatest benefit the researchers gained as a result of participating? 4. Would the researchers participate again?
Instrument Online questionnaire 6-point Likert scale ( Strongly Agree to Strongly Disagree) 22 close-ended items • six constructs loosely based on Nambisan and Baron (2007) sense of belonging • reliability of each construct ranged from .71 to .96. five opened-ended items • Why did you volunteer for the FIPSE research project? • What did your learn from your participation in the research project? • What did your learn about CPED? • What was the greatest benefit you gained as a result of participating? • If you were asked to volunteer again would you do it?
Participants • twenty-seven (out of possible 38) completed the questionnaire (71% response rate) • from various CPED-institutions
Construct Group Faculty Fellows Overall Cognitive Benefits 4.38 (1.40)* 5.90 (0.32) 4.94 Connectedness to Research Group 4.85 (0.68) 4.50 (1.44) 4.72 (Social-integrative Benefits) Connectedness to CPED 4.94 (0.87) 4.78 (0.95) 4.88 (Social-integrative Benefits) Personal Expectations 4.85 (0.77) 5.40 (0.70) 5.05 (Personal-integrative Benefits) Needs Fulfilled (Hedonistic) 5.12 (0.60) 4.83 (1.25) 5.01 Usefulness of Training Materials 4.65 (0.89) 5.50 (0.50) 4.96 *Note: SD are in parentheses.
Open-ended Items Question 1—Reasons for volunteering Faculty • learn about research, programs, and change • reciprocate • network Fellows (students) • learn about qualitative research and case study methodology • strengthen research skills • apply what they had learned • socialize • encouraged to join
Question 2—What participants learned Faculty • information about CPED as an organization • variation in programs • struggles associated with making changes Fellows • learned about CPED in ways that were different from faculty • variations in programs • faculty relationships • change process
Question 3—Benefits of participating Faculty • learning about change • validation of changes in their programs (reputation) • self-efficacy Fellows • research • relationships/networks
Question 4—would they participate in the future? Faculty 78% would, 17% would not, and 5% would, but had reservations would - they and their programs benefited would with reservations - if it fit their research agenda (tenure) would not – too time consuming Fellows 50% would, 20% would not, and 30% would, but had reservations • would - relationships, exposure to like-minded people, work with faculty members • would with reservations , time and resources • would not – not enough involvement, took too much work
Conclusions cognitive benefits (4.94) • learned about CPED, research, programs, and change from like-minded individuals closest to the source social-integrative benefits (4.72) • network • socialize –form relationships personal-integrative Benefits (5.52) • reputation and self-efficacy (faculty) hedonistic (5.01) • most would volunteer again
Copyright 2014 by the Carnegie Project on the Education Doctorate, Inc. (CPED). The foregoing material may be used for noncommercial educational purposes, provided that CPED is acknowledged as the author and copyright holder. Any other use requires the prior written consent of CPED.
1. What We Know and How We Know it FIPSE Results Prepared by: Jill Perry Debby Zambo Susan Wunder Contributors: Ray R. Buss, Ron Zambo, and Tiffany R ...
Fund for the Improvement of Postsecondary Education - Finding a FIPSE Project Evaluator
In the final year (2013-2014) of the CPED-FIPSE Grant, the lead research team will be publishing results from the cross-case analysis. A preliminary paper ...
Fund for the Improvement of Postsecondary Education -- FIPSE Competitive Grant Evaluation Requirements
Fund for the Improvement of Postsecondary Education iv Foreword During the early 1970s, the Fund for the Improvement of Postsecondary Education (FIPSE ...
This section provides information addressing three research questions raised by the proposal. First was whether or not the higher visibility and more ...
FIPSE Findings at the Center of Our Focus Where did these findings come from? 10 Suggestions Ready to Be Here? Freshmen Academic Abilities and Preparation
FIPSE grants are awarded to organizations focused on improving postsecondary educational ... present and disseminate the findings and products of ...
Increasing Access to Quality Learning Through Effective Use of Peer Feedback ... Findings to date suggest that the quality of ... (FIPSE), a program of the ...