Published on February 28, 2008
Using Research to Build Effective School Reading Programs in High Poverty Settings: Home grown and bottom up ain’t so bad!: Using Research to Build Effective School Reading Programs in High Poverty Settings: Home grown and bottom up ain’t so bad! P. David Pearson, UC Berkeley Barbara Taylor, U of Minnesota Debra Peterson , U of Minnesota Michael Rodriguez , U of Minnesota www.ciera.org A word about CIERA: A word about CIERA The federal funding ended Sept 30, 2002 Extended through Sept 30, 2003 No more funding Name and website still exists www.ciera.org CIERA lives on at Michigan website The work of Barbara Taylor at Minnesota and my work at UC Berkeley But first…, a pre-test…: But first…, a pre-test… The term, Scientifically-Based Reading Research, occurs ____ times in the Reading First Portion of NCLB: The term, Scientifically-Based Reading Research, occurs ____ times in the Reading First Portion of NCLB 115 118 110 An average of 114.3 Phonemic Awareness is…: Phonemic Awareness is… The ability to discern separate sounds in the stream of speech The process by which a unit of sound can experience self-actualization Something a person needs to practice a bit more IF he consistently says NUCULAR for NUCLEAR. Reading Recovery is…: Reading Recovery is… A much missed part of our current portfolio of early interventions The plaintiff in a law suit with the federal Department of Education A 12 step program designed to assist 1st graders who have overdosed on Accelerated Reader Reading First…: Reading First… Is the reading part of NCLB Has provided more resources for compensatory reading than any previous piece of legislation in our history Means that math is a curricular second and science and social studies are not even on the radar screen (not to mention art and music)… Overview: Overview Report on a six year line of work, with extensions into subsequent work over 10 years. Now that our project has ended, review the entire scope of work to assess what it adds up to Talk about our work in elementary schools that “beat the odds” Talk about our work with aspiring schools Expand outward to other studies of “outlier” schools and teachers Talk about follow up work extending into Reading First and other voluntary reform movements The real goal: to provide an alternative model to the top-down approach of current reform movements What have we learned about effective programs and classroom instruction in our now decade long line of work with low income schools?: What have we learned about effective programs and classroom instruction in our now decade long line of work with low income schools? Beat the Odds Study: Phase 1: Beat the Odds Study: Phase 1 Fall of 1997 Fundamental logic If you are going to advise schools about initiating school reform, you had better start by asking the experts, … those schools and teachers who are pros at beating the odds. Participants: Participants 14 Schools from around the country, ranging from 28 - 92 percent poverty 8 had a reputation of beating the odds 3 had nationally recommended programs 3 were “originally” comparison schools Logic: examine differences in growth on a composite set of measures Explain growth differences across classrooms and schools by examining classroom and school inputs ¯ Reasons For Success Offered By Educators In Most Effective Schools.: Reasons For Success Offered By Educators In Most Effective Schools. School Factors Having research-based intervention for those most at risk Ongoing professional development across building Collaboration across grades, across classroom and resource teachers (weak collab: r = -.46) Strong connections to homes (linking to parents) (r = .76) Using data on student performance to set goals, assess progress toward goals Establishing reading as the number one building priority. NOTE: underscore means that our statistical analysis concurs Reasons For Success Offered By Educators In Most Effective Schools.: Reasons For Success Offered By Educators In Most Effective Schools. Teacher Factors Time for students to read authentic texts Small group instruction Teaching strategies, not skills Improving instruction and increasing instructional time High expectations for students’ achievement. NOTE: underscore means that our statistical analysis concurs Teachers in effective schools provide reading opportunity: Time in Independent Reading most effective schools 28 minutes= moderately effective schools 27 minutes> least effective schools 19 minutes Teachers in effective schools provide reading opportunity Teachers in effective schools add coaching to their WR repertoire: Teachers in effective schools add coaching to their WR repertoire Approach to Word Recognition Instruction in Grades 1 and 2 Coaching During Reading Practice on Sight Words Explicit Phonics Instruction 1. Least Effective 13% 60% 40% 2. Moderately Effective 17% 61% 0% 3. Most Effective 53% 60% 27% 3>2=1 3>2, 1>2 Teachers in effective schools emphasize higher order questions: Teachers in effective schools emphasize higher order questions Approach to Comprehension Instruction in Grades 1 through 3 Text-Based Questions Writing in Response to Reading Higher-Level Questions 1. Least Effective Schools 45% 0% 27% 2. Moderately Effective Schools 34% 7% 24% 3. Most Effective Schools 37% 37% 47% 3>2=1 Accomplished teachers organize for small group instruction: Accomplished teachers organize for small group instruction Time in Whole Group Instruction most accomplished 25 minutes = somewhat 29 minutes < least accomplished teachers: 48 minutes Time in Small Group Instruction most accomplished teachers: 48 minutes* somewhat 39 minutes least accomplished 25 minutes *48 min. > 25 min. Accomplished Teachers engage students in their work: Accomplished Teachers engage students in their work .Student Engagement (3.0 possible) most accomplished : 2.9> (96%)* somewhat 2.2 > (84%) least accomplished 1.3 (61%) *% of students on task Slide19: Preferred Coaching Most accomplished: 48% > Somewhat: 21% > Least accomplished: 6% Preferred Telling Most accomplished: 7% < Somewhat: 38% < Least accomplished: 75% Accomplished Teachers work along side their students Accomplished Teachers have a balanced comprehension portfolio: Accomplished Teachers have a balanced comprehension portfolio Approach to Comprehension Instruction in Grades 1 through 3 Text-Based Questions Writing in Response to Reading Higher-Level Questions 1. Least accomplished 24% 0% 18% 3>1, 3>2 3>1 2. Somewhat accomplished 46% 8% 42% 3. Most accomplished 48% 31% 48% Assessment: Assessment All of our effective schools had in place a locally developed OR locally selected assessment system that they used to monitor student progress AND hold themselves to account. The perils of performance assessment: or maybe those multiple-choice assessments aren’t so bad after all…….: "The alimentary canal is located in the northern part of Indiana." The four seasons are salt, pepper, mustard, and vinegar. The perils of performance assessment: or maybe those multiple-choice assessments aren’t so bad after all……. The perils of performance assessment, or watch out for those open-ended questions:: Thunder is a rich source of loudness "Nitrogen is not found in Ireland because it is not found in a free state" The perils of performance assessment, or watch out for those open-ended questions: The perils of performance assessment: "The tides are a fight between the Earth and moon. All water tends towards the moon, because there is no water in the moon, and nature abhors a vacuum. I forget where the sun joins in this fight." The perils of performance assessment The perils of performance assessment: "Water is composed of two gins, Oxygin and Hydrogin. Oxygin is pure gin. Hydrogin is gin and water." To keep milk from turning sour, keep it in the cow. The perils of performance assessment The perils of performance assessment: "Germinate: To become a naturalized German." "Vacumm: A large, empty space where the pope lives.” Genetics explains why you look like your father, and if you don't, why you should. The perils of performance assessment The perils of performance assessment: You can listen to thunder and tell how close you came to getting hit. If you don't hear it, you got hit, so never mind. "When you breath, you inspire. When you do not breath, you expire." The perils of performance assessment Unique aspects of our work: Unique aspects of our work We examined school and teacher factors in the same study We looked much more specifically at aspects of reading instruction Combination of descriptive and quantitative data collection approaches and analyses More information: More information Book on this project Taylor, B.M., & Pearson, P.D. (Eds.) (2002). Teaching reading: Effective Schools, accomplished teachers. Mahwah, NJ: Lawrence Erlbaum. National Studies School cases Classroom portraits 1999-2003: 1999-2003 Working with aspiring schools Schools that would like to be beating the odds but are not yet there Just finished our analyses across 3 years Using the findings from our work and the work of others, we have developed...: Using the findings from our work and the work of others, we have developed... The CIERA School Change Framework, dedicated to… Improving reading achievement through school- wide collaboration and choice Originally called Variations of Choice CIERA School Change Framework Assumptions: CIERA School Change Framework Assumptions No single solution to reform exists. Schools are at different places with different needs. Schools will benefit from becoming collaborative, learning communities. Teachers will benefit from feedback, reflection and change efforts related to teaching practices. School staff must be committed to improving students’ opportunity and performance. Description of the CIERA School Change Framework: Description of the CIERA School Change Framework Six main topics (school change model, professional development, school-wide reading program, classroom instruction, early reading interventions, school/home/communities connections) divided into subtopics. Each subtopic developed through text, short readings (RT articles), research references, video clips of effective practice (where applicable), suggested whole-school and study group activities, links to other resources. Internet-based delivery system. Discussion site for schools participating in project to share ideas. Mini Lesson: Don’t count on technology The Fundamental Elements: The Fundamental Elements Schools examine data (e.g., self-study questionnaire on school-level factors; school report based on data from interviews, classroom observations, student assessments; other existing school data) to decide where to focus in improving reading instruction within school and classrooms Teachers meet 3 times a month in study groups (ideally cross-grade) to examine student work and to improve their teaching of reading; 1 time/month as whole school. Study groups get support from an external facilitator and from the internal leadership team and perhaps the principal Whole Faculty Study Groups - From Murphy and Lick (Corwin Press): Whole Faculty Study Groups - From Murphy and Lick (Corwin Press) The focus is on instruction, specifically research-based practices found to be effective in the teaching of reading. Study groups have action plans that focus on what group members can learn and do to change what and how they teach to improve student learning. Reporting back accountability Whole Faculty Study Groups -From Murphy and Lick : Whole Faculty Study Groups - From Murphy and Lick Teachers collect and analyze data over time on student work and their own teaching to assess the effectiveness of the changes they have made in classroom practices Other Activities for Reflection: Other Activities for Reflection Engage in video sharing Engage in peer coaching Bring a lesson plan and student work to a study group and use a set of questions to reflect on one’s own teaching during the lesson Take 5 minutes at a study group session to reflect in writing on students’ progress and one’s own teaching related to the study group focus. Share with a study group member. What we hoped to promote: What we hoped to promote Building Collaboration among Faculty Links to Parents Instructional reflection, plans for and evidence of change Changing views of of Professional Development Leadership: Principal Facilitator Peer Outcome measures: Outcome measures Gates Comprehension (pre and post) HM (open-ended) Comprehension (pre and post) Writing (rubric driven) (pre and post) WCPM (pre, mid, and post) K-1 (add LN, PA, Dictated Sentences as pre or post measures) Classroom Observation Scheme : Classroom Observation Scheme Kindergarten Listening to Just Grandpa and Me 9:00 – T is reading aloud to the class. Kids initiate conversation about the pictures. She initiates a few comments, “It’s a pretty loud suit,” Now I want you to think in your head – like we do before we share. Is there a time you’ve done something special with a grandpa or grandma, or other relative. Kids bow their heads to think. Then she starts to call on kids. As they say something, she gets them to elaborate. 9:05 10/16 ot c/w/r l/ n/ ra/ l m3 /n/ r, c/ or-tt level 1/2/3 4/ 5/ 6/ 7 4/ 5/ 6 / 7 Creating Classroom Process Variables: Creating Classroom Process Variables Who is working with whom Grouping Practices - large/small group/independent Word-Level Activities: phonemic awareness, phonics skill instruction, coaching in word recognition strategies Comprehension: Skill Instruction/ Comp. Strategy Instruction (routinized versus strategic/flexible) Cognitive Level/Challenge of the activities (questions, assignments): Higher (greater reliance on inference) versus lower (greater reliance on transparent links to text) Creating Categories to Look at Practices: Creating Categories to Look at Practices Teacher Interaction Stance- Teacher Directed (telling, recitation) vs. Student-Support(coaching, modeling, listening and giving feedback) Student Response Active Responding -reading, writing, manipulating, choral responding, oral responding with a partner Passive Responding - reading turn-taking, oral turn-taking, listening to the teacher Key articles: Key articles Taylor, B.M., Pearson, P.D., Peterson, D.S., & Rodriguez, M.C. (2003). Reading growth in high-poverty classrooms: The influence of teacher practices that encourage cognitive engagement in literacy learning. Elementary School Journal, 104 (1), 3-28. Taylor, B.M., Pearson, P.D., Peterson, D.S., & Rodriguez, M.C. (2005).The CIERA school change framework: An evidence-based approach to professional development and school reading improvement. Reading Research Quarterly, 40 (1), 2-32. Looking at Classroom Level Effects: Looking at Classroom Level Effects Fluency: Fluency Grade 1: 35% of the variance in student scores between teachers 35% of the between teacher variance due to HLQ differences: 1 s.d. of an increase in HLQ-->11.4 wcpm More fluency findings: More fluency findings Grades 2-5 46% of the variance in student scores between teachers 13% of that variance accounted for by Coaching + (1 sd 4.2 wcpm ) Active responding + (1 sd 3.0 wcpm Phonics instruction - (1 sd 5.2 wcpm (Perhaps fluency is a proxy for more complex reading outcomes) Looking across grades and measures: Looking across grades and measures HLQ shows up most often (more often than not) Some element of Student-centered stance (coaching or modeling) shows up and telling was negative when it did enter. Some element of active responding (active response or time on task) shows up. Treating processes as encapsulated skills shows up as a negative predictor, but treating them as an embedded activity as a positive predictor (phonics is positive in Gr 1 only) Slide48: CIERA School Change Findings - Grades 2-5 BTO Year 1 Year 2 Year 3 + HLQ (1-3) + CW Rec Str (1-2) + Coaching (1-3) + TOT (1-3) Reading Comp Fluency Telling - (2-3) Reading Comp (std test) HLQ + (2-5) TOT + (2-5) C Skill Prac - (2-5) PSV Rsp - (2-5) Fluency Writing HLQ + (2-5) Modeling + (2-5) Reading Comp (Std tests) Fluency HLQ + (2-5) C Skill Prac - (2-5) Writing Coaching (2-5) + HLQ (4-6) + Reading Comp (Bsl Rdr) HLQ + (2-5) Reading Comp (Bsl Rdr) Actv Rdg + (2-5) C Skill Prac-(2-5) Coaching + (2-5) Active Rdg + (2-5) Phonics Skills - (2-5) Slide49: Grade 1 BTO Year 1 Year 2 Year 3 + HLQ (1-3) + CW Rec Str (1-2) + Coaching (1-3) + TOT Reading Comp Fluency Act Rsp + Reading Comp HLQ + Fluency Writing Comp Str + Telling - Reading Comp HLQ + Ph Skills + Fluency TOT+ Writing HLQ + Lstn/w/gf + HLQ + Teaching for Cognitive Engagement: Teaching for Cognitive Engagement A model of teaching for cognitive engagement Student support stance (coaching, modeling, scaffolding learning) not a teacher directed stance (telling or recitation) Active (reading, writing, doing) not passive (listening to others in turn taking situations) student response mode A clear focus on the higher, not the lower order tasks--for ALL kids. Looking at School Level Factors: Looking at School Level Factors We combed our interviews and our notes from PD meetings to look for evidence of the degree to which the reform effort at the school exhibited the criteria we set out to nurture: Our Reform Effort Rubric: Our Reform Effort Rubric Meeting for 1 hour three times per monthin study groups (at least 80% of the time). Meeting in cross-grade study groups (at least 80% of the time). Reflecting on instruction and student work (demonstrated at least 80% of the time). Considering research-based practices (demonstrated at least 80% of the time). Being guided by action plans (yes or no). Rest of Rubric: Rest of Rubric Sticking with substantive topics for 3-4 months or more (yes or no). Meeting once a month as a whole faculty to share and set goals (at least 80% of the time). Working on a plan to involve parents as partners (yes or no). Effective use of an external facilitator (yes or no). Effective use of an internal leadership team (yes or no). What we found at the school level: What we found at the school level Gates Comprehension: Gates Comprehension 10% of the variance between schools Reform effort explained 17% of the BtwSch variance Effect size of .29 (qualifies as moderate) 1 sd in RE 1.34 in NCE scores Aside: In this analysis (CompSkill and Grade were negatively related to student growth) In general, the more comprehension was taught as a skill to be mastered, the less growth students made. In general, the higher the grade, the lower the gain (and the overall scores) across time points We were able to make greater progress over a year in the lower grades… Fluency: Fluency 22% of the variance BtwSch 35% of the BtwSch variance due to RE Effect Size of .38 1 sd in RE 4.87 in wcpm Taking a longer view:Looking at the Schools, Teachers, and Kids in for at least 2 years: Taking a longer view: Looking at the Schools, Teachers, and Kids in for at least 2 years 8 schools Gates Comprehension: Gates Comprehension 15% of the growth was Betw Schools RE explained 65% of the BtwSch variance 1 pt inRE 2.5 in NCE pts. ES = .49: something to write home about! Grade negatively explained entering status and did not interact with time (we had a harder time showing growth in the higher grades) Fluency: Fluency 15% of the growth was Betw Schools RE explained 44% of the BtwSch variance 1 pt inRE 12.5 in NCE ES = .41 Comparing High and Low Reform Schools: Comparing High and Low Reform Schools Differences in reform efforts across HRE and LRE schools: Differences in reform efforts across HRE and LRE schools sticking with a substantive study group topic for at least 3 - 4 months meeting once a month as a school to share study group activities effective internal leadership team What do we mean by substantive?: What do we mean by substantive? A review of research-based reading practices in comparison to current practices share study group successes with the whole group deal with school-wide issues, such as assessment, grouping practices, curricular tools Leadership team success: Leadership team success Led by a knowledgeable teacher Convene and facilitate study groups and whole group meetings discuss the progress of study groups and work to solve problems supportive principal who was enthusiastic about the reform Some had external facilitators; others internal Slide64: Mean Spring NCE Scores by Reform Effort High reform Ss staying above Mean NCE of 40. Low reform Ss falling off more in grades 3-5. Slide65: Mean Spring NCE Scores for Schools in Project for 2 Years Note: limited to schools in TWO years: note the take-off in Year 2 for the HR schools Considerations for Successful Wide-Scale Use: Considerations for Successful Wide-Scale Use Flexible but specific structure for reform is needed Teacher buy-in is essential Leadership team must keep the effort going during the year and for the long haul System for getting data from observations to the teachers and for getting school reports to schools is needed Opportunities for teachers to see exemplary teaching are essential Constant reminders to reflect on and improve instruction are needed Questions for Further Research: Questions for Further Research How can schools be encouraged or motivated to stay in a reform effort for the long haul? How can schools be helped when there is initial teacher buy-in but no real leadership? Common Ground: We are not alone: Common Ground: We are not alone Characteristics of Effective Schools:A National View: Characteristics of Effective Schools: A National View Characteristics of Accomplished Teachers: A National View: Characteristics of Accomplished Teachers: A National View Barbara Taylor’s work with REA and RF schools in Minnesota: Barbara Taylor’s work with REA and RF schools in Minnesota Gone to Scale with 80+ schools Continue to replicate and extend the CIERA findings Cognitive Engagement Reform Effort School climate/effectiveness ratings Collaborative Leadership Collaboration in teaching Reflection Reflecting on Teaching Practices: Reflecting on Teaching Practices Considering HOW we teach is as important as considering WHAT we teach. Why don’t we get better faster: Getting sidetracked at the school level:: Why don’t we get better faster: Getting sidetracked at the school level: Search for a single, magic bullet to solve all our ills OR In response to a multitude of pressures from state or district mandates, flit from topic to topic in search of something that works Flitting from flower to flower: Flitting from flower to flower Increase time for reading instruction; OR put a new delivery model in place; OR purchase a new reading curriculum; OR focus on getting students ready for the state-mandated reading test. Getting sidetracked at the classroom level, : Getting sidetracked at the classroom level, A teacher may implement a new but idiosyncratic teaching routine; OR switch students with a colleague to meet students at their instructional level in more homogeneous groupings; OR send the students with the greatest instructional needs to a resource teacher who can apply the latest intervention; OR put extra effort into getting parents more involved. A better model: A better model There is nothing inherently misguided about any of these individual efforts. But they serve as a substitute for what really matters sustained collaborative work with colleagues in school-wide efforts and refocusing classroom instruction to maximize students’ cognitive engagement in their literacy learning. At school level, teachers and principals need to: At school level, teachers and principals need to Work toward shared leadership; AND collaborate in the delivery of reading instruction with a model that puts the students first; AND use data to inform instruction and improve teaching practices; AND engage in collaborative, job-embedded professional development. AND be open to new ideas, ideas that may trump well-established practices Within classrooms teachers must: Within classrooms teachers must Reflect on their instruction; AND implement research based practices that focus on providing support to promote active student involvement in higher level cognitive tasks; AND develop and maintain high expectations for student learning. What we need to do: What we need to do To achieve all of these goals simultaneously requires hard work, deep analyses of teaching and learning, and a commitment to collective problem-solving; teachers who accept the challenge of this agenda will need to change the way they work with their colleagues and their students. And be accountable for gaining and using the very best and most current knowledge in crafting and delivering their classroom programs. Above all, staying the course… So What: So What Home grown reforms promote as much success as pre-packaged reforms, maybe more… But they won’t work without buy-in, leadership, and commitment to collaboration Note the irony: we have demonstrated the efficacy of these sorts of distributed models of reform and pd at a time when our choices are being severely limited by federal and state policy. Go figure!