Published on February 19, 2014
Strong performers and successful reformers Lessons from PISA for Turkey OECD EMPLOYER BRAND Playbook Andreas Schleicher Istanbul, 15 February 2014 1
2 PISA in brief • Over half a million students… – representing 28 million 15-year-olds in 65 countries/economies … took an internationally agreed 2-hour test… – Goes beyond testing whether students can reproduce what they were taught… … to assess students’ capacity to extrapolate from what they know and creatively apply their knowledge in novel situations – Mathematics, reading, science, problem-solving, financial literacy – Total of 390 minutes of assessment material … and responded to questions on… – their personal background, their schools and their engagement with learning and school • Parents, principals and system leaders provided data on… – school policies, practices, resources and institutional factors that help explain performance differences .
Changes in the demand for skills 3 Trends in different tasks in occupations (United States) Mean task input in percentiles of 1960 task distribution 70 65 Routine manual 60 Nonroutine manual Routine cognitive 55 Nonroutine analytic Nonroutine interpersonal 50 45 40 35 1960 1970 1980 1990 2000 2006 2009 Source: Autor, David H. and Brendan M. Price. 2013. "The Changing Task Composition of the US Labor Market: An Update of Autor, Levy, and Murnane ( 2003)." MIT Mimeograph, June.
4 What do 15-year-olds know… …and what can they do with what they know?
High mathematics performance Mean score … Shanghai-China performs above this line (613) 580 Singapore 570 560 Chinese Taipei Korea 550 540 Macao-China Japan Liechtenstein Switzerland 530 520 510 500 490 480 470 Hong Kong-China Poland Belgium Germany Austria Slovenia New Zealand Denmark France Czech Republic Latvia Luxembourg Portugal Spain Slovak Republic United States Hungary Netherlands Estonia Finland Canada Viet Nam Australia Ireland United Kingdom Iceland Norway Italy Russian Fed. Lithuania Sweden Croatia Israel 460 450 Greece Serbia Turkey Romania 440 430 420 410 Chile … 12 countries perform below this line Bulgaria U.A.E. Kazakhstan Thailand Malaysia Mexico Low mathematics performance Average performance of 15-year-olds in Mathematics Fig I.2.13
High mathematics performance Singapore Chinese Taipei Hong Kong-China Average performance of 15-year-olds in mathematics Korea Macao-China Japan Liechtenstein Switzerland Strong socio-economic impact on student performance Poland Belgium Germany Austria Slovenia New Zealand Denmark France Czech Republic Latvia Luxembourg Portugal Spain Slovak Republic United States Hungary Netherlands Estonia Finland Canada Viet Nam Australia Ireland United Kingdom Iceland Norway Italy Russian Fed. Lithuania Sweden Croatia Israel Greece Serbia Turkey Romania Chile Bulgaria U.A.E. Kazakhstan Thailand Malaysia Mexico Low mathematics performance Socially equitable distribution of learning opportunities
2012 Shanghai-China Singapore Hong Kong-China Chinese Taipei Korea Macao-China Japan Switzerland Liechtenstein Estonia Netherlands Poland Canada Belgium Finland Viet Nam Germany Strong socio-economic Austria Australia impact on student New Zealand Denmark Slovenia Ireland Iceland Czech Rep. performance 22France 26 24 20 18 16 14 12 10 8 6 UK Latvia Luxembourg Norway Portugal Italy Russian Fed. US Spain Lithuania Sweden Slovak Rep. Hungary Croatia Israel Romania Bulgaria Greece Turkey Serbia United Arab Emirates Kazakhstan Thailand Chile Malaysia Mexico Socially equitable distribution of learning opportunities 4 2 0
Australia Austria Belgium Canada Chile Czech Rep. Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel socio-economic Strong Italy impact on student Japan performance Korea Luxembourg Mexico Slovak Rep. Netherlands New Zealand Norway Poland Portugal Slovak Rep. Slovenia Spain Sweden Switzerland Turkey UK US 2012 Korea Japan Switzerland Netherlands Poland Belgium Germany Estonia Canada Finland Socially equitable Austria Australia New Zealand Denmark Ireland Slovenia distribution of learning Iceland Czech Rep. opportunities France UK Luxembourg Norway Portugal Italy US Spain Sweden Hungary Israel Greece Turkey Chile Mexico
Australia Austria Belgium Canada Chile Czech Rep. Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Luxembourg Mexico Slovak Rep. Netherlands New Zealand Norway Poland Portugal Slovak Rep. Slovenia Spain Sweden Switzerland Turkey UK US Korea Japan Switzerland Netherlands Poland Belgium Germany Estonia Canada Finland Austria Australia New Zealand Denmark Ireland Slovenia Iceland Czech Rep. France UK Luxembourg Norway Portugal Italy US Spain Sweden Hungary Israel Greece Turkey Chile Mexico
Australia Austria Belgium Canada Chile Czech Rep. Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Luxembourg Mexico Slovak Rep. Netherlands New Zealand Norway Poland Portugal Slovak Rep. Slovenia Spain Sweden Switzerland Turkey UK US Shanghai 2003 - 2012 Singapore Singapore Korea Japan Switzerland Netherlands Poland Belgium Germany Estonia Canada Finland Austria Australia New Zealand Denmark Ireland Slovenia Iceland Czech Rep. France UK Luxembourg Norway Portugal Italy US Spain Sweden Hungary Israel Greece Turkey Chile Mexico
11 Performance of countries in a level playing field How the world would look if students around the world were living in similar social and economic conditions
340 Shanghai-China Singapore Hong Kong-China Chinese Taipei Viet Nam Macao-China Korea Japan Liechtenstein Poland Switzerland Estonia Netherlands Germany Belgium Finland Canada Portugal Austria Czech Republic New Zealand Latvia France Slovenia Ireland Australia OECD average Turkey Slovak Republic Spain Hungary Luxembourg Italy Russian Federation United Kingdom Denmark Lithuania Croatia United States Norway Sweden Iceland Romania Israel Serbia Thailand Greece Bulgaria Chile Uruguay Malaysia Kazakhstan Cyprus5, 6 Mexico Costa Rica United Arab… Brazil Montenegro Tunisia Indonesia Peru Argentina Colombia Jordan Qatar Mean mathematics score 12 Mathematics performance in a level playing field Mean mathematics performance after accounting for socio-economic status Fig II.3.3 Mean score at the country level before adjusting for socio-economic status Mean score at the country level after adjusting for socio economic status 600 580 560 540 520 500 480 460 440 420 400 380 360
13 The dream of social mobility In some countries it is close to a reality
Percentage of resilient students 14 Fig II.2.4 20 18 A resilient student is situated in the bottom quarter of the PISA index of economic, social and cultural status (ESCS) in the country of assessment and performs in the top quarter of students among all countries, after accounting for socio-economic status. 16 14 12 Socio-economically disadvantaged students not only score lower in mathematics, they also report lower levels of engagement, drive, motivation and self-beliefs. Resilient students break this link and share many characteristics of advantaged highachievers. % 10 8 6 4 2 More than 10 % resilient Between 5%-10% of resilient students Less than 5% Shanghai-China Hong Kong-China Macao-China Viet Nam Singapore Korea Chinese Taipei Japan Liechtenstein Switzerland Estonia Netherlands Poland Canada Finland Belgium Portugal Germany Turkey OECD average Italy Spain Latvia Ireland Australia Thailand Austria Luxembourg Czech Republic Slovenia United Kingdom Lithuania France Norway Iceland New Zealand Russian Fed. United States Croatia Denmark Sweden Hungary Slovak Republic Mexico Serbia Greece Israel Tunisia Romania Malaysia Indonesia Bulgaria Kazakhstan Uruguay Brazil Costa Rica Chile Colombia Montenegro U.A.E. Argentina Jordan Peru Qatar 0
15 It is not just about poor kids in poor neighbourhoods… …but about many kids in many neighbourhoods
B 60 20 80 Albania Finland Iceland Sweden Norway Denmark Estonia Ireland Spain Canada Poland Latvia Kazakhstan United States Mexico Colombia Costa Rica Russian Fed. Malaysia Jordan New Zealand Lithuania Greece Montenegro United Kingdom Argentina Australia Brazil Portugal Indonesia Chile Thailand Romania Tunisia Switzerland Peru Uruguay Croatia U.A.E. Macao-China Serbia Viet Nam Korea ong Kong-China Singapore Austria Italy Luxembourg Czech Republic Japan Bulgaria Israel Qatar Shanghai-China Germany Slovenia Slovak Republic Turkey Belgium Hungary Liechtenstein Netherlands Chinese Taipei Variation in student performance as % of OECD average variation 16 Variability in student mathematics performance between and within schools Fig II.2.7 100 80 Performance differences between schools 40 OECD average 20 0 Performance variation of students within schools 40 60 OECD average 100
High impact on outcomes 22 22 Quick wins Lessons from high performers Must haves Catching up with the top-performers Low feasibility High feasibility Money pits Low hanging fruits Low impact on outcomes
High impact on outcomes 23 23 Quick wins Must haves Lessons from high performers Commitment to universal achievement Capacity at point of delivery Resources where they yield most Gateways, instructional systems Coherence A learning system Low feasibility High feasibility Incentive structures and accountability Money pits Low hanging fruits Low impact on outcomes
High impact on outcomes 24 24 Lessons from high performers Quick Must to education and the belief that wins A commitmenthaves Commitment to universal therefore competencies can be learned andachievementall children can achieve Capacity personalization as at Universal educational standards andResources point of delivery the approach to heterogeneitywhere they yield most in the student body… … as opposed to a belief that students have different Gateways, instructional destinations to be met with different expectations, and systems selection/stratification as the approach to Coherence heterogeneity A learning system Clear articulation who is responsible for ensuring Low feasibility High feasibility student success and to whom Incentive structures and accountability Money pits Low hanging fruits Low impact on outcomes
25 Countries where students have stronger beliefs in their abilities perform better in mathematics Fig III.4.5 OECD average 650 Mean mathematics performance 600 550 500 450 400 350 300 -0.60 Shanghai-China Singapore Hong Kong-China Korea R² = Chinese Taipei Macao-China Japan Switzerland Netherlands Estonia Canada Liechtenstein Finland Germany Poland Belgium Viet Nam Slovenia Denmark New Zealand Latvia Portugal Italy Austria Australia Russian Fed. Hungary Luxembourg Spain Croatia Slovak Republic Greece Norway Turkey Israel Sweden Serbia Czech Republic Lithuania U.A.E. Iceland Romania United Kingdom Malaysia Thailand United States Ireland Bulgaria Kazakhstan Chile Montenegro France Costa Rica Mexico Uruguay Albania Brazil Argentina Tunisia Colombia Qatar Jordan Indonesia Peru -0.40 -0.20 0.00 0.20 0.40 0.60 Mean index of mathematics self-efficacy 0.80 0.36 1.00 1.20
26 Perceived self-responsibility for failure in mathematics Fig III.3.6 Percentage of students who reported "agree" or "strongly agree" with the following statements: Turkey Shanghai-China OECD average Sometimes I am just unlucky The teacher did not get students interested in the material Sometimes the course material is too hard This week I made bad guesses on the quiz My teacher did not explain the concepts well this week I’m not very good at solving mathematics problems 0 20 40 60 % 80 100
High impact on outcomes 30 30 Quick wins Must haves Lessons from high performers Commitment to universal achievement Clear ambitious goals that are shared across the Capacity system and aligned with high stakes gateways and Resources at point of delivery where instructional systemsthey yield most Coherence Low feasibility Well established delivery chain through which Gateways, instructional curricular goals translate into instructional systems, systems instructional practices and student learning (intended, implemented andlearning system A achieved) High level of metacognitive content of instruction … High feasibility Incentive structures and accountability Money pits Low hanging fruits Low impact on outcomes
31 High impact on outcomes 31 Capacity at Lessons from high performers the point of delivery Quick wins Must haves Attracting, developing and retaining high quality Commitment a universal achievement teachers and school leaders andto work organisation in which they can use their potential Capacity Instructional leadership and human resource Resources at point of delivery management in schools where they yield most Keeping teaching an attractive profession Gateways, instructional System-wide career development … systems Coherence A learning system Low feasibility High feasibility Incentive structures and accountability Money pits Low hanging fruits Low impact on outcomes
Luxembourg Jordan Thailand Turkey Shanghai-China Israel Colombia Peru Chile Netherlands Mexico Germany Viet Nam Russian Fed. Uruguay Norway Kazakhstan Indonesia Belgium Italy Malaysia Australia Brazil Iceland U.A.E. Singapore New Zealand Korea Switzerland Estonia Macao-China Costa Rica OECD average Sweden Argentina Tunisia Austria Qatar Ireland Chinese Taipei France Denmark United Kingdom Hong Kong-China Albania Japan Canada Slovak Republic Latvia Greece United States Czech Republic Croatia Finland Montenegro Romania Hungary Lithuania Slovenia Spain Serbia Portugal Bulgaria Poland Mean index 2.5 1.5 Lessons from high performers 32 32 Teacher shortage Mean index Fig IV.3.5 Top quarter of this index Bottom quarter of this index 3 2 1 0.5 0 -0.5 -1 -1.5
High impact on outcomes 33 33 Lessons from high performers Quick wins Must haves Incentives, accountability, knowledge management Commitment to universal achievement Aligned incentive structures For students Capacity Resources How gateways at point of delivery affect the strength, direction, clarity and nature of the incentives operating on students at each stage of their education where they yield most Degree to which students have incentives to take tough courses and study hard Gateways, Opportunity costs for staying in school and performing well instructional For teachers Coherenceinnovations in pedagogy and/or organisation Make A learning system Low feasibility Improve their own performance and the performance of their colleagues Pursue professional development opportunities that lead to stronger pedagogical practices systems High feasibility Incentive structures and A balance between vertical and lateral accountability accountability Effective instruments to manage and share knowledge and spread innovation – communication within the system and with stakeholders around it Money pits Low hanging A capable centre with authority and legitimacy to act fruits Low impact on outcomes
Countries that grant schools autonomy over curricula and assessments tend to perform better in mathematics 650 Fig IV.1.15 Shanghai-China Mathematics performance (score points) 600 Chinese Taipei Viet Nam 550 500 450 400 Korea Estonia Singapore Hong Kong-China Japan Poland Latvia Slovenia Belgium Czech Rep. Switzerland Canada Germany Finland New Zealand Lithuania Netherlands Portugal Hungary Austria Croatia Italy Spain France Australia Serbia UK Macao-China Turkey Norway Iceland Denmark R² = 0.13 Slovak Rep. Bulgaria Thailand Greece Romania Kazakhstan Israel Malaysia Chile Uruguay USA Sweden Jordan Costa Rica Indonesia Brazil Albania Luxembourg Tunisia Colombia UAE Argentina Peru 350 Qatar 300 -1.5 -1 -0.5 0 0.5 Index of school responsibility for curriculum and assessment (index points) 1 1.5
Schools with more autonomy perform better than schools with less autonomy in systems with more collaboration School autonomy for resource allocation x System's level of teachers participating in school management Across all participating countries and economies Score points 485 480 475 470 465 460 Teachers participate in management 455 Teachers don't participate in management Less school autonomy More school autonomy Fig IV.1.17
Schools with more autonomy perform better than schools with less autonomy in systems with more accountability arrangements Fig IV.1.16 School autonomy for curriculum and assessment x system's level of posting achievement data publicly Score points 478 476 474 472 470 468 466 School data public 464 School data not public Less school autonomy More school autonomy
Schools with more autonomy perform better than schools with less autonomy in systems with standardised math policies Fig IV.1.16 School autonomy for curriculum and assessment x system's extent of implementing a standardised math policy (e.g. curriculum and instructional materials) Score points 485 480 475 470 465 460 Standardised math policy 455 No standardised math policy Less school autonomy More school autonomy
38 Quality assurance and school improvement Fig IV.4.14 Percentage of students in schools whose principal reported that their schools have the following for quality assurance and improvement: Singapore OECD average Implementation of a standardised policy for mathematics Regular consultation with one or more experts over a period of at least six months with the aim of improving… Teacher mentoring Written feedback from students (e.g. regarding lessons, teachers or resources) External evaluation Internal evaluation/self-evaluation Systematic recording of data, including teacher and student attendance and graduation rates, test results… Written specification of student-performance standards Written specification of the school's curriculum and educational goals 0 20 40 % 60 80 100
39 The issue is not how many charter schools a country has… …but how countries enable every school to assume charter type autonomy
High impact on outcomes 40 40 Quick wins Lessons from high performers Must haves Commitment to universal achievement Investing resources where they can make most of Capacity a difference Resources Alignment of resources with key challenges (e.g. at point of delivery where they teachers attracting the most talentedyield mostto the most challenging classrooms) Gateways, instructional Effective spending choices that prioritise high quality systems teachers over smaller classes Coherence A learning system Low feasibility High feasibility Incentive structures and accountability Money pits Low hanging fruits Low impact on outcomes
41 Money makes a difference… …but only up to a point
Spending per student from the age of 6 to 15 and mathematics performance in PISA 2012 Fig IV.1.8 650 Cumulative expenditure per student less than USD 50 000 Mathematics performance (score points) Shanghai-China Cumulative expenditure per student USD 50 000 or more 600 Singapore Korea 550 Japan Switzerland PolandCanada Finland Netherlands Viet Nam Estonia Belgium Germany Czech Republic Australia Austria New Zealand Slovenia Ireland Denmark Latvia France UK Norway Portugal Iceland Lithuania Slovak Republic Croatia Italy Sweden United States Israel Hungary Spain Turkey 500 R² = 0.01 Luxembourg 450 Bulgaria Thailand Chile Mexico Montenegro Uruguay Malaysia 400 Tunisia Brazil Jordan Colombia Peru 350 R² = 0.37 300 0 20 000 40 000 60 000 80 000 100 000 120 000 140 000 160 000 Average spending per student from the age of 6 to 15 (USD, PPPs) 180 000 200 000
Countries with better performance in mathematics tend to allocate educational resources more equitably 700 Adjusted by per capita GDP 650 Mathematics performance (score points) Fig IV.1.11 Shanghai-China 600 550 500 450 Mexico Costa Rica 400 Chinese Taipei Korea R² = 0.19 Viet Nam Singapore Hong Kong-China Estonia Japan Poland Slovenia Switzerland Latvia Finland Canada Belgium Germany Macao-China Slovak Rep. New Zealand UK IrelandIceland France DenmarkSpain Austria Australia Croatia Hungary Israel Romania Portugal Sweden Bulgaria Turkey USA Greece Norway Italy Serbia Thailand Malaysia Chile Kazakhstan Uruguay Jordan Brazil Indonesia UAE Montenegro Colombia Tunisia Argentina Luxembourg Peru 350 Qatar 300 1.5 SHA 1 Less equity 0.5 Equity in resource allocation (index points) 0 -0.5 Greater equity
High impact on outcomes 44 44 Quick wins Must haves Lessons from high performers Commitment to universal achievement Capacity at point of delivery Coherence of policies and practices Alignment of policies across all aspects of the system Coherence Coherence of policies over sustained periods of time LowConsistency of implementation feasibility Fidelity of implementation (without excessive control) Money pits CAN Resources where they yield most Gateways, instructional systems A learning system High feasibility Incentive structures and accountability Low hanging fruits Low impact on outcomes
High impact on outcomes 45 45 Quick wins Must haves Lessons from high performers Commitment to universal achievement Capacity at point of delivery Resources where they yield most Gateways, instructional systems Coherence A learning system Low feasibility High feasibility Incentive structures and accountability Money pits Low hanging fruits Low impact on outcomes
What it all means 46 46 Lessons from high performers The old bureaucratic system The modern enabling system Student inclusion Some students learn at high levels All students need to learn at high levels Curriculum, instruction and assessment Routine cognitive skills, rote learning Learning to learn, complex ways of thinking, ways of working Teacher quality Few years more than secondary High-level professional knowledge workers Work organisation ‘Tayloristic’, hierarchical Flat, collegial Accountability Primarily to authorities Primarily to peers and stakeholders
Find out more about PISA at www.pisa.oecd.org • All national and international publications • The complete micro-level database Thank you ! Email: Andreas.Schleicher@OECD.org Twitter: SchleicherEDU and remember: Without data, you are just another person with an opinion
Do you have an idea on how to use this data to improve education in your country? Would you like to work with us to develop that idea? Apply to the Thomas J. Alexander fellowship programme! http://www.oecd.org/edu/thomasjalexanderfellowship.htm
... Strong Performers and Successful Reformers in Education: Lessons from PISA for ... Strong Performers and Successful Reformers ... Strong Nationwide ...
Strong Performers and Successful Reformers in Education Lessons from PISA 2012 for the United States Programme for International Student Assessment
Turkey; Turkmenistan; Turks ... Home / Books / Strong Performers and Successful Reformers in Education / Lessons from PISA for ... Strong Performers and ...
chapter 1 strong performers, successful reformers: ... reformers in education: lessons from pisa ... successful reformers in education: lessons ...
The report Strong Performers and Successful Reformers in Education: Lessons from PISA for Korea aims at helping Korea to identify ... Policy Lessons for ...
Strong Performers and Successful Reformers in Education ... PISA 2009 in brief ... Strong Performers in Education Quality
This volume draws lessons from ... average performance is high and the top quarter of performers place ... Strong Performers and Successful Reformers ...
Successful Reformers in Education LESSONS FROM ... The PISA approach to assessing Student Performance in mathematics 96 STRONG PERFORMERSAND ...
1.Strong performers and successful reformers Lessons from PISA for TurkeyOECD EMPLOYER BRAND PlaybookAndreas Schleicher Istanbul, 15 February 20141