advertisement

2007 Annual Session Michael Quinn Patton

33 %
67 %
advertisement
Information about 2007 Annual Session Michael Quinn Patton
Education

Published on January 11, 2008

Author: Carmela

Source: authorstream.com

advertisement

Deepening:  Deepening Extension’s Knowledge Base Seeing Through an Evaluation Lens:  Seeing Through an Evaluation Lens Slide3:  In the beginning… Extension :  Extension as a knowledge-based learning organization Slide6:  LEARNING FRAMEWORK Action Beliefs Knowledge Premises:  Premises Action flows from some combination of beliefs/values and knowledge. Informed action has a strong knowledge base. By testing and evaluating beliefs, we build knowledge, thereby making our actions more informed and empirically based. Slide8:  In short, ACTIONS flow from some combination of… BELIEFS and KNOWLEDGE . Learning Organization:  Learning Organization Moving from beliefs to knowledge as the foundation for and basis of action. Slide10:  What constitutes high quality, useful knowledge? Aspirations:  Aspirations Scientifically-based knowledge Research-validated findings Evidence-based practice Best practices/Promising practices Generalizable Lessons Learned Slide12:  The Debate About Randomized Controls in Evaluation: The Gold Standard Question GOLD STANDARD: METHODOLOGICAL APPROPRIATENESS not Methodological orthodoxy or rigidity:  GOLD STANDARD: METHODOLOGICAL APPROPRIATENESS not Methodological orthodoxy or rigidity Slide14:  Observational Data Suffices A study in the British Medical Journal by Gordon Smith and Jill Pell (2003) found that…. http://bmj.bmjjournals.com/cgi/content/full/327/7429/1459?ck=nck Slide15:  There have been NO randomized control trials of parachute use. Slide17:  Smith and Pell concluded: “Only two options exist. The first is that we accept that, under exceptional circumstances, common sense might be applied when considering the potential risks and benefits of an intervention.” OR Slide18:  those who criticize interventions that lack a RCT evidence base will not hesitate to demonstrate their commitment by volunteering for a double blind, randomized, placebo controlled, crossover trial.” What is possible and appropriate::  What is possible and appropriate: Multiple sources of data about each proposition Triangulation of sources High Quality Lessons Slide20:  The Metaphor of Triangulation Multiple Data Sources:  Multiple Data Sources to build Knowledge Highly Rated Knowledge:  Highly Rated Knowledge “Star” Sources:  “Star” Sources basic and applied research Limitations to published research as the sole or primary source of knowledge:  Limitations to published research as the sole or primary source of knowledge Errors Inadequate peer review processes Corrections over time Time for replication Generalizability issues Corruption: publishing pressure, funding pressures “Star” Sources:  “Star” Sources basic and applied research evaluation findings --patterns across programs “Star” Sources:  “Star” Sources basic and applied research evaluation findings--patterns across programs practice wisdom and experience of practitioners: individuals & teams “Star” Sources:  “Star” Sources basic and applied research evaluation findings--patterns across programs practice wisdom and experience of practitioners experiences and knowledge of consumers, program participants, clients, intended beneficiaries… More “Star” Sources:  More “Star” Sources expert opinion “Star” Sources:  “Star” Sources expert opinion cross-disciplinary connections and patterns “Star” Sources:  “Star” Sources expert opinion cross-disciplinary connections and patterns THEORY – capturing and organizing knowledge The Theory-Practice Connection:  The Theory-Practice Connection There’s nothing so practical as a good theory. Lazarsfeld Slide32:  Theory of Change Program Theory Logic Models Theory of Action Intervention Strategy Theory-driven evaluation Focus of High Quality Lessons Learned & Knowledge::  Focus of High Quality Lessons Learned & Knowledge: Enhancing Our Theory of Change High Quality Lessons :  High Quality Lessons Those lessons that are supported by and triangulated from multiple sources that reinforce and affirm the knowledge encompassed in the lesson, yielding: Slide35:  High Quality, Triangulated Knowledge to inform a Theory of Change Assessing Lessons:  Assessing Lessons Assess the importance of the lesson learned for a particular action Examine and evaluate the strength of the connection to outcomes attainment. Compare case examples to extract principles. Reflective Practice Example:  Example Extension’s Mission and Theory of Change Slide38:  LEARNING FRAMEWORK Action Beliefs Knowledge Slide39:  Extension’s Mission BELIEF DEBATE: Knowledge Transfer (Education) or Slide40:  Knowledge Transfer (Education) or Behavior & Practice Change Bennett’s Hierarchy:  Bennett’s Hierarchy 7. End results --impact on the overall problem   6. Practice and behavior change   5. Knowledge, attitude, and skill changes 4. Reactions & satisfaction  3. Participation  2. Activities 1. Inputs & resources Basic Education Model Belief:  Basic Education Model Belief Changed Behavior Knowledge change Basic Assumptions:  Basic Assumptions Rational, information-based decision-making and Linear, direct cause-effect outcomes model Reality-testing:  Reality-testing Poor information processors, beliefs trump evidence Selective perception Humans as emotional animals Neuroeconomics Evidence:  Evidence Evaluation findings – knowledge alone generally doesn’t work to produce lasting behavior and practice changes for difficult and major issues Evidence:  Evidence Research findings – lasting behavior change is multi-faceted, requiring significant incentives, on-going reinforcement of new behaviors, and integration with other beliefs, values, and knowledge Evidence:  Evidence Theory – Systems theory emphasizes understanding behavior change in a systems context, that is, in relation to other factors and dynamics, not as an isolated phenomenon: Farming systems Family systems work Complex adaptive systems approaches Other Evidence:  Other Evidence Feedback from clients and program participants Expert opinion Cross-disciplinary findings from -- behavioral economics -- systems psychology -- organizational sociology -- adult learning Slide49:  The nature of EXPERTISE The Knowledge Challenge::  The Knowledge Challenge: Matching the knowledge transfer and program intervention strategy to the nature of the situation, the desired changes, system understandings, resources, and timeline to optimize impact within a local knowledge framework. Some premises::  Some premises: Evaluation is part of initial program design, including conceptualizing the theory of change Evaluator’s role is to help users clarify their purpose, hoped-for results, and change model. Evaluators can/should offer conceptual and methodological options. Evaluators can help by questioning assumptions. Conceptual Options:  Conceptual Options Simple Complicated Complex Types of Community Issues The Stacey Matrix:  Types of Community Issues The Stacey Matrix Certainty Agreement Close to Far from Far from Close to Simple:  Simple Certainty Agreement Close to Far from Far from Close to Simple Plan, control Slide55:  Following a Recipe A Rocket to the Moon Raising a Child Complicated Complex The recipe is essential Recipes are tested to assure replicability of later efforts No particular expertise; knowing how to cook increases success Recipes produce standard products Certainty of same results every time Simple Technically Complicated:  Technically Complicated Certainty Agreement Close to Far from Far from Close to Simple Plan, control Technically Complicated Experiment, coordinate expertise Slide57:  Following a Recipe A Rocket to the Moon Raising a Child Formulae are critical and necessary Sending one rocket increases assurance that next will be ok High level of expertise in many specialized fields + coordination Rockets similar in critical ways High degree of certainty of outcome Complicated Complex The recipe is essential Recipes are tested to assure replicability of later efforts No particular expertise; knowing how to cook increases success Recipes produce standard products Certainty of same results every time Simple Socially Complicated:  Socially Complicated Certainty Agreement Close to Far from Far from Close to Simple Plan, control Technically Complicated Experiment, coordinate expertise Socially Complicated Build relationships, create common ground Socially complicated:  Socially complicated Implementing human rights agreements, like gender equity or outlawing child labor Environmental Initiatives Many different and competing stakeholders Diverse vested interests High stakes Socially complicated:  Socially complicated situations pose the challenge of coordinating and integrating many parts Know When Your Challenges Are In the Zone of Complexity:  Know When Your Challenges Are In the Zone of Complexity Certainty Agreement Close to Far from Far from Close to Simple Plan, control Zone of Complexity Technically Complicated Experiment, coordinate expertise Socially Complicated Build relationships, create common ground Systems Thinking Relationship Building Collaboration Good Enough Vision Chunking Around Drivers Minimum Specifications Multiple Actions Adaptability & Organic Slide62:  Following a Recipe A Rocket to the Moon Raising a Child Sending one rocket increases assurance that next will be ok High level of expertise in many specialized fields + coordination Rockets similar in critical ways High degree of certainty of outcome Formulae have only a limited application Raising one child gives no assurance of success with the next Expertise can help but is not sufficient; relationships are key Every child is unique Uncertainty of outcome remains Complicated Complex The recipe is essential Recipes are tested to assure replicability of later efforts No particular expertise; knowing how to cook increases success Recipes produce standard products Certainty of same results every time Simple Know When Your Challenges Are In the Zone of Complexity:  Know When Your Challenges Are In the Zone of Complexity Certainty Agreement Close to Far from Far from Close to Simple Plan, control Zone of Complexity Technically Complicated Experiment, coordinate expertise Socially Complicated Build relationships, create common ground Chaos Massive Avoidance Slide64:  Following a Recipe A Rocket to the Moon Raising a Child Formulae are critical and necessary Sending one rocket increases assurance that next will be ok High level of expertise in many specialized fields + coordination Separate into parts and then coordinate Rockets similar in critical ways High degree of certainty of outcome Formulae have only a limited application Raising one child gives no assurance of success with the next Expertise can help but is not sufficient; relationships are key Can’t separate parts from the whole Every child is unique Uncertainty of outcome remains Complicated Complex The recipe is essential Recipes are tested to assure replicability of later efforts No particular expertise; knowing how to cook increases success Recipe notes the quantity and nature of “parts” needed Recipes produce standard products Certainty of same results every time Simple Slide65:  Challenges: Situation Recognition and Knowledge Appropriate to the Situation Slide66:  Systems Parts are interdependent such a change in one part changes all parts The whole is greater than the sum of the parts Focus on interconnected relationships Systems are made up of sub-systems and function within larger systems Slide68:  Logic Model for Pregnant Teens Program 1. Program reaches out to pregnant teens 2. Pregnant teens enter and attend the program (participation) 3. Teens learn prenatal nutrition and self-care (increased knowledge) 4. Teens develop commitment to take care of themselves and their babies (attitude change) 5. Teens adopt healthy behaviors: no smoking, no drinking, attend prenatal clinic, eat properly (behavior change) 6. Teens have healthy babies (desired outcome) Slide69:  Systems web showing possible influence linkages to a pregnant teenager Slide70:  Program systems web showing possible institutional influences affecting pregnant teenagers' attitudes and behavior Slide71:  Sustainable change: Systems dynamic reinforcing feedback loops Slide72:  Using Different System Lenses to understand a “particular” System Biologic System Energy Well-being Cycle status Vitality Sociologic System Relationships Conversations Interdependence Status issues Meaning/sense Environment Volatility Predictability Spatial Proximities Diversity Economic System Resources Fears & rewards Aspiration Political System Power Connections Possibility Locus of control Anthropologic System Values Culture/Milieu Information System Education Speed & access Fidelity/utility Knowledge Storage Psychological System Incentives Heuristics Reinforcements Behavior Change Slide73:  Extension as a knowledge-based organization Sharing Knowledge:  Sharing Knowledge to change behavior and practice, and solve problems, based on a Theory of Change Slide75:  HYMN to KNOWLEDGE USE Sung to the tune of Auld Lang Syne Slide76:  May all Ex-ten-sion programs use Knowledge that consumers trust, That tell them how to separate What works from what’s a bust. We gather data near and far To see what we can learn The findings help us to decide What to share and what to burn. Slide77:  There comes a time for each of us When doubts may give us pause We wonder what results will last Will our knowledge turn out flawed? But be assured there's naught to fear Shared learning is what we seek Tri-angulate your sources now With authority you can speak. Slide78:  Our knowledge base we can affirm, Extension shares the truth. Our lessons learned we will apply On farms, in homes, with youth. Extension staff go forth as one, Researchers here stand tall, The world will be a better place Sharing knowledge is our call. References:  References Utilization-Focused Evaluation, 3rd. Edition, by Michael Q. Patton, Sage Publications, 1997. (New 4th edition forthcoming at the end of 2007.) Getting to Maybe: How the World Is Changed by Frances Westley, Brenda Zimmerman, and Michael Q. Patton, Random House Canada, 2006. http://www.randomhouse.ca/catalog/display.pperl?isbn=9780679314431 Slide80:  Systems Concepts in Evaluation – An Expert Anthology. 2006. Bob Williams and Iraj Imam AEA Monograph, EdgePress/AEA Point Reyes CA.

Add a comment

Related presentations

Related pages

Oral History Project__Michael Quinn Patton | Michael Q ...

Oral History Project__Michael Quinn Patton. ... 2007; 28; 102 American ... use led to my doing workshops for the Evaluation Network annual conferences ...
Read more

Qualitative research and evaluation methods (Book, 2002 ...

Qualitative research and evaluation methods. [Michael Quinn Patton] -- This book has been completely revised with hundreds of new examples and stories ...
Read more

A Guide to Using Qualitative Research Methodology

A Guide to Using Qualitative Research Methodology © 2002 Michael Quinn Patton and Michael Cochran Reproduced with kind permission of Michael Quinn Patton
Read more

Patton and Schwandt Present at Inaugural Eleanor Chelimsky ...

Patton and Schwandt Present at Inaugural Eleanor Chelimsky Forum. ... especially through its annual ... Michael Quinn Patton is an independent ...
Read more

Michael Quinn Patton - YouTube

Michael Quinn Patton's presentation as part of the inaugural Eleanor Chelimsky Forum, at the Eastern Evaluation Research Society's (EERS) 2013 ...
Read more

The Evaluators' Institute at Claremont Graduate University

The Evaluators’ Institute is excited to announce two new programs in evaluation ... Individuals participating in these sessions represented BiH ...
Read more

The Annual - Wikipedia, the free encyclopedia

The Annual 2008 (2007) The Annual 2009 (2008) ... Michael Gray Feat. ... Madison Avenue & Sean Quinn: 2001: The Summer Annual ...
Read more

AEAweb: 2007 AEA Conference Papers Listings

... January 5, 2007, 8:00 AM Session: Labor Flows, Vacancies and Wages ... Quinn, Michael A. ... Annual Meetings | EconLit | JOE | RFE ...
Read more

Professional Development Workshops - Claremont Graduate ...

Professional Development Workshop Series in Evaluation and Applied Research ... Each morning begins with a brief group session ... Michael Quinn Patton ...
Read more