Complexity-Aware Monitoring: Briefing on USAID’s New Discussion Note

33 %
67 %
Information about Complexity-Aware Monitoring: Briefing on USAID’s New Discussion Note
News & Politics

Published on March 4, 2014

Author: washeval



Washington Evaluators Brown Bag
by Heather Britt and Melissa Patsalides
February 25, 2014

How can we monitor effectively in a dynamically changing and unpredictable situation? Many monitoring approaches measure the predicted – desired results, planned implementation strategies and forecasted pathways of change – using indicators expected to provide useful information over the life of the project. Complexity-aware monitoring is intended to compliment performance monitoring by tracking the unpredictable. Three principles and five recommended approaches monitor the unforeseen and unforeseeable so that projects can remain relevant and responsive. Recommended approaches include sentinel indicators, stakeholder feedback, process monitoring of impacts, most significant change, and outcome harvesting. Complexity is commonly misconstrued as synonymous with conflict. However, most development contexts contain a mix of simple, complicated and complex aspects. Complex aspects are characterized by interrelationships, non-linear causality, and emergence. Complexity-aware monitoring approaches are useful in a wide variety of programming contexts. USAID’s Complexity-Aware Monitoring Discussion Note is intended to raise questions, stimulate dialogue, and -- most of all -- inspire experimentation. Heather Britt and Melissa Patsalides will discuss complexity-aware monitoring and USAID’s efforts to support experimentation with these approaches in the Agency. Stay tuned for more news on Learning Lab!

The Note can be found on USAID's Learning Lab website:

Heather Britt is a Monitoring & Evaluation Specialist with USAID/PPL’s Program Cycle Service Center. Heather is a skilled M&E capacity builder with experience working with a diversity of development actors including INGOS, UN agencies, foundations, and local organizations. She has been based in the Middle East for the past twelve years and speaks Arabic. Her evaluation work has included programming related to gender and to fragile states. She has a special interest in systems thinking applications, complexity and emerging methods in evaluation.

Melissa Patsalides is the Monitoring and Evaluation Team Leader in USAID’s Office of Learning, Evaluation and Research (LER). This team is responsible for improving the practices of performance monitoring and evaluation within USAID and for encouraging the use of evidence gained from these practices, by providing guidance and tools, building capacity, and supporting an organizational culture that values M&E. Melissa worked with the Peace Corps and a number of NGOs before joining USAID more than 10 years ago. She draws on considerable experience in difficult programming environments as she promotes cutting-edge M&E solutions in the Agency.

Complexity-Aware Monitoring Briefing on USAID’s Discussion Note Heather Britt & Melissa Patsalides Washington Evaluators, 25 February 2014

What is Complexity? Certainty / Agreement Matrix Cynefin Framework

Complicated versus Complex Complicated • Many interacting elements; central organizing element • Repeatable patterns of cause and effect • Multiple causal pathways Complex • Interrelations: web of diverse elements; no central organizing element. • Unpredictable cause & effect: Patterns visible retrospectively. Can be non-linear. • Emergence: Dynamic and unpredictable. Unknown elements. Patterns discernible retrospectively.

Complexity Myth Busters Myth #1 • Complexity is synonymous with conflict. (Closer to the) Truth • Most development contexts include complex aspects. • Terms like “complex environments” can be confusing.

Complexity Myth Busters Myth #2 • Everything is complex. (Closer to the) Truth • Projects and their environments most often include a mix of simple, complicated aspects and complex aspects. • Avoid blanket descriptors like “complex projects” and “complex environments”

Complexity Myth Busters Myth #3 • Operating effectively in complexity requires expensive technical tools. (Closer to the) Truth • Simple rules and principles steer effective practice in complexity.

Operating in Complexity

Operating in Complexity Frisbee Flight Simulation • Models based on initial speed, shooter angle, and shooter height • Holding all other factors constant • Can’t account for changes in wind speed and direction, and other factors

Operating in Complexity

Performance Monitoring & Complexity-Aware Monitoring Performance Monitoring measures the predicted. Results intended by us Pathways of change planned by us Implementation strategies designed by us Indicators Targets

Performance Monitoring & Complexity-Aware Monitoring Complexity-Aware Monitoring tracks the unpredictable. Results (beyond those originally intended by us) Factors & actors outside the project Multiple pathways of change & feedback loops Systems qualities Triggers

Performance Monitoring & Complexity-Aware Monitoring Performance monitoring & complexity-aware monitoring complement one another. Performance monitoring for simple & complicated aspects of projects and strategies. Complexity-aware monitoring for complex aspects of projects and strategies.

Practicing Complexity-Aware Monitoring 3 Principles 5 Approaches

Principles of Complexity-Aware Monitoring 1. Synchronize monitoring with the pace of change 2. Attend to performance monitoring’s 3 blind spots a. broader range of outcomes b. alternative causes c. full range of non-linear pathways of contribution 3. Consider relationships, perspectives, and boundaries (3 key systems concepts) a. the structures, processes, and exchanges linking actors and factors within a system b. different perspectives within a system c. what is in and what is outside the system

5 Recommended Approaches 1. Sentinel Indicators 2. Stakeholder Feedback 3. Process Monitoring of Impacts 4. Most Significant Change 5. Outcome Harvesting

5 Recommended Approaches A Sentinel Indicator: • Is a proxy for complex processes that are difficult to study in their entirety – A sentinel species is a proxy for an eco-system – Stock-outs are a proxy of market efficiency • Is easily communicated • Signals the need for further analysis and investigation Why important? • A simple way to monitor complexity

5 Recommended Approaches Stakeholder Feedback • Family of approaches • Privileging perspectives of partners, beneficiaries or those excluded from a project • Seek diversity rather than consensus Why important? • Complexity is diverse • Knowledge of the system is partial and predictability is low.

5 Recommended Approaches Process Monitoring of Impacts • How a result at one level is used to achieve results at the next level (outputs to 1st level results) • Predicted and emergent processes • Between results boxes in a LogFrame Why important? • Bounds areas of complexity most critical to success • 2 blind spots: factors outside the project, feedback loops

5 Recommended Approaches Most Significant Change • Collection and analysis of stories describing the most important project outcomes • Perspectives of different stakeholder groups are represented in the criteria for determining a significant change Why important? • Captures broad range results (intended/unintended, positive/negative). • Makes diverse perspectives on results explicit.

5 Recommended Approaches Outcome Harvesting • Discover results without reference to predetermined objectives, and work backwards to determine the contribution. • Emphasis on verification and describing contribution Why important? • Captures broad range results (intended/unintended, positive/negative).

Complexity Champions to Test New Methods LER support

Next Steps Work collaboratively to -• Identify complex aspects of activity/project/strategy – Distinguish complex from complicated and simple • Select approach – Suitable to monitoring purpose – Suitable to activity/project/strategy – Suitable for a trial (timing, resources, learning culture) • Build learning circle for trial

What LER is Offering LER will provide support for complexity-aware M&E • Are We Working in Complexity? Questionnaire & workshop (February – March) • Decision support – Which approach is best? • Webinars, case examples & resources for approaches • TA for trials of new methods space for innovation. • Support intentional learning at all stages of trials • Complexity-Aware Evaluation Discussion Note (forthcoming)

Add a comment

Related presentations

Cfbp barometre octobre

Cfbp barometre octobre

November 10, 2014

VITOGAZ vous présente: CFBP baromètre gpl carburant

Ata Escrita da 16ª Sessão Ordinária realizada em 16/10/2014 pela Câmara de Vereado...

Ata Escrita da 10ª Sessão Extraordinária realizada em 16/10/2014 pela Câmara de Ve...

Rx1 nasil kullanilir

Rx1 nasil kullanilir

November 8, 2014

Rx1 zayiflama hapi, kullanimi nasildir, yan etkileri var mi? yan etkiler var ise h...

Esposto del MoVimento 5 Stelle sul Patto del Nazareno

Slide Servizi postali

Slide Servizi postali

November 7, 2014

Slides per i servizi postali presentati in occasione dell'incontro azienda e organ...

Related pages

Complexity-Aware Monitoring: Briefing on USAID’s New ...

About. 2016 Board of Directors
Read more

Complexity-Aware Monitoring - USAID Learning Lab

Complexity-Aware Monitoring USAID's Learning, Evaluation, and Research Office Briefing on USAID’s Discussion Note ... Test New Methods .
Read more

Discussion Note

Discussion Note . Complexity-Aware Monitoring . ... USAID with new approaches and methods for monitoring ... Note that will broaden USAID's ...
Read more

Discussion Note: Complexity Aware Monitoring | Better ...

USAID’s Office of Learning, ... Discussion note: Complexity aware monitoring. ... ...
Read more

VERSION 1.0 | SEPTEMBER 2014 Discussion Note

Complexity-aware monitoring is intended to complement and enhance performance monitoring, USAID’s ... Complexity-Aware Monitoring Discussion Note.
Read more

Complexity-Aware Monitoring | The Aspen Institute

Complexity-aware monitoring is ... Complexity-Aware Monitoring Discussion Note" is ... discuss complexity-aware monitoring and USAID’s efforts ...
Read more

IOCE survey of Voluntary Organizations for Professional ...

IOCE survey of Voluntary Organizations for Professional Evaluation (VOPEs) ... Monitoring Progress, ... Briefing on USAID’s New Discussion Note. 25 Feb.
Read more

M&E Plan Monitoring Component | Project Starter — USAID

M&E Plan Monitoring ... complexity is an important issue to consider and USAID’s Discussion Note ... USAID Discussion Note on Complexity Aware Monitoring.
Read more