PCM - Project Cycle Management, Training on Evaluation

0 %
100 %
Information about PCM - Project Cycle Management, Training on Evaluation

Published on December 4, 2007

Author: rexcris

Source: slideshare.net

P roject C ycle M anagement Training Course on " Evaluation"

Definitions Evaluation is the systematic assessment of the worth and merit of some object. Evaluation is the systematic acquisition and assessment of information to provide useful feedback about some object.

Evaluation is the systematic assessment of the worth and merit of some object.

Evaluation is the systematic acquisition and assessment of information to provide useful feedback about some object.

Evaluation: A Definition An evaluation is an assessment, as systematic and objective as possible, of an on-going or completed project, programme or policy, its design, implementation and results. The aim is to determine the relevance and fulfilment of objectives, develop - mental efficiency, effectiveness, impact and sustainability. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision - making process of both partner countries and donors. OECD, DAC, 1991

An evaluation is an assessment, as systematic and objective as possible, of an on-going or completed project, programme or policy, its design, implementation and results. The aim is to determine the relevance and fulfilment of objectives, develop - mental efficiency, effectiveness, impact and sustainability.

The Project Cycle Key characteristics: breaks down the life of a project / programme into phases defines key decisions, information requirements, & responsibilities at each phase each phase must be completed for the next to be tackled with success allows for corrective measures during implementation, based on monitoring and mid-term evaluation draws on evaluation experience for the design of future programmes & projects

Key characteristics:

breaks down the life of a project / programme into phases

defines key decisions, information requirements, & responsibilities at each phase

each phase must be completed for the next to be tackled with success

allows for corrective measures during implementation, based on monitoring and mid-term evaluation

draws on evaluation experience for the design of future programmes & projects

When to do evaluations? Mid-term evaluation End of project/ programme evaluation Ex-post evaluation During implemen-tation At the end of implemen-tation After imple-mentation Also used as term: Ex ante evaluation

Evaluations: What for? To improve Decision making Resource allocation Accountability Through: Informing the public Informed key decision-making processes Encouraging on-going organisational learning

To improve

Decision making

Resource allocation

Accountability

Through:

Informing the public

Informed key decision-making processes

Encouraging on-going organisational learning

Logframe Basics ‘ ... IF results are delivered, AND assumptions hold true, THEN the project purpose will be achieved ...’ Objectively Verifiable Indicators Intervention Logic Sources of Verification Assumptions Overall Objectives Project Purpose Results Activities Means Cost Pre-condi-tions

Good indicators Do partners and stakeholders agree that this indicator makes sense to use? Owned Will the data have utility for decision-making and learning? Useful Can data be collected easily, on a timely basis at reasonable costs? Accessible Is the data consistent or comparable over time? Reliable Is the definition precise and unambiguous about what is to be measured? Objective Does the indicator directly represent the objectice it is intended to measure? Valid

Indicators: An Example Objective: Pollution load of wastewater discharged into the Blue river is reduced Select the indicator: Concentration of heavy metal compounds (Pb, Cd, Hg) Define the targets: Define the quantity: Co ncentration of heay metal compounds (Pb, Cd, Hg) is r educed by 75 % compared to year x levels … (particular attention should be paid to the availability of baseline information) Define the quality: ... to meet the limits for irrigation water . .. Define the target group: ... , used by the farmers of Blue village, ... Define the place : ... in the Blue river section of the District ... Determine the time: ... 2 years after the project has started

Objective: Pollution load of wastewater discharged into the Blue river is reduced

Select the indicator: Concentration of heavy metal compounds (Pb, Cd, Hg)

Define the targets:

Define the quantity: Co ncentration of heay metal compounds (Pb, Cd, Hg) is r educed by 75 % compared to year x levels … (particular attention should be paid to the availability of baseline information)

Define the quality: ... to meet the limits for irrigation water . ..

Define the target group: ... , used by the farmers of Blue village, ...

Define the place : ... in the Blue river section of the District ...

Determine the time: ... 2 years after the project has started

Special challenges facing the donor agencies (1) Work in many different countries and contexts Have a wide diversity of projects in multiple sectors Often focus on capacity building and policy reform, which are harder to measure than direct service delivery activities

Work in many different countries and contexts

Have a wide diversity of projects in multiple sectors

Often focus on capacity building and policy reform, which are harder to measure than direct service delivery activities

Special challenges facing the donor agencies (2) Are moving into new areas such as good governance, where there's little performance measurement experience Often lack standard indicators on results that can easily be aggregated across projects Are usually only minor actors affecting impacts, with subsequent problems in attributing them to their agency's activities

Are moving into new areas such as good governance, where there's little performance measurement experience

Often lack standard indicators on results that can easily be aggregated across projects

Are usually only minor actors affecting impacts, with subsequent problems in attributing them to their agency's activities

Special challenges facing the donor agencies (3) Typically rely on outcome and impact data collected by partner countries, who have limited technical capacity and resources, with subsequent quality, coverage and timeliness problems.

Typically rely on outcome and impact data collected by partner countries, who have limited technical capacity and resources, with subsequent quality, coverage and timeliness problems.

Criteria for Evaluating Development Assistance Relevance Efficiency Effectiveness Impact Sustainability

Relevance

Efficiency

Effectiveness

Impact

Sustainability

The Logframe and the 3 basic evaluation criteria Overall Objectives Project Purpose + Assumptions Results + Assumptions Activities + Assumptions + Pre-conditions allocation action utilisation change Efficiency Means Impact Overall Objectives + Assumptions Results + Assumptions Activities + Assumptions allocation action utilisation change + Pre-conditions Means Overall Objectives Project Purpose + Assumptions Results + Assumptions Activities + Assumptions allocation action utilisation change Effective- ness Project Purpose + Pre-conditions Means

Logframe and basic evaluation criteria: Major questions Overall Objectives Project Purpose + Assumptions Results + Assumptions Activities + Assumptions + Pre-conditions allocation action utilisation change Means Efficiency Effective- ness Impact How were inputs and activities converted into Results? How well did the Results contribute to the achievement of the Project Purpose? Which benefits on society and sector?

The 4th evaluation criterion: Relevance Overall Objectives Project Purpose + Assumptions Results + Assumptions Activities + Assumptions + Pre-conditions allocation action utilisation change Means Efficiency Effective- ness Impact Relevance Quality of planning and adaptation, including relevance of problems to correct beneficiaries, OVIs, means, cost, assumptions, risks How were inputs and activities converted into Results? How well did the Results contribute to the achievement of the Project Purpose? Which benefits on society and sector?

The 5th evaluation criterion: Sustainability Overall Objectives Project Purpose + Assumptions Results + Assumptions Activities + Assumptions + Pre-conditions allocation action utilisation change Means Efficiency Effective- ness Impact Relevance Sustain-ability Have and will services and benefits be maintained? Quality of planning and adaptation, including relevance of problems to correct beneficiaries, OVIs, means, cost, assumptions, risks How were inputs and activities converted into Results? How well did the Results contribute to the achievement of the Project Purpose? Which benefits on society and sector?

Summary: The 5 evaluation criteria The fact that the results were obtained at reasonable cost, i.e. t he cost, speed and management efficiency with which Means/ Inputs and Activities were converted into Results , and the q uality of the Results achieved . Efficiency The effect of the project on its wider environment, and its contribution to the wider objectives summarised in the Overall Objectives . Impact The appropriateness of project o bjectives to the real problems of the intended beneficiaries that it was supposed to address, and to the physical and policy environment within which it operate d . Relevance The likelihood of a continuation in the stream of benefits produced by the project after the period of external assistance has ended. Sustainability How well the Results contribute to the achievement of the Project Purpose , and how Assumptions have affected project achievements. Effectiveness

Relevance = The extent to which the aid intervention is suited to the priorities and policies of the target group, partner country and donor Possible questions: To what extent are the objectives of the programme still valid? Are the activities and outputs of the programme consistent with the broader objectives and do they contribute to the attainment of these objectives? Are the activities and outputs of the programme consistent with the intended impacts and effects? … … …

= The extent to which the aid intervention is suited to the priorities and policies of the target group, partner country and donor

Possible questions:

To what extent are the objectives of the programme still valid?

Are the activities and outputs of the programme consistent with the broader objectives and do they contribute to the attainment of these objectives?

Are the activities and outputs of the programme consistent with the intended impacts and effects?







Efficiency = Efficiency measures the outputs – qualitative and quantitative – in relation to the inputs. It is a term which signifies that the aid uses the least costly resources in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted Possible questions: Were the activities cost-efficient? Were objectives achieved on time? What were the major factors influencing the achievement or non-achievement of the objectives? … … …

= Efficiency measures the outputs – qualitative and quantitative – in relation to the inputs. It is a term which signifies that the aid uses the least costly resources in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted

Possible questions:

Were the activities cost-efficient?

Were objectives achieved on time?

What were the major factors influencing the achievement or non-achievement of the objectives?







Effectiveness = A measure of the extent to which an aid intervention attains its objectives Possible questions: To what extent were the objectives achieved/are likely to be achieved? What were the major factors influencing the achievement or non-achievement of the objectives? … … …

= A measure of the extent to which an aid intervention attains its objectives

Possible questions:

To what extent were the objectives achieved/are likely to be achieved?

What were the major factors influencing the achievement or non-achievement of the objectives?







Impact = The positive and negative changes produced by an intervention, directly or indirectly, intended or unintended. Possible questions: What has happened as a result of the programme or project? What real difference has the intervention made to the beneficiaries? How many people have been affected? …… …… ……

= The positive and negative changes produced by an intervention, directly or indirectly, intended or unintended.

Possible questions:

What has happened as a result of the programme or project?

What real difference has the intervention made to the beneficiaries?

How many people have been affected?

……

……

……

Sustainability = Sustainability is concerned with measuring whether the benefits of an intervention are likely to continue after donor funding has been withdrawn. Possible questions: To what extent did the benefits of a programme or project continue after donor funding ceased? What were the major factors which influenced the achievement or non-achievement of sustainability of the programme or project? … … …

= Sustainability is concerned with measuring whether the benefits of an intervention are likely to continue after donor funding has been withdrawn.

Possible questions:

To what extent did the benefits of a programme or project continue after donor funding ceased?

What were the major factors which influenced the achievement or non-achievement of sustainability of the programme or project?







Principles of evaluation Integrative part of planning and management : closely connected to decision making processes Impartiality & independence of the evaluation process in its function from the process concerned with policy making, the delivery and management of assistance (= separation of evaluation and responsibility for the project/ programme/policy) Credibility depending on expertise and independence of the evaluators & transparency to be seeked through an open process, wide availability of results, distinction between findings and recommendations Usefulness : relevant, presented in a clear and concise way, reflecting the interests and needs of the parties involved, easily accessible, timely and at the right moment  improved decision-making Participation of stakeholders (donor, recipient...) ; if possible: views and expertise of groups affected should form integral part of the evaluation; involving all parties  capacity building DAC 1991

Integrative part of planning and management : closely connected to decision making processes

Impartiality & independence of the evaluation process in its function from the process concerned with policy making, the delivery and management of assistance (= separation of evaluation and responsibility for the project/ programme/policy)

Credibility depending on expertise and independence of the evaluators & transparency to be seeked through an open process, wide availability of results, distinction between findings and recommendations

Usefulness : relevant, presented in a clear and concise way, reflecting the interests and needs of the parties involved, easily accessible, timely and at the right moment  improved decision-making

Participation of stakeholders (donor, recipient...) ; if possible: views and expertise of groups affected should form integral part of the evaluation; involving all parties  capacity building

DAC 1991

Evaluation, Monitoring and Audit (1) once or twice , e ssentially at the end or 'ex-post' draw ing lessons from the past in order to orient future policies and actions but also during implementation: mid-term evaluation to (re-) orient implementation When? external evaluators specialised in the subjects evaluated Who? in-depth analysis How? mainly analysis of the efficiency, effectiveness, impact, relevance and sustainability of aid policies and actions What? Evaluation:

once or twice , e ssentially at the end or 'ex-post' draw ing lessons from the past in order to orient future policies and actions but also during implementation: mid-term evaluation to (re-) orient implementation

external evaluators specialised in the subjects evaluated

in-depth analysis

mainly analysis of the efficiency, effectiveness, impact, relevance and sustainability of aid policies and actions

Evaluation, Monitoring and Audit (2) regularly , several times per year When? internal and external (staff, monitors.....) Who? rapid and continuous analysis, immediately useful to improv e on-going actions ; of key importance to improving performance How? mainly analysi s of efficiency and effectiveness (i.e. measuring actual against planned deliverables ) ; i s a systematic management activity What? Monitoring:

regularly , several times per year

internal and external (staff, monitors.....)

rapid and continuous analysis, immediately useful to improv e on-going actions ; of key importance to improving performance

mainly analysi s of efficiency and effectiveness (i.e. measuring actual against planned deliverables ) ; i s a systematic management activity

Evaluation, Monitoring and Audit (3) . during or after implementation When? external, professional auditors Who? verification of financial records (financial audit) How? traditionally checks whether financial operations and statements are in compliance with the legal and contractual obligations . M ore concerned with compliance, but better financial management can also contribute to improving current and future actions . More recently: Performance audit is strongly concerned with questions of efficiency amd good management What? Audit:

.

during or after implementation

external, professional auditors

verification of financial records (financial audit)

traditionally checks whether financial operations and statements are in compliance with the legal and contractual obligations . M ore concerned with compliance, but better financial management can also contribute to improving current and future actions . More recently: Performance audit is strongly concerned with questions of efficiency amd good management

Evaluation Evaluation is the systematic assessment of the value and merit of some object (in this case, a project or a programme) Evaluation is the systematic acquisition and assessment of information to provide useful feedback about the evaluated object.

Evaluation is the systematic assessment of the value and merit of some object (in this case, a project or a programme)

Evaluation is the systematic acquisition and assessment of information to provide useful feedback about the evaluated object.

Evaluation: major criteria Relevance Efficiency Effectiveness Impact Sustainability

Relevance

Efficiency

Effectiveness

Impact

Sustainability

Relevance = The extent to which the aid intervention is suited to the priorities and policies of the target group, partner country and donor Possible questions: To what extent are the objectives of the programme valid for the beneficiaries? Are the activities and results of the programme consistent with the overall objectives? … …

= The extent to which the aid intervention is suited to the priorities and policies of the target group, partner country and donor

Possible questions:

To what extent are the objectives of the programme valid for the beneficiaries?

Are the activities and results of the programme consistent with the overall objectives?





Efficiency = Efficiency measures the outputs – qualitative and quantitative – in relation to the inputs. It is a term which signifies that the aid uses the least costly resources in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted Possible questions: Were the activities cost-efficient? Are the unit costs comparable to … … …

= Efficiency measures the outputs – qualitative and quantitative – in relation to the inputs. It is a term which signifies that the aid uses the least costly resources in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted

Possible questions:

Were the activities cost-efficient?

Are the unit costs comparable to …





Effectiveness = A measure of the extent to which an aid intervention reaches its objectives Possible questions: To what extent was the project purpose achieved/is likely to be achieved? What were the major factors influencing the achievement or non-achievement of the project purpose? … …

= A measure of the extent to which an aid intervention reaches its objectives

Possible questions:

To what extent was the project purpose achieved/is likely to be achieved?

What were the major factors influencing the achievement or non-achievement of the project purpose?





Impact = The positive and negative changes produced by an intervention, directly or indirectly, intended or unintended. Possible questions: What real difference has the activity made to the beneficiaries? How many people have been affected? … …

= The positive and negative changes produced by an intervention, directly or indirectly, intended or unintended.

Possible questions:

What real difference has the activity made to the beneficiaries?

How many people have been affected?





Sustainability = Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Possible questions: To what extent did the benefits of a programme or project continue after donor funding ceased? What were the major factors which influenced the achievement or non-achievement of sustainability of the programme or project? … …

= Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn.

Possible questions:

To what extent did the benefits of a programme or project continue after donor funding ceased?

What were the major factors which influenced the achievement or non-achievement of sustainability of the programme or project?





Add a comment

Related presentations

Related pages

P roject C ycle M an agem en t T rain in g H an dbook

Project Cycle Management Handbook Training Action for ... of the project. þ The PCM training programme offers ... evaluation of projects and programmes ...
Read more

Project Cycle Management (PCM) Training in Bangkok

PROGRAM TITLE Project Cycle Management (PCM ... Improve the Monitoring and Evaluation (M&E) of the projects ... Management Training in Bangkok 2016;
Read more

Project Cycle Management - TACSO

by the European Union Project Cycle Management ... projects Regional Training on ... for monitoring & evaluation Regional Training on ...
Read more

Project Cycle Management (PCM) - Training Courses in Dubai ...

PROGRAM TITLE Project Cycle Management (PCM) ... Improve the Monitoring and Evaluation (M&E) of the projects ... © Superior Management Trainings ...
Read more

Training Project Cycle Management (PCM): How to succeed ...

Training Project Cycle Management (PCM): ... Cycle Management principles - The 5 cycles of a project ... and e ciency evaluation of a project ...
Read more

Supporting effective implementation of EC External Assistance

In 1992 the European Commission adopted “Project Cycle Management” (PCM) ... food security training project, ... to support a project evaluation ...
Read more

Networklearning

On the basis of the evaluation, adaptation of the project can take place and lessons learnt ... (PCM) Project cycle management is the term given to the ...
Read more

Project Cycle Management (PCM) - Training in Bali ...

Project Cycle Management (PCM) ... Improve the Monitoring and Evaluation (M&E) of the projects ... Publimaks International *** Management Training ...
Read more

Training in Dubai : Project Cycle Management (PCM)

PROGRAM TITLE Project Cycle Management (PCM) ... Training in Dubai : Project Cycle Management ... Improve the Monitoring and Evaluation (M&E) of the ...
Read more