Published on March 7, 2014
CSI PROFESSIONALS BRIEFING 2014 Reana Rossouw Next Generation Consultants
Session Two: Impact Investment Index
Our Context Started in 2009 – global benchmarking – 40 models currently being applied internationally Conclusion: Africa needs its own solution – applicable to both the funding and development sectors Practitioners need capacity, support and knowledge Practitioners don’t need complex methodologies or approaches requiring specialist skills, licenced software or specific hardware or expensive solutions The industry needs a transparent, comparable, and flexible solution that contextualises and take into consideration the complexities, relationships and fundamental development principles of our specific development context
Our Objective The purpose of Impact Investment Index is to create a shared performance measurement system to be utilized by all organisations in the social investment and community development sector The current system lacks coordination, leading to added expense, limited learning, and inadequate ability to assess shared value and collective impact
Why the pressure to measure? The debate on impact and return on investment are playing out in three arenas: In private foundations and corporate CSI divisions Aiming to be more strategic about their philanthropy, grant making and social/community investments In nonprofit organisations in response to pressures from corporates, foundations and government To be more accountable for the resources received and program outcomes expected Among international development organisations such as bilateral government agencies and intermediary organisations Seeking to improve development effectiveness and lessen dependency on grant/development aid
Why now? Funders are increasingly asking for demonstrable results – to understand the difference they make, directly and indirectly This trend is accelerating, as the development and in particular the funding sector is increasingly looking to pay by results – to learn from what they, and those they fund, do It is not just donors that care about impact. In a age of competition, transparency, recessionary economies, there is growing competition for resources
Three primary applications of Impact Assessments Prospective Looking forward to determine whether or not the projected costs and benefits indicate a favorable investment Ongoing Testing assumptions and projections along the way in order to aid course correction Retrospective Looking back to determine whether or not it was a favorable investment given the costs incurred, and therefore to inform future decisions
Variety of purposes – our school of thought One can and should use impact data to make funding allocation decisions across program areas You can compare programs once you get in the sector of health, but you cannot compare health vs. arts vs. education vs. sport. One can and should use impact data to make funding decisions within program areas It is not about building a unifying measurement across domains, but to build a conceptual framework for understanding the biggest impact across a Rand value unit. So it is not about comparing health to education to sport or the arts, but to determine which program yields the highest return for the most effective use of resources
The conundrum Funders and non-profits often use the words “evaluation” and “impact” loosely, stretching these terms to include any type of report on the use of funds or the results they achieve Practitioners should distinguish between measuring performance (monitoring inputs, activities, and outputs), measuring outcomes (near-term results), and evaluating impact (long-term changes that can be attributed to the investor’s activities) as well as return on investment (benefits accrued to the investor) as a result of funds and other resources invested
What is impact “Measure of the tangible and intangible effects (consequences) of one thing's or entity's action or influence upon another. An impact evaluation/assessment assesses changes in the wellbeing of individuals, households, communities or firms that can be attributed to a particular project, program or policy. The central impact evaluation question is what would have happened to those receiving the intervention if they had not in fact received the program. ” For us this means: The broad or longer-term effects of a project or organisation’s work (also referred to as the difference it makes). This can include effects on people who are direct users/beneficiaries of a project or organisation’s work, effects on those who are not direct users/indirect beneficiaries, or effects on a wider field/aspect such as government policy, processes, systems, infrastructure or support systems
What did we want to achieve? To provide evidence To demonstrate performance To prove accountability To show program/ investment effectiveness To demonstrate value To empower and capacitate communities and funders Ultimately - to alleviate, eliminate and eradicate poverty
What we have achieved Assessed R1 billion worth of investment but also considered all types of input resources from money to hours, products and services, infrastructure and skills and volunteerism Across both socio economic and enterprise development sectors Including 15 focus areas 400 programmes – from flagship to donations, once off to 3-5 year programs, sponsorships and cause related marketing, social and business enterprises Across 15 dimensions of impact and 25 dimensions of return Developed a library of more than 500 indicators
But more than that Impact across the value chain: Outcomes of partnerships, relationships and applied resources (to be more sustainable and effective) Outcomes of the initiative (policy change, organisations working more effectively together reducing/alleviating/ eradicating poverty) Outcomes at the community level (are we moving the needle against the strategic objectives/mandates indicators?) Returns for the donor/funder (are we getting value, recognition, licence to operate, etc.)
Guiding Principles of our work Impact means impact The goal is to understand what changes as a result of the investment from funders in communities as beneficiaries and recipients of interventions The impact is shared The goal is to understand who is impacted along the impact value chain – including funders , intermediaries and beneficiaries Impact includes and involve all stakeholders Analysis must be comprehensive. Instead of cherry picking something that’s working and leaving out what is not, the analysis should include all aspects of impact and those impacted Results must be transparent Companies should report to their investors, and investors should aggregate and report results. What is left out should be stated. Assumptions and sources should be stated. It should be possible for a third party to replicate the analysis based on the documentation of it and get the same result. Context matters It is harder to create a stable job in a rural area than in a city. The qualitative and quantitative context should be provided to inform the impact as well as an understanding of how much of the problem may exist or remain.
Challenges remain There are two main challenges to measuring impact One: how to do it well Two: how to do it cheap Getting both to happen can be done in either of two ways: Force it, by making it a requirement Or attract it, by creating an environment in which the parties whose efforts are necessary to make it happen want it
The breakthrough – a Comparative Performance System 1 2 3 • Shared measurement system • Provides a menu of common indicators and a common platform to report on different outcomes and indicators • Comparative performance system • Involves asking participants to report on the same indicators, using the same process and measure to compare performance • Adaptive learning system • Supports ongoing collaboration and learning among organisations to align efforts and goals, and to measure common outcomes and indicators The Outcome: The Impact Investment Index
Our process for impact assessment • Share and act on learning and results • Seek to improve the investment and development practice • Understanding the difference/change the funder wanted to make. Who the intermediaries were that was supported and who beneficiaries was that was affected 4 Review 3 Assess • Identify and calculate the difference that was made and assess how and why it was made 1 Plan 2 Do • Design the assessment • Conduct the assessment • Collect the information needed about impact
Impact Value Chain - Our Focus Hierarchy Inputs Activities and Outputs The activities whose effects are to be measured. Goods and services generated by the use of inputs (shortterm) Outputs and Outcomes The results from activities. Expected changes in access, usage, behaviour or performance (medium-term) Impacts Returns Definition Resources invested (e.g. money, skills, hours, infrastructure, services, products, etc.) Ultimate (long-term) effect of the intervention on a key dimension of the development (e.g. living standards) (longterm) Direct or indirect business benefits/ value generated by activities/ programs/ interventions Quantitative Indicators Value of investment, number of hours, number of books, number of stakeholders Number of schools, number of volunteer, number of teachers, jobs, income, etc. % change – access to health services, education, government infrastructure, pass rates or sustainable jobs Students finding employment, people with skills finding employed due to increased health status Number of graduates hired by company, % change in grievance, complaints received by company Qualitative indicators Stakeholders satisfaction with the programme and measurable difference in their lives, environment, context Perceptions of Beneficiaries schools, educators, reporting benefits learners, community and application of skills, education, improved health Quality of links to local employment opportunities, perceptions of improved socio economic status or opportunities Changes in community / customer/ employee perceptions attributable directly/ indirectly to CSI
The Impact Investment Index - III Inputs Outputs/Outcomes Impacts • How • Cash, time, in kind, management costs • Why • Community Relations, community investments, commercial initiatives • Where • Geographic Location • Who • Stakeholders Affected, Impacted • What • Focus Areas, Investment Portfolios • Community Benefits • Qualitative • Quantitative • Dimensions of impact (15) • Community Impacts • Business Impacts/Return • Business Benefits • Qualitative • Quantitative • Dimensions of Impact and Return (25) • Shared Value • +500 Indicators
Impact Assessment – HOW?
Impact Assessment – WHAT?
Dimensions of Impact – FOR WHOM? How do we calculate? 1. We count each and every stakeholder group 2. We count each and every impact 3. We distinguish between community and business impact 4. Get to a figure: X:Y
Calculating Impact – The Process
Proving Impact and Return THE RESULTS
Some examples of impact (1) Greater output and productivity The economic value of increased revenue and reduced costs. The increase in local economic output could be estimated with multiplier analysis Employees of Suppliers Accumulation of human capital, empowerment and self confidence The cost of external skills courses could be used. The increase in wages resulting from greater levels of human capital. Willingness to pay for courses Wider Local Community Increased welfare – i.e. lower unemployment, improved health, education The value of reduction in public expenditure on health care services and unemployment benefits Local Suppliers ED Supplier Development Program Supplier capacity building and access to banking services
Some examples of impact (2) Local businesses Greater output and productivity The economic value of increased revenue and reduced costs. The increase in local economic output could be estimated with multiplier analysis Local Students Improved quality of education, better grades Increased wages resulting from improved education levels Local Households Increased communications, and quality of life Monetary value of savings Rural electrification of village Increased publicity Investor/Donor Value of increased publicity Increased employee morale, participation, skills development Value of retained employees
Some examples of impact (3) Farmers and their families Greater output and productivity, income and empowerment The economic value of increased revenue and reduced costs. Local Authorities Increased tax revenue The increase of local economic output could be estimated with multiplier analysis The value of increased income tax and export duties The value of increased savings Training of local farmers Local Consumers and Communities Improved quality and quantity of food The value of increased consumption The value of leveraged resources Attraction of new resources/donors Intermediary Increased income, increased employment Value of salaries Value of publicity The value of Increased quality of life, in particular health which lead to increased productivity
How much was spent – Where?
Spent per: Year, focus area, region
Testing Expectations Focus Area Ranking 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Bottom 10 Education Education Community Development Economic Development Economic Development Community Development Education Environment Economic Development Health Community Development Community Development Community Development Infrastructure Infrastructure Other Community Development Community Development Disaster Relief Safety and Security Number of Responses 91 62 56 52 46 44 42 40 36 35 33 29 25 16 11 11 10 8 5 4 Top 10 Community Investment Areas Education Skills Development Youth Job Creation Entrepreneurs Children Bursaries Environment Business Development Health Physically Challenged Women The Aged Housing Infrastructure Other Culture Arts Emergencies and Disasters Safety and Security
Comparing and Benchmarking Across Industry Across Focus Areas Across Perceptions Including local and global benchmarks Vision, Mission, Strategy and Policy, guidelines, frameworks Strategic intent, objectives, mandate Focus Areas Grant making criteria Beneficiaries Intermediaries Operational alignment and integration Governance and compliance Reporting structure Budget and resources Monitoring and evaluation Employee Volunteerism Marketing, communication and awareness Sponsorship, donations and cause related marketing Partnerships Recognition, Awards, perceptions Future focus, GAP and SWOT analysis
Strategic and Operational Review Theory of change to measure impact
Co-define and co-develop indicators to measure impact
Co-Development of impact assessment guidelines Section One: Project Name: Basic information: Project Budget: Section Two: Section Three: Stakeholders who benefited or were impacted by the program Stakeholder 1 Stakeholder 2 Stakeholder 3 Stakeholder 4 Stakeholder 5 Section Four: Specific aspects of Impact: Project Description and Specific Outcomes: In your opinion – what do you think Nedbank gained/ benefitted by funding this program? In other words what was Nedbank’s return on investment from funding this program? What do you think about or is your opinion or impressions of Nedbank as a funder? What do you think Nedbank can do in future to be a better funder? Do you think the Nedbank Foundation engage in a respectful, honest, approachable way? Perceptions of Nedbank: Project Owner: Organisation Project Manager: Contact Details: Do you know what the Nedbank Foundation Corporate Social Investment strategy is and do you think it is applicable in the South African context? Do you have any other comments or feedback or suggestions for the Nedbank Foundation? Qualitative Impact Qualitative Impact Qualitative Impact Qualitative Impact Qualitative Impact Economic Impact Environmental Impact Social Impact Short Term Impact Positive Impact Intended Impact Medium Term Impact Negative Impact Unintended Impact Quantitative Impact Long Term Impact Combined Impact Any other Impact Section Five: Lessons learnt, insights, additional comments, feedback, suggestions Section Six: Consultants Comments Section Seven: Scores: SG: NF: AOI: EC: EN: SO: S: M: L: P: N: C: I: UI: OI: T
Section One: Project Name: Basic information: Leliebloem House Child and Youth Care Centre Project Budget: Project Owner: Organisation Leliebloem House Child and Youth Care Centre Transparency in assessment Project Manager: R5,6m pa Francisco G Cornelius Contact Details: Section Three: Stakeholders who benefited or were impacted by the program Stakeholder 1 Leliebloem House Child and Youth Care Centre Qualitative Impact Qualitative Impact Basic care and protection, provided. The children’s basic needs are provided for. 84 children receive basic social, health, educational and developmental services. Less dependent on other support (government) Leverage funds of other donors 2 Section Four: Specific aspects of Impact: Qualitative Impact Jobs provided to staff 2 021-6964947/ 0827719233 Qualitative Impact Qualitative Impact Children cared for – behaviour and motivation changes – basic quality of life improved 2 Economic Impact It was a top up to the overall budget we need to provide these services. 1 Intended Impact Section Five: Lessons learnt, insights, additional comments, feedback, suggestions Section Six: Consultants Comments Section Seven: Scores: Project Services to Vulnerable Children in Description and Residential Child Care. Specific Outcomes: Offering various therapeutic programmes to both the children and their families that eventually allows the children to be reunified with family or community. Environmental Impact Yes so that we can render Unintended Impact the services to the children. 1 Operational expenses leveraged 1 Marketing and publicity exposure 1 Quantitative Impact 84 Children Number of staff Number of Jobs None evident 0 Social Impact Media exposure and recognition as an organisation with good financial practices. 1 Any other Impact Value of funds leveraged / generated 4 It gave these children a second chance they may not have received under normal circumstances. 1 Support and leverage government funds/ resources & support work of government – providing basic services to communities 1 We have a reputation of not saying no to opportunities. We pride ourselves in staying up to date with training for all staff and trying to do staff development programs 2 a year, if the funding permits. We have dedicated staff that could leave any time because of the poor salary structures or lack of funding, but they stay on and work in the life-space of these vulnerable children on a daily basis. In our opinion – some consideration should be given to assisting organisations to leverage funds better – i.e. capacity building (training and other assistance) for organisations to use funds not only for operational expenses but also to raise additional funds. Some organisations will never be sustainable, but similarly organisations should not be overly dependent on a single organisation as dependencies are created and it may be considered unsustainable development. SG: 1 EC: 1 S: 1 P: 1 I: 1 T:25 BOE: 1 EN: 0 M:1 N: 1 UI: 1 QL: 8 SO: 1 L: 1 C: 1 OI: 1 ON: 4
Scorecard per project per focus area per rating
Ratings and Rankings
Impact Across Portfolios Community Impact - High: Business Impact - High: Community Impact - High: Business Impact - High: •A & C Trust (215:3) •WESSA (98:6) •Sports Trust (84:3) •PPT Housing (80:10) •Kidshaven (78:1) •SIFE (73:4) •Childwelfare (69:1) •PPT Housing (80:10) •WESSA (98:6) •Hippo Roller (63:7) •LearntoEarn (56:2) •Siyabonga (47:1) •Holdinghand (40:1) •Inchanga (39:1) •Inst Deaf (38:1) •Siyazisa (31:3) •Cape Winem (35:2) •RayMhalaba (30:2) Community Development Business Impact - Low: Average - 65 : 3 •Habitat (42:2) •Lebone (34:2) •Leliebloem (28:1) •Childline (24:1) •Princess A (24:1) •WATTF (18:0) Community Impact - Low: •Wylie (62:2) •Epworth (54:1) •St Josephs (52:2) Community Impact - High: Business Impact - High: •Doctors/Bor (82:3) •Leratong (38:0) •Doctors/Bor (82:3) Health Average - 36 : 2 Community Impact - Low: Business Impact - Low: •Ramik-Gast (29:2) •ST Lukes (28:2) •Reakgona (20:1) •Ramik-Home (19:1) Skills Development Average - 36 : 1 Community Impact - Low: Business Impact - Low: •Readucate (24:1) •FurnTech (23:0) •Readucate (24:1) •FurnTech (23:0) Community Impact - High: Business Impact - High: •MyFmyCa (104:2) •Edutainer (90:2) •Sediba (90:1) •Fort Hare (72:2) •SEED (69:1) •Penreach (66:1) •Enke (64:1) •Maths NPC (62:1) •SADET (59:1) •Asset (58:1) •Matric Rev (75:5) •Pinelands (47:4) •Mobile Labs (39:4) •Fundisa (47:3) Community Impact - Low: •Bursaries (54:2) •Leap (54:2) •Protec (54:2) Education Average - 56 : 2 Business Impact - Low: •WC Essay (53: 1) •Heartlines (51:0) •Sparrow (47:1) •Acorn/BHT (40:1) •Dominican (31:1) •Chess Kids (27:0) •TracSA (26:1) •Spell It (24:0)
Collective impact and return
Impact and return score cards
Some examples of indicators Business Return Mitigate climate change risk Reduce environmental accidents and remediation Reduce water use and management; waste management and effluents Reduce and assist with energy management GHG emissions and air pollution Reduce biodiversity impacts Increase communications and engagement Increase customer satisfaction Increase access to tenders Increased access to new markets Increased reputation Increased sales Increased staff motivation Increased BEE Scorecard Strengthened value chain Maintained/obtained social licence to operate Increased product innovation and services development Community Impact Increased quality of life – 45 dimensions Increased social cohesion Increased access to services, infrastructure, support, income, jobs – 45 dimensions Skills Development – 30 dimensions Infrastructure – transport, hospitals, schools, electricity, water, housing – 60 dimensions Education – infrastructure, in general, specific – early childhood, primary, secondary, tertiary Economic, Environmental, Social Health, education, housing, safety & security, enterprise development, sport, agriculture, social justice, social participation, technology Impact on individuals, communities, government, companies, funding partners Quantitative, qualitative Volunteerism - dimensions
The impact of the impact assessment
What our clients say - investors Improved financial analysis and reporting as well as management capacity and operational efficiency Improved operational efficiency and information management processes and systems Improved board governance and oversight, understanding, knowledge and capacity Improved M&E; reporting; communication; stakeholder relationships and partnerships Improved operational processes from application requirements; due diligence processes; reporting requirements; responsiveness; knowledge and expertise of staff Improved access to further investment
What our clients say – intermediaries and beneficiaries Intermediaries We feel comfortable with the transparency of the process The process have added value to our own work – especially M&E and reporting practices The processes have increased our effectiveness and own performance; increased our learning and knowledge; built internal capacity; and increased our credibility We believe we were assured independently by someone who can verify our claims – it validated our own beliefs Beneficiaries We had an opportunity to talk without being judged – we could be honest We learnt to document our own work and the contribution we made We feel we are being trusted, being heard and someone asks our opinion We had an opportunity to share and learn
What we have learnt (1) One size of evaluation does not fit all Funders should take extra time in their planning to learn which evaluation techniques will work with indigenous populations and specific communities. Trying to define and measure empirical changes is difficult and “a slippery process,” Understanding, defining, qualifying and quantifying long-term social change is an incremental effort and on-going process. Although evaluations are significant for their organisational development, requiring intermediaries to perform them is a challenge. A suggested solution is to consider evaluation results as one – but not the only – source of information and to couple it with knowledge, experience, strategy, and context to obtain a fuller picture.
What we have learnt (2) Impact assessments describes three relationships: between evaluation and the funder’s approach; between evaluation and the funder’s strategy; and between the intermediary evaluation and the beneficiary evaluation. Grantmakers may have to change their funding approach to better accommodate intermediaries’ capacity to conduct evaluations, or provide funding to help them develop it Funders need to be clear about their overall goals and how individual investments fit within that model. Intermediary and funder evaluations should be linked: Both partners face the same issues of inadequate resources, coordination and expertise for conducting evaluations.
What we have learnt (3) ANY resources CAN be measured – books, wheelchairs, buildings, time – cash and non-cash The same project can deliver varied results for different funders How and on what you spend the money (inside the program) has a direct influence on the impact and return The strategy and focus areas has to clearly define the return and impact required - upfront Sustainability has to be clearly defined for exit and completion Indicators have to be developed, agreed, and documented as part of the contractual phase Internal monitoring and external evaluation processes has to be established and adhered too - Impact Assessment does not replace evaluation and monitoring
What we have learnt (4) You have to consider the impact of the impact assessment on so many levels As a result of our work we can now categorically state that most programs: Have only short term impact Those that have medium term impact are not necessarily sustainable The long term impact is mostly social only as opposed to economic impact which really contributes to poverty alleviation and eradication! It is possible to determine impact and return The real value lies in independent, verifiable, assurance of social investment expenditure, program results, outcomes and impact
What I have learnt (5) There is so much return for business – we just need to prove it Growth Return on Capital Risk Management Management Quality New products Price premium / market share Regulatory risk Leadership New markets Capital efficiency Reputational risk Employee development New customers Human efficiency Licence to operate Adaptability Supply chain /security of supply Long term strategic view Innovation Reputation Differentiation
What I now know… We all have impact – but it is not necessarily measurable and sustainable impact Do we want economic or social impact? By implication social impact help people right now – but may not help them in the future – which renders the project/our intervention UNSUSTAINABLE Sometimes it is our own (CSI Practitioner’s) fault we don’t have higher impact as we decide what, who and how to fund/not to fund The most sustainable projects/programs with the highest impact have social, socio economic and ECONOMIC impacts i.e. the number of jobs created Sometimes there is negative impact – i.e. dependencies are created Mostly there is only short term impact –which makes our interventions UNSUSTAINABLE
Moving from Strategic to Catalytic Responsive Strategic Catalytic Responsive to society Strategic to core business Catalytic change for the good and survival of mankind
Going Forward and Doing it better Impact assessment can help funders, intermediaries, and beneficiaries they support to: Plan how their work will make a difference, and determine how much of a difference they are making Understand what does or does not work and why and detect unintended consequences Build an (scientific) evidence base to share with others, thus influencing and informing debate, and increasing the sector’s body of knowledge Challenge yourself and others by looking critically at your/their work in order to improve, to replicate good work, or to innovate and develop new processes, products and services Inspire and motivate staff, trustees and other stakeholders including volunteers, beneficiaries, policy makers and other practitioners, funders and investors to build relationships with others, communicate added value and raise the profile of their work
In closing Involve stakeholders Establish the scope and identify the key stakeholders impacted Understand what changes Map the outcomes – identify the indicators to measure impact Value things that matter Community Impact Look for evidence, don’t forget baseline and impact studies If you can count and define indicators you can determine impact and return Only include what is material Focus on agreed outcomes first then incidental impact Do not over claim Calculate the impact based on evidence Be transparent Report the outcome – the good and BAD news Verify the result Share the learning Business Return
Questions and Discussion How to start Where to start How to do it When to do it Just do it
Contact Reana Rossouw Next Generation Consultants - Specialists in Development E-mail: firstname.lastname@example.org Web: www.nextgeneration.co.za PLEASE NOTE: THIS PRESENTATION IS PART OF A LARGER BODY OF RESEARCH! THIS INFORMATION IS COPYWRITED AND THE INTELLECTUAL PROPERTY OF NEXT GENERATION CONSULTANTS.
... in 2006 in the book Social Return on Investment: ... platform for measuring impact and late 2014 they have ... social and corporate ...
Measuring the impact and return on investment of social investment and community development. ... Social Investment | Next Generation.
community investment, enhance local development ... measuring return on corporate social ... of the impact their community investments are ...
This new guide to Social Return on Investment is timely, ... measuring social, ... The total impact. 4 Social return calculation
Social Return On Investment ... measuring the social impact of community investment ... difficulty in accessing corporate IT development or problem ...
Measuring the impact and return on investment of ... on community/social investment and development ... development, corporate ...
... you're earning a 50% return on investment. ... and received a good return on your AdWords investment. ... About return on investment (ROI) Measuring ...
Evaluation and Social Return on Investment ... of approaches to measuring social value and impacts, ... impact of investments in community and ...