8 Major Monitoring & Evaluation Challenges and How To Overcome Them!
There is a quiet revolution happening in a corporate boardroom in how they align their business goals with social and environmental impact. We see daily headlines such as Amazon wants 50% of its shipments to be carbon neutral by 2030. As per GIIN research, Impact Investment is growing twice - $114B in 2017, $228B in 2018, and $502B in 2019, and expected to grow to $1Trillion by 2020. GIIN 2019 Forum confirmed the same sentiment this year that days of impact washing is over, and every impact investors will have to collect and share impact results going forward.
Even traditional philanthropy is rapidly moving to venture philanthropy or impact first. I have been going to SOCAP for last five years. Earlier I used to conduct social impact measurement circle (now LinkedIn group) attended by leaders from asset owners, asset managers, assets (nonprofits, social enterprise) like Acumen, Kiva, Grameen, NextBillion, U of Michigan and many others! The common challenge at the time was
- Lack of agreement on what social impact measurement is!
- A common fallacy that social impact measurement expensive and time-consuming
Most early stage social enterprise would not even want to indulge into impact measurement as they felt that it was creating additional friction in the initial business development cycle
Fast forward SOCAP 2018. Most people didn't start a conversation with 'social impact measurement is costly'... instead, they were interested in learning how 'impact evidence can help in impact capital raising process, whether you are an asset manager or asset'. New generations of asset managers are almost always required to even define or select 'impact measurement & management solution' before even they can raise $50M or $250M for their asset management funds. While Impact Evidence is becoming important for impact capital raising and distribution, it is not straightforward. NY Times article beautifully explains Investing for Social Impact Is Complicated. Here Are 4 Ways to Simplify It.
Monitoring and Evaluation Challenges
So, let's start this with most challenging obstacles in Monitoring and Evaluation and see how we are able to solve them! Step by Step -
1. Impact Measurement & Evaluation Is Luxury
This misconception cannot be further from the truth! There is a fallacy in the social impact world that impact measurement is a luxury and necessary evil to satisfy a funder. I have met so many social entrepreneurs who throw the kitchen sink at you like this
- We know that we are already doing social good, why do we need to measure?
- We are an early stage social enterprise just barely struggling to build a business model; measurement is an unnecessary burden
- No one is paying for impact measurement
- on and on ....
Most of these arguments are so self-serving or demonstrates a lack of fundamental understanding behind impact measurement.
In our previous blog, SDG 3: Scaling Mobile Healthcare Through Evidence-Based Impact Measurement we describe how hundreds of mobile healthcare project remains at the pilot level, whereas a healthcare delivery organization in India is able to scale an entire program statewide with a careful integration of impact measurement. Here is a clue - carefully designed, iterative process with a systematic data collection and community engagement can do magic.
Read more: SDG 3: Scaling Mobile Healthcare Through Evidence-Based Impact Measurement
How Impact Measurement and Management is now easily accessible
Read More - 5 Steps to Design a Systematic Impact Evaluation Plan
2. Lack of data trust in impact ecosystem
There are billions of dollars being spent at the behest of donors, just to find that most reporting data are collected just because congress or board members require every international development project or foundation requires them to report, etc. Many of these program organization take long time and cost a lot with RCT, collecting survey data etc. While this is a necessary evil, it is hardly useful in creating trust with the donors.
Raising the bar for impact management practice, with stakeholders at the center creates better alignment between funders (asset owners, asset managers) and organization working close to the stakeholders (assets). This process requires a culture of impact & outcome alignment, the better language of impact measurement and agreement on improving data culture to improve stakeholder aligned impact. This is a single most missing reason for lack of data trust and perhaps the most colossal waste of precious resources.
system change bringing valuable data in M&E
Read More: Important M&E Tools and Techniques that are a must know
3. Missing theory of change driven data collection
Most organizations data collection is either non-existent or missing robust data strategy. If they are collecting data, often they focus on activity and output data which usually do not align and validate the primary mission and vision of the organization. This lack of alignment between the theory of change and data collection ultimately is the most significant barrier to understand social change. Without this alignment it is impossible for an organization to realize WHAT, WHO, CONTRIBUTION, HOW MUCH and RISK - a critical prerequisite of understanding and communicating social impact.]
Themes & SDG-05
Create Strategy
Read More: How can Impact Reporting be used for effective storytelling?
4. Data islands and rudimentary data aggregation
To better understand program outcome or results, it is essential that the data collection system is aligned with the theory of change based approach.
- Where do you collect activity and output data?
- Are these data complete?
- What is the core outcome that you seek to achieve?
- How do you know if you are making the right kind of change?
- How can you aggregate results and learn valuable insight in a short time?
The challenge here is that most organizations data collection and program management data system is all over the place. An overwhelming number of organizations still collect results in MS-Excel, Google Spreadsheets or custom database solution. While MS-Excel and Google Spreadsheet based solutions are easy to set up and learn, there are many limitations --
- Without robust data management, organizations create data islands over period time making it difficult to track data
- Impact framework and metrics can keep changing year after year, making it challenging to manage data
- Even within excel, after every event or field data collection, someone must merge data and re-apply formula adding lots of error and time
- Excel inherently doesn't understand the theory of change
- Excel is not the best analytics platform
Ultimately, a typical organization ends up using 4 to 6 different tools to achieve all the objectives of impact measurement, increasing significant time and resources wastage.
excel to advance Impact Evaluation tools
Read More: Your go-to guide to evaluating social innovation programs
Lean Data Measurement & Analytics
Impact Cloud provides scalable lean data measurement & analytics, by
- Theory of change & data alignment
- Scalable data warehouse
- Integrate standardize outcome & evidence library
Our ultimate goal is to make impact measurement & management simple by making MS-Excel / Google Sheet based approach obsolete and provide the most compelling solution compared to a platform such as Salesforce.
Read More: Essential Monitoring and Evaluation tools to make your life easy
5. Weak impact framework is barrier to demonstrating impact
While there are many frameworks - result framework, impact frameworks out there, depending on your organization role, chances are you will start with one of them and modify to meet internal goals. The best framework is the one that often helps align different impact ecosystem players. But where do you start today?
While new global reporting formats such as Sustainable Development Goals (SDG) and other reporting frameworks like IRIS and GRI have emerged, most organizations not have the capacity to align all their internal theory of change or measurement goals with global standards and structure.
Impact Strategy Through Impact Knowledge Graph
What if we have a platform that easy that simplifies all the taxonomy of standards and custom measurement approach? Wikipedia of Impact Knowledge Graph Structured knowledge platform for
- Impact Themes
- Impact Metrics
- Impact Survey & Data
- Outcome & Evidence
This impact knowledge platform is designed for impact practitioners by impact practitioners. With this approach, our goal is to make an impact strategy actionable for everyone and ultimately reduce any barrier of social impact measurement.
Why switch to Impact Cloud for Monitoring and Evaluation
Read More: Some Impact Investing Trends to follow in 2020.
6. Lack of stakeholder voice in measuring social impact
The primary purpose of impact measurement is to drive a better outcome for the beneficiary. Who are our stakeholders? According to principles of Social Value International, stakeholders are who affects and are affected by your activities. However, do you have a system that communicates stakeholder's user experience or satisfaction or dissatisfaction?
- Do we think reporting how many beneficiaries served makes sense?
- Does it make sense to report the number of meals to children while needs often tend to outstrip any intervention?
- The real questions that you should answer is,
- Don't talk about how many beneficiaries benefited from the solar lantern. Talk about how many of them are happy or dissatisfied with a new purchase?
- Are they expressing job satisfaction?
- Are they truly able to improve outcome with the new lending program?
Actionable Impact Scorecard
Impact Scorecard - How Evaluation of Impact Lookslike
Read More: 5 Ways: How can AI help in Actionable Impact Management?
7. Distance between donor and beneficiary is too high
The most significant barrier for funders in making a decision is a lack of visibility between funding and actual impact. Provide the best way to provide impact capital distribution through a better impact evidence.
- Reduce data distance through a better beneficiary voice & stakeholders
- AI-based data analytics and integrate with actionable impact scorecard
- Reduce a distance between funder and stakeholder by providing large scale data integration for distributed & disconnected hierarchical organizations
- Build the best impact evidence approach
Read More: Uniting the Impact Ecosystem: End-to-End Impact Management
Impact Manager and its use in monitoring social impact
8. Custom vs Configure: Customization is expensive, non-flexible, time consuming, and complex.
Unlike corporate counterparts, the impact ecosystem organization needs much more scalable and comprehensive solutions. All the above barriers sums up to this question. Only if we have a flexible, comprehensive, configurable system we can have Theory of Change led impact measurement with less resources and have all the stakeholders at the same table.
Impact Cloud is one the most innovative platform that has the potential to make Excel & Customizable Application Platforms (like Salesforce) obsolete for Impact Measurement.
Impact practitioners have demanded such platform and helped us build one to reduce any barrier of social impact measurement & management. Together we are on a mission to achieve the goal of making impact measurement and management simple for everyone. Interested in learning more?
Read More:Simplify Impact Management with Impact Knowledge Graph
DOWNLOAD MONITORING AND EVALUATION TOOLS SELECTION GUIDE.
M&E-1
WRITTEN BY Unmesh Sheth
October, 7 2019
| The text being discussed is available at
and
| |
