A study from 2013 cited that 88% of funders think that impact measurement makes charities more effective – and 89% of them think it makes funders more effective too. While the figure is a bit dated, with the growth of results-based financing, it is fair to expect that there is an even heavier emphasis on impact measurement among funders than before.
Based on our experience working with funders, here are the common pitfalls we see and some suggestions on how to overcome them.
1. One size fits all approach VS too flexible and can’t draw conclusions
Problem: Funders may be drawn to a common approach for their grantees so that they can articulate a coherent narrative for their Trustees and supporters, but of course we know that this often does not work as contexts are different, and it is very difficult to create standard outcomes. So, failing that, some funders may want to adopt a flexible approach and ask for reports based on what grantees think are their outcomes. While this comes from place of trust, the downside is that this approach makes it hard to draw conclusions on a portfolio level to inform grants strategies.
Solution: The middle way is to take a “basket of indicators approach” on a programme and portfolio level. This means defining common learning questions, allowing grantees to pick from suggested indicators and adding additional ones as they see fit.
2. Trust based relationship (learning) VS compliance needs (monitoring)
Problem: Good impact measurement should include monitoring, evaluation and learning (MEL). But from our work with funders, we understand that there could be tensions between learning, which is rooted in a trust-based paradigm, and monitoring, which is based on a command-and-control paradigm. Funders seem to find the tensions hard to reconcile.
Solution: Recognise the importance of both learning and monitoring. However, we see the best results coming from funders that strip down on the monitoring side of things and focus on facilitating learning with their grantees, as well as among grantees.
3. Asking for too much
Problem: we (as evaluators) as well as our clients have the tendency to want to know everything about the programmes being funded. Most people are aware that working on impact measurement takes time on the side of the grantees, but we may not be fully mindful of the burden it really places on them and beneficiaries. Because of power imbalances, grantees often need to comply with impact measurement requests, creating fatigue or even resentment towards the idea.
Solution: Recognise that accountability is as much upward (to funders) as downward (to beneficiaries). Don’t ask for anything you won’t do anything about. Ideally, what you want to know is also aligned with what the grantees need in order to improve – taking a co-production approach in defining outcomes and learning questions can therefore overcome this challenge.
4. Too technocratic and fall in the trap of jargon of “M&E”
Problem: Evaluators may make life slightly harder than it needs to be. There is a lot of jargon in the “M&E” industry, from attribution/contribution to Most Significant Change (MSC) to Social Return on Investment (SROI). Funders can find themselves in the jargon jungle when they approach impact measurement, and then this gets passed onto the grantees, who become overwhelmed.
Solution: There are quick wins and easy steps in impact measurement, starting with Theory of Change. Provide support for your grantees to ease their way into impact measurement, such as paying for training. Start by meeting your grantees, especially frontline staff, where they are at and empower them with language that makes sense to them.
5. Not facilitating space for grantees to fail and come with bad news
Problem: Social change isn’t always positive, as it’s inherently complex. Many research studies have shown that some social programmes, which produce nice stories and testimonials, actually cause more harm than good. (We recommend the Freakonomics podcast on this issue)
Solution: Funders need to learn to be okay with bad news. When you only get good news from your grantees, you should be asking yourself some questions. We need a wider shift around culture, but funders can start with framing expectations around what good looks like from the beginning. Funders need to lead the way on how to recognise their own failure and set the example.
6. Not willing to share with others external to the organisation about the learnings (or not so glossy parts that come out of impact measurement)
Problem: Even the best funders who are doing well on impact measurement seem hesitant to share their learnings externally. We have done many projects for funders and charities, hoping to share externally the real learnings (some of the bad news) that can inform social change, but are not able to do so because of fear of criticisms (such as from media on charitable funds not being spent appropriately).
Solution: If sharing externally to the public is daunting, funders should at least share with each other to begin with. We need to remember that social change is incredibly complex and if we can all just be a bit more honest with each other, we can solve the problems together faster.