The growing interest in impact measurement has raised concern over the lack of shared, reliable principles of how to assess the outcomes related to social impact. S&O Center’s researchers in fact testify to an absence of consensus and a frequency of measurement challenges which both jeopardize support for measuring social impact and valuing it positively within an organization’s total performance.
Consequently, Rodolphe Durand, Zachariah Rodgers and Sookyoung Lee, researchers at HEC Paris, have implemented a social impact measurement tool that standardizes the economic and social evaluation of corporate responsibility initiatives described in the Social Impact Assessment Strategy Report. In this report, in addition to their overview of impact measurement – its approaches and major evaluators, the researchers share with us the main practical, technical and strategic challenges of impact assessment and deliver recommendations.
THE SEVEN TECHNICAL CHALLENGES OF IMPACT ASSESSMENT
These measurement challenges, prevalent and damaging to impact efforts, are related to the evaluation of impact, to mistaken beliefs about the impact efforts themselves or to both of them.
Confusion: failing to distinguish between input, output, outcome, and impact. If you do not distinguish between different parts of the impact value chain for your social impact effort, it may be very difficult for you to develop a good understanding of the current status of your social impact or how to improve it.
Inconsistency: unreliability of measurement. When comparing numbers across time, projects, or locations, low measurement reliability means that equivalent amounts are not actually equivalent amounts without adjustment. A means of increasing the reliability of social impact measurements is to use field standards shared by more than one geography or organization and to test the reliability of measurements.
Misunderstanding: causal validity errors. Causal validity errors proceed from as well as lead to flawed information. With flawed information, for instance, you may over-invest in social impact efforts that are not effective, or you may fail to grasp the actual social impact value that you are indeed creating. Stronger grasp of causal validity for what is causing social ills and how to solve them will help your social impact efforts to be more likely to succeed, and also to generate business value.
Blindness: hidden factor correlation. Hidden factor correlation, also known as “spurious” correlation refers to two factors appearing at the same time, and therefore mistaken for causing each other, when in reality they are caused by other, unobserved factors. Hidden factor correlation is a specific type of causal invalidity: decision makers are blind to the unobserved factor and conclude wrongly about the impact of their actions.
Over-simplification: ignoring multi-determination. Ignoring multi-determination occurs when the multiple or most important causes and effects in a cause-effect relationship are ignored. It will lead you to strategic short-sightedness, where you only attend to what is immediately in front of you. In the first place, organizations will rarely attend to factors that are not measured. It can lead to the neglect of multiple solutions, measuring multiple types of outcomes, or marshalling multiple types of resources—especially including those in the hands of other organizations or stakeholders.
Partiality: failing to capture both downside and upside risks. Although appearing to mathematically cancel each other out, positive social impacts and negative social impacts are not treated as conceptually equivalent, so they often deserve to be examined separately. Loss aversion—that losses are treated as more painful than equivalent gains are pleasurable—is one example of a bias that would alter the way downside risks are treated as compared to upside risks. By ignoring one or the other, an incomplete picture leads to blindness about weaknesses and avoids fixing important ones. Or, it leads to blindness about potential opportunity to contribute to social impact.
Over-assuming: lack of generalizability. The challenge of over-assuming occurs when one instance is taken for being applicable across the board to many other situations whereas this instance is specific i.e. not generalizable. If you fail to appreciate and properly fine-tune for contextual differences, then impact evaluation efforts could lead to gathering of data and results that cannot be properly compared to data gathered in other contexts. Then, it will be difficult to make informed decisions about where resources are best invested on impact efforts
Besides the practical and measurement challenges, the S&O Center introduces a different sort of challenge associated with impact measurement : the strategic challenges of impact measurement. It consists of determining whether or not to communicate about it: too little communication and some opportunities are missed; too much, and the risks associated with hypocrisy or social-washing loom large. These strategic challenges are essential because they precede the taking of measures to address them proactively to bolster social impact initiatives and business benefits related to them.
FINALLY, THE RESEARCHERS DEVELOP A 4-STEP METHOD TO IMPROVE SOCIAL IMPACT ASSESSMENT.
1st STAGE. Setting/revising the goal(s) of social initiatives
Recommendation 1 – Check assumptions : Rather than presuming that social challenges and initiatives will perform as anticipated, follow-up to check the assumptions that your impact efforts rely upon.
Recommendation 2 – Do not focus on too narrow objectives: Rather than get focused on narrow initiatives, aim at addressing consensual objectives among your stakeholders.
2nd STAGE. Planning the assessment
Recommendation 3 – Invest in assessment : Rather than write off social impact measurement as too difficult to tackle, remember that financial measurement is difficult too, and requires rigorous training, extensive protocols, and dedicated personnel to do well.
Recommendation 4 – Compare contexts before measuring impact : Rather than rolling out measures too widely at once, ensure that your measures are reliable across different measurement-takers and across different contexts.
3rd STAGE. Improving the assessment
Recommendation 5 – Constantly improve your impact assessment : Rather than leave the many potential benefits of upgrading your impact assessment approach on the table, take impact measurement seriously. Seek and keep continuous impact assessment improvement.
Recommendation 6 – Map and engage the impact value chain : Rather than overlook the distinctions between different parts of the impact value chain, actively map and engage the impact value chain for troubleshooting problems as well as building up strategic vision.
Recommendation 7 – Do not over-rely on quantitative metrics : Rather than becoming too data-driven too soon by relying heavily on limited quantitative data, make room for collecting qualitative feedbacks from your teams and beneficiaries.
4th STAGE. Strategic use of social initiatives
Recommendation 8 – Do not only focus on benefits or risks : Rather than focusing only on positive or negative aspects that your initiative addresses, pay attention both to benefits and risks.
Recommendation 9 – Search for and extend the benefits : Rather than only assuming that some business benefit is probably occurring, actively measure the business benefits from impact efforts. In measuring your business benefit, be flexible about the payoff horizon. Business benefit takes time to accumulate.
Recommendation 10 – Leverage goodwill for increased business and social impact : Rather than keep secret or hide your impact initiatives, make sure that the right audiences know about your contributions.
To know more about it: https://www.hec.edu/fr/faculte-et-recherche/centres/centre-society-organizations-so/reflechir/social-impact-assessment
L'Observatoire du BoP 