(Image from http://sewing.about.com/od/beginner1/ig/Sewing-Tools/Tape-Measures.htm)
So what happens when you measure the wrong thing? This is one of those issues that tends to either stop discussions in their tracks or lead us all down the path of rhetorical perfection that is in fact little more than the real-world version of a filibuster: "hey, I use a slightly different version of jargon than you do, so rather than actually hash out our differences and get on to making some decisions, we'll just keep the discussion going ad infinitum." This has happened in defining social enterprise, discussing the potential of metrics of social return, and on and on.
So, before I go one step further, let me say this: I believe that we can measure social and environmental impact and that we should. In fact, I'm on record (here and at least a zillion other places) opining that the problem is not that we can't measure these things, but that we have so many ways to measure them that we don't know which measures matter.
We will get measures wrong. Recently, two cases came to mind that show this. One has to do with an article in the Washington Post claiming that Facebook Causes had driven very little money to charity and was therefore a failure. The counter-argument - well worth consideration is that money is, at best, one metric - Causes and other similar tools are best for creating connections and awareness. You can follow this all here.
Here's the second case. Twitter lets a user easily re-tweet (read: "forward") really easily. So some have argued that the number of times you are re-tweeted (your comments passed on to others' networks) is a measure of your influence. Others say that the real measure is how many followers you have. And then others jump in and point out that both re-tweeting and automatic-following is so easy and so common (and so spam-automatable) that these measures don't matter. Here's one argument that the measure that matters is how often something that I tweet is marked as a "favorite" by others.
I don't know what the best measures are of Twitter, Causes, or most other social media tools. This isn't really my area - I usually wait for Beth or Allison or Katya or others to figure these things out and then decide if what they say aligns with my experience and makes sense to me. If not, I keep thinking about, asking about it, reading about and going to places like NTEN to learn more (If you couldn't get tickets to the sold-out conference #09NTC you can follow it here)
But the debate about these measures does matter. These discussions are out in the open, the sides are (mostly) cogently argued, and folks can contribute to the discussion, learn something, and move on. This is what matters. This is what we need to make sense of the hundreds of ways of measuring social and environmental progress. This is what I hope the Alliance for Effective Social Investing, the NPC "Valuing Impact" meeting, the GSIX, and the OECD -led Global Project on Measuring the Progress of Societies will move us toward. We may not reach universal agreement on any of these things* but let's get the debate, the evidence, the arguments for and against, and the determinations of what works for what out there. And keep moving forward.
*I think it is better if we don't reach universal agreement. There will not be one-size fits all to these issues. We can use lots of metrics, indices, and methodologies. AND we also need to know which ones are really methodologically sound, which ones are comparable to others, and which ones are being used by someone reporting a certain accomplishment.
Tags:
2 comments:
Yes, exactly. Do we recognize that the act of defining metrics causes shifts in behavior and gaming of the system - that the metrics themselves become a currency? Defining metrics is not only about trying to measure what matters but it is also about incenting what we want more of.
For example, I hardly favorite anything on twitter. If you say that is the metric that matters, I will start "favoriting" all the things I want to see uplifted in the attention sphere.
(see http://openmoney.info/sophia/index.html for explanation of levels of wealth)
And if we think of metrics as currency at the measurable wealth level, then can we imagine translators/currency exchangers in some cases?
Lucy - loved the brilliant filibuster/ad infinitum insight into defining social enterprise and potential metrics. I read that sentence, and thought, we need a dictionary organization for the social sector like Black's Law Dictionary for the legal sector that defines what everything is so we can actually make progress (instead of acting like ships passing in the night).
I also wanted to raise a question on one of your assumptions, which is this need for metrics, especially in the context of the mission investing space. I had a conversation with a fund manager of a mission investing financial services firm and asked him whether metrics will be all that important for mission investments. He doesn't think so and I probably agree. Imagine if you had an idea like Google or some other worldchanging idea that could generate market rates of return or better - would you really spend the time developing social impact metrics? And perhaps more importantly, how many mission investors will be significantly influenced based on social return metrics, as opposed to financial metrics and the theory of change itself? I think it's important for "social entrepreneurs" to think about their social impact, but quantifying their social impact may be an unnecessary step.
That's not to say social impact metrics don't matter - they probably matter more at the nonprofit level than at the for-profit level (I think it's telling that SVT, one of the leaders in social impact assessment (SIA), has a client base that's primarily nonprofit). Anywho, these are just some random thoughts I have this morning.
Post a Comment