Of course, neither data nor data visualization are silver bullets. The data have to be good, clean, comparable, reliable, and open. The visualizations have to be accurate, credible, and contextual.
Even then, you have to be willing to use these tools and then change the way you work. For example, here's the latest whiz bang online tool - real time monitoring of water wells built by the group Water for The People. We can watch this cool app on our (android) phones to see which wells work and which ones don't. But what really matters is if Water for the People and its partners in Rwanda use the technology to keep the wells working.
See this wonderful piece by Clay Johnston on bad data visualization (he calls it "the lies visualizations tell.") Just because you have data doesn't mean you know anything. Just because you have a map doesn't mean you're an explorer. And just because you have data and cool visualization tools doesn't mean you'll make better decisions or be a more effective grantmaker.
For donors and foundation program officers, data and data visualizations have to meet an even higher threshold - they are going to have to make it easier to know something than to not know it. Right now, most of the structural incentives in philanthropy make it OK to not know what everyone else in your field is doing. Plus, it's hard to find out. Those two factors - making it necessary to know and making it easy to know - are big barriers to adopting these tools or changing the way foundations make decisions. Technology can make it easier to access data and easier to show the data in new ways. But unless the incentives for knowing and doing the work change, we'll have a lot of very cool tools that few will use.
A very nice new Strategy Landscape tool from The Center for Effective Philanthropy and The Monitor Institute proves this point. There's no magic in the pictures, no secret key in the data. What matters is the sense making - the collective effort to understand an issue in a new way, to know what your peers are doing, and to make funding decisions with that information at hand. The interactive demo lets you play with this very impressive visualization of data. You can "see" by foundation, by geography, by date and by strategy. Who funds energy efficiency in the Midwest? How much of the foundation's funding goes to that strategy? How do they define the strategy? These questions can all be answered by clicking and looking. No magic here - just good clean graphics and data that have been coded and categorized.
Let's talk about that last little bit. Cleaning and coding and categorizing data are tough things to do. They take time, money, expertise, and storage space. They require "definitional conversations" - what do you mean by "energy efficiency?" Doing this work to develop the Strategy Landscape tools means that Monitor consultants and staff from CEP will have to engage clusters of funders and help them through these conversations. This is not rocket science, but like all collaboration, it's not a walk in the park either. Some good comes from the process. People realize there are better terms for things and there is a way to avoid jargon. They find peers and potential collaborators. They may find an ally in making an argument for a certain approach that they've been failing to make before. That's all good. The Strategy Landscape tools will help with this.
Somewhat similar is a Social e-Valuator SROI tool being launched by the SVT group and an enterprise in the Netherlands known as Social Evaluator. The Social Evaluator Tool is a guided, online Theory-of-Change maker that yields common indicators and outcome measures and allows for the monetization of SROI. (Demo Video) Like the Strategy Landscape Tool it involves a cool website and some careful data coding and management. But mostly it involves thinking hard about what you're trying to achieve, identifying outcomes and indicators, and working together - as funders and grantees - to name, track, record, and report those things.
Both these tools show that 1) we have the data and 2) we have the visualizations. The hard part is changing how we work to share our data, think collectively, and put in the time and effort to hold ourselves accountable to outcomes.
There is another tricky part here. Public access to the data. The data that power the Strategy Landscape tools are foundation grants. These are publicly reported data. Anyone could download a lot of 990s and answer the questions that the Strategy Landscape tool answers for themselves if they the time and inclination. The tool makes it easy. But it needs to make it visible to those whose data is not yet included. It's a good first step for the foundations that provide the data to have access to the visualization, but what really matters is 1) letting others see it so they make strategic decisions from it and make their own visualizations, 2) let other data be added to it and 3) make the data storage solution accretive, so years from now you only need to fold in new data not do the whole thing all over again.
For the SROI tool, the data are different. They're subjective inputs from the funders and grantees involved. But these too should be public so that others can use them, improve and iterate on them. After all, if every group of funders needs to come up with its own proxy indicators for youth development outcomes what's the point? We need to get to a shared set of public measures, like the IRIS Standards.
Unfortunately, the default position on the data in both the Strategy Landscape tool and the SROI Tool is "not public." Those involved say they want the data to become transparent and open, but they can't start there. I think if they don't start there they won't get there.
If these tools are going to get used, get traction, get better, and be useful the processes that they inform need to be open. The data need to be accessible, machine readable, and mixable. The data are public and the tools are commodities. It's the knowledge and decision making from the data and the tools that really matter - and in order to fuel real change these need to be open. Otherwise, it remains too easy to not know, to not ask the hard questions, and not to hold ourselves accountable.
Tags:
10 comments:
Lucy,
As usual I agree with most of your points, especially all the work that goes into getting data, defining categories, taxonomies, cleaning the information and designing effective visual interfaces. The point about all of this being publicly available through 990-PFs, however, is misleading. Have you looked at a 990-PF lately? More often than not, the name of the grantee is incomplete and may not have an address. There may well be no grant description whatsoever and no indication of the duration. You can't build a tool like that of Monitor based on a grantee name and and an amount. As you say that takes agreement among foundations, working with consultants on definitions, terms, and taxonomies AND the willingness to provide this kind of agreed-upon data on a consistent basis over time AND someone to clean and maintain it. For anyone who thinks that this kind of thing can be whipped up from publicly available 990s, here is a widget that gives you free access to hundreds of thousands of them http://foundationcenter.org/widgets.html Have at it!
Brad
Brad - of course, I exaggerate. And if all the Foundations submitted their forms electronically (and completely) and the IRS tracked these data in the 21st Century definition of public (open, machine readable, downloadable) then the technological challenges would diminish but the quality of the data problems would still remain.
My point, which may have gotten lost in my overenthusiastic presentation, is that the data are a beginning (a platform). They are public information. The visualizations are tools. What matters is the sense-making and the incentives we work by that would actively encourage knowing over not knowing. Because we now have the tools to do so.
Thanks
Lucy
Lucy,
Your IF is a very big one. The quality of data provided by foundations varies widely. For some examples of what good, open grants data looks like, try this:
http://maps.foundationcenter.org/rss/grantsfire/
or this:
http://feeds.feedburner.com/FoundationsRespondToTheEconomicCrisis
The first is live feed of foundations publishing their grants via Grantsfire and the second is a compilation of nearly $450 million in grants made by foundations in response to the Economic Crisis.
Brad
A girl can dream, can't she? And useful open public data will, one day, be the norm - we'll kick, scram and dig in our heels. We'll file incomplete forms and cleaning and completing the data will always be needed, but we're headed in that direction
Lucy
Lucy,
I think your main point -- that we have to be able to make sense of the data, share it in accessible ways, and use that to drive outcomes -- is very important. And I think we often understate what "accessible" means there -- it's not just about the APIs or the tools that munch the data -- it's about the ability to have the resources to understand and do something with it. A great post from Mike Gurstein on the data divide. In it are some good examples of how people actually made sense (and change) from the data and all the skill sets that were required: http://gurstein.wordpress.com/2010/09/02/open-data-empowering-the-empowered-or-effective-data-use-for-everyone/
Hi Lucy -
Yes, we can all dream. It always comes down to the pesky "when" question. To that point, I was struck by your saying:
"... most of the structural incentives in philanthropy make it OK to not know what everyone else in your field is doing."
And in one of your comments to Brad you refer to "... the incentives we work by that would actively encourage knowing over not knowing."
Are you able to elaborate on the incentives/disincentives you refer to?
- Rick
To add another perspective on your question above Rick, we've found that one of the biggest disincentives that prevents funders from knowing what others are doing is the simple logistical difficulty of finding out the information. If you want to know what others are supporting, it takes work -- phone calls, meetings, and research -- and you still end up only kinda knowing what others are up to. So it’s easier to put on your blinders and do what you want to do, and you keep your fingers crossed that what you do maybe works well with what others are doing, or at least doesn’t duplicate their efforts or work at cross-purposes. That’s how funders too often operate, because, simply put, it’s the easiest way to go. And there’s nothing that forces funders to do anything different. The Strategy Landscape aims to at least help funders overcome that most simple of hurdles, making it just as easy to know as to not know.
That said, if you’re interested in a longer discussion of the structural barriers to change in philanthropy, I’d suggest taking a look at our recent report, What’s Next in Philanthropy, (www.monitorinstitute.com/whatsnext), where we have a section early in the piece (on pages 4-5) about precisely that subject…
Marnie
Thanks for that link - critically important post and set of issues to take into account. It's why just having data, just having visualizations is not enough. We - ALL of us - need to know how to read them, use them, generate them - it's a new literacy. And a critical one. And one that, as the link says, won't be equitably distributed by magic.
Lucy
Rick
Good question - I think Gabriel covers some of this in his comment but lets push it a little further. Program officers' jobs' are rarely designed for maximum community impact. They have pay out requirements to meet, not effectiveness or impact requirements. Their job descriptions - and those of VPs and executives at most foundations - rarely provide incentives for or even requirements/expectations that their work will be judged against some sense of successful strategy design and implementation.
In any given issue, with any particular sense of outcomes, going it alone, working together, partnering, or leveraging others funds might be maximizing choices - and these tools can help you know that. But if knowing that doesn't matter in how your job is set up, and lots of others things - meetings, reporting, payout, internal knowledge, etc - are required and structured into the job, then this other stuff is icing.
Now, there are foundations that do built results into the job descriptions, incentives, and tenure of their program staff. Not many (that I know of) but some. And there are foundations where the private foundation chief executives are held accountable to how much other money they influence. That is an example of a structural incentive that fosters "knowing" over "not knowing."
Donors have freedom to do anything - look for gaps, complement or leverage funding, or ignore everything else going on and follow their own hunches or interests. There is nothing in this structure pushing donors to work together. Some have created new ways to do so - giving circles, pooled funds, etc. and these could be great users of the kinds of tools CEP/Monitor have launched.
Thanks for the question -
Lucy
Hi Lucy,
Great piece. I wanted to brief you on a few developments in the works on the "making it public" part of helping the field get the value from the SROI tools.
1. The SROI Network headquartered in London is in the process of building a database of indicators used to measure impact and their sources. This is slated to be available sometime in 2011. http://www.thesroinetwork.org
2. SROI Canada has a (creative commons license) database of financial proxies (indicators with monetary values) that a governmental entity might find useful. You can download the latest list of metrics here: http://tinyurl.com/2dqc7b6
3. IRIS and the SROI Network are beginning to coordinate starting with drafting a statement about how the two approaches/systems relate to and complement one another. That will be up on each parties' website and on SVT's site shortly!
Warm regards,
Sara
Post a Comment