The data ecosystem

Open up the data!

I feel like I should tattoo this on my forehead. This is what I think about, write about, speak about, and blog about. My forecast for next year (and the years beyond) includes a whole section on big and open data as a resource for social good.

But I don't think data are going to change individuals' behavior directly. Most of use a lot more than data when we make decisions. Daniel Kahneman, a psychologist, won the Nobel prize in Economics for proving that we're not as rational (read: data driven) as we think we are. We factor in lots of other interests and issues besides data when we make choices.

This is especially true in philanthropy. We give for reasons of the heart, personal connections, feeling good, looking good, and doing good. Efforts to shift our giving toward more rational, data driven, informed practices know this - they aim to shift the margins (which are quite big) in order to eventually, perhaps shift some of the middle.

So why am I so focused on opening up data when individuals may not use them? Because data are the most basic organic matter for the ecosystem of social good. They provide fodder for how we identify "the problem," which then plays a huge role in "the solutions" that we build.

I believe that opening up philanthropic data will help enough innovators to think differently about how we change the world. Their efforts will yield new opportunities for the existing system and for all of us.

Stop for a second and think about how data now subtly guide or inform your choices in all sorts of realms - air fares, music, book and movie recommendations, jobs, directions, bank rates, cupcake shops, hairdressers - we all have the option for using more data than ever before when we make these daily decisions. And it's not just the easy stuff, like price. It's the tough stuff, like opinions and reviews, that are now available anywhere, everywhere on seemingly everything. When Angie's list started advertising that you could compare plumbers and doctors I knew we'd turned a corner.

I didn't personally seek out these data. Entrepreneurs saw the value of data as raw materials from which they could put new tools into my hands. Those tools help me do the things I like to do faster, easier, and with better results. And when enough of us start expecting this information to be available it spills over onto how the whole system works.

In philanthropy, we're just moving beyond the most basic information. Basic data on operational overhead is widely available and is beginning to power some new tools that let you contrast nonprofits. But that approach still assumes that you and I care about that administrative ratio comparisons first and foremost when we make a gift. Some of us do, but most of us don't.

This approach also sees data as an end-point in the decision making process, not as an input to thinking about solutions.

What we need is an approach to data that goes beyond the basic quantitative comparisons and gets to the level of how we solve problems. If we shared information on where private and public money flows in a community, or where needy people spend their days, or how much food gets wasted and where, or we could hear directly from our elderly neighbros, or help artists connect with each other we might imagine whole new approaches to our shared problems.

What if we could match something like RelayRides (neighborhood car swaps) with TaskRabbit (small job doers) with volunteers for the elderly so that we could help our neighbors keep their doctor appointments and avoid the ER? Or use data on car sharing services to reroute busses so they serve the areas that really need them? Or use the Twitter patterns of food trucks to help identify cohorts of professionals who might be willing to volunteer? Used the opt-in text messages of young people to engage them in community or public service?

What can we learn from giving patterns? Of individuals, corporations and foundations? We don't really know because we don't really have these data in the form that would allow us to know. We also don't have the ability to mix giving data with shopping data, political giving, voting patterns, faith traditions, or other potentially useful information. Charitable giving is an enormous part of our communities, yet we haven't cracked a way to use the aggregate information on money flow, causes, organizations, or donors so we can see this pervasive activity in any kind of meaningful context.

I see philanthropic data as a "nutrient" in a healthy, diverse ecosystem of social solutions. The open government movement has accomplished some of this with public data - and we have better 311 systems, better public transit, and quicker response times for public works departments as just some of the results.

Early, proprietary efforts at mashing up philanthropic data are being used to develop strategy maps and plot grants by location. These are great first steps.

When we get to the point when you can track funding from all investors (impact investors, philanthropy, government) or monitor particular organizations or enterprises that interest you, then we can influence the flow of capital.

When we can see entire  ecosystems of organizations and funding by issue, geography and population, we can engage communities, guide public policy, and fund accordingly.

And when we can map and mine patterns of success and supply, we will inspire the next era of change makers to expand what works and build what's missing.

Open data are the fuel to make all of this happen.