Influence of evaluation and evaluating influence

Just a few days ago I noted the engaging quality of an interactive evaluation report put out by the Knight Foundation. I called it "the most fun evaluation report you've ever read." I wasn't being snide, I meant it (although the bar is admittedly low).

Well, the bar just got a little higher. Today, the front page of that Arts section in The New York Times included a story about an evaluation report. It's a study of the costs of raising millions to build signature buildings. Costs so high that they can kill the entire organization.

The website for the study (from the University of Chicago, that well known comedy shop) features a two-minute animated video summarizing the findings. The website also provides easy access to a "bookshelf" of research and commentary on similar subjects, particularly much of the good work done by Clara Miller and the Nonprofit Finance Fund over the years. It's nice to see the findings being released in a variety of useful ways now, before the 2013 publication of two full books on the research. The research was supported by the The Andrew W. Mellon Foundation, The Kresge Foundation, the John D. and Catherine T. MacArthur Foundation, and the Rockefeller Brothers Fund.

The report's findings aren't terribly surprising. Big buildings cost a lot of money. Especially after they're built. Institutions should think twice, and then twice again, when forecasting the hoped-for revenue and the actual costs of that new modern art wing or performing arts center. The data are comprehensive, the examples diverse, the interviews compelling, and the presentation engaging. The case studies are all examples of philanthropic and public partnerships and feature both big city and smaller community ambitions. For an evaluation study like this to be featured in the "newspaper of record" is impressive and unusual. Let's hope it's also influential.

**************

Speaking of influence, how do you measure the impact of a documentary film? That's the question being asked at The Harmony Institute, a nonprofit research center that measures the influence of entertainment. They've done studies of Waiting For Superman and work on the intersection of telecommunications policy and communities (download summary here).

They're introducing a new metric, the HI Score, that aims to capture the influence of films toward positive social change. You can read all about it and see the comparative scores for 14 documentaries nominated films on their blog, The Ripple Effect. What do I like most about this? They've made their data, their algorithm, and the code to run the formulas available on the website. For free download. And tinkering. And feedback. And improvement. Please. Go. Check it out.

 **************

And while you're at it check out Sparkwise, a free open source dashboard for measuring media and reach. Fast Company talks about them here. 

One minute, please

Big Data. It's gotta be a buzzword, doncha think?

I'll get to that, but in the meantime I wanted to share this visual, created by DOMO, of how much data gets created, sent, moved every minute of the day. A friend emailed it to me - the original came from visualnews. Put all these 1s and 0s together, throw an algorithm or one thousand at them, and, yep, we're talking biiiiiig data.

FYI, at the #recodegood charrette last week one of the participants noted, "What we don't know about big data - what it means to have all this information stored, possibly forever, aggregatable and separate from us as individuals - is at least as big as the data themselves."

More soon.

Opening 990 Data

Carl Malamud, who would win an "open data genius award" from me*, is proposing to move the boulders that stand in the way of open, machine readable, free access 990 data. Check out the proposal to the Knight News Challenge (category "data"). Look at the price tag - $600K - amazing to think what could be done for such little money.

This is a big deal. There is an important infrastructure that has been built up around 990 data (Guidestar and several of their partner platforms - Great Nonprofits, GiveWell, etc) That infrastructure is important. So is the infrastructure built around foundation grants data (Foundation Center). These organizations are critical  - opening the data once is great, but maintaining and keeping these data open and clean and accessible over time is what will make them a real resource for change.

Malamud has a track record in opening data and then changing the ways systems work around them. He's responsible for getting the SEC to provide corporate filings in this way. Long story short - way back in the early 1990s government wouldn't do it, Malamud opened the data, gave it back to the government, and said now you keep it going. New firms, new website accessibility, new products have been built from the data to keep them useful. This is the "edge" of change that opening the 990 data will bring us to - public data made available in a free and useful form incites existing firms to adapt and provide higher value services on top of those data. New enterprises, products, apps, licensing partners also emerge.

Opening up 990 data is not an end goal, it is a beginning. It is a beginning that will make possible much of what I call data philanthropy and many of the ideas listed here. It will require adaptation and innovation by existing data purveyors and open up opportunities for new ones. It will change the way long term data access and data sets are maintained (but it doesn't remove the need for those activities).

Check out the proposal. Think about what you could do differently if these data were readily available, mixable, portable, presentable. And let Malamud, the Knight Foundation and me know what you think might happen. Whether or not the proposal to the News Challenge is successful this is something we should make happen.

On a related note - here's a very cool new website of mapping data possibilities that Micah Sifry found and shared on Twitter this morning - Harvard's WorldMap Collection.

Now - imagine the future with data on nonprofits ready to be displayed graphically using these free mapping tools. OK, never mind, imagine that as the present. Because that's where we're about to be.


*Listen to this great webinar from Tim O'Reilly about Malamud and Government 2.0.

Digital Public Goods (?)

On Thursday we held the fourth in our series of #ReCoding Good charrettes at Stanford. The topic - digital public goods. The question - do such things exist and, if so, how do they matter in the new social economy.

Here are links to the resources for the charrette and for the blog post from SSIR that sets up what we thought going into the conversation.

"As we try to theorize digital public goods, we need to distinguish among three possible conceptions:

1. The digital platform. Digital public goods (the architecture of the Internet and ubiquitous access to it) reside in the digital domain itself—in net neutrality, access, and open source code.
2. Availability of data online. Is the public good held in the data that are provided and distributed data online or via mobile technology (as part of the open data movement, for example).
3. Digital social capital. Digital public goods reside not in some output, but in the social connections and engagement that the web makes possible. For example, the freedom of association and the liberty each of us possesses in a democracy to associate with whom we wish, without government scrutiny or regulation, takes form digitally in many respects on Facebook and is given political salience via platforms like Change.org.

Each of these domains currently operates within a different set of policy spheres. Questions of Internet access fall under the purview of national telecommunications policies and multinational cooperative bodies such as ICANN. The data are often governed by intellectual property law, but may also be used for health care or educational regulations, or viewed as corporate assets. The third conception—social capital—has been seen as the realm of nonprofits and philanthropy, but is increasingly intermediated by commercial businesses or informal networks without institutional form, and without anything like the privacy and associational protections that go along with the First Amendment.

A growing body of research and writing on the nature of the Internet, networks, and data can inform our thinking. From the perspective of the ReCoding Good project, we focus on three sets of questions. The first set has to do with how our current systems work given the nature of these goods. For example, do we need to protect or incentivize digital public goods? If digital public goods are fundamentally different from analog public goods, do we need new enterprise models, funding, and/or legal code to protect and catalyze them? Do current models of private sector, public sector, nonprofit sector funding, creation, and distribution work in the digital economy, or are creating new ones?
A second set of questions looks at whether or not digital life changes how we associate with one another. Take, for example, networks of volunteer activists using public datasets to make smart phone apps. Do these “hacker communities,” which tend to thrive locally and share their products openly and globally, constitute a new form of civic engagement? Is there a new associational form emerging from these simultaneously local and global, organized but non-institutional, private (time) and public (data) groups? Does the digital domain change the nature of associational life? Does this have implications for the structure and role of nonprofits and philanthropy? Does it have implications for the future shape and/or definition of civil society?

A third set of questions turns the relationship between digital action and governance inside out, and asks whether the digital age presents an opportunity for new forms of governance or for improved forms of governance through enhanced transparency and accountability. What new opportunities do digital platforms raise for how we govern ourselves? Can digital platforms and online interactions change the incentive structure for how rulers are responsive to citizens, how accountability works between state and citizens, and, perhaps, how the nation-state itself might be transformed."

By the time Thursday was over we were well on our way to reconsidering the notion of digital public goods. Here's some of what happened - captured in photos and some quick reflections on Tumblr.

Stay tuned for the follow up post on SSIR where we'll look more closely at what we learned and why our thinking changed so dramatically due to the charrette. All in all a GREAT day's work (Thanks to the Stanford d.School for their help in making it happen)

The most fun evaluation report you've ever seen

The Knight Foundation does #DataVis again - check out this "report" on their grant to Macon, Georgia for a social game focused on community building.

Impact investing, philanthropy and the new social economy

In May the #ReCoding Good project looked at philanthropy, impact investing and the new social economy. Here's the recap on the charrette from the ongoing "From the Field" series in the Stanford Social Innovation Review.

This month we're looking at Digital Public Goods - here are some of the resources for our discussion. I'm grateful to the folks at Stanford's design school who are helping organize this upcoming charrette.

Where good happens

"What's up this weekend?" I asked, reaching around my friend to the sugar packets while trying to stay out of the way of the long line of Friday-excited coffee purchasers behind me.
"Volunteering. Hacking." She said.
"Where?"
"DMV." She bent to pick up her backpack and her words got lost in the crowd of knees and briefcases.
"What did you say?" I asked, "It sounded like DMV. You're volunteering at the DMV?"
"Yep," she said. "Some cool data projects that will inform the city's bike planning down the line."
We walked out of the cafe. "Volunteering for the DMV." I said, "That's weird."
"Yeah, maybe. But only if you think about it." She said, "It will help with the bike lanes, move traffic along faster, and, I don't know, maybe eventually we'll get Sunday Streets every Sunday."
"You're a dreamer," I smiled.
"Yeah, I know."

The scenario above has been fabricated to suit my narrative needs. But only to the degree that I put myself in the picture. A conversation just like this has probably already happened somewhere in New York, Manor, Texas, Philadelphia, Boston, Chicago or any of the 1500+ cities around the country. Folks are volunteering to make city services run better. They're creating apps and mashing together data sets. Connecting community organizations that fight the proliferation of alcohol licenses with health departments that track substance abuse. They photograph graffiti so it can get cleaned up, "tag" potholes so they can get filled, track damage to parks, and help map playground equipment for repair.

I think this is great. It redefines volunteering. It shows resourcefulness in how we use our private resources for public good. It creates new opportunities for municipal bodies and challenges nonprofits to engage with their communities. And it's happening all over the place. At MIT two weeks ago at the #Tech4Engagement summit the folks doing this kind of work got together to share ideas, network, and brainstorm more ways to work this way. The Bloomberg Philanthropies just launched a challenge to spark even more of such efforts - check out the Mayor's Challenge. It's a matter of human connectors and existing institutions finding ways to make things better. Or, as I've been known to call it, using private resources for public good.

One hundred years ago in the U.S. this same kind of energy fueled the progressive movement which launched many of the organizational norms and structures which we now take as a given. I think these kinds of citizen-civic-community movements are doing that again - launching new forms of association, change, and improvement for this century. We're doing nothing short of changing where and how good happens.

Data Philanthropy

On Monday I posted a brain dump on how data can change philanthropy. (I hope you'll add to that list of ideas. Feel free to put other examples up as well)

But the real change coming from data is much bigger - because it's everything in that post and more. Data are reshaping our associational life. Our associations - neighborhood groups, nonprofits, mutual aid organizations, temporal networks - are how we use private resources (time, wisdom) for public (collective) good. Philanthropy is one way we participate in associational life with private money. We are beginning to see half-step change in philanthropies participating in associational life with data.

Here's the half-step practice change:
Data need to be seen as both an input and output of philanthropic dollars and institutions. The dollars are limited. The information they create and can catalyze can be re-used and re-applied over and over (if let loose into the world with that intention).

If foundations really valued data as an input, they'd rethink their grants management departments. These data experts wouldn't just deal with compliance issues, they'd be unleashed on relevant external data sets that matter to the foundation's program strategies. They'd be let loose to map, crunch, remix public data sets, peer data sets, industry data sets that would be used by program officers to develop their funding strategies. Grants management isn't just compliance, its strategy and learning. They'd partner with program and evaluation staff to make teams with data, domain, and grantmaking expertise, as well as appropriate external networks to assess and validate internal ideas. They'd be part of building the learning cycles and tools that would keep the grants meaningful and the foundation catalytic, not just compliant. They'd lead the charge in re-inventing evaluation at foundations, a process that data, shared data systems and data software will accelerate.

If foundations really valued data as an output, they'd rewrite their grant agreements and contracting language. Creative Commons or other public re-use licensing would be the norm not the exception. Sharing data sets produced by philanthropic dollars (in machines readable form) would be standard practice (as would be the real-time publishing of grants data). Foundations would promote, experiment with,  and build communities around their data and their grantees data - they would structure their grants to produce and release data for others - designing the grants from the beginning for the multiplying effect of the data products (and the open source tools to make sense of the data).

What if philanthropic investments were designed to supply data into a data commons? What if philanthropy took on public access to data as one element of civil society? What if we provided incentives for the open sharing of knowledge and data in the regulatory streams that guide philanthropy?

Here's a full-step practice change:
To me, the most exciting development in philanthropy right now is the spread of data philanthropy.  Data philanthropy as Robert Kirkpatrick of Global Pulse views it, is when big BIG DATA companies (telecomms, search engines, social networks) donate their public use data (privacy rights protected, opt in only) to a data commons. If more foundations and private companies (Data CSR?) did this we'd be able to start addressing shared social and humanitarian goals in the same way we now work on developing viral advertising. As The Economist recently noted:  "It's now remarkably easy to share data around the world, mine massive data sets for interesting relationships, test those relationships with powerful statistical software, and publish and share results with audiences the world over, all in a matter of hours." Shouldn't we be thinking this way about hunger, global warming, migration routes, poverty, and education? Here's what Kirkpatrick describes Global Pulse trying to do:

"...analysis of patterns within big data could revolutionize the way we respond to events such as global economic shocks, disease outbreaks, and natural disasters. Our team of data scientists, open source hackers, and international development experts functions the way an R&D lab does: asking questions, formulating and testing hypotheses, building prototypes and collaborating with partners within and outside the United Nations to develop methods for harnessing real-time data to gain a real-time understanding of human well being."

We need to think about the "common good" of aggregated data - in economics, health care, environmental justice, land use policy, education - you name it. The bipolar data system we are developing - data either owned by companies or governments or solely owned and not shared by individuals  - leaves nothing in the commons. The rights of the individual - their ownership of their data and the privacy rights to make decisions about it- are critical. We are new to this digital landscape - and would be naive to assume that today's tools will be tomorrow's, that businesses have individuals' best interests at heart, or that we, as individuals can't shape our associational life online to meet our own, and our collective (non market, non government) best interests. But we have to be intentional about it. Doc Searls' new book - The Intention Economy - makes this clear from an individual point of view. (Here are Ethan Zuckerman's notes on the book).

Here's the full-scale change that reconsiders the frames of operation:
There is still a larger shift underway. Data and digital exchange have shifted the bounds of organizations. Lee Rainie argue that networked individuals - not groups (families, workplaces, associations) are the organizing force in this environment. And individuals create alliances, move among multiple associations, and create new relationships over the course of their lives.

Many in the commons movement (along with thinkers and doers about open government, digital ownership, the sharing economy, free press, public libraries, net neutrality, the electronic frontier -) have been leading the thinking is how do we want to associate digitally - what new rules do we want or need around sharing our data, using it for collective problem solving, what new structures, what new revenue streams, and what new governance. Our history is shaped by an active associational sector - influencing, not just responding to, markets and governments. We need this same intention when it comes to digital spheres. We need to think about what data we are willing to share to solve shared social problems, how we will share these data, how we will use them, and how we will get them back.

Just as our forebears designed our current associational systems, we need to define a space for networked private individuals to create public goods in the digital age. Just as we've defined and protected an independent sector in the analog world, we need to provide a voice for the commons when it comes to using private data for public good. Our existing nonprofits and foundations can lead is in this this via practice and experimentation (see the half steps up above).

But it is likely that our analog associational forms will themselves need to change. We need the role that they play, but do we need new forms of associations? The nature of data - which is very different from the nature of shared physical space - makes it worth considering what shared digital associational space, shared digital associational assets, and shared digital associational governance structures will best serve us.

The real barrier we face now is changing how our institutions are run, how they are governed, the incentives and compliance requirements we put on them. For foundations, there is an enormous opportunity to innovate on the form - and the currency (data) - that they deploy in service of their missions.  This is an incredible time for philanthropists to redesign foundations - to experiment, re-imagine, and think differently about the structures that will best use data as a philanthropic resource.

The technology is here to let us think about solving public problems in whole new ways - but the legal and regulatory frames for doing so are not yet developed. "It takes a long time for institutions and economies to rearrange themselves to take best advantage of new technologies." We will need to re-write the rules and regulations that define philanthropy, from their bases in 20th century financial assets to those that fit a data-abundant world.

How can data change philanthropy?

I've written about this at length (search this blog for "data").

So here are some ideas in short form that consider all kinds of data - trend, legitimacy, performance, outcome, digital exhaust - and how they can change philanthropy:

• Pattern finding - aggregated revenue and organization giving, coded by strategy or outcome, would allow us to see patterns. Find strengths, networks, gaps. 
• Mash with other data - giving data mapped against health disparities, census, income, educational levels, second home ownership - you name it
• Turn online platforms into instruments for understanding - use online platforms as instruments to understand donor motivation, not just assume we understand their actions - bring persuasive tech, A/B testing, and behavior change knowledge to these platforms
• Network mapping - aggregate digital giving data to see networks of support, find influencers
• Predictive - does "flash mob" giving tell us anything?
• Predictive - which data on organizations actually influence donors? Which influence organizations?
• Predictive - non c3 actions and networks - which ones matter?
• Follow all money - a single dataset (or common taxonomy at least) of CSR, government contracts and grants, foundation dollars, development aid - real revenue understanding
• New revenue - data on impact investing, actual money flows would allow us to see if this is real, hype, or bubble. If real, code it all so Impact Investing data can be seen in light of bullet above
• Use geo-location data with volunteers to see who, what, how, where
• Use organizational information to improve program services (see datadives from DataKind)
• Create applications and software that improve sectors such as human rights work or disability services (see Benetech, TopCoder and SocialCoding4Good)
• Protect an open internet - see the Mozilla Foundation, Wikimedia
• Create fluid networks of activists - see SOPA/PIPA fight, #PDF12 for complete catalogue
• Open proposal applications - propose once, reach many (similar to Awesome Foundation structure)
• New forms of public service and geek volunteering - see CodeForAmerica, CrisisCommons, etc.
• "Liquid philanthropy" - mash up monitoring information from a PWX with feedback mechanisms such as "liquid feedback" to drive global engagement around water issues
• Form new forms - data backbone on an issue, used by shared communities of activists and donors - create 21c century organizations
• Video.
• Use funding platforms (everything from DonorsChoose to Kickstarter to SpaceHive) as "community assessments" - what are communities asking for? Same with foundation proposal processes.
• Find how (if) people move on a ladder of data engagement - from legitimacy to performance to outcomes
• Seed baseline repository of outcome data so can answer questions about organizational form and revenue mix by outcomes (down the road)
• Data communities, such as communitycommons.org
• Data as a public good - in the "next great natural resource race" over corporate/government ownership and personal privacy what is in the Digital Public Good commons?
• More coming on Wednesday on Data Philanthropy
• Add your ideas in the comments...

I think some of the most interesting possibilities are in the "data backbone" created by online and  digital communications. Using these as starting points for understanding behavior and use of private resources for public good, not just talking about outcome data.

Awesome!

Four answers to several questions:

A: Forty-six in nine countries.
Q. How many chapters of the Awesome Foundation are there?

A. 2009
Q: When did Awesome Foundation start?

A: The power of networks, good ideas easily replicated, and people like you. Plus, the Institute on Higher Awesome Studies.
Q: How did it grow so fast?

A: Awesome Summit, MIT Media Lab, July 23, 2012 (Register here)
Q: Where can I go to hack on peer-to-peer giving, democratizing grants, and making our communities more awesome?
Q: How can I get involved in redesigning philanthropy for the 21st Century?
Q: I want to help foster decentralized organizations and open brands, encourage engaged donors, and conjure up the legal and political infrastructure we need, where can I go?


I go to a lot of conferences. Many of them I'd be happy to miss. This is one I'm really sorry not to be able to attend. You should go to the Awesome Summit and tell me all about it. Thanks.

What would your community do for you?

In the last several years a trillion independent book stores have gone out of business. OK, that might be an exaggeration, but a lot of them have closed. Over the last decade the numbers have dropped from about 4000 to 1700 or so. Some have opened - membership in the American Booksellers Association is up by 55 over last year. The press is filled with daily stories about the demise of the press.

Two of MY (yes, I love them enough to think of them as mine) local bookstores have faced down closure in the last several years with the help of community rescues. Cover To Cover, in San Francisco, made it past their first imminent shutdown, reconfigured, moved to a smaller space, and hung on for a few more years after neighbors chipped in, covered costs, and signed on as "subscription members" (yes, to a bookstore). But it still closed.

Kepler's, in Menlo Park (20+ miles south of where I live, but still MY bookstore) also almost closed and was saved by community members. They raised money, restructured debt, helped with finances and the store is still open as a business. I go there every time I'm in town and buy a book. Like a pilgrimage - it's a community asset that I depend on and I know that my purchase matters.

So the question is, what do these organizations do that drives such loyalty and commitment? It's clearly a long term effort which crisis can make visible. It's the slow accumulation of one-to-one, meaningful interactions that become, over time, a community. 

Now Kepler's is trying something else. It has launched a drive, Kepler's 2020, to involve the community in designing and leading a transition to a community-owned bookstore and event space. In full 21c crowdfunding fashion it's raising money (donate here) to make this transition possible. These are not tax deductible donations. These are donations to a business to help it restructure as a community resource.The Kepler's 2020 project is also working on defining the bookstore of the future - not just saving itself. I first met the folks running this effort at a meeting for the Digital Public Library of America, which was wise enough to realize that it's community includes booksellers (and science museums, and ebook publishers.

Got that? Next, I'll probably tell you that people volunteer for the Department of Motor Vehicles. It's true. They do. I just came from a fantastic meeting about using technology for engagement. The techie for good community is alive and growing all over the place, working with and for cities, making things run better. There are loose networks of people in places all around the country, volunteering their time to make things better. And there are a few cities and government agencies wise enough to realize they need these people - and are welcoming their help.

We're volunteering for the DMV.
We're donating money to businesses to save and reinvent them.

If you run an institution that thinks it has a community purpose, my question for you is this - what would your community do for you?