Suggestions to Overcome the Difficulties of Measuring Success

 Peter Campbell reported (click HERE) that lots of the big players in philanthropy (like Guidestar) sent out a report this month saying the best way to choose which charities to give money to is NOT by looking at operational costs as a measure of efficiency, but focusing on measurable differences. As Bernstein said in the article, “When you think about making your donations, use your heart to pick the causes you care about and your mind to pick the charities you support.” This mindful consideration is the “accountability and measurement” piece (one of the 5 things to Know in 2010) that charities will need to respond to in order to get more funds. Originally, I would have provided a blanket approval for this “measurable outcomes” approach. But, now, I must admit that I am in conditional agreement with the idea.

Let me explain. My conditional reservation is not particular to me, but a caution that has plagued researchers for years. Specifically and simply, how do you measure outcomes and which outcomes should be measured? Subjective self-reports don’t always square-up with objective reality; success can be defined in so many ways; and some outcomes in some areas of social service take hours or days to assess, while others take years. Research is as much an art as it is a science, and it requires creativity to get as close as possible to defining what is “really” happening. I think social service organizations need to have lots of careful conversations about how to operationalize their mission statements. IF (and there’s the condition) we can get the measurement right (or really close to right), I am in full support. If we get the measurement “wrong,” though, we’re essentially back to where we started, making decisions on measurements that are incomplete or inaccurate.

One potential solution is to consider multiple types of assessment. For example, by leaning towards quantifiable data alone, we may be underestimating the powerful personal experiences that can be collected and shared through qualitative data. Or, maybe we can be measuring an organization’s value to the community through social network analyses, seeing how community publications refer to the organization, or researching how people in the community talk about the organization. Rather than jumping wholesale onto the Quantifiable Outcomes Train, it is my perspective that the most beneficial and careful approach for social service organizations will be to use “triangulation,” or many sources of information that can confirm a consistent picture of the the value and contributions of the organization. For nonprofits to be viewed as desirable by funding groups, they will need to track data and provide hard numbers that “prove” their value. And, they should also be collecting personal and organizational stories that show their value, and provide the compelling, personal and emotional hook that will resonate with potential donors and their communities. Also, they need to be able to show that they are operationally efficient, and do whatever they can to make sure that funders’ money gets to where the funders thought it would originally go…to move the mission forward. I’m not at all against measuring outcomes. I’m just against measuring outcomes without careful consideration of what those outcomes should be.

I’d like to know your thoughts on this. How should we measure outcomes? How do you measure your outcomes?