For the past 20 years or so, we've seen a growing demand for accountability in the public and nonprofit sectors. It's not enough to document your program's outputs—say, the number of students who took a class, or the percentage of children who received an inoculation; the emphasis is increasingly on outcomes—what did the students learn, or are fewer children getting a certain disease?
I happen to be a big believer in focusing on outcomes. That said, I'm also quite aware of the difficulties in determining outcomes. Outcomes can be difficult to measure, some of them take years (or even decades) to become visible, some cannot be quantified, and many outcomes are beyond the control of managers and their programs.
In recent years, some agencies have begun using what are called "logic models" to analyze and demonstrate the cause-effect chains that can lead to the results they are seeking. A logic model usually spells out a series of steps or activities that, when done well and in a timely manner, lead to a desirable outcome.
Here's an example of a logic model, from the Washington State Department of Social and Health Services. This logic model was developed to address child abuse, a problem of great concern to Gov. Christine Gregoire. She believed that repeated abuse of children could be reduced through prompt reporting of suspected cases by child-protective-service social workers. So she set a goal: Social workers would report any suspected child abuse within 24 hours (the existing statute gave social workers 10 days). The logic model the department came up with spells out the theory behind this change:
As the graphic shows, the model is based on some assumptions: Faster reporting leads to faster investigations, which results in more effective safety plans, and these plans (when followed) reduce the incidence of children being abused multiple times.
If you're thinking that surely there's much more involved in keeping children safe than simply fast reporting and quick investigations, you're right. But the state wanted to start somewhere, and its staff chose a change that it could control. The downward arrow demonstrates which actions are more or less under staff members' control.
Logic models have several advantages, as well as a number of potential downsides.
• They help staff see how their actions contribute to the ultimate outcome.
• They show what is within the staff's control.
• They focus accountability very specifically; people know what they're responsible for doing.
• They help managers determine whether long-term strategies are working and give them places to look when the strategy falls short.
And some of the caveats:
• By design, logic models simplify reality, which can hide significant factors that contribute to complex problems.
• Because logic models usually include quantitative measures, some staff may be tempted to game the system in order to "make their numbers."
• They can limit creativity. As Roosevelt Johnson of the National Science Foundation puts it, "Innovations can be logical, but they aren't necessarily linear."
• Logic models rely on theories of cause-effect; if those theories aren't based on solid evidence, the whole exercise is suspect.
Here's an example of that last caveat:
In the 1980s, a well-regarded community action/anti-poverty agency in Iowa faced a challenge. When a man with a business background joined the board, he asked the staff a tough, but reasonable, question: "How's business? Are we achieving our goal of bringing people out of poverty?" The staff assured him that they were doing just that. "How do you know?" he asked. "Where's the evidence that we're succeeding?"
The staff did a study that they expected would demonstrate their success. What they found, however, shocked them. Very few of their clients were climbing out of poverty. And most of those who did improve their economic situations did so by marrying someone with a good income. The agency director's comment summed up the dilemma: "If we were running a dating service, we could declare success!"
This study forced the staff to rethink their strategy for helping people out of poverty. They made significant changes in their programs, based on different assumptions about what works—that is, they changed their logic model. The new strategy was based on evidence from other anti-poverty programs that were showing real success. And within two years, this agency was producing real and positive change for its clients.
This story illustrates one of the most useful aspects of using logic models: the process of developing them can produce important conversations about staff members' assumptions, which can lead to a search for the evidence on which those assumptions are based. When the evidence isn't there, it's time to reevaluate your assumptions.