For about six years, the program operated based on Sanford's assumption--and $29,000 a year in funding from the local Alcohol, Drug Addiction, and Mental Health Services Board, which disburses federal and state money to addiction and mental health treatment programs for three rural counties in southeastern Ohio.
A few years ago, however, the tri-county mental health board decided that too many of the programs it was funding were operating on similar hunches. So the board announced that all the agencies it funded should start measuring whether the services they were providing were actually doing anyone any good. "Our whole reimbursement system was based not on outcomes but on units of service," says Diane Pfaff, who coordinates the performance-measurement effort for the board. "We wanted to get a better sense of what it was we were buying."
Despite the additional work involved in assessing clients, the Athens Photographic Project embraced the move as a welcome opportunity. It could prove that what seems at first glance to be an amorphous and touchy-feely program is actually delivering consistent and concrete results. It turns out that Elise Sanford's hunch was right on the money.
...what seems at first glance to be an amorphous and touchy-feely program is actually delivering consistent and concrete results.
What's particularly interesting about the trend is that the policy areas where the practice appears to be making the most significant inroads are health, mental health and social services--the very areas in which many experts have argued that government and vendors tend to have the least direct influence over program outcomes.
According to Phil Basso, head of organizational effectiveness for the American Public Human Services Association, there has been "a wave that's rippled out because of the federal government's increased interest in outcomes."
One of the earliest recognized systems to directly connect dollars and results in human services was the Oklahoma Department of Rehabilitative Service's "milestones" program, which won an Innovations in American Government Award back in 1997.
Under the program, which is aimed at returning disabled Oklahomans to the workforce, the state reimburses providers based on a series of milestones ranging from whether initial client evaluations have been completed, to whether a client actually gets a job, to how long a client has stayed in a job.
In the wake of the Oklahoma program, dozens of jurisdictions began adopting a milestones approach to contracting.
While acknowledging that the trend has been toward formal performance-based contracts, James Moore, director of government programs for the Rensselaerville Institute, which advocates for results-based government, argues that the practice doesn't necessarily have to take on the structure of a contract. In fact, in many cases, it may be better that it doesn't.
The problem, Moore argues, is that too often performance-based contracts are more about "saving money and about black-and-white 'Did you make your targets or didn't you?'" than about improving outcomes. "It puts providers and governments in an adversarial position from the start," he says. "And it also tempts providers to game the system."
That is why Moore and his organization advocate for a practice he simply calls "investing for outcomes." In other words, whether embodied in a formal contract or not, there is an understanding between governments and providers that vendors will develop outcome measures. These will be used to gauge the effectiveness of programs--and also to improve them--rather than merely decide whether a reimbursement is justified, and for how much.
But where formal contracts are operating in the ideal way, they aren't simply being applied to reward or punish providers. They're also being used to spur a more sophisticated discussion about what is--and isn't--working, and how programs might change to improve. The other key to avoiding conflict, says Basso: Funders shouldn't simply dictate outcomes from on high. "Where we see success is where funders and providers have put a group together to develop outcomes mutually," he says. "And the really good news around all this is that everyone in the system really wants the same outcomes."
Oklahoma Milestones is still going strong, says Teri Egner, who oversees it for the state Rehabilitative Services Department. In no small part that's because the program has managed to steer away from the punitive and adversarial and toward the cooperative and instructive. Data on outcomes, she says, always have been used first and foremost to figure out which job-training and readiness strategies worked best for which type of client. And Oklahoma providers generally think the system has worked well.
"The state and its taxpayers are definitely getting a better deal now," says Robin Thompson, who was working for a nonprofit job-training and placement agency when Oklahoma officials decided to do performance-based contracting.
It took some time and hard work, she recalls. But she believes that going to an outcomes-based system of reimbursement definitely spurred providers to overhaul services in order to become more effective. "In the past, we simply billed for the time that clients spent with a job coach. It was wonderful and we loved it, and we made lots of money," says Thompson, who now runs her own for-profit job-training and placement service. "What we weren't focused on was actually getting people into jobs."
While the new system, according to Thompson, "has definitely made us work harder," she's a convert. "In the long run, this is better for everyone; everyone wants to do a good job."
Although it may be more work for providers, "it requires a lot more from the funder, too," says Diane Pfaff, of the tri-county mental health board in Ohio. "It's definitely easier to just hand out money. So it takes an investment and organizational change on the funder's part to do this right. We didn't want our providers to simply file reports on performance. We wanted to infuse this with learning."
Part of the strategy for doing that, Pfaff says, was to make a conscious decision to "decouple funding from performance to give people time to learn about the milestone concept and test it out." That has allowed organizations to focus on the learning side and not be distracted by the new math of provider reimbursement.
Conscious decision or not, the tri-county board's executive director, Earl Cecil, adds that in many cases, his organization simply doesn't have the authority to enter into performance-based contracts because of federal and state rules concerning reimbursement.
I have 18 lay people on my board, and now, rather than just talk about buying a bunch of clinical services, they're actually discussing things like whether clients are successful.
While the push has been more of a struggle for some organizations to get used to than for others, Cecil says, "I think all providers would admit that they've improved programming by analyzing what they do and why--what they're trying to accomplish--and then figuring out if they're doing as good a job as they can or whether they can improve programs."
For several years now, the Athens Photographic Project has been giving "before and after" psychological tests sanctioned by the state mental health department to those entering and leaving the program. The "empowerment surveys," as they're known, measure feelings of self-worth and confidence.
"We test our students at the beginning of the course and at the end," says Nate Thomson, who took over as executive director of the photo-therapy project in 2007. "And in everything from self-esteem to self-efficacy, we have shown consistent, quantifiable positive results."
Another program payoff: "These activities are actually reducing the burden and dependency on clinical care," Thomson adds. "Rather than being isolated in clinical care, our students now have one foot in the community."
And that, in turn, is paying off for the tri-county board, says Cecil. "I have 18 lay people on my board, and now, rather than just talk about buying a bunch of clinical services, they're actually discussing things like whether clients are successful."
