They call it "Opportunity NYC" and it was controversial from the start. New York City sought to incent poor people to improve their behavior by giving them cash for making good choices: keeping a doctor's appointment, making sure their children attend school or opening a bank account.

Administered by the city using money donated from private sources, the program began in 2007 with high hopes.

Close to 2,500 families participated, receiving an average of $6,000 in "conditional cash transfers" -- critics called them bribes -- earning, for example, $100 for taking their child to the dentist.

A recent study showed that the program had produced a complicated mix of results. Subsequently, Mayor Michael Bloomberg announced that the city would discontinue the program.

Please don't call this effort a failure. Improving the behaviors of those in poverty is no simple task. Not only did this effort show some marginal improvements, but the city conducted this pilot project like a social science experiment. Participants were carefully selected and balanced against a control group, and the city invested heavily in a transparent, rigorous analysis of the results. This was an attempt at social innovation done right. "The lesson of what doesn't work is just as important as what works," says Linda Gibbs, New York City's deputy mayor for health and human services.

As Mayor Bloomberg noted, innovation without experimentation is impossible. "If you never fail, I can tell you, you've never tried new, innovative things," Bloomberg told the New York Times. "[Try] anything new you're going to have that diversity of results."

In government, a constant tension exists between the desire to try innovative approaches to social service delivery and an aversion to risk on the part of public officials.

This is understandable, since failure is treated mercilessly -- sometimes gleefully -- by a media ready to pounce on even the most well-intentioned, transparent effort to improve the status quo. If you keep doing the same thing you've always done, no matter how miserable the results, you will stay out of the press. But if you try something different and it doesn't work, be prepared for a media onslaught.

The public sector desperately needs to deliver better results for the dollars spent in the social service area. President Obama embraced this goal when he set up the White House Office of Social Innovation and Civic Participation. "Government needs to be more efficient, more effective, more accountable and more transparent," says Michele Jolin, senior advisor for social innovation, whose job involves "creating a better environment for innovation in the social sector."

Many nonprofits exhibit the same sort of reluctance to try novel approaches, sticking with familiar but ineffective approaches to the enduring conundrum: How can government provide aid to the needy without fostering a culture of dependence?

The Opportunity NYC program focused on changing the behavior of those who receive assistance. That's great, but we also need to change the behavior of those who provide aid.

When Linda Gibbs first took over the homeless services department in New York City, she inherited a culture that she described as "disempowered." Like so many public aid agencies, the employees felt the frustration of facing an ever-growing problem. In addition, the nonprofits that ran many of the city's shelters had repeatedly joined other plaintiffs in suing the department, which resulted in court orders that dictated department practices and removed discretion.

These providers were wedded to a culture that saw their role as one of providing shelter without requiring improved behaviors of the recipients of shelter.

When Gibbs announced a shift toward greater client responsibility, poverty advocates and the nonprofit providers took her back to court.

One day, Gibbs was touring a shelter with one of the heads of the nonprofit contractor who was going to testify against her agency. During the tour, one of the shelter directors whispered to Gibbs: "Don't tell the boss, but I really think we need more leverage with these families. Otherwise they just won't listen to us and they know that we can't do anything."

Her anecdote provides three insights. First, the tangled relationship between government and large nonprofit institutions means that the providers of a service also participate, often aggressively politically and legally, in shaping policy. Second, too often nonprofit leaders lose touch with the reality of the front-line providers. Third, any innovation in social services faces significant barriers.

Transforming social service delivery requires a focus on results first, compliance second.

"Everybody is talking about results," says Jolin. "At the core of what we are talking about with social innovation is how to make better, faster and more lasting progress on social problems where the lack of results has too long frustrated us."

A promising group of social entrepreneurs is leading the way. As chronicled in my book "The Power of Social Innovation," local officials who have the confidence to clear away obstacles and take on the media can improve how we deliver social services. We need to support and encourage the sort of innovation shown in the Opportunity NYC program, and acknowledge that while new approaches bring some risk, sticking with a failed status quo guarantees poor results.