The Public’s Problem with Performance Measurements

Last month we wrote about the growing volume of health-care quality information and its potential benefits to policymakers and the public. There is no question in our minds that there will be dramatic possibilities for the use of this kind of results-based data -- if only the general public could be convinced to tap into it.

Consider, for example, the reams of performance data that governments are pouring out about themselves. Are citizens taking advantage of it? Not so much, says Jeff Tryens, deputy director for performance management at the Mayor’s Office of Operations in New York City. Over the past 17 years, Tryens has been involved in three major performance reporting projects in Oregon, south Australia and now New York City. He has yet, he tells us, “to find a formula to get people who really don’t have a reason to be interested in government to go and look at a website.”

But let’s take things a step further. While there may not be an outpouring of fascination about nitty-gritty government functions, people are often interested in government-tracked measurements that directly affect their lives, such as fire or police response time, trash pick-up and school ratings.

“While people are interested in government and what it is doing,” says Barbara Cohn, director of the Fund for the City of New York’s Center on Government Performance, “if you were to call them and say, ‘Do you want to see some performance measures of government?’, they would probably hang up on you.” But “if you were to say, ‘Do you want to know what your health department is doing or your school system is doing?’, that would draw some attention.”

For the past nine years, Cohn has been working with dozens of governments that range in size from Toronto (home to 2.7 million people) to Brisbane, Calif., (4,282 people) through the Trailblazers program, which helps governments involve citizens in developing the measures they want their governments to track -- measures that are often different than the ones officials pick.

One success story involves a measure used by New York City and Los Angeles: In both places, restaurants must post on their premises the grade they’ve received from local health authorities. People see those grades and react accordingly, as do restaurateurs who are embarrassed by any rating below an A. In New York City, for example, the first full year after such grades were posted (July 2010 to July 2011), the portion of restaurants that were awarded an A rose from 65 percent to 72 percent. In a city with about 24,000 restaurants, that’s almost 1,700 that cleaned up their acts in just a year.

With restaurant grades, citizens can’t help but see them before they order dinner. In a number of other instances, however, information useful to the public is more difficult to find. That leads to a key Cohn suggestion: The performance measure “must be easily accessible. It can’t be mixed in with pages and pages of data about things that are irrelevant” to the person seeking the information.

The federal government, which publishes lots of important information to aid people in making health-care choices, has been concerned that citizens are not making good use of it. Shoshanna Sofaer, professor of health-care quality at Baruch College in New York City, is the co-author of three short reports that were put together for the Agency for Healthcare Research and Quality, which address the challenge of getting people to use information about quality to make better health-care decisions. The reports spell out the importance of avoiding jargon, simplifying complex information and explaining numerical data. Of course, the performance information also needs to be disseminated in a way that citizens can see it. We see no reason why these same principles wouldn’t apply to any other kind of citizen-centric performance measures.

In 2007, the California HealthCare Foundation embarked on a rigorous effort to get comparative information about hospitals into the hands of the public. Although the foundation had been publishing the information on a website, it had become clear that only 23 percent of Californians knew that the data was available and that it was relevant to decisions they could make about their health care. The marketing campaign used Google and Yahoo for “ad word” purchases on key search terms, and tried other mechanisms -- including a “birth and baby fair” -- to convince young women that looking at comparative information was important.

Maribeth Shannon, program director with the foundation, says the marketing campaign, which was run in several cities in 2008 and 2009, emphasized that hospitals differed significantly. One ad, for example, showed the rates at which cesarean births were taking place at two Sacramento hospitals. At one, it was 1 in 5; at another it was 1 in 12.

The ad campaigns had considerable impact. There was an elevenfold increase in visits to the foundation’s website during the months of the campaign.

One of the keys to engaging people, says Shannon, is to get them at a point where they need to make a decision. “People aren’t interested in globally finding out which hospitals in their community are good,” she says. “They engage in a website when they have an immediate health-care decision they have to make.”

That goes for data for other decisions as well -- as restaurateurs in Los Angeles and New York can attest.