When it comes to public-sector performance measures, collecting and verifying data are hard enough. Then comes the challenge of trying to actually get the attention of citizens, elected officials or -- probably toughest of all -- the press, all audiences that often don't seem to evince much natural curiosity in results-based government.
Yet there are a growing number of examples where savvy players in public policy are having some significant success in getting key audiences to focus on performance measures.
For example, Adam Wilson, a reporter who covers Washington state government for The Olympian , says once he discovered the state's regular "GMAP" sessions, he decided they were among the key "meetings to go to," if he wanted to stay current on what's really happening in the capitol.
GMAP, the state equivalent of Baltimore's "CitiStat," includes meetings at which various upper-level managers present information to Governor Christine Gregoire or her top executive staff related to performance in a variety of high-level policy areas, ranging from environmental health to social services. What makes GMAP -- which stands for "Government Management, Accountability and Performance" -- different than most other "stat" efforts is that the sessions are open to the public and the press.
"Those sessions have been fairly enlightening," says Wilson. "Partly just because you have a format where the governor is listening to reports on progress from cabinet-level staff and so you can actually watch the governor direct government."
On the data side, the sessions are handy for a couple of reasons, as well. "You get the big picture handed to you in a statistical way, whether it involves trying to reduce traffic accidents or incidences of child abuse," says Wilson. You also learn what the government is trying to do to turn things around, he adds.
Still, when it comes to relying on government-provided charts, polls, maps and graphs, it's always good to maintain some healthy skepticism. "The last few GMAP sessions there have been some pretty glowing reports on parts of government that don't seem to be so glowing," Wilson says, "and I'm thinking to myself, 'Wait a minute, are they just turning those charts upside down?' "
What has helped give him confidence in the GMAP sessions, though, is that when those feelings of skepticism bubble up, it's not uncommon for the governor or her top staff to chime in with just the same concerns. When a chart shows that trends are either flat or improving, and Wilson wonders about the accuracy of the presentation, "often someone from the governor's staff is saying, 'No, there's a problem there.' "
The value of using data to detect problems is what led Bob Allers, the social services commissioner in Dutchess County, New York, to embrace the state's push on collecting and reporting data. He can cite numerous instances where he's used performance data that indicated trouble to successfully argue for more resources for his perennially resource-starved agency.
One of the most recent examples Allers cites is in the critical area of child protective services. When he needed to make his case for more child protective services workers, Allers spelled it out for budgeteers graphically. The graph was just a simple line that charted overdue child protective services case reports in relation to staffing levels. But it illustrated rather starkly the direct correlation between overdue reports and how many child protective services workers the county had on its payroll at any given time.
"We not only used the chart to say that we need more workers in this area" Allers says, "but also to make the point that when people leave and the workload on everyone else increases, then staff turnover actually increases, as well."
Using the graph to illustrate the cascading consequences of overdue reports, staffing levels and turnover worked; Allers got the money he needed to boost staff to try to reverse the trend. As for using such graphs and charts to make his case, Allers says that he's been doing that on an increasingly routine basis in a variety of program areas over the past three years or so, in line with an increased emphasis statewide on collecting and reporting performance data in human and social services.
In fact, in New York -- where health and social services are a county responsibility with state oversight -- all county social services commissioners now have access to what's called the "commissioner's dashboard." The dashboard is a statewide database that includes a wide variety of longitudinal and comparative data on how the state's 62 counties are doing in a variety of social and health services program areas.
The use of so-called "dashboards" is becoming increasingly common around the country, and they take a wide variety of forms, from graphics that actually mimic dashboards, to pie charts, bar graphs, line graphs, color-coded columns and ranked lists.
Whatever one chooses to call such graphic presentations, though, being clear and concise is only the beginning when it comes to using data to communicate an important message or make the case for policy or budget changes.
For performance measures to be compelling and connect with policy makers, citizens, managers and the media, a number of other variables go into ensuring that final presentations get attention.
First and foremost, argues Shelley Metzenbaum, a performance-management consultant, who for years has been studying the fine art of using performance data to make an impression, data -- however they are presented -- must in some way or another communicate more than dry facts. "The most effective measures relate to things that people really care about and that tell a story," she says.
For example, Metzenbaum cites the widespread media attention that the National Highway Traffic Safety Administration gets each time it releases its data on traffic fatalities. Yes, the data are presented in a way that doesn't require a doctorate in statistics to decipher, but it's what the data communicate that makes all the difference. "This is data that actually tells you about dangerous roads, right down to specific stretches of highway and intersections."
Besides illustrating for a very local audience an issue in which they're naturally interested, the NHTSA data does one other thing that catches people's attention: They allow for inter-jurisdictional comparisons of road and highway safety. "So you're not only measuring something that people care about," Metzenbaum says, "but you're putting a very local situation into a national context, which makes it incredibly powerful."
How do you figure out which issues and what kind of presentation will light up the intended target? Sometimes it's obvious, but sometimes it takes some work, says Donna Sines. Sines is executive director of Community Vision, an Osceola County, Florida, nonprofit dedicated to improving local quality of life by tracking community well-being indicators. As part of that effort, her group frequently works to mobilize community and political action around moving those indicators in the right direction.
Indeed, her organization makes a point of actively -- one might even say aggressively -- reaching out to every segment of the community, asking what issues citizens regard as most important when it comes to community health and well being. That thoroughness, says Sines, has been one key to the organization's credibility when it publishes something like a list of citizen priorities for local action.
For example, early on in the organization's life, Community Vision asked 2,500 citizens to rank issues that they thought ought to be of paramount importance to the county. When her organization presented the ranked list to county elected officials, "they were shocked," Sines says.
It wasn't crime, education or economic development that topped the list but rather the environment. Because Sines had solid data, and because that data told a story that clearly connected with its intended audiences -- elected officials and citizens -- it proved powerful enough that county officials agreed to put a referendum on the ballot to raise money to buy environmentally sensitive land. The referendum passed, and the county now leads all others in Florida in locking up environmentally important parcels.
Besides ranked lists, Sines has found that mapping is another graphically powerful way to get important messages across. One of her organization's recent mapping efforts was aimed at highlighting where county residents without health insurance lived. The effort was inspired by a very sobering statistic: that Osceola County harbored twice the national average of uninsured residents. Using that data and maps, Community Vision was able to catalyze support to raise $10 million for a mobile health unit. "It's so much more than looking at evaluation or outcome data," says Sines. "It's about bringing the right people in and putting them around that data and then bringing the resources to bear."
Choosing the Messengers
That combination of good data and an underlying story was part of a recently launched push to curb underage binge drinking in Santa Cruz County, California. But the effort there included an interesting twist. While officials had statistics on how widespread the problem was -- mostly based on emergency services calls -- that data didn't always pinpoint where the drinking occurred. Nor did the numbers say anything about where kids were getting the alcohol. "So we did peer-to-peer interviews," says Mary Lou Goeke, with the Santa Cruz County United Way, which tracks a variety of community health indicators through its "community assessment project."
"What we found is that kids were getting alcohol either from family or from older friends -- it wasn't by using fake IDs," Goeke says. Furthermore, they learned that the drinking itself was often taking place in the homes of the binge drinker, sometimes even when parents were there.
Even more significant, though, the peer interviewers were able to put meat on the data related to the incidence of binge drinking by recording actual stories from those who had either been hurt by or who had hurt someone else because of the drinking.
The statistics on the incidence of underage binge drinking, combined with new information on its logistics and the personal stories to back all that up, were enough to bring the county's drug and alcohol abuse program and the sheriff's department together to take action. Drug and alcohol abuse and law enforcement officials are currently working with the Santa Cruz County Council on hammering out new laws that impose an escalating series of fines on the owner of any property where binge drinking takes place.
Besides learning how to flesh out data with personal stories, Goeke has discovered one other thing as her organization has sought to present data in a way that will influence policy: Who actually delivers the data and the message to policy makers matters a lot. After United Way had gathered public health statistics on overweight children and mapped where the greatest prevalence of obesity was occurring in the county, for instance, it helped focus the effort on one particular school district and city.
Goeke's group then hired students in that school district and city to try to identify the root causes of the problem, which ranged from a limited availability of healthy food to limited opportunities for recreation. "We paid farm workers' kids $10 an hour to go out and do an analysis of opportunities and barriers to healthy eating, and environmental scans of why kids aren't doing things like ride their bikes more."
So it was only natural that when it came time to lobby school and city officials for such things as better access to healthy food and safer, more available recreational opportunities, United Way turned that job over to kids, as well.
In their presentation to city officials, Goeke says youngsters used everything from posters with pictures of parks without bike racks, non-existent sidewalks and filthy drinking fountains, to PowerPoint charts and graphs illustrating the obesity problem more generally to make their case. The city is now in the process of amending its general plan to build and improve recreational facilities.
These articles are part of a continuing series on public performance measurement focusing on citizen involvement. Support has been provided by the Alfred P. Sloan Foundation. Although the foundation may assist some of the programs described in these articles, it had no control or influence over the editorial content, and no one at Sloan read the material prior to publication. All reporting and editing was done independently by Governing staff.