|
|
THE GOVERNMENT PERFORMANCE PROJECT How We Grade Them
Down deep, the only people who really like grades are those who get all As. And there aren’t too many of those.
In fact, if the authors of the Government Performance Project had their choice, they’d eliminate the grades altogether, and focus on the narrative portions that follow. Unfortunately, there’s a huge likelihood that were that the case, nobody much would read the narrative writeups. What’s more, the grades do provide an easy means for states to benchmark upon those who are leaders in various elements of government management.
So, grades it is. The introduction found on page 20 of this issue gives some insight into the vagaries of the system; and into why certain grades changed in the ways they did. Following is the basic guide to how the GPP does the research and evaluation that culminates in those grades:
The process all begins with the design of a survey instrument. This task largely falls to academics associated with the Maxwell School of Syracuse University, and partners they have brought in from the University of Nebraska at Omaha. Five separate surveys were sent out to all 50 states in conjunction with this effort, one for each area covered. The surveys are based, as much as possible, on a series of criteria for these topics that were developed several years ago, and have been modified somewhat since. The most significant change this year was in Capital Management, where transportation was added to the mix.
As the surveys pour in, researchers at Maxwell code the responses in such a way that they can be dispassionately analyzed and reviewed. In addition, they pore through stacks of supplementary documents that help clarify the surveys. The documents help to prevent the states from either over-stating or under-stating the quality of their activities. As this process moves forward, the academics pass along the fruits of their labor to journalists at Governing.
The journalists take a very different approach. They, too, begin by carefully reading the surveys as well as the document-based information procured at Maxwell. But their evaluations are based on hundreds of interviews with officials in the states. Some of these sources turn out to be the same men and women who filled out the surveys. Efforts are made to utilize interviews in one area of evaluation to help get more information about another. For example, the final word on the utility of human resource technology doesn’t usually come from the IT people in a state but from the personnel department.
Governing staff also utilize a variety of sources outside the governments. One staffer, for example, spent many weeks interviewing government research groups and academics across the country. Data are also obtained from the bond rating agencies — Moody’s, Fitch, and Standard & Poor’s — and from the National Association of State Budget Officers, the Government Finance Officers Association and other state government-related organizations.
Then, as deadlines approach, the academics and the journalists meet to discuss the grades. In a remarkably high percentage of cases, the two groups come to similar conclusions. In others, the differences are easily explained (often by the fact that the journalists had somewhat more up-to-date material with which to work). In yet a third group, disagreements are more profound, and require a fresh look by both sides to determine precisely where the differences are and how they can be resolved. In most cases, that is done without too much pain.
However, at the end of the day, there were some instances in which the academic and journalistic approaches simply could not be made to mesh. In those cases, the grades in this report’s pages are the responsibility of Governing editors and staffers as informed by the academic analysis.
The final stage of this year-long process is fact-checking the entire package. While sources are not given advance word about their grades, the write-ups that accompany them — and provide substantial underpinnings to the grades — are carefully reviewed. Changes are made to the text, and, where a factual change impacts the grade, that changes as well.
Of course, some readers will challenge the published evaluations. There are generally three major reasons why this will happen:
(1) The GPP analysis was incorrect, or based on incorrect information in the survey or other sources.
(2) The reader has a bias for or against a government that fuels his or her sentiments.
(3) There is a basic disagreement about the criteria used, which underlie this entire process.
In an effort to facilitate readers’ understanding of the criteria, following are the essential ones used:
2. Does the government have mechanisms that preserve stability and fiscal health, including: structural balance between revenues and expenditures; use of contingency-planning devices such as rainy day funds; appropriate management of long-term liabilities; rational investment and cash management policies?
3. Does the government provide sufficient financial information to policy makers, managers and citizens, including: accurate and thorough financial reports; useful financial data; a means for communicating budgetary and financial data to citizens; timely financial reporting; the capacity to gauge the cost of delivering programs or services; a budget that is delivered on time?
4. Does the government have appropriate control over financial operations, including: sufficient control over expenditures; appropriate managerial flexibility; solid management of procurement, including contracts?
2. Does government adequately monitor and evaluate projects through their implementation at both the entity-wide and agency levels?
3. Does the government conduct appropriate maintenance of capital assets with sufficient data to adequately plan for sufficient maintenance and funding?
2. Can the government obtain the employees it needs by hiring in a timely manner and giving managers appropriate discretion in hiring and conducting effective recruiting efforts?
3. Is the government able to maintain an appropriately skilled work force by: conducting appropriate training; retaining skilled employees; and disciplining or terminating employees without undue constraints?
4. Can the government motivate employees to perform effectively by: rewarding superior performance through cash or non-cash incentives; evaluating the performance of employees effectively; providing sufficient opportunity for employee feedback; maintaining productive labor-management relations?
5. Does the government have a civil service structure that supports its ability to achieve work-force goals including: a classification system that is coherent and of appropriate size; personnel policies that permit flexibility in civil service and pay structure; and good communications of human resource policies and goals to employees?
2. Does the government develop indicators and evaluative data that can measure progress toward results and accomplishments and does it take steps to ensure that this data is valid and accurate?
3. Do leaders and managers use results data for policy making, budgeting, management and evaluation of progress?
4. Are there organizations within the government whose responsibility it is to evaluate programs or agencies and are their conclusions utilized?
5. Does government communicate the results of its activities to stakeholders?
2. Do government information technology systems form a coherent architecture and are strategies in place to support present and future coherence in architecture?
3. Does the government conduct meaningful multi-year technology planning, including: an information technology planning process that is sufficiently centralized; providing managers appropriate input into the planning process; creating government-wide and agency IT plans?
4. Is information technology training adequate for end-users and technology specialists?
5. Can the government evaluate and validate the extent to which information technology system benefits justify investment?
6. Can needed IT systems be procured in a timely manner with appropriate controls to prevent going over budget?
7. Do IT systems support the government’s ability to communicate with and provide services to its citizens?
|
 |