Why We Need to Get Away From Innovation by Anecdote

Feel-good stories are nice, but there's a role for academia in bringing scientific rigor to the process.
May 12, 2015
By Kevin C. Desouza  |  Contributor
A Foundation Professor in Arizona State University's School of Public Affairs

If you've been to a conference on government performance in recent years, you've heard the stories: cases presented on the latest innovations, accompanied by statements like "We tried this extremely cool way to engage citizens and everyone was engaged" or "We implemented a new model for budgeting and received positive reactions from stakeholders." While there is nothing wrong with feel-good case studies or anecdotes, it is troubling to see these artifacts become the basis of how we advance the state of practice in local government.

We have organizations that give out awards for these artifacts without much, if any, serious evaluation of whether the purported innovations actually worked or could be scaled or adopted by other communities. To the best of my knowledge, none of the case studies that are presented at our major professional conferences and other networking events have gone through even the basics of scientific scrutiny. Instead, anecdotes laud the success of a project without providing detailed information that could help others learn its full scope, benefits, weaknesses or unintended consequences.

In my professional role in academia, I have a responsibility to create and diffuse solutions that advance how we design, govern and evaluate our public institutions. But what "solutions" should we diffuse? I believe that there is an unexploited role for our higher-education research institutions to provide the scientific rigor that is needed to help determine what is genuinely innovative.

Let me be clear: I am not calling for us to rely solely on academia to come up with the next transformative innovation for government. Academics have long struggled with bridging the disconnect that occurs when they engage in research that should be applicable and use-inspired. My own home discipline, the management of information systems, has been caught up in a cycle of generating mostly irrelevant research papers for decades and has struggled to keep up with advances in technological innovations and societal transformations. There are no signs that this will change anytime soon.

However, I do think there could be a happy middle ground. In the search for innovation in the public sector, I am engaged with a number of practitioner networks at the local, federal and global levels. I have the deepest respect for my practitioner colleagues. Between the practitioner and academic communities, there could be a truly beneficial mix of research and practical efforts.

We need to think like designers and engineers and engage in applied sciences. We should be creating innovations and solutions for our challenges through a scientific process that is valuable and useful. We should embrace experimentation, collect data on our creations through rigorous but not cumbersome processes, and analyze those data to understand the effects of our innovations within communities and between those of different types. We might even embrace the concept of running randomized controlled trials. And we might consider creating "living labs" to bring practitioners, academics, community partners and citizens together to co-create, analyze and test solutions. Models on how to design living labs exist in the engineering and health sciences, where academics and practitioners engage in fruitful collaboration.

Clearly there is much that we could be doing, in partnership, to advance the state of innovation to address vexing local-government challenges. But in addition to lauding our wins, we should be comfortable with sharing our failures. I've yet to see a conference case-study category entitled anything like "Local Government Experiments that Led to Unintended Consequences." Our conferences and other networking platforms need to be safe spaces to share what has not worked, why it did not work and what lessons have been learned.

What's most important here is that we move away from simply accepting anecdotes by acclamation. We must be more critical and evidence-driven. We need to infuse scientific rigor into public-sector knowledge discovery and diffusion. If not, we might all be making the same mistakes and continue to keep going in circles.