He was watching Tim Wormus, an evangelist for Tibco/Spotfire's visualization software, track a lot of tomatoes from the moment it left a California grower to its ultimate destination on someone's plate. As Wormus clicked through screens that showed a flowing stream of circles, lines and dots, the tomatoes made their way from the grower/shipper, to packing shed, and on to a repack house. The biggest circle then fragmented and scattered across the screen- the repack house had broken the lot apart, combined it with other lots, and sent it on its way to distribution centers, retail outlets and restaurants for life as salsa, fresh tomatoes, fast food sliced product, or a dozen other tomato products.
If weeks later one of these tomatoes made a diner sick, Federal and state food safety officials would be facing an arduous manual of trying to trace it back through all the handoffs in the supply chain to an apparent source, confirm by testing, and then trace it forward to all its other points of sale.
As experience shows, that is a hard, hard slog - error-prone, unforgiving, labor intensive, taken in the glare of national spotlights, all the while farms closing, consumers falling ill, evidence seeping into the ground, memories fading, and officials wondering: is this an attack, or an accident?
With visualization tools, the right data, and agreements in place for its use, however, investigators would have answers much faster. They could illuminate the supply chains, visually trace back to likely sources, and trace forward to the other points of sale where the same contaminated produce had shipped. They could nip the outbreak of killer Salmonella or cyclosporine in the bud -avert illness and death, understand whether it was an accident or intentional, and possibly avoid the general shutdown of large segments of the nation's food supply.
All this in hours, at most days - with the right preparation, infrastructure, and collaboration between industry and government. In the absence of any of that today it takes months, and costs jobs, and health. 2008's Salmonella Saint Paul outbreak, for example, cost the tomato industry $100 million. These losses hit America's farm laborers and small town businesses hard.
A Vision for the Future
In a real outbreak, investigators start knowing only that tomatoes in mama's spaghetti in the Bronx, Harry's soup in Buffalo, and Earl's Famous Barb-B-Q in Brooklyn had made diners ill.
With data ready to use and flowing into the visualization tools, investigators would quickly identify the first common ancestor in the tomato supply chain serving all three outlets. One of Spotfire's pretty colored circles would actually be the source of the outbreak. Which one? Investigators couldn't miss it: "Click-click-click" through the displays and the first common ancestor in the supply chain would stand out like- well, like a piece of rotten fruit.
Investigators would need to confirm that through testing, of course, and that on-site process could still take weeks. But at least they would have gotten to a high-probability source much faster, shaving days, weeks, or even months off the investigation.
If another outbreak popped up somewhere distant investigators could quickly compare ancestors and determine whether the two outbreaks were linked - or whether the new outbreak was an entirely new event. In a terrorism investigation, seeing the unexpected fast is critical to localizing the problem or sounding a national alert.
This is powerful stuff: it lets investigators act fast on highly-suspect supply chain segments. With the right training and planning, they could, perhaps, signal all clear on tomatoes not in the Mama-Harry-Earl supply chain, and let commerce resume normally elsewhere, saving millions of perfectly fine tomatoes from needless destruction.
Technology is Great, But It's Nothing Without Data
A data visualization tool such as Spotfire is only as good as the data, of course. So can this tool really be useful?
Visualization technology is a massive filtering mechanism which time and again is proving invaluable. It would here, too. "It's not information overload that is killing us," as NYU's Clay Shirkey reminds us. "It's filter failure."
Visualization technologies can help make sense of scads of data. That's important. Whether it's food, climate, and geography; children, foster care, and health; narco-terrorists, financiers, and quartermasters for bomb plots; understanding these cross-boundary relationships, managing the pieces for the whole, as MIT's Stephen Spear warns, is time-critical, mission-critical, and life-saving. Spear "guarantees failure" for any who don't.
Today, by law every business in the supply chain must keep "one up/one down" data as part of every commercial transaction, showing what they receive, and from whom; and what they shipped, and to whom, and on what date.
It seemed perfect. But no one had ever stitched all the "one-up/one down" data together by lot across an open supply chain from a farm source to point of sale.
Industry hadn't. Government hadn't. Researchers hadn't. Till last winter, that is, when all three groups sat together to figure out how to rescue America's broken food safety systems.
It was then that Harvard University convened a unique gathering of government officials, members of the tomato industry, and information technologists. Working together, participants created a temporary collaboration platform for government and industry to see what could be done with real data. Within the safe environment Harvard provided, tomato growers and shippers from California, Florida, and Mexico provided the actual raw data straight from their accounting systems. The gave the data, in Excel form, to members of Microsoft's Advanced Technology in Governments Group, who then linked the data to Spotfire.
It was real data, in fact, that was creating the jaw-dropping visualization. Once the data was loaded--which took a good deal of time and effort--the network visualization was nearly instantaneous.
What impressed the federal and state investigators - not to mention industry leaders -- was that the visualization tool would allow them do in minutes what government investigators today spend months doing in the field.
Pooling data to create information
Data in these "open" supply chains - unlike data, for example, in Walmart's or Darden's vertically integrated and highly disciplined chains -- is literally all over the place. Every business keeps its own, often in unique formats, with different naming conventions for things as simpl e as addresses. This is highly confidential data - "competitive industrial" in nature. Throughout the supply chains there are thousands of independent data sources, all closely held, little standardized, and few existing all in one place.
It is a huge cross-boundary challenge: Could regulators and industry participants pool their assets to turn data into actionable information? Generating greater public value in this way is what Harvard's Mark Moore has long urged -- in this case, pooling data that currently exists as dispersed atoms but not in usable form-to reap better, faster, cheaper food safety investigations.
During the meeting at Harvard, it became clear that the wall between regulators and regulatees, put there for the public good, had been strangling collaboration.
Government investigators, for example, knew little of the tomato business, and in the Harvard sessions they turned to industry executives to help explain the meaning of the data they were seeing displayed. (The tomato business itself is so complex that industry subject matter experts more than once debated the meaning of data among themselves.)
On the other side of the equation, industry knew little of the regulators' forensic process or its requirements, often being baffled by the seemingly capricious demands of regulators.
In a world requiring collaboration industry and government had grown up apart. The regulators, much as Malcolm Sparrow shares in The Character of Harms - were struggling to treat today's problems with out-of-date tools now far too blunt for the job. Industry was struggling to get their data to regulators - but could not fathom government's requirements or get industry's dispersed data all in one place, ready.
In fact, it emerged that government and industry have no collaboration platform to manage the complexities of farm-to-fork safety: they do not share information, except under duress; they do not share information systems; they have no rules, no governance, no process for coming together before, during or after crisis to manage it successfully for the nation. As Steve Goldsmith notes in his book on networking, all are critical to cross-boundary success.
The results have been painfully obvious: botched investigations and perhaps preventable illness, costly market failure, and battered consumer confidence.
Visualizing Food Safety - And More
The initial displays last winter were thus a remarkable moment.
But it takes cross-boundary collaboration built around the formation of a collaboration platform - with rules, governance and infrastructure, as Harvard Business School's Tom Eisenmann describes - where none today exists.
How to foster such cross-boundary collaboration? In my next post, I will share some of the key implementation challenges ahead.
Zachary Tumin is the Associate Director for Programs in Technology, Networks and Governance at the Ash Institute of Harvard University's John F. Kennedy School of Government. He is also the contributing editor, technology, for the Better, Faster, Cheaper site. "Collaborate or Perish!" by William J. Bratton and Zachary Tumin will be published by Crown/Broadway Business, a division of Random House, in 2011.
