Who Do The Voodoo With The Data?

Who Do The Voodoo With The Data? -- Vol. 19, No. 3, p. 26

A client contracted with us for research about the market for labor in an important Pennsylvania industry sector.

On one hand, the client expected that our research would lead to conclusions that would endorse their business plan and comfort investors. We, on the other hand, told the client that we should wait and see. After all, research is conducted when the answers are not known.

The client wanted to receive clear, unambiguous advice that would support their business plans. And, we always prefer to let the data lead us to conclusions—not to have conclusions lead us to the data.

As it turns out, the findings from our research did not favor the client’s expected conclusions. So, we worked with the client to scale
their expectations to our actual outcomes.

All’s well that that ends well. However, this little tale holds a more
general truth: It often is difficult for knowledge producers and users to obtain a comfortable match between each other’s needs and constraints.

Consider “climategate,” a hot (pardon the pun) story about the global warming controversy that recently took hold of the news worldwide. This katzenjammer rose up after review of e-mails and other documents, obtained through an alleged hacking of a server housed at University of East Anglia in England, created questions about the truth and accuracy of conclusions made by a United Nations panel that global warming probably is due to human activity.

The surface temperature of the earth increased about 1.33°F during the 20th century. The Intergovernmental Panel on Climate Change, a panel of scientists created in 1988 by the United Nations, concluded that a substantial portion of this temperature increase probably was caused by a mounting concentration of greenhouse gases created by burning fossil fuels and deforestation. For its work, the Panel shared the 2007 Nobel Peace Prize with former Vice President of the United States Al Gore.

The East Anglia e-mails and documents are reported to contain unflattering comments about detractors against the Panel’s conclusions and to divulge behind–the–scenes discussions of means to prevent dissenting scientific papers from being published.

More serious, however, is the accusation that data leading to the Panel’s conclusions about human–caused global warming were destroyed to avoid being revealed under the Freedom of Information Act. Believing that the Panel has something to hide, editorial writers, dissenting scientists, blog writers, and conservative political commentators declared that the Intergovernmental Panel on Climate Change colluded to withhold scientific information that could challenge its conclusion about global warming.

And, the noise has not stopped since.

During December 2009, the term, “climategate,” was the subject of approximately 1% of all of the world’s blog posts. The Pew Research Center’s Project for Excellence in Journalism News Coverage Index counted global warming as the 15th most frequently cited subject covered in print media, online sites, network TV, cable, and radio during 2009.

The discussion about global warming in the media and blogosphere is shrill, and attacks on climate scientists are personal. According to The Daily Climate (http://www.dailyclimate.org), many climate scientists have received insulting, crass, and menacing e–mail messages from detractors, and some members of Congress have sent “intimidating letters” threatening dire consequences for scientists studying climate change.

A look behind the dust and spit of this pub brawl reveals a complex mix of serious problems that commonly affect using knowledge from research in practice.

The Wall Street Journal recently published interviews with several scientists on the Intergovernmental Panel on Climate Change. Keith Briffa, a climate scientist at East Anglia, pointed out that “There is pressure tell a nice tidy story” about sophisticated and, at times, inconclusive science to boil down research findings to usable advice for legislators.

Richard Alley, a Penn State geoscientist, said that legislators who toured Greenland with him to explore the causes of rising sea levels told him that explicit advice is required from scientists “because, if the sea rises, the levee has to be built at some height.” A French senator told Mr. Alley that more precision about scientific outcomes is required because it is “difficult for politicians to make a decision” otherwise. Alley said that answers to some of society’s big questions, like those about climate change, that are “very difficult to quantify, must be quantified to keep the policymakers happy….It’s a very frustrating thing.”

In the same Wall Street Journal article, John Marburger, former science advisor to G.W. Bush, observed that “science tells you what the situation is, but it doesn’t tell you what to do.” The Intergovernmental Panel on Climate Change itself strives to be “policy–relevant,” but “never policy–prescriptive.” But, the imperative from policymakers to hear a “nice tidy story” that reduces scientific findings to bullet points on a PowerPoint slide makes statements such as Marburger and the Panel’s sound wooly and milksoppy.

Users of information based on research often regard researchers as people who see something that works in practice and wonder if it would work in theory. Researchers often consider their work in the same way that Vincent van Gogh described painting: It is “a faith, and it imposes a duty to disregard public opinion.”

The gulf between knowledge producers and users is not likely to be closed soon. However, in a country faced with crises related to, among other issues, health care, energy, jobs, housing, disease, poverty, justice, war, peace, banking policy, safe food, and clean water, there is nothing so practical as good research used well.