Discoveries, Unintended Consequences, and the Values of Science

Of all the great events in domestic history since the war, the Law of Unintended Consequences might later claim this Kennedy legislation as one of its finest demonstration pieces. A new jurisprudence was opening up...[whereby] a law intended to unify would divide Americans by categories.^[1]

Unintended consequences from physics discoveries and their applications are legion:

• The Industrial Revolution was never intended to send the planet down a path toward global warming and climate change.^[2]
• When Marie Curie discovered polonium and gave radioactivity its name, no one immediately visualized nuclear waste or nuclear medicine.

  

  

• In December 1938 when Otto Hahn and Fritz Strassmann stumbled across nuclear fission and their data was correctly interpreted by Lise Meitner, they were merely trying to understand the nucleus. They did not intend weapons research. Nuclear fission was found because it was possible to find it.^[3]
• When John von Neumann envisioned programmable computers facilitating weather predictions using complex meteorological models, I doubt that he intended the subsequent development of facial recognition algorithms that would enable a dictatorship to intimidate a population.
• I’m sure the internet founders never intended that instant access to information would result in the demise of deep reading.^[4]
• Surely the college students who launched Facebook never intended for their creation to become a weaponized means for rendering suspect the notion of public “truth” or for foreign governments to disrupt national elections.
• Machines were originally intended to be tools for serving people. In his historical survey Civilisation, Kenneth Clark remarked that “in the world of action a few things are obvious....One of them is our increasing reliance on machines. They have ceased to be tools and have begun to give us directions. And unfortunately machines, from the Maxim gun to the computer, are for the most part a means by which a minority can keep free men in subjection.”[5]

  

Hindsight is always 20/20. The creators of something new almost always have good intentions. The law of unintended consequences, for good and for bad, inevitably applies. While we as scientists owe it to our fellow human beings to give our best to whatever we do, we are not society’s saviors or censors. Although we must be part of the conversation, what society decides to do with our discoveries is society’s business. Jacob Bronowski passionately argues this point:

• Now the man who makes the discovery...must make the discovery even if his creation produces works which are rather ambivalent and can be used for one purpose or another. Society has to make some judgment about this, but I do not think the creator...should be a censor on behalf of society. Any man who discovered the destructive effects of atomic energy and withheld this discovery from mankind is in my opinion a maniac. And I use this word deliberately; he is a maniac because he has no notion of what can be made of the forces that he has discovered... for you to ask the scientist to keep a secret is just as bad as for him to go and give it to a foreign embassy. What business has he to decide what the nation wants to do with the knowledge which it possesses? This is not a subject for an individual’s decision at all. He has no special competence in this.[6]
• No one can judge Jacob Bronowski to be naïve about the unintended consequences of science on society. He was a member of a team that investigated damage inflicted on Nagasaki by the atomic bomb. Of that experience he wrote elsewhere,
• The power of science for good and for evil has troubled other minds than ours. We are not fumbling here with a new dilemma; our subject and our fears are as old as the tool-making civilizations. Men have been killed with weapons before now: what happened at Nagasaki was only more massive...Nothing happened in 1945 except that we changed the scale of our indifference to man.[7]
• The dangerous unintended consequences of science say more about human nature than they do about science. Nevertheless, as scientists and science appreciators who are people of conscience, we seem to be caught in a dilemma.

The Values of Science
The values of science may be more important than the discoveries of science. Values are not imposed on scientists like the oath recited in a congressional swearing-in ceremony. They don’t have to be, because those values are necessary conditions for doing science. Let us review some of them.

Respect for Doubt, Respect for Truth. Science can only be done with a robust commitment to both doubt and truth. Nature never lies, and if we would understand nature we must accept the answers she gives to the questions we put to her. Honesty between scientists is enforced through peer review and the replication of claims. But truth can be a slippery concept. Niels Bohr famously wrote that there are two kinds of truth: “simple truth” and “deep truth"—the opposite of a simple truth is false, but the opposite of a deep truth is also true.[8] Absolute truths are few; questioners who expect binary yes/no answers typically miss the point; existential questions seldom have universal answers. Nevertheless, the attitude is so widespread in popular culture today that evidence-based reasoning is indistinguishable from opinion (recall measles vaccination controversies and climate change denials) and the unthinking fearful acceptance that truth is whatever the latest demagogue du jour says it is are not only antithetical to the spirit of science but contain within them the seeds of their own undoing. Whatever else science may be, it is an obligation to doubt everything while acknowledging the reality of truth, even when we can only approach truth asymptotically. Some claims are so and others are not so, and we must recognize the difference and act accordingly. Ideology and wishful thinking do not negate nature’s realities. As Richard Feynman reminded everyone, “Nature cannot be fooled.”^[9]

Living with Uncertainty. An essential boundary condition for respecting both doubt and truth is appreciating the limits of knowledge. In practice, science depends on measurement, and no measurement can be exact. And quantum mechanics teaches us that, in principle, the more precisely we know one observable the less precisely we can know a complementary observable. Scientists have learned to live with uncertainty as an integral part of doing science. Meanwhile, popular culture, as represented, for example, by whipped-up noisy crowds at political rallies, seems to expect simple black-and-white answers to infernally complex questions and expects those answers to be delivered with absolute certainty. Walter Kaufmann rightly observed that “certainty should not be purchased at the price of honesty."[10] Even though uncertainty is the price of doing science, we are not ignorant; when we know something, we know it to within such-and-such percent, an increment of tolerance.

Honorable Consideration of Alternatives. Before the cosmic background radiation was discovered, the steady state and the big bang were alternative cosmological models, and both were taken seriously. Although the steady state model was eventually ruled out by evidence, the existence of two models helped the proponents of both to think straight. Ideas that become obsolete are still honored as part of the scientific journey because they were tentative steps toward a deeper understanding. In science we give hearing to alternative hypotheses, not to show how open-minded we are but because the alternative may be right. But in the end the data decides, not our egos or desires or who can shout the loudest. And we still respect those with whom we disagree.

The proponents of the big bang and steady state models shared a common goal—the greater good of finding a genuine understanding of the universe. Such a shared vision seems to have been forgotten in modern society. Too often current public discourse proceeds as described by Jennifer Hecht: “There is a kind of mutual blindness, as if personal affiliation with one camp or another means more than does interest in the truth."[11] Anna Quindlen expressed her similar observations this way: “Today’s graduates have also learned that having strongly held beliefs means expressing contempt for those of others.”^[12]

Change is inevitable, but when in the public arena a disruptive idea or technology is pushed by those with agendas that do not spring from a shared vision of the greater good, deliberate thought about consequences and alternatives can be drowned out by those with large marketing budgets or bully pulpits. Whenever a disruptive change is proposed, the question “What might be gained?” should be accompanied by the question “What will be displaced?” Unintended consequences need not include regret that we appreciate what we have only after it’s gone.^[13]

Dissent, Creativity, and Tolerance. Great discoveries in science occur because someone dared to challenge embedded assumptions. For dissent to be followed with the creativity needed to propose viable alternatives, the environment must, again, be one of tolerance. Bronowski continues:

I hold that you cannot carry out the activity of science if you do not have a society organized in this way: a society rich in dissent and yet rich in tolerance and rich in honor. I think that in this are the beginnings of principles which the scientist can teach the world at large.^[14]
Doubt, uncertainty, honesty, dissent, creativity, respect, tolerance—these are values necessary for doing science. To the degree that they influence the wider society and civilization, we can avoid or manage some unpleasant unintended consequences and look to the future with optimism for the cause of human dignity.

References

[1] Theodore White, In Search of History: A Personal Adventure (New York: Harper & Row, 1978), pp. 304–306, 510–511.
[2] By comparing the rate of fossil fuel consumption to the rate of nature’s production of hydrocarbons, the ecologist Jeffrey Dukes estimated humanity’s rate of consumption exceeded the rate of nature’s production by 1888. See Charles Day, “Running Out of Energy,” Physics Today 1 (January 2020): 8. Day’s editorial cites J. S. Dukes, Clim. Change 61 (2003): 31.
[3] Otto R. Frisch and John A. Wheeler, “The Discovery of Fission,” Physics Today 20, no. 11 (1967): 43.
[4] Nicholas Carr, The Shallows: What the Internet is Doing to Our Brains (New York: Norton, 2011).
[5] Kenneth Clark, Civilisation: A Personal View (New York: Harper & Row, 1969), p. 346.
[6] Jacob Bronowski, A Sense of the Future (Cambridge, MA: MIT Press, 1977), pp. 19–20.
[7] Jacob Bronowski, Science and Human Values (New York: Harper Torchbooks, 1956 & 1965), p. 4.
[8] Niels Bohr, “Discussions with Einstein,” in Albert Einstein, Philosopher-Scientist, Paul Schilpp, ed. (La Salle, IL: Open Court, 1949 & 1970), p. 240.
[9] Richard Feynman, The Pleasure of Finding Things Out (New York: Perseus Publishing, 1999), p. 169.
[10] Walter Kaufmann, The Faith of a Heretic (New York: Anchor Books, 1963), p. 36.
[11] Jennifer Hecht, Doubt: A History (New York: Harper, 2003), p. xii.
[12] Anna Quindlen, “Life of the Closed Mind,” Newsweek (May 30, 2005): p. 82.
[13] One is reminded of Joni Mitchell’s lyrics in “Big Yellow Taxi” (1970):
Don’t it always seem to go
That you don’t know what you’ve got
‘Till it’s gone.
They paved paradise
And put up a parking lot.
[14] J. Bronowski, Ref. 6, p. 20.