Role of chance in scientific discoveries

The role of chance, or "luck", in science comprises all ways in which unexpected discoveries are made.

Many domains, especially psychology, are concerned with the way science interacts with chance — particularly "serendipity" (accidents that, through sagacity, are transformed into opportunity). Psychologist Kevin Dunbar and colleagues estimate that between 30% and 50% of all scientific discoveries are accidental in some sense (see examples below).^[2] Scientists themselves in the 19th and 20th century acknowledged the role of fortunate luck or serendipity in discoveries.^[3]

Psychologist Alan A. Baumeister says a scientist must be "sagacious" (attentive and clever) to benefit from an accident.^[4] Dunbar quotes Louis Pasteur's saying that "Chance favors only the prepared mind". The prepared mind, Dunbar suggests, is one trained for observational rigor. Dunbar adds that there is a great deal of writing about the role that serendipity ("happy accidents") plays in the scientific method.^[2][5][6][7]

Research suggests that scientists are taught various heuristics and practices that allow their investigations to benefit, and not suffer, from accidents.^[2][8] First, careful control conditions allow scientists to properly identify something as "unexpected". Once a finding is recognized as legitimately unexpected and in need of explaining, researchers can attempt to explain it: They work across various disciplines, with various colleagues, trying various analogies in order to understand the first curious finding.^[2]

Accidental discoveries have been a topic of discussion especially from the 20th century onwards. Kevin Dunbar and Jonathan Fugelsang say that somewhere between 33% and 50% of all scientific discoveries are unexpected. This helps explain why scientists often call their discoveries "lucky", and yet scientists themselves may not be able to detail exactly what role luck played (see also introspection illusion). Dunbar and Fugelsang believe scientific discoveries are the result of carefully prepared experiments, but also "prepared minds".^[2]

The author Nassim Nicholas Taleb calls science "anti-fragile". That is, science can actually use — and benefit from — the chaos of the real world. While some methods of investigation are fragile in the face of human error and randomness, the scientific method relies on randomness in many ways. Taleb believes that the more anti-fragile the system, the more it will flourish in the real world.^[9][10] According to M. K. Stoskopf, it is in this way that serendipity is often the "foundation for important intellectual leaps of understanding" in science.^[11]

The word "Serendipity" is frequently understood as simply "a happy accident", but Horace Walpole used the word 'serendipity' to refer to a certain kind of happy accident: the kind that can only be exploited by a "sagacious" or clever person.^[11][12] Thus Dunbar and Fugelsang talk about, not just luck or chance in science, but specifically "serendipity" in science.^[2]

Dunbar and Fugelsang suggest that the process of discovery often starts when a researcher finds bugs in their experiment. These unexpected results lead a researcher to self-doubt, and to try and fix what they think is an error in their own methodology. The first recourse is to explain the error using local hypotheses (e.g. analogies typical of the discipline). This process is also local in the sense that the scientist is relatively independent or else working with one partner. Eventually, the researcher decides that the error is too persistent and systematic to be a coincidence. Self-doubt is complete, and so the methods shift to become more broad: The researcher begins to think of theoretical explanations for the error, sometimes seeking the help of colleagues across different domains of expertise. The highly controlled, cautious, curious and even social aspects of the scientific method are what make it well suited for identifying persistent systematic errors (anomalies).^[2][8]

Albert Hofmann, the Swiss chemist who discovered LSD's psychedelic properties when he tried ingesting it at his lab, wrote

It is true that my discovery of LSD was a chance discovery, but it was the outcome of planned experiments and these experiments took place in the framework of systematic pharmaceutical, chemical research. It could better be described as serendipity.^[13]

Dunbar and colleagues cite the discoveries of Hofmann and others as having involved serendipity. In contrast, the mind can be "prepared" in ways that obstruct serendipity — making new knowledge difficult or impossible to take in. Psychologist Alan A. Baumeister describes at least one such instance: Researcher Robert Heath failed to recognize evidence of "pleasure brain circuits" (in the septal nuclei). When Heath stimulated the brains of his schizophrenic patients, some of them reported feeling pleasure — a finding that Heath could have explored. Heath, however, was "prepared" (based on prior beliefs) for patients to report alertness, and when other patients did, it was on the reports of alertness that Heath focused his investigations. Heath failed to realize he had seen something unexpected and unexplained.^[4]

Fugelsang and Dunbar observe scientists while they work together in labs or analyze data, but they also use experimental settings and even neuroimaging.^[2] fMRI investigation found that unexpected findings were associated with particular brain activity. Unexpected findings were found to activate the prefrontal cortex as well as the left hemisphere in general. This suggests that unexpected findings provoke more attention, and the brain applies more linguistic, conscious systems to help explain those findings. This supports the idea that scientists are using particular abilities that exist to some extent in all humans.^[2][14]

On the other hand, Dunbar and Fugelsang say that an ingenious experimental design (and control conditions) may not be enough for the researcher to properly appreciate when a finding is "unexpected". Serendipitous discoveries often requires certain mental conditions in the investigator beyond rigor. For example, a scientist must know all about what is expected before they can be surprised, and this requires experience in the field.^[2] Researchers also require the sagacity to know to invest in the most curious findings.^[4]

Royston Roberts says that various discoveries required a degree of genius, but also some lucky element for that genius to act on.^[15] Richard Gaughan writes that accidental discoveries result from the convergence of preparation, opportunity, and desire.^[16]

An example of luck in science is when drugs under investigation become known for different, unexpected uses. This was the case for minoxidil (an antihypertensive vasodilator that was subsequently found to also slow hair loss and promote hair regrowth in some people) and for sildenafil (a medicine for pulmonary arterial hypertension, now familiar as "Viagra", used to treat erectile dysfunction).

The hallucinogenic effects of lysergic acid diethylamide (LSD) were discovered by Albert Hofmann, who was originally working with the substance to try and treat migraines and bleeding after childbirth. Hofmann experienced mental distortions and suspected it may have been the effects of LSD. He decided to test this hypothesis on himself by taking what he thought was "an extremely small quantity": 250 micrograms. For comparison, a typical dose of LSD for recreational use in the modern day is 50 micrograms. Hofmann's description of what he experienced as a result of taking so much LSD is regarded by Royston Roberts as "one of the most frightening accounts in recorded medical history".^[15]

Look up chance in Wiktionary, the free dictionary.

Look up luck in Wiktionary, the free dictionary.

Look up unexpected in Wiktionary, the free dictionary.

Look up accidental in Wiktionary, the free dictionary.

Wikiquote has quotations related to Louis Pasteur.

• Belief in luck
• Discovery (observation)
• List of multiple discoveries
• Multiple discovery