Experiment

In scientific inquiry, an experiment (from the Latin term ex- periri, meaning "of (or from) trying") is a means of investigating the natural world, answering research questions, solving practical problems, and supporting (or falsifying) theoretical assumptions. The experiment is a cornerstone in the empirical approach used in both the natural sciences and social sciences.

Design of experiments

An experiment can be thought of as a specific type of method used in scientific inquiries, and personal questioning, usually to study causality. Often the objective is to test a hypothesis: that is, a tentative explanation of a phenomenon or mechanism of causality. The essence of an experiment is to introduce a change in a system (the independent variable) and to study the effect of this change (the dependent variable). Two fundamental considerations of experimental design are:

1. The independent variable is the only factor that varies systematically in the experiment; in other words, that the experiment is appropriately controlled, so that confounding variables are eliminated.
2. The dependent variable truly reflects the phenomenon under study (a question of validity) and that the variable can be measured accurately (that is, that various types of experimental error, such as measurement error can be eliminated).

In a strict application of the experimental method, hypotheses are tested by critical experiments: ones that can falsify the hypothesis in the case of a non-result (that is, an experiment showing that the independent variable did not affect the dependent variable as predicted). Such pure applications are rare, however, in part because a result can sometimes be challenged on the basis that an experiment was not sufficiently controlled, that the dependent variable was not valid, or that various forms of error compromised the experiment. The scientific method, as a result, builds in the need for reproducibility (usually termed "replication") and convergent evidence.

In designing experiments, scientists attempt to balance the requirements and limitations of their particular field of science, so that the experiments can provide the best conclusions about the hypothesis being tested. In some sciences, such as physics and chemistry, it is relatively easy to meet the requirements that all measurements be made objectively, and that all conditions can be kept controlled across experimental trials. On the other hand, in other cases such as biology and medicine, it is often hard to ensure that the conditions of an experiment are performed consistently; and in the social sciences, it may even be difficult to determine a method for measuring the outcomes of an experiment in an objective manner.

For this reason, sciences such as physics and chemistry are sometimes informally referred to as "hard sciences," while social sciences are sometimes informally referred to as "soft sciences." These descriptions attempt to capture the idea that objective measurements are often far easier in the former, and far more difficult in the latter.

In addition, in the social sciences, the requirement for a "controlled situation" may actually work against the utility of the hypothesis in a more general situation. When the desire is to test a hypothesis that works "in general," an experiment may have a great deal of "internal validity," in the sense that it is valid in a highly controlled situation; at the same time, it may lack "external validity," when the results of the experiment are applied to real-world situations.

Based on these considerations, experimental design in the "hard" sciences tends to focus on the elimination of extraneous effects, while experimental design in the "soft" sciences focuses more on the problems of external validity, often through the use of statistical methods. Occasionally, events occur naturally from which scientific evidence can be drawn, which is the basis for natural experiments. In such cases the problem of the scientist is to evaluate the natural "design."

Controlled (laboratory) experiments

Many hypotheses in the sciences (such as in physics) can establish causality by noting that if and only if a certain phenomenon occurs or a certain condition prevails, a second phenomenon is observed. Often, however, such situations are difficult to generate.

To demonstrate a "cause and effect" hypothesis, an experiment must often show that, for example, a phenomenon occurs after a certain treatment is administered to a subject, and that the phenomenon does not occur in the absence of the treatment.

A controlled experiment generally compares the results obtained from an experimental sample against a control sample, which is practically identical to the experimental sample except for the one aspect whose effect is being tested. A good example would be a drug trial. The sample or group receiving the drug would be the experimental one; and the one receiving the placebo would be the control one.

In many laboratory experiments, it is good practice to have several replicate samples for the test being performed and have both a positive control and a negative control. The results from replicate samples can often be averaged, or, if one of the replicates is obviously inconsistent with the results from the other samples, it can be discarded as being the result of an experimental error (some step of the test procedure may have been mistakenly omitted for that sample). Most often, tests are done in duplicate or triplicate.

A positive control is a procedure that is very similar to the actual experimental test, but it is known from earlier experience to give a positive result. A negative control is known to give a negative result. The positive control confirms that the basic conditions of the experiment were able to produce a positive result, even if none of the actual experimental samples produce a positive result. The negative control demonstrates the base-line result obtained when a test does not produce a measurable positive result; often the value of the negative control is treated as a "background" value to be subtracted from the test sample results. Sometimes, the positive control takes the quadrant of a standard curve.

One example used in teaching laboratories is a controlled protein assay. Students may be given a fluid sample containing an amount of protein that is unknown to them. It is their job to perform a controlled experiment in which they determine the concentration of protein in the fluid sample (called the "unknown sample"). The teaching lab would be equipped with a protein standard solution with a known protein concentration. Students could make several positive control samples containing various dilutions of the protein standard. Negative control samples would contain all of the reagents for the protein assay but no protein. In this example, all samples are performed in duplicate. The assay is a colorimetric assay in which a spectrophotometer can measure the amount of protein in samples by detecting a colored complex formed by the interaction of protein molecules and molecules of an added dye. In the illustration, the results for the diluted test samples can be compared to the results of the standard curve (the blue line in the illustration) in order to determine an estimate of the amount of protein in the unknown sample.

Controlled experiments can be performed when it is difficult to exactly control all the conditions in an experiment. In this case, the experiment begins by creating two or more sample groups that are probabilistically equivalent, which means that measurements of traits should be similar among the groups and that the groups should respond in the same manner if given the same treatment. This equivalency is determined by statistical methods that take into account the amount of variation between individuals and the number of individuals in each group. In fields such as microbiology and chemistry, where there is very little variation between individuals and the group size is easily in the millions, these statistical methods are often bypassed and simply splitting a solution into equal parts is assumed to produce identical sample groups.

Once equivalent groups have been formed, the experimenter tries to treat them identically except for the one variable that he or she wishes to isolate. Human experimentation requires special safeguards against outside variables such as the placebo effect. Such experiments are generally double blind, meaning that neither the volunteer nor the researcher knows which individuals are in the control group or the experimental group until after all of the data has been collected. This ensures that any effects on the volunteer are due to the treatment itself and are not a response to the knowledge that he is being treated.

In human experiments, a subject (person) may be given a stimulus to which he or she should respond. The goal of the experiment is to measure the response to a given stimulus by a test method.

Natural (quasi) experiments

The term "experiment" usually implies a controlled experiment, but sometimes controlled experiments are prohibitively difficult or impossible. In such cases, researchers resort to natural experiments, also called quasi experiments. Natural experiments rely solely on observations of the variables of the system under study, rather than manipulation of just one or a few variables as occurs in controlled experiments. To the degree possible, they attempt to collect data for the system in such a way that contribution from all variables can be determined, and where the effects of variation in certain variables remain approximately constant so that the effects of other variables can be discerned. The degree to which this is possible depends on the observed correlation between explanatory variables in the observed data. When these variables are not well correlated, natural experiments can approach the power of controlled experiments. Usually, however, there is some correlation between these variables, which reduces the reliability of natural experiments relative to what could be concluded if a controlled experiment were performed. Also, because natural experiments usually take place in uncontrolled environments, variables from undetected sources are neither measured nor held constant, and these may produce illusory correlations in variables under study.

Much research in several important science disciplines—including astronomy, geology, paleontology, ecology, meteorology, economics, and political science—relies on quasi-experiments. For example, in astronomy it is clearly impossible, when testing the hypothesis "suns are collapsed clouds of hydrogen," to start out with a giant cloud of hydrogen, and then perform the experiment of waiting a few billion years for it to form a sun. However, by observing various clouds of hydrogen in various states of collapse, and other implications of the hypothesis (for example, the presence of various spectral emissions from the light of stars), we can collect data we require to support the hypothesis. An early example of this type of experiment was the first verification in the 1600s that light does not travel from place to place instantaneously, but instead has a measurable speed. Observation of the appearance of the moons of Jupiter were slightly delayed when Jupiter was farther from Earth, as opposed to when Jupiter was closer to Earth; and this phenomenon was used to demonstrate that the difference in the time of appearance of the moons was consistent with a measurable speed of light.

Observational studies

Observational studies are very much like controlled experiments except that they lack probabilistic equivalency between groups. These types of experiments often arise in the area of medicine where, for ethical reasons, it is not possible to create a truly controlled group. For example, one would not want to deny all forms of treatment for a life-threatening disease from one group of patients to evaluate the effectiveness of another treatment on a different group of patients. The results of observational studies are considered much less convincing than those of designed experiments, as they are much more prone to selection bias. Researchers attempt to compensate for this with complicated statistical methods such as propensity score matching methods (see hierarchy of evidence).

Field experiments

Field experiments are so named in order to draw a contrast with laboratory experiments. Often used in the social sciences, and especially in economic analyses of education and health interventions, field experiments have the advantage that outcomes are observed in a natural setting rather than in a contrived laboratory environment. However, like natural experiments, field experiments suffer from the possibility of contamination: experimental conditions can be controlled with more precision and certainty in the lab.

Quotes

"We have to learn again that science without contact with experiments is an enterprise which is likely to go completely astray into imaginary conjecture." — Hannes Alfven

"Today's scientists have substituted mathematics for experiments, and they wander off through equation after equation, and eventually build a structure which has no relation to reality." — Nikola Tesla

See also

• Astronomy
• Biology
• Chemistry
• Ecology
• Economics
• Geology
• Paleontology
• Physics
• Political science
• Science
• Technology

References

ISBN links support NWE through referral fees

• Anderson, Mark J., and Patrick J. Whitcomb. 2007. DOE Simplified: Practical Tools for Effective Experimentation, 2nd ed. New York: Productivity Press. ISBN 978-1563273445
• Antony, Jiju. 2003. Design of Experiments for Engineers and Scientists. Oxford, UK: Butterworth-Heinemann. ISBN 978-0750647090
• Barrentine, Larry B. 1999. An Introduction to Design of Experiments: A Simplified Approach. Milwaukee: ASQ Quality Press. ISBN 0873894448
• Montgomery, Douglas C. 2005. Design and Analysis of Experiments, 6th ed. Hoboken, NJ: John Wiley & Sons. ISBN 978-0471487357

External links

All links retrieved August 8, 2017.

• Trochim, William M. Experimental Design The Research Methods Knowledge Base, 2nd Edition.