Timeline of experiment design

This is a timeline of experiment design, attempting to describe significant and illustrative events in the history of the field.

Sample questions

The following are some interesting questions that can be answered by reading this timeline:

• What landmark experiments illustrate the evolution of experimental design and its methodological pitfalls?
  • Sort the full timeline by "Event type" and look for the group of rows with value "Experiment". You will see landmark experiments spanning medicine, physics, psychology, ecology, and economics, along with the methodological lessons each one illustrates.
• What are the core statistical and methodological concepts that underpin modern experimental design, and when were they introduced?
  • Sort the full timeline by "Event type" and look for the group of rows with value "Concept". You will see the introduction of foundational statistical ideas as well as more recent methodological concerns.
• Which individuals and institutions pioneered specific experimental design methods, and what techniques did they introduce?
  • Sort the full timeline by "Event type" and look for the group of rows with value "Method". You will see the introduction of a range of design and analysis techniques, from classical randomization schemes to modern computational approaches.
• How have ethical and regulatory frameworks for human experimentation developed over time?
  • Sort the full timeline by "Event type" and look for the group of rows with value "Policy". You will see the emergence of research-ethics codes and regulatory requirements shaping how experiments involving human subjects are conducted and reported.
• Which organizations and institutions have shaped the field of experimental design?
  • Sort the full timeline by "Event type" and look for the group of rows with value "Organization". You will see the founding of academic departments, professional bodies, and international collaborative networks central to the field.
• Other event types not covered by a dedicated question above include Statistical method, Publication, Concept development, Ethical oversight, Recognition, and Literature.

Big picture

Time period	Development summary	More details
Ancient Times – 1700s	Early foundations	Experimental thinking has ancient roots, notably in medicine and agriculture. One of the earliest recorded episodes resembling a clinical trial dates to around 500 BCE, when King Nebuchadnezzar compares a meat-and-wine diet against one of beans and water, observing the latter group appears healthier. Greek physicians like Hippocrates and Galen emphasize observation and rudimentary comparative methods. However, formal experimentation remains rare. The Scientific Revolution in the 1600s shifts focus from authority to empirical investigation. Francis Bacon promotes inductive reasoning and systematic observation, laying philosophical groundwork for modern science. Nonetheless, experiments lack standardized design or statistical rigor. Scientists like Galileo and Boyle use controlled observations, but their methods are more qualitative than quantitative. The period ends with the emergence of probability theory by Pascal, Fermat, and Bernoulli, which begins to offer tools later essential for experimental statistics.
1700s – Late 1800s	Classical period	The Age of Enlightenment brings advances in both scientific methodology and mathematical statistics. Controlled experiments become more common in chemistry, physics, and agriculture. Notably, James Lind conducts one of the first controlled clinical trials in 1747, testing treatments for scurvy. In parallel, Carl Linnaeus systematizes biological classification, influencing observational rigor. Pierre-Simon Laplace and Carl Friedrich Gauss develop early statistical tools such as the normal distribution and least squares estimation, crucial for data analysis. Agricultural trials, especially in Europe, begin to use basic comparison and replication. However, randomness, replication, and formal control are not yet standardized. This period sets the stage for modern experimentation by linking empirical testing with early statistical theory, but experiments still lack formal design structures and reproducibility standards. The idea of a placebo effect—a therapeutic outcome derived from an inert treatment— is already discussed in 18th century psychology
Early 1900s – 1950s	Formalization and statistics	This period marks the birth of modern experimental design. Ronald A. Fisher revolutionizes the field in the 1920s and 1930s with work at Rothamsted Experimental Station in England. Fisher introduces core principles: randomization, replication, and blocking. He develops the analysis of variance (ANOVA) and emphasizes the role of statistical inference in drawing conclusions. His 1935 book, The Design of Experiments, would remain foundational. Experimental design becomes central to agriculture, biology, and later psychology and social sciences. Jerzy Neyman and Egon Pearson introduce hypothesis testing frameworks, further refining experimental methodology. This era institutionalizes statistics as an essential scientific tool and formalizes the structure of experiments, enabling reproducibility and greater objectivity in scientific inference. The need to blind researchers becomes widely recognized in the mid-century.
1960s – present	Modern and computational era	Post-1950s, experimental design expand into diverse fields: psychology, economics, medicine, and engineering. With the rise of computers, simulation, factorial designs, and response surface methodology become widespread. Clinical trials adopt strict protocols, often randomized and double-blind, guided by ethical oversight. In social sciences, randomized controlled trials (RCTs) gain ground, supported by behavioral economics and development studies. The Bayesian approach gains traction, offering alternative inference frameworks. From the 1990s, machine learning and A/B testing transform experimental design in tech industries, allowing real-time, large-scale experimentation. Today, experimental design is deeply intertwined with data science, AI, and evidence-based policy, continually evolving to address high-dimensional data, ethical concerns, and reproducibility challenges across disciplines.

Time period	Development summary	More details
1910s	Concept	Ronald A. Fisher develops the concept of variance as a measure of population variability, laying the foundation for later statistical methods such as analysis of variance (ANOVA).	United Kingdom
1920s	Method	Ronald Fisher develops randomized block design and formal principles of blocking to control variation in experiments.	United Kingdom
1920s	Method	Fisher introduces key principles of modern experimental design, including confounding, randomization, replication, blocking, Latin square designs, and other factorial designs.	United Kingdom
1920s	Method	analysis of variance and maximum likelihood estimation are developed as major statistical tools for analyzing experimental data.	United Kingdom
1920s	Organization	The first academic statistics department is established at Iowa State University under George Snedecor, helping institutionalize statistical experimental methods.	United States
1920s	Method	George Snedecor develops the F-test and F-distribution, which become central tools for decisions in ANOVA-based experimental designs.	United States
1930s	Concept	The size of an experiment is recognized as a key design decision, with sample size linked to precision, significance, and statistical power.	Global
1930s	Concept	The advantages of factorial experimentation over one-factor-at-a-time methods are articulated, especially its economy of resources and ability to estimate interrelationships among factors.	Global
1930s	Field development	Early applications of statistical experimental design begin to appear in industrial settings, extending beyond agriculture.	Global
1930s	Method	Randomization is formalized as the use of random numbers or equivalent devices to assign treatments and reduce bias from extraneous variation.	Global
1930s	Concept	Experimental error is formally distinguished as a major obstacle to precision, arising from extraneous sources of variation beyond the treatments themselves.	Global
1930s	Concept	Bias is distinguished from random error, emphasizing that systematic error can mislead conclusions and often cannot be detected through statistical analysis alone.	Global
1930s	Method	double-blind experiments and placebo controls are recognized as important safeguards against bias in experiments involving subjective or clinical judgments.	Global
1930s	Method	Refinements of experimental technique, such as practice runs, clearer instructions, environmental control, and improved measurement instruments, are emphasized as ways to reduce experimental error.	Global
1930s	Method	blocking is presented as a method for increasing experimental precision by balancing treatments across subjects or units with similar initial characteristics.	Global
1930s	Method	matched pairs design and matched groups designs are used with human subjects as forms of blocking analogous to blocks in agricultural experiments.	Global
1930s	Concept	Core principles of experimental design—randomization, replication, and blocking—are established as fundamental techniques to reduce bias and improve precision in experiments.	United Kingdom
1930s	Method	analysis of covariance is introduced as a more accurate statistical adjustment for initial differences when blocking is not feasible or when further precision is needed.	Global
1930s	Method	factorial experimentation is formalized as a way to investigate several factors simultaneously, improving efficiency and enabling study of interactions among variables.	Global
1940s	Method	George Box and collaborators develop early central composite design methods for response surface optimization.	United Kingdom
1940s	Method	sequential analysis emerges during World War II to improve efficiency in military experimentation and decision-making.	United States
1940s	Method	Development of orthogonal designs and Latin squares as structured experimental layouts for controlling variation in experiments.	United Kingdom
1950s	Method	W. Edwards Deming promotes statistical quality control in Japan, initiating the quality revolution and widespread industrial use of experimental design.	Japan
1950s	Method	Statistical methods and experimental design spread rapidly through the medical literature, extending techniques developed for agriculture into biomedical research.	Global
1950s	Method	Factorial experimental designs become widely used in industrial experimentation to study multiple variables simultaneously	United States
1950s	Method	Control groups are emphasized as essential in evaluatory studies for distinguishing treatment effects from other changes over time.	Global
1950s–1970s	Field development	Statistical experimental design expands from agriculture into industrial applications, particularly in chemical and process industries.	Global
1950s–1980s	Field development	Response surface methods and statistical design techniques spread across chemical and process industries, especially in research and development.	Global
Mid-20th century	Method	Standardized procedural steps for experimental design are established, including problem definition, variable selection, factor identification, experimental execution, and statistical analysis.	Global}
Mid-20th century	Method	analysis of variance (ANOVA) becomes the central statistical framework for analyzing experimental data and guiding experimental design decisions.	Global
1960s	Method	Japanese industries adopt experimental design and statistical quality control, leading to major improvements in manufacturing quality and global competitiveness.	Japan
1960s	Experiment	randomized controlled trial becomes the gold standard in clinical research, replacing anecdotal medical evidence with controlled experimental methods.	United States
1960s–1970s	Concept	Industrial experiments are recognized as distinct due to immediacy and sequential learning, enabling rapid iteration and adaptive experimentation.	Global
1970s	Method	Multi center clinical trials become standard, enabling large-scale randomized evaluation across diverse populations	Global
1970s–1980s	Method	Genichi Taguchi develops robust parameter design methods to reduce variability and improve product quality.	Japan
1970s–1990	Method	Quality improvement methodologies such as Total Quality Management (TQM) and Continuous Quality Improvement (CQI) integrate experimental design into industrial processes.	Global
Late 1970s	Method	Genichi Taguchi promotes robust parameter design, emphasizing quality improvement and reduction of variability under noise conditions using orthogonal arrays and fractional factorial designs.	Japan
Late 1970s–1980s	Field development	Growing interest in quality improvement in Western industry leads to broader adoption of experimental design methods.	Global
1980s	Method	Quality engineering approaches emphasize robustness and variance reduction in manufacturing processes	Japan
1980s	Method	Meta analysis emerges as a method to systematically combine results from multiple experiments	Global
1980s	Field development	Designed experiments become widely adopted in manufacturing industries (automotive, aerospace, electronics), driven by quality improvement initiatives.	Global
1980s	Method	Large-scale randomized trials such as the ISIS heart studies demonstrate the feasibility of very large clinical experiments.	Global
Late 1980s	Concept	Critical evaluation of Taguchi methods leads to refinement of statistical foundations and development of alternative robust design methodologies.	Global
1990s	Method	Six Sigma integrates statistical experimental design into industrial quality control and process improvement,	United States
1990s	Method	optimal design approaches gain prominence, focusing on maximizing information and efficiency under resource constraints, supported by advances in computing.	Global
1990s	Field development	Designed experiments expand into new domains including services, finance, and government operations.	Global
1990s	Method	The modern era of experimental design begins, driven by globalization and economic competition, expanding DOE across industries and services.	Global
1990s–present	Field development	The Optimal Design Era emerges, characterized by widespread use of computational algorithms, software tools, and expansion of DOE across diverse sectors.	Global
2000s	Method	Online randomized controlled experiments (A/B testing) become standard in internet companies for product and interface optimization, enabling continuous data-driven decision making.	United States
2000s	Method	Bradley Jones introduces Bayesian D-optimal design approaches, incorporating prior knowledge into experimental planning.	United States
2000s	Method	JMP (software)'s Custom Design platform, part of SAS Institute's design-of-experiments software, enables flexible, computer-generated optimal experimental designs tailored to specific problems; constraint-handling features for the platform are progressively extended starting with JMP 6.	United States
2000s	Method	Advances in supersaturated designs and split-plot designs improve efficiency in experiments with many factors and constraints.	United States
2000s–present	Method	Development of group-orthogonal supersaturated designs (GO-SSDs) improves factor screening when variables exceed experimental runs.	United States
2010s	Method	Continuous A/B testing systems enable rapid, large-scale experimentation in digital platforms, allowing ongoing optimization based on user behavior data.	Global
Late 20th century	Application	Experimental design expands beyond agriculture into engineering, business, and scientific research, becoming a general-purpose methodology.	Global

Full timeline

Year	Event type	Details	Geographical location
ca. 500 BCE	Experiment	According to the Book of Daniel, Daniel and his companions request a ten-day trial in which they consume only vegetables and water, while other youths at the royal court continue eating King Nebuchadnezzar's rich diet; the results of the two regimens are compared, and the steward acts based on the observable outcome. Several scholars of ancient literature and the history of science have cited this account as one of the earliest descriptions of a controlled comparative test, and biblical historian David C. Lindberg specifically includes it among ancient examples where a trial is deliberately arranged to evaluate competing conditions.	Babylon (Mesopotamia)
ca. 500 BCE	Experiment	According to the Book of Daniel, Daniel and his companions request a ten-day trial in which they consume only vegetables and water, while other youths at the royal court continue eating the king's rich diet; the results of the two regimens are compared, and the steward acts based on the observable outcome. Several scholars of ancient literature and the history of science have cited this account as one of the earliest descriptions of a controlled comparative test, and biblical historian David C. Lindberg specifically includes it among ancient examples where a trial is deliberately arranged to evaluate competing conditions.	Babylon (Mesopotamia)
ca. 400 BCE	Concept	Hippocrates promotes systematic observation and naturalistic explanations in medicine, emphasizing empirical study over supernatural interpretations.	Greece
ca. 587 AD	Concept	Indian astronomer-mathematician Varahamihira, in his text Brhat Samhita, describes what is considered one of the earliest datable applications of combinatorial design: a method for making perfumes by selecting 4 substances from 16 available substances using a magic square arrangement.	India
c. 1021	Method	Arab polymath Ibn al-Haytham completes the Book of Optics, applying a methodical, inductive-experimental approach to optical and mathematical problems inherited from Ptolemy — emphasizing self-criticality, reliance on visible experimental results over inherited authority, and rigorous skepticism toward earlier scholars' conclusions. He explicitly instructs that a researcher should "make himself an enemy of all that he reads" and interrogate it from every angle, while remaining alert to his own susceptibility to prejudice and leniency. Historians of science regard Ibn al-Haytham as one of the first scholars to systematically apply controlled, inductive experimentation to derive knowledge, prefiguring elements of the modern scientific method by roughly six centuries.	Egypt
1025 AD	Concept	Persian polymath Avicenna completes The Canon of Medicine, a major medical encyclopedia synthesizing Greco-Roman and Islamic knowledge. The work describes diseases, treatments, and hundreds of drugs, and outlines principles for experimentally testing medicines, including reproducibility of results, shaping medical education in both the Islamic world and Europe for centuries.	Persia
1620	Concept	Francis Bacon publishes Novum Organum, proposing a new empirical approach to scientific inquiry. He argues that knowledge should be based on systematic observation and inductive reasoning rather than tradition or pure logic, helping establish methodological foundations for the modern scientific method and experimental science.	England
1648	Concept	Flemish physician and chemist Jan Baptist van Helmont, in his posthumously published Ortus Medicinae, proposes what historians later identify as the first documented design for a randomized comparative trial: dividing 200–500 poor patients suffering from fevers or pleurisy into two groups by lot, treating one group with the standard Galenic methods of bloodletting and purging and the other without them, and comparing the resulting death counts between groups. There is no evidence the trial was ever actually conducted, but the proposal anticipates by nearly three centuries the logic of random allocation between a treatment and a comparison group.	Flanders
1676	Experiment	Danish astronomer Ole Rømer demonstrates that light does not travel instantaneously but has a measurable finite speed, by observing that the apparent timing of Jupiter's moons' eclipses was delayed when Jupiter was farther from Earth compared to when it was closer — an early landmark example of a natural experiment, relying on observation of a system as it naturally occurs rather than direct manipulation of variables.	Denmark
1683	Method	Robert Boyle solidifies his reputation as a pioneer of modern experimental science by publishing New Experiments and Observations Touching Cold. During this period, Boyle and his contemporaries, including Robert Hooke, emphasize that chemical understanding must be grounded in controlled experimentation, precise documentation of apparatus, and the replication of results.	England
1700	Concept	Korean mathematician Choi Seok-jeong is the first to publish an example of Latin squares of order nine, in order to construct a magic square, predating Leonhard Euler by 67 years. Latin squares are used in combinatorics and in experimental design.	Korea
1710	Concept	John Arbuthnot publishes "An Argument for Divine Providence, taken from the constant regularity observed in the births of both sexes" in Philosophical Transactions of the Royal Society, examining London birth records for each of 82 years from 1629 to 1710 and finding that male births exceeded female births in every single year. Applying what becomes known as the sign test, Arbuthnot calculates that the probability of this occurring by chance alone (if male and female births were equally likely) is 0.582 — an astronomically small figure — and concludes the pattern reflects design rather than chance. The paper is widely credited as the earliest recorded use of statistical hypothesis testing, predating the formal apparatus of significance testing by over two centuries.	England
1713	Concept	Jacob Bernoulli formulates the law of large numbers, establishing convergence of sample averages to expected values.	Switzerland
1747	Experiment	Scottish doctor James Lind conducts one of the earliest controlled clinical trials when investigating the efficacy of citrus fruit in cases of scurvy, dividing twelve scurvy patients, whose "cases were as similar as I could have them", into six pairs — without randomization, but with an early attempt at controlling for similar starting conditions. Each pair is given a different remedy. According to Lind's 1753 Treatise on the Scurvy in Three Parts Containing an Inquiry into the Nature, Causes, and Cure of the Disease, Together with a Critical and Chronological View of what has been Published of the Subject, the remedies were: one quart of cider per day, twenty-five drops of elixir vitriol (sulfuric acid) three times a day, two spoonfuls of vinegar three times a day, a course of sea-water (half a pint every day), two oranges and one lemon each day, and electuary, (a mixture containing garlic, mustard, balsam of Peru, and myrrh). Lind would note that the pair who had been given the oranges and lemons were so restored to health within six days of treatment that one of them returned to duty, and the other was well enough to attend the rest of the sick.	United Kingdom
1772	Concept	Scottish physician William Cullen, then the most influential medical lecturer in the Anglo-American world at the University of Edinburgh, becomes the first person known to use the word "placebo" in a medical sense, in his manuscript clinical lectures. Cullen prescribes a "placebo" — typically a weak but genuinely active substance such as mustard or Dover's powder — to a patient he considers "absolutely incurable," stating he does so "in pure placebo" while trying, "even in employing placebos," "to give what would have a tendency to be of use to the patient." For Cullen, a placebo is defined less by its chemical inertness than by the physician's lack of curative intent — what later historians term an "active placebo," distinct from the inert (bread- or sugar-pill) sense the term would come to carry by the early 20th century. The word itself derives from the Latin Placebo Domino ("I shall please the Lord," Psalm 116:9), used since the medieval period to name the Vespers for the Dead, and had by Chaucer's time already acquired a secular, pejorative sense — a flatterer who "sings placebos."	Scotland
1778	Concept	Pierre-Simon Laplace examines birth statistics from almost half a million births in "Mémoire sur les probabilités," finding a consistent excess of male births over female births and concluding through probabilistic calculation that the excess is a real, though unexplained, effect rather than a product of chance — extending the sex-ratio hypothesis-testing tradition begun by John Arbuthnot in 1710 to a substantially larger dataset and a more developed probabilistic apparatus.	France
1782	Concept	Leonhard Euler publishes "Recherches sur une nouvelle espèce de quarrés magiques" in the Verhandelingen of the Zeeland Society of Sciences (first presented to the Imperial Academy of Sciences of St. Petersburg in 1779), using Latin characters as symbols in his arrays and thereby giving rise to the name "Latin square." Though Korean mathematician Choi Seok-jeong had published an example of order-nine Latin squares 67 years earlier, Euler's work begins the general mathematical theory of Latin squares and their combinatorial properties.	Russia
1780	Concept	Franz Mesmer, then already well known in Paris, refuses a request from the Paris Faculty of Medicine's Joseph-Marie-François de Lassone to have his therapeutic method scrutinized through new cures on unfamiliar patients, arguing instead that his prior "cures" should be taken as an objective matter of record — a position that both 1784 Royal Commissions would later reject in designing their own investigations. Separately, Mesmer proposed principles for objectively testing his method that anticipate elements of controlled experimental design.	France
1784	Experiment	The first blinded experiment is conducted by the French Academy of Sciences to investigate the claims of mesmerism as proposed by Franz Mesmer. In the experiment, researchers blindfolded mesmerists and asked them to identify objects that the experimenters had previously filled with "vital fluid". The subjects are unable to do so.	France
1784	Experiment	Two independent French Royal Commissions — a nine-member "Franklin Commission" (four physicians from the Paris Faculty of Medicine and five scientists from the Royal Academy of Sciences, including Benjamin Franklin and Antoine Lavoisier) and a five-member "Society Commission" from the Royal Society of Medicine — are appointed by Louis XVI to investigate Charles d'Eslon's claims for the physical existence of "animal magnetism." Lavoisier designs the Franklin Commission's investigative protocol: rather than assessing long-term cures, the commissioners test the "momentary" physiological effects of magnetization under conditions systematically varying whether subjects are genuinely or falsely told they are being magnetized, and whether investigators and subjects are blindfolded. In one representative test, a "sensitive" subject who believed he had been led to a magnetized tree fainted at the foot of the wrong tree; in another, a subject who drank ordinary water believing it magnetized displayed magnetic symptoms. Both Commissions conclude that d'Eslon's magnetic fluid does not exist and that all observed effects are attributable to touch, imagination, and imitation. The Franklin Commission's report, presented 11 August 1784 and printed in at least 20,000 copies, is widely regarded by historians of science as one of the earliest classic examples of a systematic controlled trial, notable in particular for its use of literal blindfolding of both investigators and subjects.	France
1785	Publication	The word "placebo" appears in medical print for the first time, in the second edition of George Motherby's New Medical Dictionary, which defines it as "a commonplace method or medicine"; the 1775 first edition had contained no such entry. Historian of the placebo Arthur Shapiro would later call the reasons for the term's entry into medicine "largely unknown" — a gap subsequent research traces to William Cullen's clinical teaching over the preceding decade.	England
1785	Publication	The word "placebo" appears in medical print for the first time, in the second edition of George Motherby's New Medical Dictionary, which defines it as "a commonplace method or medicine"; the 1775 first edition had contained no such entry. The word had entered English via the Vulgate's rendering of Psalm 116:9 as "Placebo Domino" ("I shall please the Lord"), likely chosen by Jerome for its metre rather than as an accurate translation of the Hebrew; by the 13th century "Placebo" named the Vespers for the Dead, and by Chaucer's time had acquired the pejorative sense of a flattering sycophant. Historian of the placebo Arthur Shapiro would later call the reasons for the term's entry into medicine "largely unknown" — a gap subsequent research traces to William Cullen's clinical teaching over the preceding decade.	England
1796	Experiment	English physician Edward Jenner inoculates eight-year-old James Phipps with cowpox matter taken from dairymaid Sarah Nelms's lesions on 14 May 1796, then deliberately exposes him to smallpox matter that July; Phipps develops no disease, and Jenner concludes the boy is protected. Jenner expands his findings into An Inquiry into the Causes and Effects of the Variolae Vaccinae, privately published in 1798 after the Royal Society rejected his initial 1797 submission. A landmark early medical experiment, it tests a single subject without any untreated comparison group to rule out chance or an unrelated cause for the observed protection — the same limitation found in Lady Mary Wortley Montagu's earlier 1721 variolation trial on six condemned prisoners at Newgate, whose survival and subsequent immunity likewise had no control group to benchmark against.	England
1798	Statistical method	German mathematician Carl Friedrich Gauss develops the mathematical foundations of the method of least squares. Years later, the method would enable astronomers to predict the orbit of the asteroid Ceres after its discovery by Giuseppe Piazzi and help Franz Xaver von Zach successfully relocate it.	Germany
1799	Concept	English physician John Haygarth, working in Bath, tests the popular but ultimately fraudulent "Perkins tractors" — a pair of metal rods invented by Elisha Perkins and marketed as curing pain and disease through "animal magnetism" — by comparing their effects against plain wooden rods disguised to look identical. Finding the wooden sham tractors work just as well as the metal originals, Haygarth demonstrates the "cure" comes from patient expectation rather than the device itself, in what historians regard as one of the earliest uses of a placebo control in a single-blind clinical trial.	England
1811	Concept	The word "placebo" is defined in Hooper's Medical Dictionary as "any medicine adapted more to please than benefit the patient" — a definition markedly closer to the modern sense of the term than Motherby's 1785 entry ("a commonplace method or medicine"), and one that explicitly separates a placebo's pleasing effect from any genuine therapeutic benefit for the first time in print. Physician and pharmacologist Jeff Aronson later speculates the shift may reflect a turn-of-the-century reinterpretation of the word's derivation, a form of medical snobbery toward "popular" remedies, or a reversion to the term's older, sycophantic connotation.	England
1815	Concept	An article on optimal designs for polynomial regression is published by Joseph Diaz Gergonne.	France
1817	Experiment	The first blinded experiment recorded outside of a scientific setting compares the musical quality of a Stradivarius violin to one with a guitar-like design. A violinist plays each instrument while a committee of scientists and musicians listen from another room so as to avoid prejudice.	France
1827	Method	Pierre-Simon Laplace uses least squares methods to address analysis of variance problems regarding measurements of atmospheric tides.	France
1835	Experiment	An early example of a double-blind protocol is the Nuremberg salt test performed by Friedrich Wilhelm von Hoven, Nuremberg's highest-ranking public health official.	Germany
1835	Concept	Belgian statistician Adolphe Quetelet introduces the concept of the “average man” (l’homme moyen), arguing that human physical and social traits follow statistical distributions. By applying probability and quantitative analysis to crime, mortality, and demographics, he helps establish statistical approaches in sociology, demography, and the emerging social sciences.	Belgium
1843	Concept	Irish doctor James Henry proposes principles for conducting a controlled trial comparing cold-water therapy against sulphur treatment for scabies — an early articulation of controlled-trial methodology predating the formalization of randomized experimental design by nearly a century.	Ireland
1844	Concept	Wesley S. B. Woolhouse poses the problem of what becomes known as Steiner triple systems as Prize Question #1733 in the Lady's and Gentlemen's Diary — asking for a systematic partition of elements into triples such that every pair of elements appears together in exactly one triple.	England
1847	Method	English clergyman and mathematician Thomas Kirkman solves Wesley S. B. Woolhouse's 1844 prize problem in "On a Problem in Combinations," published in The Cambridge and Dublin Mathematical Journal — the first solution to what becomes known as the Steiner triple system problem, three years before Kirkman posed his own, more elaborate variant asking for resolvable systems (later known as Kirkman's schoolgirl problem).	England
1850	Concept	Thomas Kirkman poses the problem in the Lady's and Gentleman's Diary of 1850: "Fifteen young ladies of a school walk out three abreast for seven days in succession: it is required to arrange them daily so that no two shall walk abreast more than once." This becomes known as the Fifteen Schoolgirls Problem, or Kirkman's schoolgirl problem. A solution to this recreational puzzle is equivalent to a resolvable balanced incomplete block design with 15 points, block size 3, and λ = 1 — an early landmark in the theory of resolvable designs. Arthur Cayley published a solution first, followed by Kirkman's own (which he had, of course, already worked out before posing the question); James Joseph Sylvester also studied the problem and later disputed priority with Kirkman.	England
1853	Concept	Jakob Steiner independently reintroduces triple systems in "Combinatorische Aufgabe," published in Crelle's Journal, unaware of Thomas Kirkman's 1847 solution to the same problem. Because Steiner's paper becomes more widely known within the mathematical community than Kirkman's earlier work, the systems come to be named Steiner systems in his honor rather than Kirkman's, despite Kirkman's priority.	Germany
1854	Experiment	English physician John Snow investigates a cholera outbreak that kills over 500 people in ten days in London's Soho district, mapping cholera deaths against residence and finding that victims were disproportionately concentrated among those drinking from a shared pump on Broad Street. He also finds that brewery workers and workhouse residents nearby, who drew water from separate wells rather than the Broad Street pump, were largely spared. Snow persuades local officials to remove the pump's handle; the outbreak, already declining as residents fled the area, ceases entirely within days. The bacterial mechanism of cholera transmission would not be confirmed until Robert Koch's later work.	England
1860	Method	German psychologist and physicist Gustav Theodor Fechner makes a groundbreaking contribution to the development of experimental psychology with the publication of Elements of Psychophysics. In this work, he seeks to test and justify the relationship between physical stimuli and the sensations they produce. Fechner proposes that mental experiences can be quantified by linking them to measurable physical changes, laying the foundation for psychophysics. Using experimental data, he formulates mathematical laws—most notably the Weber-Fechner law—that describe how perceived intensity varies with stimulus magnitude. His efforts help establish psychology as a quantitative science rooted in empirical observation.	Germany
1861	Experiment	French chemist and microbiologist Louis Pasteur conducts controlled experiments demonstrating that microorganisms originate from airborne particles rather than spontaneous generation. In his memoir examining this doctrine, he shows that sterilized broth remain free of life unless exposed to contaminated air, helping establish the foundations of modern microbiology and germ theory.	France
1865	Concept	French physiologist Claude Bernard publishes Introduction to the Study of Experimental Medicine, advocating that researchers should not know the hypothesis being tested while making observations — a recommendation that directly contradicted the prevailing Enlightenment-era view that valid scientific observation required a well-educated, informed observer.	France
1876	Concept	American scientist Charles S. Peirce contributes the first English-language publication on an optimal design for regression models.	United States
1877	Concept	Charles Sanders Peirce formalizes inquiry as a structured experimental process: hypotheses (abduction) generate testable predictions (deduction), which are evaluated through observation and experiment (induction). This integrates reasoning with empirical testing, establishing a cyclical, self-correcting model of experiment design focused on hypothesis testing and iterative refinement, developed across his "Illustrations of the Logic of Science" series.	United States
1879	Organization	Experimental psychology emerges as a modern scientific discipline in Germany with the establishment of the first experimental laboratory by Wilhelm Wundt at the University of Leipzig. This marks a pivotal moment in the history of psychology, as Wundt seeks to separate psychology from philosophy by applying rigorous scientific methods. He introduces a mathematical and experimental approach to studying the human mind, emphasizing observation, measurement, and controlled experimentation. Wundt's work lays the foundation for psychology as an empirical science, influencing future researchers and schools of thought.	Germany
1879–1880	Experiment	The Milwaukee Academy of Medicine conducts a trial testing a homeopathic remedy against a sugar-pill control, in which both the patients and the physician-experimenters (apart from those managing the code) are masked as to which substance had been administered — a design later historians describe in modern terms as "double-blind." The trial, reported in the Homoeopathic Times as the "Final report of the Milwaukee test of the thirtieth dilution," emerges from the broader 19th-century dispute between homeopathy and orthodox medicine, in which both sides increasingly turned to masked assessment to adjudicate rival therapeutic claims.	United States
1882	Concept	In his published lecture at Johns Hopkins University, Peirce introduces experimental design with these words: Logic will not undertake to inform you what kind of experiments you ought to make in order best to determine the acceleration of gravity, or the value of the Ohm; but it will tell you how to proceed to form a plan of experimentation. [....] Unfortunately practice generally precedes theory, and it is the usual fate of mankind to get things done in some boggling way first, and find out afterward how they could have been done much more easily and perfectly.	United States
1884	Organization	Frederick Akbar Mahomed, who worked at Guy's Hospital in London and separated chronic nephritis with secondary hypertension from what would later be termed essential hypertension, founds the Collective Investigation Record for the British Medical Association — an organization collecting data from physicians practicing outside the hospital setting, considered the precursor of modern collaborative, multi-site clinical trials.	England
1885	Concept	Analysis of variance. An eloquent non-mathematical explanation of the additive effects model becomes available.	United Kingdom
1887	Experiment	Albert A. Michelson and Edward W. Morley conduct an interferometry experiment designed to detect Earth's motion through the hypothesized luminiferous aether by measuring expected variations in the speed of light in different directions. The experiment fails to detect the predicted aether drift, instead measuring a drift far too small to account for the theoretically expected effect and generally attributed to noise — becoming one of the most famous null results in the history of science. The unexpected absence of a positive result would go on to contribute significantly to the development of special relativity.	United States
1880s	Experiment	Charles Sanders Peirce and Joseph Jastrow introduce randomized experiments in the field of psychology.	United States
1894	Concept	French psychologist Alfred Binet publishes "La psychologie de la prestidigitation" in the Revue des Deux Mondes, examining how magicians exploit blind spots in human attention and perception to perform their tricks — among the earliest documented scientific studies of magic (illusion) as a subject in its own right. An English translation, "Psychology of Prestidigitation," appears in 1896 in the Smithsonian Institution's annual report. The field of magic as a subject of scientific study would not be substantially revived until the early 21st century.	France
1897	Experiment	Norman Triplett conducts one of the first social psychology experiments on cyclist performance.	United States
1900	Method	The P-value is first formally introduced by Karl Pearson, in his Pearson's chi-squared test, using the chi-squared distribution and notated as capital P. Since then, P-values would become the preferred method to summarize the results of medical articles.	United Kingdom
1900	Concept	French mathematician Gaston Tarry proves that Leonhard Euler's 36 officers problem has no solution, confirming Euler's own 1782 conjecture that no pair of orthogonal Latin squares of order six exists. Tarry's proof, published in Comptes Rendus de l'Association Française pour l'Avancement des Sciences, works by exhaustively checking all essentially distinct Latin squares of order six — a set Tarry reduces to 9,408 cases — and confirming that no two of them are orthogonal to each other. The conjecture would later prove false for every other order of the form 4k+2 above 6, as R. C. Bose, S. S. Shrikhande, and E. T. Parker would show in 1959–1960.	France
1900	Concept	The United States Army's Yellow Fever Commission, led by Walter Reed, conducts mosquito-transmission experiments on yellow fever at a camp in Cuba using American and Spanish volunteers. The study marks the first documented instance of test subjects signing a consent form before participating in a medical experiment, with researchers stating they wanted volunteers to understand the hazards involved beforehand. None of the volunteers died, and the experiments confirm that mosquitoes transmit the disease.	Cuba
1903	Concept	American physician Richard Clarke Cabot concludes that the placebo should be avoided because it is deceptive.	United States
1904	Concept	English psychologist Charles Spearman becomes the first psychologist to discuss common factor analysis, in his paper "General Intelligence, Objectively Determined and Measured." Spearman observes that schoolchildren's scores across a wide variety of seemingly unrelated subjects are positively correlated, leading him to postulate a single general mental ability, "g," underlying human cognitive performance — the origin of factor analysis as a statistical technique, though Spearman's paper provides few methodological details and is concerned only with single-factor models.	England
1907	Experiment	German philosopher and psychologist Carl Stumpf, together with his assistant Oskar Pfungst, investigates the claims surrounding "Clever Hans," an Orlov Trotter horse whose owner, Wilhelm von Osten, claimed could perform arithmetic and other cognitive tasks. Ruling out deliberate fraud, Pfungst determines the horse answers correctly 89% of the time when the questioner knows the answer, but only 6% of the time when the questioner does not — and shows that questioners unconsciously tensed their posture and expression as the horse's taps approached the correct answer, providing an unintentional cue the horse had learned to use as a stopping signal. The episode becomes the classic illustration of the observer-expectancy effect and a foundational argument for blinded experiment the questioner or experimenter, not just the subject, from knowledge that could unconsciously bias results.	Germany
1907	Experiment	The first study recorded to have a blinded researcher is conducted by W. H. R. Rivers and H. N. Webber to investigate the effects of caffeine.	United Kingdom
1908	Method	British statistician William Sealy Gosset, working at Guinness, introduces the Student’s t-distribution to analyze small sample data when population variance is unknown. Publishing under the pseudonym “Student” in the journal Biometrika, he provides a statistical method widely used for inference with limited data.	United Kingdom
1908–1909	Concept	Irish economist and statistician Francis Ysidro Edgeworth publishes a series of papers "On the Probable Errors of Frequency-Constants" that anticipate several elements of what would become known as Fisher information, over a decade before Fisher's own 1922 formal treatment — work later recognized by statisticians and historians of statistics as an important, underacknowledged precursor.	Ireland
1911–1918	Experiment	German physician Adolf Bingel conducts a large-scale blinded, controlled comparison of diphtheria treatments, alternately assigning 937 patients between diphtheria antitoxin serum and plain horse serum; all patients and participating physicians other than Bingel himself remain unaware of each patient's allocation. Published in 1918 in the Deutsches Archiv für klinische Medizin, the trial becomes an influential early example of blinded assessment applied at substantial scale in mainstream (rather than "irregular") medicine, feeding into a wider German tradition of blind clinical assessment.	Germany
1918	Concept	English statistician Ronald Fisher introduces the term variance and proposes its formal analysis in his article The Correlation Between Relatives on the Supposition of Mendelian Inheritance.	United Kingdom
1918	Method	Kirstine Smith publishes a major study in Biometrika analyzing the statistical precision of polynomial regression estimates. She derives principles for choosing observation points that minimize estimation variance, introducing foundational methods for optimal experimental design in polynomial models and influencing later developments in statistical design theory.	Denmark
1918–1940s	Method	Ronald A. Fisher and collaborators establish the foundations of modern experimental design in agricultural research, introducing factorial designs and analysis of variance (ANOVA).	United Kingdom
1919	Organization	Ronald A. Fisher is hired as a statistician at the Rothamsted Experimental Station, where his work on poorly designed agricultural data helps trigger the modern statistical approach to experimental design.	United Kingdom
1919	Method	R. A. Fisher at the Rothamsted Experimental Station in England starts developing modern concepts of experimental design in the planning of agricultural field experiments.	England
1920	Concept	English physician T.C. Graves is the first to define and discuss the "placebo effect" in print, in a commentary published in The Lancet on a case of hystero-epilepsy with delayed puberty. Graves describes "the placebo effects of drugs" as occurring in cases where "a real psychotherapeutic effect appears to have been produced" despite the absence of any specific pharmacological action — using the phrase decades before Henry K. Beecher's more widely cited 1955 paper of the same name.	England
1921	Method	Ronald Fisher publishes his first application of the analysis of variance.	United Kingdom
1922	Concept	Ronald Fisher formally defines what becomes known as Fisher information in "On the Mathematical Foundations of Theoretical Statistics," measuring the amount of information an observable random variable carries about an unknown parameter of the distribution generating it. Fisher information becomes foundational to the theory of maximum-likelihood estimation and to optimal design of experiments, where maximizing the Fisher information of a design corresponds to minimizing the variance of parameter estimates obtainable from it.	United Kingdom
1923	Field development	The first randomization model is published in Polish by Jerzy Neyman.	Poland
1923	Method	Ronald A. Fisher, together with Winifred Mackenzie, publishes "Studies in Crop Variation II," extending his 1921 "Studies in Crop Variation I" (which had partitioned the variation of a time series into annual and slow-deterioration components) to study yield variation across plots sown with different potato varieties and subjected to different fertiliser treatments — early applied work at Rothamsted Experimental Station that would feed directly into the formal analysis-of-variance framework Fisher developed over the following years.	United Kingdom
1924–1925	Experiment	Political scientist Harold Gosnell conducts an early field experiment on voter participation in Chicago, testing whether nonpartisan mailings encouraging registration and voting increased turnout — widely regarded as one of the earliest field experiments in political science, using randomization outside a laboratory setting to study a real-world civic behavior.	United States
1925	Publication	British statistician Ronald Fisher publishes Statistical Methods for Research Workers, developing tests of statistical significance, proposing the 0.05 threshold for hypothesis testing, and popularizing analysis of variance as a method for partitioning variation in experimental data. The book would later be identified as a seminal, and controversial, influence — some retrospective critics tracing aspects of the modern replication crisis in science to conventions it popularized.	United Kingdom
1926	Publication	John Russell (agricultural scientist) publishes "Field Experiments: How They Are Made and What They Are", summarizing contemporary practices and principles of experimental design in agricultural research.	United Kingdom
1926	Experiment	Janet Lane-Claypon publishes a retrospective investigation into breast cancer risk factors in the United Kingdom, widely regarded as the first major case–control study — comparing women with the disease against a control group without it to isolate differences in reproductive history. Her data provide the first epidemiologic evidence that low fertility (fewer pregnancies, later age at first birth, and shorter duration of breastfeeding) increases breast cancer risk, findings a 2010 statistical reanalysis would confirm as consistent with modern epidemiologic evidence. The study is replicated in the United States in 1931 by J.M. Wainwright.	United Kingdom
1926	Concept	Ronald Fisher argues in "The Arrangement of Field Experiments" that studying multiple factors simultaneously in "complex" (factorial) designs is more efficient than the traditional one-factor-at-a-time approach, writing that "Nature... will best respond to a logical and carefully thought out questionnaire; indeed, if we ask her a single question, she will often refuse to answer until some other topic has been discussed" — arguing that a well-designed factorial experiment can determine the effects of several factors, and their interactions, using no more trials than would be needed to determine just one factor's effect alone.	United Kingdom
1926	Concept	Soviet mathematician Sergei Natanovich Bernstein introduces the "blocks method" in probability theory, splitting a sample into blocks separated by smaller subblocks so the blocks can be treated as approximately independent — a technique later used to prove limit theorems for sums of dependent random variables and applied in extreme value theory. This is a distinct lineage from Fisher's contemporaneous statistical blocking in experimental design, converging on the same term for a different underlying idea.	Soviet Union
1930	Concept	Torald Sollmann, head of the American Medical Association's Council on Pharmacy and Chemistry, advocates that clinical drug testing adopt the rigor of other scientific disciplines, explicitly using the terms "comparative method," "double-blind procedure," and "blind test" in doing so — among the earliest documented uses of "double-blind" terminology in American medicine, predating its wider adoption by German clinical pharmacology under Paul Martini two years later.	United States
1931–1935	Method	American psychologist Louis Leon Thurstone develops common factor analysis with multiple factors across two papers in the early 1930s, summarized in his 1935 book The Vectors of Mind: Multiple-Factor Analysis for the Isolation of Primary Traits. Thurstone introduces several concepts still central to the technique, including communality, uniqueness, and factor rotation, and advocates for achieving "simple structure" in factor solutions through appropriate choice of rotation.	United States
1932	Method	German clinical pharmacologist Paul Martini publishes Methodenlehre der Therapeutischen Untersuchung ("Methodology of Therapeutic Investigation"), codifying blind assessment as a formal methodological principle for evaluating treatments — consolidating the German tradition of blinded experimentation that had developed since Adolf Bingel's 1918 diphtheria trial, shortly before the Nazi period disrupts German academic medicine.	Germany
1932	Experiment	American physician Harry Gold, an early and influential advocate for masked assessment in American clinical pharmacology, tests xanthine against a lactose placebo in a single-blind trial for cardiac pain — a milestone experiment in the American adoption of placebo-controlled methodology, conducted the same year Paul Martini codifies double-blind testing in Germany.	United States
1933	Concept	Jerzy Neyman and Egon Pearson publish "On the Problem of the Most Efficient Tests of Statistical Hypotheses" in Philosophical Transactions of the Royal Society A, part of a series of papers begun in 1928 that formalizes the statistical hypothesis test as a proposed refinement of Ronald Fisher's significance-testing approach. The papers supply much of the standard terminology still used today, including the term "alternative hypothesis" and the H0/H1 notation for the null and alternative hypotheses, and also introduce the simple/composite hypothesis distinction. Fisher and Neyman would go on to quarrel over the relative merits of their competing formulations until Fisher's death in 1962; the two approaches were later merged into a single hybrid framework by textbook writers and practitioners without direct input from either originator.	United Kingdom
1933	Method	English mathematician Raymond Paley publishes "On orthogonal matrices" in the Journal of Mathematics and Physics, introducing two methods for constructing infinite families of Hadamard matrix using Galois fields — one for orders that are a multiple of 4 where q = N−1 is an odd prime power, and a second for orders N = 2q+2 under related conditions. The two-level orthogonal arrays derived from Paley's Hadamard matrices become known as Paley designs, and his construction underlies the Plackett–Burman designs Robin Plackett and James P. Burman would introduce in 1946.	United Kingdom
1933	Experiment	At the Royal London Hospital, William Evans and Clifford Hoyle conduct an early comparative trial of drugs used continuously to treat angina pectoris, testing 90 patients by comparing the outcomes of active drug treatment against a dummy ("placebo") treatment within the same trial. Finding no significant difference between the two, the researchers conclude that the drugs tested exert no specific pharmacological effect on the condition — an early instance of the placebo-controlled comparative design that Harry Gold and colleagues would apply at much larger scale four years later.	England
1935	Method	Jerzy Neyman, with K. Iwaszkiewicz and St. Kolodziejczyk, publishes "Statistical Problems in Agricultural Experimentation" in the Supplement to the Journal of the Royal Statistical Society, extending the randomization-based model Neyman had developed in his 1923 essay "On the Application of Probability Theory to Agricultural Experiments" to cover randomized block designs.	Poland
1935	Concept	Ronald Fisher introduces the term "confounding" in The Design of Experiments to describe a specific consequence of blocking (statistics) in a factorial experiment — when partitioning treatment combinations into blocks causes certain interaction effects to become indistinguishable from ("confounded with") block effects. Fisher's usage popularizes the term within statistics, though his concern was controlling heterogeneity among experimental units rather than causal inference in the modern sense.	United Kingdom
1935	Publication	Ronald Fisher publishes The Design of Experiments, a foundational work that formalizes principles such as randomization, replication, and blocking, and emphasizes the importance of efficient experimental design; the book becomes the basis of modern experimental science and remains one of the field's foundational texts.	United Kingdom
1935	Experiment	Ronald Fisher describes the "lady tasting tea" experiment in The Design of Experiments, the original exposition of his concept of the null hypothesis — which he describes as never proved or established, only possibly disproved through experimentation. The example is loosely based on a real event: phycologist Muriel Bristol claimed she could tell whether tea or milk was poured into a cup first, and her future husband William Roach suggested testing her with eight cups, four of each preparation, in random order. Fisher's treatment, using what becomes known as Fisher's exact test, calculates that correctly identifying all eight cups would occur by chance alone in only 1 of 70 possible combinations (about 1.4%), below the conventional 5% significance threshold — establishing the logic of randomized assignment combined with a combinatorial significance test as a template later applied throughout experimental science. According to statistician H. Fairfield Smith, as later reported by David Salsburg, Bristol did in fact identify all eight cups correctly in the real trial.	United Kingdom
1935	Concept	The term "factorial" (in the experimental-design sense) appears to enter print for the first time when Ronald Fisher uses it in The Design of Experiments.	United Kingdom
1937	Method	Researcher Meredith Crawford, at the Yerkes National Primate Research Center, invents the cooperative pulling paradigm — an experimental design in which two or more animals must pull a reward toward themselves via an apparatus neither can operate alone — publishing a study of two young chimpanzees, Bula and Bimba, coordinating to pull ropes attached to a box too heavy for either to move alone. The paradigm becomes the most widely used experimental design for testing cooperation in animals, later applied to bonobos, orangutans, capuchins, elephants, wolves, dogs, ravens, kea, and dolphins, among other species, to investigate the evolution and mechanisms of cooperative behavior.	United States
1937	Experiment	American cardiologist Harry Gold, with Nathaniel Kwit and Harold Otto, publishes a placebo-controlled trial of xanthine drugs (theobromine and aminophylline) for cardiac pain in the Journal of the American Medical Association, testing roughly 700 subjects — a substantial scale-up from earlier placebo-controlled comparisons such as William Evans and Clifford Hoyle's 1933 angina trial, and part of Gold's broader advocacy (through the Cornell Conferences on Therapy) for masked, placebo-controlled assessment in American clinical pharmacology.	United States
1937	Method	Frank Yates formalizes what becomes known as the Yates analysis in "The Design and Analysis of Factorial Experiments," a technical communication published by the Commonwealth Bureau of Soils in Harpenden, England. The technique exploits the special structure of full and fractional factorial experiments — arranging response data in a specific "Yates' order" and repeatedly summing and differencing pairs of values — to generate least-squares estimates of factor effects and interactions for all factors simultaneously, without needing to separately fit each model.	United Kingdom
1939	Concept	A publication by Bose and Nair underlie the concept of association scheme. In their paper, they introduced the concept of association schemes as a way to study the structure of contingency tables. They show that association schemes can be used to represent the dependencies between the variables in a contingency table, and that they can be used to derive statistical tests for independence.	India
1940	Method	Ronald A. Fisher publishes "An Examination of the Different Possible Solutions of a Problem in Incomplete Blocks" in Annals of Eugenics, extending his earlier work on blocking (statistics) to systematically examine balanced incomplete block designs — designs in which not every treatment can be tested in every block, a situation Fisher's original randomized block design did not address.	United Kingdom
1940	Method	Raj Chandra Bose and K. Kishen at the Indian Statistical Institute independently find some efficient designs for estimating several main effects.	India
1942	Concept	K. Kishen generalizes Latin squares and mutually orthogonal Latin squares to Latin cubes and Latin hypercubes in "On latin and hyper-graeco cubes and hypercubes," published in Current Science.	India
1943	Method	Economist Robert Dorfman introduces group testing in a short report published in the Annals of Mathematical Statistics, motivated by the United States Public Health Service's wartime effort to screen soldiers for syphilis without the expense of testing every individual blood sample separately. Dorfman's method pools blood samples into groups and tests each pooled sample; groups testing negative allow every soldier within them to be cleared with a single test, while positive groups require individual follow-up testing — dramatically reducing the expected number of tests needed when disease prevalence is low. Dorfman tabulates the optimal group size as a function of prevalence rate.	United States
1944	Experiment	The Patulin Clinical Trials Committee of the Medical Research Council (United Kingdom) conducts a multicentre trial of the antibiotic patulin on the course of the common cold. Long overlooked, the trial has more recently been identified by historians of medicine as the first properly controlled, randomized multicentre trial conducted under the Council's aegis — predating the Medical Research Council (United Kingdom)'s better-known 1948 streptomycin tuberculosis trial, which had long been popularly credited as the first randomized clinical trial.	United Kingdom
1945	Method	American statistician Abraham Wald publishes Sequential Tests of Statistical Hypotheses, pioneering the field of sequential analysis — a framework for statistical tests where the number of observations is not fixed in advance, and the decision to stop collecting data (accept, reject, or continue sampling) can depend on the accumulated results as the experiment proceeds. Wald's work, developed partly in a wartime context, provides the formal apparatus later applied throughout industrial and clinical sequential experimental design.	United States
1945	Concept	British statistician D. J. Finney publishes "The Fractional Replication of Factorial Arrangements" in Annals of Eugenics, containing the first statistical use in print of the term "aliasing (factorial experiments)" — the phenomenon in fractional factorial designs where some effects become indistinguishable from each other because only a fraction of all possible treatment combinations is observed. The term would later be adopted into signal processing theory, possibly influenced by this earlier statistical usage.	United Kingdom
1945	Method	British statistician D. J. Finney introduces fractional factorial design, extending Ronald Fisher's earlier work on full factorial experiments at the Rothamsted Experimental Station by showing how to test only a carefully chosen fraction of all possible factor-level combinations — reducing the number of experimental runs required while deliberately confounding selected effects, based on the assumption that higher-order interactions are typically negligible (the sparsity-of-effects principle). Originally developed for agricultural applications, fractional factorial design later spreads to engineering, business, and other sciences.	United Kingdom
1946	Experiment	Psychologist E.M. Jellinek conducts a crossover trial for a U.S. headache-remedy manufacturer testing whether removing a scarce wartime ingredient would reduce drug efficacy, randomly assigning 199 subjects with frequent headaches to rotate through four formulations (three real combinations of ingredients plus a lactate placebo) over eight weeks. Initial analysis across all subjects suggests the scarce ingredient is unnecessary, but Jellinek discovers that 120 of the 199 subjects are "placebo reactors" whose inclusion masks the ingredient's real effect; restricting analysis to the 79 non-reactors reveals the ingredient does contribute significantly to efficacy. The trial becomes an influential early demonstration that placebo responsiveness varies systematically across individuals and can confound drug-efficacy comparisons if not accounted for.	United States
1946	Policy	The Cornell Conferences on Therapy, a series of consensus meetings among American clinical researchers, decree that new pharmaceutical substances should be tested against placebos under blinded, randomized conditions — an early institutional codification, ahead of any government regulation, of what would become the standard modern clinical trial design.	United States
1946	Method	R.L. Plackett and J.P. Burman publish a renowned paper titled "The Design of Optimal Multifactorial Experiments". The paper introduces what would be called Plackett–Burman designs, which are highly efficient screening designs with run numbers that are multiples of four. These designs are particularly useful for experiments where only main effects are of interest. In a Plackett-Burman design, main effects are often heavily confounded with two-factor interactions, making them suitable for screening experiments. For instance, a Plackett-Burman design with 12 runs can be utilized for an experiment containing up to 11 factors.	United Kingdom
1946–1947	Concept	C. R. Rao generalizes Kishen's hypercube constructions to arrays of arbitrary strength t in "Hypercubes of strength d leading to confounded designs in factorial experiments" (1946), then introduces the general concept of the orthogonal array — a tabular structure in which every selection of t columns contains all possible t-tuples of symbols the same number of times — in "Factorial experiments derivable from combinatorial arrangements of arrays" (1947), the paper credited as the origin of the modern notion of orthogonal arrays used throughout combinatorial design theory, coding theory, cryptography, and software testing.	India
1946–1948	Experiment	Sir Geoffrey Marshall of the Medical Research Council's Tuberculosis Research Unit leads the first randomized curative clinical trial, testing the efficacy of streptomycin for pulmonary tuberculosis under double-blind, placebo-controlled conditions — though the trial is more commonly credited to the Medical Research Council (United Kingdom) as an institution than to its individual lead investigator. Results, published in 1948, show reduced mortality alongside emerging drug resistance and side effects, establishing modern clinical trial methods such as random allocation and blinding and shaping the future of evidence-based medicine.	United Kingdom
1947	Policy	The Nuremberg Code is articulated by the U.S. military tribunal in USA v. Brandt (the "Doctors' Trial"), part of the Subsequent Nuremberg Trials against German physicians who conducted unethical experiments on concentration-camp prisoners and carried out over 3.5 million forced sterilizations. Concerned that the defendants — who argued their experiments differed little from prewar research and that no law distinguished legal from illegal experimentation — might escape conviction, prosecution medical experts Andrew Conway Ivy and Leo Alexander drafted a memorandum in April 1947 outlining principles for legitimate medical research; an expanded version requiring explicit voluntary consent followed on 9 August 1947. The judges' verdict, delivered 20 August 1947 against Karl Brandt (physician) and 22 others, revised these into ten points establishing principles including informed consent, freedom from coercion, and beneficence toward research subjects — the first codification of research ethics standards, influencing medical experiment codes of practice worldwide despite going largely unenforced (and even dismissed by some as inapplicable to "ordinary physicians") for roughly two decades after it was written.	Germany
1948	Concept	Francis J. Anscombe, at Rothamsted Experimental Station, discusses and develops design-based (randomization-based) analysis of variance, later extended by Oscar Kempthorne at Iowa State University, who introduces the assumption of unit-treatment additivity — the idea that an experimental unit's observed response can be written as the sum of the unit's baseline response and a treatment effect that is constant across all units receiving that treatment. This randomization-based approach differs from the more commonly taught normal-linear-model approach in making no assumption of a normal distribution or of independence between observations, though the two approaches' test statistics closely approximate each other in practice.	United Kingdom
1948	Concept	British statistician Frank Yates introduces the concept of restricted randomization.	United Kingdom
1949	Method	Psychologist Richard Solomon (psychologist) develops the Solomon four-group design, a research method addressing the problem of pretest sensitization — the possibility that administering a pre-intervention test itself influences how subjects respond to a subsequent treatment. In addition to the standard pre-test/treatment/post-test and pre-test/control/post-test groups, the design adds two further groups that skip the pre-test entirely (treatment-only and control-only, each still post-tested), allowing researchers to separate the effect of the treatment itself from the effect of having taken the pre-test.	United States
1949	Method	Genichi Taguchi begins developing his experimental design techniques while working at Japan's Electrical Communications Laboratories (ECL) in the post–World War II reconstruction period. Tasked with improving research and development productivity, he formulates a new approach to quality improvement that emphasizes off-line quality control, robust design, and statistical experimentation. These early efforts lay the foundations of what would later become known as the Taguchi Methods, integrating engineering design with statistical principles to systematically reduce variability, improve product quality, and lower societal and manufacturing costs.	Japan
1949	Method	Kenneth Arrow, David Blackwell, and M.A. Girshick publish "Bayes and Minimax Solutions of Sequential Decision Problems" in Econometrica, an early and influential contribution to sequential decision theory building on Abraham Wald's wartime work on sequential analysis, formalizing Bayes and minimax approaches to problems where the decision of when to stop sampling is itself part of the optimization.	United States
1949	Concept	R. H. Bruck and H. J. Ryser prove a nonexistence result for finite projective planes — special cases of symmetric block designs with λ = 1 — showing that if a projective plane of order q exists and q ≡ 1 or 2 (mod 4), then q must be expressible as the sum of two squares. The result rules out projective planes of certain orders (such as 6) that satisfy the basic combinatorial parameter equations for a symmetric design but are nonetheless impossible.	United States
1950	Publication	Gertrude Mary Cox and William Gemmell Cochran publish Experimental Designs, following the tradition established by Ronald Fisher's 1935 The Design of Experiments as a practical reference for statisticians and scientists. The book opens with three chapters on the philosophy of statistics in experimentation before covering specific designs — completely randomized designs, randomized blocks, Latin squares, factorial designs, split-plot designs, and various incomplete block designs — illustrated throughout with results from actual experiments. Cox, a pioneering woman in a field with little female visibility at the time, had helped found the Statistics Laboratory at Iowa State and was the first head of the Department of Experimental Statistics at North Carolina State; Cochran had worked at Rothamsted under Frank Yates before moving to Iowa State. Oscar Kempthorne would soon follow with a complementary theoretical treatment, The Design and Analysis of Experiments (1952), and Cochran and Cox would publish a second edition of their own book in 1957.	United States
1950	Method	P.M. Grundy and Michael Healy (statistician) publish "Restricted Randomization and Quasi-Latin Squares" in the Journal of the Royal Statistical Society, Series B, developing restricted randomization — a method for excluding intuitively poor or undesirable treatment allocations from the space of possible randomizations (for example, preventing a new obesity treatment from being randomly assigned only to the heaviest patients) while still preserving the theoretical statistical benefits of randomization. The concept had also been introduced independently by Frank Yates, whose earlier work on the subject this timeline's 1948 entry describes.	United Kingdom
1950	Concept	K. Bush coins the term "orthogonal array" in his PhD thesis at the University of North Carolina, naming the structure C. R. Rao had introduced in 1947 (Rao himself had used the unmodified term "array," meaning simply a subset of treatment combinations, before the tabular/matrix framing made the possibility of non-simple arrays — those with repeated rows — apparent).	United States
1950	Method	Cochran and Cox formalize principles of sample size determination in experimental design, providing more accurate tables and methods.	United States
1950	Experiment	Richard Doll and Austin Bradford Hill publish a preliminary report demonstrating a statistically significant association between tobacco smoking and lung cancer, using a large case–control study design comparing lung cancer patients against controls without the disease. Critics initially argued the case–control design could not establish causation; subsequent cohort studies by the same researchers over following decades confirmed the causal link the case–control study had suggested, and smoking is now recognized as the cause of roughly 87% of lung cancer deaths in the United States.	United Kingdom
1950	Concept	S. Chowla and H. J. Ryser extend the 1949 Bruck–Ryser nonexistence result from projective planes to general symmetric block designs, establishing what becomes known as the Bruck–Ryser–Chowla theorem: necessary conditions (a square-difference condition when the number of points is even, and a solvable Diophantine equation when odd) that any symmetric (v, b, r, k, λ)-design's parameters must satisfy, ruling out entire classes of otherwise combinatorially plausible designs.	United States
1950	Concept	Randomized controlled trials begin to emerge as the gold standard in medical research, enabling systematic causal inference	Global
1951	Method	Formal placebo-controlled and multi-arm clinical trial designs are developed to improve reliability of treatment comparisons	United States
1951	Method	George E. P. Box and K. B. Wilson introduce response surface methodology (RSM), using sequences of designed experiments and second-degree polynomial models to approximate and optimize responses, even with limited process knowledge. They develop a method to build a quadratic model where the number of data points scales linearly, rather than exponentially, with the number of inputs, striking a balance between accuracy and ease of application — enabling sequential, adaptive optimization of industrial processes even under limited prior knowledge of the underlying system.	United Kingdom
1952	Concept	American mathematician and statistician Herbert Robbins recognizes the significance of a problem where a gambler faces a trade-off between "exploitation" of the machine with the highest expected payoff and "exploration" to learn about other machines' payoffs. This problem involves pulling levers on different machines, each providing random rewards from unknown probability distributions. The gambler aims to maximize the total rewards earned over a sequence of lever pulls. Robbins devised convergent population selection strategies in his work on "some aspects of the sequential design of experiments."	United States
1952	Method	D.G. Horvitz and D.J. Thompson publish "A Generalization of Sampling Without Replacement from a Finite Universe" in the Journal of the American Statistical Association, introducing what becomes known as the Horvitz–Thompson estimator — a method for producing unbiased population estimates from samples with unequal selection probabilities by weighting each observation by the inverse of its probability of inclusion. Originally developed for survey sampling, the estimator later becomes a standard tool via inverse probability weighting for correcting bias in the analysis of experiments with unequal exposure probabilities, including studies of spillover and network interference effects.	United States
1952	Concept development	Bose and Shimamoto introduce the term association scheme.	United States
1954	Experiment	The Salk polio vaccine trial becomes one of the first large-scale randomized controlled trials, involving over one million participants	United States
1954	Concept	Paul Lazarsfeld and others formalize factor analysis as a tool for latent variable modeling in experiments.	United States
1954	Publication	Edwin Boring discusses the history and meanings of control treatments, clarifying the role of controls in experimental research.	United States
1954	Publication	American experimental psychologist Edward Boring writes an article titled The History of Experimental Design. In this article, Boring notes that the early history of ideas on the planning of experiments has been "but little studied".	United States
1955	Experiment	An influential study entitled The Powerful Placebo firmly establishes the idea that placebo effects are clinically important.	United States
1955	Method	M. B. Wilk introduces the randomization analysis of the generalized randomized block design (GRBD), which replicates each treatment at least twice within each block — unlike the classic randomized block design, which has no within-block replication — allowing block-treatment interaction (statistics) to be estimated and tested without relying on parametric assumptions about the distribution of experimental error.	United States
1956	Concept	D. V. Lindley publishes "On a Measure of Information Provided by an Experiment," proposing that the utility of an experimental design be measured as the Kullback–Leibler divergence between the posterior and prior probability distributions over the parameters being estimated — showing this expected utility is coordinate-independent and equals the mutual information between the parameters and the observations, exactly the expected information gain from running the experiment. This becomes a foundational formulation within Bayesian experimental design.	United Kingdom
1956	Experiment	Richard Doll and Austin Bradford Hill publish a second report on the mortality of British doctors, a prospective cohort study confirming the causal link between smoking and lung cancer that their 1950 case–control study had first suggested — an early demonstration of how case–control findings could be validated through a stronger, prospective study design.	United Kingdom
1959	Concept	Leslie Kish uses the term "confounding" in the sense of "incomparability" between two or more groups (such as exposed and unexposed groups) in an observational study — a distinct usage from Fisher's blocking-based sense, and closer to the modern causal-inference meaning of the term as it would later develop in epidemiology.	United States
1959	Method	Raymond H. Pierson and Edward A. Fay publish "Guidelines for Interlaboratory Testing Programs" in Analytical Chemistry, an early formalization of round-robin testing — interlaboratory studies in which multiple independent laboratories perform the same measurement or analysis, using the same or different methods and equipment, to assess the reproducibility of a test method or verify a new method of analysis against an established one.	United States
1959	Concept	Raj Chandra Bose and D. M. Mesner publish "On Linear Associative Algebras Corresponding to Association Schemes of Partially Balanced Designs" in the Annals of Mathematical Statistics, introducing what becomes known as the Bose–Mesner algebra — the commutative algebra generated by the adjacency matrices of an association scheme's relations. This publication turns association schemes, previously a statistical tool for classifying partially balanced incomplete block designs, into an object of independent algebraic interest.	United States
1959	Concept	W. M. S. Russell and R. L. Burch publish The Principles of Humane Experimental Technique, introducing the "Three Rs" — Replacement (preferring non-animal methods when they can achieve the same scientific aims), Reduction (obtaining comparable information from fewer animals, or more information from the same number), and Refinement (minimizing pain, suffering, or distress and enhancing welfare for animals that must be used) — as guiding principles for more ethical animal research. The Three Rs would go on to become explicit in animal-research legislation in many countries and remain foundational to the ethical design of experiments involving animals.	United Kingdom
1959	Concept	Arthur Samuel publishes "Some Studies in Machine Learning Using the Game of Checkers" in IBM Journal of Research and Development, describing a self-improving checkers program that, given only the rules of the game and a list of parameters of unknown relative importance, learns within 8–10 hours of play to outperform the person who wrote it. The paper is widely credited with popularizing the term "machine learning," though the commonly quoted definition attributed to Samuel — giving computers "the ability to learn without being explicitly programmed" — is now generally considered apocryphal, as it does not appear verbatim in his published work.	United States
1959–1961	Method	Jack Kiefer and Jacob Wolfowitz develop formal optimal design theory, introducing criteria-based selection of experimental designs for maximum precision.	United States
1960	Industrial statistics	Japanese engineer and statistician Genichi Taguchi publishes Design of Experiments for Engineers, advancing statistical methods for industrial experimentation. His work helps formalize what later would become known as Taguchi methods, integrating experimental design and statistical analysis to improve product quality, optimize manufacturing processes, and reduce costs in engineering and industry.	Japan
1960	Method	George E. P. Box and Donald Behnken publish "Some New Three Level Designs for the Study of Quantitative Variables" in Technometrics, devising what becomes known as Box–Behnken designs — three-level experimental designs for response surface methodology that place each factor at one of three equally spaced values and are efficient enough to fit a quadratic model. The seven-factor design, whose estimation variance depends almost exactly on distance from the centre point, was found first; designs for other numbers of factors were subsequently derived to approximate this same rotatability property.	United States
1960	Method	Damaraju Raghavarao publishes "Some Aspects of Weighing Designs" in the Annals of Mathematical Statistics, establishing the statistical theory connecting weighing matrices to the classical problem of estimating the weights of multiple objects with minimum variance using a balance scale. Raghavarao shows that the variance of least-squares weight estimates is minimized if and only if the design matrix used to allocate objects to the scale's pans is a proper weighing matrix, connecting a combinatorial mathematics object to a concrete experimental design problem.	United States
1960	Experiment	English psychologist Peter Wason conducts an experiment in which participants must discover a rule governing triples of numbers, given that (2,4,6) fits it. Participants overwhelmingly test only triples that confirm their current hypothesis rather than triples designed to falsify it, and rarely discover the true (very broad) rule, "any ascending sequence." Wason interprets the results as showing a preference for confirmation over falsification in hypothesis testing, coining the term "confirmation bias."	United Kingdom
1960	Method	Donald Thistlethwaite and Donald T. Campbell introduce the regression discontinuity design in "Regression-Discontinuity Analysis: An alternative to the ex post facto experiment," published in the Journal of Educational Psychology, applying it to evaluate the effect of merit-based scholarship programs on students' career plans. The design estimates a local causal treatment effect by comparing outcomes for units lying just above and just below a predetermined threshold that determines treatment assignment (e.g. a scholarship awarded only to students scoring above a cutoff), exploiting the idea that units on either side of a narrow cutoff should otherwise be comparable even without formal random assignment. It becomes a widely used quasi-experimental design across economics, political science, epidemiology, and psychology.	United States
1961	Concept	George E. P. Box and J. S. Hunter introduce the concept of resolution in their paper "The 2^(k-p) Fractional Factorial Designs," providing a way to measure the degree to which a fractional factorial design avoids aliasing between main effects and important interactions — a design has resolution R if every effect involving p factors is unaliased with every effect having fewer than R−p factors. This concept becomes a standard way of summarizing and comparing the aliasing properties of fractional designs.	United States
1961	Experiment	Ecologist Joseph H. Connell publishes "The Influence of Interspecific Competition and Other Factors on the Distribution of the Barnacle Chthamalus stellatus" in Ecology, a field study examining how competition from the barnacle species Balanus balanoides shapes the distribution of Chthamalus stellatus on rocky shores. The study becomes a widely cited early example of manipulative field experimentation in ecology.	England
1961	Concept	The term nocebo (Latin nocēbō, "I shall harm", from noceō, "I harm") is coined by Walter Kennedy to denote the counterpart to the use of placebo (Latin placēbō, "I shall please", from placeō, "I please"; a substance that may produce a beneficial, healthful, pleasant, or desirable effect). Kennedy emphasized that his use of the term "nocebo" refers strictly to a subject-centered response, a quality inherent in the patient rather than in the remedy".	United States
1962	Policy	The Kefauver Harris Amendment requires proof of efficacy through controlled trials before drug approval, institutionalizing experimental design in regulation.	United States
1962	Experiment	Researchers including Walter N. Pahnke conduct the Marsh Chapel Experiment at Boston University, a double-blind study giving divinity students either the psychedelic substance psilocybin or an active placebo — a large dose of niacin chosen specifically because it produces noticeable physical sensations that could lead control subjects to believe they too had received a psychoactive drug. The design becomes an influential early example of using an active rather than inert placebo to counter unblinding in trials of drugs with strong subjective effects.	United States
1962	Method	Japanese statistician Kazumasa Kôno publishes "Optimum designs for quadratic regression on k-cube" in the Memoirs of the Faculty of Science, Kyushu University, deriving optimal experimental designs for quadratic response surface methodology models that require fewer experimental runs than George E. P. Box's earlier central-composite designs. Together with later work by Jack Kiefer (mathematician), the Kôno–Kiefer analysis explains why such optimal designs, despite their mathematical construction as probability measures over continuous spaces, can be supported on a small number of discrete points closely resembling the traditional designs already used in practice.	Japan
1962	Experiment	Vernon L. Smith publishes An Experimental Study of Competitive Market Behavior in the Journal of Political Economy. Using controlled laboratory market experiments with human participants, he tests hypotheses of neoclassical competitive theory and demonstrates price convergence toward equilibrium, helping establish experimental economics as a systematic empirical research methodology.	United States
1962	Method	British statistician John Nelder proposes a set of systematic, circular experimental designs as an alternative to the replicated, full factorial spacing experiments. These designs, known as the Nelder 'wheel' design, are developed to address limitations related to space and plant material. The design consists of a circular plot with concentric circumferences radiating outward, connected by spokes that extend from the center to the farthest circumference. Trees are planted at the intersections of spokes and circumferences within the plot.	United Kingdom
1963	Concept	Campbell and Stanley discuss design according to the categories of preexperimental designs, experimental designs, and quasi-experimental designs.	United States
1963	Method	J.J. Boren introduces "repeated acquisition of new behavioral chains" as a single-subject research design in the American Psychologist, offering an alternative to reversal designs for studying behaviors that cannot practically or ethically be reversed once learned — instead repeatedly measuring how quickly a subject acquires a new behavioral sequence under different experimental conditions.	United States
1964	Policy	The World Medical Association adopts the Declaration of Helsinki at its 18th World Medical Assembly in Helsinki, Finland, setting out "Recommendations Guiding Physicians in Biomedical Research Involving Human Subjects." The Declaration requires that research protocols be reviewed by an independent committee, that subjects give freely obtained informed consent after being told the study's aims, methods, and risks, and that the subject's wellbeing always take precedence over the interests of science and society. It becomes a foundational reference document for research ethics committees worldwide and would be amended repeatedly in subsequent decades, including in Tokyo (1975) and Venice (1983).	Finland
1965	Concept	Leslie Kish coins the term "design effect" in his book Survey Sampling, proposing the general definition as the ratio of the variance of an estimator under a given (often complex) sampling design to its variance under simple random sampling of the same size — along with formulas for the design effect under cluster sampling (incorporating intraclass correlation) and under unequal-probability sampling. These become known as "Kish's design effect" and remain foundational to survey methodology.	United States
1966	Concept	Hanan Selvin and Alan Stuart describe "data-dredging procedures" in survey analysis, formalizing an early statistical critique of selecting which explanatory variables to retain in a model based on the same data used to test them. Using the metaphor of fish that "don't fall through the net," they argue that variables retained after such a selection process are systematically biased toward appearing more significant than they truly are, altering the validity of standard statistical tests applied afterward.	United Kingdom
1966	Publication	American psychologist Robert Rosenthal (psychologist) publishes Experimenter Effects in Behavioral Research, a foundational study establishing that researchers can subtly and unconsciously communicate their expectations to experimental subjects, biasing outcomes toward those expectations — later demonstrated experimentally by showing that researchers told to expect positive ratings from subjects obtained significantly more positive data than researchers told to expect negative ratings, using the identical task.	United States
1971	Publication	Raymond H. Myers publishes Response Surface Methodology, a textbook establishing widely used practical guidelines for central composite design parameters — including standard methods for selecting the axial-point distance α (orthogonal and rotatable design criteria) — that remain in use in statistical software decades later.	United States
1972	Publication	Herman Chernoff writes an overview of optimal sequential designs In the design of experiments, optimal designs is a class of experimental designs that are optimal with respect to some statistical criterion.	United States
1973	Concept	Belgian mathematician Philippe Delsarte's thesis, "An Algebraic Approach to the Association Schemes of Coding Theory," recognizes and develops the deep connections between association schemes and both coding theory and design theory, becoming widely regarded as the most important contribution to association scheme theory after its statistical origins in design of experiments.	Belgium
1973	Method	Donald Rubin publishes "Matching to Remove Bias in Observational Studies" in Biometrics, formalizing statistical matching — pairing treated units in an observational study with non-treated units sharing similar observable characteristics — as a technique for reducing confounding bias in estimated treatment effects when random assignment is unavailable. The paper predates and lays groundwork for Rubin's later development of propensity score matching with Paul Rosenbaum in 1983.	United States
1973	Concept	Psychologist Edwin Wike publishes "Water beds and sexual satisfaction: Wike's law of low odd primes (WLLOP)" in Psychological Reports, a humorous methodological note observing that whenever an experiment has a low odd prime number of treatment conditions (three being the most common case), the design tends to be unbalanced and partially confounded. Wike illustrates the principle with an invented three-group study of water beds and sexual satisfaction, showing that a fourth group is needed to disentangle the effect of the water bed from that of an accompanying seasickness pill. Wike explicitly notes that naming the "law" after himself is itself an example of Stigler's law of eponymy.	United States
1974	Concept	Donald Rubin formalizes the potential outcomes framework, defining causal effects using counterfactual comparisons between treated and untreated states for the same unit.	United States
1974	Method	R.H.F. Denniston solves a problem posed by James Joseph Sylvester in 1860 as an extension of Kirkman's schoolgirl problem: whether thirteen disjoint Steiner triple systems S(2,3,15) exist, such that "Kirkman's schoolgirls" could march in triples for an entire 13-week term without any pair of girls being grouped together twice. Denniston's computer search, run for seven hours on an Elliott 4130 computer at the University of Leicester, finds a valid week-one solution from which all subsequent weeks are generated by a fixed relabeling scheme; the number of non-isomorphic solutions to Sylvester's problem remains unknown.	United Kingdom
1975	Concept	Dijen K. Ray-Chaudhuri and R. M. Wilson prove a generalization of Fisher's inequality to t-designs, showing that in a 2s-(v, k, λ) design the number of blocks is at least the binomial coefficient v choose s — extending the classical Fisher's inequality (b ≥ v for balanced incomplete block designs) to a broader class of combinatorial designs.	United States
1975	Concept	A.C. Atkinson and V.V. Fedorov publish "The design of experiments for discriminating between two rival models" in Biometrika, introducing T-optimality — a criterion for constructing experimental designs specifically intended to maximize the statistical discrepancy between two competing candidate models at the chosen design points, rather than to minimize estimation variance within a single assumed model. The criterion becomes particularly important in biostatistics applications supporting pharmacokinetics and pharmacodynamics, building on earlier work by David R. Cox and Atkinson on model-discrimination experiments.	United Kingdom
1975	Method	Stuart Pocock and Richard Simon introduce minimisation in Biometrics, an adaptive stratified sampling technique for balancing treatment groups across multiple prognostic factors in clinical trials. Unlike traditional blocking (statistics), which requires a separate randomisation list for each combination of stratification factors — a number of lists that grows exponentially as more factors are added — minimisation calculates, for each new patient, the imbalance that would result from allocating them to each treatment group, sums the imbalance across all factors, and assigns the patient to whichever group minimises the overall imbalance (optionally with a random element retained). The method comes to be described by some as maintaining better balance than blocked randomisation, particularly as the number of stratification factors grows.	United Kingdom
1976	Method	James Scheirer, William S. Ray, and Nathan Hare publish "The Analysis of Ranked Data Derived from Completely Randomized Factorial Designs" in Biometrics, introducing the Scheirer–Ray–Hare test, a non-parametric extension of the Kruskal–Wallis one-way analysis of variance for examining whether a measured outcome is affected by two or more factors, without requiring the assumption of a normal data distribution. Though more conservative in statistical power than parametric multi-factorial ANOVA and later described than most comparable variance-analysis tests, the Scheirer–Ray–Hare test finds continued use, particularly in the biological sciences, as a non-parametric alternative when normality assumptions cannot be met.	United States
1976	Publication	Douglas C. Montgomery publishes Design and Analysis of Experiments, a comprehensive textbook on the design and analysis of experiments. The book covers a wide range of topics, including principles of experimental design, different types of experimental designs, analysis of experimental data, and use of experimental design in a variety of fields, such as agriculture, industry, and medicine.	United States
1976	Method	Neil Sloane and Martin Harwit publish "Masks for Hadamard transform optics, and weighing designs" in Applied Optics, extending weighing-matrix theory from its origin in physical balance-scale weighing to the design of optical masks used in spectrometers and image scanners — where each matrix element determines whether incoming light is transmitted to one detector, absorbed, or reflected to a second detector, mathematically equivalent to the balance-pan weighing problem.	United States
1976	Method	Olli Miettinen formalizes the conditions under which the odds ratio of exposure in a case–control study can be used to estimate relative risk, refining earlier work by Jerome Cornfield that had shown this approximation held specifically when the disease outcome under study is rare.	United States
1977	Method	Stuart Pocock introduces group sequential methods for the design and analysis of clinical trials, in which patient entry is divided into equal-sized groups so that repeated significance tests on the accumulated data after each group determine whether the trial continues; the associated stopping threshold becomes known as the Pocock boundary.	United Kingdom
1977	Method	Latvian engineer Vilnis Eglājs and P. Audze propose a space-filling multifactor experimental design technique — later known as the Audze–Eglājs (AE) method — in Russian-language literature, minimizing a potential-energy-like criterion between design points to spread them evenly across the design space. The technique is later recognized as equivalent in spirit to Latin hypercube sampling, predating Michael McKay's independently-derived and far more widely cited 1979 description of Latin hypercube sampling by two years, though Eglājs and Audze's work remains comparatively obscure in the Western statistical literature, likely due to the Russian-language original and limited circulation outside Soviet-bloc engineering circles.	Latvia
1978	Concept	Donald Rubin introduces the concept of "ignorable assignment mechanisms" in causal inference, formalizing the condition under which the way individuals were assigned to treatment groups can be disregarded during statistical analysis, given everything else recorded about them — a refinement of his earlier Rubin causal model potential-outcomes framework, and part of what becomes known as the Neyman-Rubin causal inference model.	United States
1978	Concept	According to Box et al., experimental design refers to the systematic layout of combinations of variables. The layouts in the case of concepts are test concepts or test vignettes.	United States
1978	Concept	Ulrich Krengel and Louis Sucheston (with David J. H. Garling) formulated the Prophet Inequality in optimal stopping theory. They show that a gambler observing sequential random rewards can secure at least half the expected payoff of a “prophet” who knows all outcomes in advance, establishing a foundational result in probability theory and decision processes.	United States
1979	Method	Marvin Zelen publishes his new method, which would later be called Zelen's design.	United States
1979	Method	Harvard biostatistician Marvin Zelen (biostatistician) proposes a new randomized clinical trial design in "A New Design for Randomized Clinical Trials," published in the New England Journal of Medicine, in which patients are randomized to treatment or control before informed consent is sought — allowing consent to be obtained conditionally, and control-group patients to be enrolled with minimal disruption to standard care. The design is intended to reduce clinician discomfort with randomization and patient-side effects such as resentful demoralization and the Hawthorne effect, but draws criticism for its lack of allocation concealment and, in trials on serious or life-threatening conditions, for being seen by participants as "inappropriately deceptive and manipulative."	United States
1979	Publication	Thomas D. Cook and Donald T. Campbell publish Quasi-experimentation: Design & Analysis Issues for Field Settings, extending Campbell and Julian C. Stanley's 1963 categorization of experimental, quasi-experimental, and pre-experimental designs into a comprehensive treatment of quasi-experimental methodology for applied field research where randomization is impractical or unethical. The book becomes a foundational reference for quasi-experimental design across social science, public health, education, and policy analysis, and is substantially revised and expanded by Cook, Campbell, and William R. Shadish in 2002.	United States
1979	Method	Peter C. O'Brien and Thomas R. Fleming publish "A Multiple Testing Procedure for Clinical Trials" in Biometrics, introducing the O'Brien–Fleming boundary for group sequential design clinical trial monitoring. The boundary sets a very conservative (high) significance threshold at early interim analyses, becoming progressively less stringent as the trial proceeds until it approaches the nominal significance level (e.g. 0.05) at the final analysis — protecting against premature stopping due to random early fluctuations while still preserving the overall Type I error rate specified for the trial. Despite requiring the number and timing of interim analyses to be prespecified, the O'Brien–Fleming boundary becomes one of the most widely used methods for monitoring clinical trials.	United States
1979	Method	Michael McKay at Los Alamos National Laboratory makes a significant contribution to the field of statistical sampling by introducing the concept of latin hypercube sampling.	United States
1979	Concept	John C. Gittins publishes "Bandit Processes and Dynamic Allocation Indices," proving that the optimal solution to the multi-armed bandit problem — first posed as an unsolved question in Herbert Robbins's 1952 paper on sequential experimental design — takes the form of an index policy: at each stage, choosing the option (or "arm") with the highest computable "dynamic allocation index," now known as the Gittins index. The result resolves optimal-stopping questions in clinical trial design that had been open since the 1940s, and independent economist Martin Weitzman would establish the equivalent result in economics the same year.	United Kingdom
1980	Concept	Researchers publish long-term results of the Coronary Drug Project, a study of drugs for long-term treatment of coronary heart disease in men, reporting that participants in the placebo group who adhered to their placebo regimen as instructed showed nearly half the mortality rate of those who did not adhere — despite the placebo itself being pharmacologically inert. The finding becomes a widely cited illustration of the "healthy adherer" effect, in which apparent treatment benefits attributed to adherence may instead reflect underlying differences between compliant and non-compliant patients (health consciousness, psychological effects of following a protocol, or general diligence), a confound later replicated in women with nearly 2.5 times greater survival among placebo-adherent patients.	United States
1980	Concept	Statistician John Tukey writes about the choice between confirmatory analysis (testing or rejecting existing hypotheses) and exploratory analysis (searching for new hypotheses) in statistical practice, examining how practicing statisticians decide between the two modes of reasoning at different stages of research — an influential early articulation, within statistics specifically, of the same exploratory/confirmatory distinction later applied more broadly to human reasoning by psychologists Jennifer Lerner and Philip Tetlock in 2002.	United States
1981	Concept	Allen Neuringer first proposes the idea of using single case designs (sometimes referred to as n-of-1 trials) for self-experimentation.	United States
1982	Literature	British statistician George Box publishes Improving Almost Anything: Ideas and Essays, which gives many examples of the benefits of factorial experiments.	United States
1983	Concept	Donald Rubin and Paul R. Rosenbaum define the stronger condition of a treatment assignment being "strongly ignorable" in their paper on propensity score matching, establishing propensity scores — the probability of treatment assignment given observed covariates — as central to estimating causal effects from observational data when random assignment is unavailable.	United States
1983	Method	K.K. Gordon Lan and David L. DeMets publish "Discrete Sequential Boundaries for Clinical Trials" in Biometrika, proposing a method that allows the boundary values of a group sequential trial to be allocated dynamically as the study progresses, addressing a key limitation of the O'Brien–Fleming boundary and other fixed methods that require prespecifying the number of interim analyses and the proportion of total information used at each one.	United States
1984	Method	French pharmacologist Bernard Bégaud describes the challenge–dechallenge–rechallenge protocol as one of the standardized methods used in France for assessing adverse drug reactions, monitoring whether an adverse event resolves on withdrawal of a medication (dechallenge) and recurs on its re-administration (rechallenge) — a design suited to idiosyncratic, individual-level reactions where conventional population-level statistical testing is unsuitable.	France
1984	Concept	Stuart Hurlbert publishes a paper in Ecological Monographs where he analyzes 176 experimental studies in ecology. He discovers that 27% of these studies suffer from 'pseudoreplication,' meaning they use statistical testing in situations where treatments are not replicated or replicates were not independent. When considering only studies that use inferential statistics, the percentage of pseudoreplication increases to 48%. To address this issue, Hurlbert suggests interspersing treatments in experiments, even if it means sacrificing randomized samples, particularly in smaller experiments. This approach aims to overcome the problem of pseudoreplication in ecological studies.	United States
1985	Experiment	Belgian experimental psychologist Jozef Nuttin publishes "Narcissism beyond Gestalt and awareness: the name letter effect" in the European Journal of Social Psychology, using a yoked control design in which pairs of subjects each rate the same sequence of individual letters for preference, unaware that some letters are drawn from their own name and others from their partner's. Because both members of a yoked pair see an identical stimulus sequence, any systematic difference in preference between them can only be attributed to whether a given letter appears in their own name. Nuttin finds that people reliably prefer the letters in their own name over other letters, without being aware of doing so — establishing what becomes known as the name-letter effect.	Belgium
1986	Experiment	Robert LaLonde finds that findings of econometric procedures assessing the effect of an employment program on trainee earnings do not recover the experimental findings. This is considered to be the start of experimental benchmarking in social science.	United States
1986	Concept	Fred N. Kerlinger describes the MAXMINCON principle, emphasizing maximizing systematic variance, controlling extraneous variance, and minimizing error variance in experimental design.	United States
1987	Publication	Australian mathematician Anne Penfold Street and her daughter, statistician Deborah Street, publish Combinatorics of Experimental Design, a textbook connecting the combinatorial mathematics of block designs, Latin squares, and factorial designs to their applications in statistics. Reviewers praised the book for making the combinatorial side of experimental design accessible to statisticians, with Marshall Hall calling it "very readable" and "very satisfying," though some noted it omitted certain topics covered by more comprehensive contemporary texts.	Australia
1987	Concept	S.K. Wang and Anastasios A. Tsiatis publish "Approximately optimal one-parameter boundaries for group sequential trials" in Biometrics, introducing a general parameterized family of stopping boundaries of which both the O'Brien–Fleming boundary (1979) and the Pocock boundary (1977) are shown to be special cases.	United States
1988	Publication	Roger Mead publishes The Design of Experiments: Statistical Principles for Practical Applications, a textbook presenting practical principles of experimental design and analysis.	United Kingdom
1988	Method	Organizational psychologists Miriam Erez and Gary P. Latham, in a dispute over the effect of participation on goal commitment and performance in goal setting research, design four experiments together with Edwin Locke serving as a neutral third party to resolve their disagreement — one of the earliest documented modern examples of what would later be termed adversarial collaboration, though the term itself was not yet coined.	United States
1988	Organization	Stat-Ease releases its first version of Design–Expert, a statistical software package specifically dedicated to performing design of experiments (DOE), offering comparative tests, screening, characterization, and optimization tools.	United States
1989	Experiment	The National Institute of Mental Health's Treatment of Depression Collaborative Research Program, led by Irene Elkin, publishes results comparing the tricyclic antidepressant imipramine, a placebo (each administered with supportive clinical management), and two forms of psychotherapy (cognitive-behavioral and interpersonal) for depression. The study finds no statistically significant differences in outcome across the four conditions overall, a result widely attributed to the unexpectedly strong performance of the placebo-plus-clinical-management condition — an influential demonstration of how effective a well-administered placebo control can be in psychotherapy and psychiatric drug research.	United States
1989	Publication	Perry D. Haaland publishes Experimental Design in Biotechnology, presenting statistical experimental design and analysis as a problem-solving tool in biotechnology.	United States
1989	Method	Jerome Sacks, William Welch, Toby Mitchell, and Henry Wynn publish a landmark paper summarizing the Bayesian statistical framework for computer experiments, modeling a deterministic computer simulation's output as an unknown function of its inputs and placing a Gaussian process prior over that function — establishing an approach to designing and analyzing simulation experiments distinct from classical experimental design for physical systems, where criteria like replication and A/D-optimality (suited to parametric models with random error) do not directly apply.	United States
1989	Publication	Jerome Sacks and collaborators discuss statistical issues in the design and analysis of computer and simulation experiments, helping establish the field of computer experiments.	United States
1989	Concept	C. W. H. Lam and collaborators complete a large-scale computer search establishing that no finite projective plane of order 10 exists — a case the Bruck–Ryser–Chowla theorem's necessary conditions do not themselves rule out, showing that theorem's criteria, while powerful, are not sufficient to determine existence in general. The proof, which combined coding theory with an extensive computer search, was notable enough to draw coverage in The New York Times questioning whether a computer-assisted proof that no human could fully verify by hand should count as a mathematical proof.	Canada
1990	Organization	The International Council for Harmonisation (ICH) holds its inaugural meeting in Brussels, Belgium, hosted by the European Federation of Pharmaceutical Industries and Associations (EFPIA), bringing together regulatory authorities and pharmaceutical industry representatives from Europe, Japan, and the United States. The initiative aims to harmonize technical requirements for drug development and registration across jurisdictions, reducing redundant testing while ensuring the quality, safety, and efficacy of medicines. ICH would go on to produce more than 80 guidelines and expand to around 50 participating medicine authorities worldwide by its 30th anniversary in 2020.	Global
1993	Concept	Judea Pearl introduces the Back-Door criterion, a graphical condition using causal graphical model for identifying a sufficient set of variables to adjust for in order to obtain an unbiased estimate of a causal effect — providing a formal, graph-based alternative to the counterfactual definitions of confounding developed in epidemiology, later shown to be formally equivalent to them.	United States
1993	Policy	The Council for International Organizations of Medical Sciences (CIOMS) — founded in 1949 under the auspices of the World Health Organization and UNESCO — publishes International Ethical Guidelines for Biomedical Research Involving Human Subjects, a revision of its original 1982 guidelines prompted by the HIV/AIDS pandemic, proposed large-scale prevention and treatment trials, and the growing use of multinational field trials involving vulnerable populations. Though carrying no legal force, the guidelines become influential in shaping national approaches to research ethics review, and CIOMS would go on to revise them again in 2002 and 2016.	Global
1994	Method	American psychiatrist Peter Breggin applies the challenge–dechallenge–rechallenge (CDR) protocol — administering, withdrawing, then re-administering a medication while monitoring for adverse effects — to investigate a suspected association between fluoxetine (Prozac) and suicidal ideation. Breggin had observed that only certain individuals responded to the medication with increased suicidal thoughts; given the low occurrence rate of this reaction, conventional statistical testing across a study population was considered inappropriate, making the individual-level CDR protocol a more suitable design. Eli Lilly and Company subsequently adopted the CDR protocol, rather than a randomized controlled trial, when testing for increased suicide risk associated with the drug.	United States
1994	Experiment	David Card and Alan Krueger publish a study using the difference in differences design to evaluate the employment effects of New Jersey's April 1992 minimum wage increase, comparing fast-food employment in New Jersey against neighboring Pennsylvania (used as a control not subject to the wage increase) before and after the change. Contrary to standard economic theory's prediction that a minimum wage increase would reduce employment, they find no such decrease — an early, influential demonstration of using a natural experiment and a control region to estimate a causal treatment effect from observational data. Card received the 2021 Nobel Memorial Prize in Economic Sciences in part for this and related work.	United States
1994	Method	The Neyer-d optimal test is first described by Barry T. Neyer.	United States
1995	Method	Chris Nachtsheim and Ruth K. Meyer introduce the coordinate exchange algorithm in "The Coordinate-Exchange Algorithm for Constructing Exact Optimal Experimental Designs," published in Technometrics, enabling computational generation of optimal experimental designs. The cyclic coordinate-exchange algorithm constructs D-optimal and linear-optimal designs without needing to explicitly construct or enumerate candidate sets — which grow exponentially in the number of factors — and handles convex or mixed convex/discrete design spaces directly, without requiring sophisticated nonlinear programming routines. For design problems with 10 or more factors, the approach typically achieves reductions in computing time of two or more orders of magnitude compared to standard candidate-set-based procedures, with no loss of design efficiency.	United States
1995	Concept	Leslie Kish proposes the "Design Effect Factor" (Deft), a refinement of his 1965 design effect that uses simple random sampling with replacement in the denominator rather than without, arguing this better isolates the effect of sampling design from the nuisance of finite-population correction and is simpler to use directly in confidence-interval calculations.	United States
1995	Publication	Kathryn Chaloner and Isabella Verdinelli publish "Bayesian Experimental Design: A Review" in Statistical Science, a widely cited survey establishing the now-standard approach of assuming approximate normality of posterior probabilities in order to calculate expected utility using linear theory — becoming a standard reference point for later work in Bayesian experimental design.	United States
1996	Ethical oversight	International Conference on Harmonisation (ICH) guidelines for Good Clinical Practice (GCP) established.	Global
1996	Method	Colombian-Canadian physician Alex Jadad, working as a Research Fellow at Oxford's Pain Relief Unit, and colleagues introduce the Jadad scale (also known as Jadad scoring or the Oxford quality scoring system) in an appendix to a paper on blinding in randomized clinical trials. The instrument scores a trial report from zero to five based on three yes/no questions covering randomization, double-blinding, and reporting of withdrawals and dropouts. Despite later criticism for over-emphasizing blinding, showing low inter-rater consistency, and omitting allocation concealment, the scale becomes the most widely used trial-quality assessment tool worldwide, with its seminal paper cited in over 25,000 scientific works by 2024.	United Kingdom
1996	Method	Stat-Ease releases Design–Expert Version 5, the first version of the software designed for Microsoft Windows, broadening accessibility of dedicated design-of-experiments software beyond earlier DOS-based tools.	United States
1996	Policy	The Consolidated Standards of Reporting Trials (CONSORT) Statement is first published, the product of a 1995 Chicago meeting merging two independent efforts to improve randomized-trial reporting: the 1993 Ottawa meeting's Standardized Reporting of Trials (SORT) proposal and the concurrent Asilomar Working Group's recommendations from California. Convened at the suggestion of JAMA's Drummond Rennie, the merged CONSORT Statement provides a standardized checklist and participant flow diagram intended to reduce bias and aid critical appraisal of trial reports.	United States
1997	Concept	German researchers Gunver Kienle and Helmut Kiene publish a reanalysis of Henry K. Beecher's influential 1955 paper "The Powerful Placebo" in the Journal of Clinical Epidemiology, re-examining the original studies Beecher had cited as evidence of a roughly 35% placebo response rate. They report finding no genuine evidence of a placebo effect in any of the studies Beecher relied upon, attributing his conclusions instead to factors such as natural disease progression, statistical regression to the mean, and co-administered treatments — a finding that substantially undercuts the empirical foundation of the "powerful placebo" narrative that had gone largely unchallenged for four decades.	Germany
1997	Concept	Computer scientist Tom M. Mitchell proposes a widely cited formal definition of machine learning: "A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E." — offering an operational, experimentally-grounded definition rather than an aspirational one, directly framing learning in terms of measurable performance under controlled tasks.	United States
1998	Policy	The U.S. Department of Health and Human Services announces a policy change to the population standard used for age-adjusting death rates in its publications.	United States
1998	Experiment	The UK Prospective Diabetes Study Group publishes UKPDS 38 in the BMJ, reporting results from a trial comparing tight blood pressure control (<150/85 mmHg) against less strict control (<180/105 mmHg) in patients with type 2 diabetes. The trial is stopped before its planned completion because tight control proves so much more effective at preventing macrovascular and microvascular complications — including death, myocardial infarction, and stroke — that continuing to withhold it from the control group is no longer considered ethical.	United Kingdom
1999	Organization	The first International Data Farming Workshop is held at the Maui High Performance Computing Center, the start of a series that would go on to meet roughly twice a year for decades, eventually organized by the SEED Center for Data Farming at the Naval Postgraduate School. The first four workshops focus on methodology — complex adaptive systems modeling, agent-based representation, and statistical experiment design — before subsequent workshops shift toward applications, with participation from countries including Canada, Singapore, Mexico, Turkey, and the United States assigning teams to apply data farming techniques to domains such as robotics, homeland security, and disaster relief. By the 30th workshop in February 2016, data farming — a term Gary Horne had coined in 1997 — had become an established methodology within NATO's modeling and simulation community.	United States
1999	Method	Rajeev Dehejia and Sadek Wahba re-examine Robert LaLonde's original 1986 dataset using additional non-experimental methods, arguing that when there is sufficient overlap between treated and untreated subject pools and unobservable covariates do not substantially impact outcomes, non-experimental methods can in fact estimate treatment effects accurately — offering a more optimistic counterpoint to LaLonde's original benchmarking critique.	United States
1999	Policy	The oral rotavirus vaccine RotaShield, licensed in the United States in 1998, is withdrawn less than a year later after post-marketing Phase IV surveillance identifies an elevated risk of intussusception (a form of bowel obstruction) among vaccinated infants. The episode becomes a widely cited case study illustrating the necessity of Phase IV surveillance for detecting rare adverse events that trials of the size typically used in Phase III cannot reliably capture.	United States
1999	Concept	Basili et al use the term family of experiments to refer to a group of experiments that pursue the same goal and whose results can be combined into joint—and potentially more mature—findings than those that can be achieved in isolated experiments.	United States
2000 (January 19)	Publication	A First Course in Design and Analysis of Experiments.	United States
2001	Method	The World Health Organization publishes a new standard population for age standardization, intended to allow health statistics from different countries to be compared despite differing population age profiles.	Global
Late 20th century	Concept	Fisher’s principles of randomization, replication, and blocking become standard features of statistically rigorous experiments in the biological and biomedical sciences.	Global
Late 20th century	Concept	Recognition that all experiments are inherently designed, with emphasis on the importance of proper planning to avoid wasted resources and invalid results.	Global
Late 20th century	Method	Advances in computing and algorithms enable practical implementation of optimal design, expanding its use in scientific and industrial applications.	Global
21st century	Application	Designed experiments are widely applied across service sectors including finance, business operations, and government, reflecting the broad adoption of statistical experimentation.	Global
2001	Method	Daniel Kahneman independently develops a protocol for adversarial collaboration roughly a decade after Erez, Latham, and Locke's earlier unnamed example, and may have been the first to use the term itself.	United States
2001	Concept	A study led by P.J. Devereaux, published in JAMA, surveys practicing physicians and reviews textbook definitions of "single-blind," "double-blind," and "triple-blind" terminology in randomized controlled trials, finding substantial disagreement both among physicians and between physicians and published textbook definitions as to which parties (patients, treating clinicians, data collectors, outcome assessors, data analysts) each term is understood to cover. The study becomes an early, influential demonstration — predating Haahr and Hróbjartsson's 2006 trial-level analysis of the same problem — that "blinding" terminology is used inconsistently enough to obscure rather than clarify a trial's actual methodology.	Canada
2001	Policy	Following further CONSORT Group meetings in 1999 and 2000, a revised CONSORT Statement is published, updating the original 1996 recommendations for reporting parallel-group randomized trials in light of growing empirical evidence on reporting quality.	Global
2002	Policy	The World Medical Association issues a "Note of Clarification" on Paragraph 29 of the Declaration of Helsinki, addressing controversy over the ethics of placebo-controlled trials. The note reaffirms that placebo-controlled methodology should generally be used only in the absence of an existing proven therapy, but carves out two exceptions where it may remain ethically acceptable even when proven therapy exists: when compelling and scientifically sound methodological reasons make a placebo necessary to determine efficacy or safety, or when the condition under investigation is minor and placebo recipients face no additional risk of serious or irreversible harm. The clarification becomes a widely cited reference point in ongoing debates over the ethics of withholding proven treatment from trial participants.	Global
2002	Method	Jochen Musch and Karl Christoph Klauer, and separately Ulf-Dietrich Reips, describe the "seriousness check" for web-based psychological experiments: asking respondents at the start of an online study whether they intend to participate seriously or merely browse the pages, in order to identify and exclude low-quality data entries before analysis. Later research finds the technique to be a strong predictor of dropout — roughly 75% of respondents who say they only want to look at the pages subsequently drop out, versus only 10–15% of those who say they intend to seriously participate — and that overall 30–50% of visitors to a typical online study fail the check.	Germany
2002	Method	Howard Bloom, Charles Michalopoulos, Carolyn Hill, and Ying Lei conduct a large-scale experimental benchmarking study of mandatory welfare-to-work programs, testing which non-experimental methods come closest to recovering experimentally estimated program effects. They conclude that none of the non-experimental methods tested approach the accuracy of a randomized experiment for recovering the parameter of interest, reinforcing concerns first raised by Robert LaLonde's 1986 benchmarking study.	United States
2002	Concept	The terms exploratory thought and confirmatory thought are introduced by social psychologist Jennifer Lerner and psychology professor Philip Tetlock in their book Emerging Perspectives in Judgment and Decision Making.	United States
2003	Organization	The Abdul Latif Jameel Poverty Action Lab is founded to scale randomized evaluations in development economics.	United States
2003	Concept	Economist Colin Camerer coins the term "experimetrics" in his book Behavioral Game Theory: Experiments in Strategic Interaction, to describe econometric techniques customized for the analysis of experimental economic data — ranging from simple treatment comparisons to complex multi-equation structural models. The term comes to describe the broader intersection of experimental economics and econometrics, encompassing both the statistical methodology for analyzing experimental data and, less commonly, the optimal design of the experiments themselves.	United States
2003	Method	Japanese researcher S. Hirata designs the loose-string task, which becomes the standard apparatus for cooperative pulling experiments: a single string threaded through loops on a movable platform such that if only one participant pulls, the string comes loose and the reward becomes unretrievable, requiring genuinely coordinated pulling for success. This design, simpler and more broadly applicable across species than Crawford's original box-and-rope apparatus, is subsequently adopted in cooperative pulling studies of rooks, ravens, wolves, elephants, capuchins, and many other species.	Japan
2003	Method	Alberto Abadie and Javier Gardeazabal introduce the synthetic control method in "The Economic Costs of Conflict: A Case Study of the Basque Country," published in the American Economic Review. The method estimates a counterfactual for a single treated unit (such as a region or country) by constructing a "synthetic" control as a data-driven weighted average of untreated comparison units, chosen so the weighted combination closely matches the treated unit's pre-intervention characteristics and outcome trajectory. Unlike difference in differences, the approach can account for confounders whose effects change over time.	Spain
2004	Policy	The U.S. Food and Drug Administration publishes the report "Innovation or Stagnation: Challenge and Opportunity on the Critical Path to New Medical Products," calling attention to a "slowdown... in innovative medical therapies reaching patients" and identifying areas of drug development needing improvement, including predictive models, biomarkers, and new clinical evaluation techniques. Through the resulting Critical Path Initiative, FDA works to foster innovation via collaborations among government, industry, academia, and patient advocacy groups; adaptive designs for clinical trials initially emerge under this regulatory framework.	United States
2004	Policy	The CONSORT group publishes an extension to the CONSORT statement specifically for cluster randomised trials, addressing the additional reporting requirements these designs need — such as accounting for intraclass correlation — beyond standard individually randomised trial reporting guidelines.	United Kingdom
2005	Experiment	Study determines that most clinical trials have unclear allocation concealment in their protocols, in their publications, or both.	United Kingdom
2005	Publication	Stuart Pocock publishes the editorial "When (not) to stop a clinical trial for benefit" in JAMA, addressing the range of practical and ethical considerations bearing on the decision to halt a trial early when interim results favor the treatment group — beyond the purely statistical threshold his own Pocock boundary had provided in 1977 — including the risk that early, promising results may not persist or generalize, and the tension between statistical stopping rules and the ethical pressure to give patients access to an apparently superior treatment.	United Kingdom
2005	Concept	John Ioannidis publishes "Why Most Published Research Findings Are False" in PLOS Medicine, arguing on theoretical and probabilistic grounds that a majority of published research claims across many fields are likely to be false, due to factors including small sample sizes, small effect sizes, flexibility in study design and analysis, and bias from financial or other interests. The paper becomes one of the most-cited and most-discussed articles in the history of the reproducibility crisis debate, predating and helping motivate the preregistration and open-science reforms of the following decade.	United States
2005	Publication	Jack Kleijnen, Susan Sanchez, Thomas Lucas, and Thomas Cioppa publish "A User's Guide to the Brave New World of Designing Simulation Experiments" in INFORMS Journal on Computing, addressing a key limitation of early data farming work: initial reliance on brute-force full factorial designs meant only a small number of factors could be investigated due to the curse of dimensionality. The paper helps establish improved experimental designs specifically suited to large-scale simulation experiments, developed in collaboration between Project Albert and the Naval Postgraduate School's SEED Center for Data Farming.	United States
2006	Publication	American psychologist Seth Roberts publishes The Shangri-La Diet, a popular diet book based on conclusions he drew from self-experimentation — informally applying the N-of-1 trial logic Allen Neuringer had proposed for self-experimentation in 1981. Roberts, who documented his self-experiments on his blog, becomes a prominent early figure associated with the later quantified self movement, in which the growing ease of personal data collection and analysis drives a proliferation of N-of-1-style personal experiments.	United States
2006	Policy	The U.S. Food and Drug Administration issues its Guidance on Exploratory Investigational New Drug (IND) Studies, formally introducing "Phase 0" trials — optional, exploratory human microdosing studies that administer a single subtherapeutic dose of a candidate drug or imaging agent to a small number of subjects (10–15) to gather early pharmacokinetic data. Because doses are too low to produce any therapeutic effect, Phase 0 trials yield no safety or efficacy data, but let developers rank candidate drugs and make go/no-go decisions based on relevant human data rather than solely on animal models, before committing to a full Phase I trial. The designation, initially unusual, becomes generally adopted as standard practice in drug development.	United States
2006	Concept	Danish researchers Mette T. Haahr and Asbjørn Hróbjartsson examine a random sample of 200 randomized controlled trials described as "double blind" and survey their authors, finding that blinding of patients, care providers, and assessors was clearly and separately described in only three trials (2%), while 56% failed to describe the blinding status of any individual involved at all. The study demonstrates that labelling a trial "double blind" does not reliably indicate which parties were actually blinded, undermining the term's usefulness as reported in trial literature.	Denmark
2006	Method	Alicia Melis, Brian Hare, and Michael Tomasello design a cooperative pulling experiment that explicitly controls for social tolerance between partners — a variable earlier studies had not accounted for — by comparing captive chimpanzee pairs known to share food readily against pairs less inclined to do so. They find food-sharing tolerance strongly predicts cooperative success, resolving much of the inconsistency across earlier chimpanzee cooperative pulling studies and establishing social tolerance as a key confounding factor that subsequent cooperative pulling experiments across many species would need to control for.	Germany
2007 (1 April)	Organization	The National Research Ethics Service (NRES) launches in the United Kingdom, a body requiring principal investigators to obtain approval for proposed research studies involving human participants before proceeding — with unapproved studies prohibited. NRES describes its purpose as reviewing research proposals "to protect the rights and safety of research participants and enable ethical research which is of potential benefit to science and society," extending the institutional ethics-committee review model established internationally by the Declaration of Helsinki decades earlier into a dedicated national service. The word "National" is later dropped from the name at an unrecorded point, and its functions are absorbed into the NHS Health Research Authority's Research Ethics Service.	United Kingdom
2008	Method	Economist Justin McCrary proposes the "density test" for regression discontinuity designs, examining whether the density of observations of the assignment variable is continuous at the treatment cutoff. A discontinuity in this density — for example, an unusually large number of students who "just barely" pass an exam relative to those who "just barely" fail — suggests that some participants may have been able to manipulate their treatment status, undermining the "as good as random" assumption the design's validity depends on. The test becomes a standard diagnostic check in applied regression discontinuity research.	United States
2008	Method	Bradley Jones, Dennis K. J. Lin, and Chris Nachtsheim introduce Bayesian D-optimal supersaturated designs in the Journal of Statistical Planning and Inference, a new class of supersaturated designs that can accommodate arbitrary sample sizes, blocks of any size, and categorical factors with more than two levels — matching or outperforming the best previously published designs on the E(s²) criterion for two-level experiments with even sample size.	United States
2008	Concept	A meta-epidemiological study of 146 meta-analyses finds that randomized controlled trials with inadequate or unclear allocation concealment tend to show results biased toward beneficial treatment effects — but only when trial outcomes are subjective rather than objective, refining earlier, less qualified claims about the relationship between allocation concealment and bias.	United Kingdom
2009	Method	Adversarial collaboration is recommended by Daniel Kahneman and others as a way of resolving contentious issues in fringe science, such as the existence or nonexistence of extrasensory perception.	United States
2010	Concept	Asbjørn Hróbjartsson and Peter C. Gøtzsche argue in a meta-analysis that observed placebo effects can arise from bias due to lack of blinding, emphasizing the importance of proper control and blinding in experimental design.	Denmark
2010	Policy	The National Centre for the Replacement, Refinement and Reduction of Animals in Research (NC3Rs) publishes the ARRIVE guidelines (Animals in Research: Reporting In Vivo Experiments), a 20-item checklist for improving experimental design and reporting standards in animal research, modeled on the CONSORT statement for reporting randomized trials. The guidelines follow a 2009 NC3Rs review finding that most biomedical journals provided little guidance on animal research design and reporting, and that large proportions of published animal studies failed to state a hypothesis, describe animal characteristics, use randomization or blinding, or fully report statistical methodology.	United Kingdom
2010	Policy	The U.S. Food and Drug Administration issues draft guidance on adaptive design for clinical trials, an early regulatory step preceding the agency's more comprehensive 2019 guidance.	United States
2010	Method	Researchers at the European Bioinformatics Institute (EMBL-EBI), led by J. Malone, publish a paper describing the Experimental Factor Ontology (EFO), an open-access ontology for systematically modeling experimental and sample variables — covering disease, anatomy, cell type, cell lines, chemical compounds, and assay information — originally developed to describe experimental variables in EBI's Expression Atlas resource. EFO is built to interoperate with existing biomedical ontologies such as ChEBI and the Ontology for Biomedical Investigations, and becomes a cross-cutting resource used for curation, querying, and data integration across resources including Ensembl and ChEMBL.	United Kingdom
2010	Method	Alberto Abadie, Alexis Diamond, and Jens Hainmueller formalize and popularize the synthetic control method in "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California's Tobacco Control Program," published in the Journal of the American Statistical Association, applying the technique to estimate the causal effect of California's 1988 tobacco-control program by constructing a synthetic "California" from a weighted combination of other U.S. states. The paper establishes synthetic control as a systematic, data-driven alternative to ad hoc comparison-group selection in policy evaluation, and the method sees subsequent application across economics, political science, health policy, criminology, and drug development.	United States
2010	Policy	The CONSORT 2010 Statement is published, consisting of a 25-item checklist and participant flow diagram, alongside an "Explanation and Elaboration" document detailing the reasoning behind each recommendation. By this point the CONSORT Statement has been endorsed by over 600 journals and editorial groups, including The Lancet, BMJ, JAMA, and the New England Journal of Medicine, and has directly inspired parallel reporting-guideline initiatives for other study types, including STROBE for observational studies and PRISMA for systematic reviews.	Global
2010	Experiment	The I-SPY 2 trial launches as an adaptive Phase 2 clinical trial platform for breast cancer, linking academic cancer centers, the FDA, the NCI, and pharmaceutical partners. Building on predictive biomarkers developed in its predecessor I-SPY 1 (2002–2006), the trial evaluates multiple experimental drug combinations against standard chemotherapy simultaneously, dropping ineffective regimens early and advancing promising ones to confirmatory trials — a widely cited real-world demonstration of adaptive experimental design accelerating drug development.	United States
2011	Method	Bradley Jones, Principal Research Fellow for the JMP Division of SAS, and Chris Nachtsheim, Chair of the Operations and Management Science Department at the University of Minnesota's Carlson School of Management, publish "A Class of Three-Level Designs for Definitive Screening in the Presence of Second-Order Effects" in the Journal of Quality Technology, developing definitive screening designs — three-level designs providing main-effect estimates unbiased by any second-order effect, requiring only one more than twice as many runs as factors, while avoiding confounding between any pair of second-order effects.	United States
2011	Method	Stefano M. Iacus, Gary King, and Giuseppe Porro introduce Coarsened Exact Matching (CEM) in the Journal of the American Statistical Association, a matching method they present as simpler, more statistically powerful, and monotonic imbalance bounding compared to earlier techniques such as propensity score matching.	United States
2013	Method	Alex Deng, Ya Xu, Ron Kohavi, and Toby Walker introduce CUPED (Controlled-experiment Using Pre-Experiment Data) at the WSDM international conference, a technique for improving the statistical sensitivity of online A/B tests by using data from before an experiment began to reduce the variance of outcome measurements — allowing web companies to detect smaller effects with the same sample size, or the same effects with less traffic. The technique becomes widely adopted in large-scale online experimentation at companies running many concurrent experiments on millions of users.	United States
2014	Concept	Uri Simonsohn, Leif Nelson, and Joseph Simmons — the researchers behind the blog Data Colada — coin the term "p-hacking" to describe the practice of running many statistical analyses on the same data set and reporting only those that yield a statistically significant result, dramatically understating the true risk of false positives. The paper also introduces the "p-curve" technique for detecting p-hacking in published literature by examining the distribution of significant p-values across studies.	United States
2014	Concept	Peter Keevash proves the existence of nontrivial Steiner systems for all t ≥ 6, resolving a long-standing open problem in design theory, in "The existence of designs." His proof is non-constructive, and as of 2019 no explicit Steiner systems are actually known for large values of t despite the existence proof.	United Kingdom
2014	Concept	Andrew Gelman and John Carlin (statistician) propose Type S (sign) and Type M (magnitude) errors as complements to the traditional Type I/Type II framework, in "Beyond power calculations: Assessing Type S (sign) and Type M (magnitude) errors," published in Perspectives on Psychological Science. A Type S error occurs when a statistically significant result's estimated effect has the wrong sign relative to the true effect — a risk that grows in low-powered studies — while a Type M error concerns how much a significant result's estimated effect size is inflated relative to the true effect, a consequence of selection bias introduced by using significance itself to filter which results get reported.	United States
2014	Method	Researchers at McMaster University led by Michael Walsh introduce the Fragility Index (FI) in "The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index," published in the Journal of Clinical Epidemiology. The FI measures the minimum number of participants in a trial whose outcome would need to switch from a non-event to an event (or vice versa) to change a statistically significant result to a non-significant one, exposing how easily the conclusions of many published, adequately-powered randomized controlled trials can be reversed by the reclassification of only one or a few patients — a concern the authors argue is especially acute for trials with small sample sizes and few observed events.	Canada
2015	Experiment	Journalist John Bohannon deliberately conducts and publishes a fraudulent study claiming chocolate consumption accelerates weight loss, using p-hacking techniques — considering 18 different variables during testing until one produced a statistically significant result — to demonstrate publicly how easily scientific-sounding claims can be manufactured from real but hollow statistical analysis. The hoax was picked up uncritically by numerous media outlets before Bohannon revealed it was an intentional social experiment exposing weaknesses in science journalism and nutrition research.	Germany
2015	Concept	Megan L. Head, Luke Holman, Rob Lanfear, Andrew T. Kahn, and Michael D. Jennions publish "The Extent and Consequences of P-Hacking in Science" in PLOS Biology, analyzing the distribution of p-values reported across a large sample of published papers to empirically estimate how widespread p-hacking actually is in scientific practice, rather than treating it as a purely theoretical risk. The study also examines factors — including pressure for early stopping mandated by some animal ethics boards when interim results reach significance — that can inadvertently encourage the practice.	Australia
2016	Concept	A study finds that symphony orchestra auditions conducted behind a curtain, blinding judges to a performer's gender, increase the hiring of women — a widely cited real-world example of blinding used outside clinical or laboratory settings.	United States
2016	Method	Susan Athey and Guido Imbens publish "Recursive partitioning for heterogeneous causal effects" in PNAS, pioneering machine learning techniques — adapting decision-tree methods to causal inference — for detecting and characterizing how treatment effects vary across subpopulations, addressing the external validity question of whether a treatment's effect generalizes across different subsets of people, times, and contexts rather than assuming a single homogeneous effect. Stefan Wager and Athey extend the approach using random forests in a 2018 follow-up paper.	United States
2017	Policy	Nature Human Behaviour launches with its inaugural issue offering authors the option of publishing a registered report — a format in which the study proposal is peer-reviewed and receives in-principle acceptance before the research outcomes are known, guaranteeing publication regardless of result. The journal's founding editorial frames the format as a way to neutralize publication bias and remove incentives for practices that undermine the validity of research, shifting editorial evaluation from a study's results to the soundness of its questions and methods.	United Kingdom
2017	Organization	The International Collaborative Network for N-of-1 Trials and Single-Case Designs (ICN) is established, co-chaired by Jane Nikles and Suzanne McDonald, as a global network of clinicians, researchers, and consumers interested in N-of-1 trials and single-case experimental designs; it grows to over 400 members across more than 30 countries.	Global
2017	Method	Peter M. Aronow and Cyrus Samii publish "Estimating average causal effects under general interference, with application to a social network experiment" in the Annals of Applied Statistics, developing a general method for computing exposure probability matrices in experiments where SUTVA (the stable unit treatment value assumption) does not hold — such as experiments conducted over social networks where treated units can influence untreated neighbors — enabling inverse probability weighting approaches, including the Horvitz–Thompson estimator, to correct for the resulting bias when estimating average treatment and spillover effects.	United States
2017	Concept	A historical case study documents the subversion of allocation concealment in a randomized controlled trial, part of a broader body of evidence that sealed-envelope and even centralized allocation concealment methods remain vulnerable to circumvention by study personnel — including researchers opening envelopes prematurely, holding them up to light, or keeping lists of previous allocations (reported by up to 15% of study personnel surveyed).	United Kingdom
2018	Concept	A meta-epidemiological study of 540 randomized controlled trials in oral health research, led by Humam Saltaji, formally critiques the terms "double-blind" and "triple-blind," noting that blinding can be implemented at multiple distinct levels of a trial — participants, outcome assessors, care providers, data analysts, and investigators — and that "double blinding" or "triple blinding" may refer to any two or three of these levels without specifying which, producing "conceptual and operational ambiguity." The study finds that two-thirds of the trials examined were not in fact described as double-blind at all, and recommends, in line with CONSORT guidance, that researchers abandon the "double"/"triple" blind terminology altogether in favor of explicitly stating who was blinded and to which components of the trial.	Canada
2018	Experiment	Brett Gordon, Florian Zettelmeyer, Neha Bhargava, and Dan Chapsky use data from large-scale field experiments on Facebook advertising to test whether standard observational methods — including propensity score matching, stratification, and regression adjustment — can recover the true causal effects of online ads on checkout, registration, and page-view outcomes. Despite the unusually rich variation available in social-media advertising data, they find observational methods are unable to accurately recover the causal effects established by the underlying randomized experiments, providing a striking contemporary demonstration of the same experimental-benchmarking concerns first raised by LaLonde in 1986.	United States
2018	Concept	A study by Brian Nosek and colleagues proposes preregistration as a safeguard against p-hacking: researchers submit their data-analysis plan to a journal before beginning data collection, precluding after-the-fact manipulation of the analysis to reach statistical significance.	United States
2018	Experiment	In a satirical study published in the BMJ's annual Christmas issue, Robert Yeh and colleagues randomize 23 volunteers to jump from an aircraft wearing either a parachute or an empty backpack, finding no significant difference in death or major injury between the two groups — a result made possible only because the aircraft was parked on the ground and participants jumped roughly two feet. The trial becomes a widely cited illustration, aimed at critics of evidence-based medicine's reliance on randomized controlled trials over clinical judgment, of how a methodologically valid RCT can nonetheless produce a misleading or non-generalizable conclusion when applied to a scenario where the intervention's benefit is already self-evident, and of the necessity of peer review in contextualizing trial results.	United States
2018	Concept	Kenneth Schulz and colleagues publish a retrospective account of the introduction and adoption of the term "allocation concealment," distinguishing it from the related but distinct concept of blinded experiment: allocation concealment prevents foreknowledge of treatment assignment before randomization (addressing selection bias), while blinding conceals group identity after allocation (addressing ascertainment bias). The authors argue the earlier common term "randomization blinding" had confusingly conflated these two distinct sources of bias.	United States
2019	Recognition	Nobel Prize in Economics awarded to Abhijit Banerjee, Esther Duflo, and Michael Kremer for experimental approaches to alleviating global poverty.	Sweden
2019	Policy	The U.S. Food and Drug Administration provides guidance on the use of adaptive designs in clinical trials, formalizing regulatory standards for flexible experimental designs.	United States
2019	Concept	Gary King and Richard Nielsen publish "Why Propensity Scores Should Not Be Used for Matching" in Political Analysis, arguing that propensity score matching — developed by Rubin and Rosenbaum in 1983 and widely adopted since — actually increases model dependence, bias, and inefficiency relative to other matching methods, and should no longer be the default recommendation. The paper marks a significant reversal in applied best practice among political scientists, economists, and other users of observational causal-inference methods.	United States
2019	Concept	Aleksander Fabijan, Jayant Gupchup, Somit Gupta, Jeff Omhover, Wen Qin, Lukas Vermeer, and Pavel Dmitriev publish "Diagnosing Sample Ratio Mismatch in Online Controlled Experiments" at the ACM SIGKDD conference, formalizing sample ratio mismatch (SRM) — a statistically significant discrepancy between the expected and actual ratio of treatment and control group sizes in an experiment, often caused by failures in randomization or measurement instrumentation in online A/B testing — and proposing a taxonomy and detection rules of thumb, typically using a chi-squared goodness-of-fit test, to help practitioners identify SRM and avoid drawing conclusions from biased experimental data.	United States
2019	Method	Bradley Jones, Ryan Lekivetz, Dibyen Majumdar, Christopher J. Nachtsheim, and Jonathan W. Stallrich publish "Construction, Properties, and Analysis of Group-Orthogonal Supersaturated Designs" in Technometrics, developing group-orthogonal supersaturated designs (GO-SSDs) that improve factor screening when the number of variables exceeds the number of experimental runs.	United States
2020	Experiment	In response to the COVID-19 pandemic, the World Health Organization launches the Solidarity trial, European researchers launch the Discovery trial, and UK researchers launch the RECOVERY Trial — each a large-scale, multi-arm adaptive design clinical trial of candidate treatments for hospitalized patients with severe COVID-19 infection. The adaptive designs allow ineffective experimental treatments to be dropped quickly and replaced with others as evidence accumulates, letting researchers adjust trial parameters in near real time rather than waiting for a trial's predetermined endpoint, and become widely cited examples of adaptive trial methodology deployed at unprecedented speed and scale during a public health emergency.	Global
2020	Policy	An international working group supported by the National Centre for the Replacement, Refinement and Reduction of Animals in Research publishes ARRIVE 2.0, a revision of the 2010 ARRIVE guidelines splitting the original 20-item checklist (effectively 38 items counting sub-items) into an "Essential 10" checklist of basic minimum requirements and a "Recommended Set" of 11 additional items, in response to studies finding the original guidelines — despite endorsement by over 600 journals by 2016 and over 1,000 by 2020 — had been largely ignored by researchers and made little measurable impact on reporting quality.	United Kingdom
2020	Method	Researchers Elias Garcia-Pelegrin, Alexandra Schnell, Clive Wilkins, and Nicola Clayton argue in Science that magic tricks offer a novel approach to hypothesis testing and experimental design in the study of animal cognition: presenting a trick that reliably fools humans to a nonhuman animal, then using its behavioral response (e.g. prolonged looking time, taken as an indicator of surprise) to probe which cognitive blind spots — such as expectations about object permanence or attention control — the animal shares with humans. The approach faces practical challenges distinct from conventional experimental designs, including getting animals to attend to a human demonstrator and inferring "surprise" without verbal report.	United Kingdom
2020	Concept	Jérôme Adda, Christian Decker, and Marco Ottaviani publish "P-hacking in clinical trials and how incentives shape the distribution of results across phases" in PNAS, providing empirical evidence that the distribution of statistically significant results in clinical trials shifts in ways consistent with p-hacking as financial and career incentives change across different phases of drug development — connecting the general phenomenon of p-hacking directly to the economics of pharmaceutical research.	Italy
2020	Concept	Gonville and Caius College, Cambridge removes a stained-glass window honoring Ronald Fisher — which had depicted a 7×7 Latin square in reference to his Design of Experiments — because of Fisher's association with eugenics. The removal, following a vote by the college's governing body, becomes a widely covered episode in the broader reckoning of statistics and genetics with the eugenic commitments of several of the field's founding figures.	United Kingdom
2021	Method	Researchers Marianne Gunderson, Kristian A. Bjørkelo, and Jill Walker Rettberg, together with a team of larp designers led by Anita Myhre Andersen, stage Sivilisasjonens venterom ("Civilization's Waiting Room") in Bergen, Norway — a live-action roleplaying game (larp) designed as a research methodology in its own right, intended to let participants practice ethical decision-making around emerging surveillance and machine-vision technologies (facial recognition, deepfakes, VR) within a fictional AI-governed society. The project is analyzed in subsequent scholarship as a "mimetic method" related to design fiction, illustrating how immersive, participatory fiction can function as a qualitative research design distinct from conventional controlled experimentation.	Norway
2021	Experiment	Garcia-Pelegrin, Schnell, Wilkins, and Clayton perform three sleight-of-hand magic tricks — palming, the French drop, and fast pass — on six Eurasian jays, an early empirical test of the magic-based experimental paradigm they had proposed the previous year. The jays are not deceived by palming or the French drop, both of which depend on human hand movements setting expectations about object location, but are successfully deceived by the faster "fast pass" technique — the first demonstrated case of a magic trick fooling a nonhuman animal.	United Kingdom
2022	Organization	Philip Tetlock and Cory Clark propose adversarial collaboration as a vehicle for scientific self-correction, arguing it helps expose false claims by forcing exploration of rival hypotheses; their work leads to the University of Pennsylvania School of Arts & Sciences establishing the Adversarial Collaboration Project to formally support and encourage the approach across research questions.	United States
2024	Concept	Cochrane (organisation) publishes an updated meta-epidemiological review by Ingrid Toews and colleagues comparing healthcare outcomes assessed in observational studies against those assessed in randomized controlled trials, finding little evidence of significant effect differences between the two designs regardless of the specific observational design used — a notably more optimistic finding for observational methods than the experimental-benchmarking critiques raised by Robert LaLonde (1986) and others in economics and social science.	Global
2025	Policy	The CONSORT Group issues the CONSORT 2025 checklist, superseding the 2010 statement by adding seven new items, revising three, removing one, and introducing a new open-science reporting section covering trial registration, protocol and statistical-analysis-plan access, data sharing, and conflict-of-interest disclosures.	Global
2025	Concept	Physician-scientist Pavlos Msaouel publishes "The curious rise of randomised non-comparative trials" in Significance, critiquing a design — traced by a companion meta-epidemiological review to oncology studies dating back to 2002 — in which participants are randomized to different treatment arms, but each arm is compared only against a historical control or predefined benchmark rather than against each other, functioning in practice as multiple concurrent single-arm studies. Msaouel argues the term "randomisation" in these designs is largely "talismanic," misleadingly suggesting the methodological benefits of a true randomized controlled trial despite providing no formal between-arm comparison; a companion analysis by Alexander D. Sherry, Msaouel, and Ethan B. Ludmir finds that despite this, roughly half of published randomised non-comparative trials still report some form of comparison between arms anyway.	United States
2025	Policy	Miguel Hernán, Aidan G. Cashin, and an international group of collaborators publish the TARGET Statement in JAMA, a reporting guideline for observational studies that use the "target trial emulation" framework — designing an observational study to explicitly emulate the randomized controlled trial that would ideally answer the same causal question, in order to reduce common biases in observational causal inference. The statement extends the lineage of trial-reporting guidelines established by CONSORT (for randomized trials) and STROBE (for observational studies generally) to this specific and increasingly widely used design.	United States
2020s	Method	Machine learning is integrated with experimentation to estimate heterogeneous treatment effects and optimize interventions	Global

Numerical and visual data

Google Scholar

The following table summarizes per-year mentions on Google Scholar as of December 14, 2021.

Year	"experimental design"
1900	30
1910	17
1920	13
1930	19
1940	62
1950	425
1960	1,590
1970	6,240
1980	11,400
1990	17,000
2000	53,200
2010	162,000
2020	90,600

Google Trends

The chart below shows Google Trends data for Design of experiments (Topic), from January 2004 to December 2021, when the screenshot was taken. Interest is also ranked by country and displayed on world map.

Google Ngram Viewer

The chart below shows Google Ngram Viewer data for Design of experiments, from 1900 to 2019.

Wikipedia Views

The chart below shows pageviews of the English Wikipedia article Design of experiments, from July 2015 to November 2021.

Meta information on the timeline

How the timeline was built

The initial version of the timeline was written by Sebastian Sanchez.

Funding information for this timeline is available.

Feedback and comments

Feedback for the timeline can be provided at the following places:

• FIXME

What the timeline is still missing

• Vipul: "will this timeline eventually talk of things like double-blinding✔ , triple-blinding✔ , placebos✔ , RCTs✔ , etc., right? You have blinding but I guess the rest are variants on the idea".✔
• Vipul: "Cover "Statistical significance"✔ , "p-values" and preregistration."✔

• Glossary of experimental design
• Category:Design of experiments
• Design of experiments (check See also list)