I have had the pleasure to conduct a variety of experiments, watch numerous management talks and take many classes on the scientific method. During these adventures I've noticed that people seem to misunderstand (or not comprehend the full power of) replication.
Replication is the repetition or duplication of an experiment. On the surface this seems pretty straightforward; why would someone want to replicate or duplicate an experiment? They do it to double-check that the results are correct.
If I want to test to see if hummingbirds prefer red or yellow feeders, I can put out two colored feeders and count the number of visits. Great! Right? Not exactly.
The experiment is not replicated. What if one feeder is closer to something the hummingbirds do not like? They would avoid that feeder and prefer the other no matter what the color. Replication solves this problem.
Professionals get it wrong sometimes too. I just visited a colleague who has a growing facility with 3 replicate greenhouses (3 repeated greenhouses in a row). One of the researchers using his facility had all the trials in one greenhouse. It was the south-facing greenhouse. What if the south-facing greenhouse was slightly warmer than the average greenhouse? They will not know if this influenced the result.
How many weed trails have we seen that have been conducted at a site, spray herbicide A here, herbicide B there. Done. Great! Right? Not exactly. It certainly helps to have the initial trial, and its even more helpful to replicate the experiment.
Replications can occur across space or time (this sounds like Star Wars), and on the ground it means experiments can be repeated over a variety of distances, or over different seasons.
Replications Across Space
Conducting an experiment is good; conducting the same experiment (especially when it comes to weed management) in multiple field sites, counties or states is even better. There are more herbicide resistant weeds in the Central Valley than outside that area. Same species, different resistance. How do we know? Replicated experiments across the state.
Replication Across Time
Weed scientists like to conduct herbicide trials, and I do them myself, and how many of them are conducted every year for 5 years at the same site? This type of replication allows one to determine if the result is the same during different years or seasons. Some populations of hairy fleabane (Conyza bonariensis) are resistant to the herbicide glyphosate during the summer, (they will not die when sprayed) but when those same resistant plants are sprayed during the winter they are susceptible (see footnote*). This is fascinating! Spray in summer and it grows like a weed, spray in winter and growth is reduced. The only way this was discovered was by replicating experiments across time; applications were made in spring, summer, fall and winter. On a side note the researchers in charge of this project thought they mixed the resistant with susceptible seeds and repeated (i.e. replicated) the experiment several times until they understood the results!
Replication Across Space and Time
It's the gold standard; the researcher conducts an experiment at multiple sites and repeats it over different durations. It also takes a lot of work. One simple experiment now has multiple sub-experiments. But you know how well it really works.
In Conclusion
Just because we see weed trials doesn't mean it has been fully vetted. Simple trials are good, they teach us a lot. Multiple trials are better. Control plot, treated plot, good. Control plot, treated plot, repeat, repeat, repeat here, repeat there, repeat next year, and repeat in the winter, ...even better.
*Note: I am not advocating spraying resistant hairy fleabane with glyphosate, you should consider an IVM (or IPM) program to tackle this problem.