Repeat when necessary
One of my fave colleagues used to use Pooh-sticks to illustrate an important methodological permise. “Let’s see if it happens again”. You find something interesting, let’s see if it can be repeated. Have I mentioned it before? Well, it bears repeating.
And, I fully understand Greg Francis’ admonition that apparent repetition may not be the one think that is important (especially when the non-repetitions are hidden in the reedy banks of the file-drawer, to mix pooh-sticks with research again). But, to actually trust that something is there, it needs to be replicated, and tested for effects.
There were a couple of papers/posts out this past week about the need for replication, and I’ll link them here. In part so I can find them again, in part to share, of course.
The first is John Ioannides short Frontier’s paper. I especially like the middle in his title “replication is science”. He focuses now on the embarrasment of riches that are genetics. With so much data, you have to handle discovery differently.
The second is Jonathan Russel’s “If a job is worth doing, it is worth doing twice” in Nature. This iis about the problem with non-reproducibility in bio-medical research, and proposes that more emphasis needs to be put on reproducibility. Not a novel thought. Needs to be repeated. Perhaps, if we repeat it enough, it will come true (like, 1000 elvises…oh, never mind). He goes through a number of possible objections, and answers them.
I also liked Kas Thomas’ blog post on the “we only publish the positive” issue. Discussed before. Must be discussed again. Discussed and presented, until we actually begin to publish our non-findings.
Unlike voting, replication should be done early and often!