Preregistration – counter arguments
More and more places are following the pre-registration initiative, as my twitter stream noted (and, of course, I forgot to collect so I could share), which I think is a nice thing. It wasn’t the first thing I thought of when thinking about fixing science, but I think it has a place.
But, not everybody is enthusiastic about it. A couple of months back, there was a back and forth from Neuroskeptic with a Sebastiaan Mathot (I stuck it in here too).
Today, James Kilner comes with his counter argument, which I think are similar. It is the worry that exploratory research is now going to become the b-side, that we are not going to trust researchers, and thus straight jacket them, and possibly the rigidity of it.
The way I see it (and I think I’m reiterating Neuroskeptic’s argument) is that first it won’t be the only way of publishing science. There is a place for the exploratory. You just should not pass it off as confirmatory if it is not. For trust, there is a problem right now. Sure, you want to trust scientists, but scientists are people who respond to incentives, and the incentives have been…. well… not in the direction of robust science. You know, I am a scientist, and I’m not sure I trust the scientific record. Not because I distrust the scientists per se, but, because of the incentives. (I think most scientists are honest scientists. But there are enough horror stories around. And, let’s face it, lots of us thought that n=30 was powerful enough without ever having performed a power calculation, or figuring out how the power differs between repeated measures and one shots)…
Anyway, it is well worth a read, and I think these voices need to be heard also, and it is worth pondering. Nobody knows all of the possible directions of the fallout from change. As it is, the changes are gradual, and we can evaluate them and tweak things, which I think is a good way to go.