Skip to content

Debating the merits of pre-registering research protocols.

July 25, 2013

When the Chris Chambers/Marcus Munafo Manifesto was published in the Guardian, a twitter-debate broke out about the pro’s and con’s of pre-registration. Not everybody is enamored with the idea (as I have linked in on these pages before), and one of them distinctly against was Sophie Scott. She promised to put her thoughts down. Here they are, in Times Higher Education. Also, along with the article, she linked in this series of commentaries from people who have reservations against pre-registration, and their reasons for being either ambivalent or against.

Right now (July 25, 2013) there is a Twitter debate going on, and Pete Etchell’s has more or less promised to collate them. I’ll link that in when that happens. Debates are always good. It is so easy to go “what could possibly go wrong” when one is enthusiastic.*

One of the worries is that this is even more hoops to run through, puts even more power in the hands of journals and peer-reviewers (and the possibility to block science that goes against… well. Some people clearly shun no means to try to stop a view that they deem wrong, so I can kind of see it).

There is also a worry that non-preregistered and exploratory work will be seen as “lesser” than the pre-registered work, and a worry that it will water down the research even further.

Lots of interesting points being brought up.

I don’t think registered reports are a panacea for fixing what ails science. But, hey, let’s try something, or let’s try a lot of different things. Rolf Zwaan and Micah Allen had an exchange where one of the points was that let it be a trial run with business as usual (or other alternative models) as comparisons. See how things work out. We are researchers after all. Kind of…. Exploratory. Not all journals are implementing it right away, and I don’t think anyone is touting it as the one fix.

I’m sure there will be more blogging about this, as it evolves, but here is one from Daniel Lakens.

*On edit – Pete is quick! Here is the storify!

And, his accompanying post.

Advertisements
3 Comments
  1. Rock the boat permalink

    ” But, hey, let’s try something, or let’s try a lot of different things. Rolf Zwaan and Micah Allen had an exchange where one of the points was that let it be a trial run with business as usual (or other alternative models) as comparisons. See how things work out. We are researchers after all. Kind of…. Exploratory.”

    Hell yes !! Do something, anything. I don’t understand why you scientists are so hung up on these “old ways” of doing things. Try something, see what it does to citations, replicability, etc. Be a freakin’ scientist and try and better science, use logical argumentation and facts, and investigate things !! It’s fun !

    (p.s. You could even think of pre-registering this experiment 😛 or simply call it “groundbreaking” or “exploratory” to do so. That always seems to be a “scientific” thing to say :P)

    Shake things up a bit. “Rock the boat” as they say:

  2. Rock the boat permalink

    “One of the worries is that this is even more hoops to run through, puts even more power in the hands of journals and peer-reviewers (and the possibility to block science that goes against… well. Some people clearly shun no means to try to stop a view that they deem wrong, so I can kind of see it).

    There is also a worry that non-preregistered and exploratory work will be seen as “lesser” than the pre-registered work, and a worry that it will water down the research even further.”

    Yes, maybe these things can be resolved by having different rules. You could, for instance, have journals accept findings as “significant” when they are p <.47, or you could also have journals not have their publications adhere to the APA-style of writing. You could just come up with your own style and have everything less than p = .47 be accepted as something "significant". This could al help in having researchers jump through less hoops and puts less power in the hands of journals and peer-reviewers.

Trackbacks & Pingbacks

  1. The big pre-registration debate › Counterbalanced

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: