On incentives

I wanted to share a bunch of links that I think addresses the problem with the incentive structure in science – at least in psychology (but, I’ll link in Athene Donald too, and she’s in physics).

First up, Tal Yarkoni’s post of five excuses for status-quo, of which he only takes one seriously. Good commentary on that one too. It’s the last one “but it will hurt my career to be overly honest” that has some merit to it. You need to publish, and it needs to be interesting results (supposedly) otherwise no grants, no tenure, no career. And this is a distorting force, one where the pressure to eat the marshmallow may be too tempting for some.

Akira O’connor writes about finding out someone he knows have committed research fraud. In the second to last paragraph, he discusses the incentive structure, and its corrosive effect.

This paper called Reforming Sciences: Methodological and Cultural reforms, which is in part written by Ferric Fang (who was in the news last fall from his findings on retractions due to fraud – I link in Carl Zimmer’s write-up). I think it outlines the issues quite well, and also suggests a number of reforms. But, as their Macchiavelli quote on top suggests, it may not be so easy.

Today, Athene Donald writes that it is time to resist the pressure. The pressure to publish in the most prestigious journals. The slavery to impact factors, that in the end distort science. Chris Chambers puts in a nice response. He links to Björn Bremb’s suggestion of publishing going back to the libraries.

Speaking of Björn, here is a review of his unintended consequences of impact factors paper written up by Mathbabe.

I can’t help but linking in this somewhat older blog post from Robert Trivers (because, Robert Trivers … oh hopeless fan girl set sin). It is his take/review of Jelte Wichert’s work that I brought up in my earlier post on the February 5th reforming practices symposium.

Keith Laws bring up the distortions due to incentive structure in this recent BBC debate with a philosopher (who think psychology is not a science, and should take its cue from humanities – humanities are all wonderful and such, and I even collaborate with some, but that kinda made my geeky quantitative heart shudder). It is about 35 minutes in, and worth a listen. Keith recently wrote about these issues in his article Negativland, which I don’t think I have linked in earlier. (And if I did, it is worth another link).

After the symposium, I did feel that the hurdle is the incentive-structure. The apparatus within which we work, that we need to satisfy – and I have no idea right now what can be feasibly done with it. Talking and papers are fine, but it needs doing. File-drawers, power and effect sizes have been shouted about since well before I started doing science, and it is only now it seems that something is changing. The over-production of PhD’s and shrinking funding in the US pushes the competition into the “mean chicken” stage (see Fang’s paper under “nobody here but us chickens”.

But, I think this is the next thing that needs to be tackled.

Edit on March 19.  Aaaaack. Richard Harper (who you should follow on Twitter) kindly pointed out to me that there should only be ONE e in Negativland, as it refered to a band (who I now, after perusing Wikipedia, got their name from a song, and who was then sued by U2, but, that was clearly after I went back to all classical all the time, and embrased my inherent dorkiness.) I knew there were refs to bands/experimental music in that article (from twitter again). I didn’t spot wich ones. Will I get some cool cred back if I let it slip I actually owned a record by the Residents once upon a time?