Failure – what doesn’t get published in Psychology (for good reason?)
In the comments recently, Sandgroper said that Engineering publishes failures all the time. I was curious about that, and, pretty much as I thought, the failures are of the Tacoma Bridge type – but I guess not always so spectacular and freaky. When does the system, the engineered piece break down, I take it. You could (and sometimes do) have this in psychology also – looking for boundary conditions. Where does it work, where not. Supposedly for things that are fairly robust. (You would be unlikely to hear about it, because it can be pretty boring about the little details of some phenomenon that does not seem to have any bearing on anything which is anywhere near as interesting as bridges).
But, the failures we don’t publish are of another nature. Even if you build on earlier work (don’t invent a paradigm, if you can borrow one and retrofit it), you can get messy weird results that you have no clue what they mean, and you can’t build a good story to tell, and have no idea where to really go next (yet). Or, eventually, you work out the kinks, you get a working paradigm and you publish that, and not the prototypes that crashed and burned.
Maybe an example is best, so I take my first 5 unpublished data-collections (among many).
Emotional states influence cognition. This has face-validity. That is, intuitively it makes sense, from just crude consultation with experience. You are down in the dumps, and you remember a lot of other times when things were sad and went against you, and you look at the future as perhaps sad and bleak. But, in good moods, the memories are different the judgments are different etc. There is quite a bit of research on this by now: Make your participants sad or happy or angry or fearful (movies, writing stories, music, presents etc are useful – emotion inductions work. Always beautifully significant results there). Oatley, Keltner and Jenkin’s text book “understanding emotions” describes a bunch of them (I use that one as my textbook).
When I started at the Niedenthal Lab, one of the questions Paula had looked at was tracing how early emotions influences cognition, and her thesis was that it begins even at the perceptual state. The formal theory we used to reason with (Bower’s network theory) predicted this, and she had found evidence that emotional state influenced perception of words with a congruent meaning – as long as this was specific (sad state influenced perception of sad words but not other negative words in a lexical decision task). She had pilot data that suggested this was also the case for emotional expressions – that suggestive p = .07 level. So, she figured she would stick her new graduate student at the task.
This was the basic task: the students would come in, and be seated in a nice, dim cubby. They got to see a series of short movie clips (that they were told they were going to rate). Then, they put headphones on, where they got some music piped in, and did the face-perception task while listening to the music. They simply had to indicate whether the face belonged to a man or a woman. The expressions on the faces varied: some were happy, some were sad, some were neutral. We simply recorded reaction times and error rates (they were told to be as accurate and quick as possible – which is called the speed-accuracy tradeoff). The movies and the music were designed so they would put you in a happy, neutral or sad emotional state during the task. As I mentioned, this works beautifully. Worked every time. I don’t think I have ever had a marginal emotion-induction.
The hypothesis was that people would process the expressions that were congruent with their emotional state faster than those that were incongruent – and thus they would respond quicker to whether the face was male or female. SO, we expected faster responding to happy faces by the happy students, and sad faces by the sad students. (There are some inherent differences also in response-times when you are sad or happy, as well as certain varying advantages to the different emotions, but we figure we could partial those out).
We didn’t tell them beforehand what it was about, and didn’t ask them to sort on expression, because we did not want to alert them to the hypothesis. The idea we had was that this was incidental. The emotion itself influenced perceptual fluency for congruent stimuli.
I ran 30 participants in each condition. No support for the hypothesis. Lots of other things going on (gender differences, etc), but no evidence of emotion congruence. I ran it again. Nothing. I ran a version where we messed with the stimuli to make it more difficult to perceive (in one version introducing noise. In another dimming the images so you saw the faces as if in a dark room. Expression still visible, but stimuli harder to process). Nothing. In a final version, I used chimeric stimuli – faces that were composites of the left and right sides of faces. Sometimes from the same individual, other times from different individuals, and differing expressions. (I don’t remember if the expressions were the same on both halves, actually – but the stimuli was somewhat different from the ones that are used in the chimeric face-test which tests for laterality of emotion expression). Again, nothing that was publishable. I recall tons of results, but none that we expected. The research was enough for me to do my required internal research papers, but I never got a publication.
What we suspect is that when expressions are so clear and unambiguous as these were, emotional state will do nothing to facilitate or inhibit processing. In some ways, this could be seen as a boundary condition, of course, and might be interesting, but nobody would publish this.
Emotional state does influence perception of emotional expressions, though. Just not the high intensity ones. In other research that we did in the lab, participants looked at morphed movies, where the person in the photograph apparently changed expression from one to another. And, here we found an effect in offset. (Published – the link is a PDF, and yours truly is a lowly last contributor). But, even here, it was not a straight-forward story, and the first attempt to publish the research in a single bigger paper failed. The results were just contradictory, it seemed (although perhaps not). When the expressions were morphed between two expressions (happy to sad), offset was seen earlier in congruence – happy persons saw the shift to another expression at an earlier point. But, when it was from emotion to neutral, the offset was seen longer. The idea that Paula advanced is that one would use mimicry as information about where the offset took place. When the morph went from happy to sad, there may be early information about the shift to an incongruent emotion, whereas when it goes from happy to neutral, there is no such mimicked indicator. When we stuck pens in their mouths, the offset was all over the place, suggesting a disruption.
Yes, that was complicated, wasn’t it, and that is probably why it got rejected, but then got accepted as two separate smaller papers, where the story could be more coherent in each of them.
Hmmmm – yes. We had to break it up in bite-sizes, rather than submitting a messy but in some ways more telling story.
The work on expressions and internal states is still ongoing in psychology (and I’m still at it). These are messy, subtle, contextual effects. They are small, they are varying, they depend on senders and receivers and cultures, and ethnicities, and expressions, and attitudes, and group-belonging, and mimicry or not. Effects are small – you need a lot of people to find it. But, small effects in these kinds of emotional communications could have large consequences, so it is not unimportant. (I set my own work up as: you are meeting a scowling man – does it matter if it is down town, or on campus? During daytime or at night? Does it matter if he looks like a Ralph or a Razib? Does it matter what you think about people like Ralph or Razib?).
And, because the effects are subtle, it is a pain getting them, and you have datasets with data that you don’t quite understand lingering, which may say something about the phenomenon, and maybe not, but will never be published, because you cannot say anything coherent about them. For most of what I did, ideology may not matter much on what I do, or whether it is published (there are controversies, but they do not follow the political lines these days. In the old days there were the culture vs biology controversy, but then there weren’t that many publishing. This still goes on, but has a bit different flavor).
Sexiness seem to matter though. In the textbook I used – which had a chapter on emotion processing that would have been OK in 1995, they used to have a section on how emotional stimuli captures attention.
This is another area where I have like, oh 10-15 datasets laying around that will never see the light of day, because, well, can’t interpret.
Yes, you do get a search effect – seems like expressive faces/threat stimuli are found faster – kinda maybe perhaps, and mostly if you actually are phobic, so that you are permanently on the outlook for the scary stuff.
It isn’t attentional capture though (which means that the stimulus will draw your attention, even if you didn’t look for it. Very few things will). The seach for the anomalous face, or the anomalous animal takes quite a long time – up to a second, which is an eternity for these kind of things. You will actually look at several of the stimuli – the scary stuff will not pop out.
This is also an area where I know there is other data-sets lingering out there, where no effect of emotional stimuli can be found.
But, sexy enough to put in a textbook that hadn’t updated the Emotion section with research that was an awful lot more robust than that.
I haven’t looked closely at that literature since 2005 – but my hunch is that you need to be emotional for the attentional effects to take place. Maybe I will look for that one of these days, because it is interesting (nice evolutionary connection too).
The stuff I do that involves prejudice could easily be subject to the liberal bias, and I think it is. Possibly not by me (since I belong to the stupid political middle), but there is a distinct activism flavor in the literature, which I think is problematic. The ability to be prejudiced, the ability to discriminate against those not you is, I think, a human universal, and does not follow skin-color. Besides, skincolor is not needed for that. (Where I grew up it was talking funny – because we were all pale, blueeyed and mottle-haired anyway, so how else could you discriminate).
Yet, as I think about this – and I do think about this, I do think psychology has come up with interesting stuff, and interesting valuable information. Which does not mean that everything is hunky-dory, and I’m far from the only one unhappy. (There will be a cool and interesting issue in one of the APS journals fairly soon, and I will be sure to tweet and blog it).