Jun 21, 2019 01:59 AM
(This post was last modified: Jun 21, 2019 02:01 AM by C C.)
https://slate.com/technology/2019/06/sci...reats.html
EXCERPT: Science is supposed to be self-correcting. Ugly facts kill beautiful theories, to paraphrase the 19th-century biologist Thomas Huxley. But, as we learned recently, policies at the top scientific journals don’t make this easy. Our story starts in 2008, when a group of researchers published an article (here it is without a paywall) that found political conservatives have stronger physiological reactions to threatening images than liberals do. The article was published in Science, which is one of the most prestigious general science journals around. It’s the kind of journal that can make a career in academia.
It was a path-breaking and provocative study. For decades, political scientists and psychologists have tried to understand the psychological roots of ideological differences. [...] In 2018, 10 years after the publication of the study, the findings were featured on an episode of NPR’s Hidden Brain podcast.
[...] Fast forward to 2014. All four of us were studying the physiological basis of political attitudes, two of us in Amsterdam, the Netherlands ... and two of us in Philadelphia ... We had raised funds to create labs with expensive equipment for measuring physiological reactions, because we were excited by the possibilities that the 2008 research opened for us. ... But both teams independently failed to find that people’s physiological reactions to these images correlated with their political attitudes. Our first thought was that we were doing something wrong. [...eventually...] we had become more skeptical of the rationale animating the original study. Neuroscientists can often find a loose match between physiological responses and self-reported attitudes. The question is whether this relationship is really as meaningful as we sometimes think it is.
[...] We drafted a paper that reported the failed replication studies along with a more nuanced discussion about the ways in which physiology might matter for politics and sent it to Science. ... we thought the editorial team would at least send it out for peer review. It did not. About a week later, we received a summary rejection with the explanation that the Science advisory board of academics and editorial team felt that since the publication of this article the field has moved on and that, while they concluded that we had offered a conclusive replication of the original study, it would be better suited for a less visible subfield journal.
We wrote back asking them to consider at least sending our work out for review. (They could still reject it if the reviewers found fatal flaws in our replications.) We argued that the original article continues to be highly influential and is often featured in popular science pieces in the lay media (for instance, here, here, here, and here), where the research is translated into a claim that physiology allows one to predict liberals and conservatives with a high degree of accuracy. We believe that Science has a responsibility to set the record straight ... We were rebuffed without a reason and with a vague suggestion that the journal’s policy on handling replications might change at some point in the future.
[...] Our takeaway is not that the original study’s researchers did anything wrong. To the contrary, members of the original author team ... were very supportive of the entire process [...] We believe that it is bad policy for journals like Science to publish big, bold ideas and then leave it to subfield journals to publish replications showing that those ideas aren’t so accurate after all. Subfield journals are less visible, meaning the message often fails to reach the broader public. They are also less authoritative, meaning the failed replication will have less of an impact on the field if it is not published by Science. (MORE - details)
EXCERPT: Science is supposed to be self-correcting. Ugly facts kill beautiful theories, to paraphrase the 19th-century biologist Thomas Huxley. But, as we learned recently, policies at the top scientific journals don’t make this easy. Our story starts in 2008, when a group of researchers published an article (here it is without a paywall) that found political conservatives have stronger physiological reactions to threatening images than liberals do. The article was published in Science, which is one of the most prestigious general science journals around. It’s the kind of journal that can make a career in academia.
It was a path-breaking and provocative study. For decades, political scientists and psychologists have tried to understand the psychological roots of ideological differences. [...] In 2018, 10 years after the publication of the study, the findings were featured on an episode of NPR’s Hidden Brain podcast.
[...] Fast forward to 2014. All four of us were studying the physiological basis of political attitudes, two of us in Amsterdam, the Netherlands ... and two of us in Philadelphia ... We had raised funds to create labs with expensive equipment for measuring physiological reactions, because we were excited by the possibilities that the 2008 research opened for us. ... But both teams independently failed to find that people’s physiological reactions to these images correlated with their political attitudes. Our first thought was that we were doing something wrong. [...eventually...] we had become more skeptical of the rationale animating the original study. Neuroscientists can often find a loose match between physiological responses and self-reported attitudes. The question is whether this relationship is really as meaningful as we sometimes think it is.
[...] We drafted a paper that reported the failed replication studies along with a more nuanced discussion about the ways in which physiology might matter for politics and sent it to Science. ... we thought the editorial team would at least send it out for peer review. It did not. About a week later, we received a summary rejection with the explanation that the Science advisory board of academics and editorial team felt that since the publication of this article the field has moved on and that, while they concluded that we had offered a conclusive replication of the original study, it would be better suited for a less visible subfield journal.
We wrote back asking them to consider at least sending our work out for review. (They could still reject it if the reviewers found fatal flaws in our replications.) We argued that the original article continues to be highly influential and is often featured in popular science pieces in the lay media (for instance, here, here, here, and here), where the research is translated into a claim that physiology allows one to predict liberals and conservatives with a high degree of accuracy. We believe that Science has a responsibility to set the record straight ... We were rebuffed without a reason and with a vague suggestion that the journal’s policy on handling replications might change at some point in the future.
[...] Our takeaway is not that the original study’s researchers did anything wrong. To the contrary, members of the original author team ... were very supportive of the entire process [...] We believe that it is bad policy for journals like Science to publish big, bold ideas and then leave it to subfield journals to publish replications showing that those ideas aren’t so accurate after all. Subfield journals are less visible, meaning the message often fails to reach the broader public. They are also less authoritative, meaning the failed replication will have less of an impact on the field if it is not published by Science. (MORE - details)