In psychology & other social sciences, many studies [still] fail reproducibility test


EXCERPT: The world of social science got a rude awakening a few years ago, when researchers concluded that many studies in this area appeared to be deeply flawed. Two-thirds could not be replicated in other labs. Some of those same researchers now report those problems still frequently crop up, even in the most prestigious scientific journals. But their study, published Monday in *Nature Human Behaviour* also finds that social scientists can actually sniff out the dubious results with remarkable skill.

[...] The results were better than the average of a previous review of the psychology literature, but still far from perfect. Of the 21 studies, the experimenters were able to reproduce 13. And the effects they saw were on average only about half as strong as had been trumpeted in the original studies.

The remaining eight were not reproduced. "A substantial portion of the literature is reproducible," [Brian] Nosek concludes. "We are getting evidence that someone can independently replicate [these findings]. And there is a surprising number [of studies] that fail to replicate."

[...] As part of the reproducibility study, about 200 social scientists were surveyed and asked to predict which results would stand up to the re-test and which would not. "They're taking bets with each other, against us," says Anna Dreber, an economics professor [...] and coauthor of the new study. It turns out, "these researchers were very good at predicting which studies would replicate," she says. "I think that's great news for science."

[...] But if social scientists were really good at identifying flawed studies, why did the editors and peer reviewers at Science and Nature let these eight questionable studies through their review process? "The likelihood that a finding will replicate or not is one part of what a reviewer would consider," says Nosek. "But other things might influence the decision to publish. It may be that this finding isn't likely to be true, but if it is true, it is super important, so we do want to publish it because we want to get it into the conversation."


Possibly Related Threads...
Thread Author Replies Views Last Post
  Experimental philosophy is more reliable & robust than psychology C C 0 163 Jul 14, 2018 08:28 AM
Last Post: C C
  Is evolutionary psychology a deeply flawed enterprise? C C 13 858 Mar 16, 2018 06:40 AM
Last Post: Syne
  Were These Well-Known Gender Studies Just Made Up? C C 0 190 Dec 3, 2017 05:55 AM
Last Post: C C
  How Reliable Are Cancer Studies? C C 1 275 Jan 21, 2017 04:24 PM
Last Post: Zinjanthropos
  Psychology's anti-conservative bias is damaging the field C C 0 193 Sep 16, 2016 07:13 PM
Last Post: C C
  Schizophrenia is non-existant + Politics affects sci issues + Reproducibility crisis C C 0 332 Mar 11, 2016 07:44 PM
Last Post: C C
  The Tyranny of Evidence: Do the social sciences border on being junk science? C C 0 534 Nov 18, 2015 08:41 PM
Last Post: C C

Users browsing this thread: 1 Guest(s)