Oct 11, 2015 03:37 PM
http://www.nature.com/news/let-s-think-a...as-1.18520
EXCERPT: [...] The sources and types of such cognitive bias — and the fallacies they produce — are becoming more widely appreciated. Some of the problems are as old as science itself, and some are new [...] Advocates of robust science have repeatedly warned against cognitive habits that can lead to error. Although such awareness is essential, it is insufficient. The scientific community needs concrete guidance on how to manage its all-too-human biases and avoid the errors they cause.
That need is particularly acute in statistical data analysis, where some of the best-established methods were developed in a time before data sets were measured in terabytes, and where choices between techniques offer abundant opportunity for errors. Proteomics and genomics, for example, crunch millions of data points at once, over thousands of gene or protein variants. Early work was plagued by false positives, before the spread of techniques that could account for the myriad hypotheses that such a data-rich environment could generate.
Although problems persist, these fields serve as examples of communities learning to recognize and curb their mistakes. [...] The scientific community must design research protocols that safeguard against these errors, and devise methods that ferret out sloppy analyses. [...] Finally, the scientific community must go beyond statistical safeguards, and improve researchers’ behaviour....
EXCERPT: [...] The sources and types of such cognitive bias — and the fallacies they produce — are becoming more widely appreciated. Some of the problems are as old as science itself, and some are new [...] Advocates of robust science have repeatedly warned against cognitive habits that can lead to error. Although such awareness is essential, it is insufficient. The scientific community needs concrete guidance on how to manage its all-too-human biases and avoid the errors they cause.
That need is particularly acute in statistical data analysis, where some of the best-established methods were developed in a time before data sets were measured in terabytes, and where choices between techniques offer abundant opportunity for errors. Proteomics and genomics, for example, crunch millions of data points at once, over thousands of gene or protein variants. Early work was plagued by false positives, before the spread of techniques that could account for the myriad hypotheses that such a data-rich environment could generate.
Although problems persist, these fields serve as examples of communities learning to recognize and curb their mistakes. [...] The scientific community must design research protocols that safeguard against these errors, and devise methods that ferret out sloppy analyses. [...] Finally, the scientific community must go beyond statistical safeguards, and improve researchers’ behaviour....