Feb 16, 2024 07:05 PM
(This post was last modified: Feb 16, 2024 07:08 PM by C C.)
https://www.skeptic.com/reading_room/why...ompelling/
EXCERPTS (Anthony Fowler): The public is frequently told to “trust the science,” and then ridiculed for holding any views that differ from what is reported to be the scientific consensus. Should non-experts then naively accept the authorized narrative, or are there good reasons to be skeptical?
[...] Herding can be a rational behavior. It would not be a good use of time or money for every dentist to conduct an independent study to assess the evidence and determine whether sugarless gum is good for dental health. However, herding can lead an entire scientific community to converge on the wrong answer, and they typically won’t know whether they’ve converged on the right or the wrong answer.
We can see how a dangerous emperor-has-no-clothes situation could easily arise. [...] So they decide to keep quiet, the field moves on, nobody’s beliefs are challenged, and no new evidence is collected.
[...] herding is a problem in many scientific fields, including those studying arguably more important questions, such as the health of democracy.
Consider this example from an academic subfield I happen to know well. Among scholars of political behavior, there is a broad consensus that American voters don’t know or care much about policy, and their voting decisions are largely driven by party identity. [...] To a member of the general public who has never taken a political science class, this claim might seem absurd. ... How can such a strong claim unsupported by good evidence be the scientific consensus?
When I challenged this scientific consensus, I received significant public and private criticism from scholars of political behavior. A few of my critics engaged with my arguments and evidence, but most did not. Instead, they typically made appeals to authority, such as, “How dare you challenge what’s been established wisdom for seven decades?”.
In other words, they were herding. They assumed that something must be right because that’s been the consensus view in their field for a long time. They were not able or willing to provide further evidence or arguments in support of their position, and they simply dismissed anyone who challenged them, thereby creating a strong incentive for other scholars to uphold the consensus.
[...] Roughly speaking, there are two different ways in which an apparent scientific consensus might arise. In the good scenario, scientists are conducting genuinely good work, rigorously vetting each other’s work, and the theory, the evidence, and the analyses supporting the consensus view are all really strong...
[...] In the bad scenario, the scientists are not always conducting good work, don’t rigorously vet each other’s work (or they engage in selective vetting based on whether or not they like and/or agree with the conclusions of a study), and the theory, the evidence, or the analyses supporting the consensus are not robust. ... Dissenters, whether scientists themselves or not, are ostracized.
Unfortunately, the bad scenario occurs too often— much more often than many scientists, commentators, and cultural leaders presume. We already saw one way in which the bad scenario can arise—herding. Here are some additional ways in which the bad scenario can arise and why skeptics should view appeals to scientific consensus, on their own, as uncompelling. I also discuss how non-experts can better distinguish between the good and bad scenarios, and how scientists can do more to avoid the latter.
[...] The Illusion of Scientific Consensus. Commentators and leaders often assert that their position is the consensus view, but without providing direct evidence of that consensus...
[...] Broad Consensus Doesn’t Mean High Certainty. ... the percentage of scientists who agree with a statement is not a very informative statistic...
[...] Correlated Errors. A scientific estimate can diverge from the truth for many reasons, but the hope of the scientific community is that if we conduct a lot of studies, the errors will cancel out, and when we conduct meta-analyses, our estimates will converge to the truth. The problem with this logic is that not all errors cancel each other out...
[...] Selective Reporting. Scientific results that happen to align with the predispositions of journal editors and peer-reviewers are more likely to be written up and published than those that go against the accepted wisdom in a field....
[...] Fraud. Another obvious but important reason the scientific record might fail to reflect the truth is that some scientists engage in fraud...
[...] Career Incentives. Partly because of the phenomena discussed above, we can’t know if a scientist who publicly supports a conclusion genuinely holds that view. To publish papers, secure grants, get a good job, get tenure, receive praise, and avoid banishment, scientists must not question the key tenets of their field...
[...] Science vs. Values. Not all important questions are scientific questions. ... Unfortunately, the difference between science questions and value judgments is often forgotten or ignored by scientists themselves...
[...] Politicization. This article presents a number of different explanations for the potential emergence of an unreliable scientific consensus. All of these concerns are exacerbated when a scientific question becomes politicized or is of great public interest...
[...] All that said, avoid nihilism or worse. Consumers of scientific information should be skeptical of an apparent scientific consensus, and they should think about some of the factors discussed here when deciding how skeptical they should be. How politicized is this topic? What are the career incentives for the scientists? How easy would it be for scientists to selectively report only the favorable results? Would a study have been published if it had found the opposite result or a null result? The answers to these questions will not definitively tell us whether the scientific consensus is right or wrong, but they should help us decide the degree to which we should simply trust the consensus or investigate further.
Although skepticism is warranted, nihilism is not. Even when a topic is highly politicized and when there are good reasons to worry about biased studies, selective reporting, herding, and so on, the scientific community can still find the right answer...
[...] Science is a process, not a result. If we want to learn more about the universe ... science is our best hope. So don’t become a nihilist, and don’t replace science with something worse, such as random guessing or deference to authority, religious or political.
Remember that science is just the word we use to describe the process [...] It involves repeated iterations of hypothesizing, experimenting, analyzing, empirical testing, and arguing.
If a group of so-called scientists stop theorizing, testing, and challenging, then they’re no longer engaged in science. Perhaps they’re engaged in advocacy, which is a respectable thing to do, particularly if the theory, evidence, and arguments on their side are strong. Yet advocacy and science are distinctly different activities and shouldn’t be conflated.... (MORE - missing details)
EXCERPTS (Anthony Fowler): The public is frequently told to “trust the science,” and then ridiculed for holding any views that differ from what is reported to be the scientific consensus. Should non-experts then naively accept the authorized narrative, or are there good reasons to be skeptical?
[...] Herding can be a rational behavior. It would not be a good use of time or money for every dentist to conduct an independent study to assess the evidence and determine whether sugarless gum is good for dental health. However, herding can lead an entire scientific community to converge on the wrong answer, and they typically won’t know whether they’ve converged on the right or the wrong answer.
We can see how a dangerous emperor-has-no-clothes situation could easily arise. [...] So they decide to keep quiet, the field moves on, nobody’s beliefs are challenged, and no new evidence is collected.
[...] herding is a problem in many scientific fields, including those studying arguably more important questions, such as the health of democracy.
Consider this example from an academic subfield I happen to know well. Among scholars of political behavior, there is a broad consensus that American voters don’t know or care much about policy, and their voting decisions are largely driven by party identity. [...] To a member of the general public who has never taken a political science class, this claim might seem absurd. ... How can such a strong claim unsupported by good evidence be the scientific consensus?
When I challenged this scientific consensus, I received significant public and private criticism from scholars of political behavior. A few of my critics engaged with my arguments and evidence, but most did not. Instead, they typically made appeals to authority, such as, “How dare you challenge what’s been established wisdom for seven decades?”.
In other words, they were herding. They assumed that something must be right because that’s been the consensus view in their field for a long time. They were not able or willing to provide further evidence or arguments in support of their position, and they simply dismissed anyone who challenged them, thereby creating a strong incentive for other scholars to uphold the consensus.
[...] Roughly speaking, there are two different ways in which an apparent scientific consensus might arise. In the good scenario, scientists are conducting genuinely good work, rigorously vetting each other’s work, and the theory, the evidence, and the analyses supporting the consensus view are all really strong...
[...] In the bad scenario, the scientists are not always conducting good work, don’t rigorously vet each other’s work (or they engage in selective vetting based on whether or not they like and/or agree with the conclusions of a study), and the theory, the evidence, or the analyses supporting the consensus are not robust. ... Dissenters, whether scientists themselves or not, are ostracized.
Unfortunately, the bad scenario occurs too often— much more often than many scientists, commentators, and cultural leaders presume. We already saw one way in which the bad scenario can arise—herding. Here are some additional ways in which the bad scenario can arise and why skeptics should view appeals to scientific consensus, on their own, as uncompelling. I also discuss how non-experts can better distinguish between the good and bad scenarios, and how scientists can do more to avoid the latter.
[...] The Illusion of Scientific Consensus. Commentators and leaders often assert that their position is the consensus view, but without providing direct evidence of that consensus...
[...] Broad Consensus Doesn’t Mean High Certainty. ... the percentage of scientists who agree with a statement is not a very informative statistic...
[...] Correlated Errors. A scientific estimate can diverge from the truth for many reasons, but the hope of the scientific community is that if we conduct a lot of studies, the errors will cancel out, and when we conduct meta-analyses, our estimates will converge to the truth. The problem with this logic is that not all errors cancel each other out...
[...] Selective Reporting. Scientific results that happen to align with the predispositions of journal editors and peer-reviewers are more likely to be written up and published than those that go against the accepted wisdom in a field....
[...] Fraud. Another obvious but important reason the scientific record might fail to reflect the truth is that some scientists engage in fraud...
[...] Career Incentives. Partly because of the phenomena discussed above, we can’t know if a scientist who publicly supports a conclusion genuinely holds that view. To publish papers, secure grants, get a good job, get tenure, receive praise, and avoid banishment, scientists must not question the key tenets of their field...
[...] Science vs. Values. Not all important questions are scientific questions. ... Unfortunately, the difference between science questions and value judgments is often forgotten or ignored by scientists themselves...
[...] Politicization. This article presents a number of different explanations for the potential emergence of an unreliable scientific consensus. All of these concerns are exacerbated when a scientific question becomes politicized or is of great public interest...
[...] All that said, avoid nihilism or worse. Consumers of scientific information should be skeptical of an apparent scientific consensus, and they should think about some of the factors discussed here when deciding how skeptical they should be. How politicized is this topic? What are the career incentives for the scientists? How easy would it be for scientists to selectively report only the favorable results? Would a study have been published if it had found the opposite result or a null result? The answers to these questions will not definitively tell us whether the scientific consensus is right or wrong, but they should help us decide the degree to which we should simply trust the consensus or investigate further.
Although skepticism is warranted, nihilism is not. Even when a topic is highly politicized and when there are good reasons to worry about biased studies, selective reporting, herding, and so on, the scientific community can still find the right answer...
[...] Science is a process, not a result. If we want to learn more about the universe ... science is our best hope. So don’t become a nihilist, and don’t replace science with something worse, such as random guessing or deference to authority, religious or political.
Remember that science is just the word we use to describe the process [...] It involves repeated iterations of hypothesizing, experimenting, analyzing, empirical testing, and arguing.
If a group of so-called scientists stop theorizing, testing, and challenging, then they’re no longer engaged in science. Perhaps they’re engaged in advocacy, which is a respectable thing to do, particularly if the theory, evidence, and arguments on their side are strong. Yet advocacy and science are distinctly different activities and shouldn’t be conflated.... (MORE - missing details)