Motivated Reasoning Is Disfiguring Social Science

C C Offline

EXCERPT: . . . Many people in the general public believe that organizations such as the APA (American Psychological Association) or American Academy of Pediatrics (AAP) are neutral, objective scientific organizations or that they are even part of the government. But they are largely professional guilds wherein members like me pay dues to support their profession (both, but particularly the APA, are also publication houses). As a result, such organizations tend to market their fields much like any business markets its products. Impressive sounding science brings the field prestige, captures the attention of policy makers and helps members with grants, newspaper headlines, and career advancement. This isn’t necessarily a bad thing, and I certainly don’t wish to imply any bad faith. But failing to understand this basic fact may cause the general public to overestimate the degree of objectivity with which such organizations speak when talking about research.

Spanking is one of those ideologically and emotionally loaded cultural crusades that is ripe for the misuse of inconvenient data. Most of the members of the APA task force that produced the resolution are long-time anti-spanking advocates, reviewing their own research and declaring it beyond further debate. This has been a consistent problem for the APA and other professional guilds such as the AAP (more about which in a moment), and they have repeatedly allowed scholars with fairly straightforward conflicts of interest to produce resolutions that might reasonably be expected to confirm their ideological biases. This process is not remotely objective.

Ideologically loaded statements have become more common in recent years. The APA has had a particularly bad run of it. Just this past January, the APA’s practice guidelines for men and boys became a polarizing lightning rod, beloved by some cultural progressives, but largely panned in the centre and on the Right as stereotyping and arguably sexist in their portrayal of men and “traditional masculinity.” The guidelines were long on progressive terminology but relied on weak evidence and ignored entire fields of research related to the biological roots of gender. [...]

And then there is the long-running controversy over the APA’s 2015 resolution on violent video games. [...] the task force appeared largely stacked with people who had taken prior anti-game views and portrayed a research field that simply doesn’t exist. Indeed, most scholars now agree that violent games play no role in societal violence. But the APA’s resolution has become a go-to source for politicians and the National Rifle Association whenever they want to distract the populace from talking about guns after a high-profile shooting.

Just this year a committee (which I chair) of the APA’s own Society for Media Psychology and Technology published a report documenting serious errors, biases, and misstatements in multiple resolution statements produced by the APA, AAP, and other professional groups when talking about media effects. In each case, the reports misrepresented the evidence for media effects, often simply ignoring any research that conflicted with the group’s stated position.

The bottom line is that professional guilds such as the APA and AAP have a demonstrable track record of unreliability when speaking on matters of science. This means that parents, the general public, and policy makers may base decisions on erroneous pseudo-scientific claims that can’t be backed by good data. Perhaps the most egregious issue is when such bodies simply pretend no controversy exists in fields that are, in fact, highly controversial. This behavior, known as “citation bias,” has been described by some scholars as one of the seven deadly sins of research scholarship. As professors, we would give a student a failing grade for such behavior in an academic paper. And yet professional guilds engage in such behavior on a fairly regular basis, at least with respect to behavioral research.

Why does this happen? There are a number of possible explanations.

The first is a lack of intellectual diversity. It’s been known for decades that the social sciences are heavily weighted with individuals who identify as sociocultural liberals or progressives. It probably isn’t just a happy coincidence that so much of social science is in lockstep with liberal and progressive social advocacy positions.

The second is the culture of institutions. From my experience and perspective, these tend to function on a corporate structure. Although they are typically non-profits, they increasingly behave like businesses rather than academic centers. As such, they do not appear to foster an appropriate level of critical thinking, skepticism, caution, or solicitation of opposing views (indeed on the spanking resolution, one skeptical scholar, Robert Larzelere, who volunteered to help was explicitly rebuffed). This is a recipe for conformity and groupthink.

Third, and related, the review processes these resolution statements undergo is obviously failing. Often, on council, advocates for a position will boast that a resolution passed through multiple boards and committees, all internal to the APA. And yet, I find myself thinking, the resolution still sucks. Indeed, the “this has been reviewed so many times before” line of argument is simply an appeal to consensus and authority, both of which expose the failure of such organizations’ review process, not any superior quality of the resultant product.

Worryingly, professional guilds are producing resolution statements with increasing frequency. Indeed, our review of media effects policy statements found that the AAP, in particular, is producing them like a moral panic pez dispenser. Do the 2010s really require so many more policy statements than did the 1990s? I doubt it. Increasingly, they resemble products produced by a business, and so they should be treated by the general public as if that is what they are: the advertising may not always tell the full story.

As Steven Pinker argues in The Blank Slate, producing nostrums of pseudo-science to support an advocacy position is highly risky and may be counterproductive to that advocacy position in the long run. If moral arguments for equality or against corporal discipline are constructed on a foundation of faulty science, then the foundations of that morality collapse as the faults are exposed. It is far safer to ensure that moral arguments remain independent of scientific claims—to argue, for instance, that gender equality is a moral imperative irrespective of whether or not gender differences are biological, or that hitting children is wrong irrespective of whether it leads to negative outcomes for the child.

Failing to be honest about a research field’s inconsistencies and shortcomings makes the whole project look like an instrument in a fear-mongering culture war....

MORE (details):
Syne Offline
It's this kind of stuff that both makes leftists less scientifically informed and people on the right less trusting of science. Postmodernism is destined to undermine its own academic tools to undermining the culture.
Yazata Offline
It's true.

Personally, I'm inclined to believe in the credibility of ostensible "science" is inverse to how "relevant" it is to highly politicized or highly moralized social issues. The more "relevant" a scientific subject is, the less reliable the results.

Especially in so-called "social sciences" like sociology, we see individuals starting out with their conclusions already in mind and then shaping their surveys (or whatever sociological research consists of) to produce those desired results. Then it's all written up in the form of a so-called "study" (whatever that word means) and the fake-news media promote it because it conforms with their chosen "narrative".

The rot has most definitely hit psychology, at least when the topic has anything to do with "gender". (I'm inclined to suspect that the explosion in 'autism' diagnoses is a bit of a fad too.)

Anyone who doesn't immediately fall on their knees believing whatever they are told, and instead expresses a little skepticism, is denounced as being "anti-science". (Anti-bullshit is more like it.)

Possibly Related Threads…
Thread Author Replies Views Last Post
  On menthol cigarettes, social justice theory shouldn’t trump science + Climate sci C C 0 13 Jul 27, 2021 07:56 PM
Last Post: C C
  Do political beliefs affect our ability to crunch numbers? (motivated numeracy) C C 1 67 Apr 22, 2021 03:35 AM
Last Post: Syne
  Is social science akin to a cargo cult? C C 0 45 Feb 8, 2021 08:40 PM
Last Post: C C
  Women led nations haven't fared much better in pandemic + Social science lacking bite C C 0 76 Jan 1, 2021 09:32 PM
Last Post: C C
  Social scientists revisit anthropology’s science wars C C 0 206 Oct 15, 2019 07:55 PM
Last Post: C C
  The Devolution of Social Science (unreliable, ideological pseudoscience, etc) C C 1 536 Oct 10, 2018 05:12 PM
Last Post: Syne
  Why It Took Social Science Years to Correct a Simple Error About ‘Psychoticism’ C C 3 613 Apr 14, 2017 11:35 PM
Last Post: Syne
  The Tyranny of Evidence: Do the social sciences border on being junk science? C C 0 853 Nov 18, 2015 08:41 PM
Last Post: C C
  Let’s Abolish Social Science C C 1 762 Sep 4, 2015 08:10 PM
Last Post: Yazata

Users browsing this thread: 1 Guest(s)