Research ‘Cognitive surrender’ is a new and useful term for how AI melts brains

[C C]

https://gizmodo.com/cognitive-surrender-...2000742595

EXCERPTS: “cognitive surrender” [...] was, it appears, coined in this context by the Wharton Business School marketing researchers Steven Shaw and Gideon Nave. Their paper is incredibly troubling, and once you read about these findings, the term “cognitive surrender” will be stuck in your head too.

[...] At any rate, in the part of the study where the subjects were allowed to consult the chatbot, they did so about half the time. When it gave correct answers, they accepted them 93 percent of the time. Unfortunately, when it was wrong, they accepted answers 80 percent of the time. And keep in mind, they didn’t have to use it at all. They let the bad advice trump their own brains. Even worse, those who used AI rated their confidence 11.7 percent higher than those who didn’t, even though it was wrong.

The authors write that in addition to Kahneman’s fast and slow “systems” of cognition, this new artificial crutch is creating what they call “System 3.”

The authors write:

Our findings demonstrate that people readily incorporate AI-generated outputs into their decision-making processes, often with minimal friction or skepticism. This seamless engagement with System 3 underscores its potential to enhance everyday cognition by reducing cognitive effort, accelerating decisions, and supplementing or substituting internal cognition with externally processed, vastly resourced, AI-powered insights.

Cognitive surrender isn’t necessarily all bad in their view. It “illustrates the value and integration of System 3, but also highlights the vulnerability of System 3 usage.”

This isn’t the first time the phrase cognitive surrender has existed. The theologian Peter Berger used it in a religious context in the 1990s, but it meant something more like surrendering faith in God to relieve cognitive dissonance. And if you’re like me, you’ve probably noticed that AI-assisted cognitive surrender looks like older forms of mental laziness... (MORE - details)

[Magical Realist]

Makes sense to me. When you freely outsource your ability to think and acquire factual information to an unconscious machine, you might as well hang it up. Ofcourse many do this already with social media, jumping on any well-stated position and adopting it as their own without thinking it thru. So the transition for these mindless bottom feeders to AI will be quite seamless. It's all part of what Jacques Ellul called "the betrayal of technology."

“The stage in which the human being was a mere slave of the mechanical tyrant has been passed. When man himself becomes a machine, he attains to the marvelous freedom of unconsciousness, the freedom of the machine itself.”― Jacques Ellul, The Technological Society

https://www.facebook.com/reel/2424504931332624

[Syne]

At least for AI, those on the right are well insulated against "cognitive surrender," as LLMs are often trained on vastly more left-wing content. Social media surrender on the right is isolated to the horseshoe-right, as they meet up with the far left.

[Magical Realist]

It is entirely conceivable to me that with the advance of AI systems and the growing dependency of ourselves upon them, we may reach a stage where those systems will become more and more human and “conscious” while we become more and more machine-like and "unconscious". We will thereby meet at some half-way point, each new and opposite state dialectically defining and enabling the other. At some point, much to our own belated horror, we will have completely replaced each other!

[C C]

It is entirely conceivable to me that with the advance of AI systems and the growing dependency of ourselves upon them, we may reach a stage where those systems will become more and more human and “conscious” while we become more and more machine-like and "unconscious". We will thereby meet at some half-way point, each new and opposite state dialectically defining and enabling the other. At some point, much to our own belated horror, we will have completely replaced each other!

The dependency that younger generations have on AI to do their thinking will eventually be so great that when cyborg tech is finally ready for transplantation into the masses, they will be as gung-ho for it as bling and tattoos.

While there will be the convenience of directly talking to AI on a smartphone, that outward speech reveals just how mentally lazy one is. And the privacy of internal communication should be speedier and less clumsy than having to mess with an external device. It also implements technological telepathy, where one can make and receive telephone conversations in silence.

The eventual hive-mind could be potentially ground-breaking, but at the individual level of the collective humans will have devolved into idiots and essentially become zombie Borg.

[Magical Realist]

CC said: The eventual hive-mind could be potentially ground-breaking, but at the individual level of the collective humans will have devolved into idiots and essentially become zombie Borg.

Ironically perhaps the only glimmer of salvation from this dystopian fate seems to lie in man's welcoming embrace of his dawning machine-like nature. The mythic narrative of the noble and free human will be shed like old snake skin, unleashing a new being and consciousness of oneself as a sort of gateway of new possibilities and capabilities. It's a Faustian deal we will be happy to strike, surrendering what is left of a human soul for a purely informational and networked sort of hive mind, stripped of the troublesome burdens of emotion and feeling and moral values. Language will likely change as well, ridding itself of the old categorical descriptors of "machine-like" or "cyborg" or "freewill" or "self" or "human personhood". Subjectivity itself now a distributive and interchangeable and disembodied state of being, a new being increasingly defined more in terms of flexibility and fluidity instead of some old and defunct model of invariant selfness and discrete identity. We may see this as the bleak realization of a Borg society, but to them it would be a utopian age of collective oneness and endless new experiences.

“Man is a machine, but a very peculiar machine. He is a machine which, in right circumstances, and with right treatment, can know that he is a machine, and having fully realized this, he may find the ways to cease to be a machine. First of all, what man must know is that he is not one; he is many. He has not one permanent and unchangeable “I” or Ego. He is always different. One moment he is one, another moment he is another, the third moment he is a third, and so on, almost without end.”― P.D. Ouspensky