6 hours ago
https://thereader.mitpress.mit.edu/why-a...ever-meet/
EXCERPTS: A philosophical position called functionalism says that consciousness is about what something does, not what it’s made of. So, if AI gets to the point where it can hold a conversation, reason, remember, change its mind, exhibit curiosity and emotions, and so forth, then it doesn’t matter if it’s made of flesh, silicon, or LEGO bricks. In fact, some have suggested that denying consciousness to such a creature would be a new type of small-minded bias.
I am willing to go out on a limb and say that, despite what leaders at some AI companies have suggested, I know the LLMs I talk with aren’t conscious … not for empirical reasons, but because I don’t think the concept makes sense.
My line of thought begins with a famous thought experiment called the Chinese Room, posed by the philosopher John Searle (1932–2025), who, in 1980, asked us to imagine finding ourselves in a room with a slot and no other openings.
[..] Searle maintains that this is exactly what computers do, including ones that seem conscious: They’re manipulating symbols without any awareness of what the symbols mean, or even that they’re symbols at all. The Chinese Room shows us that symbol manipulation is not enough to constitute consciousness because the meaning of a symbol comes from a consciousness separate and apart from it. In Searle’s terms, that makes the meaning of a symbol extrinsic to it: it has to be applied by a consciousness apart from the symbol.
[...] Consciousness like ours, on the other hand, is different. When you burn your hand on a hot pot handle, take a bite of a banana, or put your left foot in front of your right as you stroll, you don’t need to translate those into pain, the taste of bananas, or walking. Your brain is part of a living body that directly feels pain, tastes food, and knows how to walk without looking down. No symbols required, no translations needed. In Searle’s terms, that means consciousness is intrinsic: it doesn’t need another consciousness to assign meaning to it.
But where does the intrinsic nature of consciousness come from? Searle doesn’t tell us.
A half-century before Searle, Martin Heidegger (1889–1976) already had. In his 1927 masterpiece, “Being and Time,” he looks at the world not as we explain it, but as we experience it in ordinary, everyday circumstances. He found that, with surprising regularity, our understanding is out of sync with our experience.
Instead, he noted, in everyday lived experience, things almost always show themselves to us in light of our current project or intentions [...] But why do things matter at all? Heidegger says it’s because we know we’re at risk. But that still wouldn’t explain anything unless we care about being at risk. And that puts us in a loop: We care because we know our projects and our own lives are always at risk. But care is why that risk matters to us. Around and around we go... (MORE - details)
- - - - - - - - - - - - -
The actual problem in symbol grounding (Chinese Room) is not lack of meaning, but symbols circularly referring to other symbols for that. There's no access to actual perceptions, or jumping beyond the relationships between tokens. But machines don't experience the symbols on a screen, either. It's the processes or computational routines that the machine is performing that underlie the symbols that humans see, which the machine just does without manifestation as anything.
EXCERPTS: A philosophical position called functionalism says that consciousness is about what something does, not what it’s made of. So, if AI gets to the point where it can hold a conversation, reason, remember, change its mind, exhibit curiosity and emotions, and so forth, then it doesn’t matter if it’s made of flesh, silicon, or LEGO bricks. In fact, some have suggested that denying consciousness to such a creature would be a new type of small-minded bias.
I am willing to go out on a limb and say that, despite what leaders at some AI companies have suggested, I know the LLMs I talk with aren’t conscious … not for empirical reasons, but because I don’t think the concept makes sense.
My line of thought begins with a famous thought experiment called the Chinese Room, posed by the philosopher John Searle (1932–2025), who, in 1980, asked us to imagine finding ourselves in a room with a slot and no other openings.
[..] Searle maintains that this is exactly what computers do, including ones that seem conscious: They’re manipulating symbols without any awareness of what the symbols mean, or even that they’re symbols at all. The Chinese Room shows us that symbol manipulation is not enough to constitute consciousness because the meaning of a symbol comes from a consciousness separate and apart from it. In Searle’s terms, that makes the meaning of a symbol extrinsic to it: it has to be applied by a consciousness apart from the symbol.
[...] Consciousness like ours, on the other hand, is different. When you burn your hand on a hot pot handle, take a bite of a banana, or put your left foot in front of your right as you stroll, you don’t need to translate those into pain, the taste of bananas, or walking. Your brain is part of a living body that directly feels pain, tastes food, and knows how to walk without looking down. No symbols required, no translations needed. In Searle’s terms, that means consciousness is intrinsic: it doesn’t need another consciousness to assign meaning to it.
But where does the intrinsic nature of consciousness come from? Searle doesn’t tell us.
A half-century before Searle, Martin Heidegger (1889–1976) already had. In his 1927 masterpiece, “Being and Time,” he looks at the world not as we explain it, but as we experience it in ordinary, everyday circumstances. He found that, with surprising regularity, our understanding is out of sync with our experience.
Instead, he noted, in everyday lived experience, things almost always show themselves to us in light of our current project or intentions [...] But why do things matter at all? Heidegger says it’s because we know we’re at risk. But that still wouldn’t explain anything unless we care about being at risk. And that puts us in a loop: We care because we know our projects and our own lives are always at risk. But care is why that risk matters to us. Around and around we go... (MORE - details)
- - - - - - - - - - - - -
The actual problem in symbol grounding (Chinese Room) is not lack of meaning, but symbols circularly referring to other symbols for that. There's no access to actual perceptions, or jumping beyond the relationships between tokens. But machines don't experience the symbols on a screen, either. It's the processes or computational routines that the machine is performing that underlie the symbols that humans see, which the machine just does without manifestation as anything.