May 8, 2024 01:40 AM
https://nautil.us/what-counts-as-conscio...5038895331
EXCERPTS: In Christof Koch's new book, Then I Am Myself The World, Koch, currently the chief scientist at the Allen Institute for Brain Science in Seattle, ventures through the challenging landscape of integrated information theory (IIT), a framework that attempts to compute the amount of consciousness in a system based on the degree to which information is networked. Along the way, he struggles with what may be the most difficult question of all: How do our thoughts—seemingly ethereal and without mass or any other physical properties—have real-world consequences? We caught up with him recently over Zoom.
[...] What does integrated information theory say about consciousness?
IIT says, fundamentally, what exists is consciousness. And consciousness is the only thing that exists for itself. You are conscious. Tonight, you’re going to go into a deep sleep at some point, and then you’re not conscious anymore; then you do not exist for yourself. Your body and your brain still have an existence for others—I can see your body there—but you don’t exist for yourself. So only consciousness exists for itself; that’s absolute existence. Everything else is derivative.
It says consciousness ultimately is causal power upon itself—the ability to make a difference. And now you’re looking for a substrate—like a brain or computer CPU or anything. Then the theory says, whatever your conscious experience is—what it feels like to see red, or to smell Limburger cheese, or to have a particular type of toothache—maps one-to-one onto this structure, this form, this causal relationship. It’s not a process. It’s not a computation. It’s very different from all other theories.
[...] Toward the end of the book, you write, “I decide, not my neurons.” I can’t help thinking that that’s two ways of saying the same thing—on the macro level it’s “me,” but on the micro level, it’s my neurons. Or am I missing something?
Yeah, it’s a subtle difference. What truly exists for itself is your consciousness. When you’re unconscious, as in a deep sleep on anesthesia, you don’t exist for yourself anymore, and you’re unable to make any decisions. And so what truly exists is consciousness, and that’s where the true action happens.
I actually see you on the screen, there are lights in the image; inside my brain, I can assure you, there are no lights, it’s totally dark. My brain is just in a goo. So it’s not my brain that sees; it’s consciousness that sees. It’s not my brain that makes a decision, it’s my consciousness that makes a decision. They’re not the same.
For as long as we’ve had computers, people have argued about whether the brain is an information processor of some kind. You’ve argued that it isn’t. From that perspective, I’m guessing you don’t think large language models have causal powers.
Correct. In fact, I can pretty confidently make the following statement: There’s no Turing test for consciousness, according to IIT, because it’s not about a function; it’s all about this causal structure. So you actually have to look at the CPU or the chip—whatever does the computation. You have to look at that level: What’s the causal power?
[...] Now you can of course simulate perfectly well a human brain doing everything a human brain can do—there’s no problem conceptually, at least. And of course, a computer simulation will one day say, “I’m conscious,” like many large language models do, unless they have guardrails where they explicitly tell you “Oh no, I’m just an LLM—I’m not conscious,” because they don’t want to scare the public.
But that’s all simulation; that’s not actually being conscious. Just like you can simulate a rainstorm, but it never gets wet inside the computer, funnily enough, even though it simulated a rainstorm. You can solve Einstein’s equation of general relativity for a black hole, but you never have to be afraid that you’re going to be sucked into your computer simulation. Why not? If it really computes gravity, then shouldn’t spacetime bend around my computer and suck me, and the computer, in? No, because it’s a simulation. That’s the difference between the real and the simulated. The simulated doesn’t have the same causal powers as the real.
So unless you build a machine in the image of a human brain—let’s say using neuromorphic engineering, possibly using quantum computers—with that, you can get human-level consciousness. If you just build them like we build them right now, where one transistor talks to two or three other transistors—that’s radically different from the connectivity of the human brain—you’ll never get consciousness. So I can confidently say that although LLMs very soon will be able to do everything we can do, and probably faster and better than we can do, they will never be conscious.... (MORE - details)
EXCERPTS: In Christof Koch's new book, Then I Am Myself The World, Koch, currently the chief scientist at the Allen Institute for Brain Science in Seattle, ventures through the challenging landscape of integrated information theory (IIT), a framework that attempts to compute the amount of consciousness in a system based on the degree to which information is networked. Along the way, he struggles with what may be the most difficult question of all: How do our thoughts—seemingly ethereal and without mass or any other physical properties—have real-world consequences? We caught up with him recently over Zoom.
[...] What does integrated information theory say about consciousness?
IIT says, fundamentally, what exists is consciousness. And consciousness is the only thing that exists for itself. You are conscious. Tonight, you’re going to go into a deep sleep at some point, and then you’re not conscious anymore; then you do not exist for yourself. Your body and your brain still have an existence for others—I can see your body there—but you don’t exist for yourself. So only consciousness exists for itself; that’s absolute existence. Everything else is derivative.
It says consciousness ultimately is causal power upon itself—the ability to make a difference. And now you’re looking for a substrate—like a brain or computer CPU or anything. Then the theory says, whatever your conscious experience is—what it feels like to see red, or to smell Limburger cheese, or to have a particular type of toothache—maps one-to-one onto this structure, this form, this causal relationship. It’s not a process. It’s not a computation. It’s very different from all other theories.
[...] Toward the end of the book, you write, “I decide, not my neurons.” I can’t help thinking that that’s two ways of saying the same thing—on the macro level it’s “me,” but on the micro level, it’s my neurons. Or am I missing something?
Yeah, it’s a subtle difference. What truly exists for itself is your consciousness. When you’re unconscious, as in a deep sleep on anesthesia, you don’t exist for yourself anymore, and you’re unable to make any decisions. And so what truly exists is consciousness, and that’s where the true action happens.
I actually see you on the screen, there are lights in the image; inside my brain, I can assure you, there are no lights, it’s totally dark. My brain is just in a goo. So it’s not my brain that sees; it’s consciousness that sees. It’s not my brain that makes a decision, it’s my consciousness that makes a decision. They’re not the same.
For as long as we’ve had computers, people have argued about whether the brain is an information processor of some kind. You’ve argued that it isn’t. From that perspective, I’m guessing you don’t think large language models have causal powers.
Correct. In fact, I can pretty confidently make the following statement: There’s no Turing test for consciousness, according to IIT, because it’s not about a function; it’s all about this causal structure. So you actually have to look at the CPU or the chip—whatever does the computation. You have to look at that level: What’s the causal power?
[...] Now you can of course simulate perfectly well a human brain doing everything a human brain can do—there’s no problem conceptually, at least. And of course, a computer simulation will one day say, “I’m conscious,” like many large language models do, unless they have guardrails where they explicitly tell you “Oh no, I’m just an LLM—I’m not conscious,” because they don’t want to scare the public.
But that’s all simulation; that’s not actually being conscious. Just like you can simulate a rainstorm, but it never gets wet inside the computer, funnily enough, even though it simulated a rainstorm. You can solve Einstein’s equation of general relativity for a black hole, but you never have to be afraid that you’re going to be sucked into your computer simulation. Why not? If it really computes gravity, then shouldn’t spacetime bend around my computer and suck me, and the computer, in? No, because it’s a simulation. That’s the difference between the real and the simulated. The simulated doesn’t have the same causal powers as the real.
So unless you build a machine in the image of a human brain—let’s say using neuromorphic engineering, possibly using quantum computers—with that, you can get human-level consciousness. If you just build them like we build them right now, where one transistor talks to two or three other transistors—that’s radically different from the connectivity of the human brain—you’ll never get consciousness. So I can confidently say that although LLMs very soon will be able to do everything we can do, and probably faster and better than we can do, they will never be conscious.... (MORE - details)