Sep 21, 2026 02:47 PM
https://3quarksdaily.com/3quarksdaily/20...lding.html
EXCERPTS: There’s something ironic in this arrangement if you think about it. Science fiction imagined that advancements in computing and AI will one day free humans from the drudgery of physical work. If these trends continue, then we may end up with an opposite arrangement.
Imagine a world where there’s a machine, an AI model that searches the literature and generates a hypothesis. Based on the hypothesis, it recommends certain experiments be done. It would then gather the data from the experiments, interpret it, and suggest conclusions. The human scientists are then left to just do the experiments.
As strange as that sounds, there are even more parts of this process which could be automated. The next step would be automated laboratories, many of which already exist, but in this imagined near future, they will be tightly coupled with the hypothesis generation and experimentation process. This is not to say that scientists will disappear. However, how science is done will greatly change.
[...] Modern science works because understanding is distributed. One person understands the assay, another understands the statistical method, and another knows the instrumentation. Scientific institutions allow us to rely on this division of labor. Peer review, replication, professional specialization, and scientific reputation all help create trust between people who cannot independently verify everything they use.
The potential risk that we may be seeing here is that there may be parts of this process which cease to be understandable to humans at all. Thus, a machine could arrive at a reliable result through an inferential path involving an enormous number of interacting variables, simulations, model calls, searches, and intermediate representations. We might be able to test the output without being able to reconstruct the route that produced it.
It may be tempting. to treat this as a philosophical luxury. If a drug works, does it really matter whether anybody understands the route by which it was discovered? If a model predicts a hurricane accurately, the people in its path would presumably prefer a correct warning to an elegant explanation. Engineering has always tolerated a certain amount of pragmatism, and scientists themselves frequently use tools whose inner workings they do not fully understand.
[...] A sufficiently powerful machine could be extraordinarily productive while pursuing questions that human beings find utterly uninteresting. Scientific importance is not a property that can always be read directly from the natural world. It reflects human priorities, needs, aesthetics, and values. Even a system vastly better than us at solving scientific problems would still require some account of which problems ought to matter... (MORE - missing details)
EXCERPTS: There’s something ironic in this arrangement if you think about it. Science fiction imagined that advancements in computing and AI will one day free humans from the drudgery of physical work. If these trends continue, then we may end up with an opposite arrangement.
Imagine a world where there’s a machine, an AI model that searches the literature and generates a hypothesis. Based on the hypothesis, it recommends certain experiments be done. It would then gather the data from the experiments, interpret it, and suggest conclusions. The human scientists are then left to just do the experiments.
As strange as that sounds, there are even more parts of this process which could be automated. The next step would be automated laboratories, many of which already exist, but in this imagined near future, they will be tightly coupled with the hypothesis generation and experimentation process. This is not to say that scientists will disappear. However, how science is done will greatly change.
[...] Modern science works because understanding is distributed. One person understands the assay, another understands the statistical method, and another knows the instrumentation. Scientific institutions allow us to rely on this division of labor. Peer review, replication, professional specialization, and scientific reputation all help create trust between people who cannot independently verify everything they use.
The potential risk that we may be seeing here is that there may be parts of this process which cease to be understandable to humans at all. Thus, a machine could arrive at a reliable result through an inferential path involving an enormous number of interacting variables, simulations, model calls, searches, and intermediate representations. We might be able to test the output without being able to reconstruct the route that produced it.
It may be tempting. to treat this as a philosophical luxury. If a drug works, does it really matter whether anybody understands the route by which it was discovered? If a model predicts a hurricane accurately, the people in its path would presumably prefer a correct warning to an elegant explanation. Engineering has always tolerated a certain amount of pragmatism, and scientists themselves frequently use tools whose inner workings they do not fully understand.
[...] A sufficiently powerful machine could be extraordinarily productive while pursuing questions that human beings find utterly uninteresting. Scientific importance is not a property that can always be read directly from the natural world. It reflects human priorities, needs, aesthetics, and values. Even a system vastly better than us at solving scientific problems would still require some account of which problems ought to matter... (MORE - missing details)
