May 10, 2020 07:40 PM
(This post was last modified: May 10, 2020 07:43 PM by C C.)
https://backreaction.blogspot.com/2020/0...rated.html
EXCERPT (Sabine Hossenfelder): The world, it seems, is full with people who mistakenly think that a theory which makes correct predictions is a good theory. [...] I blame this confusion on the many philosophers, notably Popper and Lakatos, who have gone on about the importance of predictions, but never clearly said that it’s not a scientific criterion.
You see, the philosophers wanted a quick way to figure out whether a scientific theory is good or not that would not require them to actually understand the science. This, needless to say, is not possible. But the next best thing you can do is to ask how much you can trust the scientists. It is for this latter purpose, to evaluate the trust you can put in scientists, that predictions are good. But they cannot, and should not, ultimately decide what the scientific value of a theory is.
[...] See, suppose I plan to convince someone that I can correctly predict the stock market. What do I do? Well, I pick, say, 3 stocks and make “predictions” for a week ahead, but that are really just guesses which cover all reasonably possible trends. I then select a large group of victims. To each of them I send one of my guesses. Some of them will coincidentally get the correct guess. A week later, I know which people got the correct guess. To this group, I then send another set of guesses for the week ahead. Again, some people will get the correct guess by coincidence, and a week later I will know which one it was. I do this a third time, and then I have a group of people who have good “evidence” that I can tell the future.
Amazing, no? What’s the problem here? The problem is that correct predictions don’t tell you whether someone’s theory is good science.
As we have just seen, one of the problems with relying on predictions is that they may be correct just by coincidence. The larger the pool of predictions – or the pool of scientists making predictions! – the more likely this is to happen. The other problem is that relying on predictions makes fundamentally no sense. If I have a scientific theory, it is either a good description of nature, or it is not. At which time someone made a calculation for an observable quantity is entirely irrelevant for a theory’s relation to nature.
This is a point which is often raised by string theorists, and they are correct to raise it. String theorists say that since string theory gives rise to general relativity, it deserves as much praise as general relativity. That’s because, if string theory had been discovered before general relativity, it would have made the same predictions: light deflection on the sun, precession of Mercury, black holes, gravitational waves, and so on.
And indeed, this would be a good argument in favor of string theory – if it was correct. But it isn’t. String theory does not give rise to general relativity. It gives rise to general relativity in 10 dimensions, with supersymmetric matter, a negative cosmological constant, and dozens of additional scalar fields. All this extra clutter conflicts with observations. To fix this conflict with observations, string theorists then have to make several additional assumptions. With that you get a theory that is considerably more complicated than general relativity, but that does not explain the data any better. Hence, Occam’s razor tells you that general relativity is preferable.
Of course, it’s this adding of ad hoc assumptions to fix a mismatch with observation that the philosophers were trying to prevent when they requested testable predictions. But it’s the ad hoc assumptions themselves that are the problem, not the time at which they were made. To decide whether a scientific theory is any good what matters is only its explanatory power. Explanatory power measures how much data you can fit from which number of assumptions. The fewer assumption you make and the more data you fit, the higher the explanatory power, and the better the theory.
Ok, I admit, it’s somewhat more complicated than that. That’s because it also matters how well you fit the data. If you make more assumptions, you will generally be able to fit the data better. So there is a trade-off to be made, which needs to be quantified: At which point is the benefit you get from more assumptions not worth a somewhat better fit to the data? There are statistical tools to decide that. One can argue which one of those is the best for a given purpose, but that’s a fight that experts can fight in the case at hand. What is relevant here is only that the explanatory power of a theory is quantifiable. And it’s the explanatory power that decides whether a theory is good or not.
That’s obvious, I know. But why then do philosophers go on (and on and on) about predictability? Because it’s a convenient rule of thumb. It prevents scientists from adding details to their theory after they have new data, and doing so tends to reduce explanatory power. So, in many cases, asking for predictions is a good idea.
However, if you rely on predictions, you may throw out the baby with the bathwater. Just because no one made a prediction doesn’t mean they necessarily will add assumptions after an observation. In fact, the very opposite can happen. Scientists sometimes remove unnecessary assumptions when they get new data. A theory, therefore, can become better when it has been updated.
Indeed, this has happened several times in the history of physics... (MORE - details) [...] In brief, I think the world would be better place if scientists talked less about predictions and more about explanatory power.
RELATED: Imre Lakatos and the philosophy of bad science
- - -
Reply: hybridwebtech 1:25 PM, May 04, 2020
So, I'm a little confused... I thought that what separates science from philosophy was that scientific assertions (i.e. hypotheses and theories) are testable. That is, theories and hypotheses explain phenomena AND can be falsified by proposing in some form tests. In simplistic form, "If theory xxx is correct then experiment yyy should produce result zzz".
If a proposed description of some aspect of reality does not result in testable predictions, is it scientific?
Replies: Sabine Hossenfelder 12:31 AM, May 05, 2020
hybrid,
"I thought that what separates science from philosophy was that scientific assertions (i.e. hypotheses and theories) are testable."
I don't know why people believe this. Making testable predictions is trivial, and does not tell you that someone used good scientific practice. I explained this here.
"If a proposed description of some aspect of reality does not result in testable predictions, is it scientific?"
As I explained in the blogpost that you are commenting on, the answer is yes.
Reply: Bram Boroson 2:16 PM, May 04, 2020
I think your argument against falsification is a bit of a straw-man argument. Popper himself was never that crude, and other philosophers of science have debated and amended his ideas (Lakatos, et al.) [...]
Replies: Sabine Hossenfelder 12:34 AM, May 05, 2020
I am not arguing against falsification here. I did this here. This blogpost is about the requirement that a theory makes predictions. You can falsify a theory very well with data that predates the theory, so these are two separate things.
EXCERPT (Sabine Hossenfelder): The world, it seems, is full with people who mistakenly think that a theory which makes correct predictions is a good theory. [...] I blame this confusion on the many philosophers, notably Popper and Lakatos, who have gone on about the importance of predictions, but never clearly said that it’s not a scientific criterion.
You see, the philosophers wanted a quick way to figure out whether a scientific theory is good or not that would not require them to actually understand the science. This, needless to say, is not possible. But the next best thing you can do is to ask how much you can trust the scientists. It is for this latter purpose, to evaluate the trust you can put in scientists, that predictions are good. But they cannot, and should not, ultimately decide what the scientific value of a theory is.
[...] See, suppose I plan to convince someone that I can correctly predict the stock market. What do I do? Well, I pick, say, 3 stocks and make “predictions” for a week ahead, but that are really just guesses which cover all reasonably possible trends. I then select a large group of victims. To each of them I send one of my guesses. Some of them will coincidentally get the correct guess. A week later, I know which people got the correct guess. To this group, I then send another set of guesses for the week ahead. Again, some people will get the correct guess by coincidence, and a week later I will know which one it was. I do this a third time, and then I have a group of people who have good “evidence” that I can tell the future.
Amazing, no? What’s the problem here? The problem is that correct predictions don’t tell you whether someone’s theory is good science.
As we have just seen, one of the problems with relying on predictions is that they may be correct just by coincidence. The larger the pool of predictions – or the pool of scientists making predictions! – the more likely this is to happen. The other problem is that relying on predictions makes fundamentally no sense. If I have a scientific theory, it is either a good description of nature, or it is not. At which time someone made a calculation for an observable quantity is entirely irrelevant for a theory’s relation to nature.
This is a point which is often raised by string theorists, and they are correct to raise it. String theorists say that since string theory gives rise to general relativity, it deserves as much praise as general relativity. That’s because, if string theory had been discovered before general relativity, it would have made the same predictions: light deflection on the sun, precession of Mercury, black holes, gravitational waves, and so on.
And indeed, this would be a good argument in favor of string theory – if it was correct. But it isn’t. String theory does not give rise to general relativity. It gives rise to general relativity in 10 dimensions, with supersymmetric matter, a negative cosmological constant, and dozens of additional scalar fields. All this extra clutter conflicts with observations. To fix this conflict with observations, string theorists then have to make several additional assumptions. With that you get a theory that is considerably more complicated than general relativity, but that does not explain the data any better. Hence, Occam’s razor tells you that general relativity is preferable.
Of course, it’s this adding of ad hoc assumptions to fix a mismatch with observation that the philosophers were trying to prevent when they requested testable predictions. But it’s the ad hoc assumptions themselves that are the problem, not the time at which they were made. To decide whether a scientific theory is any good what matters is only its explanatory power. Explanatory power measures how much data you can fit from which number of assumptions. The fewer assumption you make and the more data you fit, the higher the explanatory power, and the better the theory.
Ok, I admit, it’s somewhat more complicated than that. That’s because it also matters how well you fit the data. If you make more assumptions, you will generally be able to fit the data better. So there is a trade-off to be made, which needs to be quantified: At which point is the benefit you get from more assumptions not worth a somewhat better fit to the data? There are statistical tools to decide that. One can argue which one of those is the best for a given purpose, but that’s a fight that experts can fight in the case at hand. What is relevant here is only that the explanatory power of a theory is quantifiable. And it’s the explanatory power that decides whether a theory is good or not.
That’s obvious, I know. But why then do philosophers go on (and on and on) about predictability? Because it’s a convenient rule of thumb. It prevents scientists from adding details to their theory after they have new data, and doing so tends to reduce explanatory power. So, in many cases, asking for predictions is a good idea.
However, if you rely on predictions, you may throw out the baby with the bathwater. Just because no one made a prediction doesn’t mean they necessarily will add assumptions after an observation. In fact, the very opposite can happen. Scientists sometimes remove unnecessary assumptions when they get new data. A theory, therefore, can become better when it has been updated.
Indeed, this has happened several times in the history of physics... (MORE - details) [...] In brief, I think the world would be better place if scientists talked less about predictions and more about explanatory power.
RELATED: Imre Lakatos and the philosophy of bad science
- - -
Reply: hybridwebtech 1:25 PM, May 04, 2020
So, I'm a little confused... I thought that what separates science from philosophy was that scientific assertions (i.e. hypotheses and theories) are testable. That is, theories and hypotheses explain phenomena AND can be falsified by proposing in some form tests. In simplistic form, "If theory xxx is correct then experiment yyy should produce result zzz".
If a proposed description of some aspect of reality does not result in testable predictions, is it scientific?
Replies: Sabine Hossenfelder 12:31 AM, May 05, 2020
hybrid,
"I thought that what separates science from philosophy was that scientific assertions (i.e. hypotheses and theories) are testable."
I don't know why people believe this. Making testable predictions is trivial, and does not tell you that someone used good scientific practice. I explained this here.
"If a proposed description of some aspect of reality does not result in testable predictions, is it scientific?"
As I explained in the blogpost that you are commenting on, the answer is yes.
Reply: Bram Boroson 2:16 PM, May 04, 2020
I think your argument against falsification is a bit of a straw-man argument. Popper himself was never that crude, and other philosophers of science have debated and amended his ideas (Lakatos, et al.) [...]
Replies: Sabine Hossenfelder 12:34 AM, May 05, 2020
I am not arguing against falsification here. I did this here. This blogpost is about the requirement that a theory makes predictions. You can falsify a theory very well with data that predates the theory, so these are two separate things.