Jan 13, 2025 11:28 PM
(This post was last modified: Jan 14, 2025 07:56 PM by C C.)
We need new security systems designed to deal with the weirdness of AI mistakes
https://spectrum.ieee.org/ai-mistakes-schneier
EXCERPTS: Humanity is now rapidly integrating a wholly different kind of mistake-maker into society: AI. Technologies like large language models (LLMs) can perform many cognitive tasks traditionally fulfilled by humans, but they make plenty of mistakes. It seems ridiculous when chatbots tell you to eat rocks or add glue to pizza. But it’s not the frequency or severity of AI systems’ mistakes that differentiates them from human mistakes. It’s their weirdness. AI systems do not make mistakes in the same ways that humans do.
Much of the friction—and risk—associated with our use of AI arise from that difference. We need to invent new security systems that adapt to these differences and prevent harm from AI mistakes.
[...] To the extent that AI systems make these human-like mistakes, we can bring all of our mistake-correcting systems to bear on their output. But the current crop of AI models—particularly LLMs—make mistakes differently.
AI errors come at seemingly random times, without any clustering around particular topics. LLM mistakes tend to be more evenly distributed through the knowledge space. A model might be equally likely to make a mistake on a calculus question as it is to propose that cabbages eat goats.
And AI mistakes aren’t accompanied by ignorance. A LLM will be just as confident when saying something completely wrong—and obviously so, to a human—as it will be when saying something true. The seemingly random inconsistency of LLMs makes it hard to trust their reasoning in complex, multi-step problems. If you want to use an AI model to help with a business problem, it’s not enough to see that it understands what factors make a product profitable; you need to be sure it won’t forget what money is.
[...] This situation indicates two possible areas of research. The first is to engineer LLMs that make more human-like mistakes. The second is to build new mistake-correcting systems that deal with the specific sorts of mistakes that LLMs tend to make.
[...] When it comes to catching AI mistakes, some of the systems that we use to prevent human mistakes will help. To an extent, forcing LLMs to double-check their own work can help prevent errors. But LLMs can also confabulate seemingly plausible, but truly ridiculous, explanations for their flights from reason.
Other mistake mitigation systems for AI are unlike anything we use for humans. Because machines can’t get fatigued or frustrated in the way that humans do, it can help to ask an LLM the same question repeatedly in slightly different ways and then synthesize its multiple responses. Humans won’t put up with that kind of annoying repetition, but machines will... (MORE - missing details)
Could AI replace politicians? A philosopher maps out three possible futures
https://theconversation.com/could-ai-rep...res-246901
INTRO: While the idea of AI politicians might make some people uneasy, survey results tell a different story. A poll conducted by my university in 2021, during the early surge of AI advancements, found broad public support for integrating AI into politics across many countries and regions.
A majority of Europeans said they would like to see at least some of their politicians replaced by AI. Chinese respondents were even more bullish about AI agents making public policy, while normally innovation-friendly Americans were more circumspect.
As a philosopher who researches the moral and political questions raised by AI, I see three main pathways for integrating AI into politics, each with its own mixture of promises and pitfalls.
While some of these proposals are more outlandish than others, weighing them up makes one thing certain: AI’s involvement in politics will force us to reckon with the value of human participation in politics, and with the nature of democracy itself... (MORE - details)
https://spectrum.ieee.org/ai-mistakes-schneier
EXCERPTS: Humanity is now rapidly integrating a wholly different kind of mistake-maker into society: AI. Technologies like large language models (LLMs) can perform many cognitive tasks traditionally fulfilled by humans, but they make plenty of mistakes. It seems ridiculous when chatbots tell you to eat rocks or add glue to pizza. But it’s not the frequency or severity of AI systems’ mistakes that differentiates them from human mistakes. It’s their weirdness. AI systems do not make mistakes in the same ways that humans do.
Much of the friction—and risk—associated with our use of AI arise from that difference. We need to invent new security systems that adapt to these differences and prevent harm from AI mistakes.
[...] To the extent that AI systems make these human-like mistakes, we can bring all of our mistake-correcting systems to bear on their output. But the current crop of AI models—particularly LLMs—make mistakes differently.
AI errors come at seemingly random times, without any clustering around particular topics. LLM mistakes tend to be more evenly distributed through the knowledge space. A model might be equally likely to make a mistake on a calculus question as it is to propose that cabbages eat goats.
And AI mistakes aren’t accompanied by ignorance. A LLM will be just as confident when saying something completely wrong—and obviously so, to a human—as it will be when saying something true. The seemingly random inconsistency of LLMs makes it hard to trust their reasoning in complex, multi-step problems. If you want to use an AI model to help with a business problem, it’s not enough to see that it understands what factors make a product profitable; you need to be sure it won’t forget what money is.
[...] This situation indicates two possible areas of research. The first is to engineer LLMs that make more human-like mistakes. The second is to build new mistake-correcting systems that deal with the specific sorts of mistakes that LLMs tend to make.
[...] When it comes to catching AI mistakes, some of the systems that we use to prevent human mistakes will help. To an extent, forcing LLMs to double-check their own work can help prevent errors. But LLMs can also confabulate seemingly plausible, but truly ridiculous, explanations for their flights from reason.
Other mistake mitigation systems for AI are unlike anything we use for humans. Because machines can’t get fatigued or frustrated in the way that humans do, it can help to ask an LLM the same question repeatedly in slightly different ways and then synthesize its multiple responses. Humans won’t put up with that kind of annoying repetition, but machines will... (MORE - missing details)
Could AI replace politicians? A philosopher maps out three possible futures
https://theconversation.com/could-ai-rep...res-246901
INTRO: While the idea of AI politicians might make some people uneasy, survey results tell a different story. A poll conducted by my university in 2021, during the early surge of AI advancements, found broad public support for integrating AI into politics across many countries and regions.
A majority of Europeans said they would like to see at least some of their politicians replaced by AI. Chinese respondents were even more bullish about AI agents making public policy, while normally innovation-friendly Americans were more circumspect.
As a philosopher who researches the moral and political questions raised by AI, I see three main pathways for integrating AI into politics, each with its own mixture of promises and pitfalls.
While some of these proposals are more outlandish than others, weighing them up makes one thing certain: AI’s involvement in politics will force us to reckon with the value of human participation in politics, and with the nature of democracy itself... (MORE - details)