Sep 30, 2024 04:01 PM
https://futurism.com/sophisticated-ai-likely-lie
INTRO: It seems that this logic also applies to large language models, which are becoming more powerful with each iteration. New research suggests that this smarter crop of AI chatbots are actually becoming less trustworthy, because they're more likely to make up facts rather than avoiding or turning down questions they can't answer.
The study, published in the journal Nature, examined some of the leading commercial LLMs in the industry: OpenAI's GPT, and Meta's LLaMA, along with an open source model called BLOOM created by the research group BigScience.
While it found that their responses are in many cases becoming more accurate, they were across the board less reliable, giving a higher proportion of wrong answers than older models did.
"They are answering almost everything these days. And that means more correct, but also more incorrect [answers]," study coauthor José Hernández-Orallo, a researcher at the Valencian Research Institute for Artificial Intelligence in Spain, told Nature.
Mike Hicks, a philosopher of science and technology at the University of Glasgow, had a harsher assessment. "That looks to me like what we would call bullshitting," Hicks, who was not involved in the study, told Nature. "It's getting better at pretending to be knowledgeable." (MORE - details)
INTRO: It seems that this logic also applies to large language models, which are becoming more powerful with each iteration. New research suggests that this smarter crop of AI chatbots are actually becoming less trustworthy, because they're more likely to make up facts rather than avoiding or turning down questions they can't answer.
The study, published in the journal Nature, examined some of the leading commercial LLMs in the industry: OpenAI's GPT, and Meta's LLaMA, along with an open source model called BLOOM created by the research group BigScience.
While it found that their responses are in many cases becoming more accurate, they were across the board less reliable, giving a higher proportion of wrong answers than older models did.
"They are answering almost everything these days. And that means more correct, but also more incorrect [answers]," study coauthor José Hernández-Orallo, a researcher at the Valencian Research Institute for Artificial Intelligence in Spain, told Nature.
Mike Hicks, a philosopher of science and technology at the University of Glasgow, had a harsher assessment. "That looks to me like what we would call bullshitting," Hicks, who was not involved in the study, told Nature. "It's getting better at pretending to be knowledgeable." (MORE - details)